The GreenHPC initiative at Rutgers is a research and educational initiative aiming at addressing several efforts in the intersection of energy efficiency, scalable computing and high performance computing. Key focus areas include:
- Energy efficiency of scientific data analysis pipelines at scale
- In-situ data analytics and co-processing at extreme scales
- Application-aware cross-layer power management for High Performance Computing systems
GreenHPC also acts as a forum for researchers and the educational community to exchange ideas and experiences on energy efficiency by disseminating research results, educational activities at different levels (PhD, MS, undergraduate - REU, K12 - GSET) and organizing events and editorial activities of related topics.
DataSpaces is a data management framework for scientific workflows running on high-performance computing platforms. It provides a scalable, in-memory shared-space abstraction that allows the component applications that comprise a workflow to interact and exchange data using a very simple put/get API. This shared space is built using a dedicated set of staging nodes, which are used exclusively for handling I/O operations. In addition, communication between applications and this shared space can be asynchronous and are performed with RDMA transport when available, enabling applications to take advantage of high-speed interconnect protocols without rewriting their applications for different vendors. DataSpaces has been ported to many of the current large-scale platforms, including:
- InfiniBand clusters
- Cray machines (uGNI/Gemini)
- IBM Blue Gene systems (PAMI/DCMF)
In addition, DataSpaces is integrated with and deployed as part of the Adaptive I/O System (ADIOS) framework distributed by Oak Ridge National Laboratories. ADIOS is an open source I/O middleware package that has been shown to scale to hundreds of thousands of cores and is being used by a very wide range of applications. DataSpaces/ADIOS has been used to support coupled applications in combustion, fusion, material science, chemistry, and FEM+AMR.
The overreaching goal of CometCloud is to enable highly heterogeneous, dynamically federated computing and data platforms that can support end-to-end application workflows with diverse and dynamic changing application requirements. This is achieved through:
- Autonomic on-demand federation of geographically distributed compute and data resources as needed by the application workflow
- Exposing the resulting software-defined federated cyberinfrastructure using elastic cloud abstractions and science-as-a-service platforms.
As a result, CometCloud is able to create a nimble and dynamically programmable environment that autonomously evolves over time, adapting to changes in both the federated infrastructure and the application requirements.