(back to 2006 C.I.P.A. Winners)
SHARCNET, Western Science Centre, University of Western Ontario
Shared Hierarchical Academic Research Computing Network
Harnessing Grid Computing Leads to World-Class Computational Research Power and Produces Faster Results
Challenge
In the past, the lack of competitive high-performance computing (HPC) programs and services in Canada was generally defined by a critical mass of researchers as a "failure." In 2001, a group of researchers resolved to address this disconcerting gap through a joint grant application involving a number of institutions in Southwestern Ontario. This initiative led to the formation of the Shared Hierarchical Academic Research Computing Network, a.k.a. SHARCNET.
SHARCNET is a consortium of colleges and universities in a "cluster of clusters" of high-performance computers, linked by advanced fibre optics. Its unique computational infrastructure combines an active academic-industry partnership, enabling world-class computational research.
Due to the rapid obsolescence of HPC hardware among Canadian institutions, SHARCNET's key objective was to build an "HPC culture" in Ontario and Canada: a community of skilled researchers that would support the growing demands in academia and the private sector for HPC, while generating the critical mass of users to ensure long-term support for, and provision of, leading hardware resources. In particular, SHARCNET would target the computational needs of researchers working on large problems that could not otherwise be contemplated.
A number of infrastructure challenges confronted the organization as it worked to meet its objectives. They needed to:
- Design the technical infrastructure so as to concurrently support a large number of researchers working at institutions located across a wide geographic region;
- Provide efficiency for users while maintaining a manageable computing environment;
- Accommodate additional member institutions and growth and evolution within the existing components of the infrastructure itself.
Solution
SHARCNET focused on the development and deployment of a high-performance computing grid (HPC-Grid) using leading commercial technology. The initial deployment of equipment occurred in 2001 as a result of a grant from the Canada Foundation for Innovation (CFI) and matching funds from the Government of Ontario. It consisted of three large clusters and several smaller development systems. These clusters were connected via a dedicated 1 Gb/second link through the ORION network. This solution provided a proof of concept for the HPC-Grid.
The consortium's initial success attracted more member institutions. A second CFI award along with matching provincial funds in 2004 resulted in four new large HPC clusters of varying architectures, several smaller clusters for development, testing and visualization, and an archival storage facility. The four large clusters from Hewlett-Packard (Canada) Co. consisted of AMD Opteron multi-core processors configured in different combinations of processors for each type of node, memory and cluster interconnection network, to create large parallel clusters that could meet different application needs.
These systems enable researchers to use the best architectural configuration (hardware and software) to meet the particular computing characteristics of an application. A shared-memory SGI system was also installed, as were six smaller clusters from Hewlett-Packard and a 200-Tb archival facility.
Each of the compute clusters uses a new parallel file system designed to meet the input-output needs of HPC applications. These systems provide the computing capabilities, but rather than installing all of them at a single location, the systems were distributed at various member institutions. Duplicating all the equipment would be expensive and an inefficient use of resources.
The increasing number of member institutions, their geographical separation, the breadth and size of the equipment and the decision to install it at the individual member sites necessitated a state-of-the-art network.
Rather than acquiring its own fibre at considerable cost, SHARCNET entered into partnership with ORANO (Optical Regional Advanced Network of Ontario), which provided a dedicated 10 Gb/second network, with some 1 Gb/second links, as the connection for the computational and storage facilities at the partner institutions. The network permits researchers at any of them to access computing, storage and software resources across the consortium and to move large amounts of data.
Results
With the high-speed network in place, SHARCNET is able to provide the services that are transforming use of HPC for researchers at its member institutions. The number of users has grown exponentially since 2001 to 1,400 researchers representing 16 academic institutions.
The network has already had a significant impact on the ability of researchers from different institutions and disciplines, such as engineering, chemistry and physics, to access the infrastructure. Further, by providing hardware, comprehensive training and personnel support, it is also empowering a variety of disciplines that are new to HPC, such as financial mathematics, bioinformatics, economics and the humanities.
Innovative Use of Technology
The uniqueness and innovation of SHARCNET's grid is based on the diversity of computing architectures available to researchers, the dedicated optical network to interconnect the computing systems, the use of existing fibre with seamless integration to ORANO's existing optical equipment, and the sets of services that enable researchers at any location to access and use compute clusters or storage.
A 2006 CIPA Winner!
For its exceptional application of information technology to transform processes and bring benefits to the public, SHARCNET has been awarded a 2006 CIPA Silver Award of Excellence in the Efficiency and Operational Improvements, Not For Profit category.
(back to top)
