Caltech is Partner in First Distributed Terascale Facility,NSF Awards $53 Million for Project

The facility will be built and operated with $53-million from the National Science Foundation (NSF). It will perform 11.6-trillion calculations per second and store more than 450-trillion bytes of data, with a comprehensive infrastructure called the "TeraGrid" to link computers, visualization systems and data at four sites through a 40-billion bits-per-second optical network.

The National Science Board (NSB) approved a three-year NSF award, pending negotiations between NSF and a consortium led by the National Center for Supercomputing Applications (NCSA) in Illinois and the San Diego Supercomputer Center (SDSC) in California, the two leading-edge sites of NSF's Partnerships for Advanced Computational Infrastructure (PACI). NCSA and SDSC will be joined in the DTF project by Argonne National Laboratory (ANL) in suburban Chicago and Caltech.

"The DTF will be a tremendous national resource," said NSF director Rita Colwell. "With this innovative facility, NSF will demonstrate a whole new range of capabilities for computer science and fundamental scientific and engineering research, setting high standards for 21st Century deployment of information technology."

"Terascale" refers to computers that perform more than one trillion floating-point operations per second, called "teraflops." The DTF would begin operation in mid-2002, reaching peak performance of 11.6 teraflops by April 2003. The facility will support research such as storm, climate and earthquake predictions; more-efficient combustion engines; chemical and molecular factors in biology; and physical, chemical and electrical properties of materials.

"This facility will stretch the boundaries of high-performance computing and give U.S. computer scientists and other researchers in all science and engineering disciplines access to a critical new resource," said NSB chair Eamon Kelly.

Adds Ruzena Bajcsy, NSF assistant director for Computer and Information Science and Engineering, "The DTF can lead the way toward a ubiquitous 'Cyber-Infrastructure' in which the national Grid of research networks will permit calculations, storage and throughput at tera levels. This facility will serve the high-end computational science community, help train the next generation of information-technology professionals and propagate the latest technology for maximum public benefit."

The partnership will work primarily with IBM, Intel Corporation and Qwest Communications to build the facility, along with Myricom, Oracle Corporation and Sun Microsystems. "The DTF will be the most comprehensive information infrastructure ever deployed for open scientific research, and we feel privileged to have a leadership role in this historic effort," said NCSA director Dan Reed and SDSC director Fran Berman in a joint statement. "The TeraGrid will integrate the most-powerful computers, software, networks, data access systems and applications, creating a unique national resource that will catalyze new breakthroughs and yield unforeseen benefits for all of society." Berman and Reed are DTF co-principal investigators.

Each of the four DTF sites will play a unique role in the project:

NCSA will lead the project's computational aspects with an IBM Linux cluster powered by Intel's second-generation 64-bit Itanium family processor, code-named "McKinley." Peak performance will be 6.1 teraflops with the cluster, which will work in tandem with existing hardware to reach 8 teraflops with 240 terabytes of secondary storage.

SDSC will lead the project's data- and knowledge-management effort with a 4-teraflops IBM Linux cluster based on Intel's McKinley processor, with 225 terabytes of storage and a next generation Sun high-end server for managing access to Grid distributed data.

Argonne will have a 1-teraflop IBM Linux cluster to host advanced software for high-resolution rendering, remote visualization and advanced Grid software.

Caltech will focus on scientific data, with a .4-teraflop McKinley cluster and a 32-node IA-32 cluster that will manage 86 terabytes of on-line storage.

The DTF project director will be Rick Stevens, who is a computer science faculty member at the University of Chicago and director of the mathematics and computer science division of ANL, a U.S. Department of Energy laboratory. "I'm excited by this opportunity to help build on prior NSF and PACI successes," said Stevens, "and it is a wonderful example of interagency cooperation."

The DTF will join a previous terascale facility commissioned by NSF in 2000. That system, located at the Pittsburgh Supercomputing Center, came on-line ahead of schedule in early 2001 and is expected to reach peak performance of 6 teraflops in October.