Energy Sciences Network

The Energy Sciences Network (ESnet) is the Office of Science’s high-performance network user facility, delivering highly-reliable data transport capabilities optimized for the requirements of large-scale science. In essence, ESnet is the circulatory system that enables the DOE science mission. ESnet is stewarded by the Advanced Scientific Computing Research Program, [1] and managed and operated by the Scientific Networking Division at Lawrence Berkeley National Laboratory.

ESnet interconnects the DOE’s national laboratory system, dozens of other DOE sites, and ~200 research and commercial networks around the world—enabling tens of thousands of scientists at DOE laboratories and academic institutions across the country to transfer vast data streams and access remote research resources in real-time. ESnet exists to provide the specialized networking infrastructure and services required by the national laboratories, large science collaborations, and the DOE research community. ESnet provides the high-bandwidth, reliable connections that enable many thousands of the nation’s scientists to collaborate on some of the world's most important scientific challenges including energy, biosciences, materials, and the origins of the universe.

ESnet was formed in 1986, combining the operations of earlier DOE networking projects known as HEPnet (for high-energy physics) and MFEnet (for magnetic fusion energy research).[2] While growing within the terrestrial US over the years, in December 2014, ESnet deployed three 100 Gbit/s links and one 40 Gbit/s connection between the United States and Europe to enhance collaborative research.

From 1990 to 2019, ESnet's average traffic has grown by a factor of 10 every 47 months.[3] By 2005, the core network used packet over SONET links at 10 Gbit/s.[4] In 2009, ESnet received $62 million in American Research and Recovery Act (ARRA) funding from the Department of Energy Office of Science to invest in its next-generation infrastructure to provide the necessary support for research discovery in this new era of data-intensive science. After three years of planning and deployment, ESnet5 – ESnet’s fifth generation network – launched in November 2012. The new network provided an immediate tenfold increase in bandwidth to DOE research sites at the same cost as providing 10G connections – with the ability to now scale to 44 times the current capacity. ESnet is currently working on its next-generation upgrade named ESnet6.

In October 2011, ESnet rolled out its 100 Gbit/s backbone network, known internally as ESnet 5. The network is the world’s fastest science network, serving the entire DOE national laboratory system, its supercomputing centers, and its major scientific instruments at speeds 10 times faster than ESnet’s previous generation network.

ESnet partnered with Internet2, the network that connects America’s universities and research institutions, to deploy its 100 Gbit/s network over a new, highly-scalable optical infrastructure that the two organizations share for the benefit of their respective communities.

The project was funded in 2009 by $62 million in American Recovery and Reinvestment Act funds. Call the Advanced Networking Initiative, the Gbit/s network was a project to handle the expanding data needs between DOE supercomputing facilities. In December 2014, ESnet extended its reach by deploying four new high-speed transatlantic links, giving researchers at America’s national laboratories and universities ultra-fast access to scientific data from the Large Hadron Collider (LHC) and other research sites in Europe. ESnet’s transatlantic extension delivers a total capacity of 340 gigabits per second (Gbit/s), and serve dozens of scientific collaborations. To maximize the resiliency of the new infrastructure, ESnet equipment in Europe will be interconnected by dedicated 100 Gbit/s links from the pan-European networking organization GÉANT. The new trans-Atlantic links build on the success of the US LHCNet, a collaboration between Caltech and CERN, led by Harvey Newman of Caltech.

The scientific community is facing a growing challenge: dramatically increasing data volumes. Telescopes are scanning the universe with greater precision. Supercomputers are simulating scientific phenomena at higher resolutions. Sensors and detectors are gathering experimental results with greater sensitivity and speed. ESnet’s next-generation network, ESnet6, is designed to help the DOE research community navigate this “data deluge” by giving them more bandwidth, greater flexibility, and faster data transfer capabilities.

With a projected early finish in 2023, ESnet6 will feature an entirely new software-driven network design that enhances the ability to rapidly invent, test, and deploy new innovations. The design includes:

• State-of-the-art optical, core and service edge equipment deployed on ESnet’s dedicated fiber optic cable backbone
• A scalable switching core architecture coupled with a programmable services edge to facilitate high-speed data movement
• 100 to 400 Gbit/s optical channels, with up to eight times the potential capacity compared to ESnet5
• Services that monitor and measure the network 24/7/365 to ensure it is operating at peak performance, and
• Advanced cybersecurity capabilities to protect the network, assist its connected sites, and defend its devices in the event of a cyberattack.

ESnet6 represents a transformational change in network capacity, resiliency, and flexibility that will bring tangible benefits to the DOE mission, including exascale. As the top science data network in the world, ESnet will continue supporting effective collaborations and consistent access to data, computing, and experiments, thus contributing significantly to U.S. competitiveness.

ESnet research and development programs have included:

• Software Defined Networking: ESnet has developed ENOS, the ESnet Network Operating system is a prototype next-generation architecture for handling data-intensive science workflows. The concept of a “network operating system,” a software layer that enables applications to get information about the network and to program it to meet its needs, has been discussed academically for awhile but now appears feasible with the wide adoption of the Software-Defined Networking (SDN) paradigm. The ENOS approach will enable ESnet to build better application-engaged scientific networks and provide the interfaces, data and programmability for applications or science virtual organizations to better orchestrate their multi-site complex big data projects and optimize their use of storage, compute and cloud resources.
• Science DMZ: A network design pattern for securely routing large datasets around institutional firewall developed by ESnet and the National Energy Research Scientific Computing Center. Typically, located at the network perimeter, a DMZ has its own security policy and is specifically dedicated to external-facing high-performance science services – exchanging data with the outside world. With support from the National Science Foundation, more than 100 universities across the country have deployed Science DMZs.
• 400 Gbit/s network: In 2015, ESnet and the National Energy Research Scientific Computing Center (NERSC) have built a 400 gigabit-per-second super-channel, the first-ever 400G production link to be deployed by a national research and education network. The connection provided critical support for NERSC’s 6,000 users as the facility moved from its previous location in Oakland, Calif. to the main campus of Lawrence Berkeley National Laboratory in Berkeley, Calif. The team also set up a 400 Gbit/s research testbed for assessing new tools and technologies without interfering with production data traffic.
• OSCARS: The On Demand Secure Circuits and Reservation System software developed at ESnet, and now open-sourced to the community, creates multi-domain, virtual circuits guaranteeing end-to-end data transfer performance on the network. OSCARS was recognized with an R&D 100 Award in 2013.
• perfSONAR: A test and measurement framework that provides end-to-end monitoring of multi-domain network performance. ESnet is a key member of the international perfSONAR collaboration, along with GÉANT, Indiana University and Internet2. As of 2016, perfSONAR had more than 1,600 installations.

Secretary of Energy Achievement Award: In May 2015, Energy Secretary Ernest Moniz recognized ESnet staff members with a DOE Secretarial Honor Award for their development of OSCARS, the On-demand Secure Circuits and Reservation System. The Secretarial Honor Awards are the department's highest form of non-monetary employee recognition. Individual and team awardees are selected by the Secretary of Energy.[5]

CENIC Innovations in Networking Award: The Corporation for Education Network Initiatives in California (CENIC) named ESnet as a recipient of the 2015 Innovations in Networking Award for High-Performance Research Applications for their 100-Gigabit Software-Defined Networking (100G SDN) Testbed.[6]

IEEE-SA Standards Medallion: Michael Bennett, head of ESnet’s Network Engineering Group, has been awarded an IEEE-SA Standards Medallion for his work in helping to create energy-saving standards for devices with Ethernet connections. The standards are expected to help save terawatts of otherwise-wasted electricity by automatically switching networked components to energy-saving modes when not in use.[7] R&D100 Award: ESnet received the R&D100 award for OSCARS (the On-demand Secure Circuits and Reservation System) version 0.6, which was finalized and released in June 2012. OSCARS can automatically create end-to-end circuits, crossing multiple network domains.[8]

InformationWeek’s Top 10 Government IT Innovators: For the second time in four years, ESnet was named one of the year’s top government IT innovators by InformationWeek Government magazine. Ihe magazine cited ESnet for its new 100 gigabit-per-second nationwide network funded as the Advanced Networking Initiative and launched into full production in November 2012.[9]

Fierce Innovators: Fierce Government, a publication covering the U.S. government, named the ESnet5 Deployment Team, charged with rolling out the 100-Gigabit-per-second national network that entered production in November, to the first annual "Fierce 15" list of the top federal employees and teams who have done particularly innovative things.[10]

Internet2 IDEA Awards: Two ESnet projects received IDEA (Internet2 Driving Exemplary Applications) awards in Internet2’s 2011 annual competition for innovative network applications that have had the most positive impact and potential for adoption within the research and education community. Internet2 recognized OSCARS (On-Demand Secure Circuits and Advance Reservation System), developed by the ESnet team led by Chin Guok, including Evangelos Chaniotakis, Andrew Lake, Eric Pouyoul and Mary Thompson. Contributing partners included Internet2, USC ISI and DANTE. ESnet’s MAVEN (Monitoring and Visualization of Energy consumed by Networks) proof of concept application was also recognized with an IDEA award in the student category.[11]

InformationWeek’s Top 10 Government IT Innovators: ESnet was honored as one of Top 10 Government IT Innovators in 2009 by InformationWeek for the development and deployment of On-Demand Secure Circuits and Reservation System (OSCARS).[12]

Excellence.Gov Award: ESnet was honored with a 2009 Excellence.Gov award for its achievements in leveraging technology. The Excellence.Gov awards are sponsored by the Industry Advisory Council’s (IAC) Collaboration and Transformation Shared Interest Group and recognize the federal government’s best information technology (IT) projects.[13]

• "Energy Sciences Network". Retrieved October 8, 2011.
• "Energy Sciences Portal"