Today at her keynote address to the AMS Washington Forum, U.S. Secretary of Commerce Penny Pritzker announced that NOAA is forming five new alliances to help bring its vast data resources to the public. The partnerships with Amazon Web Services, Microsoft Azure, IBM, Google, and the Open Cloud Consortium address the growing need for access to NOAA’s huge—and rapidly growing—environmental data resource.
That Secretary Pritzker’s announcement came at the opening of this year’s Forum is a testament to the sustained focus of these annual AMS gatherings in Washington, D.C. The Forum revisits recurring themes to build year-to-year unity—and progress—to the discussions. Last year, for example, the AMS Washington Forum participants focused on how data integration across disciplines and sectors drives the effectiveness of the weather, water, and climate enterprise. The Forum found that
Working across agencies and across sectors (e.g., health, energy) is becoming a new “normal” for solving problems. All agree the needs and demands for data, information and forecasts are continuing to change, so our enterprise must remain flexible and agile.
Though the context last year was more about the use of commercially provided data, this continuing Forum theme resonates with Secretary Pritzker’s announcement today. The new government-private sector partnerships are part of the overall movement toward “open government”–accessible, consistent data practices—that should enhance the flexibility and agility emphasized at the AMS Forum last year.
Forum participants also generally agreed last year that “while the private sector needs to take on a bigger role in the provision of weather data, the public and private sectors need more time to jointly determine the best path forward.” And indeed at that time NOAA was in an information-gathering phase preparing for the partnerships announced today. The agency issued a Request for Information (RFI) in February 2014 to see who might be able to help move NOAA data onto the cloud. Commercial partnerships would, according to the RFI, help pull together disparate NOAA sources and web sites and help people “find and integrate data from these sources for cross-domain analysis and decision-making.”
Data integration was not the only motivation. Being the main provider of its own data saddles government agencies with burgeoning information technology needs.
In a separate email news letter today, NOAA Administrator Kathryn Sullivan elaborated on the scope of the Big Data need:
Of the 20 terabytes of data NOAA gathers each day — twice the data of the entire printed collection of the United States Library of Congress — only a small percentage is easily accessible to the public.
The cloud was a way to alleviate this situation, as the RFI stated:
NOAA anticipates these partnerships will have the ability to rapidly scale and surge; thus, removing government infrastructure as a bottleneck to the pace of American innovation and enabling new value-added services and unimaginable integration into our daily lives.
Private sector cloud services have a history of meeting such challenges. The cloud services are able not only to store the huge quantities of data NOAA produces each day but also to provide opportunities for cloud-based applications. This means information processing is possible remotely so that each user does not need to have his or her own advanced infrastructure to move and manipulate vast troves of data. Thus, working in parallel with traditional NOAA data distribution channels, cloud services are expected to enable widespread use of Big Data and to drive private-sector development of applications.
The continued AMS discussions here in D.C. over Wednesday and Thursday will further amplify such continuing themes as Big Data, providing an especially rewarding venue for participants who can return year after year to the Forum. For example sessions tomorrow on “Rail and Trucking” and “Information Needs for Water Related Extremes” hinge in part on data dissemination. Surface transportation was one of the panel topics last year, as well, meaning repeat participants this year will have an opportunity to update their earlier impressions and find out how opportunities in that field are progressing.
By reaching out to the innovators of the cloud, NOAA stated it was
looking for partners to incite creative uses and innovative approaches that will tap the full potential of its data, spur economic growth, help more entrepreneurs launch businesses, and to create new jobs.
That’s pretty much the same reason leaders of the weather, water, and climate enterprise return year after year to the AMS Washington Forum.
Informative post!!