Frequently Asked Questions
What is the NOAA Big Data Project?
The NOAA Big Data Project (BDP) is a combination of three incredibly powerful resources: the expansive collection of high-quality environmental data and expertise of NOAA, the vast infrastructure and scalable computing capabilities of our industry collaborators, and innovation and energy of the American economy. Under the BDP, NOAA works with infrastructure-as-a-service (IaaS) providers to broaden access to NOAA’s data resources. The collaboration, designed in order to facilitate full and open data access at no net cost to the taxpayer, helps bring together these resources to makes NOAA's data more publicly accessible in order to foster innovation and collaboration.
In what way is the BDP different from what NOAA is already doing with its data?
The BDP differentiates itself from NOAA's current offerings by positioning NOAA's data within modern platforms with high-performance computer, analytic, and storage services to create a new economic space and expand public access to the data. NOAA has a long and well-established history of sharing its environmental data with private industry,the academic research community, and the general public. There are no current plans to change these existing offerings. Instead, the BDP is designed to augment these by creating another means of access to NOAA's open data resources.
What is NOAA’s timeline for success with the BDP?
Currently, NOAA and the BDP collaborators are in the midst of researching and testing solutions for bringing NOAA’s vast wealth of information to the cloud, where both the public and industry can easily and equally access, explore, and create new products from it to foster new ideas and spur economic growth. While no decisions have yet been made regarding the long-term future of the BDP, the collected information, best practices, and lessons learned from its efforts will help inform such a decision about the BPD’s future.
What are the milestones that you hope to meet?
The BDP is a research project, and like any research project, the most important aspect is learning and discovery. The NOAA BDP team has been collecting lessons learned, performance metrics and best practices since the inception of the project, and will do so going forward as the project continues. Subsequent iterations will be informed by lessons learned from that initial research, and NOAA hopes that there will be prototypes available within the near future. The most important milestone that the BDP will achieve is providing a recommendation and prototype for continued operations while simultaneously expanding access to NOAA's data resources.
By joining with several large commercial cloud providers, is NOAA limiting the opportunities for academia and other parts of the nonprofit sector to collaborate with NOAA on big data issues?
Quite the opposite. The intention of the NOAA Big Data Project is to provide full and open access to NOAA's data, including providing everyone equal access. NOAA spent over a year in close communication and consultation with both the private industry and the open government data community during the project's initial research. This included producing two requests for information, consulting several private industry at a widely-attended Industry Day, and conducting many individual discussions. Taking the information gained from this rigorous process, NOAA sought out its initial BDP collaborators by approaching the organizations we felt would best serve as anchors around which broader ecosystems (data alliances) could develop. The BDP Collaborators are all granted the same access to NOAA’s data in a “fair and level” manner, with no privileged access that could limit the opportunities for the Collaborators, the Data Alliances, and the wider research community. NOAA specifically included multiple collaborators, including representatives from academia and the nonprofit sector, to democratize access and ensure that all interested parties could participate in this opportunity. Additionally, NOAA continues to work closely with its existing academic research partners to maximize the value of its data.
What kind of access to NOAA data will the general public have?
The purpose of the BDP is to expand upon the existing full and open access to NOAA’s data to all parties, including the general public. NOAA continues to offer access to its existing data services with no interruptions to its customers and there are no current plans to discontinue these existing services. The BDP’s activities are instead supplementing these services. The general public can access data made available through the Collaborators, who may charge a marginal cost for distribution. However, the data remain free and available to the public without restriction on use. These data will include the original and unaltered NOAA data content, and may additionally be provided using any broadly accessible and open format or formats.
Will this change how the public receives its environmental data for its day-to-day purposes?
No. The only change this will have on how the public currently receives its environmental data is by providing an additional means of access to the data. While data remains accessible through NOAA’s existing data access portals, they will now be co-located with powerful cloud computing platforms to make its analysis more readily available to everyone. Based on the initial results, this means that new applications can be developed faster with a potentially greater speed of service while also reducing the load on NOAA’s servers and networks.
In addition to the project being conducted at no net-cost to the taxpayer, it will not alter the day-to-day operations within NOAA. This means that all of NOAA’s offerings, from the National Weather Service 122 weather forecast offices providing daily weather predictions to the National Ocean Services’ work monitoring changes in the coastline, will remain available to the public through the same channels on a day-to-day basis.
Will the public have to pay for data that is currently accessible to them free of charge?
No. NOAA’s data will remain free and available to the public without restriction on use, including through existing NOAA portals. The BDP collaborators are tasked with distributing the original data content, and may recover their costs for that distribution by charging a nominal cost for distribution only. Additionally, any developer of value-added products that use NOAA data, including the collaborators, can charge for those products, in the same way that they do today. However, a collaborator must ensure everyone wishing to access the data is given the same opportunity to do.
Where can I find a copy of the Cooperative Research and Development Agreements (CRADAs) that support the BDP activities?
NOAA entered into five separate yet distinct agreements with its collaborators when the BDP began. The NOAA Big Data Project CRADA Template used for each of these agreements is available for download as a PDF.