NAO 216-115A: RESEARCH AND DEVELOPMENT IN NOAA
Issued 10/3/2016; Effective 10/3/2016; Last Reviewed: 11/27/2018
SECTION 1. PURPOSE
.01 As a science-based services agency, The National Oceanic and Atmospheric Administration (NOAA) maintains a robust, high quality research and development (R&D) portfolio that continually improves NOAA’s products and services in response to:
- Growing demand for new and improved services;
- Changes in NOAA’s mission needs in response to evolving environmental conditions or directives from the Legislative, Judicial and Executive branches;
- Emerging understanding of the dynamic Earth system and its ecosystems;
- Innovations in R&D methods, tools and approaches; and
- NOAA’s role in cooperative environmental research and management efforts.
.02 This Order builds upon existing best practices to promote scientific and technological excellence and enable scientists and science leaders to pursue the R&D necessary to inform NOAA's service and stewardship responsibilities.
.03 This Order provides guidance by which the R&D throughout NOAA can be continually reviewed, evaluated and rebalanced in light of evolving mission needs, thus allowing the Agency to apply a logical approach to its R&D investment portfolio.
.04 The coordinated efforts described in this Order include support for the successful transitions of R&D into operations, applications, commercialization and other uses as defined in NOAA Administrative Order (NAO) 216-105A, Policy On Research and Development Transitions.
.05 The Office of the NOAA Chief Scientist is the organization responsible for preparing and maintaining this NAO and has the expertise to answer questions regarding its provisions or subject matter.
.06 The implementation of this NAO will be guided by an associated Procedural Handbook.
SECTION 2. SCOPE.
.01 This NAO applies to all NOAA R&D activities, whether internal or external R&D, and includes R&D conducted by NOAA and sponsored by others.
SECTION 3. PRINCIPLES.
The following principles will guide the planning, execution and evaluation of NOAA’s R&D portfolio:
.01 Mission alignment: NOAA’s R&D portfolio will be focused on NOAA’s explicitly defined mission needs, which include “to understand and predict changes in climate, weather, oceans, and coasts,” “to share that knowledge and information with others,” and “to conserve and manage coastal and marine ecosystems and resources.” All participants in NOAA R&D will know the specific mission(s) they are supporting with their research or technology development and view each of their activities as directly relevant to (a) specific NOAA mission(s) need.
.02 Transitioning research and development: Continually improving NOAA’s products and services to meet the needs of the Nation is an integral part of NOAA’s R&D portfolio. These improvements occur by developing the most promising research, including new or improved observing, modeling and information technologies and the results of field- and laboratory-based process studies, to the point they can be transitioned into operations, applications, commercialization or other use (R2X). Participants in NOAA R&D will understand their activities in the framework of readiness levels, as defined in NAO 216-105A: Policy on Research and Development Transitions.
.03 Research balance: Meeting NOAA’s evolving mission needs requires a vigorous and forward-looking R&D portfolio that includes a wide range of natural and social sciences. Careful planning will ensure that NOAA’s R&D portfolio is balanced across disciplines, across readiness levels, among interdisciplinary foci, between natural and social science, and between short-term and long-term efforts, and will tolerate a degree of risk commensurate with the goal of sustaining and advancing world-class capabilities in support of the Agency’s mission needs.
.04 Partnerships: NOAA has a wide range of tools for accomplishing R&D by engaging external partners. These include contracts; grants; Sea Grant, Cooperative Institute and Cooperative Science Center agreements with universities; joint activities with other Federal agencies; “Citizen Science” projects; Cooperative Research and Development Agreements and other arrangements with the private sector; and a variety of nontraditional partnerships such as with tribes, states, Councils and Commissions, non-governmental organizations, foreign and international organizations and partners within industries affected by NOAA management actions. These tools and arrangements with partners provide considerable flexibility to the NOAA R&D portfolio as well as access to a diverse range of talented personnel. Different approaches to supporting R&D are appropriate for different degrees of maturity in a given area of research and allow for considerable risk tolerance. In evaluating the NOAA R&D portfolio, NOAA will determine what tools, or portfolio of tools, are optimal for which mission need, and adjust the current set of tools to best meet NOAA’s evolving mission needs while carefully managing partner expectations.
.05 Facilities and infrastructure: The complex, transdisciplinary and long-term R&D essential to accomplishing NOAA’s missions depends on sophisticated research facilities and hardware, including laboratories, instrumentation, ships, aircraft, satellites, moored platforms, autonomous vehicles, high performance supercomputing, test beds, proving grounds, data repositories, computer models and information systems. NOAA will prioritize infrastructure investments based on a corporate view of mission needs and R&D.
.06 Workforce excellence: A diverse, creative and vibrant scientific workforce is at the core of NOAA’s R&D and mission services enterprise. NOAA will seek to recruit, develop and retain a diverse scientific workforce in a range of relevant disciplines and provide opportunities for scientific staff to excel individually, as team members and leaders, and as mentors for the next generation of NOAA scientists, including those from underrepresented groups. Training and professional development will be made available such that the scientific workforce can adapt, expand beyond traditional disciplinary boundaries and remain agile as mission needs change. NOAA will encourage and reward engagement with mission-relevant professional societies and communities of practice.
.07 Scientific integrity: Scientific research sponsored and conducted by NOAA catalyzes innovative breakthroughs, informs regulatory and policy decisions and enables the development of new industries. A robust scientific enterprise requires transparency, traceability, reproducibility and scientific integrity at all levels of practice and management. To maintain NOAA’s place among the most trusted and credible sources of scientific information in the world, the NOAA R&D portfolio will embody both the NOAA Scientific Integrity Policy (NAO 202-735D: Scientific Integrity) and the NOAA Plan for Increasing Public Access to Research Results.
.08 Accountability: NOAA’s R&D will be regularly evaluated and adjusted based on objective reviews. Those responsible and accountable for R&D activities will be assigned authority to manage and direct the efforts. Accountability for NOAA’s R&D portfolio rests with the NOAA Chief Scientist in collaboration with Line Office Assistant Administrators (AAs).
SECTION 4. DEFINITIONS.
.01 Chief Scientist: A Presidential appointee serving as senior scientist for NOAA. The Chief Scientist drives policy and program direction for science and technology priorities.
.02 Development: The systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, which is directed to producing new products or processes or to improving existing products or processes (Organization for Economic Co-operation and Development [OECD], 2015).
.03 Evaluation: A study conducted periodically or on an ad hoc basis to assess how well a program is working against specified benchmarks or standards. Evaluation teams often include experts external to the program to ensure independence.
.04 External Research and Development (R&D): R&D conducted by any entity outside of NOAA (e.g., Cooperative Institute, academic institution, state or local government entity, other federal agency, etc.).
.05 Internal Research and Development (R&D): R&D conducted at NOAA facilities and/or by NOAA employees.
.06 NOAA Invention: A new, useful process, machine, manufacture or composition of matter, or any new and useful improvement to a process, machine, manufacture or composition of matter, developed by NOAA.
.07 Peer Review: A widely used, time-honored practice in the scientific and engineering community for judging and potentially improving a scientific or technical plan, proposal, activity, program, or work product through documented critical evaluation by individuals or groups with relevant expertise who had no involvement in developing the object under review (NRC, 2000).
.08 Readiness Levels (RLs): A systematic project metric or measurement system that supports assessments of the maturity of R&D projects for transition from research to operation, application, commercial product or service, or other use and allows the consistent comparison of maturity between different types of R&D projects. (Note: NOAA's RL's are similar to Technology Readiness Levels developed by NASA (Mankins, 1995) and embody the same concept for quantifying the maturity of research). A program may include projects at different RLs depending on the goals of each project. Inventions may be generated at any RL. NOAA’s Policy on Research and Development Transitions can be found in NAO 216-105A.
There are nine RLs as follows:
- RL 1: Basic research, experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view. Basic research can be oriented or directed towards some broad fields of general interest, with the explicit goal of a range of future applications (OECD, 2015);
- RL 2: Applied research, original investigation undertaken in order to acquire new knowledge. It is, however, directed primarily towards a specific, practical aim or objective. Applied research is undertaken either to determine possible uses for the findings of basic research or to determine new methods or ways of achieving specific and predetermined objectives (OECD, 2015);
- RL 3: Proof-of-concept for system, process, product, service or tool; this can be considered an early phase of experimental development; feasibility studies may be included;
- RL 4: Successful evaluation of system, subsystem, process, product, service or tool in laboratory or other experimental environment; this can be considered an intermediate phase of development;
- RL 5: Successful evaluation of system, subsystem process, product, service or tool in relevant environment through testing and prototyping; this can be considered the final stage of development before demonstration begins;
- RL 6: Demonstration of prototype system, subsystem, process, product, service or tool in relevant or test environment (potential demonstrated);
- RL 7: Prototype system, process, product, service or tool demonstrated in an operational or other relevant environment (functionality demonstrated in near-real world environment; subsystem components fully integrated into system).
- RL 8: Finalized system, process, product, service or tool tested, and shown to operate or function as expected within user's environment; user training and documentation completed; operator or user approval given;
- RL 9: System, process, product, service or tool deployed and used routinely.
.09 Research: Research can be classified as basic research or applied research.
- Basic Research: Basic research is experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view. Basic research can be oriented or directed towards some broad fields of general interest, with the explicit goal of a range of future applications (OECD, 2015).
- Applied Research: Applied research is original investigation undertaken in order to acquire new knowledge. It is, however, directed primarily towards a specific, practical aim or objective. Applied research is undertaken either to determine possible uses for the findings of basic research or to determine new methods or ways of achieving specific and predetermined objectives (OECD, 2015).
.10 Research and Development (R&D) Portfolio: Research and development (R&D) occur throughout NOAA under a wide variety of arrangements including small, short-term programs that support NOAA, academic and industry researchers to answer specific, NOAA-relevant questions to multi-million dollar NOAA laboratories and science centers, and Cooperative Institutes and Science Centers engaged in long-term work critical to meeting NOAA’s mission needs. The term “NOAA R&D Portfolio” includes the full range of R&D activities undertaken by NOAA and for NOAA.
.11 Research Council: The NOAA Research Council coordinates all matters of research and development within NOAA and provides strategic advice to NOAA leadership on matters of science and research, including social sciences. The Chief Scientist serves as Chair of the Research Council and the Assistant Administrator for Oceanic and Atmospheric Research serves as Vice Chair. Each Line Office has a member on the Research Council. The Research Council Terms of Reference can be found at: http://nrc.noaa.gov/About.aspx (NRC, 2016).
.12 Science Advisory Board: NOAA’s Science Advisory Board (SAB) is a Federal Advisory Committee with responsibility to advise the Under Secretary of Commerce for Oceans and Atmosphere on long- and short-range strategies for research, education and the application of science to resource management and environmental assessment and prediction. The SAB Charter can be found at: http://www.sab.noaa.gov/Charter.aspx.
.13 Transition: The transfer of an R&D output to an operation, application, commercial product or service, or other use. NOAA’s policy on R&D transitions is defined in NAO 215-105A.
SECTION 5. POLICY.
.01 Managers and performers across the R&D portfolio will ensure that NOAA’s R&D is planned, executed and evaluated in accordance with the principles articulated in this NAO, and complies with the NAO on Research and Development Transitions (NAO 216-105A) and the NAO on Scientific Integrity (NAO 202-735D).
.02 A long-range vision is essential to a thriving R&D portfolio. The Chief Scientist, advised by the Research Council, and, if needed, in consultation with the NOAA Science Advisory Board, shall periodically update NOAA’s 20-Year Research Vision, not less than every five years.
.03 Each year the Chief Scientist, advised by the Research Council, shall issue a Strategic Research Guidance Memorandum highlighting those areas of R&D that merit special consideration for budget formulation.
.04 The Chief Scientist, advised by the Research Council, may develop plans to facilitate alignment of NOAA’s R&D with NOAA’s corporate strategic plan and the Department of Commerce strategic plan.
.05 The Research Council, chaired by the Chief Scientist, provides a venue for highlighting cross-cutting areas of need for R&D. Research Council members are expected to identify challenges and opportunities associated with R&D in their respective Line Offices and to seek areas of cross-line collaboration.
.06 Each year the Chief Scientist, advised by the Research Council, shall issue a Chief Scientist’s Report that highlights important trends in the NOAA R&D portfolio.
.07 Consistent with the principle on accountability expressed in Section 3.08, the Research Council will maintain information on NOAA’s R&D investments.
.08 Evaluation of NOAA’s R&D activities will include regular, independent peer reviews performed at least every five years. These reviews shall assess R&D activities for quality of the science, as well as how well the activities meet NOAA’s mission needs and/or requirements (i.e., relevance and performance). These reviews shall be separate from and not duplicative of existing reviews for grants, cooperative agreements, contracts, purchase orders, interagency agreements, or project agreements. Science managers may seek assistance from NOAA’s Science Advisory Board (SAB) to conduct or participate in laboratory, cooperative institute, or program reviews, in accordance with their charter and concept of operations (SAB, 2015). The outcome of these reviews shall be communicated to the NOAA Chief Scientist, via the Research Council.
.09 Free and open scientific communication is a fundamental element of the NOAA Scientific Integrity Policy (NAO 202-735D: Scientific Integrity). NOAA’s dissemination of research results adheres to and is fully responsive to all appropriate statutes and guidance, including but not limited to: Departmental Administrative Order (DAO) 219-1: Public Communications; the Information Quality Act (Section 515 of Public Law 106-554) and NOAA’s Plan for Public Access to Research Results (NOAA, 2015). The NOAA Research Council shall develop and maintain guidance for internal review and approval of fundamental research communications to assist participants in NOAA R&D in complying with appropriate statutes and guidance.
.10 This NAO and its associated procedural handbook will be periodically reviewed and reconfirmed or revised, as needed.
SECTION 6. GOVERNANCE AND RESPONSIBILITIES.
.01 The Under Secretary of Commerce for Oceans and Atmosphere and the NOAA Chief Scientist shall provide top management support for implementation of this Order. Furthermore, the NOAA Chief Scientist will:
- Drive policy and program direction for science and technology priorities for the NOAA R&D portfolio;
- Chair the NOAA Research Council;
- Lead the implementation of this NAO to continually strengthen and optimize the quality, relevance, and performance of NOAA’s R&D portfolio;
- Be accountable to the Undersecretary for maintaining an optimized R&D portfolio;
- Provide direction to the Research Council and Line Offices regarding execution of this NAO; and
- Champion the NOAA R&D portfolio in interactions with NOAA, DOC, OMB, the Congress, other Federal agencies, other parties, and the public.
.02 The NOAA Research Council will assist the NOAA Chief Scientist in execution of this NAO, and the Assistant Administrator for the Office of Oceanic and Atmospheric Research will provide additional support for execution, including providing a Vice Chair for the Research Council. The Research Council will:
- Advise the Chief Scientist concerning R&D priorities for budget formulation;
- Maintain a website to provide access to relevant R&D plans, reviews and reports;
- Develop, review, approve and promulgate a Procedural Handbook as needed to facilitate implementation of this NAO;
- Ensure the intent of this NAO is being fulfilled.
.03 Line Office AAs and the Director, Office of Marine and Aviation Operations (OMAO), will:
- Ensure R&D activities comply with the intent of this NAO;
- Oversee input to the Research Council for plans and reports; and
- Ensure evaluations are conducted and reported in accordance with this NAO.
.04 In accordance with their charter and concept of operations (SAB, 2015), the NOAA SAB will conduct activities which support the intent of this NAO, including:
- Advise the Under Secretary on strategies for research and application of science to operations and information services, so as to better understand and predict changes in the Earth’s environment and conserve and manage coastal and marine resources to meet the Nation’s economic, social and environmental needs;
- Advise the Under Secretary on other matters upon request;
- Submit reports to the Under Secretary; and
- Conduct or participate in laboratory, cooperative institute, and program reviews, as requested.
SECTION 7. REFERENCES.
.01 Mankins, John C. (6 April 1995). "Technology Readiness Levels: A White Paper" (PDF). NASA, Office of Space Access and Technology, Advanced Concepts Office. http://www.hq.nasa.gov/office/codeq/trl/trl.pdf.
.02 NOAA Administrative Order 202-735D: Scientific Integrity, effective 7 December 2011: http://www.corporateservices.noaa.gov/ames/administrative_orders/chapter_202/202-735-D.html.
.03 NOAA Administrative Order 216-105A: Policy on Research and Development Transitions, effective 3 December 2016: http://www.corporateservices.noaa.gov/ames/administrative_orders/chapter_216/216-105A.html.
.04 NOAA (2015). NOAA Plan for Increasing Public Access to Research Results issued February 2015: http://docs.lib.noaa.gov/noaa_documents/NOAA_Research_Council/NOAA_PARR_Plan_v5.04.pdf
.05 OECD (2015), Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and Experimental Development, The Measurement of Scientific, Technological and Innovation Activities, OECD Publishing, Paris. DOI: http:/ldx.doi.org/10.1787/9789264239012-enoffsite link.
.06 NRC (2016): The current Terms of Reference for the NOAA Research Council were approved on 11 January 2016: http://nrc.noaa.gov/sites/nrc/research_council_TOR_final.pdf These terms of reference are reviewed and potentially revised every two years.
.07 SAB (2015): The current Charter for NOAA’s Science Advisory Board was approved on 27 June 2015: http://www.sab.noaa.gov/Charter.aspx. This Charter is reviewed and potentially revised every two years.
SECTION 8. EFFECT ON OTHER ISSUANCES.
.01 This Order supersedes NOAA Administrative Order 216-115 dated July 20, 2011.
.02.The Under Secretary of Commerce for Oceans and Atmosphere signs because there is no delegation of authority for this NAO.
An electronic copy of this Order will be posted in place of the superseded Order on the NOAA Office of the Chief Administrative Officer website under the NOAA Administrative Issuances Section.
Under Secretary of Commerce
for Oceans and Atmosphere
Office of Primary Interest:
Office of Oceanic and Atmospheric Research