Common measures: Educators

Educators engage people in science and inspire them to use science and data to make informed decisions. Informal and formal educators are one of our core audiences. Through educator resources and professional development, we give educators the tools they need to teach about the ocean and atmosphere and use environmental data and the process of science in their lessons. The output measure captures three numbers: 1) the number of educators reached by our programming; 2) the number of educators who intend to use what they learned in their teaching settings; and 3) the number of educators who actually use what they learned in their teaching settings.

Common measures

Output: Number of educators who participated in NOAA-supported professional development programs.

Outcome: Number of educators who intend to integrate what they learned in NOAA-supported professional development programs.

Definitions

Professional development programs: An organized set of educational activities that are designed: 1) for educators as the primary target audience, 2) to enhance their understanding of NOAA-related topics, and 3) to provide guidance on how to integrate knowledge, skills, and NOAA educational and scientific resources to educate others.

  • Examples include all-day and multi-day workshops, teacher research experience programs, online courses, outdoor field experiences. Presentations at conferences should be reported if the intention of the presentation is to meet a professional development objective with respect to educational content or pedagogy
  • Does NOT include presentations that just describe NOAA offerings or other marketing types of outreach activities.

Educators: Those who facilitate learning in various roles: public school teachers, private or independent school teachers, informal educators, interpreters, volunteers serving as docents or educators, homeschool educators, or preservice teachers.

  • The number of educators is the cumulative number of participants in professional development programs that were offered in the federal fiscal year. The performance measure reported is the sum of all Education Council member reported counts.
  • Educators can be counted twice if they were present at two distinct professional development programs. Educators can be counted once per year for programs that occur over multiple years.
  • Educators cannot be counted twice when there is one-to-one consultation or mentoring as a follow up to a professional development program.
  • Educators should be counted as participants upon completion of the workshop, program or course, not just based on registration.
  • Educators should NOT be counted as chaperones on student programs.

NOAA-supported: Report on all programs that are provided by NOAA staff or by partner institutions with a "substantial involvement" by NOAA. Substantial involvement includes funding, facilities, or staff time; in other words convened, coordinated, co-developed, funded, presented or co-presented by NOAA staff. Program content or design must involve NOAA-related science, NOAA data, or other NOAA assets.

Intend to integrate: The educator plans to deliver or use knowledge, skills or NOAA scientific or educational resources as part of their educational activities, instruction, curriculum or informal learning environments within a year of program completion.

    NOAA resources: Educational and scientific products and services that can be used in educational settings to educate others about NOAA and disciplines that support NOAA's mission (ocean, coastal, Great Lakes, weather, and climate sciences).

    • Examples of scientific products and services include scientific data, tools, websites, books, place-based learning areas, field sites, ships, aircraft, labs, or connections to NOAA scientists and other personnel.
    • Examples of educational products and services include websites, curriculum, lesson plans, tools, games, books, field sites, exhibits, in-person and online programs, connections to NOAA educators.

    Data collection and reporting

    Leading outcome data collection: Collect data from all participants at the end of program completion using the standardized item or common forms.

    Do you plan to integrate [any of the resources from workshop] into your work within the next year?
    * I do not plan to [use them].
    * I plan to [use them].
    * I am not sure yet whether I will [use them].

    Standardized items: Common questions that have been designed to collect consistent outcome data that can be aggregated across units. Each question can be modified as needed based on the program content. The parts of the question that can be customized are in [square brackets].

    • Standardized items should be embedded as part of an existing instrument or can be asked on their own.
    • Collect responses from all participants or as many as possible.
    • For programs that present multiple resources, a matrix item type is recommended with individual components listed for each row and the response options as the column header.
    • Common Forms (in development): Common survey forms are being designed in order to aid implementation of the outcome data. Questions include the standardized items, follow up questions based on the response to the standardized item, typical questions based on the type of program delivery, and demographics.

    Administrative records: Staff for each NOAA-supported professional development program are responsible for maintaining administrative records that document the number of participants for the current fiscal year.

    • B-WET exception: National Estuarine Research Reserves (NERRS) and Sea Grant can be grantees of other NOAA grant programs, including the Bay-Watershed Education and Training (BWET) Program and the Environmental Literacy Program. As a result, we acknowledge that there will likely be some double-counting of data reported.

    Unified Data Call: An annual collection of output and outcomes measures and outyear estimates data from each Education Council member seat. Data collected in the Unified Data Call is used for both internal NOAA reporting purposes, which includes the Annual Operating Plan (AOP), and for external reporting purposes, including NOAA Education Annual Accomplishments Report and Congressional briefings.

    History

    • FY2010: This output measure was first reported.
    • FY2011: The following two measures were reported: 1) Number of teachers and educators participating in in-person professional development that integrates NOAA sciences, data and other information, 2) Number of teachers and educators participating through distance learning in professional development that integrates NOAA sciences, data and other information.
    • FY2012-2013: The following measure was approved by the Education Council on August 31, 2012 and reported as: Number of educators who participated in professional development programs that enhance understanding and use of ocean, coastal, Great Lakes, weather, and climate information to promote stewardship and to increase informed decision making.
    • FY2014: The output measure was modified on March 24, 2014 to align with annual accomplishments for FY13.
    • FY2015: The outcome measures were first collected and reported.
    • FY2016: The outcome measures were reported as a number rather than a percent. 
    • FY2017: The definitions were revised for clarity and published on the NOAA Education website.
    • FY2020: The Education Council decided on March 18, 2020 to stop collecting and reporting on the following lagging measure: Number of educators who integrated what they learned in NOAA-supported professional development programs. Guidance language has been moved into the Footnotes section of this document.

    Footnotes

    • PRA (Paperwork Reduction Act) clearance is not required if the grantee, on their own, decided to use an information collection to achieve a broader goal defined in the grant language. PRA clearance is required when you, "as a representative of the Federal government," obtain facts or opinions from ten or more persons by the use of standard questions presented in forms, telephone or personal interviews, websites, requests for narrative responses to questions, or almost any other means. If a grant or contract is specifically given to conduct an information collection and/or you must approve any plans for an information collection, then you are the sponsor of an information collection conducted by a grantee.
    • Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38, 181-199.
    • Lagging outcome: Number of educators who integrated what they learned in NOAA-supported professional development programs.  Integrated: The delivery or use of knowledge, skills or resources as part of educational activities, instruction, curriculum or informal learning environments within a year of program completion. 1) Minimum level of use is one activity once for one group of learners. 2) Program design may include stipends and professional support to ensure use. 3) NOAA-related topics: A set of knowledge and competencies about NOAA and disciplines that support NOAA's mission (ocean, coastal, Great Lakes, weather, and climate sciences).

    • Lagging Outcome Data Collection: Collect data from all participants on programs implemented in the prior federal fiscal year using the standardized item or common forms. The period of time following program completion can be a maximum of one year, this will vary by program and depends on the program theory. Three and six-month intervals are common. PRA clearances are needed for programs delivered by NOAA staff. For additional guidance see footnote 1. Since the [workshop], have you integrated [lessons, content, activities from the workshop]? * I have not [used them]. * I have not but I still plan to use them. * I have [used them in the past year]. * I plan to learn more before [using them]. * I am not sure yet whether I will [use them].