Content

B-WET national evaluation frequently asked questions

Questions about the B-WET national evaluation system are answered here.

Contents:

FAQs for all grantees

  1. Am I required to participate in the evaluation system?
  2. What will I be asked to do?
  3. Can I change or add other contacts from my organization to the evaluation system?
  4. What type of questions are on the grantee survey and the two teacher surveys?
  5. I don’t provide formal professional development for teachers, only student activities. Can I still participate in this project?
  6. Is my grant project being evaluated for performance as part of this evaluation system?
  7. Can I review the questionnaires ahead of time?
  8. Does this evaluation replace the project evaluation I proposed as part of my grant?
  9. I missed the overview webinar. Is there any place I can review this information?
  10. I’m also a NOAA grantee as a National Estuarine Research Reserve (NERRS) and already report to NOAA. Do I still need to participate in the B-WET evaluation system?
  11. Who can I contact for more information?

For projects with teacher audiences:

  1. What information will you request about my teacher participants?
  2. What if I don’t have my teacher participants identified when I receive the request for their contact information?
  3. How do I define a cohort of teachers?
  4. How do I determine the "end date" of my professional development?
  5. What if I work with the same group of teachers over several years?
  6. May I contact my teacher participants myself to notify them about this evaluation system?
  7. What if my teacher participants don’t want to participate?
  8. How can I monitor my teachers’ participation if the survey is anonymous?
  9. How can I ensure a good response rate for my teacher participants?
  10. Some of my teacher participants have email addresses that include their names. How will their anonymity/identities be protected?;
  11. How will I know when my teachers participate, and how do I get access to the results for my project?
  12. Will I have access to the raw data for my project?
  13. Do I need to go through my Institutional Review Board (IRB) approval process for this data collection?
  14. Can I link to your survey from my own? Is there a way I can include your survey questions in my own evaluation? Can I administer a paper survey?
  15. How do I update teacher contacts or professional development dates that I’ve already submitted?

For projects with student audiences:

  1. How should I collect student data?
  2. How should I use the Student Item Bank?

 

FAQs for all grantees

  1. Am I required to participate in the evaluation system?

No, participation is strongly encouraged but optional for all B-WET grantees. We anticipate that this evaluation system will have short-term and long-term benefits for programs, funders, and the field of environmental education broadly. The results will provide insight into how we can improve the NOAA B-WET program and tailor our services to you and the teachers and students you serve. Your input is valued and will contribute to a better B-WET program in the future, so please consider participating to the extent possible for your grant. Back to top

  1. What will I be asked to do?

All grantees focused on MWEE implementation with teachers and students and related professional development will be asked to complete an online survey at the end of your grant (or once per year if your grant is multiple years). This includes grantees who work exclusively with students. Grantees who are funded under a special priority area, such as Capacity Building, may not be surveyed since the evaluation system is not designed to collect information on activities outside of traditional MWEE implementation.

In addition, if you have a Teacher Professional Development or Exemplary grant, we will ask you to fill out an online teacher contact information form to provide contact info for the teachers who participate in your professional development. You will be asked for the professional development end date, your contact information and participating teachers’ email addresses. No other teacher contact information is requested. The teachers will be asked to complete an online survey following their professional development, and another at the end of the following school year. Please see the Grantee instructions on the National Evaluation System page for more information. Back to top

  1. Can I change or add other contacts from my organization to the evaluation system?

Due to the nature of the evaluation system, it is not possible to maintain multiple contacts for each grant. Grantees should identify one primary point of contact for the evaluation. That point of contact may always forward our email request to other partners so they can respond to the teacher contact information survey. Only one grantee survey is requested per year, per grant, so if necessary we ask that partners collaborate on the response. If the person who has been receiving the evaluation system emails is not the appropriate point of contact, please provide updated contact information to the B-WET National Coordinator: Bronwen Rice, 202-482-6797, E-mail: Bronwen Rice. Please include your grant number in any information change requests. Back to top

  1. What type of questions are on the grantee survey and the two teacher surveys?

The survey grantees complete at the end of the grant year asks questions about grantee satisfaction with the B-WET program and grantee’s needs for additional support, the project implementation strategies used by grantees (e.g., length, location, goals), the type of content addressed in the grantee’s project, instructional practices used during the grantee’s project, the grantee’s perceptions of teacher and student outcomes, and the evaluation practices used for the grantee’s project.

The survey completed by teachers at the end of their B-WET-funded professional development asks questions that measure teachers’ perceptions of the way the professional development was implemented, changes in teachers’ knowledge of watershed concepts, changes in teachers’ confidence toward MWEE instruction, and teachers’ intentions to change their instructional practice.

Finally, the survey teachers will complete at the end of the school year following their professional development asks them about the strategies teachers used to implement MWEEs with their students (e.g., length, location, goals), the content addressed during the MWEEs, the instructional practices teachers used during MWEEs, and the teacher’s perceptions of student outcomes.

See the National Evaluation System pageto review the survey instruments. Back to top

  1. I don’t provide formal professional development for teachers, only student activities. Can I still participate in this project?

Yes. The grantee survey is for all grantees working on MWEE implementation. However, we will not need you to complete the teacher contact information survey and will not collect data from teachers associated with your project. We are not currently collecting data at the student level as part of the national evaluation system, though those data are valuable and should be reported as part of your regular grant reports, as applicable. We do have a Student Item Bank with guidance that is completely optional and is available as a resource to you and your teacher participants. The student item bank can be found on the National Evaluation System pageBack to top

  1. Is my grant project being evaluated for performance as part of this system?

No. NOAA will not review the data at the individual grant project level, but will only look at it regionally or nationally. The intent of the evaluation system is to review implementation practices and outcomes for the program as a whole, not performance of any single grant project, grantee, or teacher. Back to top

  1. Can I review the questionnaires ahead of time?

All of the questionnaires, as well as the entire evaluation plan, are available for your review at any time on the National Evaluation System pageBack to top

  1. Does this evaluation replace the project evaluation I proposed as part of my grant?

No, this request is separate and if you choose to participate in the evaluation system, you will still be responsible for completing the evaluation you proposed as part of your grant project. While you will have access to teachers’ results for your project, because this system is looking at common elements across the seven regions in the national B-WET program, it may not provide the project-specific information you need to answer all your questions about your project. In future years we hope to provide applicants with more information during the application period so that they may better incorporate these two levels of evaluation into their projects. Back to top

  1. I missed the overview webinar. Is there any place I can review this information?

The presentation slides are available for your review on the National Evaluation System pageBack to top

  1. I’m also a NOAA grantee as a National Estuarine Research Reserve (NERRS) and already report to NOAA. Do I still need to participate in the B-WET evaluation system?

You will still be asked to participate in the B-WET national evaluation system for the B-WET funded components of your NERRS work. NOAA has developed guidance specifically for NERRS grantees to help navigate the multiple requests for information. If this applies to you,contact your regional B-WET program managerfor more information. Back to top

  1. Who can I contact for more information?

You may contact your regional B-WET program manageror the B-WET National Coordinator: Bronwen Rice, 202-482-6797, E-mail: Bronwen RiceBack to top

 

For projects with teacher audiences:

  1. What information will you request about my teacher participants?

We will request teachers’ email addresses only. We will not ask for their names. NOAA will not share the email addresses or use them for any other purpose. We will also ask you for some project specific information, such as the expected start and end dates of your grant project activities. Back to top

  1. What if I don’t have my teacher participants identified when I receive the request for their contact information?

The online teacher contact information form can be completed at any point, and can be submitted multiple times for the same grant. Grantees will receive a request with the link to the form monthly, and are asked to complete it each time they have a new cohort of teachers identified. If you have no new information to provide you may disregard these monthly reminders. Back to top

  1. How do I define a cohort of teachers?

A cohort of teachers is defined as appropriate for your project. A single cohort should all participate in activities associated with a single grant award, and those activities should be completed around the same time. For example, if you work with one group for a summer workshop only, and another for the summer workshop and follow-up school year training, it would make sense to define these as two cohorts. The first would receive their survey following the workshop, the second following completion of all of the follow-up activities. Note that the grantee is required to provide the anticipated end date for professional development with a given cohort. Back to top

  1. How do I determine the "end date" of my professional development?

Grantees will be required to provide the anticipated end date for the entire professional development activity for each cohort of teachers. For the purposes of this evaluation system, we define the end date of the professional development as when all of your interactions with that group of teachers are complete. For example, if you offer an intensive summer workshop, but then also provide in-classroom support throughout the school year, the end date should be the end of the school year, once all of your support activities are likely to be complete. It is OK to estimate this date, but your thoughtful consideration of this will help ensure that we are able to collect information about your entire program intervention.

Note that all of these are considered to be B-WET professional development that you might be providing to teachers. The "end date" is when all of your professional development ends for this cohort of teachers.

  • One day workshops (usually less than 8 hours)
  • Teacher institute (usually on consecutive days that cumulatively consist of 40 hours or more
  • Multi-day workshops (events that last at least 6 hours, but are less than 40 hours, e.g., a three-day workshop on a specific topic or a series of five Saturday sessions)
  • A college-level course
  • Professional development provider training (training for individuals who provide teacher professional development)
  • Individual teacher coaching and support (e.g., curriculum planning, shared teaching, demonstrations and/or other forms of in-school or in-field support)
  • On-line professional development support (e.g., courses, webinars, discussion forums)

Back to top

  1. What if I work with the same group of teachers over several years?

If you work with the same cohort of teachers over several years of a multi-year grant, please list the end date of the entire professional development interaction, even if it is several years in the future. You will receive the grantee survey yearly, but teachers should only receive the teacher professional development survey at the end of their entire period of work with your organization under this award.

The only exception to this guidance is if you are working on a project that, while it may be planned as a multiple year project, has only been awarded funding for a specific period. Grantees should only plan for the period of their project that has been awarded and set professional development end dates accordingly. If a project continuation is granted and you plan to continue working with the same cohort(s) of teachers, please contact the B-WET national coordinator (Bronwen Rice, 202-482-6797, E-mail: Bronwen Rice) or your regional B-WET program manager to discuss how to update the professional development end dates for any contacts you have already provided. New cohorts may always be submitted through the Teacher Contact Info Survey. Back to top

  1. May I contact my teacher participants myself to notify them about this evaluation system?

Yes, and you are strongly encouraged to do so to help ensure teacher participation! Sample notification text is provided on the National Evaluation System page. NOAA will also provide this information when we contact teachers directly, but if you want to contact them ahead of time, you may. This is also an opportunity for teachers to opt out of the data collection to you, prior to you submitting their contact information to us. Back to top

  1. What if my teacher participants don’t want to participate?

The surveys include the option to opt out of the project at any time. NOAA will not share teacher contact information (email address) or use it for any other purpose. In addition, you may choose to provide information about this data collection to your teacher participants, and allow them to opt out to you prior to submitting their contact information to us. Sample notification text is provided on the National Evaluation System pageBack to top

  1. How can I monitor my teachers’ participation if the survey is anonymous?

Teachers will receive a thank you email after they complete the post professional development questionnaire, which will be sent within 30 days of the end date you provided with their contact information. You may ask the teachers to forward that thank you email to you as proof of their participation. The results you may view in your project report are in aggregate, so you won't be able to identify individual teacher responses, maintaining anonymity. Back to top

  1. How can I ensure a good response rate for my teacher participants?

Grantees are our conduit to teacher participants and can help improve evaluation results by providing teachers information about the importance of the data collection and encouraging them to respond. Here are some strategies to help ensure good response rates from your teacher participants:

  • Make sure the professional development end date you provide is as accurate as possible.
  • Communicate about this project with your teachers in advance. Sample notification text is provided on the National Evaluation System page.
  • Discuss the importance of this data, both for your information as the project implementer, and for B-WET program improvement at the national level, to motivate responses.
  • Introduce the term “MWEE” with teachers so they are familiar with it when they see it in the surveys (though it is defined there for them as well.)
  • Provide incentives. Teachers will receive a thank you email after they complete the post professional development questionnaire. They may then forward that email to you as proof of participation. If you are able to provide an incentive, such as credit or a stipend for participating in your grant project, then you could include their participation in this data collection as one of the incentive requirements. Another option would be to offer a thank you “gift” (small token, inclusion in a raffle, etc.) for proof of participation.

These strategies will help B-WET achieve the response rates we need and we strongly encourage grantees to integrate participation in this evaluation system into their formal project requirements for teachers whenever possible. Reach out to your regional B-WET program manager to discuss specific strategies for your project. Back to top

  1. Some of my teacher participants have email addresses that include their names. How will their anonymity/identities be protected?

Only the evaluation system administrator will have access to the teachers’ email addresses when they are submitted, and individual teacher responses will not be associated with their email addresses. Our policy is to protect teachers identities, so those emails will never be used for any other purpose than this data collection. Back to top

  1. How will I know when my teachers participate, and how do I get access to the results for my project?

You will receive individualized, password-protected report links (one for the post-professional development survey responses, and one for the follow-up MWEE survey) when you are contacted by NOAA to participate in the project, and these will be resent to you monthly in a reminder email. You may visit these links at any time to review results associated with your project. This report will update automatically as teachers participate. You will not be able to identify teachers individually, but will only see the aggregate results for your project. Back to top

  1. Will I have access to the raw data for my project?

It may be possible to access the anonymous raw data for your grant project (beyond the report you will receive). If you need access to these data, please contact B-WET program staff to inquire about this. Back to top

  1. Do I need to go through my Institutional Review Board (IRB) approval process for this data collection?

Maybe. If you went through an IRB process for your grant project, we recommend contacting your IRB office to see if you need to get any additional clearance to participate in this evaluation system. If you would like to publish results in a peer-reviewed journal, then it is very likely that you will need to get formal IRB approval. Note that this data collection plan has gone through the federal Office of Management and Budget (OMB) Paperwork Reduction Act (PRA) process (OMB Control Number: 0648-0658 Expires: 01/31/2019). If you need to pursue IRB approval, B-WET program staff can provide more information about the project to help facilitate this. Back to top

  1. Can I link to your survey from my own? Is there a way I can include your survey questions in my own evaluation? Can I administer a paper survey?

No. The evaluation system surveys are set up so that teachers must receive individual links generated by the data collection system, so we are unable to provide a single link for you to distribute to all of your participants. Likewise, we cannot import data collected in other ways, so we are unable to take data collected as part of other surveys, or via paper forms. Back to top

  1. How do I update teacher contacts or professional development dates that I’ve already submitted?

If the professional development end date or other information changes after submission, you should contact Bronwen Rice, B-WET National Coordinator (Bronwen Rice, 202-482-6797, E-mail: Bronwen Rice) with updates. Please include your grant number in any information change requests. The B-WET National Coordinator will need to work with you to make the changes in the system. For this reason, we'd ask that we only make edits if there are significant changes. If the professional development end date shifts by only a few days or even a week or two it isn't generally an issue, since we send teacher emails out on a monthly basis. If teacher participants drop they will always have the opportunity to opt-out of the emails, and to add new teachers you can just complete the survey again. Back to top

 

For projects with student audiences:

  1. How should I collect student data?

NOAA provides an optional Student Item Bank: elementary (grades 4-5) and secondary (grades 6-12) measures developed by the University of Michigan and eeEvaluations to assess students’ science learning, watershed literacy, and environmental stewardship outcomes. The Student Item Bank is provided on the National Evaluation System pageBack to top

  1. How should I use the Student Item Bank?

The B-WET program provides guidance on the use of the student item bank. For the time being its use is completely optional and is available only as a resource to you and your teacher participants. These data will not be collected as part of the national evaluation system, though they are valuable and should be reported as part of your regular grant reports, as applicable. The Student Item Bank is provided on the National Evaluation System pageBack to top