{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "7w5yf_G4nA3G" }, "source": [ "# Training Random Forest Regression model for NOAAOffshoreWindProfiles-USA\n", "\n", "* Creator(s): James Frech (1,2), Huai-Min Zhang (2)\n", "* Affiliation(s):\n", " * 1. National Centers for Environmental Information ([NCEI](https://www.ncei.noaa.gov/)), National Oceanic and Atmospheric Administration (NOAA),\n", " * 2. Earth System Science Interdisciplinary Center (ESSIC), Cooperative Institute for Satellite Earth System Studies (CISESS), University of Maryland, College Park, MD, USA, 20740\n", "\n", "---\n", "\n", "\n", "## Overview\n", "This notebook covers the process of training and validating the random forest regression (RFR) model used to generate the NOAA Offshore Wind Profiles - USA dataset. The random forest is a popular and powerful machine learning method that averages predictions over many decision trees to make a final ensemble prediction. The random forest in particular works well for regression and classification problems with nonlinear tabular data. More information on the model can be found online through many resources (simple: https://towardsdatascience.com/random-forest-regression-5f605132d19d/; original publication: https://www.stat.berkeley.edu/~breiman/randomforest2001.pdf). In this notebook, the random forest model is trained using cross validation to tune hyperparameters, tested at independent locations, and applied to new data to generate new wind profiles.\n", "\n", "The problem that the NOAA Offshore Wind Profiles is designed to solve is the lack of available wind speeds around the rotor swept layer of wind turbines. This knowledge is important for being able to predict and forecast the power output of a given turbine. Many datasets contain surface wind speeds, including our surface NOAA Blended Seawinds (NBSv2), however these winds must be extrapolated vertically for optimal use in the offshore wind energy industry. There are a few tradition methods for this purpose, however, in recent years machine learning has shown to outperform these methods. This project seeks to make use of surface conditions (surface winds, air temperature, sea surface temperature) to extrapolate full vertical wind profiles up to 200 m. The surface wind speed is chosen as an input due to its obviously high correlation with the wind speeds in the wind profiles above, and the difference in air temperature and sea surface temperature is chosen as an input variable to provide an estimate of atmospheric stability which affects the shape of the vertical wind profiles. Our target variables are the wind speeds from 40 m to 200 m (20 m intervals).\n", "\n", "### Prerequisites\n", "Moderate knowledge of the python programming language is necessary for understanding this notebook. In addition experience in machine learning and knowledge of the RFR in particular would be very helpful for understanding the notebook. The NOAA Blended Seawinds notebook would also be helpful as an introduction for the dataset used to generate wind profiles in this notebook.\n", "\n", "### Learning Outcomes\n", "* Learn how to create and tune a random forest model.\n", "* Use the multiprocessing module to run tasks in parallel.\n", "* Apply the random forest model to generate new data.\n", "\n", "### Time Estimate\n", "* Estimated text reading time: 10 to 20 min\n", "* Estimated code reading time: 10 to 20 min\n", "* Estimated total reading time: 30 to 40 min\n", "---\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "HK8zv84EpPmN" }, "source": [ "## Tutorial\n", "The tutorial has four sections.\n", "1. Tuning hyperparameters for the random forest regression model.\n", "2. Training the final model with optimal parameters.\n", "3. Applying the model to test data and checking model accuracy.\n", "4. Implementing the model on the NOAA Blended Seawinds gridded dataset." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 0: Install and import libraries" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before running the code, we must make sure our environment contains and imports the necessary python libraries" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# import libraries and install them if necessary\n", "import subprocess, sys\n", "packages = [\"pandas\", \"xarray\",\"numpy\",\"scikit-learn\",\"matplotlib\",\n", " \"cartopy\",\"itertools\",\"tqdm\",\"random\"]\n", "for package in packages:\n", " try:\n", " __import__(package)\n", " except ImportError:\n", " subprocess.check_call([sys.executable, '-m', 'pip', 'install', package])" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# Import packages for notebook, taking just parts or using aliases\n", "import pandas as pd\n", "import xarray as xr\n", "import numpy as np\n", "from sklearn.ensemble import RandomForestRegressor\n", "from tqdm import tqdm\n", "import matplotlib.pyplot as plt\n", "import itertools\n", "import cartopy.crs as ccrs\n", "import cartopy.feature as cfeature\n", "import cartopy.mpl.ticker as cticker\n", "from random import seed\n", "import zipfile\n", "import io\n", "import requests" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set the random seed for reproducibility. \n", "Note: Results won't be exact in the end due to other packages not using this seed, but good practice anyways." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "seed(1234)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Step 1: Tuning Hyperparamters for the RFR" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Tuning hyperparameters is an important task when creating any machine learning model as it affects the performance of the model. In the random forest, there are a few important hyperparameters to tune. The first is the number of trees in the model. If not enough trees are chosen to build the model, the model may not be complex enough to learn all the patterns in the data to create accurate predictions, in which case the model will be underfitting. The second parameter is the minimum number of samples to be considered at each split within the model. This parameter helps with regularization and stops the trees from growing too deep to avoid overfitting. Alternatively, the max depth of the trees can be tuned. Third, tuning the number of inputs allowed to be considered to create an optimal split will create more randomness in the model and ensures that the trees don't all build similarly only utilizing the most important features. This part of the tutorial will focus on tuning these parameters." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Before doing anything else, the data must be downloaded from zenodo. All data used in the notebook is stored in this Zenodo record (https://doi.org/10.5281/zenodo.15830540)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Downloading the data from Zenodo record\n", "url = 'https://zenodo.org/api/records/15830540/files-archive'\n", "with zipfile.ZipFile(io.BytesIO(requests.get(url).content)) as zfile:\n", " zfile.extractall()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Read in the data here. The data provided consists of cleaned data from five different offshore floating lidar buoys: Hudson North, Hudson South, Hudson South West, Humboldt, and Morro Bay locations. The file contains columns for (1) the location of the data, (2) the time of the observation (10 minute resolution), (3) the surface wind speed at 10 m above sea level (w10), (4) the difference between surface air temperature and sea surface temperature, (5) the vertical wind speed profiles between 40 m and 200 m (w40, w60, ..., w200)." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| \n", " | location | \n", "time | \n", "w10 | \n", "temp_diff | \n", "w40 | \n", "w60 | \n", "w80 | \n", "w100 | \n", "w120 | \n", "w140 | \n", "w160 | \n", "w180 | \n", "w200 | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "Hudson South | \n", "2019-09-04 00:00:00 | \n", "3.679043 | \n", "-1.1400 | \n", "3.8218 | \n", "3.8193 | \n", "3.8479 | \n", "3.8223 | \n", "3.8095 | \n", "3.8747 | \n", "3.9162 | \n", "3.9656 | \n", "4.0334 | \n", "
| 1 | \n", "Hudson South | \n", "2019-09-04 00:10:00 | \n", "3.727250 | \n", "-1.1100 | \n", "3.8689 | \n", "3.8416 | \n", "3.8418 | \n", "3.8766 | \n", "3.9057 | \n", "3.9557 | \n", "3.9679 | \n", "4.0759 | \n", "4.0526 | \n", "
| 2 | \n", "Hudson South | \n", "2019-09-04 00:20:00 | \n", "3.737587 | \n", "-1.0900 | \n", "3.9316 | \n", "3.8941 | \n", "3.9455 | \n", "3.9555 | \n", "4.0147 | \n", "4.0512 | \n", "4.0993 | \n", "4.0715 | \n", "4.1344 | \n", "
| 3 | \n", "Hudson South | \n", "2019-09-04 00:30:00 | \n", "3.680078 | \n", "-1.0400 | \n", "3.7904 | \n", "3.8552 | \n", "3.8465 | \n", "3.8560 | \n", "3.8717 | \n", "3.8937 | \n", "3.9033 | \n", "3.9032 | \n", "3.9683 | \n", "
| 4 | \n", "Hudson South | \n", "2019-09-04 00:40:00 | \n", "4.139271 | \n", "-0.9900 | \n", "4.2358 | \n", "4.3596 | \n", "4.3647 | \n", "4.3550 | \n", "4.3521 | \n", "4.3278 | \n", "4.3442 | \n", "4.3641 | \n", "4.4056 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 276572 | \n", "Humboldt | \n", "2022-06-29 21:00:00 | \n", "4.460659 | \n", "-1.6405 | \n", "3.1200 | \n", "3.2500 | \n", "3.3000 | \n", "3.3900 | \n", "3.3400 | \n", "3.1800 | \n", "2.9500 | \n", "2.6800 | \n", "2.5700 | \n", "
| 276573 | \n", "Humboldt | \n", "2022-06-29 21:10:00 | \n", "4.894301 | \n", "-1.6430 | \n", "3.6500 | \n", "3.8300 | \n", "3.9500 | \n", "4.0200 | \n", "3.9900 | \n", "3.8600 | \n", "3.6400 | \n", "3.3700 | \n", "3.2600 | \n", "
| 276574 | \n", "Humboldt | \n", "2022-06-29 21:20:00 | \n", "4.935618 | \n", "-1.5585 | \n", "3.7900 | \n", "4.1600 | \n", "4.3700 | \n", "4.3800 | \n", "4.3300 | \n", "4.2000 | \n", "3.9900 | \n", "3.7700 | \n", "3.5200 | \n", "
| 276575 | \n", "Humboldt | \n", "2022-06-29 21:30:00 | \n", "5.171281 | \n", "-1.5140 | \n", "3.7800 | \n", "3.9800 | \n", "4.0500 | \n", "4.1400 | \n", "4.1000 | \n", "4.0500 | \n", "3.9900 | \n", "3.9200 | \n", "3.7500 | \n", "
| 276576 | \n", "Humboldt | \n", "2022-06-29 21:40:00 | \n", "5.688028 | \n", "-1.4890 | \n", "3.8500 | \n", "3.9300 | \n", "3.9800 | \n", "4.1000 | \n", "4.2100 | \n", "4.2200 | \n", "4.2100 | \n", "4.1900 | \n", "4.1500 | \n", "
276577 rows × 13 columns
\n", "| \n", " | n_estimators_values | \n", "min_samples_leaf_values | \n", "max_features_values | \n", "location | \n", "OOB Score | \n", "OOB Error | \n", "
|---|---|---|---|---|---|---|
| 0 | \n", "20 | \n", "50 | \n", "1 | \n", "Hudson North | \n", "0.943946 | \n", "0.056054 | \n", "
RandomForestRegressor(max_features=1, min_samples_leaf=30, n_estimators=1000,\n",
" n_jobs=-1, oob_score=True, random_state=0)In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. RandomForestRegressor(max_features=1, min_samples_leaf=30, n_estimators=1000,\n",
" n_jobs=-1, oob_score=True, random_state=0)| \n", " | time | \n", "air_temp | \n", "SST | \n", "w10 | \n", "w40 | \n", "w60 | \n", "w80 | \n", "w100 | \n", "w120 | \n", "w140 | \n", "... | \n", "temp_diff | \n", "w40_pred | \n", "w60_pred | \n", "w80_pred | \n", "w100_pred | \n", "w120_pred | \n", "w140_pred | \n", "w160_pred | \n", "w180_pred | \n", "w200_pred | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "2021-05-14 16:30:00 | \n", "14.00391 | \n", "13.82178 | \n", "2.57422 | \n", "2.52954 | \n", "2.64124 | \n", "2.61890 | \n", "2.67102 | \n", "2.87952 | \n", "2.72314 | \n", "... | \n", "0.18213 | \n", "3.202995 | \n", "3.436669 | \n", "3.607690 | \n", "3.750191 | \n", "3.894891 | \n", "4.031077 | \n", "4.156238 | \n", "4.273528 | \n", "4.390401 | \n", "
| 1 | \n", "2021-05-14 16:40:00 | \n", "14.11133 | \n", "13.84229 | \n", "2.48486 | \n", "2.44763 | \n", "2.45508 | \n", "2.53699 | \n", "2.86462 | \n", "2.93164 | \n", "2.75293 | \n", "... | \n", "0.26904 | \n", "3.300581 | \n", "3.687703 | \n", "3.978933 | \n", "4.229158 | \n", "4.465177 | \n", "4.675429 | \n", "4.868366 | \n", "5.045922 | \n", "5.203589 | \n", "
| 2 | \n", "2021-05-14 16:50:00 | \n", "14.16504 | \n", "13.87305 | \n", "2.26147 | \n", "2.28381 | \n", "2.33594 | \n", "2.63379 | \n", "3.13269 | \n", "3.26672 | \n", "2.98376 | \n", "... | \n", "0.29199 | \n", "2.970773 | \n", "3.294819 | \n", "3.548855 | \n", "3.754991 | \n", "3.949965 | \n", "4.127610 | \n", "4.295326 | \n", "4.454996 | \n", "4.607855 | \n", "
| 3 | \n", "2021-05-14 17:00:00 | \n", "14.21875 | \n", "13.92432 | \n", "2.05298 | \n", "2.00085 | \n", "2.22424 | \n", "2.58911 | \n", "3.08057 | \n", "2.95398 | \n", "2.82739 | \n", "... | \n", "0.29443 | \n", "2.771960 | \n", "3.091511 | \n", "3.370940 | \n", "3.609623 | \n", "3.854418 | \n", "4.082239 | \n", "4.300752 | \n", "4.485152 | \n", "4.677612 | \n", "
| 4 | \n", "2021-05-14 17:10:00 | \n", "14.32617 | \n", "14.00635 | \n", "1.90405 | \n", "2.06787 | \n", "2.24658 | \n", "2.67847 | \n", "3.11780 | \n", "3.03589 | \n", "3.02100 | \n", "... | \n", "0.31982 | \n", "2.378450 | \n", "2.560306 | \n", "2.745461 | \n", "2.938255 | \n", "3.147448 | \n", "3.350747 | \n", "3.557519 | \n", "3.760784 | \n", "3.988345 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 14526 | \n", "2021-09-30 11:00:00 | \n", "18.62305 | \n", "21.18408 | \n", "6.62500 | \n", "7.01965 | \n", "7.15369 | \n", "7.01221 | \n", "6.98242 | \n", "6.94519 | \n", "7.33984 | \n", "... | \n", "-2.56103 | \n", "6.853582 | \n", "6.928769 | \n", "6.981764 | \n", "7.020855 | \n", "7.078331 | \n", "7.112477 | \n", "7.154793 | \n", "7.191700 | \n", "7.235197 | \n", "
| 14527 | \n", "2021-09-30 11:10:00 | \n", "18.73047 | \n", "21.18408 | \n", "6.32715 | \n", "6.52075 | \n", "7.10901 | \n", "7.12390 | \n", "7.33240 | \n", "7.25793 | \n", "6.98242 | \n", "... | \n", "-2.45361 | \n", "6.513777 | \n", "6.596303 | \n", "6.649153 | \n", "6.672806 | \n", "6.691013 | \n", "6.692535 | \n", "6.670843 | \n", "6.659621 | \n", "6.678562 | \n", "
| 14528 | \n", "2021-09-30 11:20:00 | \n", "18.62305 | \n", "21.19434 | \n", "5.01660 | \n", "6.34949 | \n", "6.47607 | \n", "6.62500 | \n", "6.61011 | \n", "6.72925 | \n", "6.69202 | \n", "... | \n", "-2.57129 | \n", "5.223161 | \n", "5.293555 | \n", "5.346480 | \n", "5.392743 | \n", "5.441021 | \n", "5.483677 | \n", "5.522090 | \n", "5.577330 | \n", "5.618545 | \n", "
| 14529 | \n", "2021-09-30 11:30:00 | \n", "18.99902 | \n", "20.60986 | \n", "4.45068 | \n", "6.17822 | \n", "6.46118 | \n", "6.28992 | \n", "6.53564 | \n", "6.53564 | \n", "6.83350 | \n", "... | \n", "-1.61084 | \n", "4.548341 | \n", "4.599918 | \n", "4.626269 | \n", "4.624768 | \n", "4.640347 | \n", "4.646031 | \n", "4.653937 | \n", "4.679275 | \n", "4.737834 | \n", "
| 14530 | \n", "2021-09-30 11:50:00 | \n", "19.05273 | \n", "18.30273 | \n", "1.93384 | \n", "7.89832 | \n", "6.44629 | \n", "6.69202 | \n", "7.13879 | \n", "6.99731 | \n", "6.55798 | \n", "... | \n", "0.75000 | \n", "2.824650 | \n", "3.284962 | \n", "3.626603 | \n", "3.921677 | \n", "4.222763 | \n", "4.534529 | \n", "4.847942 | \n", "5.163178 | \n", "5.493389 | \n", "
14531 rows × 24 columns
\n", "| \n", " | time | \n", "lat | \n", "lon | \n", "crs | \n", "mask | \n", "u_wind | \n", "v_wind | \n", "w10 | \n", "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", "2024-12-12 00:00:00 | \n", "15.0 | \n", "190.00 | \n", "-9999.0 | \n", "1.0 | \n", "-10.592870 | \n", "-1.297211 | \n", "10.672003 | \n", "
| 1 | \n", "2024-12-12 00:00:00 | \n", "15.0 | \n", "190.25 | \n", "-9999.0 | \n", "1.0 | \n", "-10.508510 | \n", "-1.385846 | \n", "10.599497 | \n", "
| 2 | \n", "2024-12-12 00:00:00 | \n", "15.0 | \n", "190.50 | \n", "-9999.0 | \n", "1.0 | \n", "-10.455713 | \n", "-1.541452 | \n", "10.568728 | \n", "
| 3 | \n", "2024-12-12 00:00:00 | \n", "15.0 | \n", "190.75 | \n", "-9999.0 | \n", "1.0 | \n", "-10.440518 | \n", "-1.749787 | \n", "10.586131 | \n", "
| 4 | \n", "2024-12-12 00:00:00 | \n", "15.0 | \n", "191.00 | \n", "-9999.0 | \n", "1.0 | \n", "-10.448401 | \n", "-1.973764 | \n", "10.633195 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 436743 | \n", "2024-12-12 18:00:00 | \n", "71.5 | \n", "309.00 | \n", "-9999.0 | \n", "0.0 | \n", "NaN | \n", "NaN | \n", "NaN | \n", "
| 436744 | \n", "2024-12-12 18:00:00 | \n", "71.5 | \n", "309.25 | \n", "-9999.0 | \n", "0.0 | \n", "NaN | \n", "NaN | \n", "NaN | \n", "
| 436745 | \n", "2024-12-12 18:00:00 | \n", "71.5 | \n", "309.50 | \n", "-9999.0 | \n", "0.0 | \n", "NaN | \n", "NaN | \n", "NaN | \n", "
| 436746 | \n", "2024-12-12 18:00:00 | \n", "71.5 | \n", "309.75 | \n", "-9999.0 | \n", "0.0 | \n", "NaN | \n", "NaN | \n", "NaN | \n", "
| 436747 | \n", "2024-12-12 18:00:00 | \n", "71.5 | \n", "310.00 | \n", "-9999.0 | \n", "0.0 | \n", "NaN | \n", "NaN | \n", "NaN | \n", "
436748 rows × 8 columns
\n", "| \n", " | time | \n", "lat | \n", "lon | \n", "number | \n", "expver | \n", "t2m | \n", "sst | \n", "temp_diff | \n", "
|---|---|---|---|---|---|---|---|---|
| 0 | \n", "2024-12-12 00:00:00 | \n", "71.5 | \n", "190.00 | \n", "0 | \n", "0005 | \n", "266.309509 | \n", "271.459717 | \n", "-5.150208 | \n", "
| 1 | \n", "2024-12-12 00:00:00 | \n", "71.5 | \n", "190.25 | \n", "0 | \n", "0005 | \n", "266.256775 | \n", "271.459717 | \n", "-5.202942 | \n", "
| 2 | \n", "2024-12-12 00:00:00 | \n", "71.5 | \n", "190.50 | \n", "0 | \n", "0005 | \n", "266.219666 | \n", "271.459717 | \n", "-5.240051 | \n", "
| 3 | \n", "2024-12-12 00:00:00 | \n", "71.5 | \n", "190.75 | \n", "0 | \n", "0005 | \n", "266.182556 | \n", "271.459717 | \n", "-5.277161 | \n", "
| 4 | \n", "2024-12-12 00:00:00 | \n", "71.5 | \n", "191.00 | \n", "0 | \n", "0005 | \n", "266.172791 | \n", "271.459717 | \n", "-5.286926 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 436743 | \n", "2024-12-12 18:00:00 | \n", "15.0 | \n", "309.00 | \n", "0 | \n", "0005 | \n", "297.999847 | \n", "300.692139 | \n", "-2.692291 | \n", "
| 436744 | \n", "2024-12-12 18:00:00 | \n", "15.0 | \n", "309.25 | \n", "0 | \n", "0005 | \n", "298.113129 | \n", "300.663818 | \n", "-2.550690 | \n", "
| 436745 | \n", "2024-12-12 18:00:00 | \n", "15.0 | \n", "309.50 | \n", "0 | \n", "0005 | \n", "298.361176 | \n", "300.710693 | \n", "-2.349518 | \n", "
| 436746 | \n", "2024-12-12 18:00:00 | \n", "15.0 | \n", "309.75 | \n", "0 | \n", "0005 | \n", "299.017426 | \n", "300.783936 | \n", "-1.766510 | \n", "
| 436747 | \n", "2024-12-12 18:00:00 | \n", "15.0 | \n", "310.00 | \n", "0 | \n", "0005 | \n", "299.794769 | \n", "300.822021 | \n", "-1.027252 | \n", "
436748 rows × 8 columns
\n", "