Weather Prediction: Data Sources

Gathering weather predictions was accomplished by accessing the following APIs: WeatherUnderground API, AccuWeather API, DarkSky API, and the National Weather Service API.  These forecast services were chosen since they have free (but low throughput) developer APIs.  Some forecast services such as the Weather Channel do not have an API.  (The Weather Channel and WeatherUnderground use similar data and methodology and are both owned by IBM.)  Other forecast services such as the MeteoGroup require contacting a sales office to access their API.  There were a few times that the National Weather service had missing data for a day.  In this case, average values for the other days was used.  Forecast data for 12 US cities was downloaded at around 1000-1200 and at around 2200-0000 in the preceding days; however, only forecast data from 07/20 from the evening download was used for analysis since that was available on 07/22/2017 when the study was done.  This data allowed for validating the weather forecast for 07/21/2017.  Historical forecasts could not be found.

Weights for each city location were manually scraped from the Forecast Advisor website.  For each weather forecast service, the website gives a percentage of the predictions that were correct from the preceding weather predictions.  The website tracks accuracy for maximum temperature, minimum temperature, and precipitation, so these accuracy measures were used as weights for a weighted average of the forecast values of the four weather services.  Since the site samples weather forecasts at night, the maximum temperature predictions are generally better than the minimum temperature predictions.  According to the website, the WeatherUnderground and the Weather Channel are generally the best for the US.  Based on my observation of Forecast Advisor, the DarkSky service does well in predicting precipitation, and the DarkSky website claims to use a hyperlocal methodology for precipitation prediction.  The forecast services chosen for the study had generally good prediction performance.

The previous day's temperatures and precipitation amount were manually scraped from the Fast Weather website using the Yesterday's Weather tool.  The DarkSky API was accessed to get historic weather conditions for the 20 years prior for a given day.  This was used to compute the prior distributions.  The DarkSky API was used since it was the only API that offered this service, although the National Weather Service has historical data in a different format.  If the historic weather for a day at a location did not exist in the DarkSky data, then that day was ignored.  This only happened for years 2001, 2002, and 2003 for Juneau, AK on day 07/21.

Latitude and longitude locations for each city in the study was gotten from the Google search engine.  The following cities were used in the study:
Blacksburg, VA
Chicago, IL
Honolulu, HI
Houston, TX
Juneau, AK
Lebanon, KS
Miami, FL
New_York, NY
Salt_Lake_City, UT
San_Francisco, CA
Seattle, WA
Washington, DC