Weather Catalog Beginnings

Weather data can come in a wide variety of sources and formats. Historical datasets are useful, and forward looking datasets are amazing for future looking analysis. I have yet to come across a "one-stop-shop" solution for API access on weather data that covers a wide geospatial region, and has the temporal granularity that my analysis has needed. Additionally, some datasets have incredibly long query times, so grabbing something once and keeping it handy is a very strong capability. This I started working on designing a flexible "Weather Catalog" framework which provides a unified API for weather data requests and can be connected up to a wide variety to external weather sources and data types. additionally, it has built in data cacheing so long external queries are a "once-and-done" ordeal.

Initial API design was focused on getting a SDK library for back-end use cases, but I've decided to pivot into making this into a more full App using FastHTML and AirFlow. Initial testing is quite promising!

Structural Overview

Process Flow

Repo Link

Airflow Dev UI