Hirlam numerical weather prediction on AWS S3 - Finnish Meteorological Institute

Hirlam Weather Model

HIRLAM (High Resolution Limited Area Model) is an operational synoptic and mesoscale weather prediction model. The model is developed by an international program formed by institutes from Sweden, Norway, Denmark, Iceland, the Netherlands, Ireland, Spain, Estonia and Lithuania. There is also close co-operation with France.

FMI provides its operational Hirlam model runs to be downloaded freely from AWS S3 bucket. The data is licenced by the Creative Commons Attribution 4.0 International license (CC BY 4.0).

The data is updated four times a day with analysis hours 00, 06, 12 and 18. Corresponding model runs are available roughly five hours after analysis time (~ after model run has started).

The integration area of HARMONIE-AROME covers Scandinavia, while that of HIRLAM includes Europe, Northern Atlantic, parts of northwest Canada, and Russia approxmately to the Ural mountains.

Intergration areas of the two FMI's limited area weather forecast models (inner HARMONIE-AROME, outer HIRLAM).

Content

The current horizontal resolution of the HIRLAM RCR model is 7.5 km. At present, the current operational setup produces daily four 54 hour regional forecasts for extended European area (Figure below). The model is initiated by the ECMWF boundary condition files.

The data is divided in two sets: surface data and pressure levels data. The surface level data provide data for surface of the earth and pressure level data for constant air pressure levels in the atmosphere. The following pressure levels are provided: 1000, 925, 850, 700, 500, 400, 300, 250, 200, 100, 50 hPa.

The data is in projection epsg:4326. Parameters

Following parameters are available in the surface data:

Parameter	GRIB ID	GRIB Name	Level Type	Other Information
Pressure	151	msl	meanSea
GeopHeight	256	Z
Temperature	167	2t	heightAboveGround
DewPoint	168	2d	heightAboveGround
Humidity	157	r	heightAboveGround
WindUMS	131	u	heightAboveGround
WindVMS	132	v	heightAboveGround
PrecipitationAmount	201113	rain_con
TotalCloudCover	164	N	244
LowCloudCover	186	Cl	214
MediumCloudCover	187	Cm	224
HighCloudCover	188	Ch	234
Precipitation1h	2059	rr1h	entireAtmosphere
MaximumWind	201187	MaximumWind	heightAboveGround
WindGust	29	WindGust	105
RadiationGlobalAccumulation	169	ssrd		"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60
RadiationLWAccumulation	175	strd		"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60
RadiationNetSurfaceLWAccumulation	177	str		"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60
RadiationNetSurfaceSWAccumulation	228242	fdif

And following in the pressure level data:

Parameter	GRIB ID	GRIB Name
GeopHeight	256	Z
Temperature	167	2t
DewPoint	168	2d
Humidity	157	r
WindUMS	131	u
WindVMS	132	v
VelocityPotential	135	w
PseudoAdiabaticPotentialTemperature	3014

Accessing the Data on AWS

The data is uploaded into two AWS S3 Buckets:

fmi-opendata-rcrhirlam-surface-grib (for surface data)
fmi-opendata-rcrhirlam-pressure-grib (for pressure levels data)

Every model run is stored in separate directories divided into files based on parameters. Every file contains all valid times for one parameter and one model run.

Files are named with a convention:

numerical-hirlam74-forecast-<parameter>-<year><month><day>T<HOUR>0000Z.grb2

For example:

numerical-hirlam74-forecast-DewPoint-20170322T000000Z.grb2

The data is stored into following directory structure:

<year>/<month>/<day>/<hour>

Where times are derived from origin time of the model run.

Content of the buckets can be browsed here:

SNS

Public Amazon SNS topics are available for every new object added to the Amazon S3.

arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-surface-grib
arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-pressure-grib

For more information on subscribing to SNS topics, visit http://docs.aws.amazon.com/sns/latest/dg/SubscribeTopic.html.

For more information on Amazon S3 event message structure, visit http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html.

Tools to handle the data

The data is stored in GRIB2-files. Below is listed few possible tools to handle the data.

PanoPly

PanoPly provides an easy UI to view GRIB, NetCDF and HDF files.

Example image of PanoPly's user interface visualizing total cloud cover in northern hemisphere

Integrated Data Viewer (IDV)

Compared to PanoPly Integrated Data Viewer (IDV) provides a slightly more complicated but more versatile user interface for browsing and analysing geographical data.

Example image of Intergrated Data Viewer's user interface visualizing long wave radiation data

GRIB-tools

Bunch of tools exist for batch processing the data:

ECMWF GRIB tools provides number of convenient command line tools to process the data. For more information, please consult their wiki:

NOAA's WGRIB2 (and WGRIB for GRIB1 files) can for example: inventory and read grib2 files, create subsets and export data to ieee, text, binary, CSV, netcdf and mysql.

http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/

Geospatial Data Abstraction Library GDAL supports GRIB as well

http://www.gdal.org/frmt_grib.html

SmartMet Server

If you are using the data in an operational manner and live in web environment, SmartMet Server is a great resource to extract the data as JSON or GML and visualize it via WMS.

For more information, please consult: https://github.com/fmidev/smartmet-server

Additional Resources

http://en.ilmatieteenlaitos.fi/numerical-weather-prediction http://en.ilmatieteenlaitos.fi/open-data-manual-forecast-models https://aws.amazon.com/public-datasets/fmi-hirlam/