FMI Hirlam Weather Model on Amazon AWS S3

HIRLAM (High Resolution Limited Area Model) is an operational synoptic and mesoscale weather prediction model. The model is developed by an international program formed by institutes from Sweden, Norway, Denmark, Iceland, the Netherlands, Ireland, Spain, Estonia and Lithuania. There is also close co-operation with France.

FMI provides its operational Hirlam model runs to be downloaded freely from AWS S3 bucket. The data is licenced by the Creative Commons Attribution 4.0 International license (CC BY 4.0).

The data is updated four times a day with analysis hours 00, 06, 12 and 18. Corresponding model runs are available roughly five hours after analysis time (~ after model run has started).

Content

The current horizontal resolution of the HIRLAM RCR model is 7.5 km. At present, the current operational setup produces daily four 54 hour regional forecasts for extended European area (Figure below). The model is initiated by the ECMWF boundary condition files.

Hirlam-mallin peittoalue

The data is divided in two sets: surface data and pressure levels data. The surface level data provide data for surface of the earth and pressure level data for constant air pressure levels in the atmosphere. The following pressure levels are provided: 1000, 925, 850, 700, 500, 400, 300, 250, 200, 100, 50 hPa.

The data is in projection epsg:4326.

Parameters

Following parameters are available in the surface data:

Parameter

GRIB ID

GRIB Name

Level Type

Other Information

Pressure

151

msl

meanSea

 

GeopHeight

256

Z

   

Temperature

167

2t

heightAboveGround

 

DewPoint

168

2d

heightAboveGround

 

Humidity

157

r

heightAboveGround

 

WindUMS

131

u

heightAboveGround

 

WindVMS

132

v

heightAboveGround

 

PrecipitationAmount

201113

rain_con

   

TotalCloudCover

164

N

244

 

LowCloudCover

186

Cl

214

 

MediumCloudCover

187

Cm

224

 

HighCloudCover

188

Ch

234

 

Precipitation1h

2059

rr1h

entireAtmosphere

 

MaximumWind

201187

MaximumWind

heightAboveGround

 

WindGust

29

WindGust

105

 

RadiationGlobalAccumulation

169

ssrd

 

"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60

RadiationLWAccumulation

175

strd

 

"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60

RadiationNetSurfaceLWAccumulation

177

str

 

"templatenumber": 8 / "aggregatetype" : "accum" / "aggregatelength" : 60

RadiationNetSurfaceSWAccumulation

228242

fdif

   
 

And following in the pressure level data:

Parameter

GRIB ID

GRIB Name

GeopHeight

256

Z

Temperature

167

2t

DewPoint

168

2d

Humidity

157

r

WindUMS

131

u

WindVMS

132

v

VelocityPotential

135

w

PseudoAdiabaticPotentialTemperature

3014

 
 

Accessing the Data on AWS

The data is uploaded into two AWS S3 Buckets:

  • fmi-opendata-rcrhirlam-surface-grib (for surface data)

  • fmi-opendata-rcrhirlam-pressure-grib (for pressure levels data)

Every model run is stored in separate directories divided into files based on parameters. Every file contains all valid times for one parameter and one model run.  

 

Files are named with a convention:

numerical-hirlam74-forecast-<parameter>-<year><month><day>T<HOUR>0000Z.grb2

For example:

numerical-hirlam74-forecast-DewPoint-20170322T000000Z.grb2

The data is stored into following directory structure:

<year>/<month>/<day>/<hour>

Where times are derived from origin time of the model run.

Content of the buckets can be browsed here:

SNS

Public Amazon SNS topics are available for every new object added to the Amazon S3.

  • arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-surface-grib
  • arn:aws:sns:eu-west-1:916174725480:new-fmi-opendata-rcrhirlam-pressure-grib

For more information on subscribing to SNS topics, visit http://docs.aws.amazon.com/sns/latest/dg/SubscribeTopic.html.

For more information on Amazon S3 event message structure, visit http://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html.

Tools to handle the data

The data is stored in GRIB2-files. Below is listed few possible tools to handle the data.

PanoPly

PanoPly provides an easy UI to view GRIB, NetCDF and HDF files.  

Integrated Data Viewer (IDV)

Compared to PanoPly Integrated Data Viewer (IDV) provides a slightly more complicated but more versatile user interface for browsing and analysing geographical data.

 

GRIB-tools

Bunch of tools exist for batch processing the data:

  1. ECMWF GRIB tools provides number of convenient command line tools to process the data. For more information, please consult their wiki:

    1. https://software.ecmwf.int/wiki/display/ECC/GRIB+tools

    2. https://software.ecmwf.int/wiki/display/ECC/GRIB+tools+examples

  2. NOAA's WGRIB2 (and WGRIB for GRIB1 files) can for example: inventory and read grib2 files, create subsets and export data to ieee, text, binary, CSV, netcdf and mysql.

    1. http://www.cpc.ncep.noaa.gov/products/wesley/wgrib2/

  3. Geospatial Data Abstraction Library GDAL supports GRIB as well

    1. http://www.gdal.org/frmt_grib.html

SmartMet Server

If you are using the data in an operational manner and live in web environment, SmartMet Server is a great resource to extract the data as JSON or GML and visualize it via WMS.

For more information, please consult: https://github.com/fmidev/smartmet-server

Additional Resources

http://en.ilmatieteenlaitos.fi/numerical-weather-prediction

http://en.ilmatieteenlaitos.fi/open-data-manual-forecast-models

http://fmi-opendata-rcrhirlam-pressure-grib.s3-website-us-east-1.amazonaws.com/

http://fmi-opendata-rcrhirlam-surface-grib.s3-website-us-east-1.amazonaws.com/

https://aws.amazon.com/public-datasets/fmi-hirlam/