Embee is a manufacturer and retailer of IQAir-compatible air filters based in Hong Kong.

In addition to their main ecommerce sales site, I built a website that reports live air quality across Hong Kong at embee.hk/a/air-quality.

The site retrieves pan-HK pollutant data from data.gov.hk hourly, and dynamically produces a set of air quality information pages, with pollution severity levels calculated according to both Hong Kong and US EPA standards, along with relevant recommendations.

Interpolating samples for better guidance

There are 18 pollutant monitoring stations in Hong Kong, but many more locations where access to accurate, local pollution levels are helpful. To provide more accurate and tailored information, the application interpolates pollutant readings to estimate the concentrations at over 1000 primary and secondary school (POI - Point of Interest) locations across the territory.

The geo-referenced data for school locations was sourced from data.gov.hk.

Design and hosting

The application is hosted on AWS and uses Lambda, S3, DynamoDB and API gateway services.

Static and dynamic page generation

The air quality pages for the 18 primary monitoring locations are rendered every hour following raw data download and processing. The output data (AQI, time series etc.) is also produced for the PIO location pages at the same time, but the pages are not rendered, as there is a low chance that a particular POI page will be requested before the next update 1-hour later.

These POI pages are instead rendered on demand and cached until the next hourly data update, saving on both compute and S3 put requests.

The diagram below shows the entire system design:

Each hour the following Lambda functions run:

1. Download readings
   Retrieve raw pollutant reading data from Hong Kong EPD and several other government sites (UK, US etc.).

2. Process readings
   Convert readings into consistant units, calculate AQIs, geographically interpolate Hong Kong readings for all POI locations, prepare time series, etc.

3. Generate pages
   Render liquid (Shopify HTML templating language) templates for each HK recording station.

4. Generate images
   Render a PM2.5 heatmap for Hong Kong based on the individual station readings. Upload it to the main Shopify site via the Assets API

Shopify app proxy

The main Embee website runs on Shopify.

Shopify offers an extensive (and regularly changing) API for site developers. This application makes use of two APIs in order to present the application through the Shopify site. The first is the app proxy:

App proxies take requests to Shopify links, and redirect them to external links. This allows you to fetch and display data on a merchant’s store from an external source.

Shopify developer documentation

It is effectively a reverse proxy for a particular path on a Shopify site, except that instead of just passing the response back to the client, it first renders the page template that it has retrieved. Shopify uses the liquid template language.

This is useful as it allows mixing of Shopify and application content without resorting to iframes etc. and keeps everything on the same domain.

For this site, the app proxy forwards all requests from embee.hk/a/air-quality => AWS. AWS then either retreives or renders the template, which is rendered by Shopify and returned to the client. The key steps are:

1. Client requests /a/air-quality/* resource.
2. Shopify forwards the request to API Gateway.
3. API Gateway verifies the request.
4. API Gateway either returns the liquid template from S3 (standard location), or renders it with Lambda (PIO) and returns it.
5. Shopify renders the liquid template (add headers, sidebars etc.).
6. Completed HTML page is returned to the client.

Assets API

The PM2.5 heatmap is used on many pages elsewhere on the site. To avoid:

S3 -> API Gateway -> Shopify App Proxy -> Client

for images on those pages, the heatmaps are stored within the Shopify store assets. They are transferred from AWS after generation using the Assets API*.

*Note: From March 2024 ,PUT/DEL operations using the Asset API are not allowed for public apps. This does not affect our custom application.