CONVERGING DIVERGENCE Hong Kong Meteorological Society

盛放‧匯聚 E-BULLETIN Vol. 28, 2 (2024)

Analogue Forecast System for Daily Precipitation

Prediction Using Autoencoder Feature Extraction:
Application in Hong Kong
Yee Chun TSOI1, Yu Ting KWOK2, Ming Chun LAM2, Wai Kin WONG2

1
The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China
2
Hong Kong Observatory, 134A Nathan Road, Tsim Sha Tsui, Kowloon, Hong Kong, China

*Corresponding author email: [email protected]

ABSTRACT: In the Hong Kong Observatory, the Analogue Forecast System (AFS) for
precipitation has been providing useful reference in predicting possible daily rainfall scenarios for
the next 9 days, by identifying historical cases with similar weather patterns to the latest outputs
from the deterministic model of the European Centre for Medium-Range Weather Forecasts
(ECMWF). Recent advances in machine learning allow more sophisticated models to be trained
using historical data and the patterns of high-impact weather events to be represented more
effectively. As such, an enhanced AFS has been developed using the deep learning technique
autoencoder. The datasets of the fifth generation of the ECMWF Reanalysis (ERA5) are utilised
where more meteorological elements in higher horizontal, vertical and temporal resolutions are
available as compared to the previous ECMWF reanalysis products used in the existing AFS. The
enhanced AFS features four major steps in generating the daily rain class forecasts: (1)
preprocessing of gridded ERA5 and ECMWF model forecast, (2) feature extraction by the pre-
trained autoencoder, (3) application of optimised feature weightings based on historical cases, and
(4) calculation of the final rain class from a weighted ensemble of top analogues. The enhanced
AFS demonstrates a consistent and superior performance over the existing AFS, especially in
capturing heavy rain cases, during the verification period from 2019 to 2022. This paper presents
the detailed formulation of the enhanced AFS and discusses its advantages and limitations in
supporting precipitation forecasting in Hong Kong.
KEYWORDS: Analogue forecast, Daily precipitation forecast, Numerical weather prediction,
Autoencoder Feature Extraction, Machine learning

1. INTRODUCTION (HKO) for the members of the public and

Hong Kong is a subtropical city located in the stakeholders.
coastal areas of southern China (near 22°N and Recent years see the advances of the global
114°E) with a hot and humid summer from May to numerical weather prediction (NWP) in terms of
September each year. Due to the southwest enhancements in model configurations including
monsoon and tropical cyclones, thunderstorms and horizontal resolution, data assimilation and model
heavy precipitation events often occur during the physical processes, etc. In complement to NWP, the
period. Taking June as an example, the analogue forecasting technique has been adopted for
climatological monthly mean rainfall (1991-2020) is predicting more detailed and realistic local weather
491.5 mm, and the maximum record was 1346.1 mm primarily driven by mesoscale dynamics or subgrid
in 2008. Moreover, the annual mean number of days scale processes which may not be captured by global
with heavy rainfall, defined by daily rainfall of 25 NWP models. The word analogue here refers to two
mm or above, of the same period is about 30. states of atmospheric conditions that are relatively
Therefore, the forecast of significant or heavy similar (Lorenz, 1969). An analogue forecast system
rainfall is important in support of the forecasting (AFS) aims to search for historical weather patterns
and warning services of the Hong Kong Observatory that are analogous to the latest NWP model outputs,

Received: 15 Aug 2024 | Accepted: 26 Nov 2024 https://www.meteorology.org.hk/new/bulletin/

P a g e |2

which are assumed to be sufficiently representative model forecast (ECMWF deterministic model run)
of the expected synoptic conditions. With the from different initial times. Additionally, the
notion that “history repeats itself”, the occurrences performance of this AFS in forecasting extreme
and intensities of weather events like thunderstorms weather events could have rooms for improvement.
and tropical cyclones are supposed to match with Two possible causes of the above shortcomings are
historical cases bearing similar weather patterns, due to the underrepresentation of weather patterns
a.k.a. analogues (Bergen & Harnack, 1982). These by the self-defined variables and the need for more
analogues thus can serve as useful references for the cases for verification during optimisation.
expected local weather, range of possible scenarios, With a longer record of gridded reanalysis data
and extremity of weather events for the forecasted from the fifth generation of the European Centre for
days. Medium-Range Weather Forecasts (ECMWF)
Analogue forecasting is widely used in Reanalysis (ERA5) and an increasing number of
operational predictions of precipitation (Hamill & model variables, an enhanced AFS has been
Whitaker, 2006; Ben Daoud et al., 2016; Zhou & developed in this study to overcome the
Zhai, 2016), temperature (Bergen & Harnack, 1982; shortcomings of the existing AFS. Moreover,
Kruizinga & H. Murphy, 1983), extreme events improvements in computer hardware shorten the
(Chattopadhyay et al., 2020), and short-term climate time for model training and operation, enabling the
(Van Den Dool, 1994; Dayon et al., 2015). application of more sophisticated techniques based
Furthermore, the method is applied for supporting on machine learning (ML) and optimisation
impact-based forecasts of flooding (Marty et al., algorithms. The development of an enhanced ML-
2012; Bellier et al., 2016) and streamflow (P. based AFS for precipitation has therefore become
Salathé, 2003; Caillouet et al., 2022). In most cases, feasible.
the analogy of large-scale meteorological patterns ML-based models are becoming increasingly
can be associated with the corresponding observed popular in the field of weather forecasting. Unlike
local meteorological and/or hydrological element. the traditional physics-based numerical models,
The first AFS in the HKO was developed in the these models are more robust to perturbations and
early 1990s (Poon & Ma, 1992) for weather may adapt better to global climate change by
forecasts. The meteorological variables for the generalising hidden patterns in historical data
inputs of the predictor included the surface (Holmstrom et al., 2016; Singh et al., 2019).
pressure, winds, and temperature at 850 hPa, the Applications of ML range from the post-processing
geopotential height (Z) at 500 hPa, and the winds at of observations (Lo and Lau, 2021) and NWP model
200 hPa, each with a 2.5° ´ 2.5° resolution. outputs (Cho et al., 2022; Schulz and Lerch, 2022),
Similarity metrics expressed as the anomaly nowcasting (Prudden et al., 2020), to purely data-
correlation coefficient and S1 score were used for driven global weather models with neural networks
selecting analogues from the historical analysis (Pathak et al., 2022; Bi et al., 2023; Lam et al., 2023).
archive. Amongst ML methods, the autoencoder emerges as
The AFS was reformulated in 2014 by Chan et al. a deep-learning-based feature extraction method
and has been used for providing daily precipitation that is particularly suitable for generating abstract
forecasting in support of operation at HKO. The representations for multi-dimensional data and
forecasted and historical weather patterns are anomaly detections (Gogna & Majumdar, 2018;
compared in the following three aspects: the Meng et al., 2017; Xing et al., 2016). As such, it is
geopotential height pattern, gradient of geopotential believed that features can be learnt from the gridded
height, and moisture elements (Chan et al., 2014). data predicted by the ECMWF models using
The system employs the self-defined similarity autoencoder. In light of the above, this paper
scores of the above predictors, aiming to capture attempts to construct an enhanced AFS for
precipitation events, especially those with heavy precipitation by comparing the feature
rainfall. Optimised scores determined by the cuckoo representations extracted by an autoencoder and
search (Yang and Deb, 2009) are then applied to applying the optimised scores for the selection of
devise the selection criteria of analogues. Although analogues. In the subsequent sections, the training
the AFS has shown skills in forecasting daily data and the forecast workflow are first introduced,
precipitation intensity and usefulness in capturing followed by the model verification and performance
various scenarios, its predictions could vary quite evaluation, as well as a brief discussion on potential
substantially or lack continuity with the inputs of model enhancements in the conclusion.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |3

2. DATA AND METHODOLOGY

2.1 Data
Three sets of data are used for the enhanced
AFS, namely, (a) the gridded reanalysis data (ERA5),
(b) the gridded forecast data from the deterministic
model of the ECMWF (HRES), and (c) the daily
rainfall observation. The data formats, time periods,
and their corresponding purpose at various stages of
the model training and operation are summarised in
Table 1.

2.1.1 Reanalysis Data

ERA5 is the fifth-generation global atmospheric
reanalysis dataset provided by the ECMWF. The Figure 1: Spatial domain of the gridded data. Also shown
subset of ERA5 data used in this study contains ten is the schematic representation of the Gaussian weight
parameters: divergence, geopotential height, function G(rk) applied, where rk is the distance (in
degrees) from Hong Kong (HK).
potential vorticity, relative humidity, relative
vorticity, specific humidity, temperature, zonal
2.1.2 Forecast Data
component of wind (U), meridional component of
The parameters and spatial information of the
wind (V), and vertical wind (W). 2-D gridded data
ECMWF deterministic model (HRES) data used in
of these parameters are obtained for six isobaric
this study are identical to that of the reanalysis data.
levels, namely 1000, 925, 850, 700, 500, and 200
The dataset is formed by forecast data with a model
hPa, forming a total of 60 model features. The
base time of 12 UTC every day, and the forecast
gridded data extends over an area defined by 0° –
valid time of interest is from the Day 1 (T+36 hours)
45°N and 90° – 155°E, covering the region of East
to the Day 9 (T+228 hours) where T denotes the
Asia (Figure 1) where synoptic-scale weather
base time of HRES forecast. Day 1 of the forecast
systems affecting Hong Kong (HK) can be
data from 2017 to 2020 are used for feature
sufficiently captured. The reanalysis dataset for the
optimisation, while three years of data from May
AFS is formed by the ERA5 data of 00 UTC every
2019 to April 2022 are used for subsequent forecast
day from 1979 to 2020. It is used in the training of
verification. For daily operational forecasts, the
the autoencoder model and serves as the historical
latest ECMWF HRES model outputs are input into
data archive of analogues.
the trained AFS model to be compared with the
historical reanalysis data.

Table 1: Overview of the datasets used for model training and operation of the enhanced AFS.
Dataset Data Format Time Period Purpose (see Section 2.2 for details)
Reanalysis Gridded numerical (.grb); 1979 – 2020 • Autoencoder model training
(ERA5) 0.25° x 0.25° spatial • Historical data archive
resolution
Forecast Gridded numerical (.grb); Since 2017 • Feature optimisation (2017 – 2020)
(ECMWF HRES) 0.125° x 0.125° spatial • Verification (May 2019 – Apr 2022)
resolution • Operational forecast
Daily Rainfall Numerical values (.csv) Since 1979 • Feature optimisation (2017 – 2020)
Observation • Verification (May 2019 – Apr 2022)
• Operational forecast (rain class
calculation)

2.1.3 Rainfall Observation rain gauges located in different regions in Hong

Another set of data essential for the model Kong to represent the overall rainfall intensity of
training and prediction is the daily rainfall amount the territory. It is used for feature optimisation,
in Hong Kong since 1979. The daily rainfall amount forecast verification, and the rain class calculation
is obtained as an average of seven selected reference step of the operational forecast.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |4

2.2 Workflow The term rk in Equation(1) is the distance of the grid

The model training consists of three main stages: point from Hong Kong in degrees, where 1° equals
data preprocessing, autoencoder model training, and to about 103 km close to Hong Kong. Furthermore,
feature optimisation. On the other hand, the the term r0 is empirically set to be 7, such that when
workflow of the daily operational forecast is similar, rk is equal to r0, the corresponding weight of the
which includes data preprocessing, feature grid point G(rk) will be approximately halved.
extraction using the pretrained autoencoder,
application of the optimised feature weightings, and 2.2.2 Autoencoder Model Training and
a final step to obtain the rain class prediction. The Application
overview of the workflow is illustrated in Figure 2 The preprocessed ERA5 data from 1979 to 2020
and further explained below. are shuffled randomly and split into two datasets,
with 80% data for training data and 20% data for
2.2.1 Data Preprocessing validation. This strategy ensures that the two
2.2.1.1 Reduction of Resolution datasets can adequately represent the overall
The ERA5 and HRES data have initial spatial distribution of the weather characteristics over the
resolutions of 0.25° ´ 0.25° and 0.125° ´ 0.125° in 42 years, so that the exposure of the trained model
regular latitude / longitude grid respectively. Before can be maximised to the historical weather data
inputting the datasets for model training and variability and hence be more robust when
operational forecast, the spatial resolution of both handling unseen data. They are used for training the
datasets is remapped to 1° ´ 1° by average pooling to autoencoder, a deep-learning algorithm suitable for
harmonize the data resolutions and reduce extracting meaningful features and detecting
computational costs of training and inference of the anomalies from multi-dimensional data by
ML model. representation learning (Bank et al., 2020).
An autoencoder consists of an encoder and a
2.2.1.2 Normalisation decoder, each comprising two convolutional layers
Next, the input gridded datasets are normalised. and with the inception module applied so to allow
For each of the 60 model features, values are feature extraction at multiple scales (Figures 3a and
remapped to a range between 0 and 1 for 4). The encoder compresses the 2-D gridded data
convenient processing. This would not cause any into 1-D extracted vectors while the decoder
significant loss of information since the AFS is more attempts to recreate the input with the extracted
concerned with the pattern and relationship vectors as much as possible (Bank et al., 2020). The
between grids rather than the absolute values of neural network architecture is defined arbitrarily by
each feature. The maximum value of a particular testing different combinations and minimising the
feature within the data archive is set to be 1, while mean squared errors (MSE) between the input and
the minimum value is set to 0. The same reconstructed images. The training process is
normalisation process is applied to both gridded automatically stopped when there is no
datasets in a consistent manner. improvement for five consecutive epochs.
The pre-trained autoencoder can generate
2.2.1.3 Gaussian Weight Function vectors that represent the feature maps well as
With reference to Chan et al. (2014), a Gaussian illustrated using the example input and output
weight function is applied to assign more weight to images for the case of 00 UTC 16-09-2018 when
grid points closer to Hong Kong and less weight to severe typhoon Mangkhut skirted around 100 km
those further away (Figure 1). In so doing, the AFS SSW of Hong Kong (Hong Kong Observatory, 2018).
can be more sensitive to proximal weather patterns It can be seen that the patterns and magnitudes
which have the largest effect on local precipitation shown on the original input images are preserved on
events. The maximum value of the function is 1 the corresponding reconstructed image for four
centered at the location of Hong Kong, while its selected features (Figure 5). The more alike the
minimum value tends to 0. The equation of the input and recreated images are, the more
function is as follows: representable the extracted vectors are.
𝑟2
Quantitatively, the similarity between two images is
− 𝑘
1 𝐺(𝑟𝑘 ) = 𝑒 (1.2×𝑟 0 )2
(1) represented in terms of MSE.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |5

Figure 2: Overview of the AFS workflow for model training and operational forecast.

Figure 3: (a) Structure of an autoencoder; (b) Application of the pre-trained autoencoder for operational forecast.

After the autoencoder model is trained, a pre- the forecasting month. For example, the extracted
trained model is obtained for the feature extraction vectors from the forecast gridded data on any date
of both ERA5 data archive and HRES forecast data. in December are only compared to cases within the
The extracted vectors of the former are saved in 5-month interval from October to February in the
binary format after passing through the autoencoder data archive. This is done to ensure that seasonal
once to be compared with the extracted vectors characteristics of precipitation events are considered
from the latest forecast data daily for the operational when searching for past cases with similar
forecast (Figure 3b). It is noted that the comparison meteorological patterns.
process is restricted to cases within 2 months from

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |6

Figure 4: Neural network architecture of the encoder (left) and decoder (right) used in the AFS.

Figure 5: Original and reconstructed images from the pre-trained autoencoder of 500 hPa geopotential height, 700 hPa
relative humidity, 925 hPa U wind component, 925 hPa V wind component for the case of 00 UTC 16 September 2018.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |7

2.2.3 Feature Optimisation and Weightings 3. These optimised weightings are applied to
Among the 60 model features, some features calculate the weighted MSE and similarity (i.e. 1 /
may be more relevant and influential to the weighted MSE) between each forecast and
occurrence and intensity of precipitation events reanalysis pair during operational forecasts.
than others. Therefore, it is necessary to deduce an
optimised weighting for each feature when
identifying the best analogue. The process is Table 2: Definition of rain class by daily rainfall in this
performed with a hyperparameter optimisation study. Classes Heavy, Very Heavy, Torrential, and
framework named Optuna (Akiba et al., 2019). Extreme are grouped together for model optimisation and
With the pre-trained autoencoder model, the verification due to the very few number of cases.
Rain Class Daily Rainfall (mm)
gridded data of the 60 features can be represented
No Rain < 0.05
with the 60 corresponding extracted vectors every
Light ≥ 0.05 and < 10
day. The extracted vectors from each day during the
Moderate ≥ 10 and < 25
optimisation period of 2017 – 2020 are compared to Heavy ≥ 25 and < 50
the ERA5 data archive with the same vector Very Heavy ≥ 50 and < 100
expression. MSE are calculated day-by-day and Torrential ≥ 100 and < 200
feature-by-feature, and then saved as input files for Extreme ≥ 200
the optimisation. This comparison process is
restricted from 1 January 1979 to 9 days before the
verified date and to cases within 2 months from the 2.2.4 Rain Class Calculation for Operational
forecasting month. The normalisation, which Forecast
remaps MSE values into a range of 0 – 1, is applied With the procedures mentioned above, the
to each feature in every file in advance. analogue forecast model can be used to predict daily
During the optimisation, the search space for rain classes for the next 9 days. To avoid large
each parameter is defined at the beginning of each fluctuations of results between model runs and
run, with an initial range of 0 – 1 and a step size of increase the resilience of the model to outliers, an
0.001. The objective is to minimise the MSE ensemble of similar analogues formed by the 25
between the rain classes (Table 2) of each day most similar analogues is considered in the final rain
during the optimisation period and its class calculation. A weighted score (WS) is assigned
corresponding best analogue, which is selected from to each of the 25 analogues according to its
the data archive as the case with least weighted weighted MSE and the sample-to-population ratio
MSE deduced by the feature weightings from the for each rain class. If the proportion of a certain rain
current run. class within the 25 analogues (i.e. sample ratio) is
Each run contains 5000 trials, with the first 1500 smaller than its occurrence frequency in the past
trials searching randomly within the space given ~20 years (i.e. population ratio; Table 4), the rain
and the last 3500 trials applying an algorithm of the class is disregarded from the prediction. The
Tree-structured Parzen Estimator Approach equation, obtained empirically, to calculate the WS
(Bergstra et al., 2011). As the number of trials (n denotes n-th analogue) is as follows:
increases, the parameters start converging while the
𝑆𝑎𝑚𝑝𝑙𝑒 𝑅𝑎𝑡𝑖𝑜𝑛 2
loss decreases at the same time. At the end of each 1
1
𝑊𝑆𝑛 = %𝑊𝑒𝑖𝑔ℎ𝑡𝑒𝑑 𝑀𝑆𝐸𝑛 × 1𝑃𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑅𝑎𝑡𝑖𝑜𝑛 : ,
𝑆𝑎𝑚𝑝𝑙𝑒 𝑅𝑎𝑡𝑖𝑜𝑛
𝑃𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑅𝑎𝑡𝑖𝑜𝑛
≥1

run, the parameters from the best trial will be 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 (2)
obtained and taken as the result of the optimisation
from that particular run. The weighted scores for the 25 analogues are
After the first few runs, the range of each applied to the corresponding daily rainfall amount
parameter will be modified and narrowed down (RF) of the analogue to yield the weighted mean
gradually. This step will be performed manually rainfall (WMR) for a particular forecast day
based on the results from previous runs. The process (Equation (3)). Finally, the predicted rain class is
continues until the parameters converge between obtained based on the WMR and according to the
runs, meaning that the optimised values of the rain class definitions (Table 2).
parameters are very similar (within a range of 0.05)
∑25
𝑛=1 𝑊𝑆𝑛 × 𝑅𝐹𝑛
for consecutive runs. The resultant 1 optimised 𝑊𝑀𝑅 =
∑25
𝑛=1 𝑊𝑆𝑛 (3)
weightings of all model features are given in Table

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |8

Table 3: Optimised weightings of all model features for the AFS. Features with the top ten weightings are highlighted in
bold.
Layer 200 hPa 500 hPa 700 hPa 850 hPa 925 hPa 1000 hPa
Feature
Divergence 0.011 0.106 0.560 0.568 0.941 0.666
Geopotential Height 0.662 0.662 0.662 0.67 0.364 0.001
Potential Vorticity 0.095 0.004 0.09 0.407 0.017 0.246
Relative Humidity 0.014 0.163 0.014 0.987 0.968 0.770
Relative Vorticity 0.322 0.053 0.057 0.736 0.807 0.968

Specific Humidity 0.753 0.827 0.046 0.521 0.522 0.12

Temperature 0.927 0.293 0.574 0.120 0.385 0.416
U Wind 0.67 0.933 0.905 0.968 0.656 0.647
V Wind 0.247 0.012 0.625 0.979 0.699 0.524
Vertical Wind 0.077 0.207 0.984 0.818 0.077 0.615

Table 4: Statistics on the rain class population ratio based alarms of ‘light rain’ when there is ‘no rain’
on observations from 2001 to 2020. increased slightly for the enhanced AFS.
Rain Class Number of days Ratio The model performances are also evaluated in
No Rain 1779 48.70% terms of verification metrics, namely the critical
Light 1320 36.13% success index (CSI = hit / (total forecasted + misses)),
Moderate 295 8.08%
probability of detection (POD = hits / total observed),
Heavy or above 259 7.09%
and false alarm ratio (FAR = false alarms / total
forecasted), for each rain class. Verification metrics
are compared between the HKO 9-day Forecast
3. RESULTS AND DISCUSSIONS
bulletins issued by the forecasters, the enhanced
3.1 Model Verification and Performance
AFS, the existing AFS, and the averaged ECWMF
Evaluation
direct model output for grids near Hong Kong in
The performance of the newly developed ML-
Figures 6-8. With a CSI of almost 0.4 and a POD
based AFS by autoencoder (hereafter, enhanced
over 0.5 for forecast Day 1, the enhanced AFS
AFS) is evaluated and compared to that of the AFS
outperforms the forecasters and other models for
(Chan et al., 2014) that has been serving forecasters
heavy rain cases from Day 1 to Day 9. Its FAR is also
at HKO for almost a decade (hereafter, existing
generally lower than the existing AFS. The
AFS). The model verification period covers three
enhanced AFS also has a similar, if not better,
years from May 2019 to April 2022.
performance compared to the existing AFS in terms
The confusion matrices in Table 5 show the rain
of verification metrics for other rain classes. Besides,
class observations against predictions for forecast
the CSI for ‘no rain’ of the enhanced AFS is
Day 1, Day 4, and Day 9 from the two AFS. The
consistently around 0.2 higher than that of ECMWF
enhanced AFS is clearly more skillful in capturing
direct model output, indicating its ability to correct
heavy rain cases while maintaining similar skill for
the NWP model overestimation of rainfall on non-
other rain classes compared to the existing AFS
rainy days. Note that the performance of models
throughout the 9-day forecast period. In particular,
and forecasters alike appear to be the worst in the
over half of the observed heavy rain cases during
predication of ‘moderate rain’ owing to the narrow
the verification period are correctly predicted, and
range of rainfall amount defined for this rain class
this number of cases is more than double of that
(Table 2) and the relatively fewer samples.
correctly predicted by the existing AFS. It is worth
Nevertheless, for forecast operations considering the
noting that the enhanced AFS has nil Day 1
impact to the public, it is more important for a
forecasts with an error of more than two rain
model to be able to discern rainy days from non-
classes, and such unideal cases are also greatly
rainy days, and give forecasters improved guidance
reduced for other forecast days. However, false
for potential days with heavy rain.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e |9

Table 5: Confusion matrices of the actual versus forecasted rain classes by the existing and enhanced AFS for forecast Day 1,
Day 4, and Day 9. Green cells indicate correct forecasts and grey cells indicate acceptable forecasts of the neighbouring rain
class. Refer to Table 2 for definitions of rain classes. Note that the total number of forecasts are slightly different for the two
AFS due to missing data.

Day 1 Existing AFS (Chan et al., 2014) Enhanced AFS (this study)

Fc.
No Rain Light Moderate Heavy ≤ No Rain Light Moderate Heavy ≤
Obs.

No Rain 457 79 6 2 420 112 6 0

Light 129 205 30 10 75 258 36 13

Moderate 11 34 21 5 4 32 21 16

Heavy ≤ 7 29 22 16 0 21 14 38

Day 4

No Rain 447 90 3 3 385 130 10 2

Light 121 209 33 12 71 253 39 16

Moderate 11 36 19 6 6 34 16 19

Heavy ≤ 6 38 18 12 0 30 21 22

Day 9

No Rain 410 103 14 12 346 136 20 13

Light 134 182 37 19 94 215 42 28

Moderate 16 37 13 7 6 37 17 12

Heavy ≤ 13 37 14 10 4 40 11 18

3.2 Heavy Rain Case Study Meanwhile, the existing AFS gave fluctuating
In view of its promising performance, the predictions of ‘moderate’ and ‘heavy’ for this case in
enhanced AFS is put into real-time operation and the nine days prior to 13 May.
has been launched as an internal forecaster tool To further highlight the superior performance of
(Figure 9) at HKO since May 2022. An episode of the enhanced AFS compared to the existing AFS,
heavy rain from 11 to 13 May 2022 is selected to weather charts of selected layers from the
illustrate how the enhanced AFS is able to identify a corresponding best analogue identified with respect
similar analogue from the historical archive and to the ECMWF HRES forecast with model base time
provide early alert for forecasters regarding the at 20220504 12 UTC are shown in Figure 10. The
heavy rain event. best analogue of the enhanced AFS is a day with
A total rainfall of 331.5 mm was recorded during ‘torrential’ rain (11 June 1979). It exhibits more
11 to 13 May 2022, with the rainfall amounts similar synoptic patterns versus the best analogue
classified as ‘very heavy’, ‘torrential’, and ‘very found by the existing AFS (22 April 2010).
heavy’ (Table 2), respectively. The enhanced AFS Specifically, similarities can be seen in the
showed clear signals of a 3-day heavy precipitation orientation and location of the surface trough and
event up to five days ahead (Figure 9). Looking into associated rain band (Figure 10a), the locations of
more detail for 13 May 2022, the enhanced AFS low-level shear line and jets in addition to the
consistently and accurately predicted ‘very heavy’ moisture content and temperature near Hong Kong
rain since more than a week ahead on 5 May, thus (Figure 10b), as well as the narrow ridge and area of
giving forecasters more confidence to include this strong divergence along the southeastern coast of
message in the public forecast bulletins on 7 May. China at the upper level (Figure 10c).

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 10

Figure 6: Confusion Comparison of model performance in terms of critical success index (CSI) between HKO forecaster,
enhanced AFS, existing AFS, and ECMWF direct model output.

Figure 7: Confusion Comparison of model performance in terms of probability of detection (POD) between HKO forecaster,
enhanced AFS, existing AFS, and ECMWF direct model output.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 11

Figure 8: Confusion Comparison of model performance in terms of false alarm ratio (FAR) between HKO forecaster,
enhanced AFS, existing AFS, and ECMWF direct model output.

3.3 Characteristics of the Enhanced AFS Although the parameter weightings are deduced
Other than the improvement in performance, from a result-oriented optimisation, it can be seen
major differences between the existing and that some highly weighted parameters match with
enhanced AFS are summarised in Table 6. Two the physical understanding of precipitation
features of the enhanced AFS that could benefit the processes in a weather forecast. From Table 3, the
practical application of the model and enhance the top ten parameters are temperature on 200 hPa
understanding of the potential precipitation events layer, U on 500 hPa layer, U and W on 700 hPa
are discussed in this section. layer, relative humidity, U and V on 850 hPa layer,
divergence and relative humidity on 925 hPa layer,
3.3.1 Meteorological Parameters and the relative vorticity on 1000 hPa layer. Most of
Unlike the existing AFS, the enhanced AFS takes the above results coincide with the self-defined
advantage of more meteorological parameters parameters used in the existing AFS and the
included in the HRES and ERA5 data from understanding of the major contributing factors of
ECMWF. Parameters such as divergence and precipitation, including low-level moisture supply
relative vorticity, which may indicate favourable and low- to mid-level synoptic pattern in terms of U
conditions for the development of convective and V. Moreover, low-level convergence and
systems, are added into consideration in the relative vorticity relate to convective weather
enhanced AFS. Optimised weightings are also associated with low-pressure systems and tropical
assigned to each parameter so the differential cyclones, which often bring heavy rainfall to Hong
importance of each parameter can be taken into Kong and the coastal areas of southern China (Wu
account. This eliminates the insufficient et al., 2020). These prove that the optimisation
representativeness of the self-defined parameters results actually show consistency with both the
used in the existing AFS and hence yields more existing AFS and theoretical analyses.
consistent results for the enhanced AFS.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 12

Figure 9: Web portal of the enhanced AFS internal forecaster tool at HKO. Predictions by the latest and past 9 model runs
from 20220506 12 UTC are displayed in the top section; Information of the 25 most similar analogues are listed in the
middle section; Forecasted and reanalysed weather charts for the corresponding model run and analogue, respectively, are
shown in the bottom section.

3.3.2 Ensemble Forecast preprocessing, autoencoder model training, and

Another modification of the enhanced AFS feature optimisation by the Optuna framework.
compared to the existing AFS is the method for With the aid of machine learning, the
obtaining the final rain class output. It considers the implementation of the autoencoder feature
top 25 analogues and assigns a score to each extraction is able to capture information from the 60
analogue for the estimation of the WMR and features of the ECMWF forecasts and find analogues
subsequently the rain class. By considering multiple more effectively. For the operational forecast, the
analogues, the model not only reduces the pre-trained autoencoder and optimised feature
probability of the output being fluctuated by weightings are applied to the latest forecasts. Then,
outliers, but also results in a traceable ensemble by considering the similarity and past occurrence
forecast with information of the members clearly frequency of the top 25 analogues, a final rain class
listed. Furthermore, the rainfall distribution from output is obtained. The enhanced AFS demonstrates
the 25 analogues is summarised and provided as promising ability in capturing heavy rain cases and
reference for the forecasters. This could give an outperforms the existing AFS consistently over the
indication of the range of anticipated weather 9-day forecast period during the 3-year verification
scenarios and more insight on the uncertainty of period. The analogues identified by the enhanced
possible weather events when formulating the AFS serve as additional information to forecasters
weather forecast. on the plausible local rainfall amounts for a similar
synoptic pattern depicted by ECMWF HRES, while
4. CONCLUSION AND FUTURE WORK the final rain class from the system provide
An enhanced AFS has been developed utilising guidance on the rainfall intensity summarised from
the machine learning framework. There are three the ensemble of analogues. The new system has
main stages in the model training, including data been in real-time operation since May 2022 and has

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 13

ECMWF forecast Best analogue of existing AFS Best analogue of enhanced AFS
(20220504 12 UTC T+204h) (reanalysis of 20100422 00 UTC) (reanalysis of 19790611 00 UTC)
‘Very Heavy’ rain class ‘Moderate’ rain class ‘Torrential’ rain class
(a) Surface level

(b) 925 hPa level

(c) 200 hPa level

Figure 10: Comparison of (a) surface level, (b) 925 hPa level, and (c) 200 hPa level weather charts from ECMWF forecast
with model base time at 20220504 12 UTC against the best analogues found by the existing and enhanced AFS for the case
on 13 May 2022. The location of Hong Kong is marked by a black cross.

been well-received by forecasters at HKO, Kong, as the atmospheric environment may be

providing useful forecast guidance on the changing rapidly in a relatively small spatiotemporal
anticipated rain class and early warning for episodes scale under these conditions (Chen et al., 2020;
of heavy rain. Marks et al., 1998). Besides, the performance of an
Despite the improvements achieved by the AFS is highly dependent on the skill and accuracy of
enhanced AFS, there are limitations due to its the NWP model forecasts, in this case the ECMWF
methodology and the complexity of weather HRES data. Another intrinsic weakness of an AFS is
systems around the coastal areas of southern China. its inability to find matching cases for unforeseen
Events like tropical cyclones, convective storms and extreme weather events brought by climate change.
cold fronts are very hard to predict in terms of how In order to overcome some of these limitations,
much precipitation they would be bringing to Hong several modifications could be explored.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 14

Table 6: Major differences between the existing AFS and enhanced AFS.
Existing AFS (Chan et al., 2014) Enhanced AFS

Historical ERA-Interim reanalysis (1979-2008) ERA5 reanalysis (1979-2020)

Archive

Search Window Forecast date ± 50 days (101 days) Forecast month ± 2 months (5 months)
Model Geopotential Height, Gradient of Geopotential Divergence, Geopotential Height, Potential &
parameters Height Field (Z at 700, 850 & 925 hPa only), Relative Vorticity, Relative & Specific Humidity,
Relative Humidity (at 700 & 850 hPa only) Temperature and U, V & W wind fields
Model layers 200, 500, 700, 850 & 925 hPa 200, 500, 700, 850, 925 & 1000 hPa
Grid Weighting Gaussian weight function, except for similarity Also uses the same Gaussian weight function to
score of Z at 500 hPa (synoptic pattern matching) give more weight to grid points closer to HK
Methodology Z pattern & gradient matching (anomaly Autoencoder feature extraction & vector
correlation coefficient & S1 score), Moisture comparison (in terms of MSE)
matching (comparison of average RH at upstream
grids)

Optimisation Cuckoo Search; to determine system parameters Optuna; to deduce appropriate weightings for
that maximize CSI of heavy rain class and each feature in order to minimize MSE between
minimize cases with no rain class best analog and actual rain class
Rain Class From mean rainfall of analogs considered From weighted mean rainfall of top 25 analogs
Calculation sufficiently similar, otherwise only 1 best analog weighted based on similarity and rain class
population ratio

4.1 Number of Analogues Considered required to determine the optimal threshold to be

The number of analogues used to determine the adopted for this filtering.
final rain class is set to be 25 based on the tested
results. However, there may not be that many cases 4.2 Method to Calculate the Final Rain Class
from the data archive matching particular events On some occasions, there may be a wide
that have seldom occurred in the past. This results spectrum of weather scenarios among the top-
in a higher chance that some less similar analogs are ranked analogues, which may contribute to the
taken under consideration in this circumstance. overestimation or underestimation of the WMR. In
Therefore, fewer analogues might be considered for the WS calculation step, adjustments of parameters
such calculation to prevent biases caused by like the thresholds of the sample-to-population ratio
irrelevant analogues. One potential solution is to for each rain class and the relationship between the
vary the number of analogues considered for ratios and the weighted score may be implemented
different seasons. For dry seasons like fall and through further optimisations. Nevertheless, this
winter, precipitation is less likely to happen, and modification would not be effective if the output
hence similar cases may be more challenging to rain class has been misled by outliers among the
find. Lowering the number of analogues in the top-ranked analogues, probably due to the intrinsic
ensemble may be preferred. Another possible model error from ECMWF HRES forecast data.
modification is to apply thresholds to filter less Besides, care must be taken not to overfit the
similar analogues. In this case, analogues with empirical equation to cases in the verification
weighted mean scores less than the thresholds period.
would not be considered even if they are ranked at
the top. However, a more in-depth study would be

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 15

4.3 Increase in Resolution Chattopadhyay, A., Hassanzadeh, P., & Nabizadeh, E.

(2020). Analog forecasting of extreme-causing weather
The data used for the model is currently reduced
patterns using deep learning. Journal of Advances in
to a spatial resolution of 1° ´ 1°. Although the Modeling Earth Systems, 12(2).
reduction of data size lowers the running time of https://doi.org/10.1029/2019MS001958
multiple procedures, there may be loss of Chen, R., Zhang, W., & Wang, X. (2020). Machine learning
in tropical cyclone forecast modeling: A Review.
information during the process. As Hong Kong is Atmosphere, 11(7), 676.
small compared to the grid size of the NWP models, https://doi.org/10.3390/atmos11070676
the gridded data may not be reflecting sufficient Cho, D., Yoo, C., Son, B., Im, J., Yoon, D., & Cha, D. H.
details for some small-scale atmospheric processes (2022). A novel ensemble learning for post-processing of
NWP Model's next-day maximum air temperature forecast
leading to precipitation. Therefore, one way to in summer using deep learning and statistical approaches.
improve the model is to maximise the resolution to Weather and Climate Extremes, 35, 100410.
the gridded data while maintaining an acceptable Dayon, G., Boé, J., & Martin, E. (2015). Transferability in
the future climate of a statistical downscaling method for
efficiency of the model. Since most parts of the
precipitation in France. Journal of Geophysical Research:
models are optimised and finalised in the model Atmospheres, 120(3), 1023–1043.
training, it is feasible to maintain high efficiency for https://doi.org/10.1002/2014jd022236
the operational forecasts even with finer gridded Gogna, A., & Majumdar, A. (2018). Discriminative
Autoencoder for feature extraction: Application to
data, given advancements in computer hardware. character recognition. Neural Processing Letters, 49(3),
1723–1735. https://doi.org/10.1007/s11063-018-9894-5
REFERENCES Hamill, T. M., & Whitaker, J. S. (2006). Probabilistic
Akiba, T., Sano, S., Yanase, T., Ohta, T., & Koyama, M. quantitative precipitation forecasts based on reforecast
(2019). Optuna: A Next-generation Hyperparameter analogs: Theory and application. Monthly Weather
Review, 134(11), 3209–3229.
Optimization Framework. Proceedings of the 25th ACM
https://doi.org/10.1175/mwr3237.1
SIGKDD International Conference on Knowledge Holmstrom, M., Liu, D., & Vo, C. (2016). Machine
Discovery & Data Mining. Learning Applied to Weather Forecasting. Machine
https://doi.org/10.1145/3292500.3330701 Learning Applied to Weather Forecasting - CS229.
Bank, D., Koenigstein, N., & Giryes, R. (2020). Retrieved 2023, from
Autoencoders. ArXiv. https://cs229.stanford.edu/proj2016/report/HolmstromLiu
Vo-MachineLearningAppliedToWeatherForecasting-
https://doi.org/10.48550/arXiv.2003.05991
report.pdf.
Bellier, J., Zin, I., Siblot, S., & Bontron, G. (2016). Hong Kong Observatory. (2018, October 29). A Wake up
Probabilistic flood forecasting on the Rhone River: Call from Mangkhut. Retrieved 2023, from
Evaluation with Ensemble and analogue-based https://www.hko.gov.hk/en/blog/00000216.htm
precipitation forecasts. E3S Web of Conferences, 7, Kruizinga, S., & Murphy, A. H. (1983). Use of an analogue
18011. https://doi.org/10.1051/e3sconf/20160718011 procedure to formulate objective probabilistic temperature
Ben Daoud, A., Sauquet, E., Bontron, G., Obled, C., & forecasts in the Netherlands. Monthly Weather Review,
Lang, M. (2016). Daily quantitative precipitation forecasts 111(11), 2244–2254. https://doi.org/10.1175/1520-
based on the analogue method: Improvements and 0493(1983)111<2244:uoaapt>2.0.co;2
application to a French Large River Basin. Atmospheric Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger,
Research, 169, 147–159. P., Fortunato, M., Alet, F., ... & Battaglia, P. (2023).
https://doi.org/10.1016/j.atmosres.2015.09.015 Learning skillful medium-range global weather
Bergen, R. E., & Harnack, R. P. (1982). Long-range forecasting. Science, eadi2336.
temperature prediction using a simple analog approach. https://doi.org/10.1126/science.adi2336
Monthly Weather Review, 110(8), 1083–1099. Lo, K. W., & Lau, C. S. (2021). Application of Tephigram
https://doi.org/10.1175/1520- in Forecasting Diurnal Variation of Surface
0493(1982)110<1083:lrtpua>2.0.co;2 Temperature—Preliminary Findings based on Deep
Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Learning. The 34th Guangdong - Hong Kong - Macao
Algorithms for hyper-parameter optimization. Advances in Seminar on Meteorological Science and Technology.
neural information processing systems, 24, 2546–2554. Lorenz, E. N. (1969). Atmospheric predictability as revealed
Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., & Tian, Q. by naturally occurring analogues. Journal of the
(2023). Accurate medium-range global weather Atmospheric Sciences, 26(4), 636–646.
forecasting with 3D neural networks. Nature, 619(7970), https://doi.org/10.1175/1520-
533-538. 0469(1969)26<636:aparbn>2.0.co;2
Caillouet, L., Celie, S., Vannier, O., Bontron, G., & Marks, F. D., Shay, L. K., & PDT-5. (1998). Landfalling
Legrand, S. (2022). Operational hydrometeorological tropical cyclones: Forecast Problems and associated
forecasting on the Rhône River in France: Moving toward research opportunities. Bulletin of the American
a seamless probabilistic approach. LHB, 108(1). Meteorological Society, 79(2), 305–323.
https://doi.org/10.1080/27678490.2022.2061312 https://doi.org/10.1175/1520-
Chan, S.-tick, Fong, C.-hang, & Chan, M.-yee. (2014). An 0477(1998)079<0305:ltcfpa>2.0.co;2
Analog Forecast System for Precipitation Forecasting. Marty, R., Zin, I., Obled, C., Bontron, G., & Djerboua, A.
28th Guangdong-Hong Kong-Macao Seminar on (2012). Toward real-time daily PQPF by an analog sorting
Meteorological Science and Technology. Retrieved 2023, approach: Application to flash-flood catchments. Journal
from https://www.hko.gov.hk/publica/reprint/r1097.pdf.

Tsoi et al. HKMETS, Vol. 28, 2

P a g e | 16

of Applied Meteorology and Climatology, 51(3), 505–520. Singh, N., Chaturvedi, S., & Akhter, S. (2019). Weather
https://doi.org/10.1175/jamc-d-11-011.1 forecasting using machine learning algorithm. 2019
Meng, Q., Catchpoole, D., Skillicom, D., & Kennedy, P. J. International Conference on Signal Processing and
(2017). Relational autoencoder for feature extraction. 2017 Communication (ICSC).
International Joint Conference on Neural Networks https://doi.org/10.1109/icsc45622.2019.8938211
(IJCNN). https://doi.org/10.1109/ijcnn.2017.7965877 Van den Dool, H. M. (1994). Searching for analogues, how
Pathak, J., Subramanian, S., Harrington, P., Raja, S., long must we wait? Tellus A: Dynamic Meteorology and
Chattopadhyay, A., Mardani, M., ... & Anandkumar, A. Oceanography, 46(3), 314.
(2022). Fourcastnet: A global data-driven high-resolution https://doi.org/10.3402/tellusa.v46i3.15481
weather model using adaptive fourier neural operators. Wu, Y., Chen, S., Li, W., Fang, R., & Liu, H. (2020).
arXiv preprint arXiv:2202.11214. Relative vorticity is the major environmental factor
Poon, H. T., & Man, M. W. (1992). The Royal Observatory controlling tropical cyclone intensification over the
Synoptic Analogue Forecast System. The Seventh western North Pacific. Atmospheric Research, 237,
Guangdong-Hong Kong-Macau Joint Seminar on 104874. https://doi.org/10.1016/j.atmosres.2020.104874
Hazardous Weather, Macau. Retrieved 2023, from Xing, C., Ma, L., & Yang, X. (2016). Stacked Denoise
https://www.hko.gov.hk/tc/publica/reprint/files/r233.pdf. Autoencoder based feature extraction and classification for
Prudden, R., Adams, S., Kangin, D., Robinson, N., Ravuri, Hyperspectral Images. Journal of Sensors, 2016, 1–10.
S., Mohamed, S., & Arribas, A. (2020). A review of radar- https://doi.org/10.1155/2016/3632943
based nowcasting of precipitation and applicable machine Yang, X. S., & Deb, S. (2009, December). Cuckoo search
learning techniques. arXiv. via Lévy flights. In Proceedings of World Congress on
https://doi.org/10.48550/arXiv.2005.04988 Nature & Biologically Inspired Computing (NaBIC), USA
Salathé, E. P. (2003). Comparison of various precipitation (pp. 210-214). IEEE.
downscaling methods for the simulation of streamflow in a Zhou, B., & Zhai, P. (2016). A new forecast model based on
Rainshadow River Basin. International Journal of the analog method for persistent extreme precipitation.
Climatology, 23(8), 887–901. Weather and Forecasting, 31(4), 1325–1341.
https://doi.org/10.1002/joc.922 https://doi.org/10.1175/waf-d-15-0174.1
Schulz, B., & Lerch, S. (2022). Machine learning methods
for postprocessing ensemble forecasts of wind gusts: A
systematic comparison. Monthly Weather Review, 150(1),
235-257. https://doi.org/10.1175/MWR-D-21-0150.1

Tsoi et al. HKMETS, Vol. 28, 2