Article

Development of a rainfall nowcasting algorithm based on optical flow techniques

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

A new advection-based nowcasting scheme for precipitation has been developed for the Gandolf system. The method employed removes the need to split the radar analysis into contiguous rain areas (CRA's) and uses a smoothness constraint. Thus, it is similar to the Variational Echo Tracking approach. Optical flow ideas are used to diagnose the advection velocity of blocks within rain analyses, which involve the direct solution of the Lagrangian persistence equation. In block-based approaches, the radar analyses are partitioned a-priori and a CRA may be split over a number of blocks. This scheme is compared with the old Gandolf advection scheme, which is based on CRA's, and the new scheme performs better-both in cases associated with severe flooding and over a continuous verification period of 3 months. The benefit is conjectured to be due mainly to the difficulty in unambiguously identifying CRA's, which is particularly troublesome on small domains, such as the UK. Thus, it is concluded that block-based methods are likely to be superior to object-based methods in the majority of cases.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... This approach allows the probability of precipitation to be derived from an ensemble of forecasts for several hours ahead. It has been further improved by Bowler et al. (2004Bowler et al. ( , 2006 as the Short-Term Ensemble Prediction System (STEPS), which merges an extrapolated nowcast with downscaled NWP model forecasts. ...
... is the main stream length, the effective rainfall intensity, the Horton's length ratio. The Eq.(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19) can be rewritten as: be estimated since all other components of Eq.(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22) can be determined or measured. ...
... is the main stream length, the effective rainfall intensity, the Horton's length ratio. The Eq.(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19) can be rewritten as: be estimated since all other components of Eq.(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14)(15)(16)(17)(18)(19)(20)(21)(22) can be determined or measured. ...
Technical Report
Full-text available
This report presents a review of flash flood forecasting methods and systems as an initial phase of a study aiming to develop a forecasting system that will be used by the Bureau of Meteorology, Australia, to provide a flash flood warning service. In the report, characteristics of flash floods and causal factors such as hydro-meteorological processes, and hydrologic and hydraulic processes are described. Intense rainfall is the most common causal factor for flash flood formation. Advances in rainfall forecasting techniques and usage of remotely sensed data for flash flood forecasting are presented. Advance techniques for merging different sources of information (e.g. ground data, radar and satellite observations, numerical weather prediction model outputs) for producing better rainfall estimates and long lead-time rainfall forecasts are discussed. A number of flash flood forecasting models and methods are briefly described. Authors recognize that Physically-based Distributed Hydrological Models (PDHM) are more appropriate than data driven models and conceptual hydrological models for flash flood forecasting. The ability of PDHMs considering spatial distribution of rainfall and catchment state variables (e.g. soil moisture), and the potential for application to poorly-gauged catchments are advantages. Real-time updating of hydrological model parameters in the flash flood forecasting system is recommended. The two potential methods for flash flood forecasting in Australia are recognized as the flash flood guidance (FFG) method (section 4.2) and the statistical-distributed modelling (SDM) method (section 4.1.5). Their characteristics and operational feasibility are discussed. Further a probabilistic forecast approach that better represents the uncertainty associated with forecasts is recommended. Some of the operational flood forecasting systems used in different countries are described. Their advantages and limitations are suggested. Finally, future research directions to improve the quality of flash flood forecasts are discussed.
... It calculates the motion vector based on the correlation between the neighborhood of each point. The optical flow method in image processing is also used for radar extrapolation [21], [22]. This method can be regarded as a particular case of cross-correlation methods, which also obtains a dense motion vector field through pixellevel matching. ...
... The advantage of the optical flow method is that its principle has already included the global smoothing constraint, which could reduce the noise in motion vector fields more effectively. After determining the dense motion vector field, the extrapolation of each point is usually determined by the forward or backward semi-Lagrangian scheme [21], [23]. Different from the above perspectives, some studies have tried to handle extrapolation from the signal processing view. ...
... Our extrapolation GAN model is denoted as ExtGAN. Farneback optical flow uses local polynomial matching for dense motion vector field estimation, and the forward-intime scheme proposed in [21] is used as the corresponding extrapolation method. TrajGRU is an effective method that combines gated recurrent unit (GRU) and 2D-CNN for radar extrapolation. ...
Article
Full-text available
Radar echo extrapolation is a basic but essential task in meteorological services. It could provide radar echo prediction results with high spatiotemporal resolution in a computationally efficient way, and effectively enhance the operational system's forecasting capability for meteorological hazards. Traditional methods perform extrapolation by estimating echo motions between contiguous radar data. This strategy is difficult to characterize complex nonlinear meteorological processes effectively, and it is difficult to benefit from large historical data. Recently, machine learning (ML) models have been used for radar echo extrapolation. These methods have effectively improved extrapolation quality in a data-driven way and from the statistical perspective. Although the ML-based methods show excellent performance, they usually produce blurry extrapolations. This leads to underestimating radar echo intensity and making echo lack small-scale details. Moreover, it makes models difficult to predict severe convective hazards. To solve this problem, a two-stage extrapolation model based on 3D Convolutional Neural Network (3D-CNN) and Conditional Generative Adversarial Network (CGAN) is proposed. These two models form the ‘'pre-extrapolation" and "post-processing" paradigm. The pre-extrapolation model is trained in the traditional way and performs rough extrapolation. The post-processing model uses the pre-extrapolation result as input and is trained with the adversarial strategy. It could correct the echo intensity and increase the echo’s details. In the experiment, our model could provide more precise radar echo extrapolations than other methods, especially for intense echoes and convective systems, in the data of North China from 2015 to 2016.
... The innovation is the smoothness term J 2 , which enforces a degree of spatial consistency. Various choices of J 2 are possible; Li et al. (1995) use the divergence ∂u ∂x + ∂v ∂y , while Bowler et al. (2004) instead use the Laplacian ∇ 2 V. ...
... The full variational problem involves an expensive minimisation, and various tricks have been used to improve convergence. In Li et al. (1995) and Bowler et al. (2004), variational techniques are used as a form of post-processing for a solution found using a cheaper block-based method. Another option is to solve the full variational problem using a hierarchical approach, where solutions are computed at successively higher resolutions with each field being used as an initial guess for the next (Germann and Zawadzki, 2002;Zawadzki, 1994, 1995). ...
... The forward version of the scheme is conceptually similar, except that it follows the air parcels forward in time instead of backward. Its basic steps are as follows (Bowler et al., 2004): ...
Preprint
Full-text available
A 'nowcast' is a type of weather forecast which makes predictions in the very short term, typically less than two hours - a period in which traditional numerical weather prediction can be limited. This type of weather prediction has important applications for commercial aviation; public and outdoor events; and the construction industry, power utilities, and ground transportation services that conduct much of their work outdoors. Importantly, one of the key needs for nowcasting systems is in the provision of accurate warnings of adverse weather events, such as heavy rain and flooding, for the protection of life and property in such situations. Typical nowcasting approaches are based on simple extrapolation models applied to observations, primarily rainfall radar. In this paper we review existing techniques to radar-based nowcasting from environmental sciences, as well as the statistical approaches that are applicable from the field of machine learning. Nowcasting continues to be an important component of operational systems and we believe new advances are possible with new partnerships between the environmental science and machine learning communities.
... In the second step, we use that information to advect the most recent 2 Chapter 1. Introduction rain field, i.e., to displace it to the imminent future. For these two computational steps -tracking and extrapolation -the utilization of optical flow-based and semi-Lagrangian advection algorithms, respectively, is of the most common use (Bowler et al., 2004;Foresti et al., 2016;Germann and Zawadzki, 2002b;Reyniers, 2008). ...
... The original term was inspired by the idea of an apparent motion of brightness patterns observed when a camera or the eyeball is moving relative to the objects (Horn and Schunck, 1981). Today, optical flow is often understood as a group of techniques to infer motion patterns or velocity fields from consecutive image frames, e.g. in the field of precipitation nowcasting (Bowler et al., 2004;Liu et al., 2015;Woo and Wong, 2017). For the velocity field estimation, we need to accept both the brightness constancy assumption and one of a set of additional optical flow constraints (OFCs). ...
... There is also a distinct group of spectral methods where the Fourier transform is applied to the inputs, and an OFC is resolved in the spectral (Fourier) domain (Ruzanski et al., 2011). Bowler et al. (2004) introduced the first local optical flow algorithm for precipitation nowcasting, and gave rise to a new direction of models. Bowler's algorithm is the basis of the STEPS (Bowler et al., 2006) and STEPS-BE (Foresti et al., 2016) operational nowcasting systems. ...
Thesis
Full-text available
Precipitation forecasting has an important place in everyday life – during the day we may have tens of small talks discussing the likelihood that it will rain this evening or weekend. Should you take an umbrella for a walk? Or should you invite your friends for a barbecue? It will certainly depend on what your weather application shows. While for years people were guided by the precipitation forecasts issued for a particular region or city several times a day, the widespread availability of weather radars allowed us to obtain forecasts at much higher spatiotemporal resolution of minutes in time and hundreds of meters in space. Hence, radar-based precipitation nowcasting, that is, very-short-range forecasting (typically up to 1–3 h), has become an essential technique, also in various professional application contexts, e.g., early warning, sewage control, or agriculture. There are two major components comprising a system for precipitation nowcasting: radar-based precipitation estimates, and models to extrapolate that precipitation to the imminent future. While acknowledging the fundamental importance of radar-based precipitation retrieval for precipitation nowcasts, this thesis focuses only on the model development: the establishment of open and competitive benchmark models, the investigation of the potential of deep learning, and the development of procedures for nowcast errors diagnosis and isolation that can guide model development. The present landscape of computational models for precipitation nowcasting still struggles with the availability of open software implementations that could serve as benchmarks for measuring progress. Focusing on this gap, we have developed and extensively benchmarked a stack of models based on different optical flow algorithms for the tracking step and a set of parsimonious extrapolation procedures based on image warping and advection. We demonstrate that these models provide skillful predictions comparable with or even superior to state-of-the-art operational software. We distribute the corresponding set of models as a software library, rainymotion, which is written in the Python programming language and openly available at GitHub (https://github.com/hydrogo/rainymotion). That way, the library acts as a tool for providing fast, open, and transparent solutions that could serve as a benchmark for further model development and hypothesis testing. One of the promising directions for model development is to challenge the potential of deep learning – a subfield of machine learning that refers to artificial neural networks with deep architectures, which may consist of many computational layers. Deep learning showed promising results in many fields of computer science, such as image and speech recognition, or natural language processing, where it started to dramatically outperform reference methods. The high benefit of using "big data" for training is among the main reasons for that. Hence, the emerging interest in deep learning in atmospheric sciences is also caused and concerted with the increasing availability of data – both observational and model-based. The large archives of weather radar data provide a solid basis for investigation of deep learning potential in precipitation nowcasting: one year of national 5-min composites for Germany comprises around 85 billion data points. To this aim, we present RainNet, a deep convolutional neural network for radar-based precipitation nowcasting. RainNet was trained to predict continuous precipitation intensities at a lead time of 5 min, using several years of quality-controlled weather radar composites provided by the German Weather Service (DWD). That data set covers Germany with a spatial domain of 900 km x 900 km and has a resolution of 1 km in space and 5 min in time. Independent verification experiments were carried out on 11 summer precipitation events from 2016 to 2017. In these experiments, RainNet was applied recursively in order to achieve lead times of up to 1 h. In the verification experiments, trivial Eulerian persistence and a conventional model based on optical flow served as benchmarks. The latter is available in the previously developed rainymotion library. RainNet significantly outperformed the benchmark models at all lead times up to 60 min for the routine verification metrics mean absolute error (MAE) and critical success index (CSI) at intensity thresholds of 0.125, 1, and 5 mm/h. However, rainymotion turned out to be superior in predicting the exceedance of higher intensity thresholds (here 10 and 15 mm/h). The limited ability of RainNet to predict high rainfall intensities is an undesirable property which we attribute to a high level of spatial smoothing introduced by the model. At a lead time of 5 min, an analysis of power spectral density confirmed a significant loss of spectral power at length scales of 16 km and below. Obviously, RainNet had learned an optimal level of smoothing to produce a nowcast at 5 min lead time. In that sense, the loss of spectral power at small scales is informative, too, as it reflects the limits of predictability as a function of spatial scale. Beyond the lead time of 5 min, however, the increasing level of smoothing is a mere artifact – an analogue to numerical diffusion – that is not a property of RainNet itself but of its recursive application. In the context of early warning, the smoothing is particularly unfavorable since pronounced features of intense precipitation tend to get lost over longer lead times. Hence, we propose several options to address this issue in prospective research on model development for precipitation nowcasting, including an adjustment of the loss function for model training, model training for longer lead times, and the prediction of threshold exceedance. The model development together with the verification experiments for both conventional and deep learning model predictions also revealed the need to better understand the source of forecast errors. Understanding the dominant sources of error in specific situations should help in guiding further model improvement. The total error of a precipitation nowcast consists of an error in the predicted location of a precipitation feature and an error in the change of precipitation intensity over lead time. So far, verification measures did not allow to isolate the location error, making it difficult to specifically improve nowcast models with regard to location prediction. To fill this gap, we introduced a framework to directly quantify the location error. To that end, we detect and track scale-invariant precipitation features (corners) in radar images. We then consider these observed tracks as the true reference in order to evaluate the performance (or, inversely, the error) of any model that aims to predict the future location of a precipitation feature. Hence, the location error of a forecast at any lead time ahead of the forecast time corresponds to the Euclidean distance between the observed and the predicted feature location at the corresponding lead time. Based on this framework, we carried out a benchmarking case study using one year worth of weather radar composites of the DWD. We evaluated the performance of four extrapolation models, two of which are based on the linear extrapolation of corner motion; and the remaining two are based on the Dense Inverse Search (DIS) method: motion vectors obtained from DIS are used to predict feature locations by linear and Semi-Lagrangian extrapolation. For all competing models, the mean location error exceeds a distance of 5 km after 60 min, and 10 km after 110 min. At least 25% of all forecasts exceed an error of 5 km after 50 min, and of 10 km after 90 min. Even for the best models in our experiment, at least 5 percent of the forecasts will have a location error of more than 10 km after 45 min. When we relate such errors to application scenarios that are typically suggested for precipitation nowcasting, e.g., early warning, it becomes obvious that location errors matter: the order of magnitude of these errors is about the same as the typical extent of a convective cell. Hence, the uncertainty of precipitation nowcasts at such length scales – just as a result of locational errors – can be substantial already at lead times of less than 1 h. Being able to quantify the location error should hence guide any model development that is targeted towards its minimization. To that aim, we also consider the high potential of using deep learning architectures specific to the assimilation of sequential (track) data. Last but not least, the thesis demonstrates the benefits of a general movement towards open science for model development in the field of precipitation nowcasting. All the presented models and frameworks are distributed as open repositories, thus enhancing transparency and reproducibility of the methodological approach. Furthermore, they are readily available to be used for further research studies, as well as for practical applications.
... These methods generally compute an optical flow based on the last few precipitation fields by calculating an approximate flow velocity and then using semi-Lagrangian advection to move areas of precipitation along the calculated flow field, see e.g. [2]. A main complication of these methods is that they require appropriate pre-processing of the raw radar image data, such as smoothing of the input data or image segmentation to label contiguous areas of precipitation. ...
... A main reason for the prevalence of extrapolation based methods compared to direct numerical weather predictions is that numerical models generally do not capture the initial precipitation well [11]. Various different methods using extrapolation-based approaches can be found in [2,11,20], as well as in the references therein. ...
Preprint
We propose the use of a stochastic variational frame prediction deep neural network with a learned prior distribution trained on two-dimensional rain radar reflectivity maps for precipitation nowcasting with lead times of up to 2 1/2 hours. We present a comparison to a standard convolutional LSTM network and assess the evolution of the structural similarity index for both methods. Case studies are presented that illustrate that the novel methodology can yield meaningful forecasts without excessive blur for the time horizons of interest.
... Nowcasting aims to tracking the movement of storms to extrapolate the radar rainfall field into the future with a forecasting lead time of a few hours [131][132][133]. Radar-based nowcasting methods include Tracking radar echoes by correlation (TREC and COTREC methods), tracking the centroids of rain cells, use of wind fields from NWP forecasts to advect the precipitation field, the Variational Echo Tracking (VET) method and optical flow techniques. ...
... The Variational Echo Tracking (VET) method is similar to the COTREC method, but also takes into account radial velocities from Doppler radar. Bowler et al. [131] developed a method to compute the advection field using optical flow techniques that has shown better performance in cases involving embedded convection. This method assumes that features in a sequence of radar scans only change shape, but do not change in size or intensity. ...
Chapter
Full-text available
Weather radar is a remote sensing instrument that has been increasingly used to estimate precipitation for a variety of hydrological and meteorological applications, including real-time flood forecasting, severe weather monitoring and warning, and short-term precipitation forecasting. Weather radar provides unique observations of precipitating systems at fine spatial and temporal resolutions, which are difficult to obtain through conventional raingauge networks. The potential benefit of using radar rainfall in hydrology is huge, but practical hydrological applications of radar have been limited by the inherent uncertainties and errors in radar rainfall estimates. Uncertainties in radar rainfall estimates can lead to large errors in flood forecasting applications, so radar rainfall measurements must be corrected before the data are used quantitatively. This chapter discusses some of the latest advances in the measurement and forecasting of precipitation with weather radar and some of the techniques proposed in the literature to correct and adjust radar rainfall estimates.
... Large efforts are devoted to the study of these phenomena and to the improvement of nowcasting (i.e., 0-6 h forecasting) [1] to accurately predict the time and location of the occurrence of such convective storms, thus providing an early warning tool to reduce socio-economical impacts. A wide plethora of nowcasting methods exists in literature, based on satellite and radar observations or products [2][3][4][5][6][7][8][9][10][11][12][13][14][15][16]. Generally, these methods rely on different approaches, ranging from statistical to physical ones [17]. ...
... µm P 11 5-min trend: 12-10.8 µm Updraft Strength P 12 5-min trend: 6.2-7.3 µm P 13 15-min trend: 10.8 µm P 14 10-min trend: 10.8 µm P 15 5-min trend: 10.8 µm P 16 5-min trend: 6.2-10.8 µm P 17 5-min trend: 6.2-12 µm P 18 5-min trend: 9.7-13.4 ...
Article
Full-text available
In this study, we investigate the ability of several convective initiation predictors based on satellite infrared observations to distinguish convective weak precipitation events from those leading to intense rainfall. The two types of precipitation are identified according to hourly rainfall, respectively less than 10 mm and greater than 30 mm. The analysis is conducted on a representative dataset containing 92 severe and weak precipitation events collected over the Italian peninsula in the period 2016–2019 over June-September. The events are selected to be short-lived (i.e., less than 12 h) and localized (i.e., less than 50×50km2). Italian National Radar Network products, namely the Vertical Maximum Intensity (VMI) and the Surface Rain Total (SRT) variables (from Dewetra Platform by CIMA, Italian Civil Protection Department), are used as indicators of convection (i.e., VMI greater than 35 dBZ echo intensity) and cumulated rainfall, respectively. The considered predictors are linear combinations of spectral infrared channels measured with the Rapid Scan Service (RSS) Spinning Enhanced Visible and InfraRed Imager (SEVIRI) aboard Meteosat Second Generation (MSG) geostationary satellites. We select a 5×5 SEVIRI pixel-box centered on the storm core and perform a statistical analysis of the predictors up to 2.5 h around the event occurrence. We demonstrate that some of the proxies—describing growth and glaciation storm properties—show few degrees contrast between severe and nonsevere precipitation cases, hence carrying significant information to help discriminate the two types. We design a threshold scheme based on the three most informative predictors to distinguish weak and strong precipitation events. This analysis yields accuracy higher than 0.6 and the probability of false detection lower than 0.26; in terms of reducing false alarms, this method shows slight better performances compared to related works, at the expense of a lower probability of detection. The overall results, however, show limited capability for these infrared proxies as stand-alone predictors to distinguish severe from nonsevere precipitation events. Nonetheless, these may serve as additional tools to reduce the false alarm ratio in nowcasting algorithms for convective orographic storms. This study also provides further insight into the correlation between early infrared fields signatures prior to convection and subsequent evolution of the storms, extending previous works in this field.
... Nowcasting aims to tracking the movement of storms to extrapolate the radar rainfall field into the future with a forecasting lead time of a few hours [131][132][133]. Radar-based nowcasting methods include Tracking radar echoes by correlation (TREC and COTREC methods), tracking the centroids of rain cells, use of wind fields from NWP forecasts to advect the precipitation field, the Variational Echo Tracking (VET) method and optical flow techniques. ...
... The Variational Echo Tracking (VET) method is similar to the COTREC method, but also takes into account radial velocities from Doppler radar. Bowler et al. [131] developed a method to compute the advection field using optical flow techniques that has shown better performance in cases involving embedded convection. This method assumes that features in a sequence of radar scans only change shape, but do not change in size or intensity. ...
Chapter
Accurate soil moisture indicator is critically important for hydrological applications such as water resource management and hydrological modelling. Modern satellite remote sensing has shown a huge potential for providing soil moisture measurements at a large scale. However its effective utilisation in the aforementioned areas still needs comprehensive research. This chapter focuses on exploring the advances and potential issues in the current application of satellite soil moisture observations in hydrological modelling. It has been proposed that hydrological application of soil moisture data requires the data relevant to hydrology. In order to meet the requirement, the following two research tasks are suggested: the first is to carry out comprehensive assessments of satellite soil moisture observations for hydrological modelling, not merely based on evaluations against point-based in situ measurements; the second is that a soil moisture product (e.g. soil moisture deficit) directly applicable to hydrological modelling should be developed. Only fully accomplishing these two steps will push forward the utilisation of satellite soil moisture in hydrological modelling to a greater extent.
... Use a backward constant-vector [48] to interpolate or extrapolate each pixel according to the velocity field. 3. ...
... 1. Calculate velocity field using the global DIS optical flow algorithm [47] based on the radar images at times and . 2. Use a backward constant-vector [48] to interpolate or extrapolate each pixel according to the velocity field. 3. Obtain an irregular point cloud that consists of the original radar pixels. ...
Article
Full-text available
Missing data in weather radar image sequences may cause bias in quantitative precipitation estimation (QPE) and quantitative precipitation forecast (QPF) studies, and also the obtainment of corresponding high-quality QPE and QPF products. The traditional approaches that are used to reconstruct missing weather radar images replace missing frames with the nearest image or with interpolated images. However, the performance of these approaches is defective, and their accuracy is quite limited due to neglecting the intensification and disappearance of radar echoes. In this study, we propose a deep neuron network (DNN), which combines convolutional neural networks (CNNs) and bi-directional convolutional long short-term memory networks (CNN-BiConvLSTMs), to address this problem and establish a deep-learning benchmark. The model is trained to be capable of dealing with arbitrary missing patterns by using the proposed training schedule. Then the performances of the model are evaluated and compared with baseline models for different missing patterns. These baseline models include the nearest neighbor approach, linear interpolation, optical flow methods, and two DNN models three-dimensional CNN (3DCNN) and CNN-ConvLSTM. Experimental results show that the CNN-BiConvLSTM model outperforms all other baseline models. The influence of data quality on interpolation methods is further investigated, and the CNN-BiConvLSTM model is found to be basically uninfluenced by less qualified input weather radar images, which reflects the robustness of the model. Our results suggest good prospects for applying the CNN-BiConvLSTM model to improve the quality of weather radar datasets.
... Recently, radar echo extrapolation-based methods have been noticed and widely adopted [2,4] these years. ...
... These algorithms rely on the extrapolation of observations by ground-based radars via optical flow techniques or neural network models. Optical flow-based [4][5][6] methods, as typical extrapolation-based methods, have drawn increasingly more attention, owing to their fast speeds and high accuracies. The approach is conducted in two stages by the extrapolation of radar observations. ...
Preprint
With the goal of predicting the future rainfall intensity in a local region over a relatively short period time, precipitation nowcasting has been a long-time scientific challenge with great social and economic impact. The radar echo extrapolation approaches for precipitation nowcasting take radar echo images as input, aiming to generate future radar echo images by learning from the historical images. To effectively handle complex and high non-stationary evolution of radar echoes, we propose to decompose the movement into optical flow field motion and morphologic deformation. Following this idea, we introduce Flow-Deformation Network (FDNet), a neural network that models flow and deformation in two parallel cross pathways. The flow encoder captures the optical flow field motion between consecutive images and the deformation encoder distinguishes the change of shape from the translational motion of radar echoes. We evaluate the proposed network architecture on two real-world radar echo datasets. Our model achieves state-of-the-art prediction results compared with recent approaches. To the best of our knowledge, this is the first network architecture with flow and deformation separation to model the evolution of radar echoes for precipitation nowcasting. We believe that the general idea of this work could not only inspire much more effective approaches but also be applied to other similar spatiotemporal prediction tasks
... (1) where z(x, y, t) denotes the reflectivity field in space and time and u(x, y) and v(x, y) denote the x-and y-component of advection velocities. While a majority of the existing techniques solve this equation in the spatial domain via correlation-based or variational techniques [17]- [20], in [4], the formulation is done in the spectral domain. This approach allows efficient solution and also eliminates the need for separately computing the DFT of the advection field, which is utilized within the SF-DARTS framework described in the following sections. ...
... In addition, Fig. 9(d) shows decorrelation times for different scales. That is, the time after which the weighted AR(2) autocorrelation estimate falls below 0.1 and the truncation takes place according to (20). We obtain approximately 10 and 25 min for Table II. ...
Article
Full-text available
Short-term forecasts (nowcasts) of severe rainfall and flooding are of high importance to the society. In the collaborative adaptive sensing of the atmosphere (CASA) project, a high-resolution X-band radar network was deployed in the Dallas–Fort Worth (DFW) urban area. The dynamic and adaptive radar tracking of storms (DARTS) is a key component of the precipitation nowcasting system that was developed in the CASA project. In DARTS, the advection field determination is formulated in the spectral domain using the discrete Fourier transform (DFT). Building on the earlier work, an extension of DARTS is proposed. The novelty of the proposed scale filtering (SF-DARTS) method is the formulation of the extrapolation also in the spectral domain. The extrapolation method is combined with autoregressive AR(2) models applied to Fourier frequency bands together with adaptive truncation of DFT coefficients. This effectively filters small spatial scales having low predictability. It is shown that the proposed approach improves forecast skill and gives improved computational efficiency compared to conventional methods. Another important contribution is that DARTS is being evaluated for the first time beyond the urban scale. DARTS and SF-DARTS are evaluated using data from two different sources, namely the urban-scale CASA DFW network (200 km), and the country-wide radar network operated by the Finnish Meteorological Institute (1000 km).
... Once the correspondence and their displacement have been established, the position of these cells is extrapolated to the desired time horizon. The second category relies on the estimation of a dense field of apparent velocities at each pixel of the image and modeled by the optical flow [6,7]. The forecast is also obtained by extrapolation in time and advection of the last observation with the apparent velocity field. ...
Article
Full-text available
Short- or mid-term rainfall forecasting is a major task with several environmental applications such as agricultural management or flood risk monitoring. Existing data-driven approaches, especially deep learning models, have shown significant skill at this task, using only rainfall radar images as inputs. In order to determine whether using other meteorological parameters such as wind would improve forecasts, we trained a deep learning model on a fusion of rainfall radar images and wind velocity produced by a weather forecast model. The network was compared to a similar architecture trained only on radar data, to a basic persistence model and to an approach based on optical flow. Our network outperforms by 8% the F1-score calculated for the optical flow on moderate and higher rain events for forecasts at a horizon time of 30 min. Furthermore, it outperforms by 7% the same architecture trained using only rainfall radar images. Merging rain and wind data has also proven to stabilize the training process and enabled significant improvement especially on the difficult-to-predict high precipitation rainfalls.
... Figure 4 shows an example of an hourly rainfall accumulation for a fast-moving line storm that is simply the sum of the observations. A field of motion vectors for the rainfall patterns can be estimated using the optical flow technique of Bowler et al. (2004). Figure 5 shows the effect of accounting for the motion of the field when making the accumulation. ...
Conference Paper
Full-text available
The Bureau of Meteorology has developed the Rainfields system to produce quantitative radar based rainfall estimates. This paper discusses some of the technical and scientific aspects of the rainfields system for quantitative rainfall estimation, including radar quality control, issues in converting reflectivity to rainfall, and other initiatives to improve the rainfall estimates. This paper also discusses how radar quantitative precipitation estimates are being tested on real time hydrologic models within the Bureau, and presents an example of their use.
... Precipitation nowcasting is usually performed in two steps through the extrapolation of radar observations [4,5,35]. First, the wind is estimated by comparing two or more precipitation fields as seen by radar. ...
Preprint
Full-text available
Precipitation nowcasting is a short-range forecast of rain/snow (up to 2 hours), often displayed on top of the geographical map by the weather service. Modern precipitation nowcasting algorithms rely on the extrapolation of observations by ground-based radars via optical flow techniques or neural network models. Dependent on these radars, typical nowcasting is limited to the regions around their locations. We have developed a method for precipitation nowcasting based on geostationary satellite imagery and incorporated the resulting data into the Yandex.Weather precipitation map (including an alerting service with push notifications for products in the Yandex ecosystem), thus expanding its coverage and paving the way to a truly global nowcasting service.
... Then, we tracked backward from this cell to find all related convective cells in the same hail case because report time was commonly lagging and there were also WERs in the cumulus stage. There are, however, automatic tracking algorithms such as thunderstorm identification, tracking, analysis, and nowcasting (TITAN) [48], SCIT [46], optical flow [49], and some other advanced methods [50,51]. Errors were unavoidable for automatic algorithms, especially on the condition that there was cell merging and splitting. ...
Article
Full-text available
The identification of some radar reflectivity signatures plays a vital role in severe thunderstorm nowcasting. A weak echo region is one of the signatures that could indicate updraft, which is a fundamental condition for hail production. However, this signature is underutilized in automatic forecasting systems due to the lack of a reliable detection method and the uncertain relationships between different weak echo regions and hail-producing thunderstorms. In this paper, three algorithms related to weak echo regions are proposed. The first is a quasi-real-time weak echo region morphology identification algorithm using the radar echo bottom height image. The second is an automatic vertical cross-section-making algorithm. It provides a convenient tool for automatically determining the location of a vertical cross-section that exhibits a visible weak echo region to help forecasters assess the vertical structures of thunderstorms with less time consumption. The last is a weak echo region quantification algorithm mainly used for hail nowcasting. It could generate a parameter describing the scale of a weak echo region to distinguish hail and no-hail thunderstorms. Evaluation with real data of the Tianjin radar indicates that the critical success index of the weak echo region identification algorithm is 0.61. Statistics on these data also show that when the weak echo region parameters generated by the quantification algorithm are in a particular range, more than 85% of the convective cells produced hail.
... Precipitation nowcasting is usually performed in two steps through the extrapolation of radar observations [4,5,35]. First, the wind is estimated by comparing two or more precipitation fields as seen by radar. ...
Conference Paper
Full-text available
Precipitation nowcasting is a short-range forecast of rain/snow (up to 2 hours), often displayed on top of the geographical map by the weather service. Modern precipitation nowcasting algorithms rely on the extrapolation of observations by ground-based radars via optical flow techniques or neural network models. Dependent on these radars, typical nowcasting is limited to the regions around their locations. We have developed a method for precipitation nowcasting based on geostationary satellite imagery and incorporated the resulting data into the Yandex.Weather precipitation map (including an alerting service with push notifications for products in the Yandex ecosystem), thus expanding its coverage and paving the way to a truly global nowcasting service.
... In the modified version of the S-PROG model (the STEPS model), the advection velocity is determined using the optical flow method described by Bowler, et al. [27]. The cascade is respectively advected using optical flow in Lagrangian coordinates and stochastically evolves over time according to a hierarchy of auto-regressive processes of order two [28]. ...
Article
Full-text available
Radar rainfall nowcasts are subject to many sources of uncertainty and these uncertainties change with the characteristics of a storm. The predictive skill of a radar rainfall nowcasting model can be difficult to understand as sometimes it appears to be perfect but at other times it is highly inaccurate. This hinders the decision making required for the early warning of natural hazards caused by rainfall. In this study we define radar spatial and temporal rainfall variability and relate them to the predictive skill of a nowcasting model. The short-term ensemble prediction system model is configured to predict 731 events with lead times of one, two, and three hours. The nowcasting skill is expressed in terms of six well-known indicators. The results show that the quality of radar rainfall nowcasts increases with the rainfall autocorrelation and decreases with the rainfall variability coefficient. The uncertainty of radar rainfall nowcasts also shows a positive connection with rainfall variability. In addition, the spatial variability is more important than the temporal variability. Based on these results, we recommend that the lead time for radar rainfall nowcasting models should change depending on the storm and that it should be determined according to the rainfall variability. Such measures could improve trust in the rainfall nowcast products that are used for hydrological and meteorological applications.
... The lifetime of convective thunderstorm cells can be as short as tens of minutes, posing a tremendous challenge to the spatio-temporal modeling of radar reflectivity data. Note that, although it has been pointed out that the errors in the linear extrapolation of the radar echo field assuming a persistent reflectivity level are mainly due to the growth and decay of reflectivity ( Browning et al., 1982), many operational radar-based QPF systems do not take into account the rapid small-scale growth and decay, such as the GANDOLF system in UK ( Bowler et al., 2004) and the SWIRLS system developed in Hong Kong ( Li and Lai, 2004). In addition, the rapid growth-decay of weather systems also makes the Numerical Weather Prediction (NWP) models ineffective in predicting the exact location and intensity of individual thunderstorms (RMI, 2008). ...
Preprint
This paper proposes a statistical modeling approach for spatio-temporal data arising from a generic class of convection-diffusion processes. Such processes are found in various scientific and engineering domains where fundamental physics imposes critical constraints on how data can be modeled and how statistical models should be interpreted. We employ the idea of spectrum decomposition to approximate the physical processes. However, unlike existing models which often assume constant convection-diffusion and zero-mean source-sink, we consider a more realistic scenario with spatially-varying convection-diffusion and nonzero-mean source-sink. As a result, the temporal evolution of spectrum coefficients is closely coupled with each other, which can be seen as the non-linear transfer or redistribution of energy across multiple scales. Because of the spatially-varying convection-diffusion, the space-time covariance is naturally non-stationary in space. A systematic approach is proposed to integrate the theoretical results into the framework of hierarchical dynamical spatio-temporal models. Statistical inference using computationally efficient algorithms is investigated. Some practical considerations related to computational efficiency are discussed in order to make the proposed approach practical. The advantages of the proposed approach are demonstrated by numerical examples, a case study, and comparison studies. Computer code and data are made available.
... Specifically, it extrapolates radar reflectivity by estimating the velocity fields. Despite its lower computational complexity, optical flow often shows better predictive performance than NWP for precipitation nowcasting [11,12]. Moreover, optical flow has been combined with NWP to realize the advantages of both approaches [13,14]. ...
Preprint
Full-text available
Deep learning has been successfully applied to precipitation nowcasting. In this work, we propose a pre-training scheme and a new loss function for improving deep-learning-based nowcasting. First, we adapt U-Net, a widely-used deep-learning model, for the two problems of interest here: precipitation nowcasting and precipitation estimation from radar images. We formulate the former as a classification problem with three precipitation intervals and the latter as a regression problem. For these tasks, we propose to pre-train the model to predict radar images in the near future without requiring ground-truth precipitation, and we also propose the use of a new loss function for fine-tuning to mitigate the class imbalance problem. We demonstrate the effectiveness of our approach using radar images and precipitation datasets collected from South Korea over seven years. It is highlighted that our pre-training scheme and new loss function improve the critical success index (CSI) of nowcasting of heavy rainfall (at least 10 mm/hr) by up to 95.7% and 43.6%, respectively, at a 5-hr lead time. We also demonstrate that our approach reduces the precipitation estimation error by up to 10.7%, compared to the conventional approach, for light rainfall (between 1 and 10 mm/hr). Lastly, we report the sensitivity of our approach to different resolutions and a detailed analysis of four cases of heavy rainfall.
... The components of the STEPS nowcast blend include the radar extrapolation forecast, the NWP forecast and stochastic noise. The radar component is achieved using semi-Lagrangian advection of a radar analysis (e.g. the Radarnet composite), at time t , using motion vectors derived using an Optical Flow technique (Bowler et al., 2004), based on analyses from times [t , t − ∆t , t − 2∆t ] (∆t = 5mi n. for Radarnet). The NWP component is derived from a high res-5 olution rainfall rate diagnostic that is output from an NWP model. ...
Article
Full-text available
This article begins with a review of the scale decomposition and stochastic noise generation aspects of the ensemble nowcasting process, as used in the Short Term Ensemble Prediction System (STEPS). Alternative methods are then suggested, with case study examples, which could lead to enhanced performance and may also be applicable in other fields. The standard FFT‐band‐pass‐filtering scale decomposition, essential to this process, is shown to be practically the same as a redundant Continuous Wavelet Transform (CWT). This immediately makes accessible a number of more advanced methods from the wavelet literature. Particular attention is given to directional wavelet transforms and their use for analysing anisotropic scaling behaviour, common in weather images. With a view to numerically efficient operational usage, the focus is then shifted to Discrete Wavelet Transforms (DWTs). While the standard DWT has a number of disadvantages, including limited orientational selectivity in two dimensions, the relatively recent Dual Tree Complex WT (DTCWT) overcomes most of these. Examples are given that show how its six‐fold directionality can be exploited, to create synthetic features that replace the unpredictable small‐scale parts of a (weather radar‐based) extrapolation nowcast with more realistic spatially‐varying anisotropy. Further, the Bounded Log‐normal Cascade noise model, as used in STEPS, is reconsidered and a proposal to use Universal Multifractal realizations is presented. For the noise parameters, some operationally‐viable estimation methods are considered, such as the Double Trace Moment technique and a structure function‐based approach, which could work on readily‐available DTCWT coefficients. This article is protected by copyright. All rights reserved.
... Sparse model differs from SparseSD that instead of using a constant displacement for each feature, Sparse model uses a linear regression model for every tracked features through time t − 10 to t. Dense and DenseROT models use Dense Inverse Search optical flow algorithm [32] for calculating displacement field using the previous t − 1 timestamps. The pixel values are then extrapolated according to the displacement field using a backward constant-vector [10] and semi-Lagrangian scheme [8] in case of Dense and DenseROT, respectively. It should be noted that a constant-vector approach does not allow for the representation of roational motion while a semi-Lagrangian consider rotational motion with the assumption of motion field to be persistent. ...
Article
Full-text available
Nowcasting of precipitation is a difficult spatiotemporal task because of the non-uniform characterization of meteorological structures over time. Recently, convolutional LSTM has been shown to be successful in solving various complex spatiotemporal based problems. In this research, we propose a novel precipitation nowcasting architecture ‘Convcast’ to predict various short-term precipitation events using satellite data. We train Convcast with ten consecutive NASA’s IMERG precipitation data sets each at intervals of 30 minutes. We use the trained neural network model to predict the eleventh precipitation data of the corresponding ten precipitation sequence. Subsequently, the predicted precipitation data are used iteratively for precipitation nowcasting of up to 150 minutes lead time. Convcast achieves an overall accuracy of 0.93 with an RMSE of 0.805 mm/h for 30 minutes lead time, and an overall accuracy of 0.87 with an RMSE of 1.389 mm/h for 150 minutes lead time. Experiments on the test dataset demonstrate that Convcast consistently outperforms other state-of-the-art optical flow based nowcasting algorithms. Results from this research can be used for nowcasting of weather events from satellite data as well as for future on-board processing of precipitation data.
... The main goal of STEPS is to generate ensembles of rainfall forecasts that exhibit similar space-time structures to those of observed rainfall over a range of space and time scales. Originally, this system blended an advection forecast from radar observations with a noise model possessing the space-time properties of observed rain fields (Bowler et al., 2004(Bowler et al., , 2006. This method has since been extended to allow radar and numerical weather prediction (NWP) forecasts to be blended (Seed et al., 2013). ...
Technical Report
Full-text available
Large verification of STEPS nowcasts done as part of preparing to run STEPS over the entire Australian network of 60 radars
... The latter uses a moving reference frame and seeks its optimal velocity by variational analysis, which can be iterated as the reference frame sequentially splits into many small frames at the targeted scale. Both area tracking methods have been utilized by several nowcasting systems as well, such as the Continuity of TREC Vectors (COTREC [17]), the McGill Algorithm for Precipitation Nowcasting by Lagrangian Extrapolation (MAPLE [18]), the Optical Flow Scheme for the Gandolf System [19], and the Collaborative Adaptive Sensing of the Atmosphere (CASA [20]). It is noteworthy that, trying to grasp high reflectivity signatures within the radar coverage area and meet the demand for fast computing, almost all radar echo extrapolation methods perform the horizontal extrapolation of composite reflectivity rather than three-dimensional reflectivity. ...
Article
Full-text available
This study utilized a radar echo extrapolation system, a high-resolution numerical model with radar data assimilation, and three blending schemes including a new empirical one, called the extrapolation adjusted by model prediction (ExAMP), to carry out 150 min reflectivity nowcasting experiments for various heavy rainfall events in Taiwan in 2019. ExAMP features full trust in the pattern of the extrapolated reflectivity with intensity adjustable by numerical model prediction. The spatial performance for two contrasting events shows that the ExAMP scheme outperforms the others for the more accurate prediction of both strengthening and weakening processes. The statistical skill for all the sampled events shows that the nowcasts by ExAMP and the extrapolation system obtain the lowest and second lowest root mean square errors at all the lead time, respectively. In terms of threat scores and bias scores above certain reflectivity thresholds, the ExAMP nowcast may have more grid points of misses for high reflectivity in comparison to extrapolation, but serious overestimation among the points of hits and false alarms is the least likely to happen with the new scheme. Moreover, the event type does not change the performance ranking of the five methods, all of which have the highest predictability for a typhoon event and the lowest for local thunderstorm events.
... e difference between these two models is the extrapolation step. e Dense model uses a constant-vector advection scheme [16], while the DenseRotation model uses a semi-Lagrangian advection scheme [17]. In our experiments, we feed the 15 radar echo maps of the past 1.5 hours into the Dense group models. ...
Article
Full-text available
Unpredicted precipitations, even mild, may cause severe economic losses to many businesses. Precipitation nowcasting is hence significant for people to make correct decisions timely. For traditional methods, such as numerical weather prediction (NWP), the accuracy is limited because the smaller scale of strong convective weather must be smaller than the minimum scale that the model can capture. And it often requires a supercomputer. Furthermore, the optical flow method has been proved to be available for precipitation nowcasting. However, it is difficult to determine the model parameters because the two steps of tracking and extrapolation are separate. In contrast, current machine learning applications are based on well-selected full datasets, ignoring the fact that real datasets quite often contain missing data requiring extra consideration. In this paper, we used a real Hubei dataset in which a few radar echo data are missing and proposed a proper mechanism to deal with the situation. Furthermore, we proposed a novel mechanism for radar reflectivity data with single altitudes or cumulative altitudes using machine learning techniques. From the experimental results, we conclude that our method can predict future precipitation with a high accuracy when a few data are missing, and it outperforms the traditional optical flow method. In addition, our model can be used for various types of radar data with a type-specific feature extraction, which makes the method more flexible and suitable for most situations.
... Once the correspondence and their displacement have been established, the position of these cells is extrapolated to the desired time horizon. The second category relies on the estimation of a dense field of apparent velocities at each pixel of the image and modeled by the optical flow [7,8]. The forecast is also obtained by an extrapolation in time and advection of the last observation with the velocity field. ...
Preprint
Short or mid-term rainfall forecasting is a major task for several environmental applications, such as agricultural management or monitoring flood risks. Existing data-driven approaches, especially deep learning models, have shown significant skill at this task, using only rain radar images as inputs. In order to determine whether using other meteorological parameters such as wind would improve forecasts, we trained a deep learning model on a fusion of rain radar images and wind velocity produced by a weather forecast model. The network was compared to a similar architecture trained only on rainfall data, to a basic persistence model and to an approach based on optical flow. Our network outperforms the F1-score calculated for the optical flow on moderate and higher rain events for forecasts at a horizon time of 30 minutes by 8%. Furthermore, it outperforms the same architecture trained using only rainfalls by 7%.
... It is obtained from seven S-band radars, which are located at Guangzhou, Shenzhen, Shaoguan, etc. Radar echo reflectivity intensities are colored corresponding to the class thresholds defined in Figure 1. Traditional radar echo extrapolation includes centroid tracking [13], tracking radar echoes by cross-correlation (TREC) [14], and the optical flow method [15][16][17]. The centroid tracking method is mainly suitable for the tracking and short-term prediction of heavy rainfall with strong convection. ...
Article
Full-text available
Precipitation nowcasting by radar echo extrapolation using machine learning algorithms is a field worthy of further study, since rainfall prediction is essential in work and life. Current methods of predicting the radar echo images need further improvement in prediction accuracy as well as in presenting the predicted details of the radar echo images. In this paper, we propose a two-stage spatiotemporal context refinement network (2S-STRef) to predict future pixel-level radar echo maps (deterministic output) more accurately and with more distinct details. The first stage is an efficient and concise spatiotemporal prediction network, which uses the spatiotemporal RNN module embedded in an encoder and decoder structure to give a first-stage prediction. The second stage is a proposed detail refinement net, which can preserve the high-frequency detailed feature of the radar echo images by using the multi-scale feature extraction and fusion residual block. We used a real-world radar echo map dataset of South China to evaluate the proposed 2S-STRef model. The experiments showed that compared with the PredRNN++ and ConvLSTM method, our 2S-STRef model performs better on the precipitation nowcasting, as well as at the image quality evaluating index and the forecasting indices. At a given 45dBZ echo threshold (heavy precipitation) and with a 2 h lead time, the widely used CSI, HSS, and SSIM indices of the proposed 2S-STRef model are found equal to 0.195, 0.312, and 0.665, respectively. In this case, the proposed model outperforms the OpticalFlow method and PredRNN++ model.
... Traditionally, optical flow based models are the most popular techniques among classical methods for precipitation nowcasting tasks (Bowler, Pierce, & Seed, 2004;Li et al., 2018). However, machine learning and deep learning based approaches are dominating this field of research in recent years. ...
Article
Full-text available
Weather nowcasting consists of predicting meteorological components in the short term at high spatial resolutions. Due to its influence in many human activities, accurate nowcasting has recently gained plenty of attention. In this paper, we treat the nowcasting problem as an image-to-image translation problem using satellite imagery. We introduce Broad-UNet, a novel architecture based on the core UNet model, to efficiently address this problem. In particular, the proposed Broad-UNet is equipped with asymmetric parallel convolutions as well as Atrous Spatial Pyramid Pooling (ASPP) module. In this way, the Broad-UNet model learns more complex patterns by combining multi-scale features while using fewer parameters than the core UNet model. The proposed model is applied on two different nowcasting tasks, i.e. precipitation maps and cloud cover nowcasting. The obtained numerical results show that the introduced Broad-UNet model performs more accurate predictions compared to the other examined architectures.
... The movement will appear clearly if the images capture the raindrop front. However, the actual process includes horizontal movement, raindrop development (growth), evaporation back of raindrops (dissipation), and diffusion (spreading) [13][14][15]. Therefore, to estimate raindrop vertical time travel, it needs to evaluate more than one VCUT and CAPPI image. ...
... Indeed, precipitation maps based on radar images are obtained with a lower frequency (1 frame every 5-15 minutes) than video images (25-30 frames every second). For instance, some computer vision methods are based on optical flow (Bowler et al., 2004) which estimates object movement in a sequence of images. However, optical flow is unable to represent the sudden change in weather since it makes assumptions that are clearly violated, e.g., the amount of rain will not change over time. ...
Preprint
Full-text available
Precipitation nowcasting is of great importance for weather forecast users, for activities ranging from outdoor activities and sports competitions to airport traffic management. In contrast to long-term precipitation forecasts which are traditionally obtained from numerical models, precipitation nowcasting needs to be very fast. It is therefore more challenging to obtain because of this time constraint. Recently, many machine learning based methods had been proposed. We propose the use three popular deep learning models (U-net, ConvLSTM and SVG-LP) trained on two-dimensional precipitation maps for precipitation nowcasting. We proposed an algorithm for patch extraction to obtain high resolution precipitation maps. We proposed a loss function to solve the blurry image issue and to reduce the influence of zero value pixels in precipitation maps.
... The results show that the proposed algorithm can effectively track and predict severe storm events in the next few hours. Bowler et al. [31] proposed a new Gandalf system precipitation prediction scheme based on advection. The method does not need to divide the radar analysis into continuous rain areas (CRA) and uses smoothing constraints to diagnose the block advection velocity in rainfall analysis by using the idea of optical flow. ...
Article
Full-text available
The tremendous progress made in the field of deep learning allows us to accurately predict precipitation and avoid major and long-term disruptions to the entire socio-economic system caused by floods. This paper presents an LSTM–CP combined model formed by the Long Short-Term Memory (LSTM) network and Chebyshev polynomial (CP) as applied to the precipitation forecast of Yibin City. Firstly, the data are fed into the LSTM network to extract the time-series features. Then, the sequence features obtained are input into the BP (Back Propagation) neural network with CP as the excitation function. Finally, the prediction results are obtained. By theoretical analysis and experimental comparison, the LSTM–CP combined model proposed in this paper has fewer parameters, shorter running time, and relatively smaller prediction error than the LSTM network. Meanwhile, compared with the SVR model, ARIMA model, and MLP model, the prediction accuracy of the LSTM–CP combination model is significantly improved, which can aid relevant departments in making disaster response measures in advance to reduce disaster losses and promote sustainable development by providing them data support.
... Traditionally, optical flow based models are the most popular techniques among classical methods for precipitation nowcasting tasks [36,37]. However, machine learning and deep learning based approaches are dominating this field of research in recent years. ...
Preprint
Full-text available
Weather nowcasting consists of predicting meteorological components in the short term at high spatial resolutions. Due to its influence in many human activities, accurate nowcasting has recently gained plenty of attention. In this paper, we treat the nowcasting problem as an image-to-image translation problem using satellite imagery. We introduce Broad-UNet, a novel architecture based on the core UNet model, to efficiently address this problem. In particular, the proposed Broad-UNet is equipped with asymmetric parallel convolutions as well as Atrous Spatial Pyramid Pooling (ASPP) module. In this way, The the Broad-UNet model learns more complex patterns by combining multi-scale features while using fewer parameters than the core UNet model. The proposed model is applied on two different nowcasting tasks, i.e. precipitation maps and cloud cover nowcasting. The obtained numerical results show that the introduced Broad-UNet model performs more accurate predictions compared to the other examined architectures.
... A feature common to modern operational nowcasting systems (including NWCSAF/ GEO) is the automated forward propagation of existing storms through the application of storm motion or optical flow vectors (Dixon and Wiener, 1993;Hand, 1996;Mueller et al., 2003;Sills et al., 2003;Bowler et al., 2004Bowler et al., , 2006Brovelli et al., 2005;James et al., 2018). While extrapolation of this type can work very well, it has limitations. ...
Article
Full-text available
The high frequency of intense convective storms means there is a great demand to improve predictions of high‐impact weather across Africa. The low skill of numerical weather prediction over Africa, even for short lead times highlights the need to deliver nowcasting based on satellite data. The Global Challenges Research Fund African SWIFT (Science for Weather Information and Forecasting Techniques) project is working to improve the nowcasting of African convective systems and so the ability to provide timely warnings.
... The main goal of STEPS is that any predicted rainfall field exhibits similar space-time structures to those of observed rainfall fields over a range of space and time scales. STEPS was formulated to blend an advection forecast from radar observations with a noise model possessing the space-time properties of observed rain fields [15,27,28]. In this study, the STEPS method was used to predict ensembles of rainfall fields up to 90 minutes ahead. ...
Article
In this paper, we propose a novel approach to produce attenuation forecasts for microwave links using a probabilistic approach. It uses ensembles of forecast rainfall fields to easily derive attenuation forecasts for specific frequencies. The proposed approach uses the Short Term Ensemble Prediction System (STEPS) to generate ensembles of high, spatial and temporal, resolution forecast rainfall fields based on observed weather radar fields with lead times of 15 to 90 minutes. Attenuation forecasts could eventually be used by telecommunication operators to drive the operation of wireless networks and ensure their maintenance during severe and extreme rainfall events. This study used 109 microwave links ranging from 15 to 40 GHz to verify the results of this probabilistic attenuation forecast. Results suggest that the STEPS-based attenuation forecasts were within the narrow span of the 90 percent confidence region for all microwave links tested up to 30-minute lead time, decreasing for longer lead times. Examples of how the proposed approach can be used to derive detailed probabilistic attenuation forecast for multiple lead times within a domain of few kilometers, as well as the probability of attenuation maps for large areas are shown.
... On average (over +30, +60, and +90 minute lead times), CNC-D and CNC-R architectures have 16% and 8% better performance than the persistence benchmark, 18% and 9% better performance than LR, and 3% and 2% better performance than RF. [32] In terms of FAR (Figure 7-B; optimal value is 0.0), the CNC-R architecture and the persistence BM approach achieve the best performance (mean FAR values are 0.271 and 0.275, respectively). Notably, the DL and ML models tend to produce very small precipitation rate values (e.g., 1E-5), which affects their performance in terms of categorical metrics, but this problem does not plague the persistence benchmark BM approach which simply repeats the IMERG estimate. ...
Preprint
Full-text available
Accurate and timely estimation of precipitation is critical for issuing hazard warnings (e.g., for flash floods or landslides). Current remotely sensed precipitation products have a few hours of latency, associated with the acquisition and processing of satellite data. By applying a robust nowcasting system to these products, it is (in principle) possible to reduce this latency and improve their applicability, value, and impact. However, the development of such a system is complicated by the chaotic nature of the atmosphere, and the consequent rapid changes that can occur in the structures of precipitation systems In this work, we develop two approaches (hereafter referred to as Nowcasting-Nets) that use Recurrent and Convolutional deep neural network structures to address the challenge of precipitation nowcasting. A total of five models are trained using Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG) precipitation data over the Eastern Contiguous United States (CONUS) and then tested against independent data for the Eastern and Western CONUS. The models were designed to provide forecasts with a lead time of up to 1.5 hours and, by using a feedback loop approach, the ability of the models to extend the forecast time to 4.5 hours was also investigated. Model performance was compared against the Random Forest (RF) and Linear Regression (LR) machine learning methods, and also against a persistence benchmark (BM) that used the most recent observation as the forecast. Independent IMERG observations were used as a reference, and experiments were conducted to examine both overall statistics and case studies involving specific precipitation events. Overall, the forecasts provided by the Nowcasting-Net models are superior, with the Convolutional Nowcasting Network with Residual Head (CNC-R) achieving 25%, 28%, and 46% improvement in the test set MSE over the BM, LR, and RF approaches, respectively, for the Eastern CONUS. Results of further testing over the Western CONUS (which was not part of the training data) are encouraging, and indicate the ability of the proposed models to learn the dynamics of precipitation systems without having explicit access to motion vectors and other auxiliaries as features, and to then generalize to different hydro-geo-climatic conditions.
... Latent Gaussian fields are also the core of the model proposed by Niemi et al. (2016), which generalized the so-called STEPS model (Short Term Ensemble Prediction System; Bowler et al., 2004;Seed et al., 2013) introducing scale-varying anisotropy by GSI (hereinafter STEPS-GSI). Similar to STREAP, the STEPS-GSI model assumes lognormal distribution for positive rainfall, and it requires the generation of larger fields (as it uses FFT simulation) as well as a post-processing of simulated rainfall to adjust field mean and standard deviation. ...
Article
Full-text available
Realistic stochastic simulation of hydro-environmental fluxes in space and time, such as rainfall, is challenging yet of paramount importance to inform environmental risk analysis and decision making under uncertainty. Here, we advance random fields simulation by introducing the concepts of general velocity fields and general anisotropy transformations. This expands the capabilities of the so-called Complete Stochastic Modeling Solution (CoSMoS) framework enabling the simulation of random fields (RF's) preserving: (a) any non-Gaussian marginal distribution, (b) any spatiotemporal correlation structure (STCS), (c) general advection expressed by velocity fields with locally varying speed and direction, and (d) locally varying anisotropy. We also introduce new copula-based STCS's and provide conditions guaranteeing their positive definiteness. To illustrate the potential of CoSMoS, we simulate RF's with complex patterns and motion mimicking rainfall storms moving across an area, spiraling fields resembling weather cyclones, fields converging to (or diverging from) a point, and colliding air masses. The proposed methodology is implemented in the freely available CoSMoS R package.
... Most of the optical flow estimation methods are efficient in capturing the motion of rigid body surfaces containing smooth regions. Precipitation images contain non-rigid regions with discontinuities as precipitation clouds grow or decay rapidly [33]. Therefore, we need a robust edge-preserving optical flow estimation method for tracking precipitation features. ...
Preprint
Full-text available
div>Radar-based precipitation nowcasting refers to predicting rain for a short period of time using radar reflectivity images. For dynamic nowcasting, motion fields can be extrapolated using an approximate and localized reduced-order model. Motion field estimation based on traditional Horn-Schunck (HS) algorithm suffers from over-smoothing at discontinuities in non-rigid and dissolving texture present in precipitation nowcasting products. An attempt to preserve the discontinuities using an L1 norm formulation in HS led to the use of Total Variation L1 norm (TVL1). In this paper, we propose a radar-based precipitation nowcasting model with TVL1-based estimation of motion field and Sparse Identification of Non-linear Dynamics (SINDy)-based estimation of non-linear dynamics. TVL1 is effective in preserving the edges especially in the case of the eye of typhoons and squall lines while estimating motion vectors. SINDy captures the non-linear dynamics and generates the subsequent update values for the motion field based on a reduced-order representation. Finally, the SINDy-generated ensemble of motion field is used along with the radar reflectivity image for generating precipitation nowcasts. We evaluated the effectiveness of TVL1 in preserving edges while capturing the motion field from non-rigid surfaces. The performance of the proposed TVL1-SINDy model in nowcasting weather events such as Typhoons and Squall lines are evaluated using performance metrics such as Mean Absolute Error (MAE), and Critical Success Index (CSI). Experimental results show that the proposed nowcasting system demonstrates better performance compared to the benchmark nowcasting models with lower MAE, higher CSI at higher lead times.</div
Article
Accurate and timely estimation of precipitation is critical for issuing hazard warnings (e.g., for flash floods or landslides). Current remotely sensed precipitation products have a few hours of latency, associated with the acquisition and processing of satellite data. By applying a robust nowcasting system to these products, it is (in principle) possible to mitigate this latency and improve their applicability, value, and impact. However, the development of such a system is complicated by the chaotic nature of the atmosphere, lack of sufficient knowledge about the evolution of precipitation systems based on previous observations, and the consequent rapid changes that can occur in the structures of precipitation systems. In this work, we develop two approaches (hereafter referred to as NowCasting-nets) that use Recurrent and Convolutional deep neural network structures to address the challenge of precipitation nowcasting. A total of five models are trained using Global Precipitation Measurement (GPM) Integrated Multi-satellitE Retrievals for GPM (IMERG) precipitation data over the Eastern Contiguous United States (CONUS) and then tested against independent data for the Eastern and Western CONUS. The models were designed to provide forecasts with a lead time of up to 1.5 hours and, by using a feedback loop approach, the ability of the models to extend the forecast time to 4.5 hours was also investigated. Performance of the models was compared against the Random Forest (RF) and Linear Regression (LR) machine learning methods, a persistence benchmark (BM) that uses the most recent observation as the forecast, and Optical Flow (OF). Independent IMERG observations were used as a reference, and experiments were conducted to examine both overall statistics and case studies involving specific precipitation events. Overall, the forecasts provided by the NowCasting-net models are superior, with the Convolutional NowCasting-net (CNC) achieving 42%, 24%, 18%, and 16% improvement on the test set MSE over the BM, LR, RF, and OF models, respectively, for the Eastern CONUS. Results of further testing over the Western CONUS (which was not part of the training data) are encouraging and indicate the ability of the proposed models to learn the dynamics of precipitation systems without having explicit access to motion vectors and other auxiliary features, and to then generalize to different hydro-geo-climatic conditions.
Article
A rain-type adaptive pyramid Kanade-Lucas-Tomasi (A-PKLT) optical flow method for radar echo extrapolation is proposed. This method introduces a rain-type classification algorithm that can classify radar echoes into six types: convective, stratiform, surrounding convective, isolated convective core, isolated convective fringe, and weak echoes. Then, new schemes are designed to optimize specific parameters of the PKLT optical flow based on the rain type of the echo. At the same time, the gradients of radar reflectivity in the fringe positions corresponding to all types of rain echoes are increased. As a result, corner points that are characteristic points used for PKLT optical flow tracking in the surrounding area will be increased. Therefore, more motion vectors are purposefully obtained in the whole radar echo area. This helps to describe the motion characteristics of the precipitation more precisely. Then, the motion vectors corresponding to each type of rain echo are merged, and a denser motion vector field is generated by an interpolation algorithm on the basis of merged motion vectors. Finally, the dense motion vectors are used to extrapolate rain echoes into 0–60-min nowcasts by a semi-Lagrangian scheme. Compared with other nowcasting methods for four landfalling typhoons in or near Shanghai, the new optical flow method is found to be more accurate than the traditional cross-correlation and optical flow methods, particularly showing a clear improvement in the nowcasting of convective echoes on the spiral rainbands of typhoons.
Article
Precipitation nowcasting is an important task in operational weather forecasts. The key challenge of the task is the radar echo map extrapolation. The problem is mainly solved by an optical-flow method in existing systems. However, the method cannot model rapid and nonlinear movements. Recently, a convolutional gated recurrent unit (ConvGRU) method is developed, which aims to model such movements based on deep learning techniques. Despite the promising performance, ConvGRU tends to yield blurring extrapolation images and fails to multi-modal and skewed intensity distribution. To overcome the limitations, we propose in this letter a generative adversarial ConvGRU (GA-ConvGRU) model. The model is composed of two adversarial learning systems, which are a ConvGRU-based generator and a convolution neural network-based discriminator. The two systems are trained by playing a minimax game. With the adversarial learning scheme, GA-ConvGRU can yield more realistic and more accurate extrapolation. Experiments on real data sets have been conducted and the results demonstrate that the proposed GA-ConvGRU significantly outperforms state-of-the-art extrapolation methods ConvGRU and optical flow.
Preprint
Weather forecasting is a long standing scientific challenge with direct social and economic impact. The task is suitable for deep neural networks due to vast amounts of continuously collected data and a rich spatial and temporal structure that presents long range dependencies. We introduce MetNet, a neural network that forecasts precipitation up to 8 hours into the future at the high spatial resolution of 1 km$^2$ and at the temporal resolution of 2 minutes with a latency in the order of seconds. MetNet takes as input radar and satellite data and forecast lead time and produces a probabilistic precipitation map. The architecture uses axial self-attention to aggregate the global context from a large input patch corresponding to a million square kilometers. We evaluate the performance of MetNet at various precipitation thresholds and find that MetNet outperforms Numerical Weather Prediction at forecasts of up to 7 to 8 hours on the scale of the continental United States.
Article
Radar quantitative precipitation estimation (RQPE) is the most common measurement for area rainfall estimation with high spatial and temporal resolution. The radar reflectivity (Z) measured by the Doppler weather radar is strongly related to precipitation rate (R). However, conventional RQPE methods have limited capability of modeling the complex relationship between the radar echoes and the precipitation field. In this paper, we propose a graph neural network (GNN) based RQPE model named categorical node graph attention network (CNGAT) to model complex spatial-temporal features of precipitation field reflected by the radar echo field. CNGAT is derived from graph attention networks (GAT) utilizing attention mechanism to learn the importance of neighboring points to the central point, which is beneficial for learning varying local spatial patterns. Furthermore, CNGAT can handle multiple types of graph nodes by using different transform functions for different types of nodes, which makes it better at capturing diverse features of precipitation field indicated by strong and weak radar echo areas. The proposed model was trained and tested on radar network and rain gauge data distributed in East China during 2017 and 2018. The results of several experiments show that CNGAT greatly improves the estimation precision and detection rate than Z-R relation models and conventional data-driven RQPE methods, and alleviates under-estimation of higher precipitation rates, which validates that CNGAT can effectively represent complex spatial-temporal features of precipitation field.
Article
PurposeComplex fetal behavior involving multiple parts of the body, called general movement (GM), has been considered an essential predictor of neurological functional development because it directly reflects the integrity of the brain and central and peripheral nervous systems. We have developed a novel method for quantitative analysis of fetal behavior using four-dimensional ultrasound (4DUS) and conducted a pilot study for quantitative assessment of fetal GM in the early second trimester.Methods All subjects underwent 4DUS to depict the whole fetal body, and maximum velocity (MAXV), median velocity (MV), average velocity (AV), and mode velocity (MOV) were calculated by utilizing optical flow analysis. Receiver operating characteristic (ROC) curve analysis was performed to analyze the optimal speed parameters for detecting GM in the fetus. The Mann–Whitney U test was used to validate MAXV, AV, and MV ability to detect fetal GM.ResultsThe presence of fetal GMs and the absence of fetal GMs were 226 and 107, respectively, based on optical flow analysis. Mann–Whitney U test revealed a significant difference in the presence or absence of fetal GM in MAXV, MV, AV, and MOV. ROC analysis showed that the area under the curve (AUC) of MAXV was 0.959; the threshold was 0.421, the sensitivity was 86%, and the specificity was 93%. In contrast, the AUC/threshold for AV and MV was 0.700/0.110 (sensitivity 71% and specificity 76%) and 0.521/0.119 (sensitivity 21% and specificity 90%), respectively. Spearman's rank correlation analysis also showed a weak negative correlation between GM and MAXV (r = − 0.235, P < 0.01) and AV (r = − 0.28, P < 0.01).Conclusion In this study, we conducted a quantitative analysis of fetal behavior based on optical flow using 4DUS and demonstrated that it was highly accurate for detecting GMs and for evaluating developmental changes in GMs. The implementation of quantitative analysis of fetal GMs in the early second trimester has been very preliminary, and there is much debate on how it will be clinically applied to perinatal assessment.
Article
Full-text available
Subdaily rainfall extremes have been found to intensify, both from observations and climate model simulations, but much uncertainty remains regarding future changes in the spatial structure of rainfall events. Here, future changes in the characteristics of heavy summer rainfall are analyzed by using two sets (1980–2000, 2060–2080) of 12-member 20-year-long convection-permitting ensemble simulations (2.2 km, hourly) over the UK. We investigated how the peak intensity, spatial coverage and the speed of rainfall events will change and how those changes jointly affect hourly extremes at different spatial scales. We found that in addition to the intensification of heavy rainfall events, the spatial extent tends to increase in all three subregions, and by up to 49.3% in the North-West. These changes act to exacerbate intensity increases in extremes for most of spatial scales (North: 30.2%–34.0%, South: 25.8%). The increase in areal extremes is particularly pronounced for catchments with sizes 20–500 km².
Article
Deep learning has been successfully applied to precipitation nowcasting. In this work, we propose a pre-training scheme and a new loss function for improving deep-learning-based nowcasting. First, we adapt U-Net, a widely-used deep-learning model, for the two problems of interest here: precipitation nowcasting and precipitation estimation from radar images. We formulate the former as a classification problem with three precipitation intervals and the latter as a regression problem. For these tasks, we propose to pre-train the model to predict radar images in the near future without requiring ground-truth precipitation, and we also propose the use of a new loss function for fine-tuning to mitigate the class imbalance problem. We demonstrate the effectiveness of our approach using radar images and precipitation datasets collected from South Korea over seven years. It is highlighted that our pre-training scheme and new loss function improve the critical success index (CSI) of nowcasting of heavy rainfall (at least 10 mm/hr) by up to 95.7% and 43.6%, respectively, at a 5-hr lead time. We also demonstrate that our approach reduces the precipitation estimation error by up to 10.7%, compared to the conventional approach, for light rainfall (between 1 and 10 mm/hr). Lastly, we report the sensitivity of our approach to different resolutions and a detailed analysis of four cases of heavy rainfall.
Article
Full-text available
Due to the unpredictable swings in climatic and atmospheric conditions, weather forecasting is becoming a more important topic of study. Scientists have been developing novel strategies for training models to achieve accuracy over nonlinear statistical datasets for the past few decades in order to prevent future environmental damage and world disaster. Predicting the climatic condition in advance is easy for farmers to know the favorable climatic conditions for the crops to be grown, which leads to higher yields. Agriculture, which plays a crucial role in the Indian economy, predicts the climatic condition in advance is easy for farmers to know the favorable climatic conditions for the crops to be grown, which leads to higher yields. Artificial intelligence and machine learning have added a new dimension to the area of weather forecasting, requiring only a few perplexing mathematical equations. This study examines a variety of traditional strategies, ranging from classical weather forecasting to modern methodology such as data mining and artificial intelligence. This research also shows a proposed model that predicts with high accuracy and can be used in various time series forecasting applications.
Article
Short-term intense precipitation (SIP; i.e., convective precipitation exceeding 20 mm h ⁻¹ ) nowcasting is important for urban flood warning and natural hazards management. This paper presents an algorithm for coupling automatic weather station data and single-polarization S-band radar data with a graph model and a random forest for the nowcasting of SIP. Different from the pixel-by-pixel precipitation nowcasting algorithm, this algorithm takes the convective cells as the basic units to consider their interactions and focuses on multicell convective systems. In particular, the following question could be addressed: Will a multicell convective system cause SIP events in the next hour? First, a method based on spatiotemporal superposition between cells is proposed for multicell systems identification. Then, the graph model is used to represent cell physical attributes and the spatial distribution of the entire system. For each graph model, a fusion operation is used to form a 42-dimensional graph feature vector. Finally, combined with the machine learning approaches, a random forest classifier is trained with the graph feature vector to predict the precipitation. In the experiment, this algorithm achieves a probability of detection (POD) of 79.2% and a critical success index (CSI) of 68.3% with the data between 2015 and 2016 in North China. Compared with other precipitation nowcasting algorithms, the graph model and random forest could predict SIP events more accurately and produce fewer false alarms.
Article
Full-text available
Short-duration rainfall characteristics in the form of certain intensity, time, and spatial distribution become valuable contribution for lahar flow disaster mitigation in a mountainous region. Due to mitigation purpose, such information can be provided through the rainfall nowcasting process. One of the promising rainfall nowcasting applications is the extrapolation-based method. Rain motion tracking is a crucial part of the rainfall nowcasting based on this method. This paper discusses the application of Pyramid Lucas-Kanade Optical Flow (PLKOF) method on the rain motion tracking analysis using 150x150m resolution radar image. The study of rain motion tracking is carried out using 112 successive rainfall images with 10-minutes time interval originating from Mt. Merapi X-band multiparameter radar. The rainfall movement patterns in short duration are presented in the displacement vector (u,v) images and scatter diagrams of rain motions at x- and y-directions. From the simulations, it was found that the average displacement of rain motions in the Mt. Merapi region is 9 pixels (8.3 km/h) with the dominant direction is northeast. The results show that PLKOF is relatively good at detecting small displacements, yet unable to identify the occurrence of rain growth and decay properly. The ability of PLKOF method in predicting the position of rain cell displacement is satisfied as indicated by the POD, CSI, and FAR indexes.
Article
Nowcasting based on weather radar uses the current and past observations to make estimations of future radar echoes. There are many types of operationally deployed nowcasting systems, but none of them are currently based on deep learning, despite it being an active area of research in the last few years. This paper explores deep learning models as alternatives to current methods by proposing different architectures and comparing them against some operational nowcasting systems. The methods proposed here, harnessing residual convolutional encoder-decoder architectures, reach a level of performance expected of current systems and in certain scenarios can even outperform them. Finally, some of the potential drawbacks of using deep learning are analyzed. No decay in the performance on a different geographical area from where the models were trained was found. No edge or checkerboard artifact, common in convolutional operations, was found that affects the nowcasting metrics.
Article
Full-text available
The lifetime of precipitation patterns in Eulerian and Lagrangian space derived from continental-scale radar images is used as a measure of predictability. A three-step procedure is proposed. First, the motion field of precipitation is determined by variational radar echo tracking. Second, radar reflectivity is advected by means of a modified semi-Lagrangian advection scheme assuming stationary motion. Third, the Eulerian and Lagrangian persistence forecasts are compared to observations to calculate the lifetime and other measures of predictability. The procedure is repeated with images that have been decomposed according to scales to describe the scale-dependence of predictability. The analysis has a threefold application: (i) determine the scale-dependence of predictability, (ii) set a standard against which the skill for quantitative precipitation forecasting by numerical modeling can be evaluated, and (iii) extend nowcasting by optimal extrapolation of radar precipitation patterns. The methodology can be applied to other field variables such as brightness temperatures of weather satellites imagery.
Article
Full-text available
Accurate storm identification and tracking are basic and essential parts of radar and severe weather warning operations in today's operational meteorological community. Improvements over the original WSR-88D storm series algorithm have been made with the Storm Cell Identification and Tracking algorithm (SCIT). This paper discusses the SCIT algorithm, a centroid tracking algorithm with improved methods of identifying storms (both isolated and clustered or line storms). In an analysis of 6561 storm cells, the SCIT algorithm correctly identified 68% of all cells with maximum reflectivities over 40 dBZ and 96% of all cells with maximum reflectivities of 50 dBZ or greater. The WSR-88D storm series algorithm performed at 24% and 41%, respectively, for the same dataset. With better identification performance, the potential exists for better and more accurate tracking information. The SCIT algorithm tracked greater than 90% of all storm cells correctly. The algorithm techniques and results of a detailed performance evaluation are presented. This algorithm was included in the WSR-88D Build 9.0 of the Radar Products Generator software during late 1996 and early 1997. It is hoped that this paper will give new users of the algorithm sufficient background information to use the algorithm with confidence.
Article
Full-text available
Motion vectors of radar echo patterns can be obtained by applying a cross-correlation method (e.g., the TREC method) to radar data collected several minutes apart Here an extension of TREC, called COTREC, is presented. Based on constraints and a variational technique, this extension is an efficient objective analysis method for smoothing the motion vectors and forcing them to fulfill the continuity equation. COTREC corrects the apparently wrong vectors that are often caused by failures of TREC. This allows us to identify regions of growth and decay of radar echoes.For different types of precipitation (convective and widespread), radar data were collected for evaluation of COTREC in complex orography. A comparison between the radial velocity components of retrieved fields of echo motion and the measured Doppler velocity has been made. A marked reduction of the differences with respect to the measured Doppler field was obtained for COTREC, as compared to TREC vectors.A retardation of COTREC-derived motion compared to Doppler-derived motion was found in orographic precipitation. This retardation may have two causes: 1) a tendency of radar patterns to become stationary (triggered) on upsloping orography; and 2) the influence of ground clutter and shielding, also highly correlated with orography. While the first reflects the fact that propagation of echoes (by growth/decay) and translation of echoes (with the wind) are two different phenomena, the second cause is an artifact produced by the method of observation (radar) but mitigated with Doppler techniques (by suppressing the stationary ground clutter).COTREC may be useful for nowcasting, especially in orographically complex areas: for orographic precipitation as well as for severe convective storms, the technique predicts the echo development approximately 20 min ahead, and there is good hope to extend the forecasting period.
Article
Full-text available
This paper reviews the status of forecasting convective precipitation for time periods less than a few hours (nowcasting). Techniques for nowcasting thunderstorm location were developed in the 1960s and 1970s by extrapolating radar echoes. The accuracy of these forecasts generally decreases very rapidly during the first 30 min because of the very short lifetime of individual convective cells. Fortunately more organized features like squall lines and supercells can be successfully extrapolated for longer time periods. Physical processes that dictate the initiation and dissipation of convective storms are not necessarily observable in the past history of a particular echo development; rather, they are often controlled by boundary layer convergence features, environmental vertical wind shear, and buoyancy. Thus, successful forecasts of storm initiation depend on accurate specification of the initial thermodynamic and kinematic fields with particular attention to convergence lines. For these reasons the ability to improve on simple extrapolation techniques had stagnated until the present national observational network modernization program. The ability to observe small-scale boundary layer convergence lines is now possible with operational Doppler radars and satellite imagery. In addition, it has been demonstrated that high-resolution wind retrievals can be obtained from single Doppler radar. Two methods are presently under development for using these modern datasets to forecast thunderstorm evolution: knowledge-based expert systems and numerical forecasting models that are initialized with radar data. Both these methods are very promising and progressing rapidly. Operational tests of expert systems are presently taking place in the United Kingdom and in the United States.
Article
Full-text available
Digital remotely sensed observations of the atmosphere, particularly from radars and satellites, have become increasingly available over the last 15 years. Together with developments in computer technology this has stimulated the design and operation of a variety of systems to exploit these data in weather forecasting. Growing awareness of the importance of detailed site specific weather information and forecasts from zero to a few hours ahead has led to the emergence of a particular kind of forecasting called nowcasting which depends on the exceptionally detailed knowledge of the current pattern of weather that remote sensing can provide. This type of weather forecasting is reviewed, with emphasis on the measurement and extrapolation up to about 2 hours ahead of fields of various weather parameters, especially rainfall. Trends in the design of nowcasting systems are discussed, and potential benefits summarized.
Article
Full-text available
In December 1999, central Europe was hit by three violent storms, which claimed more than 130 lives and caused about 13 billion Euros worth of total economic losses. The first of these storms hit Denmark and the northern-most part of Germany on 3 December 1999 (Fig. l(a)). It was associated with a cyclone named Anatol by the German Weather Service ec 1999 1800 GMT 3 1 27 Dec 1999 1800 GMT (DWD) (the DWD gives a name to each North Atlantic cyclone). At the Danish and German North Sea coast, gales of more than 50 m s ~ (97kn) and a record storm surge were observed. Extreme wind speeds also occurred in central and eastern Denmark, with gusts as strong as 43m s (83kn) in Copenhagen. According to a report issued by the Danish 4 4 26 Dec 1999 0600 GMT
Article
Full-text available
Digital weather radar data have been used with a simple pattern recognition procedure to automatically generate precipitation forecasts in the zero to three hours range. Such a technique has been in real time operation for two years. The verification of the procedure has led to a preliminary `radar climatology' for the Montreal area in the form of a map of areas showing a predominant growth or decay of precipitation patterns.
Article
Full-text available
Four methods for retrieval of the horizontal wind field are described and compared using single-Doppler observations of a sea-breeze front measured during the Convective and Precipitation/Electrification Experiment. The first method examined is the TREC (tracking radar echoes by correlation) technique similar to the one proposed by Tuttle and Foote. Two other methods, similar to TREC, in which wind vectors are estimated by minimizing the difference between successive patterns of reflectivity, are then examined. These methods conceptually link the TREC method and the velocity volume processing (VVP) approach to the variational wind retrieval method described here. The variational formulation uses the conservation of reflectivity and the radial momentum equation as physical constraints and in this way it incorporates the concepts on which TREC and VVP are based. The performance of the methods is compared using the dual-Doppler wind analysis as ground truth. Results show that the variational method ...
Article
Full-text available
A variational method for retrieval of the three-dimensional wind field from single-Doppler radar observations is developed and tested. The method uses the conservation equation for reflectivity and the continuity equation as a constraining model. Weak and strong constraint formalisms in data analysis am reviewed and compared using a one-dimensional advection equation for reflectivity considered as a passive tracer. The authors show that a model equation should be used as a weak constraint when the model does not predict exactly the evolution of the observations (such as the conservation equation for reflectivity). Consequently, the variational method presented here combines both formalisms: the conservation equation for reflectivity is used as a weak constraint, while the continuity equation is used as a strong constraint. The method is applied to retrieve detailed three-dimensional wind field of a microburst observed by two C-band Doppler radars during the Phoenix II Convective Boundary Layer Experiment. Retrieved wind fields are compared with dual-Doppler wind analysis. Results of experiments show that the cost function has multiple minima, and consequently retrievals are sensitive to the initial guess. To find the true minimum the retrieval is performed from large to small scale. The results are very encouraging.
Article
Full-text available
This paper formulates the optic flow problem as a set of over-determined simultaneous linear equations. It then introduces and studies two new robust optic flow methods. The first technique is based on using the Least Median of Squares (LMedS) to detect the outliers. Then, the inlier group is solved using the least square technique. The second method employs a new robust statistical method named the Least Median of Squares Orthogonal Distances (LMSOD) to identify the outliers and then uses total least squares to solve the optic flow problem. The performance of both methods are studied by experiments on synthetic and real image sequences. These methods outperform other published methods both in accuracy and robustness.
Article
Full-text available
Optical flow cannot be computed locally, since only one independent measurement is available from the image sequence at a point, while the flow velocity has two components. A second constraint is needed. A method for finding the optical flow pattern is presented which assumes that the apparent velocity of the brightness pattern varies smoothly almost everywhere in the image. An iterative implementation is shown which successfully computes the optical flow for a number of synthetic image sequences. The algorithm is robust in that it can handle image sequences that are quantized rather coarsely in space and time. It is also insensitive to quantization of brightness levels and additive noise. Examples are included where the assumption of smoothness is violated at singular points or along lines in the image.
Article
Full-text available
An object-oriented verification procedure is presented for gridded quantitative precipitation forecasts (QPFs). It is carried out within the framework of “contiguous rain areas” (CRAs), whereby a weather system is defined as a region bounded by a user-specified isopleth of precipitation in the union of the forecast and observed rain fields. The horizontal displacement of the forecast is determined by translating the forecast rain field until the total squared difference between the observed and forecast fields is minimized. This allows a decomposition of total error into components due to: (a) location; (b) rain volume and (c) pattern.Results are first presented for a Monte Carlo simulation of 40,000 synthetic CRAs in order to determine the accuracy of the verification procedure when the rain systems are only partially observed due to the presence of domain boundaries. Verification is then carried out for operational 24-h forecasts from the Australian Bureau of Meteorology LAPS numerical weather prediction model over a four-year period. Forty-five percent of all rain events were well forecast by the model, with small location and intensity errors. Location error was generally the dominant source of QPF error, with the directions of most frequent displacement varying by region. Forty-five percent of extreme rainfall events (>100 mm d−1) were well forecast, but in this case the model's underestimation of rain intensity was the most frequent source of error.
Article
This study investigates ways of quantifying the skill in forecasts of dichotomous weather events. The odds ratio, widely used in medical studies, can provide a powerful way of testing the association between categorical forecasts and observations. A skill score can be constructed from the odds ratio that is less sensitive to hedging than previously used scores. Furthermore, significance tests can easily be performed on the logarithm of the odds ratio to test whether the skill is purely due to chance sampling. Functions of the odds ratio and the Peirce skill score define a general class of skill scores that are symmetric with respect to taking the complement of the event. The study illustrates the ideas using Finley's classic set of tornado forecasts.
Article
Comparisons are made between the characteristics of several types of rainbands observed in an extratropical cyclone and dynamical mechanisms relevant on the mesoscale.The warm-sector flow ahead of the cold front and above the cold-frontal zone aloft was unstable to conditional symmetric instability, and theoretical predictions for this mechanism am consistent with several aspects of the warm-sector and wide cold-frontal rainbands. In the case of the warm-sector rainbands, other mechanisms (e.g., wave-CISK and mixed dynamic/convective instabilities) may have also played a role.The core structure of the narrow cold-frontal rainbands appeared to be affected by an instability that derived its energy from the horizontal shear across the surface front. Also, many aspects of the narrow cold-frontal rainband were similar to a density current. Shear-induced gravity waves appeared to be responsible for the wavelike rainbands observed in the vicinity of the cold-frontal zone aloft.The orientation of the postfrontal rainbands suggests that energy from the mean flow was responsible for their organization. Convection, in the presence of horizontal temperature gradients and vertical shear, could explain the existence of the postfrontal rainbands through either wave-CISK or a mixed dynamic/convective instability. Since the postfrontal rainbands are often aligned along the thermal gradient, the symmetric instabilities may also play a role in their formation. Buoyant vertical motions under relatively uniform conditions can explain the hexagonally-shaped convective cells observed well behind the cold front.
Article
KNOWLEDGE of the kinematic structure of storms is important for understanding the internal physical processes. Radar has long provided information on the three-dimensional structure of storms from measurements of the radar reflectivity factor alone. Early users of radar gave total storm movement only, whereas later radar data were used to reveal internal motions based on information related to cloud physics such as the three-dimensional morphology of the storm volume. Such approaches have continued by using the increasingly finer scale details provided by more modern radar systems. Both Barge and Bergwall2 and Browning and Foote3 have used fine scale reflectivity structure to determine airflow in hailstorms. Doppler radar added a new dimension to our capabilities through its ability to measure directly the radial component of motion of an ensemble of hydrometeor particles. Two4 or three5 Doppler radars collecting data in conjunction, the equation of mass continuity, and an empirical radar reflectivity–terminal velocity relationship have enabled the estimation of the full three-dimensional airflow fields in parts of storms. Because of the inherent advantage of Doppler radar in motion detection, little effort has been directed toward developing objective schemes of determining internal storm motions with conventional meteorological radars. Pattern recognition schemes using correlation coefficient techniques6, Fourier analysis7, and gaussian curve fitting8 have been used with radar and satellite data, but primarily for detecting overall storm motions, echo merging and echo splitting. Here we describe an objective use of radar reflectivity factor data from a single conventional weather radar to give information related to the three-dimensional motions within a storm.
Article
Accurate forecasting of heavy showers and thunderstorms with associated hazards is vitally important for many business sectors and national utilities. In the UK a fully automated procedure is being developed in the Met. Office for the National Rivers Authority. The GANDOLF (Generating Advanced Nowcasts for Deployment in Operational Land surface Flood forecasting) system seeks to provide warnings of heavy rain and forecast accumulations in sensitive river catchments to flood hydrologists. GANDOLF will automatically choose the most appropriate nowcasting technique depending upon synoptic conditions. In a convective situation an important method available to GANDOLF is an object-oriented nowcasting procedure. Multi-beam, high resolution radar data and Meteosat IR satellite data are used to analyse convective cells in all stages of growth; subsequent movement and development up to 3 hours ahead is then predicted using a conceptual life-cycle model combined with mesoscale NWP data. This paper describes the object-oriented technique and demonstrates its usefulness in a severe convective situation with a case study.
Article
During the past decade hydrologists have become increasingly aware of the problems of fluvial flood prediction during periods of intense convection, particularly in urbanised catchments whose rainfall-runoff responses tend to be rapid. Existing approaches to deterministic, short-range rainfall prediction are often deficient in their treatment of convective precipitation because they cannot resolve individual convective clouds or effectively model their evolution. In 1994 the UK Met. Office established a joint R&D programme with the Environment Agency (responsible for flood prediction in England and Wales) to explore the benefits of an Object-Oriented conceptual Model (OOM) of convection in the nowcasting of fluvial floods. This involved the development of an automated nowcasting system (GANDOLF) designed to run the OOM during episodes of air mass convection. This paper describes the structure and function of the GANDOLF system and compares the performance of the OOM with that of two other precipitation models routinely used by Thames Region of the Agency. Copyright © 2000 Royal Meteorological Society
Article
A very short range forecasting system has been developed which integrates nowcasting techniques with Numerical Weather Prediction (NWP) model products to provide forecasts over the UK and surrounding waters up to six hours ahead. There are three main components, producing analyses and forecasts of precipitation, cloud and visibility, respectively. The precipitation rate analysis uses processed radar and satellite data, together with surface reports and NWP fields. The forecast is based on an object advection technique, modified for growth and decay using model products. Related variables, such as precipitation type, are also diagnosed using the NWP fields. The cloud analysis is based largely on satellite imagery and surface reports, the forecast being carried out in a similar way to precipitation rate. The visibility analysis combines surface reports with NWP model fields and satellite imagery: Meteosat during the day and NOAA–AVHRR at night. The forecast is an extrapolation using trends from the NWP model, and relaxing towards the model values themselves. Results show a substantial improvement over both persistence and raw NWP model products. Copyright © 1998 Royal Meteorological Society
Article
Nowcasting for hydrological applications is discussed. The tracking algorithm extrapolates radar images in space and time. It originates from the pattern recognition techniques TREC (Tracking Radar Echoes by Correlation, Rinehart and Garvey, J. Appl. Meteor., 34 (1995) 1286) and COTREC (Continuity of TREC vectors, Li et al., Nature, 273 (1978) 287). To evaluate the quality of the extrapolation, a parameter scheme is introduced, able to distinguish between errors in the position and the intensity of the predicted precipitation. The parameters for the position are the absolute error, the relative error and the error of the forecasted direction. The parameters for the intensity are the ratio of the medians and the variations of the rain rate (ratio of two quantiles) between the actual and the forecasted image. To judge the overall quality of the forecast, the correlation coefficient between the forecasted and the actual radar image has been used.
Article
A physical model for high-resolution (Δx = 1–2 km) rainfall over complex terrain that was recently verified against radar-derived observations is shown to be capable of complementing rainfall normals in Israel. Two examples illustrate that even high resolution rain-gauge networks may miss important small-scale rainfall features over highly complec terrain, which are effectively detected by a simplified linear and adiabatic model.
Conference Paper
The authors present a scheme for motion detection exploiting temporal integration and local contextual information. A multiscale temporal decomposition is supplied to the original sequence. Change detection is performed using a likelihood test at each temporal scale. The decision process is formalized within a statistical regularization framework and takes advantage of a tracking module. Motion detection is achieved by minimizing an energy function. This function involves three terms, expressing (1) adequacy between temporal variations at different scales and motion labels, (2) local spatial regularization, and (3) coherence between temporal prediction of change area locations and motion labels. Experimental results on real scenes are reported
Article
In order to estimate both components of optical flow as well as their first spatio-temporal derivatives, it is postulated that the Optical Flow Constraint Equation (OFCE) is valid in a spatio-temporal neighborhood of pixels. So far, it has been tacitly assumed that the partial derivatives of the gray value distribution—which are required for this approach at the pixel positions involved—are independent from each other. It is shown how dropping this assumption affects the estimation procedure, based on well established approaches of estimation theory. The insight gained thereby is used to develop an approach towards merging image regions based on the compatibility of optical flow estimates obtained within the regions considered for merger.
Objective forecasting of heavy precipitation using numerical prediction model output. WMO symposium on the interpretation of broad-scale NWP products for local forecasting purposes
  • R Tatehira
  • T Nakayama
Tatehira, R., Nakayama, T., 1976. Objective forecasting of heavy precipitation using numerical prediction model output. WMO symposium on the interpretation of broad-scale NWP products for local forecasting purposes. Warsaw, 11 -16 October 1976, 121 -127.
A variational rainfall rate analysis scheme for Nimrod
  • B Wright
Wright, B., 1994. A variational rainfall rate analysis scheme for Nimrod, Forecasting Research Division technical report No. 143, Met Office, available at http://www.metoffice.com/ research/nwp/publications/papers/technical_reports/1994/ FRTR143/FRTR143.pdf
An objective extrapolation technique for semi-conservative fields with an application to radar patterns
  • G R Hilst
  • J A Russo
Hilst, G.R., Russo, J.A., 1960. An objective extrapolation technique for semi-conservative fields with an application to radar patterns. Tech. Memo 3, The Travelers Research Centre Hartford, CT, pp. 34.
Report by the independent review team to the board of the Environment Agency
  • P Bye
  • M Horner
Bye, P., Horner, M., 1998. Easter 1998 Floods, Volume 1, Report by the independent review team to the board of the Environment Agency, p. 29.
Severe storm radar echo motion and related severe weather events hazardous to aviation operation ESSA
  • P A Barclay
  • K E Wilk
Barclay, P.A., Wilk, K.E., 1990. Severe storm radar echo motion and related severe weather events hazardous to aviation operation ESSA Tech. Memo. ERLTM-NSSL 46, pp. 63.
Comparison of the performance of 2km resolution Object-Oriented Model and Nimrod advection precipitation nowcast schemes
  • C E Pierce
  • A M Cooper
Pierce, C.E., Cooper, A.M., 2000. Comparison of the performance of 2km resolution Object-Oriented Model and Nimrod advection precipitation nowcast schemes, Forecasting Research technical report No. 350, Met Office, available at http://www.metoffice.com/research/nwp/publications/papers/ technical_reports/2000/FRTR350/FRTR350.pdf
  • O ' Neil
O'Neil, P.V., 1995. Advanced Engineering Mathematics, fourth ed., PWS Publishing, pp. 189–210.