Content uploaded by Yousef Abu-Zidan

Author content

All content in this area was uploaded by Yousef Abu-Zidan on Apr 13, 2022

Content may be subject to copyright.

Accepted for publication in Fire Safety Journal (April 9, 2022)

https://doi.org/10.1016/j.firesaf.2022.103591

Machine learning-based surrogate model for calibrating

fire source properties in FDS models of façade fire tests

Hoang T. Nguyen1, Yousef Abu-Zidan1,2,*, Guomin Zhang1, Kate T. Q. Nguyen1

1 Civil and Infrastructure Engineering, RMIT University, VIC 3001, Australia

2 Department of Infrastructure Engineering, The University of Melbourne, VIC 3010, Australia

* Corresponding author. Email address: yousef.abu-zidan@rmit.edu.au

Abstract

Calibration is an important step in the development of predictive numerical models that involves

adjusting input parameters not easily measured in experiments to improve the predictive accuracy of

the numerical model compared to the real system. For complex models of façade fires, model calibration

can be difficult due to the large number of input parameters that need to be calibrated simultaneously.

This paper proposes a machine-learning-based surrogate modelling technique to help with calibrating

the fire source in simulations of façade fire tests. Two case studies are presented to assess the feasibility

of the proposed method: a simple fire source with a single burner surface based on the JIS A 1310:2015

test, and a complex fire source of a wooden crib based on the BS 8414-2:2015 test. The properties of

the fire sources are calibrated based on thermocouple temperatures measured near the cladding surface.

In both case studies, the ML-based surrogate model successfully calibrated the fire source properties,

resulting in a high level of agreement between the calibrated model and results for experiments (average

error = 2.8% and 14.3% for case studies 1 and 2). The proposed method can be applied for various

optimisation problems in fire engineering research and design.

Keywords

Surrogate model, Numerical simulations, Model calibration, Artificial neural networks, Façade test,

Fire Dynamics Simulator, Machine learning

2

1 Introduction

Modern façades provide an excellent solution for high-performance building envelopes with superior

structural characteristics, high energy efficiency, and aesthetic appeal. However, because of increased

requirements for thermal and acoustic insulation, modern façade systems often include flammable

components, making them a primary contributor to the rapid spread of building fires [1]. The

performance of façade systems is an important factor that affects the overall fire safety of buildings.

The most reliable method for assessing the fire performance of façade is by conducting experiments

according to established fire testing procedures (e.g. [2, 3]), but these are often costly and time-

consuming. Numerical modelling of façade fire tests has emerged as a cost-efficient alternative that has

allowed researchers to perform sensitivity analysis on a wide range of empirical factors [4]. While there

have been several attempts to develop numerical models for façade fire tests [5-11], the accuracy of

such models is often limited by uncertainty in the fire source properties used in the simulations. Despite

the availability of experimental data, calibrating the fire source in numerical models of façade fire tests

has proved challenging.

Model calibration is an important step in the development of numerical models that involves adjusting

input parameters to improve agreement between the simulation results and the real system. Calibration

is often needed to address uncertainty with input parameters, some of which are empirical properties

that relate to the physical system while others are numerical. However, calibrating input parameters of

fire models in Fire Dynamics Simulator (FDS) software can be very difficult and time-consuming due

to the high computational cost of these models and the large number of input parameters involved.

Optimisation algorithms can help automate the calibration process (e.g. Bayesian inversion framework

[12-15], Bayesian inversion framework combined with polynomial chaos expansions surrogate

modelling technique [16-21], posterior predictive distribution [22], variance decomposition techniques

[23, 24], genetic algorithm [25-27]) but these are impractical to use directly with computationally

expensive models. Optimisation algorithms generally require computations to be performed in series,

where the outcome of the previous iteration is needed to guide computations for the following iteration.

These algorithms can require thousands of iterations before arriving at a solution, and convergence is

not guaranteed in many cases. If optimisation algorithms are coupled directly with computationally

expensive FDS models, then the total computation cost will be excessive since the fire model will need

to be solved repeatedly for each iteration of the optimiser function.

To overcome this issue, this paper proposes a Machine Learning (ML) based surrogate modelling

approach for calibrating computationally-expensive FDS models of façade fire tests. Rather than

coupling the FDS model with the optimisation algorithm directly, the proposed approach involves

substituting the FDS model with a ML-based surrogate that can be efficiently combined with

optimisation algorithms for the purposes of calibration. The surrogate model is developed using

Machine Learning methods and trained using data from an array of randomly generated FDS

simulations. Because there is no dependency between the generated cases, they can be solved in parallel

using a high-performance computing (HPC) facility to reduce simulation time. The results obtained

3

from the surrogate model are verified in FDS, and the process is repeated until the calibration of the

FDS model is deemed suitable for the application of interest.

Two case studies are presented in this paper to demonstrate the practical implementation of the proposed

method for calibrating fire source properties in FDS models of façade fire tests. The first case study

involves a simple model with a single burner surface based on the Japanese façade fire test method JIS

A 1310:2015 [3]. The second case study involves a complex model with a wooden crib as the fire source

placed inside the combustion chamber of the British façade fire test BS 8414-2 [2].

1.1 Surrogate model using machine learning

Machine learning (ML) is a subset of artificial intelligence that allows for constructing statistical models

by providing sufficient sample data and employing a series of algorithms to carry out predictions. A

number of prominent studies have demonstrated the efficacy of utilising ML in the fire safety domain,

such as in making predictions on the evolution of soot mass fraction, temperature, and pressure in case

of high-rise fires [28], identifying the stages of fire growth in residential room fires with an average of

85% accuracy [29], developing early-warning systems for smoke-detection and smoke movement with

the support of support vector machines [30, 31], estimating the temperatures and velocities inside a

compartment fire model using a convolutional neural network given the input of fire location, fire sizes,

ventilation configurations, and compartment geometries [32], providing rapid and accurate predictions

of fires to support hazard or risk assessments utilising data-driven techniques [33, 34], using data mining

techniques such as artificial neural networks and support vector machines to forecast the occurrence of

wildfires [35], developing an ML-based model to predict the probability of flashover in multiple rooms

on a single floor during the compartment fire [36], and proposing a data-driven methodology to estimate

temperature-dependent material properties and determine the fire resistance of timber structures [37].

Numerical models for solving engineering problems have increased in accuracy and complexity over

the past few decades. However, the computing time necessary to run these models has not always been

reduced, making it difficult to finalise engineering tasks dependent on such models [38]. The surrogate

modelling approach has emerged to replace computationally expensive numerical simulations to

address the limitation in terms of the computational time, allowing engineers to complete tasks (e.g.,

design optimisation, sensitivity analysis) more quickly while achieving the required degree of accuracy.

Several surrogate modelling approaches can be found in the literature including Fourier surrogate

modelling [39, 40], random forests [41], support vector machines, space mapping [42], and Bayesian

neural networks [43]. The process of constructing a surrogate model for the design optimisation consists

of the following steps:

• Selecting samples or generating the training dataset based on available experimental data or

simulations cases.

• Constructing the surrogate model to predict the output variable of interest.

• Optimising the model parameters.

• Evaluating the accuracy of the surrogate model with new data.

Surrogate modelling is frequently applied in engineering design to substitute costly experiments and

time-consuming simulations. In this study, surrogate modelling is used to help automate the calibration

4

of fire source properties in FDS models of façade fire tests. The proposed modelling procedure is based

on machine learning algorithms that include the multiple linear regression (MLR) model and artificial

neural networks (ANN). Details of these models are provided in the following subsections.

1.2 Multiple linear regression

The critical steps in building an ML-based model include collecting sufficient data points to form a

valid dataset, selecting relevant input variables, dividing the dataset into training and test sets, choosing

suitable algorithms (e.g. multiple linear regression, artificial neural networks) and controlling the

complexity of the model, training the ML model with the training data, and assessing the predictive

capability of the model with the test data that it has never seen before [44].

One of the most fundamental machine learning techniques is the multiple linear regression (MLR)

model [45]. The mathematical assumption behind the MLR model is a linear relationship between the

independent and dependent variables. The linear regression formula can be expressed as given in

Equation (1):

ŷ=0+�

=1

(1)

where ŷ is the predicted value of the dependent variable, 0 and are the intercepts and regression

coefficients of each independent variable , and is the total number of independent variables.

To assess the accuracy of an ML-based model, it is recommended to use the coefficient of determination

(R2), root mean square error (RMSE) or mean absolute error (MAE) on the test set. These indices are

commonly utilised to measure the difference between the predicted values obtained from the model and

the actual values of output. The way to compute these indexes can be found in Nguyen et al. [44]. It is

strongly suggested that R2 should be used in combination with RMSE or MAE when evaluating the

prediction capacity of an ML-based model. In the case of two ML-based models with the same R2

values, the model with the lower value of RMSE/MAE determined on the test set is regarded as the

better model. An ML model that provides an R2 value approaching 1.0 and an RMSE value close to 0

is considered a well-forecasting model [44].

1.3 Artificial neural networks

Among commonly used ML algorithms, artificial neural networks (ANNs) have received significant

interest and have been widely applied for solving many engineering problems in the area of fire

engineering and sciences [44, 46-50]. ANN is a powerful method for identifying parameters for model

calibration [51, 52] and has been successfully utilised to approximate the inverse output-input

relationship of calibrated models. In fire engineering-related research, great attempts have been made

to calibrate numerical models with the integration of ANN, including the prediction of transient

temperatures at specified locations in a compartment fire [46], determining the fire origin in

compartment fires [48], estimating the smoke temperature in a single-room fire [47], and evaluating the

temperature-deformation history of reinforced-concrete beams exposed to fire [49].

5

The basic structure of a three-layer ANN is shown in Figure 1 consisting of input, hidden, and output

layers. The neurons (or nodes) in the input layer correspond to the input variables in the model, while

the output layer corresponds to the predicted variables. The number of hidden layers and topology of

the ANN affects the performance of the model, and the optimal configuration will vary depending on

the application of interest. Each of the nodes is assigned an initial weight and bias, which are then

modified through iterative computations as the model is being trained until an acceptable level of

predictive performance is achieved.

Figure 1. Structure of artificial neural network with three layers

2 Methodology

This paper proposes a ML-based surrogate modelling approach for calibrating fire source properties in

FDS models of façade fire tests. Rather than coupling optimisation algorithms with FDS models

directly, which would be very time-consuming, the proposed approach involves coupling the optimiser

with a ML-based surrogate model (e.g. MLR, ANN) instead. The optimisation results are verified by

running an FDS simulation using the calibrated values of FDS parameters suggested by the surrogate

model and optimiser. The calibration process is repeated until the performance of the calibrated FDS

model is deemed suitable for the application of interest. A flowchart of the proposed method is presented

in Figure 2, and details of each step in the process are described as follows:

Step 1: An initial FDS model of the problem being solved is developed, where the input and output

parameters of interest are explicitly defined. The FDS model should sufficiently capture the underlying

physics of the system being simulated.

Step 2: A series of preliminary sensitivity studies are performed with the initial FDS model to assess

the influence of each input parameter on the output variables of interest. The sensitivity analyses are

performed using a trial-and-error approach that involves varying the input parameters individually and

comparing FDS results against the calibration target values (e.g. from experiment). The aim is to

identify an appropriate value range for each input parameter that produces FDS results which fully

encompasses calibration target values of the output parameters of interest. This is to avoid having to

extrapolate the surrogate model beyond the range of the training dataset during the calibration process.

Input layer Hidden layer Output layer

i

1

i

2

h

1

h

2

h

3

h

4

o

1

w

1

w

2

w

3

w

4

w

5

w

6

w

8

weights

w

9

w

10

w

11

w

12

w

7

6

Step 3: Once the initial FDS model has been developed and appropriate ranges for input parameters

have been identified, the third step involves generating an array of simulation cases with randomised

values of input parameters within the specified range. In this study, a random case generator is

developed using a Python script to automate this process. For some applications, it may be necessary to

introduce physics-based constraints to define the relationship between input parameters. Each randomly

generated simulation case will provide an additional data point for training and testing the surrogate

model. There are no restrictions on the maximum number of generated cases, but increasing the number

of cases will require additional computing resources. As a rule of thumb, at least 10 simulation cases

are needed for each input parameter being considered for calibration. This number can be increased if

the size of the dataset is deemed inadequate for training and testing the surrogate model.

Step 4: The randomly generated cases in step 3 are solved in parallel using multiple compute cores.

This is ideally performed with the aid of a high-performance computing (HPC) facility, although a

standalone workstation with a large number of computing cores can be used depending on the size of

the FDS model. Solving the cases in parallel is possible because there is no dependency between the

generated cases. This allows for a considerable reduction in computational time that can be scaled

directly with available cores/nodes in the HPC facility. For each simulation case, the output of the FDS

model is saved along with the corresponding values of the input parameters to create a valid dataset for

training the surrogated model.

Figure 2. Flowchart of proposed model calibration procedure using ML-based surrogate model

Step 5: The machine learning-based surrogate model is constructed and trained using the dataset

generated in step 4. The dataset is split into a training set and a test set. The training set (~65-80% of

total data points) is used to train the model, while the test set is used to assess the predictive accuracy

of the model. Here, the aim is to identify an ML model that can adequately replicate the results from

Step 1

Build initial

FDS model

Step 2

Perform

sensitivity

analyses

Step 3

Generate

simulation

cases

Step 4

Solve cases

in parallel

Step 5

Construct and

train ML-based

surrogate

model

Step 6

Calibrate input

parameters

Step 7

Validate input

parameter

calibration

Yes

No

Start

End

Acceptable

accuracy?

(iv)

Address

limitations

of initial

model

(i)

Improve

accuracy of

surrogate

model

(iii)

Identify

additional

parameters

affecting

results

(ii)

Increase

number of

cases

7

the FDS models. For ANN models, the complexity of the model can be controlled by changing the

number of hidden layers (depth of the network) as well as the number of neurons within each layer

(width of the network). Other ML-based models can also be assessed until an adequate model is

identified.

Step 6: Once an adequate ML-based surrogate model is constructed and trained, the next step involves

combining the surrogate model with an optimisation algorithm to calibrate the input parameters

identified in step 1. The basic approach is to define an optimisation function as some measure of

deviation between the output of the surrogate model and the target values of interest (e.g. experimental

data). The most used metrics are the Mean Absolute Error (MAE) and the Root Mean Square Error

(RMSE). An optimisation algorithm is then used to identify optimal values of input parameters for

which the optimisation function is minimised. Physics-based constraints that define the relationship

between values of input parameters may be introduced in the optimiser algorithm to avoid non-realistic

results.

Step 7: Once the calibration has been completed, the next step involves validating the outcomes of the

calibration effort by running an additional FDS case with input values obtained from step 6. The results

from the calibrated FDS model are compared with the target values (e.g. experimental data) to assess

whether the calibrated FDS model achieves an acceptable level of accuracy. If the accuracy of the FDS

model is deemed adequate, then the entire procedure is completed. Otherwise, the calibration can be

further improved in the following ways:

(i) Improve the accuracy of the ML-based surrogate model by changing the algorithms or

managing its complexity (e.g. adjust the width/depth of ANN)

(ii) Increase the number of randomly generated simulation cases to create a larger dataset for

training and testing the ML-based surrogate model.

(iii) Perform additional sensitivity studies to identify additional input parameters that influence

the output of the FDS model, or modify the range selected for existing input parameters.

(iv) Modify the initial FDS model to address underlying limitations.

It should be noted that the acceptable level of accuracy from the FDS model will vary depending on

numerous parameters, some of which are project-specific, such as the consequence of failure or

underperforming systems and the risk tolerance of decision-makers involved. These are best left up to

the modeller to decide on a case-by-case basis.

It should also be stressed that the intended deliverable of the proposed framework is not the ML-based

surrogate itself, but rather the calibrated FDS model. The ML surrogate has a very limited scope of

application because, unlike the FDS model which is physics-driven, the surrogate model is data-driven

and does not account for the underlying physical constraints of the problem being addressed. The

surrogate model is merely used to assist with the calibration of the FDS model and can be discarded

once the calibration process is complete.

8

3 Results and discussion

Having provided a general description of the proposed method, results from two case studies are

presented in this section to assess the feasibility of the proposed method for calibrating the fire source

properties in FDS simulations of façade fire tests. Case study 1 involves calibrating a simple model

with a single burner surface for the Japanese façade fire test JIS A 1310:2015 [3], while case study 2

involves calibrating a more complex fire source of a wooden crib for the BS 8414-2:2015 [2] test. The

models are constructed using FDS v6.7.5 with the aid of the graphical user interface pre-processing tool

PyroSim 2020.5. For both case studies, experimental results are available for a case of non-combustible

cladding that will be utilised for calibrating the fire source properties in the simulation. A detailed

description of each case study is provided in the following subsections.

3.1 Case study 1: Calibrating FDS model with simple heat source

Case study 1 demonstrates the use of the proposed ML-based surrogate modelling approach for

calibrating the heat release rate per unit area (HRRPUA) of a simple fire source inside a combustion

chamber of a façade fire test. Experimental results from a façade fire test by Zhou et al. [10] with a non-

combustible cladding material were adopted for the purpose of calibrating the fire source properties.

The fire test was performed according to the Japanese façade fire testing method JIS A 1310:2015,

which determines the fire propagation over building facades exposed to flames ejected from an opening.

The JIS A 1310 testing layout (Figure 3a) involves a single façade surface with a 910x910 mm opening

that is attached to a combustion chamber. At the floor of the combustion chamber, a single gas burner

with a stable heat release rate is provided as the fire source. The surface area of the burner is 0.36 m2

with dimensions L × W = 600 mm × 600 mm. An FDS model was constructed with an equivalent test

layout as shown in Figure 3c. Temperatures from 5 thermocouples (M1-M5) above the combustion

chamber were used to calibrate the burner surface in the FDS model.

Results from a preliminary sensitivity analysis revealed that the heat release rate per unit area

(HRRPUA) of the burner surface has a direct influence on the temperature profiles of the

thermocouples. Hence, a single input parameter was considered in the calibration of the FDS model,

which is the HRRPUA of the burner surface. A total of 5 output parameters were defined which

correspond to the temperatures of the 5 thermocouples in the fire test (M1-M5).

9

(a)

(b)

(c)

Figure 3. Façade fire test based on JIS A 1310. (a) Test layout. (b) Experiment setup [10].

(c)FDS model

3.1.1 Generating random cases

Based on preliminary sensitivity analysis (step 2 in Figure 2), an appropriate range of HRRPUA was

defined to be between 2500 – 7500 kW/m2. Using the rule of 10 simulation cases per parameter, and

given that the model has a single input parameter, a total of 10 simulation cases were generated with

randomised values of HRRPUA within the specified range of 2500 – 7500 kW/m2. To automate this

process, a random case generator script was developed using Python 3.8 (step 3 in Figure 2). The

resulting range of the input variable is shown in Table 1. The 10 simulation cases were solved in parallel

to reduce the total simulation time (step 4 in Figure 2), and results of the simulation cases are plotted in

Figure 4 against the experimental measurement from Zhou et al. [10].

Table 1. List of input and output values for 10 randomly generated cases

Case No.

Input

HRRPUA

(kW/m2)

Output

Temperature (oC)

M1

M2

M3

M4

M5

1

2740

445.7

272.4

196.3

166.0

143.8

2

3025

491.7

314.2

223.6

187.6

158.6

3

4861

786.5

573.9

424.3

345.3

282.2

4

4944

708.5

492.7

381.9

314.8

256.0

5

5324

811.0

592.0

443.4

368.9

314.7

6

5990

862.1

676.1

511.0

422.3

362.0

7

6056

885.3

702.6

538.9

458.4

391.6

8

6740

902.3

724.8

559.2

474.9

399.4

9

7393

954.6

834.0

675.8

569.1

473.3

10

7399

943.6

797.4

666.4

584.8

495.5

Range (2740 – 7399) (446 – 955) (272 – 834) (197 – 676) (166 – 585) (144 – 496)

Non-combustible

10

Figure 4. Results of 10 randomly generated cases in case study 1

Figure 4 shows that the input parameter values of the randomly generated cases produce results (grey

plots) that fully encompass the target values from the experiment of Zhou et al. [10] (yellow dots). This

confirms that the target value of HRRPUA will fall within the previously defined range of 2500 – 7500

kW/m2. The calibration process can therefore be performed without the need to extrapolate the

performance of the ML surrogate model beyond the range of the training dataset.

3.1.2 Constructing and training the surrogate model

Due to the simplicity of this case study, a Multiple Linear Regression (MLR) algorithm was used for

constructing the surrogate model (step 5 in Figure 2). The surrogate model was composed of an array

of 5 separate models, each corresponding to an output variable of thermocouple temperature (M1-M5).

The HRRPUA was specified as the input variable, and the model was trained using results from the 10

randomly generated cases, which were split into a training set (7 cases) and a test set (3 cases). The

trained MLR model achieved an R2 value of 0.93 indicating a high level of predictive accuracy.

3.1.3 Calibrating fire source parameter

Having trained the surrogate model, the next step involved calibrating the input parameter of HRRPUA

to achieve agreement with experimental results (step 6 in Figure 2). An optimisation function based on

the RMSE was selected to quantify the level of divergence between the surrogate model predictions

and the experimental results. The optimisation function is minimised using the “minimize” function

from the SciPy v1.7.1 library in Python 3.8. The Sequential Least Squared Programming (SLSQP)

algorithm is selected with bounds from 2500 - 7500 kW/m2 and an initial value of 4000 kW/m2. The

optimisation algorithm successfully converged to a solution with HRRPUA = 4917 kW/m2 with a

corresponding local minimum value of RSME = 35.4oC. The surrogate model predictions corresponding

to the optimised value of HRRPUA shows a high level of agreement with experimental results (Figure

5a) with an average error of 7.1%.

11

(a)

(b)

Figure 5. Prediction of ML-based surrogate model (a) and FDS model (b) for the calibrated value

of input parameter HRRPUA = 4917kW/m2 showing good agreement with experiment.

Having calibrated the value of the HRRPUA in the surrogate model, the final step is to verify the results

of the FDS model using this calibrated value (step 7 in Figure 2). Hence, an additional FDS case is

solved with HRRPUA = 4917 kW/m2. The results shown in Figure 5b demonstrate a high level of

agreement with the target values from the experiment where the average error was 2.8%. This indicates

the validity of the proposed approach of using a surrogate model to calibrate input parameters of the

FDS model.

It should be noted that the calibration of the single input parameter in case study 1 could have been

performed manually due to the simplicity of the problem. A more complex case with multiple

interrelated input parameters is presented in case study 2.

3.2 Case study 2: Calibrating FDS model with complex heat source

Having demonstrated a simplified problem in case study 1, the second case study presents the

application of the ML-based surrogate modelling approach for calibrating the properties of a complex

fire source in the BS 8414-2:2015 façade fire test. The façade system consisted of a vertical main wall

that includes the combustion chamber at the bottom, and a perpendicular wing wall located on one side

of the main wall. A total of 16 thermocouples were installed at two levels on both the main and wing

walls in accordance with the BS 8414 testing standard.

Unlike case study 1 using a gas burner surface, the heat source for the façade test in case study 2 is a

wooden crib with multiple surfaces, making this case more complicated due to ambiguity in the

HRRPUA of each surface and size of the wooden crib. The schematic of thermocouple locations, the

system installation before the fire test, and the FDS model are shown in Figure 6. The calibration of the

heat source was carried out based on experimental data from Dréan et al. [7] performed for a non-

combustible cladding material (plasterboard). The temperature of 16 thermocouples at the top and

bottom levels marked in Figure 6a were considered in the calibration of the heat source.

12

(a)

(b)

(c)

Figure 6. Façade fire test based on BS8414-2. (a) Test layout showing the location of

thermocouples. (b) Experiment setup. (c). FDS model.

Based on preliminary sensitivity analyses (step 2 in Figure 2), two fire source properties in the numerical

model were identified to affect the temperatures at the thermocouples. These include the HRRPUA and

the location of the fire source surface. To provide control over the temperatures achieved at different

locations on the façade, four surfaces of the wooden crib were assigned individual parameters of

HRRPUA and location within the combustion chamber, resulting in a total of 8 parameters to be

calibrated. These parameters are listed in Table 2 along with appropriate ranges identified from

sensitivity analyses. The locations of the crib surfaces are defined in terms of Cartesian coordinates

with the origin shown in Figure 6c. The selected ranges in Table 2 fully encompassed the results of

thermocouple temperatures from the experiment.

Table 2. Calibration range of heat source parameters in case study 2

Parameter

Range

HRRPUA (kW/m2)

Top surface

0 - 1500

Front surface

0 - 1500

Left surface

0 - 1500

Right surface

0 - 1500

Location of surface (m)

Top surface (z max)

1.30 – 1.50

Front surface (y max)

0.0 – 0.30

Left surface (x min)

0.35 – 0.55

Right surface (x max)

1.90 – 2.10

13

3.2.1 Generating random cases

Given a total of 8 input parameters to be calibrated, and using the rule of 10 simulation cases per

parameter, a total of 80 simulation cases were generated using the random case generator script in

Python 3.8 (step 3 in Figure 2). The cases were generated based on random combinations of input

variables with the ranges shown in Table 2. Because there was no dependency between the cases, the

cases were solved in parallel using HPC to reduce the total simulation time (step 4 in Figure 2). The

results of the 80 randomly generated cases are presented in terms of temperature-time histories (Figure

7) and peak temperatures at t = 1000 s (Figure 8) for the 16 thermocouples in the façade test. The plots

in Figure 7 and Figure 8 show a wide range of variability in output temperature. Critically, the range of

output profiles fully encompasses the experimental profiles plotted in yellow. Results of peak

temperatures at t = 1000s were extracted from the randomly generated cases to create a valid dataset for

training the surrogate.

Figure 7. Results of 80 randomly generated cases showing temperature-time histories

for the 16 thermocouples in case study 2

14

Figure 8. Results of 80 randomly generated cases showing peak temperature at t= 1000s for the 16

thermocouples in case study 2

3.2.2 Constructing and training the surrogate model

An ANN-based surrogate model was selected in this case study due to the complexity of the calibration

problem and the large number of interrelated parameters involved (step 5 in Figure 2). For constructing

the surrogate model, a densely connected ANN was created using Python 3.8 using TensorFlow v2.3.0

library with Keras API. This network included 8 nodes in the input layer, which corresponds to the

number of FDS model parameters to be calibrated, and 16 nodes in the output layers corresponding to

the temperatures of the 16 thermocouples. Based on preliminary trials, 4 hidden layers were provided

with 2048 neurons in each layer. The rectified linear ('relu') activation function was used for the hidden

layer, and the ‘linear’ function was applied for the output layer.

To train the ANN model, the dataset generated from the 80 randomised cases was partitioned into a

training set of 56 cases (70%) and a test set of 24 cases (30%). The values of input parameters were

normalised using a scaling function to fall with a uniform range of 0 to 1. This was done to address the

variability of units and scales between the input parameters. The training set was used to solve the ANN

and the performance was monitored based on the mean absolute error (MAE) of the predicted values.

A maximum of 2000 training epochs was specified with a batch size of 80, and early stopping was

implemented to avoid overfitting the model. Overfitting implies that the ANN might produce a good

prediction on the training set but a bad predicted result given the test data. Hence, the ANN model was

trained with an increase in the number of epochs and the training was terminated once the MAE of the

test set no longer improved. A delay of 100 epochs was used before triggering the early stopping

15

function. This was done to avoid the early stopping function being triggered by spikes in the MAE of

the test set. Results of the MAE during the training process are shown in Figure 9. The training process

aborted after 374 training epochs beyond which the MAE of the test set showed no further improvement.

Figure 9. Mean absolute error per epoch on the training set and the test set

The predictive power of the ANN-based surrogate model is evaluated based on predictions of data

points not seen during the training process. The test set is used for this purpose. Figure 10 compares the

surrogate model predictions of peak temperatures at the 16 thermocouples with results from the FDS

model. The ANN model achieved R2 values of 0.976 and 0.819 for the training and test sets,

respectively. This indicates that the ANN-based surrogated model is capable of predicting results from

the FDS model with a reasonable degree of accuracy.

(a)

(b)

Figure 10. Comparison of FDS results and ANN model predictions of temperature at 16

thermocouples in case study 2. (a) Training set. (b) Test set.

3.2.3 Calibrating fire source parameters

Having developed a surrogate model that adequately predicts results from the FDS simulation, the next

step (step 6 in Figure 2) involves combining the surrogate model with an optimisation algorithm to

calibrate input values of heat source parameters. An optimisation function is selected based on the

16

RMSE between ANN predictions and experimental measurements. The optimisation function is then

minimised using an optimisation algorithm to determine the optimal values for the 8 input parameters.

The SLSQP algorithm used in case study 1, which is based on local minima, did not converge to an

adequate solution when calibrating the ANN model in this case study. Hence a more computationally

demanding global optimisation algorithm was adopted that is based on the differential evolution (DE)

method. The DE algorithm successfully minimised the optimisation function, achieving an RMSE value

of 55.6 oC. The resulting optimal values for the 8 input parameters are presented in Table 3, and the

corresponding predictions of the ANN-based surrogate model are compared against the target

experimental values in Figure 11.

It is noted that the HRRPUA of the wood crib is a dynamic parameter to be changed over time and is

difficult to measure in reality. Therefore, the HRRPUA values provided in Table 3 are not the actual

properties of the fire source but rather the FDS input parameters found by the DE algorithm to offer a

near-optimal agreement between FDS model predictions and experimental measurements. The

interaction of the surrounding air with the burning chamber may explain the significant difference in

HRRPUA values on the left and right surfaces of the wood crib. The left surface of the wood crib, closer

to the corner of the facade, is likely to get less air than the right side. However, more research is needed

to acquire a deeper understanding of the problem. In reality, the experimental results may differ even

when tested under repeatable testing conditions. As a result, the HRRPUA suggested by the ML-based

surrogate model should be interpreted as average values over the range of test conditions considered.

Table 3. Initial and calibrated values of heat source parameters in case study 2

Parameter

Initial values

Calibrated values

HRRPUA (kW/m2)

Top surface

513

745.8

Front surface

513

1437.2

Left surface

513

2.7

Right surface

513

1481.9

Location of surface (m)

Top surface (z max)

1.40

1.30

Front surface (y max)

0.00

0.00

Left surface (x min)

0.46

0.37

Right surface (x max)

2.00

2.10

17

Figure 11. Comparison of ANN predictions and experiment results of thermocouple temperatures

at t=1000s

3.2.4 Validation of the calibrated model

Once the optimal parameter values have been obtained from the surrogate model, the model is validated

by running additional cases in FDS using the initial and calibrated parameter values as listed in Table

3 (step 7 in Figure 2). The initial model is based on nominal values specified in the BS 8414 standard

for the maximum HRR and the size of the wooden crib.

Results of peak temperature at t = 1000s and temperature-time histories for the 16 thermocouples are

presented in Figure 12 and Figure 13, respectively, for both the initial and calibrated FDS model. The

results show a considerable improvement in the predictive performance of the calibrated model (average

error = 14.3%) compared to the initial model (average error = 40.7%). Before calibration (Figure 12a),

the temperature curves are seen to follow the overall trend of the experimental results, but there is a

considerable discrepancy in thermocouple readings at the centre of the main wall (M2-M4, M7-M9).

After calibration (Figure 12b), the FDS model is able to closely replicate experimental results for all 16

thermocouples, where the RMSE value fell from 240.7 oC to 59.8 oC. Similar conclusions can be drawn

from the results of temperature-time histories shown in Figure 13.

18

(a) before calibration

(b) after calibration

Figure 12. Thermocouple temperatures at t = 1000s in case study 2 before and after calibration

19

(a) before calibration

(b) after calibration

Figure 13. Time histories of thermocouple temperatures in case study 2 before and after calibration

20

4 Conclusions

This work describes the development of a surrogate modelling approach integrated with machine

learning techniques for calibrating the fire source in FDS simulations of façade fire tests. Two case

studies are presented to assess the applicability of the proposed approach; the first is an FDS model

with a single burner surface that involved the calibration of a single parameter (HRRPUA of a single

fire source), and the second is a complex FDS model with a wood crib as the fire source with 8

parameters to be calibrated (HRRPUA and location of 4 fire source surfaces). In both case studies,

experimental measurements of thermocouples temperatures from a case of non-combustible cladding

were used to calibrate the properties of the fire source.

The proposed approach involves generating and solving an array of FDS cases with randomised values

of input parameters within a specified range obtained from preliminary sensitivity analyses. Because

there is no dependency between the generated cases, the FDS cases can be solved in parallel using a

high-performance computing (HPC) facility to reduce simulation time. Results from the FDS cases are

used to train a machine learning-based surrogate model, which is then combined with an optimisation

algorithm in order to calibrate input parameters to achieve a high level of agreement with experimental

results.

The study assessed two different ML algorithms for constructing the surrogate model: the multiple

linear regression (MLR) model and artificial neural networks (ANN). The MLR model was used in case

study 1 to predict 5 output parameters from a single input parameter, while ANN was used in case study

2 to predict 16 output parameters for 8 input parameters. Both models achieved acceptable predictive

accuracy with R2 = 0.930 and 0.819 for MLR and ANN, respectively.

For calibrating the input parameters, the sequential least squared programming (SLSQP) optimisation

algorithm was used in case study 1, while the different evolution (DE) algorithm was used in case study

2. Being a global optimisation function, the DE algorithm was more suitable for optimising the large

number of interrelated input parameters in case study 2 despite requiring larger computational

resources. The optimisation algorithms in both case studies converged successfully, and the resulting

ML predictions demonstrated reasonable agreement with the target values from the experiment. Results

from the FDS model based on the calibrated values of input parameters also achieved reasonable

agreement with the experiments, with an average error of 2.8% and 14.3% for case studies 1 and 2,

respectively.

Overall, the study demonstrates the validity of the proposed surrogate modelling approach for

calibrating fire source properties in FDS models of façade fire tests. The calibrated fire source properties

can be adopted by fire safety modellers to improve confidence in the simulation of combustible

cladding, which has practical value in terms of assessing fire safety risk for various cladding materials

used in real-life buildings.

The proposed approach may be used to calibrate other model parameters in FDS models of façade fire

tests, and more generally, for solving optimisation problems in the field of fire engineering where

computationally expensive models cannot be feasibly coupled with optimisation algorithms. It should

be stressed however that the success of the framework will ultimately depend on whether an appropriate

21

ML surrogate can be found that can adequately replicate the FDS model for the specific application of

interest, and further research is needed to investigate the limitations of extending the proposed approach

to different features to those presented in this study.

Acknowledgments

This research was undertaken as part of a PhD program in Civil Engineering at RMIT University,

Melbourne, Australia. We gratefully acknowledge the support of the Australian Research Council, grant

number DE190100217. There is no conflict of interest declared by the authors.

Author Contributions

Hoang Nguyen: Conceptualisation; Methodology; Formal analysis; Investigation; Data curation;

Writing - original draft.

Yousef Abu-Zidan: Conceptualisation; Methodology; Software; Investigation; Data curation;

Writing - review & editing.

Guomin Zhang: Writing - review & editing; Supervision.

Kate Nguyen: Conceptualisation; Resources; Investigation; Writing - review & editing; Supervision;

Project administration; Funding acquisition.

References

1. Nguyen, K.T., et al., Performance of modern building façades in fire: a comprehensive

review. Electronic Journal of Structural Engineering, 2016. 16(1): p. 69-86.

2. British Standard, BS 8414-2: 2015 Fire performance of external cladding systems. Test

method for non-loadbearing external cladding systems fixed to and supported by a structural

steel frame. 2015.

3. Japanese Standards Association, JIS A 1310:2015 Test method for fire propagation over

building facades. 2015.

4. Nguyen, Q.T., et al., Experimental and numerical investigations on the thermal response of

multilayer glass fibre/unsaturated polyester/organoclay composite. Fire and Materials, 2016.

40(8): p. 1047-1069. https://doi.org/10.1002/fam.2364.

5. Anderson, J., et al., Modeling of fire exposure in facade fire testing. Fire and Materials, 2018.

42(5): p. 475-483. https://doi.org/10.1002/fam.2485.

6. Anderson, J. and R. Jansson. Façade fire tests–measurements and modeling. in MATEC Web

of Conferences. 2013. EDP Sciences. https://doi.org/10.1051/matecconf/20130902003.

7. Dréan, V., et al., Numerical simulation of the fire behaviour of facade equipped with

aluminium composite material‐based claddings‐Model validation at large scale. Fire and

Materials, 2019. 43(8): p. 981-1002. https://doi.org/10.1002/fam.2745.

8. Guillaume, E., et al., Reconstruction of Grenfell Tower fire. Part 3—Numerical simulation of

the Grenfell Tower disaster: Contribution to the understanding of the fire propagation and

22

behaviour during the vertical fire spread. Fire and Materials, 2020. 44(1): p. 35-57.

https://doi.org/10.1002/fam.2763.

9. Guillaume, E., et al., Reconstruction of Grenfell Tower fire. Part 2: A numerical investigation

of the fire propagation and behaviour from the initial apartment to the façade. Fire and

Materials, 2020. 44(1): p. 15-34. https://doi.org/10.1002/fam.2765.

10. Zhou, B., et al., Numerical and experimental study of cedar façade fire. Fire and Materials,

2021. https://doi.org/10.1002/fam.2983.

11. Abu-Zidan, Y., et al., Effect of wind speed and direction on facade fire spread in an isolated

rectangular building. Fire Safety Journal, 2022. 129: p. 103570.

https://doi.org/10.1016/j.firesaf.2022.103570.

12. Beck, J.L. and L.S. Katafygiotis, Updating models and their uncertainties. I: Bayesian

statistical framework. Journal of Engineering Mechanics, 1998. 124(4): p. 455-461.

https://doi.org/10.1061/(ASCE)0733-9399(1998)124:4(455).

13. Hewson, P., Bayesian Data Analysis 3rd edn A. Gelman, JB Carlin, HS Stern, DB Dunson, A.

Vehtari and DB Rubin, 2013 Boca Raton, Chapman and Hall–CRC 676 pp.,£ 44.99 ISBN 1-

439-84095-4. Journal of the Royal Statistical Society Series A, 2015. 178(1): p. 301-301.

https://doi.org/10.1111/j.1467-985X.2014.12096_1.x.

14. Yu, B., R. Tang, and B. Li, Probabilistic calibration for development length models of

deformed reinforcing bar. Engineering Structures, 2019. 182: p. 279-289.

https://doi.org/10.1016/j.engstruct.2018.12.047.

15. Wagner, P.-R., et al. Surrogate-based bayesian inversion for the model calibration of fire

insulation panels. in SIAM Conference on Uncertainty Quantification (UQ18). 2018. ETH

Zurich. https://doi.org/10.3929/ethz-b-000279670.

16. Blatman, G., Adaptive sparse polynomial chaos expansions for uncertainty propagation and

sensitivity analysis. 2009, Clermont-Ferrand 2.

17. Blatman, G. and B. Sudret, An adaptive algorithm to build up sparse polynomial chaos

expansions for stochastic finite element analysis. Probabilistic Engineering Mechanics, 2010.

25(2): p. 183-197. https://doi.org/10.1016/j.probengmech.2009.10.003.

18. Blatman, G. and B. Sudret, Adaptive sparse polynomial chaos expansion based on least angle

regression. Journal of Computational Physics, 2011. 230(6): p. 2345-2367.

https://doi.org/10.1016/j.jcp.2010.12.021.

19. Guo, X., et al., Reliability analysis of embankment dam sliding stability using the sparse

polynomial chaos expansion. Engineering Structures, 2018. 174: p. 295-307.

https://doi.org/10.1016/j.engstruct.2018.07.053.

20. Sudret, B., Global sensitivity analysis using polynomial chaos expansions. Reliability

Engineering & System Safety, 2008. 93(7): p. 964-979.

https://doi.org/10.1016/j.ress.2007.04.002.

21. Wagner, P., et al., Bayesian calibration and sensitivity analysis of heat transfer models for

fire insulation panels. Engineering Structures, 2020. 205.

https://doi.org/10.1016/j.engstruct.2019.110063.

22. Gelman, A., X.-L. Meng, and H. Stern, Posterior predictive assessment of model fitness via

realized discrepancies. Statistica Sinica, 1996: p. 733-760.

23. Arwade, S.R., M. Moradi, and A. Louhghalam, Variance decomposition and global sensitivity

for structural systems. Engineering Structures, 2010. 32(1): p. 1-10.

https://doi.org/10.1016/j.engstruct.2009.08.011.

23

24. Chan, K., A. Saltelli, and S. Tarantola. Sensitivity analysis of model output: variance-based

methods make the difference. in Proceedings of the 29th conference on Winter simulation.

1997. http://doi.org/10.1109/WSC.1997.640407.

25. Lautenberger, C., G. Rein, and C. Fernandez-Pello, The application of a genetic algorithm to

estimate material properties for fire modeling from bench-scale fire test data. Fire Safety

Journal, 2005. https://doi.org/10.1016/j.firesaf.2005.12.004.

26. Rein, G., et al., Application of genetic algorithms and thermogravimetry to determine the

kinetics of polyurethane foam in smoldering combustion. Combustion and Flame, 2006.

146(1-2): p. 95-108. https://doi.org/10.1016/j.combustflame.2006.04.013.

27. Yuen, A.C.Y., et al., Utilising genetic algorithm to optimise pyrolysis kinetics for fire

modelling and characterisation of chitosan/graphene oxide polyurethane composites.

Composites. Part B, Engineering, 2020. 182: p. 107619.

https://doi.org/10.1016/j.compositesb.2019.107619.

28. Buffington, T., S. Bilyaz, and O. Ezekoye, Brain-STORM: A deep learning model for

computationally fast transient high-rise fire simulations. Fire Safety Journal, 2021. 125: p.

103443. https://doi.org/10.1016/j.firesaf.2021.103443.

29. Fang, H., et al., Development of a machine-learning approach for identifying the stages of fire

development in residential room fires. Fire Safety Journal, 2021. 126: p. 103469.

https://doi.org/10.1016/j.firesaf.2021.103469.

30. Gubbi, J., S. Marusic, and M. Palaniswami, Smoke detection in video using wavelets and

support vector machines. Fire Safety Journal, 2009. 44(8): p. 1110-1115.

https://doi.org/10.1016/j.firesaf.2009.08.003.

31. Hong, Y., J. Kang, and C. Fu, Rapid prediction of mine tunnel fire smoke movement with

machine learning and supercomputing techniques. Fire Safety Journal, 2021: p. 103492.

https://doi.org/10.1016/j.firesaf.2021.103492.

32. Hodges, J.L., B.Y. Lattimer, and K.D. Luxbacher, Compartment fire predictions using

transpose convolutional neural networks. Fire Safety Journal, 2019. 108: p. 102854.

https://doi.org/10.1016/j.firesaf.2019.102854.

33. Lattimer, B., J. Hodges, and A. Lattimer, Using machine learning in physics-based simulation

of fire. Fire Safety Journal, 2020. 114: p. 102991.

https://doi.org/10.1016/j.firesaf.2020.102991.

34. Lau, C.K., et al., Fire risk assessment with scoring system, using the support vector machine

approach. Fire Safety Journal, 2015. 78: p. 188-195.

https://doi.org/10.1016/j.firesaf.2015.10.003.

35. Sayad, Y.O., H. Mousannif, and H. Al Moatassime, Predictive modeling of wildfires: A new

dataset and machine learning approach. Fire Safety Journal, 2019. 104: p. 130-146.

https://doi.org/10.1016/j.firesaf.2019.01.006.

36. Wang, J., et al., P-Flash - A machine learning-based model for flashover prediction using

recovered temperature data. Fire Safety Journal, 2021. 122: p. 103341.

https://doi.org/10.1016/j.firesaf.2021.103341.

37. Naser, M., Fire resistance evaluation through artificial intelligence - A case for timber

structures. Fire safety journal, 2019. 105: p. 1-18.

https://doi.org/10.1016/j.firesaf.2019.02.002.

38. Bouhlel, M.A., et al., A Python surrogate modeling framework with derivatives. Advances in

Engineering Software, 2019. 135: p. 102662.

https://doi.org/10.1016/j.advengsoft.2019.03.005.

24

39. Bliek, L., et al., Online optimization with costly and noisy measurements using random

Fourier expansions. IEEE transactions on Neural Networks and Learning Systems, 2016.

29(1): p. 167-182. https://doi.org/10.1109/TNNLS.2016.2615134.

40. Manzoni, L., et al., Surfing on fitness landscapes: A boost on optimization by Fourier

surrogate modeling. Entropy, 2020. 22(3): p. 285. https://doi.org/10.3390/e22030285.

41. Dasari, S.K., A. Cheddad, and P. Andersson. Random forest surrogate models to support

design space exploration in aerospace use-case. in IFIP International Conference on

Artificial Intelligence Applications and Innovations. 2019. Springer.

https://doi.org/10.1007/978-3-030-19823-7_45.

42. Bandler, J.W., et al., Space mapping: the state of the art. IEEE Transactions on Microwave

Theory and Techniques, 2004. 52(1): p. 337-361.

https://doi.org/10.1109/TMTT.2003.820904.

43. Chivatá Cárdenas, I., On the use of Bayesian networks as a meta-modelling approach to

analyse uncertainties in slope stability analysis. Georisk: Assessment and Management of

Risk for Engineered Systems and Geohazards, 2019. 13(1): p. 53-65.

https://doi.org/10.1080/17499518.2018.1498524.

44. Nguyen, H.T., et al., Review on the use of artificial intelligence to predict fire performance of

construction materials and their flame retardancy. Molecules, 2021. 26(4): p. 1022.

https://doi.org/10.3390/molecules26041022.

45. Gareth James, D.W.T.H.R.T., An introduction to statistical learning : with applications in R.

2013: New York : Springer.

46. Buffington, T., et al., Deep-learning emulators of transient compartment fire simulations for

inverse problems and room-scale calorimetry. Fire Technology, 2020: p. 1-27.

http://doi.org/10.1007/s10694-020-01037-2.

47. Di, S., et al., Temperature predictions of a single-room fire based on the CoKriging model.

Journal of University of Science and Technology of China, 2021. 51(1): p. 75.

https://doi.org/10.52396/JUST-2020-1140.

48. Li, N., et al., Multi-fidelity surrogate algorithm for fire origin determination in compartment

fires. Engineering with Computers, 2020. 36(3): p. 897-914. https://doi.org/10.1007/s00366-

019-00738-9.

49. Naser, M., H. Hostetter, and A. Daware. AI modelling & mapping functions: a cognitive,

physics-guided, simulation-free and instantaneous approach to fire evaluation. in The 11th

International Conference on Structures in Fire. 2020. Brisbane, Australia.

https://doi.org/10.14264/a0b3b36.

50. Nguyen, H.T., et al., Predicting heat release properties of flammable fiber-polymer laminates

using Artificial Neural Networks. Composites Science and Technology, 2021: p. 109007.

https://doi.org/10.1016/j.compscitech.2021.109007.

51. Mareš, T., E. Janouchová, and A. Kučerová, Artificial neural networks in the calibration of

nonlinear mechanical models. Advances in Engineering Software, 2016. 95: p. 68-81.

https://doi.org/10.1016/j.advengsoft.2016.01.017.

52. Novák, D. and D. Lehký, ANN inverse analysis based on stochastic small-sample training set

simulation. Engineering Applications of Artificial Intelligence, 2006. 19(7): p. 731-740.

https://doi.org/10.1016/j.engappai.2006.05.003.