Content uploaded by Raju Rimal
Author content
All content in this area was uploaded by Raju Rimal on Apr 30, 2015
Content may be subject to copyright.
Evaluation of Models for predicting the
average monthly Euro versus Norwegian
krone exchange rate from financial and
commodity information
Raju Rimal
ADissertation
Presented to the Faculty
of Norwegian University of Life Sciences
in Candidacy for the Degree
of Masters of Bioinformatics and Applied Statistics
Recommended for Acceptance
by the Department of
IKBM
Supervisor: Ellen Sandberg and Trygve Almøy
Dec 2014
c
Copyright by Raju Rimal, 2014.
All rights reserved.
Abstract
Many multinational companies and policy makers carry out decisions by speculat-
ing exchange rate. Exchange rate is determined by the demand and supply of a
currency. It depends highly on variables like imports, exports, interest rates, oil
prices, inflation and even with its past values. Since these macroeconomic variables
are highly correlated with each other, latent variables or principal components can
solve the problem of multicollinearity. The application of latent variables and prin-
cipal components based methods such as Principal Component Regression (PCR)
and Partial Least Square (PLS) in time series data for prediction is uncommon.
Prediction of exchange rate of Norwegian Krone per Euro using Multiple linear re-
gression, Principal Component Regression (PCR) and Partial Least Square (PLS)
regression is performed in this dissertation.
Linear models and its subsets obtained using criteria such as minimum AIC or
BIC and maximum R2adj are compared on the basis of their goodness of fit. The
selected model is then compared with models from principal component regression
and partial least square regression on the basis of predictability criteria of RMSEP
and R2predicted. The results have suggested the partial least square regression as
the best models among other. The residuals obtained from the models have no au-
tocorrelations so the application of this method has not only reduced the dimension
of data but also resolved the problem of multicollinearity and autocorrelations.
iii
Acknowledgements
IwouldliketoexpressmygratitudetomysupervisorsEllenSandbergandTrygve
Almøy for their guidance and invaluable suggestions. I am grateful to Prof. Solve
Sæbø for his helpful advice and instructions. I want to thank Statistic Norway
and Norges Bank for making valuable data easily available without which it is
impossible to complete my thesis.
Being a student of a developing country, I was unaware of programming, mod-
ern statistical methods and academic writing. With the encouragement of Prof.
Solve Sæbø and my supervisors, I have completed by thesis with extensive use of
R programming and modern statistical tools.
Finally, my special thanks also goes to my families and friends for their con-
tinuous support and encouragement.
iv
To my p arent s.
v
Contents
Abstract.................................... iii
Acknowledgements .............................. iv
ListofTables ................................. vii
ListofFigures.................................viii
1 Introduction 1
1.1 Methods opted for analysis . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Sourcesofdata ............................. 4
1.3 Objectiveofthesis ........................... 4
2 Data and Material 5
2.1 ForExMarket.............................. 5
2.2 The Norwegian krone (NOK) . . . . . . . . . . . . . . . . . . . . . 6
2.3 EURO .................................. 7
2.4 Factors influencing Exchange Rate . . . . . . . . . . . . . . . . . . 7
2.4.1 Inflation ............................. 9
2.4.2 InterestRate .......................... 11
2.4.3 IncomeLevels.......................... 14
2.4.4 Government Control . . . . . . . . . . . . . . . . . . . . . . 15
vi
2.4.5 Expectations .......................... 16
2.5 BalanceofPayment........................... 16
2.5.1 CurrentAccount ........................ 18
2.5.2 Capital and Financial Accounts . . . . . . . . . . . . . . . . 21
2.6 OilSpotPrice.............................. 23
2.7 Lagged response variable as predictor . . . . . . . . . . . . . . . . . 24
2.8 E↵ectofCrisisperiod.......................... 25
3 Models and Methods 26
3.1 AstatisticalModel ........................... 26
3.2 Linear Regression Model . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2.1 Least Square Estimation . . . . . . . . . . . . . . . . . . . . 27
3.2.2 Prediction............................ 29
3.3 Variableselection............................ 29
3.3.1 Criteria for variable selection . . . . . . . . . . . . . . . . . 30
3.3.2 Computational procedure for variable selection . . . . . . . . 31
3.4 Principal Component Analysis . . . . . . . . . . . . . . . . . . . . . 32
3.5 Principal Component Regression . . . . . . . . . . . . . . . . . . . . 35
3.6 Partial Least Square Regression . . . . . . . . . . . . . . . . . . . . 37
3.7 RidgeRegression ............................ 40
3.8 ComparisonCriteria .......................... 41
3.8.1 Goodnessoffit ......................... 41
3.8.2 Predictability .......................... 43
4 Data Analysis 47
4.1 Multiple Linear Regression . . . . . . . . . . . . . . . . . . . . . . . 50
vii
4.2 Variable Selection Procedure . . . . . . . . . . . . . . . . . . . . . . 51
4.2.1 Model selection using Mallows Cpand R2adjusted . . . . . 51
4.2.2 Model selection using AIC and BIC criteria . . . . . . . . . 51
4.2.3 Step wise procedures based on F-value . . . . . . . . . . . . 53
4.3 Principal Component Analysis . . . . . . . . . . . . . . . . . . . . . 55
4.4 Principal Component Regression . . . . . . . . . . . . . . . . . . . . 57
4.5 Partial Least Square Regression . . . . . . . . . . . . . . . . . . . . 58
4.6 RidgeRegression ............................ 59
4.7 CrossValidation............................. 60
4.8 Prediction on test Data . . . . . . . . . . . . . . . . . . . . . . . . . 62
4.9 Comparison of Models . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.9.1 Goodnessoffit ......................... 63
4.9.2 Predictability .......................... 64
4.10 CoefficientsEstimates ......................... 66
4.11 Autocorrelation and its resolution . . . . . . . . . . . . . . . . . . . 67
5 Discussions and Conclusion 68
5.1 Somediscussions ............................ 68
5.2 Conclusions ............................... 70
5.3 FurtherStudy.............................. 71
Bibliography 77
A Data Description 77
B R packages used 79
viii
C Some Relevent Plots 81
D Codes in Use 85
ix
List of Tables
2.2 Two components of Balance of Payments and their subdivision . . . 17
4.1 Summary Report of all the variables used in this report . . . . . . . 47
4.1 Summary Report of all the variables used in this report . . . . . . . 48
4.2 Variables significant at ↵=0.05 while fitting linear model . . . . . 50
4.3 Dispersion of data explained by principal components . . . . . . . . 56
4.3 Dispersion of data explained by principal components . . . . . . . . 57
4.4 Percentage of variation explained by PCR model in response and
predictor................................. 57
4.5 Percentage of variation Explained by PLS model in Response and
Predictor................................. 58
4.6 Summary statistic and information criteria for model comparison . 64
4.8 Validation result containing RMSEP and R2pred for training set,
cross-validation set and test set . . . . . . . . . . . . . . . . . . . . 66
4.9 Coefficient Estimate for PLS and PCR model . . . . . . . . . . . . 67
x
List of Figures
2.1 Exchange rate of Norwegian Krone per Euro . . . . . . . . . . . . . 6
2.2 E↵ect of shifts on demand and supply of currencies on their Ex-
changerates............................... 8
2.3 E↵ect of inflation on Exchange Rate Equilibrium . . . . . . . . . . 10
2.4 Time Series plot of Consumer Price Index (CPI) . . . . . . . . . . . 11
2.5 E↵ect of interest rate change in Exchange Rate . . . . . . . . . . . 12
2.6 Market Rate influence on demand channel, exchange rate channel
and expectation channel . . . . . . . . . . . . . . . . . . . . . . . . 13
2.7 Interest Rates from Norway and Eurozone and their comparision
with Exchange Rate showing a distinct inverse relationship . . . . . 13
2.8 E↵ect of change in relative income levels on exchange rate ceteris
paribus. ................................. 14
2.9 Current Account Balance prepared from quartely data from the year
1981to2014............................... 19
2.10 Time Series plot of major imports of Norway . . . . . . . . . . . . . 20
2.11 Time Series plot of major exports of Norway . . . . . . . . . . . . . 21
2.12 Time Series plot of variables related to capital account . . . . . . . 22
2.13 Time Series plot of oil spot price from Jan 2000 . . . . . . . . . . . 23
xi
2.14 Partial autocorrelation function for Exchange Rate of NOK per Euro 24
3.1 Model Error - Estimation Error and Prediction Error . . . . . . . . 43
3.2 Procedure adopted in the thesis . . . . . . . . . . . . . . . . . . . . 44
4.1 Correlation between response (Exchange Rate) and other predictor
variable ................................. 49
4.2 Number of variable against the criteria where the red dot corre-
sponds the number of variable to acheave the criteria, i . . . . . . . 52
4.3 Modelselectedby ........................... 52
4.4 Number of variable against the AIC vs BIC criteria . . . . . . . . . 53
4.5 Best subset model selected by AIC and BIC criteria . . . . . . . . . 54
4.6 Best subset model selected by F-test based criteria . . . . . . . . . 55
4.7 Variance Inflation Factor (VIF) of di↵erent models . . . . . . . . . 56
4.8 Variation Explained by PLS and PCR . . . . . . . . . . . . . . . . 59
4.9 RMSE and R2pred plots for di↵erent ridge regression paramter . . 60
4.10 RMSEP plot for PCR and PLS . . . . . . . . . . . . . . . . . . . . 61
4.11 Comparision of Model on the ground of calibration model, cross-
validation models and prediction model on the basis of RMSEP and 65
4.12 Cofficients estimates for predictor variables . . . . . . . . . . . . . . 67
C.1 Diagnostic plot for the subset of linear model selected from mini-
mum .................................. 81
C.2 Scatter loading plot of PLS with its first and second components . . 82
C.3 Scoreplot of first three component of PLS regression . . . . . . . . . 82
C.4 Residuals obtained after fitting the model . . . . . . . . . . . . . . 83
xii
C.5 Partial Autocorrelation Function (PACF) of Residuals obtained af-
terfittingthemodel .......................... 83
C.6 Prediction made on trained and test dataset using di↵erent models 84
xiii
Abbreviations and Symbols
Abbreviations and their full forms used in this Thesis
Abbreviation FullForm
PC Principal Components
PCA Principal Component Analysis
PLS Partial Least Square
PCR Princiapal Component Regression
AIC Akaike Information Criterion
BIC Bayesian Information Criterion
Cp Mallows’sCp
VIF Variance Inflation Factor
RMSE Root-Mean-Square Error
RMSEP Root-Mean-Square Errorof Prediction
RMSECV Root-Mean-Square Errorof Cross-validation
R2pred PredictedR-squared
VAR Vector Autoregression
ARIMA Autoregressive Integrated Moving Average
ADL Autoregressive Distributed Lag
NOK Norwegian Krone
USD United State Dollor
Symbols and their meaning used in this Thesis
Symbols Meaning
Bold Symbols like,
X,YMatrices and Vectors
Sigma (S) Popularion (Sample) variance-covariance matrix
R2adj Adjusted coefficient of determination
CVadj RMSECV adjusted for bias
cp.model Subset of linear model selected with minimum Mallow’s Cp
Criteria
r2.model Subset of linear model selected with maximum R2adjusted
Criteria
aic.model Subset of linear model selected with minimum AIC Criteria
bic.model Subset of linear model selected with minimum BIC Criteria
forward.model Subset of linear model selected based on F-test Criteria
using forward selection procedure
backward.model Subset of linear model selected based on F-test Criteria
using backward elimination procedure
train Training Dataset (From Jan 2000 to Dec 2012)
test Test Dataset (From Jan 2013 to Nov 2014)
Ridge Regression Parameter
Q2R2predicted
PerEURO Exchange Rate of Norweian Krone Per Euro (Response
Variable)
Chapter 1
Introduction
Apart from having distinct role in money market, exchange rate has influence
in almost all the sectors of economics and finance. Understanding its dynamics
enables multinational companies to make decision on their investment and assist
bureaucrats to update the monetary and fiscal policies. Di↵erent models are used
to understand the dynamics of exchange rate, however the use of latent variable in
the models is unconventional. Multicollinearity which is also a common problem
in economic researches, models based on principal components (latent variables)
such as Principal Component Regression(PCR) and Partial Least Square(PLS)
regression can resolve the problem. Although autocorrelation is a major problem
in time-series, inclusion of the past values of dependent variable in the model can
solve the problem in many situations. In this dissertation the exchange rate of
Norwegian Krone vs Euro is predicted from the classical linear regression models,
its subsets derived from various criteria, PCR and PLS models. The models are
compared on the basis of their performance. Under proper model specification
1
and wise selection of required components, Principal Component Regression and
Partial Least Square regression can forecast better than the linear models.
Tradi ng has s tar t ed fr om the ver y be ginn ing of hu man ci vili zati on. Peo ple
used to trade with goods at the time but with advancement of development people
started using gold, silver and finally money. The process is not restricted within
a country. Some countries are powerful and some are not so as their currencies.
Currency of another country becomes essential to buy things from that country.
Here comes the role of exchange rate. Buying powerful currencies requires large
sum of weak currencies.
Any international trade is conducted through more than one currencies. Par-
ticipants in the international trade require to exchange their currency which is
performed by foreign exchange market. “The foreign exchange market (ForEx)
is the mechanism that brings together buyers and sellers of di↵erent currencies”
(Appleyard, Field, and Cobb, 2014).
As any other commodity, exchange rate is also determined from its demand and
supply in money market. All those economic activities that exist between countries
create demand and supply of the currencies which consequently determine the
exchange rate. The economic activities between countries are recorded as balance
of payment account. Thus the balance of payment account captures all the demand
and supply of foreign currency (Fang and Kwong, 1991). When the domestic
demand for foreign currency exceeds the foreign demand of domestic currency i.e.
a deficit in the balance of payment, the domestic currency depreciate (Balance of
Payments Deficits and Surpluses).
Foreign currencies are involved in various activities such as, (a) imports and ex-
ports of goods and services, (b) interest and dividends payed to foreign investment
2
in domestic market, (c) interest and dividends earned from investments made on
foreign market, (d) all the currencies that enter into and leave from a country as
income and expenditure.
Three factors a↵ecting exchange rate are considered in this thesis. Primarily,
total monthly imports and exports of goods are considered. Ships, oil platform,
chemicals and food stu↵s are major imports of Norway. Petroleum products,
machinery, equipment, chemicals and fishes are the major exports. Since the
economy of Norway highly depend on petroleum products, apart from imports and
exports, the second component considered is the spot oil price. Third factor is the
financial variables such as interest rate and consumer price index are considered.
In interest rate - (a) key interest rate of Norway, (b) Loan interest rate (c) key
interest rate of euro area are taken into account as factors a↵ecting interest rate.
1.1 Methods opted for analysis
Univariate time series analysis is very common in Econometric where Autoregres-
sive (AR), Moving Average (MA) and Autoregressive integrated Moving average
(ARIMA) are used. However, dealing a time series data with many predictor
variables using latent variables and principal components methods is unconven-
tional. This thesis aims to analysis a time series with financial and commodity
data, as predictor, using statistical regression methods such as - Multiple Linear
Regression, Ridge Regression, Principal Component Regression (PCR) and Partial
Least Square (PLS) Regression. Apart from these, a subset models which selected
from the Multiparty Linear Regression using various criteria are also used. An
application of PCR and PLS on time series data makes this thesis distinct.
3
1.2 Sources of data
Data related to balance of payment such as import, export and trade balance used
here are obtained from Statistics Norway. Consumer price index is also obtained
from the same source. Interest rate variable related to Norway are obtained from
Norges’ Bank and the key interest rate for euro zone is obtained from Euro Bank
while the oil spot price is obtained from US Energy information system. The
average monthly spot price for Brent oil was on Dollar per Barrel unit which was
converted into NOK using NOK per USD exchange rate for that month.
1.3 Objective of thesis
There are three main objective of this thesis-
1. To anal yze th e rel a tio nshi p of for eign e xchan ge rat e with t he fina ncia l (pri ce,
indices and exchange rate) and commodity (imports, exports and trade bal-
ance) information
2. Prediction of out-of-sample observations (Exchange Rate) using various mod-
els
3. Comparison of the Models considered on the basis of goodness of their fit
and their predictive ability
4
Chapter 2
Data and Material
Prediction of dynamics of Exchange Rate through Economic and Financial indi-
cators is the main aim of this thesis. From these two broad categories, only those
factors were considered which are believed to be useful to understand the exchange
rate dynamics.
2.1 ForEx Market
Foreign Exchange(Fx) Market is the most traded and liquid financial market where
individuals, firms and banks buy and sell foreign currencies. Forex market consti-
tute of monetary counters connected electronically which are in constant contact
forming a single international financial market. The market remains open 24 hr a
day for five working days of a week (Introduction to the Forex Market ).
Currencies are exchanged for activities like trade, tourism and investments in
another countries. For instance, a person visiting France needs euro since euro is
accepted in France. On returning back from the visit (s)he might want to exchange
5
7.5
8.0
8.5
9.0
9.5
99Oct
00Apr
00Oct
01Apr
01Oct
02Apr
02Oct
03Apr
03Oct
04Apr
04Oct
05Apr
05Oct
06Apr
06Oct
07Apr
07Oct
08Apr
08Oct
09Apr
09Oct
10Apr
10Oct
11Apr
11Oct
12Apr
12Oct
13Apr
13Oct
14Apr
14Oct
15Apr
Date
PerEURO
Fig 2.1: Exchange rate of Norwegian Krone per Euro
back those Euros to Norwegian Krone. This transaction is a↵ected by the exchange
rate of Norwegian Krone per Euro. The exchange rate of NOK per Euro over time
is plotted in figure-2.1.
Exchange rate can be set according to di↵erent macroeconomic variables, such
as interest rate, price index, balance of payment etc. Such exchange rate deter-
mined by ForEx market transaction is called Floating exchange rate. Some country
fix exchange rate while others pegged with other currency. Norway has a floating
exchange rate.
2.2 The Norwegian krone (NOK)
After introduction of Krone in April 1875 (Brief History Of Norges Bank 2014-
11), Norway was pushed to join the Scandinavian Monetary Union established on
6
1873 (Norwegian Kroner 2014/12). Although the Union was formally abolished
on 1972, Norway decided to keep the names of its currencies. In December 1982,
due to heavy speculation, Norges Bank (Central Bank of Norway) decided to fix
Norwegian Krone which later floated on 1992 (Brief History Of Norges Bank 2014-
11).
2.3 EURO
Euro, the official currency in the Eurozone, was introduced as a virtual currency
in 1999 and later as physical in 2002. It is the single currency shared by 191of the
European Union’s Member States of Euro Area. Although European Central Bank
(ECB) manages Euro, the fiscal policy (public revenue and expenditure) are in the
hands of individual national authorities. The single currency market throughout
the euro zone not only makes traveling across the countries easier but also helps the
member country to keep their economy sound and stable. This situation removes
currency exchange cost, smooth international trade and consequently gives them
more powerful voice in the world. A stable economy and larger area protects
euro zone from external economic fluctuations, instability in currency market and
unpredictable rise in oil prices.(The euro 2015)
2.4 Factors influencing Exchange Rate
The demand of any currency relative to its supply determines its price, just like
any other commodity. For each possible price of a Norwegian Krone, there is
1https://www.ecb.europa.eu/euro/intro/html/index.en.html
7
a corresponding demand and supply to be exchanged with euro in the money
market. When demand of krone equals its supply, the price it exhibit at some
specific time is called its equilibrium exchange rate. Factors like inflation, interest
rates, expectation and government policy a↵ects the demand for any currency. But
the supply is mostly in control of the central bank. In a floating exchange rate
regime, the shift in demand (fig-2.2a) and supply(fig-2.2b) function determines
equilibrium exchange rate of any currency.
Q0
e0
ed0
e1
Q1
Supply Function
Demand Function
Shift in Demand Function
Deadweight
Loss
(a) Demand Shift and Exchange Rate Equilibrium
Q0
e0
e1
Q0
Q1
Supply Function
Demand Function
Shift in Supply Function
Deadweight
Loss
(b) Supply Shift and Exchange Rate Equilibrium
Fig 2.2: E↵ect of shifts on demand and supply of currencies on their Exchange rates
In case of demand shift, with constant currency supply, the exchange rate
will suddenly rise to edcreating dead weight loss (also known as excess burden
or allocative inefficiency2) which consequently pushes the supply from Q0to Q1
creating a new equilibrium exchange rate at e1. In the similar fashion, if the
market is over flooded with currency, shifting the supply function and creating
dead weight loss, the exchange rate is pressed from e0to create a new equilibrium
at e1. In both the situation, the quantity supplied although being increased, the
first one leads to a rise in exchange rate while the other leads to its fall.
2http://www.princeton.edu/~achaney/tmve/wiki100k/docs/Deadweight_loss.html
8
Madura (2012, p. 103) suggested an equation consisting those macroeconomic
factors that can a↵ect the demand and supply of any currency and consequently
the exchange rate as,
e=f(INF,INT,INC,GC,EXP) (2.1)
where,
e: percentage change in spot exchange rate
INF: change in inflation di↵erential between two countries (currencies)
INT: change in interest rate di↵erential between two countries
INC: change in the income level di↵erential between two countries
GC: change in government control
EXP: change in currency value expectations
2.4.1 Inflation
Inflation is the steady rise in overall price level, i.e. a decrease in the value of
currency. In other words, more amount of money is needed to buy same goods than
previous. Relative change in inflation has e↵ect on exchange rate. For instance, an
abrupt rise in the inflation in Norway relative to the Eurozone, Norwegian products
becomes relatively expensive in terms of Norwegian Currency. On one hand, this
would increase the demands for Eurozone goods, and consequently the demand
for euro increases in the short run. On the other hand, expensive Norwegian
goods becomes less attractive in Eurozone and therefore reduce the supply of
euro purchasing Norwegian kroner. In figure -2.3, the demand function of Euro
9
shift upward due to inflation of NOK, i.e. Eurozone goods are more attractive
than Norwegian goods and the downward shift on supply function occurs as the
customers are less interested in Norwegian products. As a result the value of Euro
per NOK increases from 9.10 to 9.97, i.e Norwegian Krone deprecates against the
Euro (Madura, 2012, p. 104).
S0
D0
Value of EURO per NOK
Quantity of EURO
9.10
9.97
S1
D1
QEuro
Upward shift in De-
mand of Euro due to
inflation in Norway
Downward shift in sup-
ply of Euro purchasing
NOK
Source: Madura, 2012
Fig 2.3: E↵ect of inflation on Exchange Rate Equilibrium
Statistics Norway prepares and publishes the official figures for inflation, the
consumer price index (CPI) with base year at 1998. Since the real value of money is
constantly declining, high inflation means that storing money is expensive. while
low and stable inflation contributes to an efficient distribution of resources in
a market economy (FAQ: Monetary Policy, Inflation and Interest Rates 2007).
Since this is an important factor that can influence exchange rate, data for CPI is
10
104
108
112
115
119
123
127
130
134
138
00Feb
00Dec
01Oct
02Aug
03Jun
04Apr
05Feb
05Dec
06Oct
07Aug
08Jun
09Apr
10Feb
10Dec
11Oct
12Aug
13Jun
14Apr
15Feb
Date
CPI
Source: Norges Bank
Fig 2.4: Time Series plot of Consumer Price Index (CPI)
obtained for this thesis from Norges bank. The time-series plot for CPI in figure-??
shows an steady increment over the time.
2.4.2 Interest Rate
Since Interest rate has impact on inflation and currency values, by manipulating it,
central banks exert influence over both inflation and exchange rates. For example,
a sudden increase in interest rate in Norway relative to Eurozone could have in-
crease on investment of Eurozone in Norway with interest-bearing securities. The
Eurozone investors wants to invest more in Norway which increases the demand
for NOK in Eurozone. Due to stronger incentives, Norwegians also increase their
domestic investment, as a result, the supply of NOK in currency market will re-
11
duce. The increase in Demand of NOK and decrease in its supply results a shift
in exchange rate to lower level. The process is illustrated in figure - 2.5.
Quantity of Euro
(purchasing Norwegian Krone)
Price of Euro (EUR/NOK)
S0
S1
D0
D1
QEuro
NOK
8.72
NOK
9.10
Demand Shift
Supply Shift
Source: Madura, 2012
Fig 2.5: E↵ect of Interest Rate change on Exchange Rate includes (a) Demand Shift:
Due to increased interest rate in Norway, demand of Norwegian Krone increases
creating a demand shift in demand function and (b) Supply Shift: The supply
of Krone decrease as Norwegian increase their domestic investment creating a
shortage of NOK in market.
The influence of market interest rate flows through multiple channel such as
demand channel, exchange Rate channel and expectation Channel as shown in
figure-2.6 (E↵ect of Interest Rate Changes 2004).
According to Madura (2012), change in interest rate in third country can also
a↵ect the exchange rates between NOK and Euro. For instance, the sudden in-
crease of interest rate in US would shift the European investment from Norway to
12
Exchange
Rate
Imported
Price Inflation
Norge’s Key
Interest Rate Market Rates Inflation
Expectations Inflation
Consumption
Investment
Labor Market
Wages
Margins
Source: E↵ect of Interest Rate Changes 2004
Fig 2.6: Market Rate influence on demand channel, exchange rate channel and expec-
tation channel
US which consequently reduce the demand of NOK resulting a downward pressure
on its exchange rate with Euro.
Norwegian Key Interest Rate
Norweign Loan Interst Rate
Eurozone Key Interest Rate
0.0
2.5
5.0
7.5
2000
2005
2010
2015
2000
2005
2010
2015
2000
2005
2010
2015
Date
value
Eurozone Key Interest Rate
Exchange Rate of NOK per Euro
Norwegian Key Interest Rate
Norweign Loan Interst Rate
Fig 2.7: Interest Rates from Norway and Eurozone and their comparision with Exchange
Rate showing a distinct inverse relationship
Since the interest rate is a key factor influencing exchange rate, the key interest
rate of Norway and Eurozone along with the loan interest rate of Norway is consid-
13
ered in this thesis. The time series plot of these variables are in figure - 2.7. Due
to simultaneous act of other variables, the plot does not exhibit any discrete rela-
tionship. However, the model fitted by the data collected suggest some in-depth
understanding of this relationship which is analysed and presented in chapter-4.
2.4.3 Income Levels
The rise in real income level increases the consumption level. Relative income
levels of a country is another factor which can a↵ect the demand of imported
goods which consequently a↵ect exchange rate (Madura, 2012). For instance, if
the income levels of people of euro zone rises, other factor being constant, the
demand for foreign goods in euro zone may increase which can shift the demand
function outward and subsequently increase the exchange rate (figure-2.8).
Quantity of Euro
(purchasing Norwegian Krone)
Price of Euro (EUR/NOK)
S0
D0
D1
Q(Euro)
NOK
8.72
NOK
9.10
Increased demand of
foreign goods due to in-
creased income levels
Source: Madura, 2012
Fig 2.8: E↵ect of change in relative income levels on exchange rate ceteris paribus.
14
The example considered above is on the assumption of ceteris paribus, which in
reality is not usual. The change in exchange rate due to income levels is also guided
through the e↵ect of income levels on interest rates and inflation. The increased
income levels increase the consumption cause the economy to overheat. Central
banks could increase interest rates to prevent overheating and increased inflation.
Thus the relative change in income levels can a↵ect exchange rates directly and
indirectly (Madura, 2012, p. 106).
2.4.4 Government Control
Government Control is the fourth factor Madura (2012) has considered that can
influence foreign exchange rate. Government can influence exchange rate in many
ways including, (a) imposing foreign exchange barriers, (b) imposing foreign trade
barriers, (c) intervening (buying and selling currencies) in the foreign exchange
markets, and (d) a↵ecting macro variables such as inflation, interest rates, and
income levels. Norges Bank could force the currency to depreciate by flooding the
market with NOK (i.e increasing supply) if Norway wants to boost its exports.
Similarly, the bank could used their foreign currency reserve to purchase NOK to
rise its value. Such direct interventions make considerable impact on the exchange
rate. As a indirect intervention, the government can influencing the underlying
macroeconomic factors like inflation, interest rate and income level (Madura, 2012,
p. 107).
15
2.4.5 Expectations
Response to new information in foreign exchange market is similar to other fi-
nancial market. The current expectation for the future value is reflected in the
exchange rate changes. Like in stock market, when a company publishes its pros-
perous financial statement, the stock price suddenly rises; the forex market also
exhibit similar performance. For example, a news of increasing inflation in Norway
cause currency traders to sell Norwegian Krone expecting a decrease in its future
value. This expectation is immediately seen as a downward pressure on Norwe-
gian Krone. The similar e↵ect is obtained when speculator expects the currency
to depreciate (Madura, 2012, p. 107).
A person of one country need the currency of another country for various
purposes such as trade of goods and services, foreign investment and travelling.
The actual flow of currency from one country to another is in these forms of
activities. The transaction of trade in terms of goods and services between specific
countries is kept recorded as a form of balance of payment which can even have
signal of possible shifts in exchange rate.
2.5 Balance of Payment
Although international trade is possessed in various forms, the transaction of mul-
tiple currency is common in each of them. A country keeps these transactions
with other countries as a form of Balance of Payments account. A balance of
payment account maintains a systematic records of these transactions conducted
at some specific time period between a home country and others (those countries
with which the transactions are made). A balance of payment account of a country
16
exhibit the size of its economic activities with rest of the world (Appleyard, Field,
and Cobb, 2014, p. 462).
Since Balance of Payment is a bookkeeping system for inter countries economic
activities, the items with payments inward to the home country are credited while
payments outward from the home country are debited. Exports, inflow of foreign
investment, interest and dividends obtained from the investment made on foreign
country by the home country are considered as credited items as they increase
the inward flow of currency. Similarly, Imports, investment made on foreign coun-
tries, interest and dividends paid to foreign countries for their investment in home
country are the items to be debited (Appleyard, Field, and Cobb, 2014, p. 465).
Table 2.2: Two components of Balance of Payments and their subdivision
Balance of Payment
Current Account Capital Account
•Payments for Merchandise and Ser-
vices
•Factor Income Payments
•Transfe r Payment s
•Examples of Payment Entries
•Actual Current Account Balance
•Direct Foreign Investment
•Portfolio Investment
•Other Capital Investment
•Errors and Omissions and Reserves
Source: Madura, 2012
Balance of payment can be classified into two broad categories - (a) Current
Account and (b) Capital Account. The items that lies in these subcategories are
illustrated in table-2.2.
17
2.5.1 Current Account
Current account measures net imports and exports of a country. Imports and
exports are divided into three sub categories - (a) Tra de of go od s, (b) Tr ade of
services and (c) Income which includes the interest and dividend payed to inter-
national firms operating within home country and interest and dividends earned
from domestically owned firms abroad (Krugman and Obstfeld, 2006).
The current account balance is the di↵erence between export and import.
When export of a country exceed its import, there is current account surplus
and when import exceed export there is a current account deficit.
Current Account = Total Exports Tota l Imp ort s (2.2 )
Above equation can also be expressed as a form of income and expenditure like
in equation-2.3 which is the di↵erence between Total National Income and Total
Domestic consumption (Krugman and Obstfeld, 2006).
Current Account Balance = Y
|{z}
GNP
(C + I + G)
| {z }
Total Domestic
Consumption
(2.3)
where,
C=Consumption
I=Investment
G=GovernmentPurchases
Current account incorporates a wide range of international transactions so
there is a vital role of exchange rate in each of those transaction. This thesis has
considered the monthly data for imports and exports of goods which is available
18
0
50
100
1980Q1 1985Q1 1990Q1 1995Q1 2000Q1 2005Q1 2010Q1 2015Q1
Date (in quarters)
Value (in NOK thousand)
BalanceGoods BalanceServices CurrentAccountBalance
Fig 2.9: Current Account Balance prepared from quartely data from the year 1981 to
2014
from Statistics Norway. In Norway, current balance is highly influence by the
balance in goods. Figure-2.9 shows that the balance in services in Norway is
decreasing while the balance in Goods has boost up after around 1998. Further,
the balance in services plotted in the same figure from the quarterly data exhibit
a seasonal trend which is usual in Norway.
Imports
Machinery & equipment, chemicals, metals and food stu↵saremajorimportsof
Norway. Sweden (13.6%), Germany (12.4%), China (9.3%), Denmark (6.3%), UK
(6.1%) and US (5.4%) are major import partners 3. The monthly imports of new
ships (ImpNewShip), oil platform (ImpOilPlat), old ships (ImpOldShip) and all
3https://www.cia.gov/library/publications/the-world-factbook/geos/no.html
19
ImpExShipOilPlat ImpNewShip
ImpOilPlat ImpOldShip
200
300
400
500
0
10
20
30
0
25
50
75
0
20
40
60
80
2000 2005 2010 2015 2000 2005 2010 2015
Date (Monthly)
Value (NOK hundreds)
Fig 2.10: Time Series plot of major imports of Norway
other items excluding ship and oil platform (ImpExShipOilPlat)areconsidered
as predictor variable in data analysis. The time-series plot for these variables are
presented in figure-2.10
Exports
Norway is richly endowed with natural resources - petroleum, hydro-power, fish,
forests, and minerals but the economy is highly dependent on the petroleum sec-
tor 3. Petroleum products, machinery and equipment, metals, chemicals, ships and
fishes are major exports of Norway 3. The monthly time series for the Export of
condensed fuel (ExpCond), crude oil (ExpCrdOil), natural gas (ExpNatGas), new
ships (ExpNewShip), oil platform (ExpOilPlat), old ships (ExpOldShip) and all
other exports excluding ships and oil platforms (ExpExShipOilPlat)arepresented
in figure-2.11.
20
ExpCond ExpCrdOil
ExpExShipOilPlat ExpNatGas
ExpNewShip ExpOilPlat
ExpOldShip
0
5
10
15
20
200
300
400
500
600
700
800
900
100
200
0
5
10
15
20
0
10
20
30
0
5
10
15
20
2000 2005 2010 2015
Date (Monthly)
Value (NOK hundreds)
Fig 2.11: Time Series plot of major exports of Norway
2.5.2 Capital and Financial Accounts
The following text of capital and financial accounts are adapted from International
financial management by Madura (2012). A capital account includes transaction
of inter-country transfer of financial assets due to immigration and non-financial
assets such as buying and selling of patents and trademarks. These transaction
are relatively minor in comparison to the items of financial accounts. The key
elements of financial account are,
•Direct Foreign Investment includes investment in fixed assets in foreign
countries.
21
CapitalTransferAbroad AcqPatentLeisense NetLending
−50
−40
−30
−20
−10
0
10
−2.5
0.0
2.5
5.0
0
400
800
1200
1980 1990 2000 2010 1980 1990 2000 2010 1980 1990 2000 2010
Date (Monthly)
Value (NOK hundreds)
Fig 2.12: Time Series plot of variables related to capital account
•Portfolio Investment includes transaction of long term financial assets
such as bonds and stocks.
•Other Capital Investment includes short term financial assets such as
money market securities.
•Errors, Omissions and Reserves includes adjustment for negative bal-
ance in current account.
Due to unavailability of monthly data for capital accounts, this thesis has not
included the data in the analysis. The time series plot from quarterly totals for
the variables related to capital account are plotted in the figure-2.12. The figure
shows that the economy of Norway has drastically heated after the year around
1998.
22
OilSpotPrice
0.5
1.0
1.5
2.0
2000 2005 2010 2015
Date (Monthly)
Value (NOK hundreds)
Fig 2.13: Time Series plot of oil spot price from Jan 2000
2.6 Oil Spot Price
After the discovery of oil in the North Sea in late 1969, economy of Norway has
transformed completely (Norway The rich cousin 2013). Since the economy of
Norway is highly depended on its petroleum products, oil spot price also has
influence on foreign exchange rate of Norway. However, Ferraro, Rogo↵,andRossi
(2012) argued that the predictive ability of exchange rate from oil price is more
e↵ective at a daily frequency and is hardly visible at monthly frequencies. Oil spot
price is also considered as predictive variable in this thesis. The heavy fluctuation
in the oil spot price shown in time series plot (fig-2.13) is due to the financial crisis
of 2007-2009.
23
2.7 Lagged response variable as predictor
Exchange rate, being a time-series variable, contains autocorrelation which can be
checked out (soften) by including the lagged variables of the response as predictor.
Further, the correlation of response (PerEURO) with its first lag and second lag
are 0.94 and 0.86 respectively. In addition, two spikes which are significant in the
partial autocorrelation function as plotted in figure-2.14 also indicate for the use of
auto-regressive terms in the model. This thesis has included the first and second
lag of response variable as a predictor.
−0.25
0.00
0.25
0.50
0.75
0 5 10 15 20
Lag
Partial.ACF
Fig 2.14: Partial autocorrelation function for Exchange Rate of NOK per Euro. The red
dashed line denotes the 95% level of significance.
24
2.8 E↵ect of Crisis period
Financial crisis unleashed in the United State in summer 2007. The crisis extended
towards Europe which has created a series of difficult situations in the financial
market. Inter bank interest rate rose dramatically, stock market plunged and
banks incurred serious funding problem with losses on their head (The Financial
Market in Norway 2008: Risk outlook 2009).
Norway has been a↵ected by the crisis through various channels. Sharp fall
in commodity price, devaluation of companies and low international demand has
direct impact in exchange rate of NOK. The data during those period has high
influence in the statistical model using in this thesis. The influence of crisis is
visible in the plots of Appendix-C.
25
Chapter 3
Models and Methods
3.1 A statistical Model
A statistical model describes the relationship between a cause and its e↵ect. Let a
vector ycontains nnumber of responses and Xbe a n⇥pmatrix whose columns
are predictor variables and each of them have nobservations. These variables in
Xcan a↵ect yso, the relationship between Xand ycan be written in a functional
form as,
y=f(X)+✏(3.1)
where, ✏is a vector of unknown errors usually referred as ‘white noise’ when
dealing with time-series data which is assumed to have zero mean, constant vari-
ance and no autocorrelation.
26
3.2 Linear Regression Model
The linear regression model with a single response (Y=yt1,y
t2,...,y
tp)andp
predictor variable X1,X
2,...,X
phas form,
Y
Response
=0+1Xt1+2Xt2+...+pXtp
Mean Response explained by predictors only
+✏
Error Term
(3.2)
The model - 3.2 is linear function of p+1 unknown parameters ,1,2,...,p
which is generally referred as regression coefficients. In matrix notation, equation-
(3.2) becomes,
Y
n⇥1=X
n⇥(p+1)
(p+1)⇥1
+✏
n⇥1(3.3)
3.2.1 Least Square Estimation
The estimate of the unknown parameter vector in (3.3) is obtained by minimizing
the sum of square of residuals, The sum of square of residuals is,
✏t✏=(YX)t(YX) (3.4)
On minimizing equation - 3.4, we get the OLS estimate of as,
ˆ
OLS =(XtX)1XtY(3.5)
For ordinary least square estimation, following basic assumptions (Wooldridge,
2012) are required,
1. Linear in parameter
27
2. Absence of Multicollinearity
3. No correlation between Error terms and predictor variable, mathematically,
E(✏i|X)=0,t =1,2,...,n
The equation implies that the error term at time tshould be uncorrelated
with each explanatory variable in every time period
4. Homoskedastic Error terms, i.e,
var(✏t|X)=var(✏t)=2I
5. No serial correlation (autocorrelation) in error terms, i.e,
corr(✏t,✏s)=0,8t6=s
For Hypothesis testing and inference using tand Ftest, an additional assumption
of normality is needed, i.e
✏t⇠N(0,2I)
Under the assumption from 1 to 5, the OLS estimate obtained from equation-3.5
is best linear unbiased estimator (BLUE) of .
28
3.2.2 Prediction
Using ˆ
obtained in equation-3.5, following two matrices can be obtained,
Predicted Values: ˆ
Y=Xˆ
=X(XtX)1XtY(3.6a)
Residuals:ˆ
✏=Yˆ
Y=[IX(XtX)1Xt]Y(3.6b)
Here equation-3.6a gives predicted values of Ywhich on subtracting from
observed value give the predicted error terms as is presented in equation-3.6b.
Equation-3.6a can also be written as,
ˆ
Y=Xˆ
=HY (3.7)
Here, His called Hat matrix and is the orthogonal projection of yonto the
space spanned by the columns of X.
3.3 Variable selection
Although including many variables in the model can add information, they are
also the source of unnecessary noise. In addition, many variables in a model is
also the cause of multicollinearity. So, a model that is simple yet contain useful
information is always desirable. Variable selection is intended for selecting best
subset of predictor variables. Some of the criteria for variable selection as described
in Applied linear regression by Weisberg (2005) are discussed below:
29
3.3.1 Criteria for variable selection
Suppose Xsis selected set of variable which gives the predicted output of,
ˆ
Y=E(Y|Xsxs)=0
sxs(3.8)
If Xsmisses important variables, the residual sum of squares of fitted model in
equation-3.8 will be larger than the full model. Lack of fit for selecting the set Xs
is measured by its Error sum of square.
Model statistic Approach
When a model is fitted, various statistics such as R2,R2-adj, F-statistic are
obtained which measures the quality of that model. Based on these statistic,
a model is selected as better than others.
Information Criteria
Another common criterion, which balances the size of the residual sum of
squares with the number of parameters in the model (Johnson and Wich-
ern, 2007, p. 386), for selecting subset of predictor variable is AIC (Akaike
Information Criterion). It is given as,
AIC = nlog(RSSs/n)+k (3.9)
where, RSS=Residual Sum of Square, n=number of observation and
k=Number of variables included in the model
A model with smaller value of AIC obtained from equation-3.9 is better
better than other with larger AIC. An alternative to AIC is its Bayesian
30
analogue, also known as Schwarz or Bayesian information criteria. Bayesian
Information Criteria provides balance between model complexity and lack of
fit. Smaller value of BIC is better.
BIC = nlog(RSSs/n)+klog(n) (3.10)
A third criterion that balances the complexity and lack of fit of a model is
Mallows Cp(Mallows, 1973), where the subscript pis the number of variables
in the candidate model. The formula for this statistic is given in equation-
3.11,
Mallows Cp=RSS
ˆ2+2kn (3.11)
Where, ˆ2is from the full model. A plot of Cpvs kfor each subset of
predictors indicate models that predict the responses well. Better models
usually lie near the 45line of the plot.
3.3.2 Computational procedure for variable selection
When a model is large, fitting all possible subsets is not feasible. Furnival and
Wilson (1974) suggested several algorithm to calculate residual sum of square of
all possible regression called leap and bound technique which has been widely
implemented in statistical software. However, this method is not appropriate for
criteria based on model statistic where step wise methods can be used. methods
has three basic variation (Weisberg, 2005, p. 221).
Forward selection procedure
Model is started without any variable and in each step a variable is added and
31
the model is fitted. The variable is left in the model if the subset minimizes
the criterion of interest . Similar process is repeated for other predictor
variables.
Backward elimination procedure
This process is like the reverse of Forward selection procedure. In this pro-
cess, the model is fitted with all the predictor variable and variables are
removed one at a time except those that are forced to be in the model. The
model is examined against the considered criteria. Usually, the term with
smallest t-value is removed since this gives rise to the residual sum of square.
Stepwise procedure
This combines both Forward selection procedure and Backward elimination
procedure. In each step, a predictor variable is either deleted or added so
that resulting model minimizes the criterion function of interest.
3.4 Principal Component Analysis
The purpose of PCA is to express the information in X=(X1,X
2,...,X
p)bya
less number of variables Z=(Z1,Z
2,...,Z
q); q<pcalled principal components
of X(Martens and Naes, 1992). These principal components are orthogonal and
linearly uncorrelated. Since they are computed from the linear combinations of
Xvariables, the variation in Xvariables are compressed in first few principal
components. In other words, the first principal components is the direction along
which the Xvariables have the largest variance (Massart, 1998). In this situation,
the multicollinearity in Xis not a problem any more.
32
The principal components can be performed on Covariance or Correlation ma-
trix. If the variables are of same units and their variances do not di↵er much, a
covariance matrix can be used. However the population correlation matrix is un-
known, its estimate can be used. In this thesis, sample correlation matrix is used
to compute sample principal components. Construction of principal components
requires following steps,
1. Estimate the correlation matrix Aof Xas,
corr(X) = (diag(⌃))1
2⌃(diag(⌃))1
2(3.12)
Using sample observation, equation-3.12 can be estimated as,
A= corr(X) = (diag(S))1
2S(diag(S))1
2(3.13)
Where Sis the sample estimate of covariance matrix ⌃,
S=Eh(XE[X]) (XE[X])Ti(3.14)
2. Calculate eigenvalue and eigenvector of the correlation matrix obtained in
equation-3.13. An eigenvalue ⇤of a square matrix Aof rank pis a diagonal
matrix of order pwhich satisfies,
AE =E⇤(3.15)
where,
33
⇤= diag(1,2,...,p) (3.16)
In PCA these eigenvalues are arranged in descending order, i.e. 1
2... p. For each eigenvalues there is an eigenvector. Let E=
(v1,v2,...,vp) be the matrix of eigenvector so that the correlation matrix
Acan be decomposed and expressed as,
A=E⇤E1=E⇤ET(3.17)
Equivalently, |AiIn|E= 0 which can only be realized if AiInis
singular, i.e.,
|AiIn|= 0 (3.18)
Equation-3.18 is called the characteristic equation where, Ais the correla-
tion matrix obtained from equation-3.13. The root of the equation is called
eigenvalues (Seber, 2008) and the vector Eiis called eigenvector correspond-
ing to the eigenvalue i. The eigenvector obtained from equation-3.15 are
then normalized, i.e. ||Ei||2=1.
3. Since, the variation explained in data are accumulated in first few principal
components, only keigenvalues are considered. The corresponding eigenvec-
tors of those eigenvalues is called projection matrix. The projection matrix
is,
34
P=✓ET
1ET
2... ET
k◆T
(3.19)
The projection matrix in equation-3.19 projects the data matrix into lower
dimensional subspace Zi. i.e.,
Z=PX (3.20)
The column vectors of matrix Zobtained from 3.20 are the orthogonal pro-
jections of data matrix Xinto kdimensional subspace. These components
are the linear combination of the rows of matrix Xsuch that the most vari-
ance is explained by the first column vector of Zand second one has less
variance than the first one and so on. Here,
var(Zi)=iand
cov(ZiZj)=0fori6=j
3.5 Principal Component Regression
The components of Principal Component Analysis (PCA) accumulate the varia-
tion in predictor variables on first few components. A linear regression fitted with
only those components can give a similar results as the full linear model. How-
ever, Jolli↵e(1982)inhispaper“Anoteontheuseofprincipalcomponentsin
regression”, has given many examples taken from di↵erent papers of various fields
where the components with low variance are also included in regression equation
35
in order to explain most variation in the response variable. Following are the steps
to perform Principal Component Regression. These steps are based on the paper
“A comparison of partial least squares regression with other prediction methods”
by Yeniay and Goktas, 2002.
1. First principal components are obtained for Xas explained in section-3.4.
The PCs obtained are orthogonal to each other.
2. Suppose mPC which are supposed to influence the response are taken and
a regression model is fitted as,
Y=Zm↵m+✏(3.21)
3. Here, ↵m=ZT
mZm1ZT
mYare the coefficients obtained from OLS methods.
Using this alpha, one can obtain the estimate of as,
ˆ
PCR =PPTXTXP1PTXTY(3.22)
Here, Pis a projection matrix defined in equation-3.19.
Since, PCR includes only mcomponents, the estimate obtained are biased. ;The
number of components mcan be chosen by cross-validation the prediction mean
squared error (RMSEP). If all the components are included in the model, estimates
obtained from PCR, i.e. PCR are identical to the estimates of OLS (OLS).
36
3.6 Partial Least Square Regression
Partial Least Square Regression (PLS) is relatively new method and it can be
used for both univariate and multivariate regression. It constructs a new set of
variables called latent variable (or factor or components) from the linear combi-
nation of predictor variables X1,X
2,...,X
n(Garthwaite, 1994) as in the case of
principal components, however PCR construct components (factors) maximizing
the variation of data matrix(X) while PLS construct them using the variation
in both Xand Y(Yeniay and Goktas, 2002). The intention of PLS is to create
latent variables (components) that capture most of the information in the Xvari-
ables that is useful for predicting Y1,Y
2,...,Y
p, while reducing the dimension of
the regression problem by using fewer components than the number of X-variables
(Garthwaite, 1994). Partial least square regression can be performed using follow-
ing steps. These steps are adapted from the paper “PLS-regression: a basic tool of
chemometrics” from Wold, Sj¨ostr¨om, and Eriksson (2001). The Xand Ymatrices
are column centered for the ease of computation.
1. PLS estimates the latent variables also called X-scores denoted by
ta,(a=1,2,...,A), where Ais the number of Components a model
has considered. These X-scores are used to predict both X and Y, i.e. both
X and Y are assumed to be modeled by the same latent variable. The
X-scores are estimated as linear combination of original variables with the
coefficients W(wka) as in equation-3.23, i.e,
tia =
p
X
k=1
W⇤
kaXik (T=XW⇤)(3.23)
37
Where, W⇤is a vector of weights w⇤
aof X. It is obtained as in equation-3.24
below as a normalized coefficients obtained on regressing Xon a column of
Y.
W⇤=Xty(i)
kXty(i)k(3.24)
Here, y(i)is any column of response matrix Y.
2. The x-scores (T) are used to summarize Xas in the equation-3.25. Since
the summary of Xexplained most of the variations, the residuals (E)are
small.
Xik =X
a
tiaPak +eik ;(X=TP0+E)(3.25)
A similar setup can be used to have the summary for Y-matrix as in equation-
3.26,
Yim =X
a
uiaqam +gim ;(Y=UQ0+G)(3.26)
where, U=YQand Q=TtY
3. The X-scores (T) are also good predictor of Y, i.e.,
yim =X
a
qmatia +fim (Y=TCt+F)(3.27)
Here, Fis the deviation between the observed and modeled response.
38
4. Coefficients Estimates:
Equation(3.27) can also be written as,
yim =X
a
qma X
k
w⇤
kaxik +fim
=X
k
bmkxik +fim
In matrix notation this can be written as,
Y=XW⇤Ct+F=XB +F(3.28)
Thus, the estimates of PLS coefficients are obtained as,
ˆ
bmk =X
a
qmaw⇤
ka (3.29)
i.e., BPLS =W⇤Ct(3.30)
Above process is repeated for each components (a), the matrix Xand Yare
“deflated” by subtracting their best summaries (TPtfor Xand QCtfor Y).
The Residuals obtained are used as new Xand Yin the computation process
for new component. However, the deflation of Yis not necessary since the result
is equivalent with or without the deflation (Wold, Sj¨ostr¨om, and Eriksson, 2001,
p. 5).
Various algorithm exist to perform PLS regression among which NIPLS and
SIMPLS are in fashion. This thesis has opted NIPLS (Nonlinear Iterative Partial
Least Square) regression which is performed by oscores method of pls package
in R. In the algorithm, the first weight vector (w1)isthefirsteigenvectorofthe
39
combined variance-covariance matrix XtYYtXand the following weight vectors
are computed using the deflated version. Similarly, the first score vector (t1)is
computed as the first eigenvector of XXtYYtand the following x-scores uses the
deflated version of the matrices.
3.7 Ridge Regression
When the minimum eigenvalue of XtXmatrix is very much smaller than unity (i.e.
min << 1), the least square estimate obtained from equation-3.5 are larger than
average (Marquardt and Snee, 1975). Estimates based on [XtX+Ip],0
rather than XtXcan solve these problems. A.E. Hoel first suggests that to control
instability of the least square estimate, on the condition above, can be;
ˆ
⇤
ridge =⇥XtX+I⇤1XtY;0
=WXtY(3.31)
The analysis build around equation-3.31 is called “ridge equation”. The relation-
ship of ridge estimate with ordinary least square is,
ridge =hIp+XtX1i1ˆ
OLS
=Zˆ
OLS (3.32)
40
Here, as !0,ˆ
ridge =ˆ
OLS and !1,ˆ
ridge = 0 Further, the hat matrix for
Ridge regression is given as,
Hridge =XXtX+I1Xt(3.33)
All the theory behind Ridge Regression described above are cited from “Ridge
regression: Biased estimation for nonorthogonal problems” by Hoerl and Kennard
(1970).
3.8 Comparison Criteria
After fitting models with various methods, it becomes necessary to test their valid-
ity for their results to be trusted. Models react di↵erently for the new information
during prediction as the quality of model highly depends on their estimates. Since
the purpose of this thesis is to compare di↵erent models, the basis for their com-
parison are set as their (a) Goodness of fit and (b) Predictability.
3.8.1 Goodness of fit
A model is assumed to follow some hypothetical state of being ideal. Setting up this
state as null hypothesis (H), in many situations, the test of goodness of fit for a
model construct an alternative hypothesis simply stating that the model gives little
or no information about the distribution of the data. However in other situation,
such as testing for no e↵ect of some specific variable in the model, rejection of
Hindicate that the variable is useful in the model (D’Agostino, 1986, p. 1). A
goodness of fit for a model depends on many aspects such as,
41
Residual obtained after the model fit
Residuals obtained from the fitted model are assumed to be random and
normal considering that no useful information are still content on them.
Outlier
Outliers can distort the analysis toward unintentional direction creating false
estimates. Models without such outliers are considered better.
Variance explained by the model
The variance explained by the model is generally measured by R2or R2adj
in linear models. More the variation contained in the data is explained by
the model, better the model is considered. In the case of PLS and PCR, the
residuals contains very little information left on the ignored components.
Relative value of Information Criteria such as AIC and BIC
AIC (Akaike information criterion) and BIC (Bayesian information criterion
or Schwarz criterion) measures relative quality of models. Although, it is not
an absolute measure of the model quality, it helps to select a better model
among others. AIC is defined as in equation - 3.34 which is free from the
ambiguities present in the conventional hypothesis testing system (Akaike,
1974).
AIC = (2) log(L)+2(k) (3.34)
where, L= maximum likelihood and k= number of independently adjusted
parameters within the model For least square case, above formula resembles
to equation - 3.9 (Hu, 2007).
42
3.8.2 Predictability
Prediction is highly influenced by the model in used. So, prediction strongly
depends on the estimates of a model. False and unstable estimate makes the
prediction poor and unreliable. On one side, providing more information (variable)
can well train the model resulting more precise prediction. On the other hand,
over-fitting, which attempts to explain idiosyncrasies in the data, leads to model
complexity reducing the predictive power of a model. In the case of PLS and PCR,
adding more components results in including noise in the model.
123456789
1
2
3
4
5
6
0
Under fitting Over fitting
Error of prediction
Model Error
Estimation Error
Complexity of Calibration Model
Fig 3.1: Model Error - Estimation Error and Prediction Error
The relationship between the model complexity and the prediction error is
presented in figure-3.1 with the case of under-fitting and over-fitting of a model.
Furthermore, a model exhibits an external validity if it closely predicts the
observations that were not used to fit the model parameters (Lattin, Carroll, and
Green, 2003, p. 72). An over-fitted model fails to perform well for those obser-
43
vation that are not included during model parameter estimation. The dataset in
this thesis is divided into two parts. The first part includes the observations from
Jan 2000 to December 2012 and the second one includes observation onward till
November 2014. A cross-validation approach is utilized on the first set of observa-
tion to train the model. The model is used to predict the exchange rate of NOK
per Euro from the predictors of the second set of observations. Figure - 3.2 shows
the procedure adopted for prediction in this thesis.
Tra i n i n g D a taset
(Jan 2000 - Dec 2012)
Test Dataset
(Jan 2013 - Nov 2014)
Dataset
Y
train X
train
Y
test X
test
Calibrated
Model
•Linear Model
•PCR
•PLS
•Ridge
Model Compari-
sion Criteria
•Goodness of Fit
•Predictability
Best Model
Cross
Validation
Tes t d a t a
for Prediction
Tes t S t a t i stic
AIC, BIC, R2,
R2pred, RMSEP etc
Fig 3.2: Procedure adopted in the thesis for model comparison. A cross-validation tech-
nique is used to validate the trained dataset. The trained model is used to
predict the test response from with prediction errors are obtained.
Cross-Validation
There are various cross-validation techniques among which two are described be-
low;
K-Fold Cross-validation:
The dataset are split into kequal parts. For each i=1,2,...,k, a model
44
is fitted leaving out the ith portion. A prediction error is calculated for this
model. The process is repeated for all i. The prediction error for K-fold
cross validation is obtained by averaging the prediction error of each of the
model fitted.
Leave-one-out cross validation:
This is a special case of kfold cross-validation where k=n(number of
observation), i.e, each time one observation is removed and the model is
fitted.
Prediction Error
Prediction of a model becomes precise if the error is minimum. Models can be
compared according to their predictability. Understanding of di↵erent measures
of prediction error is necessary to acknowledge their predictability and eventually
perform model comparison.
Root Mean Square Error (RMSE)
RMSE is the measure of how well the model fit the data.
RMSE = v
u
u
t1
n
n
X
i=1
(yi