Data Sources for CPPIs: An Overview and Strategy

Abstract

Commercial property price indices (CPPIs) should be based on market transaction prices. Yet in monitoring average price changes over time price data can be sparse and the properties transacted each period of a different quality-mix. Due to the heterogeneity of commercial property, CPPI measurement requires a quality-mix adjustment so that the prices of like properties are compared over time with like. An appealing way around this sparse data and quality-mix adjustment problem is to use price data on broadly the same properties over time and avoid transaction price data. Tax or investment appraisal data or market valuations of real estate investment trusts (REITs) are two commonly used alternatives. While convenient, both such series can seriously mislead macroprudential and macroeconomic-policy makers. In this overview paper we point to the deficiencies of these data sources, outline and argue for the use of hedonic methods of quality-mix adjustment that are designed to work with sparse transaction price data in thin heterogeneous commercial property markets.

Full text

Data Sources for CPPIs: An Overview and Strategy.
Mick Silver
1
Abstract
Commercial property price indices (CPPIs) should be based on market transaction prices. Yet in
monitoring average price changes over time price data can be sparse and the properties transacted
each period of a different quality-mix. Due to the heterogeneity of commercial property, CPPI
measurement requires a quality-mix adjustment so that the prices of like properties are compared
over time with like. An appealing way around this sparse data and quality-mix adjustment problem
is to use price data on broadly the same properties over time and avoid transaction price data. Tax or
investment appraisal data or market valuations of real estate investment trusts (REITs) are two
commonly used alternatives. While convenient, both such series can seriously mislead
macroprudential and macroeconomic-policy makers. In this overview paper we point to the
deficiencies of these data sources, outline and argue for the use of hedonic methods of quality-mix
adjustment that are designed to work with sparse transaction price data in thin heterogeneous
commercial property markets.
1
Formerly Principal Statistical Methodologist, IMF. The author acknowledges the support of Eurostat in the writing of
this paper. The responsibility for the contents of this paper lies entirely with the author and views expressed within
should not be attributed to Eurostat.
Paper Presented at the EC International Conference on Real Estate Statistics,
20-22 February, 2019, Luxembourg.

2
I. Introduction
Commercial property price indexes (CPPIs) are hard to measure. Commercial property comes in many types
including: hotels; offices; wholesale and retail outlets; and buildings for traffic and communications, industry
and warehouses, public entertainment, education, hospitals, and institutional care. There is much diversity
within each type, for example, shops vary in size (small traders to large hypermarkets/department stores),
location (prime downtown to local), type of location (shopping malls, high-streets, local stores) and much
more.
2
This heterogeneity of commercial property makes the measurement of CPPIs problematic. Average
prices of retail property, for example, may increase over time but this may in part be due to a change in the
quality-mix of the properties transacted. For example, more larger stores in better (more expensive) locations
sold in the current quarter compared with the previous quarter would bias upwards a meaningful measure of
the change in average property prices. There is a need to measure constant-quality property price changes
and avoid this problem of changes in the quality-mix of properties transacted polluting the measurement of
price change.
A second problem is the infrequent turnover of commercial properties, especially for industrial properties. A
market price can only be identified when the property is transacted leading to a sparseness of price
observations—thin markets. Infrequent transactions for heterogeneous properties is the perfect storm for CPPI
measurement.
The seemingly appealing way around this sparse data and quality-mix adjustment problem is to use price data
on broadly the same properties over time and avoid transaction price data. Tax or investment appraisal data or
market valuations of real estate investment trusts (REITs) are commonly used alternatives to transaction data.
Appraisal data and REITs have the attractive property of including regularly-surveyed assessed prices of
properties providing a matched/paired panel database of property price estimates. Appraisals can in principle
be undertaken on all properties irrespective of whether they are transacted thus giving at least the appearance
of not suffering from the problem of sparse data. The matching of prices of the same property further enables
price indexes to be aggregated without an undue concern about changes in the quality-mix of properties
transacted.
However, while indexes based on appraisal prices seem promising, appraisals are often only undertaken
annually, have a subjective element and possible bias, and been found to lag and overly smooth corresponding
transaction indexes. While convenient, such series can seriously mislead macroprudential and
macroeconomic-policy makers. In this overview paper we point to the deficiencies of these data sources,
outline and argue for the use of hedonic methods of quality-mix adjustment that are designed to work with
sparse transaction price data in thin heterogeneous commercial property markets.
Problems of infrequent transactions on heterogeneous properties also arise with residential property price
indexes (RPPIs) but these problems are not so severe and have been largely overcome due to developments
in data availability and measurement methodology. The Bank of International Settlements (BIS) maintains a
database of RPPIs for 60 countries and CPPIs for 16 countries, the coverage and periodicity of the CPPI
series varies from country-to country.
3
2
Commercial property can be broadly defined to include all property other than owner-occupied housing and property
used in non-market activities (social housing and, for example, most types of non-residential property owned by
government). “Types” of commercial property are taken from the Classification of Types of Constructions (United
Nations/Eurostat, 1998): a nomenclature for the classification of constructions according to their type (Jens Mehrhoff in
Eurostat (2017, page 32) and Mehrhoff (2017).
3
The BIS Property Price database is available at: https://www.bis.org/statistics/pp_detailed.htm. The CPPIs are limited
for two countries (Philippines and Korea) to the price of land only, for two further countries to annual series (Poland and
Portugal) and for one country to a bi-annual series (Greece).

3
Section II critically examines these alternative data sources used for CPPIs and outlines some valuable
research that takes hedonic CPPIs and compares the results with appraisal- and REIT-based indexes finding
empirical evidence that indexes compiled from the latter data sources unduly smooth and, for appraisal
indexes, can considerably lag hedonic transaction-based CPPIs.
Section III briefly outlines some basic sampling theory to help support the case for using the recommended
RPPI practice of a transaction-based hedonic approach (Hill et al., 2018). We argue, on the basis of the
sampling theory, that the right course of action where sub-property markets have too few transactions is not to
turn to appraisals or REITs, but to identify the underlying data as being too sparse to provide macro-
economists with reliable RPPIs due to the overly wide confidence intervals accompanying the estimated CPPI
measure.
Section IV recognizes the problem of sparseness of data for the hedonic methodology and points to methods
that have been developed to better adapt the hedonic-transaction methodology to thin commercial property
markets.
Section V points to the political economy of international organizations wanting to respond to the demands
for data as likely to mislead users formulating macro-economic policy and financial stability. The honest
and professional stance is to focus on markets segments, such as offices and retail, where sample sizes are
sufficiently large and there is not undue heterogeneity; this is in line with the statistical theory on confidence
intervals on index numbers expounded in Section III. Appropriate transaction-based hedonic methods for
sparse data are outlined in Section IV.
Section VI concludes with a brief summary of a proposed way forward.
The motivation for this paper and flow of the argument is as follows: reliable timely measures of
commercial property price inflation are essential to macroeconomic planning and financial stability. While
the need for reliable RPPIs is well recognized (Heath and Goksu, 2015), standards of measurement have
been established (Eurostat et al., 2013), and countries are now successfully implementing such measures for
RPPIs (Hill et al., 2018) though similar progress is not apparent for CPPIs (Section V).
The need for reliable and timely CPPIs is strong but data and measurement problems using transaction data
more problematic. It is argued here that the use of appraisal/valuation data based on market investments and
tax assessments or REIT prices do not serve the needs of users and are likely to actually mislead them. The
argument that some data is better than none is a dangerous one if the “some data” unduly smooth and lags
transaction price indexes. The signal that users receive from the data is that the turning point has yet to
come, when in practice the economy may have hit it, and given data that says otherwise, no action is taken.
Statisticians have a responsibility to not issue misleading data. We point out that the political economy of
international institutions is to react to international data demands, especially when such demands are
prompted by high-profile G-20 directives. We will argue that the responsible course of action is to make use
and encourage the major developments in methodology for transaction-based CPPIs that take account of the
limitations of sparse data, and be transparent as to the reasons for not publishing estimated CPPIs with wide
confidence intervals.

4
II.
Alternative data sources and empirical evidence on the relationship
between price indexes from these sources
Appraisal data
4
Appraisal-based indices—the term “appraisals” is used here in a generic sense to include valuations for
investment or tax assessment—have the major advantage of extending the effective sample of price
observations beyond the often-sparse sample of transactions. Moreover, the extended sample of
observations can be matched over time with, in principle, the price of a property being assessed, say for tax
purposes, annually resulting in a panel of matched property prices over time. The larger samples and the
matched panel structure of the database, with the prices of like properties compared with like, not only
negates the need for quality-mix adjustments but also leads to more efficient estimates, in the sense of
smaller confidence intervals, for the resulting price index measures—see Section III. However, it will be
argued below that the increase in sample size is illusionary; appraisal prices are themselves based on
transaction prices and carry with them the insecure foundation of a more limited sample. Further, it will be
argued in Section III and shown in Section IV that the compilation of hedonic transaction-based indexes can
be best formulated as a matched-paired sample.
Appraisal-based indexes have a number of serious disadvantages when contrasted with transaction-based
indexes. These include:
• Appraisal-based indexes have a subjective basis that does not accord with the (transaction-based)
market valuation principle of price statistics and the System of National Accounts (2008 SNA
paragraphs (2.59) and (2.60)). The extent and nature of the bias will vary between and within
countries depending on the appraisal standards used in practice. There is an extensive literature with
references provided below on appraisal bias and its nature.
• Guidelines to professional appraisers are that they base their appraisal on the transactions of similar
properties currently in the market, see Baum and Crosby (2008), introducing circularity in the argument
that appraisals data solves the problem of sparse data. Appraisal data are often heralded as a panacea to
the problem of sparse data but they only give the appearance of doing so, the assessments being
grounded in sparse data.
5
This is particularly apparent when regressions are used to predict assessed
4
Limitations outlined below are largely based on Eurostat (2017, section 4.24, pages 34–36). Silver (2013) provides a summary
account of the different data sources.
5
Royal Institution of Chartered Surveyors (RICS) (2017, pages 78-79), outlines valuation approaches and methods: “….the overall
valuation approach is usually classified into one of three main categories:
o The market approach is based on comparing the subject asset with identical or similar assets (or liabilities) for which price
information is available, such as a comparison with market transactions in the same, or closely similar, type of asset (or
liability) within an appropriate time horizon.
o The income approach is based on capitalisation or conversion of present and predicted income (cash flows), which may
take a number of different forms, to produce a single current capital value. Among the forms taken, capitalisation of a
conventional market-based income or discounting of a specific income projection can both be considered appropriate
depending on the type of asset and whether such an approach would be adopted by market participants.
o The cost approach is based on the economic principle that a purchaser will pay no more for an asset than the cost to obtain
one of equal utility whether by purchase or construction.
3. Underlying each valuation approach and valuation method is the need to make such comparisons as are practically possible, since
this is the essential ingredient in arriving at a market view. It may well be possible to arrive at a valuation opinion by adopting more
than one approach and one method or technique, unless statute or some other mandatory authority imposes a particular requirement.
Great care must be exercised when relying on the cost approach as the primary or only approach, as the relationship between cost and
value is rarely direct.

5
prices from sparse actual transaction prices.
• Valuations made by an appraisal firm are largely conducted irregularly, say annually, and quarterly
data may in part be (stale) estimates by the manager/owner of the property largely based on the last
formal appraisal. It may be, for example, that a quarterly index compiled from annual appraisal data
would include, on average and depending on reporting requirements, three-quarters of its price
observations each quarter based on interpolations and all of its price change measures, based on
interpolations.
• Depending on the institutional arrangements for the country, the timeliness of an appraisal index,
especially if it relates to a requirement for an adequate sample size for interpolations, may be
problematic.
• Information on capital expenditures and depreciation are used, in appraisal-based indices, as a means for
quality adjustment between appraisals. There is much in the definition of these variables that render
them inadequate as currently constructed for the needs of CPPIs.
6
• Guideline for appraisals and definitions vary between and within countries, and substantially so.
• The sample of values is for larger professionally-managed properties—there may be a sample
selectivity bias.
• The population from which the data used to create appraisal indices changes over time. Since the
purpose of these indexes, as currently used in the private sector, is to capture changes in investment
returns of properties, they are estimated either by taking investment properties which are owned by
clients or by sampling the population. As a result, if a given property is sold off and is no longer an
investment target, it is removed from the index; if a property becomes a new investment target, it
becomes part of the index. In other words, the properties, which are the target of the index, change
over time. Average prices of a bundle of properties are no longer compared with average prices of
‘like’ bundle. In this sense, these indices are not completely free from bias stemming from quality
changes over time.
• There is evidence that appraisal-based indexes unduly smooth and lag prices—see Section II.
• Users for macroeconomic analysis have an established preference for transaction-based indexes. The
European Central Bank (ECB), as part of a stocktaking exercise on CPPIs, asked end-users their views
as to their needs: the relatively uniform response was for commercial property price index based on
transaction prices; valuation indexes were, as noted by Kanutin (2013), only ‘a second-best option’.
All of this is not to negate the efforts put into developing appraisal-based CPPIs. Kanutin (2013) has
compiled, with due diligence, an extensive dataset of CPPIs for European countries as part of the work by the
4.Valuation methods may include a range of analytical tools or techniques as well as different forms of modelling, many of which
involve advanced numerical and statistical practices. In general, the more advanced the method, the greater the degree of vigilance
needed to ensure there is no internal inconsistency, for example, in relation to the assumptions adopted.
5. Further detail on the application of approaches and methods may be found in the International Valuation Standards at IVS 105
Valuation Approaches and Methods. It must be emphasized, however, that the valuer is ultimately responsible for selection of the
approach(es) and method(s) to be used in individual valuation assignments, unless statute or other mandatory authority imposes a
particular requirement.”
6
Most investment return indicator appraisal-based indices eliminate capital expenditures from the capital return (price change) index,
and explicitly report capital expenditures on the properties each period. Capital expenditures are sometimes reported only to the
extent that such expenditure brings the property back to its previous period’s quality. A change in capital expenditure quality may be
over and above this sum, say an extension to the building.

6
European Central Bank’s (ECB) Working Group on General Economic Statistics (WGGES).
Empirical evidence on transaction and appraisal indexes
There is a growing literature on the nature and extent of valuation error in appraisal-property price
assessments. This includes: Gallimmore and Wolverton (1997); Kinnard, Lenk and Worzala (1997); Wolverton
(2000); Baum and Cosby (2008); Changha, and Gallimore (2010); Crosby, Lizieri and McAllister (2010); and
Geltner, Miller, Clayton and Eichholtz (2014). The research extends to valuation bias in less-developed
countries including, Awuah, Baffour, Gyamfi-Yeboah, Proverbs, and Lamond (2017) and Caleb, Durodola,
Oloyede, Omolade, and Oni (2018).
CPPIs using transaction costs have been measured for some countries and where appraised prices for
investment valuations, tax assessments and REITs are available this provides a valuable opportunity to
examine the extent to which the alternative indexes track each other. There will remain differences in the
coverage and methodologies used for the compilation/estimation of the different indexes from the different
data sources, however, such differences may well be endemic to the source data and be part and parcel of the
actual measure. For example, if REITs cover larger and more prestigious property holdings, or tax
assessments are annual and quarterly data is interpolated, or valuations of REITs are for the (changing
composition of the) portfolio, rather than individual properties, then while these differences will mar the
purity of a comparison of data sources, they reflect the differences in actual measures of commercial property
price inflation from appraisal/assessment and REIT data, as compared with transaction data. Where there
should be concern is with failings in the coverage and methodology of transaction-based CPPI which acts as
a benchmark. However, what the studies do find is a consistent smoothing and lagging of appraisal data with
regard to their transaction-based CPPI counterparts.
Geltner (2015), in the context of US data, ascribes this to two phenomena:
“First, the appraisal of individual property values reflects procedures of professional
appraisal practice that tend to result in some temporal lagging bias, and properly so, as
appraisers need to document their valuation estimates based on historical transaction price
evidence and as they need to filter out the noise that exists in individual transaction prices
(Quan & Quigley 1989, 1991). Second, although pension funds and their property
investment managers are required …. to reappraise each property at some frequency, this
is rarely done for all properties every quarter. Yet the NPI
7
includes all properties every
quarter, including those that are not reappraised in the current quarter and that are thus
reported to the index at a prior (stale) appraisal value.”
There is a growing literature on the difference between transaction and appraisal-based price indexes,
particularly for the US and Japan. This literature includes: Cole, Guilkey and Miles (1986); Geltner, Gra
and Young (1994); Geltner (1997, 1998); Crosby (2000); Geltner and Goetzmann (2000); Bowles, McAllister,
and Tarbert (2001); Clayton, Geltner, and Hamilton (2001); Geltner, MacGregor and Schwann (2003);
McAllister, Baum, Crosby, Gallimore, and Gray (2003); Geltner and Fisher (2007); Horrigan, Case, Geltner, and
Pollakowski (2009); Cannon and Cole (2011); and Geltner (2015) as examples of US studies indexes finding
evidence of lagging, smoothing, and errors with regard to transaction-based CPPIs. Nishimura and Shimizu
(2003), Shimizu and Nishimura (2006), and Shimizu, Nishimura and Watanabe (2012), Shimizu, Diewert,
Nishimura and Watanabe (2013), Shimizu (2016) and Diewert and Shimizu (2017) compared transaction-based
RPPIs and CPPIs with appraisal-based price indexes for Japan with similar findings.
7
The NPI [National Property Index] is an appraisal-based index for commercial properties compiled by the
National Council of Real Estate Investment Fiduciaries (NCREIF)—Silver (2013)

7
Shimizu and Nishimura (2006) outline the very particular institutional and cultural factors adversely
impacting on residential and commercial property price measurement in Japan. In spite of this, there has been
a vast research output. We take as an illustrative example recent work by Diewert and Shimizu (2017).
Figure 1 shows three price indexes for office space in Tokyo taken from the exhaustive study by Diewert and
Shimizu (2017, Table 10). The PFREIT is a Fisher price index based on appraisal values of properties whose
holdings are in REITs – they are not based on stock market prices themselves but on appraised values.
PFMLITS is a smoothed price index based on transaction data and is compiled as a weighted average of the
separate price series estimated for each of the land and structures components.
8
PFLDHEDS is again a
smoothed transaction-based price index, but this time estimated without benefiting from the analytical
decomposition of land and structure. PFMLITS and PFLDHEDS can be seen to be fairly similar. These two
transaction indexes showed the fall into the recession in 2007Q3 and 2007Q4 respectively. However, the
appraisal index would have informed the central bank and other macroeconomist that commercial property
prices were not only holding up, but increasing. It was not until 2008Q3 that a downturn was signaled by the
appraisal index, and then only by 0.8 percent (annual quarterly change) and continued to fall at a much lower
rate than the transaction-based indexes. Indeed, the appraisal index showed commercial property prices to
hardly recover for the remaining quarters of the series, to 2015Q. The appraisal-based price index increased
by 11 percent from the start of the series in 2005Q1, over 10 years the 10 years of the series, while the
transaction-based indexes showed prices to recover after the dip of the recession increasing at 43 and 54
percent respectively over the same period.
Figure 2 shows the same data for quarter-on-corresponding-quarter in the previous year—percentage annual
quarterly changes—an indicator arguably more suitable for monitoring price changes. Again, it is in 2007Q3
and 2007Q4 that alarm bells would have rung as these indexes plummeted, as shown by the PFMLITS and
PFLDHEDS, but it was only in 2008Q2 that we saw similar falls with the former indexes crossing into a
negative percentage change in 2008Q1 while the appraisal index did so in 2008Q4, but with a fall of 1.4
percent contrasting with the 18 and 15 percent falls of the above two transaction-based indexes. At the trough
of the fall in the commercial property prices (both in 2009Q3) the transaction-based indexes were registering
falls of 25.5 and 20.2 percent while the appraisal index reaches fell by only 12.8 Percent (2009Q4).
This study is of course specific to a particular type of commercial property in a specific country, but is
indicative of findings in the literature. Given such results it should be hard for statisticians to advocate for the
provision of appraisal indexes.
8
The innovative framework for the decomposition of Land and structures in property price index
measurement is due to Diewert and Shimizu (2015) and Diewert, Fox and Shimizu (2016).

8
Real Estate Investment Trusts (REITs)
Shares in Real Estate Investment Trusts (REITs) are traded on the stock market. Their equity share prices
provide data on the value of the property holdings and how the value changes over time. The resulting data
0.8
0.9
1
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
2005Q1
2005Q2
2005Q3
2005Q4
2006Q1
2006Q2
2006Q3
2006Q4
2007Q1
2007Q2
2007Q3
2007Q4
2008Q1
2008Q2
2008Q3
2008Q4
2009Q1
2009Q2
2009Q3
2009Q4
20010Q1
20010Q2
20010Q3
20010Q4
20011Q1
20011Q2
20011Q3
20011Q4
20012Q1
20012Q2
20012Q3
20012Q4
20013Q1
20013Q2
20013Q3
20013Q4
20014Q1
20014Q2
20014Q3
20014Q4
20015Q1
20015Q2
20015Q3
20015Q4
Figure 1: Commercial property price indexes for Tokyo
offices, using transaction and appraisal data: 2005Q1=1.00 -
Diewert and Shimizu (2018)
PFMLITS(trans: L+S) PFREIT(appraisal) PLDHEDS(trans)
-30
-10
10
30
2006Q1
2006Q2
2006Q3
2006Q4
2007Q1
2007Q2
2007Q3
2007Q4
2008Q1
2008Q2
2008Q3
2008Q4
2009Q1
2009Q2
2009Q3
2009Q4
20010Q1
20010Q2
20010Q3
20010Q4
20011Q1
20011Q2
20011Q3
20011Q4
20012Q1
20012Q2
20012Q3
20012Q4
20013Q1
20013Q2
20013Q3
20013Q4
20014Q1
20014Q2
20014Q3
20014Q4
20015Q1
20015Q2
20015Q3
20015Q4
Figure 2: Commercial property price inflation for Tokyo:
quarter-on-quarter annual change - Diewert and Shimizu
(2018)
PFMLITS(trans: L+S) PFREIT(appraisal) PLDHEDS(trans)

9
are of a very high frequency, often compiled on a daily basis. REIT holdings are commercial real estate
portfolios. Most of their earnings are paid out as dividends. They have the advantages of being a readily-
available high-frequency (daily) indicator that can lead the private market due to the efficiency with which
trades can be made. They do not suffer from problems of thin markets and sparse transaction data. However,
they have some disadvantages:
• They do not track the transaction prices of commercial properties within the private property market, but
rather only indirectly reflect the valuation of such assets by investors and traders in the stock market.
• The holdings of properties in portfolios will change over time as new properties are added and
existing ones sold. Changes in the stock market valuation will be in part determined by such
compositional changes, though the extent to which this takes place will not be apparent to the
user.
• Not all commercial properties are publicly traded. Many will be owned and traded outside of REITs
in the private property markets. Commercial properties held in publicly-traded REIT portfolios are
likely to be larger, on average, than properties traded in the private property market and may
experience quite different price changes.
• The stock market is a more efficient vehicle for trading in commercial property, especially for
enabling a quick turnover in purchases and sales. Its movements will not just reflect (a best guess
at) the underlying market price, but also the “animal spirits” that can drive the stock market. The
market price of a commercial property is something that is only realized when the asset itself is
bought and sold in the market. The concern of CPPIs is with the price change measurement of
commercial property and REIT indexes with investment returns. They are different phenomena.
• Buyers and sellers in the private market may have different priorities regarding their purchases and
sales. The private market purchaser, for example may be looking for long-term returns.
• They are measures of total investment returns which include rents and other incomes, while CPPIs
are price indices for commercial property as a non-financial asset whose price change is in principle
the change in the market price in the current period relative to a reference period, with no account
taken of income.
Empirical evidence on transaction and REIT-based indexes
9
Figure 4 is from Geltner (2015) based on data provided in Elonen (2013). Figure 4 provides an illustrative
example of the difference between a REIT-based price index and a transaction-based one for commercial
property assets for Germany, quarterly from the end of 2009 through 2012. The VDP Property Price Index is
a hedonic transactions-based price index of commercial property. PureProperty is based on publicly traded
European REITs and listed property companies that hold properties in Germany, Eurostat (2017) and Elonen
(2013). The three-year period covered by Figure 4 includes the initial recovery from the global financial crisis
as well as the crisis in the euro in 2011, which by 2012 led to capital flight into Germany. As Geltner (2015)
notes, the PureProperty index is based on “….public securities markets, a type of market in which prices reflect
an equilibrium that moves very freely and quickly, responding to news and shifts in supply and demand. The
private property market in which German investment institutions trade the properties whose prices are reflected
in the VDP index is a much quieter and more traditional arena.”. They are two different indexes operating in
roughly the same markets showing quite different price changes.
9
This account is based on Geltner (2105).

10
Figure 4, CPPIs and REIT-based commercial property price indexes for Germany:
2009: Q4=100.00
113
111
109
107
105
103
101
99
97
95
2009 2010 2010 2010 2010 2011 2011 2011 2011 2012 2012 2012 2012
Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4
Stock market–based and transactions-based commercial real estate price indices for Germany. Abbreviations:
REIT, real estate investment trust; VDP, Verband Deutscher Pfandbriefbanken (Association of German
Pfandbrief Banks). Data are from Elonen (2013).
III.
Sampling theory and confidence intervals of index number formulas
The concern with using transaction-based CPPIs is the problem of sparse data, a problem phrased in terms of
small sample sizes. Yet it is the heterogeneity of the sample that compounds the problem. Errors arising from a
small heterogeneous sample are reflected in statistical sampling theory by the width of the confidence interval
attached to the sample estimates. The (confidence) interval estimate in which we would expect the population
mean to fall 95 percent of the time is
   

where

is the sample mean,

the population standard,
deviation (to be approximated by the sample standard deviation), n the sample size for each randomly drawn
sample, and


the standard error (SE) of the sampling distribution. By the central limit theorem, the sampling
distribution of the means,

, is normally distributed for large samples—a rule of thumb is
  
—irrespective
of the underlying distribution of the variable

. What is important is that it is not just the sample size that
determines the width of the confidence interval, but also the heterogeneity of the property prices. If all properties
transacted in a quarter were of the same price, then a sample size of one would be sufficient. Conversely, if the
standard deviation was very large, what might be considered to be a sufficiently large sample size may yield very
large confidence intervals and imprecise estimates.
However, we do not require a confidence interval for a sample mean, but for the ratio of sample means. For the
(unweighted) Dutot price index, the required ratio is of arithmetic mean prices in the current period compared
with arithmetic mean of prices in the reference period. For the Jevons index the required ratio is of geometric
mean prices in the current period compared with geometric mean of prices in the reference period. We consider
here the Jevons index because of its better axiomatic properties.
If the sample of properties compared by the two means of prices—in the current and reference period—are
independent samples, the Jevons index is the (exponent of) the difference between the arithmetic means in the
REIT-based
(PureProperty index)

11
current and reference period of the logarithms of prices and the SE of this difference is the square root of the sum
of the separate variances (standard errors squared) of the two samples. Thus where


is the difference between
the logarithms of prices in the current and reference periods for property i,






,


the mean of
these differences,

(

)
 

 


, and the 95% confidence interval for the difference is







to  —a simple illustration is provided in Altman and Bland (2003). The confidence
interval for the Jevons index is simply the exponents of these lower and upper bounds on the confidence
interval. As such the SE from sparse heterogeneous independent samples will be much higher than would be
anticipated from simply judging what it might be based on the sample size in an individual period. It is the
ratio of the means we are interested in, not the individual means, and the ratio’s standard error. The
implication is that the problem of sparse data is more serious than might be initially considered.
The large standard error for the ratio of the means from the two independent samples would substantially increase
the confidence interval making the results from spare heterogeneous data much more imprecise than might be
initially considered. Re-sampling properties transacted each period may be considered as independent sampling.
The lesson here is not to compare (geometric) average transaction prices when you have sparse heterogeneous
data. The samples are independent and the SE suffers as a result. Appraisal data does not need to use independent
samples. The data are matched/paired prices for the same property in the two periods. The (exponent) of the
average difference between the logarithm of these prices is the Jevons index and its SE is based on the standard
deviation of these differences, much smaller than the sum of the SEs for the two independent samples. This
argues for appraisal data and indeed gives a statistical validation to the argument for the data’s matched
pairs/panel structure
The question is whether transaction-based data can be used to compile a Jevons index that not only adjusts the
mean prices in the current and reference periods for changes in the quality-mix of properties, but also phrases the
calculation in terms of matched-paired price comparisons, rather than independent samples, so that the SE is no
longer the sum of the SEs of each independent sample. The answer is “yes.” The best-practice hedonic
methodology, as espoused in Hill and Melser (2008), Diewert, Heravi and Silver (2009), Hill (2013), de Haan and
Diewert (2013), and Silver (2018), does exactly that. The hedonic imputation approach takes transactions in the
current period and then for these same matched-paired properties imputes the prices they would have sold at in
the reference period. The imputation is undertaken using a hedonic regression estimated for the reference period.
A similar exercise can be undertaken using a hedonic regression to impute the current period prices of reference
period properties, as outlined in more detail in the next section. We thus have a quality-mix adjustment and a
transference of the data away from independent samples to matched paired comparisons achieved by a leading-
edge hedonic methodology.
We describe it as “leading-edge methodology” since first, international recommendations on HPI measurement
methodology advocate the hedonic methodology as the preferred method for quality-mix adjustment
(Eurostat, ILO, IMF, OECD, UNECE, World Bank (2013), European Commission (EC) (2017), and Hill
(2013). Second, it is widely used, Hill et al, 2018), and third the imputation method benefits by having been
shown to be equivalent, for quite reasonable specifications of the hedonic regression and aggregation
formulas, to the characteristics approach
10
and to be readily adapted using information available in real time,
10
A log-linear hedonic characteristics price index with constant reference-period average characteristics,
0
00
0,
1
k i k
iN
zz
N
=
,
is equals an imputation index for reference period properties:
P
HGMI:zi
0
0®t=
ˆ
b
k
t
( )
zk
0
k=0
K
Õ
ˆ
b
k
0
( )
zk
0
k=0
K
Õ=exp zk
0ln ˆ
b
k
t
k=0
K
å
æ
è
çö
ø
÷
exp zk
0ln ˆ
b
k
0
k=0
K
å
æ
è
çö
ø
÷
=exp 1
N0zi,k
0ln ˆ
b
k
t
iÎN0
å
k=0
K
å
æ
è
çö
ø
÷
exp 1
N0zi,k
0ln ˆ
b
k
0
iÎN0
å
k=0
K
å
æ
è
çö
ø
÷

12
to being weighted at the elementary level using a superlative index number formula, see de Haan and
Diewert (2013), Hill and Melser (2008), and Silver (2018).
It may be argued that the use of the hedonic imputation approach to create a matched panel data structure,
while an improvement on the use of comparisons of independent samples, only puts the transaction-based
data set on a par with the appraisal data set in the sense that both are now compiled from matched data. The
transaction-based CPPI now avoids the larger standard errors from comparing means from independent
samples. Appraisal/assessment data will still have a larger sample size being available for all properties
appraised/taxed, rather than the hedonic paring of those transactions in the (extended) reference period and
current period. However, this larger appraisal/assessment sample size is an illusion. Such prices are based on
the more limited transaction data and used in a subjective and inconsistent manner with documented bias to
arrive at the appraised prices. Further, the quarterly estimates are based on extrapolations from annual
estimates; the basis of, and extent to which, these extrapolations are employed is unknown to the user and
can swamp measured (transaction-based) price change. If, for example, there were only four properties sold
for a type of commercial property during a quarter, appraised prices in the current period for the much larger
sample will be based on these four prices, never mind other biases, lags and staleness that may be part of the
appraisal price-determining process outlined in Section II.
A final take-away from this section is that it is the confidence interval associated with an estimate from a
sample of sparse data that matters. If the underlying transaction data is based on a small sample of
transactions the confidence interval will be large and results unreliable, indeed even worse, misleading, for
policy purposes. Dressing up a small sample in an abundance of appraisal prices does not negate this
statistical principle.
IV. The hedonic imputation approach and matched-paired data
11
=exp 1
N0zi,k
0ln ˆ
b
k
t
k=0
K
å
iÎN0
å
æ
è
çö
ø
÷
exp 1
N0zi,k
0ln ˆ
b
k
0
k=0
K
å
iÎN0
å
æ
è
çö
ø
÷
=
ˆ
pi|zi
0
t
( )
1
N0
iÎN0
Õ
ˆ
pi|zi
0
0
( )
1
N0
iÎN0
Õ
and similarly, average characteristics held constant in the current period t,
zk
t=1Ntzi,k
t
iÎNt
å
is equal to an imputation
index for current period t properties:
P
HGMI:zi
t
0®t=
ˆ
b
k
t
( )
zk
t
k=0
K
Õ
ˆ
b
k
0
( )
zk
t
k=0
K
Õ=
ˆ
pi|zi
t
t
( )
1
Nt
iÎNt
Õ
ˆ
pi|zi
t
0
( )
1
Nt
iÎNt
Õ
11
The focus here is on hedonic transaction-based indexes, rather than Repeat Sales (RS) indexes. The Repeat Sales (RS)
method was developed to overcome the quality-mix problem by constraining the sample of properties used in RPPI
measurement to those sold more than once over the period in question. The inherent weaknesses of the RS method lies
in its deletion of single-sales data, a potential lemons bias, and failure to properly account for the
depreciation/improvements to properties between sales. There is also a major problem of determining how much weight
should be given to pairs of price comparisons with a long time period between sales. Leventis (2008) has found
differences in the autoregressive formulation of the RS model, used to weight such paired comparisons, can account for
significant differences in the index results. Hedonic regression methods are now more prevalent than the RS method as a
result of the increasing availability of detailed data sets of house prices and characteristics, mainly arising from the
development of on-line residential property sales databases, and the development of a more sophisticated hedonic RPPI
methodology (Hill and Melser (2008); Hill (2013); De Haan and Diewert (2013); Diewert and Shimizu (2017), and
Silver (2018) led to the development of international standards of measurement for RPPIs (Eurostat, 2013) and the
widespread development of RPPIs both in terms of the number of countries and their quality of measurement (Hill et al.,
2018).

13
A log-linear hedonic regression equation
12
for (the logarithm of) prices on
,
t
ki
z
characteristics for period t
data is given by:
(1)….
ln pi
t=ln
b
0
t+zk,i
tln
b
k
t
k=1
K
å+ln
e
i
t
An estimated OLS regression equation for equation (1) is given as:
(2)….
ln ˆ
pi
t=ln
b
0
t+zk,i
tln
b
k
t
k=1
K
å
where
ˆt
i
p
(and
t
i
p
) are the predicted (and actual) price of property i in period t;
,
t
ki
z
are the values of each
k=1,….,K price-determining characteristic for property i in period t;
ˆ
b
0
t
and
b
k
t
are the estimated (and
actual) coefficients for each characteristic
zk
t
;
t
i

are i.i.d. errors, using period t data and characteristics.
The imputation approach works at the level of individual properties, rather than the average values of their
characteristics. The rational for the imputation approach lies in the matched model method. Consider a set of
properties transacted in period t. We want to compare their period t prices with the prices of the same
matched properties in period 0. In this way there is no contamination of the measure of price change by
changes in the quality-mix of properties transacted. However, the period t properties were not sold in period
0—there is no corresponding period 0 price. The solution—in the denominator of equation (3)—is to predict
the period 0 price of each period t property. We use a period 0 regression to predict prices of properties sold
in period t to answer the counterfactual question: what would a property with period t characteristics have
sold at in period 0?
A constant-quality hedonic geometric mean imputation (HGMI) price index is a ratio of the geometric means
of prices of individual properties in period t compared with period 0 of properties transacted in the current
period t. The value in the numerator of equation (3) is the geometric mean of the period t price of period t
price-determining characteristics,
zi,k
t
. This is compared, in the denominator, with the geometric mean of the
period 0 predicted price of the self-same period t price-determining characteristics,
zi,k
t
. For each property,
the quantities of characteristics are held constant at period in period t,
zi,k
t
; only the characteristic prices
change. Where Nt is the number of properties transacted in period t:
(3)…..
P
HGMI:zi
t
0®t=
ˆ
piçzi
t
t
( )
1
Nt
iÎNt
Õ
ˆ
piçzi
t
0
( )
1
Nt
iÎNt
Õ
=exp 1Ntln ˆ
piçzi
t
t
iÎNt
å
æ
è
çö
ø
÷
exp 1Ntln ˆ
piçzi
t
0
iÎNt
å
æ
è
çö
ø
÷
And a constant period 0 characteristics,
zi
0
, hedonic imputation HGMI where N0 is the number of properties
transacted in period 0 is given by:
(4)….
P
HGMI:zi
0
0®t=
ˆ
pi|zi
0
t
( )
1
N0
iÎN0
Õ
ˆ
pi|zi
0
0
( )
1
N0
iÎN0
Õ
=exp 1N0ln ˆ
pi|zi
0
t
iÎN0
å
æ
è
çö
ø
÷
exp 1N0ln ˆ
pi|zi
0
0
iÎN0
å
æ
è
çö
ø
÷
.
12
Triplett (2006) provides a thorough overview of the concepts and methods of hedonic regressions.

14
We use predicted prices in both the numerators and denominators of equations (3) and (4): a dual imputation.
For example, in equation (3) the single imputation index could be defined to use the actual price in the
numerator and predicted price in the denominator. The denominator is a counterfactual price that a transacted
property in period t would have sold at period 0; a hedonic regression in period 0 is required.
Yet a feature of the ordinary lest squares (OLS) estimator is that the mean of actual prices is equal to the
mean of predicted prices:
(5)….
0
00
00
00
11
ˆii
i
i N i N
pp
NN

=

|z
and
11
ˆt
i
tt
tt
i
tt
i
i N i N
pp
NN

=

|z
.
Thus, while the denominator of equation (3) must be counterfactual and use predicted prices, the numerator
of equation (3) can use actual prices. Thus, when using un-weighted hedonic imputation indexes there is no
need to estimate hedonic regressions in each period for (3), actual prices can be used in the numerator.
Equations (3) and (4) can be seen to take the form of matched-paired data, rather than independent samples
with the actual (or predicted) price of property i=1 in equation (3), for example, in the numerator compared
with the counterfactual price of what this same property, that is one with identical characteristics as specified
in the hedonic regression, would have sold at in period 0, as predicted by the estimated hedonic regression
equation. The summation sign encompasses all properties transacted in period t and their imputed period 0
prices.
Tweaking the hedonic methodology for sparse data
Use an HPI methodology that only require a hedonic regression to be estimated in the reference period.
The proposed measure below is a hedonic imputation based only on a sample of period t transactions.
Equation (3) above only requires a hedonic regression to be estimated in the reference period 0 so that
properties from the sample of transactions in period t with their associated characteristics can be valued in
period 0—the denominator. While predicted prices are shown in period t for the numerator, actual prices
will suffice as shown by equation (5) above.
Limiting the regression estimation to the reference period is a major advantage. Hedonic regression
estimates are subject to the vagaries of specification and estimation procedures, particularly in thin markets.
A requirement to estimate in each period a new hedonic regression not only opens the estimation and
compilation of HPIs up to both the vagaries of rushed hedonic estimation but also to an increased delay in
publication.
Use an extended period for the reference period.
The reference period should be over an extended period, rather than a quarter. First, there may not be an
adequate number of observations and/or variation in the characteristics of the sample of properties transacted
in a say quarterly reference period to enable reliable and pertinent estimates to be made of the coefficients of
price-determining characteristics that define properties sold in the current period t. For example, there may a
relatively small number of offices sold in a prime location in period t, but none sold in a quarterly reference
period 0, for example 2019:Q1=100.00. An extended period 0 regression will be more likely to better
encompass the characteristics of period t properties as well as basing the regression on a larger sample size
providing coefficient estimates with more degrees of freedom.
13
13
The advantage of not having to re-estimate a hedonic regression on a periodic basis is well recognized by NSIs in
Europe. The repricing variant of the characteristics approach used by eight countries has an extended reference period of
a year to establish the average values of the characteristics and the commensurate estimated marginal values from the
hedonic regression. The repricing approach allows for this due to its correspondence to the characteristics approach and
equivalence to the imputation approach when crafted following the principles in section III Table 1.

15
There are thus many advantages to the use of a say reference period of say the whole year of 2019 (=100)
as the database for the hedonic regression, with a hedonic regression re-estimated on an annual basis, or
every two years allows a CPPI to be compiled as a chained index, linking together the quarterly component
sub-indexes, for example, 2021=100.00, 2023=100.00, etc. by successive multiplication. The desirable
duration of the reference period and the frequency with which the reference period is updated can be based
on exploratory work with initial experimental CPPIs compiled using reference periods with different
durations and updated at different frequencies.
Use weights at the elementary level
A benefit of matched-paired data is that weights can be applied at the elementary level. The expenditure
weight for an individual property is its relative price and data on relative prices exist in real time for each
property and can be applied to each respective property’s price change.
14
The weighted version of equation
(3) is given by equation (6) below as a quasi-hedonic formulation of a Törnqvist index.
(6)….
P
QToHGMI:zi
0
0®t=ˆ
pi|zi
0
t
ˆ
pi|zi
0
0
æ
è
ç
ç
ö
ø
÷
÷
iÎN0
Õ
ˆ
wi
t
=exp ˆ
wi
t
ln ˆ
pi|zi
0
t
ˆ
pi|zi
0
0
æ
è
ç
ç
ö
ø
÷
÷
iÎN0
å
æ
è
ç
ç
ö
ø
÷
÷=exp ˆ
wi
t
ln ˆ
pi|zi
0
t-ˆ
pi|zi
0
0
( )
iÎNt
å
æ
è
çö
ø
÷
where
ˆ
wi
t
=1
2
ˆ
pi|zi
t
t
ˆ
pi|zi
t
t
iÎNt
å+ˆ
pi|zi
t
0
ˆ
pi|zi
t
0
iÎN0
å
æ
è
ç
ç
ç
ö
ø
÷
÷
÷
, an index that has excellent properties in economic theory as a
superlative index. It is “quasi” in the sense that it does not make use of the sample of period t transactions. It
is “superlative” in the sense that the index of price changes of transactions undertaken in period 0 makes
symmetric use of reference and current period price information (Diewert, 1976 and Balk, 2008). Silver
(2018) shows how a weighted version of equation (4) can also be formulated as can a superlative index that
makes use of both base period 0 and current period t transactions as the two weighted versions of equations
(3) and (4) are put together as a weighted average to form a Törnqvist hedonic imputation index for the full
sample.
There are other methods of measuring property price indexes where there are sparse transactions and the
reader is referred to Goetzmann (1992), Geltner (1993), Bokhari and Geltner (2012) and Silver and Graf
(2014). There may be no easy solution, but it must not be for lack of serious research that can, at the very
least, identify and surmount some methodological pitfalls and confront others with eyes wide open.
V. The institutional setting
This section outlines the institutional setting for the development of CPPIs. While the primary work on
developing and compiling regular CPPIs will fall on national statistical offices, central banks and, to a lesser
extent, private and quasi-governmental organizations, the responsibility for establishing measurement
standards and promoting the methods lies with the international organizations. The primary institutional
initiative under which global standard setting for CPPs is promoted is the G-20 Data Gaps Initiative (DGI).
In April 2009, the Group of Twenty (G-20) Finance Ministers and Central Bank Governors (FMCBG)
Working Group on Reinforcing International Co-operation and Promoting Integrity in Financial Markets
called on the International Monetary Fund (IMF) and the Financial Stability Board (FSB) to explore
information gaps and provide appropriate proposals for strengthening data collection and report back to the
Finance Ministers and Central Bank Governors. This call was endorsed by the IMF’s International Monetary
14
A similar procedure can be applied to appraisal-based indexes though has not, to the author’s knowledge, been used.

16
and Financial Committee (IMFC). The work is coordinated by the Inter-Agency Group on Economic and
Financial Statistics (IAG) which comprises representatives from the Bank for International Settlements
(BIS), the European Central Bank (ECB), Eurostat, the IMF, Organisation for Economic Co-operation and
Development (OECD), the United Nations, and the World Bank.
The first Report was on October 29th 2009 and listed 20 recommendations to address information gaps
revealed by the global financial crisis, one of which was real estate indicators. Work on the DGI focused on
the RPPI making considerable progress including the development of methodological standards, Handbook
on Residential Property Price Indices (RPPIs), published in April 2013 (Eurostat et al., 2013), and a
considerable uptake in country practice, currently 60 countries.
There is another group that is responsible for international standards on price statistics. This is the Inter-
secretariat Working Group on Price Statistics (IWGPS) comprising representatives from Eurostat, ILO, IMF,
OECD, UNECE, and the World Bank. This is a Group that reports to the United Nations Statistics
Commission (UNSC). All Manuals and Handbooks must be approved by the IWGPS before going to the
UNSC for final approval. There is some overlap in membership of the IWGPS and IAG, though in practice it
is the author’s experience that the two bodies have worked productively together.
Given this progress, in September 2015, its sixth year, the G-20 FMCBG closed the first act (DGI-1) and
opened a second act of the DGI (DGI-2)—Heath and Goksu (2015). DGI-2 recommendations II.17 and 18
addresses the development of RPPIs and CPPIs respectively. The BIS is the lead agency on CPPIs:
Recommendation II.18.
Recommendation II.18 on Commercial Property Prices is:
“The IAG in collaboration with the Inter-Secretariat Working Group on Price Statistics to enhance
the methodological guidance on the compilation of Commercial Property Price Indices (CPPI) and
encourage dissemination of data on commercial property prices via the BIS website.” (IMF and FSB,
2015, page 40).
This shift in focus towards CPPIs was prompted by both an appreciation of their importance, the lack of
methodological standards, and the small number of countries disseminating CPPIs:
“Commercial Property Price Indices (CPPI) are at a less developed stage, both conceptually and in
terms of available data. To this end, methodological guidance for the compilation of commercial
property prices indices is being drafted and expected to be finalized in 2015. However, unlike the
guidance for RPPI, there remain significant differences” (IMF and FSB, 2015, pages 39–40).
However, by the Progress Report of September 2018 matters had not significantly improved. Table 1 of the
Report is a dashboard of achievements—Overall Implementation Status and Progress for the DGI-2
Recommendations. While “10 G20 economies report CPPI data” and “CPPI Sources, Methods and Issues
was published in December 2017 by Eurostat,” are listed as Fully or completed workstream; “No harmonized
methodological framework nor detailed methodological guidance available yet. Action plan still to be
elaborated” are included as Early stages of implementation or lack of timely progress. IMF and FSB (2018,
page 9).

17
The Traffic light monitoring dashboard for Recommendation II.19 is less than inspiring with 10 of the G-20
countries boxes shaded in red (target of publishing a CPPI not met, and 9 in green, target met, the Euro area
being partially met, orange).
15
Of note is the summary of the Conference on RPPIs and CPPIs organized by the IMF in coordination with
the BIS, Eurostat, and the OECD, and hosted by Instituto Nacional de Estadstica y Censos de la Repblica
Argentina in Buenos Aires, during January 29–30, 2018:
“The participants agreed that the way forward with real estate statistics, especially CPPI, should be
pragmatic, data-oriented, and take account of available private data sources for economies where no
official indicators exist.” IMF and FSB (2018, paragraph 5).
So, what is the take-home from all of this? Major decisions on international standards are taken by
international organizations, though these in turn, through the UNSC and other fora, are responsible to its
member countries. The pressure by DGI-2 is to resolve the methodological crisis and turn the red boxes in
Table 1 to green. This can most quickly be undertaken by adopting appraisal-based commercial property
data. For European countries (Kanutin, 2013 and subsequent papers) such indexes are effectively up and
running.
A more measured approach, as will be proposed below, will be more problematic. At the end of this
conference there is a summary session from which the next IMF and FSB Report on DGI-2 Progress will
include a section. So, we turn to a proposed way forward.
VI. A proposed way forward
Transaction-based CPPIs should be the recommended methodology.
They should only be used for market segments that have a sufficient sample size, where “sufficiency”
considers both the dispersion (heterogeneity) of the sample and its size and follows the statistical principles
of section III above. Compiling agencies should act transparently and responsibly by monitoring sample sizes
and the dispersion in (the difference between matched) prices to inform users as to why indexes are
constrained to particular market segments, as necessary. The proposal is that CPPIs should be provided to
inform rather than mis-inform policy makers and statistical offices should be guided by statistical principles
in doing so.
Transaction data CPPIs are based on new samples of transactions each period, the current period sample
mean price is compared with the reference period sample mean price. This ratio, be it as an arithmetic or
geometric mean, is (i) tainted by changes in the quality-mix of the sample and (ii) has an excessively wide
confidence interval, as a ratio of means of two independent samples, as explained in Section III. It is
proposed that such ratios of means not be used, even when there is some (minimal) quality-mix adjustment,
such as price per square foot.
The proposed hedonic imputation methodology takes account of both concerns (i) and (ii) above.
16
The
hedonic approach is designed to take account of quality-mix changes. The specification of the hedonic
15
IMF and FSB (2018, Annex 2): Monitoring Traffic Light Dashboard: Status of Progress in 2018 in the
Implementation of the DGI-2.
16
Hill (2013, 906) concludes his survey paper: “Hedonic indexes seem to be gradually replacing repeat sales as the
method of choice for constructing quality-adjusted house price indexes. This trend can be attributed to the inherent
weaknesses of the repeat sales method (especially its deletion of single-sales data and potential lemons bias) and a
combination of the increasing availability of detailed data sets of house prices and characteristics, including geospatial

18
regression is expected to improve over time as more characteristic and spatial (locational) data becomes
available, as is likely in this digital age. There will be a natural synergy with the RPPI measurement team.
The CPPI team should be able to blend expertise, software, dissemination practice gained from the RPPI
practice. The imputation approach, as outlined above, transforms the calculation from the difference between
mean of the logarithms of independent samples, whose exponent is the ratio of geometric means—a Jevons
index—to a panel data set for each property transacted where the standard error is calculated as the standard
deviation of matched pairs, as outlined in Section III. This substantially reduces the standard error of the
estimated price index change.
17
The hedonic imputation method can be readily applied with weights at the level of the individual property
transaction and, further, such weighting system can take a superlative form as outlined in Hill and Melser
(2018), de Haan and Diewert (2013), and Silver (2018)—Section III.
The transaction-based CPPI with sparse data can further benefit from using an extended time period for the
reference period, as advocated by de Haan and Diewert (2013) and Silver (2018)—Section III.
The transaction-based CPPI with sparse data may also benefit from estimating the hedonic regression
periodically, say annually or every two years, and chaining the results. The regular estimation of a hedonic
regression every say quarter, using sparse data, leaves the index results open to bias from undue influence
and other vagaries of econometric estimation. Compilers further benefit from using an extended time period
for the reference period, as advocated by de Haan and Diewert (2013) and Silver (2018). Such indexes can
take a quasi-superlative form as explained in Silver (2018)—Section III.
It may be argued that the use of the hedonic imputation approach to create a matched panel data structure,
while an improvement on the use of comparisons of means from independent samples, only puts the
transaction-based data set on a par with the appraisal data set in the sense that both are now compiled from
matched data.The transaction-based CPPI now avoids the larger standard errors from comparing means from
independent samples. It may be argued that appraisal/assessment data will still have a larger sample size
being available for all properties appraised/taxed, rather than the hedonic paring of those transactions in the
(extended) reference period and current period. However, this larger appraisal/assessment sample size is an
illusion. Such prices are based on the more limited transaction data and used in a subjective and inconsistent
manner with documented bias to arrive at the appraised prices. Further, the quarterly estimates are based on
extrapolations from annual estimates; the basis of, and extent to which, these extrapolations are employed is
unknown to the user and can swamp measured (transaction-based) price change.
An important role of international organizations with regard to economic statistics is to meet the needs of
data initiatives such as the DGI, as outlined in Section V, by setting and promoting standards of
measurement. However, in this case, doing something, rather than nothing, may be to mislead, rather than
lead, and that would be a disservice to central banks and macroeconomists, especially when an alternative
transaction-based methodological approach is available for sparse data, albeit one that requires more effort
and care in its development.
data, increases in computing power, and the development of more sophisticated hedonic models that in particular take
account of spatial dependence in the data.” Alternative methods are the repeat sales method, mainly used in the United
States, and the sales price appraisal method (SPAR), outlined and surveyed in Eurostat et al., (2013). A survey and
evaluation of the impact of methods is in Silver (2015).
17
.

19
References
Altman, Douglas G and J Martin Bland (2003), Interaction revisited: the difference between two estimates,
British Medical Journal, Statistics Notes, vol. 326, page 219.
Awuah, Kwasi Gyau Baffour, Frank Gyamfi-Yeboah, David Proverbs, and Jessica Elizabeth Lamond, (2017)
Sources and reliability of property market information for property valuation practice in Ghana, Property
Management, vol. 35, issue 4, July.
Balk, Bert M. (2008) Price and Quantity Index Numbers (Cambridge: Cambridge University Press).
Baum, Andrew E. and Neil Crosby (2014) Property Investment Appraisal (Hoboken: Wiley and New York:
NY John Wiley and Sons).
Bokhari. S, and D. Geltner (2012), Estimating real estate price movements for high frequency tradable indexes in
a scarce data environment, Journal of Real Estate Finance and Economics 45, pp. 522-543.
Bowles, G., P. McAllister, and H. Tarbert (2001), An assessment of the impact of valuation error on property
investment performance measurement, Journal of Property &Finance 19: pp. 139-155.
Caleb, Ayedun, D.O. Durodola, Samuel Oloyede, Akinjare Omolade, and S.A. Oni (2018) An empirical
evaluation of the factors militating against valuation accuracy in Nigeria, International Journal of Civil
Engineering and Technology, vol.9, issue 8, pages 752-762.
Cannon, S., and R. Cole (2011) How accurate are commercial real estate appraisals? evidence from 25 years
of NCREIF sales data, Journal of Portfolio Management, Vol. 37, No. 5 (2011), pp. 68–88.
Changha, Jin and Paul Gallimore, (2010) The effects of information presentation on real estate
market perceptions, Journal of Property Research, vol. 27, issue 3, pp. 239–246.
Cole, Rebel, David Guilkey and Mike Miles. 1986. Toward an assessment of the reliability of
commercial appraisals. The Appraisal Journal, July, 442 – 432.
Clayton, J., D. M. Geltner and S. W. Hamilton (2001), Smoothing in commercial property valuations: evidence
from individual appraisals, Real Estate Economics 29, pp. 337-360.
Crosby, N. (2000) Valuation accuracy, variation and bias in the context of standards and expectations,
Journal of Property Investment and Finance, vol. 18, issue 2, pp. 130–161.
Crosby, N., C. Hughes and J. Murdoch (2004), Influences on Secured Lending Property Valuations in the UK,
Working Papers in Real Estate & Planning, University of Reading.14/04.
Crosby, N., C. Lizieri and P. McAllister (2010), Means, motive and opportunity? disentangling client
influence
on performance measurement appraisals, Journal of Property Research 27(2), pp. 181-201.
Diewert, W. Erwin (1976). “Exact and superlative index numbers,” Journal of Econometrics, 4, 2, 115–145.
Diewert, W. E., K. Fox and C. Shimizu (2016), Commercial property price indexes and the system of national
accounts, Journal of Economic Surveys 30(5), 913-943.
Diewert, W. E. and C. Shimizu (2015), A Conceptual Framework for Commercial Property Price Indexes,

20
Journal of Statistical Science and Application 3(9-10),131-152.
Diewert, W. E. and C. Shimizu (2017), Alternative approaches to commercial property price indexes for
Tokyo, Review of Income and Wealth, forthcoming.
Diewert, W. Erwin, Saeed Heravi, and Mick Silver (2009). Hedonic imputation indexes versus time dummy
hedonic indexes. In W. Erwin Diewert, John Greenlees, and Charles R. Hulten eds. Price Index Concepts
and Measurement, NBER, Chicago: University of Chicago Press, 278–337.
de Hann, Jan and W. Erwin Diewert (2013). Hedonic regression methods. In Eurostat et al. (2013) op. cit.,
chapter 5
Elonen, K. (2013), Tracking and trading commercial real estate through reit-based pure-play portfolios: the
european case. Master Thesis, Massachusetts Institution of Technology, http://dspace.mit.edu/
handle/1721.1/84176.
European Commission, Eurostat, 2017. Technical Manual on Owner-Occupied Housing and House Price
Indices, https://ec.europa.eu/eurostat/documents/7590317/0/Technical-Manual-OOH-HPI-2017/.
Eurostat, European Union, International Labor Organization, International Monetary Fund, Organisation for
Economic Co-operation and Development, United Nations Economic Commission for Europe, The World
Bank (2013), Handbook on Residential Property Prices Indexes (RPPIs), Luxembourg, European Union.
Eurostat (2017). Commercial Property Price Indicators: Sources,
Methods and Issues (Luxembourg: European
Union).
Gallimmore, P and M. Wolverton (1997), Price-Knowledge-indies bias: a cross cultural comparison, Journal
of Property Valuation and Investment 15, pp. 261-273.
Geltner, David (1993) Temporal aggregation in real estate return indexes, Journal of the American Real
Estate and Urban Economics Association 21, 2, 141–166.
Geltner, D.M. (1997), The use of appraisals in portfolio valuation and index, Journal of Real Estate Finance and
Economics 15, pp. 423-445.
Geltner, D.M. (1998) How accurate is the NCRIEF index, and who cares, Property Finance 14: pp. 25-37.
Geltner, D.M. (2015) real estate price indices and price dynamics: an overview from an investments
perspective, Annual Review of Finance and Economics, vol. 7, pages 615–633.
Geltner, David and Jeffrey Fisher (2007), Pricing and index considerations in commercial
real estate derivatives, The Journal of Portfolio Management, Vol. 33, No. 5: pp. 99–118.
Geltner, D.M. and W. Goetzmann (2000), Two decades of commercial property returns: a repeated- measures
regression-based version of the NCREIF index, Journal of Real Estate Finance and Economics 21(1), pp. 5-21.
Geltner, D.M., R. Graff and M. Young (1994) Random disaggregate error in commercial property: evidence
from the Russell-NCRIEF database, Journal of Property Research 19, pp. 403-419.
Geltner, D.M., B. MacGregor, and G. Schwann (2003), Appraisal smoothing & price discovery in real estate
markets, Urban Studies 40, pp. 1047-1064.
Geltner , D.M., N.G. Miller, J. Clayton and P. Ecihholtz (2014), Commercial Real Estate Analysis and
Investments,
Third Edition, OnCourse Learning, USA.

21
Goetzmann, W. N. 1992. The accuracy of real estate indexes: Repeat sale estimators. Journal of Real Estate
Finance and Economics, 5, 5–53.
Heath, Robert and Evrim Bese Goksu (2016) G-20 Data Gaps Initiative II: meeting the policy challenge, IMF
Working Paper WP/16/43
Hill, Robert J. (2013) Hedonic price indexes for residential housing: a survey, evaluation and taxonomy,
Journal of Economic Surveys, 27, 5, 879–914, December.
Hill, Robert J. and D. Melser (2008)Hedonic imputation and the price index problem: An application to
housing, Economic Inquiry, 46, 593–609.
Hill, Robert J., Michael Scholz, Chihiro Shimizu, and Miriam Steurer (2018) An evaluation of the methods
used by European countries to compute their official house price indexes. Economie et Statistique, 500-501-
502-12, pp. 221–238.
Horrigan, H., B. Case, D.M. Geltner and H. Pollakowski (2009), REIT-Based property return indices: a new way
to track and trade commercial real estate, Journal of Portfolio Managemen, 35(5), pp. 80-91.
International Monetary Fund and Financial Stability Board, The Financial Crisis and Information Gaps,
Sixth Progress Report on the Implementation of the G20 Data Gaps Initiative, September 2015
https://www.imf.org/external/np/g20/pdf/2015/6thprogressrep.pdf.
International Monetary Fund and Financial Stability Board, The Financial Crisis and Information Gaps
Second Phase of the G20 Data Gaps Initiative (DGI-2) Third Progress Report, September 2018.
https://www.imf.org/external/np/g20/pdf/2018/092518.pdf.
Kinnard, W.N, M.M. Lenk, and E.M. Worzala (1997), Client pressure in the commercial appraisal industry: how
prevalent is it, Journal of Property Valuation and Investment 15, pp. 233-244.
Leventis, Andrew (2008), Revisiting the differences between the ofheo and s&p/case-shiller house price
indexes: new explanations, office of federal housing enterprise oversight, January, Available at:
www.ofheo.gov/media/research/OFHEOSPCS12008.pdf
Lusht (2012), Real Estate Valuation. Principles and Applications, KML Publishing, USA.
McAllister, P., A. Baum, N. Crosby, P. Gallimore, and A. Gray (2003), Appraiser behaviour and appraisal
smoothing: some qualitative and quantitative evidence, Journal of Property Research 20, pp. 261-280.
Mehrhoff, J. (2017), ‘What is “commercial property”?’, IFC Bulletin, 46.
Nishimura, K.G and C. Shimizu. (2003), Distortion in land price information. mechanism in sales
comparables and appraisal value relation, Center of International Research on the Japanese Economy
Discussion Paper (University of Tokyo), No 195.
Plazzi, A., W. Torous and R. Valkanov (2010), Expected returns and expected growth in rents of commercial
real estate, Review of Financial Studies 23(9), pp. 3469-3519.
Quan D. and J. Quigley (1991), price formation and the appraisal function in real estate markets, Journal of Real
Estate Finance and Economics 4(2), pp. 127-146.
Quan, D. and Quigley, J. “Inferring an investment return series for real estate from observations on
sales.” AREUEA Journal 17 (Summer, 1989), 218–230.

22
RICS (2012) RICS Valuation-Professional Standards. Incorporating the International Valuation Standards
(London: Royal Institution of Chartered Surveyors).
Shimizu, C (2016), Microstructure of asset prices, property income, and discount rates in tokyo
residential market-, International Journal of Housing Markets and Analysis (forthcoming), IRES-
NUS(National University of Singapore) Working Paper 2016-002.
Shimizu, C., W. E. Diewert, K. G. Nishimura and T. Watanabe (2013), Estimating quality adjusted commercial
property price indexes using Japanese REIT, CARF Working Paper Series (University of Tokyo), CARF-F-
F307.
Shimizu, C. and K.G. Nishimura (2006), biases in appraisal land price information: the case of Japan,
Journal of Property Investment and Finance 26, pp. 150-175.
Shimizu, C., K.G. Nishimura and T. Watanabe (2012), biases in commercial appraisal-based property price
indexes in Tokyo. lessons from Japanese experience in bubble period, RIPESS (Reitaku Institute of Political
Economics and Social Studies) Working Paper, No 48 (presented at: International Conference on Commercial
Property Price Indicators on 10 May 2012, the European Central Bank in Frankfurt).
Silver, M. (2013) Understanding commercial property price indexes, World Economics 14(3), July– September.
Silver, Mick 2015. The degree and impact of differences in house price index measurement, Journal of
Economic and Social Measurement, 39, 305–328.
Silver, Mick 2018. How to measure hedonic residential property price indexes bettter, Eurostat Review on
National Accounts and Macroeconomic Statistics Indicators (EURONA), volume 1, pp. 35–66.
Silver, Mick and Brian Graf 2014. Commercial property price indexes: problems of sparse data, spatial
spillovers, and weighting, IMF Working Paper WP/14/72, Washington DC, April.
Triplett, Jack E. 2006. Handbook On Hedonic Indexes And Quality Adjustments In Price Indexes Special
Application To Information Technology Products: Special Application To Information Technology Products.
Paris: OECD Publishing.
United Nations (Eurostat) (1998) Classification of Types of Constructions (New York: United Nations,
Statistics Division). https://unstats.un.org/unsd/classifications/Family/Detail/1010.