Conference PaperPDF Available

Quality Control Specifications for Large Earthworks Projects

  • Foundation Specialist Group


Content may be subject to copyright.
Burt Look
Sinclair Knight Merz Pty Ltd, 32 Cordelia Street, South Brisbane,
QLd 4101, Australia. E-mail:
Density testing has been applied widely in quality control, yet because of its
widespread usage, this now acts as an impediment to the development of the
industry. The compaction model has many underlying assumptions. These include
translating the words “optimum” and “maximum” to another English word “best”,
when the equilibrium condition is more important. The emphasis on density has
led to the belief that it is the key parameter, yet it is a second order assessment
parameter, i.e. we assume an increased density means an increased strength or
modulus. Although technology has now advanced to measure those parameters
directly, we still remain with density testing as the main quality evaluation parame-
ter because of our longstanding experience. Applying a quality control specification
involves understanding the underlying assumptions, as well as associated issues
on the depth of the compacted layers and the application of statistical schemes for
large earthwork projects.
Quality control, Earthworks, Compaction, CBR, Statistics.
The Australian Standard (AS 3798–2007) on Earthworks is specific to commercial and resi-
dential developments. Most road and rail authorities have their own earthworks standards
(eg Austroads, 2009). Design and construction concepts for earthworks construction can be
found in TRB (1990) and Ervin (1993) for earthworks specifications and control.
Proctor introduced the fundamentals of compaction in the 1930s through the standard
laboratory compaction test. This test showed the change in density when compacting at
various water contents with a maximum dry density (MDD) and corresponding optimum
moisture content (OMC). These compaction concepts are universally taught in courses in
soil mechanics, yet, this simple laboratory model has many underlying assumptions. These
The Optimum and Maximum refer to a peak on a graph and the corresponding
moisture content a very useful reference point, but not always the best condition
The laboratory model may be different to the field process or bulk material used.
Density of a particular layer is therefore dependent on the underlying layer
A laboratory compacted sample tested for its California Bearing Ratio (CBR) may
not represent the CBR in the field even at a given density and moisture content
Proceedings of the International Conference on Ground Improvement and Ground Control
Edited by
Buddhima Indraratna, Cholachat Rujikiatkamjorn and Jayan S. Vinod
Copyright © 2012 by Research Publishing Services. All rights reserved.
ISBN: 978-981-07-3560-9 :: doi:10.3850/978-981-07-3560-9 04-0413 1113
Proceedings of the International Conference on Ground Improvement and Ground Control
The CBR was developed by Porter in 1938 using Proctor’s Standard compaction curve.
The modified compaction test was introduced in the late 1940s by the U.S. Army Corps of
Engineers to represent the compaction required for aircraft loading. This increased effort
increased the MDD, while decreasing the OMC. The modified CBR values then also
Because the modified test came later and was associated with heavier equipment, there is
an assumption that it is better. However, the 1940s ”heavy” equipment does not
represent that of the 1960s or 1980s or 2000s. No one is suggesting we should be using
the modified rev 4 compaction tests. The modified test while relevant to pavements and
granular material is a dis-benefit for use on clay subgrades, where a modified soaked CBR
test can produce CBRs over 10%. Such lab values would be unrepresentative of what can
be achieved in the field.
While compaction testing is the key tool in most earthworks projects, same judgment is
required in assessment of the ”quality” of the earthworks. Its use (along with the CBR test)
represents our comfort with its historical association rather than its relevance to the 21st
century. This forms the central theme of this paper. The primary objective of compaction is
to attain a certain minimum strength, compressibility, permeability or volume change. The
density per se is of little interest as a key design parameter. A density however suggests
one of the other characteristics have been met e.g. higher density means higher strength or
Laboratory Compaction Model Compared with Field Characteristics
The MDD and OMC provide a good reference point by comparing an ”ideal” with the
compacted density in the field. The following are some compaction model assumptions.
The laboratory compaction test is an impact process, where a hammer is dropped several
times on a soil sample in a mold. In the field compaction occurs by static load, vibration
and kneading action. Impact (non round) rollers are used only on specific cases. Thus the
laboratory and field process is different.
Discarding a high percentage of oversize invalidates the test. This particle size effect and
importance is conveyed in the testing standards and AS3798 by stating the compaction
applies for soils less than 20% passing the 37.5 mm sieve after field compaction.
The test is confined within a steel mould and plate. This means that it is not appli-
cable where there is not a significant confinement e.g. at edge of embankments, adja-
cent to trenches. The steel plate means there is an assumption of rigidity at the base.
A minimum reaction modulus is implied for the layer to achieve its density. This concept
becomes important for CBR testing. Hammitt (1970) showed the CBR (base course) to CBR
(subgrade) ratio needs to be 5.2 on average (but this value ranged from 1.7 to 17) for the
laboratory CBR to be achieved in the field. Without this rigid base, material is wasted by
being pushed into the underlying layer.
Optimum Moisture Content vs. Equilibrium Moisture Content
The current emphasis is compacting to the OMC and MDD. These are very useful refer-
ence points, but it is the equilibrium density and equilibrium moisture content (EMC) that
Quality Control Specifications for Large Earthworks Projects
is more important (Look et al., 1992; Look et al., 1994; and Look, 2005). The OMC may
be a construction expedient, but does not always represent a long term value. For highly
expansive soils with a high or low rainfall environment, soils compacted at optimum is
likely to swell or shrink within 2 to 5 years, with an accompanying loss of strength and
volume change leading to cracking and uneven riding surfaces. Thus the design CBR must
be representative of the EMC and equilibrium density.
Burman et al. (2008) highlighted experiences with post-construction retesting of engi-
neered clay fills where later testing at several sites did not produce the as constructed test
values, even when the original work had tests results which provided the specified level
of compaction.
Compaction control occurs by either method specifications or end product specifications.
The latter is the more common approach in Australia, and relies on the relative compaction
(RC) achieved. This is the ratio of the field density to the laboratory MDD.
Tynan and Morris (1970) show that for a 10 ton roller, 8 passes @ OMC (Standard) can
achieve 100% MDD. However the MDD can still be achieved at other moisture contents but
at a higher No. of passes. A specified density can be achieved at various moisture contents
and passes for given equipment. The moisture content (OMC) is simply an indicator at the
likely value that can make this happen efficiently. But this varies between materials and
equipment, where the field process is different from the laboratory compaction process.
Density is not directly used in design. The design of a building pad for example is based
on the strength value. Density is used as the control parameter because it is an indica-
tor an index parameter that is a relatively low cost test that can be easily applied in
practice. The use of direct strength or stiffness control tests, is typically more expensive,
and is less familiar to inspectors and construction engineers. Such testing also introduces
the complication that these properties are dependent on both moisture content and den-
sity. The understanding is that while the OMC is used as control parameter, it allows the
MDD to be efficiently obtained. However the moisture content does change with time.
Look (2005) shows that change is significant for materials with a weighted plasticity index
(WPI) >3200, but with little change for WPI <1200.
Strength can be measured directly with Plate Loading testing and indirectly with pen-
etrometers, falling weight deflectometers, etc. Intelligent compaction which assesses the
modulus values directly is discussed in Briaud (2003). These in situ strength and modu-
lus measurements are more relevant to the designer than the various density compaction
controls. In a road pavement for example, the resilient modulus value is used in design.
This may be established from correlations with the CBR values (one step removed). Yet in
general, only density (two steps removed) is used as a control parameter.
The CBR is extensively used as a design parameter. Typically in design the lab CBR is
based on a 98% target density at OMC. Yet this target many not be achieved as lab results
can vary from 95% to 102% MDD, with a corresponding CBR of 8% or 18%, respectively.
Yet the specifications for constructions may have 90% or 95% MDD target compaction. This
shows there is a mismatch between specifications and design values used. Compacting dry
of OMC can significantly increase the CBR - which can provide good results for proof roll
testing, but may not be representative of the equilibrium condition.
Proceedings of the International Conference on Ground Improvement and Ground Control
Current specifications typically use a 300 mm limitation on lift thickness, while also asking
for a compaction requirement eg 95% MDD. In effect 2 separate criteria are being applied.
The compaction lift limitation provides confidence that a minimum allowable relative com-
paction level has been achieved, and this corresponds to the stress envelope for light to
medium compaction equipment. Larger equipment has a larger zone of influence, and can
therefore compact to a deeper level and still achieve similar compaction results. However
if a light equipment is used in clay, the 250 mm is probably is too deep a lift and 100 mm
thickness is required to achieve the required 95% RC (say). Therefore specifying a lift thick-
ness is not optimising the equipment on site, and can be too little or too much.
Lift Thickness
A 200 mm to 300 mm limit lift thickness is a common compaction practice for road authori-
ties across Australia and internationally - but that thickness is representative of equipment
50 years ago. The testing limitation of 200 mm to 300 mm of the sand cone and nuclear
density testing equipment also means our QC requires we are limited to this thickness for
The 300 mm maximum loose lift limit thickness has a direct impact on contractor time
and cost for embankment construction. Modern day earthmoving equipment has become
larger and heavier, and with specialized high-energy compaction equipment developed
by a number of equipment manufacturers. Such equipment is often promoted as being
capable of compacting soil layers of over 2 metres in thickness (since the 1980s).
We (as an industry) should recognize the need to review policies and requirements for lift
thickness based on current construction practices and equipment. However, any changes to
current requirements must be based on sound theoretical analyses and actual field monitor-
ing. Projects should rationalize the lift thickness requirement which is currently an imped-
iment to the potential production output possible from modern compaction equipment.
The current lift thickness is based on the production rates of the 1960s when 100 m3/hr
(HRB Bulletin, 1960) was an upper bound. Current maximum lift thickness; represent less
than 20% of the production cycle. However, with lift thickness possible above 1.0 m, and
with the current testing procedures maintained, the testing and reinstatement times could
be over 80% of the production time. The current potential of modern day equipments is
therefore not being realised.
The times associated with placing, testing and reinstating is compared in Figure 1. This
shows that as lift thickness increases, then the testing requirement now represents a
significant part of the cycle. At the current production rates (lifts not exceeding 300 mm),
the testing represents 15% of the total time. This increases to 42% for the 0.5 m lift thickness.
This suggests that for compaction above 1.0 m lift thickness and using the conven-
tional testing approach that the testing/reinstatement phase is likely to exceed 80% of the
total time required with less than 20% of total time devoted to the placing with the com-
paction equipments. Conventional testing procedures therefore provide delays where high
Quality Control Specifications for Large Earthworks Projects
Figure 1. Typical time (hrs) of various activities for 4,000 m3with 8 No. sand replacement tests.
Table 1. Comparison of the CBR values for the design sections.
Chainage n Value Distribution Normal Log Normal
51440–64440 95 10 Percentile CBR 1.7% 2.8%
20 Percentile CBR 2.9% 4.0%
Average CBR (50 percentile) 11.5% 8.2%
51440–54940 31 10 Percentile CBR 0.7% 3.0%
20 Percentile CBR 3.4% 3.8%
Average CBR (50 percentile) 8.6% 6.6%
54940–56440 6 10 Percentile CBR 11% 15%
20 Percentile CBR 17% 18%
Average CBR (50 percentile) 30% 25%
56440–59640 37 10 Percentile CBR 2.7% 2.2%
20 Percentile CBR 1.6% 3.2%
Average CBR (50 percentile) 9.9% 6.8%
59640–64440 21 10 Percentile CBR 1.3% 4.0%
20 Percentile CBR 5.5% 5.7%
Average CBR (50 percentile) 14% 11%
production rates are possible. Other techniques need to be considered for such large earth-
works projects and /or the use of trials.
AS3798 (2007) describes the characteristic values applicable for statistical acceptance
schemes. While not stated it infers Lower characteristic value (LCV) and a normal dis-
tribution. The LCV is derived from the Mean and Standard Deviation and a “constant”
(k) which depends on the proportion defective, the number of values and the contrac-
tor’s risk. Many standards suggest a value of k = 1.3, but this assumes 10% defective
(90% confidence) is adequate. For minor and local roads this is too tight a specification.
Another assumption is for a Normal distribution statistical model. This may be a reason-
able assumption in density testing, but not for other design values such as CBR or strength
parameters, as shown in the following case study.
Proceedings of the International Conference on Ground Improvement and Ground Control
Look (2009) discussed a case study over a 13.2 km alignment. The normal and log nor-
mal distribution functions are compared in Table 1 to illustrate the pitfalls in using the
normal distribution. Negative values would result for 10% defective for a normal distribu-
tion, whether for the whole road length or for design sections. Yet for major highways and
structural fills any statistical scheme would typically be targeting 10% values. A better fit
distribution function such as log-normal distribution function would not produce that neg-
ative value possibility. Thus many practitioners then abandon the statistical model because
of the negative value and rely on their judgment in arriving at a characteristic value.
The laboratory compaction model, while an excellent reference point may not represent the
material placement target. There is a mis-match between the compaction - testing process
and the design intent.
There is a need to review policies and requirements for lift thickness based on cur-
rent construction practices and equipment. However, any changes to current requirements
must be based on sound theoretical analyses and actual field monitoring. The current con-
struction equipment specifications and quality control specifications are mis matched
especially for large earthworks projects.
The use of a statistical prediction model for CBR design value requires a non normal
distribution function be used, especially at risk levels where the percentage defective is
required to be less than 20%. Assuming a normal distribution produces dubious results as
evident for the 10% defective risk assessment.
1. Australian Standards AS3798 (2007). Guidelines on earthworks for commercial and residential
development. Standards Australia AS3798.
2. Austroads (2009). Guide to pavement technology: part 4I - Earthwork materials. Austroads
publication No. AGPT04I/09.
3. Bomag (2001). Soil and Asphalt Compaction. Technical Brochure.
4. Briaud, J L (2003). Intelligent compaction: Overview and research needs. Texas A &M University.
5. Burman B.C, Mostyn G.R. and Piccolo D. (2008), Experiences with post-construction retesting of
engineered clay fills, Australian Geomechanics Journal, Volume 43, No 4, pp 1–29.
6. Ervin M.C. (1993). Specification and control of earthworks. Proceedings of the Conference
Engineered Fills held at Newcastle upon Tyne. Edited by BG Clarke, CFP Jones and AIB Moffat,
Thomas Telford Publications.
7. Hammitt, G.M. (1970). Thickness requirement for unsurfaced roads and airfields, bare base
support. Report S –705, U.S. Army Engineering Waterways, Experiment Station, Vicksburg.
8. Johnson A.W. and Salberg J.R. (1960). Factors that influence field compaction of soils: compaction
effects of field equipment. Highway Research Board 272
9. Look B G, Reeves I N and Williams D J (1994). Development of a specification for expansive clay
road embankments. 17th Australian Road Research Board Conference, August, Part 2, pp 249–264.
10. Look B G (2005). Equilibrium Moisture Content of volumetrically active clay earthworks in
Queensland. Australian Geomechanics Journal, Vol 40, No. 3, pp 55–66.
11. Look B G (2009). Spatial and statistical distribution models using the CBR tests. Australian
Geomechanics Journal, Vol 44, pp 37–48.
12. Transportation Research Board (1990). Guide to Earthwork Construction. State of the Art Report
No. 8, National Research Council.
13. Tynan A.E. and Morris P.O. (1970). The performance of three heavy class rollers in compacting
a medium clay. Proceedings of the 5th Australian Research Board Conference, Canberra, Volume 5,
Part 5, pp. 147–164.
... To measure strength or modulus and then correlate back to an index test shows how tradition encourages this force fit from a primary to the 2 nd order index parameter. Look (2012Look ( , 2016 discuss some of the underlying assumptions in the traditional QA compaction model. Density is only an indicator of strength or modulus. ...
... A lognormal is usually more appropriate for geotechnical data once the coefficient of variation exceeds 25% based on goodness of fit tests (Look, 2009;2012;2015). ...
Full-text available
Density testing has been applied widely in earthworks quality assurance (QA), yet because of its widespread usage, this now acts as an impediment to the development of alternative methods of testing. Many incorrect inferences are made from density testing. Modern geotechnical and pavement designs are based on modulus and strength values. However, when such measurements are correlated back to density testing, a poor correlation often results. Therefore, while alternative in situ testing to density provides significant benefits to the industry, the poor correlation is often questioned, and then site personnel default to the usual density testing for quality testing of earthworks. The reason for the poor correlations includes, the depth of influence being different, with the quality and compaction being combined into one parameter (say modulus). Another rationale for the poor correlation, is the density lot measurements are normally distributed due to its low coefficient of variation, while other measurements are not normally distributed and have a large variation. A new method on matching Probability density functions (PDFs) for quality assurance has been successfully used on a large earthworks project to overcome the correlation inconsistency and is introduced. Data from several test sites using a range of alternative testing equipment are compared. One must also distinguish between a test accuracy and its precision. Traditional testing density has focused on its key benefit of precision without appreciating the poor accuracy associated with this measurement. Case studies are used to illustrate this dichotomy between traditional and non-traditional testing for QA assessment of earthworks.
... • Roadway Performance, Look (1995a); • Specifications, ; Look et al. (1994); Look (1995b) • Construction, Look (2012) • Weighted Plasticity as a screening tool, Look (2016) • Equilibrium Moisture Content, Look (2005). ...
... The compaction curves establish a very useful reference point, but not always the best condition for field compaction. Many aspects of the laboratory model are different to the field process (refer Look, 2012). ...
Full-text available
Technical specifications provide guidance on quality levels and documents the procedures to be followed to achieve an acceptable standard. Road specifications sets out the precise requirements of the road authority and varies between Australian States and Territories, and internationally. A road construction specification cannot be overly prescriptive as it must integrate standard procedures, with varying climatic and material conditions as well as the significant testing variability. Elements used in developing road specifications using expansive clay material is presented. A key quality control parameter is density. Soil compaction control requires a lower characteristic value to obtain the required density (and associated implied increase in strength and modulus with increased density). However, increased density can result in higher suction and swell for highly expansive clay soils. An upper characteristic value needs to be considered for such materials. The Optimum Moisture Content (OMC) is associated with the Maximum Dry Density (MDD). For expansive clays, the Equilibrium Moisture Content (EMC) is more important than an OMC placement target. Thus, the standard compaction approach which applies to non-expansive material cannot be directly used to expansive soils because such soils are dependent on climate, which influence its soil suction and movement potential. Movement rather than strength often governs for expansive soils. Zonal strategies are appropriate for embankment constructed of expansive clays. Additionally, residual soils are very common in Australia, with a high granular content in "clayey" soils. The commonly used Plasticity Index (PI) used as an initial indicator of likely expansive behaviour, is not representative of the whole sample, with a significant "error" in this most basic of classification tests when used as a screening measurement to identify expansive clays in Australia. This screening is the first step in establishing appropriate design and construction procedures. The weighted plasticity index (WPI) accounts for the portion used in the PI test and this is more relevant for classification of residual soils. A simple ± 2% of Optimum Moisture Content (OMC) which may be relevant to nonexpansive materials cannot be applied to expansive clays. Additional testing variabilities also apply, which influences the interpretation of the reference maximum dry density (MDD) and even the CBR values used.
Carbide lime is a by-product obtained during the manufacturing of acetylene from the reaction of calcium carbide and water. A major portion of carbide lime is dumped in waste deposition areas, creating an environmental problem. Carbide lime and fly ash have possible applications in slope stabilization, subgrade improvement of roads, and soil treatments under shallow foundations. A series of Atterberg limits tests, compaction tests, unconfined compressive strength tests, ultrasonic pulse velocity tests, and wetting–drying tests were performed on carbide lime and fly ash treated clay soils to evaluate the effects of additive content, curing time, strength development, and the effects of wetting and drying. A total of 8% of carbide lime constituted the fixation point, and peak strength was achieved at 12% carbide lime content. A total amount of 25% additive was found as a threshold changing the Atterberg limits. Test results indicated that the strength of the treated soil improved by the existence of carbide lime and fly ash; best performance was observed in 28-day specimens with 10% carbide lime and 20% fly ash content reaching to 8 times larger strength than untreated soil. The failure patterns of the specimens reflected the curing time and wetting–drying effects. Although, the application of wetting–drying cycles deteriorated the treated soil, the presence of carbide lime partially prevented the strength loss. New relationships between normalized strength and curing time depending on carbide lime content were proposed. Furthermore, a linear relationship between the unconfined compressive strength and the ultrasonic pulse velocity of the treated soils was established.
Conference Paper
Full-text available
Volume changes in reactive clay roadway embankments cause deformation and cracking. As a result, the ride quality of the road surface deteriorates. Such a decrease in the structural integrity of roadway embankments constructed using reactive clays, results in significantly increased maintenance costs over the life of the facility. The principles of reactive clay behaviour have been known for some time. However, applying these principles to road construction requires a variation from traditional specifications , as well as a trade-off between the practicalities of construction and the application of theoretical geotechnical principles. Developing specifications for the construction of reactive clay roadway embankments, required a statistical two-tailed compliance scheme for density. This meets the requirements of strength and subgrade stability using the lower characteristic value while using an upper characteristic value to minimise the swelling behaviour of the reactive clays. ACKNOWLEDGEMENTS: This paper is published with the permission of the Director General, Queensland Transport.
Full-text available
The California Bearing Ratio (CBR) is the most common test used in the design of pavements. This simple test has many pitfalls in its application. This is an empirical test, and one must be aware of its many considerations if the design value is to be appropriately applied. A design value must consider its spatial variation, level of compaction and its relationship with its surrounding layers. The design risk is used to determine the characteristic value for a given project. Characterisation using the spatial and statistical variation of a CBR is used for a project site in Queensland to illustrate the requirement to use an appropriate prediction model. The results of this curve fitting show the normal distribution is inappropriate due to negative values and the lognormal distribution is an appropriate statistical model to characterize the design CBR.
Full-text available
A considerable amount of disputation, both legal and informal, arises where compacted clayey fills are retested some time after completion of the works and is predicated on two assumptions. Firstly, that once compacted, clayey fills remain unchanged thereafter and secondly that the results of post-construction retesting are more credible than the results of control testing carried out at the time of construction. The authors have been exposed to a number of cases, including legal proceedings, where earthworks having apparently been properly carried out and reputably tested during construction were retested some time after completion and assessed to be below specification. The simple conclusion often drawn by owners and their experts in such instances is that the earthworks were inadequately carried out at the time of construction. However, in the authors' view there has developed a body of factual evidence which does not support that simple conclusion. This evidence has arisen from a variety of sources involving actual construction works where multiple testing and/or retesting by a range of reputable authorities often under conditions that were less than ideal. Nevertheless they have provided a series of experiences or case histories to which geotechnical engineers, earthworks contractors, lawyers and owners should have regard and from which valuable insights and lessons may be drawn. This paper deals with the factual aspects of seven cases which, in the authors' view, challenge the simple conclusion based on the assumptions of essentially inert compacted clayey fills and the primacy of retests over tests during placement
Full-text available
The water changes of volumetrically active clays result in movement of the overlying pavements and in a change in the subgrade strength. This adverse effect results in damage to roads and buildings, with over one-third of Queensland covered with such clays. This paper discusses the equilibrium moisture content (EMC) operating range in southeast Queensland, and the philosophy behind the procedures for assessment and design on reactive clay earthworks. Two important considerations for wet environments with highly reactive clays are 1) the EMC is wet of the Optimum Moisture Content (OMC), and 2) the long-term density is below the Maximum Dry Density (MDD). If this placement condition is not targeted, then movements can be expected in the early years. This may result in damage to overlying structures irrespective of the design subgrade strength adopted. Targeting the OMC and MDD in such cases is building in future long term movement. This EMC condition must be considered together with construction issues.
A research program conducted to develop methods of determining thickness requirements for landing-mat- surfaced, membrane- surfaced, and unsurfaced airfields pertains to developing a method for determining design thickness requirements for unsurfaced airfields. Tests were conducted on 43 unsurfaced test items which had varying thicknesses and were trafficked under different loading conditions. California bearing ratio, water content, density, deflection, and deformation data were recorded. The design expression relates thickness requirements to soil response in terms of applications of load, load magnitude and pressure, and strength of soil.
Guidelines on earthworks for commercial and residential development
Australian Standards AS3798 (2007). Guidelines on earthworks for commercial and residential development. Standards Australia AS3798.
Guide to pavement technology: part 4I-Earthwork materials. Austroads publication No
Austroads (2009). Guide to pavement technology: part 4I-Earthwork materials. Austroads publication No. AGPT04I/09.
Soil and Asphalt Compaction
  • Bomag
Bomag (2001). Soil and Asphalt Compaction. Technical Brochure.
Intelligent compaction: Overview and research needs
  • J Briaud
Briaud, J L (2003). Intelligent compaction: Overview and research needs. Texas A &M University.
Specification and control of earthworks
  • M C Ervin
Ervin M.C. (1993). Specification and control of earthworks. Proceedings of the Conference Engineered Fills held at Newcastle upon Tyne. Edited by BG Clarke, CFP Jones and AIB Moffat, Thomas Telford Publications.