Access to this full-text is provided by IOP Publishing.
Content available from EPL (Europhysics Letters)
This content is subject to copyright. Terms and conditions apply.
August 2015
EPL, 111 (2015) 30002 www.epljournal.org
doi: 10.1209/0295-5075/111/30002
Comment
Comment on “Measurements of Newton’s gravitational constant
and the length of day” by Anderson J. D. et al.
M. Pitkin
SUPA, School of Physics & Astronomy, University of Glasgow - Glasgow, G12 8QQ, UK
received 29 May 2015; accepted in final form 30 July 2015
published online 18 August 2015
PAC S 04.80.-y – Experimental studies of gravity
PAC S 06.30.Gv – Velocity, acceleration, and rotation
PAC S 96.60.Q- – Solar activity
open access Copyright c
EPLA, 2015
Published by the EPLA under the terms of the Creative Commons Attribution 3.0 License (CC BY).
Further distribution of this work must maintain attribution to the author(s) and the published article’s
title, journal citation, and DOI.
Introduction. – In [1] the authors claim to observe a
periodic signal in measurements of Newton’s gravitational
constant, G. Specifically they find a 5.9-year-period signal
that is strongly correlated with variations in the observed
length of day [2]. They do not suggest that Gactually
varies on these time scales, but rather that there could be
some systematic effect on the measurement process that is
correlated with the mechanism that leads to the variation
in the length of day. Here I present a reanalysis of the
data used in [1] using Bayesian model selection to test
the hypothesis that the data contains a periodic signal
compared to other potential models. In light of updated
information on the times of the various Gmeasurements
given in [3] I also reanalyse this new dataset with the same
method. In both datasets I have found that a model for the
variations in Gthat only contains an additional Gaussian
noise term is hugely favoured, by factors of e30,over
models containing a sinusoid term1.
Analysis method. – Bayesian model selection pro-
vides a natural way to test multiple hypotheses by forming
the odds ratio of evidences for the different hypothe-
ses. The odds ratio for two hypotheses Hiand Hjis
given by
Oij =(p(d|Hi,I)/p(d|Hi,I)) ×(p(Hi|I))/(p(Hj|I)) ,(1)
where p(d|Hi,I) is the evidence for hypothesis Higiven
some data d,p(Hi|I) is the prior probability for Hi,andI
1The code, data tables, figures and prior ranges for this analysis
can be found at https://github.com/mattpitkin/periodicG.
is information concerning any other assumptions. When
comparing hypotheses I assume that they are equally
probable a priori, so the prior ratio is unity. Therefore,
I just calculate the ratio of evidences for each hypothesis.
If a given hypothesis is defined by a set of parameters, θi,
with their own priors, p(θi|Hi,I), then to calculate the
evidence the parameters must be marginalised over, e.g.
p(d|Hi,I)=θi
p(d|θi,H
i,I)p(θi|Hi,I)dθi,(2)
where p(d|θi,H
i,I) is the likelihood function of the data
given a set of model parameters θi.
The general model that I use for my hypotheses is
m(μG,A,P,φ
0,T
k)=Asin (φ0+2π(Tk−t0)/P )+μG,
(3)
where μGis an offset value, Ais the sinusoid amplitude,
φ0is an initial phase at an epoch t0,Pis the sinusoid
period, and Tkis the time.
In this analysis I have compared four different hypothe-
ses, Hi, to explain the measurements of G:H1) the data
is consistent with Gaussian errors, given by the experi-
mental error bars σe,k, about an unknown μG;H2)as
for H1, but also including an unknown common Gaussian
noise term σsys;H3)asforH1, but also including a si-
nusoid with unknown A,φ0and P; and, H4)asforH3,
but also including an unknown σsys. Theseeachcorre-
spond to a different set of parameters required in θand
also the number of parameters required in the integral
of eq. (2).
30002-p1
Comment
Table 1: Log odds ratios for the four hypotheses (irepresents
rows and jrepresents columns) when using the data used in [1],
with those when using the data from [3] in parentheses.
ln Oij H2H3H4
H1−133 (−140) −102.2(−66) −103 (−110)
H230 (74) 30 (30)
H3−0.3(−45)
For an initial examination of the claim in [1] I have used
their fig. 1 to read-off the experimental times and then
used table XVII of [4] for values of G(see footnote 2).
In [1] the experiment times are given no associated
error. However, many of the times used correspond to the
received date of the respective paper rather than the date
of the actual experiment. In analysing this data I specified
uncertainties on the experiment times of σt=0.25 years
(with the exception of the JILA-10 and LENS-14 measure-
ments for which I use uncertainties of one week) before the
given time. I have taken this time uncertainty into account
by marginalising over it for each data point.
Results. – The odds ratios comparing hypotheses
when using the Gdataset of [1] are summarised in table 1.
It is clear that hypotheses including extra parameters over
that for H1are hugely favoured by factors of e100.The
two hypotheses, H3and H4, containing a sinusoidal signal
are both approximately equally probable. However, H2,
just containing the additional unknown noise term σsys
and the unknown offset μG, is hugely favoured by factors
∼e30 over H3and H4. This shows that the simple model
for which variations are just due to an unknown Gaus-
sian noise term is far more likely to be the cause of the
variations than an additional sinusoidal variation. This
is due to Bayesian model selection naturally applying a
penalty for including additional parameters that do not
significantly increase the evidence.
I have also looked at the posterior probability distribu-
tions for the period and for H3I see a clear lone spike in
probability around the claimed period of 5.9 years. A sim-
ilar spike shows up for H4, but is much less pronounced.
I have assessed the significance of this period probability
peak for H3by rerunning that analysis 20 times, but each
time randomly shuffling the Gvalues to remove any real
periodicity in the data. Out of these 20 runs there is one
time when the hypothesis using the shuffled data is more
favoured than when using the unshuffled data and another
couple that are within a factor of two. The posteriors for
these cases also show very similar spikes in the period to
that from the unshuffled data.
Since the acceptance of [1] Schlamminger et al. [3] ex-
amined the claim, in particular noting that the experi-
mental times in the original work are not accurate. They
examined the literature to compile a more complete list
2For the BIPM-13 measurements I used the combined servo and
Cavendish value from [5] and for the LENS-14 measurements I used
the values from [6].
of experiments with information on the actual dates on
which the experiments were performed. I have reanalysed
this new dataset for each of the four hypotheses. When
marginalising over the time error I have now set the error
to be symmetric around the mean experiment times. For
all other parameters I have used the same prior ranges as
in the initial analysis. The odds ratios for each of these
cases are also given in table 1 from which it can be seen
that H2is still favoured over all other hypotheses by a huge
amount. However, H3is now hugely disfavoured over H4,
i.e. just including a sinusoid, but adding no additional
noise term does far worse at fitting the data than also
including the noise term.
Conclusions. – I have reanalysed the data consisting
of measurements of Gfrom [1] and [3] to asses the claim
of a periodic component with a period of 5.9 years.
Using Bayesian model selection, and four different hy-
potheses to describe the variations in the data, and in-
cluding uncertainties on the experimental times, I have
found that the best model is one in which there is an
additional unknown Gaussian noise term on top of the ob-
served experimental errors. This is favoured over a model
also containing a sinusoidal term by factors of e30. I also
find that periodic signals can easily be found in random
permutations of the data suggesting that the observed
periodicity seen in [1] is just a random artifact of the
data.
Following the publication of [1] the authors have taken
into account the work of [3] (see [7]). They fit an ad-
ditional sinusoid to the updated data and note that the
significance of the correlation with the length of day de-
creases. I expect that calculating the evidence for a model
including two sinusoids would not cause such a model to be
favoured over the simpler model containing just the extra
Gaussian noise term, as the increase in parameter space
will be penalised if the fit does not significantly improve.
I note that if there were good a priori reasons to expect
a periodic component with a specific period in the data
(i.e. if there were a good reason why the mechanism lead-
ing to changes in the length of day could couple into mea-
surements of G), then the evidence for models containing
such a periodic signal might dramatically increase. How-
ever, without such prior knowledge using such a constraint
would strongly bias us.
∗∗∗
I would like to thank Prof. J. Faller for useful discus-
sions, Prof. C. Speake for putting me in contact with
Dr. S. Schlamminger, and Dr. S. Schlamminger
for providing me with a data file of their compiled G
measurements. I am funded by the STFC under grant
ST/L000946/1.
Additional remark (16th July 2015): In responding to
my Comment the authors of the original article put for-
ward two related pieces of evidence to suggest that models
30002-p2
Comment
for the time variations of Gmeasurements containing a
constant offset and one or two periodic components are
favoured over a model with no periodic component. They
show that once the best fits for the models containing one
or two periodic components are removed from the data the
residuals have smaller variance, and have a distribution
that is closer to a Gaussian distribution, than for the
model containing no periodic component. These findings
are not at all surprising. If one adds more parameters to
the model then one can almost always find a better fit with
smaller residuals —in cases where the additional model
complexity adds no extra information this is commonly
known as over-fitting and is related to the concept of Oc-
cam’s razor. A Bayesian analysis, such as I performed,
naturally allows one to penalise extra complexity when
it is unnecessary through the incorporation of an Occam
factor. This Occam factor penalty comes about through
use of the prior volume of the parameter space for each
model. Each parameter has a range over which it is a pri-
ori thought to exist, and in my analysis the prior volume
is just the product of these ranges. So, a model with more
parameters will often have a larger prior volume. This nat-
urally incorporates a penalty for over-fitting in that larger
prior volumes will down-weight the evidence for a model,
so if the likelihood does not compensate enough for this
down-weighting then the evidence will be reduced.
In my analysis this penalty far outweighs the slightly
better fit that can be achieved with a more complex model
containing a sinusoid. It should also be noted that my
most favoured model does not just contain a constant off-
set, but also contains an additional unknown noise term.
This may suggest that the quoted errors on the Gmea-
surements are underestimates of the true noise.
A further addition that I was able to include in my
analysis, but which the authors of the original article do
not appear to have addressed is that there are also error
bars on the experimental times. This may also weaken
their fit.
Further Gmeasurement data may change my conclu-
sions, but with the current data I stand by my result that
the data is best explained by just a constant offset and an
additional unknown noise term, and no periodic compo-
nent is required.
REFERENCES
[1] Anderson J. D., Schubert G., Trimble V. and
Feldman M. R.,EPL,110 (2015) 10002.
[2] Holme R. and de Viron O.,Nature,499 (2013) 202.
[3] Schlamminger S., Gundlach J. H. and Newman
R. D.,Phys. Rev. D,91 (2015) 121101.
[4] Mohr P. J., Taylor B. N. and Newell D. B.,Rev.
Mod. Phys.,84 (2012) 1527.
[5] Quinn T., Speake C., Parks H. and Davis R.,Phys.
Rev. Lett.,113 (2014) 039901.
[6] Rosi G., Sorrentino F., Cacciapuoti L., Prevedelli
M. and Tino G. M.,Nature,510 (2014) 518.
[7] Anderson J. D., Schubert G., Trimble V. and
Feldman M. R., arXiv:1504.06604v2 (2015).
30002-p3
Available via license: CC BY 3.0
Content may be subject to copyright.