8th May, 2014

Technische Universität Braunschweig

Question

Asked 5th Jan, 2014

Moving-Average Disadvantages.

Hi Bilal Esmael,

the weight function of your moving average filter should be symmetric. Otherwise the filtered values are shifted in phase: depending on the structure of the weight function the phase lag can reach half the length of the weight function.

For example: a one-sided Kalman Filter has got an asymmetric weight function.

Further on, be careful in interpreting the filtered values at the both ends of a time-series, they've got a structural phase lag always.

Best regards,

Michael Heinert

One problem is that the moving average time series will have temporal autocorrelation at a lag determined by the length of the moving window.

5 Recommendations

**Get help with your research**

Join ResearchGate to ask questions, get input, and advance your work.

There is a bit of a confusing in the terminology in signal processing. Moving average filters are filters calculating a series of weighted means of the input signal. In addition to Balázs Kotosz’ comment, it is important that the weights are not equal, i.e. you calculate the running arithmetic mean of the input signal. This type of filter is usually called running mean. You shouldn’t use those because they eliminate some frequencies in your spectrum and others are reversed. That’s bad if you are interested in a specific frequency band, which is either eliminated (no response) or reversed (change of sign and hence causality) (see Page 177 in my textbook MATLAB Recipes for Earth Sciences, Springer 2010).

3 Recommendations

Here's a MATLAB Example to see the effect of running means. As an example, applying the filter to a signal with a period of approximately 1/0.09082 completely eliminates that signal. Furthermore, since the magnitude of the frequency response is the absolute of the complex frequency response, the magnitude response is actually negative between ~0.09082 and ~0.1816, between ~0.2725 and ~0.3633, and between ~0.4546 and the Nyquist frequency. All signal components having frequencies within these intervals are mirrored on the t-axis. As an example, we try a sine wave with a period of 7.0000, e.g., a frequency of approximately 0.1429, which is within the first interval with a negative magnitude response:

t = (1:100)';

x10 = 2*sin(2*pi*t/7);

b10 = ones(1,11)/11;

m10 = length(b10);

y10 = filter(b10,1,x10);

y10 = y10(1+(m10-1)/2:end-(m10-1)/2,1);

y10(end+1:end+m10-1,1) = zeros(m10-1,1);

plot(t,x10,t,y10)

Here is the amplitude response of the filter showing the zeros and the clipping:

[h,w] = freqz(b10,1,512);

f = 1*w/(2*pi);

magnitude = abs(h);

plot(f,magnitude)

The sine wave with a period of 7 experiences an amplitude reduction of

1-interp1(f,magnitude,1/7)

e.g., around 80% but also changed sign as you can see from the plot. The elimination of certain frequencies and flipping of the signal has important consequence while interpreting causality in earth sciences. These filters, though they are offered as standard in spreadsheet programs for smoothing, should therefore be completely avoided. As an alternative, filters with a specific frequency response should be used, such as a Butterworth lowpass filter.

4 Recommendations

One problem is that the moving average time series will have temporal autocorrelation at a lag determined by the length of the moving window.

5 Recommendations

Actually, before using the method of moving averages, one should first be able to decide the length of the smallest cycle associated with the data. Unless that is known, moving averages would lead to bad statistical analysis.

2 Recommendations

A Gaussian kernel or B-spline kernel have much better properties than the average kernel (e.g. rotation invariance, differentiability, band limitation, etc). Efficient recursive implementations exist, similar to running average. For more information see [Bouma, LNCS 4485, 2007].

PDF available at: http://home.kpn.nl/henri.bouma/research

Dis advantages of Moving avg method:

1. Requires saving lots of past data points: at least the N periods used in the moving average computation

2. Lags behind a trend

3. Ignores complex relationships in data

1 Recommendation

Which MATLAB function: ttest or ttest2 I should use to compare two paired datasets??

Question

4 answers

- Asked 29th Nov, 2017

- Mateusz Soliński

Hi,

I would like to compare two sets of data using t-Student test in MATLAB environment.

The data are the values of parameter X calculated for n=88 pairs of signals where each pair contains the signals modified by two processing methods: Y1 and Y2.

There are very small difference of mean value of X parameter between these two processing methods (0.480(0.053) for Y1 and 0.478(0.053) for Y2, see figure below).

The question is: which MATLAB function I should use to obtain the correct p-value?

According to the documentation of ttest (https://www.mathworks.com/help/stats/ttest.html) and ttest2 (https://www.mathworks.com/help/stats/ttest2.html) I think I should use the first one, because it "returns a test decision for the null hypothesis that the data in x – y comes from a normal distribution with mean equal to zero and unknown variance, using the paired-sample *t*-test ", while the second function "returns a test decision for the null hypothesis that the data in vectors x and y comes from independent random samples from normal distributions ".

But using ttest function I obtain very small p-value (p<<0.05) in contrary to ttest2 (p=0.73). The second value are more reliable, but I cannot understand why I should use the test for independent samples where I compare the pairs of dependent signals.

Could you help me with that problem?

Preprint

Full-text available

- Apr 2019

The study of correlated time-series is ubiquitous in statistical analysis, and the matrix decomposition of the cross-correlations between time series is a universal tool to extract the principal patterns of behavior in a wide range of complex systems. Despite this fact, no general result is known for the statistics of eigenvectors of the cross-corr...

Article

- Apr 2020

The study of correlated time-series is ubiquitous in statistical analysis, and the matrix decomposition of the cross-correlations between time series is a universal tool to extract the principal patterns of behavior in a wide range of complex systems. Despite this fact, no general result is known for the statistics of eigenvectors of the cross-corr...

Chapter

- Jan 2002

Time series analysis is the statistical analysis of a series of observations made at successive points in time. For waterborne disease, time series analysis has been mainly used to study effects on human health (such as hospital visits because of diarrhea) of water quality (such as turbidity). Time series studies (TSS) published on the epidemiology...

Get high-quality answers from experts.