Measuring System Analysis in Six Sigma methodology application – Case Study

Abstract

As the manufacturing industry moves toward 6-sigma capable processes, the
same requirement is becoming necessary for measuring system, as well. In the scope of
Six Sigma methodology application for the existing manufacturing system, according to
DMAIC cycle (Define-Measure-Analyse-Improve-Control), this paper presents case
study – statistical analysis of the measuring system, as a scientific method for
understanding measurement variation and determining how much the variation within
the measurement process contributes to overall measured manufacturing process
variability. The observed measuring system is used to measure variable values of the
most important product quality characteristic, directly related to majority of nonconformities
found in the observed manufacturing system. To be fully confident about
measurement readings of the product characteristic, it is necessary to understand the
extent of confidence that the observed measuring system allows.

Full text

Measuring System Analysis in Six Sigma methodology
application – Case Study
M.Sc Sibalija Tatjana 1, Prof.Dr Majstorovic Vidosav 1
1 Faculty of Mechanical Engineering, University of Belgrade,
Kraljice Marije 16, 11120 Belgrade 35 PF 34, Serbia,
sibalija@infosky.net
Abstract: As the manufacturing industry moves toward 6-sigma capable processes, the
same requirement is becoming necessary for measuring system, as well. In the scope of
Six Sigma methodology application for the existing manufacturing system, according to
DMAIC cycle (Define-Measure-Analyse-Improve-Control), this paper presents case
study – statistical analysis of the measuring system, as a scientific method for
understanding measurement variation and determining how much the variation within
the measurement process contributes to overall measured manufacturing process
variability. The observed measuring system is used to measure variable values of the
most important product quality characteristic, directly related to majority of non-
conformities found in the observed manufacturing system. To be fully confident about
measurement readings of the product characteristic, it is necessary to understand the
extent of confidence that the observed measuring system allows.
Keywords: Measuring System Analysis (MSA), Gage R&R, Six Sigma.
1. INTRODUCTION
As the requirement for Statistical Process Control (SPC) implementation, Analysis of
Measuring System must be perform to ensure that measured values are correct and
relevant for analysis based on SPC. In this paper, real case study will be presented –
analysis of measuring system, within the scope of Six Sigma methodology application
in certain Serbian metal-processing manufacturing company, for the observed
manufacturing system / process / product. The observed measuring equipment
“MiniTest 600 B” is used to measure variable values of the most important product
quality characteristic - pan enamel thickness, directly related to majority of non-
conformities found in the observed manufacturing system [Sibalija, Majstorovic, 2006].
In order to assess the measuring system overall quality level and its capability to
measure the observed product quality characteristic, analysis of the measuring system
has been performed using two methods: (1) average and range (X bar – R) control charts
method, and (2) ANOVA method, quantifying measuring system characteristics:
repeatability and reproducibility, discrimination, stability, bias and linearity. Results of
analysis show constituent components of variation occurred during measuring process:

part-to-part variation, operator variation, measuring equipment variation and variation
due to interaction effects (if there are any), presenting an input for minimization of
variation introduced by measuring process, so that full focus on part-to-part variation
(variation of the observed product quality characteristic) can be set.
Input data for this MSA are [Sibalija, Majstorovic, 2006]:
- Specification Tolerance of pan enamel thickness: T=USL–LSL=(550–180)µm=380 µm
- Discrimination (resolution) of the observed measuring equipment is: 2 µm.
2. MEASURING SYSTEM ANALYSIS
2.1. Stability
Stability of measuring system is presented at figure 1, by X bar – R control chart. One
operator measured enamel thickness of the same product 15 times, over time period of
four weeks - once per week (table 1. – all values in µm).
Table 1. Data for Stability of Measuring System
Date Readings (µm) Aver. Range
05.07.06
232; 240; 230; 212; 210; 210; 218; 216;222; 214; 208; 206; 222; 228; 216
218.933
34
12.07.06
232;
226; 222; 222; 212; 222; 202; 206; 210; 218; 214; 220;226; 222; 220
218.267
30
20.07.06
238; 232; 240; 204; 210; 208; 218; 214; 224; 220; 212; 216;216; 234; 224
220.667
36
27.07.06
224; 226; 224; 220; 204; 208; 222; 220; 212; 214; 210; 214;222; 226; 218
217.6
22
Average
218.867
30.5
Figure 1. Xbar-R control chart for Stability

Stability Range chart: R mean = 30.5; (D4 = 1.653, for sub-group size=15)
UCL = D4 · R mean = 50.416
Stability Xbar chart: X mean mean = 218.867 (A2 = 0.223, for sub-group size=15)
LCL / UCL = X mean mean - / + A2 · R mean = 212.065 / 225.668
Since there are no points out of control limits on Xbar-R chart for Stability
[Pyzdek, 2003], the observed measuring system is considered as statistically stable.
2.2. Bias
Measuring System Bias is calculated [Pyzdek, 2003] by measuring standard part / etalon
(known thickness 95 µm) repeatedly 10 times, and finding discrepancy between
measurements average value and standard value (table 2.).
Table 2. Data for Bias of Measuring System.
2.3. Gage R&R
2.3.1. Gage R&R: Xbar–R Method
In order to calculate Gage R&R, 3 operators measured 5 different products/parts 3
times, to estimate measuring equipment variation (repeatability), operator variability
(reproducibility) and variation of pan enamel thickness (part-to-part variation).
Results are presented in table 3. Xbar-R chart for Repeatability is presented at
figure 2. The same measuring data are rearranged to calculate Reproducibility and
presented in table 4 and Xbar-R chart for Reproducibility is presented at figure 3.
Results of analysis show part-to-part variation, operator variation – reproducibility and
measuring equipment variation – repeatability, as well as measurement variation
relative to the tolerance of the pan enamel thickness (table 5).
a.) Repeatability (all values presented in table 3. are in µm)
Part Reading1
Reading2
Reading3
Average
Range
1
250
252
252
251.333
2
2
254
254
254
254
0
3
258
258
258
258
0
4
270
272
270
270.667
2
Operator 1
5
266
266
266
266
0
1
254
252
252
252.667
2
2
256
256
254
255.333
2
3
258
258
25
6
257.333
2
4
270
268
270
269.333
2
Operator 2
5
264
268
268
266.667
4
1
250
250
252
250.667
2
2
252
254
252
252.667
2
3
260
258
256
258
4
4
268
268
268
268
0
Operator 3
5
264
264
264
264
0
Average
259.644
1.6
Table 3. Data for Repeatability of Measuring System.
Readings (µ
m)
Average (µ
m)
Bias (µ
m)
95; 95; 93; 95; 95; 97; 94; 95; 97; 95
95.1
0
.1

Repeatability Range chart: R mean = 1.6; (D4 = 2.574, for sub-group size=3)
UCL = D4 · R mean= 4.118
Repeatability Xbar chart: X mean mean = 259.64444 (A2 = 1.023, for sub-group size=3)
LCL / UCL = X mean mean - / + A2·R mean = 258.008 / 261.281
- Standard Deviation for Repeatability (gage variation):
Sigma repeat.=Sigmae= Rmean /d2* =0.930, (d2* = 1.72, for 3 readings and 3 inspectors x 5 parts).
- Repeatability: 5.15 · Sigma e = 4.798,
(5.15 - const. - Z ordinate which includes 99% of a standard normal distribution).
At R chart for Repeatability, all values are less than UCL - measurement system’s
variability due to repeatability is consistent – there are no special causes of variation.
At Xbar chart, more than half of points are out of control limits – variation due to gage
repeatability error is less than part-to-part variation [Pyzdek, 2003].
b.) Reproducibility (all values presented in table 4. are in µm)
Read.1
Read.2
Read.3
Read.1
Read.2
Read.3
Read.1
Read.2
Read.3
Part
Operator 1 Operator 2 Operator 3 Average
Range
1
250
252
252
254
252
252
250
250
252
251.556
4
2
254
254
254
256
256
254
252
254
252
254
4
3
258
258
258
258
258
256
260
258
256
257.778
4
4
270
272
270
270
268
270
268
268
268
269.333
4
5
266
26
6
266
264
268
268
264
264
264
265.556
4
Average
259.644
4
Table 4. Data for Reproducibility of Measuring System
Figure 2. Xbar-R control chart Figure 3. Xbar-R control chart
for Repeatability for Reproducibility

Reproducibility Range chart: R mean = 4; (D4 = 1.816, for sub-group size=9)
UCL = D4 · R mean = 7.264
Reproducibility Xbar chart: X mean mean = 259.644 (A2 = 0.337, for sub-group size=9)
LCL / UCL =X mean mean - / + A2·Rmean= 258.296 / 260.992
- Standard Deviation for reproducibility:
Standard deviation for repeatability & reproducibility: Sigma o = Ro / d2* = 1.342
(d2* = 2.98, for 9 readings and 3 inspectors x 5 parts)
Sigma o = Sigma repeat.+reprod.2 = Sigma repeat.2 + Sigma reprod.2
Sigma reprod.= SQRT (Sigma o2 – Sigma repeat.2) = 0.968
- Reproducibility: 5.15 · Sigma reprod.= 4.983
(5.15 - const. - Z ordinate which includes 99% of a standard normal distribution)
- Measurem. System Standard Deviation: Sigma m = SQRT(Sigma e2+Sigma o2) = 1.633
- Measurement System Variation: R&R = 5.15 · Sigma m = 8.410
For Reproducibility, all values at R chart are less than UCL - measurement system’s
variability due to repeatability & reproducibility is consistent – there are no special
causes of variation; more than half of points at Xbar chart are out of control limits –
variation due to gage repeatability & reproducibility error is less than part-to-part
variation [Pyzdek, 2003].
c.) Part-to-Part Variation
- Range of the parts averages: Rp = 17.778
- Part-to part standard deviation: Sigma p = Rp / d2* = 7.168
(d2* = 2.48, for 5 parts and 1 calculation for R)
- 99% spread due to part-to-part variation: PV = 5.15 · Sigma p = 36.918
d.) Overall Measuring System Evaluation
- Total process standard deviation: Sigma t = SQRT ( Sigma m2 + Sigma p2) = 7.352
-.Total Variability: TV = 5.15 · Sigma t = 37.863
- The percent R&R: 100 · ( Sigma m / Sigma t) % = 22.213%
- The number of distinct data categories that can be created with this measurement
system: 1.41 · (PV / R&R) =6.189122 = 6.
Since the number of categories for this measurement system is 6 (>5 minim. required)
[Pyzdek, 2003], this measuring system is adequate for process analysis / control.
Table 5. Analysis of spreads - Measurement variation relative to the tolerance.
Taking into consideration all relevant factors (cost of measurement device, cost of
repair, etc.), the observed Gage R&R System may be accepted, since operators and
equipment cause 22.21% (< 30%) of variation. But, it is far away from doubtless
acceptance threshold of 10% [Pyzdek, 2003].
Source
St.Dev.
Variability(5.15·St.Dev.)
% Variab.
% Tolerance (Var./Toler.)
Total gage R&R
1.63
8.41
22.21%
2.27%
Repeatability
0.93
4.79
12.65%
1.29%
Reproducibility
0.97
4.98
13.16%
1.35%
Part
-
to
-
part
7.17
36.92
97.50%
9.98%
Total variation
7.35
37.86
100.00%
10.23%

2.3.2. Gage R&R: ANOVA Method
Analysis of measuring results using ANOVA method (table 6.), includes analysis of
interaction operator * part.num . Since “alpha to remove interaction term” is set to 0.05
(for 95% of confidence), variation due to interaction operator * part.num is found as
insignificant (table 6.) [Pyzdek, 2003]. Results of analysis show constituent components
of variation occurred during measuring process (table 7), as well as variations relative to
the tolerance of the pan enamel thickness and relative to concerning manufacturing
process variation (table 8). Figure 4 gives graphical presentation of these.
Source DF SS MS F P____
PartNum 4 2066.31 516.578 338.707 0.000
Operator 2 22.04 11.022 7.227 0.002
Repeatability 38 57.96 1.525
Total 44 2146.31
(
Alpha to remove interaction term = 0.05)
Table 6. Two-Way ANOVA Table Without Interaction, for Gage R&R.
Percent
Part- to-PartReprodRepeatGage R&R
100
50
0
% Con tribution
% St udy Var
% To lerance
Sample Range
4
2
0
_
R=1.6
UCL=4.119
LCL=0
1 2 3
Sample Mean
270
260
250
_
_
X=259.64
UCL=261.28
LCL=258.01
1 2 3
PartNum
54321
270
260
250
Operator
321
270
260
250
PartNum
Average
54321
270
260
250
1
2
3
Oper ator
Gage name :
Date of study :
Reported by :
Tolerance:
Misc:
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Data by PartNum
Data by Operator
Operator * PartNum Interaction
Gage R&R (ANOVA) for Data
Figure 4. Gage R&R – ANOVA method.
Observed Gage R&R System may be accepted, since operators and equipment
cause 19.06% (< 30%) of variation. But, it exceeds doubtless acceptance threshold of
10% [Pyzdek, 2003]. Since the number of distinct categories for this measurement
system is 7 > 5 (minim. required), this measuring system is adequate for process
analysis or process control.

Table 7. Components of Variance Analysis. Table 8. Analysis of spreads.
Results for Gage R&R, from ANOVA method, differ slightly from results obtained
using Xbar-R method, since by using Xbar-R method there is no possibility to include
and calculate interaction - effect operator * part.num (in this Gage R&R, this interaction
is found as insignificant, but it still takes certain value). Thus, ANOVA method for
Gage R&R is considered as more accurate than Xbar-R method.
2.4. Linearity
Linearity is determined by choosing products/parts that cover most of the operating
range of the measuring equipment; then Bias is determined at each point of the range
[Pyzdek, 2003]. In this case, 4 parts were chosen, with following expected enamel
thickness: 100, 220, 360 and 470 µm; each part was measured 10 times; discrepancy
between their average value and expected value presents bias in that point (table 6).
Reference Value
Bias
500400300200100
15
10
5
0
-5
0
Regression
95% CI
Data
Avg Bias
Percent
BiasLinearity
10
5
0
C onstant -0.800 2.143 0.745
Slope 0.016870 0. 006703 0.128
Predictor C oef SE C oef P
Ga ge Linearity
S 1 .87525 R-S q 76.0%
Linearity 0. 63870 % Linearity 1.7
A v erage 4.05 10.7 *
100 2. 00 5.3 *
220 2. 00 5.3 *
360 3. 60 9.5 *
470 8. 60 22.7 *
Refe rence B ias %B ias P
Ga ge Bias
Gage nam e:
Date of study :
Reported by:
Tolerance:
Misc:
Percent of Pr ocess Variation
Gage Linearity and Bias Study for A verage
Figure 5. Linearity and Bias Study of Measuring System.
%Contribution
Source VarComp (of VarComp)
Total Gage R&R 2.1583 3.63
Repeatability 1.5251 2.57
Reproducibility 0.6331 1.07
Operator 0.6331 1.07
Part-To-Part 57.2281 96.37
Total Variation 59.3864 100.00
Process tolerance = 370
Study Var %Study Var %Toleran.
StdDev (SD) (5.15·SD) (%SV) (SV/Toler)
1.46911 7.5659 19.06 2.04
1.23497 6.3601 16.03 1.72
0.79570 4.0979 10.33 1.11
0.79570 4.0979 10.33 1.11
7.56492 38.9594 98.17 10.53
7.70625 39.6872 100.00 10.73
Number of Distinct Cate
gories = 7

Then, a linear regression was performed (figure 5.).
Part
Readings (µm) Average (µm)
Ref.value (µm)
Bias (µm)
1 101; 101; 102; 103. 102. 102; 102; 101; 102; 104
102 100 2
2 224; 222; 220; 222; 222; 222; 222; 220; 222; 224
222 220 2
3 362; 368; 360; 364; 368; 364; 362; 360; 364; 364
363.6 360 3.6
4 478; 478; 478; 478; 480; 478; 478; 482; 478; 478
478.6 770 8.6
Table 6. Data for Linearity and Bias of Measuring System
The equitation of linearity is (figure 5.): Bias= - 0.800 + 0.01687 · Ref.value.
Since P values are higher then 0.05 (figure 5.), gage bias is statistically
insignificant. R-Sq is 76.0 %, meaning that straight line explains about 76% of the
variation in the bias readings (>50% - acceptable). Further, the variation due to
linearity for this gage is 1.687% of the overall process variation. The variation due to
accuracy for this gage is 10.6963% of the overall process variation.
3. MEASURING SYSTEM CAPABILITY
3.1. Capability indices for Gage – Cg and Cgk
According to [Dietrich, 2006], capability indices for gage can be calculated by
measuring of standard part n times and calculating average measurement value, bias and
standard deviation of measurement. From data presented in table 2., one can find:
Average = 95.1 µm; Bias = 0.1 µm; St.Deviation = 1.197 µm;
so, capability indices for this gage are:
Cg = 0.2·T / 4·St.Deviation = 15.45 >1.33 ... (1) [Dietrich, 2006]
Cgk = (0.1·T - Bias) / (2·St.Deviation) = 15.41>1.33 ... (2) [Dietrich, 2006]
Since values for Cg and Cgk are over 1.33, one can conclude that measuring process is
capable according to this criteria.
3.2. Precision-to-Tolerance (PTR) ratio, Signal-to-Noise (STN) ratio and
Discrimination ratio (DR)
The precision-to-tolerance ratio (PTR) is a function of variance of measurement system:
PTR =(5.15·SQRT(Variancemesurement system))/(USL-LSL))·100% = 2.045 % ... (3)
(Variance mesurement system can be found in table 7. - VarComp for Total Gage R&R).
As it is stated in [Burdick, Borror, Montgomery, 2003], since PTR for this measuring
system is less than 10%, the measurement system is adequate.
The adequacy of a measuring process is more often determined by some function
of “proportion of total variance due to measurement system” [Burdick, Borror,
Montgomery, 2003], as it is signal-to-noise ratio (SNR) and discrimination ratio (DR):
SNR = SQRT((2·(Varp / Vartotal) / (1-Varp / Vartotal)) = 7 ... (4)
DR = (1+ (Varp / Vartotal)) / (1 - (Varp / Vartotal)) = 54 ... (5)
(Varp / Vartotal can be found in table 7. - VarComp for Part-To-Part / Total Variation).
AIAG (1995) defined SNR as the number of distinct levels of categories that can be

reliably obtained from the data [Burdick, Borror, Montgomery, 2003] and value of 5 or
greater is recommended. Also, it has been stated that DR must exceed 4 for the
measurement system to be adequate. Values SNR = 7 and DR = 54 indicate that the
observed measuring system is adequate.
3.2.1 Confidence Interval for PTR (95% confidence)
Limits for PTR confidence interval are [Burdick, Borror, Montgomery, 2003]:
L PTR = 5.15·SQRT(Lower Bound)/(USL-LSL) ... (6)
U PTR = 5.15·SQRT(Upper Bound)/(USL-LSL) ... (7)
where bounds are:
Lower Bound = Estimate Variance mesurement system - SQRT(V LM)/(p·r) ... (8)
Upper Bound = Estimate Variance mesurement system + SQRT(V UM)/(p·r) ... (9)
where: - p = 5 – number of different part measured for Gage R&R
- r = 3 – number of repeated measurement (readings) for Gage R&R
- o = 3 – number of operators that performed measurements for Gage R&R
- Estimate Variance mesurement system= (SDo2 + p·(r-1)·SDe2) / (p·r) ... (10)
(SDo / SDe – StdDev for Operator / Repeatability from table 8.)
- VLM = G22 · MSO2 + G42 · p2 · (r-1)2 · MSe2 ... (11)
VUM = H22 · MSO2 + H42 · p2 · (r-1)2 · MSe2 ... (12)
(MSO / MSe – MS for Operator / Repeatability, table 6.; coefficients are:
G2 = 1- 1/F(1-α/2, o-1, infinite); G4 = 1- 1/F(1-α/2, p·o·(r-1), infinite) ... (13)
H2 = 1 / F(α/2, o-1, infinite) –1; H4 = 1 / F(α/2, p·o·(r-1), infinite) – 1 ... (14)
where F(., ., .) is Fisher test value and α = 0.05 - threshold).
Results: Estimate Variance mesurement system= 1,059; VLM = 94.939, VUM =180194.689;
Lower Bound = 0.409, Upper Bound = 29.358
LPTR = 0,89% <= PTR <= UPTR = 7.54%
Since lower and upper limit for PRT are less then 10%, there is sufficient evidence
to claim that the observed measuring system is adequate for prodact characteristic
measurement!
3.2.2 Confidence Interval for SNR and DR (95% confidence)
SNR confidence interval limits are [Burdick, Borror, Montgomery, 2003]:
LSNR = SQRT((2·Lower Bound)/(1-Lower Bound)) ... (15)
USNR = SQRT((2·Upper Bound)/(1-Upper Bound)) ... (16)
where bounds are: Lower Bound = (p · L*) / ( (p · L*) + o ) ... (17)
Upper Bound = (p · U*) / ( (p · U*) + o ) ... (18)
where: L*=MSp / ((p·(r-1)· F(1-α/2, p-1, infinite)·MSe)+(F(1-α/2, p-1, o-1)·MSo)) ...(19)
U*=MSp / ((p·(r-1)· F(α/2, p-1, infinite)·MSe)+(F(α/2, p-1, o-1)·MSo)) ...(20)
(MSp / MSe / MSo – MS for PartNum / Operator / Repeatability, table 6.).
Results: L* = 1.087, U* =179.251; Lower Bound = 0.644, Upper Bound = 0.997
L SNR= 1.904 <= SNR <= USNR = 24.444
Since not all values in the interval for SNR exceed 5, there is not sufficient evidence to
claim the measurement system is adequate for monitoring the process.
Limits for DR confidence interval [Burdick, Borror, Montgomery, 2003]:

LDR = (1+Lower Bound)/(1-Lower Bound)) = 4.624 ... (21)
UDR = (1+Upper Bound)/(1-Upper Bound)) = 598.504 ... (22)
LDR = 4.624 <= DR <= UDR = 598.504
Regarding to DR confidence interval, one can says that the observed measurement
system is adequate for monitoring the process, since both DR limits exceed value 4.
Note: All above stated equitation for confidence intervals are valid only in case when interaction
effect operator * part.num. is insignificant.
4. CONCLUSION
Measuring system analysis has been performed, with good results for all criteria
considering central location of measurements. As regards to variability of the
measurements (Gage R&R), we conditionally accepted this measuring system for
considered measurement. Measuring system capability (presented over gage potential
Cg and capability Cgk) satisfies required criteria, as well as confidence interval for PTR
and DR ratio. Thus, this measurement system is adequate for monitoring the process,
according to Cg, Cgk, PTR and DR criteria. One concern is STN ratio, since its
confidence interval doesn’t satisfy required value. This could be expected, also, from
ANOVA analysis, because “Number of Distinct Categories” is 7, not far enough from
minimum required value 5. Further, this corresponds to only conditional acceptance of the
measuring system, regarding to Gage R&R value. In order to absolutely accept this
measuring system for pan enamel thickness measurement, clamping of the part or
measuring instrument during measuring process and measuring instrument maintenance
/ repair, if necessary, should be considered. Advanced training for operators and fixture,
to help the operators to use measuring instrument more consistently, are advisable.
REFERENCES
[Sibalija, Majstorovic, 2006] Sibalija, T.; Majstorovic, V.; "Application of Six Sigma
Methodology in Serbian industrial environment"; In: International Journal ’Total
Quality Management & Excellence’, Vol.34, No.3-4, YUSQ EQW 2006; Belgrade,
Serbia, 2006; ISSN 1452-0699
[Pyzdek, 2003] Pyzdek, T.; "The Six Sigma Handbook"; In: McGraw-Hill Companies
Inc., USA, 2003; ISBN 0-07-141015-5
[Dietrich, 2006] Dietrich, E.; "Using the Process Steps of a Measurement System
Capability Study to Determine Uncertainty of Measurement"; http://www.q-
das.de/homepage_e/es geht auch einfach_e.htm; Q-DAS GmbH, 2006
[Burdick, Borror, Montgomery, 2003] Burdick, R.K.; Borror, C.M.; Montgomery,
D.C.; "A Review of Methods for Measurement Systems Capability Analysis"; In:
Proceedings of the 47th Annual Technical Conference of the Chemical and Process
Industries, Journal of Quality Technology, Vol.35, No.4, El Paso, USA, 2003