MSA  Continuous data
MSA  Continuous dataProcess variation affects how resulting products and services appear to Customers. This appearance includes not only the item's variability, but variation from the item's measurement.
For example, you measured the thickness of a pair of glasses' lens. Then you
ask three other people to measure the lenses. It is highly
likely that there would be a difference in answers between
everyone. It is also highly likely that if someone handed you the
same pair of glasses later (without you knowing it was the same
pair) and asked you to measure again, you would come to a different
measurement.
The pair of glasses did not changed. The measurement system
caused the different answers and specifically errors within it. The higher the measurement
error, the harder to understand the true process capability
and behavior.
The sole purpose of a continuous data measurement system is to collect the right data
to answer the questions being asked. To do this, the team must be
confident in the integrity of the data being collected. To confirm
data integrity the team must know...
*The type of data
*If the available data is usable
*If the data is suitable for the project
*If it is not suitable, whether it can be made usable
*How the data can be audited
*If the data is trustworthy
To answer these questions, break down continuous data integrity
into two elements:
1) Validity. Are your measuring
the right aspect of the process or product? The continuous data might be
from a reliable method or source, but still not match the
operational definitions established for the project.
2) Reliability.
Is the valid measurement system producing continuous good data? This considers
the accuracy and consistency of the data.
Gage R&R
To confirm validity and reliability, one uses a Gage R&R to conduct an audit on a
continuous data measurement system. This involves using multiple people,multiple items, and multiple measurements to complete the audit . Each person measures every item at
least twice. The Gage R&R data determines...
*Percentage overall agreement (% Repeatability & Reproducibility)
*Percentage agreement within individuals (% Repeatability, agreement with themselves)
*Percentage agreement between individuals (%
Reproducibility, agreement with others)
Gage R&R vs Calibration
Gage R&R study
examines the whole continuous data measurement system including the
test samples, people, techniques, and methods. Many people confuse a Gage R&R Study with
tool calibration but the two are different. Calibration considers only the reading and comparison to a known standard.
Gage R&R is significantly different and
often far more difficult. It analyzes the entire variation within the measurement system.
Total Process Variation
The total process variation
comes from the true
process variation and the continuous data measurement system.
Total Variation = Process Variation +
Measurement Variation
Or in statistical terms
To effectively see the process variation, the variability due to the measurement system should
be small.
Gage R&R determines..
*The size of the measurement error
*The sources of measurement error
*Whether the Measurement System is stable over
time
*Whether the Measurement System is capable for
the study
Where in the Measurement System to focus
improvements
Repeatability
The Gage R&R study breaks the
total observed measured continuous data variation into two components
repeatability
and reproducibility
:
Repeatability variation occurs when repeated measurements are made of the same variable under absolutely identical conditions. It is the variation between successive measurements of the same sample, of the same characteristic, by the same person using the same instrument.
Poor repeatability causes an increase in decision
error. When the same person looks at the same characteristic and
measures different values he makes different
decisions. Some of these decisions are the
wrong decisions!
Source: SBTI's Lean Sigma Methodology training material.
Reproducibility
Reproducibility
variation comes from different people making
measurements on the same items using the same instrument. We examine this variation with different environmental conditions such as time, environment, temp, work conditions etc. When two or more individuals return the same value for a given
characteristic, that measure is said to be reproducible.
Source: SBTI's Lean Methodology training material.
P/T Ratio
The P/T compares
the size of the continuous data measurement system error with respect to
the size of the specification. Since all measurement systems have variation, this metric lets you know if your measurement system variation is acceptable (or minimal) to measure a characteristic.
The P/T Ratio represents the percent of the
specification tolerance taken up by measurement error. The metric includes both repeatability and reproducibility. An
excellent continuous data measurement system has a P/T Ratio less than
10%. A value of 30% is barely acceptable.
5.15
standard deviations account for 99% of Measurement System
variation. The use of 5.15 is an industry standard, but more
recently some texts recommend the use of 6 standard deviations,
which represents 99.73% of Measurement System variation. Either value
is appropriate provided you use it consistently across the
business.
%R&R
If you're measuring process
improvement, then we recommend using a more appropriate metric of %R&R.
This represents the percentage of the total process variation taken by
measurement error:
An excellent Measurement System has a %R&R
less than 10%. A value of 30% is barely acceptable.
Discrimination
Discrimination (sometimes called resolution) represents the number of decimal
places that can be measured by the system. Increments of measure
should be about onetenth of the width of the product specification
or process variation
Roadmap
Step 1.

Identify the metric and agree within the team
on its operational definition (see "KPOVs and Data" ). 
Step 2.

Select 3 to 12 samples. Selection of each should be
independent from the others. Samples should span the normal
variation of the process. For example, for material with a mean
thickness of 0.020 inches and a standard deviation of 0.001 inches
samples should have thickness from 0.017 to 0.023 inches (99% of the range). Do not randomly draw samples from the process as they tend to be grouped close to the mean and not represent the full
width of the process. 
Step 3.

Select a minimum of 3 appraisers to conduct the MSA. Chose people who normally conduct the measurement. If the process
uses only one operator or no operators at all, then perform the
study without operator effects (Reproducibility effects are thus
ignored). 
Step 4.

Select the number of trials. This needs to be
at least two. The total number of continuous data points (samples x
appraisers x trials) should be greater than 30. For example, for
five samples and two operators, it would be best to use three or
four trials to generate 30 or 40 data points. 
Step 5.

Calibrate the gage, or assure that it has been
calibrated. 
Step 6.

Perform the appraisal. Randomly provide the
samples to one appraiser (without them knowing which sample it is)
and have them measure the item. After the first appraiser has
measured all the entities, repeat with the remaining appraisers.
Appraisers must measure independently and out of sight of other
appraisers to minimize potential bias. After all appraisers have
measured each item, repeat the whole process for the required
number of trials. 
Step 7.

Enter the data into a statistical software
package such as Minitab and analyze it. The analysis output typically
includes *Repeatability *Reproducibility *%R&R *P/T Ratio 
Interpreting the
Output Charts
The Range Chart should show a process that is in control. Repeatability is questionable if the range chart shows outofcontrol conditions. If a point is above the UCL, that operator is having a problem making consistent measurements. If the range chart for an operator is outofcontrol and the other charts are not, then the method is probably suspect. If all operators have ranges outofcontrol, the system is sensitive to operator technique.
The Range Chart also helps
identify inadequate discrimination. There should be least five
distinct levels (points along the Y axis) within the Control Limits. Also, if there are more than 1/4 of the values
at zero, then again, discrimination is suspect.
For the Xbar Chart by operator, this is the average reading of each item measured. Due to part to part variation, the
majority of the points on the chart should fall outside the control
limits. In addition the pattern of the reading needs to be consistent for all the operators. If not then the reproducibility is suspect. If there are no points
outside the control limits, then the selected samples
did not cover the full range of the process (i.e.., there
was not enough parttopart variation).
Below is an example of an OperatorPart Interaction Plot.
For a reliable continuous data measurement system, the lines should follow the same
pattern and be reasonably parallel to each other. Crossing lines
between operators indicates significant interactions. Also the part
averages should vary enough that the differences between parts are
clear.
An example of Gage
R&R OperatorPart Interaction Plot
Below is an example of a Gage R&R by Operator plot, which shows the
average value (Circle) and the spread of the continuous data for each
operator. The spread should be similar across all operators and
there should be a flat line across the means of the operators.
An example of a Gage
R&R By Operator Plot
Interpreting the
Output Charts
An example of Gage R&R analytical results (output from Minitab v14).
Source 
Std Dev (SD) 
Study Var (5.15*SD) 
%Study Var (%SV) 
%Tolerance (SV/Toler) 

Total Gage R&R 
0.066615 
0.34306 
32.66 
68.61 
Repeatability 
0.035940 
0.18509 
17.62 
37.02 
Reproducibility 
0.056088 
0.28885 
27.50 
57.77 
Operator 
0.030200 
0.15553 
14.81 
31.11 
Operator* Sample 
0.047263 
0.24340 
23.17 
48.68 
ParttoPart 
0.192781 
0.99282 
94.52 
198.56 
Total Variation 
0.203965 
1.05042 
100.00 
210.08 
Number of Distinct Categories = 4 


The key metrics to examine
*The P/T Ratio (listed as the %Tolerance) at
68.61%. This gage is clearly not suitable as a production gage (30%
is acceptable).
*The %R&R (listed as %Study Variation) at
32.66%. This gage is less than acceptable to help make improvements
to the process in question (30% is acceptable).
*The majority of the variation comes from reproducibility, which can be seen from its standard deviation
(0.056088) versus repeatability (0.035940). Appraisers aren't
agreeing with one another.
*The largest portion of
reproducibility comes from an OperatorSample interaction. In some
way, due to the items measured characteristic, the operators measure different samples differently.
This could occur when one or more appraisers aren't good with
small parts, but can adequately measure larger parts, whereas
others can measure all samples equally well.
*The Number of Distinct Categories is an
indication of the discrimination of the measurement system. If the
number of categories is less than five, the measurement system is of
minimal value because it is difficult to distinguish one entity
from another.