Determining Statistically Significant Sample Size

Question

I am developing an internal audit process within our supply chain to determine packaging and Finalizing SOP’s are being followed. I need to determine what will be the sample size needed to accurately represent the population. We are currently shipping out 650k cartons a day. How do I determine how many audits I need a day for statistical significance?

Answer

Statistical sampling theory shows that for large populations, the sample size is not a function of the population size, assuming all units in the population have an equal probability of being selected for the sample.  To ensure a representative sample, stratified random sampling is employed to represent in the audit sample. This method requires that each category (or stratum) is specified, and that none of them overlap (i.e., items to be audited must fall in only one category).  For example, you can break the packaging records in groups of 25,000 (26 stratum for 650,000 records), sampling 1/26^th of the sample from each stratum.

To determine the sample size, we employ the binomial distribution where a records is either confirming or nonconforming. 

The basic formula for the binomial confidence interval is

For a given sample size (n) with a given number of defects (x), the probability of the sample coming from a population with probability (p) is given by the value alpha (a).  The above equation can be solved for probability (p) at a given a level or can be solved for a at a given population probability (p).

In other words, you specify the percent defective in the population you can accept.  The only when to ensure 0% defective is 100% sampling. You solve the equation for n by setting 1-alpha (1-a) equal to a high probability (i.e. 95%).  If you desire to accept zero (0) defects in the sample then set x equal to zero. In this case, the equation reduces to ln(1- a)/ln(1-p).

Hope this helps with the question.

Thanks

Steven

For more on this topic, please visit ASQ’s website.

ISO 9001 Certification for University

Question

How should I proceed to get my university ISO 9001 certified? There are nine campuses and a quality manual already exists.

Answer

If I may, let me answer this from my own experience. I worked with an organization that was geographically dispersed. In the US, there was the corporate headquarters, which had top management, sales and marketing, and design. There was also a manufacturing plant in another state. There was a manufacturing plant in Brazil and one in France.

It was decided that top management would own organization-wide processes. This included Internal Audit, which was outsourced. Top management was responsible for strategy, interested parties (also addressed on a local basis), overall quality objectives, risk and opportunity, the process for documented information on a macro level (for example, your quality manual), organizational knowledge, data analysis and management review (each quarter held in a different facility).

The plants did their own purchasing and supplier development, process risk and opportunity, hiring and training, operational control, local control of specific documented information and data reporting. The three manufacturing plants responded well to local autonomy and centralized reporting. Management Review was very rigorous with the CEO and functional vice presidents at each review. Progress toward objectives, monitoring of interested parties, risk and opportunity was stressed.  If an objective was lagging, plant managers had to present their corrective action with root cause analysis.

Considering the geographical spread, three languages and different cultures and labor laws, the system was very effective, in large part due to the commitment of the CEO.

The 3^rd party auditors, while from the same certification body, were local to the facilities. The lead auditor amassed audit data and presented that to top management with a detailed report.

With applications such as Zoom, considerable travel time and expenses can be saved.

Regards,

George Hummel

For more on this topic, please visit ASQ’s website.

Scope Statement

Question

My company is late to transition from ISO 9001: 2008 to 2015 and we are just starting. I am already stumped. I need to answer “What is our your scope statement?  This should succinctly summarize your products and/or services. A single sentence is all that is required, as this will be shown your ISO 9001:2015 certificate.”

We are a commercial printing and bindery. Is the scope from receipt of order through final acceptance by the customer? I don’t know what this means.

Answer

Scope is a difficult concept for many. Accuracy is important; however, you encounter two types of scope that may confuse you.  The first is the Scope of the Standard.  It’s in the “introduction,” in clause 01.  It’s informational and is not auditable.  The second is clause 4.3.  Scope is the outcome of the work in 4.1 & 4.2.  There can be a number of ways to address scope: your range of products and services (XYZ company provides design and printing services); different sites (XYZ designs art work in its Chicago studio and our Cleveland plant does commercial printing) – each of these sites addresses their activities; outsourced services (XYZ company coordinates outsourced design and printing services for our customers).  You can slice and dice scope for a single plant (having others in the company), specific sections (within XYZ company, implantable medical devices are manufactured).

In your case, “XYZ provides commercial printing and bindery” is sufficient.

The scope must be documented and will be verified by your third party certification body.

Thanks for your question.

George Hummel

For more information on this topic, please visit ASQ’s website.

ISO Certification without Quality Dept?

Question

There were some changes recently in the company where I work. Now, a quality manager or quality function does not exist in the our company; however, we do maintain the ISO certification.  Our ISO certification applies to our corporate office and it applies also to the manufacturing facility located in a different city.

Is acceptable to have an ISO certification and not have a quality function?

Answer

There is no requirement for a quality function in ISO 9001:2015. I believe that this can be a positive move for your organization as it puts the job of quality upon the process owners, especially top management.  This is where it should be.

It can be a little messy as these responsibilities are passed back to the process owners; but, it’s right!

George Hummel

For more on this topic, please visit ASQ’s website.

Making Management Review Meaningful

Question

I’m looking for assistance with Management Review requirement of ISO 9001:2015. We’ve been following an agenda that covers 9.3.2 a-f, but Senior Management believes that they cover much of this information in other meetings.

I found a few articles through the ASQ website, but wanted to see if there are suggestions for other resources to help make the Reviews relevant and useful rather than just checking off a box.

Answer

Management Review does not have to happen in one session but can be addressed over several meetings.  It is required that all the inputs and outputs (not addressed in your question) are recorded and accessible.

It is important that the inputs/outputs are not “checking off a box.” Management Review should be seen as “due diligence.”  For example, it is not designed to say, “internal audits were performed on xx/xx/2019.” Here is the opportunity for top management to review the audit results for improvement opportunities and determine how risks uncovered can be mitigated.

Note that the standard does not say “a meeting.” You may wish to gather the materials into one document and send it back to top management for review and approval.  This would also allow you to determine if there are any gaps to be addressed.

Try to ensure that top management notes changes in the QMS or needed. For example, does the QMS still support the strategic directions of the organization, have the requirements of interested parties changed, has corrective action found the root cause of problems, have complaints been adequately addressed and, have there been any changes in statutory requirements?

Following the review of the information, it would be my advise to publish the results to communicate these to the entire organization.

George Hummel

AS9100 7.1.5.2 Measurement Traceability

Question

Can a manufacturer use “reference only” M&M equipment to accomplish in-process checks as long as these items are verified later on by “Inspection” using calibrated and/or verified equipment?

Would all of the measurements have to be verified by properly calibrated equipment?

Answer

The practice of using “reference-only” devices for in-process measures is not noncompliant with the standard.  The formal inspections to accept product would require use of calibrated equipment.

In clause 7.1.5.1, the standard clearly states “the organization shall determine and provide the resources needed to ensure valid and reliable results when monitoring or measuring is used to verify the conformity of products and services to requirements.”  And in clause 7.1.5.2, “hen measurement traceability is a requirement, or is considered by the organization to be an essential part of providing confidence in the validity of measurement results…”

Users and auditors should look at formal inspections steps and final buy-off to determine if any nonconformities were escapes from in-process measurements being conducted without calibrated devices.  Customer complaints can also be analyzed to see if any escapes were a result of failed in-process inspections.  If so, then the organization did not properly determine and provide the appropriate resources.

Buddy Cressionnie

For more information on this topic, please visit ASQ’s website.

Z1.4 Inspection Requirements

Question

Our customers require that we follow the ANSI Z1.4 standard for attribute sampling plans; however, it is not feasible to wait until lots are completed to perform inspections. Lots can be large and run for many days and waiting until lot completion to determine the sample size, based on the finished lot size, is too late because we will have missed our chance to correct any production issues that may result in defective parts. Another limitation is a lack of space to stage product while waiting for the final inspection of the completed production lots. Product is made as orders are received, and not typically stored as inventory, so our on-time delivery demands also hinder our ability to hold product for final inspections of completed production lots. Therefore, we are seeking guidance on a practical way to implement a in-process inspection during production that follows the ANSI Z1.4 standard.

Answer

Yes, you can sample as you produce to get to the sample size.  It is important that you keep track of your accept/reject items.  Since you know how long you are running the product, you can project the approximate lot size to get the sample size.  Work with your scheduler before the product starts so you can take samples early and continue on in the process.

In addition, if you have material changes as the product is running, I am sure that you are sampling then to make sure everything is set correctly, you can use those samples also.  As an example, let’s say your product is running 4 days and based on the projected lot size, you have a sample of 28 to take, you could take 7 samples each day spaced throughout the day or you take 10 samples the first day, 7 the second day, 6 the third day and 5 the last day of production.  You need to figure the right sequence that fits your history of the product.

Jim Bossert

For more on this topic, please visit ASQ’s website.

AQL Calculation Explanation

Question

Regarding part of your answer to a post found here, you state:

“The calculation of AQL is not dependent on lot size. In other words, a sample size of 315 gives a minimum AQL of 0.04, so a larger sample is required to estimate an AQL of 0.01.”

Can you explain for the non-statistical folks like me people how that math works? Specifically, I am wondering what the minimum sample size would be for an AQL of 0.25, when using Special Inspection level S2? Would it be a minimum of 50, no mater what the lot size is?

Answer

Acceptance sampling procedures were developed during the early 1920s at Western Electric Company and later formalized at Bell Telephone Laboratories where terms like producer’s risk and consumer’s risk were established.  Later, during World War II, sampling plans such as MIL-STD-105 were developed by Harold F. Dodge and others working with the Army Quartermaster Corps (Dodge, 1967).

Two special features were employed in order to gain agreement with the large body of military suppliers.  One was the use of the acceptable quality limit (AQL) as opposed to the RQL in presenting the plans.  The goal at the time was to focus on rewarding suppliers for production whose quality levels were considered good.  RQLs were recognized but not often brought to the surface during discussions. Also, at that time, the term “AQL” was deliberately vague or inexact.  It was a close approximation, not an exact probability statement.

The other feature was the practice of increasing sample sizes with increased lot sizes.  As noted in Section 3, in most situations, the lot size does not factor in plan construction (based on the binomial).  For many, however, this lacks intuitive appeal.  Therefore, in the development of MIL-STD-105 and its derivatives a deliberate increase in sample sizes for higher lot sizes was introduced, with corresponding increases in acceptance numbers for similar AQLs.  Clearly, this practice resulted in over-sampling and consequent increased inspection costs.  Government operatives believed that the increased sampling cost was of small consequence relative to the power to persuade.

For the binomial distribution you solve for the AQL that gives a high probability of passing.  Usually this probability is set at 95%.  For example if you have a sample size of 80 units with an accept/reject of 1, an AQL of 0.65% would have a 90% probability of passing the sampling plan.

You can use Excel to solve this with the function

=BINOMDIST(1,80,0.0065,1)

Hope this helps,

Steven Walfish

For more on this topic, please visit ASQ’s website.

Sampling Plan Review?

Question

When following ANSI/ASQ Z1.4-2003 (R2018), if a product has been placed in a “reduced” sampling plan based on the previous 10 lots results, is it a requirement to convert back to a “normal” sampling plan on an annual basis, or should that decision remain based on supplier performance? I have been told that we should revert to normal sampling each year, but I do not see that in the AQL inspection manual.

Answer

The standard does not require annual (or periodic) review of the sampling plan.  The switching rules are time invariant, and reflects just the normal flow of lots, which can span more than a year.  Unless the supplier requires a change in the inspection level, the standard is silent on resetting to the normal level annually.

Steven Walfish

For more on this topic, please visit ASQ’s website.

7.1.5.2 AS9100 D

Question

Section 7.1.5.2 of AS9100D States that “the organization shall establish, implement, and maintain a process for the recall of monitoring and measuring equipment requiring calibration or verification.”

Are calibration and verification both referring to the checking of equipment to make sure that it is suitable for it’s purpose, or is verification referring to the measurements taken on product? More specifically, does this require the organization to be able to identify which piece of measurement equipment made which measurements? In other words, if I have two micrometers that are both in a system that recalls them periodically for calibration, have I satisfied the requirements of 7.1.5.2 even if I don’t record which micrometer makes which measurements during it’s daily use?

Answer

The calibration or verification both refer to the monitoring and measurement equipment.  The AS9100D additional text is consistent with the ISO 9001:2015 by referring to calibration or verification.  The monitoring and measurement equipment could require verification as meeting requirements instead of a calibration.  If monitoring and measurement equipment is found to be out-of-tolerance and there is a product conformity impact, then it is very helpful if the organization has identified which instrument was used for which job.  Otherwise, the recall of product or alerting customers is much broader since the impact is not understood.  There is not specifically an AS9100D requirement to record which monitoring and measuring equipment was used but it is a good practice.

Buddy Cressionnie

To learn more about this topic, visit ASQ’s website.