The following application article is part of a larger set of articles
published under the title ISO 9000 in Scientific
Computing as a special supplement to Scientific Computing & Automation
magazine.
The bad news first: despite the prodigious effort involved, everyone
is caught unawares on some aspect of the requirements. That's to be expected
and the presence of nonconformances does not reflect negatively on any company.
After all, you're paying the auditor to find something wrong. With a standard
so open to interpretation as ISO 9000, that's exactly what will happen.
To make it even trickier, every auditor has a different perspective and
every registrar a different approach. The experiences described below are
sometimes cumulative observations by the managers for several sites under
their supervision, sometimes the experience happened only once.
Now for the good news: everyone survives the audit, fixes their nonconformances,
and receives a certificate. Because ISO 9000 is relatively new, the organizations
who share their experiences here are the scouts who caught the arrows while
blazing the trail. You have the advantage of learning from their experiences;
but be forewarned that you will have nonconformances too. The path to process
improvement is never easy.
What happens during audits
What gets inspected during an audit depends on your type of business, your
industry, and the scope of your practices.
Most of the organizations interviewed for these articles, while being part
of large corporations, are relatively small sites of 100-200 staff. As a
result, they typically hosted two to three auditors for three to four days.
If the company sought a TickIT registration, the audits took longer.
Be prepared to have your auditors arrive early the day your assessment begins.
During the opening meeting, the auditors explain the purpose and scope of
the audit, and present an agenda of the areas they wish to cover.
Next, the auditors spend an hour or two reviewing your quality manuals.
They then split the various functions between them and take off with their
guides to examine the processes within those areas. One auditor may visit
the department handling contract review, while another may start with quality
assurance.
Essentially, auditors look for consistency between what's written and what's
in practice. A good auditor will investigate a point, find out what the
issue is, and ask for a response before asking you to fix it. Auditors will
re-word questions to ensure understanding. The object is not to catch you
but to verify the process.
"During an audit," states Richard Sisk, PE Nelson. "They
ask you to describe your operations, where you get your directions, and
what do you do when there's a problem -- not "what does paragraph 2,
section 10 say?" They ask, "if you don't know, where can you find
out? Where is that information?"
"They question everyone," he continues. "They asked our Division
Manager about his business plan, what his form of communication is, whether
there were minutes, what issues were covered, and if there was evidence
of closure on action items. The auditors were also looking for a clear definition
and statement of responsibility and delegates."
"We found that the auditors chose a project and audited that project
from start to finish; they don't wander the halls aimlessly," Varian's
Fred Klink recalls. "During the pre-assessment, the auditors found
a couple of nonconformances in operations. We had written it down and were
doing it, but hadn't documented that we'd done it. Also, in R&D we found
that part numbers had occasionally been written down in pencil. That's a
no-no."
"Auditors have the knack for finding the weakest point," explains
Geoff Belton, Fisons. "We found that they looked closest at quality
control reviews for documented evidence and evidence of planning and reporting.
They also wanted to know how we handled corrective actions."
Your auditors will use a checklist to track the clauses that they review
in each area, checking off nonconformances and making observations in each
area. Toward the end of the afternoon, the auditors assemble in a closed
meeting to discuss their findings. They'll be comparing what they've found
and looking for trends. You will also meet with them at the end of each
day for a summary of their daily findings.
When the audit is complete, a final meeting is held to announce the results.
Typically, you're passed with qualifications and given a set period of time
to address the nonconformances. If you fail at this point, it will generally
be caused by a significant collection of minor nonconformances. The auditors
will typically stop the auditing process immediately when a major nonconformance
is discovered to return at a future date when the problem is fixed. You
have the option, however, to ask them to continue if you wish to uncover
other areas of weakness.
Nonconformances
How many companies pass an audit the first time? The number varies depending
on your source from as low as 30% to as high as 75%. It also depends upon
your interpretation of "pass".
"At the end of the audit," states BSI's James Davies, "there
will be one of three recommendations: One, unqualified registration. We've
registered few companies with no nonconformances. Two, qualified registration.
Usually there's a collection of minor nonconformances which the company
has 30 days to address and provide evidence of corrective action. Three,
fail. Companies usually fail due to a collection of minor nonconformances
or one major nonconformance. Some companies have passed with 60-70 minor
nonconformances, but it depends on the type and scope of those nonconformances."
Fisons, PE Nelson, and most of Beckman's, Varian's, and Hewlett Packard's
divisions passed the first time. With the exception of Thermo Separation
Products who is in the midst of implementation, everyone interviewed is
currently registered to the standard.
"We had been under the impression that if we failed, we'd have to do
it all over again, not just where the nonconformances arose," Mettler's
Walter Kupper adds. "In fact, we were given 90 days to fix our nonconformances.
If it's a minor nonconformance that can be fixed by correcting the documentation,
all you need to do is send evidence to the auditor by mail that the nonconformance
is fixed. If it's a major nonconformance, the auditor comes back for reinspection
of that particular item. It reassured us to know that we couldn't flunk
the process."
According to Davies, "The best audits happen when you've got well-prepared
guides, the staff is briefed on what to expect, and the auditors don't find
any major nonconformances. Eight to 10 nonconformances is a good audit.
But it's not unusual to see 100 nonconformances if the site has 500-1000
employees and it's a long process."
Of the two types of nonconformances, minor nonconformances cover issues
such as not signing forms. Major nonconformances include not implementing
a process, not writing down what you've done, or writing "not applicable"
on a clause that does apply. Major nonconformances can also be a collection
of 60-70 minor incidences in the same area, such as document control.
"Our final audit didn't uncover any discrepancies in the level three
procedures," Waters' Marq Ransom observes. "They did find weaknesses
with the management review procedure because there was no history. Now we
have monthly reviews by our QAG teams. This was the only thing they checked
during their follow-up visit."
When Bio-Rad was audited, John Goetz points out that the auditors zeroed
in on training issues. "We found it difficult to assess the qualifications
of the trainer and lacked documentation to back it up. One of the key questions
was "if you were trained, who trained you?"
"We had a chapter in our manual for Internal Audits but we hadn't implemented
it," states Kupper. "This hadn't come out in the pre-assessment
audit, so we weren't prepared, but that's due to the random nature of audits
as to what they find and when. It gave me a new perspective on ISO -- that
internal as well as external enforcement is an important component of the
quality system and that this is what puts teeth into the process."
"Calibration!" declares Jim Quirk, Beckman. "Things were
not calibrated, or out of calibration, or things that should have been calibrated
weren't. The registrar wanted a record of all the details, whether in spec
or out of spec, not just a note stating "in spec". Only two groups
didn't pass the first time. They didn't have proper control over returned
goods. The biggest point is do not document something you're not doing;
you've got to be doing what you say you're going to do. Implementation is
key!"
No other company mentioned being grilled about returned goods. In fact,
that seems to be a relatively insignificant procedure when compared to the
importance of other processes. Beckman's experience points out, however,
that there are no insignificant procedures.
If something is not examined during the original assessment, it will be
looked at the next time. "Nothing escapes," stresses Will Cowan,
Hewlett Packard. "I'm sensitive to corrective action and nonconformances
because it's difficult to get all the required signatures when you have
hundreds of documents. The good news is that lately our auditors couldn't
find any noncompliances so they were looking at things like page numbers.
Now we only have minor errors such as occasionally finding an old procedure
on a wall."
When all else fails
Chris Rew of Bio-Rad brings up an interesting point. "You can argue
like hell with registrars to defend your procedures because one of the problems
with the standard is that it is not looking for efficiency or effectiveness.
Remember that you are the final decision-maker on how to run your business,
and that's why it comes down to interpretation of the standard. If you don't
agree with what the registrar says, you can actually send their decision
to be arbitrated."
Follow-up audits
Certification is good for one year to three years depending on the registrar,
with surveillance audits occurring one to four times a year depending on
the results of the initial assessment. In follow-up surveillance audits,
each time an assessment is conducted, certain clauses of the standard are
always examined. The remaining clauses are covered over a period of one
to two years of continuous assessments so that there is a complete system
reassessment over time.
"All we do now for surveillance audits is refresh our memory and keep
the binder area up-to-date," adds Klink. "There's not a lot of
detail to remember because we do process improvement all the time."