Monday, December 4, 2006

The Quality Conundrum

“Quality” is best measured by those who “use” a product rather than by those who make it. ~ Hunter S. Thompson

Sony is making another recall, a minor one, involving camera display screens, which do not appear to be catching fire or anything. Sony, as everyone well knows, has been shooting itself in the foot with an AK-47 over the last few months. First there was the rootkit fiasco. Basically, Sony DRM-protected CD's required that you place software on your machine to play them. That software, it turned out, behaved in a manner that allowed system files to be replaced, not only by Sony (which they denied was ever their intent), but by any hacker who knew where to look.

After that shot to the old corporate image, there was the battery recall that impacted a wide swath of laptops, including Sony's own Vaio. The problem was tiny little metal bits in the battery that ultimately caused the battery to overheat and rather spectacularly burst into flame. What made the situation even more damning was that Sony evidently was aware of the problem and even discussed it with Dell (who got the lion's share of these fire bombs).

None of this is particularly newsy except that in one of the references to the most recent recall, someone made a reference to the Sony's quality control people not doing their job.

As a former quality professional, I take umbrage at that sentiment. It raises my hackles -- and you know how painful that can be.

I started in quality control in 1974 and kept at it until I was “re-engineered” out of a job in 1994. To show just how stupid I was, after getting the boot in '94, I actually tried to find another job in quality for a couple of months, before I had an epiphany that it might actually be time to do something I enjoyed doing. If you wonder why it took so long, I supposed it's rather like the guy who's banging his head against a wall. When asked why he keeps doing that, he responds, “Because it feels so good when I stop.”

Boy, has it felt good.

At any rate, I wanted to make a point about how things like the Sony battery can happen. I don't know that this is what actually went on, but given the discussions with Dell mentioned above, I think I might be on the right track.

Quality Control departments don't catch everything that isn't made correctly. There are two reasons for this. First, most of them time it's prohibitively expensive to inspect every single part. With automation techniques, it is sometimes possible to test each product, but even those tests would not detect a problem that takes time to develop. Second, Production people will try to hide defective product. Yes, they will. The problem is that their goal is to generate quantity, and stopping to inspect everything or, worse, remaking a large quantity of parts due to a rejection gets in the way of that goal. For all the proud words companies spout about the importance of quality, their production supervisors know full-well that they will get a mild chastising for bad products but big rewards for making high production levels.

If it wasn't like that, you wouldn't need quality control departments in the first place.

So, what I'm saying is that sometimes, stuff gets out that shouldn't have. Sometimes, though, the quality people discover a defect issue and notify management, which proceeds to make a “management decision” to ship the stuff anyway.

Now, I can tell you in all honesty that many of these decisions are okay. Customers will over-specify products with a vengeance. When I worked in the rubber industry, I used to spend a lot of time trying to explain to customers that rubber parts are not like steel parts. Most of them are squishy. You can't hold the kinds of tolerances on a squishy material that you can a rigid one. Many's the time I decided on my own to let a washer go which was slightly too thick because I knew how the part was used, and I knew that being .001” over spec wasn't going to cause a problem.

But, I've also been a party to some bad decisions. At one company, I was overridden by a sales manager three times and by the president once. None of these involved life-threatening characteristics, but functionality could have been impaired in each case. Each one of those shipments was rejected by the customer. After a while, they stopped overruling my calls.

For the record, I often went to the sales manager to see if a customer might accept a condition, but on those occasions, he would call the customer and get a waiver. Those three occasions where he overrode me all came on the same day when the company was trying to generate some nice year-end shipping numbers.

Sony knew about the problem; their big customer Dell knew about the problem. The implication is that the quality people did their job, and the managers made one of those “decisions.”

In all the years I was in quality, I was with a company that had to deal with one of those situations (a really serious defect that could cause injury) only once. A defect was discovered during routine testing that showed that a manufacturing procedure had not been followed properly. To that company's enduring credit, it spent around $400,000 testing parts, remaking parts, revising procedures, and certifying production employees involved in the key process to feel sure that none of the defects had gotten to customers and to ensure that the problem didn't occur again.

I don't know how much it would have cost Sony to toss those defective batteries, but it would almost surely have cost a fraction of what recalling and replacing them cost, not to mention the cost of the bad press. But, I'll bet if you go looking for the people who let the situation get so far out of hand, you will find that the quality control group had been screaming like their hair was on fire long before “management decisions” were being made.

Quality control people have a tough job; they're the bad guys most of the time and get little or no credit when things go well. At one company, we were running one of those bad patches where a department was generating a lot of bad product. Our inspectors were identifying the problems, but the production supervisors just continued to run the jobs, with the result that every lot had to be inspected or reworked and a lot of stuff had to be remade. Finally, my boss and I held a council of war. We considered arming the inspectors but decided that bloodshed wasn't the answer. Eventually, my boss got the president to issue strict instructions to the head of Production that all shutdown notices from inspectors were to be honored immediately. Anyone failing to do so would be subject to disciplinary action.

Well, lo and behold, scrap and rework went way down, processes got corrected, and, just as the quality people keep saying, productivity went up. So who gets the credit? The president sends a letter to Production Manager praising him and his supervisors for the great job they did – said “great job” consisting of following the procedures they had been violating. The Quality Department? We got squat recognition.

You know the worst part of that story? It happened around 1976, yet I was dumb enough to stay in that profession for 18 more years. Oh, well, it could have been worse.

I could have stayed in quality and gone to work for Sony.

No comments:

Post a Comment