Q: Do you ever call in and pretend you're a customer to test your customer service? A: No, I don't do mystery shopping. Q: Why? Is it deceptive? A: No. I would advocate calling in like any customer if you need a product or service, but there are other ways to get the information. Q: How do you accurately measure consumer satisfaction? A: I use many internal measurements and match them to external measurements. We know how many (complaints) per 100 customers that we receive and whether it's going up or down. We measure billing errors. We're also into surveys that are not burdensome (for the customer to answer). Some of those are automatic and can be done with a voice product. Customers can say if they've had a good experience or bad, and we can get those immediately to the employee involved. We constantly compare internal measures to those done externally to be sure we're getting an unbiased view of what's going on.
Sam Walton knew better. Employees pressured to make numbers "look good" will always find a way to do so. My observations from many people at many companies is the more "bean-counting" you do, the less useful the count will be. The basic problem with "counting beans" is that what you're counting isn't the actual product (customer satisfaction), but merely an attempt to quantify the product. Once, long ago, while watching this principle in operation, I was moved to obsserve that if you reward employees for getting blue beans and punish them for getting red beans, they'll bring in cans of blue paint and spend extra time painting the red beans blue. I give him points for attempting to measure outside the system, but the difference between being first-rate and second-rate is the difference between being told how well the system works and actually enduring it yourself. Surveys and other such devices are good for capturing the extremes, but people who are resigned or merely irritated generally won't waste their time on them. I'm not a big fan on being "told" things anyway. When you hold the whip, people tend to tell you what you want to hear.
An IDE is no substitute for an Intelligent Developer.
It's been my experience that companies aren't really all that interested in knowing what their problems are. If they can report day after day "Mission accomplished", even when it's not, then they can continue to justify their worth to stockholders and the senior managers can continue to rake in their rewards. Which is ironic because that pretty well describes the principle you just described!
There's a big difference between your bean analagy and the validity of using surveys as a method of measuring customer satisfaction. A previous job of mine involved countless statistics for determining productivity. Unfortunately, the stats that made an impact on individual salary (and keeping your job) were... bean-like. Our entire department, however, was evaluated with customer surveys. We couldn't paint customers blue (although at times we would have liked to ) Surveys are only valid if those being rewarded/punished are not the same people that organize the survey's statistics. That would be a conflict of interest.
Surveys are only valid if those being rewarded/punished are not the same people that organize the survey's statistics. That would be a conflict of interest.
One of the best books I've read is Measuring and Managing Performance in Organizations. An urban-legend: (page 5) The Soviet iron factory was measured by how much weight it produced, so they created rail road spikes, the heaviest item they could create. One day the measurement changed to number of items produced, so the next day, the started producing small nails. There are two categories of measurement (page 21) - motivational: to alter the behavior of those being measured - informational: guides organizational and process improvement
When I monitor employees I make it clear to them whether or not this information will be used in evaluating them (90% of the time, it won't be used). For example, when I first joined this company, I asked all employees to track their time and go over it with me. The purpose was for them to realize how much time is wasted with multiple interruptions throughout the day (a problem before I joined this company). I did not, however, want them to think I was tracking their hours, so I made it very clear this was not my goal. I also had them track their time, and I only looked over the sheets with them, I never kept copies of it. In Peopleware DeMarco and Lister say something similar: don't shoot the messenger. People need to feel safe reporting bad news, or they will choose not to report it. This is illustrated by two examples from the metric's book. In 1991 Dick Cheney, Secretary of Defense canceled the Navy?s A-12 attack plane program because it was behind schedule, overweight, and would cost an addition $2.7B. 6 months earlier he had been told it was on schedule and under budget. An inquiry found the development team had been pressured y upper management to maximize cash flow, which meant BS-ing the government with optimistic reports as long as possible. (page 12) In 1882 the USS Omaha was build with a coal room big enough for only 4 days of travel. This problem was never elevated to a sufficient level in order to stay on schedule. The cost of fixing the problem cost was much as an up-to-date steel ship, and it ultimately was not a match for any other nations warship. (page 13)
Realworld example might be the QA department who are measured by the number of bug reports they submit rather than the number of valid bugreports or the number of bugs they do NOT find (which is the real measure of any QA department). I've encountered such a situation for real. The QA department submitted hundreds of nonsense reports just to get their quota in, often submitting the same problem several times a day and elevating simple typos in screen texts or positioning errors in fixed texts on printed reports to release critical status (the more serious the problem, the better they were evaluated to be). All that stopped when we started billing them for every non-issue reported and for every duplicate issue reported over a certain number a month (to account for some margin of error in judgement on their part).
Dashboards are at least a decade old, maybe older. Only the technology has changed. Of course, just because you can get a number doesn't mean that it's the number you want, that it's accurate, or that you know how to act on it. :-) --Mark
Joined: May 15, 2002
But, perhaps, it will be immediately apparent that one doesn't know how to act on the figures. All the offshoring, followed by diversions seems like madness run amock. Now there is a new term - smart sourcing.
Truthfully, I think the problem begins when the company gets too big. I can tell you as a customer, I've seen more good businesses ruined service-wise by growing than for any other reason. It seems almost axiomatic that you inherit the lowest common denominator of all the businesses you acquire. A crying shame, since as I understand it, we've gone from a Farming Economy to a Manufacturing Economy to an Information Economy to a Service Economy. Please stay on the line. Your call is VERY important to us. ACK! Myself, I favor the "Management-by-walking-around" style. Things where you know what's going on because you actually see what's going on and don't depend on lackeys and handlers to tell you things. Because they won't. In theory, they're empowered to fix things themselves. In practice, they see themselves as being denied the goodies at the top, limited in their ability to make improvements and the fall guys when blame gets passed out. And in today's climate of hyper-compensated executives and falling line salaries, there's a lot of room for resentment. You can get by using BellSouth's techniques if all you care about is "getting by". Until someone comes along who actually does make their employees feel like it's their company and that what they do is valued. Then you'll panic at shrinking market share, hire overpriced Dogbert consultants who'll tout some silver bullet solution, lay off your most competent (e.g. least submissive) employees and make the problem worse. But then again, this is a phone company. Their competition is other phone companies. Can you tell I don't have a lot of respect for phone companies? "For an interminable wait listening to brain-dissolving Muzak, press 7. To be hung up on abruptly, press 9". ACK!