Many times, the lessons we learned as youngsters are remembered for life. I recall trying this old standby on my father: “It’s not fair!”
“No one ever said life was going to be fair,” was the even-measured retort.
Dad didn’t have to say it too many times before I figured out that my well-thought-out objection to the injustice I had experienced was not the most enlightened approach.
It’s Not Fair
Many times, customer satisfaction index (CSI) and net promoter score (NPS) results are unfair. They come from people, and people, by definition, are flawed. We (the consumer) are ill-informed, ignorant, biased and uneducated (at least about the collision repair business). Some – not many – are intentionally spiteful.
Due to the fact that customer satisfaction scores (CSAT) often impact our wallets, our relationship with business partners (insurers, OEMs) and our pride (our sense of fairness), the results can be a jagged pill to swallow. And while it’s difficult to divorce ourselves from the emotion of the slight, best practices would suggest we attempt to do just that. One concept that may prove helpful is that of the “false negative” and “false positive.”
False Negative/False Positive
Over many years and millions of surveys, I’ve yet to have a client call or e-mail me and ask for a survey result to be lowered. I pretty consistently get “change” requests or, more often than not, just a request to throw a survey out because the customer was “wrong.”
Taken at face value, there is no doubt that customers are sometimes wrong. However, if they’re wrong on the negative side, would it not stand to reason that there are just as many (I would argue more) artificially high scores infiltrating the system? Put another way, if lack of education hurts you, can it not help you? What about ignorance and bias? Does that road only travel in one direction? A strong case can be made that scores are more likely to be inflated than deflated. Consider these three points:
- Most people do not enjoy conflict. In general, they want to have pleasant interactions with others and, unless something egregious has occurred, will err on the positive side.
- Others are just in a hurry and realize the more negatives they give, the more survey questions may be asked of them. “They weren’t bad – let’s give the top box score and get on with it.”
- A high percentage of customers are not knowledgeable about the technical aspects of collision repair. As long as the car looks good and drives well, undetected mistakes will go unnoticed.
As to the third point, a personal example takes me back to the days when I owned a bright red Toyota MR2 Turbo that was vandalized in an ill-lit parking lot one night. It required some pretty extensive repairs that included a paint blend. In natural light, the car looked good, and I took delivery of the vehicle and was happy with the result. A week later, pumping gas under fluorescent lights, I was stunned to see the inconsistency in the color and blend. Back to the shop! Certainly, my CSI result would have been different had I been surveyed one day before or after pumping gas that night.
Aside from more formal measurement programs, there are any number of options for consumers to rate and review online as well. Some review sites have had credibility issues due to their inability to verify that the user was actually a customer. Or, that competitors or bots are not artificially pumping positive or negative information into content, which, of course, impacts your company’s online brand and footprint.
In addition to customers themselves, nefarious competitors or others seeking to do you harm, some platforms have come under scrutiny with accusations of favoring advertisers and penalizing non-advertisers/participants. Getting a fake or false review off a site can be nearly impossible.
As these challenges become more widely recognized by consumers, the information is increasingly viewed with some level of skepticism. Similarly, if owners, managers and staff understand the concept of false negatives and false positives of CSAT programs, we can begin to view the data from a fresh perspective, while still appreciating the substantial impact.
All platforms and systems have flaws and challenges, but in my experience, the vast amount of information contained in what might be called closed systems – or platforms that are immune from the impact of bots – is overwhelmingly accurate.
Play the Game
In discussions that hit particularly close to home (and the wallet), sometimes it’s useful to use examples from an industry we’re all familiar with but is not quite our own. So let’s pick on automotive dealerships.
The previously mentioned example of my Toyota M2 Turbo illustrates the concept of the false negative and false positive in what would have been an unintentional, inaccurate result. However, due to the stakes involved, we all know that some attempts to alter results are far from unintentional. Stories of bribery, data manipulation and other shenanigans are legendary in the dealership world. Giant representations of the survey sit in the service area, filled out with all positive results. And the attitude is, “If you don’t give me a 10, it’s like giving me an F. There’s no in-between.”
If coaching doesn’t work, there are various ways to eliminate the customer from the survey sample. Business has been turned down and turned away due to the fear of receiving a poor score. Sales have been intentionally sabotaged – customers have been turned away and run off. Let that sink in for a minute. Sabotaging your own business! More often, phone numbers or e-mail addresses are changed (it only takes one number or letter) so that the customer doesn’t receive the text, e-mail or phone call from the survey company.
Let me pose this question to you: Would you manipulate your financial data? How would that go over with your accountant, or, worse, the government? Would this explanation work: “Well, see, I didn’t really want to include those debits on the valuation report because they really weren’t fair in my estimation.”
What’s more important to your company than customers? Without customers, you don’t have a company. So should their feedback, even the stuff we don’t agree with, be treated with any less reverence than your financial data?
There is a lot of frustration at the dealership level from the fact that every survey counts, no matter if a crazy person filled it out or not. These surveys carry a lot of weight with manufacturers and dealers, but some dealer employees view them as injustices and an unfair way to measure their performance. I have to admit that this frustration is disheartening, even on the dealership side – because certainly there is some of this sentiment in collision as well.
Hopefully, while experiencing some level of unfairness, the majority of owners, managers and employees understand the intent of trying to make sure customers are satisfied and therefore the value of companies wanting to measuring their level of satisfaction.
Appropriate Program Application
It’s worth noting that valid measurement tools such as scales cannot in and of themselves be ridiculous or unrealistic. It’s the application of the scale that causes the heartburn. This may seem like a nuanced difference, but it’s actually at the crux of the difference between a powerful, motivating, empowering process and a debilitating, morale-sapping program that encourages system manipulation and cheating. It’s the difference in utilizing CSAT information as a management tool and integral part of your overall business philosophy of empowerment, instead of a club to beat on a negative experience.
Owners and managers have the discretionary authority to determine how much credence to give a particular result.
“But wait, I’m being measured for a DRP or by an OE!” Certainly, that may be the case for some of your ROs. But, ultimately, you are in charge of that relationship and whether or not you want to be in it. It would be in your best interest to be intimately familiar with how the CSAT portion of the program works so you can have intelligent and rational discussions with your business partners about it.
If your relationship with those external client business partners is built on knowledge of the program and shared goals, the mutual trust that results will ensure success, despite the occasional false negative.
Having a process in place that flows from your strategic plan daily to your vehicle owner-external clients and at least weekly to your business partner-external clients is essential to effective program application.
Summary
CSAT can have a dramatic impact on the culture of your organization. Utilized correctly, it can be a great force for good – and much of that starts with how it’s framed, explained and executed from the top.
No, it is not always going to be fair. But, with leadership that views the process holistically, it can be a lot more fun.
