Invalid survey?

MrCreosote

A VP brought in a consultant and one of the programs he has introduced is an internal survey, where employees rate other departments based on service provided within the company. He said that 90% of the time, IT and Accounting get the worst results. Seems to me that it must be an invalid survey if the results are consistently skewed. It's being used to measure something it can't measure, services and help that many people are not aware of. If your computer works, for example, you don't pause to thank IT, you just let them know when it isn't working. Do you think the survey must be invalid or are accounting and IT people just flawed and damaged people?

Don D

Do you think the survey must be invalid
>or are accounting and IT people just flawed and
>damaged people?

While it is true that IT and Accounting people are inherently flawed and damaged, I agree with your premise about the survey. I think all surveys are basically meaningless unless one forces oneself to attach some meaning to them. People who respond to surveys typically know very little about the subject matter, or if anything at all, just a fact or two. They base their response on very little information or on how they feel at the precise moment about the subject. Like 'How do you feel about the new Mercedes horn?' If I heard it from a distance only once or twice I might rate it a 10, really a crisp, neat sound. If I live next door to the factory that makes them and they drive me up the wall, I would respond at the other end of the spectrum, as with Ray's trombone practice. Your theory is along the lines of 'what have you done for me lately?' If the department has pleased me lately, they get a good survey response. If my check has been late or my computer is really slow, they get a low rating. If I hardly know they exist, a mediocre one. Meaningless. x:-)

ray a

I agree with Don, especially about my trombone practicing. But, it could be worse, I could play the harmonica.

When I first went into HR, I thought doing surveys was neat. Then I woke up and realized that often the information was useless for the exact same reasons Don gives.

SMace

I agree with Ray and Don, sort of. Survey info can never be rock solid because it relies on individual interpretation. But I don't think you can say they are totally meaningless. If you take the time to look at the information and temper the responses with reality you should be able to get something out of it. For example, 90% of the respondents say IT is too slow to get them help. The reality may be hey, we are understaffed and doing the best we can. Without the survey IT doesn't know 90% of the people think they are slow. They can address that perception and put a focus on getting to people quicker. Although the data was flawed they got something out of it.

marc

I have been one of the flawed ones for most of my career. I would add that many surveys use these rating scales that have a different meaning for each person. Some people cannot give below a 4 or a 5 on the 1-5 rating scale with 5 being the highest and 1 the lowest. They are incapable of accurate evaluations for a lot of reasons. Anyone that has trained supervisors on the proper use of a rating scale understands the proclivity to rate in a way that avoids confrontation. They want to be friends, etc. Thus they miss the opportunity for meaningful feedback.

You find a similar problem on the other end of the scale with people who just cannot rate above a 3 - saying that is doing the job and nobody can give 110%, etc.

Maybe these are simply pessimists and optimists, but their views are rated the same. Also, many of these surveys are anonynmous - and when anonymity is possible, you get the wildest swings - people who do not have to prove their point with specifics can say anything.

All in all, I don't put a lot of weight on survey results, and with that in mind, can use them for the tepid value they give back.

blw

I think maybe part of it is that from those two departments we want "immediate response", especially IT.

If I have a three day turnaround on a request from purchasing or HR I might be okay, but dangit...I want my computer fixed this afternoon and no later!