Performance Appraisal Questions - Need your input - 5 questions

cbeetel

Hello,

I am working on a project for a client and would like to gather the following information.  Please respond if you have time.  Thanks in advance.

1. When using a rating scale for performance reviews do you use a number rating (1-5) or a letter rating (NI for needs improvement, for example). 

2. If you use a numerical scale, what are the pros and cons to it?

3.  Do you allow increments or only whole numbers?

4. Are you using a 3 point scale or a 5 point scale?  If so, why?

5. Are your reviews held on the employee's anniversary date or annually company wide (same time each year)?

 Any additional comments?

 

 

 

TXHRGuy

[quote user="cbeetel"] I am working on a project for a client and would like to gather the following information.  Please respond if you have time.  Thanks in advance.

1. When using a rating scale for performance reviews do you use a number rating (1-5) or a letter rating (NI for needs improvement, for example). 

2. If you use a numerical scale, what are the pros and cons to it?

3.  Do you allow increments or only whole numbers?

4. Are you using a 3 point scale or a 5 point scale?  If so, why?

5. Are your reviews held on the employee's anniversary date or annually company wide (same time each year)?

 Any additional comments?[/quote]

1. When you use a numbered scale, you are implicitly agreeing that the dimension being measured is a bipolar scale with no important sub divisions to confuse the rater.  For example, "are you a liberal or a conservative" will generate a lot of noise in the data because a) the terms do not relate to widely agreed upon definitions and b) the terms break down into multiple dimensions (meaning it's not simply bipolar), such as fiscal and social dimensions of  conservatism and liberalism.  The better your breakdown on the dimensions being measured, the more appropriate using numbers are.  Using letters or words that describe a continuum such as "unacceptable - needs improvement - average - above average - exceptional" has the same effect as using numbers.  The best reason avoid numbers is when you have a concept with multiple dimensions and a reason not to break it down into its constituent components.  I can't actually think of a time when both of those things has applied in my own work or research.
2. Easy to measure.  If you aren't going to use a numerical scale or something that essentially translates into one, then you probably should either spend more time on test design so that you can or use open ended questions.
3. Whole numbers only.  See anser to number 4.
4. People generally are not able to distinguish among more than 7 options.  More variation allows for more "variance" in the statistical sense, which is desirable, so using 7, if you can describe 7 points, doesn't hurt anything and may help but going beyond 7 generally creates random noise.  That is, people are better at distinguishing between 4 and 5 on a 7 point scale than 4 and 5 on a 10 point scale.  The smaller the odd numbered scale, in general, the more likely you will get neutral answers to questsions (2 on a 3 point scale, 3 on a 5 point scale).
   
   Don't leave out the "forced choice" format with, typically, only 4 options, "unacceptable - needs improvement - above average - exceptional".  If everyone is at the top of the scale or in the middle, forced choice will help break the deadlock and make supervisors really think about employee performance.
   
   If your supervisors are well trained, I like a 5 point scale.  I use forced choice if they aren't well trained or, despite training, tend to rate optimistically or give everyone "average".  One problem you can run into is a selection bias and its effect on raters.  For example, everyone at MIT's graduate school of electrical engineering is above average.  If you get a "C" there, you may actually b an "A" student if you had been at Backwoods U o' Technical Stuff.  Are they being rated against peers, or against the available workforce?  You need good, experienced supervisors to do the latter.  Whatever you do, you should say what they're rating against and then interpret the data that way.
5. I've never been able to pass this idea, but I've always wanted to do quarterly performance appraisals and do a comp review at every fourth performance appraisal.