UI LEAD: Information for Raters
Our goal is to support raters in providing objective and constructive 360-degree feedback.
This information is regarding your role as a rater, in responding to your colleague’s invitation to complete a survey on his/her leadership competencies.
Please review the following before completing the 360:
- What is a 360 and why is it a part of UI LEAD?
- Where and when do I complete the 360?
- What if I don’t get the email?
- Whom do I contact with questions?
- What is my role?
- How is confidentiality maintained?
- Standard deadline: 2.5 weeks.
- During this time, you can exit and return to the survey. The Envisia Learning system will save your responses.
- Automatic reminder: Envisia will send a reminder email one week before the deadline to all raters whom have not yet completed the online 360.
- Planning for the future – Develops clear priorities, objectives, and steps to be accomplished in future work.
- Organizing and orchestrating events – Lays the foundation for work by bringing together resources, tools, schedules, and people as needed.
- Listening to others – Gives careful attention to what others have to say. Plays back information to make sure that others’ messages are understood.
- Refers to a tendency to form a generalized positive impression of an individual, and rate him/her highly on all rating criteria even if s/he doesn't deserve a high rating for every one.
- The result is that the ratings become inflated, and inaccurate, at least for some items. The problem is that an individual may receive feedback that does not help him/her improve.
- The opposite tendency is called “the devil effect bias” when the rater generally dislikes, or has little confidence in an individual, and tends to rate him/her as low functioning on all or most of the rating items.
- Describes the situation where a rater tends to be more lenient than his or her peers, when rating an individual, OR, is more lenient with one as compared to another.
- since areas for performance improvement tend to be ignored or swept under the rug in appraisals or assessments.
- The opposite tendency is called “severity bias.”
- Also known as “average rater error” occurs when raters evaluate an individual as “average” when they apply a rating scale.
- For example, given a scale that runs with points that range from one (poor) to seven (excellent), with four being the average, many raters will refuse to use the points at either of the ends. There will be a tendency for almost all ratings to fall within the 3-5 range.
- There is a tendency for some people to focus on "what's happened lately" when evaluating or judging something. This may also be true in performance reviews.
- Be aware of the possible biases described in this tutorial.
- Address each behavior assessed as independent from all other items.
- Consider the range of experiences that you have had with this individual, not the first or the last.
- The individual may have changed his/her behaviors in recent months, as compared to the entire length of your acquaintance. If this is the case, assess the new patterns of behavior.
- If have not observed the behavior in question, select “N/A.”
- Provide objective and constructive feedback on written comments to exemplify your views. (Again, comments will be reported as written.)
Rater FAQ's
Rater Education Tutorial
Rater FAQs (Frequently Asked Questions)
What is a 360 and why is it a part of UI LEAD?
A 360 is a way for multiple individuals, with different perspectives, to share feedback with a colleague related to his/her skill, performance, and leadership. Examples of raters in a 360 assessment include direct reports, team members, peers, and managers or administrators.
Learn More . . .
UI LEAD recognizes that individual development leads to organizational success.
"The better designed development initiatives employ structured 360-degree feedback based on responses from workplace colleagues....Learning is enhanced whenever individuals have some form of behavior feedback..." (Jay Conger, Marshall School of Business at The University of Southern California and author of Learning to Lead)
When and where do I complete the 360?
After your colleague (the UI LEAD participant) announces his/her upcoming 360, you will receive an email from Envisia Learning within one week’s time.
To: (your name)
From: Envisia Learning
Subject: UI LEAD Participant - Feedback Questionnaire
This email will contain directions and a link to access the online 360.
Before you begin the 360, please continue reviewing this page, including important Rater Education information.
What if I don’t get the email?
If you don’t recall receiving the email from Envisia Learning within one week of your colleague’s notification, please check your deleted folders. If you need another invitation sent to you, email clientservices@envisia.learning.com or barbara-simon@uiowa.edu
Whom do I contact with questions?
If you have any technical questions about this online process, contact Envisia Learning at 800-335-0779 or email clientservices@envisia.learning.com or barbara-simon@uiowa.edu, Organizational Effectiveness, 335-2260.
If you have questions about the use of 360-degree feedback within UI LEAD, contact Nikole Mac, Leadership Development Coordinator, Organizational Effectiveness, 384-0535.
What is my role in providing feedback?
Your role is to provide honest and constructive feedback to aid in your colleague’s leadership development.
“The 360-degree feedback process allows people to become connected, and, in so doing, it facilitates the building of effective working relationships around work-oriented communication.” (Tornow and London, 1998)
In UI LEAD, participants carefully select their raters in consultation with their supervisor/faculty administrator, and their UI LEAD advisor. If you have been selected to participate in your colleague’s 360, it affirms that s/he considers your input valuable to his/her leadership development. While your participation is voluntary, we encourage you to honor this invitation by completing the 360.
Your feedback will be added to the feedback provided by other UI faculty and staff members to assist your colleague and UI LEAD advisors to develop an individualized talent development plan.
“Coaching is important in helping appraisees interpret the results, set improvement goals with appraisees, and follow-up on action plans.” (Antonioni, 1996)
Please note that if you are a supervisor or faculty administrator of the participant, your 360 responses will be provided separately. However some participants may report to multiple leaders; in these cases feedback is aggregated. You will receive an email from UI LEAD indicating either case, informing you of how your feedback will be reported.
More information is available about your supervisor/faculty administrator role in the overall UI LEAD process. Learn More. . . .
How is UI LEAD maintaining confidentiality in this process?
UI LEAD understands the importance of respecting the confidentiality of raters, while also providing an opportunity for direct feedback from supervisors.*
To this end, we have selected a vendor partner, Envisia Learning, to provide an online process which protects the anonymity of individual raters. Responses and written comments (reported as written) are aggregated, meaning they are batched together by rater group, such as peers, direct reports, or team members. Each rater group, with the exception of one’s supervisor/administrator, has a minimum of six raters.
Using these aggregated responses, Envisia creates a report which will only be shared with the participant and his/her UI LEAD advisor. In this report, response rates are provided, indicating the percentage of raters who have responded to the questionnaire. However, the identity of individual respondents is protected, with the exception of one’s supervisor.
*Please note: Supervisor(s)’or faculty administrator(s)’ responses and comments will be reported separately. Our experience and research in leadership coaching have affirmed that direct feedback is imperative to one’s growth and development. If you have additional questions or concerns, please contact the UI LEAD advisor with whom you have been working, or call Nikole Mac, Leadership Development Coordinator, Organizational Effectiveness, 384-0535.
“In most cases, supervisor ratings, as opposed to other sources, are not confidential. In other words, the feedback recipient is aware of the ratings given by the supervisor because these ratings are not aggregated with other raters.” (Tornow & London, 1998)
A written comments section will be available. Please provide honest and objective comments, while keeping in mind that your words will be reported as written. Comments are most valuable when they provide examples to make your point more explicit. Carefully review your written words before finalizing them. A good rule of thumb: “Is what I wrote truthful, useful, and kind?”
“Feedback recipients are less likely to react negatively if they believe the feedback is accurate. This is particularly important when open-ended comments are included.” (Atwater, Walderman, Brett, 2002)
Rater Education Tutorial
Research and best practices suggest that raters should be aware of errors and biases that naturally exist. UI LEAD recommends that you complete this online rater education module before starting your colleague’s 360.
Sample Questions and Scales used in UI LEAD 360s
The online 360 survey will ask you to rate your colleague on a series of leadership behaviors, or competencies. The number and type of questions will vary depending upon which of the Envisia assessment products that your colleague, the UI LEAD participant, has chosen.
Articulating ideas and information – Speaks clearly and expresses ideas so that others understand what is meant.
©2003 Envisia Learning.All Rights Reserved.
Evaluating others’ competencies can be a challenge, and 360 developers strive to create tools that promote greater accuracy in assessment. One case in point is the prevalent use of the Likert-type scale for rating people on leadership behaviors.
The following is an example of such a scale which you may see in a UI LEAD 360:
Frequency Rating Scale
1 = To an Extremely Small Extent
2 = To a Very Small Extent
3 = To a Small Extent
4 = To a Moderate Extent
5 = To a Large Extent
6 = To a Very Large Extent
7 = To an Extremely Large Extent
N/A = Not Observable or Not Applicable
©2003 Envisia Learning.All Rights Reserved.
Most of the 360 surveys used in UI LEAD feature a 7-point scale, such as the one above. (However, some 360s in UI LEAD have a 5-point scale to streamline responses.)
An expanded, 7-point scale effectively spreads the distribution of scores, thus aiding in the clarification of results. An individual’s strengths and opportunities for growth are more easily identified.
As you might imagine, raters often select scores in the middle range of the scale, perceiving it to be the “safe” rating. This tendency is known as the “average rater error.”
Understanding rater bias:
Literature on 360s suggests that raters may be susceptible to unintentional biases which greatly reduce the effectiveness of 360-degree feedback. The best solution is for you, the rater, to be aware of these possible biases by reviewing the following information.
The halo effect:
Leniency bias:
Central Tendency:
Recency bias:
General Recommendations for Enhancing Objectivity:
In summary, to enhance the effectiveness of the feedback provided to your colleague who is participating in UI LEAD,
Resources for Raters
Definitions of competencies (skills / behaviors) assessed:
ExecView: http://www.uiowa.edu/hr/lead/execview_360.pdf
LeaderView: http://www.uiowa.edu/hr/lead/leaderview_360.pdf
ManagerView: http://www.uiowa.edu/hr/lead/managerview_360.pdf
Emotional IntelligenceView: http://www.uiowa.edu/hr/lead/emotional_intelligence_360.pdf
Validity reports and research information from Envisia Learning:
LeaderView (Per Ken Nowack, Ph.D., Envisia Learning, see TeamView360): http://www.envisiatools.com/products/360-Degree-Feedback/Team-View-360/Team-View-360-Resources/Team-View-360-Psychometric.pdf
Emotional IntelligenceView: http://www.envisiatools.com/products/360-Degree-Feedback/Emotional-Intelligence-View-360/Emotional-Intelligence-View-360-Resources/Emotional-Intelligence-View-360-Validity.pdf