|
|||||||||||
|
Contents About PSS
To give success-minded individuals the resources they need to grow stronger for work and life.
The two co-founders, Dennis E. Coates, Ph.D. and Meredith M. Bell, M.A., have over sixty years combined experience in the fields of assessment and personal, professional and human resource development. Together, they’ve helped thousands of organizations present programs to well over 100,000 managers and professionals. To date, over a million people have used PSS products. We are the publisher of the award-winning 20/20® Insight GOLD, a uniquely flexible, affordable, and customizable multi-source feedback and development system. Contact Information Performance
Support Systems, Inc. PSS Executives
About 20/20 Insight GOLD People need timely information about their performance in order to improve it. More than a million people have participated in feedback projects since 20/20 Insight’s first release in 1994. They have seen, first hand, the benefits of the quality information that has been collected and delivered by our easy-to-use software. 20/20 Insight GOLD gives organizations the ability to collect virtually any type of feedback—ideas, opinions, ratings, surveys or facts—from any number of people about the performance of a product, service, individual, team or even an entire company. Customized feedback reports can easily be generated and distributed using this highly flexible system. The 20/20 Insight GOLD system is the world’s first fully integrated family of feedback and individual development planning software, making it the complete feedback solution.
Organizations have two options for using 20/20 Insight GOLD: • Purchase the software and manage administration in-house. Installation is quick and easy (on a PC or network drive). • Outsource survey administration to an authorized representative, who will perform the administration services for the user. Software purchase is not required, and confidential information is stored off-site. Features and Capabilities Organizations that purchase 20/20 Insight GOLD receive the world’s most versatile feedback tool: COMPLETE: A total feedback system CUSTOMIZABLE: A program that can
change with a company’s growing
needs FLEXIBLE: A tool that can be used
for multiple applications USER-FRIENDLY: Easy to use for
administrators and respondents ECONOMICAL: A very cost-effective
investment Value-Added Reseller (VAR) Program To help market and sell the product, PSS has established a formal relationship with dozens of independent firms worldwide. Our 20/20 Insight VAR Program offers resllers the following opportunities: Expand their consulting business. Add a major new revenue stream. Enjoy a supportive business alliance. Use high-quality tools. Click here for details about the 20/20 Insight VAR Program. Testimonials
Artwork These images are designed to accompany any articles or stories you publish pertaining to 20/20 Insight GOLD or Performance Support Systems, Inc.
Click
on the image to view your download options. Top Five FAQs The following FAQs will help you understand the 20/20 Insight GOLD system and the most effective ways to implement it in an organization.
What’s the purpose of 360-degree feedback? What are the most powerful ways to use it? The technology for collecting and reporting multi-source feedback was developed in the 1980s. Its original purpose was to diagnose leadership performance. By assessing a comprehensive set of skill areas, leaders obtained quantitative and qualitative information about strengths and areas that need improvement. Other innovative uses for multi-source feedback have evolved over the decades. However, when most people hear about 360-degree feedback, they still think of its traditional use: a global diagnostic of competence and skill. A much more powerful application of 360-degree feedback goes beyond the diagnosis to support changes in behavior. A doctor’s diagnosis can reveal the disease, but this information can’t cure it. Likewise, 360-degree feedback can identify priority areas for improvement, but this information isn’t enough to improve work habits. Changing a behavior pattern may require instruction, followed by months of reinforcement. Try changing the way you eat or the way you swing a golf club. Tiger Woods made changes in his swing early in 2004, and he didn’t start to win again until almost a year later, after persisting through hours of practice every day. The problem is that even with the best of intentions, when people try to do things differently, initial attempts tend to feel awkward. When these efforts don’t achieve the desired result, frustration and discouragement follow. Without a formal program of follow-through reinforcement and without support from the direct manager and others in the workplace, people tend to fall back on what feels familiar and comfortable. They eventually return to their old way of doing things. To achieve the desired changes in behavior, 360-degree feedback needs to be followed by several months of reinforcement, involving ongoing learning, ongoing feedback, coaching and accountability. It takes that long for the brain cells to grow and reconnect into new pathways that are the physical basis for new behavior patterns. After people are assessed in underdeveloped skill areas, they may need training. Either or both of these interventions must be followed by an extended period of reinforcement. This commonsense developmental sequence is the foundation of what is perhaps the most powerful 360-degree application ever devised: validating individual performance improvement. Used in this way, 360-degree feedback works both as a diagnostic assessment and as a means to check whether weak areas have improved. The concept is simple. First, integrate behavior-based assessment with behavior-based training. Then several months after training, follow through with a more focused behavior-based assessment related to the priority areas for improvement. Compare the pre-course scores with the post-course scores. Improved scores will indicate how much skills have improved. This approach has significant benefits. First, the results of the pre-course diagnostic allow participants to set quantified, behavior-based performance improvement goals. Also, knowing that follow-up assessments will be administered causes learners to be more focused and motivated as they work with trainers—the ideal mindset for learning. In addition, the post-course assessments give learners quantified and qualitative feedback about how they’re doing as they try to improve their skills. Finally, following through with post-course assessments creates accountability. The assessment results will document whether the individual has improved on-the-job performance. Repeat post-course assessments can be administered as desired to produce ongoing measures of performance improvement. To implement this system, you’ll need a fully customizable 360-degree assessment system, because the assessment items need to be tailored to exactly mirror the desired behaviors taught in training. In other words, assessment and training must be integrated. Also, the assessment needs to be affordable. Inexpensive unlimited assessment licenses for each person make it possible to give learners all the feedback they will need after training—without additional expense. For leadership development, many organizations use the 20/20 Insight GOLD 360-degree feedback system along with the Vital Learning Supervision Series GOLD curriculum. 20/20 Insight GOLD administers the Supervision Series leadership diagnostic assessment, identifying strong and weak areas. Then the Vital Learning Supervision Series GOLD curriculum is conducted as a 12-unit series, or selected blended (in-class and online) learning units are delivered to focus on high-priority weak areas. In addition to assessment and training, the two technologies provide ongoing support for the structured period of reinforcement. The trainee materials include a year of access to online video behavior models and post-course reinforcement resources. You don’t have to repeat the entire diagnostic assessment to measure how much performance has improved. Instead, post-course assessments need only focus on priority developmental areas. Since the pre-course and post-course items are identical, scores can be compared. This ability to measure improvements in performance fulfills the need for ongoing feedback and accountability. The data created by performance improvement assessment can also be used as a practical return-on-investment (ROI) calculation. For example, assume that leadership skills account for half of a supervisor’s effectiveness. Assessment scores showing an average improvement from 6.4 (before assessment and training) to 7.7 (several months after) would indicate a 20% percent improvement. Since half of a salary of $60,000 is $30,000, the organization would be getting 20% more effectiveness for this cost, worth roughly $7,500—a result many times greater (in dollars) than the cost of the individual’s training. Simple ROI calculations like this can be performed locally. They are made possible by pre-course/post-course performance improvement measurements powered by customizable 360-degree feedback. The bottom line: global diagnostic assessments serve an excellent purpose if you follow through with learning and reinforcement. Combine an affordable, customizable 360-degree feedback technology with a behavior-based leadership development curriculum, and you get a fully integrated assessment, training and reinforcement system:
More important, supervisory leaders are empowered to reinforce and ingrain their new skills over time to create permanent, measurable changes in behavior—the Holy Grail of leadership development. In the end, how well your front-line managers lead affects the bottom line—and every other aspect of your organization. Considering the billions of dollars invested annually in leadership development, organizations need a way to demonstrate whether these programs are actually changing behavior. Using multi-source feedback to measure performance improvement is the most effective way to quantify the return on your investment.
2. What was the validation research performed for "Executive Leadership" and other standard surveys contained in the 20/20 Insight GOLD Survey Library? The answer has three parts: A. The process for "validating" a behavioral feedback survey is not the same as the process for validating a psychological instrument. Since the technology of behavioral feedback surveys is an outgrowth of the tradition of psychological testing, it's natural to confuse the two kinds of assessments. However, even though they're both often referred to as "assessments," they're radically different kinds of tools, and the procedures for validating them are different. Validity means one thing when evaluating a personality assessment or a psychological test, but it means something entirely different when evaluating a tool that gives direct feedback about behavior - a completely different kind of assessment. Validity and reliability information has been important to determining the usefulness of tests that measure intelligence, values or psychological characteristics. These are important aspects of people that cannot be observed or measured directly, so questions are asked in order to draw inferences about these aspects. Therefore, it's important to determine whether the assessment really measures what it says it measures. In other words, are the constructs valid? Are the constructs that are calculated based on the answers highly correlated to phenomena in the real world? Do they make valid inferences based on the questions in the assessment? Will we get the same results, measurement after measurement? Many of the early multi-source feedback instruments evolved from these traditions. Some even focused on traits, values, characteristics and other aspects which cannot be measured through direct questioning or direct observation. Their purpose was to ask carefully researched questions from which one could draw inferences about these important qualities. The constructs created from the answers were the main purpose and product of these assessments. Therefore, research was needed to verify (validate) that the inferences were not just theoretical notions, but actually coincided with what was happening in the real world. 20/20 Insight GOLD is a wholly different technology. Quite simply, it's a computer-assisted mechanism for gathering and giving feedback about specific observable behaviors. Unlike psychological instruments, the survey items (observable behaviors) themselves are the primary focus of the feedback - not constructs inferred from the item responses. In 360-degree feedback, many people are asked to report what they observed, and their aggregate response is reported. These responses themselves, not dimensions inferred from the responses, are the purpose and product of the assessment. In fact, the 20/20 Insight GOLD surveys don't create or report any constructs. The items are clustered into categories only to make it easier to relate and analyze the feedback. Therefore, there's no need to do research to verify the validity of inferences or constructs, because no inferences or constructs are produced. Unlike personality or trait assessments, a "universal" validation which applies to all organizations is not appropriate. That is not to say that validity isn't important. With 20/20 Insight GOLD, another kind of validity is important. It's important to establish whether the surveys to be used within the organization actually address the most important workplace behaviors of that site. When the 20/20 Insight GOLD surveys are used, we encourage organizations to do internal competency research, then deselect, add and modify items in the standard survey categories to align with local practices. These customized surveys must be verified locally, because every organization has a different business, culture and priorities. Validation can be accomplished by using importance and frequency surveys, expert panel reviews and pilot assessment projects. Reliability also doesn't have the same meaning with this kind of feedback tool. Reliability refers to the consistency of measurements. In fact, you wouldn't expect or want to get the same results with each subsequent measurement, because the assessment focuses on observable skills, competencies and abilities, which are expected to improve over time. Once again, with
behavioral feedback surveys, the important validity issues are: This view of validity also applies to the organizational surveys processed by 20/20. Survey items are organized into categories because of their topical relationship, not because they are used to create constructs - none are created. In other words, the answers to these questions are what we seek. The categories simply help us focus in on areas of interest. The survey doesn't "measure" the category. It simply asks questions that leaders want to ask, and collates the replies as feedback information. The items are valid if they are, in fact, exactly the questions leadership needs to ask to find out what they need to know to improve the organization. Because a psychological instrument is intended to be used as is with all possible subjects, its validation research is normally published in psychological testing journals. This is not the case with the validation research performed for behavioral feedback surveys. B. "Executive Leadership" and other standard surveys contained in the 20/20 Insight GOLD Survey Library were researched and developed using standard behavioral feedback (competency) research and development practices. The 20/20 Insight GOLD Survey Library features no psychological instruments. All the individual multi-source feedback surveys are behavioral feedback surveys. Optimally, standardized behavioral feedback surveys are the subject of not one, but two cycles of validation research, both of which are dramatically different from procedures used to validate psychological instruments: (1) author/publisher validation and (2) local validation. The surveys featured in the Survey Library have been developed by experts who have extensive experience and learning in their specialized areas. Each used a standard method for researching competencies. The first step was a comprehensive review of the literature to determine what current research and authoritative writing has defined as highly desired competencies. Based on this research, the developer then created draft competency lists that model the target area of performance. These lists were then screened for redundancy, importance and validity, based on the survey authors' experience in organizations. The lists were then subsequently validated by the developers' client organizations, a process in which the models were examined by organization subject matter experts, used in training and development programs, checked for improvements in performance, and finally referenced in quantitative and qualitative feedback - an effort that often spanned several years. During that process, the survey items were also circulated to a variety of other experienced professionals for evaluation and suggestion. The final versions of the surveys were studied and revised as necessary by the publisher for acceptance in the Survey Library. The surveys were then offered to organizations during beta testing and later for general use in 360 degree feedback programs. Feedback from these groups was studied by the publisher and appropriate revisions were made. In the case of 20/20 Insight GOLD, the surveys included in the Survey Library have been systematically reviewed and used by hundreds of organizations. The standard behavioral feedback surveys in the Survey Library were validated for developmental use, not for linkage to personnel or compensation decisions. C. All behavioral feedback surveys must be customized and validated locally. Different industries involve dramatically different business practices; what is understood as desired performance within organizations varies widely, as does the language that describes it. For example, in the U.S. Army, the Federal Reserve Bank and the Baptist church, leadership is defined and practiced differently. Yet, each desires to use 360-degree feedback to hold a mirror to its leaders' performance. While there are many common behaviors, many are different. Therefore, the second phase of validation must inevitably involve local validation. The author's research does not - and cannot - establish universal validity. While a one-size-fits-all instrument was possible and desirable in the field of psychological testing, it is neither possible nor desirable for feedback about specific workplace behavior. A published standard competency list is intended to serve as an optimum start-point for organizational research, customization and validation locally. This process requires people within the organization (or a consultant) who are familiar with the area of competence to review the standard survey. Items inappropriate to the local workplace may be deleted and additional items added. Wording can be modified to use business-specific terms and align the items with the organizational culture. The new behavior list is then reviewed for size, frequency and importance by local subject matter experts and stakeholders; and additional revisions are made, if appropriate. A pilot assessment with a sampling of subjects tests effectiveness of the survey, and feedback from the process is studied for possible refinements. The final product is a locally validated list of observable behaviors. For behavioral feedback surveys, validity equates to relevance to workplace performance. Does it focus on the most important workplace behaviors in this particular organization? In the end, the most valid survey process may not employ an entire competency list, but focus on portions of it that are deemed by executives to have the highest potential for individual and organizational improvement. 1. For a summary
of the research conducted to develop the original "Team Player" survey,
see the monograph by Dennis E. Coates, Ph.D., "Report of Field
Test Results for 20/20® Insight." Newport News:
Performance Support Systems, 1994. This is an appropriate question when referring to a rigid survey - a fixed set of measurements that can't be customized. When every person in every organization is administered exactly the same set of items, it's possible to accumulate industry and national averages drawn from many organizations. However, when organizations use customized surveys, no two are alike, and national averages - norms - are technically impossible. Rigid surveys were popular in the 1980s, although today most organizations no longer value the "one-size-fits-all" approach to 360-degree feedback. Instead, they require unique surveys that have been customized to align with the culture and practices of their particular workplace - even if they have to develop them in-house. The use of customized surveys is an enlightened approach to feedback. For example, the leadership activities of a minister of a large Baptist church are simply not the same as those of a Marine Corps battalion commander. What a movie director does to get the best out of people doesn't involve the same actions as those of a dean of a law school. A sales manager doesn't lead people the same way a project manager of a construction site does. All these people are leaders, and it's important to give them feedback about how they lead; but each setting requires a somewhat different list of behaviors. While feedback recipients are often interested in how their ratings compare with those of other people, this is not the most meaningful way to make sense of one's scores. The ultimate payoff of 360 feedback is improved performance, and this is possible only if an individual focuses on a single area of behavior. Comparing high and low scores is the most effective start point, supplemented by comments and previous 360 scores. Actually, comparing one's scores to the averages of one's group and one's organization is far more meaningful and revealing than comparisons with national or industry averages, which are based on practices within other organizations and which involve different businesses, strategies, cultures, practices, challenges and technologies. Placing importance on how one's scores compare with those of other people can be counterproductive. Experience has shown that individuals often lose interest in working on areas of performance when they discover that their scores are "above average." They may even conclude that they don't need to focus on any areas for self-improvement, because all their scores are above average. This predictable outcome defeats the individual's need for continuous improvement and enhanced competitiveness in the career marketplace, and it defeats the desires of coworkers for that person to improve how he or she works with them. 20/20 Insight GOLD
doesn't use a fixed set of measurements. Instead, the program is designed
to permit easy customization, so that an assessment exactly mirrors
the behaviors that are important within a particular organization.
National norms are impossible to collect and considered inappropriate
to the needs of the organization. If certain surveys are used organization-wide
and if a comparison of scores is desired, the software can compute
and report local averages - both group project and organization norms. The
best feedback providers are those who have observed the performance of
the subject most closely. These people could be bosses,
peers, direct reports or customers. People are often concerned that individuals
will be selected who will give low, unfair or biased ratings. These concerns
arise when people think 360 will be linked to pay or personnel action,
which we believe to be an inappropriate use of developmental information.
Concerns can be minimized by completely removing the feedback from any
subsequent pay or personnel action. The diagnosis of skills and behaviors
are best used as a prelude to training and development, not personnel
action. One technique to minimize concerns is to ask subjects to nominate
a field of potential respondents, from which a supervisor or the team
can make the final selection. There are four typical reasons why people are motivated to improve themselves based on multi-source feedback information, even though they know that it won't be used for any kind of personnel action. First, if the feedback recipients think of themselves as professionals, they will want to work on any area of their performance that they discover needs improvement. Professional-minded employees typically seek this kind of feedback for the purpose of self-improvement. They see themselves as life-long learners, they take responsibility for their own development, and their pride creates a desire to improve their scores. Second, most people in the workplace care about their coworkers. If the people around them report that certain actions on their part are causing problems, the individuals receiving feedback will want to change these patterns, because they don't like the idea of causing pain or frustration for others. Also, people are more concerned about career development these days. For a variety of reasons, people are more likely to move to other organizations, and they are concerned about their value in the marketplace. In this light, multi-source feedback is seen as an opportunity to identify strengths and areas for improvement. Finally, people are often motivated to assess and develop themselves when the results aspects of performance are measured and linked to compensation, because a lack of ability is a common reason for not achieving desired results. People may conclude they need to improve the way they go about their work in order to get desired results. This doesn't mean that everyone is motivated by these reasons to seek self-improvement action. People who don't think of themselves as professionals, who don't care about the people who work around them, who aren't concerned about remaining competitive in the career marketplace or who aren't held accountable for results may not feel a keen motivation to take development action based on feedback data.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Email us • Toll-free: 800.488.6463 • Phone: 757.873.3700 • © Performance Support Systems, Inc. All rights reserved. |
