Be used as input to generate an application for a state or national quality award. The opportunities for improvement can be prioritized and converted into action plans for improving organizational performance.
While this article will focus primarily on collaborative assessments using the Baldrige Criteria for Performance Excellence, the approach could be used for almost any type of business or organizational assessment, provided the purpose is for understanding of actual performance and opportunities, and not as part of a judging or certification process. The collaborative assessment process can be subject to participant bias if there is a performance goal to be evaluated as part of the assessment outcome.
The Collaborative Assessment approach is not for all circumstances. For organizations that are applying for award recognition, the process for examiner assessment and feedback is a robust and high value approach. For organizations using the assessment primarily as an input for improvement, the Collaborative Assessment provides a solution to many of the inherent disadvantages that exist with traditional awards process assessments.
Disadvantages of Traditional Awards Process Assessments
- Takes too long – For organizations that apply for the Baldrige Award, it takes five to nine months from the submission of the eligibility form until you get your feedback. If a primary purpose of applying is to get feedback, then the better part of a year is lost before beginning to take improvement actions. With the Collaborative Assessment approach, most of the work is completed within one week.
- Resources required – A full self-assessment requires a great deal of time, personnel and financial investment. An awards application takes even longer. Preparation of an application covering all items has been estimated at 1800 person-hours. When resources are exhausted completing the assessment and application, there may be little remaining energy or enthusiasm for the more important task of taking action to improve.
- Untimely for business calendar – Most awards programs, including Baldrige, are on a fixed calendar with feedback reports delivered in the Fall. The feedback report is critical for organizations that anticipate developing action plans from the opportunities for improvement. Depending upon the strategic and business planning cycles for the organization, the feedback may not be in sync with business needs. A Collaborative Assessment can be scheduled to occur any time during the year or planning cycle.
- Wasted time – When writing an application for award recognition, the organizational emphasis is on the strengths of the organization. In the evaluation steps of the awards process, examiner teams use their expertise to try to figure out the opportunities for improvement. It can be similar to a game of hide-and- seek. Applicants tend to embellish the approaches and deployment to present the organization in the best light possible. Poor results are often omitted from the application. In the Collaborative Assessment process, opportunities for improvement are more transparent, thus providing a more efficient process for understanding current state performance.
- Awards vs. improvement mindset – As noted in a recent message from Debbie Collard, immediate past chair of the Malcolm Baldrige National Quality Award Foundation, “Ninety-three award recipient organizations have been publicly acknowledged as performance excellence role models and shared their best practices so that other organizations could learn and improve.” This is one of the tremendous benefits from this program. However, when the focus of the applicant is on achieving award recognition, a competitive mindset can impact the behaviors of those involved in writing the awards application. There can be a tendency to deny real opportunities for improvement, or even to shut down the sharing of best practices between organizations. This can especially occur within organizations when organizational units begin to compete against each other for awards recognition. Oftentimes, gamesmanship shows up in applications that stretch the stated performance beyond reality. With Collaborative Assessments, the focus is shifted more towards understanding the gaps between true current performance and the desired performance.
- Add-on activities – Most organizations will get “volunteers” to gather the information required for an awards application. This activity may begin two or three months prior to the deadline for submission, and can distract from other work responsibilities for several hours every week. Once the application is submitted, the effort is put aside until the feedback report is received several months later. By then, many of the application findings have become forgotten or are less clear. The Collaborative Assessment compresses the effort into a much more efficient information- and data-gathering event. Action plans are developed while the information is still very fresh on everyone’s minds.
- Lack of follow through on feedback – The feedback received from a Baldrige assessment can be incredibly enlightening for organizations wanting to improve. Unfortunately, when the feedback is received as a part of the award process, it often does not get effectively converted into action plans for improvement. This can occur for several reasons already stated. If the primary purpose for assessing is to improve, then the awards process can totally derail this objective unless there is an exceptional tenacity for follow through. Organizations that are just starting out or struggling to move forward are especially vulnerable to this risk. Collaborative Assessments are primarily focused on knowing where to improve and putting action plans into place quickly. Therefore, the likelihood of timely follow though is increased.
- Activities too score oriented – There can be exceptional pressure to improve application scores without understanding the true intent of the criteria. In the awards process, the assessment score is the primary output that is used in determining award levels and recognition. Applicants are not involved in the actual scoring and sometimes lack understanding of how the scores are derived. The Collaborative Assessment process can involve the SMEs in the scoring activities so further learning and understanding can take place. The focus of scoring in a Collaborative Assessment is to provide a baseline metric from which overall improvement can be measured.
- Lack of senior leader engagement – The tasks of writing applications and conducting self-assessments are often delegated to non-senior leaders. This is a missed opportunity for those who are setting direction and priorities for the organization. It also can lead to a lack of acceptance upon receipt of the feedback report because senior leaders missed the chance to learn more about their organization through active involvement in the assessment and application process. A Collaborative Assessment provides good opportunity for senior leader engagement and learning.
- Lack of criteria understanding – The Baldrige Criteria for Performance Excellence are incredibly rich and have gone through many iterations of improvement over the past 25 years. Let’s face it – the criteria will often overwhelm applicants, especially those who are new to them. The Collaborative Assessment lessens this concern because the process and criteria experts help guide the subject matter experts to understand the purpose and intent of the criteria questions. Learning occurs about the criteria and organizational performance.
There may be various approaches that can be applied for conducting a Collaborative Assessment. Some suggested features for an effective collaborative assessment include the following:
Focused Event – Depending upon the size of the organization, a full assessment against the Baldrige Criteria can be completed in 3 to 5 days. Preparation for the event should include planning the event logistics and scheduling of the subject matter experts. Other activities that can be completed in advance include answering the Organizational Profile questions from the criteria, gathering existing documentation about organization processes and gathering data and reports for key organization metrics. While it is important to do a good job of planning, excessive planning and front-end work can be non-value adding and negate the benefits of the Collaborative Assessment. On the other hand, it is important that the necessary people are available for the full-time they are scheduled during the collaboration efforts. This will provide for an accurate and efficient assessment for the organization.
Experts and SMEs – Two key groups of people are necessary for a Collaborative Assessment –people with expertise in the criteria and conducting assessments, and people who are knowledgeable about the organizational processes and performance. There are several sources for assessment experts including Baldrige examiners, state award examiners, consultants and internal resources who are skilled in the criteria and assessing. The subject matter experts (SMEs) can include all levels and functions of the organization. A good cross section is necessary to get a true picture of how the organization works and performs. The SMEs are scheduled to participate only during the parts that are most relevant to their involvement the organization. For example, you would expect senior leaders to participate in the assessment while addressing Leadership criteria questions. Human resource representatives should participate when addressing the Workforce Focus criteria questions.
Focus group format – Assessment information is gathered in a focus group setting. For each section of the criteria, a team of SMEs is scheduled to respond to the criteria questions. For the categories 1-6 of the Baldrige Criteria, responses should address the “what” and “how” questions of the criteria, and should also address “how well” the organization can respond in terms of approach and deployment. The “how well” responses are the basis for the strengths and opportunity information gathered. For the Results category within the Baldrige Criteria, the responses address the “what” and “how” questions and again “how well” the organization is performing. The assessment experts facilitate the discussion by helping the SMEs to understanding the intent of the questions and the scoring criteria which provide a basis for “how well” the organization is performing. The assessment experts will offer their own insights based upon participant responses and knowledge of the performance excellence criteria. The Baldrige Program collaborative assessment gathers information from interviews or small focus groups of category “champions” and their teams (usually 3 – 5 people from across the organization).
Brainstorm – During each focus group, it is best to begin the discussion by conducting a brainstorming session. The facilitator begins the conversation by explaining the specific criteria questions that the group is addressing. A recording device should be tracking responses in real-time. One technique that happens to be a personal favorite is to project the responses using an LCD projector so all the participants can see the information that is being captured. Responses are recorded on two separate lists – one for strengths and another for opportunities for improvement (OFIs). (Note that the term “weakness” is avoided.) At first, the facilitator should simply accept responses like any other brainstorming session. The facilitator may ask clarifying questions, but must refrain from judging. After the group has provided input, then the facilitator should invite discussion regarding the information collected. This is the opportunity for the facilitator to transition to a teaching/consulting mode to help the participants understanding how well the organization responds to the criteria. The facilitator will highlight some of the key strengths and OFIs from the responses, and help the group to understand where information may be missing or off base.
Consensus around strengths and opportunities – At the end of each focus group session, the group will identify the most important strengths and OFIs from the discussion. This can be done using several methods. Multi-voting techniques or rating ballots are effective approaches for getting input from a group of people very quickly. With the Baldrige Program, examiner teams work with the category champions to decide which strengths and OFIs are relevant and important. The final decision lies with the category champions.
Built-in site visit to verify – Information that is gathered in the focus group sessions will need to be verified or clarified. This is a very appropriate activity to include. These can be conducted in between the focus group sessions or perhaps by sending a small group out in the middle of a session if the information can be accessed easily.
Scoring by experts – Scoring is an optional activity in a collaborative assessment. When included, it should be secondary to the identification of key strengths and opportunities. Scoring should be done by the Criteria Experts who are well trained in the scoring guidelines. The advantage of scoring is that it can provide a metric for evaluating progress over time. The scoring becomes non-value adding when energy becomes diverted from gathering assessment information to justifying the score that was provided.
Affinitize key themes – Typically, a collaborative assessment will gather much information, including a long list of strengths and OFIs. This information needs to be synthesized, by separating the useful many from the vital few. One technique that has been proven effective is the process of affinitizing OFIs so that they are grouped into similar themes. The output of this exercise will provide a summary of the most important issues from the assessment.
Action Planning – The final step is to prioritize the opportunities and develop action plans to close the gap between current performance and desired performance. One approach is to narrow the synthesized OFIs from the prior step to create a top twenty list, effectively prioritizing the data so the most significant opportunities are the most apparent.. The selection of the top priorities should include input from the participants and the facilitators. Senior leaders should use this input to determine the most important OFI for which the organization will take immediate action. Action plans should be created with assignments and milestones for follow-up and completion. Ideally, these action plans become integrated into the organization’s annual plans and performance management systems.
Collaborative Assessment Advantages
Let’s recap. For organizations that intend to use the Baldrige Criteria or similar assessment criteria for internal assessment and continuous improvement, the Collaborative Assessment process can be described as:
Faster – most of the work is completed in a one week period.
Less Costly – because the assessment requires little preparation and is completed so quickly, it takes less labor to complete when compared to more traditional application assessments.
More Efficient – the collaborative assessment facilitates organizational learning about the criteria and performance, while building consensus and buy-in for the things that need to change.
Time to Begin
There are five key steps to follow to get started with the Collaborative Assessment as a transformational activity in your organization.
Decide – Leadership needs to decide if an assessment is strategically important to the organization, and if so, what they hope to accomplish. If the purpose is to get started or to continue a journey to performance excellence using Baldrige or similar criteria, then a collaborative assessment may be a good fit for the organization.
Prepare – The organization will need select experts to facilitate the assessment. The experts may come from many sources, including the Baldrige program, a State Awards program, consultants or internal experts. The experts can help the organization to schedule and plan the assessment, including the selection of key participants. The collaborative assessment that the Baldrige Program offers typically entails two months of planning ahead of the actual assessment. The primary purposes of the planning are getting clear on expectations, setting the right structure within the organization to carry out their roles, getting interviews and focus groups confirmed logistics, and getting a clear context of the organization. Much of this initial information can be gleaned from the Organizational Profile.
Launch – For smaller organizations, the initial assessment may be for the entire entity. Larger organizations may want to begin with a subset of the overall entity, such as a location, a business unit, or major function.
Spread – For larger organizations, the assessment can then be spread to other units and areas. All organizations should begin to schedule periodic assessments to monitor performance and identify opportunities for improvement. The follow-up assessments become a means for keeping track of progress and re-prioritizing based upon the changing needs of the organization.
Sustain – The assessment activities and finding should be integrated into annual strategic planning and continuous improvement activities. The organization may eventually apply for award recognition if there are expected benefits to the organization.
The ultimate purpose of collaborative assessments is to help the organization to achieve the highest level of performance possible in the eyes of their customers and key stakeholders.
- Collard, D. (2013). A Baldrige Message of Hope and Celebration, National Institute of Standards and
- Technology, Gaithersburg, MD. Available from Internet: http://www.nist.gov/baldrige; Accessed 07 March 2013.
- Hoyt, G. and Ralston, E. (2001). White Paper: Building on Baldrige
- NIST. (2013). The Baldrige Collaborative Assessment, National Institute of Standards and Technology,
- Gaithersburg, MD. Available from Internet: http://www.nist.gov/baldrige; Accessed 07 March 2013.
Published at :