Process Performance Professionals
Your Subtitle text

CLAM V 1.0

Process Performance Professionals

CMMI® Lean Appraisal Method, Version 1.0 (CLAM V 1.0)

Dr. Michael J. D’Ambrosa

President & CEO, Process Performance Professionals

SEI-Authorized CMMI Instructor & High Maturity Lead Appraiser

June 2013

Unlimited distribution subject to the copyright

http://pppcmmi.com


Table of Contents

Part 1:  Overview                                                        

1.1 Executive Overview

1.2 Comparison of CLAM and SCAMPI

1.3 Audience for CLAM

Part 2:  Plan for Appraisal

2.1 Identify CLAM Leader, Sponsor, and CLAM Team Members

2.2 Identify Appraisal Scope

2.3 Develop Appraisal Plan

Part 3:  Conduct Appraisal

2.1 Conduct Participant Briefing

2.2 Conduct Interviews

2.3 Generate Finding Statements

Part 4:  Generate and Deliver Results

4.1 Prepare Appraisal Report

4.2 Deliver Report to Sponsor


Part 1:  Overview                                                        

1.1 Executive Overview

Process Performance Professionals recognize the value of evaluating a company’s development procedures for its business effectiveness.  For many years the SEI’s SCAMPI method has been used to do that.  But SCAMPI was originally developed for the US DoD, resulting in a method that is often geared more for external comparisons (via its strict rules to generate ratings) than for internal needs.  Of course, there’s a significant overlap there, but they are not the same things.  It is not surprising that the number of SCAMPI appraisals has decreased in the US and Western Europe, where the high cost and limited use for some projects (e.g. Agile, infrastructure, and maintenance projects) have caused companies to scale back on their CMMI-based activities.  It is also not surprising that the number of SCAMPI appraisals has increased in other parts of the world (e.g. China and India), where the SEI-backed ratings are critical for those companies who wish to be preferred suppliers, often to those same companies who have dropped those activities themselves. 

In recognition of these issues Process Performance Professionals have developed a “lean” appraisal method, the CMMI Lean Appraisal method (CLAM).  Like SCAMPI, CLAM is primarily a method to evaluate system/software development methods against a model of best practices.  But it is far less costly, broader, and more flexible than SCAMPI.  It emphasizes business effectiveness, not model compliance.  It was developed by an experienced SEI-authorized High Maturity SCAMPI Lead Appraiser.  It is designed to be used internally to determine areas for business improvement in a cost-effective manner.  Since it has a very small rule set and no external oversight, its success depends to a much higher degree on the skills of the lead appraiser and company liaison, the insights of the sponsor, and the openness and buy-in of the interviewees.  It results in finding statements, not ratings, which should be used as the basis of a company’s business improvement initiatives.

1.2 Comparison of CLAM and SCAMPI

Like SCAMPI, CLAM has a planning/preparation phase, a conduct phase consisting of interviews and artifact reviews, and a concluding phase where results are documented and delivered.  But as its name suggests, it has been developed in the spirit of lean initiatives.  It should be noted that the author of this document is a strong supporter of SCAMPI and continues to use it when asked to do so.  CLAM is intended to be an alternate approach, not a replacement, and the comparison below is intended to help a company choose which is more appropriate for their needs.  CLAM obeys all the principles of the ARC (Appraisal Requirements for CMMI) and almost all of the practices.  Those practices omitted were consistent with the “lean” philosophy of CLAM.  CLAM should not be used as preparation for a SCAMPI A (with ratings which have business value); a SCAMPI B or C would be more appropriate.

1.2.1 Cost Comparison

Much of the cost a company absorbs for a typical SCAMPI A (and most SCAMPI Bs) occurs prior to the “conduct” phase of the appraisal, preparing for the SCAMPI by:

(1) Doing internal work to do various gap analyses and develop and implement remediation plans.  While these efforts are generally worthwhile, arguably they should be performed as BAU (Business As Usual), not as pre-SCAMPI activities.  In addition it is unfortunately a common occurrence that organizations will develop processes and artifacts which have negligible business value, but are deemed necessary to “pass” the SCAMPI.

(2) Developing a SCAMPI Plan.  While planning is almost always a good thing, the reality is that too often the primary customer for these plans is the SEI, not the company itself.  The contents of the plan are driven by SCAMPI MDD requirements, not by the company’s improvement needs.  One would hope that doesn’t happen, but experience says otherwise. The SCAMPI Plan thus becomes extensive and expensive non-lean shelfware.

(3) Gathering and reviewing the artifacts.  While the latest MDD has the potential for making the cost of gathering – and reviewing - the required artifacts smaller, the cost is still prohibitive (and there is no data of which I am aware to show the cost has indeed been lowered).  For example if there were 500 artifacts and it took 30 minutes to define, collect, review, and annotate the review (and these are very conservative estimates) the cost would be 250 man-hours.

By comparison since CLAM is truly a “snapshot” for which there are no ratings at stake, there should be no need for gap analysis and remediation activities.   Also CLAM requires no preliminary artifact collection; instead it uses selected “real time” artifact review - as part of the conduct phase.  That leaves us with the only CLAM preparation activity, planning, which primarily consists of:

(1) Meeting with the “company liaison.”  This meeting may take 2-3 man days (less if the CLAM LA is already familiar with the company) and results in a draft appraisal plan.  This plan should be no more than four pages in length, but does require some work, as explained later.

 (2) Meeting with sponsor.  The CLAM LA and the company liaison meet with the sponsor to review and/or modify the plan.  This meeting and modification should take no more than a few hours.

There is extensive cost for the SCAMPI A (and, again, most Bs) conduct phase due to:

(1) Large teams.  The MDD requires 4-10 team members.  So, for example, a 12-day Level 3 SCAMPI with seven internal team members, results in a cost to the company of almost 700 man-hours.

(2) Three meetings.  Both SCAMPI A and B now require an Opening Meeting (and/or a Participant Briefing), a Validation Activity (usually a Preliminary Findings Meeting/Presentation), and a Final Findings Presentation.  Generally all interviewees and team members attend these meetings.  If these three meetings take an hour each and there are 40 attendees, the attendance cost alone is 120 man-hours.

(3) Coverage rules.  The MDD rules for coverage, collaboration, and so on results in much time spent obtaining redundant corroborations, logging straightforward “strengths”, ensuring those practices which have negligible business value are in place,  and multiple reviews of the observation worksheets to ensure all rules are met.

In the conduct phase, CLAM may require more time for interviews, since the artifact review is only done during the interviews.  However this extra time is offset by not requiring all the “tick-the-box” questions whose answers are known and whose primary purpose is to satisfy the set of MDD rules.   Also CLAM only requires the Opening Meeting (the other meetings are optional).  With the time saved for meetings and consolidation, the CLAM conduct phase should take about two-thirds the time of the SCAMPI conduct phase.

Finally for the reporting phase, there is only one significant difference - CLAM requires no SAS entry, which saves one or two man-days of work for larger appraisals.

1.2.2 Breadth and Flexibility Comparison

Of course the SCAMPI MDD puts no restrictions on the breadth (i.e. the portion of the organization’s work covered by the appraisal) of the SCAMPI.  However, since all projects are responsible for all PAs within the model scope (with generally the only exception being SAM) – and all practices for those projects must be examined - usually some subset of project types are chosen for the OU.   This is borne out by the author’s personal experience based on his involvement in over 30 appraisals, only a handful of which have covered the whole organization.  For example for defense firms IR&D projects are generally not part of the Organizational Unit (the OU is that portion of an organization from which the sample projects are picked and to which the appraisal results apply).  In addition service projects are generally not included with development projects (multi-model appraisals – using CMMI-DEV and CMMI-SVC - are now being performed, but infrequently). 

For IT organizations, SCAMPIs generally are performed only for large development projects; infrastructure projects and maintenance projects (which frequently cover more than half of IT’s work) are generally excluded (the former may be included for a Maturity Level 2 appraisal, but not for Maturity Level 3; the latter is almost always excluded).  The rationale is simple: the excluded projects generally follow a slimmed-down version of the processes used by the included projects and thus their inclusion in the OU puts the attainment of their rating goals at great risk.  For the same reasons projects which follow non-standard development life cycles (such as Agile or other iterative approaches) are excluded from the appraisal even though those projects are sometimes pilots whose inclusion in the appraisal is clearly of highest priority.

By comparison, since CLAM has no ratings – only business improvement opportunities – and no coverage rules other than following the plan, there is no reason to exclude any type of projects.  The planning broadly categorizes the various project types to ensure that all types are covered, subject to sponsor guidance.  Careful planning is the key to making this work.  Since CLAM is not bound by any coverage rules, the planning should carefully define which process areas, goals, and possibly even practices have business significance to each project type.

The advantages of CLAM go beyond easy extension to all project types.  While the SCAMPI does allow for multi-model appraisals, there are strict rules for doing so which many companies find prohibitive.  In CLAM the planning can easily have any mix of CMMI-DEV and CMMI-SVC PAs, subject to business needs.  CLAM can also allow other models and/or methods to be brought in.  For example, if a project using Agile Scrum is included, the interviews can look at some of the techniques and artifacts unique to that (which may or may not be mapped to CMMI goals, in accordance with the plan).  There is no limit to these extensions, but in practice they will be limited by time and team expertise.  It should be noted here that CLAM has no rules for full participation by team members (since corroboration is not an issue), so part-time team members can be used to deal with team knowledge gaps.

1.2.3 Quality Comparison

If one looks at the history of the various CMM/CMMI appraisal methods, it is quite apparent that the rigor of the methods provides  somewhat of a guarantee that there is some level of confidence that the ratings truly reflect the company’s (or at least the OU’s) process maturity, the so-called “benchmark” characteristic.  That is a quality attribute of the SCAMPI method that CLAM certainly doesn’t have.  

On the other hand, the SCAMPI methodology rigor can have an adverse effect.  This can happen in various ways:

(1)  Some practices have little or no business relevance for some or all projects.   This generally results in unnecessary work, sometimes even including the creation of artifacts to pass the appraisal.  For example, consider formal alternative design decisions (TS SP 1.1 – develop alternative solutions and selection criteria).  Its value for small enhancements to existing products is questionable, since most likely any design choices were made in the past, often prior to the existence of the OU’s current procedures.

(2)  In contrast, some practices are much more important than others.  Following the MDD, once the artifacts and/or affirmations indicate that an SP is “satisfied” there is no need to delve further, even though business needs suggest otherwise.  For example, consider estimation (PP SG 1 – establish estimates).  Typically a project shows its WBS, shows estimation worksheets indicating the use of attributes and the life cycle choice, and shows traceability from effort estimates to cost estimates.   This is good, but there are so many more issues.  For example, accuracy; how accurate are the estimates and, equally important from a business point of view, how accurate do they need to be?  What about phased estimates?  And how do the PMs collect estimates from the various estimation stakeholders and review them? And so on?

The point here is that the MDD rules can result in all SGs and SPs to be treated in a near equal manner (especially with tight time constraints).  But we all know that they are not equal in business significance.  CLAM is better suited to allow more time for the more important ones and little or no time (by choice) for the less important ones.

1.3 Audience for CLAM

Clearly any company who needs the SEI (and/or the new CMMI Institute) to stand behind their rating should not use CLAM, since it is wholly internal and has no organization standing behind the appraisal results.  On the other hand any company which deals with system/software development or services and has used CMM or CMMI as a significant driver (even if that was sometimes in the past) to create its processes can use CLAM.  The primary caveat is that CLAM needs at least one internal person (as will be explained later) with appropriate knowledge of the company and its process strengths and weaknesses to effectively plan for, and participate in, the CLAM with the Lead Appraiser and truly represent the sponsor.  It should be noted that in the author’s experience this has always been possible.


Part 2:  Plan for Appraisal

2.1 Identify CLAM Leader, Sponsor, and CLAM Team Members

To keep CLAM an “open” method there is no “CLAM CLAM” (Certification of Lead Appraiser Method for CLAM).  However, the typical characteristics of a good CLAM Leader are one who:

  • Has previous experience with appraisal activities
  • Understands the fundamental difference between a CLAM appraisal and an audit
  • Has solid development and management background
  • Is very familiar with the CMMI and has appropriate familiarity with any other reference models being used (keeping in mind that gaps can be filled, as explained later)
  • Fully understands the necessity of understanding the background and organizational structure of a company as a prerequisite to having an effective appraisal
  • Has good presentation skills and great interviewing skills

The role of the CLAM sponsor is similar to, but possibly more demanding than, the role of the SCAMPI sponsor.  Specifically the CLAM sponsor:

  • Has personal belief in the business benefits of the appraisal
  • Supplies the funding and manpower for appraisal activities
  • Ensures the support of the organization
  • Supports the appraisal by clearly observable personal involvement before, during, and after the appraisal
  • Understands the organization well enough to be instrumental in helping to scope the appraisal
  • Is the recipient of the appraisal results and understands the necessity of meaningful and appropriate post-appraisal activities

CLAM requires only one additional team member, but does demand more of that team member than SCAMPI.  This team member is referred to as the “company liaison.”  Unfortunately at the present time there is also no “CLAM BAKE” (CLAM Book of Acquired Knowledge and Expertise).  But this person should be a senior person who has been designated by the sponsor to work hand-in-hand with the CLAM LA to prepare and conduct the appraisal.  It is imperative that this person is highly regarded by the sponsor and the company, is familiar with the organizational structure, understands the project portfolio, and has more than rudimentary knowledge of the CMMI and any other relevant models and/or standards to be used in the appraisal.  

Additional team members can be used, but CLAM is designed to be run effectively with the liaison as the only full-time team member required to work with the appraisal leader.  If needed, part-time team members can also be used.  For example, an Agile expert could be used to help in the interview of Agile projects and the write-up of the corresponding section of the final report.   It is expected that the company liaison has sufficient knowledge of the company to help the appraisal lead to identify the need for any additional full-time or part-time team members.  It is imperative that the sponsor understands the need for, and supports, this flexible appraisal team approach.

In summary, an effective CLAM appraisal requires a leader, sponsor, and liaison with the above characteristics.  When this is not the case, it is strongly suggested that more traditional appraisal methods, such as a SCAMPI B or C, be initially used. 

2.2 Identify Appraisal Scope

Since CLAM has no ratings – only weaknesses and improvement opportunities - there is no reason to exclude any type of projects.  The planning should broadly categorize the various project types to ensure that all types are covered, subject to sponsor guidance.  Careful planning is the key to making this work.  Since CLAM is not bound by any coverage rules, the planning should carefully define which process areas, goals, and even practices have business significance to each project type.  This represents a significance difference from SCAMPI, which allows PAs to be chosen, but not goals or practices within the chosen PAs.

Determining CLAM’s scope goes beyond the choice of project types and model components.  As mentioned above, CLAM can (and should, if appropriate) mix CMMI-DEV and CMMI-SVC, and include Agile projects, IR&D projects, infrastructure projects, and maintenance projects.  It should be noted here that CLAM has no rules for full participation by team members, so part-time team members can – and should - be used to deal with team knowledge gaps.

2.3 Develop Appraisal Plan

Following CLAM’s “lean” philosophy, the plan should contain only what is absolutely needed.  To that end the plan should contain the following:

  • The primary objectives of the appraisal (most important areas to be investigated)
  • The identity of the sponsor, lead appraiser, and company liaison
  • The project types to be included in the appraisal
  • Reference models or methods to be considered
  • Any significant constraints or risks requiring action and a description of those actions
  • Logistical needs

For each project type the following additional information is required:

  • Reference models or methods relevant to those project types
  • For CMMI-DEV and CMMI-SVC the GGs, PAs, and SGs to be investigated
  • For the SGs any notes to further direct the interview (e.g. very important SPs or subpractices)
  • For other models or methods notes similar to the CMMI notes
  • Additional team members needed for the project type (or some subset of the interviews)
  • A description of the interviewees needed by role (specific names to be chosen later)
  • A list of the type of artifacts that will be looked at (specific artifacts will be designated later) as part of the interviews.

Finally the plan should contain an approach for the interviews.  In particular decisions as to which areas can be explored in common interviews and which areas require separate interviews for each project type should be made and documented.  As an example Risk Management may be conducted in the same manner for all project types, so separate interviews for this area would not be necessary.

It is expected that the lead appraiser and company liaison work jointly on this plan, and that this may take several days.  The accuracy and thoroughness of this plan is critical to appraisal success.  Part-time team members will be used if needed and intermittent review by the sponsor will be used in accordance with the sponsor’s expressed requirement.  After final review and modifications by the sponsor, the plan is signed by the sponsor, lead appraiser, and company liaison.

The company liaison is responsible for using the plan to build an interview schedule that is consistent with the plan.  This interview schedule contains the following:

  • The time, place, and name of all the interviews
  • The areas to be covered for each interview (the plan should be referenced)
  • The attendees for each interview (including the part-time team members)
  • The specific artifacts that will be reviewed at the interview (it is expected that the documents are on-line and the necessary computer connection and projection equipment will be available)

The sponsor and lead appraiser will participate if necessary.  At a minimum they will review, modify, and approve the schedule.  The completed schedule is then added as an appendix to the plan.


Part 3:  Conduct Appraisal

3.1 Conduct Participant Briefing (aka Opening Meeting)

CLAM does not require a Preliminary Findings Presentation nor a Final Findings Presentation, consistent with its low cost objective.  However the Opening Meeting is absolutely needed.  All appraisals depend to a large extent on the openness and honesty of the interviewees.  This is especially true of CLAM since there are no preliminary artifact reviews.  The primary purpose of the Opening Meeting is to do whatever is necessary to promote that openness and honesty, to increase the “buy-in” of the interviewees, and to make the interviewees comfortable with the lead appraiser.  The lead appraiser should present the Opening Meeting and all interviewees should attend it.  There are no required contents for this meeting, but it is suggested that the meeting includes:

  • The overall – and specific – objectives
  • All the necessary “thank-you’s”
  • An overview of the method – including the output
  • An explanation of how CLAM differs from audits and other appraisals with which they may be familiar
  • An explanation of how important their openness and honesty are to the entire activity
  • Reminders about the schedule (which they should have received by then)
  • A brief description of how the interviews will go – especially the artifact review
  • A challenge to them to seek clarification if they don’t understand a question or don’t understand it’s business relevance

Typically the Opening Meeting should take about a half hour.

3.2 Conduct Interviews

Every interview should have a pre-interview and post-interview phase immediately before and after the interview.  It is important that relevant part-time team members participate in all three phases.

3.2.1 Pre-Interview Phase

Immediately before the interview the team should review and expand the relevant portions of the plan.  If necessary, internal team members should provide information about the interviewees to the lead appraiser.  Notes about SGs and SPs should be reviewed and elaborated to properly focus the interview.  The primary areas to concentrate on for the artifacts under review should be discussed.  It is suggested that all relevant team members participate in the interview (but not necessarily equally) and a plan for that participation be formulated.   All panning notes – and relevant model components – should be brought to the interview by those team members.

3.2.2 Interview Phase

Interviews should follow the CLAM Plan and the agreements made during the Pre-Interview Phase.  These two planning activities define the “depth” of the interview.  All team members should take notes, with special emphasis on identifying improvement opportunities.  It is suggested that the lead appraiser be both the moderator and time keeper.  A time keeper is necessary to ensure that the interview covers all that was planned in the allotted time.

CLAM interviews differ from the customary SCAMPI interviews in two major ways: 

(1) Since there are no strict coverage rules not all CMMI practices need to be addressed.  Those that are addressed may have more time spent on them.  Again, the team is not focused on model compliance, but is looking for improvement opportunities. Mapping observations to the reference model(s) is less important in CLAM than it is SCAMPI.

(2) Artifact review only takes place during the interview, in accordance with planning.  Generally this assumes the team knows what it is looking for, and the interviewees know where in the artifact that information can be found.  So the artifact review and the questions about the artifact occur simultaneously.  This is generally done by projecting the artifact for team and interviewee review.  Of course this technique can be (and has been) used in SCAMPIs, but it is not the norm as it is in CLAM.

Although the interview flow can proceed in any manner, there is one strong suggestion for CLAM.  Start with the first few GPs, specifically GP 2.1 to GP 2.4 (and GP 3.1, if it’s within scope).  Besides being a valuable source for potential improvement opportunities, they also provide the proper background for the SP-related questions.  The remaining GPs can be covered at any time (or skipped, according to the planning activities).  It is also useful to ask the interviewees (presumably at the end of the session) if there are any additional areas for business improvement that did not surface during the interview.

3.2.3 Post-Interview Phase

Here CLAM is very different from SCAMPI.  SCAMPI team members generally use mini-teams to categorize observations according to the relevant model component (i.e. by PA, SP/GP) and then record them in some structured worksheet or tool for later full-team modification and, ultimately, consensus .  CLAM team members use a brainstorming session to record, and ultimately agree on the draft improvement opportunities that arose from the interview and artifact review.   One team member should act as the scribe to accomplish this.  There are no strict rules for this, but there are a few guidelines:

  • This activity should be done immediately (or at least in the same day) after each interview by all the team members who conducted the interview
  • There is no need to link these improvement opportunities to model components
  • “Wordsmithing” is not necessary at this point
  • There is no need to denote “weaknesses” (as in SCAMPI)
  • There is also no need to cite strengths, unless a strength is noted for some subset of the company that should be extended to a larger part of the company, in which case the strength can still be worded as an opportunity for improvement

3.3 Generate Finding Statements

After the interviews are completed, the team synthesizes the draft improvement opportunities into the statements that will go into the Appraisal Report.  This includes deriving summary statements from related draft statements and wordsmithing those statements.  The statements could be grouped by PA (or other model components) or by project type, or some combination of the two.  The important point is that, unlike SCAMPI Final Findings, these improvement opportunities are grouped in any manner the team believes will be most effective (or, if applicable, in the manner requested by the sponsor).


Part 4:  Generate and Deliver Results

4.1 Prepare Appraisal Report

A Final Findings Presentation is not required for CLAM, although of course one can still be done as an option.  If such a presentation is given, it can - and should – serve as the Appraisal Report.  If that presentation is not given, then an Appraisal Report should be prepared.  In either event, the report (or presentation) should be prepared by the full-time appraisal team members, with optional input from part-time team members or any other personnel the team agrees on (including non-team members).  The report must contain the finding statements.  Other optional components of the report are:

  • A timeline of the appraisal activities
  • Appropriate thank-yous
  • Appraisal statistics such as number of interviewees and number of artifacts reviewed
  • Traceability to appraisal goals
  • Statements related to model coverage
  • Guidelines on how to use the finding statements to build action plans

It is important to note that (with the possible exception of the thank-yous) these optional components should not be included unless the team believes they will be useful in the future or the sponsor specifically requested their inclusion.  Once again, the lean mentality should rule!

4.2 Deliver Report to Sponsor

As we have stated above, no record is sent to the SEI or CMMI Institute.  The report is simply delivered to the sponsor.  The sponsor has the responsibility of delivering it to the company in any manner he/she deems appropriate.  If the sponsor requests it there can be a follow-up meeting.  Also a follow-up retrospective can be held, but – again – only if the team sees it as valuable.

Website Builder