Process Performance Professionals
Your Subtitle text

CLAM Summary

CLAM = CMMI Lean Appraisal Method (PPP Copyright)

We all recognize the value of evaluating development procedures for their business effectiveness.  For many years the SEI’s (and, more recently, the  CMMI Institute’s) SCAMPI method has been used to do that.  But SCAMPI was originally developed for the US DoD, resulting in a method that is geared more for external comparisons (via its strict rules to generate ratings) than for internal needs.  Of course, there’s a significant overlap there, but they are not the same things.  It is not surprising that the number of SCAMPI appraisals has decreased in the US and Western Europe, where the excessive cost and limited use for some projects (e.g. Agile, infrastructure, and maintenance projects) have caused companies like to scale back on their CMMI-based activities.  It is also not surprising that the number of SCAMPI appraisals has increased in other parts of the world (e.g. China and India) where the SEI-backed ratings are critical for those companies who wish to be preferred suppliers, often to those same companies who have dropped those activities themselves.

In recognition of these issues – especially cost constraints – PPP has developed an appraisal method that is far less costly, broader, and more flexible than SCAMPI.  It emphasizes business effectiveness, not model compliance, is partially ARC-compliant, but is not a product of the SEI or CMMI Institute.  It should be used internally to determine areas for business improvement in a cost-effective manner.  It will have a very small rule set and no external oversight.  Its success depends to a much higher degree on the skills of the lead appraiser and “company liaison”, the insights of the sponsor, and the openness and buy-in of the interviewees. 

Like SCAMPI, PPP’s method has a planning/preparation phase, a conduct phase consisting of interviews and artifact reviews, and a concluding phase where results are documented and delivered.  But as its name suggests, it has been developed in the spirit of lean initiatives.  It is intended to be an alternate approach to, not a replacement for, a SCAMPI appraisal.

The main differences from SCAMPI are:

  • There is a very essential up-front planning activity, but the output is a “lean” plan that contains only what is needed for a successful appraisal, not what is needed for the SEI and/or CMMI Institute
  • There is no required artifact review prior to the on-site appraisal; the artifact review of a small selected set of artifacts takes place during the interviews
  • There are no ratings and therefore no coverage rules; the company chooses which models, PAs, SGs, SPs, etc. are important and focuses on them (i.e. depth over breadth)
  • Different project types can – and should - make different choices (thus the appraisal can easily be used to cover infrastructure, maintenance, and Agile-based projects
  • Small full-time teams can be used; part-time team members are used as necessary
  • There is only one required general meeting; the Opening Meeting
  • The team does not spend a lot time entering data into worksheets and mapping observations to model elements; immediately after each interview business improvement opportunities are recorded from the team’s notes and these become the basis for the Final Findings Report
  • The interviews might take longer, but they may not need as many attendees
  • Findings only include Opportunities for Improvement
  • And, of course, there are no SAS entries needed

In summary, PPP’s appraisal method will cost significantly less (we estimate about 1/3 of the cost of a SCAMPI) and yet produces what is really needed – the findings which are the basis for your action plans to improve your business.  The section below contains a more detailed comparison of SCAMPI and CLAM.  The CLAM User’s Manual – CLAM V 1.0 – van be found by clicking on the web page link above.



More Detailed SCAMPI/CLAM Comparison

SCAMPI Cost

Much of the cost a company absorbs for a typical SCAMPI occurs prior to the “conduct” phase of the appraisal, preparing for the SCAMPI by:

(1) Doing internal work (or, worse yet, hiring consultants) to do various gap analyses and develop and implement remediation plans.  While these efforts may be worthwhile, arguably they should be performed as BAU (Business As Usual), not as pre-SCAMPI activities.  In addition it is unfortunately a common occurrence that organizations will develop processes and artifacts which have little or no business value, but are deemed necessary to “pass” a SCAMPI.

(2) Developing a SCAMPI Plan.  While planning is almost always a good thing, the reality is that the primary customer for these plans is the SEI, not the company itself.  The contents of the plan are driven by SCAMPI MDD requirements, not by the company’s improvement needs.  The SCAMPI Plan usually becomes extensive and expensive shelfware.

(3) Gathering and reviewing the artifacts.  While the latest MDD has the potential for making the cost of gathering – and reviewing - the required artifacts smaller, the cost is still prohibitive (and there is no data of which I am aware to show the cost has indeed been lowered).  For example if there were 500 artifacts and it took 30 minutes to define, collect, review, and annotate the review (and these are very conservative estimates) the cost would be 250 man-hours.

In addition there is extensive cost for the SCAMPI conduct phase due to:

(1) Large teams.  The MDD requires 4-10 team members.  So, for example, a 12-day Level 3 SCAMPI with seven internal team members, results in a cost to the company of almost 700 man-hours.

(2) Three meetings.  The MDD requires an Opening Meeting (and/or a Participant Briefing), a Prelimianary Findings Meeting/Presentation, and a Final Findings Presentation.  Generally all interviewees and team members attend these meetings.  If these three meetings take an hour each and there are 40 attendees, the cost is 120 man-hours.

(3) Coverage rules.  The MDD rules for coverage, collaboration, and so on results in much time spent logging straightforward “strengths” and reviewing the observation worksheets to ensure all rules are met.

CLAM Cost: 

By comparison CLAM requires no preliminary artifact collection in favor of selected artifact review as part of the conduct phase.  The only preparation activities are:

(1) Meeting with the “company liaison.”  This person should be a senior person who has been designated by the sponsor to work hand-in-hand with the CLAM LA to prepare and conduct the appraisal.  It is imperative that this person is highly regarded by the company, is familiar with the organizational structure, understands the project portfolio, and has some knowledge of the CMMI and any other relevant models and/or standards to be used in the appraisal (more on this later).  This meeting may take 2-3 man days (less if the CLAM lead appraiser is already familiar with the company) and results in a draft appraisal plan.  This plan should be more than four pages in length; its primary content is to define the areas most in need of improvement and to set up a tentative interview schedule so that these areas can be explored in depth and other areas of interest can be examined.  The plan may also include a tentative list of key artifacts that interviewees should bring with them, although this list could be prepared during the conduct phase..

(2) Meeting with sponsor.  The CLAM lead appraiser and liaison meet with the sponsor to review and/or modify the plan.  It is imperative that the sponsor understands that he or she needs to be truly involved and has the responsibility of supporting the appraisal’s needs.  After the meeting the plan is modified if necessary.  This meeting and modification should take no more than a few hours.

In the conduct phase, CLAM may require more time for interviews, since the only document review is done in “real time” during the interviews.  However this extra time is offset by not requiring all the “tick-the-box” questions whose answers are known and whose primary purpose is to satisfy the set of MDD rules.   With the time saved for meetings and consolidation, the CLAM conduct phase should take about two-thirds the time of the SCAMPI conduct phase.

SCAMPI Breadth and Flexibility: 

Of course the MDD puts no restrictions on the breadth (i.e. the portion of the organization’s work covered by the appraisal) of the SCAMPI.  However, since all projects are responsible for all PAs within the model scope (with the only exception being SAM) – and all practices for those projects must be examined - generally some subset of project types are chosen for the OU.   This is borne out by PPP’s personal experience: based on their involvement in over 30 appraisals only a handful of them have covered the whole organization.  For example for defense firms IR&D projects are generally not part of the Organizational Unit (the OU is that portion of an organization from which the sample projects are picked and to which the appraisal results apply).  In addition service projects are not included with development projects (multi-model appraisals – using CMMI-DEV and CMMI-SVC are now being performed, but infrequently and, presumably, inefficiently).  For IT organizations, SCAMPIs generally are performed only for large development projects; infrastructure projects and maintenance projects (which frequently cover more than half of IT’s work) are generally excluded (the former may be included for a Level 2 appraisal, but not for Level 3; the latter is almost always exluded).  The rationale is simple: the excluded projects generally follow a slimmed-down version of the processes used by the included projects and thus their inclusion in the OU puts the attainment of their rating goals at great risk.  For the same reasons projects which follow non-standard development life cycles (such as Agile or other iterative approaches) are excluded from the appraisal even though those projects are often pilots whose inclusion in the appraisal is clearly of highest priority).

CLAM Breadth and Flexibility

Since CLAM has no ratings – only improvement opportunities (and strengths, if desired) – and no coverage rules, there is no reason to exclude any type of projects.  The planning should broadly categorize the various project types to ensure that all types are covered, subject to sponsor guidance.  Careful planning is the key to making this work.  Since CLAM is not bound by any coverage rules, the planning should carefully define which process areas, goals, and even practices have business significance to each project type.

The advantages of CLAM goes beyond easy extension to all project types.  While the SCAMPI does allow for multi-model appraisals, there are strict rules for doing so which many companies find prohibitive.  In CLAM the planning can easily have any mix of CMMI-DEV and CMMI-SVC PAs, subject to business needs.  CLAM can also allow other models and/or methods to be brought in.  For example, if a project using Agile Scrum is included, the interviews can look at some of the techniques and artifacts unique to that (which may or may not be mapped to CMMI goals, in accordance with the plan).  There is no limit to these extensions, but in practice they will be limited by time and team expertise.  It should be noted here that CLAM has no rules for full participation by team members, so part-time team members can be used to deal with team knowledge gaps.

Quality Discussion:

If one looks at the history of the various CMM/CMMI appraisal methods, it is quite apparent that the rigor of the methods provides  somewhat of a guarantee that there is some level of confidence that the ratings truly reflect the company’s (or at least the OU’s) process maturity.  That is a quality attribute of the SCAMPI method that CLAM certainly doesn’t have.  

On the other hand, the SCAMPI methodology rigor can have an adverse effect.  This can happen in various ways:

(1)  Some practices have little or no business relevance for some or all projects.   This generally results in unnecessary work, sometimes even including the creation of artifacts to pass the appraisal.  For example, consider formal alternative design decisions (TS SP 1.1 – develop alternative solutions and selection criteria).  Its value for small enhancements to existing products is questionable, since most likely any design choices were made in the past, often prior to the existence of the OU’s current procedures.

(2)  In contrast, some practices are much more important than others.  Following the MDD, once the artifacts and/or affirmations indicate that an SP is “satisfied” there is no need to delve further, even though business needs suggest otherwise.  For example, consider estimation (PP SG 1 – establish estimates).  Typically a project shows its WBS, shows estimation worksheets indicating the use of attributes and the life cycle choice, and shows traceability from effort estimates to cost estimates.   This is good, but there are so many more issues.  For example, accuracy; how accurate are the estimates and, equally important from a business point of view, how accurate do they need to be?  What about phased estimates?  And how do the PMs collect estimates from the various estimation stakeholders and review them? And so on?

The point here is that the MDD rules can result in all SGs and SPs to be treated in a near equal manner (especially with tight time constraints).  But we all know that they are not equal in business significance.  CLAM is better suited to allow more time for the more important ones. 

Website Builder