CMMI® Assessments: Motivating Positive Change

CMMI® Assessments: Motivating Positive Change

Released February 2005
Publisher(s): Addison-Wesley Professional
ISBN: 9780321179357

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

Use Assessments to Drive Process Improvements in Software, Systems, Human Resources, and Beyond

Pioneered by the world's leading innovators in software engineering, assessments have evolved into a remarkably powerful tool for process improvement in areas ranging from systems to services, hardware to human resources.

Unlike audits, assessments are conducted from the inside. When handled correctly, assessments can help transform technical and managerial culture, dramatically improving both quality and profitability. In CMMI Assessments: Motivating Positive Change, two of the field's most respected leaders show exactly how to use them for maximum business advantage. Writing for executives, managers, technical professionals, and assessors themselves, Marilyn Bush and Donna Dunaway illuminate every phase of the assessment process—from planning through post-assessment follow-up.

The authors begin with an expert overview of what assessments entail, when they make sense, how to set achievable goals for them, and how to lead them to success. Next, they "drill down" into each stage of the process, presenting step-by-step instructions and defining the roles and responsibilities of every participant. Coverage includes creating and training assessment teams; identifying assessment products; consolidating interview data and other onsite activities; presenting results; and using those results productively.

Drawing on their unsurpassed experience leading assessments and mentoring assessors, they offer deep insights into the real-world challenges and obstacles you'll face—and proven solutions. They also present an extended case study showing how a real software development organization drove continuous improvement through four years of iterative assessments—moving from CMM Level 2 to elite Level 5 status, and driving dramatic business benefits along the way.

© Copyright Pearson Education. All rights reserved.

Table of contents

  1. Copyright
    1. Dedication
  2. Foreword
    1. How It All Started
    2. Early Software Experiences
    3. Assessment Issues
    4. The SEI and the CMM
    5. CMM Development
    6. What Next?
  3. Preface
    1. Intended Audience for the Book
    2. Assessments Are Part of the Larger Subject of Process Improvement
    3. Software Versus Systems and Hardware Process Improvement Assessments
    4. CMM/CBA IPI Versus CMMI/SCAMPI
      1. Staged Versus Continuous Assessments
    5. Anecdotal Histories
    6. A Roadmap Through the Book
  4. Acknowledgments
  5. About the Authors
  6. 1. Why do Assessments?
    1. 1.1. What Assessments Do
    2. 1.2. The Four Principal Functions of Assessments
      1. 1.2.1. Function 1: Assessments Serve as Analytical Tools
      2. 1.2.2. Function 2: Assessments Function as Fulcrums of Positive Change
      3. 1.2.3. Function 3: Assessments Transform Organizations by the Way They Work
      4. 1.2.4. Function 4: Assessments Educate Organizations in Worldwide Best Practices
    3. 1.3. The Analytical Function of Assessments
      1. 1.3.1. The Importance of Reference Models
      2. 1.3.2. Assessments Stabilize Process and Prioritize Change
    4. 1.4. Assessments Function as Fulcrums of Positive Change
      1. 1.4.1. Assessments Effect Change by Involving and Motivating Organizations in Efforts of Self-Analysis
      2. 1.4.2. Assessments Effect Change Because They Help the Workers in an Organization Understand That Processes, Not People, Need to Be Fixed
      3. 1.4.3. Assessments Effect Change Because They Provide a Voice for Change Agents
      4. 1.4.4. Assessments Effect Change Because They Foster Follow-On Activities
    5. 1.5. Assessments Transform Organizations by the Way They Work
      1. 1.5.1. Assessments Transform Organizations by Getting Different People to See the Same Things the Same Way
      2. 1.5.2. Assessments Transform Organizations by Helping Senior Management’s Efforts at Unification
      3. 1.5.3. Assessments Transform Organizations by Softening a Culture of Blame, Permitting Staff the Freedom to Think About What Goes Wrong and How to Correct It
      4. 1.5.4. Assessments Transform Organizations by Encouraging People to Think Across Boundaries
      5. 1.5.5. Assessments Transform Organizations by Consolidating a Party of Improvement
      6. 1.5.6. Assessments Transform Organizations by Helping to Institutionalize Rigorous Analysis
    6. 1.6. Assessments Educate as They Analyze, Motivate, and Transform
    7. 1.7. Why Gaming the Results of an Assessment Doesn’t Help (Though Many Try)
    8. 1.8. Can Assessments Really Change an Organization? A Preview of an Extended Case History to Be Found in Chapter 12
      1. 1.8.1. Analysis
      2. 1.8.2. Initiation of Positive Change
      3. 1.8.3. Positive Transformation
      4. 1.8.4. Education
      5. 1.8.5. A Continuous Program of Assessment and Improvement
    9. 1.9. Bottom-Line Profit and Cost Numbers: Assessments Pay
  7. 2. A Brief History of Process Improvement Methodologies and Assessment Methods
    1. 2.1. The Beginnings of Modern Software Assessment Methodology
      1. 2.1.1. The Creation of the SEI
      2. 2.1.2. Assessments Versus Evaluations
        1. 2.1.2.1. The Inconsistency of Early SPA Results and Its Upshot
    2. 2.2. The SEI Capability Maturity Model
      1. 2.2.1. Capability Maturity Levels
      2. 2.2.2. The Structure and Philosophy of the CMM
      3. 2.2.3. Benefits from Succeeding Maturity Levels
        1. 2.2.3.1. Benefits from Maturity Level 2
        2. 2.2.3.2. Benefits from Maturity Level 3
        3. 2.2.3.3. Benefits from Maturity Level 4
        4. 2.2.3.4. Benefits from Maturity Level 5
      4. 2.2.4. The CMM Assessment Method: CBA IPI
    3. 2.3. The Three Principal Advantages of a Modern Software Assessment over Traditional Manufacturing-Based Audit Procedures
      1. (1) A Focus on the Reliability of the Organization as a Whole
      2. (2) Encouraging Creativity and Initiative by Emphasizing Goals Instead of Means
      3. (3) An Action-Oriented Approach
    4. 2.4. A Second Path Toward Software Process Improvement Assessments: The History of ISO-9000-3, Bootstrap, SPICE, and the CMMI
      1. 2.4.1. ISO 9000
      2. 2.4.2. Bootstrap
      3. 2.4.3. SPICE
      4. 2.4.4. From SPICE to the CMMI: “Continuous” Versus “Staged” Assessments
    5. 2.5. The CMMI: An Enlarged Structure and Scope
    6. 2.6. A Hybrid Assessment Approach: The CMMI SCAMPI
      1. 2.6.1. Differences Between the “Staged” SCAMPI and the CBA IPI Method
      2. 2.6.2. More Differences Between the SCAMPI and CBA IPI Methods
    7. 2.7. Informal or Reduced Assessments: Class B and Class C Assessments
  8. 3. Assessments: An Executive Overview
    1. 3.1. What Are a Senior Executive’s Responsibilities?
      1. 3.1.1. Choosing an Assessment Methodology and Reference Model
      2. 3.1.2. Selecting a Lead Assessor
      3. 3.1.3. Establishing (with the Lead Assessor) the Business Goals and the Scope of the Assessment
        1. 3.1.3.1. Business Goals
        2. 3.1.3.2. Organizational Scope
        3. 3.1.3.3. Reference Model Scope
      4. 3.1.4. Appointing an Assessment Sponsor from Senior Management
        1. 3.1.4.1. The Responsibilities of the Assessment Sponsor
      5. 3.1.5. Establishing Appropriate Organizational Understanding
    2. 3.2. What Are the Phases of an Assessment?
      1. 3.2.1. Phase One of an Assessment: Planning and Preparation Activities
        1. 3.2.1.1. Choosing a Time
        2. 3.2.1.2. Appointing an Organization Site Coordinator
          1. The Responsibilities of the Organization Site Coordinator
        3. 3.2.1.3. The Assessment Plan
        4. 3.2.1.4. Creating an Assessment Team
        5. 3.2.1.5. Defining the Final Assessment Products
        6. 3.2.1.6. Selecting Projects to Be Assessed
        7. 3.2.1.7. Selecting People to Be Interviewed
        8. 3.2.1.8. Distribution of Questionnaires
        9. 3.2.1.9. Assessment Team Training
        10. 3.2.1.10. The Assessment Team’s Last Pre-Onsite Activities
      2. 3.2.2. Phase Two of the Assessment: Onsite Activities
        1. 3.2.2.1. The Kick-Off Meeting and Other Presentations
        2. 3.2.2.2. Collecting and Managing Data
          1. 3.2.2.2.1. Interviews
          2. 3.2.2.2.2. Consolidating Data from Interviews and Other Sources
      3. 3.2.3. Phase Three: Summing Up and Presenting the Results
        1. 3.2.3.1. Consolidating Draft Findings
        2. 3.2.3.2. Draft Findings Meetings
        3. 3.2.3.3. Final Findings Consolidation and the Establishment of Ratings, Usually Including a Capability Maturity Level Rating
        4. 3.2.3.4. Presenting Final Findings Informally to the Assessment Sponsor (Optional)
        5. 3.2.3.5. The Final Findings Presentation
        6. 3.2.3.6. Post-Final Findings Meeting Executive Session (Optional)
        7. 3.2.3.7. Assessment Team Wrap-Up
    3. 3.3. Cost: How Much Time and Effort Does an Assessment Require?
      1. 3.3.1. Time and Effort
        1. 3.3.1.1. Resources and Time Estimates for SCAMPI
      2. 3.3.2. Summary
  9. 4. Planning and Preparing for an Assessment, Part 1: Senior Management Responsibilities
    1. 4.1. Selecting a Lead Assessor
      1. 4.1.1. Should a Lead Assessor Be Selected from Outside or Inside the Organization?
    2. 4.2. Determining the Business Goals and the Organizational and Reference Model Scope of the Assessment
      1. 4.2.1. Organizational Scope (Choosing a Division or Cross-Section of the Company to Be Assessed)
        1. 4.2.1.1. The Special Problem of Matrix Organizations
        2. 4.2.1.2. The Special Problem of Small Organizations
      2. 4.2.2. Reference Model Scope: Which Maturity Levels Should Be Assessed?
    3. 4.3. Choosing an Assessment Sponsor
    4. 4.4. Establishing Appropriate Organizational Understanding
  10. 5. Planning and Preparing for an Assessment, Part 2: Choosing a Time. Formulating an Assessment Plan. Appointing an Organization Site Coordinator and Organizing Logistics
    1. 5.1. Choosing a Time for the Assessment
    2. 5.2. The Assessment Plan
      1. 5.2.1. How Far in Advance to Begin Formulating the Assessment Plan
      2. 5.2.2. How (and by Whom) the Assessment Plan Is Written
      3. 5.2.3. General Features
        1. 5.2.3.1. The Assessment Plan: Identifying Confidence Factors and Risks in the Coming Assessment
      4. 5.2.4. While the Details of the Assessment Plan Are Being Finalized: A List of Interim Tasks
      5. 5.2.5. A Full-Scale Template of a Working Assessment Plan
      6. 5.2.6. The Assessment Plan: Managing Resources and Estimating Costs
    3. 5.3. Managing Logistics: Appointing an Organization Site Coordinator
      1. 5.3.1. The Logistical Responsibilities of the Organization Site Coordinator and Associated Staff
      2. 5.3.2. A Sample Logistics Checklist
    4. 5.4. Assessment Readiness: When Is an Organization Ready for an Assessment?
  11. 6. Planning and Preparing for an Assessment, Part 3: Creating an Assessment Team. Selecting Projects to Be Assessed. Selecting People to Be Interviewed. Defining the Final Assessment Products. Distributing Questionnaires
    1. 6.1. Selecting the Assessment Team
      1. 6.1.1. How Big Should the Team Be?
      2. 6.1.2. Insiders Versus Outsiders
      3. 6.1.3. Official CBA IPI and SCAMPI Team Member Selection Guidelines
      4. 6.1.4. Selecting Assessment Team Members: Sensitive Areas
    2. 6.2. Selecting Projects to Be Assessed
      1. 6.2.1. Project Selection Matrix Checklist (See 5.2.5)
    3. 6.3. Selecting People to Be Interviewed
      1. 6.3.1. Selecting People to Be Interviewed: Senior and Middle Management
      2. 6.3.2. Selecting People to Be Interviewed: Project Managers
      3. 6.3.3. Selecting People to Be Interviewed: Technical Groups
      4. 6.3.4. Selecting People to Be Interviewed: A Checklist Matrix (See 5.2.5)
    4. 6.4. Defining Final Assessment Products
    5. 6.5. Distributing Questionnaires
      1. 6.5.1. Specimen Section of the CBA IPI Maturity Questionnaire (Section for the Requirements Management KPA)
  12. 7. Planning and Preparing for an Assessment, Part 4: Assessment Team Training and Post-Training Activities
    1. 7.1. Assessment Team Training
      1. 7.1.1. The Lead Assessor’s Team Training and Leadership Responsibilities
      2. 7.1.2. Training the Assessment Team in the Reference Model
      3. 7.1.3. Training the Team in the CBA IPI and SCAMPI Assessment Methodologies
      4. 7.1.4. Topics Covered During an Assessment Team Training Course
      5. 7.1.5. Assessment Team Training: Team Building
        1. Team Partnership: Sensitive Areas
    2. 7.2. The Assessment Team’s Pre-Onsite Organization and Activities
      1. 7.2.1. Managing and Facilitating Responsibilities for the Tasks Ahead
      2. 7.2.2. The Assessment Team Examines Questionnaire Responses
      3. 7.2.3. The Assessment Team Begins to Review Documents
      4. 7.2.4. The Assessment Team Elaborates the Assessment Plan
      5. 7.2.5. The Assessment Team Begins to Develop Interview Questions
    3. 7.3. Preparing Organization Participants for What Is to Come
      1. 7.3.1. Pre-Onsite Assessment Participants Briefing
      2. 7.3.2. A Sample Assessment Participants Briefing
  13. 8. Onsite Activities, Part 1: The Kick-Off Meeting and Other Presentations. Collecting and Managing Documents Throughout the Assessment. Problems Associated with Immature Organizations
    1. 8.1. The Kick-Off Meeting and Other Presentations
      1. 8.1.1. The Opening Meeting
        1. Opening Meeting Features for Organizations with Little Assessment Experience
      2. 8.1.2. Other Presentations
    2. 8.2. Collecting and Managing Documents Throughout the Assessment
      1. 8.2.1. Reviewing Organization-Level Documents
      2. 8.2.2. Reviewing Project-Level Documents
      3. 8.2.3. Reviewing Implementation-Level Documents
      4. 8.2.4. How the Assessment Team Prioritizes Documents for Review
      5. 8.2.5. Reviewing Documents for General Context
      6. 8.2.6. Library Management
    3. 8.3. Problems Associated with an Immature Organization’s Desire to “Do Well” on an Assessment
      1. 8.3.1. The Abuse of Mock Interviews
      2. 8.3.2. Violating Confidentiality
      3. 8.3.3. Intimidation
      4. 8.3.4. Deliberate Obstruction
      5. 8.3.5. How an Experienced Lead Assessor Should Handle Sensitive Situations
  14. 9. Onsite Activities, Part 2: Interviewing
    1. 9.1. Interviewing: An Overview
    2. 9.2. Interviewing Dynamics
    3. 9.3. Assessment Team Roles During the Interview
    4. 9.4. The Stages of an Interview
      1. 9.4.1. Opening the Interview
      2. 9.4.2. Phases of the Interview
      3. 9.4.3. Typical Sequences of Questions in an Interview
      4. 9.4.4. Typical Time Allotments for Interview Questions
      5. 9.4.5. Looking for Specific Information: Kinds of Interview Questions
      6. 9.4.6. Direct or Open-Ended Questions in an Interview
      7. 9.4.7. Additional Sample Questions
      8. 9.4.8. More Details: Sample Technical Questions Concerning the Area of Software Project Planning (Similar Questions Must Be Formulated for Every Technical Area)
      9. 9.4.9. Judgmental Statements Are to Be Avoided
      10. 9.4.10. Closing the Interview
    5. 9.5. Note Taking
    6. 9.6. Different Interviews for Different Jobholders
      1. 9.6.1. Project Manager Interviews
        1. 9.6.1.1. Unexpected and/or Sensitive Situations
        2. 9.6.1.2. Project Manager Seating Arrangement: A Sample Configuration
      2. 9.6.2. Group Interviews
        1. 9.6.2.1. Sensitive Situations in Group Interviews
        2. 9.6.2.2. Two Variations of a Group Interview Seating Configuration
      3. 9.6.3. Middle Manager Interviews
      4. 9.6.4. Senior Management Interviews
      5. 9.6.5. SCAMPI Interviews
  15. 10. Onsite Activities, Part 3: The Day-to-Day Consolidation of Data
    1. 10.1. Consolidating Data: An Overview
    2. 10.2. Team Members Take Notes and Prepare to Construct “Observations” About Questionnaires, Documentation Reviews, Presentations, and Interviews
    3. 10.3. Transforming Notes into Observations
      1. 10.3.1. General Rules
      2. 10.3.2. Two Common Problems: Under- and Over-Emphasizing the Relevance of Information
      3. 10.3.3. Transferring Notes into Observations: Examples
        1. 10.3.3.1. An Example of Different Kinds of Note Taking
        2. 10.3.3.2. Observation Statements Based on Notes
      4. 10.3.4. Observations: Suitable Recording Media
    4. 10.4. How Consolidation Produces Day-to-Day Alterations in the Assessment Plan
    5. 10.5. Consolidation Is a Consensus Process
    6. 10.6. Warning: Consensus Must Not Be Deferred Until the Final Stages of an Assessment
    7. 10.7. A Lurking Disaster to Consensus: Misunderstanding the Model
    8. 10.8. The Special Requirements of the SCAMPI Consolidation Approach
      1. 10.8.1. Verify Objective Evidence
  16. 11. The Final Stages of an Onsite Assessment: Summing Up and Presenting Results
    1. 11.1. Consolidating Draft Findings
      1. 11.1.1. Draft Findings Presentation—A Sample Template
    2. 11.2. Draft Findings Meetings: An Overview
      1. 11.2.1. There Can Be Multiple Draft Findings Meetings
      2. 11.2.2. Practitioner Draft Findings Meetings
      3. 11.2.3. Management Draft Findings Meetings
    3. 11.3. The Team’s Final Consolidation: Ratings, Including the Maturity Level Rating
      1. 11.3.1. Final Consolidation
      2. 11.3.2. Rating: The Process
        1. 11.3.2.1. Prerequisites for Rating
        2. 11.3.2.2. Final Rating Process
        3. 11.3.2.3. Rating Goals
        4. 11.3.2.4. Rating of Process Areas/Key Process Areas
        5. 11.3.2.5. Maturity Level Ratings
        6. 11.3.2.6. An Example of How Initial Questions Lead to Rating Determinations as Traced from a Technical Interview Through Final Findings
      3. 11.3.3. Ratings: The Process Is Not Always Smooth
        1. 11.3.3.1. Coming to Consensus About Rating KPAs/PAs
        2. 11.3.3.2. How Lead Assessors can Negotiate Instances of Intransigence as Ratings Are Consolidated and Assigned
    4. 11.4. CMMI Continuous Model
      1. 11.4.1. Rating
      2. 11.4.2. Determining the Capability Profile
    5. 11.5. The Preparation of Final Findings
      1. 11.5.1. Final Findings Presentation—A Sample Template
      2. 11.5.2. Examples from Final Findings Presentations
    6. 11.6. Presenting Final Findings Informally to Senior Management (Optional)
    7. 11.7. The Final Findings Presentation
    8. 11.8. Post-Final Findings Executive Session (Optional)
    9. 11.9. Assessment Wrap-Up
      1. 11.9.1. Organization Wrap-Up
      2. 11.9.2. Assessment Data Sent to the SEI Appraisal Database
      3. 11.9.3. Feedback Relating to the Effectiveness of the Assessment Method
      4. 11.9.4. Confidence Report Template for CBA IPI
      5. 11.9.5. Assessment Acknowledgement Letter—Template
  17. 12. How to Use the Results of an Assessment Productively
    1. 12.1. Introduction: After an Assessment
      1. 12.1.1. What Happens the First Day After an Assessment
    2. 12.2. Who? (Who Drives a Disciplined Post-Assessment Plan? Who Makes It Work?)
      1. 12.2.1. Senior Management: The Push from Above
      2. 12.2.2. Two Time-Honored Mistakes
        1. 12.2.2.1. Mistake 1: Appointing a Process Improvement Manager Who Has No Real Authority
        2. 12.2.2.2. Mistake 2: Hiring a Consultant to Make Everything Better
    3. 12.3. When Should Post-Assessment Planning Begin? How Ambitious Should It Be?
      1. 12.3.1. What Should the Duration Be of a Post-Assessment Plan?
        1. Medium Term
        2. Long Term
    4. 12.4. What Does a Post-Assessment Improvement Plan Look Like? How Should It Unfold? In What Spirit Should It Be Undertaken?
      1. 12.4.1. The Principles of Building a Post-Assessment Process Improvement Action Plan
      2. 12.4.2. The Contents of a Post-Assessment Process Improvement Action Plan: An Overview
      3. 12.4.3. A Template of a Simple Plan to Identify and Track the Execution of Improvement Goals
      4. 12.4.4. The Components of a Thorough Process Improvement Plan: A Template Designed for a Mature Organization
      5. 12.4.5. A Common Temptation: Trying to Run Before You Walk
      6. 12.4.6. Why Maturity Levels Can’t Be Skipped
      7. 12.4.7. Planning for Reiterated Assessments
    5. 12.5. After the Plan: Managing the Introduction of Improved Processes
      1. 12.5.1. The Role of Senior Management in Implementing Improved Processes
        1. 12.5.1.1. Learning to Ask the Right Questions
        2. 12.5.1.2. Extending the Authority of Senior Management Without Diluting It
      2. 12.5.2. How to Create a Workable Ladder of Authority: An Example
      3. 12.5.3. The Role of Process Groups in Immature Organizations
    6. 12.6. Creating, Tracking, and Implementing a Post-Assessment Plan for Process Improvement
      1. 12.6.1. Background
      2. 12.6.2. A Summary of the First Assessment’s Findings
        1. 12.6.2.1. Organization Z: Final Findings from the First Assessment
        2. 12.6.2.2. The Assessment Findings: A Summary
      3. 12.6.3. Organization Z’s Post-Assessment Efforts and the Difference They Made One and Two Years Afterwards
      4. 12.6.4. Two Years Later: Organization Z’s Second CBA IPI Assessment
        1. 12.6.4.1. Organization Z’s Second Assessment: Summary of Achievements
        2. 12.6.4.2. Organization Z’s Second Assessment: Remaining Issues
        3. 12.6.4.3. Organization Z Second Assessment Findings
        4. 12.6.4.4. A Sample of Organization Z’s Proposals After a Second Assessment and What Difference They Made One and Two Years Later
      5. 12.6.5. Four Years Later: Organization Z’s Third CBA IPI Assessment, Undertaken Two Years After the Second Assessment and Four Years After the First Assessment
        1. 12.6.5.1. Organization Z’s Third Assessment: Summary of Achievements
        2. 12.6.5.2. Organization Z’s Third Assessment Findings
      6. 12.6.6. The Payoffs for a Level 5 Organization
        1. 12.6.6.1. The Bottom Line: Productivity and Performance Improvements in Organization Z
        2. 12.6.6.2. The Benefits of Reiterated Assessment in Company Z: Analysis, Positive Change, Transformation, and Education
          1. Analysis
          2. Initiation of Positive Change
          3. Positive Transformation
          4. Education
          5. A Continuous Program of Assessment and Improvement
  18. References

Product information

  • Title: CMMI® Assessments: Motivating Positive Change
  • Author(s):
  • Release date: February 2005
  • Publisher(s): Addison-Wesley Professional
  • ISBN: 9780321179357