Performance Evaluation : Proven Approaches for Improving Program and Organizational Performance.

Intro -- Performance Evaluation: Proven Approaches for Improving Program and Organizational Performance -- Contents -- Acknowledgments -- Preface -- The Author -- Part One: Introduction to Evaluation -- One: Foundations of Evaluation -- A Brief Overview of Evaluation History -- Evaluation: Purpose and Definition -- Performance Improvement: A Conceptual Framework -- Making Evaluation Happen: Ensuring Stakeholders' Buy-In -- The Evaluator: A Job or a Role? -- The Relationship to Other Investigative Processes -- When Does Evaluation Occur? -- General Evaluation Orientations -- Challenges That Evaluators Face -- Ensuring Commitment -- Benefits of Evaluation -- Basic Definitions -- Key Points -- Reflection Questions -- Two: Principles of Performance-Based Evaluation -- Principle 1: Evaluation Is Based on Asking the Right Questions -- Principle 2: Evaluation of Process Is a Function of Obtained Results -- Principle 3: Goals and Objectives of Organizations Should Be Based on Valid Needs -- Principle 4: Derive Valid Needs Using a Top-Down Approach -- Principle 5: Every Organization Should Aim for the Best That Society Can Attain -- Principle 6: The Set of Evaluation Questions Drives the Evaluation Study -- Key Points -- Reflection Questions -- Part Two: Models of Evaluation -- Three: Overview of Existing Evaluation Models -- Overview of Classic Evaluation Models -- Objective-Based Evaluation: Tyler (1949) -- Consumer-Oriented Evaluation: Scriven (1967) -- Discrepancy Model of Evaluation: Provus (1971) -- Goal-Free Evaluation: Scriven (1974) -- Responsive/Client-Centered Evaluation: Stake (1975) -- Utilization-Focused Evaluation: Patton (1997) -- Selected Evaluation Models -- Selecting a Model -- Conceptualizing a Useful Evaluation That Fits the Situation -- Key Points -- Reflection Questions -- Four: Kirkpatrick's Four Levels of Evaluation.

Kirkpatrick's Levels -- Level 1 Evaluation: Reactions -- Level 2 Evaluation: Learning -- Level 3 Evaluation: Behavior -- Level 4 Evaluation: Results -- Comments on the Model -- Strengths and Limitations -- Application Example: Wagner (1995) -- Methodology -- Findings -- Issues -- Key Points -- Reflection Questions -- Five: Phillips's Return-On-Investment Methodology -- Phillips's ROI Process -- Collecting Postprogram Data -- Isolating the Effects of Training -- Converting Data to Monetary Values -- Tabulating the Costs of the Program -- Calculating the Return on Investment of the Program -- Identifying the Intangible Benefits of the Program -- Comments on the Model -- Strengths and Limitations -- Application Example: Blake (1999) -- Methodology -- Findings -- Key Points -- Reflection Questions -- Six: Brinkerhoff's Success Case Method -- The SCM Process -- Strengths and Weaknesses -- Application Example: Brinkerhoff (2005) -- Methodology -- Findings -- Key Points -- Reflection Questions -- Seven: The Impact Evaluation Process -- The Elements of the Process -- Step 1: Identifying Stakeholders and Expectations -- Step 2: Determining Key Decisions and Objectives -- Step 3: Deriving Measurable Indicators -- Step 4: Identifying Data Sources -- Step 5: Selecting Data Collection Methods -- Step 6: Selecting Data Analysis Tools -- Step 7: Communication of Results and Recommendations -- Comments on the Model -- Strengths and Limitations -- Application Example -- Methodology -- Findings -- Key Points -- Reflection Questions -- Eight: The CIPP Model -- Stufflebeam's Four Types of Evaluation -- Context -- Input -- Process -- Products -- Articulating Core Values of Programs and Solutions -- Methods Used in CIPP Evaluations -- Strengths and Limitations -- Application Example: Filella-Guiu and Blanch-Pana (2002) -- Methodology -- Findings -- Key Points.

Reflection Questions -- Nine: Evaluating Evaluations -- Evaluation Standards -- The American Evaluation Association Principles for Evaluators -- Application Example: Lynch et al. (2003) -- Methodology -- Findings -- Key Points -- Reflection Questions -- Part Three: Tools and Techniques of Evaluation -- Ten: Data -- Characteristics of Data -- Hard Data -- Soft Data -- Qualitative and Quantitative -- Scales of Measurement -- Defining Required Data from Performance Objectives -- Deriving Measurable Indicators -- Finding Data Sources -- Follow-Up Questions and Data -- Key Points -- Reflection Questions -- Eleven: Data Collection -- Observation Methodology and the Purpose of Measurement -- Person-Centered Direct Observation Methods -- Person-Centered Indirect Observation Techniques -- Instrument-Centered Methods -- Traditional Knowledge Testing -- Reliability and Validity of Tests -- Treatment-Centered Methods -- Experimental Research and Evaluation: The Relationship -- Designing the Experiment -- Problems with Classic Experimental Studies in Applied Settings -- Time-Series Studies -- Simulations and Games -- Determining What to Model in a Game or Simulation -- Using Simulation and Gaming for Evaluation -- Document-Centered Methods -- Extant Data Review -- Literature Review -- Artifacts and Work Products -- Conclusion -- Key Points -- Reflection Questions -- Twelve: Analysis of Evaluation Data: Tools and Techniques -- Analysis of Models and Patterns -- Analysis Using Structured Discussion -- Imposing Structure on Emerging Issues -- Relevance -- Controls on Structured Discussion -- Methods of Quantitative Analysis -- Statistics -- Measures of Central Tendency -- Measures of Dispersion (Variability) -- Graphical Representations of Data -- Measures of Relationship -- Inferential Statistics: Parametric and Nonparametric -- Interpretation -- Key Points.

Reflection Questions -- Thirteen: Communicating the Findings -- Recommendations -- Considerations for Implementing Recommendations -- Developing the Report -- Know the Audience -- Identify the Key Message: Take Action! -- Make Clear Distinctions and Linkages -- Be Clear About the Stakeholders' Responsibilities -- The Evaluator's Role after the Report -- Key Points -- Reflection Questions -- Part Four: Continual improvement -- Fourteen: Common Errors in Evaluation -- Errors of System Mapping -- Error of Causal Path -- Error of Misattribution -- Error of the Missing Player -- Error of Local Value -- Errors of Logic -- Error of False Conclusion -- Error of the Expert -- Error of Wishful Thinking -- Error of Association -- Error of the Quick Fix -- Error of Explanation Creep -- Errors of Procedure -- Error of the Instrument -- Error of Measurement -- Error of the Model -- Error of the Artifact -- Error of the Blind Spot -- Error of Displacement Shift -- Error of the Forked Path -- Error of Population Masking -- Error of the Average Person -- Conclusion -- Key Points -- Reflection Questions -- Fifteen: Continual Improvement -- What Is Continual Improvement? -- Monitoring Performance -- Building Continual Improvement in Organizations -- Characteristics of the Self-Evaluation Framework -- Adjusting Performance -- The Role of Leadership -- Key Points -- Reflection Questions -- Sixteen: Contracting for Evaluation Services -- The Contract -- Fixed Price -- Cost Plus Fixed Fee -- Time and Materials -- Consulting Retainer -- Contracting Controls -- Scope of Work -- Statement of Work -- Other Contract Clauses -- Management Plan -- Program Review -- Schedule and Work Breakdown Structure -- Delivery and Acceptance -- Ethics and Professionalism -- Sample Statement of Work -- Key Points -- Reflection Questions -- Seventeen: Intelligence Gathering for Decision Making.

Performance Measurement Systems -- Issues in Performance Measurement Systems -- Conclusion -- Key Points -- Reflection Questions -- Eighteen: The Future of Evaluation in Performance Improvement -- Evaluation and Measurement in Performance Improvement Today -- What Does the Future Hold? -- Conclusion -- Key Points -- Reflection Questions -- References and Related Readings -- Index.

Description based on publisher supplied metadata and other sources.

Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2024. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries.