<?xml version="1.0" encoding="UTF-8"?><document><element name="title"><value> Evaluation in Distance Education and E-Learning : The Unfolding Model</value></element><element name="body"><value> Guilford, New York, 2009 ISBN 978-1-59385-872-8 With the rapid proliferation of distance education and e-learning courses, the need is growing for a comprehensive, professional approach to evaluating their effectiveness. This indispensable book offers a road map to guide evaluation practice in these innovative learning environments. Providing practical, step-by-step guidelines and tools for conducting evaluation studies-including how to deal with stakeholders, develop surveys and interview protocols, collect other scientific evidence, and analyze and blend mixed-methods data-the work also features a template for writing high-quality reports. The "unfolding model" developed by the authors draws on Messick's influential assessment framework and applies it to program evaluation. Two case studies of actual programs (a distance learning course and an e-learning course) demonstrate the unfolding model in action. Valerie Ruhe is an Evaluation Studies Specialist at the University of British Columbia. Previously, she was an Assessment and Evaluation Consultant at the Center for Teaching and Learning, University of Minnesota. She has 10 years of professional program evaluation experience in distance education, K-12, and higher education. Bruno D. Zumbo is Professor of Measurement, Evaluation, and Research Methodology, and of Statistics at the University of British Columbia. He is widely published in research methodology, validity, and validation processes, as well as statistical science and program evaluation methodology. His work has had wide-ranging influence across many fields in the social, educational, and health sciences. Content: l. Why Do We Need a New Approach to Evaluation in Distance Education and E-learning? Distance Education versus E-Learning The Rapid Expansion of Distance Education and E-Learning What Is Evaluation? Why Do We Need a Professional Approach to Evaluation? What Does a Professional Approach to Evaluation Look Like? Responding to the Call for a Professional Evaluation Approach: The Unfolding Model Conclusions: Our Approach to Evaluation 2. The Theory and Practice of Program Evaluation Why Are Program Evaluation Models Important? Classification Frameworks for Program Evaluation Models Alkin and Christie's Evaluation Tree: The Roots and Branches Where Do We Diverge from Alkin and Christie? Conclusions 3. Evaluation Theory and Practice in Distance Education and E-Learning Evaluation Theory Models with Scientific Evidence in the Foreground Models Based on Evidence, Values, and Consequences Models Based on Messick's (1989) Framework A Summary of Evaluation Models in Distance Education Evaluation Studies Evaluation Practice Do Unintended Consequences Emerge in Authentic Evaluation Studies? Conclusions 4. Messick's Framework: What Do Evaluators Need to Know? The Overlap between Test Validity and Program Evaluation Messick's Contributions Messick's Framework The Overlap among the Four Facets The Controversy over Unintended Consequences Implications for Evaluation Conclusions 5. Getting Started Planning the Evaluation Study The Ethics Review Process The Political Context of Evaluation Using the Unfolding Model as a "Road Map" Mixed Methods: Blending Quantitative and Qualitative Data What Is Essential to Our Approach? Tailoring the Unfolding Model to Your Needs Conclusions 6. The Unfolding Model: Scientific Evidence Definition of "Scientific Evidence" Scientific Evidence How to Write Good Survey Questions Using the Unfolding Model to Write Survey and Interview Questions Administering Surveys Analyzing Survey Data Qualitative Data: Interviews, Focus Groups, and Online Ethnographies Qualitative Data Analysis Outcomes The Evaluation of Environmental Quality Relevance Cost-Benefit Analysis Bringing It All Together: Mixed Methods Conclusions 7. The Unfolding Model: Values and Consequences Underlying Values How to Identify the Values Underlying Your Course Course Goals and Objectives Writing Survey/Interview Questions about Underlying Values Analyzing Data on Underlying Values Unintended Consequences How Can You Identify Unintended Consequences? Writing Survey/Interview Questions about Unintended Consequences Analyzing Data on Unintended Consequences How to Enhance the Validity of Your Findings Recommendations for Course Improvement Writing the Evaluation Report Conclusions 8. Findings from Two Authentic Case Studies Methods and Procedures for Both Studies Distance Learning: Computing Science 200 (CPSC 200) E-Learning: Professional Writing 110 (PWRIT 110) Conclusions 9. Bringing It All Together Using Messick's Framework to Evaluate Distance and E-Learning Courses Using the Unfolding Model to Evaluate Your Courses Conducting Your Evaluation Study What Have We Learned from Two Case Studies? E-Learning and Beyond: Is the Unfolding Model the Last Word? The Future of Distance Education and E-Learning The Future of the Unfolding Model Conclusions Appendix A. Summary of the 1994 Program Evaluation Standards Appendix B. Glossary Appendix C. List of Associations Reviews: "Ruhe and Zumbo have written the premier text for evaluation of distance education and e-learning. This is the first theoretically grounded, comprehensive guide for conducting rigorous process and outcome studies of one of the fastest growing segments of curriculum development and education. It will be valuable to those involved in evaluating innovative educational practices and programs, today and for years to come. The book is unique in presenting both evaluation theory and practice, making it an excellent course text and practical resource." -Christina A. Christie, PhD, School of Behavioral and Organizational Sciences, Claremont Graduate University "I found this book intriguing and worthy of thoughtful discussion by students, practitioners, and theorists. It provides a good introduction to program evaluation in the context of distance education and e-learning, and to the authors' unfolding model. I will definitely use this book in the four graduate courses I teach on evaluation and qualitative inquiry. The book will help our Instructional Psychology and Technology students sort through the issues, form their own theories, and refine their practices. Students need this kind of deep discussion of the role of stakeholder values, how to respond dynamically to continual changes in technology, and how to integrate evaluation into the design process to enhance their distance education and e-learning processes and products." -David Dwayne Williams, PhD, Department of Instructional Psychology and Technology, Brigham Young University "A much-needed text. While there are resources available to aid in the design of e-learning materials, there is very little out there focusing on how to document their effectiveness. This book presents an approach to evaluation and also provides the theory that underlies that approach. Graduate students in evaluation and instructional technology will find it a useful resource because of its balance of theory and practice. Anyone involved in designing or evaluating e-learning or technology-supported instruction needs to take a look at this book!" -Dianna Newman, PhD, Department of Educational and Counseling Psychology, University at Albany, State University of New York "This book should have a significant impact on the field. Distance education is no longer on the periphery of teaching and learning; rather, it is now widely practiced and applied. This book is well written, clearly explained, and relevant to all those who are interested in best practices in distance education and e-learning." -Michael Simonson, PhD, Instructional Technology and Distance Education Program, Nova Southeastern University</value></element></document>