Week 2 - Quality & Evaluation

1. Why is evaluation important to you and how do you define it?
   
   Evaluation is important as it can provide a range of different and useful information to assist in future course design, planning and implementation.
   
   Evaluation is the process through which we examine the learning opportunities and experiences we offer our students and make judgements about their effectiveness and educational value of techniques and resources, as well as the costs. As Heather stated in her Week 2 posting "evaluation is a way of determining what is needed". How? By looking at a broad range of evidence in order the gauge the effectiveness of the elearning – for example, including a course assessment as a method of evaluation data collection as a way of keeping quality issues clearly in focus.
   
   "Evaluation is a continuous ongoing process (Gunn, 1999) that is "fundamentally about asking questions, and then designing ways to try to find useful answers" (Manwaring and Calverley, 1998). It is an expensive and time-consuming process and it is essential that it is worthwhile:
   
   

   "If the answer to the question 'why evaluate?' is that the results will lead to action to improve the teaching and learning within the course or institution,
   then all the effort will be worthwhile." (Shaw, 1998). "

   (Higgison, C. 2001).
   
   
2. What sort of evaluations mentioned on the presentation are familiar to you already and why?
   
   • Observation – in some assessments, observation is required to identify whether the student is able to demonstrate competency in performing a task – this can be observed by audio, video recording or F2F.
   
   • Questionnaires – I have used online surveys, hard-copy evaluation handouts and verbal questions to evaluate teaching and learning.
   
   • Focus groups – regular meeting with our stakeholders –relevant Industry Advisory Board representatives.
   
   • Expert review – internal (pre- and post-) within a School by peers and/or Senior Lecturers, by Subject Matter Experts, by Curriculum & Academic Services at UCOL and across campuses, and external moderation with NZQA.
   
   • Checklists – depending on the paper or unit standard being assessed – often combined with Observation.
   
   • Feedback in a discussion forum – by quality participation, text chat feedback and the use voting buttons (in Elluminate).
   
   
3. Why is quality important in eLearning?
   
   Learning and teaching that incorporates the use of learning technologies is complex and evaluation is seen as the key to developing an understanding of the factors that influence its success. That success will depend on the quality of its instructional design and the academic and technical supported provided to learners and online tutors.
   
   I agree with Joy's comment: "evaluation is based on asking questions, collecting data or information that answer the questions, through analysis then make the decision and take action to ensure the whole project / system fulfil planned outcome. It can be varied depends on what questions we are asking and what result we would like to get."
   
   Quality assurance standards are the responsibility of the institution and therefore it is essential they evaluate their practices. It plays an important role is satisfying the demands of external (and internal) scrutiny. This ensures that innovations are subject to the same institutional scrutiny and evaluation processes as traditional F2F delivery.
   
   However, as Brownyn has suggested in her slide presentation – “we have got away it so far, and have not really bothered to do much evaluation apart from checking about what students think of the course at the end of the course …..” therefore examples of sound pedagogy and integrity of best practice in elearning are at risk. As Michelle states "it is imperative that the course material and activities are of a high quality."
   
   Specific indicators for measuring quality include:
   
   • assessment of student learning,
   • feedback from students,
   • peers and external reviewers and
   • institutional accreditation procedures.

---

Gunn, C. (1999). They love it but do they learn from it? Evaluating the educational impact of innovations. Higher Education Research and Development 18 (2): 185-199.

Higgison, C. (2001). Online tutoring e-book. Chapter 5 Evaluation. Institute for computer-based learning. Edinburgh.