In Week 3, from the eLearning Guidelines for New Zealand I identified the following, relevant to my area of practice:
- TD5: has a representative sample of students tested the eLearning materials and any necessary modifications been made?
(It is intended to measure the usability and effectiveness of the eLearning materials after implementation by students and respond by continual improvement to ensure viability over time). - TT13: does the teacher evaluate the eLearning during their course to identify its effectiveness and how to improve it?
(It is intended to measure the usability and effectiveness of the eLearning materials after implementation by teaching staff and respond by continual improvement to ensure viability over time).
Appropriate method for evaluation project
In Week 4, we considered Evaluation paradigms and models, I related to Tom Reeves’ Educational Paradigms # 4: eclectic-mixed methods (using multiple evaluation methods) and like Adrienne, where she states:
This will allow me to pick and choose a variety of research methods based on what will fit in with what I want to study.
Why? As Rachel suggests:
This triangulation should give a reasonable indication and support for the outcomes and success of adopting flexible options to a F2F course.
Background
I am currently working with teaching and administration staff at UCOL Whanganui who teach F2F (or administer) the Certificate of Business Studies (CBS) programme. This programme also incorporates the NZIM Certificate in Management qualification. Teaching staff are expected to have an eLearning presence (blended delivery) using Moodle in all the CBS (F2F) papers. Some staff had been using Blackboard, which is still available, but being phased out. Lecturing staff are expected to develop a Moodle site for each of the 12 papers to support the students in a mainly F2F environment. They have limited support in the design plan of their Moodle site and may lack time and eLearning expertise. There are also no quality control measures in place. My role is an eLearning advisor only.
What is the purpose of the evaluation?
From the Six Facets of Instructional Product Evaluation identified by Professor Tom Reeves, the purpose of my evaluation after the eLearning materials are implemented so students and UCOL staff can measure their Moodle site for:
- usability; and
- effectiveness
From the WikiEd eLearning Guidebook Analysis of evaluation data I plan to use the following methods:
- summative evaluation; and possibly
- monitoring evaluation
I plan to follow the University of Tasmania Project evaluation framework to guide me in the development, support and application of a best (or good) practice checklist (or guidelines) to ensure best practice methods and quality control measures are recognised and supported at our institution.
Hi Kay, that looks like a good framework. It covers the bases, but isn't too huge and unwieldy.
ReplyDeleteI found a good checklist in easy to read language at http://elearning.typepad.com/thelearnedman/ID/evaluatingcourses.pdf. It is on the resources page too. I use it as a quick guide for myself to make sure I'm on track, and as a guide for staff who are floundering. See what you think of it.
great work Kay and a substantial amount of thinking portrayed here. I am thinking especially after our conversation that the guidelines need some additions. For example: TD5: has a representative sample of students tested the eLearning materials and any necessary modifications been made?
ReplyDeleteThis will work, as we discussed, if you add in a sub-question or two - perhaps 1. how easy to use do students find the online materials?
2. How effective is the design of the online course for learning?
For TT13: does the teacher evaluate the eLearning during their course to identify its effectiveness and how to improve it?
As I suggested, this could be re-worded to incorporate the range of people involved in designing and supporting the online course.
For example, What criteria can be used during the development of an eLearning course to guide best practice?
sub-question suggestion: Do different personnel involved in the development phase have different perspectives - e.g. designer versus teacher?
The checklist you showed me is very comprehensive and would certainly benefit from a bit of trimming to make it manageable for this project. I am looking forward to seeing your draft plan and hope you can manipulate it on Google documents. This is excellent progress Kay. :)