You are here:

A multi-component model for assessing learning objects: The learning object evaluation metric (LOEM)
ARTICLE

,

Australasian Journal of Educational Technology Volume 24, Number 5, ISSN 0814-673X Publisher: Australasian Society for Computers in Learning in Tertiary Education

Abstract

While discussion of the criteria needed to assess learning objects has been extensive, a formal, systematic model for evaluation has yet to be thoroughly tested. The purpose of the following study was to develop and assess a multi-component model for evaluating learning objects. The Learning Object Evaluation Metric (LOEM) was developed from a detailed list of criteria gathered from a comprehensive review of the literature. A sample of 1113 middle and secondary students, 33 teachers, and 44 learning objects was used to test this model. A principal components analysis revealed four distinct constructs: interactivity, design, engagement, and usability. These four constructs showed acceptable internal and inter-rater reliability. They also correlated significantly with student and teacher perceptions of learning, quality, and engagement. Finally, all four constructs were significantly and positively correlated with student learning performance. It is reasonable to conclude that the LOEM is reliable, valid, and effective approach to evaluating the effectiveness of learning objects in middle and secondary schools.

Citation

Kay, R., Knaack, L. & Knaack, L. (2008). A multi-component model for assessing learning objects: The learning object evaluation metric (LOEM). Australasian Journal of Educational Technology, 24(5),. Australasian Society for Computers in Learning in Tertiary Education. Retrieved August 11, 2024 from .

Keywords

References

View References & Citations Map
  1. Adams, A., Lubega, J., Walmsley, S. & Williams, S. (2004). The effectiveness of assessment learning objects produced using pair programming. Electronic Journal of e-Learning, 2(2). Http://www.ejel.org/volume-2/vol2-issue2/v2-i2-art1-adams.pdf
  2. Agostinho, S., Bennett, S., Lockyer, L. & Harper, B. (2004). Developing a learning object metadata application profile based on LOM suitable for the Australian higher education market. Australasian Journal of Educational Technology, 20(2), 191-208.
  3. Baruque, L.B. & Melo, R.N. (2004). Learning theory and instructional design using learning objects. Journal of Educational Multimedia and Hypermedia, 13(4), 343-370.
  4. Baser, M. (2006). Promoting conceptual change through active learning using open source software for physics simulations. Australasian Journal of Educational Technology, 22(3), 336-354.
  5. Bradley, C. & Boyle, T. (2004). The design, development, and use of multimedia learning objects. Journal of Educational Multimedia and Hypermedia, 13(4), 371-389.
  6. Brown, A.R. & Voltz, B.D. (2005). Elements of effective e-learning design. The International Review of Research in Open and Distance Learning, 6(1). Http://www.irrodl.org/index.php/irrodl/article/view/217/300
  7. Butson, R. (2003). Learning objects: Weapons of mass instruction. British Journal of Educational Technology, 34(5), 667-669.
  8. Buzzetto-More, N.A. & Pinhey, K. (2006). Guidelines and standards for the development of fully online learning objects. Interdisciplinary Journal of Knowledge and Learning Objects, 2006(2), 96104. Http://ijklo.org/Volume2/v2p095-104Buzzetto.pdf
  9. Caws, C., Friesen, N. & Beaudoin, M. (2006). A new learning object repository for language learning: Methods and possible outcomes. Interdisciplinary Journal of Knowledge and Learning Objects, 2006(2), 112-124. Http://ijklo.org/Volume2/v2p111-124Caws.pdf
  10. Convertini, V.C., Albanese, D., Marengo, A., Marengo, V. & Scalera, M. (2006). The OSEL taxonomy for the classification of learning objects. Interdisciplinary Journal of Knowledge and Learning Objects, 2, 125-138. Http://ijklo.org/Volume2/v2p125-138Convertini.pdf
  11. Del Moral, E. & Cernea, D.A. (2005). Design and evaluate learning objects in the new framework of the semantic web. In A. Mendez-Vila, B. Gonzalez-Pereira, J. Mesa Gonzalez& J.A. Mesa Gonsalez (Eds), Recent research developments in learning technologies (1-5). Spain: Formatux. [verified 28 Oct 2008] http://www.formatex.org/micte2005/357.pdf
  12. Deaudelin, C., Dussault, M. & Brodeur, M. (2003). Human-computer interaction: A review of the research on its affective and social aspects. Canadian Journal of Learning and Technology, 29(1), 89-110. Http://www.cjlt.ca/index.php/cjlt/article/view/34/31
  13. Downes, S. (2003). Design and reusability of learning objects in an academic context: A new economy of education? USDLA Journal, 17(1). [viewed 1 June 2007, verified 28 Oct 2008] http://www.usdla.org/html/journal/JAN03_Issue/article01.html
  14. Field, A. (2005). Discovering statistics using SPSS (2nd edition) Thousand Oaks, CA: SAGE Publications.
  15. Fourner-Viger, P. (2006). A cognitive and logic based model for building glass-box learning objects. Interdisciplinary Journal of Knowledge and Learning Objects, 2, 77-94.
  16. Friesen, N. & Anderson, T. (2004). Interaction for lifelong learning. British Journal of Educational Technology, 35(6), 679-687.
  17. Gadanidis, G., Sedig, K. & Liang, H. (2004). Designing online mathematical investigation. Journal of Computers in Mathematics and Science Teaching, 23(3), 275-298.
  18. Gibbons, A.S., Nelson, J. & Richards, R. (2000). The nature and origin of instructional objects. In D.A. Wiley (Ed.), The instructional use of learning objects: Online version. [viewed 1 July 2005, verified 28 Oct 2008] http://reusability.org/read/chapters/gibbons.doc
  19. Guadagnoli, E. & Velicer, W. (1988). On methods in the analysis of profile data. Psychometrika, 24, 95-112.
  20. Haughey, M. & Muirhead, B. (2005). Evaluating learning objects for schools. E-Journal of Instructional Science and Technology, 8(1). Http://www.ascilite.org.au/ajet/ejist/docs/vol8_no1/fullpapers/eval_learnobjects_school.htm
  21. Howard-Rose, D. & Harrigan, K. (2003). CLOE learning impact studies lite: Evaluating learning objects in nine Ontario university courses. [viewed 3 July 2007, not found 28 Oct 2008] http://cloe.on.ca/documents/merlotconference10.doc
  22. Jonassen, D.H. (2006). On the role of concepts in learning and instructional design. Educational Technology Research& Development, 54(2), 177-196.
  23. Kay, R.H. & Knaack, L. (2005). Developing learning objects for secondary school students: A multi-component model. Interdisciplinary Journal of Knowledge and Learning Objects, 1, 229-254.
  24. Kenny, R.F., Andrews, B.W., Vignola, M.V., Schilz, M.A. & Covert, J. (1999). Towards guidelines for the design of interactive multimedia instruction: Fostering the reflective decision-making of preservice teachers. Journal of Technology and Teacher Education, 7(1), 1331.
  25. Kline, P. (1999). The handbook of psychological testing (2nd edition). London: Routledge.
  26. Koohang, A. & Du Plessis, J. (2004). Architecting usability properties in the e-learning instructional design process. International Journal on E-Learning, 3(3), 38-44.
  27. Koppi, T., Bogle, L. & Lavitt, N. (2004). Institutional use of learning objects: Lessons learned and future directions. Journal of Educational Multimedia and Hypermedia, 13(4), 449-463.
  28. Koppi, T., Bogle, L. & Bogle, M. (2005). Learning objects, repositories, sharing and reusability. Open Learning, 20(1), 83-91.
  29. Krauss, F. & Ally, M. (2005). A study of the design and evaluation of a learning object and implications for content development. Interdisciplinary Journal of Knowledge and Learning Objects, 1. Http://ijklo.org/Volume1/v1p001-022Krauss.pdf
  30. Lim, C.P., Lee, S.L. & Richards, C. (2006). Developing interactive learning objects for a computing mathematics models. International Journal on E-Learning, 5(2), 221-244.
  31. Lin, A. & Gregor, S. (2006). Designing websites for learning and enjoyment: A study of museum experiences. International Review of Research in Open and Distance Learning, 7(3), 1-21.
  32. MacDonald, C.J., Stodel, E., Thompson, T.L., Muirhead, B., Hinton, C., Carson, B. & Banit, E. (2005). Addressing the eLearning contradiction: A collaborative approach for developing a conceptual framework learning object. Interdisciplinary Journal of Knowledge and Learning Objects, 1. Http://ijklo.org/Volume1/v1p079-098McDonald.pdf
  33. Malcolm, M. (2005). The exercise of the object: Issues in resource reusability and reuse. British Journal of Educational Technology, 26(1), 33-41.
  34. Maslowski, R. & Visscher, A.J. (1999). Formative evaluation in educational computing research and development. Journal of Research on Computing in Education, 32(2), 239-255.
  35. McCormick, R. & Li, N. (2005). An evaluation of European learning objects in use. Learning, Media and Technology, 31(3), 213-231.
  36. McGreal, R. (2004). Learning objects: A practical definition. International Journal of Instructional Technology and Distance Learning, 1(9). Http://www.itdl.org/Journal/Sep_04/article02.htm
  37. Metros, S.E. (2005). Visualizing knowledge in new educational environments: a course on learning objects. Open Learning, 20(1), 93-102.
  38. Muzio, J.A., Heins, T. & Mundell, R. (2002). Experiences with reusable e-learning objects from theory to practice. The Internet and Higher Education, 5(1), 21-34.
  39. Nesbit, J. & Belfer, K. (2004). Collaborative evaluation of learning objects. In R. McGreal (Ed.), Online education using learning objects. (pp. 138-153). New York: RoutledgeFalmer.
  40. Nesbit, J., Belfer, K. & Vargo, J. (2002). A convergent participation model for evaluation of learning objects. Canadian Journal of Learning and Technology, 28(3). Http://www.cjlt.ca/index.php/cjlt/article/view/110/103
  41. Nielson, J. (2003). Ten usability heuristics. [viewed 1 June 2007, verified 28 Oct 2008] http://www.useit.com/papers/heuristic/heuristic_list.html
  42. Nunnally, J.C. (1978). Psychometric theory. New York: McGraw-Hill.
  43. Nurmi, S. & Jaakkola, T. (2005). Problems underlying the learning object approach. International Journal of Instructional Technology and Distance Learning, 2(11). Http://www.itdl.org/Journal/Nov_05/article07.htm
  44. Ohl, T.M. (2001). An interaction-centric learning model. Journal of Educational Multimedia and Hypermedia, 10(4), 311-332.
  45. Oliver, R. & McLoughlin, C. (1999). Curriculum and learning-resources issues arising from the use of web-based course support systems. International Journal of Educational Telecommunications, 5(4), 419-435.
  46. Parrish, P.E. (2004). The trouble with learning objects. Educational Technology Research& Development, 52(1), 49-67.
  47. Reimer, K. & Moyer, P.S. (2005). Third-graders learning about fractions using virtual manipulatives: A classroom study. Journal of Computers in Mathematics and Science Teaching, 24(1), 5-25.
  48. Schell, G.P., & Burns, M. (2002). A repository of e-learning objects for higher education. E-Service Journal, 1(2), 53-64.
  49. Schoner, V., Buzza, D., Harrigan, K. & Strampel, K. (2005). Learning objects in use: ‘Lite’ assessment for field studies. Journal of Online Learning and Teaching, 1(1), 1-18. [verified 28 Oct 2008] http://jolt.merlot.org/documents/vol1_no1_schoner_001.pdf
  50. Siqueira, S.W.M., Melo, R.N. & Braz, M.H.L.B. (2004). Increasing the semantics of learning objects. International Journal of Computer Processing of Oriental Languages, 17(1), 27-39.
  51. Sosteric, M. & Hesemeier, S. (2002). When is a learning object not an object: A first step towards a theory of learning objects. International Review of Research in Open and Distance Learning, 3(2), 1-16. Http://www.irrodl.org/index.php/irrodl/article/view/106/185
  52. Sosteric, M & Hesemeier, S. (2004). A first step towards a theory of learning objects. In R. McGreal (Ed.), Online education using learning objects (pp. 43-58). London: Routledge Falmer.
  53. Wiley, D., Waters, S., Dawson, D., Lambert, B., Barclay, M. & Wade, D. (2004). Overcoming the limitations of learning objects. Journal of Educational Multimedia and Hypermedia, 13(4), 507521.
  54. Williams, D.D. (2000). Evaluation of learning objects and instruction using learning objects. In D.A. Wiley (Ed.), The instructional use of learning objects: Online version. [viewed 1 July 2005] http://reusability.org/read/chapters/williams.doc
  55. Van Merrienboer, J.J.G. & Ayres, P. (2005). Research on cognitive load theory and its design implications for e-learning. Education Theory, Research and Development, 53(3), 1042-1629.
  56. Van Zele, E., Vandaele, P., Botteldooren, D. & Lenaerts, J. (2003). Implementation and evaluation of a course concept based on reusable learning objects. Journal of Educational Computing and Research, 28(4), 355-372.
  57. Vargo, J., Nesbit, J.C., Belfer, K. & Archambault, A. (2002). Learning object evaluation: Computer mediated collaboration and inter-rater reliability. International Journal of Computers and Applications, 25(3), 1-8.

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References