You are here:

Basic sequence analysis techniques for use with audit trail data
Article

, , The University of Melbourne, Australia

Journal of Educational Multimedia and Hypermedia Volume 17, Number 3, ISSN 1055-8896 Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC USA

Full Text

Abstract

Audit trail analysis can provide valuable insights to researchers and evaluators interested in comparing and contrasting designers' expectations of use and students' actual patterns of use of educational technology environments (ETEs). Sequence analysis techniques are particularly effective but have been neglected to some extent because of real and perceived difficulties associated with their implementation and interpretation. This paper describes two simple sequence analysis techniques that are readily applied to audit trail data - a new sequence characterisation technique and a modification of an existing sequence comparison technique. The utility of these techniques are demonstrated through the analysis of audit trail data collected from two contrasting ETEs that were used by students in naturalistic settings. The first of these investigations examined the sequence in which students completed a single drag and drop task; the second examined the order in which students accessed the full complement of tasks within a modular ETE. The results of these analyses revealed that the order in which students completed the drag and drop task closely matched the tasks supporting narrative and that the order in which students accessed the various tasks in the modular ETE was strongly influenced by their physical location within on-screen menus. The concurrences of these findings with the various designers' expectations of use of the two ETEs are discussed.

Citation

Judd, T. & Kennedy, G. (2008). Basic sequence analysis techniques for use with audit trail data. Journal of Educational Multimedia and Hypermedia, 17(3), 285-306. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). Retrieved August 8, 2024 from .

© 2008 Association for the Advancement of Computing in Education (AACE)

Keywords

References

View References & Citations Map
  1. Agrawal, R., & Srikant, R. (1995). Mining sequential patterns. Data Engineering, 1995. Proceedings of the Eleventh International Conference on Data Engineering (pp. 3-14). Taipai, Taiwan.
  2. Cook, J. E., & Wolf, A. L. (1997). Software process validation: Quantitatively measuring the correspondence of a process to a model. Tech Report CUCS-840-97, Department of Computer Science. Bolder, CO: University of Colorado at Boulder.
  4. Hilbert, D. M., & Redmiles, D. F. (2000). Extracting usability information from user interface events. ACM Computing Surveys, 32(4), 384-421.
  5. IBM (2006). DB2 intelligent miner. Retrieved April 27, 2008, from http://www306.ibm.com/software/data/iminer/
  6. Jonassen, D. (1988). Integrating learning strategies into courseware to facilitate deeper processing. In D. Jonassen (Ed.), Instructional designs for microcomputer courseware (pp. 151–181). Hillsdale, NJ: Lawrence Erlbaum.
  7. Judd, T. (2006). Sequence analysis. [Computer software]. Retrieved April 27, 2008, from http://www.bmu.unimelb.edu.au/research/audittrail/
  8. Judd, T., & Kennedy, G. (2001). Extending the role of audit trails: A modular approach. Journal of Educational Multimedia and Hypermedia, 10(4), 377395.
  9. Judd, T. S., & Kennedy, G. E. (2004). More sense from audit trails: Exploratory sequential data analysis. In R. Atkinson, C. McBeath, D. Jonas-Dwyer, & R. Phillips (Eds.), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 476-484). Perth, WA, Australia. Retrieved April 27, 2008, from http://www.ascilite.org.au/conferences/perth04/procs/contents.html
  11. Kennedy, G., & Judd, T. (2000). Pilot testing of a system of electronic evaluation. In R. Sims, M. O’Reilly, & S. Sawkins (Eds.), Learning to choose: Choosing to learn (Short Papers and Works in Progress). Proceedings of the
  12. Kennedy, G. E., & Judd, T. S. (2004). Making sense of audit trail data. Australasian Journal of Educational Technology, 20(1), 18-32.
  13. Kennedy, G. E., & Judd, T. S. (2007). Expectations and reality: Evaluating patterns of learning behaviour using audit trails. Computers & Education, 49(3), 840-855.
  14. Kruskall, J. B. (1983). An overview of sequence comparison: Time warps, string edits, and macromolecules. SIAM Review, 25(2), 201-237.
  15. Lange, K. (2002). Mathematical and statistical methods for genetic analysis (2nd ed.) New York: Springer-Verlag.
  16. Magnusson, M. S. (2000). Discovering hidden time patterns in behavior: T-patterns and their detection. Behavior Research Methods, Instruments & Computers, 32, 93-110.
  17. Metcalfe, S. (2003). Medical genetix: Clinical and molecular aspects of human genetic disorders [CDROM]. Melbourne VIC, Australia: The University of Melbourne.
  18. Misanchuk, E. R., & Schwier, R. (1992). Representing interactive multimedia and hypermedia audit trails. Journal of Educational Multimedia and Hypermedia, 1(3), 355-372.
  19. Noldus Information Technology (2006). Noldus–Theme. Retrieved April 27, 2008, from http://www.noldus.com/site/doc200403003
  20. Oliver, R. (2000). When teaching meets learning: Design principles and strategies for web-based learning environments that support knowledge construction.
  21. Reeves, T. C., & Hedberg, J. G. (2003). Evaluating interactive learning systems. Athens GA: University of Georgia, College of Education.
  22. Sanderson, P. M., & Fisher, C. (1994). Exploratory sequential data analysis: Foundations. Human-Computer Interaction, 9, 251-317.
  23. Siochi, A. C., & Hix, D. (1991). A study of computer-supported user interface evaluation using maximal repeating pattern analysis. Reaching through technology. Proceedings of ICSE’91 Conference (pp. 301-305), New Orleans, LA. New York: ACM Press.
  24. Spiliopolou, M., & Pohle, C. (2001). Data mining for measuring and improving the success of web sites. Data Mining and Knowledge Discovery, 5, 85-114.
  25. SPSS Inc. (2006). Maximize your returns with data mining and predictive analysis. Retrieved April 27, 2008, from http://www.spss.com/clementine/

These references have been extracted automatically and may have some errors. Signed in users can suggest corrections to these mistakes.

Suggest Corrections to References

Pages You've Visited

Abstracts