You are here:

Usability Methodology and Testing for a Virtual Learning Environment

, , , Oxford university, United Kingdom

International Journal on E-Learning Volume 17, Number 3, ISSN 1537-2456 Publisher: Association for the Advancement of Computing in Education (AACE), Waynesville, NC USA


This article describes a usability evaluation that was conducted among student users of improved learning areas (sites) designed as part of the WebLearn Improved Student Experience (WISE) project conducted at Oxford University (UK). WebLearn is the university’s internal brand for the Sakai open source virtual learning environment (VLE). The project benefited from central university funding to support 19 departments, faculties and other academic units to fast-track the development and improvement of their WebLearn presence in order to enhance the student learning experience. This was the first time in the Technology-Enhanced Learning (TEL) team that formal usability testing had been carried out on the VLE used by staff and students for both full distance and blended learning study programmes. The WISE team members devised a methodology to assess certain usability criteria that would be helpful in the context of the project. The methodology included various techniques generally used in usability testing such as first-click testing, scenarios, focus group discussion and memory tasks. The methods and tasks presented in this paper could be further refined and used as guidelines for usability testing on other systems or VLE-related projects.


Laurent, X., Fresen, J. & Burholt, S. (2018). Usability Methodology and Testing for a Virtual Learning Environment. International Journal on E-Learning, 17(3), 341-375. Waynesville, NC USA: Association for the Advancement of Computing in Education (AACE). Retrieved November 19, 2018 from .

View References & Citations Map


  1. Bangor, A., Kortum, P., & Miller, J. (2009) Determining What Individual SUS Scores Mean Adding an Adjective Rating Scale. Journal of Usability Studies, 4, 114-123.
  2. Berry, C. J., Shanks, D. R., & Henson, R. N. A. (2008). A unitary signal-detection model of implicit and explicit memory. Trends in Cognitive Sciences, 12, 367-373.
  3. Bonk, C. J., Cummings, J. A., Hara, N., Fisher, R. & Lees, S.M. (2000). A tenlevel web integration continuum for higher education. In B. Abbey (Ed.), Instructional and Cognitive Impacts of Web-Based Education (pp. 56-77). Hershey, PA: Idea Group.
  4. Freire, L.L., Arezes, P.M., & Campos, J.C. (2012). A literature review about usability evaluation methods for e-learning platforms. Work 41, 1038-1044.
  5. International Standards Organisation (ISO) (1998). Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs)-Part 11: Guidance on Usability (ISO 9241-11:1998(E)). ISO, Geneva.
  6. IT Services (2013). The WebLearn Student Learning Experience (WLSE) Project. Internal report. Oxford: Oxford University Academic IT Services.
  7. Laurent, X. (2013). The impact of selective attention and action on episodic memory (Doctoral dissertation). Retrieved from
  8. Lewis, J. R., & Sauro, J. (2009). The Factor Structure of the System Usability Scale. In Proceedings of the Human Computer Interaction International Conference , San Diego CA, USA
  9. Magnussen, S., & Helstrup, T. (2007). Everyday Memory. Psychology Press.
  10. Neyman, J., & Pearson, E.S. (1933). On the Problem of the Most Efficient Tests of Statistical Hypotheses. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences. 231 (694–706): 289– 337.
  11. Nielsen Norman Group (2006). F-Shaped Pattern For Reading Web Content (original study)
  12. Sauro, J. (2011a). A Practical Guide to the System Usability Scale (SUS): Background, Benchmarks & Best Practices. Measuring Usability LLC, Denver.
  13. Sauro, J. (2011b). Measuring Usability with the System Usability Scale (SUS),
  14. Sauro, J., & Lewis, J.R. (2012). Quantifying the User Experience. Morgan Kaufmann, Amsterdam.
  15. Shneiderman, B. (1992). Designing the User Interface: Strategies for Effective Human- Computer Interaction. Reading, Addison-Wesley.
  16. Yonelinas, A. P. (1994). Receiver-operating characteristics in recognition memory: Evidence for dual-process model. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1341-1354.
  17. Yonelinas, A.P. (2001). Components of episodic memory: the contribution of recollection and familiarity. Philosophical Transactions of the Royal Society B: Biological Sciences, 356, 1363-1374.
  18. Yonelinas, A.P. (2002). The nature of recollection and familiarity: A review of 30 years of research. Journal of Memory and Language, 46, 441-517.
  19. WISE (2016). The WebLearn Improved Student Experience (WISE) Project. Oxford: Oxford University Academic IT Services.

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact