You are here:

Testing a TPACK-Based Technology Integration Assessment Rubric PROCEEDINGS

, College of William & Mary, United States ; , University of Nebraska at Omaha, United States ; , College of William & Mary, United States

AACE Award

Society for Information Technology & Teacher Education International Conference, in San Diego, CA, USA ISBN 978-1-880094-78-5 Publisher: Association for the Advancement of Computing in Education (AACE), Chesapeake, VA

Abstract

Although there is ever-increasing emphasis on integrating technology in teaching, there are few well-tested and refined assessments to measure the quality of this integration. The few measures that are available tend to favor constructivist approaches to teaching, and thus do not accurately assess the quality of technology integration across a range of different teaching approaches. We have developed a more “pedagogically inclusive” instrument that reflects key TPACK concepts and that has proven to be both reliable and valid in two successive rounds of testing. The instrument’s interrater reliability coefficient (.857) was computed using both Intraclass Correlation and a score agreement (84.1%) procedure. Internal consistency (using Cronbach’s Alpha) was .911. Test-retest reliability (score agreement) was 87.0%. Five TPACK experts also confirmed the instrument’s construct and face validities. We offer this new rubric to help teacher educators to more accurately assess the quality of technology integration in lesson plans, and suggest exploring its use in project and unit plans.

Citation

Harris, J., Grandgenett, N. & Hofer, M. (2010). Testing a TPACK-Based Technology Integration Assessment Rubric. In D. Gibson & B. Dodge (Eds.), Proceedings of SITE 2010--Society for Information Technology & Teacher Education International Conference (pp. 3833-3840). San Diego, CA, USA: Association for the Advancement of Computing in Education (AACE). Retrieved September 19, 2018 from .

Keywords

View References & Citations Map

References

  1. Allen, M.J. & Yen, W.M. (2002). Introduction to measurement theory. Long Grove, IL: Waveland Press.
  2. Archambault, L., & Crippen, K. (2009). Examining TPACK among K-12 online distance educators in the United States. Contemporary Issues in Technology and Teacher Education, 9(1). Retrieved October 19, 2009, from http://www.citejournal.org/vol9/iss1/general/article2.cfm
  3. Britten, J.S., & Cassady, J.C. (2005). The Technology Integration Assessment Instrument: Understanding planned use of technology by classroom teachers. Computers in the Schools, 22(3), 49-61.
  4. Cronbach, L.J., Gleser, G.C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability of scores and profiles. New York: Wiley.
  5. Dawson, K (2007). The role of teacher inquiry in helping prospective teachers untangle the complexities of technology use in classrooms. Journal of Computing in Teacher Education, 24(1), 5-14.
  6. Field, A.P. (2005). Intraclass correlation. In Everitt, B.S. & Howell, D.C. (Eds.). Encyclopedia of statistics in the behavioral sciences (pp. 1296-1305). Chichester, England: Wiley.
  7. Griffin, D., & Gonzalez, R. (1995). Correlational analysis of dyad-level data in the exchangeable case. Psychlogical Bulletin, 118, 430-439.
  8. Gronlund, N.E. (1985). Measurement and evaluation in teaching (5th ed.), New York: McMillian.
  9. Groth, R., Spickler, D., Bergner, J., & Bardzell, M. (2009). A qualitative approach to assessing technological pedagogical content knowledge. Contemporary Issues in Technology and Teacher Education, 9(4). Retrieved December 5, 2009, from http://www.citejournal.org/vol9/iss4/mathematics/article1.cfm
  10. Harris, J., & Hofer, M. (2006, July). Planned improvisations: Technology-supported learning activity design in social studies. Session presented at the National Educational Computing Conference, San Diego, CA. Retrieved October 19, 2009, from http://center.uoregon.edu/ISTE/NECC2006/program/search_results_details.php?sessionid=13514149
  11. Harris, J., & Hofer, M. (2009). Instructional planning activity types as vehicles for curriculum-based TPACK development. In Maddux, C.D. (Ed.). Research highlights in technology and teacher education 2009 (pp. 99-108). Chesapeake, VA: Society for
  12. Hughes, J.E. & Scharber, C.M. (2008). Leveraging the development of English TPCK within the deictic nature of literacy. In AACTE Committee on Innovation and Technology (Eds.). Handbook of technological pedagogical content knowledge for educators (pp. 87-106). New York: Routledge.
  13. Koehler, M.J., & Mishra, P. (2008). Introducing TPACK. In AACTE Committee on Innovation& Technology (Eds.). Handbook of technological pedagogical content knowledge for educators (pp. 3-29). New York, NY: Routledge.
  14. Koehler, M.J., & Mishra, P. (2005). Teachers learning technology by design. Journal of Computing in Teacher Education, 21(3), 94-102.
  15. Koehler, M.J., Mishra, P., & Yahya, K. (2007). Tracing the development of teacher knowledge in a design seminar: Integrating content, pedagogy, & Technology. Computers& Education, 49(3), 740-762.
  16. Lawless, K.A., & Pellegrino, J.W. (2007). Professional development in integrating technology into teaching and learning: Knowns, unknowns, and ways to pursue better questions and answers. Review of Educational Research, 77(4), 575-614.
  17. Litwin, M.S. (2002). How to assess and interpret survey psychometrics. The Survey Kit Series (Vol. 8). Thousand Oaks, CA: Sage Publications.
  18. McGraw, K.O., & Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients, Psychological Methods, 1 (1), 30-46.
  19. Mishra, P., & Koehler, M.J. (2006). Technological pedagogical content knowledge: A new framework for teacher knowledge. Teachers College Record. 108(6), 1017-1054.
  20. Moskal, B.M., & Leydens, J.A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research& Evaluation, 7, 71-81.
  21. Mouza, C. & Wong, W. (2009). Studying classroom practice: Case development for professional learning in technology integration. Journal of Technology and Teacher Education, 17(2), 175-202. Retrieved October 19, 2009, from http://www.editlib.org/p/26963
  22. Pierson, M. (2008). Teacher candidates reflect together on their own development of TPCK: Edited teaching videos as data for inquiry. In K. McFerrin et al. (Eds.), Proceedings of the Society for Information Technology and Teacher Education International Conference 2008 (pp. 5305-5309). Chesapeake, VA: Association for the Advancement of Computing in Education.
  23. Roblyer, M.D., & Doering, A.H. (2010). Integrating educational technology into teaching, (5th ed.). Boston: Allyn & Bacon.
  24. Schmidt, D., Baran, E., Thompson, A., Koehler, M.J., Shin, T, & Mishra, P. (2009, April). Technological Pedagogical Content Knowledge (TPACK): The development and validation of an assessment instrument for preservice teachers. Paper presented at the 2009 Annual Meeting of the American Educational Research Association, San Diego, CA. Retrieved October 19, 2009, from http://mkoehler.educ.msu.edu/unprotected_readings/TPACK_Survey/Schmidt_et_al_Survey_v1.pdf
  25. Schrader, P.G., & Lawless, K.A. (2004). The knowledge, attitudes, and behaviors (KAB) approach: How to evaluate performance and learning in complex environments. Performance Improvement, 43(9), 8–15.

These references have been extracted automatically and may have some errors. If you see a mistake in the references above, please contact info@learntechlib.org.

View References & Citations Map

Cited By

  1. Unpacking Performance Indicators in the TPACK Levels Rubric to Examine Differences in the Levels of TPACK

    Aleksandra Kaplon-Schilis, The Graduate Center, CUNY, United States; Irina Lyublinskaya, College of Staten Island, CUNY, United States

    Society for Information Technology & Teacher Education International Conference 2018 (Mar 26, 2018) pp. 2074–2083

  2. Early Career Teacher Candidate TPACK Development: Implementation of a Learning Activity Types Short Course

    Daniel Mourlam & Heather Bleecker, University of South Dakota, United States

    Society for Information Technology & Teacher Education International Conference 2017 (Mar 05, 2017) pp. 2404–2409

  3. Examining Differences in TPACK Levels Assessed through Pre-service Teachers STEM Lesson Plans: Case Study.

    Aleksandra Kaplon-Schilis, Graduate Center, CUNY, United States; Irina Lyublinskaya, College of Staten Island, CUNY, United States

    Society for Information Technology & Teacher Education International Conference 2016 (Mar 21, 2016) pp. 2893–2899

  4. Examining Teachers’ Technology Use Through Digital Portfolios

    Sarah Keenan, Joshua Rosenberg, Spencer Greenhalgh & Matthew Koehler, Michigan State University, United States

    Society for Information Technology & Teacher Education International Conference 2016 (Mar 21, 2016) pp. 1084–1091

  5. Designing a tool to support teachers with the TPACK framework

    Fei Fang, Learning Technologies, College of Education & Human Development, University of Minnesota, United States

    E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2015 (Oct 19, 2015) pp. 1079–1084

  6. A performance assessment of teachers’ TPACK using artifacts from digital portfolios

    Joshua Rosenberg, Spencer Greenhalgh & Matthew Koehler, Michigan State University, United States

    Society for Information Technology & Teacher Education International Conference 2015 (Mar 02, 2015) pp. 3390–3397

  7. Examining the Relationship between Self and External Assessment of TPACK of Pre-Service Special Education Teachers

    Irina Lyublinskaya & Nelly Tournaki, CUNY College of Staten Island, United States

    Society for Information Technology & Teacher Education International Conference 2015 (Mar 02, 2015) pp. 3331–3337

  8. A rubric for assessing teachers' lesson activities with respect to TPACK for meaningful learning with ICT

    Joyce Koh, National Institute of Education, Nanyang Technological University, Singapore

    Australasian Journal of Educational Technology Vol. 29, No. 6 (Dec 23, 2013)

  9. Designing In-service Technology Integration Training Program for Vocational Instructors

    Hatice Cilsalar & Evrim Baran, Middle East Technical University, Turkey

    EdMedia + Innovate Learning 2014 (Jun 23, 2014) pp. 1851–1857

  10. How teacher education institutions cope with challenges of teaching and learning in the digital age

    Joke Voogt, University of Amsterdam/ Windesheim University of Applied Sciences, Netherlands; Maurice Schols, Fontys University of Applied Sciences, Netherlands; Jeroen Bottema, Inholland University of Applied Sciences, Netherlands; Hans van Bergen & Nanda van der Stap, Utrecht University of Applied Sciences, Netherlands; Anna Tomson, Michael Nieweg, Jan-Willem Doornenbal & Bas Bakker, Amsterdam University of Applied Sciences, Netherlands; Anneke Smits, Windesheim University of Applied Sciences, Netherlands; Ann Thompson, Iowa State University, United States

    Society for Information Technology & Teacher Education International Conference 2014 (Mar 17, 2014) pp. 2821–2823

  11. Building Pre-Service Teachers’ Knowledge of TPACK: A Study Replication and Extension

    Amanda Goodwin, Asbury University, United States

    Society for Information Technology & Teacher Education International Conference 2014 (Mar 17, 2014) pp. 2809–2818

  12. Conducting Classroom Observations to Understand TPACK: Moving Beyond Self-reported Data

    Shu-Ju Diana Tai & Denise Crawford, Center for Technology in Learning & Teaching, Iowa State University, United States

    Society for Information Technology & Teacher Education International Conference 2014 (Mar 17, 2014) pp. 2661–2664

  13. Using Technology Pedagogical Content Knowledge Development to Enhance Learning Outcomes

    Agyei Douglas, University of Cape Coast, Ghana; Jared Keengwe, University of North Dakota, United States

    Society for Information Technology & Teacher Education International Conference 2014 (Mar 17, 2014) pp. 2473–2481

  14. Building Technological, Pedagogical Content Knowledge(TPACK) Among Pre-service Teachers In a Science Methods Course

    Laura Lowder, Pfeiffer University-Assistant Professor of Education/University of Florida-Doctoral Candidate, United States

    EdMedia + Innovate Learning 2013 (Jun 24, 2013) pp. 1755–1760

  15. Teaching Geometry with Technology: A Case Study of One Teacher’s Technological Pedagogical Content Knowledge

    Ewelina McBroom, Southeast Missouri State University, United States

    Society for Information Technology & Teacher Education International Conference 2013 (Mar 25, 2013) pp. 4828–4836

  16. I have TPCK! – What does that mean? Examining the External Validity of TPCK Self-Reports.

    Karsten Krauskopf, Knowledge Media Research Center, Germany; Karin Forssell, Stanford University, United States

    Society for Information Technology & Teacher Education International Conference 2013 (Mar 25, 2013) pp. 2190–2197

  17. Testing a TPACK-Based Technology Integration Observation Instrument

    Mark Hofer, College of William and Mary, United States; Neal Grandgenett, University of Nebraska at Omaha, United States; Judi Harris, College of William and Mary, United States; Kathy Swan, University of Kentucky, United States

    Society for Information Technology & Teacher Education International Conference 2011 (Mar 07, 2011) pp. 4352–4359

These links are based on references which have been extracted automatically and may have some errors. If you see a mistake, please contact info@learntechlib.org.