This paper provides an experience report on a co-design approach with teachers to co-create learning analytics-based technology to support problem-based learning in middle school science classrooms. We have mapped out a workflow for such applications and developed design narratives to investigate the implementation, modifications and temporal roles of the participants in the design process. Our results provide precedent knowledge on co-designing with experienced and novice teachers and co-constructing actionable insight that can help teachers engage more effectively with their students' learning and problem-solving processes during classroom PBL implementations.
What is already known about this topic
- Success of educational technology depends in large part on the technology's alignment with teachers' goals for their students, teaching strategies and classroom context.
- Teacher and researcher co-design of educational technology and supporting curricula has proven to be an effective way for integrating teacher insight and supporting their implementation needs.
- Co-designing learning analytics and support technologies with teachers is difficult due to differences in design and development goals, workplace norms, and AI-literacy and learning analytics background of teachers.
What this paper adds
- We provide a co-design workflow for middle school teachers that centres on co-designing and developing actionable insights to support problem-based learning (PBL) by systematic development of responsive teaching practices using AI-generated learning analytics.
- We adapt established human-computer interaction (HCI) methods to tackle the complex task of classroom PBL implementation, working with experienced and novice teachers to create a learning analytics dashboard for a PBL curriculum.
- We demonstrate researcher and teacher roles and needs in ensuring co-design collaboration and the co-construction of actionable insight to support middle school PBL.
Implications for practice and/or policy
- Learning analytics researchers will be able to use the workflow as a tool to support their PBL co-design processes.
- Learning analytics researchers will be able to apply adapted HCI methods for effective co-design processes.
- Co-design teams will be able to pre-emptively prepare for the difficulties and needs of teachers when integrating middle school teacher feedback during the co-design process in support of PBL technologies.
CONFLICT OF INTEREST STATEMENT
The authors have no conflicts to disclose.
DATA AVAILABILITY STATEMENT
De-identified data are available upon request.
- 2019). Designing in context: Reaching beyond usability in learning analytics dashboard design. Journal of Learning Analytics, 6(2), 70–85.
- 2021). From visible to understandable: Designing for teacher agency in education data visualizations. Contemporary Issues in Technology and Teacher Education, 21(1), 155–186.
- 2011). Learning at the boundary: An introduction. International Journal of Educational Research, 50(1), 1–5. https://doi.org/10.1016/j.ijer.2011.04.002
- 2004). Design-based research: Putting a stake in the ground. Journal of the Learning Sciences, 13(1), 1–14.
- 2003). Building sustainable science curriculum: Acknowledging and accommodating local adaptation. Science Education, 87(4), 454–467.
- 2017). Toward human-centered algorithm design. Big Data & Society, 4(2), 2053951717718854.
- 2021). Simulating more equitable discussions: Using teacher moments and practice-based teacher education in mathematical professional learning. OpenBU. https://open.bu.edu/handle/2144/44488
- 1990). Reflections on participatory design: Lessons from the trillium experience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI ’90, New York, NY. Association for Computing Machinery. https://doi.org/10.1145/97243.97307
- 2000). Creating usable innovations in systemic reform: Scaling up technology-embedded project-based science in urban schools. Educational Psychologist, 35(3), 149–164.
- 2016). Developing computational thinking in compulsory education—Implications for policy and practice. Technical Report.
- 2014). Understanding decision making in teachers' curriculum design approaches. Educational Technology Research and Development, 62(4), 393–416.
- 2019). Human-centred learning analytics. Journal of Learning Analytics, 6(2), 1–9. https://doi.org/10.18608/jla.2019.62.1
- 2021). Making sense of sensemaking: Understanding how K-12 teachers and coaches react to visual analytics. Journal of Learning Analytics, 8(3), 60–80.
- 1999). Five reasons for scenario-based design. In Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers, Track3.
- 2006). Constructing grounded theory: A practical guide through qualitative analysis. Sage.
- 2003). The use of think-aloud methods in qualitative research an introduction to think-aloud methods. Brock Education Journal, 12(2). https://doi.org/10.26522/brocked.v12i2.38
- 2021). Using teacher dashboards to assess group collaboration in problem-based learning. Educational Technology Research and Development, 15(2). https://doi.org/10.14434/ijpbl.v15i2.28792
- 2015). Teachers as participatory designers: Two case studies with technology-enhanced learning environments. Instructional Science, 43(2), 203–228.
- 2017). Participatory design for learning (pp. 15–18). Routledge.
- 2018, March). Co-creation strategies for learning analytics. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge (pp. 97–101).
- 2002). Value sensitive design: Theory and methods (Vol. 2, p. 12). University of Washington Technical Report.
- 2002). Doing qualitative research in education settings. SUNY Press.
- 2015). Problem-based learning: Goals for learning and strategies for facilitating (pp. 69–84). Purdue University Press.
- 2004). Problem-based learning: What and how do students learn? Educational Psychology Review, 16(3), 235–266.
- 2002). Creating context: Design-based research in creating and understanding CSCL. In Proceedings of the Conference on Computer Support for Collaborative Learning: Foundations for a CSCL Community (pp. 453–462).
- 2019). Co-designing a real-time classroom orchestration tool to support teacher–AI complementarity. Journal of Learning Analytics, 6(2), 27–52.
- 2021). Coherence across conceptual and computational representations of students’ scientific models. In E. Vries, J. Ahn, & Y. Hod (Eds.), 15th International Conference of the Learning Sciences—ICLS 2021 (pp. 330–337). International Society of the Learning Sciences.
- 2019). C2STEM: a System for Synergistic Learning of Physics and Computational Thinking. Journal of Science Education and Technology, 29(1), 83–100. https://doi.org/10.1007/s10956-019-09804-9
- 2002). Japanese and American teachers' evaluations of videotaped mathematics lessons. Journal for Research in Mathematics Education, 33(3), 154–175.
- 2014). Participatory design of learning environments: Integrating perspectives of students, teachers, and designers. Instructional Science, 42, 1–9.
- 2010). Mental models, visual reasoning and interaction in information visualization: A top-down perspective. IEEE Transactions on Visualization and Computer Graphics, 16(6), 999–1008.
- 2012). Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers.
- 2016). Latux: An iterative workflow for designing, validating and deploying learning analytics visualisations. Journal of Learning Analytics, 2(3), 9–39.
- 2016). Gathering requirements for teacher tools: Strategies for empowering teachers through co-design. Journal of Science Teacher Education, 27, 79–110.
- 2020). Beyond sticky notes. In Doing co-design for real: Mindsets, methods, and movements. Thorpe-Bowker Identifier Services Australia
- 1992). Taxonomy of participatory design practices: A participatory poster. In Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems, CHI ’92 (p. 34, New York, NY). Association for Computing Machinery.
- NGSS. (2013). Next generation science standards: For states, by states. The National Academies Press.
- 1994). Precedents in design: A computational model for the organization of precedent knowledge. Design Studies, 15(2), 141–157.
- 2007). Designing formative assessment software with teachers: An analysis of the co-design process. Research and Practice in Technology Enhanced Learning, 2, 51–74.
- 2019). Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology, 35(4). https://doi.org/10.14742/ajet.4314
- 2000). Investigating the mutual adaptation process in teachers' design of technology-infused curricula. In Fourth International Conference of the Learning Sciences (pp. 342–349).
- 2006). Co-design of innovations with teachers: Definition and dynamics, 2, 606–612.
- 2022). A learning analytics approach towards understanding collaborative inquiry in a problem-based learning environment. British Journal of Educational Technology, 53(5), 1321–1342.
- 2008). Co-creation and the new landscapes of design. CoDesign, 4(1), 5–18.
- 2022, March). Participatory and co-design of learning analytics: An initial review of the literature. In LAK22: 12th International Learning Analytics and Knowledge Conference (pp. 535–541).
- 2014). Teacher noticing via video: The role of interpretive frames (pp. 11–28). Routledge.
- 2020). From theory to action: Developing and evaluating learning analytics for learning design. In Proceedings of the Tenth International Conference on Learning Analytics Knowledge, LAK ’20, New York, NY (pp. 569–578). Association for Computing Machinery.