Workshop Report: EdTech Providers x EdTech Strategy Lab Summer Workshop Series
The EdTech Strategy Lab Summer Workshop Series recently concluded its third session, with the objective to delve deeper into the needs and requirements of EdTech Providers concerning evidence-driven evaluation frameworks. The goal is to bolster trust across the sector.
Insights from the Strategy Lab
Beth Havinga, the Managing Director of the European EdTech Alliance, began the session by emphasising the importance of aligning EdTech solutions with the broader frameworks of educational institutions. To succeed, EdTech providers need to articulate how their solutions align with a school's visions, values, local principles, risk and safety requirements, and the overall contributions they aim to make to the education system.
Critically, not all EdTech products directly influence learning outcomes. Many concentrate on administrative or procedural offerings, which render evaluation mechanisms, focused solely on learning outcomes, insufficient. Thus, clear communication about the outcomes they aim to achieve is essential.
The Role of Evaluation and Certification and Challenges for EdTech Providers
The workshop participants were asked if they considered evaluation or certification helpful. Five out of eight respondents agreed that such mechanisms could be useful, two remained unsure, and one disagreed. This reflects the complexity of the issue, as some countries primarily use evaluation mechanisms, while others lean towards certification.
Despite the existence of approximately 140 frameworks and evaluation systems, EdTech providers often struggle to navigate these systems. The lengthy evaluation processes, averaging nine to ten months, pose significant challenges, especially for early-stage innovations. Furthermore, there is often a lack of acceptance or recognition of certifications, and a lack of trust regarding the evidence or basis of these mechanisms.
Moreover, there is a low uptake of these certifications or evaluations by EdTech providers due to a lack of incentives and resources for implementing EdTech solutions. The lack of a clear framework for how EdTech can contribute to successful testing environments further adds to the challenges.
Defining Evidence
The participants were asked about the kind of evidence they provide or deem important. The responses ranged from design principles and pedagogical quality to alignment with user needs and curriculum, lawfulness, ethics and safety, and the evaluation of case studies. It was noted that not every criterion needs to apply to every product, and the product’s implementation should align with the pedagogical goals.
Emerging Frameworks and the Future of Evaluation Frameworks
Several different frameworks were highlighted during the workshop to provide further context for the discussion. These included the following:
A marketplace developed by EdTech Impact to help schools in the United Kingdom evaluate EdTech solutions, while also supporting Edtech providers in assessing the overall quality and impact of their products.
The Goldstar Framework which focuses on helping those making decisions around Edtech solutions evaluate an EdTech company's six super powers relating to leadership vision, learning culture, sense of purpose, teamwork, research know-how, and orientation toward action.
The EdTech Index, developed by Fachhochschule de Mittelstandes in Germany offers a more research-focused approach that looks at the usability and alignment with necessary competences in focus, outlines the makeup of the team behind the product and their expertise, as well as examines technical solutions and topics of compliance.
UNICEF’s EdTech for Good Framework, presented in a previous workshop in the series, is a framework with 132 designators that are designed with the focus of ensuring that education is a common good.
A quick overview of each of these evaluation mechanisms showcased a different approach to assessing specific aspects of EdTech solutions, which provided the context necessary to delve into an engaging discussion about the future of evaluation frameworks from the perspective of EdTech providers.
Discussion Points
During the discussion, case studies were highlighted as an area needing further exploration. While these studies provide valuable insights for end users, they are often discredited by the research community. The challenge is determining what constitutes enough evidence and what provides substantial insight into the product and its usage.
According to the workshop respondents, the most significant challenge related to certification and evaluation was time and team constraints (37.5%), followed by financial issues (31.25%), lack of understanding regarding evidence (18.75%), and lack of clear guidance and expectations (12.5%).
In summary, workshop participants highlighted the need for support in increasing trust among users, lowering the friction for trialling new technologies, establishing research protocols, providing examples of successful implementations, understanding the impact of usage, and working closely with the beneficiaries of the product. As the EdTech sector continues to evolve, these insights will be instrumental in shaping the future of education technology.
