Evaluation report: Methods and conclusions

Download the extensive final evaluation report, September 2015

For public extract of final report, click here (October 2015).

The methods

One of the evaluation methods: Bono´s 6 Thinking Hats in a transnational team

The MAPPING project team undertook a final evaluation of MAPPING´s core products, as well as of the work process during the project. The final evaluation was undertaken during the final transnational partner meeting, June 2015, Breda/Holland.

The evaluation used a triangulation of several methods (for specification, see evaluation report). The transnational joint evaluation was supplemented with qualitative and quantitative results from local test runs.

As MAPPING had been designed as an agile project, findings on interim products and processes had already been taken into account when making adjustments en route.

The core products

MAPPING´s core products have been realized as targeted in the application:

  1. MAP: A guideline containing models, methodical-didactical instruments and crucial principles for implementation and improvement of an individualized learning approach. MAP is launched as the European Web App Quality MAP, supplemented with national versions in Slovenian and Dutch.
  2. A range of VET teacher training units, based on MAP. Besides several national versions, a transnationally co-created MOOC (massive open online course) has been trial run successfully and will be further developed.
  3. A multimedia documentation of best practice cases from trial runs in the 5 partner countries during the project period. The documentation visualizes methodologies for supporting individualized learning and is merged into MAP.

The pedagogical approaches of MAP and MOOC are to a high degree representative of MAPPING´s accumulated results. Furthermore, the core products have a transnational generic character and are written in English. Therefore, they were chosen as main objects for the final joint evaluation.
The work process described pertains to transnational cooperation and co-creation in the MAPPING project.

In short: Conclusions and recommendations

Both MAP with its sub-products and the MOOC were considered valuable products, each of them with specific problems and challenges. Both have a high potential for being further developed and disseminated even more locally, in the partners´ own countries and internationally. The products interact, and this interaction could be explored more.

According to local evaluation results, re-designed teacher training has influenced the teachers´ competencies and empowered them in their new practice of facilitating individualized learning. Both present and online teacher training should follow the principle of practicing-how-to-do, beyond telling-how-to-do. Welsh measurements of impact on learners indicate that the personalized approach increases the students´ learning capability and motivation.

Adding authentic local best practice cases has benefited the products as well as the process, providing realistic solutions, and should be taken as a guideline for further development. Local development has been supported by transfer of best practice from MAPPING´s partners.

Focus on collaboration and mutual reviews had a positive impact. These principles have benefited the work process during the MAPPING project, as well as studies in the MOOC. Strengths in this context were consideration of and mutual respect for local goals, as well as the MOOC students´ individual work focuses.

Platforms for supporting learning processes, interactive communication, knowledge sharing and storage of major data are still not optimal, and the platforms selected have not been applied optimally for full outcome. MAPPING´s cumulative portfolio of project platforms did not satisfy all needs and meant a certain barrier for new project members. The MOOC platform Canvas could not serve all countries and was not flexible enough. Advantages, disadvantages, timing, steering and criteria for choice of platforms should be identified for future recommendations. In addition, the interaction of MAP and the MOOC, as well as learning activities with personal presence, could be explored more.

In all cases of further development, the MAPPING partners would be relevant experts and stakeholders to involve in their specific top interests and expertise. The MAPPING products, as well as the huge amount of data behind them, would be valuable and sustainable materials for new initiatives.