International Seminar on Evidence-based research: methodological approaches and practical outcomes. Insights for Online Education

The UNESCO Chair in Education and Technology for Social Change at UOC work and initiatives take as reference the document “Transforming our world: the 2030 Agenda for Sustainable Development” (United Nations, 2015) which underlines the commitment and the challenges to face world-wide for a sustainable and equitable development. Specifically, the Chair focuses on the Goal 4 of the Sustainable Development Goals (SDGs) which aims to ensure, by 2030, “that all learners acquire the knowledge and skills needed to promote sustainable development, including, among others, through education for sustainable development and sustainable lifestyles, human rights, gender equality, promotion of a culture of peace and non-violence, global citizenship and appreciation of cultural diversity and of culture’s contribution to sustainable development” (SDG 4.7).

This means that education is one of the main tools for achieving a sustainable development (OECD, 2007; UNESCO, 2014) and that those countries with more educated people have generally better economic incomes and more opportunities to grow and increase the wellbeing of their citizens. In addition, technology is also providing new opportunities for access to higher education, as well as the transition to new methodologies that might help to get better teaching and better learning outcomes.

Online teaching and learning can make a huge contribution to the achievement of the aforementioned goal. It provides more flexibility, easiness of access, affordability and appropriate skills for the digital age, as well as social and economic impact in the society. However online education is always under the magnifying glass as it usually brings concerns about its quality and effectiveness.

What opportunities online education is going to provide to higher education are growing is a fact (Allen & Seaman, 2016; Bichsel, 2013; Online Learning Task Force, 2011), but doubts on this educational modality still persist. Different studies (Cavanaugh et al., 2004; Means et al., 2010; Means et al., 2013) have started to provide some evidence of its effectiveness through research, but there still is a big need of further investigation in this field, which could help to increase online education reputation.

To do this, the development of an appropriate methodology is crucial. In this sense, the work from Hattie (2008), Visible Learning, becomes a benchmark. Other studies have also highlighted the importance of the meta-analysis approach for these purposes (Borenstein et al., 2009; Field & Gillet, 2010). But considering just a unique approach could be biased and, therefore, taking into account other perspectives is highly recommendable.

Building up a methodology that can be the underlying framework for carrying out a set of research looking for evidence supporting the reputation and better visibility of online education is the main purpose of this event: organizing a Seminar with experts on this topic to get the knowledge that will provide the UNESCO Chair and its collaborators with the required skills to establish an observatory on the reputation and visibility of high quality online education.

This UNESCO Chair is highly committed to identify, analyse and disseminate any evidence or data regarding the quality, the effectiveness, and social impact and economic impact of online education. In the last two years, this Chair has fostered a long-lasting Seminar on the “Economic Effects and Social Impact of Online Education” consisting in different workshops, aiming to analyse how online education can be a driver for improving the way people can have a more equitable higher education, and to identify what might be the economic effects of greater dissemination and understanding of the contributions of online education to social development.

The good outcomes of the previous workshops have led the Chair, to organise new activities addressed to go further and deeper to identify the ways in which evidence can be gathered to know more and to make more visible the economic and social effects of online education.

The current International Workshop we are organising is aiming to know and understand the methodologies underlying evidence-based research; getting the skills to translate evidence-based research to the online education field; interacting with experts gathering data for observatories and impact of policies and strategies; and establishing the methodological foundations for the creation of an observatory for the reputation of online education, that will be evidence-based. After identifying what kinds of things could be constitutive of evidence, as it was worked in precedent workshops, we would now like to know how to gather them.

In this context, the International Seminar Evidence-based research: methodological approaches and practical outcomes. Insights for Online Education will have two parts:

Workshop on Evidence-based Research: Gathering data for measuring impact

Tuesday, 21st November 2017, 10.00 – 14.00 h.

Place: Palau Macaya, Passeig de Sant Joan, 108, 08037 Barcelona

This activity is by invitation only. Five experts from different international centres and observatories will present the methodologies they use to gather data and evidence for their purposes: how they work, how they design their research and how the collect the data they later use in their observatories or studies:

  • Richard Garrett, Observatory of Borderless Higher Education, UK
  • Emmanuel (Manny) Jimenez, 3ieImpact – International Initiative for Impact Evaluation, USA
  • Jairo Cifuentes, Red Telescopi – Pontificia Universidad Javeriana, COL
  • Katie Rose, Centre for Public Impact, UK/USA
  • Donatella Persico, Consiglio Nazionale delle Ricerche, CNR, IT
  • Karsten Krueger, Fundación CYD – U-Multirank, ESP

Every speaker will have around 30 min, to present the model they work with to the audience. Invited participants, who have participated in previous seminars organised by the UNESCO Chair on the social and economic impact of online education, will discuss with the speakers.

Open Public Session (lecture + roundtable): Will the future of Higher Education be evidence-based?

Wednesday, 22nd November 2017, 17.30 – 20.00 h.

Place: Palau Macaya, Passeig de Sant Joan, 108, 08037 Barcelona

This Open Session will start with a lecture and followed by a roundtable in which it is expected to have an overview of how evidence-driven education would be, with their benefits and possible risks, too.


  • Paul Prinsloo, University of South Africa, UNISA (Lecture)


  • Olaf Zawacki-Richter, Karl von Ossietzky University Oldenburg, GER
  • Richard Garrett, Observatory of Borderless Higher Education, UK
  • Francesca Pozzi, Consiglio Nazionale delle Ricerche, CNR, IT
  • Frans Kaizer, Centre for Higher Education Policy & Studies, CHEPS, NE

These two parts are very complementary and will provide valuable outputs to go forward with the research endeavoured by the Chair. As one of the milestone of this research, information about the launching of KLEOS, the Observatory for the Reputation and Quality of Online Education will be given.

If you are interested in participating to the Open Session, you can register to the event by filling this form.

For the Spanish Version of the program click here :

Referencias bibliográficas

Allen, I.E. & Seaman, J. (2016). Online Report Card. Tracking Online Education in the United States. Babson Survey Research Group and Quahog Research Group, LLC.

Bichsel, J. (2013). The State of E-Learning in Higher Education: An Eye toward Growth and Increased Access (Research report). Louisville, CO: EDUCAUSE Center for Analysis and Research.

Biesta, G. (2014). ¿Medir los que valoramos o valorar lo que medimos? Globalización, responsabilidad y la noción de propósito de la educación. Pensamiento Educativo: Revista de Investigación Educacional Latinoamericana, 51(1), 46-57. doi: 10.7764/PEL.51.1.2014.17. Retrieved from

Borenstein, M.; Hedges, L.V.; Higgins, J.P.T. & Rothstein, H.R. (2009). Introduction to Meta-Analysis. San Francisco, CA: John Wiley & Sons, Ltd. ISBN: 978-0-470-05724-7.

Field, A. P. & Gillett, R. (2010), How to do a meta-analysis. British Journal of Mathematical and Statistical Psychology, 63, 665–694. doi:10.1348/000711010X502733.

Hattie, J. (2008). Visible Learning. Oxford, UK: Routledge.

OECD (2007). Higher Education for Sustainable Development. Final Report of International Action Research Project. OECD from Forum for the Future (2006-2007).

UNESCO (2015). Position Paper Education post-2015. Paris: UNESCO.

UNESCO (2014). Sustainable Education begins with Education. Paris: UNESCO.

United Nations (2015). Transforming our world: the 2030 Agenda for Sustainable Development. Retrieved from

Will Ranking System Be Abandoned Everywhere?

  • For many years, ranking ruled the education world, in the U.S. and everywhere. To get a better spot in existing ranking systems run mainly by private entities, colleges would invest into infrastructure, construct posh athletic and student centers, misrepresent the numbers “by twisting the meanings of rules, cherry-picking data or just lying”:
  • In an attempt to shake up this system, in 2013 President Obama requested to  create a government rating system which would  rate all 7,000 colleges and universities on metrics like graduation rate, amount of debt accumulated by students and earnings after graduating, among other data. These ratings would be used to allocate the federal grants and loans ($150 billion each year). The college presidents  bitterly opposed it:  “wrongheaded”, radical, “uncharacteristically clueless”, etc.
  • With this initiative still in works, last Saturday, Sept.11, the White House abandoned this two-year plan and released a huge new federal database, with raw data instead of explicit rating: a new College Scorecard. This is a revamped  tool which links a huge database of the receivers of Pell Grants or loans since 1996 and their income tax records and gives a clear picture of earnings progress:  (see this part of data  in a simple graph):

earnings after school


Read More

Is college worth it? A huge new federal database reveals the answer depends on the college – Vox

With Website to Research College, Obama Abandons Ranking System – The New York Times

Obama’s New College Scorecard Flop the Focus Rankings – The Atlantic