Is college overrated

Studies on the study That is why PISA is overrated

The term "study" indicates that this is a scientific study on an individual question. But what is the question actually? What or who is being examined here and how - and on whose behalf?

Every three years since 2000, 600,000 15-year-olds from around 80 countries have been tested for their reading, math and science performance. Figures from around 10,000 children are included from Germany. The "Program for International Student Assessment" (PISA) is commissioned by the Organization for Economic Cooperation and Development (OECD). The aim of the study is to compare the school systems of the participating countries with one another.

What does the PISA ranking really say?

The results are shocking regularly, like the German place 20 on December 3rd. But what does the ranking really say about the students, about the school system of a nation? Heiner Barz is head of the department for educational research and management at Heinrich Heine University in Düsseldorf. In the past few years he has done extensive research on educational studies such as PISA.

PISA is not an instrument with one hundred percent measurement accuracy. Many experts now doubt whether one can really draw conclusions for the German education system from the results.

Heiner Barz, education expert

A fair comparison?

Sampling, for example, has been criticized again and again since PISA was introduced. In Germany, care is taken to ensure that children from all types of schools, including special schools and special needs schools, are included. However, the OECD does not set any guidelines for this; the participating countries decide for themselves how they compile their sample.

In other countries, on the day of the PISA survey, poorer students may be advised to call in sick.

Heiner Barz

There are now numerous studies that scientifically document the weak points of PISA.

Methodological deficiencies disclosed

While the PISA tests were running in schools in 2018, scientists from Harvard and Boston College published their criticism of the method behind the previous studies in the science magazine "Science". Judith B. Singer (Harvard) and Henry I. Braun (Boston College) primarily question the ranking, i.e. the order of success of the individual nations. In fact, the curricula of the federal states and the individual grade levels are not taken into account. 15-year-olds are tested, regardless of whether they are still learning in grade nine or already in grade ten and how far they are "in the material".

Front runners are not always smarter

The choice of children tested has often been questioned. Singer and Braun also found that in some countries 15-year-olds from allegedly poorly educated families are being left out. In Shanghai, for example, boys and girls who had immigrated from rural areas of China had no right to a place in city schools. As a result, 27 percent of the children in this year would not be recorded at all.

So we have to look at the top position of China in 2012 together with Singapore and Hong Kong. The results of Mexico and Turkey are also controversial. There, 40 percent of young people aged 15 no longer go to school, presumably mainly from families with little educational background.

Centralized countries and federal states are difficult to compare

The researchers also complain that centrally governed countries such as China are directly compared with federal states such as Canada or Germany. Because where school education is the responsibility of the federal states or states, the spread of values ​​is large within a nation alone.

Where the students really get their knowledge is ignored

The ranking does not take into account whether pupils actually acquired the knowledge they provided in the PISA test in the classroom. In 2012, half of Korean children stated that they received private tutoring. Singer and Braun therefore warn against drawing linear conclusions about the quality of the respective school system from the ranking. However, that is the goal that PISA pursues.

Which improves expressiveness and comparability

The authors therefore recommend progress studies that show the development of the children within their school system over a longer period of time. And indeed, many of the criticisms mentioned were taken into account in the current PISA survey, says Andreas Schleicher, head of the OECD's Directorate of Education.

The Standing Conference has introduced additional country comparisons in Germany. They should help to increase comparability with centrally governed countries. The comparisons are carried out by the Institute for Quality Development in Education (IQB). The selection of the samples has also changed in the meantime:

We expanded the samples to be more representative. In China, for example, we are now also including students from additional provinces.

Andreas Schleicher, Head of the Education Directorate of the OECD

New to PISA: test in "global competence"

In order to broaden the test areas and to map more skills, the "Global Competence" was also tested for PISA 2018. The pupils should show how they move in intercultural contexts and how to deal with differences. In the next PISA round in three years, creative thinking should then play a role.