Whenever the results of the admission test for the universities (PSU) are delivered, the criticisms appear: it reduces the school education to the content evaluated by the test, it asses content that some schools do not even have the opportunity to be review and generates curricular inequality, it is such a kind of Simce for the last year of secondary school, and it doesn’t predict the students’ performance at university.
The Universidad de Chile Departamento de Evaluación, Medición y Registro Educacional (Demre) has long been working on improvements to the PSU and new proposals for selection tools that address these criticisms. For that reason, this topic was analized in the seminar where Demre celebrated 20 years old. “We’re not designing a new admission system for universities,” said Alejandra Osses, head of Demre Development and Analysis Unit. “We want to provide evidence to help us to design solutions for the future.”
In this task, the department was advised by Salomé Martínez, director of the Strategic Area of Education at Center for Mathematical Modeling, and her team.
“Being able to apply concepts and procedures to solve real problems is crucial for citizens in the modern world,” said researcher about the fundamentals that inspired the redesign.
The new test focused on the curricular bases from 7th year of primary school to 2nd year of secondary school, where fundamental skills are learned. This allowed not to contaminate the evidence with problems of curricular content covered in 3rd and 4th year of secondary school. The team sought to analyze the ability of students to formulate or transfer the mathematical problems of the real world, to use or operate mathematically, and to interpret the mathematical results in the concrete world.
In total, one hundred items of different types of answers were designed: fill in the blanks, multiple choice, and true or false questions. These corresponded to the axes of the curriculum, numbers, algebra, geometry and data and chance. So, the team designed 40-question tests of varying difficulty and cognitive demand.
“In the construction of the items, we try to have a diversity of processes, formats and difficulties,” said Flavio Guíñez, who also worked on the project and supported the Salomé Martínez’s presentation.
As an example, the speakers showed a problem that served to encourage three multiple choices questions appeared, but a fill in the blank one.
The test was implemented in Santiago, Colina and Talagante, where more than two thousand students participated. They and their teachers found it the exam was tough, Martinez said: “The current PSU evaluated much fine content. Here, on the contrary, you had to know little content and think more. This added much difficulty.”
“The questions required understanding, it was exhausting, but very interesting. Actually they measured skills and competencies. Many questions could be resolved by seeking rather a strategy than knowledge,” said a teacher from a school where the proposal was tested, who was interviewed after the test.
The CMM researcher said there are also technical problems to apply a test like this. One has to do with the number of questions. Another is the possible presence of cultural or gender bias associated with real-world contexts in which problems arised. So, some questions are easier for some students depending on the social group they belong to.
“This is not a panacea, but it is part of something. It is an experiment that allows us to understand something else,” Martinez concluded.
