Идет работа над наполнением сайта. Некоторые ссылки могут не работать. Приносим свои извинения.

Поиск по сайту

Счётчики


Русский
The English and Russian versions may differ in the amount of information and wording

 

External consultation on linking the Second Certificate Examination to the Common European Frame of Reference for Languages (Linking Project)

  • EALTA (European Association for Language Testing and Assessment) Conference in Copenhagen, May 2015
  • Main News - Headlines
  • Questions – Answers

EALTA (European Association for Language Testing and Assessment) Conference in Copenhagen, May 2015

Presentation of the Linking Project Results. Plenary meeting
The Importance of the event and its unique character Saint Petersburg University has become the first educational institution in Russia to launch and carry out a CEFR linking project with the aim of exploring linking its graduation examination in English, called the Second Certificate Examination, to the Common European Frame of Reference for Languages. It is a privilege and honor for the university to be in the list of other universities and testing centres (Cambridge, TOEFL, IELTS) that have already been involved in the same kind of project.
Applying for presenting results at the EALTA conference In November 2014 Saint Petersburg University applied to present the results of the linking project at the 2015 EALTA conference.

The university entered the EALTA contest by submitting its proposal for the plenary presentation. The proposal of Saint Petersburg University was selected among other winning proposals and the university staff members were given the floor at the plenary meeting. Moreover, the advertising information about Saint Petersburg University and the Language Testing Centre was also placed in the programme of the conference. (Proposal)
Preparation for the conference;
Presentation
From January to May 2015 the Language Testing Centre specialists analyzed data collected during the linking project. This has become the core of the presentation, aimed at sharing the experience of building and maintaining the system of external assessment of foreign languages. This kind of system is the first in Russian tertiary education, and Saint Petersburg University plays a leading role in this field of knowledge in Russia. (Presentation, Slides)
The presentation included:
  • statistical analysis data describing the linking of the Second Certificate Examination to CEFR;
  • the validity and reliability of the linkage project results;
  • Standard Setting results obtained by 33 experts from 5 European countries;
  • methods used within the linking project;
  • the impact of the external assessment system of Saint Petersburg University on improving the quality of English Language teaching.
Feedback to the presentation The presentation was successful and the received high acclaim of the EALTA experts and the professional community. The experience gained by the university during the linking project was described as interesting and enriching.

The presentation was widely discussed by conference participants. The following factors added to the competitive potential of Saint Petersburg University:
  • Saint Petersburg University has become the first educational institution in Russia to present its system of external assessment of foreign languages; it is still the only educational institution in the country that has achieved such results;
  • the quality of the materials presented is high; the professionals involved in the project are knowledgeable;
  • the validity and reliability of the linking project are high.
The importance of dissemination The facts described above lead us to assume that the dissemination of the linking project results will contribute to the growth of the competitive potential of Saint Petersburg university both in the country and worldwide.

Conference Programme,
30th May, 14.15-14.45: CEFR Linkage: Bringing together education reform policies, external assessment and classroom realities
Elena Prokhorova, Tatyana Timofeyeva, Olga Lankina

 

Main News - Headlines

Dates Events Materials
25 – 31 May 2015 Presentation of the CEFR linking project results.
Linking the Second Certificate Examination to CEFR was described and presented at the EALTA 2015 conference plenary meeting in Copenhagen in May 2015.
Presentation,
Slides
November 2014 Standard Setting for productive skills.
Cut-off scores for the Writing and Speaking papers of the Second Certificate Examination were set.
External Experts Report, November 2014
June 2014 Standard Setting for receptive skills.
Cut-off scores for Listening, Reading, and Language in Use papers of the Second Certificate Examination were set.
External Experts Report, June 2014
November 2013 Guidelines for the linking project were drawn up by external consultants. The project details were defined. External Experts Report, November 2013
June 2012 The quality of the Second Certificate Examination was checked and reported using 17 ALTE’s Quality Standards. Report of Dianne Wall, June 2012
17 ALTE’s Quality Standards

 

Questions – Answers

Question: Why do students have to take the Second Certificate Examination?
Answer: As stated in the Educational Standard (Bachelor’s Degree) of Saint Petersburg University1, “the Bachelor’s Degree graduate, if it is their first Bachelor’s Degree, must have a B2 CEFR level of English or above”2.
For this purpose, the Language Testing Centre of Saint Petersburg University has developed the Second Certificate Examination, which is now used as the graduation examination for Bachelor Degree students (four years of education) and Specialist Degree students (five years of education).
Question: Why is the linking project important?
Answer: Within this project, the Second Certificate Examination is linked to the Common European Framework of Reference for Languages. This procedure was developed by the Council of Europe and described in their documentation3. As a result, the examination certificate becomes recognized because it is linked to international standards.
Question: Who is involved into the linking project?
Answer: two groups of experts:
  • external consultants and experts from Holland, Spain, Finland and Sweden;
  • experts from Russia who have been familiarized with the procedure of the Council of Europe and proved that they met the required standards set by the Council of Europe.
The project is led by:

Dr. Norman Verhelst, leading specialist in linguistic statistical analysis, a co-author of the CEFR; a member of the authoring group of the Manual and other procedures of the Council of Europe to link examinations to CEFR.

Dr. Neus Figueras, a co-author of the CEFR; one of the authors of the Manual for Relating Examinations to the CEFR (2009) and the first elected president of the European Association for Language Testing and Assessment (EALTA), of which she is now an Expert member.

Dr. Sauli Takala, a co-author of the CEFR; the second elected president of the European Association for Language Testing and Assessment (EALTA).

Elena Prokhorova, senior researcher, and Director of the Language Testing Centre of Saint Petersburg University.

Tatiana Timofeyeva, senior researcher, and Head of the Test Development Department. Coordinator of the Standard Setting procedures.

Olga Lankina, researcher, and Coordinator of Familiarisation, Standardisation Training and Expert accreditation procedures.
Question: What are the main stages of the linking project?
Answer: The guidelines for linking examinations to CEFR were suggested by the Council of Europe and described by the Language Policy Department of the Council of Europe in January 20093. According to these guidelines, any linking project must include five stages:
1 stage - “Familiarisation”. The participants of the project become familiar with six levels of language proficiency of the Common European Framework of Reference for Languages. At this stage it is important to learn how to use CEFR descriptors for assessment purposes;
2 stage - “Specification”. This stage involves describing the exam that is going to be linked to the CEFR;
3 stage – at the “Standardisation Training and Benchmarking” stage, the would be experts continue to learn how to analyze exam tasks and link them to CEFR;
4 stage - “Standard Setting”. The main objective of this stage is to set cut-off scores for every paper of the examination and prove the claim that the exam is linked to a certain CEFR level;
5 stage – The “Validation” stage has the following objectives: collecting statistical analysis data, writing the report on the linking project, and disseminating its results.
Question: What has been done within the linking project so far?
Answer: The way has been long but now we find ourselves at the fifth stage of the project, which is the final one. We carefully followed the guidelines of the Council of Europe and planned events thoroughly.
  • January - November 2013
    • groups of international and local experts were set up;
    • two coordinators (Familiarisation/ Accreditation and Standard Setting) were chosen;
    • the main stages of the linking project were defined and discussed with external consultants (Report of the External Experts, November 2013);
    • the following documents were written (more than 300 pages): reports on the quality of the test, assessment of the productive skills; interpretation of results; 3 versions of the Second Certificate Examination with statistical analysis data; (September 2011, May 2013, September 2013); reports on setting the grades boundaries (September 2011, May 2013, September 2013); rating scales for “Writing” and “Speaking”; students oral performances and writing scripts; rating sheets for Writing and Speaking; test specifications;
    • “Specification” training workshops (48 classroom hours, January - April 2013);
    • Teaching materials for “Familiarisation”, and “Standardisation Training” were prepared.
  • December 2013-February 2014.
    • “Familiarisation” workshops (40 hours, December 2013);
    • Workshops in Statistics with the aim to prepare for the Standard Setting events (40 hours, January 2014).
  • March - May 2014.
    • The difficulty levels, discrimination indexes, and content validity of the test items were discussed with the external consultants (test versions of May and September 2013: Listening, Reading, Language in Use);
    • The “Standard Setting” procedure for the receptive skills was discussed with the external consultants;
    • OPLM computer programme for statistical analysis of the test results (June 2014) was run. IRT analysis was used;
    • Further “Familiarisation” workshops (24 hours);
    • Training for the “Standard Setting” procedure. Workshops in Reading, Listening, and Language in Use.
  • June 2014:
    • Standard Setting for receptive skills;
    • Statistical analysis data was collected and analyzed. (External Experts Report, June 2014).
  • July-October 2014.
    • 52 oral performances and 78 writing scripts were selected from 6,000 student works;
    • The procedure of the Standard Setting for productive skills was discussed with the external consultants and the necessary materials were prepared;
    • “Standardisation Training” and “Benchmarking” workshops that led to the accreditation of experts.
  • November 2014.
    • Standard Setting for productive skills (Writing and Speaking);
    • Statistical analysis data collected and analyzed. (External Experts Report, November 2014).
  • January-May 2015.
    • Collecting materials: Specification, Familiarisation, Standard Training and Benchmarking, and Standard Setting;
    • Making decisions on the ways to analyze statistical data;
    • Statistical analysis of the data received, and describing the project results;
    • Writing the report on the linking project;
  • May-June 2015.
    • Presenting the linking project results at the 2015 EALTA Conference;
    • Writing the report on the linking project results.

1. Educational Standard for the tertiary and vocational education, Bachelor’s Degree, 02.07.2014 №3535/1, decree issued by the vice rector for educational and methodical work “Setting the Educational Standard (Bachelor’s Degree).”

2. Modern Languages: Learning, Teaching, Assessment. A Common European Framework of Reference: Draft 2 of a Framework Proposal. – Strasbourg: Council of Cultural Co-operation, Council of Europe, 1998.

3. Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR). A Manual