Noelle Rouxel-Cubberly Valeria Belmonti

Noelle Rouxel-Cubberly Valeria Belmonti

Noelle Rouxel-Cubberly
Assistant Professor of French
Valeria Belmonti
Director, Modern Languages Media Center
College of Staten Island
City University of New York
CHALLENGES
CUNY realities: large classes, demographics
Requirement
Passive learning habits
TECHNOLOGY ASSETS
Digital Natives
Increased speaking time
Formative assessment
Our Goal:
ACTFL Intermediate Low level of oral proficiency by the end of 3rd semester.
Our Measurement:
Course-embedded
embedded assessment of oral proficiency.
1500+ STUDENTS PER SEMESTER
Class appointments setup at the beginning of the semester
15-30
30 minutes at the beginning or end of the lecture (“formative time”)
2-8
8 oral assignments per class per semester
Content of exam posted on online message boards
Students recorded audio answers using Audacity
Response file posted on the message boards
No mechanism to control test progression
Grading procedure
Instructors used external grading tools
No systematic way to distribute and analyze results
Mean score for Spanish 114 oral assessment was 86.1%
Students could hear questions and record answers unlimited times
No time limitations to answer questions
OWL Testing Software
www.owlts.com
Section
1
Test
Section
2
Section
3
•
•
•
•
Item
Item
Item
Item
• Item
• Item
• Item
• Item
• Item
Within the test:
•Order sections by number
•Randomize sections
Within each section:
•Order questions by number
•Randomize questions
Instructor
decides if
questions
are displayed
simultaneously
or not within
a section
Instructor decides if and
audio/video question can be
played back or not
Instructor decides if students can
change previous responses
RECORD TIME:
Test level
Section level
Item level
THINK TIME:
No time to think about answer
Unlimited time to think about answer
Set amount of time to think about answer
Students retrieve grade online
Specific corrections: efficiency, personalization, input in TL
Overall comments: visualization, references, proficiency-oriented,
proficiency
independent learning.
Spring 2011 - Mean score for SPN 114 midterm oral exam 73.5%
Film and Literature Activities
Video clips Comments
Half-guided
guided Monologues
Situations
Question to
Answer
Pronunciation
Arrivi a casa a Brooklyn e ti ricordi di aver lasciato le
chiavi nel tuo ufficio in New Jersey
Mi universidad es muy
grande y hay varios
edificios. Hay dos
gimnasios, uno con piscina.
Hay una biblioteca grande.
Hay varios laboratorios de
lenguas y de las ciencias.
También hay una librería
donde los estudiantes
compran los libros de texto
y plumas y cuadernos.
¿Qué hay en tu universidad?
Hola ¿Cómo está? Me llamo Teresa. Soy de Oaxaca, México. Mi estación preferida del año es el otoño. Durante los meses de otoño no hace mucho calor, hace fresco. Algunas veces llueve, pe
Hola ¿Cómo está? Me llamo Teresa. Soy de Oaxaca, México. Mi estación preferida del año
es el otoño. Durante los meses de otoño no hace mucho calor, hace fresco. Algunas veces
llueve, pero no siempre. También me gusta el otoño porque en esta estación celebramos una
de las fiestas más tradicionales de mi país: el día de muertos. El día de muertos se celebra el
uno de noviembre. En esta fiesta celebramos a nuestros parientes muertos. Hay muchas
decoraciones típicas de esta celebración, como el papel picado… o las calacas… y más
importante: los altares. Los altares se montan en honor de los parientes muertos. En ellos
ponemos decoraciones y también ofrendas de comida. Normalmente ponemos la comida
favorita de nuestros parientes y también la comida típica de la celebración… Algunas de las
comidas típicas de esta fiesta son: el pan de muerto… Y las calaveritas de azúcar. Si algún
día viene usted a México durante el día de muertos, pida estas comidas típicas, cómalas
y visite los altares. Estoy deseando conocerla. Escriba pronto. Hasta luego.
Vocabulary and grammar knowledge in the context of specific cultural backgrounds was evaluated in exam 1.
Students were shown a picture randomly selected from 3 movies among the 6 analyzed in the course
Oral exam 2 assessed both specific and global understanding of the film selected by each student.
Each student only saw a clip related to his/her film.
FACULTY
STUDENTS
Group workshops
1-1 meeting
Video and paper tutorials
Faculty discussion board
PowerPoint online
Sample OWL Test in English
Handout at computer desk
Proctors during exams
Spring 2010
Fall 2010
Spring 2011
Pilot Project
All French and Italian
Courses
All Spanish courses
4 classes
Spanish courses taught
by full timers
Chinese pilot project
Multiple choice, True/False, Fill-in,
in, Short Answer are self-graded
self
by OWL
CHINESE
Computerized midterm
and final exams
More tasks assessed
than in paper exams
Minimize grading
AMERICAN SIGN
LANGUAGE
Finger spelling
Receptive skills
PLACEMENT
EXAMS
Adaptive tests
System-graded
Customized to our
programs
STUDENTS
FACULTY
Effective training but some students prefer
audio instructions to written instructions
on test screen.
Effective training but need to share
ideas and practices among instructors.
OWL is simple to operate and very intuitive.
For some students, timing positively
forces to think fast in the target language.
Most students would prefer no timing
and ability to re-record answers.
OWL is easy to use but:
Too many steps required to modify grades
Somewhat difficult to view grades
Too many steps required to print reports
Verbiage not always intuitive
Concerns over CSI network performance.
Feedback time consuming.
Instructor’s feedback very useful.
[email protected]
[email protected]
We thank our colleagues, Jerry Lombardo, Jill Jegerski, Lucas Marchante, Gerry Milligan, Sarah Pollack, Kathryn Talarico,
Paola Ureni, Oswaldo Zavala for their contributions.
We thank Susan Holak and our VP for Technology Systems, Dr. Michael Kress for his ongoing support!