Assessment and Teaching of 21st Century Skills
Transcripción
Assessment and Teaching of 21st Century Skills
Assessment and Teaching of 21st Century Skills Claire Scoular Assessment Research Centre University of Melbourne 2nd April 2014 San Jose, Costa Rica Outline 1. Background of ATC21S 2. ATC21S: The Latin American Chapter 3. Costa Rican results and outcomes 4. The future of the project in Costa Rica 2 SECTION 1 : Background of ATC21S CHANGE BEGINS WITH GLOBAL PARTNERSHIP ATC21S is a joint effort to mobilize international, educational, governmental and business communities to bring 21st-century skills into the classroom. 4 ATC21S GOAL To develop new assessment approaches matched to new 21C skills and to advise systems, schools, and teachers on the uses of assessment data to help students develop higher order performances 5 ATC21S PROCESS Phase 1 Phase 2 Conceptualize C21 skills and education output needs Skill identification and hypotheses Phase 4 Phase 3 Task development Pilot studies and trials Phase 5 Dissemination scale and policy formation 6 SECTION 1 : Phase 1 Conceptualize C21 Skills PREPARING STUDENTS TO SUCCEED IN THE 21ST CENTURY • Changes in employment trends • Increase in workforce by 2050 • For Latin America expected increase of 26% (Wang) 8 PREPARING STUDENTS TO SUCCEED IN THE 21ST CENTURY “We are surrounded by technology, and kids are digital natives. They go into the classroom, and they don’t see the same use of technology in their education as they might in their outside lives. If education doesn’t start to address that issue, there will be a disconnect in what kids see in their lives growing up and when they get out into the workplace, which is one of the motivations for us to join this project.” e are surrounded by technology, and kids are digital natives. They go into the classroom, and they don’t see the same use of technology in their education as they might in their outside lives. If education doesn’t start to address that issue, there will be a disconnect in what kids see in their lives growing up and when they get out into the workplace, which is one of the motivations for us to join this project.” — Michael Timms, National Project Manager ATC21S for USA • — Michael Timms, National Project Manager ATC21S for USA9 TRANSFORM EDUCATION FOR THE 21ST CENTURY 10 PHASE 1: CONCEPTUALIZING THE SKILLS Assemble experts Define methods Explore practical and technical needs in the classroom Create a new framework 11 12 SECTION 1 : Phase 2 Theoretical and Conceptual Framework SELECTED SKILLS WAYS OF WORKING AND THINKING TOOLS FOR WORKING AND LIVING IN THE WORLD Collaborative problem solving ICT digital literacy Personal and social responsibility through social networking 14 PROCESS OF ASSESSMENT DEVELOPMENT Definition of constructs Collaborative Problem-solving ICT Literacy Formulation of developmental continua Identify the dimension/s Characterise the evidence Draft three profile levels Formulation of tasks Teacher view Cognitive laboratory Pilot Trial 15 COLLABORATIVE PROBLEM SOLVING: Definition Collaborative problem solving as a 21st Century Skill may be seen as a combination of its component parts. These are collaboration, problem solving and the new technologies currently believed to characterise the 21st century • Collaboration – working together with a common goal – Communication – Cooperation • Problem solving – Knowledge – Strategies 16 16 Purpose, functions and outcomes • Why? – When different expertise, knowledge or skills are both needed and available • What? – Formulate and express ideas; argue and defend – Value, review, evaluate • Outcomes – Problem solution – Enhanced skills and knowledge 17 COLLABORATIVE PROBLEM SOLVING FRAMEWORK Collaborative problem solving Cognitive Social Participation Perspective taking Social regulation Task regulation 18 Knowledge buiilding 18 Social skills strands Participation Action Perspective taking Adaptive responsiveness Social regulation Negotiation Self evaluation Interaction Task completion Audience awareness Transactive memory Responsibility initiative 19 Cognitive skills strands Task regulation Knowledge building Resource management Knowledge acquisition Information collection Systematicity Represents relationships Ambiguity tolerance Identifies consequences Organisation Goal setting Hypothesises 20 ICT Literacy Intellectual Capital Social Capital Producer Consumer 21 SECTION 1 : Phase 3 Task Development Process Theoretical Framework Interpret Task Design Score & Intervention Report 23 24 Laughing Clowns 25 25 Indicative Behaviours Skill Behaviour in task Interaction - Chatting with partner Presence of chat before allowing partner to make a move Audience awareness - Adapts contributions to increase understanding for partner Number of ball moves attempted before stopping and waiting for partner to respond Reponsibility initiative - Takes responsibility for progress for the group task Communicating with partner before first half of balls have been used Resource management Realises that balls are meant to be shared and only uses allocated half Relationships - Identifies patterns between information Both students coming to agreement on how their machine works 26 Balance Beam 27 Indicative Behaviours Skill Behaviour in task Action - Active in scaffolded tasks Student A passes B a mass Task completion – undertaking part of task individually Follows instructions, moves 100g to position 4 Responsiveness – responding to contributions of others Realising that some masses cannot balance. If Student A resends 50 or 500, B returns it immediately Systematicity - Implements possible solutions to a problem Trying out all combinations of masses on different beam positions Solution - Correct answer Number of successful balances achieved (3 optimum) 28 Skill progressions SOCIAL LEVELS Level 4 PARTICIPATION The student initiates and promotes interaction with their partner; sharing resources, information and ideas. The student perseveres in solving the task as shown by repeated attempts and/or multiples strategies Level 3 The student is actively participating in the task in familiar and unfamiliar contexts. The student interacts with their partner more (i.e. responds to communication cues and requests) and shares information and/or resources with partner. Level 2 The student is actively participating in the task when it is familiar and/or scaffolded. There is limited interaction with partner and this is mainly prompted by task instructions. Level 1 The student participates in the task but is working quite independently Task activity is usually directed by system instructions 29 Reporting to Students: Learning Readiness Real time reports based on empirically developed learning progressions. They will identify a point of readiness to learn for each student. This type of report can be linked to teaching interventions associated with readiness to learn as indicated by the black bar in the spine of the chart. 30 SECTION 1 : Phase 4 Pilot studies and trials SECTION 1 : Phase 5 Dissemination & Policy Formation 33 Dissemination Resources will now be placed in the public domain Atc21s.org 34 Trialing Tools Concept Checking Cognitive Laboratories Pilots Trials Teacher pedagogy workshops 35 SECTION 2 : THE LATIN AMERICAN CHAPTER Objectives and Benefits to Costa Rica • Validate the assessment through pilot and trials • Results could continue the agenda and lead Latin America in 21st Century skills • Contributing to large international project • Gain ideas about future implementation of 21C skills • Evaluate curriculum • Regional dialogues with peers as well as feedback from stakeholders in the global initiative 37 ASSESSMENT OF COMPETENCIES OF THE 21ST CENTURY Three exemplary contributions: – Language adaptions – Customization of ICT Literacy Tasks – Localized website 38 LANGUAGE ADAPTIONS 39 LANGUAGE ADAPTIONS 40 LANGUAGE ADAPTIONS Componente cognitivo Descripciones de nivel Nivel 5 Resolución refinada de problemas En este nivel, los estudiantes pueden reconocer errores previos y transferir y usar ese conocimiento en las secciones más complejas de las pruebas. Reconocen qué recursos han causado errores previos y los evitan. Están desarrollando confianza en el manejo de las pruebas y esto les permite corregir o superar las respuestas/acciones de sus parejas. Los estudiantes comparten con sus parejas recursos apropiados y usan estrategias consistentes que han identificado como efectivas en pasos anteriores de la solución del problema. Todo el planeamiento y la mayoría de las estrategias implementadas se realizan dentro del contexto del equipo. Nivel 4 Trabajo eficiente En este nivel, los estudiantes comienzan a resolver problemas de manera colaborativa y a usar estrategias de cooperación más efectivas. En este momento pueden simplificar el problema y revisar varias veces la ejecución de secciones previas de las pruebas para determinar si la regla es transferible a las páginas o secciones siguientes. En este nivel, el intercambio de información entre estudiantes incrementa y se hace más eficiente. Su secuencia y el juicio y la error acciones requieren pocos intentos y por lo tanto se concluyó en una cantidad óptima de tiempo / intento. Las completan en un tiempo óptimo. Las acciones parecen estar bien pensadas y planeadas, y cada acción parece tener un propósito. Pueden identificar causas y efectos y usar estrategias adecuadas para lograr una vía de solución correcta. El estudiante dirige y guía a su pareja hacia acciones y recursos apropiados. Nivel 3 Planeamiento y ejecución estratégicos En este nivel, los estudiantes pueden completar tareas de menor dificultad de manera correcta e independiente. Son capaces de completar el último paso o sección y ayudar a su pareja en este proceso. Incrementan su trabajo juntos al planear estrategias de trabajo, establecer objetivos y compartir sus recursos. El estudiante tiende a enfocarse en los recursos relevantes y desecha otros que no fueron beneficiosos en intentos anteriores. Son capaces de llegar a respuestas de común acuerdo cuando esto es necesario. En pruebas con varias secciones el estudiante tiende a identificar estrategias y recursos exitosos y aplicar las 41 mismas reglas repetidamente. El estudiante continúa intensificando la investigación secuencial y el ICT TASK CUSTOMIZATION Poetry 42 ICT TASK CUSTOMIZATION Antarctic Trek 43 LOCALIZED WEBSITE 44 Trialing the assessment tools Trials 2011 (data collection CPS) 16 elementary and high schools 90 11-year-old students, 222 13-year-olds, and 188 15-year-olds Trials 2013 (ICT and system testing) 9 high schools 776 students 45 SECTION 3 : SPECIFIC COSTA RICAN OUTCOMES ALL COUNTRIES COMBINED Valid CPS construct Item Separation reliability 0.999 Person separation reliability 0.814 Regression variable -0.182 47 ALL COUNTRIES COMBINED Two separate dimensions (social & cognitive) Social Cognitive Separation reliability 0.998 Person separation reliability 0.754 0.770 Mean Latent Ability 0.560 -0.824 Correlation between dimensions 0.788 48 Summary of data in Costa Rica • 575 students were scored • Sample only not representative • Focus is to validate the test not to compare countries 49 COUNTRY COMPARISON Costa Rica All other Countries Mean of Latent Ability (StD Error) -0.121 (0.021) 0.145 (0.01) EAP-PV reliability (Similar to Cronbach’s alpha) 0.711 0.783 Reliability of Item Separation 0.993 0.999 The psychometric properties of the tasks in Costa Rica were similar as in other countries, indicating that these tasks can be completed in Costa Rica with little to no language, cultural, or country specific bias. 50 Differences in the difficulty of the items were minor or non-existent Out of 150 items only two were more difficult for CR than other countries Hexagons - Answering the final answer correctly Small Pyramids – Responding to questions from partner Difficulty = Frequency Could be explained by time constraints 51 Two items were easier for CR than other countries Sunflower - presence of chat when growth appears Survey (self assessment) - Replying to chats However, there are many items that assess each skill and these differences are based on single items so they are not representative of overall ability in a specific skill There is no particular task that is more difficult for Costa Rica than the other countries. 52 SECTION 4: IMPLICATIONS AND FUTURE DIRECTIONS Now we have… Two skill constructs Assessments to test skills Share results for teaching 54 Now we need… Strategies for teaching Resources for teaching PD for teaching Developmental assessments 55 Domain specific or general curriculum Expected to teach subject content and develop 21st century skills • assess and intervene in both key learning area and general capabilities Required to think beyond discipline expertise • • current teacher training mainly discipline-based pedagogy most effective methods of teaching skills in one discipline may not be same in another 56 EMBEDDED INTO CURRICULUM : Australian Curriculum GENERAL CAPABILITIES Literacy ICT Capability Numeracy Critical and Creative Thinking Personal and Social Capability Ethical Understanding LEARNING AREAS Science History Math Intercultural Understanding English Australian Curriculum and Reporting Authority, ACARA (2013). 57 EMBEDDED INTO CURRICULUM : Singapore Curriculum http://www.moe.gov.sg/media/press/2010/03/moe-to-enhance-learning-of-21s.php 58 EMBEDDED INTO CURRICULUM : Costa Rica SEXUALIDAD ESPAÑOL MATEMÁTICAS ÉTICA, ESTÉTICA Y CIUDADANÍA PRIMARIA (MÚSICA, ARTES PLÁSTICAS, EDUCACIÓN FÍSICA, CÍVICA Y ESTUDIOS SOCIALES) VIDA COTIDIANA Y ARTES INDUSTRIALES ÉTICA, ESTÉTICA Y CIUDADANÍA SECUNDARIA (MÚSICA, ARTES PLÁSTICAS, CÍVICA, EDUCACIÓN FÍSICA) CONVIVENCIA 59 SUSTAINABILITY 60 61 62 63 Poetry 64 Antarctic Trek 65 Why engage in the assessments? Establish a baseline for academic standards Influence curricula at local, regional, national and global levels Generate new approaches to Influence curricula at local, regional, evaluation and teaching national and global levels and 66 Claire Scoular [email protected] Esther Care [email protected] www.unimelb.edu.au/arc www.atc21s.org www.actrc.org © Copyright The University of Melbourne 2014