Time

Topic

Aids

• Michele Anciaux Aoki
• Brent Hester, John Stanford International School
• Maria Buceta Miller, John Stanford International School
• Participants from Sheridan Elementary and Stewart Middle School

Tell us a bit about what you have learned so far this week.
 

Listen

What does assessment mean to you?

The topic of our workshop today is Assessment in a Language Immersion Setting. Find a partner and take a moment to share what "assessment" means to you. What is it? Why is it important?

• Give them a few minutes to talk, then come back to large group.

What did you discover?

Fundamentally, assessment is how we tell:
But are they learning?

To answer that question, we need to know:

1. What is to be assessed

2. How to assess it

What are some of the things you currently assess in your classroom or school?

• They should mention things like Reading, Writing, Math, Science, perhaps Spelling, etc.

Now, let's think about how you assess these things.

• Have them take out the materials on the left-hand side of the packet.

This Assessment Profile from the New Jersey World Languages Curriculum Framework shows a selection of types of assessments.

• Go over the four quadrants and ask them which ones they do at their schools.

• Ask if they'd like to share with the others any particularly successful assessments they've used/developed.

• Which kind of assessments do they like best? Least? Why?

• Which are the hardest to do?

• Which are the most rewarding for the students?

• Which give them the best answers to: But are they learning?

Easel: Write down comments

Show Stiggins' book

Easel: Write down comments

OH: Assessment Profile
(NJ WL Framework p. 234  (2 in PDF)

What about the immersion language?

There are many things that you assess on a regular basis, either formally or informally. Some assessments -- like WASL -- are very much on your mind. But there's something that's easy to overlook when you're teaching in a language immersion setting. And that's the language that your students are learning!

• Ask them why that might be the case.
  (Because the focus is on content, on "acquiring" the language rather than being "taught" the language.)

So, is it important to assess the immersion language too?

• Yes. Otherwise, how do you know whether you're meeting the goals of the immersion program? How do you ensure that students can understand well enough to do the academic work? How do you let parents know that it is worth supporting the immersion program?

Well, then, what are you going to assess? And how will you assess it? That's what we'll be exploring today.

To begin, what are some things about the student's knowledge or ability in the immersion language that it might be reasonable to assess?

• Show the K-1st Spanish Goals from last year.

• Ask them what some ways might be to assess some of these goals. (Refer to Assessment Profile.)

• Are they using any of the Goals sheets that we set up last year?

HO: K-1st Spanish Goals (Doc)

Tools for Assessment

How many of you have used Rubrics or Rating Scales for assessment before?

• Show example generated from teach-nology.com.

• Explain how easy it is to generate this rubric.

Rubrics are a very handy tool to use for assessment. They can help you capture useful information to guide your teaching and provide feedback to the student. They can also be a good tool for peer assessments and self-assessment.

There are two main kinds of rubrics: Holistic Rating Scales and Analytic Rating Scales.

• Have them turn to page 241 Sample Assessment Rubrics

• Explain the difference

Turn to your neighbor for a moment and talk about what the advantages and disadvantages might be of each.

• Give them a moment to look over the example and discuss this, then ask what they think.

• Which would they be more likely to use and why?

Now, let's take a brief tour through some other sample assessment rubrics for World Languages. These are all from the NJ WL Curriculum Frameworks, although many of them were adapted from the Nebraska FL Frameworks. We'll be going through these fairly quickly, but I'd like you to notice which ones appeal to you right away -- you think they might be a rubric you'd consider using in your class.

• Figure 5: Generic Rubrics for Collaborative Work, Oral Presentations (Simple Answers), Oral Presentation (Cultural Role Play), Written Materials (General), and Written Materials (Creative Writing)

Do any of these look like Rubrics you'd consider using in your classroom? Why?

• Figure 6: Portfolios
  (Ask them if any of them do Portfolios in their classrooms.)

• Figure 8: American Sign Language
  (Are they still interested in doing ASL?)

• Figure 9: Oral Activity Self-Evaluation

Have any of you incorporated self-assessments in your classes? We did one this spring at John Stanford International School. It was an online assessment and one 2nd grader's comment was: "Thank you for asking me about my Japanese. I have not got this test before. This test is fun! I want to have this test again. Do you have other tests?"

• Figure 10: Oral Report
  (Great way for the listeners to give feedback.)

• Figure 11: Story Evaluation
  (This could be feedback on language, without asking about things like grammar and vocabulary!)

• Figure 12: Expressing a Point of View
  (Very helpful for students developing skills to help them do well on the WASL.)

• Figure 13: Story Retelling Checklist
  (This could serve as a good self-check for comprehension of a story read aloud in class by the teacher.)

Incidentally, almost any of these rubrics could provide strong evidence of the learning that is happening in your immersion classroom. They would give you something concrete to show your principal and parents or the school district.

• Ask them which ones they liked best and would be most likely to use (or adapt).

HO: Presentation Rubric (Doc)

HO: Sample Assessment Rubric (NJ WL Framework p. 241)

HO: Sample Assessment Rubrics
(NJ WL Framework p. 238-248)

Many of the Rubrics we just looked at would be great for assessing student written reports, oral presentations, portfolios, understanding of story content, etc., but none of them are quite suitable for measuring a student's ability to speak and understand language in a conversation. Yet this type of interpersonal communication is fundamental to learning another language. In some ways, it's also the hardest part of learning the language because you're interacting on the spot without having time to prepare your presentation or write your report in advance. Ultimately, assessing oral proficiency is an important part of knowing: But are they learning?

Back in 2001, when we first started the John Stanford International School, we were fortunate to be able to arrange for the Center for Applied Linguistics (CAL) to send a team to assess the oral proficiency of our students in their first year of Spanish partial immersion. Over the years we have worked with CAL to help train teachers at John Stanford, as well as some students at UW who helped us in the early years. CAL also produced statistical reports to help us see how our students' language proficiency was developing year by year.

In the past two years, the teachers at John Stanford have taken on the responsibility for doing this proficiency assessment themselves, and even training new teachers to do it. Brent and Maria have really led the way. Both are fortunate to have been trained by an expert from CAL.

I invited them to work with us here today so you could see what is entailed in doing an effective Oral Proficiency Assessment, and how you can build that type of assessment right into your classroom work, using a common rubric that we'll be introducing shortly.

As the popularity of immersion and other elementary language programs grew in recent years, CAL realized that teachers and schools needed tools that would be helpful for assessing very young language learners. So, they developed the ELLOPA -- Early Language Listening and Oral Proficiency Assessment. I first saw the ELLOPA presented in Boston at the ACTFL Conference in 2000, and I recognized that it was something we should bring to John Stanford International School.

The ELLOPA has two components: an ELLOPA Interview (conducted by someone who is not the student's teacher) and the ELLOPA Teacher Observation Matrix, which is, obviously, completed by the teacher. The two together help create a balanced picture of what a student can do.

What we've learned is that the ELLOPA Interview is also a great way for teachers to gain insight into how to get students to produce as much language as they can and how to differentiate student performances using a standard rubric.

What’s the purpose of the ELLOPA Interview?
To assess proficiency – how well students use the language – not to test what they know about any particular topic. The goal, then, is to get them to demonstrate their listening comprehension and to speak at as high a level as possible.

I would now like to invite Maria to come up and give you a tour of the ELLOPA Interview protocol and share some of her experiences with the ELLOPA over the past five years.

• Introduce yourself and explain that we will be watching an interview in Spanish. It was with two children in their first year of partial immersion. Although the non-Spanish speakers might not understand all the individual words, they can follow along with the transcription of the interview. (You can also give them the English script -- used for Japanese, though it is slightly different.)
   
• Have them take out the materials on the right-hand side of the packet (ELLOPA scripts). (Hand out the English version to Japanese teachers if they want it.)
   
• Go over the script from 2001. Note the elements
  (Game 1: Warm-up: Magic Purse; Game 2: Let’s talk with Señora Vaca; Game 3: Let’s tell a story (or Classroom); Game 4: Let’s sing)
   
• View sample interview on video (Alden & Noah).
   
• Hand out the transcribed script and go over with comments from Beverly Boyson of CAL.
   
• Talk about your experience of learning to conduct an ELLOPA Interview. Was it hard? Has it been helpful to you as a teacher? What have you learned from interviewing students over the years?

[Brent: Feel free to jump in with comments too!]

HO:
2001 Script-Spanish (PDF)
2001 Script-English (PDF)

HO: Transcription of Interview

Video: Sample Interview

• Introduce the ELLOPA-RP.
• Go over the Criteria on the Rubric: Oral Fluency, Language Control (Grammar), Vocabulary, and Listening Comprehension.
• Then, consider the Levels on the ELLOPA-RP: Jr. Novice Low, Jr. Novice Mid, Jr. Novice High, and Jr. Intermediate Low.

Possible activity (if you think it's appropriate or do something different):

• Have 4 volunteers line up at the front of the room, each one taking one of the 4 ELLOPA levels. Then you say "Oral Fluency" and each level reads the text in their box (from Jr. Novice Low to Jr. Intermediate Low). Then do the same with Language Control, Vocabulary, and Listening Comprehension.
  (Goal is to leave them with an auditory and visual association with the different levels.)

HO: ELLOPA-RP (PDF)

• Show them a blank Interview Rating Sheet used for taking notes.
• Show them a copy of Beverly & Linda’s sheet from 2001 (and/or examples from this year).
• Have them think back to the Alden & Noah Interview they watched and imagine how they might have rated the students using the ELLOPA-RP.
• Hand out the Justify Ratings sheet from the Alden & Noah ELLOPA Interview and go over the actual ratings. Answer questions they might have about how to notice the distinctions in the levels.
• You might ask the non-Spanish teachers to think about comparable examples they might have seen in French or Japanese.

Example: ELLOPA-Rating Sheets

HO: Justify Ratings

The ELLOPA Scale is helpful for students at the early levels of language acquisition. It allows us to begin to measure growth even at the early stages and even with very young children. Over time, more and more students will surpass the levels in the ELLOPA Scale. At that point, you can make use of the SOPA Scale -- which measures 9 levels of performance. (SOPA stands for Student Oral Proficiency Assessment)

• Hand out the SOPA-RP.
• Work in pairs to review the levels. (Or do some other activity.)
• Talk about how you've seen students progress to these higher levels, and also how native speakers are rated.

HO: SOPA-RP (PDF)

Note: We can invite Japanese teachers to go to another room to view some Japanese interviews we have. French teachers can probably stay with Spanish teachers, but may need translations.

• Show them the SOPA Spanish Samples to Illustrate Oral Fluency Levels of the Rating Scale.
   
• Let them practice rating sample language
   
• Go back over the justifications for the sample ratings.

Take a moment to think about students you've had over the past year. If you were to rate their typical language comprehension and production, where would you place them on the SOPA Scale.

• Pair K-1, 2-3, 4-5 teachers
• Have them call to mind indications of listening comprehension, oral fluency, language control, etc.

HO: SOPA Spanish Samples (Doc)

HO: SOPA Spanish Samples for Practice Ratings (Doc)

HO: SOPA Spanish Samples Rating Justifications (Doc)

Welcome back from lunch.

• What questions do you have so far?

We invested some time with you this morning to help you really get to know the ELLOPA and the SOPA Rating Scales or Rubrics. That's because they are used not only for the ELLOPA or SOPA Interviews, but also for teacher ratings.

• Have them take out the SOPA Rating Profile again

In the spring, the immersion teachers complete one of these evaluations for each of their own students, identifying at which level the student has been performing, based on the classroom experience of the teacher.

Taken together, the on-demand ELLOPA or SOPA Interview and the Teacher Observation Matrix provide a pretty balanced assessment of the student's oral language proficiency. Working with CAL in the first couple of years, we could see that on the whole, the teacher ratings were pretty similar to the ratings from the interviews. This is encouraging. The goal would be for classroom-based observations to become a valid and reliable measure of student oral proficiency.

In years when we have not done the ELLOPA or SOPA Interviews, we have still encouraged the teachers to complete the Teacher Observation Matrix so that we can continue to gather data about the students' progress using the same tool.

Now let me turn it over to Brent and Maria to talk you some more about this from the teacher perspective.

HO: SOPA-RP (PDF)

• Talk about how you've done the TOMs in past years (given sub time, etc.) -- How long does it take, how difficult is it, do you need to do your own interviews of your students in order to check your ratings?
• Brent -- talk about how you keep track of ratings each year and provide that information to parents too.
• Debrief with them... what would it take for them to feel comfortable using the SOPA - Teacher Observation Matrix to rate their students' oral proficiency?

• Show examples of the ELLSSA used for K-1
• Show example of Student Self-Assessment developed by CAL (for higher grades)
• Talk about benefits of involving students in their own assessments
• Get ideas for how they might do this

Show: ELLSSA examples

HO: Student Self-Assessment (Doc)

• Have them break up by Language Family (French, Spanish, Japanese) and talk about how they might track development of students' proficiency across the years.

• How can you communicate with parents about how students are doing?
• Show sample reports from ELLOPA data (generated with mail merge) that have been shared with parents over the past years

Show: Examples of reports to parents