Testing the Test

Every time I design a test or project I have this underlying sense of dread. What if everyone fails? What if I haven’t prepared students correctly? What if everyone bombs it and I have to make a new test?

I really hate the reading curriculum in our school district. The tests at the end of each unit are constructed mainly of fixed response format items in the form of multiple-choice items, fill in the blank, matching, true/false, short answer and essay. It takes me almost as long to prepare my students for the tests as it does for them to read one or two of the stories in the book and to go through the entire comprehension process. There are many times I hand student a test and I still get incorrect answers to questions I went over in class only a day before and I feel frustrated. As a lover of reading I felt that some of my most wonderful reading moments in school happened when I was allowed to draw my own conclusions and create projects and assignments that showed the rest of the class how I felt about the book and how I related to it. I hated the reading tests that I would get in high school and college where I was supposed to match definitions and talk about the literary elements. Sure, students need to know that stuff, but I think there is a lot to be said in giving them the tools for discovering those things and then allowing them room to discover them. Students are supposed to know about theme and literary elements but I feel that what we are doing is forcing them to regurgitate information instead of discovering it for themselves in a meaningful way.

Another beef I have with our district are the proficiency tests that we give our students. The problem that I have seen so far with tests and giving tests is that most of the time the examiners vary in terms of how they judge the test. Quite often, the way the teacher grades or determines if a student is proficient may not be how the next teacher sees it. This causes a lot of frustration in our district.

One thing I reacted strongly to in this chapter was the idea that tests are validated enough to satisfy the politicians and other stake holders. After watching two students struggle through the HSGQE this past week I can’t help but feel angry that the people who decide if the test is good aren’t the ones sitting in a room for 5 or more hours trying to pass it so they can get a high school diploma. Yet, these students don’t have a say. They aren’t being asked how they felt about the test or given alternative opportunities to show that they are proficient in a content area. I also reacted to the fact that many tests aren’t validated because validation is hard and technical and expensive. I’m glad that all those people were able to get high school diplomas before this silly idea of high stakes testing came into play. Do you think they would still feel it too much of a burden if they had to begin more high stakes tests in their lives. Perhaps every time they wanted a raise or were up for a promotion they had to sit in a little room taking a test that they may or may not be prepared for and have that feeling in the pit of their stomach that they are running out of opportunities to pass the test. Perhaps if they sat in a student’s chair sweating it out they would think differently about the expense of making sure the test is valid.

Multiplicity

Here is a picture of me and Macy reading this article. If you can notice, she's lost a little bit of weight. She loves carrots. She also really loves the concept of "multiplism".

As we were reading this Macy and I were discussing some of the assessments I use in my classes. One class I really would love to develop stronger assessment methods for is my writing class. I often feel that I need to constantly edit and help the students revise everything they write. I often forget that every once in awhile I need to give them an assessment to see what they can do on-demand without my help. There is just as much value in allowing them to make mistakes and not talking about them as are there are in discussing key issues in their papers.

Again, the ideas of portfolios came up and again, lol, if anyone has some advice or websites I can visit to get some help in starting the implemetation of portfolios in my own clasroom I would greatly appreciate it.

One thing that I seem to tend to forget as a teacher sometimes is that we are constantly assessing the language use of our students. Today I had studnets define "apartheid" and they all looked in their books and copied down the definition the book gave. I then had them create a concept map on the board of all their different definitions. The final step was to have them partner up and pretend that I was a 10 year old and they had to explain the "book" definition in simpler terms. Many of my students struggled the activity and it really made me understand that this concept of apartheid was still new and unfamiliar to them. Although it was a quick and relatively unintrusive assessment I still was assessing where my students were in terms of understanding and defining this concept.

Top Design

In the past few years LKSD has attempted to include more authentic assessment in our curriculum. The theory and ideal behind the phase system was to create more authentic assessments for students that aligned with the state and district standards. However, the implementation of the phase system has had quite a few...hiccups (I'm being nice) .

However, I applaud the LKSD's District Office staff for one very valuable experience that I really wish they would continue. Every fall, the DO would ask students to participate in a district wide writing assessment. The students would respond, on demand to a writing prompt at their phase level. We would then send the writing samples in to the DO to be assessed at the Writing Assessment Training.

I was unable to attend the writing assessment training for two years due to the fact that the river was freezing up and we were stuck in Oscarville. Last year I was fortunate to hitchhike a helicopter ride to the assessment and found it one of the most valuable trainings I have participated in since coming to work in LKSD.

For three days teachers from all over the district joined together to grade all assessments. We were trained how to assess the papers using the same analytical scoring guide we used in our classrooms. For three days we debated, argued and reasoned with each other about how to score and why we chose to score the way we did. When I was reading this chapter I kept seeing myself at the training and although I did feel that the training was valuable while I was there I now see the value of it even more so and really wish that it was something that the district chose to keep going.

I'm kind of saddened that LKSD chose to not participate in this type of training any longer because it cost too much to fly in and house teachers for the three days. However, a few teachers from neighboring villages to Oscarville have expressed interest in doing our own little writing assessment and continuing the tradition on a much smaller scale.

One idea from this reading that I thought was so important was the idea that we need to take the assessments ourselves and score our own papers. It seems like such a no brainer that we put ourselves in our students' positions to try to understand what it is we are asking them to do.

One more idea that struck home with me is the idea that not all problems with assessments are going to be solved simply by switching to authentic assessments. I find that many of the assessments created with the phase system are much harder than some of the multiple choice tests at the end of a chapter. One example is that many of the reading assessments are very highly-language dependent. I spend much of my time showing students how to construct answers to certain questions just so they know how to answer them on the test and have a model of how to answer those questions. Although I do feel that it is necessary to show students how to answer questions on a test I feel that I spend more time showing them how to answer the test questions sometimes than I do on talking about the literature or having them create a project or participate in other meaningful activities.

Grades...I feel pretty strongly about grades. I often get frustrated at the fact that I have to sit down and do so many grade calculation for each of the classes I teach but the students seem to care very little about their grades. Grades really do not motivate me as a learner and in high school and even college grades meant very little to me. I don't get motivated by seeing A's on a report card. When I was in college I often felt like I did some mediocre work that received good grades and some pretty good work that got some hard criticism from my professors.

As a teacher I find that many of my students are not really all that motivated by grades. Most of them don't come in with their parents to conferences and when I do sit down and talk to them they nod their head and tell me what I want to hear and then walk away to do something else but their homework. It's frustrating. I want students to care about their grades but more importantly I want them to be motivated to do homework and to set goals for themselves.

What I would really like to be a part of in my school and district is to design more authentic assessments for our learners. I really wish that the district would lead unserviced in how we can guide and coach students to set their own goals and to become motivated learners and how we can create better authentic assessments.

Also, after reading this article I really, really, really, want to use portfolios and have the students present their portfolios to the community at the end of the year. If anyone uses electronic portfolios I would love to pick your brain!

State of Alaska LEP Accommodations

Here is a Power Point I found outlining the LEP Accommodations for the State of Alaska. It's kind of distorted but you can get the gist of it.

If you want to see it in its full glory go to: http://209.85.173.104/search?q=cache:0-KPdeieDGYJ:www.eed.state.ak.us/tls/assessment/presentations/Fall%252006/AlaskaLEPAccommodations.ppt+state+of+alaska+LEP&hl=en&ct=clnk&cd=1&gl=us&client=firefox-a

You could also Google "State of Alaska LEP Accommodations" to find the PowerPoint

What's going to happen?

Here is a list of discussion activities I am planning for class today. Everything should be ready to go.



So, here is a bulleted list of everything we are going to be doing in class today (at least during this discussion).


• Check-In

• Check out the synopsis on my blog

• Note activity/Discussion of Abedi's Six Issues with Assessment of English
* Each person will be assigned one issue
* They will then post at least 2-3 key ideas Abedi mentioned about that issues (please post
these in the comment section of the original post)
* As you post please be reading other people’s comments and add to them or comment
on what they wrote
* When everyone is done we will go back through and quickly discuss the main ideas of each
issue

• Lingering Questions/ Questions I still want to ask that may not have been covered in the activity

Abara Cad-abedi

Hey folks! I'm the discussion facilitator tomorrow and I was asked to provide a brief synopsis of the article. Here it is for all to see! I will be directing you back to this entry for one class activity tomorrow!

Synopsis of Article:

Within this article Abedi identifies and discusses six major LEP assessment issues as they relate to AYP reporting. The six major issues are as follows:


1. Inconsistency in LEP classification across and within states




2. Sparse LEP population




3. Lack of LEP subgroup stability




4. Measurement quality of AYP instruments of LEP students





5. LEP baseline scores





6. LEP cutoff points





(We’ll look at these more in depth in the activity!)

Abedi also identified other factors that can affect AYP. One major factor that can affect an LEP students’ ability to become proficient in the content is their ability to understand the instruction that is given. This lack of understanding may result in LEP students to participate less in class and even when LEP students did raise their hand the teacher was apt to call on them less.

Abedi goes on to discuss that LEP student don’t necessarily lag behind non-LEP students in terms of content knowledge but the biggest limiting factor on tests are the linguistic structures of the test. Also, because LEP students at such a lower baseline than native English speakers and lag farther behind native English speakers it will be near impossible for them to move at the same rate towards proficiency as native English speakers.

Abedi also discusses that the classification LEP students as well as the number of LEP students in each state also leads to inconsistent AYP reporting. Many districts and states have smaller numbers of LEP students and these small numbers make it difficult to reach reliable AYP reporting.

NCLB does not completely ignore LEP populations. Title III of NCLB does address students with Limited English Proficiency, however, Abedi believes that this wording is not enough to support and provide adequate testing for all LEP students in the entire nation. He goes on to state that the test developers should take into account recommendations to ensure that the mistakes that were made in the first tests do not happen again. Abedi discusses that many of the tests are questionable in terms of reliability and validity.

Abedi ends his article by giving ideas for improving the AYP outcome for LEP students. He says that there are four critical needs to help LEP students reach proficiency. They are 1) improve current LEP classification an assessment, 2) improve monitoring of progress, 3) improve teacher capacity and 4) consider redesignated LEP students as part of LEP subgroup that established the baseline score.

More importantly, Abedi concludes his article by discussing his purpose, to raise awareness of this issues and that it is his hope that policymakers, lawmakers, and decision makers to consider the struggles of the LEP student population when making decisions.

McNamara

One thing I liked about this chapter was that it laid out the history of language tests so that we could clearly understand how the tests and test construction has changed within the last 50 years or so.

When I read about the discrete point tests I could not help but think about the grammar drill exercises I used to receive in grade school. Then, suddenly, when I entered Junior High School the grammar exercises dropped off and I didn't get another grammar lesson until college. We then had to create sentence diagrams to "show" that we knew the parts of speech and where they all belonged on our neat little diagram. I actually liked diagramming (until it got hard) because it helped me to understand sentence structure and patterns.

So, the whole point of this ramble, is that as I was reading this chapter I really didn't feel like I had any gigantic insights or ah!ha! moments. What I did find myself doing as I read this chapter was that I was thinking more and more about all those tests we have to give and what kind of tests they are. Tests kept flashing in my head and I kept thinking about the construct and what type of tests they were.

Another thing that I kept thinking of as I read this chapter is that it has taken quite awhile for tests to change and for people to embrace these changes. I mean, it is 2008 and I'm still required to give students discrete point tests to see if they know what they need to know in terms of English grammar. The horribly sad and frustrating part is that I don't think (even about earning all those credit hours studying English) I could even pass those tests...yet I'm required to give them and have a second language learner pass them. Sigh. I cringe when I see those tests.