Zach Ronneberg - KSTF Portfolio 2007-2008

INTASC standard 8 – The teacher understands and uses formal and informal assessment strategies to evaluate and ensure the continuous intellectual, social, and physical development of the learner
Performances #1 – The teacher appropriately uses a variety of formal and informal assessment techniques to enhance her or his knowledge of learners, evaluate students’ progress and performance, and modify teaching and learning strategies.

Although my portfolio las year focused around Project Based Learning (PBL) when I stared this school year I didn’t feel like I was done learning about the process of planning, executing, and evaluating a PBL unit. I felt as though I had focused exclusively on the planning part, and spent vary little time trying to figure out exactly what it was students were learning from PBL situations. I felt like I still had a long way to go till my PBL units became fully formed teaching tools, rather than simply cool applications. I figured that the first step toward honing a PBL unit as a teaching tool was to try to get a handle on what students were actually learning during the process. To that end, this portfolio will focus on the assessment of what students are actually learning during project based learning. Evidence will show that I am trying to understand what students are getting out the sound and wave project and then hypothesize on how I might change the project to better serve my students next year. In other words, this portfolio is a kind of formative assessment of a single PBL unit on sound and waves. My aim is to figure out what students are, or are not, taking away from the unit. This information can guide my thinking on how to change the format, content, presentation, or evaluation of the unit in order to better meet the needs of my students.

Before jumping in to the evidence, I thought it might be useful to give an overview of the lesson sequence I chose to focus on. With my students I call it the ANTSR P-BLOp (America's Next Top Science Rockers Project-Based Learning Opportunity), but will just call it ANTSR or "the unit" for short. During the unit students are asked to fill out a background survey, then put into groups (bands) with a mix of skills. With the help of their band, and the rest of the class, students learn about the properties of sound and waves through designing, building, tuning, and playing an instrument. Students can choose to make any instrument they like as long as it meets some specifications about the types of sounds it can make (like it must be able to reliably play at least 5 notes on the even tempered scale). The unit takes approximately 5 weeks from start to finish, with some class time dedicated to discussion, lab work, design, building, problem solving, testing, rehearsing, and performing. In addition to building an instrument, students also are responsible for more conventional HW assignments (worksheets and problem sets). At the end of the unit, students must get together with their band and play a song for the class. This is a major part of what drives student motivation - peer pressure. They want to sound good in front of their peers, so their instrument has to function properly, and in order to function properly they have to understand what factors are making it do certain things (like be out of tune, or be really quiet, or make a buzzing sound).

Evidence

1- Class-Created Wiki

Link:
- http://ronnebergscience.wikispaces.com/ (each classes notes are accessible via the links on the left hand side)
Caption:
- During the sound unit I made students responsible generating much of the class discussion and for taking notes during these discussions. I created an online wiki for each class to use as a space to keep notes. One student would sit at the computer and take notes during the discussion and these notes would be projected onto the whiteboard for other students to see. This way there was an interactive editing process going on, and students could suggest additions changes or deletions during the discussion. By having the notes available online students they could refer back to it when they were at home or in the lab, and I could also keep a record indefinitely and refer back to it for analysis at a later date (like now). To be honest, the note taking process didn't happen every day, didn't happen as smoothly as I had envisioned, and actually kind of stopped happening once the building phase started. However, during the initial phases of learning the basics of wave behavior, there is a fairly good record of what students were discussing and what they felt important to write down. It was a good formative assessment tool for me, because student thoughts and understandings were made much more transparent when typed out in front of the class. This tool would help focus my lessons and brought up many good discussions that may have otherwise been left at the wayside. In the reflection portion of this portfolio I will discuss what this evidence shows in terms of student understanding during the early stages of the unit.

2- Unit Test Analysis

Links:
Caption:
- This piece of evidence includes an end of unit test and my analysis of it's results. I was unsure of how much of the actual analysis data to include, so I have linked it all here so it is clear what I am referring to. I chose to include this evidence because I thought that going over their unit test very closely and methodically would help me figure out what conceptual understandings (correct or otherwise) students were coming away from the unit with. It would also help me understand what types of problems students did or did not do well on. On 2a and 2b, green highlighting indicates high scoring questions (over 89%) and red or pink indicates low scoring (less than 70%). 2c is my analysis of 2a and 2b.

3- Post Unit Survey and Analysis

Links:
- 3a - Survey Results Summary
- 3b - Survey Graphs (data was filtered by student group and results of comprehension questions 9 and 10 were graphed to get a rough idea for the conceptual understanding of each student group)
Caption:
- In order to get a better idea of what kinds of things (both content related and more generally) students took away from the ANTSR unit, I decided to give them a survey which asked them about their experiences during the unit. The questions garnered information about student demographics, type of instrument built, reasoning behind decision, likes and dislikes in the unit, self-reported understanding, and a couple of content questions. My hope in administering the survey was that I might be able to get an idea for what the students themselves thought about the unit, and what they thought they took away from it. I also wanted to try to find some correlations within the data (like whether being a regular or honors level student had any effect on your comprehension level, enjoyment of certain tasks, or satisfaction with how your instrument turned out). I decided to wait until about 3 weeks after the end of the unit to administer the survey. I thought this would give me a better idea as to what actually stuck with the students for a bit longer than a few days.

Reflection

All together the 3 pieces of evidence above show that I made significant growth in finding out what students actually know, learn, and understand in the process of, and as a result of, a project based learning opportunity. One significant realization I had along the way: it is really really hard to figure out what kids know and don't know. Data is never really as clear as you would have liked, and in a classroom setting there are always many ways of interpreting a single piece of data. To make matters more confusing, data will often be conflicting, or worse, not really reveal anything useful about your subjects. Through studying the multiple pieces of data above with regards to student understanding, I believe I have come to some tenuous conclusions regarding the state of my students knowledge during the unit, at the end of the unit, and 3 weeks after finishing the unit. Here I will share those conclusions and reflect on the process of how I came to them.

The Wiki was a great tool to help students be more articulate, precise, and thoughtful during a discussion. Because their thoughts were being recorded it seemed to me as though they thought a bit more about their comments and we in fact more apt to make comments. This last bit seems kind of counter-intuitive to me, because it seems like students would be more reluctant to say something if it were going to be written down, but this didn't seem to be the case. It also allowed students to take up the role of pushing other students to clarify their statements (though I still often had to take on this role). So one way it helped me figure out what students were thinking, is just that they had to be more clear in their statements. It also allowed me the time (since I was not furiously scribbling on the board) to take in student comments and digest them a bit more before asking for clarification, extension, or refer the question or statement back the class in the form of a reflective toss. However, when looking back over the wiki I am struck by how short and non-illuminating the notes are. It is hard to tell whether their notes were representative of collective knowledge or just a single individuals understanding. It is sort of the same problem that you have during a class discussion, if one person says something indicating they understand, do the other 30 kids understand? I think there may be a better chance of the rest of the class understanding in this case because the information was had to be spoken, then interpreted by the person at the computer, then typed onto the whiteboard, then confirmed by the speaker that it was what he or she meant and perhaps revised by others in the class. The information is passed around the class more than with a simple 1-studnet to 1-teacher interaction. However, this still gives us no information about how many of the students understand this information. A classroom clicker type of situation would be wonderful for formative assessment. I will have to look into that.

Lets, for a second, just imagine that what's written on Wiki does actually represent what your average student is thinking. What information can I glean about the thoughts or understanding of that student by what is written on the Wiki? Here is a sample of the notes written by 7th period:

-as wavelength increases frequency decreases: frequency =velocity/wavelength
-as tension increases, so does frequency, and also velocity
-we don’t need a constant as long as we use correct units; wave/second
-flames show sound waves by altering air pressure in tube, forcing gas out in different amounts (Ruben's tube)
-for a given length, there’s certain frequencies that match the boundary conditions
-higher harmonics =increase in frequency

The first thing I noticed is that the notes the students take is that they are very short and choppy, without much contextual information (about the situations where these apply). They also focus on basic principles and relationships. It looks as though the students understand the frequency wavelength inverse relationship, but do they understand it in the context of an equation? do they have a picture in their head of what that actually looks like? The second bullet shows they might understand that an increase in tension increases the frequency. But do they understand that we are only really talking about standing waves on a string or spring? and it looks like they added "and also velocity" and the end, but the fundamental relationship is that the increase in tension increases the velocity of the wave, which then in turn increases the frequency. On the unit test, when asked to explain how the tuning pegs change the sound we hear on a guitar, most students didn't write down that the velocity changes, just that the frequency does. So it seems that students don't really have a complete picture of what is happening. They know that changing the tension affects the frequency, but don't really understand the mechanism. Perhaps this was because we didn't focus much on the molecular view of waves. Without understanding how the molecules interact to propagate a wave, it wouldn't make much sense why the wave travels faster when tension is increased.

While it is hard to go back and extract much information from the wiki about what students did or did not understand while they were taking notes, I will definitely not abandon it as a formative assessment tool. I think it's value lies in the act of creating it. By increasing the amount and clarity of student input, creating a class-created communal record of their discussion, and by freeing me up to put more thought into how to respond to their comments, the wiki allows me to get a much better sense of what students are thinking during the time we spent creating it, and have a better sense of where to take the class from there.

I completed the test analysis early this summer to try to get a sense of what students were and were not understanding at the end of the unit. I remember that when I was working on the analysis I felt like it was kind of futile, that it wouldn't really reveal much. When reading back over it now, I have mixed feelings. Although I still think it is hard to make any strong conclusions, I now see a few key ideas that I can take and hopefully use to improve my practice next year.

First, my overall impressions that students tended to do well on mathematical-conceptual and terminology/recall questions, but less well on conceptual-practical questions leads me to believe that students are having a harder time translating the practical experience (rather than the mathematical problem solving experience) they gained from the ANTST project onto the test. This is completely understandable. Students have had years and years of training in solving math-based word problems. They excel at it, they love it, they want it! One girl actually begged and pleaded with me to "just give me a test! I hate this stupid instrument, I just want a test!!!". I think that this is trend throughout the year, in all project settings, not just during this unit. I think it would be helpful if I consciously focus on making sure to put similar types of questions on HW assignments, and give them equal time when going over HW (usually the mathy questions get much more coverage than the practical/conceptual ones, simply because they often take a long time to solve and students want to know if they got the "right answer").

Next, in question #20, I think I came across something really important in terms of students using and understanding equations. It seems that students don't always understand that an equation represents represents an equality, not a causal relationship. There can be a causal relationship present, but this is not always the case. I wonder how my use of "if/then" statements ("if the frequency goes up then speed must also go up") affect the students perception of a causal relationship. I think it will be important to specifically address the idea of causality vs equality in class. Perhaps it would be enlightening to show the equation rearranged and discuss where the causality is also rearranged?

From question # 6, I see the importance of not just saying, but showing and having students experience as much as possible the difference between a longitudinal and transverse wave. And making very clear the idea that you can represent a longitudinal wave with a drawing of a transverse wave, but the actual wave is still longitudinal. I will definitely use the physlets (1, 2) to help show this.

From question #18, and from my thinking about the wiki notes on tension and wave speed above, it seems clear that wave-speed is a bit problematic for students. In particular, it seems that they can't predict well when a wave will move fast or slow, and why it may do to. To help with this idea I think that we must do a bit more surrounding the molecular nature of waves. Perhaps making a series of human models where molecules are loosely/tightly bound and close/far apart. This might help solidify to get a visual picture of what molecules are doing and why waves move fast or slow.

I felt the survey was a great tool to help me understand how students felt about the project and to get their perspective on what they thought they took away from it.

One of the things I found most interesting was to filter the data into various student groups and examine the results for interesting places where the groups differed. The two comparisons I made were: regular vs honors students, and students who chose to build their instrument because they thought it was interesting vs those who chose it because they thought it would be easy to build. Below are the observations I made while looking through the filtered data:

Though all of these trends interesting (whether they were expected or not) the most intriguing to me is that the regular kids liked building their instruments so much more than the honors kids. Someone suggested to me that this may be because regular students have more experience building things, which is possible, but doesn't seem likely to me. If so, I should have seen a overall difference in the quality of the instruments they produced, which wasn't the case. In fact, I would say that there was very little difference in the quality of the instruments produced between these groups. My intuition tells me that honors kids are probably less likely to enjoy trying something they don't already know they are good at. Thus the building of instruments probably felt uneasy or scary for a lot of them who wanted to be assured that they would do it right and get a good grade. Think of the girl mentioned earlier who pleaded for a test. On the other hand, the students in the regular class are likely to be less successful in school overall and thus not excel at traditional types of learning. I would imagine that this may lead them to be more enthusiastic and enjoy anything they see as being outside of traditional teaching and learning. This is just a hunch. But it lead me to think about how often our system caters to a student who learns in one singular manner, rather than offering a variety of options to cater to the diversity of learning styles present in all classrooms.

The other method of analysis I employed was to take the filtered data and see how each group of students performed on the last 2 questions - #9 tested their understanding of standing waves in a tube, and #10 tested their understanding of sounds produced by a guitar. I designed these questions to be based in a practical setting, but probe a wide array of understandings in terms of wave phenomena. The following observations are based on the graphs supplied in the evidence section (3b).

The wildest thing I noticed here was that the Interest group (who built mainly guitars - because they were interested in them) actually scored lower than the ETM group (who built mainly pan pipes and xylophones - because they were easy to build) on their understanding of how guitar bodies interact with the air around them to create a sound wave. This seems backwards on all accounts. All I can figure is that since the pan pipe folks were used to thinking about waves in air, that they had more ability to analyze what happens to the waves once they reach the air. This could also be accounted for by a statistical anomaly, because the number of students in the Interest group was not all that high.

While it was hard to draw any solid conclusions from the data I collected, the process of systematically going through it and finding places where student understanding could improve then coming up with ideas for next year was a very worthwhile process. I hope to continue using student data to inform my practice for future years. I think as I do this process more I will get a better sense of what kinds of data are most helpful for informing my practice and what kinds don't yield much helpful information.

Standard/Goal

Evidence

1- Class-Created Wiki

2- Unit Test Analysis

3- Post Unit Survey and Analysis

Reflection