Research Evaluation Checklist: How do I evaluate educational research?

As another school year begins, teachers around the nation will attend professional development where the newest educational research will be trusted upon them, which then will, in turn, be expected to read, evaluate, synthesize and implement the suggestions from the researchers. This is a lot of to ask of teachers who are already overworked and stressed out. In order to help teachers sort through the rash of information that’s thrown at them, it helps educators to have a reasonable grasp on evaluating research: is it worth our time and effort? Will this research really help my students learn? By using the suggestions below, teachers can quickly evaluate an article or books claims and decide whether or not they are advisable courses of action.
1) Who published the work? Who paid for the research? Consider the source—even the financial source-- of the work. Where is the research or author from? What credentials does the author/s have? Do they represent a special interest group or business? By evaluating the source, readers may determine the intent of the article. If the research is presented by a business, the research is most likely one sided to support the businesses interests. For example, emails suggesting research that certain “educational software” improves reading scores are often sent to educators responsible for software purchasing. Upon further look, the supporting research was conducted by the software manufacturers. This is a red flag; the research should be completed by unbiased researcher who has no financial ties to the results.
2) The research should be written clearly. Research in the social sciences should be written in an easy to understand format that allows the reader to grasp the content and purpose without needing a dictionary or a search engine. If the ideas are worthy, the research won’t have to prove his intelligence by using words and professional jargon that confuses the intended audience.
3) Does the work make sense? Can you easily identify holes in the research? Even though you may not be an official researcher (though it can easily be argued that all teachers are engaged in action research), it is easy to identify lapses in the research. If you have a question pertaining about the content, subjects being studies, methods or any other red flags that pop up, seriously look in to your concerns. There is a good chance that what doesn’t seem right to you could be an overlook or omission by the researcher. Many times these variables affect the research findings. Consider the article, “Bad Research: A Comparison of iPad, Kindle, and Book Reading Speeds” by John M Grohol PSYD. This article highlights the flaws and problems with what may seem like a plausible study. Overall, if it doesn’t make sense to you, it may not make sense at all.
By following the above advice, educators can easily and quickly determine if they are reading a study worthy of his/her time or not. Considering the author, the format and the overall methodology can offer clues to the researches legitimacy.

Buckhold Annotations

Click here to open Google Doc

Annotated Resources

Research Introduction and Background

As a teacher in a rural school, I have noticed that my students rarely have any concept of the world around them, especially when it comes to international issues. While most of my students rarely leave the county, much less the state, none of them have every left the country or have an idea of the global community that encompasses our world. While the Internet has offered them countless opportunities to expand their knowledge around the world, they don’t realize the possibilities this offers. Therefore, in the wake of the push for 21^st century skills, I often wonder, in an isolated, rural community, how does global awareness and collaboration increase students’ motivation and achievement in the English classroom?

My students struggle with understanding global points of views on issues. The biggest exposure they get to world issues is in World History and British Literature, both which teach valuable concepts, but none that are necessarily current and innovative in our modern society. So, for most of our students, this is as much exposure as they will ever get to global awareness, and in today’s global society, this isn’t enough. Our rural community has seen a complete economic devastation in the past 3 years with crop revenue dropping and most local factories closing. Now our students have to consider leaving the community, going to college, or joining the armed forces in order to support themselves. Without a complete understanding of global issues and community, my students will not have a clue about what waits on the other side of the cornfields or, more importantly, the vast and amazing opportunities that they can partake in. Now is the time to incorporate an awareness of global issues as well as global collaboration to entice my students to be aware of the rest of the world. However, I am skeptical of how this will impact their motivation for learning and their academic achievement. Will my rural students have an interest in global awareness? Will they embrace the possibilities of international collaboration? It’s a new concept to them; will it be enough to boost their achievement in English?

Teachers in any school setting, urban or rural, need to realize the importance of global awareness and collaboration in their students’ lives. The world is changing at a drastic pace, technology is evolving at an exponential rate, and jobs now have an international scope. In the next few decades, the growth in these areas is only going to continue to grow; therefore, we, as teachers, have a responsibility to prepare our students to adapt to these changes and benefit from them as much as possible. We also need to find away to ensure our students that is in a viable concept and maintain their interest in this area. Not only will teachers be interested in the incorporation of global awareness, but so will universities, future employers, and institutions around the world.

There is a definite need for globalization in the classroom. The Partnership for 21^st Century skills highlights global awareness as one of the elements necessary for students to thrive in the next few decades of the 21st century. Their directives for global awareness include the following:

SOCIAL AND CROSS-CULTURAL SKILLS

Interact Effectively with Others

• Know when it is appropriate to listen and when to speak

• Conduct themselves in a respectable, professional manner

Work Effectively in Diverse Teams

· Respect cultural differences and work effectively with people from a range of

social and cultural backgrounds

• Respond open-mindedly to different ideas and values

• Leverage social and cultural differences to create new ideas and increase both

innovation and quality of work (Partnership)

Also, as written in Wanted: Global Citizens, “the world needs young people who are culturally sophisticated and prepared to work in an international environment… Because of globalization--the ongoing process of intensifying economic, social, and cultural exchanges across the planet--young people the world over need more innovative thinking skills, cultural awareness, higher-order cognitive skills, and sophisticated communication and collaboration skills than ever before” (Suarez).While this research supports the importance of global skills and collaboration in curriculum, there is much less research available in relation to rural schools.

In Virginia, however, some attention has been given to creating a “global education”. According to Dawn Hutton of the Southwest Virginia Education and Training Network, “Students learn and retain information better if they are able to relate what they learn to real life. Without a personal connection or a social connection to someone from other parts of the world such as Asia, Europe, or Africa, it is hard for them to relate to the material and understand the impact it has on their life. This problem may result in a lower informational retention rate” (Hutton). A real world attention to curriculum will obviously make students retain information better.

Still, my question delves further into the subject: how will motivation and achievement be affected? Retention is a worthwhile goal, but students need to do much more than just remember content. They need to enjoy it, to make it their own, and use it to construct new knowledge. I believe that through the creating of global skills and collaborator experiences, students will be able to truly own their learning and achieve at much high levels than previously possible.

References
Hutton, Dawn. (2009) Global Education in Rural Schools throughout Southwest Virginia. Retrieved from the Connexions wiki at http://cnx.org/content/m23391/latest/.

Suárez-Orozco, M., Carolyn Sattin. (April 2007) Wanted: Global Citizens. Educational Leadership 64 (7), 58-62.

The Partnership for 21st Century Skills. (2009) P21 Famework Definitions. Retrieved from http://www.p21.org/documents/P21_Framework_Definitions.pdf .

Working?

Does this feeder work?

WOA Reflection

I struggled greatly with my WOA. Every idea I came up with seemed cliche, boring, or overwhelming. Trying desperately to think of something that is truly an unrecognized work of art, I drove past my favorite oak tree: one centered in the middle of a field all by itself. This is when I decided to focus on the lone tree as my WOA.
This was, however, far from the end. I had to narrow down the compelling aspects of the tree, both in relation to other trees and relation to my own experiences. Finally, after much support from my professor, I narrowed down what I was going to do.
Not being able to tread on the field that held my inspirational tree, I used an old pear tree down the road from my house, which worked better than I ever could have imagined. The day I went to take pictures, the blossoms were in full bloom, the sun was shining, and the grass was greener than ever. It made for beautiful pictures.
I didn't expect the paradoxical beauty of the tree. Part of it was strangled with massive vines while the other was teeming with flowers. That made the perfect bridge for my WOA.
I really wasn't impressed with my educational bridges until I read the responses to my video and watched it again. Now they jump out at me. The tree is a perfect metaphor for students, who we don't examine well enought to understand not only their dark side, but the beauty of them also, not matter how hidden. I wish all teachers would be able to understand that each students does have a past and a future. The difference between the tree and students is that we can find out the student's past and influence their future.

Fashion is a Verb!

To Fashion: (v) to imagine, create, and express; To choice, a fashion statement. To fashionfashion something, be it an idea, an artifact, or so forth, is to give meaning to one's existence (Wong)

While I found Wong & Henriken's article on fashion interesting, I wasn't really intrigued until I came upon the paragraphs that addressed style. As an English teacher, I specifcally teach style--as it pertains to writing-- to my students. This connection brought the idea of fashion into my mind.

While I keep reminding myself that fashion is a social construction, used to attract people as birds use feathers to attract mates, why can't the same apply to writing? Students develop a style that evolves over time as they themselves mature and learn new ideas. Over these years they hone and fashion a style of their own--well, most of them do. Those who do not see the value in writing lack this "fashion sense." This may be an interesting metaphor to use in my classroom. Their writing style is exactly how they are presenting themselves to an audience, and regardless of their intent, they are taken at face value due to superficial mistakes, such as conventions, mispellings, etc. The thoughts beneath are often masked by these problems. WRITING IS A SOCIAL CONSTRUCTION TO! It is a method of communication!

I think that I'll use this metaphor in my classroom on Monday to help relate the potential or detrement writing can have on a student's impression on others.

P.S. I cannot figure out how to get the highlighting off of my text! Sorry! :)

JAWS: Better Music Than Movie

As I was preparing the commentary on my music project, I had an epiphany: The music of Jaws tells a better story than the film itself does. So, I went on YouTube and I watched a clip once with sound and once without sound. Try it; you'll see. http://www.youtube.com/watch?v=ucMLFO6TsFM

Speilberg's adaption of Benchley's novel would be nothing without William's accompanying score. There are two plot lines running in the music; in fact, I could complete an entire plot chart on the theme song. There is an exposition, where the hook is established. Rising actions as the prey plot is introduced and elaborated as the predatory hook beats in the background, getting louder and louder. Finally, the battling tunes climax in a loud beat, but the only surviving notes are the two tell-tale notes of the hook: DAH-DAH, that calmly swim/fade away.

And the winner is....

It's very interesting how a song can truly have a plot, how notes alone can tell a story based on the emotions they create.

The Jaws theme song uses flat/sharp notes alternated with natural notes to create the suspenseful hook, which clearly is the basis to this song, as well as the underlying current of panic during the movie.

Compelling? Hell, yes! Almost any person in the world could identify this song just from the feeling it creates. That's powerful!

Music to my ears...

Of course, as an English teacher, I am automatically going to compare music and writing. The parallels are unending. While the "hook" in an essay is presented immediately to get your attention, this isn't always the case in music, but either way a feeling is immediately created. This hook can be threaded throughout the essay to create a feeling of flow and remembrance of the hook that got you interested in the first place. It's interesting to listen to a piece of music as though it is a piece of writing and see how similar notes would be like similar words, ideas. The flow of each, with intro, body, conclusion are symmetrical also.

Either way, the main objective is to get your reader involved, get your reader interested. Boring arrangements of notes are just like boring arrangements of words. While each word/note on it's own is just a word/note, when combined, there is the ability to created compelling thoughts. It's the arrangement of both that makes pieces compelling.

This comparision reverberated in my mind as I listened to the "What Makes Music Good?" shows. Every last thing that Composer Robert Kapilow said could have been translated into the critiquing of a writing teacher. The focus on how each note put together, even as simple as they may be, can create a compelling feeling, image in the reader.

Design for Contractors & Teachers--I'm on this one!

Alright...I went to town having NO idea what two stores I would choose for this project, not to mention that I was scared to death of getting kicked out for taking pictures. I figured inspiration would come to me at some point, and it did. During the remodeling of our house we spent much, much time in several stores: Menards, Lowe's and Home Depot. Since I hate Home Depot and refuse to set foot in that store, I decided I was left with two stores that I often frequented.

These stores were perfect for this project. I can't tell you how many times my husband and I debated the interior--and exterior--design of these stores. When price was not the deciding factor of which store to shop at, design was.

Our local Menards is a horribly designed store. We hated going there. They decorated in horrible colors--red, yellow, orange, green--and the whole store glowed an aged yellow. The signs are often handwritten and they sell generic junk on the end of their aisles. They shelves were disorganized and they sold weird, misplaced items like Bar-B-Que sauce in the heating and cooling department.

Then, there is Lowe's. When you walk into the store you are immediately welcomed by warm colors of blue and gray. The store feels like it was designed to invite you in and make your shopping experience the best it can be. Their is not clutter of junk attached to the end of each aisle, nor is there a yellow hue enveloping everything. It just LOOKS better. Also, all of the product is located inside the store, whereas Menards had a make-shift area outside where you had to dig through piles to find products.

All of this design reminds me of how I need to focus on design in my own classroom. While the interior design of my room is partially dictated by the administration, I strive to make my room as comfortable as possible. While it can be cluttered at times, and I will work on that, especially after this unit, I post interesting work and information along the walls. There is art work hanging on the walls that do not have cabinets. I try to take this vast, sterile space and turn it into a space where students can still concentrate and learn, but also work together and feel relaxed. Instead of the straight rows of desks/tables, I have arranged my tables in several squares, a large one around a smaller square, to encourage community, encourage discussion and working together. I feel this is much more comfortable and inviting than the typical face-forward desk arrangement. The students like it too, though they admit they can't get out of participation when we're grouped this way. There is NO way to avoid eye contact!

After this unit, I want to spend some time in my room just thinking about how to improve the interior design. I want students to feel compelled to learn in my room, not forced. Hopefully during our PD day on Monday I can take the time to assess my room and the effectiveness of it.

Compelling or Not Compelling--that is the question.

It’s obvious that we can be compelled in a variety of ways: photography, video, interior design architecture. But, the common variable in all of these mediums is one thing: emotion. No matter how one is compelled, the end result is always some sort of emotion, ranging from sadness to hatred to love. And, have we not all felt these emotions previous to this specific compelling experience? Isn’t the end result of configuring these compelling experiences always the same? How can it be unique if has been done/felt before? And along these same lines, has the same experience not already have been created? Are we not mimicking similar experiences that we’ve had? Our goal is to elicit these same feelings, just from a different method? In one of my creative writing classes, I had to read a book by Stephen King that reiterated the fact that everything has already been written. Every character, every storyline, and every emotion to be elicited—they have been written before. We may rearrange these elements and repackage them, but in the end, it has been done before. The guttural emotion that is the end result of our work has been done before…it has been felt before. I’m not sure how this translates into creating compelling experiences, but I am reminded of this fact constantly in this course. Obviously, whatever I am trying to create has been done before. In fact, when I do create something, it is based off something that has compelled me in the way I want my audience to be compelled. When we take pictures, we arrange them in a sense that we have seen before, where that arrangement worked and moved us. It’s the same with video and especially design…we work at it, but are always modeling the effective work of others. If I think about this too long, I see how this same principle applies in all areas of my life.

Also, along these same lines of insane questions, can we truly create a similar experience so that our audience can empathize, rather than sympathize with the predicament presented? Is not their emotion based on experiences they have had? If I show a picture of an animal, is not that person’s personal experiences—or lack there off—with that animal indicative of the emotion they will or will not experience? What is compelling to one person or group of people may not be as compelling to another. For example, I can show a movie of sweet, innocent deer grazing in a field and while one person may be experiencing a cut, cuddly feeling, the next person may very well be getting excited and twitching his trigger finger. How can we truly gage the effectiveness of a compelling experience?

Before I had a child, I was always moved by stories involving kids or a mother’s pictures, stories, videos, etc. it was cute…but now…I am compelled. In any and every area involving children (especially mine). I cry like a baby in movies where my emotions tie back to my son. I never would have cried like that before being a mother. The same can be applied to when I got married. My emotions have evolved to suit my life. However, if you are not in a similar situation, you will not feel the same, you will not be compelled along the lines that I am. So, how can we judge the compelling factor of a work without a broader range of audience?

One more—I have had several students tell me that the movie Johnny Darko changed their lives, was the best movie ever. I figured a movie that important had to be good right? WRONG! I felt that was the worst movie ever, a complete waste of my time. However, I still get students remarking on how moved they were by that movie. I DON”T GET IT! And I tried…I really, really tried. I find NOTHING compelling in that movie. But…thousands of people do. So, even though I was not moved by this movie, how can we say it is not compelling when many, many others will argue that it is?

Imaginative Bridges: Nature or Nuture?

"One thing I love about stories--in both movies and literature--is the
ability to align yourselves with the characters and identify with their
emotions. Some people (like my husband) can't/don't (I haven't figured
out which is correct) do this, and this makes me sad. They can't FEEL
the story."

Okay, I am quoting myself, but please, don't think I take myself too seriously. I'd just like to continue the conversation. And here's why:

I finally got my husband to take me to see Avatar, a world-renowned phenomenon that I wanted to experience. I could tell after the first 10 minutes, my husband HATED it. The following argument ensued:

Why! How could you hate this movie that was 1) a marvelous work of art and 2) a brilliant metaphor for a thousand lessons about love, survival, racism/oppression, greed, preserving natural resources, etc.

He HATES science fiction. Ok..I can understand, It's not for everyone. But can that really stop you from seeing beyond? from empathy? from aligning your emotions to the story and identifying with it?

In my case, no. Nothing stops me from identifying with a story in either film, literature, or a guy on the street--maybe this is a trait of an English teacher? I abstain from watching "heavy" films that I know will cause me to hurt for several hours or even days after watching them. For example, I refused to watch P.S. I Love You for 2 years after it's release. I know I would cry, and when I did watch it, I was heartbroken for about 2 days. When I watched that film, I could FEEL the pain. I missed MY husband, who was just at work. I cried more than any character in the movie. Yes, I am deeply affected by stories. Not just love stories. War Stories--I cried in the theater and felt a mixtures of sadness and terror in my heart for days after watching We Were Soldiers. Amistad--the injustice weighed down my heart for weeks.

I carry the emotion of whatever I watch (or read) much longer than just the duration of the movie (or book). My husband, however, cannot comprehend this attachment. He is the one who leans over as I'm crying during any movie or and says "It's just a movie." He looks at me like I'm crazy (not that it's ruled out) and rolls his eyes. I question this lack of connection often, more for an understanding of WHY he doesn't put himself in the shoes of the characters. I know that some of my students must feel this way. If only I could understand WHY he isn't affected like I am, maybe I could unlock the key to understanding why kids --and my husband--don't read anything for entertainment. If there is no connection, what investment would anyone put towards something?

My biggest question is if he DOESN'T feel empathy (or sympathy) or if he CAN'T. Are some of us just born more "sappy" than others? Why can I understand how the characters feel and find some symbolism that applies to my life in every movie I watch and book I read? And why can't he? Is he physically incapable? Or is there some magic switch deep down that can be flipped?

And how deep does this go? If this same trend follows the imaginative bridge into other areas, what else isn't my husband interested in that he should/could/would be if it wasn't for this disconnect?

I wonder, for the sake of my husband and all of my students, what CAN'T you feel? If I could anser this and find the key to unlocking that ability to identify with information, imagine the improvements I could make in education.

Film & Television

This has been an interesting unit. My minor is in communications and in previous years I have taught journalism and communication classes at the high school. I LOVED THESE CLASSES! They gave me the freedom to teach the same ideas in new ways. Everything had the ability to be more compelling, and not being tied down by benchmarks and testing requirements, we had FUN! I did a unit on televised journalism, focusing on the Edward Murrow era where television played a HUGE part in the perceptions of Americans during the Communist "scares". We watched Good Night and Good Luck and discussed how the importance of television and the impact Edward Murrow was allowed to have because of it. Had journalism been limited to print, he never would have been able to speak out against McCarthy.

I also taught a small movie unit that focused mostly on the technical aspects of film, such as angles of shooting, lighting, zooming, panning, use of color, etc. Again, I loved this unit. We read about various techniques used by directors and editors and then watched Big Fish (directed by the brilliant Tim Burton), which uses every technique we discussed. Students were amazed and loved to discuss the effects made by these techniques. I always used these techniques as a metaphor for writing techniques. It was a great way to show them the difference the techniques--in both film and writing--can make.

A third unit I still do in my English classes is the Super Bowl Commerical unit. We discuss the various propaganda techniques employed by producers and the effect it has, both on the commercial and the audience. Then, they have to apply these same propaganda techniques in their persuasive writing. These bridges are pretty concrete for my students during this unit. Once they see the techniques in action, it's easy for them to literally put them into words.


Our class focus this week reminds of the unit repeatedly. I never thought of these units as "compelling". I was just excited to be doing something new with my students. Now I've been reminded to using film and television to create imaginative bridges and not only enhance my students' learning, but also as a visual metaphor for elements in their writing.

Compelling Video

NDCE

I took floriculture courses in high school. We learned everything about flowers and the arrangment of them. From the science to the aesthetic, we studied it. However, despite everyone's equal education, some could effortlessly create beautiful arrangements of flowers, while others worked tirelessly and could never create a decent arrangement. Why? I always wonder. Yes, you can look at a poor arrangement and see the inadequacies, but to explain them often is impossible. It is a talent, just as drawing, painting, etc., and some are naturally better at it than others. The artist just KNOWS what to do. Why? I don't know.

This memory was awakened by the phrase describing NDCE, "Its nature is fleeting, evanescent, difficult to describe, or even beyond words. With too much effort or thought, the very qualities we seek to experience and understand may evaporate or become even more elusive." So much of what we do as teachers, workers, learners is sought to be characterized, classified, and boxed until the heart and soul has been beat out of it. We can learn, be taught, but the heart and soul of something has to come from within, and this is what makes something compelling. Our passion is what makes something worthwhile and interesting to others. However, this cannot be taught, cannot be replicated and dispensed in textbooks. Natural passion and ability has to play a huge part of being compelling. Right?

Photographic Bridges

Since photography is the medium I most comfortable with, I am ridden with imaginative bridges between teaching and photography. I imagine as I learn more about each other area I will also be filled with similar ideas, so I am excited about the future of this class. But, to begin, I will relish the ideas I have connecting the teacher with the photographer.

I am constantly composing shots. Everyday I have to decide the best angle/approach to come from. I know that it must be exiting, relevant, and innovative if I want to grab their attention. One of the biggest challenges I face is creating an approach that compels my learners. I am constantly working to find new angles to approach content from. How can I make this moment more interesting? I can compare my lesson plan compilation to a scrapbook. Are my photos/lesson plans interesting, captivating, compelling? Or are they boring, standard, emotionless shots of standard material?

However, on the road to composing my perfect shot, I will have to take numerous photos to get the right one. Rarely do lesson plans come out perfect on the first shot. There is always a way to get better. Even when I feel a lesson plan has reached it's potential, I seek ways to enhance it, just as a photographer may do with his photos.