With this in mind, I’ve compiled responses to the most oft-occurring questions and assumptions about SAGrader, as prompted by reader comments on the Chronicle article and Slashdot.com.

Outsourcing assessment
The Chronicle article presented SAGrader as one method of outsourcing assessment, but SAGrader is really more of an instructor-driven learning tool. SAGrader provides feedback and assessment to students based on an assignment-specific topic outline provided by the instructor. We do not rely on generic algorithms or statistics modeling. In this sense, the instructor has a great deal of direct control over how SAGrader scores each submission. Our instructors don’t see themselves as “outsourcing” their grading work; rather, they are using SAGrader to provide personal feedback to their students more efficiently 24/7.

Where’s the feedback?
A popular concern that readers expressed about automated grading revolved around the role of feedback in improving the student learning experience. This wasn’t covered in the Chronicle article, but it’s probably the most important benefit SAGrader provides. Along with an overall score, SAGrader provides students with detailed, actionable feedback about their submission. Students are encouraged to examine their feedback, make revisions to their answer and submit again.

Immediate feedback, paired with the opportunity to make assignment revisions, supports an iterative writing cycle that promotes learning. Many of our instructors allow unlimited student submissions, so that students can practice as much as they need to before the due date.  We have evidence that this process increases student comprehension and can be especially empowering for students with learning challenges.

Promoting student/instructor interactions
Another concern expressed by readers of the article is that using such an automated tool would widen the communication divide between instructors and students. The thought is that introducing another layer between students and instructors would limit interaction, and thus hinder an instructor’s ability to nurture the student/instructor relationship.

However, this is a misunderstanding of SAGrader’s intended use. SAGrader in its primary function is intended for large classes (100-1000+ students), where personal contact is already constrained by the sheer number of students and student work generated. The program is designed to give these instructors an alternative tool to combat the limits often imposed by classes of this size. Instead of being limited to multiple choice, true/false questions, and a participation grade, instructors can now test and reinforce more substantive knowledge in their students through the use of short answer and essay questions, thereby shifting student learning to higher levels of learning. As one of our instructors told us, SAGrader offers “the ability to have large classes do the same types of work as small classes.”

In addition, SAGrader includes a number of features that allow students and instructors to connect. Students can challenge any feedback they feel is unfair, giving instructors a chance to offer additional explanation or improve the grading rubric. The student performance reports allow instructors to find areas where students may be having trouble, or identify students who may be at risk of failing.

Lloyd, an SAGrader instructor, has this to say about his experience:

“I also appreciate being able to track which students are not doing well, or who are consistently late submitting assignments. It has given me the opportunity to contact those students, express concern about their performance, and in a few cases to eventually help them get back on track with their grades.”

Above all, SAGrader makes it possible to offer more writing assignments in class, giving instructors a better view into their students’ heads. Instructors tell us that allowing their students to write more has given them much more insight into what their students know and don’t know, and identify points where they may be struggling.

Can computers really do that?
As expected, the most common questions readers asked revolved around whether computer software could actually grade student responses in any sort of meaningful manner. It’s a question we at Idea Works answer every day. Advancements in artificial intelligence, natural language processing, and expert systems have come a long way, and today’s computers are capable of things that just ten years ago would have been considered in the realm of science fiction.  As Chronicle commenter becauseisaidso expressed:

“Why not prepare an article on the (obviously not the proprietary, secret) ways an algorithm can grade an essay test?  Clearly, those of us not in the AI biz can’t judge whether or not this makes sense unless we have some clue how it is done…”

Ok, here it goes: SAGrader account representatives work with course instructors to create SAGrader assignments by first developing a conceptual model of the knowledge the instructor wants the student to be able to convey. This information is then modeled in SAGrader and tested to ensure reliability and general cohesiveness of the knowledge domain. We ensure that the program accounts for the hundreds or even thousands of ways a student could express each concept.

When a student submits an answer, SAGrader takes the student response and compares it against the instructor’s conceptual model to identify which concepts in the response were expressed correctly and which were not. If the student fully expresses the concepts and accurately relates them to other ideas the instructor was looking for, they get the points. SAGrader then gives the student feedback on which concepts they got right and which ones need further revision.

So there you go; automated grading of student submissions with assistive feedback in less time than it takes to brush your teeth.  We’d love to hear what you have to say, so if you’re excited, apprehensive, or just plain curious about the technology, sound off in the comments below, or send me an email at Luis@ideaworks.com.

A link to the original Chronicle article can be found here.

I’m sure by now you’ve seen the commercials for Dos Equis beer espousing the adventures and wisdom of The Most Interesting Man in the World. No doubt you’ve been intrigued by his mysterious charm and almost Zen Master-like wisdom. Every few episodes or so we’re temporarily interrupted from our day-to-day preoccupations and invited to glimpse a new facet of his fascinating life.

Surely a person of such class, esteem, and experience would have some crucial nugget of information to share with the rest of us. Some choice words of wisdom to improve our day to day existence?

Such it is that I’ve compiled a list of 5 Things Every Teacher Can Learn From”The Most Interesting Man”™:

1. “People hang on his every word, even the prepositions.”
Learn to speak with authority. Students like to feel that the person instructing them has a good grasp of the subject matter. Filling your sentences with “um” and “uh” may feel natural in speech, but it sends the wrong message by suggesting to your students that you’re hesitant or questioning yourself as you speak.

2. “He can make orange juice, out of apples.”
It’s no surprise that the state of the economy has hit education hard. As school districts and universities face slimmer and slimmer budgets, it’s crucial for resourceful educators to consider educational technology outside of the classroom. From twitter accounts to class blogs, the options can be overwhelming.  Fortunately, we’ve got you covered with our free resources and list of low-cost textbook alternatives.

3. “He never says something tastes like chicken. Not even chicken.”
Learning how to tailor your message to your audience is crucial for being a successful instructor. Your students likely come from different cultural and even linguistic backgrounds. As a result they may not understand a concept the first time it’s explained. The best instructors use varying language and examples to better connect with their students.

4. “His reputation is expanding faster than the Universe.”
Whether or not you’re on the tenure track, it’s important to be aware of what your students and fellow colleagues think of you. Especially for a teacher starting out in a new campus, this information can provide some much needed feedback as to your performance as well as to how you’re adjusting to your new environment. It’s not just at your end-of-semester teacher evaluations that people form their opinions of you, your reputation is growing all the time.

5. Speaking on Life – “It’s never too early, to start beefing up your obituary.”
There’s a reason he’s the most Interesting man. Sure you’re an educator; maybe also a grad student, a parent, and/or a doting family member, but don’t forget to get out every once in a blue moon and spice up your life. Go rock climbing, join a local social group, take up SCUBA diving. In general, throw a little excitement into the mix every once in a while. It’ll do wonders for your physical and mental health as the semester progresses and the walls start to close in.

So there you have it, 5 important tips we can all appreciate from our mysterious, immaculately-bearded friend. Hopefully these tips will come in handy in helping you make sure your students “Stay thirsty” this semester.

Study after study shows that when students are given the opportunity (or in some cases forced) to revise their work, they will learn the material better.

This works in part because students continuously have to review their writing and course material. In turn, students end up pounding the information into their brain.

For example, in a recent Organizational Behavior course students were ask to write a comprehensive essay about one topic covered during the unit. The paper was to be reviewed once by a peer and then later by the instructor. The unit covered a variety of topics about organizational decision making. One student decided to write about groupthink and the repercussions it has in the work place. Her paper was of decent quality, but not stupendous. When it came to the test, however, she nailed every question that was even tangentially related to groupthink. The other material covered on the test she answered at about 75% accuracy.

As you can tell, the topic she chose to write on was the topic she remembered most when test time came. This works because students are going back and making sure their writing is succinct, coherent and generally of good quality and in turn the material sinks in.

This concept is pretty simple, but sometimes it’s easy to forget and abandon revisions or even writing all together in the classroom.

That being said allowing time for revisions and actually reviewing the revisions is one of the most time consuming processes in education. So the big question for us is how do we make sure we can use writing and revisions in the classroom without losing time for everything else?

How to Make Time for Revisions

The key to utilizing revisions is providing feedback. Certainly, each student is capable of reviewing their own work to help them revise, but many times it is helpful to get outside opinions. Here are a few ways to review student work:

• Teacher Review: This requires you to personally go through each student submission and give personal remarks. This type of review provides high quality feedback to students, but also requires a large time investment.
• Rubrics: Rubrics are a set of defined features that are present in a specific assignment. When using rubrics you are able to quickly analyze each student submission and give some quick generic feedback. This type of review provides lesser quality feedback to students; however, it does not require the same large time investment.
• Sampling: If you offer multiple writing assignments throughout the semester you might consider doing teacher reviews for only half the class while the other half utilizes peer reviews. Then when the next assignment rolls around switch who you provide reviews for. This saves some time by allowing you to only review half the assignments; however, students might find it unfair that other peers are receiving more personal attention for certain assignments.
• Peer Review: Peer reviews occur when you allow your students to assess each others writing. With peer reviews there is virtually no time commitment on your part, however, the feedback for students will be of varying quality and may not be accurate.
• Lastly, we would be at fault not to mention or own product, SAGrader. SAGrader will give students immediate feedback on their writing assignments allowing them to make instant revisions. Most of the time commitment for a professor is up front when building the assignments. SAGrader also provides standardized feedback that does not happen with peer reviews.

Obviously, each type of review here has its pros and cons. The best way to make use of review is typically through some combination of these five techniques. This can be accomplished by partnering a high time commitment review process with a lower time commitment review process. This way you can get the best of both worlds. High quality writing reviews with a lesser time commitment than if you were forced to hand review each student submission.

This type of process is very similar to what we accomplish by using SAGrader.

The most important thing to remember is to allow students to utilize these reviews and revise their work. When students are allowed to revise, learning happens.

Like this post? Leave a comment and add us to your RSS feed.

Sound familiar?

This problem is rampant across education at all levels including higher education. Students have a lot on their minds and zeroing in on a lecture is going to be tough. Breaking through the mind blocks students have requires effective communication.

To communicate effectively with students follow these three steps:

1. Teach content
2. Make your students care
3. Give them a way to remember it

Nailing step one is fairly straight forward. After all, teachers teach content – that’s your job. Steps two and three, however, are usually a little more tricky so let’s explore those a little more closely.

Make your students care

Teachers are used to giving lecture about material but a lot of times it’s tough to make student’s care. Student’s that care are going to be more interested in hearing what you have to say. But if a student doesn’t care you might as well not even be communicating with them.

So how do you make them care? This one is pretty tricky, but many times if you can make the material relevant or practical its easier for students to see the point. Here are a few more ideas to help students care:

• Have an answer to the question ever teacher has heard – “Why do I need to know this?” If you’ve taught it’s guaranteed that you’ve heard this question a million times. Instead of saying because so, come up with a real legitimate reason that you can spout off quickly whenever a student asks.
• Bring in a guest speaker – Guest speakers can usually bring in outside examples of how the class material is being used in real life. Knowing the material can have a real life impact will make your student’s care just a little more.
• Have a lab – If your class is well suited for it, try having a lab. Labs are fun interactive ways that are useful for demonstrating real life applications of course material.
• Know your students – Remember you know your students best. Figure out what will relate to them and make them care.

Give students a way to remember

So now your students care about what your saying but that still doesn’t make it easy to remember what your saying. Forgetfulness is pretty easy especially considering how busy students are now-a-days. Here are a few tricks to help students remember.

• Acronyms: There’s a reason your elementary teachers used these. I still remember from second grade that “My Very Education Mother Just Served Us Nine Pizzas.” (Though I suppose the Nine Pizzas has been changed to Nuggets or something to account for Pluto no longer being a planet).
• Repetition, repetition, repetition: I am not talking about mindlessly beating information into a students brain here, but instead the slow learning that takes place when a concept is allowed to soak in over time. If a student hears a lecture, reads about it in a book, does a lab over it and then takes a test, odds are the concepts will stick.
• Appeal to all parts of the brain: Students learn through by hearing, by seeing and by touching. If you can combine all three in your teaching material it will help students remember those big concepts.

There’s three steps to will get you started. If anyone has more tips for getting students to care or helping them remember please share in the comments. Our readers would love to hear them.

Like this post? Leave a comment and add us to your RSS feed.

Inertia is powerful.

It’s nearly impossible to drive the speed limit on the interstate when all the cars around you are going five miles over.

Inertia is the resistance to change from the status quo. Inertia is why it is hard drive the speed limit. Inertia is why it is hard to buck the trend.

I recently stumbled upon a Dvorak Simplified Keyboard. This keyboard was patented in the 1930′s and was designed to address the problems of inefficiency and fatigue which characterized the QWERTY keyboard layout. How many people do you see taking the time to relearn how to type in order to use the more efficient Dvorak Keyboard? Not many, including me.

While succumbing to inertia is fine for trivial issues such as driving a few mph over the speed limit or choosing what keyboard to use, it is costly for classroom instruction.

How to Buck the Trend

I have been in far too many classes where the lecture tends to go something like this:

1. Turn on the computer
2. Start up PowerPoint
3. Read the slide
4. Add a few thoughts
5. Click to next slide
6. Repeat steps 3-5 until the slide show ends
7. Repeat for next lecture

This kind of lecture happens far too often. From a students perspective these tend to be the most uninformative kinds of lectures. I for one would love to see some professors try and buck this trend.

Perhaps trying

• Lecturing without notes at all
• Role Play
• Use Props
• Bring in stuff that students can touch
• Have guests lecturers
• Teach somewhere besides the classroom

It took me roughly 2 minutes to come up with these ideas. So next time you have to give a lecture, think twice before booting up PowerPoint. Try something new. Try bucking the trend.

Like this post? Leave a comment and add us to your RSS feed.

The Ten Worst Teaching Mistakes

Like most children of the ’80s, many of my earliest memories of using a computer involve “The Oregon Trail“. Developed by a student teacher to supplement his history class, The Oregon Trail allowed students to get a glimpse of life in 19th Century America while also teaching them budgeting, decision-making skills, and how to hunt for buffalo (or waste your time shooting squirrels).

Today, instructors are using more complex video games to teach students about the scientific method, physics, math and literacy. It’s a strategic move by educators who hope to capitalize on the popularity of video games (97% of teens ages 12-17 play computer, web, portable, or console games) and keep students engaged in the learning process.

Educational Games

In the same vein as Oregon Trail or Where in the World is Carmen Sandiego? some gamemakers are designing video games with a specific educational purpose. MIT-based The Education Arcade’s newest game, Caduceus , is a puzzle game for tweens focused on logic, reason and creativity skills.

It’s also designed to promote compassion:

Caduceus is a key content element of a new pediatric medical research awareness…designed to help parents teach their children to care about others and believe they can make a difference.

Another gamemaker, Education Simulations, created “Real Lives 2007” to let students experience how people live in other countries. Based on real statistical data, the game randomly assigns the user an identity – such as a factory worker in Brazil or a policeman in Nigeria – and teaches them life skills while letting them explore the global community.

Of course, “educational” games are a double-edged sword; the same elements that make them explicitly educational discourage students from playing them. Few of these games will be played outside of the classroom.

In response, some instructors are reappropriating mainstream games for educational purposes.

Mainstream games

In a commentary for Wired, Clive Thompson tells us about the “eureka!” moment for Constance Steinkuehler – an assistant professor at University of Wisconsin-Madison – who discovered that many of the teenage boys playing online games such as Lineage or World of Warcraft were employing the scientific method to figure out how to beat game bosses.

A group of [the teenagers] were building Excel spreadsheets into which they’d dump all the information they’d gathered about how each boss behaved…Then they’d develop a mathematical model to explain how the boss worked — and to predict how to beat it…Some would offer up new data they’d collected, and suggest tweaks to the model.

That’s when it hit her: The kids were practicing science….They were using the scientific method. They’d think of a hypothesis — This boss is really susceptible to fire spells — and then collect evidence to see if the hypothesis was correct. If it wasn’t, they’d improve it until it accounted for the observed data.

Thompson goes on to point out that these same kids are “the same ones who are, more and more, tuning out of science in the classroom. ” The instructor’s job is to harness the high-level thinking skills young people use to beat a video game and apply it to real-world mysteries.

Other popular games like SimCity, Civilization IV: Warlords or Caesar III can allow students to experiment in simulated environments, while learning about history, economics and leadership.

Of course, with the bad press video games have gotten for being violent and socially isolating, schools are understandably skeptical about allowing games in the classroom. Certainly, some games shouldn’t ever be condoned by educators.

But many games are safe and provide educators with a tremendous opportunity to engage their students.