With this in mind, I’ve compiled responses to the most oft-occurring questions and assumptions about SAGrader, as prompted by reader comments on the Chronicle article and Slashdot.com.

Outsourcing assessment
The Chronicle article presented SAGrader as one method of outsourcing assessment, but SAGrader is really more of an instructor-driven learning tool. SAGrader provides feedback and assessment to students based on an assignment-specific topic outline provided by the instructor. We do not rely on generic algorithms or statistics modeling. In this sense, the instructor has a great deal of direct control over how SAGrader scores each submission. Our instructors don’t see themselves as “outsourcing” their grading work; rather, they are using SAGrader to provide personal feedback to their students more efficiently 24/7.

Where’s the feedback?
A popular concern that readers expressed about automated grading revolved around the role of feedback in improving the student learning experience. This wasn’t covered in the Chronicle article, but it’s probably the most important benefit SAGrader provides. Along with an overall score, SAGrader provides students with detailed, actionable feedback about their submission. Students are encouraged to examine their feedback, make revisions to their answer and submit again.

Immediate feedback, paired with the opportunity to make assignment revisions, supports an iterative writing cycle that promotes learning. Many of our instructors allow unlimited student submissions, so that students can practice as much as they need to before the due date.  We have evidence that this process increases student comprehension and can be especially empowering for students with learning challenges.

Promoting student/instructor interactions
Another concern expressed by readers of the article is that using such an automated tool would widen the communication divide between instructors and students. The thought is that introducing another layer between students and instructors would limit interaction, and thus hinder an instructor’s ability to nurture the student/instructor relationship.

However, this is a misunderstanding of SAGrader’s intended use. SAGrader in its primary function is intended for large classes (100-1000+ students), where personal contact is already constrained by the sheer number of students and student work generated. The program is designed to give these instructors an alternative tool to combat the limits often imposed by classes of this size. Instead of being limited to multiple choice, true/false questions, and a participation grade, instructors can now test and reinforce more substantive knowledge in their students through the use of short answer and essay questions, thereby shifting student learning to higher levels of learning. As one of our instructors told us, SAGrader offers “the ability to have large classes do the same types of work as small classes.”

In addition, SAGrader includes a number of features that allow students and instructors to connect. Students can challenge any feedback they feel is unfair, giving instructors a chance to offer additional explanation or improve the grading rubric. The student performance reports allow instructors to find areas where students may be having trouble, or identify students who may be at risk of failing.

Lloyd, an SAGrader instructor, has this to say about his experience:

“I also appreciate being able to track which students are not doing well, or who are consistently late submitting assignments. It has given me the opportunity to contact those students, express concern about their performance, and in a few cases to eventually help them get back on track with their grades.”

Above all, SAGrader makes it possible to offer more writing assignments in class, giving instructors a better view into their students’ heads. Instructors tell us that allowing their students to write more has given them much more insight into what their students know and don’t know, and identify points where they may be struggling.

Can computers really do that?
As expected, the most common questions readers asked revolved around whether computer software could actually grade student responses in any sort of meaningful manner. It’s a question we at Idea Works answer every day. Advancements in artificial intelligence, natural language processing, and expert systems have come a long way, and today’s computers are capable of things that just ten years ago would have been considered in the realm of science fiction.  As Chronicle commenter becauseisaidso expressed:

“Why not prepare an article on the (obviously not the proprietary, secret) ways an algorithm can grade an essay test?  Clearly, those of us not in the AI biz can’t judge whether or not this makes sense unless we have some clue how it is done…”

Ok, here it goes: SAGrader account representatives work with course instructors to create SAGrader assignments by first developing a conceptual model of the knowledge the instructor wants the student to be able to convey. This information is then modeled in SAGrader and tested to ensure reliability and general cohesiveness of the knowledge domain. We ensure that the program accounts for the hundreds or even thousands of ways a student could express each concept.

When a student submits an answer, SAGrader takes the student response and compares it against the instructor’s conceptual model to identify which concepts in the response were expressed correctly and which were not. If the student fully expresses the concepts and accurately relates them to other ideas the instructor was looking for, they get the points. SAGrader then gives the student feedback on which concepts they got right and which ones need further revision.

So there you go; automated grading of student submissions with assistive feedback in less time than it takes to brush your teeth.  We’d love to hear what you have to say, so if you’re excited, apprehensive, or just plain curious about the technology, sound off in the comments below, or send me an email at Luis@ideaworks.com.

A link to the original Chronicle article can be found here.

Why? For two reasons:

1) Our own research has shown that it’s a great way for students to learn the material — often resulting in a score one or two letter grades higher than they would have achieved without the opportunity.

2) It makes it much easier for the instructor to identify concepts their students are having trouble grasping, since they can track a student’s progress throughout the learning process. This not only helps in identifying students who may be struggling, but can also help instructors target and improve parts of their own instruction that may need further clarification.

We also embrace iterative learning in our own product design. We’re always in a continuous loop of testing, design, and validation; then retesting, redesign, more validation, and so forth. Why? Because it allows us to stay closer to our users! By directly asking our instructors and students for feedback at multiple points throughout the process, we make sure we’re building solutions that our customers really want and appreciate. It also helps us be more efficient by eliminating time and energy wasted on features no one really cares about.

So it’s a win-win for everyone: the instructors get what they want, students get the information they need in a manner that appeals to them, and we save development time and costs by not building unneeded features.

If you’d like to know more about how SAGrader can bring you closer to understanding your students’ learning experience contact Luis at Luis@ideaworks.com.

While the issue is multifaceted, one approach that Mark proposes involves leveraging classroom technology to help even the odds. Technology, when used effectively, can allow the instructor to engage with and assess a much larger class than would normally be possible. It can also promote a student-centered classroom by encouraging discussion and participation between students, instead of “bottlenecking” the learning process by requiring the instructor to be the conduit through which information is exchanged.

When we designed SAGrader, we originally did so with large classes in mind. We knew there had to be a way to provide quality writing assignments to classes of 100, 300, even 1,000 students without significantly increasing instructor workload. We also felt that, although other forms of learning have their merits, writing is one of the best ways for students to learn and retain information (see the recent article on WIRED.com about writing as a learning strategy).

Numerous schools and thousands of satisfied students later, we believe we’ve developed a great program to help teachers expand the capabilities of their classroom. If you’re looking for an effective way to leverage technology in your over-sized classroom, we invite you to consider SAGrader. Send us an email (support@ideaworks.com) or give us a call (574.445.4554) and we’d be happy to give you a guided tour and answer any questions you may have.

Our Context engine
As part of our grading engine, SAGrader examines the context in which students use terms and concepts to determine whether the student understands the concept correctly. For example SAGrader can tell the difference between the two “Fords” in the following sentences.

“Ford did a great job when taking over for Nixon as President.”
“By mass-producing cars, Ford really helped the American people.”

SAGrader has always used this context engine, but we’ve recently made some improvements that make it able to handle more subtle situations that arise in the English language. These improvements increase grading accuracy overall.

Metrics in Short Answer Assignments
We’ve turned on the ability to add metric scoring and feedback (things like word count, spelling and grammar) to short answer assignments. Previously these features were only available on essay assignments. This flexibility provides even more constructive feedback to our students, and allows instructors to better structure assignments the way they want them.

Feedback
Stay tuned for upcoming changes to our feedback as we work to improve its effectiveness.

Picture courtesy ralphbijker via Flickr.com

As textbook costs rise, major publisher take a new approach toward an electronic format. But will students and administrators go for it?

Recently, a lot of buzz has been generated about the future of textbooks. While some see the traditional textbook as an irreplaceable part of Higher Education, a growing number of people are calling for a reworking of how students receive their course materials. The focus has turned primarily to the growing role of electronic books, and their impact on education.

Due to this, many publishers have come to realize that the traditional textbook model is quickly becoming obsolete. But textbook publishers have adapted, and in the face of an industry-rattling shift from paper to digital formats, many publishers are coming up with ways to both differentiate themselves and capture a share of a growing e-book market.

As part of this push towards the e-book format, major publishers have taken some interesting initiatives. McGraw-Hill recently launched Create, a service that allows instructors to mix and match selections from various textbooks, articles, and even upload their own sources to create their own custom textbook. Students then have the option of downloading an electronic version or, for a higher fee, ordering a hard copy.

Macmillan Publishers has also stepped into the customizable e-book arena. They’ve launched Dynamicbooks, with the promise that it will be the “next generation of interactive textbooks”. The service allows instructors to tailor one of the existing titles for their course. Instructors are invited to rearrange, alter or delete chapters, write their own content, include study questions, even embed video and audio into the coursepack. Like McGraw-Hill, students can opt to purchase the e-format version, or pay a little more for a printed version.

While some of the most commonly cited reasons for supporting the shift to a digital format are tangible, it costs far less both monetarily and ecologically to produce and distribute digital work; many of the potential benefits of this system are yet to be seen. Will students make the switch from paper to digital course materials any time soon? Will professors make the time investment required to carefully tailor and review their course materials every semester or two? There are also hardware considerations; a switch to digital format would require every student to purchase an e-reader of some kind (even so, the cost savings over paper books would likely defray this expense).

At SAGrader we’re big fans of technology and its potential –when properly applied– to enhance the learning process. However, technology is only as good as the methods and context within which it is used. A move towards digital textbooks would require added training for instructors not only on how the technology works, but also on how to best use it effectively. As an evolving trend, the ways in which these technologies can be used to enhance learning are still being developed. It will take a combination of technology and proper teaching methods to unlock the power and capabilities that the digital format provides.

Of course at the time the Nokia devices were used more to play “snake” in their tiny night-vision-green screens than anything else. There weren’t many other people with cell phones you could call (plus all your friends were with you in class), and the texting phenomenon was still years away.

My mother’s advice to alleviate our plight: here’s 35 cents, use a payphone.

Nowadays cell phones and other forms of mobile technology are ubiquitous. It’s not uncommon to see elementary and even kindergarten-age children sporting their own cell phones, digital readers, and iPads. Wireless networks in schools are pervasive, if not robust, and communication is taking place on these platforms at unprecedented levels. The result is a whole generation whose lives encompass both a flesh-and-blood existence and an accompanying virtual identity.

Students are glued to their devices, and schools have taken notice. While the first reaction from school administrators and educators is often to do away with allowing use of the devices altogether, some schools have taken a different approach. Just recently Dysart Unified School District in Phoenix, AZ began a program to allow students to use their phones in class. Students can use the phones to conduct research, participate in quizzes, and give the teacher instant feedback.

And why not? If part of the responsibility of academia is to equip students with the tools and skills required to compete in a global and increasingly technology-dependent world, shouldn’t students be allowed, nay encouraged, to adopt and learn the latest tools involved in shaping the global economy?

Isn’t it the educator’s job to learn about and adapt to the latest technology affecting their students’ futures? And while it’s true that it’s the lesson that’s much more important than the delivery, shouldn’t we be teaching today’s youth with today’s tools? After all, we no longer use the abacus, the slide rule, or the overhead projector.

What do you think? Feel free to leave comments in the field below.

Over at the ed tech blog Free Technology for Teachers, they’ve compiled a list of 15 tech tools to help students organize demands on their time across classes, organizations, fraternities and the million other activities college students find time for.

Photo by Orcmid

Among the aps are the always popular Remember the Milk and Google Calendar aps, but they share a few sites you may never have heard of.

Nexty is a personal planning tool that falls somewhere a basic to-do list creator and a full-fledged project management tool. Nexty offers users the ability to create lists, prioritize lists, and set reminders. Nexty users can create project folders to which they can add to-do lists for completing each project. The settings in Nexty are intuitive and easy to adjust. Getting started with Nexty takes only seconds as you do not have to enter an email address to create an account.

For more ideas on keeping students productive in the new year, check out the full article. Know of other tricks to help students stay organized? Let’s hear about it in the comments.

15 Tools to Help Students Keep Organized [Free Technology For Teachers]

Inertia is powerful.

It’s nearly impossible to drive the speed limit on the interstate when all the cars around you are going five miles over.

Inertia is the resistance to change from the status quo. Inertia is why it is hard drive the speed limit. Inertia is why it is hard to buck the trend.

I recently stumbled upon a Dvorak Simplified Keyboard. This keyboard was patented in the 1930′s and was designed to address the problems of inefficiency and fatigue which characterized the QWERTY keyboard layout. How many people do you see taking the time to relearn how to type in order to use the more efficient Dvorak Keyboard? Not many, including me.

While succumbing to inertia is fine for trivial issues such as driving a few mph over the speed limit or choosing what keyboard to use, it is costly for classroom instruction.

How to Buck the Trend

I have been in far too many classes where the lecture tends to go something like this:

1. Turn on the computer
2. Start up PowerPoint
3. Read the slide
4. Add a few thoughts
5. Click to next slide
6. Repeat steps 3-5 until the slide show ends
7. Repeat for next lecture

This kind of lecture happens far too often. From a students perspective these tend to be the most uninformative kinds of lectures. I for one would love to see some professors try and buck this trend.

Perhaps trying

• Lecturing without notes at all
• Role Play
• Use Props
• Bring in stuff that students can touch
• Have guests lecturers
• Teach somewhere besides the classroom

It took me roughly 2 minutes to come up with these ideas. So next time you have to give a lecture, think twice before booting up PowerPoint. Try something new. Try bucking the trend.

Like this post? Leave a comment and add us to your RSS feed.

Like many high school students, Jenna was forced to hand write assignments for her college preparatory English course.

Instead of embracing the power of word processors, her instructor insisted that every draft be hand written to avoid plagiarism. Her instructor’s misunderstanding of technology led to misguided rules in an effort to decrease cheating in the classroom.

In Jenna’s case, plagiarism didn’t decrease, but her engagement certainly did.

Some instructors, however, see technology as a way to engage students in the classroom. A recent U.S. News & World Report article describes how several educators are going high-tech to engage students and are finding ways to use technology to enhance their classrooms.

The rise of information availability on the internet is a large part of the struggle that collegiate professors are facing. What should educators do about wireless and near 100% mobile device penetration in the classroom? Glenn Platt, professor of interactive media studies at Miami University in Oxford, Ohio has this to say:

Professors are not so much people who stand and spout facts with students taking notes. The Internet has all of the information. And students aren’t going to come to class for a lecture if it’s on a podcast. So that means many instructors are trying to make the classroom more interactive.

The question is how do we achieve the ever elusive interactive classroom? What technology is helping? Blogs, wikis, SMART Board’s and Twitter have been used with success, but other technologies, like cell phones and other mobile devices, have been harder to use for classroom benefit.

We are firmly in the camp that believes technology can be a powerful learning tool for the classroom. We have seen the powerful benefits of educational software like SAGrader, but we do realize that technology can be a distraction if used improperly.

For instance, PowerPoint is a great tool that allows instructors to prepare and distribute lectures, however, far too often instructors use PowerPoint as a crutch and in turn improperly communicate information to students.

So remember, technology is a great way to enhance your curriculum, but it cannot replace your curriculum. Make sure that you use it in a way that assists your student’s learning rather than detracts from it.

Like this article? Find something interesting? Let us know in the comments!

Like most children of the ’80s, many of my earliest memories of using a computer involve “The Oregon Trail“. Developed by a student teacher to supplement his history class, The Oregon Trail allowed students to get a glimpse of life in 19th Century America while also teaching them budgeting, decision-making skills, and how to hunt for buffalo (or waste your time shooting squirrels).

Today, instructors are using more complex video games to teach students about the scientific method, physics, math and literacy. It’s a strategic move by educators who hope to capitalize on the popularity of video games (97% of teens ages 12-17 play computer, web, portable, or console games) and keep students engaged in the learning process.

Educational Games

In the same vein as Oregon Trail or Where in the World is Carmen Sandiego? some gamemakers are designing video games with a specific educational purpose. MIT-based The Education Arcade’s newest game, Caduceus , is a puzzle game for tweens focused on logic, reason and creativity skills.

It’s also designed to promote compassion:

Caduceus is a key content element of a new pediatric medical research awareness…designed to help parents teach their children to care about others and believe they can make a difference.

Another gamemaker, Education Simulations, created “Real Lives 2007” to let students experience how people live in other countries. Based on real statistical data, the game randomly assigns the user an identity – such as a factory worker in Brazil or a policeman in Nigeria – and teaches them life skills while letting them explore the global community.

Of course, “educational” games are a double-edged sword; the same elements that make them explicitly educational discourage students from playing them. Few of these games will be played outside of the classroom.

In response, some instructors are reappropriating mainstream games for educational purposes.

Mainstream games

In a commentary for Wired, Clive Thompson tells us about the “eureka!” moment for Constance Steinkuehler – an assistant professor at University of Wisconsin-Madison – who discovered that many of the teenage boys playing online games such as Lineage or World of Warcraft were employing the scientific method to figure out how to beat game bosses.

A group of [the teenagers] were building Excel spreadsheets into which they’d dump all the information they’d gathered about how each boss behaved…Then they’d develop a mathematical model to explain how the boss worked — and to predict how to beat it…Some would offer up new data they’d collected, and suggest tweaks to the model.

That’s when it hit her: The kids were practicing science….They were using the scientific method. They’d think of a hypothesis — This boss is really susceptible to fire spells — and then collect evidence to see if the hypothesis was correct. If it wasn’t, they’d improve it until it accounted for the observed data.

Thompson goes on to point out that these same kids are “the same ones who are, more and more, tuning out of science in the classroom. ” The instructor’s job is to harness the high-level thinking skills young people use to beat a video game and apply it to real-world mysteries.

Other popular games like SimCity, Civilization IV: Warlords or Caesar III can allow students to experiment in simulated environments, while learning about history, economics and leadership.

Of course, with the bad press video games have gotten for being violent and socially isolating, schools are understandably skeptical about allowing games in the classroom. Certainly, some games shouldn’t ever be condoned by educators.

But many games are safe and provide educators with a tremendous opportunity to engage their students.