About GradeBot
Students tend to have a love/hate relationship with GradeBot. They
love it because they can get immediate feedback. They know instantly
that they work was accepted. And they can submit work any time, any
day. They hate it because it is incredibly picky. You have to get
the spacing exactly right. You have to get the wording exactly right.
This is the "sad reality" of computer programming: things have to work
exactly right or in many cases they are useless. Even so, GradeBot
seems to go above and beyond, into the realm of picky. Sigh. Deal
with it.
GradeBot was invented in 1998 to solve a simple need: most new
programming students need practice. Dr. Don Colton was involved in
teaching introductory courses in computer programming and discovered
by surveying his students that roughly 80% of them had never
programmed before, and of the 20% who had, most had not done more than
a few simple lines in BASIC.
In fact, even talking about programming was a challenge akin to
describing the taste of salt to someone who has never tasted it. The
best approach seemed to be simple: taste it. Students taste
programming by writing programs.
Partial Credit
One common problem for programming instructors is grading broken
programs. (One cannot really expect every new student to write
programs that actually work.) The challenge for grading broken
programs is to decide just how close the program is to actually
working. How close is the student to having the program fixed? These
are difficult questions.
Missed Opportunities
Another difficulty in teaching programming is teaching the art of
debugging and program repair. Test questions can be contrived to find
out whether students can spot simple mistakes and fix them. But on
the whole, debugging is another difficult challenge.
Two Birds with One Stone
GradeBot was conceived as the perfect solution to both problems.
(Well, not perfect, but good at least.) Partial credit is not
awarded. Instead, students are required to complete their work. Much
of the learning occurs in the "last mile" and GradeBot facilitates
that learning. Much of that last-mile learning is debugging of one's
own program.
How is this done? GradeBot compiles and runs the student's program.
GradeBot knows how the program should work, and provides sample inputs
to the student. The student then must create the expected outputs.
If not, GradeBot reports the offending output to the student, who is
then able to make corrections and try again.
The try-again process is very simple for GradeBot, which routinely
grades dozens of program submissions per minute. The response time to
students is often on the order of one second. Try getting that kind
of response from a live instructor. No wonder live instructors tend
to accept only one submission of the program.
Three Birds
Actually, there is another bird that GradeBot handles: specifications.
All too often the specs for small real-world projects are ad hoc
descriptions of what is wanted. Formal specification languages are
not employed. Instead, the boss wants this or the client wants that.
And once you produce it, they cheerfully tell you what to fix. Are
they never satisfied? GradeBot plays into this by giving simple
descriptions of what is wanted, and then requires something very
specific. Spacing must be exact. Wording must be exact. Creativity
is not for the output, but for the methods used to construct it. And
special cases are not even mentioned until they come up in testing.
(Oh, by the way ...)
Encouragement
GradeBot encourages its students by giving them an updated projection
of their grade in the course at any time. After completing a lab,
GradeBot sends back a Congratulations message showing how far you have
come, and what lies ahead. It can be particularly satisfying to see
that Congratulations message after a hard-fought battle to get
GradeBot to accept that previous lab.
WebBot
To simplify the interactions with GradeBot for new students, a web
interface has been developed. We call it WebBot. Rather than working
directly with the operating system and the compilers and editors,
students write their first programs in a textbox on a web page. This
gets the students up and running lots faster, and defers the nuts and
bolts to a later time when the students will better appreciate the
power and flexibility of a more professional programming environment.
In the mean time, WebBot fills the gap for new students by getting
them into the swing of programming quickly so they can taste it. The
initial trials indicate it is meeting these objectives.
But will it prove to be better than the way things were before?
Perhaps time will tell.