I've been an athletic coach every year of my teaching career. And hands down, one of my favorite features of coaching culture is the willingness of so many of us to share best practices. It's kind of wild when you think about it: a coach will absolutely dominate a league using a particular strategy, then—as soon as the season is over—invite opponents to a clinic where they explain every detail of said strategy. Not only that, but if you walk up to most coaches with a dinner napkin, a pen, and a good question, they'll likely spend 20 minutes drawing up the plays or drills you need. Even sports' most famous coaching curmudgeons have stories of lowly assistants invited into film rooms to talk shop.
Athletic coaches share their most precious strategies because they recognize a collective responsibility to push their sports forward—even if this generosity erodes a personal advantage. It is often the same for teachers. We recognize that when the teacher down the hall is great for kids, it's easier for us to be great for kids. It is in this spirit that I dedicate another EL column to ChatGPT.
For many of us teachers, grading assignments last year got very weird. We read something that felt a little out-of-line with a kid's voice or skill level, and we wondered if they'd pasted in the results of an AI prompt. Sometimes we asked them about it, and they'd emphatically deny it. We'd then paste the questionable language into online AI detectors, which sometimes worked, and sometimes led to hilariously inaccurate results. By May, many of us had decided to avoid the confrontation altogether until someone showed us a way to truly know what a student had (and had not) written.
I refused to turn a blind eye—for reasons that are both pedagogical and admittedly selfish. In my classes, writing is an important way for students to show me—or each other—how they are thinking. ChatGPT isn't a threat because it fixes spelling and grammar or tells students how to be more concise or expressive. It's a threat because, when used thoughtlessly, it replaces the creation of ideas. If writing cannot be reasonably assumed to represent a student's thinking, it loses most of its scholarly value. And the selfish reason? Grading and offering feedback on student writing takes up so much time. There is no way that I'm about to dedicate that level of energy to something a robot wrote.
Accounting for Every Keystroke
I ended last year with a strong system for catching most AI writing. Many of my colleagues found the system useful as well, in addition to folks on Twitter. So, here I am, a coach jotting down a simple but effective strategy on a napkin. I hope you find it useful!
To start, all teachers should add the Draftback extension to their Google Chrome browser. (As of this writing, I am not aware of a similar extension on other browsers. But I am sure that will change quickly.) Draftback does a few helpful things for Google Docs. Most importantly, it turns a document's revision history—which we normally have to click through piecemeal—into a video that can be played at controllable speeds. When we watch it, we can actually see a student's every keystroke on a document. Draftback also provides a detailed breakdown of the document's history that includes the overall number of revisions, the number of "distinct writing sessions" (which Draftback defines as moments when there wasn't more than a 10-minute gap between revisions), the time and duration of each of those writing sessions, and which user made them! (The images below are from my writing of this article at this very moment.)
With this information a tab away, we teachers now have the tools needed to hold students accountable. First, we can check the overall number of revisions (or individual changes) a student has made. While this might vary by grade level and discipline, I've discovered that for all of my assignments (generally projects with word counts of 1,500–2,500), students should have thousands of revisions. Anything less is an immediate red flag. If something seems fishy, we simply play the video at a quick speed, looking for moments where chunks of text appear from nowhere. If they do, a conversation with the student is in order. The student has to show us where the magical text came from. Before the chat, it helps to paste the suspicious text into a few AI detectors (I don't swear by any of them, but it's important to look at more than one). I've had a few of these conversations, and it helps to know that I can just press play and say, "Help me understand this. What happened right … there?"
Think through what it would take for kids to "game" this system. They would have to copy the AI text word for word, occasionally deleting and changing words to make the rhythm look like authentic writing. Even if they opted to engage in this strange theater, the edit count would still be suspiciously low.
Cheating is made even more difficult if we have students submit the various stages of their writing, which is good practice anyway.
Cheating is made even more difficult if we have students submit the various stages of their writing, which is good practice anyway. (Unfortunately, this process is less helpful when a kid has a legitimate last-minute burst of ideas the night before a project is due and scraps everything at 1 a.m. Here it comes down to how well we know a kid's voice.)
A "Wild New Era"
For the foreseeable future, this system—or something like it—will be a seamless part of my classroom writing culture. My class syllabus now includes the following wording:
With the rise of ChatGPT and other AI writing tools, the teaching team recognizes that plagiarism looks different than it did a few years ago. For this reason, all students are accountable for giving the teaching team edit access to the Google Doc for every major project. We advise that all parts of every draft should be written on the same Google Doc so that by looking at the revision history, the teaching team can easily determine if the student wrote it. (Students must account for any material that is pasted in from writing practices, freewrites, etc.)
Draftback is a cool tool that could be used for more than policing. I can imagine having students look at the drafting video themselves to reflect on their processes. It's been especially useful for me to see the hard work that emergent multilingual students put into their writing. But, until I find a better way to detect cheating, Draftback's main use will be to ensure that a kid's writing is that kid's writing. And I hope, by sharing this here, I can help us all continue to push our craft forward into this wild new era.
