{"id":53108,"date":"2026-10-30T13:12:06","date_gmt":"2026-10-30T12:12:06","guid":{"rendered":"https:\/\/impulseducacio.org\/?p=53108"},"modified":"2026-10-30T13:12:34","modified_gmt":"2026-10-30T12:12:34","slug":"ai-detectors-assessment-schools","status":"publish","type":"post","link":"https:\/\/impulseducacio.org\/en\/ai-detectors-assessment-schools\/","title":{"rendered":"Detectors do not work: assessing with an AI use scale"},"content":{"rendered":"

<\/div>
<\/div>
<\/div>

What the Machine Cannot Hand In for You<\/h1><\/h1><\/div>
<\/div><\/div><\/div>

Share<\/h4>
<\/i><\/a><\/span><\/i><\/a><\/span><\/i><\/a><\/span><\/i><\/a><\/span><\/div><\/div>
<\/div><\/div><\/div>
<\/div>
<\/div>
<\/div>
<\/path><\/svg><\/div>
<\/div><\/div><\/div>
<\/div><\/div><\/div><\/div><\/div>

30 October 2026<\/p>\n<\/div>

<\/div><\/div><\/div><\/div>

Detectors do not work: assessing with an AI use scale<\/h2>\n<\/div>
<\/div><\/div><\/div><\/div><\/div>
<\/span><\/a><\/span>
<\/a><\/span><\/div>
<\/div>
<\/i>Register now<\/span><\/a><\/div><\/div><\/div>

Judgement Cannot Be Delegated \u00b7 10 of 12<\/p>\n<\/div><\/div><\/div><\/div>

It is often said that the problem with artificial intelligence in assessment consists in working out who wrote the text. Jos\u00e9 Antonio Bowen and C. Edward Watson, authors of Teaching with AI, open their chapter on cheating and detection with a real case, with the names changed.<\/h5>\n<\/div>
<\/div><\/div><\/div><\/div>

A postgraduate student receives an alert: she may have violated her university\u2019s honesty policy. A professor had found one of her contributions to the course discussion board too insightful for someone who was not a major in the subject, and ran her three posts through two detectors. The first returned a 50%, a 35% and a 5% probability of having been machine-generated. The second returned 100% for all three. She knew the work was her own, and she had an idea: she looked up the professor\u2019s most recent article, single-authored and published ten months before ChatGPT existed, and ran it through the same tools. It came out at 36% in one and 100% AI content in the other (1).<\/p>\n<\/div>

<\/div>
\"AI<\/span><\/div><\/div><\/div><\/div>

The alarm that does not put out the fire<\/h3>\n

The staffroom conversation about detectors usually moves between two opposing certainties, and neither stands up in full. The data Bowen and Watson gather are uncomfortable in both directions. Turnitin announced a 1% false positive rate and revised it to 4% in autumn 2023. OpenAI\u2019s classifier was presented with a 98% accuracy rate, and its own research found that it detected real AI text only one time in four, while flagging as artificial almost one in ten human texts. It ended up being shut down (1).<\/p>\n

One study tested seven detectors with two different materials: award-winning essays by secondary school students and TOEFL exams written by Chinese students. It was accurate with the former and misclassified more than half of the latter. It was enough to ask the model to sound like a native speaker for the false positives to fall from 61% to 12% (1).<\/p>\n

That finding is the one that should keep us awake at night. Those who write with a more limited vocabulary, because they are learning the language, are more likely to be accused.<\/p>\n

But the same review reports a finding that runs in the opposite direction, and leaving it out would be cheating. Mike Perkins and his team found in 2023 that Turnitin correctly identified 91% of AI-generated content, while the teaching staff reviewing the same texts got it right in 54.5% of cases (1). The best detector performed considerably better than the human eye.<\/p>\n

Here the sources disagree. Jasper Roe, Leon Furze and Perkins himself state in 2026, without qualification, that attempts at text detection are inaccurate and unreliable (4). Bowen and Watson hold that accuracy varies enormously between products, that the best ones separate the two reasonably well and that the problem is not that they do not work. The disagreement makes sense once you look at what each is measuring: some are judging the technical reliability of the set of available tools, others are judging their practical usefulness when a teacher is supervising. And both parties reach the same conclusion. Detection cannot be the school\u2019s policy (1, 4).<\/p>\n

Bowen and Watson say it in a sentence worth keeping to hand: \u2018AI detectors do not accuse students of cheating\u2019 (1). They return a probability. They compare the tool to a smoke alarm, which is proof only that something should be investigated and not that there is a fire, and on that basis they take issue with the decision by Vanderbilt, Michigan State, Pittsburgh and DePaul to turn off Turnitin\u2019s AI detection (1). The comparison is useful, and it also has a limit: an alarm that often goes off where there is no fire stops serving to warn and starts serving some other purpose. In a school, that other purpose is distrust.<\/p>\n<\/div><\/div><\/div><\/div>

An alarm triggered by false positives generates systematic distrust within the educational community.<\/h6>\n<\/div><\/div><\/div>
\"detectors<\/span><\/div><\/div><\/div><\/div>

Assessing is the other side of teaching<\/h3>\n

Amaia Arroyo Sagasta, doctor in Communication and Education and lecturer at Mondragon Unibertsitatea, and Carlos Magro Mazo, coordinator of the educational programme of the Instituci\u00f3n Libre de Ense\u00f1anza, argue in the ODITE Report 2026 a thesis that reorders the whole problem: \u2018Assessment is the backbone of the educational model\u2019 (2). What we assess and how we assess it ends up becoming the real curriculum.<\/p>\n

Their chapter is a conceptual essay, not an empirical study, and it draws on well-established literature. They take up the three conceptions described by Lorna Earl in 2003: assessment of learning, assessment for learning and assessment as learning. In each of them, who handles the information and who decides changes. And they take up Neus Sanmart\u00ed\u2019s distinction between formative assessment, the kind teachers use to adjust their teaching, and \u2018formadora\u2019 assessment, the kind students themselves use to regulate their own process (2).<\/p>\n

The diagnosis they make of the present will be recognisable to any leadership team: we assess formatively in the rules and in our intentions, and summatively in practice and in what the administration demands (2).<\/p>\n

Onto that ground comes generative artificial intelligence. Arroyo and Magro propose looking at it as a mirror that reflects back the reality of our teaching and assessment practices. When a task consists only in reproducing information or in handing in an automatable product, assessment becomes fragile, and they are precise about why: not because AI cheats, but because the design was not conceived for this scenario (2). The answer, they write, does not lie in making assessment tamper-proof or in going back to the analogue past by banning tools, but in revising the curriculum, the instructions, the tasks and the products.<\/p>\n

There is a distinction in Bowen and Watson\u2019s book that helps decide which to revise and which to protect. David Krakauer separated complementary cognitive artifacts, like the abacus, which make us more capable even when we no longer have them in front of us, from competitive ones, like GPS, which when they disappear leave us worse off than we were. No source yet knows which of the two AI will turn out to be, and that is exactly why the friction we discussed in the article \u2018The Empty Gym\u2019<\/a><\/span> still has to appear at some point in the task (1).<\/p>\n<\/div><\/div><\/div><\/div>

\"detectors<\/span><\/div><\/div><\/div>
Artificial intelligence acts as a mirror that questions tasks based solely on automatable products.<\/h6>\n<\/div><\/div><\/div><\/div>

Raising the bar instead of pulling down the shutters<\/h3>\n

Faced with a machine that writes well, schools have two reflexes: ban or police. Both leave the bar where it was. There is a third, and it consists in raising it.<\/p>\n

Bowen and Watson put it bluntly. A second-year student at Harvard wanted to know whether ChatGPT would have passed her first year, handed her eight professors work generated by the machine and asked them to grade it. The average was 3.34 out of 4. It is not a study, it is a single student\u2019s experiment, but the pattern in the comments is revealing: they praised the style more than the content (1).<\/p>\n

From this comes their proposal: AI is the new bare pass, and the reasonable response is not to ban AI but to stop passing that level of work. They propose reworking the rubric by shifting a column, so that what used to be a C becomes a fail, and removing the points we still award for grammatical correctness or for a serviceable outline, two things the tools have been handing out for years (1).<\/p>\n

It sounds simple, doesn\u2019t it? Well, it is not so simple, because the same authors acknowledge the limit of all this: it is proving virtually impossible to design writing tasks that AI cannot do at least to pass level, and no task will be AI-proof (1). That is why transparency works better than surveillance.<\/p>\n

The most practical tool for that was published in 2024 by Mike Perkins, Leon Furze, Jasper Roe and Jason MacVaugh: a five-level scale that the teacher assigns to each task and communicates in advance (3). Level 1 excludes any use. Level 2 allows AI for generating ideas and structuring them, without any generated content appearing in what is handed in. Level 3 admits editing and improvement of the writing, with the original attached for comparison. At level 4 the machine completes one part and the student has to evaluate it, discuss it and cite it. Level 5 permits full use. The levels are cumulative, and the authors note that level 3 is a transitional point rather than a destination (3).<\/p>\n

Two warnings of theirs travel with the scale. The first is about equity: they recommend that level 1 tasks be carried out under supervision or with little weight in the mark, because outside the classroom those with more digital competence or paid tools can use them without it showing. The second is about methodological honesty: when they published the scale it had only been tried by a pilot cohort, and there was no research on its effectiveness (3).<\/p>\n<\/div><\/div><\/div><\/div>

A transparent five-level scale allows modulating the use of technology before evaluating.<\/h6>\n<\/div><\/div><\/div>
\"detectors<\/span><\/div><\/div><\/div><\/div>

Three decisions a program cannot take<\/h3>\n

In the classroom<\/strong>. Labelling each task with its AI use level before setting it, and asking for a brief declaration of use on submission. Bowen and Watson propose a process template that works well reduced to four steps: ask the AI to do the task, list its errors, improve the result with your own work and explain what value you added (1).<\/p>\n

For the leadership team<\/strong>. Taking and writing down two decisions. Whether the school uses detectors and for what exactly, knowing that they return probabilities and not proof. And what the procedure is when there is a suspicion, which should begin with a conversation with the student about their working process and not with a screenshot. Perkins and his team go further and advise against treating misuse of these tools as a breach of academic integrity, proposing that it be resolved within the rubric (3).<\/p>\n

For the teaching staff<\/strong>. A calibration session: the same piece of work assessed by everyone with the process rubric, and open discussion of the divergences (1).<\/p>\n<\/div><\/div><\/div><\/div>

Teaching faculties must agree on process rubrics that replace persecution with transparency.<\/h6>\n<\/div><\/div><\/div>
\"detectors<\/span><\/div><\/div><\/div><\/div>

To conclude<\/h3>\n

The systematic review by Guaranda-Arias and her team places diffuse authorship among the risks of the educational use of AI, together with technological dependency and uncritical acceptance, and warns that the studies reviewed are mostly cross-sectional or documentary and do not allow causality to be established (5). Diffuse authorship is not cleared up by a percentage on a screen. It is cleared up by asking for the process.<\/p>\n

Arroyo and Magro sum it up better than anyone: assessing better is not assessing more, but assessing meaningfully (2).<\/p>\n

Food for thought<\/strong>. If tomorrow my school had to assign an AI use level to every task of next term, how many of those tasks would still make sense at level 1, and how many would we discover were only asking for a product the machine already delivers?<\/p>\n<\/div><\/div><\/div><\/div>

Glossary<\/h4>\n