To help teachers identify the potential use by students of artificial intelligence (AI) to generate written content, Imagine Edgenuity offers a Generative AI Detector as an add-on feature.
This article provides information about:
- Frequently asked questions
- How the Generative AI Detector works
- Chat models detected
- Accuracy and limitations
- Best practices for avoiding false accusations
For information about accessing and reviewing the results of the Generative AI Detector, reference Viewing an assignment that was automatically scanned for plagiarism and AI-generated content.
Frequently asked questions
Grammarly use will trigger an AI alert if the student has used the tool to substantially rewrite their text. Grammarly will not trigger an AI alert if the student has only used the tool to correct spelling and grammar.
We suggest the following when using Grammarly:
- For Essays: Require students to write the Drafting activity without Grammarly and allow them to use Grammarly for Revising only. Then the teacher can compare the Drafting to the Revising and see the extent to which the student changed the text.
- For Short Writing: Require students to enter the original text (what they wrote without Grammarly) first and then paste the edited-with-Grammarly text below it. AI will scan both, and the teacher can view what triggered the AI.
How the Generative AI Detector works
Our Generative AI Detector tool functions very similarly to AI tools like ChatGPT. Content created by AI tools is created as the result of “training” that occurs by scanning numerous amounts of human-written source text. The AI tool then creates the content based on its training.
The Generative AI Detector was trained to detect AI-generated responses by using similar algorithms. The Generative AI Detector is trained on trillions of pages of publicly available human-written content from thousands of universities and enterprises worldwide. Our detector uses its training to determine the probability that the submitted content was generated by a human or an AI tool.
AI content detection is included in the PDF report with the Plagiarism Checker. The AI scan reports a confidence level, expressed as a probability, that the text was written by a human versus an AI tool like ChatGPT. This scanner is a tool to help teachers gather information about the contents of an essay; a judgement about cheating is ultimately up to the teacher.
In the Gradebook and the PDF report, the information is presented as follows:
-
Gradebook: Two icons display under Academic Integrity: P (plagiarism) and AI (AI-generated content). Click either icon to see scanner details for both.
- If use of AI is detected, a red AI Detected button displays.
- If use of AI is not detected, a green AI Not Detected button displays.
- If use of AI is detected, a red AI Detected button displays.
-
PDF Report: The percentage of text and number of words identified as being written by AI or by a human is represented in the AI Content Detection window.
The report also provides a heatmap of the areas where potential AI-generated content was detected.
Chat models detected
The Generative AI Detector detects the following chat models:
- ChatGPT (also called Chatbot)
- GPT4
- GPT3
- GPT2
- T5
- BERT
- Jasper
Accuracy and limitations
The Generative AI Detector currently works only in English with an accuracy of 99.12%. Accuracy increases as text length increases; for best results, text should contain at least 400 words. The accuracy of creative writing, including poems and song lyrics, is typically lower than other types of content.
The Generative AI Detector can detect mixed text (part human, part AI). It can detect manipulation such as deliberate typos.
Best practices for avoiding false accusations
There is a small — but not zero — chance that content written by a student will be falsely labeled as AI-generated. Unlike a plagiarism checker that provides sources of plagiarized content, the Generative AI Detector does not provide evidence. Instead, it provides a probability of AI versus human generated text.
The scan is designed to help teachers make judgements about the student submission content and if it poses a threat to academic integrity. Ultimately, the educator will decide if the submission was the result of cheating.
Generally, on an essay with potential AI-generated text, before speaking with the student, we recommend that you consider:
- The length and type of the text.
- The amount of time the student spent on the essay.
- Other essays written by the same student; how do they compare?
- The results of running the same essay manually through an online checker; how do they compare?
- Whether the text also includes plagiarism matches, because AI generators use Web sources.