ChatGPT worries many teachers because it is a fabulous cheating tool. To fight against trickery, a student has developed GPTZero, an app capable of determining whether texts have been written by AI or by a human.
ChatGPT, OpenAI’s conversational AI capable of generating precise and detailed texts in a natural language via a simple request, is panicking the writing and teaching communities. She is so effective that it is sometimes impossible to differentiate her creations from those of a human, which poses a hell of a problem. Students in particular see it as a valuable time saver as well as the possibility of getting a good grade without too much effort. One problem among many others, which pushes its developers to find a way to set up a signature within the text in order to differentiate those produced by the AI from those resulting from a human.
While some are celebrating for the New Year, Edward Tian, a computer science and journalism student at Princeton University, USA, spent it developing an application whose algorithm can identify if a text was produced by the chatbot or if it was written by a human. Called GPTZero, it analyzes the text in order to evaluate its complexity, its random character compared to a text model as well as its uniformity. Combined together, these elements make it possible – well, in theory – to determine whether the content was created by a human or by an AI.
GPTZero: a practical but perfectible anti-plagiarism tool
GPTZero can be accessed for free through any web browser from gptzero.me Where this address. Be careful, the tool is still in beta version, so it often bugs and is not very fast. You have to wait a few minutes between each step for the page to load and do not hesitate to refresh it if an error message appears. You don’t have to be in a hurry! The site is in English, but GPTZero is quite capable of analyzing a text in French. To use it, simply copy/paste the text in the corresponding field then press Ctrl + Enter for the tool to analyze it. You must then wait a few minutes for all the results to be displayed and, at the end, click on “Get GPTZero Result”.
To determine whether the text was written by an artificial intelligence or a human, GPTZero measures its Perplexity, which designates “the randomness of the text”. This is’“a measure of the ability of a language model like ChatGPT to predict a sample of text. In other words, it measures how well the computer model likes the text.” The higher the perplexity of a text, the more likely it is to have been written by a human.
In our test, ChatGPT generated a text from the following instruction: “write an exhaustive text on the place of philosophy in Harry Potter.” His perplexity score was 38. However, it does not take into account many factors, such as the length of the text. However, longer texts are less random and generally have lower perplexities. This is why it is necessary to take into account the average perplexity, that is to say over the set of sentences. This time it was 72.3. Finally, by analyzing the complexity of the text sentence by sentence, GPTZero came to the conclusion that our text on Harry Potter was written by an AI. On our article on the Apple Pencil, our total puzzlement was 190, and the tool concluded that the text was indeed written by a human. To note that some sentences written by the human may have low perplexity, but there are bound to be peaks of perplexity as he continues to write – hence the need to analyze sentence by sentence. In contrast, perplexity is evenly distributed and consistently low for machine-generated texts.