6 Detecting AI
6 Detecting AI
The initial reaction humans had to the idea of talking to machines varied, but once we embraced the idea and built it into our phones and various IoT devices, most people were comfortable with the idea. Much of our initial frustration was based on the voice assistants not understanding what we said or responding to us when we did not ask for their input. I have numerous Amazon devices in my house and have written several papers on how to perform forensic exams on them. I have also tested Amazon's claims about privacy with those devices regarding what was stored locally and transmitted to Amazon's cloud. Amazon devoted a lot of time and effort to its voice assistant, Alexa, to better understand humans and better respond to humans. Unsurprisingly, by eliminating false hits and making machines more like humans, we are now more disturbed than ever and have created new challenges and problems. The problem we are now facing in the academic world is not new to the business world, primarily with search engines, news, and social media sites. How do we detect AI?
6.1 Why Is Everyone Trying to Detect AI?
This seems like a ridiculous question for educators, and the answer for us is obvious. But, as I mentioned previously, educators are late in this game. Educators need to have at least a basic understanding of why everyone else has been trying to detect AI because many of the tools educators are now using to detect AI were developed for purposes other than to catch cheaters. This does not invalidate these tools because much of what is used in science and medicine is based on the use of tools or procedures for purposes outside of their initial intended purposes. I learned how to "JTAG" (Joint Test Action Group) mobile phones in 2014. This involved using a microscope and soldering into tiny ports and pads called "taps" on the logic boards of phones to extract the contents of the phone in a criminal investigation. Like educators, law enforcement was using a tool designed for something else and exploited by others before the police began using it for themselves. The manufacturer installed the JTAG taps on the phones for repair and diagnostics. JTAG taps were also used on the logic boards of gaming consoles, and gamers, who wanted to cheat, learned to exploit these repair taps to give their character unlimited life or some other advantage. Law enforcement also realized that these taps were a way into these devices and thus started using this technique to extract evidence in criminal investigations. Thus, unintended consequences always arise to the benefit or detriment of another. I will refer back to this paragraph when I start talking about the tools we use and designed for educators to catch cheaters.
6.2 What is SEO?
There is a universal concern in education about the use of Chat GPT by students. However, there is just as big of concern about the use of AI for generating content created for general consumption on the internet. That includes news, social media posts, business reviews, and marketing. SEO stands for Search Engine Optimization, which involves driving Internet traffic to a particular business website using techniques that promote your website in the results of searches on search engines like Google (Duo, 2019).
Google objects to certain tactics that unfairly promote one business above another, so it takes steps to detect and stop this from happening. That detection includes the detection of AI-generated or modified content. Google has devoted considerable research and resources to detecting AI-altered and "plagiarised" content in which content is stolen from other sites using a method called scraping. That scaped content can be modified and used to promote or demote certain sites, thereby gaining a market advantage \citep{30_schwartz_google_2022}. However, as with everything related to AI, there is a catch and, recently, a clarification. Depending on how it is used, AI can be a legitimate tool for many things, including developing content to be used in marketing. So if a really good business uses AI to develop really good ad content, what is wrong with that?
Google has since clarified its stance regarding SEO and AI-generated content and is apparently claiming that it will reward high-quality content, no matter how it was created - even by AI. So Google considers the use of AI to "manipulate" search engine rankings to be spam and, therefore, a violation of their policy, but the use of AI for good content creation is okay \citep{32_kashyap_ai-generated_2023}.
\subsection{Are We Trying to Detect AI or Create Undectable AI?}
For AI detection, the answer is both. This is far outside of my area of expertise, but there is a point to this brief description of SEO and the issues surrounding it. Google was or is developing methods to detect AI - for many different reasons. Some of the AI checkers I reviewed were created for the purpose of helping business optimize their content by using AI to create content. However, they want AI-created content not to look like AI. Therefore, they create their own AI checkers to make sure they are not penalized in search engine rankings because of their own AI-created content. Whether the AI checker was created to catch cheaters in education or for the purpose of SEO, I only care if they actually work. I mention this because you will notice while on sites designed for SEO and marketing, the atmosphere and purpose are different. So whether the goal is to detect AI or to create undetectable AI, the same tools (AI detectors) are used for both.
\subsection{Using AI Checkers}
There are several free tools available to check for AI-generated content. My initial experience with AI checkers was based on a Google search for "AI checker". As a side note, I would recommend leaving the "s" off of AI Checker, as a search for AI "checkers" will return matches related to the game of checkers. Quill (\url{https://www.quill.org/}) is a site designed to help students become better writers and critical thinkers. A direct link to their AI checker is the following (\url{https://aiwritingcheck.org/}). Quill claims to have tested their algorithm on 15 thousand essays with 80-90 percent accuracy, but Quill advises caution when using their tool to check for academic dishonesty.
Writer (\url{https://writer.com/}) is a site designed to be an AI writing assistant similar to Grammarly. Writer.com has an AI detection tool that allows you to paste a sample of your created or suspected AI content into their AI checker at (\url{https://writer.com/ai-content-detector/}). The checker then displays the results as a percentage that estimates how much of the suspected content is human-generated.
So, with my initial search, I found a couple of AI checkers designed for writing analysis and assistance. As a test, I pasted a paragraph from this paper, which I wrote without any writing assistance, into its content checker. I wrote this paper from a first-person perspective and used the word "I" twice in that paragraph. As expected, Writer.com identified my writing as "100\% human-generated content" (Figure \ref{fig:HumanContent}). Quill also identified my writing as "Text Written by Human" (Figure \ref{fig:HumanContent}). Therefore, despite what my friends say about me, I am human.
%%Now, all I have to do is ask Chat GPT-4 to generate a writing sample, paste that sample into the same site which correctly identifies me as human, and when that site correctly identifies a machine as a machine, this paper is done.
%%%%%%%%%%%%%%%%%% FIGURE FIGURE FIGURE FIGURE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[htpb]
\caption{Lorenz's Normal Writing - Content From This Paper}
%\vspace{-0.5em}
\label{fig:HumanContent}
\centering
\centerline{\includegraphics[width=1\textwidth]{img/writerHuman.png}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% SubSection %%%% My First Attempt to Write "Bad" Like AI %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{My First Attempt to Write "Bad" Like AI}
After the discussion at the department chair meeting on 2-17-23 about AI and the use of AI checkers to determine whether something was written by AI, I decided to conduct a quick test of an AI checker. My interest in digital tools is not just their results or conclusions but how they arrive at those results or conclusions. I initially suspected that it was merely an analysis of the writing style. If that were the case, I want to know if I could write in a style that would identify me as AI. When I conduct tests, I like to start with broad, obvious assumptions and then narrow down and fine-tune the parameters. I wrote a paragraph about a wall and intentionally wrote it in a way that I described as clunky or robotic. I needed a sample of at least 100 words for Quill.org to perform the analysis, and I wrote the following paragraph.
%%%% SUB SUB SECTION &&&&&&&&& Sample 1 - Lorenz Bad Writing &&&&&&&&&&&
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Sample 1 - Lorenz Bad Writing}
\begin{quote}
Now I wish I were here at the wall with you and I wish you could be at the wall with me. The wall is one of the best places in the world and most people enjoy the wall. We all want to see the wall but not many of us get to experience the wall. The wall is where I first found out that I had trouble seeing and may need glasses. The wall told me that I needed glasses so I decided to have an eye exam. If I were to ever visit the wall again, I think I would do it differently this time.
\end{quote}
I didn't put any thought into the 107 words I wrote except being intentionally robotic when I wrote about "the wall". I copied and pasted my first attempt to write like AI into Quill.org \url{https://aiwritingcheck.org/}, and that site immediately identified my paragraph as "Written by AI". But Quill gives no score or percentage estimating my paragraph's level of AI versus human creation. I then checked my paragraph on the wall with Writer.com \url{https://writer.com/ai-content-detector/}, which identified my writing as 1\% human-generated, therefore 99\% AI (see Figure \ref{fig:FakeAI}).
%%%%%%%%%%%%%%%%%% FIGURE FIGURE FIGURE FIGURE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[htpb]
\caption{Lorenz's Attempt to Write Like AI}
%\vspace{-0.5em}
\label{fig:FakeAI}
\centering
\centerline{\includegraphics[width=1\textwidth]{img/writer.com.png}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
This test confirmed that there is no objective analysis of metadata or checking of my IP address, as the submission method does not require me to attach any document that may contain some metadata or creation time. The analysis by the AI checkers was purely based on the only thing the AI checker could see - my writing style, spelling, grammar, etc.
The fact that AI identifies my writing as AI does not invalidate the writing tools as it can only look at the words submitted, and I was intentionally being as robotic as possible. Writing about a wall like "bad AI" was easy. That robotic, repetitive, stiff writing or speaking reminiscent of early Hollywood depictions of talking machines is easy to replicate. Each sentence is repetitive and stands on its own, without regard to the subject of the paragraph. My paragraph about the wall is more of an old stereotype of AI's writing ability. However, current AI checkers still consider this robotic writing style to be a strong indicator of AI despite the fact that voice assistants have been much more fluid than my wall paragraph for years.
\subsection{Putting Chat GPT-4 to the Human Test}
I did not start using Chat GPT until after GPT-4 was released. So after playing with it for a while, I decided to test GPT-4 and test the two AI checkers to see if the submission was detected as AI or human. I was already impressed with GPT-4's explanation of some fairly complicated procedural and substantive criminal law court decisions and statutes. So I asked GPT-4 to explain the legal term "reasonable suspicion."
%%%% SUB SUB SECTION &&&&&&&&& Sample 2 - Chat GPT-4 Reasonable Suspicion &&&&&&&&&&&
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Sample 2 - Chat GPT-4 Reasonable Suspicion}
\begin{quote}
\textbf{Lorenz}: \textit{write two paragraphs describing reasonable suspicion in simple terms}
\textbf{Chat GPT-4}:
\textit{Reasonable suspicion is an important concept in law that helps protect people's rights while allowing the police to do their job. It's a bit different from probable cause because it doesn't require as much evidence, but it still means that police officers need to have a solid reason to think that someone might be up to something illegal. They can't just stop and question someone because they have a hunch or because they don't like the way the person looks. They need to be able to point to specific facts or circumstances that make them think something might be going on.}
\textit{ For example, if a police officer sees someone walking around a store nervously and constantly looking over their shoulder, they might have reasonable suspicion to think that person is planning to shoplift. In this case, the officer could stop and question the person to find out more about what's going on. However, if the officer just sees someone walking down the street and doesn't notice anything out of the ordinary, they wouldn't have a good enough reason to stop them. Reasonable suspicion helps strike a balance between giving the police the tools they need to keep people safe and making sure that everyone's rights are respected.}
\end{quote}
I copied and pasted Chat GPT-4's answer about Reasonable Suspicion into writer.com's AI Content Detector, which determined that the text was 100\% Human-Generated Content (see Figure \ref{fig:Passing100}). I then pasted the same text into Quill.org's AI checker, which determined that the text was generated by AI. This example involving Chat GPT-4's explanation of Reasonable Suspicion helps reveal something about the algorithms AI checkers use to make predictions about whether content was created by AI or a human. It also reveals that different AI checkers use different algorithms or criteria to predict the presence of AI-created content.
%%%%%%%%%%%%%%%%%% FIGURE FIGURE FIGURE FIGURE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[htpb]
\caption{GPT-4 Reasonable Suspicion Explanation Passed for Human on One AI Checker}
%\vspace{-0.5em}
\label{fig:Passing100}
\centering
\centerline{\includegraphics[width=1\textwidth]{img/ReasonableSuspicion100percertPASS.png}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Mistaking Good Writing for AI}
At the risk of being picked apart by English professors or anyone more skilled at writing, I am slightly reluctant to say my paragraph in Sample 3 is "well written". I was striving for something much better than my writing about the wall in Sample 1, which is not a very high bar. My primary objective was to again fool AI checkers into thinking I was AI. However, with this sample, I wanted to write well and still write like AI. My first attempt to write like AI in Sample 1 was intentionally bad and robotic. Not only was it repetitive in the extreme, but what I wrote in that paragraph was also absurd. I referred to a wall as a living, sentient thing that spoke to me. If bad, inaccurate writing by a human is obviously AI according to AI checkers, can good, accurate writing by a human be mistaken for AI? Thus I set out to create a writing sample about probable cause. I wrote something that is more formal than I would write to students in an online class. Unlike my writing about the wall, what I wrote about was "correct" regarding the subject of my writing. I have described probable cause more times than I can remember - in person and in writing. However, this time I was definitely more conscious of the words I was using. Attempting to write "well" like AI affected my choice of words and sentence structure. I wouldn't describe Sample 3 as an accurate, unbiased test of AI checkers because I set out to imitate more modern AI. Figure \ref{fig:LorenzPC} shows the results of Quill.org and Writer.com.
%%%% SUB SUB SECTION &&&&&&&&& Sample 3 - Lorenz Attempt to Write Well Like AI &&&&&&&&&&&
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{Sample 3 - Lorenz Writing About Probable Cause}
Probable cause is a legal term used to describe circumstances that must exist before an arrest can be made, or a search can be conducted. Probable cause is a metric used by a neutral and detached magistrate to determine whether the facts and circumstances described in a search warrant affidavit are sufficient to justify a search or seizure. Probable cause is also the standard used by a grand jury in determining whether to issue an indictment charging that a specific crime has been committed. The determination of probable cause is based on a reasonable evaluation of facts and circumstances known at the time of the evaluation.
%%%%%%%%%%%%%%%%%% FIGURE FIGURE FIGURE FIGURE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[htpb]
\caption{Lorenz's Attempt to Write Well Like AI}
%\vspace{-0.5em}
\label{fig:LorenzPC}
\centering
\centerline{\includegraphics[width=1\textwidth]{img/Sample 3 Results with Quill and Writer.png}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% SubSection %%%% Digital Evidence vs an Educated Guess %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Digital Evidence vs. an Educated Guess}
In a criminal or civil investigation, I use forensic tools to perform a mobile phone examination which begins with an extraction of the contents of the phone. I use programs and techniques designed to examine, recover and preserve evidence in its original form. Text messages, photographs, documents, emails, and other digital evidence have metadata associated with these artifacts. Meta-data are the date and time the message was sent or received and the phone number from which the text originated. If a student submits a Microsoft Word document as an attachment in Blackboard, it can have metadata that may tell us about who and when it was created. However, text pasted into that document, pasted into Blackboard, or typed into Blackboard does not have metadata that tells us when or who created the text. Even though Blackboard can provide information on the IP address of the student, the IP address doesn't tell us if the student's submission is content originating from another source or location.
%%% SubSection %%%% AI Checking Tools Are Not Forensic Tools %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{AI Checking Tools Are Not Forensic Tools}
When AI checkers analyze text pasted into a website, none of what I described above occurs. AI checkers use algorithms to determine whether the suspected text appears to be created by an AI or a human. This is based on style, grammar, or sentence structure. This means that there is no metadata to examine or compare. For digital artifacts, metadata is the part of each artifact that is typically unseen. The "content" is the part of the digital artifact that we see. For example, in a text message from one person to another, "I love you." is the content of the message. Photographs sent via text message, chat apps, or multimedia messaging systems (MMS) are frequently stripped of their metadata, leaving only the image for us to determine when, where, who, and what camera created it.
%%% SubSection %%%% Validating the Tools We Use %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Validating the Tools We Use}
One of the most important parts of digital forensics investigations involves frequent testing and validation of the tools and techniques used to analyze digital evidence. This is especially important when the results of that analysis can affect a person's fate in a criminal or civil investigation. There many ways to validate the results of forensics tools used. The best way is to test the forensic tools on evidence you create yourself. I am most certain about a photograph I took myself. After all, I used my own phone and knew exactly where I was standing and what time it was. I can pull the photograph directly from my phone so that it retains its metadata data, and I can test any number of forensic tools I have to make sure they are all capable of seeing and displaying the same information about the photograph.
%%% SubSection %%%% Validating a Forensic Tool Using Metadata %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Validating a Forensic Tool Using Metadata}
A digital photograph contains metadata called EXIF data (Exchangeabel Image File) and it can be examined, revealing evidence of its origin. This includes the date and time the photograph was taken and information about the device or camera used to take that photograph. The camera settings can be recovered from the photograph, and depending on the settings of the device used to capture the photograph, the GPS location can be obtained. A single photograph can contain some or all of these data. When I recover the photograph, I can preserve it and, to ensure that the evidence has not been tampered with, a hash value can be generated so that the evidence can be tested and checked so that any copy is a bit-for-bit duplicate of the original. This hash value is essentially a digital fingerprint represented by a long code unique to any piece of digital evidence. As an example I took a photograph of my lawn mower I took with my iPhone 13 Pro Max. That photograph was uploaded to my Google photos and I downloaded it on my PC. But even after all of that it retains its EXIF information. (See Figure \ref{fig:Mower})
%%%%%%%%%%%%%%%%%% FIGURE FIGURE FIGURE FIGURE %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure*}[htpb]
\caption{Photo of Lawn Mower Taken on an iPhone}
%\vspace{-0.5em}
\label{fig:Mower}
\centering
\centerline{\includegraphics[width=1\textwidth]{img/MyLawnMower.png}}
\end{figure*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%% SubSection %%%% A Writing Sample Pasted in AI Checkers has no Metadata %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{A Writing Sample Pasted in AI Checkers has no Metadata}
If I were asked to prove a particular person used Chat GPT to create a forgery or commit a crime, I would use a search warrant to seize and search the person's computer so that I may examine the hard drive(s) for internet history showing visits to OpenAI and other digital evidence demonstrating how the questionable material was created or obtained. Of course, obtaining that kind of metadata and evidence is impossible when it comes to checking a student's work for plagiarism or the use of AI.
That leaves the AI-checker and us with only the content of the student's submission to determine if it was created by a human or AI. The content submitted by a student on Blackboard could have been written minutes or years earlier by a student, or it could have been created by Chat GPT-4. Date or time information, GPS information, IP addresses, or any other digital information that can be generated with original artifacts is not transferred to Blackboard, emails, or any other digital interface when the text is copied and pasted.
%%% SubSection %%%% AI Checking Tools Are Limited Tools %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Current AI Checking Tools Are Limited Tools}
This is not an attempt to undermine or discourage the use of AI checkers, as they are a very useful tool, and I am currently using them. However, we (humans) sometimes believe that a particular process is scientific or forensic just because the process is digital and unseen. It is tempting to assume that something complex and scientific is occurring in the background with AI checkers. The algorithm AI checkers use may be complex, but the algorithm cannot analyze what is not there, namely, metadata. Therefore, the opinion provided by an AI checker is based on style, structure, punctuation, and other traits common to human writing that distinguish it from AI writing. The title of this section begins with the word "current". Publicly available AI is a relatively new phenomenon, and thus the issues we are all experiencing with its release have yet to be fully realized. The problems associated with new technology often outpace safety, ethical analysis, and regulation. There are scientific detection models already deployed, in development, or in the theoretical stage.