The Motherboard

Pandora’s Zip File: What Should REALLY Terrify You About A.I. | Tom Cody

In early November, in a banquet room in Memorial Hall at Central Connecticut State University, the School of Engineering, Science and Technology (SEST) held a tech talk. The dinner served included rolls, baked chicken, and pasta salad with fresh herbs. Computer science and cybersecurity students and faculty sat at round tables, filling half the room. A girl with split-dyed hair leaned over to her tablemates, holding one of the silver-painted plastic pieces of cutlery, and whispered, “Wow, they really brought out the good stuff for this.”

Professor Samir E. Hamada, who holds a PhD in Computer Science and Engineering, approached the podium at the far end of the room to deliver the keynote on ChatGPT and data management. His posture reflected his stiff, clean-shaven upper lip. After a brief PowerPoint, he opened ChatGPT and connected it to a program called Back4App to build an app in minutes that could create and manage a database using ChatGPT as a user interface—work that, two years ago, would have taken a programmer far longer to complete.

After the talk, student Ben Rolfe explained that he uses ChatGPT as a sort of tutor to help him form outlines, bounce around ideas, and even wholesale write conclusion paragraphs in his general writing courses. In his programming courses, he copied his own work into the model to scan it for errors he would otherwise have to find manually, saving himself hours of work. He’s also used it to spitball ideas. “I would ask it to write out a tree traversal in the inorder traversal, and then I would use that to then draw back on the tree to fundamental my understanding of it.” 

For all the buzz A.I. language models have generated, even a superfan early-adopter like Rolfe acknowledges the limitations of the technology. ChatGPT will lie, and lie confidently, because it doesn’t actually understand the information it’s conveying the way a person does. “In order to use it, you still need to be a conduit, almost, for ChatGPT’s knowledge. You have to be able to take ChatGPT’s output and recognize if it’s good or not,” he offered, holding a relaxed hand face-up and drawing it towards his body to mimic a wiggly virtual beam of slippery knowledge floating directly through him.

Rolfe isn’t alone in his bright-eyed optimism regarding the chatbot’s capabilities as a study tool, or even his excitement to live in a world changed by newly-fundamental tech that may revolutionize the face of human labor.

In an article for Forbes,  futurist and business influencer Bernard Marr summarized the history of the famed chatbot. ChatGPT is an AI chatting model initially released in 2022 by OpenAI, which was founded in December 2015 by a number of big names in tech—including Sam Altman, Greg Brockman, Elon Musk, Ilya Sutskever, Wojciech Zaremba, and John Schulman. The upgrades between iterations have been exponential, as a result of both manual improvements and larger and larger breadths of input parameters—which, with GPT-3 and beyond, include hundreds of billions. The A.I. has gone from being able to predict proceeding words in sentence fragments to searching the internet and generating paragraphs of text or blocks of code in seconds. The economic impact of these developments is unquestionable; like a multitude of other cheery articles, Marr describes the tech as “shaping our world in unexpected and exciting ways.”

However, there are certainly others who are less excited, and they are most often, seemingly, the people who know what to expect.

Programmer James Somers, who understands that speeding up and automating processes is more efficient, is not as excited about the prospect of the elimination of basic programming the way people like Ben Rolfe and Professor Hamada seem to be. 

In a piece for the New Yorker, Somers explains that, over the years, professionally and personally, he has watched as A.I. has been increasingly used as a tool in the field. While novice programmers and techies see its ability to generate code for specific use-cases without the user having the requisite knowledge to do so as exciting, Somers doesn’t. He sees it as the obsolescence of many facets of technical expertise that he once was overjoyed to engage in. “I suspect that non-programmers who are skeptical by nature, and who have seen ChatGPT turn out wooden prose or bogus facts, are still underestimating what’s happening. Bodies of knowledge and skills that have traditionally taken lifetimes to master are being swallowed at a gulp. Coding has always felt to me like an endlessly deep and rich domain. Now I find myself wanting to write a eulogy for it.”

Geoffrey Hinton. Photo Courtesy of Johnny Guatto for University of Toronto. Edited by Tom Cody.

On a hike near his glass cabin on his own private island, A.I. godfather Geoffrey Hinton, who is the single most cited individual among computer science students on Google Scholar, told New Yorker writer Joshua Rothman that ChatGPT made him uneasy, explaining that these chatbots could comprehend the meanings of words and ideas. He called it an “existential threat.”

In the Willard-DiLoreto building at CCSU, a 300-level course on the portrayal of riots in the American canon let out. Dr. Susan Gilmore sat at the front of the classroom, available for any student who needed a word. As the final stragglers streamed towards the exit, she explained that she believes ChatGPT robs students of the ability to practice a couple of fairly critical skills. ChatGPT cannot read closely enough, or analyze work incisively enough. Its writing is drab and lacks a human touch. But beyond that, if there is no part of the learning process that cannot be automated through ChatGPT—from gaining summaries of works to creating outlines to writing papers—then the concern in allowing the use of the technology is to rob students of the skills they will need in higher level classes and future careers. 

Dr. Gilmore paused and glanced out the window at the growing darkness. “I just worry that people won’t learn to write, and therefore, won’t learn how to think in nuanced ways.”

This is an especially salient point in light of ChatGPT’s lack of proficiency with very complicated or publicly controversial topics. As reported in the Journal of Social Computing, sociologist John Levi Martin discovered through a number of interrogations that ChatGPT is relatively unable to develop stories that have negative endings. This is easily discovered by anyone interested in tooling around with the A.I. itself, as it frequently refuses to discuss very dark topics or develop ideas it does not consider to be essentially positive. 

Photo Credit: Tom Cody via Canva

Regarding issues of marginalized identity, things with ChatGPT are bleak. When asked about different issues related to culture, race, gender, sexuality, or ability, ChatGPT frequently either is unable to answer questions or defaults to summarizing the opinions of others. For example: ChatGPT cannot competently discuss the landmark novel Stone Butch Blues, a seminal work of LGBT history and identity, the same way it can discuss Moby Dick and its famous whale. It won’t decide whether or not wearing a Cuban chain is cultural appropriation. Any question about Israel and Palestine dissolves into comments about it being a matter of controversy. While not entirely incompetent in every subject, the chatbot’s dedication to neutrality would seem to indicate an internal politics of its own which it fairly nebulously conforms to. The effect leaves the A.I. straddling any two opinions as land at either side of the information superhighway in a way that suits this millennium’s new Colossus.

“I’m on a high horse here, technically, but this is wrong and bad

The skeptical view is reflected in the syllabi across the English department in zero-tolerance policies on the use of A.I. for class assignments. Many departments and colleges across the country have similar rules. However, as detailed by culture writer Miles Klee in Rolling Stone, these policies come with their own problems. Professors who do not understand this new technology end up flagging original student work erroneously in attempting to enforce them, and it ends up on the students to navigate baseless and widespread accusations of plagiarism.

Just after Professor Hamada’s SEST dinner, a bevy of familiar faces met up at the university’s cybersecurity club. They gathered around computer screens as Professor Thomas King showed them how to access unencrypted security cameras around the world, and make sure they are not breaking the law in doing so. There was a lot of technical jargon thrown around that blue-and-red-haired Liz Wertz explained to any pseudo-luddites present using the same stage-whisper she’d used at the SEST dinner to comment on the cutlery.

Photo Credit: Tom Cody via Canva

In the hall, Wertz fidgeted from foot to foot and explained that A.I. art is probably the arena she sees as the most controversial. She’s most interested in graphics technologies and knows a lot of artists who are watching like circling hawks as copyright lawsuits wind through the U.S. court system. A.I. images work by drawing on a database of images that have attached tags and descriptions, which it uses to predict an appropriate arrangement of pixels to suit a user’s prompt. However, it takes a lot of images to create a training database for an A.I., and often, those images are stolen. Additionally, as time passes, it may be impossible for artists to avoid the theft of their images for this purpose, as these A.I. image generators are, like ChatGPT, gaining the ability to trawl the internet for examples to use in prompt responses. Wertz explains that A.I. can even be used to prompt art in the style of individual artists, robbing those artists of a commission income. “I have close friends that take commissions to live. I’m on a high horse here, technically, but this is wrong and bad,” Wertz asserted.

In an article for the New Yorker, staff writer Kyle Chayka dutifully assembles the variety of complaints issued by a number of artists and attorneys. Plaintiff Kelly McKernan has had their name used specifically by users of A.I. image generator Midjourney to generate images copying their niche style “more than twelve thousand times in public prompts.” The lawyers representing them believe that, in terms of legal copyright, the A.I. image generators are not covered by the idea of A.I. images as “transformative works.”

As this art can be generated in seconds, there is a very real worry that it can replace flesh and blood artists. Graphic design student Campbell Karanian kept her eyes down and pressed her thumbs into one another when she spoke about the relatively libertine policy towards the use of A.I. art in the art department at CCSU. “I think I would be really frustrated if a professor accepted A.I. generated images and considered those on the same level of students who were, like, hand-drawing. It’s not the same amount of effort and care and time as doing it yourself.”

 In Chayka’s piece, McKernan reported that they’ve already lost work because of the prolific use of A.I. in commissioned illustrations that would normally be done by cover artists. “I can pay my rent with just one cover, and we’re seeing that already disappearing.”

Akin to the conversation ongoing in the English major, it seems as though there is this rising concern that, not only will work be eaten up by A.I. and render careers nonviable—but it will lead to a loss of fundamental skills, like hand-drawing or close-reading or coding, necessary for high level mastery in any discipline. Particularly as this technology only becomes more efficient, capable, and encompassing, and the upgrades have just begun.

Taking stock, there are many people who see A.I. as a bell that can’t be unrung, spelling a future where the skyrocketing arc of efficiency reaches a place in the heavens where the little humans, standing on the ground and doing their best, will be unable to see. As Somers laments, “getting computers to do precisely what you want might become a matter of asking politely.”

Tom Cody is a staff writer for Blue Muse Magazine. 

Header Image is courtesy of Tom Cody via Canva.

Blue Muse Magazine is a general interest literary magazine published by the students of the English Department at Central Connecticut State University in New Britain, Connecticut. We publish poetry, fiction, and a gamut of creative nonfiction on anything and everything the blue muse inspires us to write.

1 comment on “Pandora’s Zip File: What Should REALLY Terrify You About A.I. | Tom Cody

  1. Mary Collins

    I think this issue of Blue Muse has the most substantive features I’ve seen to date. Way to go PUBLISHING CLASS! I’m impressed with Tom’s sourcing–fact about Hinton single most cited person on Google Scholar; in-person interviews; New Yorker articles. I learned two new things: ChatGBT will lie and can only come up with positive endings. Hilarious. Thanks for the hard work on this.

Leave a Reply

Discover more from

Subscribe now to keep reading and get access to the full archive.

Continue reading