ChatGPT Is a Blurry JPEG of the Web

OpenAl’s chatbot offers paraphrases, whereas Google offers quotes. Which do we prefer?

The New Yorker, 2023

5 min read

5 take-aways

Audio & text

What's inside?

Large language models provide blurry AI answers because of their dependence on “lossy” compression.

1×

Editorial Rating

Qualities

Analytical
Concrete Examples
Hot Topic

Recommendation

Today’s large language models operate by scouring massive data inputs, much of it from the web. The sheer magnitude of that training material requires researchers to employ compression algorithms that contain inherent flaws. The results are entertaining and often useful, but a far cry from human creative prose. Author Ted Chiang compares OpenAI’s ChatGPT to a “blurry JPEG” of the internet, rather than a reliable responder.

Summary

Large language models and Xerox photocopiers share error issues, when compressing information.

File compression requires encoding, or converting, text into a more compact version – and then decoding it, or reversing the process. If the original and decoded files are identical, the compression is “lossless.” If the restored material is missing information, it is called “lossy.” Typically, lossless compression is used for text storage and computer programs, while lossy compression is used to save images and multimedia materials. Lossy compression isn’t usually noticed unless it causes “compression artifacts” such as image fuzziness or subpar audio.

Photocopiers sometimes eliminate important details, as long as the images appear similar enough to one another. In the same way, language models such as OpenAI’s ChatGPT provides an approximation of grammatical text, based on the immense volumes of material it retains and assembles to answer questions. By employing statistical regularities, GPT’s responses are generally acceptable. But because its answers are based on lossy algorithms, they represent a “blurry JPEG of all text on the...

About the Author

Ted Chiang is an award-winning author of science fiction. In 2016, the title story from his first collection, Stories of Your Life and Others, was adapted into the film Arrival. He lives in Bellevue, Washington, where he works as a freelance technical writer.

Comment on this summary

More on this topic

In our Journal

11 months ago

Journal Article

How to Get Better Results from ChatGPT

To get the most useful answers from generative AI programs, ask more useful questions. ChatGPT is a generative large language model AI composed of deep learning algorithms trained on enormous databases of textual information, much of it sourced from the internet. It understands complicated language patterns and responds conversationally to natural language queries. Whether like […]

1 year ago

Journal Article

Chatbot, Where Are You Headed?

GPT-4 and other large language models are changing our lives at a pace no one could have imagined. Where do we go from here? And how did we get here in the first place? Several breakthrough technologies have changed society during the past centuries. However, changes didn’t happen overnight. It took time from the invention […]

1 year ago

Journal Article

What Are You Talking About with ChatGPT?

How you talk to ChatGTP has an enormous impact on what it spits out. This presents many opportunities, but also some dangers, on the path to widespread and successful use of this AI. These days, everyone is talking about (or to) ChatGPT. OpenAI’s revolutionary natural language processing system GPT-3.5 represents the latest advancement in natural […]