Apple's Genmoji Might Be a Prompt Engineering Training Tool in Disguise

If a picture is worth a thousand words, what’s the value of an AI-generated emoji?

I’ve been giving that a lot of thought since Apple unveiled Genmoji at its developers’ conference on June 10. The new emoji maker, due this fall in the next version of its iOS operating system software, will use a generative AI engine to let iPhone users describe an emoji with a few words — Apple’s examples included “smiley relaxing wearing cucumber slices” and “squirrel DJ.” Apple’s iOS 18 software then creates the digital icon for you in seconds, so you can share it in your messages and as stickers. 

Many predict that Genmoji, offered as part of a new set of Apple Intelligence features for the iPhone, Mac and iPad, will lead to an explosion of custom pictograms you can use in addition to the more than 3,700 emoji that the Unicode consortium has standardized and that we’re pretty familiar with (😉👋✔️). Keith Broni, editor in chief of the emoji dictionary Emojipedia, told The Washington Post that Genmoji will “open the floodgates” of niche emoji creation.

I have no doubt Apple users will try to outdo themselves by dreaming up original emoji that are cool (💀), creative (💡) , clever (🤓), bizarre (🪱), obscure (🤺), silly or tacky (💩) or some play on all that. 

But whether an AI-powered emoji generator is good or bad is up for debate, given that Apple’s prior attempts to inject some fun into its Message app got a meh reaction — remember 2018’s personalized “Memojis” and animated “Animojis?” And while your custom emoji might seem clever to you, will your recipient actually know what a squirrel DJ represents? 

“By adding the potential for unlimited, unhinged emojis, the company could springboard AI emojis to new digital-culture heights. Or it could fail to read the room and change how we communicate for the worse,” Tatum Hunter wrote in The Washington Post.

Fair point. Will Genmojis usher in the dawn of emoji confusion and Mehmojis? I honestly don’t know.

But I have a different thought about this new emoji maker: Might Genmoji be a simple, nonthreatening way to get people who haven’t played around with gen AI to become comfortable writing “prompts,” the term used to describe what we ask of AI-powered chatbots?

Given that Apple is a consumer tech behemoth that’s really good at mainstreaming new technology and getting people to buy into new ways of doing things, and given that there are millions of people using the iPhone every day, whatever features it introduces have the potential to disrupt the consumer marketplace — and our relationship with tech.

That’s why I thought instead of just spawning variations on eggplants, unicorns and trolls, Genmoji could be a rudimentary training tool for a new generation of prompt engineers. 

The Mac’s Puzzle and ‘mouse coordination’

Before you write me off as crazy, consider The Puzzle desk accessory included in the original operating system software for the Macintosh 128K computer when it launched in January 1984. Designed by Apple’s inventive and resourceful software engineer Andy Hertzfeld, The Puzzle injected a bit of fun into the whole idea of owning a personal computer at a time when old-school IBM dominated the PC market. 

But The Puzzle’s true contribution may be that it helped people understand how to use the Mac’s groundbreaking graphical user interface and aided them to become more comfortable using its novel input device, the mouse.

Hertzfeld told me that was part of the thought behind it, when I interviewed him for the fifth anniversary of the Mac while I was working as a reporter for MacWEEK. I’m happy to say he’s documented The Puzzle’s origin story on Folklore.org, a site that features over 100 stories about the early history of Apple and the Mac written by the people who were there.

“Desk accessories were usually utilities, like the calculator or the alarm clock, but I thought that we should also have a game or two, to show that the Macintosh was fun, too,” Hertzfeld says. “I decided to write a ’15 number puzzle,’ where there are fifteen numbered tiles in a four by four space that must be arranged in sequential order.

“If you click on a tile next to the empty space, it slides into that space,” he explains. “It was a fun way to waste time and build up your mouse coordination.”

Clever, right? Early Mac users were just moving tiles around and playing a game, but what actually happened is that they got comfortable navigating Apple’s GUI and using a mouse, which helped popularize it as a computer input device. (By the way, Hertzfeld also recounts how The Puzzle almost wasn’t included in the OS because it took up too much disk space. So he rewrote the code and got it down to 600 bytes. Read his essay if you’re a Mac fan.)

Anyway, when you consider how a little game helped prepare the planet for the mouse as an input device, it’s not a stretch to view Genmoji as a way to get people thinking about writing prompts. I mean, dreaming up original emoji certainly sounds like a fun way to waste time.

Prompts, avadoas and baby daikon

To write successful prompts, you need to provide enough specifics to get a satisfying answer within the first few asks so you’re not frustrated by the AI’s early outputs. 

I know this because I’ve spent the past year understanding the implications and use cases for gen AI. I also studied over 55 prompt guides, like this one from OpenAI and this one from Adobe, and watched hours and hours of YouTube videos to create a primer for the CNET team that boils my learning down to 15 tips on how to master the art of the prompt. (You can get a copy by signing up for my weekly AI newsletter at CNET’s AI Atlas.)

The trick is to make sure your prompt is specific. Instead of asking for a “blue star” and hoping for the best, try asking for a “a five-pointed, shiny aqua blue star in the style of a graffiti artist.” Or maybe you’re into cat emoji (no judgments here); instead of a “dancing cat,” you might ask for a “moonwalking calico cat wearing aviator sunglasses and Converse Chuck Taylors in black.” 

Genmoji, Apple says, will allow you to turn photos of friends, family and I presume enemies into emoji as well. Maybe they’ll let us AI-ify that photo and turn it into a black-and-white, Ansel Adam-esque image or a Kodachrome colorized masterpiece.

Or maybe the Genmoji toolbox will be robust enough to allow us to riff off ideas like OpenAI’s text-to-image experiments with its Dall-E chatbot using fruits and vegetables, including the “avocado chair” and the “baby daikon radish in a tutu walking a dog.” 

Genmoji as a learning environment?

Again, what you’ll be able to ask Genmoji depends on what training data and capabilities Apple builds into the text-to-image tool. With Memoji, Apple provided a deck of characters, and then allowed us to customize features like skin tone, hairstyle and eyes.

When it comes to Genmoji, there aren’t that many details yet about how it will work.

I asked Apple if anyone on their design team, focused on user interface and human factors design, had considered Genmoji as a way to train people to talk to a gen AI chatbot. An Apple spokeswoman referred me to its press release and the Genmoji demo shared during the keynote at the Worldwide Developers Conference. 

There’s also an 11-minute explainer for developers that describes how they can “bring expression to your app with Genmoji.” In it, Apple input experience engineer Aaron Hurley shares the difference between Unicode emoji and Genmoji.

“Traditional emojis aren’t actually images but are instead a standardized list of Unicode characters sent as plain text and it’s up to the viewing device to render the appropriate image in its own font, just like any other text content,” Hurley says. “Personalized images like Genmoji, on the other hand, are unique, rasterized bitmaps that can’t be described by a Unicode text character.” 

He adds that Genmoji “are image glyphs” and developers can add them to their apps via a new application programming interface called NSAdaptiveImageGlyph that supports a standard image format, with a square aspect ratio that can be created in multiple resolutions and augmented with metadata, such as content description. 

OK. But I still don’t know if Apple was as clever as Hertzfeld in seeing beyond the fun factor when they came up with Genmoji. 

So I asked a user experience expert what he thought about Genmoji as a nonthreatening, easy way to get average people, some of whom may have not engaged with a chatbot before, to get comfortable talking with an AI tool.

Caleb Sponheim, a user experience specialist with the Nielsen Norman Group and a former computational neuroscientist, told me my idea isn’t that crazy. It could be a way for “Apple to kind of create a low-stakes, low-cost environment for people to have a little fun,” he said. But, he added, it all depends on how capable and user friendly the tool is.

“If we are taking Apple’s presentation at face value [and that] it will work at least as well as they’re saying, then it will be an entrance point for people to experience and start working with generative AI in an image generation context,” Sponheim said.

But, “it’s up to Apple and their implementation about how usable it is and how enjoyable of an experience it is,” he added. “Because if it’s not, if it’s challenging or frustrating, it will turn into another iMessage feature that gets hidden behind the keyboard.”

Even if Apple does create a truly inventive emoji maker, that doesn’t mean people will necessarily enjoy creating with it. A problem with popular AI text-to-image editors today, including OpenAI’s Dall-E and Midjourney, is that they don’t provide good feedback about what’s working and what’s not with your prompts, Sponheim said. The lack of a feedback loop would be frustrating to users trying to figure out how to create an emoji of a relaxing apricot with cucumber slices over its eyes instead of a smiley face if the system doesn’t have any apricot images.

“AI is not exempt from basic interface guidelines about how to make a good, usable experience,” Sponheim said.

We’ll just have to wait and see how much AI intelligence Apple builds into Apple Intelligence. In the meantime, if you’re excited about the prospects of creating your own emoji, start brainstorming. Genmoji may be just a fun way to waste time — or it could put you ahead of the class in mastering the art of the prompt. 🤷 

Editors’ note: CNET used an AI engine to help create several dozen stories, which are labeled accordingly. The note you’re reading is attached to articles that deal substantively with the topic of AI but are created entirely by our expert editors and writers. For more, see our AI policy.