New tool to make Internet memes convenient for visually impaired people

By Author  |  Published: 2nd Dec 2019  8:41 pm

In a new study researchers have developed a tool that will enable visually impaired people who find it difficult to understand memes, to enjoy them. Researchers at Carnegie Mellon University have developed a tool to automatically identify memes and apply prewritten templates to add descriptive alt text, making them intelligible via existing assistive technologies.

“Memes may not seem like the most important problem, but a vital part of accessibility is not choosing for people what deserves their attention,” said Jeff Bigham, associate professor at the HCII. “Many people use memes, and so they should be made accessible.”

Optical character recognition techniques are used to decipher the overlaid text, which can change with each iteration of the meme, reported the study published in – Access. For each meme type, it’s only necessary to make one template describing the image, and the overlaid text can be added for each iteration of that meme. But writing out what the meme is intended to convey proved difficult.

“It depends on the meme if the humour translated. Some of the visuals are more nuanced,” Gleason said. “And sometimes it’s explicit and you can just describe it.” “For example, the complete alt text for the so-called ‘success kid’ meme states “Toddler clenching fist in front of the smug face. Overlaid text on top: Was a bad boy all year. Overlaid text on bottom: Still got awesome presents from Santa.”

The team also created a platform to translate memes into sound rather than text. Users search through a sound library and drag and drop elements into a template. This system was made to translate existing memes and convey the sentiment through music and sound effects.