Edge Detection, Denoising Algorithms, and Other Ways to Look at Art

In Professor Ivo Dinov’s Data Science and Predictive Analytics class, students are learning how to manage, analyze, and make sense of Big Data in the real world. You’ve probably associated large, complex datasets with healthcare data, scientific information, or all of the information about viewing preferences of Netflix subscribers, but art?

Edge Detection, Denoising Algorithms, and Other Ways to Look at Art

Written by Olivia Ordoñez

When introducing Curriculum/Collection to students in the class, Dave Choberka, Andrew W. Mellon Curator for University Learning and Programs, said, “We’re not displaying art as examples of things that classes are already doing or to illustrate what is already there. We’re using art to pose problems and drive cutting-edge discussions in whatever discipline or field is looking at the art.”

In collaborating on Curriculum/Collection, students will extract information from art in UMMA’s collection, by applying information-processing algorithms. These skills will not only help students learn how to manage Big Data but also presents them and UMMA with the opportunity to look at art in new ways—to see things in the art that might not have been visible to us without the computer’s vision. 

And the reverse is true, too: some features of the art might be more easily visible to the human eye than to a computer program. 

As Professor Dinov said, he wants students to “Extract quantitative information that might lead us to some kind of a perception, decision-making inference, or interpretation of the art that has a point of view.”

With this in mind, Choberka and Dinov selected art from UMMA’s collection that falls into two broad categories: art with features that we can train computers to pick out better than we can observe with the unaided eye and art where humans might identify the salient features of the piece more quickly than a computer can.

When confronted with Lesley Dill’s White Poem Figure (The Soul Has Moments of Escape), students suggested they might use a denoising algorithm to extract data from the artwork. 

A denoising algorithm compares pixels to their nearby neighbors. If there’s a pixel that’s really different from all of its neighbors, and it’s not on a boundary or an edge, then the algorithm would make the pixel more similar to its neighbors. Denoising Dill’s work would reduce scratches, smudges, and stray pixels in order to make color areas more uniform. Because a lot of the pixels are affected by the artist’s scratches, the algorithm might help the letters appear darker or clearer—more legible to us—and make the white spaces on the edges appear more bright.

Students suggested they might use an edge detection algorithm on Khalid al-Saa’i’s Winter in Ann Arbor

An edge detection algorithm finds the largest difference between neighboring pixel values, which helps the algorithm mark edges in an image. Thus, using edge detection would identify the boundaries between the edges of letters and the background, which would allow students to detect the really faint brushwork in the background and make legible letters that our eyes struggle to see. 

Finally, optical character recognition might help identify features of Jasper Johns’s Figure 3 from ‘Black Numeral Series’ .

There are so many applications for which detecting handwritten characters are useful. For example, machine learning is used to very accurately sort mail at the Post Office or to make transactions at a bank. If you’ve ever used mobile deposit and taken a picture of your check, the app has to think about the image, using an algorithm trained on handwritten characters to check whether the numbers on the check match the number you claimed to be depositing. 

Because accurately identifying handwritten characters was such a priority in early machine learning efforts, many algorithms were trained on the MNIST database, which contains over 60,000 handwritten characters to feed an algorithm. Today, the MNIST database is a benchmark standard used to test and train machine learning algorithms on their handwriting detection abilities.

If students use an algorithm that has been trained on the MNIST database, or on digit recognition, the algorithm may be able to immediately recognize the “3” in Johns’s piece, whereas humans may look at other features of the image first, given that the 3 is occluded and at such a large scale.

Other Recent C/C Stories

Take our latest quiz, and challenge yourself to see unusual connections between objects and course topics this semester at the University of Michigan

The assignment is due at the end of the week, at 11:59PM (EST), but you can take all the time in the world.

Students in Professor Matias del Campo’s Space, Time, and Artificial Intelligence class are asking questions about the relationship between artificial intelligence, architecture, and creativity. Can artificial intelligence dream new visions of architecture into being?