New AI Tool converts blurry pixelated images in to 60X sharper images
Image Credits: Rudin Lab, Duke University
Artificial Intelligence is the simulation of human intelligence and thought process into machines. It’s tough to mention a technology not equipped with AI or not trying to incorporate in to it. Artificial Intelligence has proven to be a game changer for the world of technology, giving devices the ability ‘to think’ for themselves but along the lines of the code used to manoeuvre it. From toasters to rockets, artificial intelligence is marking its way across the entire world. Another field where AI is currently under development is image processing, where the system should be able to adjust according to the external environment and the data set to generate sharper images.
A good AI system in regards to generating images could be the difference between a sharper image of your pet that moved just as you were about to take that one in a million shot or clearing an image caught on a surveillance camera. But for AI to completely generate an image from scratch and giving you an indication of what the reality of a completely pixelated image could be representing, is something brilliant and unique on its own.
Contents
Duke University researchers found a way to turn blurred photos into sharper images
Recently, researchers at Duke University have managed to develop an AI tool that can improve the image quality of blurry images of unrecognizable human faces into computer-generated good quality sharper images. The technology is not entirely new, previous methods could improve the image resolution by up to 8 times the resolution of the original blurred image. Duke University researchers have come up with the AI powered technology that enables it to take a minimal amount of pixels and produce high quality computer-generated portraits. The images are developed with almost 64 times greater resolutions than the original. The algorithm lets the AI tool to create face lines, eye lashes and other features that were not present before hence the images they come up with are a resemblance of whomever the pixelated image was of.
Duke computer scientist Cynthia Rudin, who led the team, said “Never have super-resolution images been created at this resolution before with this much detail,”
What is PULSE?
The researchers at Duke University named their image processing method as PULSE which means ‘Photo Upsampling via Latent Space Exploration’. The team presented their work at the Conference of Computer Vision and Pattern Recognition (CVPR) which was held virtually from 14th to 19th June, 2020.
What does PULSE do and how?
PULSE take low resolution images and try to ‘guess’ what the missing pixels might represent, shaping the required face lines, eye lashes and other facial characteristics of a picture of a person alternately representing what could be the actual plausible image. The AI algorithm does this on the basis of high resolution images already fed in to the system. Studying how the pixels work in those images to generate an image, PULSE uses the same pattern and applies it to the low resolution images and then generates the image. It also works by taking the already fed images in to the equation and shrinking them to the same size and then observes how the low quality version of it might appear so and then proceeds to sharpen it accordingly.
PULSE not only generates one plausible outcome of the noisy pixelated image but many possibilities that can be associated to it. Rudin said PULSE can create hyper realistic portrait photos from noisy, poor-quality inputs that other methods can’t.
The system has the tendency to convert a 16 x 16-pixel image of a face to 1024 x 1024 pixels in a few seconds, adding more than a million pixels, taking it to HD resolution. It adds details such as freckles, wrinkles, eye bags and baby hairs to its final product, details that are incomprehensible to the millimeter in the original blurry and noisy image.
PULSE makes use of Generative Adversarial Network to sharpen image quality
A tool known as Generative Adversarial Network or GAN in machine learning was used by the researchers of Duke University. GAN works on the basis of two neural networks which are trained on the basis of sets of images fed to it. The first neural network takes the pixelated image and sharpens it to what could be the actual image on the basis of the types of images fed into it while the other studies the outputs of the first neural network and determines whether it is accurate enough to be mistaken for the real thing. The first neural network works better and better until there can be little or no difference detected by the second network.
Co-author Alex Damian ’20, a Duke math major says, “Our algorithm still manages to do something with it, which is something that traditional approaches can’t do,”
PULSE vs conventional image processing techniques
For the final testing, the researchers asked 40 volunteers to rate 1,440 images that were generated via PULSE and five other scaling methods, from 1-5 and not shockingly, PULSE got the best response as it portrayed high resolution images that were closest to the image of the real person.
You can view the results yourself here.
Limitation of the System
Because of the fact that AI does not give an absolute clear picture of the actual human face in the pixelated image and only has the capability to generate a plausible look-alike, it cannot be used for security purposes. The researchers say, ‘It won’t turn an out-of-focus, unrecognizable photo from a security camera into a crystal clear image of a real person. Rather, it is capable of generating new faces that don’t exist, but look plausibly real.’
Possible application of PULSE in the real world
According to the co-author Sachit Menon ’20, who just graduated from Duke with a double-major in mathematics and computer science, using the concept of producing clearer, sharper and what seems to be the closest replica of the distorted image, science can benefit from the AI tool immensely. From the field of microscopy to astronomy, scientists might be able to get what could be the closest glimpse of objects that are only known as blobs for now.
This research was supported by the Lord Foundation of North Carolina and the Duke Department of Computer Science.