Automatic Linguistic Indexing of Pictures Real-Time (ALIPR): Software Recognizes And Describes Images In Words

"It can be done!"

"No, it can not be done, you idiot!"

Those were the more friendly responses in the past when photograpers, agency owners or even visitors of advanced IEEE workshops discussed if it would ever be possible to keyword images automatically.

CBC News Canada runs an article on "Software recognizes and describes images in words" with the key message "U.S. researchers have developed a computer system that can automatically recognize the content of a photograph and describe it in English":

The system by Penn State University professors uses a vocabulary of up to 332 words to annotate a photo with subject-relevant descriptors or keywords. For example, an image of a polo match could be described by the system as "sport," "people," "horse," "polo," the researchers said.

The technology makes it possible to automatically tag images with keywords, rather than having a person manually label the photos. The system can tag online collections of images as they are uploaded.

Image search engines currently rely on text tags to help index and sort images, so those that don't have descriptions are effectively invisible to search requests.

The Automatic Linguistic Indexing of Pictures Real-Time (ALIPR) system developed by Penn State associate professors James Wang and Jia Li solves the problem by analyzing the images and comparing them against a database. The computer then suggests 15 possible tags for the image.

"By inputting tens of thousands of images, we have trained computers to recognize certain objects and concepts and automatically annotate those new or unseen images," Wang said in a statement. "More than half the time, the computer's first tag out of the top 15 tags is correct."

The analysis takes about 1.4 seconds per image and in 98 per cent of tests suggests at least one correct tag in the top 15.

The university has applied for a patent on the invention.

Wang is expressly stating that the analysis recognizes "concepts". From the perpective of a professional image keyworder, 322 words are not a big deal for the english language with some hundreds of thousands of words.

But it´s a start, and once, after leaving the scientific world, the system may help in a not so distant future the stock photo industry to have the basic keywording done automatically.

Additionally, ALIPR might in a slightly more advanced version perfectly fit the needs of micropayment stock photo companies or sites like Flickr, Webshots and Photobucket with their millions and billions of images.

"It can be done, you .....!"

 

Links:

• ALIPR:  http://www.alipr.com, where you can upload photos to test and play with the system.
• More details here.