OSTG | SourceForgeThinkGeekITMJLinux.comNewsForgefreshmeatNewslettersJobsBroadbandWhitepapers X 
Welcome to Slashdot
 Login
 Why Login?
 Why Subscribe?

 Sections
 Main
 Apache
 Apple
 AskSlashdot
  7 more
 Books
 BSD
 Developers
  1 more
 Games
  8 more
 Hardware
  1 more
 Interviews
 IT
 Linux
 Politics
 Science
 YRO
 
 Help
 FAQ
 Bugs

 Stories
 Old Stories
 Old Polls
 Topics
 Hall of Fame
 Submit Story

 About
 Supporters
 Code
 Awards

 Services
 Broadband
 PriceGrabber
 Product Guide
 Special Offers
 Jobs

Searching by Image Instead of Keywords
Technology
Graphics
Posted by samzenpus on Wed May 04, '05 09:00 PM
from the find-me-something-square-and-green dept.
Content based image retrieval (CBIR), the technique to search for images not by keywords, but by comparing features of the images themselves has been the focus of much research ever since the web emerged. Consider for instance adding CBIR to Google Images, where you would be able to search for images similar to a query image instead of using keywords. A research project at Penn State University has recently been applied to the biggest aviation photo database in the world with close to 800,000 images. You can search for images similar to a photo already in their database (click "View similar photos") or submit your own query image. Some queries generate better results than others but CBIR is certainly here to stay and will be standard in many image applications of the future.

Falk AdSolution

 

 
Slashdot Log In
Nickname:

Password:

Public Terminal

[ Create a new account ]

Related Links
Review IT Products
Compare prices
Compare prices on Graphical Software
Content based image retrieval (CBIR)
Google Images
research project
aviation photo database in the world
already in their database
submit your own query image
better results
others
More Technology stories
More Graphics stories
 
 
This discussion has been archived. No new comments can be posted.
Searching by Image Instead of Keywords | Log in/Create an Account | Top | 184 comments | Search Discussion
Threshold:
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Think of the greatness to society! (Score:5, Funny)
by qewl (671495) on Wednesday May 04, @09:01PM (#12437393)
(http://humans.com/)
I can't wait to put a nipple into it!
Arm jokes... (Score:2, Funny)
by sharkey (16670) on Wednesday May 04, @09:04PM (#12437408)
and set for goatse!
Location? (Score:5, Funny)
by poopdeville (841677) on Wednesday May 04, @09:04PM (#12437416)
What an awful beach [airliners.net].
Wow (Score:5, Interesting)
by themoodykid (261964) on Wednesday May 04, @09:04PM (#12437418)
(http://www3.telus.ne...i/ProjectHazuki.html | Last Journal: Friday September 03, @05:53PM)
I was just thinking about this the other day. I think content-based image search is one of the Next Big Things. Cameras are so ubquitous now (for better or worse), but having to rely on metadata to give meaning to images requires lots of effort up front.

It will be interesting if we ever get to a stage where we can just search for a random object (or person) in a database of photos. Then you could take pictures of everything with an always-on camera and if you need to find where you put your car keys, just do a search.
  • Re:Wow by mboverload (Score:2) Wednesday May 04, @09:19PM
    • Re:Wow by flyingsquid (Score:2) Thursday May 05, @12:13AM
      • Re:Wow by david.heyman (Score:1) Tuesday May 10, @02:44PM
    • Re:Wow (Score:4, Informative)
      by theguyfromsaturn (802938) on Wednesday May 04, @10:24PM (#12437893)

      If you are only interested in searching for images on your own computer, have a look at imgSeek. http://imgseek.python-hosting.com/ [python-hosting.com]

      It's been around for some time now. You can not only use an existing image to search, but also do a rough sketch. Check the screenshots: [sourceforge.net]

      Nice complement to what has been presented in this article.

      [ Parent ]
      • Re:Wow by Berzelius (Score:1) Wednesday May 04, @10:27PM
        • Re:Wow by jaylene_slide (Score:1) Thursday May 05, @03:49AM
          • 2 replies beneath your current threshold.
          Brilliant! (Score:1)
          by kid_wonder (21480) <sklein&scott-klein,com> on Wednesday May 04, @09:05PM (#12437427)
          (http://scott-klein.com/)
          This that I have an image to search with...which is why I am searching for images to begin with - to find an image.
          They would need major manpower to maintain this db (Score:3, Funny)
          by guardiangod (880192) on Wednesday May 04, @09:07PM (#12437437)
          This is just asking for trouble. As most of you would probably imagine, some self-proclaim "comdeian" would post either porn pictures, or pictures that resembles porn body position.

          They would need a team of outsource Indian workers to go through each picture one by one!

          I am not Indian but...can I apply for the image filtering job?
          I said this first, I should get the job ;) .
          Top Search (Score:3, Funny)
          by daishin (753851) on Wednesday May 04, @09:08PM (#12437445)
          (http://daishi.aeoth.net/)
          Something with two circles and dots in the middle of each circle.
          security issues (Score:1, Offtopic)
          by green pizza (159161) on Wednesday May 04, @09:08PM (#12437448)
          (http://slashdot.org/)
          Forget slashdotting Airliners.net, how long before the TSA shuts down that website? The trainspotting hobby has already died off following terrorism fears, I can't help but think that other enthusiast sites like Airliners.net will be next.
          The ever popular 'Breast' option... (Score:1)
          by farmkid (15226) on Wednesday May 04, @09:09PM (#12437454)
          Hmmm...

          +"34b"

          -"puffy" ... Profit! (Oops, or something, grabbing for a Kleenex)
            Similar images (Score:2, Funny)
            by iMaple (769378) on Wednesday May 04, @09:14PM (#12437487)
            Some Applications of Our Research
            1. Airliners.net
            A site with almost 1,000,000 aviation images.


            Wow !!! I tested their Sample search [airliners.net] and all the results were aeroplane photos !!! Ok, ok the site only has airplanes but still ..:)

            On a more serious note the alogorithms seem to look for similatity in the colors and lighting rather than the subjects (for example it shows the interior of a cabin in photos similar to a whole plane in the sky. To really see its effectiveness we need to test in in the real world (google images) . The 'artisticly revealing' photo you always liked ... now you should be able get similar pr0n (^H^H^H^H I mean art) with these algorithms
            The Human Brain (Score:1)
            by smokeslikeapoet (598750) <{wfpearson} {at} {gmail.com}> on Wednesday May 04, @09:15PM (#12437494)
            (http://wfpearson.blogspot.com/)
            Isn't this kind of how the human brain works to identify objects your eye hasn't seen before?

            IANABP, I am not a bio-physicist but it seems very much like artificial intelligence to me.
              And for 'lynx' users... (Score:5, Funny)
              by mikael (484) on Wednesday May 04, @09:17PM (#12437504)
              (Last Journal: Monday July 05, @08:34PM)
              ... the search engine will support ASCII art image searches.

                Some relevant research papers (Score:5, Informative)
                by FleaPlus (6935) on Wednesday May 04, @09:28PM (#12437575)
                There's a bunch of interesting papers out there on content-based image analysis and retrieval. Below is a sampling from my bibtex file. Does anyone else have others they'd like to share?

                * Finding Naked People [hmc.edu] (Fleck et al, 1996)

                * Video google: A text retrieval approach to object matching in videos [ieee.org] (Sivic & Zisserman, 2003): web page demo here [ox.ac.uk]

                * Names and Faces in the News [columbia.edu] (Berg et al, 2004)

                * FACERET: An Interactive Face Retrieval System Based on Self-Organizing Maps [springerlink.com] (Ruiz-del-Solar et al, 2002)

                * Costume: A New Feature for Automatic Video Content Indexing [www.irit.fr] (Jaffre 2005)
                  One more: automatic film character retrieval (Score:4, Informative)
                  by FleaPlus (6935) on Wednesday May 04, @09:39PM (#12437636)
                  I forgot one more, where specific faces were automatically retrieved from feature-length movies and Fawlty Towers:

                  Automatic Face Recognition for Film Character Retrieval in Feature-Length Films [cam.ac.uk] (Arandjelovic & Zisserman, 2005)

                  The objective of this work is to recognize all the frontal faces of a character in the closed world of a movie or situation comedy, given a small number of query faces. This is challenging because faces in a feature-length film are relatively uncontrolled with a wide variability of scale, pose, illumination, and expressions, and also may be partially occluded. We develop a recognition method based on a cascade of processing steps that normalize for the effects of the changing imaging environment. In particular there are three areas of novelty: (i) we suppress the background surrounding the face, enabling the maximum area of the face to be retained for recognition rather than a subset; (ii) we include a pose refinement step to optimize the registration between the test image and face exemplar; and (iii) we use robust distance to a sub-space to allow for partial occlusion and expression change. The method is applied and evaluated on several feature length films. It is demonstrated that high recall rates (over 92%) can be achieved whilst maintaining good precision (over 93%).
                  [ Parent ]
                  • Re:Some relevant research papers by XSforMe (Score:2) Wednesday May 04, @10:17PM
                  • that first one is so pointless by QuantumG (Score:2) Wednesday May 04, @11:11PM
                    • Re:Some relevant research papers by Senor_Programmer (Score:1) Thursday May 05, @07:04AM
                      • Re:Some relevant research papers by bill_mcgonigle (Score:2) Friday May 06, @10:48AM
                        and then we have reverse "Googling" for images.. (Score:5, Interesting)
                        by dotpavan (829804) <i.me.myself.pavan@nOSPAm.gmail.com> on Wednesday May 04, @09:45PM (#12437669)
                        (http://www.slashdot.org/~dotpavan)
                        Here is a google game which is reverse of google's image search:

                        One has to guess the search word which generated a given set of 20 images in google's image search [robinson.name]

                        When things are moving forward, we have soomthing to talk about "those good ole days" but frankly the game is interesting initially but later gets boring due to the frequent repetitions..

                        Is it just colour? (Score:5, Interesting)
                        by Bifurcati (699683) on Wednesday May 04, @09:47PM (#12437676)
                        (http://www.illuminatingscience.org/)
                        I just did a quick search based on this [designer.am] image of a Qantas logo (that's the main Australian airline, in case you're wondering...) It's red, with a white kangaroo in the middle. My theoretical aim was to find photos of Qantas planes.

                        What I got was an awful lot of red planes - some of which were actually Qantas planes, but I think more by coincidence (i.e., they're red) than design. Many images had nothing to do with Qantas, or even a red plane - they simply had a lot of red in the image.

                        This is impressive in some ways, but in others it seems like it's simply looking for similar patches of colour. I haven't done enough testing to see what happens if,say, I gave it a half red half green image.

                        Interesting, but not ready for public consumption just yet. A bit like A.L.I.C.E. the artifial intelligence system actually - neat, but not practical. Yet!

                        Great! (Score:5, Funny)
                        by SetupWeasel (54062) on Wednesday May 04, @09:50PM (#12437700)
                        (http://www.ministry-of-fun.com/)
                        Now I can find all the other naked pictures of Bea Arthur on the web!
                          eVision? (Score:1)
                          by Polarweasel (33867) on Wednesday May 04, @09:50PM (#12437701)
                          (http://polarweasel.org/)

                          I guess eVision [evisionglobal.com] were just too early to market with their visual search engine. Here's a demo or three [evisionglobal.com] of eVe in action.

                          It sure was cool, just too far ahead of its time...

                            Several image viewers do this already (Score:1)
                            by Ezza (413609) on Wednesday May 04, @09:51PM (#12437703)
                            Programs like GQview (unix/linux) offer functions to search for similar images, mainly used to find duplicates.

                            It's not quite "put in an image and find me all the similar ones" but the underlying technology is the same, usually creating some kind of "signature" of each image and then comparing the signatures to find others visually similar.

                            Great for de-duplicating your por^M^M^MPhoto Collection.
                              IP Enforcement Nightmare (Score:2, Interesting)
                              by buckhead_buddy (186384) on Wednesday May 04, @09:54PM (#12437718)
                              The Mona Lisa (famous and out of copyright) is often plagarized in whole or in part as part of commercial or satiric artistic works. These types of visual database engines have frequently been explained to me as being able to input the Mona Lisa and get a list of images that used the entirety of the image or just a part (such as the highly-praised subtle smile).

                              The big problem to me is specifying input. I know the "look" of the Mona Lisa's smile, but even with the best pen input methods I'd never be able to mimic DaVinci's subtle emotion of the smile; my hands just aren't capable of doing so. Using photos of the painting could simplify this, but this almost assumes that I'm only looking for the parody's and commercial exploiters of the image rather than the image itself (after all, I have the image to start with). And it raises the further issue that many photographic reproductions of the Mona Lisa that I can get my hands on are still under copyright and I'd be doing something legally questionable with an image long in the public domain.

                              Add to this the "infinite number of monkeys" issue where legally litigious companies will use technologies like this to scan the internet for litigation targets. Imagine Disney using a cell of Rafiki from the Lion King to find legally similar images that were created after the Lion King was released even if they were only superficially similar. Now do this for all movies back to Snow White or Steamboat WIlly and you could get to be a real visual mob boss with ownership (or at least threat of litigation) over huge libraries of works that weren't even created to intentionally violate Disney "Intellectual Property".

                              My need for this technology is small considering the input problems I'd have with my artistic abilities, while the litigation nightmare from large databases of "similar" visual data would seem to be more bothersome than helpful. I rather hope these visual search and categorizing methods don't catch on.

                                _Mind at Light Speed_ by David Nolte (Score:1)
                                by Kyrka (20144) on Wednesday May 04, @09:57PM (#12437734)
                                I've not finished it, but I started a book called "Mind at Light Speed" by David Nolte a while back. He describes three stages of machines of light, and I can't do the book justice here.

                                However, he put forth the concept of replacing the bit as the common unit of data with actual images - best described as holographic images of light manipulated by light. A picture really _would_ be worth a thousand words in such a system!

                                  He was my professor, it sucked. (Score:1)
                                  by dalamarian (741404) on Wednesday May 04, @10:05PM (#12437774)
                                  Yeah, in my early undergrad days I took a 200 level course for a gen ed requirement of discrete mathmatics. Wang was the professor, and too this day I haven't had a course that was as difficult or completely freaking insane as the one he gave. Glad he is doing more research and less teaching.
                                  • 1 reply beneath your current threshold.
                                  This reminds me of Gibson's Pattern Recognition (Score:3, Interesting)
                                  by WhiteDragon (4556) on Wednesday May 04, @10:05PM (#12437775)
                                  (https://dawgchain.at/ | Last Journal: Thursday September 23, @04:40PM)
                                  Pattern Rcognition [williamgibsonbooks.com] is a novel by William Gibson, basically set in the present day or very near future. Image based search plays a central role in the plot. It's a very good read.
                                    How does it do that? (Score:3, Funny)
                                    by exp(pi*sqrt(163)) (613870) on Wednesday May 04, @10:07PM (#12437792)
                                    (Last Journal: Monday January 06, @11:36PM)
                                    I was looking at a picture of a plane on that web site and there was a link that said "Click for similar images". And what do you know? It brought up more pictures of planes. This is amazing stuff. How did it understand that I was looking at a picture of a plane?
                                      more sophisticated than colour matching (Score:1)
                                      by jtangen (861406) on Wednesday May 04, @10:08PM (#12437793)
                                      Whatever algorithm they're using, it seems to be sensitve to the horizon line, colour, shading, orientation of the aircraft, etc. It seems to be operating at the level of a pigeon (who have been shown to discriminate photos depicting trees, water, and particular people - as well as art by Picasso and Monet. See http://www.pigeon.psy.tufts.edu/avc/huber [tufts.edu] for other examples. It will be some time before algorithms can match on the basis of model numbers and such. It took humans quite a while to evolve a cortex to enable such fine discriminations.
                                      There is a GNU project related to this GIFT (Score:5, Interesting)
                                      by capedgirardeau (531367) on Wednesday May 04, @10:43PM (#12438007)
                                      From gnu.org:

                                      The GIFT (the GNU Image-Finding Tool) is a Content Based Image Retrieval System (CBIRS). It enables you to do Query By Example on images, giving you the opportunity to improve query results by relevance feedback. For processing your queries the program relies entirely on the content of the images, freeing you from the need to annotate all images before querying the collection.

                                      GIFT [gnu.org] It worked pretty well for me in the demos they linked too. I have been waiting for this type of application to gain momentum.

                                      Sorry, someone else is currently using the system. (Score:1)
                                      by Ninwa (583633) <jbleau@gmail.com> on Wednesday May 04, @10:46PM (#12438030)
                                      (http://www.ninwa.net/)
                                      Got this error when testing out the system, they certainly need to fix this, especially if they want it to become popular.
                                        Would it work for animated .gifs? (Score:2, Interesting)
                                        by Tzarius (688342) <rtzarius@gmail.com> on Wednesday May 04, @10:48PM (#12438038)
                                        (http://del.icio.us/tzarius | Last Journal: Sunday May 22, @11:25PM)
                                        'Coz I'm looking for more information on this image. [pacific.net.au]

                                        It says "multi lock on" and a date, but all Google reports is other forum posts looking for the creator of the image. Apparently, there's a high-res version of it too.
                                        Apps Already do this. (Score:1)
                                        by AgNO3 (878843) on Wednesday May 04, @11:07PM (#12438148)
                                        (http://www.rslittle.com/)
                                        iView Media pro already does this. You just tell it to find dupes and set the tolerence to loose. http://www.iview-multimedia.com/ [iview-multimedia.com]
                                          It's db of airplane pictures, they are all similar (Score:1)
                                          by skeptictank (841287) on Wednesday May 04, @11:14PM (#12438185)
                                          I clicked on the search for similar pictures link for a 727 and one of the results was a helicopter.

                                          I guess anyone can get a research grant these days.

                                            Pretty controllable test (Score:2)
                                            by omahajim (723760) on Wednesday May 04, @11:19PM (#12438214)
                                            A.net is very stringent on the photos they accept. You can submit hundreds of photos, and get rejected for such things as 'badmotive' (a runway sign blocking a single tire), very mildly soft focus, and lots of other pretty anal things (IMHO). So while the image count they are dealing with is high, the obvious resulting similarity among images will result in a high number of matches.

                                            Now, do this for something like Google Images or PBase or collections spanning infinite numbers of subjects and image sizes, then I'll get excited.

                                            No, I've never had a rejection from A.net, I've never submitted there. Two minutes in their forum will tell you how anal their 'screeners' are, for whatever reason. It's just freakin' pictures of airplanes, for chrissakes.

                                            You can tell pretty quickly how they match (Score:1)
                                            by skeptictank (841287) on Wednesday May 04, @11:25PM (#12438237)
                                            They are doing it based upon the shades of color in the image. So if you query for a image of an aircraft in flight with a lot of white clouds behind it, you get more of the same, but you also get aircraft parked on snow-covered ground.
                                              Music (Score:1)
                                              by sn0wflake (592745) on Wednesday May 04, @11:27PM (#12438251)
                                              (http://home20.inet.tele.dk/sn0wflake/ | Last Journal: Monday August 16, @06:13AM)
                                              So when can we search for music? I'm trying to find this song that goes like "dah daahh, dumpiti, dum dum"? Any answers?
                                              • Re:Music by Famanoran (Score:1) Thursday May 05, @02:41AM
                                                • 1 reply beneath your current threshold.
                                                Spotlight (Score:1)
                                                by fr3nch13 (675181) <fr3nch13&yahoo,com> on Wednesday May 04, @11:29PM (#12438258)
                                                Imagine if this was incorporated into Spotlight, not having to rename your entire photo library.
                                                  Is this a joke? (Score:3, Interesting)
                                                  by Daikiki (227620) <daikikiNO@SPAMwanadoo.nl> on Wednesday May 04, @11:38PM (#12438320)
                                                  (http://slashdot.org/ | Last Journal: Sunday March 23, @01:46AM)
                                                  I've tried two different images of airplanes; one of a bright red flying car on bright green grass and one of SpaceShip One against a deep blue sky. Both times, the results looked surprisingly like my query images in color composition only. Red planes on grass and white planes against a blue sky. Inauspicious start.

                                                  Next experiment: I took a picture of a highly distinctive plane, a harrier, climbing at a steep angle and viewed in profile. I got, in return, a list of passenger jets, and even a helicopter. Hardly surprisingly, all of the result pictures had the same bluish white sky as my original image. That was literally the only similarity.

                                                  According to the introduction on the search page the heuristics used compares colors, contrast and shapes in the images themselves. I saw no correlation whatsoever between shapes, and any correlation in contrast seems to be to be the result of the search engine simply looking for images that contain the same colors in a similar ratio to the original. In short, nothing to see here, move along.

                                                  On the other hand, one of the projects listed under the Penn State University link looks fairly fascinating. The Riemann a-LIP project [psu.edu] (automatic linguistic indexing of pictures) doesn't allow user input of images, unfortunately, but it does show some fairly fascinating attempts at verbally qualifying image data. For example, it describes a blue and orange mandelbrot as pattern agate shimer abstract scene, and a sunset over a lake as Berlin Devon Namibia landscape lake scene. Okay, it may still need some work, but it sure beats the hell out of the "find the same color airplane engine".
                                                  Oh, you mean like imgseek? (Score:4, Informative)
                                                  by mr_zorg (259994) on Wednesday May 04, @11:47PM (#12438377)
                                                  Oh, you mean like imgseek [python-hosting.com]?
                                                    Requirements (Score:1)
                                                    by zxflash (773348) on Thursday May 05, @12:35AM (#12438605)
                                                    (http://www.solidz.com/)
                                                    If image search is going to be the next big thing bots are going to need to crawl deeper and more often... Most engines probably don't have the resources currently to support the extra space/load.
                                                      But what I don't understand is... (Score:1)
                                                      by triffidsting (594096) on Thursday May 05, @12:41AM (#12438639)
                                                      ...why does it show me pictures of donkeys when I use my school portrait for the search?
                                                      Oracle interMEDIA does this for a while now? (Score:1)
                                                      by baziel (822372) on Thursday May 05, @12:48AM (#12438659)
                                                      see http://www.oracle.com/technology/products/intermed ia/index.html [oracle.com] (oracle)
                                                      This document was written in 2002 (and that version is as old too)
                                                      I can remember some sales guy saying "You can look for a couch that's like this one. But blue."
                                                      So that means it should do a bit more than just color patches..
                                                      and then there is Google images..
                                                        This is similar (Score:1)
                                                        by cancer4xmas (666669) * on Thursday May 05, @01:07AM (#12438742)
                                                        This Photo ID has not been indexed in the similarity database yet.
                                                          ummm... (Score:1)
                                                          by Baseclass (785652) on Thursday May 05, @01:16AM (#12438771)
                                                          It just keeps coming up with photos of airplanes.
                                                            Purdue University's 3D Shape Search (Score:2, Informative)
                                                            by karthik_r085 (775682) on Thursday May 05, @01:20AM (#12438781)
                                                            Purdue also has a 3D shape search. More can be found at Here [slashdot.org].
                                                              altavista (Score:2, Informative)
                                                              by heatdeath (217147) on Thursday May 05, @01:40AM (#12438856)
                                                              (http://bengarrison.net/)
                                                              I'm pretty sure altavista had this feature several years ago, but removed it. I remember that it worked fairly well. Does anyone know what happened to it?
                                                              Professor Wang (Score:1)
                                                              by dampjam (779525) on Thursday May 05, @02:07AM (#12438945)
                                                              I took Professor Wang's networking class this Semester and have him for databases in the fall. He is a very interesting professor and was in the same class as the people who invented google.
                                                              • 1 reply beneath your current threshold.
                                                              Uh-oh. (Score:2)
                                                              by Shag (3737) <<danbirchall+slashdot> <at> <gmail.com>> on Thursday May 05, @03:32AM (#12439230)
                                                              (http://danbirchall.multiply.com/)
                                                              Given that it apparently has trouble telling the difference between an SR-71, a Chinook helicopter and JFK airport's formerly-TWA terminal [airliners.net], among other things... I think it could use some tweaking.

                                                              Of course, given the usual course of things, it will instead be deployed at JFK's formerly-TWA terminal, assigned facial recognition tasks, and immediately declare everyone to be among the 10-most-wanted terrorists. I can't wait.

                                                                How about just searching for an identical image? (Score:1)
                                                                by DavidD_CA (750156) on Thursday May 05, @04:08AM (#12439343)
                                                                (http://home.happyface.net/)
                                                                This is great and all, but I would love for a way to upload a graphic file to a search engine and have it report back any identical (or nearly identical.. perhaps a "threshold" setting) copies on the web.

                                                                This would be useful to me as a photographer to see if anyone out there is using my photos.
                                                                  Worked on something similar (Score:3, Informative)
                                                                  by Sirch (82595) on Thursday May 05, @05:28AM (#12439533)
                                                                  (http://www.sirch.co.uk/ | Last Journal: Friday September 19, @05:30PM)
                                                                  About a year or so ago, I and three other Masters students worked on a similar project at the University of Southampton.

                                                                  I've not RTFA (not had the time), but our approach was to split the images into segments (based on colour and texture) which were assumed to be objects. The segments would then be analyzed for various feature vectors, such as shape, texture, colour etc. These vectors would then be added into a database of numbers, and finally the segments grouped, giving a collection of classified sections which (hopefully) represent similar objects.

                                                                  From related metadata such as keywords, you could then hope to build up an idea of what keyword matches which section. You could also come up with a relevance between two images, and thus search for similar images.

                                                                  We didn't have enough time to make it bulletproof by any means, but our limited results were very promising.

                                                                  Sorry I can't find the paper, but we've got some screenshots of the application here [soton.ac.uk] and here [soton.ac.uk] (you can see false colouring applied to the original image to display the segments)
                                                                  Reee diii cuuuu looooooooous ! (Score:2)
                                                                  by Ancient_Hacker (751168) on Thursday May 05, @08:55AM (#12440401)
                                                                  Ridiculous.

                                                                  Image search will kinda work for airplanes in this database,as there are a very limited set of airplane model numbers, which are going to be attached to each photo.

                                                                  But if the database didnt have these text clues the image search is going to be unlikely to see the similarity between an 747 in the air, as seen from the ground, with a head-on view of a 747, or one at the gate, or one in a hangar, or one in twilight, or one of a different color.

                                                                  Maybe it could be done by outsourcing the task to India?

                                                                    Too Literal (Score:2)
                                                                    by suwain_2 (260792) on Thursday May 05, @09:27AM (#12440612)
                                                                    (Last Journal: Tuesday August 28, @08:17AM)
                                                                    I was trying this out a bit, and have to admit that it's cool that something like this exists at all.

                                                                    However, I think it would be better if it were able to realize what the 'background' was and filter it out. (Though I couldn't begin to guess how you'd do this.)

                                                                    For example, I searched for this image [airliners.net]. Many of the results [airliners.net] are of something completely different, such as a white jet. Which is nothing like a camo helicopter. But the sky and the ground are pretty similar, and I think that's how it's matching.

                                                                    It's incredible that we got this far, but I think there's still a long ways to go before it's at the stage where you put in an image, and are awed at how quickly it works.

                                                                    Also note that I'd tend to think this would exponentially more difficult than searching HTML files, so it might be much more expensive to implement large-scale.
                                                                      CBIR is the same problem as AI (Score:2)
                                                                      by HuguesT (84078) on Thursday May 05, @12:52PM (#12442657)
                                                                      In general image understanding is equivalent to general AI. We won't get a CBIR system that works well before we get an AI that works and vice versa, because people expect to be able to match the *content* of the image they submit as template and not the general appearance of the image. The problem is then too unspecified.

                                                                      Even in the restricted context of aeroplanes this is not a trivial problem. Someone in the list of replies submitted an image of a warthog (A-10) and got nonsensical results. Somehow the CBIR system would need to be able to infer a model of the A-10 from a given random 2D projection, and match it to the other 2D projections of the same A-10 model that they do have in the DB. This doesn't sound impossible but it is hard, and I suspect the Penn State people didn't do that. Instead they are probably matching on colour, texture, general appearance, etc.

                                                                      This is not to say that CBIR is not a nice problem to apply new image processing/image analysis algorithms to, which are developed all the time.
                                                                        Video loops etc (Score:1)
                                                                        by AaronCampbell (826767) on Thursday May 05, @03:36PM (#12444634)
                                                                        When this becomes a little more refined, it will be nice to apply to frames of video to assist in creating video loops. For example, you mount a camera near the coast, and get 3 hours of footage of the waves crashing against the rocks. You want to make a nice loop of that, which can play indefinitely, but you want to avoid that 'jump' that always seems to occur. Find small sections of the movie where say...10 frames in a row are similar to 10 other frames... It'll take out a lot of the usual pain.
                                                                          Re:wtf? (Score:5, Interesting)
                                                                          by Rei (128717) on Wednesday May 04, @09:07PM (#12437442)
                                                                          (http://www.cursor.org/)
                                                                          Because it still has problems - you'll note that the pictures seem to be compared simply based on color similarity. That's the same thing imgSeek [python-hosting.com] does (a great program for sorting and searching your photos) on photo searches. It works wonderfully if you're searching a very limited picture subset (say, airplanes), but if you search a wide variety of pictures, the results can be quite amusing.
                                                                          [ Parent ]
                                                                          • Re:wtf? by Xcott Craver (Score:2) Wednesday May 04, @09:43PM
                                                                            • Been going on for YEARS... by FatSean (Score:1) Thursday May 05, @12:03AM
                                                                              • Re:wtf? by asadsalm (Score:1) Thursday May 05, @01:05AM
                                                                                • Re:wtf? / CBIR 101 by kd4evr (Score:1) Thursday May 05, @02:36AM
                                                                                  • Re:wtf? by bcmm (Score:2) Thursday May 05, @05:25AM
                                                                                    • Re:wtf? by bcmm (Score:2) Thursday May 05, @05:28AM
                                                                                      • Re:wtf? by bcmm (Score:2) Thursday May 05, @05:32AM
                                                                                      Re:wtf? (Score:1, Funny)
                                                                                      by merpal (873013) on Wednesday May 04, @10:01PM (#12437754)
                                                                                      don't mod me as redundant you niggers
                                                                                      [ Parent ]
                                                                                        Re:wtf? (Score:3, Interesting)
                                                                                        by chefmonkey (140671) on Wednesday May 04, @11:15PM (#12438189)
                                                                                        Google actually did take this technology and try it. The first version of their image search had a "find similar" link next to every image. These tended to work okay at first (they weren't great, but you usually got enough photos back that you could visually scan them and find something of interest that was related to the original image). After a few months, for some reason, the "find similar" links started returning increasingly nonsensical results. After it degenerated to the point of near uselessness, they took the "find similar" link away from the image search results. I expected it to turn up again once they got the kinks worked out, but apparently they just decided to stop working on it.
                                                                                        [ Parent ]
                                                                                        • 26 replies beneath your current threshold.
                                                                                        •  
                                                                                            "Do not meddle in the affairs of wizards, for you are crunchy and good with ketchup."
                                                                                           All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest 1997-2005 OSTG.
                                                                                          [ home | awards | contribute story | older articles | OSTG | advertise | about | terms of service | privacy | faq | rss ]