Reverse image search

Reverse image search using Google Images.

Reverse image search is a content-based image retrieval (CBIR) query technique that involves providing the CBIR system with a sample image that it will then base its search upon; in terms of information retrieval, the sample image is what formulates a search query. In particular, reverse image search is characterized by a lack of search terms. This effectively removes the need for a user to guess at keywords or terms that may or may not return a correct result. Reverse image search also allows users to discover content that is related to a specific sample image,[1] popularity of an image, and discover manipulated versions and derivative works.[2][3]

Uses

Reverse image search may be used to:[4][5]

  • Locate the source of an image
  • Find higher resolution versions
  • Discover webpages where the image appears
  • Track down the content creator
  • Get information about an image

Algorithms

Commonly used reverse image search algorithms include:[6]

Google Images

Google's Search by image is a feature that utilizes reverse image search and allows users to search for related images just by uploading an image or image URL. Google accomplishes this by analyzing the submitted picture and constructing a mathematical model of it using advanced algorithms.[8] It is then compared with billions of other images in Google's databases before returning matching and similar results. It should be noted that when available, Google also uses metadata about the image such as description.

TinEye

TinEye is a search engine specializing in reverse image search. Upon submitting an image, TinEye creates a "unique and compact digital signature or fingerprint" of said image and matches it with other indexed images.[9] This procedure is able to match even heavily edited versions of the submitted image, but will not usually return similar images in the results.[10]

eBay

eBay ShopBot uses reverse image search to find products by a user uploaded photo. eBay uses a ResNet-50 network for category recognition, image hashes are stored in Google Bigtable; Apache Spark jobs are run in Google Cloud Dataproc for image hash extraction; and the image ranking service is deployed by Kubernetes. [11]

SK Planet

SK Planet uses reverse image search to find related fashion items on its e-commerce website. It developed the vision encoder network based on the TensorFlow inception-v3, with speed of convergence and generalization for production usage. A recurrent neural network is used for multi-class classification, and fashion-product region-of interest detection is based on Faster R-CNN. SK Planet's reverse image search system is built in under 100 man-months.[12]

Pixsy

Pixsy reverse image search technology detects image matches[13] on the public internet for images uploaded to the Pixsy platform[14]. New matches are automatically detected and alerts send to the user. For unauthorised use, Pixsy offers a compensation recovery service[15][16] for commercial use of the image owners work. Pixsy partners with over 25 law firms and attorneys around the world to bring resolution for copyright infringement.

Pinterest

Pinterest acquired startup company VisualGraph in 2014 and introduced visual search on its platform.[17] In 2015, Pinterest published a paper at the ACM Conference on Knowledge Discovery and Data Mining conference and disclosed the architecture of the system. The pipeline uses Apache Hadoop, the open-source Caffe convolutional neural network framework, Cascading for batch processing, PinLater for messaging, and Apache HBase for storage. Image characteristics, including local features, deep features, salient color signatures and salient pixels are extracted from user uploads. The system runs on Amazon EC2, and only requires a cluster of 5 GPU instances to handle daily image uploads onto Pinterest. By using reverse image search, Pinterest is able to extract visual features from fashion objects (e.g. shoes, dress, glasses, bag, watch, pants, shorts, bikini, earrings) and offer product recommendations that look similar.[18][19]

Research systems

Microsoft Research Asia's Beijing Lab published a paper in the Proceedings of the IEEE on the Arista-SS (Similar Search) and the Arista-DS (Duplicate Search) systems. Arista-DS only performs duplicate search algorithms such as principal component analysis on global image features to lower computational and memory costs. Arista-DS is able to perform duplicate search on 2 billion images with 10 servers but with the trade-off of not detecting near duplicates.[20][21]

Production reverse image search systems

See also

References

  1. ↑ "How to search by image". Google. Retrieved 2 November 2013.
  2. ↑ "Find Your Images Online Using Reverse Image Search on Google". PhotoShelter Blog.
  3. ↑ "Video searching with Frompo". Frompo.com. Retrieved 2 November 2013.
  4. ↑ "MAKING A MARK". makingamark.blogspot.com.
  5. ↑ "FAQ - TinEye - Why use TinEye?". TinEye.
  6. ↑ Bundling Features for Large Scale Partial-DuplicateWeb Image Search Microsoft.
  7. ↑ A New Web Image Searching Engine by Using SIFT Algorithm computer.org
  8. ↑ "How does Google's reverse image search work? - Quora". quora.com.
  9. ↑ "FAQ - TinEye - How does TinEye work?". TinEye.
  10. ↑ "FAQ - TinEye - Can TinEye find similar images??". TinEye.
  11. ↑ "Visual Search at eBay". acm.org.
  12. ↑ Visual Fashion-Product Search at SK Planet
  13. ↑ "Find stolen images - Pixsy". Pixsy. Retrieved 2017-10-20.
  14. ↑ "Pixsy.com review: Find & Fight Image Theft - Online Marketing for Artists -". Online Marketing for Artists. 2015-07-02. Retrieved 2017-10-20.
  15. ↑ https://plus.google.com/+SteveSchlackman (2014-10-18). "Pixsy: Find and Get Paid for Image Theft". artlawjournal.com. Retrieved 2017-10-20.
  16. ↑ "Resolve image theft - Pixsy". Pixsy. Retrieved 2017-10-20.
  17. ↑ Josh Constine. "Pinterest Acquires Image Recognition And Visual Search Startup VisualGraph". TechCrunch. AOL.
  18. ↑ "Visual Search at Pinterest". acm.org.
  19. ↑ "Building a scalable machine vision pipeline". Pinterest Engineering.
  20. ↑ Duplicate-Search-Based Image Annotation Using Web-Scale Data Microsoft.
  21. ↑ "Arista - Dr. Xin-Jing Wang's Homepage". Google.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.