Projects > Security > Imagination

click to enter the IMAGINATION demo system

Picture-based CAPTCHAs

CAPTCHA = Completely Automated Public Turing test
to tell Computers and Humans Apart

Demonstration :: Realistic Alternative

Natural image based CAPTCHAs - try the IMAGINATION demonstration (test it by yourself)

Other Info: user study; presentation slides [.PPT]


When looking at the image above, are you able to discern where one image begins and another ends? Are you able to tell the semantic content of the images? Since computers look at the pixels of an image, this is a very difficult task and something that separates humans from computers. Many Websites are currently using the text-based CAPTCHA technologies to ensure that the user filling out a form is indeed human by requesting that the user input a set of numbers and letters from a distorted image. Some examples of these sites are listed in the Project Background section below. Text-based systems have been defeated by computer vision and especially shape matching algorithms. In this project, we aim to develop image-based CAPTCHAs that only human beings can solve. We call the project IMAGINATION because we want to leverage the endowed imagination power of humans.

Project Personnel

This project is being conducted within the research group of Prof. James Z. Wang and Prof. Jia Li at The Pennsylvania State University. Ritendra Datta and Dhiraj Joshi are the main researchers.


Ritendra Datta, Jia Li and James Z. Wang, ``IMAGINATION: A Robust Image-based CAPTCHA Generation System,'' Proceedings of the ACM Multimedia Conference, pp. 331-334, Singapore, ACM, November 2005. (download)

Project Background

Example websites

Websites which use text-based CAPTCHAs to prevent denial--of-service attacks or mass-scale registration

  1. Yahoo! Mail - Sign Up
  2. PayPal - Sign Up.
  3. MSN Hotmail account Sign Up.
  4. Domain name owner search at
  5. Search for availability of tickets at Ticketmaster (including tickets to Penn State football games !)

Proposed Security Protocols which rely on CAPTCHAs

Countering Distributed Denial-of-service attacks: W. G. Morein, A. Stavrou, D. L. Cook, A. D. Keromytis, V. Mishra, and D. Rubenstein, "Using Graphic Turing Tests To Counter Automated DDoS Attacks Against Web Servers," Proc. ACM Conference on Communications and Computer Security, 2003. [PDF]

Preventing attacks on user-defined passwords: B. Pinkas and T. Sander, "Securing Passwords Against Dictionary Attacks,"Proc. ACM Conference on Communications and Computer Security, 2002. [PDF]

Related Publications :: Breaking conventional text-based CAPTCHAs

[A] Relevant Publications

  1. K. Chellapilla and P. Y. Simard, "Using Machine Learning to Break Visual Human Interaction Proofs (HIPs)," Proc. NIPS, 2004. [PDF].
  2. G. Moy, N. Jones, C. Harkless, and R. Potter, "Distortion Estimation Techniques in Solving Visual CAPTCHAs," Proc. IEEE CVPR, 2004. [PDF]
  3. G. Mori and J. Malik, "Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA," Proc. CVPR, 2003. [PDF]
  4. A. Thayananthan, B. Stenger, P. H. S. Torr, and R. Cipolla, "Shape Context and Chamfer Matching in Cluttered Scenes," Proc. IEEE CVPR, 2003. [PDF]

[B] Public-domain resources on breaking CAPTCHAs

  1. PWNtcha - captcha decoder
  2. The OCR Research Team (commercial).

More Information

  1. Research Group
  2. Related media annotation projects done by this group
  3. Carnegie-Melon Universitiy's CAPTCHA project page.
  4. A Wikipedia reference page on CAPTCHAs.