Pranav Mistry Sixth Sense

About Sixth Sense :

'SixthSense' is a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

We've evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly all available online. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no link between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. SixthSense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures. ‘SixthSense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer. 

The SixthSense prototype is comprised of a pocket projector, a mirror and a camera. The hardware components are coupled in a pendant like mobile wearable device. Both the projector and the camera are connected to the mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks user's hand gestures and physical objects using computer-vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tip of the user’s fingers using simple computer-vision techniques. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus SixthSense also supports multi-touch and multi-user interaction. 

The SixthSense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system. The map application lets the user navigate a map displayed on a nearby surface using hand gestures, similar to gestures supported by Multi-Touch based systems, letting the user zoom in, zoom out or pan using intuitive hand movements. The drawing application lets the user draw on any surface by tracking the fingertip movements of the user’s index finger. SixthSense also recognizes user’s freehand gestures (postures). For example, the SixthSense system implements a gestural camera that takes photos of the scene the user is looking at by detecting the ‘framing’ gesture. The user can stop by any surface or wall and flick through the photos he/she has taken. SixthSense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions. For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail. The SixthSense system also augments physical objects the user is interacting with by projecting more information about these objects projected on them. For example, a newspaper can show live video news or dynamic information can be provided on a regular piece of paper. The gesture of drawing a circle on the user’s wrist projects an analog watch. 

The current prototype system costs approximate $350 to build. Instructions on how to make your own prototype device can be found here (coming soon). 

Download :

      

    Download the full source code of this project free   Download

PICTURES

                   
                  
                 
                 

                   
 

  . . . some more pictures are coming soon. 

PUBLICATIONS

• P. Mistry, P. Maes. SixthSense – A Wearable Gestural Interface. In the Proceedings of SIGGRAPH Asia 2009, Sketch. Yokohama, Japan. 2009
• P. Mistry, P. Maes. SixthSense – A Wearable Gestural Interface. In the Proceedings of SIGGRAPH Asia 2009, Emerging Technologies. Yokohama, Japan. 2009
• P. Mistry. The thrilling potential of SixthSense technology. TEDIndia 2009. Mysore, India 2009
• P. Mistry, P. Maes, L. Chang. WUW - Wear Ur World - A Wearable Gestural Interface. In the CHI '09 extended abstracts on Human factors in computing systems. Boston, USA. 2009
• P. Maes, P. Mistry. Unveiling the "Sixth Sense," game-changing wearable tech. TED 2009. Long Beach, CA, USA 2009

AWARDS

   SixthSense wins  2009 INVENTION AWARD  by Popular Science

     TR35 2009 Young Inventor Under 35 Award  by Technology Review

PRESS COVERAGE

•  WIRED News   MIT Students Turn Internet Into a Sixth Human Sense
•  FOX News   MIT Team Shows 'Sixth Sense' Wearable Computer
•  BBC   SixthSense blurs digital and the real
•  The Sydney Morning Herald   MIT researchers make 'sixth sense' gadget
•  The Inquirer   MIT makes sixth sense gadget
•  Mumbai Mirror   Wear ur world
•  THE TIMES OF INDIA   'Sixth sense' gizmo makes sci-fi a reality
•  Gizmodo   Sixth Sense Technology May Change How We Look at the World Forever
•  Fast Company   MIT's Sixth Sense Machine Makes Reality Better
•  Slashdot   MIT Researchers Create a Cheap "6th Sense" Device
•  engadget   MIT's "sixth sense" augmented reality device demonstrated on video
•  engadget   MIT concocts wearable "sixth sense" device, Bruce Willis is like "what?"
•  SLASH GEAR   Wearable projection computer project: Internet ’sixth sense’
•  World Changing   Pattie Maes Invents a Sixth Sense
•  SOFTPEDIA   Next-Gen 'Sixth Sense' Device Created at MIT
•  Gearlog   The Internet as an Extension of Self
•  DVICE.com   Minority Report-like wearable interface gives you a sixth sense
•  MAXIMUMPC   TED 2009: The Sixth Sense is Actually a Minority Report-like Internet Interface
•  redOrbit   MIT Develops Device That Offers 'Sixth Sense'
•  AFP   MIT researchers make 'sixth sense' gadget
•  TMC   MIT Researchers Unveil Computing Device that Turns Any Surface into a Touchscreen
•  betanews   MIT develops an even more augmented reality device
•  GEEK.com   MIT develop Wear Ur World, 6th sense hand gesture gadget
•  GIGAOM   MIT develop Wear Ur World, 6th sense hand gesture gadget
•  Crunch Gear   Video: That MIT “sixth sense” device in action
•  GottaBeMobile.com   MIT Student Doing Amazing “Sixth Sense” Work
•  sina.com   美推出可穿戴电脑将网络变成人类第六感
•  io9.com   MIT Student Turns His Body Into a Computer
•  Wired Vision   身体と世界と情報を統合：情報をどこにでも投影するウェアラブル・マシン：動画
•  CNET NEWS   MIT's 6th Sense device could trump Apple's multitouch
•  New York Times   At TED, Virtual Worlds Collide With Reality
•  hexun.com   “少数派报告”成为现实 用手指控制虚拟屏(组图)
•  CNET JAPAN   MIT研究グループ、「第6感」ガジェットを開発
•  CNET TAIWAN   MIT的Sixth Sense裝置挑戰蘋果的多重觸控技術
•  news.com.au   MIT's 'Wear Ur World' (WUW) device puts internet literally at your fingertips
•  The Sun   New Minority Report internet
•  The Gazette, CANADA   MIT researchers make 'Minority Report' gadget
•  JOINS NEWS   손가락 컴퓨터가 펼치는 4차원 세계
•  EXPRESS COMPUTER   Windows 7, Netbooks, Kindle 2 and WUW
•  The Tech, MIT   New Invention Gives People 'Sixth Sense'
•  Le Monde, France   La réalité augmentée du bout des doigts
•  Mail Today, India   Indian geek invents 'SIXTH SENSE' gizmo
•  COMPUTERWORLD   Future shock: The PC of 2019
•  computerra, Russia   Все свое носи с собой
•  American Observer   SixthSense Makes the Entire World your Computer
•  Clarin.com   Diseñan la primera computadora que se maneja con gestos
•  Business Standard   Indian IT geek develops digital 'sixth sense' at MIT
•  IndianExpress   The world at your fingertips
•  CORE77   MIT's wearable "Sixth Sense" device turns any surface
•  CIO   MIT Wearable Gadget Gives You Sixth Sense, a la Minority Report
•  PCWORLD   MIT Wearable Gadget Gives You Sixth Sense
•  The Christian Science Monitor   Sixth Sense: A Web you can wear
•  ACM News   MIT Student Develops Wearable Interface That Relies on Hand Gestures
•  CNN IBN   Superstars of Science
•  The Times of India   Gujarati boy rules world's senses
•  Financial Cronicle   The world @ your fingertips
•  CNN IBN   Superstars of Science
•  Sina.com   麻省理工開發出可佩戴式計算機
•  ELTIEMPO.COM   Sexto Sentido Digital