Skip to main content

Tobii wants you to control your PC with just your eyes and we show you how

Tobii eye tracking
Tobii eye tracking (Image credit: Windows Central)

Gaming and even regular user interaction on computers have largely remained unchanged for the last 20 years. Take one display and add a keyboard, mouse or a trackpad and call it a day. Sure, sensitivity and trackpad gestures have added a little, but it's all just riffing on the same tune.

Tobii Tech, however, wants to change all of that. No, they don't want you to give up your favorite keyboard or high-end laser mouse, but they do want you to consider just using your eyes.

Yes, your gaze, literally one of the most fundamental things we all do can now be used to control your PC. This technology is not something coming "later this year" like so many vaporware products we see at CES but it is ready now. We're going to show you how it works and why it may be the coolest thing at CES.

Video hands-on

Watch our video walkthrough of the Tobii EyeX to see eye-tracking on a PC with Windows 10.

Hello, why eye tracking?

Here at CES 2016 we were lucky enough to sit down with the president of Tobii Tech, Oscar Werner, for a hands-on demo as well as a discussion of the technology, their plans, and long-term goals.

The immediate question you may have is why even bother? As it turns out, there is a long history of eye tracking technology used in academic research (I used to use it for reading studies, for instance), but now the technology is being developed to a level where it can be commercialized and mass produced.

And it's not expensive either. The Tobii EyeX is just $139.

Besides being an eye tracking camera, it can be used with Microsoft's Windows Hello too for unlocking your PC just be looking at it. That makes it one of the first readily available options for people who want Hello on their home PC without buying a new display or computer.

But there are some other real-world uses for EyeX right now:

  • Gaming including using your glance to target objects
  • Controlling mouse cursor movement in Windows 10 or Alt-Tab selection
  • Dim Screen: The eye tracker knows if you are looking at the screen or not. When you look away, the device automatically dims the screen - saving power and giving privacy.
  • Stay awake: Imagine leaving your laptop idle while being in a meeting. Today's devices can't anticipate when you would like to perform an action. With Tobii eye tracking, devices can understand that you are in front of the computer and will stay awake until you step away from it.

Like many technologies, EyeX is considered an emerging experience. That means drivers, features, and abilities are not only constantly evolving, but you the end user can influence the technology. Tobii Tech is very interested in getting this technology into people's hands to let them shape where it goes with OS and gaming integration.

Gaming

Currently, Tobii EyeX supports over 30 gaming titles (you can see them here including official support from Ubisoft for Assassin's Creed: Syndicate and user mods for others like Grand Theft Auto V.

The company expects to have 100 titles in the coming months and partnerships with MSI including direct integration of the EyeX technology in MSI's new GT72 Dominator Pro Tobii gaming laptop, suggests that companies are taking notice. Werner informs us that they have had interest from a few companies about using their hardware in future laptops and PCs.

Usage of the technology varies. In Grand Theft Auto V a user can just use their glance to target enemies. The player looks where they want to shoot and simply presses the RT button to fire with no need to use the scene changer joystick for aiming. Likewise, you can even do this while driving any vehicle in GTAV, which means you can be driving in one direction, but shooting at the same time in another.

In a game like Assassin's Creed: Syndicate the eye tracking puts a reticle on objects that you can immediately repel to using your grappling hook. Once gain, no more aiming. You can also auto-tag enemies who for taking out when you are positioned.

Finally, in something like Microsoft Flight Simulator players get a more immersive experience:

"…adjusting the view with the mouse is no longer necessary and you are able to interact with the game more freely and in a more natural way. Look around the cockpit as if you are really there, or lean back and just enjoy the spectacular view of the world from above."

The opportunities here are truly limitless and since EyeX does not replace current input methods, but merely augments them, there are no downsides with the technology.

How is it?

The EyeX device is like a long stick with multiple cameras and a mini USB plug. The kit comes with two magnetic mounts so that you can position it at the bottom of your PC display. Since there are two, you can swap it in and out of multiple computers. The magnet simply keeps it attached to the screen and the USB cable powers it.

Besides installing the PC drivers using EyeX is a Plug-n-Play experience.

The issue with new technology especially ones that alter user input strategies is the learning curve aka re-training and breaking old habits. Xbox owners have had this experience when using Kinect for instance.

Luckily, using Tobii EyeX is very straightforward. For the first time user, a calibration program is run to get the system in line with the user's gaze. From there it was more about trusting the system that it was going to work each time. That part may take a little while to adjust too resulting in a slight hesitation before you inmate the action with your controller, mouse, or keyboard. But once you let yourself go, so to speak, EyeX just works.

The reason for the ease is the technology only extends what is already a natural function: looking at an object of interest. There is nothing else more basic in gaming or using a computer.

Next generation

Although the current EyeX camera can be used on any laptop or PC, the company is already readying the next generation of the device on their IS4 Platform. The newer one is even smaller and it has a proprietary chip on board to handle the eye tracker processing. Much like how a dedicated GPU offloads graphics duties from the main CPU, this new eye-tracking chip – a first of its kind – means your PC will have even less to do when using the hardware.

The reduced footprint of the next generation tracker also means that the technology can be integrated into laptops smaller than 17-inch gaming ones. We saw a prototype for an Ultrabook with the gear built right into the display hinge adding a barely noticeable bump. Any PC manufacturer can use Tobii's technology if they want it.

And yes, if you thought that someday having eye tracking on your phone would be great Tobii's Werner is already ahead of you.

The road ahead

Needless to say, I was very impressed with the Tobii EyeX and the small Swedish company who have built this all on their own. It's not a prototype, but something you can buy right now (we have one in the house for even more demos in the coming weeks). The company is looking for user feedback to take the technology even further and partnerships with game developers and hardware manufacturers to make the technology readily available to the masses.

To editorialize a bit, I think that eye tracking could be the next big thing in PCs. Today, we are already just getting used to Intel's RealSense 3D cameras and Windows Hello. The Tobii EyeX does Windows Hello too, but for gamers and regular users, is much more useful for everyday tasks than just 3D scanning.

My hope is companies like Microsoft and others take notice of Tobii here at CES. Imagine for a second a Surface that not only unlocks your computer but one that dims the display when you are not looking at it (or turns it back on when you do). How about saving your index finger from slowly moving the mouse cursor around the screen all day and instead cut that by ¾ since you can just navigate with your peepers?

Those are jaw-dropping technologies, and you can have it right now. Will the rest of the industry get on board? We'll have to wait and see, but I'm hopeful. Folks, this stuff, is great.

Windows Central will be doing a lot more with Tobii in the coming weeks including creating user-forums, some contests, and regular coverage of EyeX for gamers and more. Let us know what you think in comments!_

Where to buy

If you are interested in Tobii EyeX, you can find more information from their website here including ordering a unit directly from the company for $139 (+$25 for shipping).

Daniel Rubino
Executive Editor

Daniel Rubino is the Executive Editor of Windows Central, head reviewer, podcast co-host, and analyst. He has been covering Microsoft here since 2007, back when this site was called WMExperts (and later Windows Phone Central). His interests include Windows, Microsoft Surface, laptops, next-gen computing, and arguing with people on the internet.

52 Comments
  • I don't need to read the article to say how useful this is really is; especially for handicapable people. Bjorn, a graphics designer I know is wheel chair bound - he cannot walk, he cannot even use his hands due to limited movement yet he is extremely efficient at what he does and that is great amazing pieces of art - all using a pen which he holds with his mouth. He moves the mouse with his face, clicks the buttons with the pen and also operates the keyboard - with the pen. Bloody inspirational. So before anyone says this is redudant, think twice. Edit: read the article, glad tech like this flowing down to the affordable spectrum. Dan, what about selecting and dragging the mouse - for instance can a user say single blink to imitate a left click, blink twice slowly to imitate the double click, blink twice in rapid succession to imitate a right click? Or use individual eyes to simulate the respective left and right buttons?
  • Yup, and a lot of Tobii's work is exactly in that area. Now it's being pushed out for other uses and it's truly remarkable stuff.
  • Awesome and it sure is. A combination of speech and eye tracking will help transform the lives of alot of people.
  • "Dan, what about selecting and dragging the mouse - for instance can a user say single blink to imitate a left click, blink twice slowly to imitate the double click, blink twice in rapid succession to imitate a right click?"
    No, right now it's mostly about cursor movement and a few other tricks (we'll be covering more if it next week, including initial setup, etc.). Part of the issue is Tobii would need deeper OS access to do more with eye tracking as a peripheral, so hopefully MS will take note and maybe work with them.
  • Oh okay, understood. Likewise I too hope Microsoft do take note and start working with them as tech like this literally falls into category of "empowering users to do more" :). As the potential for this huge especially for handicapable people who aren't really that well off financially.
  • Yup! Microsoft should seriously partner up with Tobii to integrate this technology deeply on Windows 10 down to low-level. It would be even better if Kinect-team and Tobii team will share their tech and collaborate with each other for Windows and Xbox integration. This is gonna be a very exciting future of Windows and in PC's in general if they did this and might even become mainstream sooner if they collaborate.
  • It would be a big bonus for the windows ecosystem and maybe also a great selling point for windows (especially mobile)
  • MS needs to buy out Tobii now before google or apple does
  • Are you listening Microsoft? Make this happen now!!
  • Maybe they are that (seemingly rare these days) sort of company that isn't looking for a buyer. Maybe they are passionate about what they do and don't want the man standing on thier neck.
  • Do you realize that people DO BLINK even if they don't want to?
  • Oh crap, just triple blinked while looking at the C: drive. Sent to trash bin. D'oh! Triple blinked while looking at the trash bin :(
    Seriously now, this camera system is the future.
  • There are different speeds and methods of blinking though. Natural blinks can be very different from controlled blinks. With the power here, shouldn't be that hard at all to implement something basic (not hard for them, because they're obviously good at what they do. It still might be hard overall) Posted via the Windows Central App for Android
  • So use slower than natural blinks. This tech is not new at all, the Israeli air force almost used this kind of tech in their helicopters, it was just too expensive at the time. Posted via the Windows Central App for Android
  • Very promising technology at a very reasonable price. As you said hopefully Microsoft takes notices. Interesting to see if projects like this and HoloLens can usher in a new era of eye tracking for traditional computing.
  • If you're looking to buy one, keep in mind that there are a ton of USB driver incompatibilities with this.  For example, if you're connecting to a USB port on an Intel motherboard, it won't work without jumping through a few hoops (in my case, I had to track down modified drivers that would actually install on my hardware).  I'd recommend perusing their developer forums before purchasing one, because this stuff is very common with the EyeX because of the custom libUSB implementation they do.
  • Interesting. Thanks for the heads-up. FWIW it installed just fine on my Surface Book, but will be looking more to use this at home or in the office on a desktop.
  • To be fair, both the Surface Pro 2 and the custom-built gaming PC I tested on are using the same generation chipset and CPU (H87 on the Surface, Z87 on the gaming PC), but from looking into the problem I didn't get the impression that it was all that uncommon. A large number of their developer forum's posts are all about USB connection problems based on drivers, whether Intel, AMD, Renesas, ASMedia, etc.  I was able to get it to work after using the modified drivers found in section D here:  http://www.win-raid.com/t834f25-USB-Drivers-original-and-modded.html​  but it wasn't at all the plug and play experience I was hoping for... and in case anyone ends up needing to do this, I want to stress the additional risk for the Surface Pro (and other similar devices) that if you upgrade the wrong USB driver in device manager, you're basically bricked - can't go into Device Manager to fix the problem with no cover, touchscreen, digitizer, or USB support.
  • Thanks for the info Jhoff80 :).
     
  • This sounds like having that third hand for a mouse you've always wanted, but even better. It could make things like programming which are both typing- and navigation-heavy significantly faster.
  • Kinect 3.0? Wonder how long it'll take Microsoft to take notice and possibly pursue an acquisition?
  • you took the words out of my mind.
  • Beauty of Windows, a whole new dimension of usage/interaction implemented via a simple USB accessory...amazing.
  • Very nice article Windowscentral. Great read.  
  • Microsoft needs to buy Tobii!! Like, right now!!!! Or atleast make an exclusive deal with them!!
  • I have always wished for this to be possible. Technology is really advancing fast.
  • Any blink detection? I imagine detecting 4 blinks in rapid succession could be a useful way to interact.
  • It does have blink detection, but not utilized yet as a function.
  • Wow, I just watched the demo video and you weren't kidding - this is awesome! I have always felt that touchpads and even mice were a pretty unnatural way to control something that we do naturally with our eyes. I think you're right that limiting this to pointing works well at this time - so much of Windows and gaming development has been focused around pointing (that Alt-Tab menu, for one) that it's a really rich area for reinventing the interaction.
  • Each year CES shows how much closer man is to building their future master.
  • Many times gamers 'accidentally' move their head when navigating even though it 'was' ineffective. This should be easily adapted. Haha.
  • This feature MS could added in build to the w10, like hello they have added. .
  • yay more things to take up my processor.
  • Awesome techology. I would see this becoming the norm one day.
  • looks great @dan. Let me know how good it is. and if it works. i didn't buy leap motion.
  • An amazing piece of technology. Can it still be used if wearing glasses, dark or ordinary, and can it be used in bad visibility?
  • good point. Mine block blue light and uv rays will that be effected?
  • Yeah I used Tobii eyetrackers in the past for usability research. It can still be used with glasses. I wear glasses myself and didn't really had much problems. The only thing you need to watch out for is reflections from light
  • @fndlumia. In all my time reading these board you are the first user I've seen sully the sight with needless casual racism. . That aside. Agree with the consensus this is a great product, Halo 5 is apparently due for the PC by nice to see game focus changing with the gamers eye movement. Wonder what distance restrictions there are, would this tech work at very close range.. Like inside a VR helmet?
  • Military technology was used many years ago in the Eurofighters then by the tracking your eyes for targeting and destroy the object by the missiles
  • There is better software out there for this
  • Like?
  • A really great device, but not for most games. Examples:
    FPS: As long you are not playing a sniper you have to constantly look around after you have locked your gun at a target and shoot it (wihtut looking at it the most time).
    Racing game: It's natural to look around, even when you think we only look straight forward. Think of a racing game. You have to take a look in the rear view mirror, to the sides... Basically any game that has more than one "opponent" will not work with this technology. Also it seems way too slow in the GTA V demo. But it seems fine to me for using it in menus, Point-and-click games and basic Windows navigation. It's just not advertised like this.
  • This technology is important. Maybe not for gamers or regular users, but it's good that it's out there for people with disabilities. Other than that, besides how spectacular it is and the potential of use, to me it just adds to a series of similar technologies like the Motion Leap, the RealSense cameras, even MS's Kinect, that are all looking to bring new ways of interaction. So there needs to be some kind of standard, or one of them should emerge as the default option. I was excited about the Leap controller but I ended up buying a RealSense camera, which has somewhat similar capabilities of controlling things by waving your hands at it. But in the end it all comes down to how much support a device has from OEMs, Microsoft and the developers, besides being bought by the public. This is what holds the future down. Also there's the potential of interaction with the soon-arriving VR generation of hardware. I personally don't consider VR is here yet, the Oculus is just a very early attempt to do it, not only because of hardware but also because for it to work software needs to change a lot. It's not the same to just put on a VR helmet and try the same games. For example, when I just read you can control Assassin's Creed with your eyes, it made no sense to me. There's probably games better suited for this specific kind of controller. So yeah, this is exciting in itself but in the big picture it's just a gadget.  
  • Wow, a game changer. (No pun intended.) Freedom from having to drag the mouse cursor all over the place alone is enough for me to buy one (after researching the USB driver issue). I agree, Microsoft should buy them immediately if possible. Tech like this can solidify Windows 10 as the computing platform of the future. Posted via the Windows Central App for Android
  • Since the SteelSeries Sentry is technically the same as the Tobii Eye-X, does the SteelSeries Sentry software work with the Tobii Eye-X?
  • Awesome Sauce seems built in.
  • This is "TAKE MY MONEY THING"
  • Surely, this ia a game changer, especially for those who has disabilities... though I read the first comment, and, yeah, that's true. I think Tobii was inspired with the disabled artist, and begin to make this awesome invention. Kudos for Tobii!
  • what about multi-monitor setup? 1 per?
  • Nice!!!! I'm thinking that this is not going to be isolated to the Windows ecosystem. I believe that there would be drivers for MacOS as well. As I have laptops with less than 17"screen, I'll wait till the next version is available. By then hopefully there will be further advancements and adoption of this technology. Definitely something to adopt and supprt as it has many benefits as described by various people here. Please keep up the good work. Looking forward to hearing more on this.
  • i wonder now i have 3 monitors will this camera get lost?