Skip to main content

Windows Phone developer hooks up Cortana to control his lights

We’re very excited to see what developers can do by integrating their apps with Cortana. It’s a very promising future for those on Windows Phone 8.1. One developer has already started to play around with Cortana. Matt Cavanagh, a guy from South Africa who is well-known in the Windows Phone developer scene, has hooked up Cortana to control his lights. Check the video after the break.

It’s a quick and early look at the possible integration of Cortana and your favorite apps. Matt used a Cortana on Windows Phone, a Netduino, a Bluetooth module and a normal 240v desk lamp to get this early concept up and running. You don’t even have to work too hard to see Microsoft getting into home automation and have Cortana truly be your personal assistant by controlling everything around you.

Pretty cool concept from Matt and we're excited to see him take it a bit further. In the meantime check out VALA: Alpha, his new Windows Phone 8 game 

Any other developers out there have fun concepts with Cortana running? Be sure to share with us!

Via: @roguecode

108 Comments
  • Shut up and take my money
  • Fuck how did he do that. How hr connected it??
  • App controlling lights over Bluetooth.. There's a lot of these projects on the net and it's fairly easy to program.
  • Yup. I did some stuff that Microsoft's IOT "traffic light" demo at BUILD was based on. Checkout the source and tutorial here: http://blog.roguecode.co.za/Internet-of-Things-Azure-cloud-Netduino-othe....
  • Read the article butt nugget.
  • +1 ಠ益ಠ
  • Why does she keep on calling us Chief??
  • You should play more video games or read up on "who" Cortana really is.
  • Dude I know Cortana more than you do...maybe before u were born but at least we are not Master Chief. Shouldn't she call us by our names?
  • the guy has set his name as Chief.
  • Nope, i gave the name "David" yet it calls me chief no matter what. Kinda silly, why should I give it a name if zero fucks given? 
  • This is exactly what I'm trying to explain here. I've set my name, but she never calls me by it. And when I ask some questions she replies with chief before answering. She should at least call my name. Trying to explain this and people don't get it.
  • You call your boss by his name?
  • Hahahahahahahhahahahahaha EPIC!!! That was so funny but I think maybe she can't really pronounce those names so they automatically make her say chief.... Maybe
  • hahahaha i have to agree that was funny as hell.. it weired though cause she asks you how your name is pronounced and stuff in the initial setup.. but never says it.. asking her what is my name? who am I? etc. doesn;t work either.. I tought her to say Kakarot (DBZ) adn she said it perfectly but I've never heard her that that since i set it up.. which is pretty sad.. 
  • How hard is the name David to pronounce?
  • Just be glad you're having a female voice calling you chief..
  • Cortana is the artificial intelligence of the master chief.
  • But we are not Master Chief. She should call me by my name right?
  • Every one should strive to be like the master chief, he saved the universe you know.
  • So did Shepard..
  • @kenzibit you can have Cortana call you whatever you want, you are asked when Cortana is first launched and can be edited in the settings via the tile or settings - application. Plus many prefer to have her call them "chief" :P.
  • I have my regular name set in Cortannas settings and all of the appropriate boxes checked and have not had her call me by name yet. Tried resetting the app as well with no luck.
  • Speak for yourself. JOHN 117 REPORTING FOR DUTY! ಠ益ಠ
  • Guys relax its beta... It will evolve... Don't worry it wud be done.... For now enjoy 8.1 and cortana
  • Maybe its her habit. Bad habit (or good) dies hard.
  • From what I see, it's "chief" not "Chief". Probably meant in a generic way.
  • Goto apple store and buy Philips light with changing colors and get ONI light control app, it works with Cortana
  • You lost me at "go to apple store" Lol
  • Haha
  • There's a store that just sells apples? Hope it at least sells a variety of Fresh Apples, but seriously there's so much more fruit out there. Why get stick with an apple.
  • U can buy Philips lights on amazon, no need to go to the enemy's camp :)
  • Philips hues are a tad over priced tbh
  • I got it at 200, it's over priced but worth it I guess, I like it though
  • That's cool. So developers can program Cortana's speech capabilities? Awesome.
  • Well they could in 8 too, but this is cooler just because.
  • For launching apps, you were bound to specific phrases, but with Cortana you can now pass in random grammars enabling much richer voice apps.
  • MS provides developers Cortana API which can directly access to the databases and applications.  The results of voice query can be programmed to send command to external devices via BT.  For this case, a BT supported electrical switch is used to turn on and off the light.  In theory, all kinds of control can be initiated by Cortana.  Neither Apple or Google provides API to Siri/Google Now developers.  So this is a unique strengh of Cortana.
  • Now, thats Awesome!! Could you also please let me know other functionaliteis of Cortana?   Thank you, Chaitanya chaitanya.hps@live.in
  • Once the Cortana API is on the developers' hand, the creativity starts.
  • that is one major advantage over siri and google reviewers have been stating.
  • Sweet!
  • Sploosh!
  • It's worth noting that the same speech capabilities have been around since WP8 launched. Just in TellMe form and not Cortana form. So... yeah... cool idea though.
  • Well not the same actually, in WP8 developer had to program all specific voice commands word by word manually, OS provided developers with just the feature of translating audio to the text and that's it. WP8.1 gives developer not only the speech recognition, but also a natural language processing - so that there're machine learning/AI algorithms that process the recognized text to sort of get the meaning of the used wordings. That simplifies things for developers like A LOT.
  • I'd be very interested in seeing the documentation you're referencing on TellMe vs Cortana integration from a developer standpoint, then. Having done the Code Challenge for Cortana implementation at BUILD, I saw no difference. And to-date have not seen any new documentation saying "for Cortana, do this!" - it's all still VCD (voice command definition) files from what I have found and implemented on 8.0 and 8.1
  • http://channel9.msdn.com/Events/Build/2014/KEY01 Head to 1:33:40, there's the mention of what I've described.
  • I was at the keynote. I want hard implementation details. Wp8.1 is out, the SDK is out, and i've seen no change or any new doco on how developers integrate w/ voice reco.
  • If you were at the BUILD why didn't you ask WP team about it in person? Anyway I don't think WP8.1 is fully out yet, the documentation and SDK seem in preview at the moment, some subjects are not fully covered.
  • lol. yeah just walk to up belfiore and ask him. why didn't i think of that.   after some cursory Binging I've come across the following: http://www.bing.com/blogs/site_blogs/b/search/archive/2014/04/14/integrating-store-apps-with-cortana-in-windows-phone-8-1.aspx&adlt=strict which directs devs to look at the source code for the MSDN Voice Search application. Indeed we still use the VCD files, but can put special keys in to the file to get different results.
    The salient point here is this is only valid in-app, and system-level commands still require knowing the phrase ahead of time, based on how I read that page anyway. Also given this, it's interesting to note this is the only doco i know of; this app being released and open-sourced. And what is being shown in the video on this post is *still* 110% possible *without* Cortana.
  • You are right, the difference in the Speech APIs between WP8 and WP8.1 are minimal you still create your VCD, and you can use custom SGRS grammar for speech recognition, like on WP8. Here there is a Demo showed at the Build: http://channel9.msdn.com/Events/Build/2014/9-2663 Here there is the MSDN documentation: http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj206958(v=vs.105).aspx And as you can see are essentially the same. So for now the real benefit of Cortana, for the developer is the speech engine that interprets the voice. But for the future the idea is to expose more capabilities to the third-party developers.
  • But you are wrong on one thing: To compare TellMe and Cortana. Are two different thing. Cortana substantially is an interface but under the hood there are many component on the cloud and on the device. For example there is the engine that track the information, there is also an engine that recognize what the user says, there is Satori, ect. The combination of all this part give as Cortana. But TellMe was only the engine that recognize hat the user says, so in some way is still in Cortana and is par of Cortana. This speech engine is what intercept a speech and translates it into a bunch of 0 and 1 which can be converted into a text string. But not only you have to translate a speech in a texst string, but also to perform the right action with that string. Moreover a digital personal assistant is not only about speech, is also about to know an find the information that the user want and to inform the user about this. Remind to wish happy birthday to a frient when he call you or to buy the milk when you are near a market, don't necessarily involve a tecnology like TellMe (in fact you can make a reminder also with keyboard), but a speech engine is involved when you use your voice to create this reminder, to inform the personal assistant that you want to be allerted about this. TellMe was an engine that provided the speech capabilities in some Microsoft product, in the 2011 the team of TellMe (TellMe originally was a company that provide speech engine and software base on speech tecnology acquired by Microsoft in the 2007) was moved under the Bing Division and, starting from that point, Microsoft no more refer to this speech engine like TellMe but with a more genereic Bing Speech and it is not a real product that you can install or something similar, but is more like a tecnology, a distribuited tecnology (not only in the sense of in the cloud but also in the sense of a many interdisciplinary technologies that work together). So is wrong to compare Cortana vs TellMe because "TellMe" is part of Cortana and that can take advantage of TellMe tecnologies
  • Mind blown!
  • This was the first thing I told my friend, when Belfiore announced that Cortana can be used in apps. I was gonna do this, dammit!
  • Now you can do using wifi or the internet :p
  • Sweet!
  • so I think e can update Cortana w/o global update?(like a music, video, game hubs?
  • Cortana is essentially updated "on the fly" all the time since most of what you ask her to do is processed in the cloud and not on your phone itself.
  • Some part is processed on the cloud some other on the phone
  • Like a Xbox music"some part in phone, some in the cloud ☁
  • Nicely done matt
  • As soon as I saw the headline, I thought that it sounded like something that South African Llama wrangler would make. Genius. So jealous.
  • Ha, thanks :)
  • I console myself with the delusion that the only thing that separates our achievements in programming is my lack of spare time.... It isn't that, turns out you are more able too! Ha ha. Kudos to you and thanks for responding previously with regards to the Bluetooth remote control vehicle. Don't stop.
  • It helps that I do this stuff for a living too ;-)
  • Advantage Rogue Code. Keep it up. Hopefully we may see an Xbox One game at some point.
  • Been waiting to see what your next project would be after you had posted this one! My fav is still the car for going under your house, lol.
  • Well done !!! Awesome!
  • Cortana, when I do this, <redacted>  I need you to make this face :O
  • It's like "show me the news" then the WPCentral open up!
  • Mzantsi fo sho fly that flag!!!
  • Wow, now that is amazing. Hook this up to the xbox one and have your home entertainment controlled wherever you are in the house
  • It's nice!
    I'd like MS could implement a voice command to activate Cortana instead of pressing the button, something  like "Calling Cortana....>>> request...." and 'til the end with something like "end calling Cortana", to stop it. I understand probably the OS should always check for a command coming in the mic and maybe it could drain the battery, but, well, I'm not a technician so I don't know if it's really doable, but I'd like a voice command to activate it.
     
  • So people can cry about 'always on' like the XboxOne kinect? ... But it would be nice if they give you option to have Cortana always on too ...
  • How much more awesome would this be if he could do it hands free?
  • Hope Cortana in the future can give us cricket updates here in India.Satya might love it when he plays cricket
  • Yeah, she already giving me basketball, baseball and soccer updates here.
  • Not the cricket updates but when I ask her. In India cricket apps and info are much needed.
  • But they said very clearly that cortana is only American for now. Obviously Americans don't care about cricket. You went around ms to get her on your phone in India what did you think you should get. I'm sure when she comes to India you could get ask the cricket scores you want. Currently Google now does not recognise "did west indies win" for me. Lol
  • Awesome!!
  • It works with oni light control too
  • Good now make some apps.
  • Cortana can do more than siri and srewgle now.
  • Good work man!
  • This is pretty damn cool I love our developers
  • Cortana FTW!
  • Snap Switch has had this before Cortana. Now it's even cooler. Works for all of my insteon and x10 devices.
  • Microsoft Agent was command based, but it had the capability years ago. I had "Merlin" turning on my lights, giving me the time.... and small preprogrammed conversations. No where near as cool as Cortana, but MS has had the speech capabilities to do this since 2000.
  • A small trick with cortana. To get ur normal Bing search screen with wallpaper and Bing VISION immediately while using cortana, switch off the location and click the search key. Default Bing search will appear. Cortana and Bing search with vision can be toggled by using location on/off. May be someone else might have already covered this, but I just wanted to say :)
  • There is a 'QR for Cortana' app you can install from the Store.  Tap on that Tile to reach Bing Vision.
  • Cortana activated using voice would be cool .
  • How it should be commanded then? 'Cortana'
  • Yes.
  • Unfortunately, even if MS adds this, it wont be available on existing phones. You'll need a specific chip to handle the always listening part like the Moto X. Oh well maybe a future Lumia.
  • And he didn't have to hack it using a man in the middle vulnerability like some guys did with Siri.
  • Great creativity
  • This is sweet!
  • Or, just get a Wemo and use IFTTT to have Cortana send a text to turn lights on or off. This seems overworked to me.
  • Didn't they already have something like this with Oni for Phillips Hue? http://m.wpcentral.com/oni-light-control-philips-hue-support-windows-phone
  • It's a shame that Nest Thremostat does't develop an integrated app with Cortana like that! I would love to have it. But again, they don't even have a Windows Phone app to start with! If you have Nest you can vote for the WP app request here. https://community.nest.com/ideas/1014
  • Good luck with that. You can't even get Google to give us an official YouTube app much less a Nest app.
  • Can't wait to use Cortana to control my flying suit of armor!
  • The possibilities if added with CastleOS which already supports Kinect. https://www.castleos.com  
  • Grasshopper already takes control commands through Cortana to control your entire z-wave home network. But I wish the commands were simpler and more adaptable. The Grasshopper dev needs to make these commands simpler.
  • I want this
  • Now this what makes iOS and Android fellas envious about us.
  • When does Cortana say your name out loud? I've seen it onscreen but never heard her say it out loud
  • You can do this and much more with the Snap Switch app for WP8. It understands lots of voice commands and works with several off-the-shelf home automation controllers: Hue, Vera (Z-Wave), and ISY (Insteon). I am the developer of Snap Switch. Apparently I should have submitted a video to WPCentral.