Windows Phone developer hooks up Cortana to control his lights

Cortana Lights

We’re very excited to see what developers can do by integrating their apps with Cortana. It’s a very promising future for those on Windows Phone 8.1. One developer has already started to play around with Cortana. Matt Cavanagh, a guy from South Africa who is well-known in the Windows Phone developer scene, has hooked up Cortana to control his lights. Check the video after the break.

It’s a quick and early look at the possible integration of Cortana and your favorite apps. Matt used a Cortana on Windows Phone, a Netduino, a Bluetooth module and a normal 240v desk lamp to get this early concept up and running. You don’t even have to work too hard to see Microsoft getting into home automation and have Cortana truly be your personal assistant by controlling everything around you.

Pretty cool concept from Matt and we're excited to see him take it a bit further. In the meantime check out VALA: Alpha, his new Windows Phone 8 game 

Any other developers out there have fun concepts with Cortana running? Be sure to share with us!

Via: @roguecode


Reader comments

Windows Phone developer hooks up Cortana to control his lights


App controlling lights over Bluetooth.. There's a lot of these projects on the net and it's fairly easy to program.

Dude I know Cortana more than you do...maybe before u were born but at least we are not Master Chief. Shouldn't she call us by our names?

Nope, i gave the name "David" yet it calls me chief no matter what. Kinda silly, why should I give it a name if zero fucks given? 

This is exactly what I'm trying to explain here. I've set my name, but she never calls me by it. And when I ask some questions she replies with chief before answering. She should at least call my name. Trying to explain this and people don't get it.

Hahahahahahahhahahahahaha EPIC!!! That was so funny but I think maybe she can't really pronounce those names so they automatically make her say chief.... Maybe

hahahaha i have to agree that was funny as hell.. it weired though cause she asks you how your name is pronounced and stuff in the initial setup.. but never says it.. asking her what is my name? who am I? etc. doesn;t work either.. I tought her to say Kakarot (DBZ) adn she said it perfectly but I've never heard her that that since i set it up.. which is pretty sad.. 

@kenzibit you can have Cortana call you whatever you want, you are asked when Cortana is first launched and can be edited in the settings via the tile or settings - application. Plus many prefer to have her call them "chief" :P.

I have my regular name set in Cortannas settings and all of the appropriate boxes checked and have not had her call me by name yet. Tried resetting the app as well with no luck.

Guys relax its beta... It will evolve... Don't worry it wud be done.... For now enjoy 8.1 and cortana

Goto apple store and buy Philips light with changing colors and get ONI light control app, it works with Cortana

There's a store that just sells apples? Hope it at least sells a variety of Fresh Apples, but seriously there's so much more fruit out there. Why get stick with an apple.

For launching apps, you were bound to specific phrases, but with Cortana you can now pass in random grammars enabling much richer voice apps.

MS provides developers Cortana API which can directly access to the databases and applications.  The results of voice query can be programmed to send command to external devices via BT.  For this case, a BT supported electrical switch is used to turn on and off the light.  In theory, all kinds of control can be initiated by Cortana.  Neither Apple or Google provides API to Siri/Google Now developers.  So this is a unique strengh of Cortana.

It's worth noting that the same speech capabilities have been around since WP8 launched. Just in TellMe form and not Cortana form. So... yeah...

cool idea though.

Well not the same actually, in WP8 developer had to program all specific voice commands word by word manually, OS provided developers with just the feature of translating audio to the text and that's it. WP8.1 gives developer not only the speech recognition, but also a natural language processing - so that there're machine learning/AI algorithms that process the recognized text to sort of get the meaning of the used wordings. That simplifies things for developers like A LOT.

I'd be very interested in seeing the documentation you're referencing on TellMe vs Cortana integration from a developer standpoint, then. Having done the Code Challenge for Cortana implementation at BUILD, I saw no difference. And to-date have not seen any new documentation saying "for Cortana, do this!" - it's all still VCD (voice command definition) files from what I have found and implemented on 8.0 and 8.1

I was at the keynote. I want hard implementation details. Wp8.1 is out, the SDK is out, and i've seen no change or any new doco on how developers integrate w/ voice reco.

If you were at the BUILD why didn't you ask WP team about it in person?

Anyway I don't think WP8.1 is fully out yet, the documentation and SDK seem in preview at the moment, some subjects are not fully covered.

lol. yeah just walk to up belfiore and ask him. why didn't i think of that.


after some cursory Binging I've come across the following:


which directs devs to look at the source code for the MSDN Voice Search application.

Indeed we still use the VCD files, but can put special keys in to the file to get different results.
The salient point here is this is only valid in-app, and system-level commands still require knowing the phrase ahead of time, based on how I read that page anyway.

Also given this, it's interesting to note this is the only doco i know of; this app being released and open-sourced. And what is being shown in the video on this post is *still* 110% possible *without* Cortana.

You are right, the difference in the Speech APIs between WP8 and WP8.1 are minimal you still create your VCD, and you can use custom SGRS grammar for speech recognition, like on WP8. Here there is a Demo showed at the Build: http://channel9.msdn.com/Events/Build/2014/9-2663 Here there is the MSDN documentation: http://msdn.microsoft.com/en-us/library/windowsphone/develop/jj206958(v=vs.105).aspx And as you can see are essentially the same.

So for now the real benefit of Cortana, for the developer is the speech engine that interprets the voice.

But for the future the idea is to expose more capabilities to the third-party developers.

But you are wrong on one thing:

To compare TellMe and Cortana.

Are two different thing.

Cortana substantially is an interface but under the hood there are many component on the cloud and on the device.

For example there is the engine that track the information, there is also an engine that recognize what the user says, there is Satori, ect.

The combination of all this part give as Cortana.

But TellMe was only the engine that recognize hat the user says, so in some way is still in Cortana and is par of Cortana. This speech engine is what intercept a speech and translates it into a bunch of 0 and 1 which can be converted into a text string.

But not only you have to translate a speech in a texst string, but also to perform the right action with that string.

Moreover a digital personal assistant is not only about speech, is also about to know an find the information that the user want and to inform the user about this.

Remind to wish happy birthday to a frient when he call you or to buy the milk when you are near a market, don't necessarily involve a tecnology like TellMe (in fact you can make a reminder also with keyboard), but a speech engine is involved when you use your voice to create this reminder, to inform the personal assistant that you want to be allerted about this.

TellMe was an engine that provided the speech capabilities in some Microsoft product, in the 2011 the team of TellMe (TellMe originally was a company that provide speech engine and software base on speech tecnology acquired by Microsoft in the 2007) was moved under the Bing Division and, starting from that point, Microsoft no more refer to this speech engine like TellMe but with a more genereic Bing Speech and it is not a real product that you can install or something similar, but is more like a tecnology, a distribuited tecnology (not only in the sense of in the cloud but also in the sense of a many interdisciplinary technologies that work together).

So is wrong to compare Cortana vs TellMe because "TellMe" is part of Cortana and that can take advantage of TellMe tecnologies

This was the first thing I told my friend, when Belfiore announced that Cortana can be used in apps. I was gonna do this, dammit!

Cortana is essentially updated "on the fly" all the time since most of what you ask her to do is processed in the cloud and not on your phone itself.

As soon as I saw the headline, I thought that it sounded like something that South African Llama wrangler would make. Genius. So jealous.

I console myself with the delusion that the only thing that separates our achievements in programming is my lack of spare time.... It isn't that, turns out you are more able too! Ha ha. Kudos to you and thanks for responding previously with regards to the Bluetooth remote control vehicle. Don't stop.

Been waiting to see what your next project would be after you had posted this one! My fav is still the car for going under your house, lol.

It's nice!
I'd like MS could implement a voice command to activate Cortana instead of pressing the button, something  like "Calling Cortana....>>> request...." and 'til the end with something like "end calling Cortana", to stop it.

I understand probably the OS should always check for a command coming in the mic and maybe it could drain the battery, but, well, I'm not a technician so I don't know if it's really doable, but I'd like a voice command to activate it.

So people can cry about 'always on' like the XboxOne kinect? ...

But it would be nice if they give you option to have Cortana always on too ...

Hope Cortana in the future can give us cricket updates here in India.Satya might love it when he plays cricket

But they said very clearly that cortana is only American for now. Obviously Americans don't care about cricket. You went around ms to get her on your phone in India what did you think you should get. I'm sure when she comes to India you could get ask the cricket scores you want. Currently Google now does not recognise "did west indies win" for me. Lol

Snap Switch has had this before Cortana. Now it's even cooler. Works for all of my insteon and x10 devices.

Microsoft Agent was command based, but it had the capability years ago. I had "Merlin" turning on my lights, giving me the time.... and small preprogrammed conversations.

No where near as cool as Cortana, but MS has had the speech capabilities to do this since 2000.

A small trick with cortana. To get ur normal Bing search screen with wallpaper and Bing VISION immediately while using cortana, switch off the location and click the search key. Default Bing search will appear. Cortana and Bing search with vision can be toggled by using location on/off. May be someone else might have already covered this, but I just wanted to say :)

There is a 'QR for Cortana' app you can install from the Store.  Tap on that Tile to reach Bing Vision.

Unfortunately, even if MS adds this, it wont be available on existing phones. You'll need a specific chip to handle the always listening part like the Moto X. Oh well maybe a future Lumia.

And he didn't have to hack it using a man in the middle vulnerability like some guys did with Siri.

Or, just get a Wemo and use IFTTT to have Cortana send a text to turn lights on or off. This seems overworked to me.

Grasshopper already takes control commands through Cortana to control your entire z-wave home network. But I wish the commands were simpler and more adaptable. The Grasshopper dev needs to make these commands simpler.

You can do this and much more with the Snap Switch app for WP8. It understands lots of voice commands and works with several off-the-shelf home automation controllers: Hue, Vera (Z-Wave), and ISY (Insteon).

I am the developer of Snap Switch. Apparently I should have submitted a video to WPCentral.