Microsoft's computer vision technology may be in your next 'smart' refrigerator

Microsoft is collaborating with household appliance company Liebherr that will have Microsoft's Computer Vision API used inside future Liebherr refrigerators and freezers.


In a blog post (opens in new tab), Microsoft says that the collaboration will create an image processing system that will be able to detect and identify food products inside Liebherr refrigerators:

Starting with a general purpose computer vision model learned from millions of generic images, the Microsoft system can learn to recognize new types of objects – milk cartons, ketchup bottles, pickle jars and much more – from example images. When a new image from inside a refrigerator is provided to the newly learned model, it can detect the presence of the objects it has seen before during training.

This technology will help people with future Liebherr refrigerators quickly identify what food and other items are inside from anywhere, via a smartphone app. That will allow them to create a shopping list from that app so they can better manage their grocery needs. There's no word on when Microsoft's Computer Vision API will be included in Liebherr's products.

John Callaham
  • Cool
  • No, not cool. It's really chilling hot!
  • After failing in mobile, they are now trying to their luck with refrigerators.
  • This concept is a mistake. You are not supposed to put tomatos in the fridge. They will rotten very quickly. I work in the produce department at a Shop Rite supermarket. I know what I'm talking about. Posted via the Windows Central App for Android
  • This whole concept is a mistake just because, Tomato in the fridge? This is more or like Fridge is a mistake because, tomatoes shouldn't be in it.
  • I thought the reason why people don't put tomatoes in the fridge is because they'll lose their nutritional value quickly.
  • Then why are they sold in the fridge.
  • Tomatoes do not spoil faster in the fridge. Infact, like with most produce, it slows the spoilage process. The problem is, that again like most produce, putting it in the fridge causes "damage", which effects the taste, texture, and in some cases the nutritional value.
  • You make sure they ripen nicely on your counter but if you still don't have time to get to them, you should put them in the fridge to slow down the ripening process
  • This isn't about tomatoes
  • Wish we had that much space in our fridge, normally so jam packed its impossible to find anything.
  • Hahahaha, true that!
  • That's what I was thinking about also. It seems that items would block other items from view of the camera. Unless there are multiple angles that it takes into account.
  • People are so demanding that they think everything will start at top notch technology from the very begining. 
    As if people had phones with 12GB RAM in their hand 20 years ago. 
  • This is ridiculous. What's next...the pantry?
  • Yes actually, the pantry will be next. It's excessive in my opinion, but not ridiculous.
  • Noooooo. People are gonna starve......
  • Interesting!
  • 404: Milk not found?
  • Waw! I wonder how Is that possible to measure everything in the fridge!
  • Awesome
  • That is one empty fridge. And soy, mustard and Tabasco shouldn't be refrigerated anyway.
  • Just like Cortana? First you have to make a reminder to get reminded.
  • Wow... refrigerators with a privacy statement.
  • Is this the method Microsoft wants to feed Cortana when she's hungry?
  • Warning: Window 10 is updating the foods; they can't be eaten for now, until after update. ⚠: Windows is updating to Redstone 2; foods are also updating too. But beware of BSOD in your food. Can't wait for Microsoft to launch " windows 10 for food" in the future and Apple to follow suit with 'food OS'
  • iFood ?
  • iEat :P
  • I want a toilet to analyze my stool and then have that data sold to advertising companies so that when I go online I see advertisements for Metamucil fibre and laxatives.
  • Leftovers   ---    Leftovers   ---    Leftovers   ---    Ketchup   ---    Leftovers   ---    Leftovers   ---    Leftovers   ---    Salad dressing from 2014   ---    Leftovers   ---    Pickles   ---    Leftovers   ---    Leftovers...
  • lol
  • Bizarre. The things people will think of for the sake of technology.
  • So does the fridge scan the barcode or is it reading the text on the product container?!
  • My guess is neither. I think it looks at the general shape and color to determine what the object is.
  • It's called trained AI, you don't know what they look at, just like when you recognize something it's a sum of factors you learned in time.
  • wow
  • I'd like to buy Windows Refrigerator
  • Samsung and now LG have (or will have in LG's case) Fridges that are 'Hubs' for a kitchen. They show pics of the inside of it but don't yet analyze the contents. It's visual. Adding scanning of UCP's, reading RFID's will be next. In the meantime there working on connecting everything in the house.... and you and your phone, watch and any other wearable.
  • Are people so absent minded and lazy that they can't walk to the refrigerator and check what they need to buy? Plain ridiculous in my opinion. It just freaks me out thinking of the things coming out in the name of technology.
  • Well you could be at the movies and think of something to cook. Beam up the trusty phone app and see if everything you need is there. If not pick it up on the way home. Saves an extra trip back out. Maybe fuel prices will go back down lol
  • Are people so absent minded and lazy that they need a stove instead of making a fire? It's called progress. Hell, have you never been at the supermarket and been unsure if you had milk or eggs? This solves that.
  • No. This is not progress like "fire to stove". If you are unsure of whether you need one or not, you will call home and find it from people who are there. If all the family is on a trip, you would buy extra to be safe. You can, you know, refrigerate in case there is extra. There could be different use cases, like the life time of a product in the fridge, how many days the item can be in the fridge, detect and warn in case some thing ripens, and so on. But definitely not going to help you making the shopping list.
  • Just give me a really well made refrigerator that will last the test of time. Like the really old ones. That's what I want. But no company makes things to last like they use too.
  • Smart fridge with win10 os if success whats next? Smart oven? :) heheh
  • ...i won't be surprised if the toasterfridge will rise
  • Just thought about this, who's account will it be linked to, at the moment Windows seems to be anti sharing (Yes you can manually share, but the family room and things like it have gone) so in our flat there are three of us, all with Windows accounts, could be fun managing (Apart from the stuffed look of our fridge compared to the pictured one).
  • I'm not sure that the computer vision model is best applied here as it seems that RFID would be more practical but the general principle of knowing what foods are in a fridge, freezer and cupboard would be useful in reducing the amount of food wastage as long as the 'use by' dates are also included in the data. For example, an app could suggest recipes based on food that is going out of date, combined with other items that you already have at home. In addition the app could suggest items that you could buy in order to make use of items that are going out of date in a recipe. Food wastage is a terrible First World problem.
  • With their gaming position not having an answer for PlayStation VR, and their mobile platform less than 1%, I don't think MSFT should be concerning themselves with crap like this. IMO.
  • Just remember, some jack-ass is going to have to fix this overpriced pile of dung somewhere down the line. Good luck with that!
  • Interesting, I guess, but I can't come up with a single scenario why I'd want my refrigerator to be 'smart'. In fact, the rule of thumb is, when appliance shopping, go for the units that have fewer electronics features, as those are the things on appliances that tend to stop working first, and will have high repair costs.