Not all Steam VR games work with Windows Mixed Reality, at least not officially. While there are many that do seem to support WMR that aren't listed officially, it's at your own risk when you're spending money on games like this.
Fortunately, there's a really easy way to tell which Steam VR games do support WMR, and an equally easy way to separate them off from the rest of the Steam store.
To only view games that are officially WMR supported, first filter the games by virtual reality. Then, keep scrolling down until on the right-hand side you see the box highlighted on the right in the image above. Select the Windows Mixed Reality category button to filter out everything that isn't supported.
If you're just browsing the Steam store generally and come across a game that takes your fancy, you'll always be able to see if it supports WMR right from the list. Underneath the game title, you'll see a little logo for WMR if it's supported, as shown by the left-hand red box above.
That's all there is to it. It's not to say other games won't work, but Steam VR support for WMR is still early days and it's a little messy in places, not least with using the motion controllers. Nevertheless, it automatically opens up the WMR library beyond what's in the Microsoft Store, and we're never going to complain about that.