Microsoft Kinect and iPad turned into augmented reality 3D recording/projection system
The Kinect has gotten a lot more attention as a hackable 3D camera system than it has as a game system. The people at LAAN Labs used it to record a 3D video clip, and by that I mean a video that is actually a (somewhat buggy) moving 3D model, not a static layered video clip. They then got the model to play nice with the iPad 2 and an augmented reality system, getting the 3D video to play back as a layer on top of the camera stream wherever the iPad 2’s camera sees a specific pattern. It’s hard to explain, so just check out the video – the result is pretty darn cool in any case.
It seems complicated but it really is quite simple (relatively speaking). The Kinect’s two cameras sees a person from two slightly different angles and can then join those two images into a 3D model that shows the person’s front, part of the sides and the depth of different items in the picture. On the iPad side of things, the app is using the camera to constantly look for a predetermined pattern, in this case the card on the tablet. By seeing how this pattern is twisted as a result of the angle of the iPad’s camera, it knows what direction the camera is filming from relative to the card. All it then needs to do is superimpose the 3D model onto where the card is (and virtually remove the card by cloning the texture around it onto where it should have been in the video) and adjust the 3D model’s angle so it matches that of the camera. As the iPad is moved around the augmented reality card on the table changes in the camera’s view (as the perspective changes) and the model is adjusted accordingly. The effect is that it looks like a person is actually standing on the tablet and the camera is panning around him.
The reason I’m pointing out the genius simplicity of it (again, relatively speaking) is that it has potential as a consumer product. There are already phones, tablets and cameras out there with dual lenses for 3D recording, which means that it’s only missing the software to remove the background and create the 3D model, plus a corresponding playback app. It’s not something the current iPad could do on its own (though there are tablets that could) but maybe a future iteration will have this as standard. Even a 2D video would be much cooler if combined with augmented reality. Imagine being on vacation, sending home a postcard and having a receiving family member or friend use his/her iPad to see the video. Or just email the video, of course, but that’s so 2010.
[Engadget]
