Photosynth – reconstruction of 3D spaces from photos

Photosynth

I was fortunate enough to go to Siggraph this year in San Diego, which is the premier conference for new research relating to computer graphics. While the technology I’m going to write about in this article was not presented there this year, it another example of one of an increasing number of research projects to use the vast array of photos available through online communities such as Flickr.

Microsoft have recently released a techology preview of a stunning new piece of technology called “Photosynth”. The aim is to create three dimensional “spaces” of real places and buildings by analysing and processing photos taken from differing viewpoints. The idea of creating spaces is subtly different from creating models. The former places the emphasis on something that is navigable and in which you feel immersed. Model reconstruction, on the other hand, is the creation of a 3D representation that can be places within a virtual environment, perhaps as part of a larger scene.

It is incredibly cool stuff. This article introduces the ideas and shows where to go for more information.

(Update August ’08: See this link for another post on the recently released version that anyone can use to make 3D spaces from their photos.)

The technology to create a 3D model from a set of photos is not all that new. Indeed, I worked on a project with Real3D Inc a few years ago that was all about creating 3D models of people’s faces from two or more photos. But many of these approaches depend on a photographer setting up his shots in a certain way, and explicitely defining the relationships between images. What Photosynth does, however, is use the millions of images available in online photo communities to assemble a 3D space of a real place using a montage of pictures taken by many different people at different times. This montage could consist of thousands of individual source images.

A great way to see this in action is to watch the demo video on the excellent TED site. It is only about seven or eight minutes long. If you are interested in technology and want to feel inspired by some great thinkers, then a browse around TED will be time worth spending.

If you fancy having a go yourself, then you can access an online prototype at the Photosynth research homepage. It is free, and just requires you to download a small plugin that works with both Internet Explorer and Firefox. Here’s a screenshot from my PC:

Photosynth

The scene is of the Piazza San Marco in Venice. The main image is a combination of one or more photos from an online collection placed within a 3D scene represented by a pointcloud that Photosynth generated from processing the photos. The strip along the bottom shows the primary contributing image to the scene, and also related images that are spatially close. Remember that these pictures could have been taken by hundreds of different people, and many are just holiday snaps. You can navigate around the Piazza in 3D using the controls on the image or the mouse.

At the moment the creation of spaces can only be done by the authors using their own specially processed database of images derived from a public photo community. But imagine the possibilities if you combined this with the Panoramio geotagged image database? Of course there is a small issue of corporate politics that would prevent this, unless Google and Microsoft merge. Yeah, right….

Expect to see more and more applications that exploit vast online image libaries for various uses. In upcoming articles I will describe a couple of recent research projects presented at Siggraph.

1 thought on “Photosynth – reconstruction of 3D spaces from photos

  1. Pingback: Photosynth goes live!

Leave a Reply