About a year ago I wrote about the Photosynth technology from Microsoft that creates 3D spaces from collections of photographs. At the time it only worked with a few sample data sets created by Microsoft, but there was a very nice demo to download and it was a pretty exciting piece of tech. Just a few days ago Microsoft released a full version that allows you to create your own “photosynths” by uploading collections of photos.

I just had to have a go. The image above shows a screen-grab of the Photosynth viewer with some data that I took.

Read on for more on the new release, how I got on with it, videos and links to the 3D scenes that I created.

It may be worth reading my original posting for a description of what Photosynth does, but in a nutshell it attempts to make a 3D spatial model by matching up lots of photos of the same subject taken from different viewpoints. What Microsoft has just released is a new version that can take your own photos, a website to host these, and plugins for Internet Explorer and Firefox to view the scenes in 3D. You get all you need from photosynth.net.

When you get to this site you get three things:

1. A plugin for your web browser just for viewing photosynths.
2. An application to install on your computer that will process and upload your own photos to create a photosynth.
3. A brief “how-to” guide in PDF form that tells you how to go about taking photos to make good photosynths. There is an online video too, but to date I have not watched it.

The first two are wrapped up in the same installation packet. You will need a WindowsLive ID or MSN Messenger account to use the latter, but you don’t need to create an account if you just want to view other people’s work. Also note that the application and, I think, the viewing plugins are Windows only so Mac users are out of luck.

If you have had the earlier demo version of the viewer installed on your machine, then either uninstall or disable the original plugin. In Firefox I had the problem that it failed to recognise I had the viewer installed, and I think it was getting confused with the two plugins there. Once I disabled the old one everything worked fine.

Browse the existing photosynths to get a good feeling for it – I would recommend only choosing those that have been marked at or near “100% synthy”. If not, then things look very boring indeed. The user interface is quite intuitive, and you certainly don’t need to read a manual to use the basic navigation features.

I went out to our local church that this weekend was having its annual Flower Festival. This gave me the opportunity to wander around both inside and outside snapping away lots of shots to use with Photosynth. The following video shows the results when uploaded and processed by Photosynth, and put into “play” mode (which moves the viewpoint around the scene between camera viewpoints).

The scene was reconstructed fom just 35 photos. It was incredibly easy to do – just point the Photosynth application at the images, press “upload”, and then wait. Once complete you are given the option of viewing the result immediately in your browser. Part way through the video I used the “p” key option to toggle the display of the point cloud that Photosythc has calculated that represents the 3D information extracted from the photos. Normally this is obscured as the photos are shown by default, but as you can see it has done a fine job of picking out the shape of the church building. The following two stills show the default scene, and then with point cloud display turned on.

Of course, viewing this as a video does not do Photosynth justice. Once you have the viewer installed, then jump straight to this dataset using this link (note, will open in new window/tab).

And the final bit of niceness is that you can also geo-locate your photosynth from within the viewer, assuming you are logged in as the creator of the scene. This uses Virtual Earth as its back-end, and is very easy to do.

Taking photos to make nice 3D spaces is pretty straghtforward – the short guide on the Photosynth website is worth a read. The main rule is to ensure lots of overlap between images, as this gives the software something to use to help work out areas of similarity. For the church shoots I just wandered around with my digital SLR on medium resolution mode (2.4k x 1.6k pixels) and storing as JPEG. There is really no point in using RAW for this, and picking a lower resolution image saves on upload time. I think even the res I used is higher than is needed.

I also took about 30 interior shots, and I was very pleased with the results. The video below shows it running live. You can also see the actual Photosynth at this link here.

The videos were captured using the excellent free CamStudio application.

Summary

I think Photosynth is quite simply awesome. I highly recommend at least viewing some of the content created using the system. And creating them yourself is so easy you may as well have a go.

Start Slide Show with PicLens Lite

I was fortunate enough to go to Siggraph this year in San Diego, which is the premier conference for new research relating to computer graphics. While the technology I’m going to write about in this article was not presented there this year, it another example of one of an increasing number of research projects to use the vast array of photos available through online communities such as Flickr.

Microsoft have recently released a techology preview of a stunning new piece of technology called “Photosynth”. The aim is to create three dimensional “spaces” of real places and buildings by analysing and processing photos taken from differing viewpoints. The idea of creating spaces is subtly different from creating models. The former places the emphasis on something that is navigable and in which you feel immersed. Model reconstruction, on the other hand, is the creation of a 3D representation that can be places within a virtual environment, perhaps as part of a larger scene.

It is incredibly cool stuff. This article introduces the ideas and shows where to go for more information.

(Update August ’08: See this link for another post on the recently released version that anyone can use to make 3D spaces from their photos.)

The technology to create a 3D model from a set of photos is not all that new. Indeed, I worked on a project with Real3D Inc a few years ago that was all about creating 3D models of people’s faces from two or more photos. But many of these approaches depend on a photographer setting up his shots in a certain way, and explicitely defining the relationships between images. What Photosynth does, however, is use the millions of images available in online photo communities to assemble a 3D space of a real place using a montage of pictures taken by many different people at different times. This montage could consist of thousands of individual source images.

A great way to see this in action is to watch the demo video on the excellent TED site. It is only about seven or eight minutes long. If you are interested in technology and want to feel inspired by some great thinkers, then a browse around TED will be time worth spending.

If you fancy having a go yourself, then you can access an online prototype at the Photosynth research homepage. It is free, and just requires you to download a small plugin that works with both Internet Explorer and Firefox. Here’s a screenshot from my PC:

The scene is of the Piazza San Marco in Venice. The main image is a combination of one or more photos from an online collection placed within a 3D scene represented by a pointcloud that Photosynth generated from processing the photos. The strip along the bottom shows the primary contributing image to the scene, and also related images that are spatially close. Remember that these pictures could have been taken by hundreds of different people, and many are just holiday snaps. You can navigate around the Piazza in 3D using the controls on the image or the mouse.

At the moment the creation of spaces can only be done by the authors using their own specially processed database of images derived from a public photo community. But imagine the possibilities if you combined this with the Panoramio geotagged image database? Of course there is a small issue of corporate politics that would prevent this, unless Google and Microsoft merge. Yeah, right….

Expect to see more and more applications that exploit vast online image libaries for various uses. In upcoming articles I will describe a couple of recent research projects presented at Siggraph.

Start Slide Show with PicLens Lite