Rendering, viewing and annotating panoramic images, and applications thereof

The present invention relates to rendering, viewing and annotating panoramic images. In an embodiment, a system may be used for viewing panoramic images. The system includes a panorama viewer that receives at least a portion of a first panoramic image and presents a viewport that displays the portion of the first panoramic image. The viewport includes a three-dimensional overlay rendered with the panoramic image. The panorama viewer changes the three-dimensional overlay's orientation in three-dimensional space as it is rendered with the panoramic image so as to match a change in orientation of the panoramic image within the viewport.