Take a good look around
"Spherical video" system offers 720-degree views of the world
In the 1980s, G. David Ripley, then at RCA's Sarnoff Labs, had the job of figuring out what in the heck the tantalizing but unplumbed new digital technology might be good for. In an initial foray into multimedia, Ripley helped develop a panoramic still photograph tour of Mayan ruins in the jungles of Mexico.
iMove Spherical Video
" dg.o2002 Presentation (.pdf)
" iMove home page
CLICK TO ENLARGE
The iMove system is being developed for civilian and military use, as in this tank periscope system, which allows 360-degree battlefield views with less vulnerability to the operator.
"In order to really appreciate the ruins, you had to be able to look all around," Ripley explained recently.
Ripley and others wrote software to correct for the distortion created by the extremely wide-angle, fisheye lens used to capture the pictures, and a new technology was born. Soon, researchers moved on to panoramic video. In its most recent and sophisticated iteration, spherical video imagery, as it is known, promises to set new standards for everything from military reconnaissance to real estate sales and entertainment DVDs.
Ripley, a presenter at the dg.o2002 conference, says spherical imagery really burst into public view with the spectacular footage sent back from Mars. More recently, real estate agents have been offering panoramic video tours of hot properties, and hospitals and architects have mounted spherical video previews of their facilities. But if researchers can meet the technical challenge of compressing, correcting and stitching together enormous amounts of data to stream panoramic video in real time, the imagery someday will serve much graver functions. Work is underway to put spherical video at the center of homeland security efforts, Ripley said.
The iMove system consists of an all-digital, 6-lens camera that captures images in every direction, including up and down. In playback, users are able to move forward and backward through the video, stopping to look at any point from different vantages.
"It looks exactly like if you were sitting with your head on a desk and looking all around," said Ripley, founder and chief technology officer of iMove Inc., of Portland, Ore., a technology transfer company.
Ripley said he is at the demonstration stage with a number of government partners on spherical imaging projects, including:
iMove camera mounted in a military periscope.
- Military reconnaissance: The Office of Naval Research is exploring the use of iMove's spherical camera system on an unmanned submarine The pole-mounted spherical camera system is thrust out of a torpedo tube in shallow waters along the shore, where a sub cannot travel., then raised six feet above the waterline, where it begins shooting video. The video can be queried from the sub for weapons, buildings, enemy positions and other targets.
- Mapping: The U.S. Special Operations Command, which has been very active in Afghanistan, is trying out a spherical video mapping program. Essentially, someone driving into a new city, say, pops a spherical video camera out of the sunroof and records every street and alley. The video can be replayed on a handheld personal digital device, or on CD or DVD, wireless or wired, and overlaid with various graphical information systems tied to global positioning data. Possible applications include future mission rehearsal and plain navigation, Ripley says. Current mapping targets include U.S. embassies around the world, he adds.
"The U.S. Embassy comes under attack, and everybody pulls out their CD or DVD that has the entire embassy captured, including the grounds outside of it," Ripley says. "It's a lot more useful than blueprints."
- Security: In extremely high-security locations, spherical camera systems can be used to exhaustively surveil the area. The video can then be computer tracked for intruders and other threats, Ripley says. "Let's say airport security discovers a bag with a gun, but the person who brought it in is already gone," Ripley said. "No problem, you go over to the display console, roll the video back until you spot who put the bag on the security belt., then roll forward and track the person to the current moment. Afterward, you can run the video in reverse and watch the person go back out, what car he got out of and the license number."
- Entertainment: George Lucas is looking into spherical video for location scouting, and James Cameron is considering spherical video for the DVD of his upcoming TV show Return to Titanic, Ripley said.
Stephen Bogner, research engineer at the Defense Research Establishment Suffield, near Medicine Hat, Alberta, is developing iMove technology for a next-generation armored vehicle prototype, he said. The Canadian defense department-funded project melds GPS and other GIS with panoramic video in an "immersive imaging system" that will transform battlefield navigation and tactical awareness, he added.
Friendly and enemy positions will show up as icons, both to limit command-and-control failures that are the bane of the battlefield, and to end friendly-fire incidents, Bogner hopes, such as the four Canadian soldiers killed last April near Kandahar by U.S. fighter pilots. Two U.S. Air Force officers are on trial after mistaking the Canadian training exercise for an artillery blast and dropping a bomb, in the first criminal case ever for wartime friendly-fire fatalities.
"In a tank or armored personnel carrier, the visual situation is very poor, particularly in combat when the hatches are down," said Bogner. "People were constantly getting lost , they never knew where they were. They were out there in the middle of the night, trying to read maps in the rain. It was a huge problem." Now, from any position within the tank, soldiers will be able to scan 360 degrees of vista on their computer screens, he said.
With the Mars videos, technicians were able to take their time to stitch multiple images into panoramas and transport them back to Earth. Soldiers, however, need instantaneous visual information.
"It wouldn't do any good to see images that are a half-second old," Bogner said. The real feat will be to make real-time transmission cheap enough for a real-world military budget.
"If cost were no object we could do this with arrays of equipment," Bogner said. Where the project really gets interesting, Bogner says, is in the area of graphically augmented virtual reality. Researchers are combining the spherical imagery with graphical overlays of tactical data to provide soldiers with precise roadmaps for battles. In place of the grease pencils that army grunts traditionally used to mark routes and targets, graphics will detail paths, obstacles and landmarks.
"When a soldier drives to an objective, he typically drives over phase lines with funny names, so as he crosses over the lines, he can radio, ÔI'm at crazy black cat now', for example," Bogner said. "Nos he'll see himself driving over a line as if it was painted on the ground, even though it is just rendered in a graphics way."
All friendly and enemy positions will show up as icons, Bogner added. Eventually, the entire battlefield is expected to be networked with access to both satellite and ground data, presenting soldiers with a package of integrated, easily assimilated information to help them do their jobs. "One of the advantages if you know where the enemy vehicles are, you can present the soldier with information relevant to the success of his mission, without overwhelming him with too much information," Bogner said. "What it means is you'll be able to delegate more authority down to soldiers, who will be able to draw on a much bigger range of resources."
"We want to avoid fratricide and all the other negative things that can happen on the battlefield," Bogner said.
The spherical video tank is still 7 to 15 years away, Bogner says, but if the U.S. goes to war in Iraq, some of the component technology will probably be tested.
"The soldier will be empowered will all kinds of possibilities that haven't been possible in the past," Bogner said. "It will be a huge revolution in capability."