Posted on Sep 3, 2018 in Discoveries, Science, Technology
“These images are hidden to us, but they are all around us, all the time,” says Antonio Torralba at MIT’s Computer Science and Artificial Intelligence Lab.
While vacationing on the coast of Spain in 2012, the computer-vision scientist Torralba noticed stray shadows on the wall of his hotel room that didn’t seem to have been cast by anything. Torralba eventually realized that the discolored patches of wall weren’t shadows at all, but rather a faint, upside-down image of the patio outside his window.
The window, continues Natalie Wolchover in Quanta, was acting as a pinhole camera—the simplest kind of camera, in which light rays pass through a small opening and form an inverted image on the other side. The resulting image was barely perceptible on the light-drenched wall. But it struck Torralba that the world is suffused with visual information that our eyes fail to see.
The experience alerted him and his colleague, Bill Freeman, both professors at the Massachusetts Institute of Technology, to the ubiquity of “accidental cameras,” as they call them: windows, corners, houseplants, and other common objects that create subtle images of their surroundings. These images, as much as 1,000 times dimmer than everything else, are typically invisible to the naked eye. “We figured out ways to pull out those images and make them visible,” Freeman explained.
The patio outside of the hotel room where Antonio Torralba noticed that his window was acting as an accidental pinhole camera (1). The faint image of the patio on the wall (2) could be sharpened (3) by covering up most of the window with cardboard to decrease the size of the pinhole. Viewed upside-down (4), the image reveals the scene outside. (Antonio Torralba and William T. Freeman)
The pair discovered just how much visual information is hiding in plain sight. In their first paper, Freeman and Torralba showed that the changing light on the wall of a room, filmed with nothing fancier than an iPhone, can be processed to reveal the scene outside the window. Last fall, they and their collaborators reported that they can spot someone moving on the other side of a corner by filming the ground near the corner. This summer, they demonstrated that they can film a houseplant and then reconstruct a three-dimensional image of the rest of the room from the disparate shadows cast by the plant’s leaves. Or they can turn the leaves into a “visual microphone,” magnifying their vibrations to listen to what’s being said.
“Mary had a little lamb …” a man says, in audio reconstructed from the motion of an empty chip bag that the scientists filmed through a soundproof window in 2014. (These were the first words Thomas Edison recorded with a phonograph in 1877.)
Research on seeing around corners and inferring information that’s not directly visible, called non-line-of-sight imaging, took off in 2012 with Torralba and Freeman’s accidental-camera paper and another watershed paper by a separate group at MIT led by Ramesh Raskar. In 2016, partly on the strength of those results, the Defense Advanced Research Projects Agency (darpa) launched the $27 million reveal program (for Revolutionary Enhancement of Visibility by Exploiting Active Light-fields), providing funding to a number of nascent labs around the country. Since then, a stream of new insights and mathematical tricks has been making non-line-of-sight imaging ever more powerful and practical.
Torralba said he and Freeman didn’t have any particular application in mind when they started down this road. They were simply digging into the basics of how images form and what constitutes a camera, which naturally led to a fuller investigation of how light behaves and how it interacts with objects and surfaces in our environment. They began seeing things that no one had thought to look for.
Psychological studies have shown, Torralba noted, “that humans are really terrible at interpreting shadows. Maybe one of the reasons is that many of the things that we see are not really shadows. And eventually the eye gave up on trying to make sense of them.”
Continue reading…
Image credit top of page: With thanks to photograoher Ekaterina Elizarova
Most Viewed Space & Science Headlines (2018)
The Alien Observatory –“When We see Something Unusual, That’s Out of Whack, It Can Be a Sign of Life”
“Point of No Return” –MIT Scientist Predicts the Event Horizon for Earth’s 6th Mass Extinction
Artificial Intelligence Is Already Out There, and It’s Billions of Years Old” (WATCH Video)
‘Ghost Signals’ of Extraterrestrial Civilizations Haunt the Milky Way –Suggests the New Drake Equation (WATCH Video)
“Odds That There has Never Been Another Civilization in the Universe One in Ten Billion Trillion” –A Joe Rogan Interview
Stephen Hawking’s Great Question –“Why Isn’t the Milky Way Crawling With Mechanical or Biological Life?”
The Alien Observatory –“The Mystery of Where Extraterrestrial Life is Hiding Deepens”
“We’re Entering Uncharted Territory” –The Exoplanet Revolution May Reveal that Rise of Civilizations May Not be Unusual
“The Big Rip” –When Matter and Spacetime are Gradually Torn Apart Through Expansion of the Universe
Alien Ocean Worlds–“There May be Life There, but Could It be Technology-Based”
“Humans are the First to Arrive at the Interstellar Stage” –Physicist Answers the Fermi Paradox
