E-skin lets you manipulate objects in real and virtual worlds

Researchers integrate magnetoelectronic sensors into electronic skin that tracks motion.

When it comes to virtual reality systems, sensors that ‘fit like a glove’ aren’t good enough anymore. Now, we want such sensors to fit more like skin. That’s what researchers have accomplished in a paper published today in Science Advances.

Helmholtz-Zentrum Dresden-Rossendorf, AAAS Illustration

By Emily Waltz, IEEE Spectrum 19 January 2018

Denys Makarov at Helmholtz-Zentrum Dresden-Rossendorf, in Germany, and his colleagues, have engineered a magnetosensitive electronic skin, or e-skin, with directional perception. The device can track the subtle motions of the human hand, and is so thin and flexible that is tangibly imperceptible.

The e-skin could be a substitute for the bulky, often motion-restraining gloves of virtual reality environments, the researchers say. It could also enable a user to interact, using mere gestures, with everyday objects in the physical world and in augmented reality applications.

Makarov and his PhD student Gilbert Santiago Cañón Bermúdez say that in addition to gaming, they envision applying the technology to software design, business, physical therapy and security.

“Imagine that you want to press a button that is located in a restricted environment that you can’t touch. You would have to use some touchless means of interacting with this button,” says Makarov. “A combination of magnetic field sensors and permanent magnets can do this job.”

When placed on a user’s hand, the e-skin tracks its position and turning motion in relation to the angle of an external magnetic field. The information is sent to a computer where the motions are digitally reconstructed and translated into a function or command.

The team demonstrated the e-skin’s capability by having a user dim an image on a computer screen and type keyboard characters, all without touching anything. In the light dimming demo, a plastic ring with a permanent magnet in the middle represents a dial for dimming a light bulb. The user places a sensor on her palm, holds her hand above the dial, and rotates her hand as if she were turning thea knob, but without touching it. A computer connected to the sensor translates that information, and dims a light source image on a screen.

Electronic Skin Manipulates Interactive Light. A demonstration of interactive light dimming by touchless manipulation via a new electronic skin. Helmholtz-Zentrum Dresden-Rossendorf e.V. IEEE Spectrum. Youtube Jan 18, 2018

In the virtual keypad demonstration, the sensor is placed on a wristband and a tiny magnet is placed on the fingertip of the opposite hand. The user can choose one of four keyboard characters by positioning her fingertip above, but not touching, four different areas of the wrist sensor that represent the characters.

New Electronic Skin Manipulates a Virtual Keyboard. This new electronic skin can manipulate objects virtually, including this keypad. Helmholtz-Zentrum Dresden-Rossendorf e.V. IEEE Spectrum. Youtube Jan 18, 2018

Makarov’s team built the sensor using miniaturized magnetosensitive components arranged in a novel way. “It’s the placement and packaging of a high performance spin valve stack in a Wheatstone bridge on ultrathin foils—that’s where a lot of knowhow came,” says Cañón Bermúdez.

Using magnetic fields to track movement is not new in virtual reality. Scientists at Oculus Research and the University of Washington demonstrated fingertip-sized magnetic sensors for virtual reality in a system called Finexus. And a start-up called Ommo is developing a pair of gloves with embedded magnetic sensors for hand motion tracking.

Those devices are an elaboration on conventional magnetic sensors, which involve bulky materials, says Makarov. “The difference is that our device does not restrain motion due to its unique mechanical properties of thinness and extreme flexibility,” he says.

The concept of e-skin, or artificial skin, isn’t new either. Such devices have been built for a range of functions, from monitoring health conditions, vital signs, and temperature changes to making more sensitive prosthetic hands. Makarov’s team in 2015 demonstrated e-skin with proximity sensing, which imitates touch. But that previous system didn’t have directional perception of turning-type movements.

Makarov’s new prototype is the first to shrink magnetosensing into e-skin materials with directional perception. It does not require a direct line of sight between a virtual object and the sensors, unlike optical approaches.

Indeed, Makarov hopes the system, like many e-skins, will have medical applications as well. For example, the sensors could be placed on the skin and aligned with joints to monitor patients undergoing rehabilitation after orthopedic surgery, he says.

Having to put a permanent magnet on objects in order to alter the magnetic field profile of its environment is not always practical for everyday life. So the next step for Makarov is to eliminate the need for permanent magnets and instead rely on geomagnetics—earth’s magnetic fields—to track positioning. “This is something for which our community has been waiting for a long time and is the topic of our current development,” Makarov says. “We’re very close.”

After all, bacteria, insects and even birds and sharks can perceive geomagnetic fields and use them for orientation and navigation. Some of us humans would like to be able to do that too.

Source IEEE Spectrum

References

Magnetosensitive e-skins with directional perception for augmented reality, Gilbert Santiago Cañón Bermúdez, Dmitriy D. Karnaushenko, Daniil Karnaushenko, Ana Lebanov, Lothar Bischoff, Martin Kaltenbrunner, Jürgen Fassbender, Oliver G. Schmidt, and Denys Makarov. Sci Adv. 2018 Jan; 4(1): eaao2623. Published online 2018 Jan 19. doi: 10.1126/sciadv.aao2623