A doorknob that knows to lock or unlock based on how it is grasped. A smartphone that silences itself if the user holds a finger to her lips. A chair that adjusts room lighting.
They are among the many possible applications of Touché, a new sensing technique developed by a team at Disney Research, Pittsburgh, and Carnegie Mellon University.
Touché is a new touch and gesture sensing technology that allows to make everyday objects, such as a common door knob, touch and gesture sensitive with just a single wire.
Touché can sense touch and hand postures on everyday objects, hand grasp on hand-held devices, human gestures directly on their bodies and with unusual materials such as water.
Touché is a form of capacitive touch sensing, the same principle underlying the types of touchscreens used in most smartphones. Touché is a new sensing technology that proposes a novel Swept Frequency Capacitive Sensing technique that can not only detect a touch event, but simultaneously recognize complex configurations of the human hands and body during touch interaction. This allows to significantly enhances touch interaction in a broad range of applications, from enhancing conventional touchscreens to designing interaction scenarios for unique use contexts and materials. For example, in our explorations we added complex touch and gesture sensitivity not only to computing devices and everyday objects, but also to the human body and liquids. Importantly, instrumenting objects and material with touch sensitivity is easy and straightforward: a single wire is sufficient to make objects and environments touch and gesture sensitive.
But instead of sensing electrical signals at a single frequency, like the typical touchscreen, Touché monitors capacitive signals across a broad range of frequencies.This Swept Frequency Capacitive Sensing (SFCS) makes it possible to not only detect a “touch event,” but to recognize complex configurations of the hand or body that is doing the touching. An object thus could sense how it is being touched, or might sense the body configuration of the person doing the touching.
SFCS is robust and can enhance everyday objects by using just a single sensing electrode. Sometimes, as in the case of a doorknob or other conductive objects, the object itself can serve as a sensor and no modifications are required.
Even the human body or a body of water can be a sensor.
Touché can recognize how the user touches his own body.
The basic principles of operation in most common capacitive sensing techniques are quite similar: A periodic electrical signal is injected into an electrode forming an oscillating electrical field. As the user’s hand approaches the electrode,a weak capacitive link is formed between the electrode and conductive physiological fluids inside the human hand, altering the signal supplied by the electrode. This happens because the user body introduces an additional path for flow of charges, acting as a charge “sink”. By measuring the degree of this signal change, touch events can be detected.
There is a wide variety of capacitive touch sensing techniques.One important design variable is the choice of signal property that is used to detect touch events, e.g., changes in signal phase or signal amplitude can be used for touch detection.
Touché can recognize hand gestures on liquids
SWEPT FREQUENCY CAPACITIVE SENSING
Touché is measuring capacitive response of object and human at multiple frequencies, a technique that we called Swept Frequency Capacitive Sensing. The signal travels through different paths depending on it’s frequency, capturing the posture of human hand and body as well as other properties of the context. The resulted data is classified using machine learning algorithms to identify gestures that are then used to trigger desired responses of the user interface.
The human body is conductive, e.g., the average internal Resistance of a human trunk is ~100 Ω.Skin, on the other hand, is highly resistive, ~1M Ω for dry undamaged skin.This would block any weak constant electrical(DC) signal applied to the body. Alternating current (AC) signal, however, passes through the skin, which forms a capacitive interface between the electrode and ionic physiologic fluids inside the body.The body forms a charge “sink” with the signal flowing though tissues and bones to ground, which is also connected to the body through a capacitive link.The resistive and capacitive properties of the human body oppose the applied AC signal. This opposition, or electrical impedance1, changes the phase and amplitude of the AC signal. Thus, by measuring changes in the applied AC signal we can 1) detect the presence of a human body and also 2) learn about the internal composition of the body itself. This phenomenon, in its many variations, has been used since the 1960s in medical practice to measure the fluid composition of the human body,in electro-impedance tomography imaging and even to detect the ripeness of nectarine fruits.More recently, it has been used in a broad variety of capacitive touch buttons, sliders and touchscreens in human-computer interaction.The amount of signal change depends on a variety of factors.It is affected by how a person touches the electrode, e.g., the surface area of skin touching the electrode. It is affected by the body’s connection to ground, e.g., wearing or not wearing shoes or having one or both feet on the ground. Finally, it strongly depends on signal frequency.This is because at different frequencies, the AC signal will flow through different paths inside of the body. Indeed, just as DC signal flows through the path of least resistance, the AC signal will always flow through the path of least impedance. The human body is anatomically complex and different tissues, e.g., muscle, fat and bones, have different resistive and capacitive properties. As the frequency of the AC signal changes, some of the tissues become more opposed to the flow of charges, while others less, thus changing the path of the signal flow (see for an overview of the bioelectrical aspects of human body impedance).Therefore, by sweeping through a range of frequencies in capacitive sensing applications, we obtain a wealth of information about 1) how the user is touching the object, 2) how the user is connected to the ground and 3) the current configuration of the human body and individual body properties.
TOUCHÉ IMPLEMENTATION
The user interacts with an object that is attached to a Touché sensor board via a single wire. If the object itself is conductive, the wire can be attached directly to it. Otherwise, a single electrode has to be embedded into the object and the wire attached to this electrode. Touché implements SFCS on a compact custom-built board powered by an ARM Cortex-M3 microprocessor. The on-board signal generator excites an electrode with sinusoid sweeps and measures returned signal at each frequency.The resulting sampled signal is a capacitive profile of the touch interaction. Finally, the capacitive profile is sent to a conventional computer over Bluetooth for classification. Recognized gestures can then be used to trigger different interactive functions.While it is possible to implement classification directly on the sensor board, a conventional computer provided more flexibility in fine-tuning and allowed for rapid prototyping.
EXAMPLE TOUCHÉ APPLICATIONS
The application space of Touché is broad, therefore at least some categorization is pertinent to guide the development of the interfaces based on this technology. We identified five application areas where we felt that Touché could have the largest impact – either as a useful enhancement to an established application or a novel application, uniquely enabled by our approach:
- • making everyday objects touch gesture sensitive
- • sensing human bimanual hand gestures
- • sensing human body configuration (e.g., pose)
- • enhancing traditional touch interfaces
- • sensing interaction with unusual materials (e.g., liquids)
Making objects touch and grasp sensitive
If analogue or digital objects can be made aware of how they are being touched, held or manipulated, they could configure themselves in meaningful and productive ways The canonical example is a mobile phone which, when held like a phone, operates as a phone.However, when held like a camera, the mode could switch to picture-taking automatically. Touché offers a lightweight, non-invasive sensing approach that makes it very easy to add touch and gesture sensitivity to everyday objects. Doorknobs provide an illustrative exa to operate. Yet, in general, doorknobs have not been infused with computational abilities. A smart doorknob that can sense how a user is touching it could have many useful features. For example, closing a door with a tight grasp could lock it, while closing it with a pinch might set a user’s away message, e.g., “back in five minutes”. A sequence of grasps
could constitute a “grasp password” that would allow an authorized user to unlock the door
Body Configuration Sensing
Touché can be used to sense the configuration of the entire human body. For example, a door could sense if a person is simply standing next to it, if they have raised their arm to knock on it, are pushing the door, or are leaning against it.Alternatively, a chair or a table could sense the posture of a
seated person – reclined or leaning forward, arms on the armrests or not, one or two arms operating on the surface, as well as their configuration .More importantly, this can occur without instrumenting the user. Similar to
everyday objects, conductive tables can be used as is, just by connecting a Touché sensor. Non-conductive tables would require a single flat electrode added to their surface or could simply be painted with conductive paint.
Enhancing Touchscreen Interaction
Touché brings new and rich interaction dimensions to conventional touch surfaces by enhancing touch with sensed hand posture. For example, Touché could sense the configuration of fingers holding a device, e.g., if they are closed into a fist or held open, whether a single finger is touching, all five fingers, or the entire palm. The part of the hand touching could be also possibly be inferred,e.g., fingertips or knuckles, a valuable extra dimension