Last week, Facebook released a blog post exploring human-computer interaction (HCI) and its future. It was in this post the company promised to unveil the features of its upcoming AR glasses. A week later, Facebook has now shared a wrist-based AI input mechanism to go with the AR glasses. Facebook mentions in the new blog that it uses limited yet contextualized AI that is capable of dynamically adjusting to the user and the environment.
Facebook Wrist-based AI Input Mechanism
The wrist-based AI input mechanism is crucial in Facebook’s vision of how people will connect with people in the future. The company is working on its always-available AR glasses that will allow users to connect to people conveniently irrespective of the distance. The glasses will help fuse the virtual world with the real world. According to Andrew Bosworth, lead at Facebook Reality Lab, AR glasses will enhance daily life for the better. Unlike smartphones, it will project the digital world in three dimensions.
The wrist-based AI input mechanism is far from the ideal HCI for the future. Facebook itself is considering an “ultra-low-friction input” that combines contextually-aware AI that can interpret users’ actions with a reliable communication technology that can complement it. But this ideal HCI mechanism is not possible with present-day technology. As a result, we need to be content with a mechanism that has limited but usable contextualized AI.
Facebook has also explained its decision to go with a wrist-based mechanism. First of all, it is much closer to hand which we use to interact with the world. And it’s more normalized to wear smart wearable like band and watches on the wrist. The proximity with our body means it’s both private and reliable.
EMG on the Facebook Wrist-based AI Input
The prototype of the wrist-based AI input makes use of electromyography. There are sensors that detect the nerve signal moving from the wrist to the hand and translate it into digital signals. These signals serve as a 1-bit command that can control the functions of the device in focus. Facebook says the degree of control is highly “personalizable” and adaptable. The sensors in use are so sensitive that they can even detect just the intention of moving a finger.
The sensors are not reading your mind. In fact, they are sort of a bridge between the peripheral nervous system and the world. When you decide to perform a task, your mind sends nervous impulses to your hand for the same. Here, the EMG sensors detect the signal from the wrist and convert them to digital commands. It doesn’t have access to your thoughts.
Click and Contextualized Click
The wrist-based AI input mechanism with EMG supports up to 2 bits of control commands, otherwise called ‘click.’ The click could be a simple tap or complex gesture like pinching and releasing of thumb and forefingers. It is a lot faster than other conventional input methods like tapping on a touchscreen click on a mouse and typing on a keyboard.
Facebook wants to expand these controls to a point where users can use them to navigate through virtual AI and control objects. It also working on custom keyboard models to adjust to users’ speed and method as it is working to replace traditional keyboards with EMG inputs.
Facebook says the new wrist-based AI input mechanism will be combined with contextualized AI for an “intelligent click”. On its own, the click has very limited functionality. However, when it is combined with a contextualized AI, it can do a whole lot more. An intelligent click is when AI recommends a certain task based on the daily routine that best aligns with your past activities. You just have to click to approve it.
The wrist-based AI mechanism can also reproduce a wide range of haptic responses for a more realistic experience. Facebook says it can realistically re-create the sensation of pulling back arrow while using a virtual bow. This will be used to notify users of the incoming notifications. For instance, Facebook proposes a custom haptic to let the user know the caller. Users, after receiving it, can use click to receive calls or reject them. All this is completed without a visual interface. Facebook is also working on new concepts such as haptic emojis, Tasbi (Tactile and Squeeze Bracelet Interface), and Bellowland.
In the past, Facebook has been a subject of different controversies regarding user’s privacy, safety, and security. Thus, it has called for opinions about their latest venture. It is open to share its learning with the community and be part of open discussions for addressing potential concerns. The company also has a neuroethics program at FRL which looks for loopholes in the technology to avoid any potential harm.
- Meanwhile, check out our impressions of the Galaxy A52 4G.