Convert mental handwriting into on-screen text

NS Madhurjya Chowdhury
July 30, 2021

Next-generation brain interface computer changes write mechanism

The brain-computer interface monitors brain activity, extracts information from it, and converts that data into output to replace, repair, enhance, supplement, or improve human activity.

BCI can be used to replace lost features such as speech and movement. For example, you can regain control of your body by activating the neurons and muscles that move your hands. BCI is also used to improve function, such as training individuals to improve the function of impaired grip paths. BCI can also improve functionality, such as warning drowsy drivers to get out of the car. Finally, BCI can be used to complement the normal output of the body, such as a third hand.

BCI employs a variety of methods to monitor brain activity. Electrical signals are monitored in most of the cerebral cortex using electrodes that are invasively implanted in the cortex or on the surface of the cortex, or non-invasively implanted on the surface of the head. Some BCIs are based on non-invasive metabolic activity measurements such as functional magnetic resonance imaging (fMRI).

Mental handwriting on the text on the screen

Scientists are exploring different ways to enable people with disabilities to communicate with their minds. Go back to handwriting, the latest and fastest way to express yourself.

Researchers have identified brain activity associated with trying to write letters by hand for the first time. Researchers used algorithms to recognize letters when they tried to write them in collaboration with paralyzed volunteers who had sensors embedded in their brains. The text was then displayed on the screen in real time by the system.

According to Krishnachenoy, a researcher at the Howard Hughes Medical Institute at Stanford University, who co-authored the study, and co-directed the study with Stanford neurosurgeon Jaimie Henderson, the technique allowed people with paralysis. You will be able to type quickly without using your fingers.

According to Shenoy and his colleagues, study participants wrote 90 characters per minute while trying to handwrite. This is more than four times the previous benchmark for typing in such a “brain computer interface”.

According to Jose Carmena, a brain engineer at the University of California, Berkeley who was not involved in the study, this and other technologies could benefit people with a variety of disabilities. “This is an important advance in this area,” he adds.

According to Carmena, Brain Computer Connection turns thoughts into actions. “This document is a great example. The interface decodes the written concept and generates the action.”

Communication supported by thought

Even if a person becomes immobile due to an accident or illness, the neural activities of the brain such as walking, drinking coffee, and uttering phrases remain. Researchers can use this exercise to help patients who are incapacitated due to paralysis or amputation.

The need depends on the type of impairment. Some people with hand injuries can continue to use PCs with voice recognition software and other programs. Other techniques that help people communicate have been developed by scientists for those who have difficulty speaking.

For the past few years, Shenoy’s team has deciphered speech-related brain activity with the goal of replicating the speech. They also created a way for individuals with embedded sensors to move the pointer on the screen, using ideas related to arm movement attempts. By pointing and pressing a character in this way, I was able to enter about 40 characters per minute, breaking the previous speed record for writing using the Brain Computer Interface (BCI).

On the other hand, no one glanced at the handwriting. Frank Willett, a neuroscientist in Shenoy’s lab, wondered if the urges of the brain triggered by placing a pen on paper could be used. “We want to discover new ways for individuals to communicate faster,” he says. He was also fascinated by the possibility of trying something new.

The researchers collaborated with participants in a BrainGate 2 clinical study evaluating the safety of BCI, which sends data directly from the brain to a computer. (Leigh Hochberg, a neurosurgeon and neuroscientist at Brown University Massachusetts General Hospital, and Providence VA Medical Center are responsible for the study.) Henderson puts two small sensors in the brain area to adjust his hands and arms. I inserted it. An individual who manipulates a robot’s arm or a cursor on a computer screen by moving his paralyzed arm.

A subject, 65 years old at the time of the study, was paralyzed from the neck down due to a spinal cord injury. The machine learning system used data obtained by sensors from sensory cells to detect patterns created by his brain at each letter, as he imagined writing. The man was able to use this technique to duplicate the phrase and answer the question as fast as someone of his age was writing on his smartphone.

According to Willett, this so-called “brain-to-text” BCI is very fast because each letter evokes a different pattern of activity, making it very easy for algorithms to distinguish from each other.

New system implemented

Shenoy’s team plans to employ attempted handwriting for text input as part of a larger system that incorporates point-and-click navigation and attempted voice decoding similar to those found on today’s smartphones. I am. “It’s what we do naturally to have these two or three modes and switch between them,” he explains.

According to Shenoy, the team will then work with people who can’t speak, such as those with amyotrophic lateral sclerosis and degenerative brain disease that causes loss of mobility and voice.

Henderson says the new method may benefit people who are paralyzed due to various illnesses. Among them is Jean-Dominique Bauby, a diving bell and butterfly writer who suffered from a brainstem attack. “He was able to create this emotional and beautiful novel by using eye movements to carefully select characters one by one,” Henderson wrote. “Imagine what he did with Frank’s writing interface!” Exclaims the narrator.


BCI has proven to be effective in communication in patients with chronic paralysis or in LIS. BCI allows users to communicate their goals directly without the need for motor perimeters. Patients who lose control of the effects of eye movements on disease progression and injury due to the introduction of non-visual BCI can benefit from this technology. Normative studies are needed to establish the predictive usefulness of ERP or mental image classification for the use of BCI before using the proposed hierarchical methodology for cognitive processing in DOC patients.

Convert mental handwriting into on-screen text

Source link Convert mental handwriting into on-screen text

Related Articles

Back to top button