Brain-Machine Interface and the Walk Again Project
In 1999, exactly two decades ago, Dr. John Chapin, Dr. Miguel Nicolelis, and collaborators published the first scientific study demonstrating that electrical signals generated by populations of motor neurons, recorded in awake rats, could be extracted and converted into real-time commands for a robotic arm. This innovative experiment evidenced the enormous potential of this technology, called the brain-machine interface (BMI), to restore movements in persons with paralysis.
Since then, research in BMI has advanced, and the studies of Dr. Nicolelis and his team stand out in the following areas: creation and improvement of technologies for registration of populations of neurons; control of robotic devices, assistive devices, and virtual arms and legs; treatment of Parkinson's disease and epilepsy symptoms; development of brain-brain interface and brain networks for task synchronization; among others.
In 2009, based on the experience gained in his career, Dr. Nicolelis brought together a team of neuroscientists, roboticists, neuroengineers, computer scientists, neurosurgeons, and rehabilitation professionals to form the international non-profit scientific consortium called the Walk Again Project, created with the mission of developing a robotic suit (exoskeleton) controlled by the human brain capable of restoring locomotion in persons with paralysis. Furthermore, in 2009, the IINN-ELS, the scientific research development center of AASDAP, became responsible for the coordination of the Walk Again Project in Brazil.
As a way of popularizing the potential of BMI to rehabilitate persons with paralysis, the Walk Again consortium proposed to the Brazilian government and FIFA the first public demonstration of the exoskeleton during the opening ceremony of the World Cup in Brazil in 2014. This proposal was accepted and, in December 2012, AASDAP, manager of the Walk Again Project in Brazil, signed an agreement with the Financing Agency for Studies and Projects (FINEP) for the development of the brain-controlled exoskeleton. In partnership with the Association for Assistance of Disabled Children (AACD), the first laboratory of the Walk Again Project was set up within the AACD facilities in São Paulo, and the clinic staff of this institution assisted in the recruitment of paraplegic volunteers for the research and clinical professionals for the Project.
Between December 2012 and June 2014, a multidisciplinary team of 156 professionals from various countries and institutions worked uninterruptedly to design, develop, and test the entire Exo’s structure and its BMI communication system.
In summary, Exo works as follows: (1) the user imagines the movement of one of his legs (for example, the left) and this thought generates brain signals, (2) the electroencephalography (EEG) system captures these brain signals through electrodes placed in a cap on the user's head, exactly above the areas that represent the legs in the brain, (3) the received signals are sent to a processor that encodes them into Exo activation commands, (4) Exo moves the left leg, (5) pressure and distance sensors installed on Exo's feet generate information that is encoded in real time as tactile stimuli applied to the user's forearm (body area with intact sensitivity), providing the virtual sensation of the foot touching the floor or ball. These tactile stimuli are important because the paraplegic volunteers have no sensitivity below the level of the spinal cord injury; in this way, when receiving the stimuli in the forearm during the walk with the Exo, the user has the sensation of walking with his own legs and not of being carried by the equipment.
Exo's tests were successfully performed on eight paraplegic volunteers. On June 12, 2014, Juliano Pinto, representing the other volunteers of the research, gave the kick-off of the World Cup and was able to demonstrate, for an audience of more than 70,000 fans, besides the billions of viewers worldwide, the viability of the control of a robotic device by the human brain.
Neurological Improvement and the Neurorehabilitation Protocol
After the World Cup Ceremony, the Walk Again Project continued, and during medical evaluations, the clinical team noticed that the volunteers were developing some level of sensory perception and motor control in the areas below the spinal cord injury.
To investigate this neurological improvement, our team continued the training routine that the volunteers performed during Exo’s development research. This routine, which we call the Neurorehabilitation Protocol, consists of conventional physiotherapy (strengthening and stretching), robotic gait training with Lokomat, BMI robotic gait training with Exo, conventional gait training, and virtual reality training with BMI.
After 12 months of training, some volunteers partially recovered their ability to control their leg muscles and feel touch and pain in their paralyzed limbs, even though they were originally diagnosed with chronic paraplegia, in some cases for more than a decade. Volunteers also recovered some level of bladder and bowel control and had improved cardiovascular function. Thus, four volunteers (50% of the study population) had a change in their AIS classification (scale for the neurological classification of spinal cord injury given by the American Spinal Injury Association): three went from AIS A to AIS C and one went from AIS B to AIS C. These results are published in the journal Scientific Reports.
In the 28-month training period, neurological improvement continued for the seven volunteers who remained on the protocol. Neurological improvement did not occur only for one volunteer who interrupted participation in the study after 12 months. The seven volunteers who continued the neurorehabilitation protocol recovered significant levels of nociceptive sensation below the injury level, with improved voluntary motor functions and sensory functions for several modalities (proprioception, tactile, pressure, and vibration). They also partially recovered intestinal, urinary, and sexual functions. At the end of 28 months, all volunteers had their AIS scale improved (six went from AIS A to C, and one went from AIS B to AIS C). This clinical evolution also resulted in an improvement in their quality of life, as assessed by standardized psychological tests. The psychological follow-up of the volunteers showed that, in general, participation in the neurorehabilitation protocol also promoted a sense of empowerment, which strengthened the volunteers' sense of competence, self-worth, and self-esteem. These results are published in the journal PLOS ONE.
Improvement of the neurological classification of the volunteers of the Walk Again Project. The y-axis indicates the neurological rating of each volunteer according to the AIS scale and the x-axis indicates the months of participation in the protocol.
In addition to the application of the neurorehabilitation protocol to paraplegic volunteers, our team developed, in partnership with the Laboratory of Robotic System of the École Polytechnique Federale de Lausanne (EPFL), a non-invasive system of functional electrical stimulation (FES) controlled by the brain (BMI-sFES). This system integrated the following technological innovations: 16 sFES channels for electrical stimulation of muscles to generate steps, taking into account all subphases of the physiological gait; activation of sFES from the detection of the electrical signals of the user's right and left legs (captured by EEG); and, application of vibratory stimuli in the user's forearm to provide the virtual sensation of the lower limbs.
When triggered by the electrical activity of the user's brain, generated by the thought of moving his leg, the BIM-sFES emits electrical pulses to activate individually and orderly 16 leg muscles (eight in each leg), following the sequence of muscle activation performed in normal gait. This orderly activation allows the individual to carry out the gait movements naturally (smoothly and continuously), using only the support of a walker and partial body support. Vibratory stimuli applied to the user's forearm give the virtual feel of the touch of the feet on the ground, which helps the user to perceive the events of the gait.
The BMI-sFES was tested and validated with two volunteers with chronic paraplegia who walked safely with 65-70% body weight support and accumulated a total of 4,580 steps with this setting. For these volunteers, we observed gait improvement (less dependence on walking assistance), increased muscle volume, and partial neurological recovery, and one of them had substantial rates of motor improvement. These results are in the process of being published in the scientific journal Scientific Reports.
Functional electrical stimulation system controlled by the brain (BMI-s FES), developed in Walk Again Project.
To date, our studies are the only ones in the scientific literature to demonstrate that the use of non-invasive, non-drug-based brain-machine interfaces affects the recovery of sensory and motor functions in persons with complete paraplegia. These results bring hope to thousands of persons affected by paralysis or movement disorders caused by neurological diseases and this is certainly a great step towards developing treatments that ensure their quality of life.
The international consortium of the Walk Again Project had the collaboration of several institutions:
Association for Assistance of Disabled Children – AACD (Brazil);
BIA Turnkey Systems, Paris (France);
Brain-Machine Interface Institute of Science and Technology – INCeMaq (Brazil);
Cardinal Hill Rehabilitation Hospital, Health Care Kentucky University, (USA);
Colorado State University (USA);
Duke Immersive Virtual Environment, Duke University (USA);
Duke University Center for Neuroengineering (USA);
École Superieure de Physique et Chimie Industrielles de La Ville de Paris (France);
Edmond and Lily Safra International Institute of Neuroscience of Natal – IIN-ELS (Brazil);
International Neuroscience Network Foundation (INNF);
Laboratoire de Systèmes Robotiques, Ecole Polytechnique Federale de Lausanne – EPFL (Switzerland);
Neuroprosthetic Center at Ecole Polytechnique Federale de Lausanne – EPFL (Switzerland);
Robotics, Autonomous Systems, and Controls Laboratory, UCDavis, (USA);
Robotics Group at ATR Laboratories of Kioto (Japan);
Technical University of Berlin (Germany);
Technical University of Munich (Germany).
The panel installed in the Laboratory in São Paulo, with the names of all the collaborators of Walk Again Project
Articles in scientific journals:
A. Selfslagh, S. Shokur*, D. S.F. Campos, A. R. C. Donati, S. Almeida, S.Y. Yamauti, D.B. Coelho, M. Bouri, M. A. L. Nicolelis (2019). Non-invasive, Brain-controlled functional electrical stimulation for locomotion rehabilitation in paraplegic patients. Scientific Reports. 9:6782. https://doi.org/10.1038/s41598-019-43041-9
Solaiman Shokur, Ana R.C. Donati, Debora S.F. Campos, Claudia Gitti, Guillaume Bao, Dora Fischer, Sabrina Almeida, Vania A.S. Braga, Patricia Augusto, Chris Petty, Eduardo J.L. Alho, Mikhail Lebedev, Allen W. Song, Miguel A.L. Nicolelis. (2018). Training with brain-machine interfaces, visuo-tactile feedback and assisted locomotion improves sensorimotor, visceral, and psychological signs in chronic paraplegic patients. PLOS ONE 13(11):e0206464. https://doi.org/10.1371/journal.pone.0206464
Solaiman Shokur, Simone Gallo, Renan C. Moioli, Ana Rita C. Donati, Edgard Morya, Hannes Bleuler & Miguel A.L. Nicolelis. (2016). Assimilation of virtual legs and perception of floor texture by complete paraplegic patients receiving artificial tactile feedback. Scientific Reports 6, 32293. https://doi.org/10.1038/srep32293
Ana R. C. Donati, Solaiman Shokur, Edgard Morya, Debora S. F. Campos, Renan C. Moioli, Claudia M. Gitti, Patricia B. Augusto, Sandra Tripodi, Cristhiane G. Pires, Gislaine A. Pereira, Fabricio L. Brasil, Simone Gallo, Anthony A. Lin, Angelo K. Takigami, Maria A. Aratanha, Sanjay Joshi, Hannes Bleuler, Gordon Cheng, Alan Rudolph, Miguel A. L. Nicolelis. (2016). Long-Term Training with a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery in Paraplegic Patients. Scientific Reports 6, 30383; doi: 10.1038/srep30383
M. Bouri, A. Selfslagh, D. Campos, S. Yonamine, A. Donati, and S. Shokur (2019). "Closed-Loop Functional Electrical Stimulation for Gait Training for Patients with Paraplegia." In 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1489-1495. IEEE.
B. Hannes, T. Vouga, A. Ortlieb, R. Baud, J. Fasola, J. Olivier, S. Shokur, and M. Bouri (2018). Exoskeletons as Mechatronic Design Example. New Trends in Medical and Service Robotics (MESROB), pp. 109-117. Springer, Cham.
D. J. Zielinski, R. P. McMahan, S. Shokur, E. Morya, and R. Kopper (2014). Enabling Closed-Source Applications for Virtual Reality via OpenGL Intercept-based Techniques. IEEE 7th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS), pp. 1–6.
Poster in scientific meetings:
Shokur, S.; Asnis, F.; Almeida S.; Nicolelis, M.A.L. The peripersonal space representation in paraplegic patients depends on the level of lower-limb residual neurological functions. In: 48th Society for Neuroscience, 2018, San Diego.
Shokur, S.; Donati, A.R.C; Nicolelis, M.A.L. Long-term training with no-invasive brain machine-interfaces and locomotion promotes neurological improvements in patients with chronic complete paraplegia: a pilot clinical trial. In: 47th Society for Neuroscience, 2017, Washington.
Selfslagh, A.; Shokur, S.; Donati, A. R. C; Campos, D. S. F; Almeida, S. B de; Padula, N.; Bleuler, H.; Bouri, M.; Nicolelis, M.A.L. Locomotion training with closed-loop brain-machine interface and lower-limb functional electrical stimulation for complete paraplegic patients. In: 47th Society for Neuroscience, 2017, Washington.
Campos, Debora S. F.; Selfslagh, Aurélie; Shokur, Solaiman; Donati, Ana R. C.; Fisher, Dora; Bouri, Mohamed; Bleuler, Hannes; Nicolelis, Miguel A. L. Developing a new synergic muscle contraction gait model produced by surface functional electrical stimulation (FES) in humans after complete spinal cord injury (SCI). In: Annual Meeting of the International Functional Electrical Stimulation Society (IFESS), 2017, London.
Donati, Ana R. C.; Shokur, Solaiman; Campos, Debora S. F.; Nicolelis, Miguel A. L. Development of a New Motor Assessment for Spinal Cord Injury Patients. In: International Neurorehabilitation Symposium (INRS), 2017, London.
Nicolelis, Miguel A. L.; Donati, Ana R. C.; Shokur, S.; Morya, E. Brain-Machine-Interface Based Neurorehabiltation Induces Partial Neurological Recovery in Paraplegic Patients. In: 9th World Congress for Neurorehabilitation, 2016, Philadelphia.
Donati, A.R.C.; Shokur S.; Campos, D. S. F.; Pires, Cristhiane G.; Fischer, D.; Morya, E.; Nicolelis, M.A.L. Improvement of Trunk Stability in Chronic Paraplegic Patients After Long-Term Training With Robotic Orthotic Trainers. In: 9th World Congress for Neurorehabilitation, 2016, Philadelphia.
Shokur S.; Donati, A. R. C.; Moioli, R. C.; Nicolelis, M. A. L. Tactile Feedback Restoration Using Sensory Substitution In Chronic Paraplegic Patients. In: 9th World Congress for Neurorehabilitation, 2016, Philadelphia.
V. Braga, A. Donati, S. Shokur, M. Nicolelis. A melhora da função intestinal dos pacientes com lesão medular crônica submetidos a treinamento de neuroreabilitação de longo prazo. In: 25° Congresso Brasileiro de Medicina Física e Reabilitação, 2016, São José do Rio Preto.
D. Campos, A. Donati, D. Fisher, S. Shokur, M. Nicolelis Programa de reabilitação ativa para lesão medular completa: impacto sobre a recuperação neurológica motora. In: 25° Congresso Brasileiro de Medicina Física e Reabilitação, 2016, São José do Rio Preto.
P. Augusto, C. Gitti, A. Donati, S. Shokur, M. Nicolelis Mudanças na imagem corporal de pacientes com lesão medular crônica e motoramente completa após experimento com realidade virtual. In: 25° Congresso Brasileiro de Medicina Física e Reabilitação, 2016, São José do Rio Preto.
Shokur, S; Gallo, S; Moioli, R; Bouri, M; Morya, E; Bleuler, H; Nicolelis, Miguel A. L. Inducing paraplegic patients to perceive distinct ground textures using tactile feedback generated by virtual feet. In: 45th Society for Neuroscience Meeting, 2015, Chicago.
Moioli, Renan Cipriano; Shokur, S.; Gallo, S.; Brasil, Fabricio Lima; Morya, Edgard; Nicolelis, Miguel A. L. Cortical incorporation of virtual legs in spinal cord injured patients. In: 45th Society for Neuroscience Meeting, 2015, Chicago.
Aratanha, Maria Adelia; Shokur, S.; Brasil, Fabricio Lima; Donati, A. C.; Gallo, S.; Morya, Edgard; Nicolelis, Miguel A. L. Closed loop brain controlled avatar training for locomotion with spinal cord injured patients. In: 45th Society for Neuroscience Meeting, 2015, Chicago.
Donati, A. C.; Shokur, S.; Morya, Edgard; Gitti, C. M.; Augusto, P. B.; Campos, D.; Yoshihara, D.; Nicolelis, Miguel A. L. Twelve months of physical rehabilitation protocol integrating brain controlled locomotor training and tactile feedback for patients with chronic spinal cord injury. In: 45th Society for Neuroscience Meeting, 2015, Chicago.
Brasil, Fabricio Lima; Shokur, S.; Aratanha, Maria Adelia; Moioli, Renan Cipriano; Donati, A. C.; Morya, Edgard; Nicolelis, Miguel A. L. Walk using single leg control at BMI-driven exoskeleton. In: 45th Society for Neuroscience Meeting, 2015, Chicago.
Brasil, F.; Moioli, R. C.; Shokur, S.; Fast, K.; Lin, A.; Peretti, N.; Takigami, A.; Lyons, K.; Zielinski, D.; Sawaki, L.; Joshi, S. ; Morya, Edgard; Nicolelis, Miguel A. L. The Walk Again Project: An EEG/EMG training paradigm to control locomotion. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Lin, A. ; Schwarz, D. ; Sellaouti, R. ; Shokur, S. ; Moioli, R. ; Brasil, F. ; Fast, K. ; Peretti, N. ; Takigami, A. ; Gallo, S. ; Lyons, K. ; Mittendorfer, P. ; Lebedev, M. ; Joshi, S. ; Cheng, G. ; Morya, Edgard ; Rudolph, A. ; Nicolelis, Miguel A. L. The walk again project: Brain-controlled exoskeleton locomotion. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Moioli, R. C.; Brasil, F.; Shokur, S.; Lin, A.; Fast, K.; Peretti, N.; Takigami, A.; Schwarz, D.; Morya, Edgard; Nicolelis, Miguel A. L The Walk Again Project: Analysis of brain activity of spinal cord injury patients during training with a BMI. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Nicolelis, Miguel A. L.; Shokur, S.; Lin, A.; Moioli, R. C.; Brasil, F.; Peretti, N.; Fast, K.; Takigami, A.; Morya, Edgard; Cheng, G.; Sawaki, L.; Kopper, R.; Schwarz, D.; Gallo, S.; Lebedev, M.; Joshi, S.; Bleuler, H.; Rudolph, A. The Walk Again Project: Using a Brain-Machine Interface for establishing a bi-directional Interaction between paraplegic subjects and a lower limb exoskeleton. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Sawaki, L.; Donati, A. C.; Nogueira, A. N.; Garabello, C.; Gitti, C. M.; Campos, D.; Yoshihara, D.; Pereira, G. A.; Araujo, I.; Augusto, P. B.; Tripodi, S.; Morya, Edgard; Nicolelis, Miguel A. L. Novel rehabilitative strategy to facilitate EEG-triggered locomotor training in chronic spinal cord injury patients: Preliminary results of an ongoing study. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Shokur, S.; Gallo, S.; Olivier, J.; Takigami, A.; Lin, A.; Fast, K.; Moioli, R.; Brasil, F.; Morya, Edgard; Cheng, G.; Bleuler, H.; Nicolelis, Miguel A. L. The walk again project: Sensory feedback for brain controlled exoskeleton. In: 44th Society for Neuroscience Meeting, 2014, Washington.
Zielinski, D.; McMahan, R. P.; Shokur, S.; Morya, Edgard; Kopper, R. Enabling Closed-Source Applications for Virtual Reality via OpenGL Intercept Techniques. In: 7th Workshop on Software Engineering and Architectures for Realtime Interactive Systems, 2014, Minneapolis. v.7Show more... Show less...