Mechanics, mobility, and motion capture: From Pixar to reality

When you watch a Pixar movie, have you ever wondered why the characters’ movements and facial expressions appear so vivid and realistic?

Illustration by Eric Chung for rehabINK

By Yilina Liubaoerjijin, John Christy Johnson, & Martin Ferguson-Pell, rehabinkmag February 5, 2020

The next time you sit down for family movie night, you may recognize that these animations are possible thanks to cutting-edge motion capture (mocap) technology.

Three-dimensional (3D) mocap systems are commonly used for a variety of applications ranging from computer graphics in cinema, video game design, and virtual reality renderings. However, mocap is not limited to animating cartoon characters – it is also a powerful tool that helps us learn about human biomechanics.

In this knowledge summary, we briefly describe two applications of mocap:

monitoring gait in persons with obesity, and
tracking propulsion styles in manual wheelchair users.

We conclude this article by envisioning exciting future avenues for using mocap to deliver more effective and efficient rehabilitation.

Mocap systems can measure the ranges, angles, and speed of movements of people and objects with sub-millimetre accuracy. Extracting these types of metrics for assessment purposes is valuable in rehabilitation contexts. The applications of mocap are numerous even within this one field, being used for everything from prosthetics to remote haptics, which is a kind of user-computer interface.

A typical mocap session in a research or clinical setting looks something like this: a participant or patient enters a recording space where they wear attire outfitted with infrared markers on different body landmarks (Figure 1). As the person performs different functional tasks, the mocap system records movements of the body parts with its attached markers.

Figure 1. The motion capture suit with infrared markers attached. University of Alberta

The mocap system then generates a video of moving dots meant to reconstruct the person/limb completing the tasks. The x, y, and z coordinates for each marker from the motion data are read into software called Motive, which produces a point-light display of the individual as they walk (Figure 2). In turn, we can use this data to do motion analysis, clinical diagnosis, and animations.

Figure 2. The point-light display represented as a full-body skeleton in Motive 2.1.1 software

Stepping forward: Gait for persons with obesity

People with obesity often experience altered walking patterns, poor balance, and postural control problems due to excessive body fat. Information on the way people move around in 3D space can help clinicians provide exercise prescriptions and identify risks of falling and signs of compensation that can lead to injury. By using this mocap information to identify signs of movement impairments, clinicians may be able to introduce treatments and prevention strategies at early stages of musculoskeletal disorders which are caused by or related to obesity.

Consequently, the risk of severe falling incidents, which may lead to bone fractures and other complications, is reduced. For instance, gait parameters such as posture, walking speed, and movement range of joints are used as indicators of irregular or injurious forms of walking. Gait analysis―the systematic study of walking patterns―in people with obesity has been done in various studies, many of which utilized mocap for data collection.

In addition to the types of data mocap systems can capture, assessments using these systems are accurate and reliable compared to other wearable biomechanical analysis systems (e.g., pressure and inertial sensors). Therefore, using mocap systems minimizes the number of trials needed to collect accurate data. This efficiency is important because obesity is associated with general fatigue; from an assessor standpoint, swift execution and collection of data is vital to create a seamless, non-fatiguing experience for the individual. With mocap, lightweight markers can be quickly attached/removed, which is a minimal burden for participants with obesity.

Most of the mocap systems are compatible with other devices used in rehabilitation, like force plates which capture specific biomechanical metrics. By using mocap with these other rehabilitation devices, much more information (like reaction forces acting on the body or the center of mass during gait) is captured at once. These additional parameters allow quicker and more accurate clinical assessments versus conventional methods like manual measurement and naked-eye observation.

However, one challenge to using mocap systems with people with obesity is that the movements of excess skin and fat may interfere with the musculoskeletal movements data. Fortunately, we can minimize this issue by grouping distinct markers (instead of a single marker on the desired landmark) and by cleaning data (removing errors and ensuring consistency).

Mocap allows for “quicker and more accurate clinical assessments versus conventional methods like manual measurement and naked-eye observation.”

In clinical settings, the mocap system can be especially beneficial for rehabilitation in rural areas with limited access to healthcare providers. Rural healthcare facilities can set up the mocap system; local clients wearing mocap attire can then perform tasks asked by clinicians located elsewhere. Clinicians in another location can therefore see the clients virtually.

Another feature about the mocap system is that the video of each task can be recorded for clinicians to revisit. With these recordings, clinicians can easily identify differences before and after treating the clients. The quantified data may also improve diagnostic accuracy, thus enhancing the likelihood of prescribing proper treatment.

Pushing forward: Propulsion styles and wheelchair biomechanics

Imagine having to live in an oversized house as a person with small stature – reaching objects in high cabinets is challenging, detecting hazards is harder as your field of vision is limited, and navigating becomes generally more difficult.

Barriers like these are inconvenient and dangerous, yet commonplace for wheelchair users. Because the hands, arms, and shoulders are not suited to adapt to inaccessible terrain designed for those who walk, overuse injuries are highly common in manual wheelchair users. Moreover, up to 75% of wheelchair users report upper limb pain following longer-term usage of their wheelchair.

Consequently, using a manual wheelchair may be daunting for people who are not as strong and who have decreased control of their abdominal or back muscles. People with spinal cord injury represent a large percentage of wheelchair users and typically have poorer sensation of touch and control over their arm movements. Therefore, adequate wheelchair setup and training are vital to empowering them to carry out their everyday tasks.

This problem beckons us to ask – how can we better detect, measure, and prevent such cases of wheelchair-related injury? One plausible answer suggested by scientists is through the study of wheelchair biomechanics. Wheelchair biomechanics involves examining the forces, ranges, and patterns of movement to draw insights about the cause of injury. Unsurprisingly, mocap can be used to investigate these stylistics of motion.

Mocap is a compelling piece of technology to study the complexities of a person self-pushing a wheelchair. With mocap, we can develop an upper body skeleton model that accurately records postures and patterns while the user is pushing. Regarding wheelchair propulsion, mocap can revolutionize the way we inform exercise and clinical guidelines.

For example, manual wheelchair stroke patterns are typically categorized into four styles of propulsion (Figure 3). Mocap can empower us to break free from these “convenient” classifications and collect individualized kinematics of propulsion, as fit for each wheelchair user. This individualized biomechanical data can then inform specific clinical considerations or recommendations, whether for the user, the wheelchair, or clinicians.

Figure 3. A simple classification system of four wheelchair propulsion styles, which is not representative of the peculiarities of the push itself.

Several studies have used mocap to examine the specific propulsion styles of wheelchair users and how improper styles can lead to injury. However, the true value of mocap is in the way it can be leveraged. The University of Alberta Rehabilitation Robotics Lab is exploring ways we can inform clinical guidelines by visualizing injury mechanisms through the creation of biomechanical models (Figure 4). In conjunction with force-measuring technology, we can provide biomechanical models as a way for healthcare professionals, academics, and policymakers to understand, recommend, and educate manual wheelchair users on how to push properly and safely, thus avoiding injury.

Figure 4. A biomechanical model produced using TheMotionMonitor software illustrating the hand motion and stresses on shoulder, elbow, wrist and neck joints while pushing a wheelchair.

Next steps: Markerless mocap

The next generation of mocap has the potential to reimagine rehabilitation as we know it. With some companies using markerless mocap, people no longer need to wear special equipment for tracking. All that is required would be a 3D camera and body landmark detection software. This simplicity is considered mocap’s “Holy Grail” by some, despite it being less accurate in identifying and tracking anatomical landmarks than conventional marker-based mocap. Markerless mocap applications have tremendous potential for remote rehabilitation.

For example, consider Jane, a prospective patient for shoulder joint repair who lives in remote northern Alberta. Jane needs an assessment to determine if she qualifies for surgery. Since there are no rehabilitation centres nearby, travelling to a hub clinic with access to specialist care costs Jane a significant amount of time, effort, and resources. Quite possibly, Jane may then find out that she does not meet the criteria for surgery. This painstaking process could be avoided by holding a teleconference meeting with the surgical specialist at the remote clinic and by conducting the preliminary assessment using a markerless mocap system (Figure 5).

Figure 5. User interface of Kinetisense, an Alberta-based markerless motion capture system, providing wheelchair push style (kinematic) data.

Beyond the importance of mocap in providing relevant measures for kinematic research, these innovative applications also have potential clinical impact. Further, this suggests the importance of using these kinds of mocap techniques to improve the quality of diagnosis and functional mobility for individuals living with diverse disabilities.

Whether it be cartoon animation or rehabilitation, using mocap to record and recreate the precise way we move saves us time, resources, and effort.

Source rehabinkmag

References

Increased risk for falling associated with obesity: mathematical modeling of postural control, Corbeil P, Simoneau M, Rancourt D, Tremblay A, Teasdale N. IEEE Trans Neural Syst Rehabil Eng. 2001 Jun;9(2):126-36.
Effects of obesity on functional capacity, Pataky Z, Armand S, Müller-Pinget S, Golay A, Allet L. Obesity (Silver Spring). 2014 Jan;22(1):56-62. doi: 10.1002/oby.20514. Epub 2013 Aug 13. Full text
Measurement of lower extremity kinematics during level walking, Kadaba MP, Ramakrishnan HK, Wootten ME. J Orthop Res. 1990 May;8(3):383-92.
Three-dimensional gait analysis of obese adults, Lai PP, Leung AK, Li AN, Zhang M Clin Biomech (Bristol, Avon). 2008;23 Suppl 1:S2-6. doi: 10.1016/j.clinbiomech.2008.02.004. Epub 2008 Apr 18.
Biomechanical gait analysis in obese men, Spyropoulos P, Pisciotta JC, Pavlou KN, Cairns MA, Simon SR. Arch Phys Med Rehabil. 1991 Dec;72(13):1065-70.
Technologies for Advanced Gait and Balance Assessments in People with Multiple Sclerosis, Shanahan CJ, Boonstra FMC, Cofré Lizama LE, Strik M, Moffat BA, Khan F, Kilpatrick TJ, van der Walt A, Galea MP, Kolbe SC. Front Neurol. 2018 Feb 2;8:708. doi: 10.3389/fneur.2017.00708. eCollection 2017. Review. Full text
The association of obesity, cytokine levels, and depressive symptoms with diverse measures of fatigue in healthy subjects, Lim W, Hong S, Nelesen R, Dimsdale JE. Arch Intern Med. 2005 Apr 25;165(8):910-5.
Detailed shoulder MRI findings in manual wheelchair users with shoulder pain, Morrow MM, Van Straaten MG, Murthy NS, Braman JP, Zanella E, Zhao KD. Biomed Res Int. 2014;2014:769649. doi: 10.1155/2014/769649. Epub 2014 Aug 11. Full text
Functional outcomes and disability after nontraumatic spinal cord injury rehabilitation: Results from a retrospective study, New PW. Arch Phys Med Rehabil. 2005 Feb;86(2):250-61.
Manual wheelchair propulsion patterns on natural surfaces during start-up propulsion, Koontz AM, Roche BM, Collinger JL, Cooper RA, Boninger ML. Arch Phys Med Rehabil. 2009 Nov;90(11):1916-23. doi: 10.1016/j.apmr.2009.05.022.
Shoulder pain and jerk during recovery phase of manual wheelchair propulsion, Jayaraman C, Beck CL, Sosnoff JJ. J Biomech. 2015 Nov 5;48(14):3937-44. doi: 10.1016/j.jbiomech.2015.09.018. Epub 2015 Oct 9. Full text
Comparison between performance with a pushrim-activated power-assisted wheelchair and a manual wheelchair on the Wheelchair Skills Test, Best KL, Kirby RL, Smith C, MacLeod DA. Disabil Rehabil. 2006 Feb 28;28(4):213-20. Review.
Motion capture and data elaboration to analyse wheelchair set-up and users’ performance, Regazzoni D, Vitali A, Rizzi C, Zefinetti FC. Proceedings of the ASME 2018 International Mechanical Engineering Congress and Exposition. Volume 13: Design, Reliability, Safety, and Risk. Pittsburgh, Pennsylvania, USA. November 9–15, 2018. V013T05A005. ASME. https://doi.org/10.1115/IMECE2018-87531

Running Analysis – markerless motion capture – Simi Shape. Exemplary treadmill running analysis done with Simi Shape. Tracking was done without any markers. 3D bone model on the right. On the left you can see our runner from different camera perspectives which are necessary for markerless 3D analysis. We also included a look at his center of gravity and knee angles. Analysis options are not limited to the ones in this video – Let’s talk about it! Simi Systems. Youtube May 4, 2017

CAMERA Showreel. University of Bath. Vimeo August 14, 2017

Hollywood motion capture technology finds a new role in hospital rehab. Alex Hogan, STAT February 12, 2020

Xsens MVN BIOMECH – Wearable Real-time Kinematics. Wearable full-body human measurement system based on inertial sensors, biomechanical models and sensor fusion algorithms. Easy to use, short setup time and instant data output. Can be used anywhere. Xsens. Youtube Feb 9, 2015