Helmet study highlighted in The New York Times is flawed

Cranial remolding expert Ellie Boomer CPO, shares her commentary on the helmet therapy article highlighted in the May 1 issue of The New York Times.

UCSF Benioff Children’s Hospital

By Ellie Boomer CPO LPO, The O&P Edge August 4, 2014

As a professional who specializes in treating children with cranial deformation by using a cranial remolding orthosis (CRO), I was surprised to see The New York Times (NYT) article Helmets Do Little to Help Moderate Infant Skull Flattening, Study Finds, which discussed a study from the Netherlands that called into question the efficacy of CROs. For a moment, I was forced to consider the possibility that I have been correcting cranial deformations that would have corrected on their own.

Over the last 12 years, I have seen effective correction of cranial deformation when parents and caregivers are compliant with a 23-hours-per-day, full-time wear schedule with a well-fitting, active-design CRO. I have also remeasured infants’ skulls after they were involved in repositioning programs and then recommended CRO treatment because the deformational asymmetries failed to correct with that protocol. Also, in follow-up with patients one to two years post-CRO treatment, I have typically seen the head increase in size, a slight decrease in the cephalic ratio/cranial index, and the cranial vault holding.

Thus, even though Mark Proctor MD, vice chair of neurosurgery at Boston Children’s Hospital, Massachusetts, and an assistant professor at Harvard Medical School, Boston, was quoted as saying that the study was “rigorous,” my own clinical experience led me to believe that there were factors in this study that had not been included in the NYT article’s summary of methods and results that contradict Proctor’s assessment.

The study referred to in the NYT article, was published online May 1 in BMJ, an open-access, peer-reviewed journal. This is the first randomized controlled trial on helmet therapy, and I agree with Proctor that the method of enrolling patients, follow-up with the enrolled patients, and methods of measuring the asymmetries were strong.

However, the study’s conclusion that there “was no conclusive evidence that a significant or clinically meaningful difference in improvement of skull shape was found at the 24-month follow-up between the two groups” is not consistent with my own clinical results or with previous literature. It appears that the reason van Wijk et al.’s study was unable to show the effectiveness of treatment with a CRO lies in the many weaknesses of the study, among which are the low participation rate, incomplete reporting, dissimilar cohort groups, and questionable averaging of data.

Study Introduction

The study cohort was found by asking parents of five-month-old infants to participate in a randomized controlled study. Of the initial 883 infants enrolled for baseline measurements, only 401 met the inclusion criteria, and only 84 parents (21 percent of the eligible infants) agreed to participate in the study. This number is low but perhaps understandable since the study parameters included the possiblity the child would not receive treatment.

The 84 infants were randomly assigned (1:1) to either helmet therapy for six months or no treatment, and were remeasured at eight, 12, and 24 months of age. Primary outcomes were evaluated using anthropometric measurements at 24 months of age, which were blinded at 24 months only. The measurements were taken using plagiocephalometry (PCM) – a thermoplastic material that is wrapped around the child’s head, at which time the position of the child’s nose and ears are marked on the material. This material is then photocopied and the measurements are taken from the paper. Previous scholarly literature has documented the intrarater and interrater reliability of this method.^[1]

With the exception of the diagonal measurements having been taken at 40 degrees from midline, instead of the standard 30 degrees used in the United States, and the plagiocephaly index having been calcuated differently from the U.S. method of calcuating the cranial vault asymmetry index (CVAI), the results provide similar measures of plagiocephaly and brachycephaly, as shown in Table 1.

Table 1

According to the study, the researchers had a 94 percent follow-up rate, and they did statistical analysis on two outcome variables: change in the oblique diameter difference index (ODDI), which is equivalent to the U.S. definition of CVAI, at 24 months of age versus five months of age; and change in the cranio proportional index (CPI), equivalent to the U.S. definition of cephalic ratio (CR) or cranial index (CI), at 24 months of age versus five months of age. The effectiveness of treatment was then presented as the difference of these two variables in the helmet treatment group versus the natural course group.

The study authors hypothesized that although helmet therapy is expected to give slightly better results in the short term, the natural course of skull deformation would catch up with the effects of helmet therapy over time and that no clinically meaningful differences would be present between the two groups at two years of age.

Commentary on the Study Analysis

Of the children who were assigned to the helmet treatment, it appears that there were inconsistencies with regard to the brand of helmet used, who delivered the treatment, and the facility at which the patients were treated. Because the study mentions two brands of custom-made helmets that use the “same mechanism to redirect skull growth” but does not list the brands or fabrication methods, it is not reproducible.

Cranial Remolding Helmets. UCSF Benioff Children’s Hospital

Further, some of the ill-fitting helmets depicted in a press release for the study represent a passive-design CRO, use chin straps, and appear to be excessively large. This type of design is not comparable to the active-design CROs that are typically used in the United States (represented in the photos that accompany this article), which have neither a chin strap nor move excessively.

The study does not contain details of the material used to fabricate the CROs; impression methods (cast, scan, or measurements); modification methods (reliefs and/or build-ups on the molds and modifications to the helmet throughout treatment); or how the modifications differed between patients with plagiocephaly and brachycephaly. Nor is there commentary about how many orthotists were involved, or their training or CRO experience. Additionally, the patients treated were seen in four different institutions.

The study notes that all patients assigned to helmet therapy had one or more side effects — an outcome contrary to existing literature, which puts the incidence of side effects closer to 25 percent.^[2] The study attributes this difference to the researchers’ broader definition of side effects, including sweating and unpleasant odor; however, only 23 percent were related to skin irritation while 73 percent of the side effects were associated with fit problems. One parent even reported that the helmet came off spontaneously. I have not heard of any experienced orthotists who encounter CRO fit issues in three out of every four of their CRO patients. This indicates poor-quality helmets rather than an increase in side effects, and further weakens the study’s conclusions. As a result of side effects reported in the study, ten parents discontinued helmeting before their infants reached 12 months of age. However, all infants assigned to the CRO group were included in the calculations.

The study’s reporting on the number of children that actually received treatment is unclear. It states that of the 42 children initially assigned to helmet therapy, six infants did not start treatment. (In three cases the parents preferred no treatment, and in three other cases the physician advised against helmet therapy). Further, there was an infant who was allocated to the natural course whose parents preferred helmet therapy. But again, it is unclear if this child’s results were included in the treated or untreated group. This means we would expect that either 36 or 37 infants were fitted with a CRO. What is further confusing is that the study states that only “ten of 30 infants received helmet therapy until 12 months of age.” So is the number of infants who actually underwent treatment only 30? And if so, what happened to those other six or seven children? Basically, the reported numbers for treated infants do not add up.

It is also difficult to prove or disprove the first part of the researchers’ hypothesis — that they expected to see slightly better results at the eight- and 12-month follow-ups compared to the 24-month follow-up since they failed to report their short-term follow-up results.

In addition, the helmet treatment was described as starting in infants no later than six-and-a-half months of age, and families were instructed to have their infant wear the helmet for 23 hours per day until 12 months of age. At six months of age, a moderate asymmetry does not usually indicate full-time treatment for six months. In my experience, at six months of age, treatment of a moderate asymmetry should take about three to four months with full-time wear of an active-design CRO, and compliance is critical to achieve these results. The study protocol included monitoring compliance using an electronic device built into the helmet; however, the study authors stated that “data from the measuring devices proved to be unreliable and we therefore omitted them from further analysis.”

Although the researchers relied on a parent questionnaire to measure compliance in absence of the measuring devices, a skilled orthotist can tell from hand measurements and the fit of the orthosis if the family has been compliant with full-time wear. However, the study fails to provide specific information regarding follow-up care with the orthotist. It does mention that the “helmet was modified or replaced to accommodate skull growth,” but no details are provided, nor does the study account for how many helmets were replaced.

Even if we were to ignore all of the obvious CRO fit and followup issues and assume a perfectly designed and fitted orthosis, any clinical relevance to the study is removed because the final statistical analysis involved averaging the results for the three different conditions (plagiocephaly, brachycephaly, and combination).

Table 2 demonstrates how averaging different head-shape measures dilutes the results and therefore the clinical relevance. To have a clinically relevant study, the researchers should have changed their inclusion criteria to only one head-shape type — or they should have had a larger patient population to show statistical significance with each of the three head-shape types.

Table 2

Moreover, the treated and non-treated groups were not comparable and were significantly different in the baseline assessments. The group treated with a CRO had higher brachycephaly scores and occipital lift, and the nontreated group had higher plagiocephaly and facial asymmetry scores. In other words, because the researchers randomized the group selection, they actually ended up with two dissimilar groups, which further dilutes clinical relevance between the treated and non-treated group.

Finally, the exclusion criteria omitted patients with torticollis. Because the incidence of those diagnosed with both plagiocephaly and torticollis has been documented to be as high as one in six,^[3] the diagnosis of torticollis is clinically relevant to the population of infants that were studied. Since the researchers excluded this diagnosis, they should have focused on children with brachycephaly only.

Ever seen a baby in a helmet? Here’s why. Healthline

Literature Analysis

Numerous other studies, including those by Katzel et al., which found that “helmet usage produces reproducible changes in head shape,” and Kluba et al.,^[6] which found that “helmet treatment significantly improves an initial malposition of the external ear in infants with positional plagiocephaly,” show the effectiveness of treatment with a CRO.^[4-8] There have also been studies that looked at the effectiveness of CRO treatment in different ages of infants. Graham et al.’s study found “orthotic therapy was more effective than repositioning, and early orthosis was significantly more effective than later orthosis.” ^[9] Another study, by Seruya et al., also indicates that early helmet intervention is most effective because the “correction rate of plagiocephaly with helmet therapy decreases with increasing infant age.” ^[10]

Further, there is a long list of studies that show that deformational asymmetries do not always correct on their own.^[11-15] Thus, van Wijk et al.’s conclusion that “skull deformation does not completely resolve in all cases by natural course,” and that 75 percent of infants continue to have some degree of skull deformation at two years of age is the only part of the study’s conclusion that is in agreement with other literature regarding treatment with a CRO.

Take-home Message

At some point, you may have parents ask you about this study and insurance providers may decide not to pay for treatment because of this single, not repeatable study. It is important that as a profession we come together to deliver a similar message regarding the weaknesses of this study that actually make it clinically inadequate at measuring the true effectiveness of CRO treatment.

Further, the take-home message from the study is not that CROs do not work, but that deformational asymmetries will not correct on their own, as seen countless times in previous studies. This study, with a low participation rate, incomplete reporting, dissimilar cohort groups, and questionable averaging of data, should not be used by parents, payers, or practitioners in CRO treatment decisions.

Ellie Boomer CPO LPO practices at Level 4 Orthotics Prosthetic & Cranial Remolding, San Antonio, Texas. She received her prosthetics certificate from California State University, Dominguez Hills, and her orthotics certificate from the Northwestern University Prosthetics-Orthotics Center, Chicago, Illinois. Boomer completed her O&P residency at Mary Free Bed Rehabilitation Hospital, Grand Rapids, Michigan, where she received specialized training in CROs, scoliosis, and pediatric prosthetics. She is also an active member of the American Academy of Orthotists and Prosthetists (the Academy) and the chair of its Craniofacial Society.

Source The O&P Edge

References