Editor’s Note: To read another perspective on metrics and neurosurgery, read “Counterpoint: Randomized Controlled Trials in Neurosurgery — the Advantages.”

Data collection and analysis drives change in practice, whether used locally or published for a broad readership. In the 1990s, databases and spreadsheets provided increasingly efficient means to collate and store data. More recently, electronic medical records facilitate access to medical information but typically do not contain information of specific value for research and analysis.

Neurosurgery includes a spectrum of disorders that affect the nervous system. Stereotactic radiosurgery is applicable to many brain disorders that include all benign and malignant brain tumors, blood vessel malformations, and selected disorders that affect brain function. Each disorder shares some common elements and others that vary markedly, including disease specific scoring systems, patient scales and parameters of unique interest. Our own efforts over 25 years have allowed the publication of numerous reports. Most have been retrospective reviews. Some have been randomized trials, matched cohort or case control studies. Randomized controlled trials can provide strong medical evidence. However, they can be difficult to conduct. Thus, the technique is not utilized as often as we might like.

We have experienced the challenges of both retrospective and prospective data collection and think that continued prospective data collection is preferred. Large, non-randomized datasets can provide pragmatic data from real patients over time that reflect actual practice. Radiosurgery is an interesting challenge for a broad registry because it includes many different disease processes; is both text- and imaging-based for diagnosis, planning and outcome; and requires collection of both short- and long-term data. A current radiosurgery registry is supported by Elekta (Stockholm). Another registry sponsored by the American Association for Neurological Surgeons (AANS) and American Society for Radiation Oncology (ASTRO), together with BrainLab, is in development.

Ensuring Data Quality
Physicians desire tools that allow them to collect useful medical information efficiently, and to use that information for a diverse set of tasks. These could include practice assessments, meeting reporting requirements, data benchmarking and research. The perceived or measured quality of our practice will be based on the data that we document. Thus, it is increasingly important that practices maintain their own databases that can be coupled with useful analytical tools. Although this may sound obvious, data collection often is relegated to an afterthought in clinical practice because it can be time-consuming and not effectively used. Registries that collected large amounts of data from multiple institutions have been effective at changing practice. As one example, despite the important and diverse set of observations and conclusions that were generated from the 5,209 patients in the Framingham study of the 1960s, the tool of the large-scale clinical registry remains underutilized.

The maintenance and collection of good-quality data is challenging. In our own radiosurgery registry, we created data fields that described the patients and their diseases, followed the logical flow of a procedure and defined clinical and imaging outcomes, together with any related management. We used accepted definitions of disease parameters and validated rating scales. Some free text entry was necessary. The challenge was to collect comprehensive data that would eliminate the need to use a separate clinical chart, and yet collect so many fields as to make the data entry process overwhelming and laborious. We aimed for data entry at point of care to improve recording accuracy.

Quality control is crucial. During the test phase of our first 300 patients, clinical data was entered by only one person to test consistency and system reliability. Demographic data was entered by an administrative assistant and then checked by the surgeon. Nevertheless, errors occurred. Errors at data entry (via entering an incorrect decimal point, choosing the wrong selection on a scroll-down menu) were corrected by a separate review, prior to saving the record. For both local or shared use, auditing of data will be required. For multicenter studies, proof of data fidelity by evaluating a subset of records against another archive (ie. hospital electronic medical record) should be part of study design. Security features must be incorporated into the system to allow a specific set of privileges for any user. All centers and individual users must meet the regulatory requirements of institutional and national agencies.

Who will commit to data entry, and how can this be made more efficient? Text from the electronic medical record could be used to populate some fields, but this will likely require change in the way text is entered primarily so that it could be understood and sorted. Second, real images should be acquired for archival and analytic purposes. We need tools that fuse images from planning and follow-up, measure target response in an automated or assisted fashion, and then record it. Third, we need sophisticated analytical tools to probe registry data in different ways, using numerical or text filters. Natural language processors could also assist to suggest clinical follow-up or create predictive analytics on an individual case, based on interrogation of the entire dataset.

Maintaining Momentum
Any registry, at its core, is a set of rules or directive. Once ready for productive use, the maintenance of such a registry can be challenging. Periodic reviews by data stewards is key to ensure the data, at its core, is pristine. Asher et al. have described the regulatory issues important in the creation and use of large-scale registries for local or national use. Their report in Neurosurgical Focus is detailed and serves as an excellent resource on this topic. For any registry, there must be a clearly defined description of the planned utility, which can include quality assurance, safety and /or research. The ability to obtain a waiver of informed consent is key to maximizing data collection, and indeed in reducing bias since those who are willing to provide consent often comprise a unique subset of the subject population.

The value of registries, while still underutilized and in development, remains largely unproven. Although we are committed to the value of data collection for our own analyses and research, there are concerns that outside entities (ie. payers) may use registries for payment approval or denial. Who will approve and oversee a big data registry? Will this be a national organization, a government office or some other group? Will a new registry “industry” be created, similar to that associated with the electronic medical record? Clearly, the design and implantation of any large registry requires great planning. However, it is likely that registry-based data and analytics will grow to become a key part of our learning foundation.

Douglas Kondziolka, MD, MSc, FAANS, , is professor and vice-chair of Neurosurgery at NYU Langone Medical Center. He serves as associate editor for the Journal of Neurosurgery, is a director of the American Board of Neurological Surgery, and is a past-president of the Congress of Neurological Surgeons. The author reported no conflicts for disclosure.