What We Don't Know About Our Outcomes
Ask any physician about the outcomes of their procedures. Whatever the answer, ask them how they know. You may be surprised.
Stone Research team reviewing patient-reported data from its AI-driven outcome tracking software.
Doctors, especially surgeons, are proud of their results. Ninety percent of all doctors are sure they are in the top 10% of their field. Your doctor may in fact be one of the very best. But how do they actually know?
Doctors, in almost every field, are not incented to find out. Although they want to know that their interventions actually work, most do not follow up to find out. Surgeons do not see their patients more than a few times after surgery. They do not send follow-up questionnaires or hire outcomes evaluation specialists. They don’t work with software that follows patients automatically. They are not paid to see a patient after the initial postoperative period and certainly not at one, two, or five years after surgery. In fact, most employers and insurance companies discourage non-required follow-up visits as they use resources and take up new patient slots. Doctors say “Well, my patients come back if they have a problem.” Possibly; but they may also go to other doctors instead.
Surgeons are not trained in rehabilitation techniques nor in how to work with physical therapists. A few have a relationship with a rehab team in their office or nearby, but most just send a prescription for generic rehabilitation — if they send one at all.
All of this means that the surgeon is separated from the actual performance of their surgical technique or implants. They don’t know how soon patients can run, jump, or ski. And they don’t know how to help them do that earlier in the healing process. Without the direct connection to the rehabilitation team and the patient, the surgeon only knows what he or she may be told or wants to hear.
This is not unique to surgeons. Emergency room doctors have similar problems. They often see patients in the middle of the night, treat them as best they can and then either refer them to their primary care doctor or admit them to the hospital for care. There is little or no feedback. Did the procedure performed during the busy ER shift lead to the best outcome? Were the infections they saw viral or bacterial, requiring time or antibiotics? Did the medications they started the patient on resolve the disease? Did the patient even pick up their prescriptions? Did the intervention prolong the patient’s life or just get them out of the emergency room? Even studies evaluating emergency interventions have the limitations of shortsightedness. While epinephrine during cardiac arrest can help restore a heartbeat, does it restore the patient’s post-ER life with a mind and body intact enough to enjoy it?
Doctors often do not have the bandwidth, incentive, training or resources to follow up on the outcomes of all their interventions. And the systems are not in place for them to do so automatically. Nor are they trained to extract data from electronic medical records to answer questions about their own effectiveness. Doctors are not trained in how to do outcome studies or how to collaborate with research teams. Therefore, they simply do not know how effective they are as physicians. If they rely primarily on the data from published studies, they are basing the effectiveness of their own interventions on poorly generalizable data: techniques and results that are often out of date, performed years ago in major universities rather than private practices, and typically based on only the most favorable outcomes.
All of this is to say, when you ask your doctor how their patients do, trust their good intentions but realize that they don’t always know.
This can, and will, change. Nearly all patients now have devices that follow them throughout their entire lives (smart phones). It is possible to design apps to track outcomes and send back data to their doctors (or implant manufacturers, the FDA, or research teams). In this way, every procedure and implant could have a true track record. All data could be published anonymously and crowd-sourced outcome evaluations performed.
This feedback loop could be enormously helpful to surgeons curious about how their implants are performing around the world and to emergency physicians wondering if their patients get better after visiting the ER. Real-time reporting of successes and failures will improve techniques, correct mistaken assumptions about performance and recovery, and make every surgery and medical intervention better. Why we rely on limited data to assess the most important things we have done to ourselves is a baffling question in our increasingly transparent universe. It’s time to find out.
Learn how the Stone Research Foundation tracks every major procedure performed at The Stone Clinic—gathering and sharing insights to drive progress in orthopaedic care. Find out how in SRF’s most recent Annual Report.
