Medical Director Cole Stanley continues his journey though the Circles of Healthcare Data Hell with Circle Seven. You can find the other circles here

Down to the next circle, where we see a negative consequence of abundant data available at low or no cost. This makes it too easy to jump from a partially formed thought to a data request, while remaining fuzzy in our purpose for looking at this particular data and being unclear about what we would or could do with the results. In the extreme, picture a well-intentioned manager known for their verbosity. They are having a sort of stream-of-consciousness discussion with the team about possible measures. “I wonder what this measure would show.” “It might be interesting to look at this measure.” Since it seems data is easy to get and abundant, the assumption becomes that the data team can go away and gather these measures for reporting back, and thus they are kept busy working away on things that may have little meaningful use.  I’ve been on both sides of this, both guilty of shooting from the hip with my data requests without thinking it through, and frustrated or worried about how to deal with fuzzy data requests I have received.

Data, data, everywhere. Nor any time to think.

Our partially formed questions or unclear purposes create other problems. As reviewed above, our human brains have a knack for telling stories about data, filling in the blanks, and discounting information that doesn’t fit. It’s just too easy for us to look at data and say “oh yes this makes sense because x, y, z.” We are great at post-hoc rationalization for what we see or experience. We have righteous minds [1]. meaning that our innate emotional responses are often in the driver seat, and our rational brain acts as our internal lawyer, providing the rationale and logic post-hoc. More often than not, this means that the following sequence takes place:

  1. Data request for unclear purpose, no clear questions, no predictions made
  2. Data results released
  3. Innate emotional response to the data
  4. Post-hoc rationalization for what we are seeing and why we need to act on it or not

We are at especially high risk of this when we don’t have a clear purpose, questions, predictions, and ideas about how we might act based on the result. Here our QI methodology is again to the rescue, as we fit the data request into our approach that includes being clear on purpose. The CDSC recently discussed this issue in relation to custom aggregate data requests, guiding our operations team on what we should ask before these requests can be actioned.

We can even structure a data request as a PDSA cycle, wherein we make a plan to request the data. A great PDSA Plan includes the overall purpose for requests, specific questions we are wanting to answer by enacting the plan, and predictions about what the results of the test (answering the question) will be. Let’s take an example:

  Unappreciative Inquiry A better way to request
Initial reason for data request I feel like I had a lot of No Shows today and am curious about what the No Show data shows. It seems to be increasing.
Refined reason for data request (better explaining the Why) No time, just GIVE ME THE DATA! I suspect that we have significant waste of resources at our clinic due to a high No Show Rate for physician appointments, and I suspect there are steps our team can make to improve this over time.
Questions to answer Can you share the data with me on No Show rates? What proportion of booked physician appointments were No Shows over the last six months?

Were there differences based on the day of the week?

Were there differences based on morning or afternoon?

Were there differences based on provider?

Predictions made none About 25% of physician appointments are No Shows, and this is increasing over time.

There are more No Shows on Fridays, in the mornings, and for Dr. X who has a very complex panel (about 50% more).

Data team response Unclear on purpose of looking at this, they struggle to pull something together: Overall No Show rate for the clinic on a line graph over time since clinic opening (including allied health staff appointments, it’s gone from 10% to 25%) Clear on what data to pull and rationale, they quickly pull the data: The No Show rate is only about 10% now, down from 20% six months ago. Fridays indeed have a higher rate, at about 25%. Dr. X’s panel doesn’t though, and mornings actually have a lower No Show rate.
Interpretation of results It looks like our overall No Show rate is increasing, just as suspected. We need to do something about this! My prediction wasn’t quite right, and our No Shows for physicians are not as bad. I guess I was working on Friday when I initially thought of this problem.
Actions taken based on result Strike a physician working group to deal with this problem (spend time and money) No need for focused improvement efforts at this time, continue to monitor

 

Consider some modifications to our day schedule to move booked appointment slots to times with lower No Show rates (mornings or Monday to Thursday)

 

Prompts idea to have a discussion with allied health team members to see how their No Show rates might compare

As our former HDC Board Chair Dr. Anthon Meyer would often remind us: Clarity is kindness. The example in the table above demonstrates this. By being more intentional and detailed in our data requests, we can save time and money, learn more about our complex system, and ensure that the actions we take are data-informed (as opposed to data-appeased?).

CDSC exaltations

  • Our team at HDC and folks trained in QI can help you transform your rough ideas into exemplary data requests
  • Be clear on the following before submitting a data request
    • Purpose: why is looking at this important to you and/or your patients?
    • Questions: what do you want to learn from this data?
    • Predictions: what do you predict the answers to your questions will be? Be specific.
  • Think about what possible actions you might take based on the results of your data request
  • Consider the relative value of the data you are requesting, and the resources potentially required to collect and report this data

Return to Healthcare Data Hell Table of Contents 

[1] I highly recommend this thought provoking book.