The Wrong Toolkit: Matching Tool to Task – Circle Nine of Healthcare Data Hell

A quick guide to matching measurement to purpose in primary care 

For my family practice colleagues, this might seem like a straightforward scenario. Imagine you notice that too many of your patients with a chronic condition are falling off treatment. You want to fix it. Do you design a randomized controlled trial? Commission a program evaluation? Or start testing a small change next week and track what happens? 

The answer depends on what you’re trying to do – and this is where we often trip up. In primary care, we tend to reach for the measurement tools we were trained in (usually research tools) even when the job calls for something different. It’s like bringing a microscope to a job that needs a mirror. 

Before you measure anything, ask three questions: 

• 1.  What’s the purpose?  (Learning? Discovery? Accountability?) 
• 2.  Who’s asking?  (Your team? A funder? A journal?) 
• 3.  What will they do with the answer? 

These questions point you toward one of three broad approaches. Each is valuable. None replaces the others. The trick is matching tool to task. 

	Quality / QI	Research	Evaluation
Purpose	Make it work better here and now	Discover what’s true and generalizable	Is this program working? Why or why not?
Typical tools	Run charts, PDSA cycles, driver diagrams	RCTs, regression, significance testing	Mixed methods, logic models, pre-post designs
Speed	Days to weeks	Months to years	Weeks to quarters
Who uses it	Frontline teams, clinic leadership	Academics, scientists	Funders, program leads, policymakers, project managers

Most improvement doesn’t require novel discovery. It requires applying what we already know – systematically, in our own context, with feedback loops that help us learn fast. That’s Quality Improvement (QI). If you’re trying to make your clinic work better for your patients right now, start there. 

Research helps when genuine uncertainty remains – when nobody knows the answer yet. Evaluation helps when funders or system leaders need to understand whether a program worked, for whom, and at what cost. But neither should be the default starting point for everyday improvement at the frontline. 

Tools like HDC Discover provide the clinical data infrastructure that supports all three approaches. The same data that powers a frontline run chart can also support a research study or program evaluation. The data is agnostic to purpose – it’s on us to choose the right lens. 

From HDC’s Clinical Data Stewardship Committee (CDSC) – Exaltations  

• Don’t conflate Quality, Research, and Evaluation. They serve different purposes and can complement each other.  
• Understand what Quality means in your local context, and use Quality Planning, QI, and Quality Control to move towards your goals and sustain outcomes – this is Whole System Quality 
• Use Research when gaps persist, but start with a Quality framework and improvement cycles 
• Most improvement in complex systems ought to come from applying known solutions systematically, not discovering novel approaches through research or relying on Predictive Analytics 
• Consider engaging with Evaluators for assessing larger initiatives, connecting outcomes to the broader system, and building capability for Whole System Quality within your organization 
• Ask: What’s the purpose? Who’s asking? What will they do with the result? 
• Empower teams to measure over time and not just compare snapshots 
• Align financial incentives with quality infrastructure – frontline quality is classically underfunded compared to research and evaluation. It needs to be built into the structures of our frontlines and supported financially, not bolted on as an afterthought 

Sometimes the wrench is exactly what you need. But not if you’re trying to cut down a tree.   

Want to go deeper? Circle 9: The Wrong Toolkit – Clinical Data Stewardship Committee (CDSC) version  explores these distinctions in detail, with clinical examples, a comparison of theory-for-change approaches, and the case for funding quality infrastructure in primary care. 

Acknowledgements 

Thanks to the many people who provided feedback (see a full list in the CDSC version of this article).