A Data Science Approach to Evaluating Innovative Healthcare Models

Ontario’s new Family Health Team and Health Links models aim to transform frontline healthcare delivery with the goal of improving patient care and reducing costs. These programs will likely be evaluated based on analyses of the billing data that the government receives from healthcare providers, but there are significant gaps in the knowledge this information can offer. Data science approaches to program evaluation seek to analyze data from a variety of sources to gain a more comprehensive understanding of program outcomes. We discuss how this could work in a healthcare context.

As the most expensive area in provincial budgets, with costs continually rising, healthcare is constantly targeted by governments for reform. In Ontario, recent years have seen new healthcare delivery models introduced with the aim of improving access to care and reducing costs. The Family Health Team and Health Links models fundamentally reorganize frontline health services with the promise of better coordination between different health professionals seeing the same patients.

The Family Health Team (FHT) model focuses on primary healthcare in community settings, connecting patients to a team of primary healthcare providers including family doctors, nurses, social workers, and dietitians. It is hoped that FHTs will provide patients with rapid access to the health professional best able to meet their needs, while also helping to expand preventative health programs, improve the management of chronic health conditions, and increase access to care outside of regular office hours.

The Health Links model, on the other hand, is targeted to the five percent of patients with the most complex health needs, who account for two thirds of healthcare spending. These patients are typically seniors, people with multiple chronic health conditions, and people with mental illness and addictions. Health Links seeks to increase cooperation between the multiple health professionals treating these high-needs patients by establishing circles of care, which may include family doctors, specialists, hospitals, home care coordinators, long-term care facilities, and community support agencies. It is hoped that Health Links will reduce specialist wait times, emergency room visits and unnecessary hospital admissions.

Challenges of evaluating healthcare delivery

These new healthcare delivery models, like those implemented by governments worldwide, aim to improve healthcare quality and efficiency. To confirm whether these objectives are actually met, health system planning has to include evaluation of the processes and results of introducing new models. In healthcare, measuring success is a complex question. Different health system stakeholders tend to focus on different indicators: for example, clinics may want to lower administrative costs, but may be less concerned about overprescribing medications, while hospitals, which are usually funded on the basis of the numbers of procedures they perform and patients they admit, may find it difficult to measure the benefits of prevention. Also, evaluating new models often takes years. Setting up new health teams takes months to years, and a variety of problems usually surface at each stage of the process. Once these initial hurdles are overcome, health ministries usually examine billing data from healthcare providers to see where costs have increased and decreased. Of course, collecting enough billing data to draw reliable conclusions also takes months to years. This makes it very difficult to take an evidence-based approach to healthcare reform.

Healthcare billing information, which is the usual source of data for evaluating healthcare models, shows areas where costs have increased or decreased, but it many other ways it is not an ideal metric. For one thing, decreased costs are not a realistic goal in all areas of healthcare; an aging population means that certain types of care will be in more demand. Billing data provides little information about health outcomes or about the connections between different areas, which can result in short-sighted planning: for example, releasing patients from hospital sooner after surgeries may successfully reduce hospital costs, but can contribute to poorer recoveries and increase the burden on family physicians and community care resources.

A data science-informed approach

How, then, can healthcare models be evaluated in greater depth? Data science promises that a broad base of detailed data on program outcomes, together with effective data analytics, makes it possible to understand how programs succeed or fail, and to judge which programs will be most appropriate for specific contexts. A better understanding of the relationships between different patient populations, healthcare delivery models, health outcomes, and costs could have a huge impact on healthcare planning in Canada, where healthcare accounts for over 40% of provincial budgets.

One of the key features of the FHT and Health Links models is that they promise to set up electronic health records which will be shared among teams of health professionals who treat the same patients. These will provide a broader, more detailed, and more frequently updated source of data than billing data. In short, electronic health records are ideal for data analytics. Using thoroughly de-identified (anonymized) data from health records to evaluate healthcare delivery models could make it possible to go beyond simply tracking costs, to understanding how a new model affects patients’ trajectories of care. Only this type of clinical data can directly trace the types of complex problems targeted by FHT and Health Links: emergency room visits by patients unable to access primary care, nursing home admissions due to a lack of coordinated care in the community, and negative interactions between multiple prescriptions from different doctors.

To summarize, current means of evaluating new healthcare delivery models tend to be too slow to effectively guide decision-making, and often do not show how well different groups of patients are served by different models of care. Studying patient trajectories through the healthcare system, using de-identified information from electronic health records, can confirm whether new models of care are actually improving patient health, rather than simply reducing costs. Understanding the impact of healthcare delivery models on patient outcomes as well as costs can contribute to better health system planning by showing which models help patients to stay healthier and more independent, reducing the need for emergency and institutional care.


Ontario Ministry of Health and Long-term Care. Community Health Links.

Ontario Ministry of Health and Long-term Care. Family Health Teams.

Ontario Ministry of Health and Long-term Care. Improving Care for High-needs Patients.