Pine River Institute is a youth treatment agency in Canada. They are challenging the landscape of accountability in residential treatment by investigating their social return on investment. The picture is pretty good too.
While at a conference in FL in February, I attended a session on “Social Return on Investment for Residential Treatment.” What I heard and saw was jaw dropping. I followed up with Laura Mills (Ph.D. in Quantitative Methodology, (Psychology) and Pine River’s Director of Research & Evaluation) and Vaughan Dowie (CEO) to talk and discuss their study and Pine River’s other outcome evaluations. I needed to learn more about how the study was done and results that might resonate with parents and families.
A couple of details that you need to know first: Pine River Institute is a non profit, co-educational 36 bed treatment program with offices located in Toronto, Canada and is accredited by the Council on Accreditation (CoA). Pine River’s mission, vision and values are lived each day and what jumped out at me is their dedication to embed outcome based evaluation (in-house and third party) into their programming, since the Institute’s inception in 2006.
AKoT: First, thank you both for taking time out of your schedules to talk about social return on investment. Let me start with Vaughan Dowie, CEO, before we jump into jaw dropping stats that you both shared. I understand that Pine River outcomes have already promoted an evolution in public policy and this evidence is becoming a central part of the conversation in Canadian mental healthcare. Could you give us a quick (if that is possible) overview of when that happened and how?
V.D.: I don’t think we have made public policy changes in Canada but we are part of the debates. For example, we have been quite involved in the discussions around the upcoming legalization of cannabis in Canada, advocating for an increase in both youth-centered public education regarding cannabis and the adolescent brain, plus calling for an increase in youth treatment linked to the cannabis revenue. In October, we held a by-invitation-only symposium on youth and cannabis legalization that was co-sponsored by the Governments of Canada and Ontario as well as Ryerson University, St. Michael’s Hospital and the Canadian Council on Substance Use and Addiction.
AKoT: Ok, so that is impressive that a small 36 bed program is causing profound national conversations. Clearly you are doing interesting and solid work with the focus on your outcomes. Would you share how long this “Social Return on Investment for Residential Treatment” research took?
L.M.: Developing and nurturing relationships is key to our programming, and that extends to our evaluation & research as well. We didn’t want to just hire an agency to run some data. We wanted to ensure deep understanding with our programming, and our data. McMaster University DeGroote School of Business was a strategic choice for us
V.D.: We wanted an objective 3rd party to undertake the study so as to lend it both independence and credibility. McMaster’s business school because they housed their Health Policy stream in the Business School and therefore understood both the policy issues and the business metrics.
Although the data collected were done so as part of our ongoing research and evaluation efforts, the planning, data orientation and relationship development took over a year. Pine River funded the McMaster portion of the resources and our own human resources in-kind.
AKoT: To all the readers, these numbers are only about clients who completed treatment at Pine River, this does not include the students who never enrolled or left Pine River during the treatment process (“against medical advice”).
Let’s jump into some of the numbers that excited me — the first one was the student’s ROI over the course of their working life. These returns are based on saved health care and justice costs, and income contribution from youths over their working life and their parents (who missed much less work) in relevant years after the program.
L.M.: Yes. Although the numbers vary by year of youth admission and were calculated with very conservative estimates of outcomes, the average ROI is about 9:1. For every dollar invested, expect 9 dollars in return just on the available outcome measures.
Worth noting is the non-financially-quantifiable returns – those arising from better family relationships, stronger identities, and healthier approaches to decisions. These are profoundly impactful yet don’t have an easily attached dollar figure.
AKoT: Solid points about the non-financial measures. Another measureable number that jumped out at me is time expense from parents/family missing work while their child was struggling. You were able to quantify that cost. Would you share that finding?
L.M.: The measurable missed work over the three months before applying for treatment is an average of 9 days for moms and 5 for dads. This is reduced to less than one day for either parent after the program. This is a very real social savings, but also opens the conversation to nuances.
Some parents have quit their jobs in order to emotionally (and logistically) support their families, so they wouldn’t have reported any missed days. As such, our averages are underestimated before treatment. Others report that even though they only missed so many days, their time at work had significantly diminished effectiveness due to their focus on family needs, so again this is not a measurable financial outcome but has significant impact in the everyday of families.
AKoT: What did you find out about education/academics (attendance) through the study? This identifier is regularly a critical part of families’ struggles, leading up to an intervening treatment.
L.M.: One thing I know is that parents are intensely interested in their child completing school as a foundation for a life of working. Many of our youths have unsuccessful, stalled or abandoned academic careers before Pine River. This is particularly distressing to parents, many of whom recognize the personal and far reaching value of engagement with the working world.
Most of our youths finish secondary school during or after the program, and subsequently engage with post-secondary or work-related studies. Others engage with the workforce. As much as we are so proud of our high post-secondary academic engagement among our youths, we also want to recognize the success of workforce engagement. That said, I don’t like to attach a ‘success’ benchmark when it comes to academic progression, but I can say that our youths are typically meaningfully occupied after they complete the program.
AKoT: Final Question…. What outcome-based research is Pine River doing currently?
L.M.: Can I LOL on a blog? I have about a dozen projects in the hopper. The most exciting thing we’re doing now is feedback-informed treatment. So, youths engage with their health and behaviour scores and it’s a tool for clinical dialogue. But what is really exciting is that we are going to start this for parents too – we measure parent health and parenting skills and now we’ll be able to offer them the same opportunity to dialogue with their therapist at Pine River. Here are the titles of a few of our ongoing projects:
- What Happens When Families Wait for Treatment
- Is Residential Youth Treatment Associated With Different Health Trajectories Than an Untreated Counterpart?
- For Whom Are We Best Suited? A Look at Predictors of Attrition from Intensive Residential Youth Treatment
- Emotional Intelligence and Maturity Among Residential Youth Clients: Conceptualizations and Changes Over Time
AKoT: What would you like to see in terms of research at Pine River?
V.D.: There are two broad areas that are our focus. The first is to continue to work both here in Canada and with our colleagues in the U.S. to increase capacity regarding outcome evaluation and research. It is very important to provide credible information to stakeholders such as families, referring professionals and funders regarding the impact of our work.
The second area of focus for us relates to the mechanisms of change in youth and families, investigating which approaches create positive change and what works (or doesn’t) for whom so that we are better able to tailor our approach to various client groups overtime.
AKoT: While looking over the Pine River website I found your Equity & Inclusion statement. Many of the teen and young adult treatment programs in the United States are just starting to include these formally. I am mentioning this because again, our neighbor to the north has moved forward and is waiting for us.
THANK YOU BOTH!!!
Dr. Laura Mills, Director of Research & Evaluation
Vaughn Dowie, CEO