Does Wilderness Therapy work? As an industry professional, I need to understand more about “outcomes based research.” Like, what does “the research” even really mean? What is actually being studied? There is a lot of discussion about ‘outcomes’ so it’s important to understand what outcomes are being studied and why those are the ones chosen. I have come to recognize that I support something but don’t really understand what exactly I am supporting. I have been a cheerleader of research, and have witnessed its growth and development over the last several years, but now it is time for me to dig in and understand. An example: every time there is an article written about the industry, ‘the research’ is called into question for various validating reasons. This blog is a beginning conversation to clarify and provide context to the research about the Family Choice Behavioral Healthcare industry (FCBHI)* becoming evidence based treatment.
This long form article questioned the efficacy of wilderness therapy based on its research limitations. I had to sit down and ask many questions of Steve DeMille, PhD, LCMHC Executive Director of RedCliff Ascent (UT) and Neal Christensen, PhD, LP, Clinical Director of Elements Wilderness Program (UT) and the Research Committee Chair of the Outdoor Behavioral Healthcare Council (OBHC). They explained to me the problems and the solutions that this blog is based on. They enjoy research and since I never took a stats class or research methods class, they explained it more thoroughly.
The first take away I learned from them is that this body of research is still early in its development with relatively few, but quality research scientists investigating the field of Wilderness Therapy and the broader context of Outdoor Behavioral Healthcare. Demille said, “OBH has a growing and promising research foundation; however, we still have a long way to go.” The growing body of empirical research has focused largely on specific symptom-based and behavioral outcomes, as well as a few other areas of interest. There is so much more work to be done, as I have come to understand, to learn about its mechanisms of change, to use Neal’s jargon. Researchers and clinicians through OBH are partnering to research this intervention looking at what factors of wilderness therapy and other OBH programs can be shown to positively affect treatments for particular clients, and for which client populations it is contraindicated.
Anyone who wants to poke holes in the research does not have to work very hard and those who do the research are WELL AWARE of this fact. This does not change; that research has to start somewhere and it builds upon itself. The Outdoor Behavioral Health Council demands of its members to contribute data to investigating the quality and safety of its treatment practices. “The initial research from [Keith] Russell informed significant changes to wilderness therapy models like dynamic start and end dates and families needing more family (therapy) work while a student is enrolled,” said Will White, PhD, co-founder of Summit Achievement (Maine), who reminded me that programmatic changes occur, even when the research is created and executed by the industry.
PROBLEMS WITH THE RESEARCH
Control Groups: I learned from Demille, in order for quantitative research to be considered really strong and meaningful, or as scientists often say “valid and generalizable”, it needs to have a comparison group or a ‘control group.’ Unfortunately, there are not many of these types of studies to date in the OBH literature. There are inherent challenges in identifying such control groups, which might include a ‘wait-list’ group or ‘treatment as usual’ group. Control groups are important in research because they control for many of the limitations inherent in doing research. If a program does a pre-treatment and post-treatment study, how do they know that it was the PROGRAM that worked? The change could be from the client getting older and more mature, thus the client would have improved without the intervention. Including a control group allows researchers to minimize the many variables that may explain the change or difference measured. Without a control group as part of a study, researchers are very limited in the conclusions they can make.
One of the common studies referenced by OBH and in the media is a control group study conducted by the OBH Center and representatives at RedCliff Ascent. The study was conceptualized by Steven DeMille in 2012 and took about 5 years to organize and execute the study. Additionally, it took many contributors to conduct, fund, and publish the study. The study is a good step in producing control group research but the outcomes only represent one program so generalizing the findings is very limited.
Funding: Funding research continues to be a barrier for OBH due to the population it serves and ‘for-profit’ status of many of the OBH programs. Demille and Christenson shared with me that federal funding is nearly non-existent for social science research. To compensate for this, OBH programs have dedicated their own resources to make sure research was happening, and has purposely partnered with independent researchers and research institutions.
photo credit: RedCliff Ascent Currently, OBH has partnered with the University of New Hampshire and two full-time professors to direct OBH research practices. This professional distance offers the kind of objectivity needed to improve the quality of research, and validity of its findings. Unlike the tobacco companies funding and researching its products internally, OBH releases its data to independent researchers so that there is objectivity and transparency in the conclusions drawn.
When OBHC was founded in 1996, the five original programs determined that research was going to be one of the pillars of the OBHCs mission. Those 5 programs started supporting and engaging with researchers from various universities to study the effectiveness, safety, processes, experiences, and many other aspects of treatment in an OBH program. From 5 programs to now over 20 programs, OBHC programs continue to routinely collect data from all of its member programs and share this data for the use of independent researchers that follow rigorous research standards and practices of their respective universities. Each program dedicates both personnel and financial support to collect data at their respective program. As an organization, OBH dedicates funds to support research and its associated technologies largely because funding sources are few and far between.
As positive as that sounds, one of the critics that the Media and others will always be able to levy against the OBH research is the funding of the research. Because OBH programs have been willing to fund research when others have not been, this critique will exist. Just look at any critic of OBH treatment, they will include some version of this problem. The critics are not totally wrong as there are many examples in our society of companies/industries funding research to serve their personal benefit (again, think about the tobacco industry a few decades ago).
What I did not know before writing this blog is how research is done. As consumers, you can find research to support anything and everything. In the past few years, I have come to find out that I am healthier and happier and have a robust vocabulary because I like to use four-letter words, a lot. And like all results motivating critical review, my mother completely disagrees with all of the findings that I share with her upon their publication.
SOLUTIONS TO THE PROBLEMS IN THE RESEARCH
Replication: It is important that research studies don’t stand alone, instead stand together and this is why it is important to replicate them. The more times something is found by various sources, the more confident you can feel in the message. It’s like the game of ‘Life Line’, when an audience is polled. When you have a large majority that say the same thing, you can begin to feel more confident in the message that many people share. Additionally, this is the reason why OBH conducts a ‘multi-site’ study. There is now the ‘Golden Thread’ that was launched in 2019.
photo credit: RedCliff Ascent The Golden Thread is a piece of software that tracks participants in the study with a unique but anonymous identifier to see how the participant (teen or young adult) enrolls in treatment programs or does not enroll. The Golden Thread assists to generate control studies. Additionally, it will help researchers and practitioners understand long-term differences among the various client profiles FCBHI serves. Treatment programs participating in the research have the Golden Thread will hopefully contribute to the control data, contributing to the teens and young adults in treatment. This will make the research more powerful, credible and different questions can be approached with this new data. DeMille noted in my interview that many of the popular evidence-based treatments have dozens of control group studies supporting their effectiveness. Scientific inquiry (aka good research) has multiple researchers looking at the data from different perspectives and diagnoses like anxiety, depression, autism spectrum, levels of meds before and after interventions or treatment — there are so many places to go with the research. “As our body of research grows we will be able to make stronger claims about effectiveness. We will be able to advocate for insurance funding for our families. It is a critical part in our field’s ability to move forward,” said Christensen.
Doing Good Work and raising the ethics, accreditation standards and revealing to the next that as an industry, policing themselves. Recently, NATSAP announced that all of their members will need to be nationally accredited by 2023. This is not something that the members were in opposition to, instead of having CARF, Joint Commission, COA having a third party organization inspect is good for clients and the operation of a program. FCBHI is constantly evolving. Quality work and intentional change are values that wilderness therapy programs and all levels of residential care strive for; the motivation for credibility is why the Outdoor Behavioral Healthcare Council was created in 1996. It was created because there were unanswerable questions about quality of care, questions and outcome efficacy and anecdotes were insufficient. Research that challenges beliefs and reinforces valuable practices is important to the growth and development of treatment programs who are clinically driven.
Collaboration After the ‘Undark’ article in January 2020 was published, Steve DeMille, Ph.D. who some might say was criticized in the article, was excited and said, “Dr. Alan Kazdin wrote the book on adolescent psychotherapy and research methods. I have the book in my office and I am going to reach out to him to inquire more,” As a field, we are starting to get the attention of the foremost experts in the mental health field. We need to be ready for that attention. As it stands now it will be easy to critique the literature but as we take the steps above we will be able to make stronger claims and gain more prominence in the mental health fields.
The troubled teen industry has many labels and regrettably, not all treatment programs are ethical, transparent or equal. There is a lot of misinformation about them because it is outdated.
In order to be part of the Outdoor Behavioral Healthcare Council and be accredited by the Association of Experiential Education (AEE) all members have to be part of the research. NATSAP as an organization does not require members to be part of the research, however, many of their members are part of it.
* Boot Camps are not part of the Family Choice Behavioral Healthcare industry.
About the Author
Jenney Wilder, M.S.Ed launched All Kinds of Therapy in 2015, as the only independent online directory for the Family Choice Behavioral Healthcare Industry. With an impressive case of ADHD and her starter career in the 90’s in Silicon Valley, the dream for creating a website with features like side-by-side comparison and an integrated newsletter was born. Jenney stopped counting treatment centers and all types of schools that she has visited when she hit 500 many years ago. She was the sponsoring author of the only Economic Impact Study of the Family Choice Behavioral Healthcare Industry, which revealed the only true financial figures about this industry (in Utah). Jenney has a Masters in Special Education from Bank Street College (NY) and a Bachelors of Arts focused on History from Wheaton College (MA).