Research ArticleABSTRACT
Objective: To assess the feasibility of providing interventions to adolescent inpatients that promote community mental health engagement following discharge.
Method: This randomized parallel two-arm pilot trial was conducted in an acute inpatient mental health ward at a tertiary pediatric hospital over 16 months. Adolescents aged 13-18 years presenting with suicidal behaviors and/or deliberate self-harm and their caregiver were recruited during admission and randomized to receive a structured Therapeutic Assessment and Attachment-Based Family Therapy intervention (TA-ABFT; n=15) or treatment as usual (n=15).
Results: TA-ABFT demonstrated good patient, caregiver and clinician acceptability and adherence, with adolescents and their caregivers describing the experience as valuable and supportive. Readmissions to the emergency department and acute mental health ward were substantially reduced post-intervention compared with controls. Clinician-rated engagement at 1- and 3-months post-discharge was similar between groups. Recruitment and retention rates indicated that study modifications are required prior to proceeding with the full trial.
Conclusion: TA-ABFT is a promising inpatient intervention for increasing readiness for change and reducing subsequent inpatient readmissions. Results highlighted several important lessons pertinent to clinicians and researchers working with adolescents admitted to inpatient mental health settings following suicidality and deliberate self-harm.
Keywords: Deliberate Self-Harm; Suicidal Behaviors; Pilot; Feasibility; Therapeutic Assessment; Adolescent
Abbreviations: TA: Therapeutic Assessment; ABFT: Attachment-Based Family Therapy; DSH: Deliberate Self-Harm
Introduction
Levels of suicide and deliberate self-harm (DSH) are of growing concern within the adolescent population [1], with suicide representing the second leading cause of death among young people aged 16 to 29 globally [2]. Risk for deliberate selfharm (DSH) [3] and completed suicide increases dramatically during adolescence [4], highlighting this as a critical opportunity for intervention. Despite promising developments in treatments to reduce DSH and suicidal behavior in young people [5-11], poor adherence to follow-up treatment continues to prevent the provision of appropriate support for these young people. Just 30% of adolescents who attempt suicide report complying with referral recommendations [12,13], with 16% to 60% not engaging in any further support [14,15]. It is critical to maintain focus on supporting adolescents and their families to develop the skills, understanding and motivation necessary to engage in community treatment following inpatient admission. This is supported by meta-analyses [19-21] evaluating treatment for adolescents who engage in DSH and/or attempt suicide which highlight the importance of both individually focused support and family participation. Therapeutic Assessment (TA) is a brief, structured safety planning intervention designed to help patients learn skills to manage crisis situations [22]. Recent studies have shown that TA is feasible and effective for use with adolescents presenting with DSH [23] and is associated with increased engagement with mental health services [24,25]. Complementing TA, Task One of Attachment-Based Family Therapy (ABFT) Relational Reframe seeks to improve the relationship between the young person and their caregiver by enhancing the help seeking behaviors in the young person and the containing role of the parent [26].
This has been identified as a key step in motivating engagement in further intervention to treat the underlying psychopathology [27]. Although TA and ABFT have proven effective treatments for adolescent depression and suicidality [28,29], few studies have examined how these treatments may impact outpatient engagement when initiated during inpatient admission [30]. A recent meta-analysis identified one randomized study that assessed adolescent engagement after TA and another that assessed engagement after ABFT [30]. To our best knowledge, no studies have yet assessed the impact of targeting engagement at both the caregiver and adolescent levels using these interventions during inpatient admission. Given that family involvement has been associated with increased engagement, targeting both levels of the family system may serve to maximize treatment engagement. Research within inpatient settings consistently reports on the challenges in evaluating the effectiveness of treatment options for adolescents [31]. These methodological challenges include difficulty engaging this population in research, high attrition rates, frequent use of retrospective data, a lack of comparison groups and the use of idiosyncratic measures. To overcome these difficulties and inform the development of rigorous clinical trials, studies evaluating the challenges and barriers to conducting research within this adolescent inpatient population are needed [32-34]. This pilot randomized control study therefore aimed to assess the feasibility of providing a structured intervention to promote engagement in community mental health services with adolescent inpatients admitted following suicidal behaviors and/or DSH. The intervention comprised two parts: Structured TA with the adolescent and ABFT with their caregiver (henceforth referred to as TA-ABFT). Four areas of feasibility were evaluated: Acceptability, recruitment and retention, preliminary outcomes and barriers and challenges.
Methods
This trial is registered with the Australian New Zealand Clinical Trials Registry (#ACTRN12618000085279p). Ethics approval was obtained by the Hospital Human Research Ethics Committee (Project #37243).
Design
Single site (tertiary pediatric hospital) pilot randomized control trial, with four time points: baseline (T1), discharge (T2) and 1 (T3) and 3 (T4) months post-discharge.
Participants
Adolescents aged 13 to 18 years who were admitted to the inpatient mental health ward following a suicide attempt and/or DSH within 30 days prior to the admission were recruited. To be eligible, adolescents needed to have a stable living situation, no immediate plans for out-of-home placement and a caregiver who was available and willing to participate in the study. Exclusion criteria were substance dependence, learning disability, autism spectrum disorder, acute psychotic episode, direct transfer to a medical unit, not being referred to an outpatient mental health service or insufficient understanding of English to consent or complete study procedures. Patients who identified as Aboriginal or Torres Strait Islander were excluded as present interventions have not yet been validated as culturally appropriate within these groups.
Primary Outcomes
Feasibility was assessed across all four time points, by measuring acceptability, recruitment and retention, preliminary outcomes and barriers and challenges encountered. Criteria for success were defined using a traffic light system approach [35] with a. ≥ 30.00% recruitment rate,
b. ≥ 80.00% retention of study participants at 3 months (final data collection time point) and
c. ≥ 90.00% patient adherence to allocated interventions indicating that the trial is feasible. Recruitment rate <15.00%, retention below 50.00% and <60.00% patient adherence to allocated interventions were set as the threshold indicating that the trial is not feasible. Values between the above-specified ranges were set to indicate that the trial may be feasible if appropriate changes are made to the protocol.
Secondary Outcomes
Readiness for Change: The University Rhodes Island Change Assessment Scale (URICA) [36,37] is a measure of readiness for change across four stages: Pre-contemplation, Contemplation, Action and Maintenance, and was completed at baseline (T1) and discharge (T2). As well as producing an overall Readiness to Change score, higher scores on each subscale indicate greater readiness at each stage of change. The URICA has exhibited good internal consistency, reliability and validity in clinical adolescent psychiatric populations [38].
Experience of Service Questionnaire: An adaption of the Experience of Service Questionnaire (ESQ) [39] was used to measure service satisfaction and acceptability from both adolescent and caregiver perspectives and were completed at Baseline and Discharge. The adapted questionnaire consisted of 10 items rated on a 4-point Likert scale (1=Agree Strongly to 4=Disagree Strongly) and free text sections exploring what the respondent liked about the service, what they felt needed improving and inviting further comments about the service. Lower scores on the ESQ represent greater satisfaction, where a score of 10 represents maximum satisfaction and a score of 40 indicates minimal satisfaction with the service.
Service Engagement: The Service Engagement Questionnaire (SEQ) [40] was completed 1 (T3) and 3months (T4) postdischarge. It is a clinician-rated measure of the adolescent’s level of engagement with outpatient mental health services. This 11-item questionnaire assesses six dimensions of engagement:
1) Appointment keeping,
2) Client-therapist interaction,
3) Communication/openness,
4) Client’s perceived usefulness of treatment,
5) Collaboration with treatment, and
6) Compliance with medication. Responses are combined to form a single measure of engagement, with higher scores reflecting higher levels of engagement. A recent systematic review [41] identified this measure as assessing the three key areas of treatment engagement and having acceptable internal consistency, test re-test reliability and being associated with other indices of treatment engagement.
Readmission Rates: The number of readmissions to the hospital emergency department and inpatient mental health ward were calculated using patient medical records and were assessed at 1 (T3) and 3-months post-discharge (T4). Reasons for readmission were recorded and coded as either DSH/suicidal behavior or ‘other’.
Procedure: Young people admitted to the hospital inpatient mental health ward following a suicide attempt or DSH and their caregiver were screened for eligibility and invited to participate in the trial. Consenting adolescent-caregiver dyads were randomized to intervention or control groups using a computer-generated block randomization sequence. Blinding was not possible due to the naturalistic setting of this study and randomization was not stratified by any factors due to the pilot nature of the study. Participants and their caregivers completed measures during their admission prior to intervention (T1) and post intervention (T2). Adolescents’ community case managers completed the SEQ 1-month post-discharge (T3) and three-month post-discharge (T4) via an online survey form. Study data, including randomization procedures, were collected and managed using REDCap (Research Electronic Data Capture) secure web-based software [42,43].
The Intervention: TA is a structured safety-planning approach that draws upon the principles of cognitive behavior therapy [23] and cognitive analytic therapy [24] to support patients to recognize the need for change and implement required adjustments in their lives. Further information on the specifics of this method and the implementation in a clinical setting is available in Appendix A [44- 47].
Results
Figure 1 outlines the participant flowchart for this randomized feasibility study, consistent with CONSORT guidelines.
Table 1: Participant characteristics by group.
Feasibility
Recruitment & Retention: Recruitment and retention data are presented in Figure 1. Of note, five participants in the intervention group and two participants in the control group dropped out after randomization, prior to completing baseline measures (see Figure 1 for reasons provided). To account for this attrition and allow thorough piloting of intervention procedures, two participants were recruited and non-randomly assigned to the intervention group. In total, participant retention rate at T2 was 59.37%. At discharge, Table 1 outlines baseline demographic characteristics for each group.
Acceptability: Data from the ESQ indicated that the intervention was viewed positively by both caregivers and adolescents and provided the family the support they needed to care for the adolescents’ needs. Participants and caregivers who received the intervention felt that they were treated with dignity (adolescents: 88.89%, caregivers: 100.00%) and that staff were friendly and approachable (adolescents: 100.00%, caregivers: 100.00%). The majority of adolescents (66.67%) and caregivers (88.89%) reported that the therapies and support offered helped manage the adolescent’s condition. Handwritten comments on the ESQ form demonstrated that adolescents who received the intervention felt that staff were aware of and understanding of the adolescents individual needs (“The staff are aware of my needs, which makes it easier for me to manage”) and safety (“I know that I’m safe and that I can get better the more I keep up my attitude”). Handwritten comments by caregivers indicated a desire for structured interventions during the adolescent’s inpatient admission to be included as standard practice (“These kids need treatment whilst on the ward. Therapy sessions [...] need to be standard practice so that there is continuity of care. If [the hospital] can’t treat these kids who are critical, then who can?!”). The process of completing the intervention was also perceived positively by the clinicians completing the intervention with adolescents and their caregiver. Clinician feedback described satisfaction with the ease of delivery, time required and relevance to the challenges facing caregivers and adolescent inpatients presenting with DSH or suicidal behaviors. Rapport was identified as a key factor influencing engagement in the intervention itself (“The better the rapport, the better the engagement [in the intervention] and therefore the more the young person seemed to get out of the whole experience.”). Clinicians also described the intervention techniques as effective drivers of insession reappraisal of their emotional and behavioral functioning (“Often, the young person found the diagram enlightening, they often said ‘I never thought of it like that’, and mentioned they were surprised when they saw their maladaptive cycles on paper”). Clinicians completing the intervention met once weekly for supervision, including one supervision session each fortnight with a clinician involved in the design and implementation of ABFT. No changes were made to the treatment protocol over the course of the study.
Figure 1: Outlines the participant flowchart for this randomized feasibility study, consistent with CONSORT guidelines.
Barriers & Challenges: There were no adverse events related to study participation. Challenges facing study personnel included difficulty contacting the treating clinician and having adequate time with patients and caregivers to complete the intervention prior to discharge. Median duration of admission was 6 days (M = 6.52, SD = 3.04) with participants completing baseline procedures a median 2 days after admission (M = 6.53, SD = 1.35). Due to scheduled activities for patients within working hours there was a relatively short period of time to arrange and implement separate child and caregiver intervention sessions. Despite an average survey completion time of just 4.88 minutes (SD = 2.75), only 47.37% of community clinicians responded to the invitation to complete the adolescent engagement survey 1 and 3-months post-discharge. Possible reasons for this low response rate include adolescents failing to attend the first scheduled appointment following referral and therefore not being admitted into the service, high volume workload in these outpatient settings and transfer to new casemanager.
Preliminary Outcomes
The small sample of this pilot randomized trial precluded the use of inferential statistics and therefore data reported on preliminary outcomes should be interpreted conservatively. Mean scores and standard deviations are presented as measures of central tendency for continuous variables, while frequencies and percentages are presented for categorical variables (Table 2).
Table 2: Preliminary outcome data.
Note: University of Rhodes Island Change Assessment Scale (URICA); Experience of Service Questionnaire (ESQ), Service Engagement Questionnaire (SEQ); mean (M); standard deviation (SD).
Readiness for Change: At baseline, adolescents in both groups were pre-contemplative or contemplative about change. Whilst URICA scores showed an increase in readiness for change in both groups, only adolescents who completed the intervention showed an elevation to the action stage by discharge.
Service Engagement: Clinician ratings of engagement tended to decrease over time for both groups, with the intervention group showing similar levels of engagement at both time points compared with controls. Again, this difference was small and fell within 1 standard deviation. The variability within the intervention group was 1 to 2 times larger than the control group, suggesting that individual responses to the intervention may be more varied than responses to treatment as usual. All participants in both groups attended their first appointment with their outpatient mental health clinician following their inpatient admission.
Readmission Rates
Compared with 60.00% of controls, just 11.11% of adolescents in the intervention group were admitted to the hospital emergency department for DSH or suicidal behaviors within 1 month of the intervention. This was maintained up to 3 months postintervention (11.11% of intervention group vs 70.00% of controls). No participants in the intervention group were readmitted to the inpatient mental health ward for DSH or suicidality within 1- or 3-months post-intervention, compared with 50.00% and 70.00% of controls respectively. Similarly, the mean duration of admission was 1.6 days shorter in the intervention group compared with controls.
Discussion
This study sought to examine the feasibility of undertaking a randomized pilot trial of a structured intervention with adolescent inpatients following suicidal behaviors and/or DSH and their caregiver. TA-ABFT demonstrated good patient, caregiver and clinician acceptability and adherence, with adolescents and their caregivers describing the intervention as valuable and supportive. This was also reflected in the substantial reduction in readmissions to the emergency department and inpatient mental health ward in the intervention group compared with controls. Given that more adolescents in the intervention group also reported readiness to actively implement change in their lives, this promising finding suggests that brief inpatient TA-ABFT may have a positive impact on help seeking behavior, including potentially reducing the burden associated with repeated inpatient mental health admissions on both the family and the healthcare system. For instance, observed reductions in admission duration may partly reflect the effects of the intervention in supporting inpatient recovery. While a fullscale trial is required to explore these trends further, present findings represent a critical first step towards increasing treatment engagement which is essential for achieving the dose-response relationship required for effective intervention.
Recruitment and retention rates fell below predetermined success criteria defined by the research team, suggesting that modifications to the research strategy are needed prior to going ahead with the full-scale trial. As expected, researching a population who show low levels of outpatient engagement, recruiting and retaining participants was a key challenge of this pilot study. Interestingly, the proportion of young people who declined to participate (47.72%) and/or dropped out of the study (40.63%) is consistent with the proportion of young people who did not access recommended care in previous research (35-59%). At follow-up clinicians reported several key challenges engaging most young people in treatment, with many clinicians rating their clients as rarely or sometimes perceiving therapy as useful. This reinforces the importance of motivation and readiness for change as key targets for intervention and further research. Although most adolescents in this study had been engaged with community mental health services at some point, 78.95% were not engaged in outpatient treatment at the time of their inpatient admission which further highlights the importance of interventions to support young people to actively engage with mental health services following inpatient admission.
Present data on recruitment suggest that future studies recruiting inpatient mental health patients should account for moderate to high levels of attrition in sample size calculations across adolescent, caregiver and clinician response rates. These results also emphasize the importance of flexibility in scheduling, particularly in the context of arranging separate caregiver and child intervention sessions during admission that includes school, group therapy sessions and family meetings as standard care. Despite these recruitment challenges, participants and their caregivers represented a range of ages, sex, gender identity and family educational attainment, and included both single and twocaregiver families. To overcome the methodological challenges faced in this study, future research should continue to minimize the burden on clinical staff and involve all key stakeholders in the development of research procedures including screening, recruitment, data collection and follow-up processes. Given the difficulties experienced obtaining clinician feedback in the present study, future studies may also benefit from including adolescent and caregiver ratings of engagement in outpatient mental health services. Adolescent and caregiver perceptions of engagement may also serve to shed light on the barriers underlying disengagement from these services, representing critical perspectives to weigh in on possible solutions to address this very issue. In light of qualitative acceptability data, measures of inpatient intervention engagement and rapport should also be included in future research as potential modifiable factors with flow-on effects to outpatient engagement.
Conclusion
In summary, the challenges and barriers addressed in this pilot randomized trial illuminate critical processes in the development and delivery of TA-ABFT to adolescent inpatients following DSH and/or suicidality and their caregiver. Preliminary results suggest that a brief introduction to TA-ABFT during inpatient admission may be a promising intervention to increase readiness for change and reduce subsequent readmissions to acute psychiatric inpatient services. Evaluating the feasibility and challenges associated with adolescent inpatient mental health research provides important information that is valuable in the design and implementation of larger research studies aiming to improve adolescent engagement in mental health services following DSH or suicidal acts.
Declaration of Interest
All authors declare that they have no conflicts of interest.
Acknowledgement
The authors would like to thank the participating families and Banksia staff at The Royal Children’s Hospital in Melbourne, Australia for their support of this study. The authors also extend their gratitude to the dedicated volunteers and research assistants without whom this study would not have been possible.
Author Disclosure Statement
All authors declare that they have no financial relationships relevant to this article to disclose.
Funding Source
This study was funded by The Royal Children’s Hospital Foundation, the Gannon Family, and the Victorian Government’s Operational Infrastructure Support Program.
The funding organizations did not have a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.
Ethical Approval
This study has been approved by the Human Research Ethics Committee of the Royal Children’s Hospital Melbourne (study number #37243).
References
- Hawton K, Saunders KE, O'Connor RC (2012) Self-harm and suicide in adolescents. The Lancet 379(9834): 2373-2382.
- (2014) WHO. Preventing suicide: A global imperative.
- Moran P, Coffey C, Romaniuk H, Olsson C, Borschmann R, et al. (2012) The natural history of self-harm from adolescence to young adulthood: a population-based cohort study. The Lancet 379(9812): 236-243.
- Gould MS, Fisher P, Parides M, Flory M, Shaffer D (1996) Psychosocial risk factors of child and adolescent completed suicide. Archives of general psychiatry 53(12): 1155-1162.
- Asarnow JR, Baraff LJ, Berk M, Grob CS, Devich Navarro M, et al. (2011) An emergency department intervention for linking pediatric suicidal patients to follow-up mental health treatment. Psychiatric services 62(11): 1303-1309.
- Ougrin D, Zundel T, Ng A, Banarsee R, Bottle A, et al. (2010) Trial of Therapeutic Assessment in London: randomised controlled trial of Therapeutic Assessment versus standard psychosocial assessment in adolescents presenting with self-harm. Archives of disease in childhood 96(2): 148-153.
- Donaldson D, Spirito A, Esposito Smythers C (2005) Treatment for adolescents following a suicide attempt: results of a pilot trial. Journal of the American Academy of Child & Adolescent Psychiatry 44(2): 113-120.
- Spirito A, Boergers J, Donaldson D, Bishop D, Lewander W (2002) An intervention trial to improve adherence to community treatment by adolescents after a suicide attempt. Journal of the American Academy of Child & Adolescent Psychiatry 41(4): 435-442.
- Henggeler SW, Rowland MD, Randall J, Ward DM, Pickrel SG, et al. (1999) Home-based multisystemic therapy as an alternative to the hospitalization of youths in psychiatric crisis: Clinical outcomes. Journal of the American Academy of Child & Adolescent Psychiatry 38(11): 1331-1339.
- Katz LY, Cox BJ, Gunasekara S, Miller AL (2004) Feasibility of dialectical behavior therapy for suicidal adolescent inpatients. J Am Acad Child Adolesc Psychiatry 43(3): 276-282.
- Asarnow JR, Berk M, Hughes JL, Anderson NL (2015) The SAFETY Program: a treatment development trial of a cognitive-behavioral family treatment for adolescent suicide attempters. J Clin Child Adolesc Psychol 44(1): 194-203.
- Litt IF, Cuskey WR, Rudd S (1983) Emergency room evaluation of the adolescent who attempts suicide: compliance with follow-up. Journal of Adolescent Health Care 4(2): 106108.
- Swedo SE (1989) Post discharge therapy of hospitalized adolescent suicide attempters. Journal of Adolescent Health Care 10(6): 541-544.
- Cummins R, Allwood C (1984) Suicide attempts or threats by children and adolescents in Johannesburg. S Afr Med J 66(19): 726-729.
- Spirito A, Lewander WJ, Fritz G, Levy S, Kurkjian J (1994) Emergency department assessment of adolescent suicide attempters: factors related to short-term follow-up outcome. Pediatric emergency care 10(1): 6-12.
- Borschmann R, Stark P, Prakash C, Sawyer SM (2018) Risk profile of young people admitted to hospital for suicidal behaviour in Melbourne, Australia. J Paediatr Child Health 54(11): 1213-1220.
- De Kloet L, Starling J, Hainsworth C, Berntsen E, Chapman L, et al. (2011) Risk factors for self-harm in children and adolescents admitted to a mental health inpatient unit. Aust N Z J Psychiatry 45(9): 749-755.
- Katz LY, Cox BJ, Gunasekara S, Miller AL (2004) Feasibility of dialectical behavior therapy for suicidal adolescent inpatients. Journal of the American Academy of Child & Adolescent Psychiatry 43(3): 276-282.
- Daniel SS, Goldston DB (2009) Interventions for suicidal youth: a review of the literature and developmental considerations. Suicide and life-threatening behavior 39(3): 252-268.
- Robinson J, Hetrick SE, Martin C (2011) Preventing suicide in young people: systematic review. Australian and New Zealand journal of psychiatry 45(1): 3-26.
- Ougrin D, Tranah T, Stahl D, Moran P, Asarnow JR (2015) Therapeutic interventions for suicide attempts and self-harm in adolescents: systematic review and meta-analysis. Journal of the American Academy of Child & Adolescent Psychiatry 54(2): 97-107.
- Tharinger DJ, Finn SE, Wilkinson AD, Schaber PM (2007) Therapeutic assessment with a child as a family intervention: A clinical and research case study. Psychology in the Schools 44(3): 293-309.
- Beck JS (2011) Cognitive behavior therapy: Basics and beyond. In: Beck JS (Edt.)., Guilford Press, USA.
- Ryle A, Kerr IB (2003) Introducing cognitive analytic therapy: Principles and practice. In: Ryle A, Kerr IB (Eds.)., John Wiley & Sons, USA.
- Ougrin D, Zundel T, Ng AV, Habel B, Latif S (2013) Teaching therapeutic assessment for self-harm in adolescents: training outcomes. Psychology and psychotherapy 86(1): 70-85.
- Diamond G, Siqueland L, Diamond GM (2003) Attachment-Based Family Therapy for Depressed Adolescents: Programmatic Treatment Development. Clinical Child and Family Psychology Review 6(2): 107-127.
- Diamond G, Russon J, Levy S (2016) Attachment‐based family therapy: A review of the empirical support. Family Process 55(3): 595-610.
- Diamond GS, Reis BF, Diamond GM, Siqueland L, Isaacs L (2002) Attachment-based family therapy for depressed adolescents: A treatment development study. Journal of the American Academy of Child & Adolescent Psychiatry 41(10): 1190-1196.
- Ougrin D, Boege I, Stahl D, Banarsee R, Taylor E (2013) Randomised controlled trial of therapeutic assessment versus usual assessment in adolescents with self-harm: 2-year follow-up. Archives of disease in childhood 98(10): 772-776.
- Yuan SNV, Kwok KHR, Ougrin D (2019) Treatment Engagement in Specific Psychological Treatment vs. Treatment as Usual for Adolescents With Self-Harm: Systematic Review and Meta-Analysis. Front Psychol 10: 104-104.
- Blanz B, Schmidt MH (2000) Preconditions and outcome of inpatient treatment in child and adolescent psychiatry. J Child Psychol Psychiatry 41(6): 703-712.
- Leon AC, Davis LL, Kraemer HC (2011) The role and interpretation of pilot studies in clinical research. Journal of Psychiatric Research 45(5): 626-629.
- Lancaster GA, Dodd S, Williamson PR (2004) Design and analysis of pilot studies: recommendations for good practice. Journal of evaluation in clinical practice 10(2): 307-312.
- Craig P, Martin A, Browne S, Simpson SA, Wight D, et al. (2018) Development of guidance for feasibility studies to decide whether and how to proceed to full-scale evaluation of complex public health interventions: a systematic review. The Lancet 392: S7.
- Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, et al. (2016) CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Pilot and feasibility studies 2: 64.
- Di Clemente CC, Hughes SO (1990) Stages of change profiles in outpatient alcoholism treatment. Journal of substance abuse 2(2): 217-235.
- McConnaughy EA, DiClemente CC, Prochaska JO, Velicer WF (1989) Stages of change in psychotherapy: A follow-up report. Psychotherapy: Theory, Research, Practice, Training 26: 494.
- Greenstein DK, Franklin ME, McGuffin P (1999) Measuring motivation to change: An examination of the University of Rhode Island Change Assessment Questionnaire (URICA) in an adolescent sample. Psychotherapy: Theory, Research, Practice, Training 36(1): 47-55.
- Brown A, Ford T, Deighton J, Wolpert M (2014) Satisfaction in child and adolescent mental health services: Translating users’ feedback into measurement. Administration and Policy in Mental Health and Mental Health Services Research 41(4): 434-446.
- Hall AMJSCJM (2001) Brief report: The development and psychometric properties of an observer-rated measure of engagement with mental health services. Journal of Mental Health 10(4): 457-465.
- Tetley A, Jinks M, Huband N, Howells K (2011) A systematic review of measures of therapeutic engagement in psychosocial and psychological treatment. Journal of Clinical Psychology 67(9): 927-941.
- Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, et al. (2019) The REDCap consortium: Building an international community of software platform partners. J Biomed Inform 95: 103208.
- Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, et al. (2009) Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of biomedical informatics 42(2): 377-381.
- Ougrin D, Zundel T, Ng A, Banarsee R, Bottle A, et al. (2011) Trial of Therapeutic Assessment in London: randomised controlled trial of Therapeutic Assessment versus standard psychosocial assessment in adolescents presenting with self-harm. Archives of disease in childhood 96(2): 148-153.
- Ougrin D, Tranah T, Stahl D, Moran P, Asarnow JR (2015) Therapeutic interventions for suicide attempts and self-harm in adolescents: systematic review and meta-analysis. J Am Acad Child Adolesc Psychiatry 54(2): 97-107.
- Diamond GS, Diamond GM, Levy SA (2014) Attachment-based family therapy for depressed adolescents. American Psychological Association 17(2): 1-21.
- Diamond GS, Stern RS (2003) Attachment-Based Family Therapy for Depressed Adolescents: Repairing Attachment Failures. Attachment processes in couple and family therapy 6(2): 107-127.
