G.C.M. Kersten, G.M. Schippers, T.G. Broekman, J.M. van Rijswijck & J.P.A. Joosten
University of Nijmegen Research Group on Addictive Behaviors
Bureau Bêta Nijmegen
SUMMARY
The complete report of the project is
available in Dutch
The accurate matching of addiction treatment and care possibilities to a particular individual recently has become more important. This is the result of the increasing differentiation of the treatment facilities within a single organization, the increasing complexity of the problems of the clients with addictive behaviors, and the increasing knowledge of the effectiveness of different treatment modalities. This situation calls for a thorough evaluation of the matching and referral practices in the Dutch treatment of addiction. Evaluation can only be undertaken when these practices are explicit, however, and this is currently not the case: The matching and referral rules for the treatment of addiction have simply not been well described. The rules are often invisible, impossible to evaluate, difficult to systematically improve, and largely nontransferable. The goals of the present research were therefore as follows.
The research questions were as follows.
The client-treatment matching is conceived in
the present research as a decision leading to the
appropriate form of treatment. A matching decision consists of
two steps. (1) The collection of client information, and (2) the
application of particular rules to this information. We then
introduce a distinction between two types of client
characteristics that can play a role in client-treatment
matching: namely, treatment-dependent and treatment-independent
client characteristics. Treatment-dependent characteristics
concern the perspective of the client and are the product of
existing treatment possibilities and/or the interaction between
the client and the therapist (for example, the particular motives
and treatment desires of the client and also previous treatment
experience). Treatment-independent characteristics do not concern
either the provision of treatment or the therapist.
Treatment-independent characteristics are related to the nature
of the problem, the characteristics of the individual client, and
the social circumstances of the client (for example, addiction
severity, psychiatric diagnosis, and living situation).
The conclusion of the literature review
undertaken with regard to the results of prediction and matching
studies and existing matching and referral models is that
relatively few 'hard' empirical facts are available. For a number
of client characteristics (and rules), it is clear that they
should play a role in client-treatment matching. These include
the severity and nature of the psychiatric problem along with the
social stability of the client (treatment-independent variables).
They also include the particular motivation and treatment desires
of the client (treatment-independent variables). The client
characteristics and concomitant rules encountered in the
literature are divided into six major dimensions: (I) client
perspective, (II) previous treatment experience, (III)
demographic characteristics, (IV) substance abuse-related
characteristics, (V) intra-personal characteristics, and (VI)
inter-personal characteristics.
Three addiction-treatment centers - the Bouman
House (Boumanhuis) in Rotterdam, the Gelders Center for
Addiction Treatment (Gelders Centrum voor Verslavingszorg)
and the clinical facility affiliated with the Gelders Center, the
Oolgaardt House (Oolgaardthuis) in Arnhem - participated
in our study. The Boumanhuis is a big urban treatment center and
it consists of two divisions, one alcohol-treatment and one
drug-tre atment and every division has several outpatient and
inpatient units (nineteen in total). The Gelders Centrum voor
Verslavingszorg is a rural outpatient addiction center with
locations in five cities with several alcohol- and drug-treatment
modalities (such as beha vior thera py, self control training,
family therapy, methadone maintenance). The locations Arnhem and
Nijmegen participated in the research. The Oolgaard thuis is a
clinic with a detoxification and observation unit, and a hier
archical therapeu tic community. In these treatment centers in
total circa 7000 clients are treated per year.
The matching and referral rules of the three
centers were observed, made explicit, and described. In these
descriptions, a flow chart has been used to represent each
referral point (each of the points where matching decisions are
made) and the rules for flowing from one unit to the next. The
description and specification occurred in close collaboration
with the therapists of the treatment centers. The process was a
long, intense, and difficult process of observation,
interviewing, description, feedback, and continual adjustment,
which took more than two-and-a-half years. The relevant teams
were stimulated to improve their matching and referral practices
by making them explicit. The result is a series of extensive
descriptions of nonprioritized flow rules, that were accepted by
the teams as representing acual matching practice. The
descriptions of flow rules have been presented in a series of
bundles for the different locations. In a total of more than 200
pages, the rules for 44 referral points and more than 100
different referral possibilities (varying from general hospital,
several psychiatric inpatient and outpatient modalities to
methadone maintenance, group therapy, family therapy,
counselling, rehabilitation programs etc.) are described. In the
description of the matching and referral rules, a number of
concepts (for example, addiction severity, psychiatric problems)
appeared to be insufficiently operationalized. This is in part
because objective diagnostic instrumentation was either lacking
or not in use. The descriptions of the matching and referral
practices were also so bulky, poorly organized, and redundant
that they simply could not be evaluated.
The model chosen to facilitate the efficient
representation and evaluation of the matching and referral
practices in addiction treatment was the decision tree.
The descriptions of flow rules were transformed into decision
trees for the eight largest and most representative referral
points in the three addiction-treatment centers: the central
screening point for drug addiction, the outpatient observation
program for drug addiction, and the inpatient observation program
for drug addiction from the Bouman House drug division; the
Consultation bureau for Alcohol addiction (CAD) Rotterdam and two
alcohol clinics (House ter Schie en Boerhaave Clinic) from the
Bouman House alcohol division; the Consultation Bureau for
Alcohol and Drugs (CAD) Nijmegen; and the Oolgaardt House. The
decision trees were constructed in such a manner that with a
minimum of client characteristics and rules an unambiguous
indicated treatment alternative could be established. This
implied a considerable reduction of the number of necessary
rules. To determine the validity of the decision trees, the
correspondence between the matching decisions delivered by the
decision trees and the matching decisions made in actual practice
had to be evaluated. Such validation is above all needed to
verify that our decision trees actually fit matching and referral
practice or, put differently, that actual practice can be
represented in such a manner.
For validation, those client characteristics that occur in the decision trees must first be reliably measured. For this purpose the Assessment Instrument Matching variables (AIM) (Beoordelings-Instrument Cliëntkenmerken, BIC) is developed. The AIM consists of 40 items with regard to client characteristics from the decision trees in the following areas: substance abuse, previous treatment experience, social circumstances of the client, psychiatric and psycho-social problems, and treatment desires of the client.
The reliability and utility of the Asessment
Instrument were then studied. Eighteen therapists from nine
different Dutch addiction-treatment centers each evaluated 20 (of
a total of 60) clients with the aid of the AIM. The clients were
recruited from the three centers participating in this research
and were interviewed by the researchers. The interviews were
recorded on video, and every client was then evaluated by six
therapists. In order to determine the inter-rater reliability,
kappas were calculated. The inter-rater reliability of 12 items
was substantial (.60 < k <= .80), of 11 items moderate (.40
< k <= .60), of 9 items fair (.20 < k <= .40), and 9
items where judged to be insufficent reliable. The most important
explanation for the insufficient or fair degree of reliability
was commonly the involvement of difficult to operationalize
concepts like crisis, psychiatric problems, personality
disorders, intro spec tive capacity and fami ly problems .
The utility of the AIM was studied by asking
the raters whether they could make a matching decision on the
basis of the information from the instrument and the interview or
not. In more than 80% of the cases, this question was answered
positively. Those instances where the therapists reported not
being able to make a matching decision on the basis of the
collected information appeared to be largely the result of the
occurrence of seven particularly 'difficult' cases in the sample.
The results of these analyses suggest that the AIM provides
enough information for making a matching decision. It is
concluded that the AIM is an economic and pragmatic instrument
for client-treatment matching on condition that unreliable items
are better operationalized.
Keeping in mind those client characteristics
that were difficult to establish and thus less reliable (the
crisis items were particularly problematic because they occur
high at the 'top' of the decision trees), the validity of the
decision trees was next determined. Validation involves a
comparison of the matching decisions produced in actual practice
with those produced according to the models. The validation
research was set up as follows. During a period of one-half to
three-quarters of a year at eight key referral points in the
three aforementioned addiction-treatment centers, the client
characteristics for a particular client were assessed with the
aid of the AIM whenever a matching decision was made. The
therapists involved in this assessment also recorded their own
matching decision and actual referral behavior for the relevant
client. With the aid of a computer algorithm specially developed
for this purpose, a matching decision on the basis of the client
characteristics for each observed client was made in keeping with
the relevant decision-tree model. A
total of circa 750 client decisions was observed.
The deviations of the matching decisions
according to the model from those according to actual practice
were analyzed further. There appeared to be two types of causes
for deviations: (1) Reliability-related causes; these are
deviations between model and practice that are primary caused by
ill defined client variables or unclear treatment options of the
decision tree models. These observations are left out of
consideration in further (validity) analysis. (2)
Validity-related causes; these are deviations between model and
practice that refer to the validity of the decision trees. There
are two kinds of validity-related causes, namely (a) construction
inaccuracies in the decision trees (for instance are there recent
changes in matching practice that are not yet represented by the
decision tree), (b) unclear or inconsistent matching rules. The
first kind of deviations could be 'solved' by making the
appropriate adjustments in the decision trees. The second kind of
deviations were taken as invalidations. That is, the possibility
of representing the matching decision within the relevant model
is open to question. The percentage of invalidating deviations
varied per decision tree and ranged between 3% and 35%: The
central screening point for drug addiction 7%, the outpatient
observation program for drug addiction 21%, the inpatient
observation program for drug addiction 33%; the CAD for alcohol
addiction Rotterdam 18%, clinic House ter Schie 19%, and the
Boerhaave Clinic 3%; the CAD Nijmegen 35%, and the Oolgaardt
House 20%. The percentage of invalidating deviations was in part
a product of the chance of error, which varied per decision tree.
To the degree that the decision tree contained more decision
points and possibilities for referral, the chances of error
increased. Similarly, the involvement of a greater number of
therapists in the matching meant a greater chance of error. Such
an increased chance of error partially explained the relatively
high percentage of deviations for the CAD in Nijmegen. The chance
of error was also greater when the therapists responsable for the
assessments and matching decisions were 'new' and had not, thus,
been involved in the development of the descriptions of flow
rules. This situation appears to have played a role in the
outpatient observation program and the inpatient observation
program for drug addiction from the Bouman House. The most
important (and intrinsic) threat to the validity of the decision
trees consists of ongoing discussions in the treatment teams on a
number of the fundamental problems associated with
client-treatment matching. These discussions concerned the
following points: (1) criteria for addiction severity, (2)
matching rules for family therapy, (3) the role of previous
treatment-experience in choosing treatment setting (outpa tient
versus inpatient), and (4) the role of client's treatment
desires.
As in the reliability research, the validation
research shows a number of the client characteristics to be
insufficiently operationalized. The variables crisis, psychiatric
problems, personality disorders characterized by a lack of
structure, and introspective capacity appear to be particularly
in need of attention. Moreover, a number of the therapeutic
alternatives are insufficiently described and distinguished from
each other. These include, for example, guidance versus
treatment, follow-up care, and outpatient versus ambulatory care.
In other words, the better operationalization of both the client
characteristics and the treatment alternatives appears to be a
prerequisite for formal and verifiable client-treatment matching.
Our conclusion is that the decision trees are
reasonably valid. This conclusion does not apply to the decision
trees of the CAD Nijmegen and the clinical observation program
for drug addiction from the Bouman House. These decision trees
accurately represented only about two-thirds of the matching
decisions.
In order to more generally evaluate the
matching and referral practices, the relative importance of the
different client characteristics and thereby the role of the
treatment-dependent and treatment-independent client
characteristics in matching were determined on the basis of the
data from the validation research. The frequency with which the
characteristics were found to play a role in the matching
decisions and the distinct importance of a particular client
characteristic were determined for this purpose. The
characteristic of psychiatric problems occurred in all eight of
the decision trees, and at least 96% of all the clients were
sorted in the decision trees according to this characteristic;
this characteristic was found to have influence on the matching
decision for less than 6% of the clients, however. Items
pertaining to a possible crisis occurred in six of the
decision trees and were assessed for 84% of the clients; one can
speak of only a limited number of actual cases of crisis, however
(somatic crisis 11%, psychiatric crisis 4%, and psycho-social
crisis 26%). The characteristic of previous experience with
outpatient treatment occurred in four of the decision trees, and
66% of the clients were sorted according to this characteristic;
61% of the clients also had previous outpatient treatment, which
suggests that this is a particularly important characteristic for
sorting. Less than one-half of the clients were found to be
sorted according to any one of the remaining client
characteristics.
When we combine the number of clients with a
particular characteristic with the number of clients actually
sorted according to that characteristic, the order of importance
appears to be: use of opiates at this moment, addiction severity,
and client desire for methadone (for drug addicted); addiction
severity, family problems, and introspective capacity (for
alcohol addicted).
The different client characteristics (or items
from the AIM) were grouped according to the six major dimensions
found in the literature. The frequency with which the
characteristics were found to play a role in the matching
decisions was determined for the major dimensions. The
intra-personal characteristics were found to play the
(quantitatively) most important role followed by the
inter-personal and substance abuse-related characteristics (with
almost equivalent roles), client perspective, previous treatment
experience, and demographic characteristics (which were found to
hardly play role). The percentages of treatment-dependent and
treatment-independent characteristics used in the decision trees
to sort clients were 22% (dependent) and 88% (independent).
On the basis of the rank ordering of the client
characteristics and our knowledge of the decision trees, a
decision hierarchy was constructed to represent the most
important client characteristics and treatment dimensions. This
hierarchy makes decisions at three different levels: (1)
addiction treatment versus psychological/health care, (2)
outpatient versus part-time versus inpatient treatment, and (3)
the choice of treatment modality. The second level appears to be
the most important with regard to the number of clients sorted
and capacity to distinguish. A number of decisions at the third
level follows - in particular: non-directive versus directive
treatment and family versus individual treatment. From this
decision hierarchy, the relative contributions of
treatment-dependent and treatment-independent client
characteristics also become clear in terms of the degree to which
these characteristics differentiate between treatment options.
The role of the treatment desires of the client, for example, was
found to be particularly important at the second and third levels
where a large investment by the client (for inpatient treatment)
may be expected or a specific treatment motivation (for group
treatment, family treatment or treatment in a Hierarchical
Therapeutic Community).
The matching and referral practices isolated in
this research were next evaluated in light of the existing
literature. The question was whether or not the client
characteristics and matching rules that have stood out as
important in previous empirical research could be detected in the
decision trees. The conclusion of this evaluation is that
client-treatment matching in the three treatment centers is at
places not specific or refined enough. This situation appears to
be caused by less than explicit assessment of a number of client
characteristics and in particular the psychiatric problems,
addiction model entertained by the client, and stage of change
that the client is in. The therapeutic alternatives are also not
sufficiently differentiated with respect to the aforementioned
client characteristics. Those treatment modalities directed at
specific psychiatric (DSM-IV Axis I) problems (for example,
anxiety disorders, mood disorders) were simply not encountered in
any of the decision trees. Those treatment modalities directed at
the addiction model of the client and the client's stage of
change, such as minimal interventions, self-control procedures,
self-help books, and motivation enhancement techniques appear to
be lacking in most decision trees.
In general, the matching and referral practices in addiction treatment and care in the Netherlands can, with some effort, be made explicit using relatively simple decision- tree models. The nature of the decision trees corresponds reasonably well to empirical knowledge although further differentiation on the basis of this information may be desired at particular points.