TOMS

Behavioral Health Outcomes Tracking

The TeleSage Outcomes Measurement System, TOMS, is a state-of-the art system for collecting and reporting client self-report outcomes and obtaining regular client progress reports.
- Evaluate Client Progress
- Evaluate Treatment Effectiveness
- Identify Best Practices

Outcomes are assessed using TeleSage self-report items from Item Response Theory (IRT)-validated item banks. All items are reliable for use with inpatient, partial hospital, and outpatient (public and private) mental health and substance use clients. Self-report item banks are available for adult (18 years of age and older) and youth (12-17 years of age) clients, as well as parents/guardians of youth 5-12 years of age. All items were designed and validated to be maximally sensitive to clinically significant change over time and exhibit minimal to no differential functioning across specific populations (e.g., gender, race, ethnicity, public vs. private clinics).

Evaluate Outcomes
Over Time

Track, assess and report self-reported outcomes over time. Evaluate client progress, evaluate treatment effectiveness, and identify best practices.

Real-time Reports Available

Assessments may be taken via web, phone, or paper and pencil. Reports and data associated with these assessments are available in real time via our Web portal or an Electronic Health Record (EHR) system.

Reliable Self-Reported Assessment

Outcomes are assessed using self-report items from Item Response Theory (IRT)-validated item banks. All items are reliable for use with inpatient, partial hospital, and outpatient (public and private) mental health and substance use clients.

HISTORY

1999
Under a state mandate for adult and youth outcomes tracking, 18 counties in the state of Ohio, including Cincinnati, hired TeleSage to host their outcomes tracking system. From 1999-2003, TeleSage created and hosted a large-scale project titled KidsPeace Outcomes Tracking for Youth in the state of Pennsylvania.

2001
The state of Washington hired TeleSage (via an open RFP process) to implement an outcomes measurement system for the entire state. More than 300,000 IVR, RDE, and fax-based surveys were administered in Washington.

Also in 2001, the Idaho Department of Health and Welfare hired TeleSage to track the outcomes of all public sector substance abuse clients.

2006
TeleSage began a successful working relationship with the state of Tennessee (via an open RFP process) to offer statewide outcomes tracking utilizing IVR and Web-based surveys with touchscreens.

Pennsylvania hired TeleSage for its IVR outcomes tracking pilot project concerning behavioral health outcomes of residential patients, with website for administrative-level outcomes tracking.

2008 / 2010
Then in 2008 and 2010, respectively, the states of Iowa (via an open RFP process) and Louisiana also entered into statewide contracts with TeleSage to meet their outcomes tracking needs. Currently, TeleSage, Inc. is engaged in statewide outcomes tracking in Tennessee and Louisiana.

TeleSage’s outcomes measurement systems and software have been used for other projects across North America, South America, Europe, the Middle East, Asia, and Australia.

ASSESSMENT & DOMAINS

CLIENT OUTCOMES ASSESSMENT

The TeleSage Outcomes Measurement System (TOMS) assessment measures outcomes, including symptom presentation, functioning, and National Outcomes Measurement System (NOMS)-specific outcomes.

Customers may create custom assessments using the following outcomes (administered using a “past 7 days” or “past 30 days” timeframe, as appropriate):

Adult Populations
Depression, Anxiety, Anger, Psychosis, Social Functioning – Peer, Social Functioning – Family, Social Functioning – Spouse, Work Functioning, Daily Functioning, Recovery, Medication Adherence, Medication Side Effects, Drug Use, Alcohol Use, Tobacco Use, Arrests, Criminal Justice Contact, Living Situation, Transportation, Victimization, & Emergency/ Hospitalization.

Youth Populations 12-17 Years of Age & Parent/Guardian Report for Youths 5-12 Years of Age
Depression, Anxiety, Anger, Risky Behavior, Hyperactivity, Inattention, & Conduct, and School Functioning, Social Functioning –Peer, Social Functioning – Peer Substance (assesses affiliation with substance using peers), Social Functioning – Family, Daily Functioning, Resiliency, Medication Adherence, Medication Side Effects, Drug Use, Alcohol Use, Tobacco Use, Arrests, Criminal Justice Contact, Living Situation, Victimization, & Emergency/ Hospitalization.

Addictions Programs
TeleSage has been involved in tracking outcomes in residential and outpatient substance use treatment facilities, both adult and youth, since the start of our long tenure in outcomes tracking. The majority of domains for these populations stem from our work on the Self-report Addiction Severity Index (ASI-SR) and the Self-report Teen Addiction Severity Index (TASI-SR).

ADMINISTRATION

Frequency of Administration
The frequency with which assessments are administered to clients represents a tradeoff between client/staff burden and the value of the information that can be obtained. Our assessments may be administered as often as once a week given the “past 7 days” timeframe. Based on our experience and analyses comparing private and public-sector clients, shorter assessment periods appear to be beneficial for private sector clients, who largely experience acute disorders (e.g., depression, anxiety) and therefore shorter treatment durations and steeper, truncated recovery curves compared to public-sector clients. Public-sector clients, especially severely and persistently mentally ill (SPMI) clients, are more likely to suffer from Bipolar Disorder (I & II), Schizoaffective Disorder, Schizophrenia and other chronic, severe disorders which result in longer treatment durations and less steep, elongated recovery curves. Given the level of administration burden on staff and public-sector clients, we find that longer periods between assessment administrations (e.g., monthly or quarterly) achieves an optimal balance between level of client/staff burden and the utility of data for evaluation of effectiveness.

Modality of Administration
Assessments can be administered via PC (using touchscreen, tablet, or standard setup), phone, or paper. Computer and phone data are immediately entereed into the database and reports are generated automatically and delivered via a link in the EHR system. Paper survey data can be entered via Rapid Data Entry (RDE) Web-interface and reports are generated and avalable in EHR once data are entered.

RELIABILITY & VALIDITY

The TOMS instruments are evidence-based and have been developed and refined by Telesage, Inc. through a rigorous series of studies over the past several years. A detailed summary of this process and findings are included in “TeleSage Outcome Measurement System (TOMS): An Overview of Item Development and Psychometric Properties“. The following is a brief synopsis:

The development of items and domains of TOMS instruments followed a systematic, multi-step process. The original item pool was developed through a review of core symptoms/ impairments and quality of life issues found in the DSM-IV and popular clinical assessment and outcome instruments now in frequent use (e.g., BASIS-32; SF-36). A panel of behavioral health experts then selected a core set of items and domains which showed high “face validity.” The item set was then presented to a group of behavioral health clients by an interviewer who has experience with cognitive interviewing. Items that were difficult to understand or confusing to clients were deleted or revised and retested. Validity and reliability was established for the final set of items using Modern Measurement Theory and Classical Test Theory.

The psychometric validity of the TOMS was established by statistically correlating domain scores with well-established instruments, such as the PHQ-9, BDI-II, and BAI, etc. The domain scores correlated in expected ways with these ‘gold-standard’ instruments, establishing both convergent and discriminant validity of the TOMS.

The psychometric reliability of the TOMS was established using rigorous Item Response Theory (IRT) analyses. IRT analyses indicate that the items within each domain fit the unidimensional model well and demonstrated low standard error (high reliability) across the entire range of severity (theta).

The sensitivity of the TOMS to detect differences was established using IRT analyses. The TOMS instruments are sensitive to detect differences in symptom severity and functionality across all races, genders, and clinic sites (public vs. private).

Studies have shown that the TOMS items can be successfully administered via various methods of administration, e.g., pencil/paper, touchscreen, PDA, and telephone. In particular, clients rated the touchscreen as the easiest modality for viewing the questions, but no significant difference was detected among modalities for ease of understanding the survey questions.

In summary, results of the psychometric studies indicate the TOMS instruments have:

High clinical relevance as indicated by breadth of coverage of relevant domains of client quality of life and as judged by panels of experts and clients.

High validity (convergent and discriminant) when tested against other ‘gold-standard’ instruments.

High reliability as indicated by classical test-retest and IRT analyses.

High sensitivity and relevance for all behavioral health clients.

REPORTING

Clinician Reports

Clinician Reports are generated instantaneously for all TOMS assessments upon data entry and are available for immediate access via the a hosted secure TOMS web page and/or as an integrated part of the client’s EHR record. These reports allow clinicians to easily view consumers progress over time with graphs showing the consumer’s current assessment along with the two or three most recent assessments for each domain, as well as detailed information about responses to individual questions.

Aggregate Reports

TeleSage, Inc. offers a few different Aggregate Reports. These reports are often aggregated by clinic/sub-clinic, agency, or state. These reports can be used by administrators and policymakers to be accountable to the state & also make decisions pertaining to allocation of resources, and include:

Implementation Reports (pilot phase): Session & item-level statistics

Quarterly Reports: Comparing across clinics, agencies, & the state

Report Cards: Wellness at a glance (reported quarterly)

SUPPORT

SUPPORT DOCUMENTATION INCLUDES:

About: Basic one-page information sheet explaining the system and its functions.

Instructions for Administering Surveys:
Step-by-step instructions for staff describing how to properly administer the survey on paper, web and phone

Client guides – Phone & Web Survey
Step-by-step instructions to guide clients through the survey process

Client Information Sheet
Basic one-page information sheet for clients which explains the survey and the client’s rights.

Rapid Data Entry Instructions
Step-by-step instructions for staff describing how to properly and quickly enter paper surveys into the system using Rapid Data Entry (RDE).

Frequently Asked Questions (FAQ)
List of FAQs that covers every part of the system; it should be the first place to look when any problem is encountered.

SUPPORT FEATURES INCLUDE:

Report a Technical Problem
If there is any problem with the technical or functional aspects of the system, please use this page to report directly to TeleSage.

Training Sessions (Flash Animations)
A series of short instructional animations that should be viewed before using the system. These real-time demonstrations are available 24/7 for training staff when the system is first implemented and also for training new members of staff after the system is in place. Current sessions include: Introduction to the Outcomes Tracking System ; Clinical Utility ; Getting Started ; TOMS Surveys ; Online Surveys: Computer, Monitor and Mouse ; Online Surveys: Touchscreen or PDA, Paper Surveys with Rapid Data Entry; Phone Surveys; Administrative Features; Clinician Reports; and Getting Help.

Support via email
Provides email links for support problems.
Note: Response within one working day.

Support to clinicians via phone and email
Support provided by project manager dedicated to this project to handle any issues during and after normal business hours.

For more information, to order, or to schedule a free webinar

Standardized Behavioral Health Diagnoses and Outcomes
Chapel Hill, North Carolina 27514 | USA
866.942.8849

Enter your keyword