TOMS

In response to the growing need for behavioral health outcomes tracking, Dr. Brodey founded TeleSage in 1996 for the purpose of gathering patient-centered outcomes tracking data for mental health clinical and research institutions. With R&D funding from NIH, TeleSage, Inc. pioneered the development and use of research tools for tracking patient-reported outcomes, including diagnostic assessment, treatment evaluation, and reporting to regulatory agencies.

In 1999, under a state mandate for adult and youth outcomes tracking, 18 counties in the state of Ohio, including Cincinnati, hired TeleSage to host their outcomes tracking system. From 1999-2003, TeleSage created and hosted a large-scale project titled KidsPeace Outcomes Tracking for Youth in the state of Pennsylvania.

In 2001, the state of Washington hired TeleSage (via an open RFP process) to implement an outcomes measurement system for the entire state. More than 300,000 IVR, RDE, and fax-based surveys were administered in Washington.

Also in 2001, the Idaho Department of Health and Welfare hired TeleSage to track the outcomes of all public sector substance abuse clients. Beginning in 2006, TeleSage began a successful working relationship with the state of Tennessee (via an open RFP process) to offer statewide outcomes tracking utilizing IVR and Web-based surveys with touchscreens.

In 2006, Pennsylvania hired TeleSage for its IVR outcomes tracking pilot project concerning behavioral health outcomes of residential patients, with website for administrative-level outcomes tracking. Then in 2008 and 2010, respectively, the states of Iowa (via an open RFP process) and Louisiana also entered into statewide contracts with TeleSage to meet their outcomes tracking needs. Currently, TeleSage, Inc. is engaged in outcomes tracking in Tennessee and Louisiana.

All of the systems described above were completed on time and within budget. Of note, TeleSage’s outcomes measurement systems and software have been used for other projects across North America, South America, Europe, the Middle East, Asia, and Australia.

The TOMS instruments are evidence-based and have been developed and refined by Telesage, Inc. through a rigorous series of studies over the past several years. A detailed summary of this process and findings are included in “TeleSage Outcome Measurement System (TOMS): An Overview of Item Development and Psychometric Properties“. The following is a brief synopsis:

The development of items and domains of TOMS instruments followed a systematic, multi-step process. The original item pool was developed through a review of core symptoms/ impairments and quality of life issues found in the DSM-IV and popular clinical assessment and outcome instruments now in frequent use (e.g., BASIS-32; SF-36). A panel of behavioral health experts then selected a core set of items and domains which showed high “face validity.” The item set was then presented to a group of behavioral health clients by an interviewer who has experience with cognitive interviewing. Items that were difficult to understand or confusing to clients were deleted or revised and retested. Validity and reliability was established for the final set of items using Modern Measurement Theory and Classical Test Theory.

The psychometric validity of the TOMS was established by statistically correlating domain scores with well-established instruments, such as the PHQ-9, BDI-II, and BAI, etc. The domain scores correlated in expected ways with these ‘gold-standard’ instruments, establishing both convergent and discriminant validity of the TOMS.

The psychometric reliability of the TOMS was established using rigorous Item Response Theory (IRT) analyses. IRT analyses indicate that the items within each domain fit the unidimensional model well and demonstrated low standard error (high reliability) across the entire range of severity (theta).

The sensitivity of the TOMS to detect differences was established using IRT analyses. The TOMS instruments are sensitive to detect differences in symptom severity and functionality across all races, genders, and clinic sites (public vs. private).

Studies have shown that the TOMS items can be successfully administered via various methods of administration, e.g., pencil/paper, touchscreen, PDA, and telephone. In particular, clients rated the touchscreen as the easiest modality for viewing the questions, but no significant difference was detected among modalities for ease of understanding the survey questions.

In summary, results of the psychometric studies indicate the TOMS instruments have:

High clinical relevance as indicated by breadth of coverage of relevant domains of client quality of life and as judged by panels of experts and clients.

High validity (convergent and discriminant) when tested against other ‘gold-standard’ instruments.

High reliability as indicated by classical test-retest and IRT analyses.

High sensitivity and relevance for all behavioral health clients.

Clinician Reports

Clinician Reports are generated instantaneously for all TOMS assessments upon data entry and are available for immediate access via the a hosted secure TOMS web page and/or as an integrated part of the client’s EHR record. These reports allow clinicians to easily view consumers progress over time with graphs showing the consumer’s current assessment along with the two or three most recent assessments for each domain, as well as detailed information about responses to individual questions.

Aggregate Reports

TeleSage, Inc. offers a few different Aggregate Reports. These reports are often aggregated  by clinic/sub-clinic, agency, or state. These reports can be used by administrators and policymakers to be accountable to the state & also make decisions pertaining to allocation of resources, and include:

Implementation Reports (pilot phase): Session & item-level statistics

Quarterly Reports: Comparing across clinics, agencies, & the state

Report Cards: Wellness at a glance (reported quarterly)