A Peer Reviewed Publication of the College of Allied Health & Nursing at Nova Southeastern University

Correspondence:
Prof Karen Grimmer
Director
Centre for Allied Health Evidence
University of South Australia
City East Campus
North Tce
Adelaide 5000
Email: Karen.Grimmer@unisa.edu.au

Citation:
Grimmer, K., et al. CAHE outcomes calculator: An allied health initiative to improve practice. The Internet Journal of Allied Health Sciences and Practice. October 2005. Volume 3 Number 4.

Introduction
A common finding when evaluating the uptake of clinical guidelines has been the importance of clinicians monitoring patient progress using standard outcome measures, in order to demonstrate and reflect on, the effectiveness of intervention.^1-4 This information is important for quality assurance purposes within clinical practices, and it is also integral for continuity of care, by informing the patient themselves, other health care providers, referring doctors and/ or funding agencies about patient progress.³ There is a plethora of outcome measures available for use by therapists, particularly when treating musculoskeletal conditions, however there are few practical supports to assist in immediate calculation of change using these measures, particularly within an episode of care for the one patient.^5,6 Outcome measures can reflect impairment, functional capacity and participation (reflecting current World Health Organization diagnostic classification criteria).⁷ Recently published good quality clinical guidelines on the management of acute low back pain illustrate the point. These guidelines all agree on the importance of clinicians using standard clinical outcomes in order to benchmark within- and between- practices, and to demonstrate the effectiveness of their care on a patient-by-patient basis, as well as a group basis.^8-14

Many barriers to uptake of evidence into practice have been proposed, not least of which is the lack of readily available clinical information about performance to allow comparison with best-practice.^15,16 Following our publication regarding best practice when treating elderly veterans (which incorporates regularly applied standard outcome measures), and our review of clinical guidelines, which identified the lack of consensus on what constituted good therapy practice, our group identified the urgent need to provide clinicians with a tool which could be used in the clinical setting to record and evaluate outcome of care.^17,18  Thus we developed a software-based instrument to assist clinicians in calculating patient outcomes, using a selection of outcome instruments.  This paper reports on the process and the resultant product.

Method
Purpose and Aims
Our purpose was to develop an inexpensive, simple and effective mechanism by which therapists could efficiently demonstrate their accountability in clinical practice.  

Our aims were to:

• identify measures of outcome common in clinical practice and also commonly reported in the literature, which had good published evidence of psychometric properties, and for which we could obtain permission from the developers to use free of charge in our product

• produce software (Centre for Allied Health Evidence (CAHE) Outcomes Calculator) for use in clinical practice by therapists with minimum computing skills, to collect and report information on health outcomes throughout the episode of care⁶  

• design mechanisms for automating the calculation of change in outcome measures (metrics) by the Outcomes Calculator software, which could be reported in graph or table format for discharge plans, or for letters to referrers and insurers

• develop a detailed manual for instruction, self training and reference purposes, to accompany the software, which provides information about how the outcome measures were chosen, and how change within the episode of care could be interpreted for accountability, quality improvement and research purposes.  

Framework for developing the software
The project team involved a software developer (BM), a website manager (TM) and clinicians / health researchers (KG, SM, SK, AB, YD, LD, PN, SH) who have taken various roles during product development, including literature reviewing, evaluation of information on psychometric properties, corresponding with instrument developers, writing the instruction manual, assisting in software development, testing and modification, and providing backup support for software users. For a number of reasons, the team decided that the software would be available free of charge. This firstly reflected the generosity of the instrument developers in allowing their instruments to be used in the software free of charge. The availability of freeware to assist therapists with outcome calculation was also congruent both with the mission statement of the Centre for Allied Health Evidence (University of South Australia), and the context of the industry relationship between CAHE, its funder (the Dept of Health, South Australia), and its parent body (the Joanna Briggs Institute). CAHE is auspiced with providing overt support and leadership for allied health therapists to consider and adopt evidence-based practices, to improve the quality of their care and the health outcomes of their patients. Ready availability of software which could assist therapists to compute change in common outcome measures was one tangible way in which CAHE could address this objective.

Framework for choosing outcome measures
Following extensive discussions with clinicians and academics associated with the Division of Health Sciences, University of South Australia, on what was required to assist clinicians to compute change in outcome measures, we focused on those musculoskeletal outcome measures:

• which were commonly used in clinical practice and high quality research reporting,

• which had published evidence of their psychometric properties,

• whose publications included clear, accurate and reproducible instructions on calculation of the metrics associated with the outcome measure (how to interpret change),

• which had published population norms or clearly defined expected endpoints that indicated improvement,

• which fit with the World Health Organization disease classifications using an episode of care model (in which therapists may treat patients a number of times (linked occasions of service) for the one condition7,27 and

• whose developers provided written permission for our team to use the outcome measure without charge.**  

We took an initial approach which primarily focused on spinal problems, because management of spinal problems was core business for most musculoskeletal therapists in Australia.^37,38 Moreover, we were not in a financial position to extend this version of the calculator to incorporate peripheral-joint-specific outcome measures. Future versions of the CAHE Outcomes Calculator will incorporate peripheral joint musculoskeletal measures.

Process of outcome measure choice
The steps we took to identify the outcome measures which were included in our Outcomes Calculator included:

• collating a list of the outcome measures commonly used in clinical reports and in high methodological quality experimental studies on management of musculoskeletal problems (generic and for the spine) in the previous five years (2000-2005),

• identifying those measures which fitted within the WHO model of disease classification,

• identifying those measures with published psychometric properties, and information on metric calculation, and

• seeking and receiving written permission from the outcome measure developer(s) to use their instrument in the Outcomes Calculator free of charge.

User manual
For each selected outcome instrument, we constructed a users’ manual with common subsections including brief background information on the measure, a description of the purpose and construction of the outcome instrument, how it was  scored (metrics), desired direction and amount of change that indicated improvement, population norms (if available), the outcome measure itself (which could be copied for use in the clinic), how often the outcome measure should be applied throughout the episode of care, the formulae to calculate metric change (as appropriate), published evidence of psychometric properties (usually validity, reliability, sensitivity to change), and appropriate reference material.

Baselines / benchmarks
An important element when assessing improvement in outcomes is comparison with "normal" or "patient’s usual performance." Thus integral to the CAHE Outcomes Calculator was a report for each outcome measure of how patients progressed towards expected outcomes. We sought outcome measures which clearly stated from psychometric testing, the desired direction and amount of change. For each set of outcome measure metrics we incorporated assessment against patients’ own baseline (their initial treatment score), as well as against expected "normal." For many of the instruments, improvement towards "normal" reflected movement towards zero, where higher scores indicated greater levels of functional loss or impairment. For joint range of movement, we provided a composite table of population norms derived from a number of common texts for different age groups. We also reported raw and standardized scores so that comparisons could be made within and between patients. Scores were standardized as percentages, with a choice of denominator (initial treatment as baseline, or previous treatment within the episode of care, as baseline).

Collection of outcome information
Repeated collection of outcome measures in the clinical setting takes time, and thus we also favored outcome measures whose administration by therapists was efficient, or where outcome assessment could be undertaken with patients prior to (or after) treatment using paper copies of the instrument. Administrative staff in clinical practices could then enter patients’ data into the calculator for time efficiency. Where patients were computer-literate and practices could provide access to computer terminals, patients could also enter their data directly into their own records on the software.

Software development
The software was written in Java script and was mounted on a MS (Microsoft) Access database. The development team designed algorithms to underpin each of the calculator screens, and navigation through the software program. The initial data collection screen sought demographic information, and subsequent screens identified body area requiring treatment, the selection of outcome measures relevant to each body part, dates of episode commencement and completion, the dates of occasions of service within the episode, and data collection screens for each of the outcome measures at each point of data collection. The software allowed outcome data to be collected on more than one body part at each occasion of service, and in addition, more than one outcome measure could be used for each body part. The opening screen of the CAHE Outcomes Calculator is provided in Figure 1.

Figure 1. Opening screen of the CAHE Outcomes Calculator

We designed the demographic data screen to also identify mechanism of funding (self, private health insurance, compensable etc) and common risks for poor outcome from therapy. These classifications would allow therapists to reflect on why patients failed to achieve the desired outcome (benchmark) at the end of the episode of care, and would provide additional explanatory information during data analysis. We sought to identify a range of risks which have been widely reported as likely to influence outcome of therapy musculoskeletal conditions, including psychosocial factors, physiological factors (overweight, unfit, chronic / multiple health conditions) and occupational factors such as repetitive or lowly paid work.^19-26 The software is not designed to assess risk, thus identification of potential risks by the therapists need to be based on clinical reasoning, or prior subjective or objective assessment using standard risk assessment tools. Risks are flagged by tick-box options only.

Keeping track of users
The software was designed to be downloaded free-of-charge from the website of the Centre for Allied Health Evidence, University of South Australia, using a preliminary registration process. Completion of this registration process guides users to the software download site. Knowing who is using the calculator has allowed our team to initiate feedback activities, and to publicise upgrades as they are brought online. The website for calculator download is www.unisa.edu.au\cahe.

Trialing and modification
The early versions of the CAHE Outcomes Calculator were trialed by volunteer therapists for utility, errors and applicability of reporting. Modifications were made each time to improve screen design, navigation, accuracy and ease of data entry and reporting. We anticipate that the CAHE Outcomes Calculator will always be a work in progress, given the ongoing changes to programming languages and IT (information technology) platforms, and opportunities to incorporate more outcome measures, and better data handling functions.  

Results
We identified 18 potentially useful outcome measures, of which we retained 12 for use in the first version of the CAHE Outcomes Calculator. Reasons for not including potential outcome measure were lack of convincing evidence of psychometric properties and / or lack of information on benchmarks / population norms (2), lack of permission from the developers to use the instrument free of charge (3), and lack of information on metric calculation (1).

The outcome instruments selected for use in the Outcomes Calculator were:

• Joint range of movement

• Uni-dimensional measures of pain

• Neck Disability Index²⁸

• Graded Chronic Pain Scale²⁹

• Patient Satisfaction subscales²³

• Patient Specific Scale³⁰

• Glasgow Pain questionnaire³¹

• Roland-Morris questionnaire³²

• Oswestry questionnaire³³

• WL-26³⁴

• Timed Up and Go Test^35,36

Relevant to any body part were the pain scales and range of movement information addressed the impairment element of the WHO Disease Classifications, the Patient Specific Scale, WL-26 and the Timed Up and Go Test addressed the functional / participation element of this classification.⁷ The Patient Satisfaction Subscales allowed collection of post-hoc reflections of patients on their satisfaction with the clinical elements of care, thus providing a useful mechanism for reflection by therapists. The remaining scales are commonly used to measure function in patients with spinal problems.⁷ The outcome measures in context of the WHO International classification of diseases are outlined in Table 1.

Table 1. Chosen measures of outcome

The first two drafts of the CAHE Outcomes Calculator were trialled by 20 volunteer physiotherapists, mostly in Australia and New Zealand. Modifications were undertaken as a result of feedback, and ranged from correcting errors in data entry screens, enhancing the accompanying manual, improving graphs of outcome measure change over the episode of care, providing more metrics to demonstrate change (percentage improvement from baseline at each occasion of service, and over the entire episode), and options to report domain scores as well as total instrument scores (for instance in the Patient Specific Scale). We also developed a demonstration aspect to the calculator using a dummy episode of care (setting up the number and frequency of treatments within an imaginary episode of care), which removed the constraints of working in "real time" as one would in a clinic environment with a real patient.

Technical support
One of our concerns was the amount of technical support that the CAHE Outcomes Calculator might require. Installation onto a server system was the only issue which posed problems however, and CAHE staff became adept at trouble shooting the minor problems over the telephone. To date we have over 1000 registrations from therapists world-wide.

Feedback to date
Informal feedback to date from users has indicated high levels of satisfaction with the Calculator, with respect to clinical utility, ease of data entry, choice of outcome tools (including online and academic support in the manual), metric options and graphical representation of the outcome tool change scores. Feedback has also drawn attention to the level of sophistication of many therapists and their referring doctors in understanding what outcome measure change actually means (in particular the functional scales). This highlights the need for ongoing clinical research and development into choice, application and interpretation of outcome measures in terms of patients’ clinical progress and their capacity to function safely and effectively in their community.

Conclusion
The CAHE Outcomes Calculator provides a rare opportunity for therapists to readily apply standard outcome instruments in clinical practice for musculoskeletal conditions, using a range of outcome measure choices. The calculator assists therapists to demonstrate their effectiveness to patients, referrers and funding agencies, and to undertake quality assurance activities in order to provide practice based on evidence. The choice of outcome measures was based on pragmatics of published psychometric properties, the underlying metrics, permission to use the instrument, and clinical utility.  Given the huge number of published outcome instruments, our decisions were often based on financial viability, practicality and volume of publication available.

Feedback about the tool and its clinical application has been overwhelmingly positive to date, with a common finding being the difficulties many therapists have in translating paper-based information on outcome measurement into something useful. The outcomes calculator appears to transcend cultural and language barriers as approximately 40% of its downloads have come from 14 overseas countries. Thus it appears that the CAHE Outcomes Calculator provides therapists with a readily accessed, low cost, readily reported tool to assist in uptake of evidence and review of clinical practice.

At present, we have not progressed the capacity of the calculator to provide standard queries, for instance those written for subsets of patients (such as those with neck pain). We plan that this option will be available in a future version, which will reflect clinician input into what information would be useful in a practice improvement sense. We plan to release upgraded versions of the Calculator later in 2005, including Incontinence and Neurological measures. A persuasive element in deciding on these two clinical areas as our next focus was the current availability of compendiums of relevant clinical outcome measures developed by practitioners and researcher in Australia. This documentation (also available free of charge) saved our team hours of work by providing us with copies of instruments, summaries of psychometric properties, and basic metric calculation formulae.

We would like to acknowledge the support and enthusiasm of the many therapists around the world who have assisted us in bringing our dream of efficient outcome measurement for therapists to life.

References

1. Bekkering GE, Engers AJ, Wensing M, Hendriks HJM, van Tulder MW, Oostendorp RAB, Bouter LM. Development of an implementation strategy for physiotherapy guidelines on low back pain. Australian Journal of Physiotherapy 2003; 49: 208-216.

2. Grol R, Zwaard A, Mokkink H, Dalhuijsen J, Casparie A. Dissemination of guidelines: which sources do physicians use in order to be informed? International Journal for Quality in Health Care 1998;10(2):135-40.

3. Li C, Bombadier C. Physical therapy management of low back pain: an exploratory survey of therapist approaches. Phys Th 2001; 81: 1018-1028.

4. Van Tulder M, Tuut M, Pennick V, Bombadier C, Assendeft W. Quality of primary care guidelines for acute low back pain. Spine 2004; 29: E357-362

5. Beatie P. Measurement of health outcomes in the clinical setting: applications to physiotherapy. Physiotherapy Theory and Practice 2001; 17: 173-185.

6. Grimmer K, Bowman P, Roper J. Episodes of allied health outpatient care: an investigation of service delivery in acute public hospital settings. Disability and Rehabilitation 2000; 22(1/2): 80-87.

7. World Health Organization. International classification of functioning, disability and health. Geneva: World Health Organization 2001.

8. University of Michigan Health System. Acute Low Back Pain. Guidelines for Clinical Care. Available at: http://cme.med.umich.edu/pdf/guideline/backpain03.pdf. Accessed November 19, 2004.

9. PRODIGY. Guidance- Back Pain- Lower. Available at: http://www.prodigy.nhs.uk/guidance.asp?gt=back%20pain%20-%20lower. Accessed November 19, 2004.

10. The Norwegian Back Pain Network- The communication unit. Acute Low Back Pain. Interdisciplinary clinical guidelines. Available at: http://www.ifomt.org/pdf/Norway_Acute_Low_Back.pdf. Accessed November 19, 2004.

11. Institute for Clinical Systems Improvement. Health Care Guideline: Adult Low Back Pain. Available at: http://icsi.org/index.asp. Accessed November 19th, 2004.

12. Australian Acute Musculoskeletal Pain Guidelines Group. Evidence-based Management of Acute Musculoskeletal Pain. Available at: http://www7.health.gov.au/nhmrc/publications/pdf/cp94.pdf. Accessed November 19, 2004.

13. Bekkering GE, Hendriks HM, Koes BW, Oostendorp RB,  Ostelo R, Thomassen JM. Dutch physiotherapy guidelines for low back pain. Physiotherapy 2003;89:82-96.

14. American Academy of Surgeons (AAOS). AAOS Clinical guideline on Low Back Pain/ Sciatica (acute) (phases I and II). Available at: http://www.guideline.gov/summary/summary.aspx?view_id=1&doc_id=5369. Accessed November 19, 2004.

15. Bero LA, Grilli R, Grimshaw J, Harvey E, Oxman A, Thomson M (1998): Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ 317: 465-468.

16. Metcalf C, Lewin R, Wisher S, Perry S, Bannigan K, Klaber Moffett J (2001): Barriers to implementing the evidence base in four NHS therapies. Physiotherapy 87: 433-440

17. Grimmer K (2002): Providing evidence of good allied health care for the veteran population: the development of a unique management system. Journal of Allied Health 31 (1): 15-21.

18. Grimmer K, Milanese S, Bialocerkowski A. Clinical guidelines for low back pain: Physiotherapy perspective. Physiotherapy Canada 2003;55:185-94.

19. Burton AK, Tillotson KM (1991): Prediction of the clinical course of low back trouble using multivariate models. Spine 16: 7-14.

20. Burton AK, Tillotson M, Main CJ, Hollis S (1995): Psychosocial predictors of outcome in acute and subchronic low back trouble. Spine 20: 722-728.

21. Croft PR, Papageorgiou AC, Ferry S, Thomas E, Kayson MIV, Silman AJ (1996): Psychological distress and low back pain: evidence from a prospective study in the general population. Spine 20: 2731-2737.

22. Frymoyer JW and Cats-Baril W (1987): Predictors of low back pain disability. Clin Orthop 221: 89-98.

23. Cherkin DC, Deyo RA, Street JH, Barlow W (1996): Predicting poor outcomes for back pain seen in primary care using patients’ own criteria. Spine 21: 2900-2907.

24. Gatchel RJ, Polatin PB, Mayer TG (1995): The dominant fole of psychosocial risk factors in the development of chronic low back pain disability. Spine 20: 2721-2630.

25. Waddell G, Newton M, Henderson I, Somerville D, Main CJ (1993): A fear-avoidance belief’s questionnaire (FABQ) and the role of fear-avoidance beliefs in chronic low back pain and disability. Pain 52: 157-168.

26. Waddell G, Somerville D, Henderson I, Newton M (1992): Objective clinical evaluation of physical impairment in chronic low back pain. Spine 17: 617-628.

27. Grimmer K, Bowman P, Roper J (2000): Episodes of allied health outpatient care: an investigation of service delivery in acute public hospital settings. Disability and Rehabilitation 22(1/2): 80-87.

28. Vernon H and Mior S (1991): The neck disability index: a study of reliability and validity. Journal of Manipulative and Physiological Therapeutics 14: 409-415.

29. Von Korff M, Jesen M and Karoly P (2000): Assessing global pain severity by self-report in clinical and health services research. Spine 25: 3140-3151.

30. Stratford P, Gill C, Westaway M and Binkley J (1995): Physiotherapy Canada 47 (4): 262.

31. Thomas R, McEwen J and Ashbury A (1996): The Glasgow pain questionnaire: a new generic measure of pain: development and testing. International Journal of Epidemiology 25: 1060-1067

32. Roland M and Morris R (1983): A study of the natural history of low back pain: Part 1. Development of a reliable and sensitive measures of disability in low back pain. Spine 8: 141-144.

33. Fairbank J and Pynsent P (2000): The Oswestry disability index. Spine 25: 2940-2953.

34. Amick BC, Lerner D, Rogers WH, Rooney T, Katz JN (2000):A review of health related work outcome measures and their uses, and recommended measures. Spine 25(24): 3152-3160

35. Mathais S, Nayak US, Isaacs B (1986): Balance in elderly patients: the ‘get-up and go’ test. Archives of Physical Medicine and Rehabilitation 67: 387-389.

36. Podsaidlo D and Richardson S (1991): The timed ‘up and go’: a test of basic functional mobility for frail elderly persons. Journal of the American Geriatrics Society 39: 142-148.

37. Grimmer K, Sheppard L, Pitt M, Magarey M and Trott P. Differences in Stakeholder Expectations In the Outcome Of Physiotherapy Management of Acute Low Back Pain. International Journal for Quality in Health Care (IJQHC) 11 (2): 155-162, 1999

38. Grimmer K, Kerr J, Hughes K, Trott P and Pitt M. An Overview of the Australian Physiotherapy Association Accredited Practice Data Collection 1995-6 Australian Journal of Physiotherapy 44 (1):61-63, 1998

[**] The development of the calculator was self-funded by CAHE, and was not intended as a profit-making venture. Thus the development team was not in a position to pay for license fees or ongoing outcome measure management costs.

Print Version