Joule - A Preliminary Report on Effectiveness and Client Satisfaction


Joule Home Page Or Back to Articles

“Functional capacity evaluations (FCEs) have become part of practice in work injury prevention and rehabilitation. These tools are supposed to define an individual's functional abilities or limitations in the context of safe, productive work tasks.” Thus is introduced a current article by King, Tuckwell, and Barrett (1998), titled “A Critical Review of Functional Capacity Evaluations.” The article appeared in the August edition of Physical Therapy, and not only reviews existing FCEs, but also makes recommendations regarding important elements of the well-designed FCE.

Joule is a new functional capacity evaluation (FCE) system by Valpar International Corporation. It is an entirely new system of equipment and protocols with a unique guiding philosophy that sets it apart from all other FCE products and systems. We believe that Joule substantially meets the requirements set forth in the King, Tuckwell and Barrett article (1998) for the well-designed FCE.

This report will: 1) briefly introduce Valpar and Joule, touching upon Joule's painstaking development and unique qualities; 2) summarize a preliminary follow-up study of Joule clients in terms of their satisfaction with various elements of their FCE-related experiences, and of their post-FCE experiences; and 3) discuss plans for future Joule research.

Valpar International Corporation

For many years, Valpar has been widely known as a developer of work samples and other assessment products that are criterion- referenced to the U.S. Department of Labor's (DOL) job analysis and classification system. That system comprises the revised 4th edition of the Dictionary of Occupational Titles (DOL, 1991)(DOT) and the Revised Handbook for Analyzing Jobs (DOL, 1991)(RHAJ). The DOL used the approach described in the RHAJ to analyze and classify over 12,700 occupations defined in the DOT. Each of those occupations has a list of work-related factors, rated at specific levels that are required of successful workers.

Valpar's work samples were originally developed in the early 1970s for use in the vocational rehabilitation field, but have been used extensively in allied health as elements within FCEs and work hardening services since the late 1970s. The criterion- referenced nature of Valpar's work samples appealed to physical therapists, occupational therapists, and other allied health professionals who were by training inclined toward that method of interpreting assessment results. Many in allied health suggested that Valpar develop a product designed specifically for FCE purposes. Joule is that product: it is the culmination of over four years of research, and it incorporates technologies gleaned from Valpar's twenty-five years of experience and the expertise of nine professional providers of FCEs, each with many years of relevant experience.

Development of Joule

In early 1994, Valpar convened a panel of allied health professionals to study the need for new products to serve the allied health professional community, including an FCE product. Over the course of several days that meeting produced several useful recommendations, but it was clear that more study was needed on issues related to the development of an FCE product.

In the summer of 1995, several of the original panel members were consulted specifically to finalize a broad-based survey for presentation to another panel of professional therapists. The survey would, it was hoped, yield the information necessary for Valpar to develop a superior FCE product. In the fall of 1995, a working panel of five therapists was established to respond to the survey questions.

The survey contained 45 questions. It elicited responses on a range of issues from the “ideal” FCE to the affordable cost of such a product. The survey responses were clarified and synthesized into recommendations that were to form the basis for Joule.

Mary Ruprecht, OTR, the Director of Industrial Medicine at the Two Rivers Center in Coon Rapids, Minnesota, was retained as the primary consultant to finalize the panel's recommendations. Ms. Ruprecht then assumed the lead in Joule's development, designing and testing protocols and coordinating with Valpar engineers and other staff.

Important Features of Joule

Several salient issues common to all FCEs emerged during the course of Joule's development and became the focus of much attention. The solutions to those issues became key features of Joule and have served to make it unique among such commercial systems.

All FCEs must deal with the issue of safety: Since the overall goal of an FCE is to estimate safe levels of work, how can that be done with any confidence based upon a series of short duration functional tests? How should physically demanding tests be conducted safely? When should such tests be terminated? Who should make those decisions, clients or therapists?

Furthermore, it is often the case that, for various reasons, clients do not exert themselves sufficiently to allow accurate assessments of their true capacities. How should that issue be handled?

In Valpar's view, none of the commercially available FCE systems address these and other issues satisfactorily, and it was Valpar's goal from the beginning to provide superior answers to these problems. We believe we have succeeded.

All of Joule's protocols emphasize a functional approach to more closely simulate real world work tasks than the “isokinetic” approaches used by some FCE systems. This serves to enhance FCE accuracy, not only because test protocols are more realistic, but because the more realistic functional tests increase “face validity” which, in turn, encourages fuller client participation.

The Joule FCE process incorporates computer technologies to customize and organize the tests, process and analyze information, and automatically produce customized reports. This feature alone saves much time and greatly enhances the accuracy of the entire FCE.

The Joule system is completely flexible, and is designed to accommodate the particular situation of each client. Joule's flexibility is illustrated in the following several points.

Another Joule feature that sets it apart from other commercial FCE systems is the manner in which its “core” lifts are administered. Typically, in other FCEs, clients complete an entire lift series (say, lifting weights from waist height to eye level) before beginning a new one. This approach makes little sense in Valpar's view, because by the time the client reaches his or her maximum weight in that lift, he or she is under some degree of fatigue that surely affects subsequent lifts, no doubt serving to reduce client performance and produce inaccurate results.

The Joule approach is to administer all three core lifts (waist to waist, waist to mid-shin, waist to eyes) at the same time; all three lifts are performed simultaneously, alternating the lifts after one time at each weight level. In this fashion, clients are not unduly fatigued by any one lift series and their maximum safe weight levels more closely reflect their actual capacities.

An important FCE issue concerns the general approach taken in determining the capacities of clients as they undergo the various protocols. Most FCEs may be classified into two kinds depending on how they deal with the issue of who controls the test (client or evaluator) and whether objective or subjective information is used to determine functional maximums. The “psychophysical” FCE relies almost entirely upon subjective client reports with client-controlled termination points. The “kinesiophysical” approach, on the other hand, minimizes client reports and, instead, relies upon the evaluator's observations. Joule does neither; instead, several kinds of information are synthesized to determine safe physical capabilities. These sources include:

Subjective Client Input

Joule evaluators gather information from clients regarding symptoms, perceived exertion, and client perception of safety at predetermined points in the FCE (at start of FCE, during subtests, at subtests' completion, and at FCE termination). The evaluator uses his or her knowledge of the client's diagnosis, functional and medical history, and pattern of recovery to understand the connection between subjective client reports and functional capacities to decide when tests should be terminated. Throughout the FCE, subjective client reports are assessed in light of the particular test activity to evaluate the congruity of the client reports. Subjective client reports are classified into one of three categories based upon clear, objective criteria:

“Meaningful” client reports provide important information that helps the evaluator determine safe levels of maximum function. Meaningful reports are specific and correspond to the client's diagnosis, the activity being assessed, and the client's body mechanics. Meaningful reports are timely, and they directly correspond to objective performance. Joule views meaningful client reports as valid information for determining functional abilities in specific activities and overall tolerances.

“Relevant” client reports provide information of a lower level of importance. The information is generally appropriate to client diagnosis and test performance, but may not be specific in nature. Such reports are not necessarily timely ones; they may be provided by the client during the protocol or at its termination. The reports generally correspond with objective client performance. Relevant client reports are viewed as valid adjunct information to be used subordinately to primary objective criteria.

“Extraneous” client reports provide useless information. Such reports are usually non-specific, and the same report may be offered several times throughout the FCE without reflecting any change in symptoms. The report may or may not correspond to the client's diagnosis or the protocol activity. They are often offered several minutes after the protocol termination, and they have little or no relationship to objective functional performance. They may directly contradict performance or reports in other areas of the FCE. Extraneous client reports are not considered in determining the client's functional capacities.

Some FCEs provide no instruction to clients on safe lifting techniques prior to protocol administration. In spite of possible safety issues, this is done because it is feared that in real work situations, clients will revert to their customary lifting styles, regardless of any instruction received during the FCE. Therefore, to more “realistically” simulate the client's work situation, no instruction is given. Other FCEs provide instruction in proper lifting prior to protocol administration in the interest of safety. In the Joule FCE, the therapist provides no instruction during the first (safest) lift. The client is observed for any deviation from safe lifting technique, and, if any such deviations are observed, the therapist provides corrective instructions. The therapist intervenes at any point in the FCE when the client fails to use proper body mechanics. If the problem is amenable to correction, the protocol continues until safe maximum levels have been established.

Client safe maximum weight levels are established based upon seven generally accepted and clearly defined criteria. Once those levels have been established, determination of recommended work weight frequencies is based upon the recommendations of Astrand and Rodahl (1986) as cited in The Advanced Ergonomics Manual (Selan, 1994), as follows.

Continuous:
15% - 30% of safe maximum lift
Frequent:
30% - 50% of safe maximum lift
Occasional:
50% - 100% of safe maximum lift
Rarely:
Last safe weight lifted at termination of lift

Adjustment factors recommended by Mital, Nicholson, and Ayoub (1993), as cited in The Advanced Ergonomics Manual (Selan, 1994) are applied to recommended weight levels for situations in which ideal body mechanics are not possible. These situations include lifting with limited head room, lifting while twisting, lifting various types of load handles, load clearance limitations, and load asymmetries.

Participation Index

Another important FCE dimension is the assessment of the level of client's sincerity of effort. For various reasons, clients may not put forth a maximal effort during the FCE administration. Obviously, in such cases, FCEs cannot make valid recommendations. Commercial FCEs have adopted several approaches designed to assess the level of client sincerity of effort. Referring to this issue, King, Tuckwell, and Barrett (1998) wrote: “...a reliable and valid method of determining subject participation is vital, but none have been supported by current research.” (p.863). Valpar has developed a novel approach to this important issue that we believe will be shown to have superior reliability and validity.

Joule's participation index assigns clients into one of three categories: “consistent,” “marginally consistent,” or “inconsistent.” FCEs in the “consistent” category are viewed as valid and reliable representations of safe work abilities. “Marginally consistent” FCEs contained occasional inconsistencies. The FCE results represent safe work levels, but some of the client's abilities may actually be higher than demonstrated in the FCE. “Inconsistent” FCEs contain frequent inconsistencies and are not considered valid representations of the client's functional work abilities.

Joule makes a determination of client level of effort in two ways. First, and primarily, for every protocol, subjective indicators (client reports and behaviors) are compared to objective findings of the FCE, client diagnosis, and other information. The consistency category is determined based upon the calculation of the percent of comparisons falling into the “meaningful,” “relevant,” or “extraneous” classifications described above.

In borderline cases, the client's reported functional abilities, as recorded on the functional history questionnaire, are compared to objective findings, and those comparisons are also entered into the calculation.

The Survey

In January of 1997, Ms. Ruprecht began conducting FCEs with Joule at the Two Rivers Center. By March of 1998, she had conducted 68 FCEs on clients with work-related injuries. Those individuals were invited to respond to a survey (see Appendix A) which solicited their responses to a range of questions concerning their satisfaction with their FCE and certain post-FCE experiences. They were also asked for their permission to gather relevant information from their client files at the Two Rivers Center. Eighteen Joule FCE recipients completed the survey. (14 males and 4 females. Mean age of respondents: 45.3; standard deviation: 9.7. The mean time in months between client injury and the survey response was 38.6 with a standard deviation of 48.3 months; range= 267 months.) Despite the low response rate (about 26.5%), the results were encouraging and have served to guide the planning of continuing Joule client follow-up studies.

The results of the survey are summarized in the following discussion.

The satisfaction questions of the survey revealed a high level of satisfaction with Joule among the 18 respondents. The satisfaction items were:

A. Explanation of the purpose of the FCE
B. Explanation of the results of the FCE
C. Accuracy of the FCE results as compared to your physical capabilities
D. Respectfulness shown by the therapist during the FCE
E. Overall satisfaction with the FCE
F. Assistance provided by your physician(s)
G. Assistance provided by your therapist(s)
H. Contacts with your insurance contact
I. Contacts with your employer at the time of your injury or medical difficulties
J. Assistance provided by your case manager or rehabilitation counselor
K. Your current work status
L. Your job before your injury

All items were rated on a Likert scale; 1= very satisfied, 5=very dissatisfied.

The results were as follows.


N

MeanStandard Deviation
A.18

1.40.7
B.18

1.61.0
C.18

1.91.0
D.18

1.10.3
E.18

1.71.1
F.14

1.91.1
G.17

1.61.1
H.15

2.71.6
I.16

2.51.5
J.15

2.01.4
K.16

2.61.5
L.18

1.30.6

Irrespective of all other considerations, the satisfaction items relevant to Joule itself showed a high level of client satisfaction. The implications of this are extremely important in light of the inherent potential for litigation in industrial injury cases.

Survey question 15 asked whether the respondents had an attorney working with them on their injury. Ten of the 18 did not; 8 did. The only significant relationship between that status and satisfaction responses had to do with satisfaction with their current job status (item K). The point-biserial correlation coefficient was used to assess relationships between the satisfaction responses and various item responses. (The t-test was used to calculate the probability of the correlation occurring by chance.) Those with attorneys were less satisfied with their current job status than those without attorneys (r= -.51; t= -2.22; p< .05). It is well-known that correlational relationships do not imply causation, and, obviously, satisfied people are less likely to seek legal representation than unsatisfied ones. This finding, therefore, would seem to lend credence to the survey as a research tool.

Four persons did not follow the FCE recommendations: 1 is working and in litigation, 1 is receiving continued medical treatment, 1 has retired on permanent disability pension, and 1 is not working and in litigation. Taken together, the satisfaction responses of these individuals varied from those who did follow the FCE recommendations on two of the items: C (accuracy of the results of the FCE) and F (assistance provided by their physicians): C: r= -76; t= -4.05; p< .002; F: r= -.43; t= -3.00; p< .02).

Two of the four who did not follow the FCE recommendations were also the only ones to report having been re-injured since returning to work. These individuals were also the only persons to make suggestions for improving the FCE. Their satisfaction responses varied from the rest of the group on two items: C (accuracy of the results of the FCE) and E (overall satisfaction with the FCE). They were much less satisfied: C: r= -.77; t= -4.34; p< .001; E: r= -.86; t= -6.08; p< .001.

The one individual who was assigned a less than “consistent” rating on the FCE, mentioned above, was one of the four who did not follow the FCE recommendations. That person was one of the two persons to report a re-injury, and was one of the two persons to make recommendations for the improvement of the FCE. That person's satisfaction responses were as follows.

A. 3
B. 5
C. 5
D. 2
E. 5
F. 4
G. 5
H. 5
I. none
J. none
K. none
L. 1

It is apparent that this individual was much less satisfied on most of the items than the group as a whole. Again, the point- biserial correlation coefficient was used to estimate the probability of such relationships occurring by chance:



r
t
p
A.
-.71
-3.02
<.02
B.
-.89
-5.86
<.001
C.
-.90
-5.92
<.001
D.
-.63
-2.43
<.05
E.
-.93
-7.59
<.001
F.
-.78
-2.74
<.05
G.
-.85
-4.56
<.01
H.
-.44
-1.30
<.20
L.
+.22
+.68
>.80

Note that the one response of that individual indicating a higher level of satisfaction (although slight) than the mean response concerned the pre-injury job. In synthesizing these results two general interpretations suggest themselves. First, while not wishing to put a cynical interpretation on the findings, it is nevertheless tempting to wonder if the fact that the individual with the less than “consistent” rating is now litigating to win a higher compensation has something to do with the responses to the satisfaction items, including item L; for if it can be shown in court that the FCE was unsatisfactory, and that while things were totally satisfactory prior to the injury, quite the opposite state of affairs now prevails, might not a judge or jury be inclined to be more sympathetic and generous?

Of course, it is possible that the individual concerned was simply misjudged by the therapist, and that a “consistent” rating was deserved. It is possible that the person is displeased with the FCE because of a sincere feeling that the FCE results were inaccurate. Hence, the fact that the person contributed suggestions for improving the FCE should not be remarkable. Also, if the individual does not agree with the FCE recommendations, why should those recommendations be followed? If a re-injury has occurred (as reported), even though the FCE recommendations were not followed, it could hardly be expected that the individual would be satisfied. Nevertheless, either way one chooses to interpret the various associations between the person's survey responses and status — with cynicism or acceptance — the responses are mutually consistent. This seems to lend credence to the survey as a research tool, and points to several interesting questions that will be explored in the future:

What is the relationship between various dimensions of satisfaction with the FCE and following FCE recommendations?

Does having legal representation predict satisfaction with the FCE?

Does not following FCE recommendations predict re-injury?

Does the FCE consistency rating predict satisfaction and/or the likelihood of following the FCE recommendations?

Is there are relationship between having legal representation and consistency rating?

These and other issues will be explored in planned Joule research. Inter-rater reliability studies are planned for the fall of 1998 and winter of 1999 at the Nova Care Airport Medical Clinic in Bloomington, MN and Rehabilitation Services, Inc. clinic in Arlington Hts., IL.

Several studies are planned over the same time frame at Knox Community Hospital in Mt. Vernon, OH. First, normative data on 30 non-injured volunteers will be gathered on various exercise protocols for work rate and errors. Second, a follow-up study will be conducted on 40 injured clients at three and six months post-FCE. The study will survey clients on a variety of issues including their satisfaction levels. And their responses will be compared to their chart information and their subsequent work- and injury-related experiences to establish various elements of Joule's utility and predictive validity. Finally, a test-retest study will be conducted on approximately 25 clients (some of whom will have also participated in the follow-up study) who will complete pre- and post-work hardening FCEs. That study will also assess the utility of Joule in guiding client treatment plans.

Summary and Conclusion

Joule is a new FCE system by Valpar. It is the culmination of over four years of intensive research and development effort and was designed specifically to offer unique solutions to problems other FCE systems do not address satisfactorily.

This paper has introduced Joule, describing its development and touching upon a few of Joule's many unique elements. The results of a preliminary follow-up study were presented, and the plans for further research were discussed. For those interested in more information about Joule, please contact Valpar.

Overall, the preliminary Joule follow-up study showed not only a high level of client satisfaction with the Joule FCE, but also a high level of client cooperation with Joule's recommendations, and a high level of improved work status subsequent to the FCE. A correspondingly low level of re-injury was reported, and by the only two of the respondees who returned to work without following the FCE recommendations. The findings, while preliminary, and based on a small sample, are, nevertheless quite encouraging. Joule has demonstrated a high degree of validity in terms of its ability to help injured persons return to work or increase their work demands without any re-injury.

Appendices

Articles Page/ Joule Home Page/ Home

Copyright © 1999 - Valpar International Corporation; All Rights Reserved.