Summary
The evaluation of computer expert systems, a promising diagnostic tool for future application in clinical medicine, is of great importance. We present here the evaluation of our expert system, “RHEUMA”. It is stressed, that repeated retrospective testing and updating of an expert system and its subsequent repeated assessment in clinical use and surroundings is mandatory. This increases the diagnostic accuracy of the system. For our system this is demonstrated under three separate conditions. In the first study the information available for the computer system (mainframe) came from medical histories only. Here an error rate of about 25% — similar to that of physicians themselves using the same information — was observed in 358 outpatients, compared to the final diagnoses of physicians also relying solely on information from medical histories. In a second step a completely new system on a personal computer was developed with all relevant diagnostic information. The error rate of this system (0.4%) was much too optimistic because the knowledge base was changed during the study, affecting about 30% of the 282 prospectively recruited outpatients. In a third step the efficacy of the expert system was tested in an additional hospital without the diagnostic involvement of the first testing clinic. The error rate of the system without changing the knowledge base reached 11% in 51 outpatients in this rheumatology clinic. This result reflects the diagnostic accuracy of the system today. Its ability to specify the same diagnoses which clinical experts reached approached 90%. Considerable time is needed for such prospective testing, with repeated updating of the knowledge base — in our case for both the two systems and field studies of 2 years each. Further prospective field testing with physicians not specialized in rheumatology and with a larger number of patients is necessary before the system can be used in clinical routine.
Similar content being viewed by others
References
Bernelot Moens HJ, van der Korst JK (1991) Comparison of rheumatological diagnoses by a Bayesian program and by physicians. Methods Inf Med 30:187–193
Herzer P, Schattenkirchner M (1986) Schmerzen im Bereich des Bewegungsapparates. In: Zöllner N, Hadorn W (eds) Vom Symptom zur Diagnose. Karger, München, pp 100–121
Hilden J, Habbema JDF (1990) Evaluation of clinical decision aids — more to think about. Med Inf 15:275–284
Kolarz G, Adlassnig KP (1986) Problems in establishing the medical expert systems CADIAG-1 and CADIAG-2 in rheumatology. J Med Syst 10:395–405
McCarty DJ (ed) (1991) Current opinion in rheumatology, vol 3. Current Science, Philadelphia
Pauker SG, Gorry GA, Kassirer JP, Schwartz JB (1976) Towards the simulation of clinical cognition: taking a present illness by computer. Am J Med 60:981–996
Porter JF, Kingsland LC, Lindberg DAB, Shah I, Benge JM, Hazelwood SE, Kay DR, Homma M, Akizuki M, Takano M, Sharp GC (1988) The AI/RHEUM knowledge based computer consultant system in rheumatology. Performance in the diagnosis of 59 consecutive tissue disease patients from Japan. Arthritis Rheum 31:219–226
Puppe F (1987) Requirements for a classification expert system shell and their realization in MED1. Applied Artificial Intelligence 1:163–171
Puppe F, Gappa U (1992) Towards knowledge acquisition by experts. In: Belli F, Radermacher FJ (eds) Fifth international conference of industrial and engeneering applications of artificial intelligence and expert systems. Lecture Notes in Artificial Intelligence 604. Springer, Berlin Heidelberg New York, pp 546–555
Sandler G (1979) Costs of unnecessary tests. BMJ 2:21–24
Schewe S, Herzer P, Krüger K (1990) Prospective application of an expert system for the medical history of joint pain. Klin Wochenschr 68:466–471
Schewe S, Müller-Nordhorn J, Mitterwald S, SchreiberMA (1991) Patient's and physician's opinion about computer expert systems. In: Stefanelli M, Hasman A, Fieschi M, Talmon J (eds) Lecture Notes in Medical Informatics 44, AIME91. Springer, Berlin Heidelberg New York, pp 297–305
Schewe S, Heinrich P, Stadler B (1992) Development and clinical trial of the expert system “Rheuma” based on a personal computer. In: Chytil MK, Duru G, von Eimeren W, Flagle CD (eds) Fifth international conference on system science in health care. Omnipress, Prague, pp 1301–1305
Wyatt J, Spiegelhalter D (1990) Evaluating medical expert systems: what to test and how? Med Inform 15:205–217
Author information
Authors and Affiliations
Additional information
Dedicated to Prof. Dr. N. Zöllner on the occasion of his 70th birthday
Rights and permissions
About this article
Cite this article
Schewe, S., Schreiber, M.A. Stepwise development of a clinical expert system in rheumatology. Clin Investig 71, 139–144 (1993). https://doi.org/10.1007/BF00179995
Received:
Revised:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF00179995