Abstract
The paper examines in detail one particular measure of variable importance for linear regression that was theoretically justified by Pratt (1987), but which has since been criticized by Bring (1996) for producing “counterintuitive” results in certain situations, and by other authors for failing to guarantee that importance be non-negative. In the article, the “counterintuitive” result is explored and shown to be a defensible characteristic of an importance measure. It is also shown that negative importance of large magnitude can only occur in the presence of multicollinearity of the explanatory variables, and methods for applying Pratt's measure in such cases are described. The objective of the article is to explain and to clarify the characteristics of Pratt's measure, and thus to assist practitioners who have to choose from among the many methods available for assessing variable importance in linear regression.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
REFERENCES
Bring, J.: 1994, ‘How to standardize regression coefficients’, The American Statistician 48, pp. 209–213.
Bring, J.: 1996, ‘A geometric approach to compare variables in a regression model’, The American Statistician 50, pp. 57–62.
Budescu, D. V.: 1993, ‘Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression’, Psychological Bulletin 114, pp. 542–551.
Green, P. E., J. D. Carroll and W. S. DeSarbo: 1978, ‘A new measure of predictor variable importance in multiple regression’, Journal of Marketing Research 15, pp. 356–360.
Healey, M. J. R.: 1990, ‘Measuring importance’, Statistics in Medicine 9, pp. 633–637.
Hocking, R. R., F. M. Speed and M. J. Lynn: 1976, ‘A class of biased estimators in linear regression’, Technometrics 18, pp. 425–437.
Hoerl, A. E. and R. W. Kennard: 1970, ‘Ridge regression: Biased estimation for nonorthogonal problems’, Technometrics 12, pp. 55–67.
Huberty, C. J.: 1994, Applied Discriminant Analysis (Wiley, New York).
Kruskall, W.: 1987, ‘Relative importance by averaging over orderings’, The American Statistician 41, pp. 6–10.
Kruskall, W. and R. Majors: 1989, ‘Concepts of relative importance in recent scientific literature’, The American Statistician 43, pp. 2–6.
Kumar, V. and U. Kumar: 1992, ‘Technological innovation in Canadian manufacturing industry: An investigation of the speed and cost of innovation’, Report for Industry Canada (Carleton University, Ottawa).
Montgomery, D. C. and E. A. Peck: 1982, Introduction to Linear Regression Analysis (Wiley, New York).
Morrison, D. F.: 1983, Applied Linear Statistical Methods (Prentice-Hall, Englewood Cliffs, NJ).
Pratt, J. W.: 1987, ‘Dividing the indivisible: Using simple symmetry to partition variance explained’, in T. Pukkila and S. Puntanen (eds.), Proceedings of the Second International Conference in Statistics (University of Tampere, Tampere, Finland) pp. 245–260.
Saville, D. J. and G. R. Wood: 1986, ‘A method of teaching statistics using N-dimensional geometry’, The American Statistician 40: 205–214.
Stevens, J.: 1992, Applied Multivariate Statistics for the Social Sciences (2nd edn.) (Lawrence Erlbaum Associates, Hillsdale NJ).
Thomas, D. R.: 1992, ‘Interpreting discriminant functions: A data analytic approach’, Multivariate Behavioural Research 27, pp. 335–362.
Thomas, D. R.: 1997, ‘A note on Huberty and Wisenbaker's “Views of variable importance”’, Journal of Educational and Behavioral Statistics 22, pp. 309–322.
Thomas, D. R., E. Hughes and B. D. Zumbo: 1996, ‘Variable importance in regression and related analyses’, Working Paper, WPS 96–01, School of Business, Carleton University, Ottawa, Canada.
Thomas, D. R. and B. D. Zumbo: 1996, ‘Using a measure of variable importance to investigate the standardization of discriminant coefficients’, Journal of Educational and Behavioural Statistics 21, pp. 110–130.
Thomas, D. R. and B. D. Zumbo: 1997, ‘Variable importance in logistic regression based on partitioning an R 2 measure’, Working Paper, WPS 97–01, School of Business, Carleton University, Ottawa, Canada.
Wetherill, G. G.: 1986, Regression Analysis with Applications (Chapman and Hall, London).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Thomas, D.R., Hughes, E. & Zumbo, B.D. On Variable Importance in Linear Regression. Social Indicators Research 45, 253–275 (1998). https://doi.org/10.1023/A:1006954016433
Issue Date:
DOI: https://doi.org/10.1023/A:1006954016433