Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.
ADVANCED SEARCH
Log in
Top
Sports Medicine
Published in: Sports Medicine 4/2014

01-04-2014 | Systematic Review

Tests Examining Skill Outcomes in Sport: A Systematic Review of Measurement Properties and Feasibility

Authors: Samuel J. Robertson, Angus F. Burnett, Jodie Cochrane

Published in: Sports Medicine | Issue 4/2014

Login to get access

Abstract

Background

A high level of participant skill is influential in determining the outcome of many sports. Thus, tests assessing skill outcomes in sport are commonly used by coaches and researchers to estimate an athlete’s ability level, to evaluate the effectiveness of interventions or for the purpose of talent identification.

Objective

The objective of this systematic review was to examine the methodological quality, measurement properties and feasibility characteristics of sporting skill outcome tests reported in the peer-reviewed literature.

Data Sources

A search of both SPORTDiscus and MEDLINE databases was undertaken.

Study Selection

Studies that examined tests of sporting skill outcomes were reviewed. Only studies that investigated measurement properties of the test (reliability or validity) were included. A total of 22 studies met the inclusion/exclusion criteria.

Study Appraisal and Synthesis Methods

A customised checklist of assessment criteria, based on previous research, was utilised for the purpose of this review.

Results

A range of sports were the subject of the 22 studies included in this review, with considerations relating to methodological quality being generally well addressed by authors. A range of methods and statistical procedures were used by researchers to determine the measurement properties of their skill outcome tests. The majority (95 %) of the reviewed studies investigated test–retest reliability, and where relevant, inter and intra-rater reliability was also determined. Content validity was examined in 68 % of the studies, with most tests investigating multiple skill domains relevant to the sport. Only 18 % of studies assessed all three reviewed forms of validity (content, construct and criterion), with just 14 % investigating the predictive validity of the test. Test responsiveness was reported in only 9 % of studies, whilst feasibility received varying levels of attention.

Limitations

In organised sport, further tests may exist which have not been investigated in this review. This could be due to such tests firstly not being published in the peer-review literature and secondly, not having their measurement properties (i.e., reliability or validity) examined formally.

Conclusions

Of the 22 studies included in this review, items relating to test methodological quality were, on the whole, well addressed. Test–retest reliability was determined in all but one of the reviewed studies, whilst most studies investigated at least two aspects of validity (i.e., content, construct or criterion-related validity). Few studies examined predictive validity or responsiveness. While feasibility was addressed in over half of the studies, practicality and test limitations were rarely addressed. Consideration of study quality, measurement properties and feasibility components assessed in this review can assist future researchers when developing or modifying tests of sporting skill outcomes.
Literature
1.
Currell K, Jeukendrup AE. Validity, reliability and sensitivity of measures of sporting performance. Sports Med. 2008;38(4):297–316.PubMedCrossRef
2.
Cale AA. The investigation and analysis of softball skill tests for college women [dissertation]. Maryland: University of Maryland; 1962.
3.
Cobb JW. The determination of the merits of selected items for the construction of a baseball skill test for boys of Little League age [dissertation]. Indiana: Indiana University; 1958.
4.
Collins DR, Hodges PB. A comprehensive guide to sports skills tests and measurement. Maryland: Rowman & Littlefield Education; 2001.
5.
Fein JT. Construction of Skill Tests for Beginning Collegiate Women Fencers [dissertation]. Iowa: University of Iowa; 1964.
6.
Safrit MJ. Construction of skill tests for beginning fencers [dissertation]. Madison: University of Wisconsin; 1962.
7.
Sopa AM. The construction of an indoor batting skills test for junior high school girls [dissertation]. Madison: University of Wisconsin; 1967.
8.
Robertson SR, Burnett AF, Newton RU. Development and validation of the approach-iron skill test for use in golf. Eur J Sport Sci. Epub 2013 Jan 10.
9.
Sunderland C, Cooke K, Milne H, et al. The reliability and validity of a field hockey skill test. Int J Sports Med. 2006;27(5):395–400.PubMedCrossRef
10.
Russell M, Kingsley M. Influence of exercise on skill proficiency in soccer. Sports Med. 2011;41(7):523–39.PubMedCrossRef
11.
Duncan MJ, Taylor S, Lyons M. The effect of caffeine ingestion on field hockey skill performance following physical fatigue. Res Sports Med. 2012;20(1):25–36.PubMed
12.
Currell K, Conway S, Jeukendrup AE. Carbohydrate ingestion improves performance of a new reliable test of soccer performance. Int J Sport Nutr Exerc Metab. 2009;19(1):34–46.PubMed
13.
Bottoms LM, Hunter AM, Galloway SD. Effects of carbohydrate ingestion on skill maintenance in squash players. Eur J Sport Sci. 2006;6(3):187–95.CrossRef
14.
Russell M, Benton D, Kingsley M. Influence of carbohydrate supplementation on skill performance during a soccer match simulation. J Sci Med Sport. 2012;15(4):348–54.PubMedCrossRef
15.
Russell M, Benton D, Kingsley M. The effects of fatigue on soccer skills performed during a soccer match simulation. Int J Sports Physiol Perform. 2011;6(2):221–33.PubMed
16.
McKay B, Wulf G. A distal external focus enhances novice dart throwing performance. Int J Sport Exerc Psychol. 2012;10(2):149–56.CrossRef
17.
McCann P, Lavallee D, Lavallee R. The effect of pre-shot routines on golf wedge shot performance. Eur J Sport Sci. 2001;1(5):1–10.CrossRef
18.
Russell M, Rees G, Benton D, et al. An exercise protocol that replicates soccer matchplay. Int J Sports Med. 2011;32(7):511–8.PubMedCrossRef
19.
Nicholas CW, Nuttall FE, Williams C. The Loughborough Intermittent Shuttle Test: a field test that simulates the activity pattern of soccer. J Sports Sci. 2000;18(2):97–104.PubMedCrossRef
20.
Williams JD, Abt G, Kilding AE. Ball sport endurance and sprint test (BEAST90): validity and reliability of a 90-minute soccer performance test. J Strength Cond Res. 2010;24(12):3209–18.PubMedCrossRef
21.
Robertson SJ, Burnett AF, Newton RU, et al. Development of the Nine-Ball Skills Test to discriminate elite and high-level amateur golfers. J Sports Sci. 2012;30(5):431–7.PubMedCrossRef
22.
Gabbett TJ, Georgieff B. The development of a standardized skill assessment for junior volleyball players. Int J Sports Physiol Perform. 2006;1(2):95–107.PubMed
23.
Pyke F. Introduction. In: Gore C, editor. Physiological tests for elite athletes. Champaign, Illinois: Human Kinetics; 2000. p. xii–xiv.
24.
Russell M, Benton D, Kingsley M. Reliability and construct validity of soccer skills tests that measure passing, shooting, and dribbling. J Sports Sci. 2010;28(13):1399–408.PubMedCrossRef
25.
Gabbett T, Wake M, Abernethy B. Use of dual task methodology for skill assessment and development: examples from rugby league. J Sports Sci. 2011;29(1):7–18.PubMedCrossRef
26.
Brent S, Draper N, Hodgson C, et al. Development of a performance assessment tool for rock climbers. Eur J Sport Sci. 2009;9(3):159–67.CrossRef
27.
28.
Lidor R, CôTé JE, Hackfort D. ISSP position stand: to test or not to test? The use of physical skill tests in talent detection and in early phases of sport development. Int J Sport Exerc Psychol. 2009;7(2):131–46.CrossRef
29.
Ali A, Williams C, Hulse M, et al. Reliability and validity of two tests of soccer skill. J Sports Sci. 2007;25(13):1461–70.PubMedCrossRef
30.
Kingsley M, Russell M, Benton D. Authors’ response to letter to the editor: “The need for ‘representative task design’ in evaluating efficacy of skills tests in sport: a comment on Russell, Benton and Kingsley (2010)”. J Sports Sci. 2012;30(16):1731–3.CrossRef
31.
Vilar L, Araújo D, Davids K, et al. The need for ‘representative task design’ in evaluating efficacy of skills tests in sport: a comment on Russell, Benton and Kingsley (2010). J Sports Sci. 2012;30(16):1727–30.PubMedCrossRef
32.
Philips E, Davids K, Renshaw I, Portus M. Expert performance in sport and the dynamics of talent development. Sports Med. 2010;40(4):271–83.CrossRef
33.
Vaeyens R, Lenoir M, Williams MA, et al. Talent identification and development programmes in sport: current models and future directions. Sports Med. 2008;38(9):703–14.PubMedCrossRef
34.
McNamara A, Collins D. Comment on “Talent identification and promotion programmes of Olympic athletes”. J Sports Sci. 2011;29(12):1353–6.CrossRef
35.
Pinder RA, Renshaw I, Davids K. The role of representative design in talent development: a comment on “Talent identification and promotion programmes of Olympic athletes”. J Sports Sci. 2013;31(8):803–6.PubMedCrossRef
36.
Glazier P. Movement variability in the golf swing: theoretical, methodological and practical issues. Res Q Exerc Sport. 2011;82(2):157–61.PubMed
37.
Fleisig G, Chu Y, Weber A, et al. Variability in baseball pitching biomechanics among various levels of competition. Sports Biomech. 2009;8(1):10–21.PubMedCrossRef
38.
Langdown BL, Bridge M, Li F. Movement variability in the golf swing. Sports Biomech. 2012;11(2):273–87.PubMedCrossRef
39.
Betzler NF, Monk SA, Wallace ES, et al. Variability in clubhead presentation characteristics and ball impact location for golfers’ drives. J Sports Sci. 2012;30(5):439–48.PubMedCrossRef
40.
Bartlett R. Movement variability and its implication for sports scientists and practitioners: an overview. Int J Sports Sci Coach. 2008;3(1):113–24.CrossRef
41.
Ovens A, Smith W. Skill: making sense of a complex concept. J Phys Educ N Z. 2006;39(1):72–84.
42.
Smith W. Skill acquisition in physical education: a speculative perspective. Quest. 2011;63(3):265–74.CrossRef
43.
Moher D, Liberati A, Tetzlaff J, et al. Reprint-preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys Ther. 2009;89(9):873–80.PubMed
44.
Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49.PubMedCentralPubMedCrossRef
45.
Auld ML, Boyd RN, Moseley GL, et al. Tactile assessment in children with cerebral palsy: a clinimetric review. Phys Occup Ther Pediatr. 2011;31(4):413–39.PubMedCrossRef
46.
Fliess-Douer O, Vanlandewijck YC, Manor GL, et al. A systematic review of wheelchair skills tests for manual wheelchair users with a spinal cord injury: towards a standardized outcome measure. Clin Rehabil. 2010;24(10):867–86.PubMedCrossRef
47.
Helmerhorst HJ, Brage S, Warren J, et al. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act. 2012;9(1):103–57.PubMedCentralPubMedCrossRef
48.
Williams MA, McCarthy CJ, Chorti A, et al. A systematic review of reliability and validity studies of methods for measuring active and passive cervical range of motion. J Manip Physiol Ther. 2010;33(2):138–55.CrossRef
49.
Packham T, MacDermid JC, Henry J, et al. A systematic review of psychometric evaluations of outcome assessments for complex regional pain syndrome. Disabil Rehabil. 2012;34(13):1059–69.PubMedCrossRef
50.
Dobson F, Morris ME, Baker R, et al. Gait classification in children with cerebral palsy: a systematic review. Gait Posture. 2007;25(1):140–52.PubMedCrossRef
51.
Howe TE, Dawson LJ, Syme G, et al. Evaluation of outcome measures for use in clinical practice for adults with musculoskeletal conditions of the knee: a systematic review. Man Ther. 2012;17(4):100–18.PubMedCrossRef
52.
Mirkov D, Nedeljkovic A, Kukolj M, et al. Evaluation of the reliability of soccer-specific field tests. J Strength Cond Res. 2008;22(4):1046–50.PubMedCrossRef
53.
Knudson DV, Morrison CS. Qualitative analysis of human movement. 2nd ed. Illinois: Human Kinetics; 2002.
54.
Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 3rd ed. Oxford: Oxford University Press; 2005.
55.
Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 3rd ed. New Jersey: Pearson/Prentice Hall; 2008.
56.
Westen D, Rosenthal R. Quantifying construct validity: two simple measures. J Pers Soc Psychol. 2003;84(3):608–18.PubMedCrossRef
57.
Squires A. A valid step in the process: a commentary on Beckstead. Int J Nurs Stud. 2009;46(9):1284–5.PubMedCrossRef
58.
Braeken AP, Kempen GI, Eekers D, et al. The usefulness and feasibility of a screening instrument to identify psychosocial problems in patients receiving curative radiotherapy: a process evaluation. BMC Cancer. 2011;11(1):479–90.PubMedCentralPubMedCrossRef
59.
Gabel CP, Melloh M, Burkett B, et al. Lower limb functional index: development and clinimetric propertie. Phys Ther. 2012;92(1):98–110.PubMedCrossRef
60.
Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):10.PubMedCentralPubMedCrossRef
61.
Gardner MJ, Machin D, Campbell MJ, et al. Statistical checklists. In: Altman DG, Machin D, Bryant TN et al., editors. Statistics with confidence: confidence intervals and statistical guidelines. 2nd ed. Bristol: BMJ Books; 2000. p. 191–201.
62.
Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193–205.CrossRef
63.
Hill BE, Williams G, Bialocerkowski AE. Clinimetric evaluation of questionnaires used to assess activity after traumatic brachial plexus injury in adults: a systematic review. Arch Phys Med Rehabil. 2011;92(12):2082–9.PubMedCrossRef
64.
Solway S, Brooks D, Lacasse Y, et al. A qualitative systematic overview of the measurement properties of functional walk tests used in the cardiorespiratory domain. Chest. 2001;119(1):256–70.PubMedCrossRef
65.
Lohr KN, Aaronson NK, Alonso J, et al. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clin Ther. 1996;18(5):979–92.PubMedCrossRef
66.
Campion MA. Article review checklist: a criterion checklist for reviewing research articles in applied psychology. Pers Psychol. 2006;46(3):705–18.CrossRef
67.
Valderas JM, Ferrer M, Mendivil J, et al. Development of EMPRO: a tool for the standardized assessment of patient-reported outcome measures. Value Health. 2008;11(4):700–8.PubMedCrossRef
68.
69.
Beck TW. The importance of a priori sample size estimation in strength and conditioning research. J Strength Cond Res. 2013;27(8):2323–37.PubMedCrossRef
70.
Nevill AM, Holder RL, Cooper SM. Statistics, truth and error reduction in sport and exercise sciences. Eur J Sport Sci. 2007;7(1):9–14.CrossRef
71.
Cohen J. A. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37–46.CrossRef
72.
Haynes SN, Richard DC, Kubany ES. Content validity in psychological assessment: a functional approach to concepts and methods. Psychol Assess. 1995;7(3):238–47.CrossRef
73.
74.
Stinson JN, Kavanagh T, Yamada J, et al. Systematic review of the psychometric properties, interpretability of self-report pain measures for use in clinical trials in children and adolescents. Pain. 2006;125(1 & 2):143–57.PubMedCrossRef
75.
Ravens-Sieberer U, Auquier P, Erhart M, et al. The KIDSCREEN-27 quality of life measure for children and adolescents: psychometric results from a cross-cultural survey in 13 European countries. Qual Life Res. 2007;16(8):1347–56.PubMedCrossRef
76.
Munro BH. Statistical methods for health care research. 5th ed. Philadelphia (USA): Lippincott, Williams & Wilkins: 1999.
77.
Van Saane N, Sluiter JK, Verbeek JH, et al. Reliability and validity of instruments assessing job satisfaction—a systematic review. Occup Med. 2003;53(3):191–200.CrossRef
78.
Beckerman H, Roebroeck ME, Lankhorst GJ, et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10(7):571–8.PubMedCrossRef
79.
Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chron Dis. 1987;40(2):171–8.PubMedCrossRef
80.
81.
Crossley KM, Bennell KL, Cowan SM, et al. Analysis of outcome measures for persons with patellofemoral pain: which are reliable and valid? Arch Phys Med Rehabil. 2004;85(5):815–22.PubMedCrossRef
82.
Yilla AB, Sherrill C. Validating the Beck battery of quad rugby skill tests. Adapt Phys Act Q. 1996;15(2):155–67.
83.
Porter JM, Landin D, Hebert EP, et al. The effects of three levels of contextual interference on performance outcomes and movement patterns in golf skills. Int J Sports Sci Coach. 2007;2(3):243–55.CrossRef
84.
Lam ET, Zhang JJ. The development and validation of a racquetball skills test battery for young adult beginners. Meas Phys Educ Exerc Sci. 2002;6(2):95–126.CrossRef
85.
Bock-Jonathon BB, Venter RE, Bressan ES. A comparison between skill and decision-making ability of netball players at club level: pilot study. S Afr J Res Sport Phys Educ Rec. 2007;29(1):29–38.
86.
Downs SB, Wood TW. Validating a special olympics volleyball skills assessment test. Adapt Phys Act Q. 1996;13(2):166–79.
87.
De Groot S, Balvers IJ, Kouwenhoven SM, et al. Validity and reliability of tests determining performance-related components of wheelchair basketball. J Sports Sci. 2012;30(9):879–87.PubMedCrossRef
88.
Ali A, Foskett A, Gant N. Validation of a soccer skill test for use with females. Int J Sports Med. 2008;29(11):917–21.PubMedCrossRef
89.
Royal KA, Farrow D, Mujika I, et al. The effects of fatigue on decision making and shooting skill performance in water polo players. J Sports Sci. 2006;24(8):807–15.PubMedCrossRef
90.
Vergauwen L, Spaepen AJ, Lefevre J, et al. Evaluation of stroke performance in tennis. Med Sci Sports Exerc. 1998;30(8):1281–8.PubMedCrossRef
91.
Lemmink KA, Elferink-Gemser MT, Visscher C. Evaluation of the reliability of two field hockey specific sprint and dribble tests in young field hockey players. Br J Sports Med. 2004;38(2):138–42.PubMedCentralPubMedCrossRef
92.
Bartlett J, Smith L, Davis K, et al. Development of a valid volleyball skills test battery. J Phys Educ Rec Dance. 1991;62(2):19–21.CrossRef
93.
Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med. 1998;26(4):217–38.PubMedCrossRef
94.
95.
Beaton DE, Wright JG, Katz JN. Development of the QuickDASH: comparison of three item-reduction approaches. J Bone Joint Surg Am. 2005;87(5):1038–46.PubMedCrossRef
96.
Kilkens OJ, Post MW, Dallmeijer AJ, et al. Wheelchair skill tests: a systematic review. Clin Rehab. 2003;17(4):418–30.CrossRef
97.
Faber MJ, Bosscher RJ, van Wieringen PC. Clinimetric properties of the performance-orientated mobility assessment. Phys Ther. 2006;86(7):944–54.PubMed
98.
Stevens B, Gibbins S. Clinical utility and clinical significance in the assessment and management of pain in vulnerable infants. Clin Perinatol. 2002;29(3):459–68.PubMedCrossRef
99.
Robinson JM, Cook JL, Purdam C, et al. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med. 2001;35(5):335–41.PubMedCentralPubMedCrossRef
100.
Baumgartner TA, Jackson AS. Measurement for evaluation in physical education and exercise science. 5th ed. Michigan: Brown & Benchmark; 1998.
101.
Law MC, MacDermid J. Evidence-based rehabilitation: A guide to practice, Slack Incorporated, 2007 Outcome measures rating form. CanChild Centre for Childhood Disability Research. 2nd ed. Thorofare (NJ); 2004. p. 367.
102.
de Boer MR, Moll AC, de Vet HC, et al. Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004;24(4):257–73.PubMedCrossRef
103.
Landis J, Koch G. The measurement of observer agreement of categorical data. Biometrics. 1977;33(1):159–74.PubMedCrossRef
104.
Pietrobon R, Coeytaux RR, Carey TS, et al. Standard scales for measurement of functional outcome for cervical pain or dysfunction: a systematic review. Spine. 2002;27(5):515–22.PubMedCrossRef
105.
Copay AG, Subach BR, Glassman SD, et al. Understanding the minimum clinically important difference: a review of concepts and methods. Spine. 2007;7(5):541–6.CrossRef
106.
Streiner DL. A checklist for evaluating the usefulness of rating scales. Can J Psychiatry. 1993;38(2):140–8.PubMed
Metadata
Title
Tests Examining Skill Outcomes in Sport: A Systematic Review of Measurement Properties and Feasibility
Authors
Samuel J. Robertson
Angus F. Burnett
Jodie Cochrane
Publication date
01-04-2014
Publisher
Springer International Publishing
Published in
Sports Medicine / Issue 4/2014
Print ISSN: 0112-1642
Electronic ISSN: 1179-2035
DOI
https://doi.org/10.1007/s40279-013-0131-0

Other articles of this Issue 4/2014

Sports Medicine 4/2014 Go to the issue

Systematic Review

Biomechanical Factors Influencing the Performance of Elite Alpine Ski Racers

Current Opinion

What is a Sports Injury?

Review Article

Physiological Mechanisms of Post-Exercise Balance Impairment

Review Article

Assessment, Management and Knowledge of Sport-Related Concussion: Systematic Review

Systematic Review

Effects of Protein in Combination with Carbohydrate Supplements on Acute or Repeat Endurance Exercise Performance: A Systematic Review

Current Opinion

Physical Exercise and Epigenetic Modulation: Elucidating Intricate Mechanisms