Review or meta-analysis
Patricia Heyn, Beatriz C Abreu, Kenneth J OttenbacherThe Effects of Exercise Training on Elderly Persons With Cognitive Impairment and Dementia: A Meta-Analysis
Archives of Physical Medicine and Rehabilitation
2004 Oct, Vol.85(10):1694-1704
- Submitted by:Stefan Tino Kulnik - Physiotherapist
- Institution:Imperial College Healthcare NHS Trust
- Date submitted:25th November 2009
1
Objectives and hypotheses
1.1
Are the objectives of the study clearly stated?
Yes. To determine by meta-analysis whether physical exercises are beneficial for people with dementia and related cognitive impairments.
2
Design
2.1
Is the study design suitable for the objectives?
Yes. A sufficient number of studies has been published to warrant a systematic review and meta-analysis.
2.2
Were the search methods used to locate relevant studies comprehensive?
Yes.
PubMed, MEDLINE, Ageline, CINAHL, PsychLIT, PsychINFO, Sport Discuss (SIRC/CDC), Cochrane Register, PEDro, Educational Resources Information Centre, and Dissertation Abstracts International were searched in a computer-aided search. A wide variety of relevant search terms was used. In addition, an extensive manual search and cross-referencing from review and original articles were performed. The 'ancestry search approach' (reference provided) was used in order to ensure comprehensive coverage.
PubMed, MEDLINE, Ageline, CINAHL, PsychLIT, PsychINFO, Sport Discuss (SIRC/CDC), Cochrane Register, PEDro, Educational Resources Information Centre, and Dissertation Abstracts International were searched in a computer-aided search. A wide variety of relevant search terms was used. In addition, an extensive manual search and cross-referencing from review and original articles were performed. The 'ancestry search approach' (reference provided) was used in order to ensure comprehensive coverage.
2.3
Was this the right sample to answer the objectives?
Yes. All 30 included studies were RCTs and investigated a physical training intervention.
Inclusion criteria:
- RCTs that included a nonintervention control or comparison group or control or comparison period
- subjects older than 65 years
- reported baseline mini-mental state examination score (MMSE) of less than 26, or subjects diagnosed by physician as having some degree of cognitive impairment or preexisting diagnosis of dementia reported by the original author
- any exercise program or form of rehabilitative exercises, physical activity, fitness, or recreational therapy
- reported means, standard deviations, t test or F test, and n values
- minimum of 5 subjects per group
- at least one dependent variable from one of the following categories: health-related physical fitness (cardiovascular, strength, flexibility, body mass index), functional, cognitive, and behavioral
- journal articles, master's theses, and doctoral dissertations published in English and indexed between Jan 1970 and October 2003
Exclusion criteria:
- not enough statistical information to calculate effect size
- outcomes not including physical fitness or function, behavior, or cognitive function
- studies based on qualitative designs or narrative case reports
- 5 or fewer subjects (? - inclusion criterion is minimum of 5 subjects)
Inclusion criteria:
- RCTs that included a nonintervention control or comparison group or control or comparison period
- subjects older than 65 years
- reported baseline mini-mental state examination score (MMSE) of less than 26, or subjects diagnosed by physician as having some degree of cognitive impairment or preexisting diagnosis of dementia reported by the original author
- any exercise program or form of rehabilitative exercises, physical activity, fitness, or recreational therapy
- reported means, standard deviations, t test or F test, and n values
- minimum of 5 subjects per group
- at least one dependent variable from one of the following categories: health-related physical fitness (cardiovascular, strength, flexibility, body mass index), functional, cognitive, and behavioral
- journal articles, master's theses, and doctoral dissertations published in English and indexed between Jan 1970 and October 2003
Exclusion criteria:
- not enough statistical information to calculate effect size
- outcomes not including physical fitness or function, behavior, or cognitive function
- studies based on qualitative designs or narrative case reports
- 5 or fewer subjects (? - inclusion criterion is minimum of 5 subjects)
2.4
Is the study large enough to achieve its objectives?
Yes. 30 included studies, 2020 subjects across all included studies, with 1023 subjects in the treatment group and 997 subjects in the control group.
2.5
Were all the studies accounted for?
No QUOROM flow-chart provided. No exact numbers of initial search results ('more than 300 articles') and discarded results provided. No description of the selection process.
2.6
Were all appropriate outcomes considered?
Yes.
In the context of the authors' reasoning, all the appropriate outcomes have been considered. The authors' perspective is to investigate whether 'physical activity' or 'exercise' in general show positive effects on health in general in the population of older people with dementia, similarly to the positive effects of physical activity on cardiovascular conditions and related diseases shown in the 1970s and 1980s.
From my perspective (falls prevention and falls rehabilitation in older people with dementia), only some of the reported outcomes are of interest, which are strength and flexibility. Balance and falls outcomes are not reported in the included studies.
In the context of the authors' reasoning, all the appropriate outcomes have been considered. The authors' perspective is to investigate whether 'physical activity' or 'exercise' in general show positive effects on health in general in the population of older people with dementia, similarly to the positive effects of physical activity on cardiovascular conditions and related diseases shown in the 1970s and 1980s.
From my perspective (falls prevention and falls rehabilitation in older people with dementia), only some of the reported outcomes are of interest, which are strength and flexibility. Balance and falls outcomes are not reported in the included studies.
2.7
Has ethical approval been obtained if appropriate?
Not appropriate for systematic review. For the included studies ethical approval is not commented on.
3
Measurement and observation
3.1
Is it clear what was measured, how it was measured and what the outcomes were?
Yes.
To assess methodological quality of the included studies, authors used a quality score adapted from the Delphi List and the PEDro Scale with a maximum score of 25. This scoring system was based on other reviews that used a similar system, references are provided. Validity and reliability of the scoring system are not commented on. 2 authors scored the included studies independently.
With regards to the outcome measures used in the included RCTs, the authors only mention the MMSE. The outcomes of included RCTs are described, e.g. cardiovascular, strength, behavior, etc., but the actual outcome measures used in the individual RCTs, and their validity and reliability for a population with cognitive impairment and dementia is not commented on. Measurement in the included studies has been assessed as part of the methodological quality assessment, but is not commented on by the authors.
To assess methodological quality of the included studies, authors used a quality score adapted from the Delphi List and the PEDro Scale with a maximum score of 25. This scoring system was based on other reviews that used a similar system, references are provided. Validity and reliability of the scoring system are not commented on. 2 authors scored the included studies independently.
With regards to the outcome measures used in the included RCTs, the authors only mention the MMSE. The outcomes of included RCTs are described, e.g. cardiovascular, strength, behavior, etc., but the actual outcome measures used in the individual RCTs, and their validity and reliability for a population with cognitive impairment and dementia is not commented on. Measurement in the included studies has been assessed as part of the methodological quality assessment, but is not commented on by the authors.
3.2
Were explicit methods used to determine which studies to include in the review?
Yes.
Inclusion and exclusion criteria are described (minor conflict between inclusion criterion 'minimum of 5 subjects in each group' and exclusion criterion '5 or fewer subjects').
The inclusion criterion that possibly allows for some degree of subjective interpretation is 'at least 1 dependent variable from one of the following categories: health-related physical fitness (cardiovascular, strength, flexibility, body mass index), functional, cognitive and behavioral'.
The selection process is not described.
Inclusion and exclusion criteria are described (minor conflict between inclusion criterion 'minimum of 5 subjects in each group' and exclusion criterion '5 or fewer subjects').
The inclusion criterion that possibly allows for some degree of subjective interpretation is 'at least 1 dependent variable from one of the following categories: health-related physical fitness (cardiovascular, strength, flexibility, body mass index), functional, cognitive and behavioral'.
The selection process is not described.
3.3
Was the selection of primary studies re-producible and free from bias?
Unsure.
The search strategy is briefly described, but ideally the full search strategy would have been made available. The selection process is not described.
The search strategy is briefly described, but ideally the full search strategy would have been made available. The selection process is not described.
3.4
Was the methodologic quality of the primary studies assessed?
Yes.
To assess methodological quality of the included studies, authors used a quality score adapted from the Delphi List and the PEDro Scale with a maximum score of 25. This scoring system was based on other reviews that used a similar system, References are provided. 2 authors scored the included studies independently.
To assess methodological quality of the included studies, authors used a quality score adapted from the Delphi List and the PEDro Scale with a maximum score of 25. This scoring system was based on other reviews that used a similar system, References are provided. 2 authors scored the included studies independently.
3.5
Are the measurements valid?
See 3.1. of this appraisal.
3.6
Are the measurements reliable?
See 3.1. of this appraisal.
3.7
Are the measurements reproducible?
See 3.1. of this appraisal.
4
Presentation of results
4.1
Are the basic data adequately described?
Yes. Table 2 summarizes characteristics of included trials, including MMSE scores, study quality scores, and type of intervention (including frequency, intensity and duration).
Main criticism: rather than just describe the type of outcome (e.g. cardiovascular, strength, etc.) the authors should have included the used outcome measures. This would have made it easier for the reader to judge the reliability and validity of the outcome measures used in the individual studies. Adding the type of setting the intervention was delivered in (e.g. care home or in person's own home) would have added to the picture, since this is an important issue in older people with dementia.
Main criticism: rather than just describe the type of outcome (e.g. cardiovascular, strength, etc.) the authors should have included the used outcome measures. This would have made it easier for the reader to judge the reliability and validity of the outcome measures used in the individual studies. Adding the type of setting the intervention was delivered in (e.g. care home or in person's own home) would have added to the picture, since this is an important issue in older people with dementia.
4.2
Were the differences between studies adequately described?
Yes.
See 4.1. of this appraisal.
See 4.1. of this appraisal.
4.3
Are the results presented clearly, objectively and in sufficient detail to enable readers to make their own judgement?
Unsure.
Due to the choice the authors made to describe general improvement in general outcomes, it is difficult for the reader to actually translate the improvement into clinically meaningful context. E.g. the summary effect size for strength was 0.75 (the intervention group was stronger): other than stating that the intervention group was stronger, it does not illustrate the context of the improvement. Similarly, and probably more importantly, for the functional outcomes: the summary effect size was 0.59 (the intervention group performed better in functional assessment): without knowing the actual functional assessment tool, it is difficult for the reader to appreciate the actual value of that improvement.
Again, it brings the reader back to the lack of information on the actual outcome measures used in the individual studies, and on lack of information on validity and reliability of these outcome measures.
Due to the choice the authors made to describe general improvement in general outcomes, it is difficult for the reader to actually translate the improvement into clinically meaningful context. E.g. the summary effect size for strength was 0.75 (the intervention group was stronger): other than stating that the intervention group was stronger, it does not illustrate the context of the improvement. Similarly, and probably more importantly, for the functional outcomes: the summary effect size was 0.59 (the intervention group performed better in functional assessment): without knowing the actual functional assessment tool, it is difficult for the reader to appreciate the actual value of that improvement.
Again, it brings the reader back to the lack of information on the actual outcome measures used in the individual studies, and on lack of information on validity and reliability of these outcome measures.
4.4
Are the results internally consistent, i.e. do the numbers add up properly?
Yes.
5
Analysis
5.1
Were the results of primary studies combined appropriately?
The authors used a quantitative method of review proposed by Glass (reference provided), which estimates a population effect size from effect size values obtained from individual studies. A fixed effects model was used to decide whether a summary effect size was statistically significant.
5.2
Has a sensitivity analysis been performed?
Not commented on.
5.3
Were all the important outcomes considered?
Yes, the selected outcomes cover the relevant issues in older people with cognitive impairment and dementia:
1) outcomes of health related physical fitness (cardiovascular, strength, flexibility, or body mass index)
2) functional outcomes
3) cognitive outcomes
4) behavioral outcomes
Summary effect size is presented for: all studies and all outcomes combined; for each individual outcome; for each of the subcategories of the health related fitness outcome.
Summary effect size is analyzed according to mode of intervention delivery (intensity, frequency, and duration) and according to level of cognitive impairment. The largest effect sizes were linked to the most cognitively impaired subjects, and to a higher number of mean sessions per week.
Data is not analyzed according to quality scores of included studies. 80% of studies were rated to be of moderate or high quality.
1) outcomes of health related physical fitness (cardiovascular, strength, flexibility, or body mass index)
2) functional outcomes
3) cognitive outcomes
4) behavioral outcomes
Summary effect size is presented for: all studies and all outcomes combined; for each individual outcome; for each of the subcategories of the health related fitness outcome.
Summary effect size is analyzed according to mode of intervention delivery (intensity, frequency, and duration) and according to level of cognitive impairment. The largest effect sizes were linked to the most cognitively impaired subjects, and to a higher number of mean sessions per week.
Data is not analyzed according to quality scores of included studies. 80% of studies were rated to be of moderate or high quality.
5.4
Are the data suitable for analysis?
Yes (?).
Studies with insufficient data for statistical analysis were excluded.
However, authors state that the selected studies provided 'most of the information needed for statistical analysis' (?). In their discussion, authors acknowledge as a limitation of their analysis that 'because of lack of raw data and the need to estimate ES values, the combined ES estimates for selected outcomes may not be precise' (?).
Studies with insufficient data for statistical analysis were excluded.
However, authors state that the selected studies provided 'most of the information needed for statistical analysis' (?). In their discussion, authors acknowledge as a limitation of their analysis that 'because of lack of raw data and the need to estimate ES values, the combined ES estimates for selected outcomes may not be precise' (?).
5.5
Are the methods appropriate to the data?
Yes (?) - Unable to determine from my knowledge and understanding of statistics.
Published in a well established journal (Archives of Physical Medicine and Rehabilitation).
Published in a well established journal (Archives of Physical Medicine and Rehabilitation).
5.6
Are any statistics correctly performed and interpreted?
As 5.5. of this appraisal.
6
Discussion
6.1
Are the results discussed in relation to existing knowledge on the subject and study objectives?
Partly.
Authors state that to their knowledge this is the first research synthesis to evaluate the effects of physical training on older people with cognitive impairment. They mention studies that have investigated effects of physical training in cognitively normal older people. It would have been valuable to briefly compare results of this systematic review with data from a cognitively normal population, i.e. do both groups benefit to the same extent from physical training or does one group show better improvement.
See 4.1. and 4.3. of this appraisal: The authors do not seem to acknowledge the value of knowing the actual outcome measures used in the individual studies.
In their discussion, authors acknowledge 'heterogeneity in the outcomes that could be explained by differences in study methods, subjects' mental status, or other characteristics not reported'.
Authors state that to their knowledge this is the first research synthesis to evaluate the effects of physical training on older people with cognitive impairment. They mention studies that have investigated effects of physical training in cognitively normal older people. It would have been valuable to briefly compare results of this systematic review with data from a cognitively normal population, i.e. do both groups benefit to the same extent from physical training or does one group show better improvement.
See 4.1. and 4.3. of this appraisal: The authors do not seem to acknowledge the value of knowing the actual outcome measures used in the individual studies.
In their discussion, authors acknowledge 'heterogeneity in the outcomes that could be explained by differences in study methods, subjects' mental status, or other characteristics not reported'.
6.2
Is the discussion biased?
Unsure.
Authors draw positive conclusions from their analysis, but at the same time openly discuss limitations and criticisms to their systematic review.
Overall, the authors' criticism seems somewhat disconnected with the general positive tenor of the conclusion. E.g. authors acknowledge 'important limitations related to the quality of study design, including the absence of blinding procedures and small sample sizes' and 'heterogeneity in the outcomes that could be explained by differences in study methods, subjects' mental status, or other characteristics not reported'. However, this is not reflected in preceding discussion and conclusion.
Authors draw positive conclusions from their analysis, but at the same time openly discuss limitations and criticisms to their systematic review.
Overall, the authors' criticism seems somewhat disconnected with the general positive tenor of the conclusion. E.g. authors acknowledge 'important limitations related to the quality of study design, including the absence of blinding procedures and small sample sizes' and 'heterogeneity in the outcomes that could be explained by differences in study methods, subjects' mental status, or other characteristics not reported'. However, this is not reflected in preceding discussion and conclusion.
7
Interpretation
7.1
Are the author's conclusions justified by the data?
Unsure.
On the one hand the authors conclude that there is a 'medium to large treatment effect for health-related physical fitness components, and an overall medium treatment effect for combined physical, cognitive, functional and behavioral outcomes'. On the other hand the authors acknowledge 'important limitations related to the quality of study design' and call for 'future studies involving multicenter trials with rigorous experimental controls' to resolve some of the issues not addressed or left unanswered in their investigation.
On the one hand the authors conclude that there is a 'medium to large treatment effect for health-related physical fitness components, and an overall medium treatment effect for combined physical, cognitive, functional and behavioral outcomes'. On the other hand the authors acknowledge 'important limitations related to the quality of study design' and call for 'future studies involving multicenter trials with rigorous experimental controls' to resolve some of the issues not addressed or left unanswered in their investigation.
7.2
What level of evidence has this paper presented?
CEBM level of evidence: 1a- (minus-sign due to query about heterogeneity and quality of included studies).
7.3
Does this paper help me to answer my problem?
Yes and no.
From the point of view of my question (physiotherapy interventions for falls prevention and falls rehabilitation in older adults with cognitive impairment and dementia) it would have been interesting to gain information on benefits of physical training in people with dementia, as some of the investigated outcomes, e.g. strength, play an important part in falls prevention/rehabilitation. Having appraised this paper, I am sceptical about the authors' conclusions, mainly due to questionable homogeneity of included studies.
Still, the paper pools and brings together available literature on the topic and informs the reader to a certain extent about the current evidence base. Whether the statistical analysis and conclusion are entirely appropriate or not, the paper still contributes to the literature on the topic and highlights gaps in knowledge and need for further research.
From the point of view of my question (physiotherapy interventions for falls prevention and falls rehabilitation in older adults with cognitive impairment and dementia) it would have been interesting to gain information on benefits of physical training in people with dementia, as some of the investigated outcomes, e.g. strength, play an important part in falls prevention/rehabilitation. Having appraised this paper, I am sceptical about the authors' conclusions, mainly due to questionable homogeneity of included studies.
Still, the paper pools and brings together available literature on the topic and informs the reader to a certain extent about the current evidence base. Whether the statistical analysis and conclusion are entirely appropriate or not, the paper still contributes to the literature on the topic and highlights gaps in knowledge and need for further research.
8
Implementation
8.1
Can any necessary change be implemented in practice?
Not applicable. No change in my current area of work (physiotherapist on acute hospital ward in area of acute elderly medicine) is indicated from the evidence presented in this paper. The team practices a multidisciplinary approach for prevention and rehabilitation following a fall according to current UK guidelines, including multidisciplinary assessment and individualised multifactorial intervention. The approach is adapted for the person with cognitive impairment or dementia as thought appropriate.
8.2
What aids to implementation exist?
Not applicable.
8.3
What barriers to implementation exist?
Not applicable.