SM?RT

1
SM?RT technical aspects

• UNICEF, New York
• June 23rd 2005
• Michael Golden on behalf of the
• Technical Advisory Group

2
What is new in SM?RT methods?

• The basis of the SM?RT methodology has indeed
  been drawn from many established manuals and
  guidelines, particularly that recently published
  by SCF. Elements have also been taken from,
  among others, the MSF, FANTA, ACF and WHO
  publications as well as standard epidemiological
  and statistical texts.
• These publications are good and have served us
  well. So why a new guide?

3
What is new in SM?RT methods? 2

• None of these guides answers all the questions
  that are regularly faced by field workers.
• The current guidelines are not living documents
  that are regularly updated
• There is great emphasis on obtaining a sound
  representative sample (and all the statistics are
  taught) but less on the quality of the
  measurements.
• There is an old adage
• epidemiologists make the wrong measurements on
  the right people and laboratory scientists make
  the right measurements on the wrong people.
  Clinicians make the wrong measurements on the
  wrong people
• we just have to make the right measurements on
  the right people to have a good survey
• The software has not been integrated with the
  manual the current software is not
  non-epidemiologist friendly.

4
What SM?RT attempts to do

• Integrates anthropometric, mortality and
  food-security data together.
• Where possible avoids epidemiological and
  statistical jargon, yet uses sound
  epidemiological and statistical methods to derive
  the results.
• Tries to make the survey as easy as possible for
  the person in the field.
• Tries to generate accurate data as easily and
  rapidly as possible
• Tries to anticipate practical difficulties that
  arise at field level, and offer solutions that
  are theoretically sound.

5
What SM?RT attempts to do 2

• Gives software that addresses issues of survey
  design
• Gives software to assess the quality of training
  and indicates which individuals require
  additional training or replacement
• Helps with identification of errors during data
  entry
• Examines the internal structure of the data to
  see where the quality of the survey is inadequate
• Produce a report in a standard format that
  contains all the information needed to judge the
  quality of the survey as well as presenting the
  data in a standard format. (Analogous to the
  CONSORT guidelines for clinical trail reporting)

6
What is new in SM?RT

• Rationalization of nomenclature
• Changing precision with expected prevalence
• New method for estimating death rates
• Variable recall period method for dealing with
  mass migration
• Avoidance of Epi method of proximity sampling
  where feasible

7
What are the contentious issues? 1 - Nutrition

• Cut-off points for flags
• Calculation of prevalence from the mean when the
  data are suspect
• No substitution of children
• Handling of oedematous cases
• Reporting of both Z-score and median
• Choosing a design effect for cluster sampling
• Correction for clothing
• Calculation of sample size for each survey and
  avoiding the standard 30x30 design, where a
  smaller sample can be used

8
What are the contentious issues? 2 Death rates

• Nomenclature for Mortality/ Death rates
• The new method for Death rates, although
  theoretically sound has not been fully explored
  in practice
• Advice to treat catastrophic events (tsunami,
  genocidal attacks, etc) as events and not to
  express the mortality as a rate for these
  events. But useful to calculate Death rates
  before and after the event.

9
The importance of random measurement error.

• Many consider that random error is neutral in
  terms of the reported data.
• This is not the case.
• Small random errors of measurement or recording
  can lead to a large variation in the reported
  prevalence.
• If it is suspected that the data are not sound
  then it is better to use the calculated
  prevalence of malnutrition (from the mean with an
  SD of 1) than report the unsound data.

10
Effect of measurement errors on survey results

• Suppose an imprecise error moves a value from one
  segment to another (up or down)
• If the errors are random then the same number of
  values will move from one half of the
  distribution to the other (orange to green and
  green to orange).
• There will be no change in the mean of the
  distribution provided that there are as many
  positive as negative measurement errors

11
Effect of measurement errors on survey results

• If there is an error that moves a value from
  one segment to the other in the tail then there
  will be more points moving from the orange to the
  green than from the green to the orange in
  relation to the respective areas
• There will be an increase in the Standard
  Deviation.
• There will be an increased prevalence of
  values below 2Z and also below 3Z

12
Effect of measurement errors on survey results

• A change in the standard deviation from 1.0 to
  1.2 will have a major effect upon the prevalence
  of moderate and severe malnutrition.
• Imprecise measurements are potentially a major
  cause of error in surveys.
• The prevalence will be exaggerated even if the
  positive and negative errors balance each other
  out.

13
Effect of SD of survey on prevalence of wasting

• Wide SD, from measurement error can increase
  prevalence dramatically
• Narrow SD from over cleaning, selection or bias
  can reduce the prevalence of malnutrition

14
Effect of measurement errors on survey results
15
SM?RTs response

• SM?RT lays particular emphasis upon training and
  provides software to assess the quality of
  anthropometric training
• SM?RT advocates for different use of flags for
  cleaning data
• SM?RT suggests using calculated prevalence for
  data that has not got an internal structure
  consistent with best practice measurements

16
Cut-off points for flags

• Conventional to use very extreme values that are
  biologically implausible to exclude data when
  analyzing. All other data assumed correct.
• The number of cleaned subjects rarely reported.
• However, we would like to remove all erroneous
  measurements and keep all correct measurements.

17
Cut-off points for flags - 2

• Alternative The flags can be applied at a
  level where the data are MORE LIKELY
  (statistically) to be errors than correct
  measurements.
• If the mean of the sample is calculated then any
  value that is more than 3Z (SD) or less than -3Z
  is more likely to be an error than a true
  measurement. (using this on a perfect survey
  would result in removal of 1 in 1000 subjects
  incorrectly, which would make almost no
  difference to the reported result. There are
  usually many more than 1/1000 such subjects.
  Retaining many such subjects will inflate the
  reported rate of malnutrition

18
Calculation of prevalence from the mean when the
data are suspect

• The SM?RT software reports the SD, skewness and
  kurtosis of the sample (and for each team), and
  the Poisson distribution of the malnourished
  cases.
• It counts the prevalence of children below the
  cut-off points
• It calculates the prevalence from the mean and
  survey SD
• It calculates the prevalence from the mean with
  an SD of 1.0
• With a good survey all these estimates are
  similar
• It recommends that the calculated values are used
  where the internal structure of the data is
  suspect

19
No substitution of children

• Many organizations return to houses with absent
  children twice and then substitute a child from
  the nearest household. (to ensure the sample
  size)
• They do not report the number of absent children
• SM?RT methodology does not advise substitution
  numbers of absentees have to be reported.

20
Handling of oedematous cases

• At the moment weight and height of oedematous
  children are disregarded although these cases
  are counted as severely malnourished
• If they are included then there is no correction
  for the weight of the oedema fluid.
• This can not be resolved without data on the
  magnitude of the correction that should be made.
• We have generated such data for SM?RT in order to
  make this correction possible as an option

21
Oedema as of body weight
No. oedema W.Afr Sahel E. Afr
Kwash 1066 2.7 5.4 2.9 1.4 3.2
Kwash 1319 4.3 6.6 4.3 2.5 4.7
Kwash 459 8.4 8.2 8.4 7.0 11.1
Data of Yvonne Grellety from 23 centers in 13
countries in Africa
22
Correction for oedema - option
Marasmus 0 0.0
Kwash 2000 2.7
Kwash 1000 4.3
Kwash 250 8.4
weighted mean 3.6
Mean of and 3.6
23
(No Transcript)
24
Reporting of both Z-score and median

• median used for admission of children to
  therapeutic and supplementary feeding
• Data needed to plan intervention
• median can be used for adolescents whereas
  Z-score cannot
• median is much easier to teach and understand
• The prevalence reported is quite different with
  Z-score and median. Important to be consistent
  when comparing survey data

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Choosing a design effect for cluster sampling

• Conventionally this has been set at 2.0
• A recent small series (8 surveys) gives design
  effects from 0.8 to 2.4 with a mean of 1.6
• The effects of an error in choosing an
  inappropriate design effect are not symmetrical.
  If to large, additional children are measured
  unnecessarily if to small the results may be
  discarded and a new survey requested.
• There is a need for research into the
  distribution of design effects and the sort of
  variables that affect them.

30
Correction for clothing

• In cold climes where the clothes are heavy
  (central Asia, North Korea for example)
  correction must be made.
• If light pants for example are worn say of 30g,
  this systematic error, even though less than the
  divisions on the scale, make a real difference to
  the reported prevalence.

31
Wasting by height group as the population
nutritional state deterioratesAll groups are
affected as the situation becomes desperate
older children have a high prevalence
32

• Nomenclature, The SMART mortality method, mass
  migration and catastrophic events are addressed
  in the mortality presentation.
• Thank you