Value Documentation and Quality Assessment Testing

1
Value Documentation and Quality Assessment
Testing Compiler Network Meeting Geel November
07
Mark Roe Institute of Food Research WP1.8
Compiler Network and Supporting Task Groups
2

• Background
• Value Documentation Testing Results
• Quality Assessment Testing Results
• Conclusion Possible revisions to quality
  assessment

3
Why Document Data?

• To allow effective management, updating,
  extending and comparability of data across
  Europe.
• Documentation at the value level allows data to
  be combined with confidence that the data is
  comparable across different datasets

4
Progress in 2007

• Value Documentation Workshop Paris, March 2007
• EuroFIR standard description and technical annex
  revised - July 07
• Guidelines for quality index attribution to
  original data July 07 (joint WP1.3 WP1.8)

• Deliverable 1.8.6. Month 28
• Report on test value documentation
• by compilers using prototype standard

5
Testing protocol

• 21 Compilers (IFR, NUBEL, DTU, KTL, AFSSA, MATIS,
  BfEL, NKUA, UCC, CSPO, UiO, NFNI, INSA, FRI, NFA,
  TUBITAK, NNC, ETHZ, IMR, FVS, FC, NEVO)
• 8 Value References
• 7 papers from peer reviewed journal
• 1 product information
• 1 Value per reference to be documented with
  comments
• Excel template provided for documentation and QA
• Results from 12 compilers

6
Value Documentation Results 1

• Food Entity
• Mandatory properties (Original Food Name,
  Original Food Code, English Food Name, Product
  Type ) ok
• Guidance needed for documenting optional
  properties
• When should they be completed?
• e.g. additional food
  classifications
• Should assumptions be made?
• e.g. colour of food, nature of
  edible portion/waste
• Should they describe typical values or only
  actual values reported?
• e.g. edible proportion

7
Value Documentation Results 2

• FoodNames Entity
• Not clearly understood names in additional
  language, synonyms etc.
• Recipe, Ingredients, Yield Factor Entities
• Not applicable to references tested
• Sample Entity
• Some problems -
• Number of primary samples and primary sample unit
  size
• Primary sample handling not clearly understood

8
Value Documentation Results 3

• Component Identification Entity
• Method Specification Entity
• Difficult for many compilers
• Analytical key steps not easy to define
• Laboratory performance properties difficult for
  non-analysts
• Free-text method description needs clarification

9
Value Documentation Results 4

• Value Entity
• No major problems
• Need to clarify
• Date of generation
• Analytical Statistics Properties (n, analytical
  portion size, number of analytical portion
  replicates)
• Quality Assessment Entity
• Contributing Values Entity
• Not relevant to references tested
• Reference Entity
• Clarify citation property

10
Quality Assessment Overview

• Categories assessed
• Food description
• Component Identification
• Sampling Plan
• Number of Analytical samples
• Sample Handling
• Analytical Method
• Analytical Performance

All recorded as part of value documentation All
contribute to overall quality index and quality
confidence code
Designed for assessment of scientific literature
and laboratory reports
11
Quality Assessment Results1

• Food Description Category
• Wide variation in scores
• Better when description more detailed
• Interpretation difficult when criteria not
  explicitly stated
• When should assumptions be made?
• Inconsistent use of not applicable
• Consider reducing or weighting criteria?

12
Quality Assessment Results2

• Component Identification Category
• Scored consistently
• Some cases unit or matrix unit not considered
  unequivocal
• Sampling Plan Category
• Quite consistent but some criteria open to
  interpretation
• When is a sample plan representative?
• What is the number of primary samples?
• Not applicable needs to be used consistently
  rephrase criteria to include relevance?

13
Quality Assessment Results3

• Number of Analytical Samples Category
• Variable if not clear in reference examples
  needed
• Score should be 1,3 or 5?
• Sample Handling Category
• Very variable
• Difficult for non-analysts to assess
• Key criterion should be was any treatment for
  stabilisation necessary?
• Reduce score if stabilisation required and
  measures not taken or not reported?

14
Quality Assessment Results4

• Analytical Method Category
• Guidelines for methods/components required
• Analytical Performance Category
• Scores variable
• Difficult to interpret criteria related to
  reference materials and analytical replicates
• Appropriate number of analytical replicates not
  defined should be gt1?
• Quality Index
• Quite consistent despite variability in some
  categories

15
Conclusions

• Some revisions required to improve clarity and
  consistency
• Additional and improved guidelines required
• Clear guidelines essential for criteria related
  to analytical method
• Training will be necessary
• References tested were good quality, need to
  assess some low quality
• Value documentation is a time consuming and
  testing task!

16
Possible revisions to quality assessment

• Food Description
• Reduce number of criteria? Delete criteria
  related to container/wrapping and food contact
  surface
• Component Identification
• No changes necessary
• Sampling Plan
• Add relevance to most criteria so that not
  applicable can be used e.g. if relevant, were
  samples taken from more than one geographical
  region/sales outlet

17
Possible revisions to quality assessment
analytical method

• Categories related to analytical method
• Merge number of analytical samples, sample
  handling, analytical method and analytical
  performance categories into new sample
  analysis category
• To include
• Is the analytical method appropriate?
• Are the key analytical method steps appropriate?
• If relevant, was appropriate stabilization
  treatment applied?
• Is the number of analytical samples gt5?
• Were analytical portion replicates used?
• Was laboratory accredited or was method validated
  by performance testing?
• Was an appropriate reference material used?

If no then quality score 1(low quality)