PowerPoint-Pr

1

• On the Epistemic Value of Reputation 
• The Place of Ratings and Reputational Tools in
  Knowledge Organization
• 11th International ISKO Conference
• Rome, February 23-26 2010
• Gloria Origgi Judith Simon
• Institut Jean Nicod
• ENS-EHESS-CNRS
• Paris, France

2
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

3
REPUTATION Overview

• Guiding Questions
• How to use reputation for epistemic purpose?
• Whats the epistemic value of reputation?
• Is this a good thing or a bad thing?

4
REPUTATION Background LiquidPub
Different methods of quantifying, assessing
propagating reputation
Further Information on http//project.liquidpub.o
rg http//liquidpub.wordpress.com
5
REPUTATION Background
Thesis Types of Epistemic Sociality
6
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

7
REPUTATION Introduction
Hero
Sinner
Drunkard
Reputation as Heuristic Reputation as a way to
classify social types within the community that
will allow its member to manage their relations
with others, to make inferences and predictions
about their behavior, i.e. to construct a basic
"social map" that will help them orient in their
society.
Source http//en.wikipedia.org/wiki/FileMatteson
_Scarlet_Letter.jpg http//en.wikipedia.org/wiki/T
he_Scarlet_Letter
8
REPUTATION Introduction
Hero
Sinner
Drunkard
Morally questionable! CENSORED!
Reputation as Heuristic Reputation as a way to
classify social types within the community that
will allow its member to manage their relations
with others, to make inferences and predictions
about their behavior, i.e. to construct a basic
"social map" that will help them orient in their
society.
Source http//en.wikipedia.org/wiki/FileMatteson
_Scarlet_Letter.jpg http//en.wikipedia.org/wiki/T
he_Scarlet_Letter
9
REPUTATION and Knowledge Organization

• Reputation as social information about the value
  of people, systems and processes that release
  information.
• Focus relationship between this special form of
  social information and the processes of knowledge
  organization and evaluation.
• More precisely, we argue not only that
• (1) we make use of other people's reputations to
  evaluate information in various ways
• (2) within systems, like the Web, that make
  possible the easy and dynamic organization and
  re-organization of knowledge, we also use our own
  rankings to determine new content and generate
  new categories.

10
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

11
REPUTATION as Evaluative Social Information

• Reputation is the informational track of our past
  actions, it is the credibility that an agent or
  an item earn through repeated interactions.
• Reputational Cues are indicators/proxies of
  reputation where quality of objects or agents
  cannot be directly assessed
• Relying on reputational cues is an efficient way
  of shaping the too rich informational landscape
  around us by creating new relevant categories.

12
REPUTATION as Evaluative Social Information

• In an information-dense environment, where
  sources are in constant competition to get
  attention and the option of the direct
  verification of the information is often simply
  not available at reasonable costs, evaluation and
  rankings are epistemic tools and cognitive
  practices that provide an inevitable shortcut to
  information
• Modest Prediction The higher the uncertainty on
  the content of information, the stronger is the
  weight of the opinions of others in order to
  establish the quality of this content.

13
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

14
REPUTATION A Model for the Epistemic Use of
Reputation

• Lehrer Wagner (1981) Rational Consensus in
  Science and Society
• Proposes a model for rational decision making
  processes in science, society and the arts that
  makes epistemic use of reputation
• It rests upon the employment of consensual
  probabilities, utilities and weights
• For decision making processes to be rational, it
  is central that all evidence or empirical
  information available for the topic of concern
  has to be used
• Experimental Social Information

15
REPUTATION A Model for the Epistemic Use of
Reputation

• Social information information about the
  expertise of other experts on issues at hand
  Reputation
• Example Expert Dilemma Do we need to vaccinate
  large parts of the population to prevent a
  pandemie?
• Step 1 each expert gives a weights other
  experts competency
• Step 2 weights are laid open
• Step 3 revision of own weights taking the
  others assessment into account depending on the
  weights assigned to them
• Repeat cycle till consensus is achieved
• Once these consensual weights are achieved, they
  can be applied to answering the question of
  concern by weighting each members votes on the
  issue with their consensual personal weight.

16
REPUTATION A Model for the Epistemic Use of
Reputation

• Lehrer Wagner propose a model of how to
  rationally reach consensus that rests upon the
  epistemic use of reputation
• This implies that reputational information, i.e.
  social information about other people that is
  evaluative, is epistemically useful.
• Do we need such formal models?
• Epistemic use of reputational cues does not have
  to follow such a formal method. But on the Web,
  models similar to this one are embedded and
  hidden within different applications.

17
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

18
REPUTATION Reputational Tools on the Web
http//www.briansolis.com/2008/08/introducing-conv
ersation-prism/
19
REPUTATION Reputational Tools on the Web

• What the Web makes possible today is an
  algorithmic treatment of methods of gathering
  social information to extract knowledge. Ratings
  and rankings on the Web are the result of
  collective human registered activities with
  artificial devices.
• However, the control of the heuristics and
  techniques that underlie this dynamics of
  information may be out of sight or
  incomprehensible for the users who find
  themselves in the very vulnerable position of
  relying on external sources of information
  through a dynamic, machine-based channel of
  communication whose heuristics and biases are not
  under their control.
• Thus, the reputational tools that are
  proliferating on the Web should be scrutinized by
  epistemically responsible users who do not want
  to accept too naïvely the outcome of a process
  they do not control.

20
REPUTATION Reputational Tools on the Web
21
REPUTATION Reputational Tools on the Web
22
REPUTATION Reputational Tools on the Web
23
REPUTATION Reputational Tools on the Web
Interestingness! There are lots of elements that
make something 'interesting' (or not) on Flickr.
Where the clickthroughs are coming from who
comments on it and when who marks it as a
favorite its tags and many more things which are
constantly changing. Interestingness changes over
time, as more and more fantastic content and
stories are added to Flickr. http//www.flickr.c
om/explore/interesting/
24
REPUTATION Reputational Tools on the Web
Interestingness! There are lots of elements that
make something 'interesting' (or not) on Flickr.
Where the clickthroughs are coming from who
comments on it and when who marks it as a
favorite its tags and many more things which are
constantly changing. Interestingness changes over
time, as more and more fantastic content and
stories are added to Flickr. http//www.flickr.c
om/explore/interesting/
Interestingness is a new category based on
reputational mechanisms, making use of different
proxies whose weight and combination is not
obvious!
25
REPUTATION Reputational Tools on the Web

• Reputational tools get more and more central on
  the Web
• Rankings and Ratings provide new arrangements of
  information
• Early years of 2000 focus on personalized
  information (My-Features)
• Now trend towards systems of shared preferences,
  were people can rely on others preferences and
  rankings to construct there own access to and
  categorization of information
• Examples
• Flickrs Interestingess
• Twitter-Logic of Followers and Leaders
• LiquidJournal people or groups create their own
  journals by selecting (existing) content and
  making it available via their selection

26
REPUTATION Problems for the Epistemic Use of
Reputation
So, all is well, or?
27
REPUTATION Problems for the Epistemic Use of
Reputation
So, all is well, or? Well, not quite
28
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

29
REPUTATION Problems for the Epistemic Use of
Reputation

• The danger of misuse of reputation danger of
  epistemic injustice (Fricker 2007), judging
  epistemic credibility and social identity (Alcoff
  2001)
• Using proxies that are not valid to assess the
  reputation and epistemic credibility of epistemic
  agents (gender, race, nationality, institutional
  background,)
• testimonial injustice occurs when prejudice
  causes a hearer to give a deflated level of
  credibility to a speakers word ((Fricker 2007)
  1)

30
REPUTATION Problems for the Epistemic Use of
Reputation

• 2. Limits of the epistemic usefulness of
  reputation itself
• How to calculate reputational values in the first
  place?
• What are the pros and cons of different methods
  e.g. peer review versus Amazon-type ratings?
• Which proxies should be used and how should they
  be combined?
• Stability of reputation over time?
• Transferability of reputation over domains?
• 3. Lack of transparency of reputational
  algorithms and metrics
• How should users be responsible knowers if they
  do not understand the functioning, the strengths
  and weaknesses of different mechanisms?
• How to detect biases, if the mechanisms are not
  laid open?
• Need to make these mechanisms visible and
  understandable

31
REPUTATION Overview

• Background
• Introduction
• Reputation as Evaluative Social Information
• A Rational Model for the Epistemic Use of
  Reputation
• Reputational Tools on the Web
• Problems with the Epistemic Use of Reputation
• Conclusions

32
REPUTATION Conclusions

• Ratings and reputational tools in knowledge
  organization have epistemological, practical as
  well as ethical implications.
• Epistemological questions How epistemically
  warranted is the use of these tools?
• Practical questions How to develop these
  mechanisms? Which proxies to use, how to combine
  and weigh them? Whats the status of these new
  types of classes, such as interestingness? Can
  ratings and ranking serve as middle-ground
  categorizations?
• Ethical and political question Epistemic
  injustices lack of transparency Once
  reputation mechanisms become formalized and are
  embedded within tools, there is a clear danger
  that epistemic injustices are inscribed in and
  reinforced by technology.

33
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?

34
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?
• Both - it is useful and dangerous. But either
  way, reputational information, different
  reputational proxies and methods of quantifying
  and combining them are being used extensively on
  the Web and elsewhere.

35
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?
• Both - it is useful and dangerous. But either
  way, reputational information, different
  reputational proxies and methods of quantifying
  and combining them are being used extensively on
  the Web and elsewhere.
• An additional problem on the Web concerns the
  lack of visibility for the users the metrics and
  algorithms behind different reputation tools are
  often unknown.

36
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?
• Both - it is useful and dangerous. But either
  way, reputational information, different
  reputational proxies and methods of quantifying
  and combining them are being used extensively on
  the Web and elsewhere.
• An additional problem on the Web concerns the
  lack of visibility for the users the metrics and
  algorithms behind different reputation tools are
  often unknown.
• There is an epistemic duty of epistemologists and
  knowledge organization scholars to thoroughly
  analyze these different reputational practices
  from epistemological, ethical and political
  perspectives.

37
REPUTATION

• Thank you for your attention!
• Contact
• Judith Simon
• Institut Jean Nicod
• Ecole Normale Supérieure
• 29, rue d'Ulm
• F-75005 Paris
• email judith.simon_at_ens.fr
• www http//www.institutnicod.org
• tel [phone]
• fax [phone]

38
REPUTATION Problems for the Epistemic Use of
Reputation

• Two major problems of using reputation for
  epistemic purpose
• the use of reputation to assess content can be
  epistemically beneficial while being morally
  questionable
• 2) limits of the epistemic usefulness of
  reputation itself

39
REPUTATION as Evaluative Social Information

• We want to explore the epistemic value of
  reputation, while being aware of the ethical and
  political problems that might come with using it
  for epistemic purpose.
• Using the judgment on past records to classify an
  agent or an item can be epistemologically useful
  in the absence or - as is especially relevant
  today - overabundance of information. But it has
  to be and remain open to constant scrutiny and
  revision to be epistemically useful and ethically
  just.

40
REPUTATION Reputational Tools on the Web

• Early years of 2000 focus on personalized
  information (My-Features)
• Now trend towards systems of shared preferences,
  were people can rely on others preferences and
  rankings to construct there own access to and
  categorization of information
• Examples
• Flickrs Interestingess
• Twitter-Logic of Followers and Leaders
• LiquidJournal people or groups create their own
  journals by selecting (existing) content and
  making it available via their selection

41
REPUTATION Background

• Epistemic Use and Value of Reputation as ongoing
  inquiry by two authors fuelled by different
  sources

42
REPUTATION Background
Two authors -Two perspectives
43
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?
• Both - it is useful and dangerous.

44
REPUTATION Conclusions

• What is the epistemic values of reputation? Is it
  useful? Or rather dangerous?
• Both - it is useful and dangerous. But either
  way, reputational information, different methods
  or reputational cues of assessing it are being
  used extensively on the Web and elsewhere.