Relevance in academic research - PowerPoint PPT Presentation

1 / 31
About This Presentation
Title:

Relevance in academic research

Description:

Relevance in academic research Dan Remenyi PhD Visiting Professor School of Systems and Data Studies Trinity College Dublin dan.remenyi_at_tcd.ie ... – PowerPoint PPT presentation

Number of Views:378
Avg rating:3.0/5.0
Slides: 32
Provided by: DanR200
Category:

less

Transcript and Presenter's Notes

Title: Relevance in academic research


1
Relevance in academic research
  • Dan Remenyi PhD
  • Visiting Professor
  • School of Systems and Data Studies
  • Trinity College Dublin
  • dan.remenyi_at_tcd.ie

2
Establishing the context
  • The issues of relevance and rigor are complex
    concepts and need careful understanding.
  • Some words have a large semantic footprint and
    relevance, rigor, validity, reliability,
    generalisability and integrity are good examples
    of these.
  • No one can proceed with any understanding of
    research without having a clear agreement with
    regards the meaning of terms.

3
The effectiveness and efficiency parallel
  • A quick rule of thumb understanding of relevance
    is doing the right thing or looking at the right
    issues
  • A quick rule of thumb understanding of rigor is
    doing the thing right
  • These are the colloquialisms coined by Peter
    Drucker to explain effectiveness and efficiency
  • Thus relevance equates to effectiveness and rigor
    relates to well not so much efficiency but
    something more like adequacy

4
Relevance and Rigor
  • Relevance is a high level issue in that you can
    say that the research project is relevant or not
    relevant as a whole it could also be said
    that the use of case studies was not relevant to
    the objectives of the study
  • Rigor is an issue which permeates the whole
    project in that you can say that the literature
    review was not rigorously approached or the
    sample was not rigorously chosen, or the
    conclusions were not rigorously written up.

5
Academic Research is a Challenge
6
Academic research
  • What is the nature of academic research?
  • It is an intellectual activity
  • It is an attempt to answer a question
  • It is a process of question, data and analysis
  • Academic research has two primary attributes-
  • It adds something of value to the body of
    theoretical knowledge.and practice!
  • It demonstrate a high level of scholarship

7
Something of value to the body of theoretical
knowledge.and practice
  • Something of value is reflected in the concept of
    relevance.
  • Relevance needs to be perceived by both the
    non-academic stakeholders and also the academic
    community.
  • Without some contribution to theory the research
    will not be much valued by academics.
  • Mode 2 brings both together.

8
Scholarship and rigor
  • Being well read and being able to demonstrate
    that in conversation
  • Being able to craft an argument in writing
  • Scholarship is connected to rigor
  • An argument is a series of logical steps by which
    someone is persuaded. In this sense is not
    confrontational.
  • Academic work i.e. either a dissertation or a
    peer reviewed paper needs to argue for relevance,
    rigor, validity, reliability, generalisability.

9
Quality academic research
  • Dimensions of quality in academic research
  • addresses a pertinent question
  • uses academic language with care - remember the
    fog factor
  • uses appropriate data
  • is written up in intelligible English and correct
    format
  • is presented in an attractive way - structure
  • invoking authority
  • finds an acceptable/useful answer
  • Being parsimonious quality academic research
    needs to be relevant and rigorous

10
Double hurdle
  • Academic research needs to be relevant i.e. of
    interest and of use to a wide range of management
    at different levels and rigorous i.e. display
    academic excellence.
  • Being both relevant and rigorous has been
    described as a synthesis strategy.

11
Research question
  • The research question has been reached through a
    careful examination of and understanding of
  • The field of study
  • The topic
  • The specific questions and sub-questions

12
Some critical issues
  • Something of value
  • As judged by the academic community
  • Body of theoretical knowledge
  • Add something to established domains
  • Parsimonious
  • Expressed in a precise and economical way
  • Authority
  • Ideas/concepts/theories which have through the
    peer review process
  • Relevance
  • Has value to a group of stakeholders
  • Rigor
  • Complies with the rules of the research paradigm

13
Three more issues
  • Validity ..Has the research addressed the
    intended issues i.e. addressed the issues which
    are required to answer the research question?
  • Reliability..Has the research been conducted
    sufficiently well (rigorously) to provide results
    in which we can have confidence?
  • Generalisabilty..To what extent can the
    research be applied else where?

14
Rationalism versus Empiricism
  • Rationalism (theory)
  • The mind is superior to the senses dating from
    the Greeksat least Plato..thought experiments
  • Reason
  • Logic
  • Secondary data
  • Empiricism
  • If it cannot be held, felt or shaken etc it does
    not exist!
  • In its pure form it fails due to particle physics
    etc
  • Positivism versus interpretivism
  • Primary data

15
Assumptions - preliminary concepts
  • What are the assumptions under which academic
    research is conducted?
  • The world is understandable.
  • We are interested in understanding it.
  • Researchers will be open minded.
  • Researchers are capable of objective reasoning.
  • We can find data/evidence to explore the
    phenomena and therefore understand them.
  • The nature of the understanding can be both broad
    and also detailed.
  • Occams razor points out that too much complexity
    can obscure what is happening.
  • We always have to use ceteris paribus all other
    things being equal.
  • That some degree of cause and effect can be
    identified/assumed.
  • We all operate in terms of our cognitive capacity
    which changes, and hopefully develops/increases
    over time.

16
The Issue of Integrity Fleischmann and Pons
17
Integrity and Professor Woo Suk Hwang And
Professor Sir Cyril Burt
Sir Cyril Burt Reported nonexistent studies on
I.Q. tests in the 20s and 30s, and added
fictitious collaborators to his papers
18
Some ideas to examine
  • Relevance ..doing the right thing
  • Relevance ..to whom or to what?
    theory/academics/practitioners......Is it
    significant?
  • Rigor .doing the thing right
  • Is rigor a dimension of validity?
  • If the research is not valid then do not worry
    about reliability?
  • If the research is not reliable can it be valid?
  • If the research is neither valid or reliable then
    is it worthless?
  • What can go wrong question, data, analysis,
    finding conclusions etcthreats to the research

19
Evaluation of relevance and rigor
  • Relevance is normally evaluated holistically i.e.
    is the research important or is it not. Sometimes
    some aspects of the research may be more useful
    than others but this question is essentially a
    holistic one.
  • Rigor is normally evaluated on an issue by issue
    basis. This requires comment of individual
    aspects of the research. But there is also a
    holistic view. There may be some tension between
    the individual evaluations and a holistic view of
    the research. The sum of the parts may be
    different to the whole.

20
Relevance
  • In information science Relevance is the extent
    to which some information is pertinent,
    connected, or applicable to the matter at hand.
    It represents a key concept in the fields of
    documentation, information science, and
    information retrieval. Pehcevski and Larsen
  • Although academic relevance is today of essential
    importance to research, surprising little has
    been written about it.
  • It used to have been assumed that if the research
    is good then it is academically relevant and
    this attitude has only recently been challenged.
  • The main drivers of this challenge to relevance
    may be seen in the debate surrounding Mode 1 and
    Mode 2 research.

21
A stumbling block issue
  • Previously relevance was not a visible issue. If
    the research and his/her supervisor thought that
    the question was important then that was all that
    was needed.
  • Now the research has to convince the examiners or
    the reviewers that it is adequately relevant. An
    argument has to be made.
  • A research dissertation/paper could be
    refused/rejected on the grounds that the
    research is not adequately relevant

22
Weak and strong relevance
  • According to Blake Ives (US IS academic) there
    are two forms of relevance - weak relevance and
    strong relevance - http//www.misq.org/archivist/v
    ol/no20/issue3/edstat.html
  • Weak relevance occurs when a research question
    addresses an issue which is known and documented
  • Strong relevance requires an action research
    approach to answering the research question which
    produces concrete findings

23
A closer look at relevance
  • The statement that a research question is
    relevant begs the question of relevant to whom?
    and in which way?
  • A research question is relevant if there is-
  • A distinct interest among the stakeholders but
    who are the stakeholders? A material stakeholder
    group
  • The research question will provide interesting
    results and/or useful guidelines to professionals
  • Improvements to practice i.e. there is clear
    application for the findins

24
How are relevant research questions to be
identified?
  • Relevant research questions may be found in-
  • the extant literature
  • work place experiences
  • special interest groups
  • Business/professional/practitioner groups
  • government requests
  • general business press
  • Remember that a relevant question leads to a
    solution for someone
  • Is there pure or basic research in BM
    Studies?

25
What do we need to find a relevant research
question?
  • Imagination!
  • .. But also needed is imagination to create
    from these hints the great generalizations to
    guess at the wonderful, simple, but very strange
    patterns beneath them all, and then to experiment
    to check again whether we made the right guess.
    Richard Feynman, Six Easy Pieces, p2, Penguin
    Books, London (1995)
  • Finding a suitable research question should not
    be rushed.
  • Activities which can help to find relevant
    research questions include-
  • Talking to knowledgeable informants
  • Focus groups
  • Qualitative questionnaires
  • Blue sky and scenario sessions
  • Talking to your supervisor/s
  • Literatureacademic and popular press

26
A caveat!
  • The relevance will be judged by both academics
    and by practitioners but academics will/may have
    a greater say than practitioners.
  • The degree of relevance is often tested by the
    So What? or Whats new ....... and who cares?
    question. This question needs to be directly
    answered in a research dissertation and in a peer
    reviewed academic paper.
  • Relevance or the lack thereof will always be
    known post hoc or a posteriori but you have to
    address this issue on an a priori basis

27
The time dimension
  • Relevance has a time dimension.
  • What is relevant today may not be relevant
    tomorrow.
  • The time value of a research question is a
    function of the field of study and the topic and
    directly influences the relevance of the research.

28
The argument
  • Relevance may not be taken for granted.
  • The researcher needs to argue that the research
    is relevant because it-
  • answers a well known /important question/s
  • addressed a non-obscure question
  • will produce a useful result usually a paradigm
  • will allows several other questions to be
    explored
  • it will attract people and funding to the
    university or business school
  • etc

29
The argument is one of the key issues for
establishing relevance and rigor
30
Assessing relevance
  • Feedback from informants and peers is the most
    effective way of ensuring that the research is
    relevant.
  • It is useful to have in mind the need to test
    this relevance occasionally through out the
    duration of the research
  • Also test the findings by asking the stakeholders
    of the results of the research are likely to be
    useful to them.

31
Journal titles......Relevant?
  • Stark, S., Chernyshenko, O., Guenole, N.
    (Accepted).  Can subject matter expert ratings of
    statement extremity be used to streamline the
    development of unidimensional pairwise preference
    scales. (18 no ?)  Organizational Research
    Methods
  • Chernyshenko, O., Stark, S., Guenole, N.
    (2007). Can the discretionary nature of certain
    performance criteria lead to differential
    prediction across cultural groups? (15 yes or no
    answer) International Journal of Selection and
    Assessment, 15, 175-184
  • Be careful of papers with very long titles and
    many authors.
Write a Comment
User Comments (0)
About PowerShow.com