Method and Research Design

1
Method and Research Design

• Gerry S. Doroja, MSCS
• Associate Professor
• Department of Computer Science
• Xavier University - Ateneo de Cagayan
• gsd_at_xu.edu.ph
• August 31, 2004

Adapted from Writing up research online,
Language Center, Asian Institute of Technology,
Bangkok, Thailand, 2003 (http//www.languages.ait.
ac.th/EL21OPEN.HTM)
2
Purpose of Methods Chapter

• Answers these two main questions
• How was the data collected or generated?
• How was it analyzed?
• Shows your reader how you obtained your results

3
Why do you need to explain how you obtained your
results?

• We need to know how the data was obtained because
  the method affects the results.
• Knowing how the data was collected helps the
  reader evaluate the validity and reliability of
  your results, and the conclusions you draw from
  them.
• Often there are different methods that we can use
  to investigate a research problemyour
  methodology should make clear the reasons why you
  chose a particular method or procedure.

4
Why do you need to explain how you obtained your
results?

• The reader wants to know that the data was
  collected or generated in a way that is
  consistent with accepted practice in the field of
  study.
• The research methods must be appropriate to the
  objectives of the study.

5
Why do you need to explain how you obtained your
results?

• The methodology should also discuss the problems
  that were anticipated and explain the steps taken
  to prevent them from occurring, and the problems
  that did occur and the ways their impact was
  minimized.
• In some cases, it is useful for other researchers
  to adapt or replicate your methodology, so often
  sufficient information is given to allow others
  to use the work (i.e.,this is particularly the
  case when a new method had been developed, or an
  innovative adaptation used.

6
Common Problems

• irrelevant detail unnecessary explanation of
  basic procedures
• Remember that you are not writing a how-to guide
  for beginners.
• Your readers will be people who have a level of
  expertise in your field and you can assume that
  they are familiar with basic assessments,
  laboratory procedures etc, so do not explain
  these in detail.

7
Common Problems

• problem blindness
• Most of us encounter some problems when
  collecting or generating our data.
• Do not ignore significant problems or pretend
  they did not occur.
• Often, recording how you overcame obstacles can
  form an interesting part of the methodology, and
  means you can also give a rationale for certain
  decisions, plus a realistic view of using the
  methods you chose.

8
This is how method fits into your work

• Introduction introduction of research problem
  introduction of objectives introduction of how
  objectives will be achieved (methodology) and,
  optional introduction of main findings and
  conclusions or expected outcomes and results.
• Literature review review of previous work
  relating to research problem (to define, explain,
  justify) review of previous work relating to
  methodology (to define, explain, justify)and,
  review of previous work relating to results
  (particularly reliability, etc.)
• Method (how the results were achieved)
  explanation of how data was collected/generated
  explanation of how data was analyzed
  explanation of methodological problems and their
  solutions or effects
• Results and discussion presentation of results
  interpretation of results discussion of results
  (e.g. comparison with results in previous
  research effects of methods used on the data
  obtained)
• Conclusions has the research problem been
  solved? to what extent have the objectives been
  achieved? what has been learnt from the results?
  how can this knowledge be used? what are the
  shortcomings of the research, or the research
  methodology? etc.

9
Some Examples of Different Types of Research
Designs

• Analysis classes of data are collected and
  studies conducted to discern patterns and
  formulate principles that might guide future
  action
• Case Study the background, development, current
  conditions and environmental interactions of one
  or more individuals, groups, communities,
  businesses or institutions is observed, recorded
  and analyzed for stages of patterns in relation
  to internal and external influences.
• Comparison two or more existing situations are
  studied to determine their similarities and
  differences.
• Correlation-Prediction statistically significant
  correlation coefficients between and among a
  number of factors are sought and interpreted.
• Evaluation research to determine whether a
  program or project followed the prescribed
  procedures and achieved the stated outcomes.

10
Some Examples of Different Types of Research
Designs

• Design-Demonstration new systems or programs are
  constructed, tested and evaluated
• Experiment one or more variables are manipulated
  and the results analyzed.
• Survey-Questionnaire behaviors, beliefs and
  observations of specific groups are identified,
  reported and interpreted.
• Status a representative or selected sample of
  one or more phenomena is examined to determine
  its special characteristics.
• Theory Construction an attempt to find or
  describe principles that explain how things work
  the way they do.
• Trend Analysis predicting or forecasting the
  future direction of events.

11
Design-Demonstration

• Follows the System Development Lifecycle (SDLC)
  or one of its variants
• Define System (Requirements)
• Design System
• Build System (e.g., program)
• Test System (viz-a-viz Design)
• Evaluate System (viz-a-viz Requirements)
• Use a good software engineering book as reference!

12
Research Design determines the Methods Used
13
Research Project Phases, Research Methods the
Literature Review
Research Methods
Research Phase-Method Matrix

Research Phases
14
What is expected out of research?

• Some examples
• a definition of a problem or task
• a unit for solving a problem, performing a task
• identification of factors in influencing the
  cost, effectiveness, or applicability of a unit
  (perhaps with some idea of the relative
  importance of the factors)
• development of an ideal model
• a finished unit that can be distributed to users
  and,
• measurement of some properties of a unit e.g.,
  run time, chip area, representation requirements,
  reliability, usability, etc.

15
What are the research methodologies? (Some
generic examples)

• Design units
• Implement units
• Run units
• Find and prove theorems
• Analyze and consolidate

• Study users
• Import techniques
• Read literature
• Write paper

There are many possible research methodologies
(depending on the field/discipline).
The Research Methodology section in a research
proposal or paper is normally an elaboration of
one or a combination of these methodologies used
in the research as well as the materials/resources
to be used/used.
16
What are the research methodologies? (Some
Computer Science examples)

• Writing programs
• Building systems
• Developing architectures
• Developing content architectures (ontologies,
  knowledge bases, class libraries, graphics
  toolboxes, etc.)
• Measuring properties of units
• Finding and proving theorems
• Analyzing and consolidating previous research
• Interviewing experts, customers

• Performing psychological experiments, surveys,
  observations
• Building hardware
• Reading literature
• Importing techniques and results from other
  fields
• Measuring and predicting constraints on future
  units (e.g., VLSI technology, government
  regulation, user expectations and requirements)
  and,
• Writing papers, monographs, and textbooks.

17
What are the research project phases?

• An individual research project follows phases
  related to the research life-cycle
• Choose research question/problem/tradeoff
• Determine current state of knowledge (literature
  review)
• Apply appropriate methods to produce research
  results and,
• Write up research results.
• To address the basic questions/framework for
  research
• What is the problem? (INTRODUCTION)
• What did you use to tackle it? (MATERIALS AND
  METHODS)
• What results followed? (RESULTS AND DISCUSSION)

Research is not complete until it is written up!
18
What are the required supporting skills?

• Different research methods require different
  supporting skills.
• Some examples of required SKILLS

• Programming
• Design
• Organization
• Mathematics
• Psychological techniques
• Protocol analysis
• Experimental manipulations

• Survey methods
• Statistics
• Writing proposals
• Writing papers
• Critiquing papers
• Designing experiments and,
• Giving talks

19
How to design a research project?

• Some factors to consider
• Impact and significance
• Required skills
• Inherent interest
• Feasibility and competition
• Phase of research extendibility
• Opportunities for learning new skills

20
TIPS Writing your Method Chapter

• Bear in mind the purpose of the method section.
• Keep notes of what you did, why you did it, and
  what happened. Some researchers keep research
  diaries so that they have a record of the methods
  they used. Make sure you develop some way of
  recording your work, and that you then carefully
  select which material to include in your final
  methodology section.
• Remember who your audience will be, and be
  careful not to include unnecessary details.
• Avoid using "I" to write about what you did. Do
  not use "we" unless you really were working with
  one or more other researchers. One way to avoid
  this problem is to use passive voice.
• Verb tenses - be consistent, and choose the
  correct one!(Future tense in proposal past tense
  in final paper).

21
Voice in the Method Chapter

• Verbs can be used in either in the passive voice
  (The biscuit was eaten by the dog OR The biscuit
  was eaten) or the active voice (The dog ate the
  biscuit).
• What does it matter? Well, using passive or
  active voice changes the emphasis of a piece of
  writing. For example
• "The biscuit was eaten by the dog." This sentence
  is passive because the main focus of the sentence
  is on the biscuit, but the biscuit does not do
  anything - instead something is done to the
  biscuit (by the dog). In fact, we can even leave
  out the part about who performed the action The
  biscuit was eaten.
• "The dog ate the biscuit." This is active because
  the main focus of our attention is on the dog,
  and the dog is the one who does something (it
  eats the biscuit).

22
Why use passive voice?

• People reading your thesis or dissertation are
  going to be far less interested in you than in
  your work so the emphasis should be on what you
  did and not on you.
• Also, by not saying "I weighed the sample" but
  "The sample was weighed" you make your writing
  sound more objective.

23
How do you make the passive voice?

• Passive voice is the verb to be followed by a
  past participle
• For example
• Rice is grown in Thailand.(simple present is
  plus past participle grown)
• The film is being shown at Future Park
  Mall.(present continuous is being plus past
  participle shown)
• The sample was weighed to find its dry
  weight.(simple past was plus past participle
  weighed)
• The samples were being dried . . .(past
  continuous were being plus past participle dried)
• The interviews will be conducted in
  groups.(future will be plus past participle
  conducted)

24
Writing about what the researchers have done

• Research writing usually avoids using "I" or "we"
  (although "we" is becoming more acceptable in
  co-authored papers). By convention, if the
  passive voice is used we assume that an action
  was carried out by the researcher/s, and we don't
  say directly who did it.
• For example
• INCORRECTThe temperature inside the chamber was
  increased from 0 C to 20 C by the researcher.
• CORRECTThe temperature inside the chamber was
  increased from 0 C to 20 C. (We assume the
  researcher increased the temperature.)
• INCORRECTFour thermocouples were monitored
  hourly by the researcher.
• CORRECTFour thermocouples were monitored hourly.
  (We assume the researcher monitored them.)

25
Writing about what the equipment has done

• The active voice is usually used when the
  equipment has performed an action (i.e. when it
  is not the researcher/s who have performed the
  action).
• For example
• A 200hp generator provided power to the
  piezometers.
• Control gauges monitored air pressure inside the
  chamber.
• The use of active voice indicates that the
  researchers were not directly involved in the
  functioning of the equipment.
• The passive voice can be used to describe an
  action involving equipment, but a "by" clause
  must be included to which equipment performed the
  action.
• For example
• Power was supplied by 14 generators with
  capacities ranging from 90 to 300 KW.

26
Verb Tense and Method - Proposal

• Because you haven't yet carried out your
  research, you should write about what you plan to
  do in the future tense (because you will do the
  work at some point in the future).
• For example
• A multiple choice questionnaire will be
  administered to the top managers of fifty
  information technology companies in the country.
• Use "will" to make the future tense rather than
  the more informal "going to."
• For facts or information that are true and
  unlikely to change, write about them in the
  present tense.
• For example
• A multiple choice questionnaire will be used
  since it offers a way to reduce the time
  respondents will need to complete it, and
  therefore to increase the number of completed
  questionnaires.
• The fact that the multiple choice questionnaire
  offers a way to reduce completion time is true
  whether the questionnaire is administered today
  or next year (it is always true), so "offers" is
  used instead of "will offer."

27
Verb Tense and Method Final Paper

• Most of your methodology section will be written
  in the past tense because you are recording what
  you have done. Notice too that it is usually
  written in the simple past (the verb tense used
  for events that are now finished).
• For example
• The sample was weighed.
• Fish seed were added to the pool.
• A thermometer recorded changes in external
  temperature.

28
Verb Tense and Method Final Paper

• For facts or information that are true and
  unlikely to change, write about them in the
  present tense.
• For example
• Vietnam was chosen for this study because it has
  a long coastline.(Present tense is used because
  we assume that the length of Vietnam's coastline
  is unlikely to change.)
• Cornmeal was used to feed the fingerlings because
  it provides high nutritional content at a
  relatively low cost.(Present tense is used
  because we assume that neither the nutritional
  content nor the cost of corn meal is likely to
  change.)

29
Verb Tense and Method Final Paper

• Other verb tenses may also be used, for example
  to describe one event happening during another.
• Example
• Ethyl alcohol was added while the sample was
  being dried. (past continuous passive)
• The present tense is used to talk about the
  thesis or dissertation itself and what it
  contains, shows, etc. For example
• Table 3 shows that the main cause of weight
  increase was nutritional value of the
  feed.(Table 3 will always show this - it is now
  a fact that is unlikely to change, and will be
  true whenever this sentence is read, so present
  tense is used.)
• "to" is often used to mean "in order to" and "by"
  is usually followed by a verb ending in -ing 

30
Proof by Demonstration?

• Perhaps the most intuitively persuasive model for
  research is to build something and then let that
  artifact stand as an example for a more general
  class of solutions. There are numerous examples
  of this approach being taken within the field of
  computer science. It is possible to argue that
  the problems of implementing multi-user operating
  systems were solved more through the
  implementation and growth of UNIX than through a
  more measured process of scientific enquiry.
• However, there are many reasons why this approach
  is an unsatisfactory model for research. The main
  objection is that it carries high risks. For
  example, the artifact may fail long before we
  learn anything about the conclusion that we are
  seeking to support. Indeed, it is often the case
  that this approach ignores the formation of any
  clear hypothesis or conclusion until after the
  artifact is built. This may lead the artifact to
  become more important to the researcher than the
  ideas that it is intended to establish.
• The lack of a clear hypothesis need not be the
  barrier that it might seem. The proof by
  demonstration approach has much in common with
  current engineering practice. Iterative
  refinement can be used to move an implementation
  gradually towards some desired solution. The
  evidence elicited during previous failed attempts
  can be used to better define the goal of the
  research as the work progresses. The key problem
  here is that the iterative development of an
  artifact, in turn, requires a method or
  structure. Engineers need to carefully plan ways
  in which the faults found in one iteration can be
  fed back into subsequent development. This is,
  typically, done through testing techniques that
  are based upon other models of scientific
  argument. This close relationship between
  engineering and scientific method should not be
  surprising engineering n. an application of
  science to the design, building and use of
  machines, construction etc. (The Oxford Concise
  Dictionary).
• Source What is Research in Computing Science?
  By C. Johnson, Glascow University.