Understanding Research: What is Research in Computer Science

1
Understanding Research What is Research in
Computer Science?

• Henning Schulzrinne
• Columbia University
• updated Oct. 2006

2
My biases

• Systems research in networking
• Protocols and software systems for multimedia
• Focused on solving practical problems rigorously

3
Modes of CS research

• New algorithm
• faster on average or under certain conditions
• New system
• e.g., new network protocol
• new application
• system combining lower-level components into
  new interesting artifact
• Characterization of systems
• measure performance of OS, networks, compilers,
  CPUs,
• Apply known technique X to new area
• Neural networks for picking pizza toppings
• Other disciplines, less common in CS systems
  research
• taxonomy of things
• confirm or contradict earlier work
• meta-studies

4
Research maturity of field

• Initially, almost no related work
• premium on being first, even if incomplete or
  partially wrong
• CS, in most areas, no longer virgin territory
• higher emphasis on correctness, measurement,
  comparison, differences to related work
• large sub-communities
• products often more advanced than research
  prototypes
• product development has dozens of people, not 1-2

5
Two criteria for research impact

• Impact matters, not (just) paper count
• Used by others
• other researchers citation, tool use, data use,
  common knowledge, text books
• industry products or product ideas, standards
• Significant improvement
• allows new things (new service, new application)
• improves existing applications, services
• often, danger of 5 improvements that only pan
  out under specially controlled circumstances
• e.g., network QOS work
• but 5 may matter if resource is very expensive
• better algorithms may fail due to secondary
  reasons
• too complex to understand
• too brittle (sensitive to parameter changes)
• danger optimize to death (and delay publication)

6
The cynics view of research

• Research
• whats publishable
• what gets you through a thesis defense
• what gets you a job interview
• Not always related to quality
• reviewers often harshest on things they
  understand
• ? sometimes easier to publish at the fringes of
  field
• fashion dictates
• technology driven
• funding cycle driven (active networks, sensor
  networks, )
• e.g., ATM and DQDB used to be hot
• you know the topic is dead if most of the
  conference papers are submitted from China ?

7
On publishing

• PhD program expect at least one major
  publication/year
• competitive conference, journal, not just
  workshop or TR
• optimists approach publish in 10 conference
• high risk, but looks good on resume
• may get scooped by others if rejected
• may not get published in time for graduation or
  job interview
• pessimists approach pick 50 conference
• many papers are found by IEEE Xplore, CiteSeer or
  Google ? venue matters somewhat less
• resume looks thin if only workshops
• results may become inaccessible ? at least,
  publish in ACM, IEEE or Usenix-sponsored
  conferences (digital libraries)

8
Technical reports arXiv

• Publish results early as tech report
• avoids scooping
• establishes prior art for patent suits
• doesnt preclude conference or journal
  publication
• Consider arXiV.org for tech reports
• wider visibility
• Announce to relevant mailing lists
• use sparingly
• e.g., TCCC list, IETF list (e.g., for IETF
  measurements)

9
Publishing is (also) marketing

• Many results and projects become well-known not
  because of quality
• lots of presentations at workshops
• invited talks
• famous university (the MIT/Berkeley/CMU
  syndrome)
• nice web pages
• impact on standards
• Cant just rely on good paper to rise to top

10
Good student ? good researcher

• Good student find correct solution, quickly
• well-defined problems, with likely solution
• Good researcher
• find new topics
• synthesize new results from older ones
• know when to quit (good enough, diminishing
  returns)
• able to advertise and get funding
• Good researchers were usually good students, but
  not vice versa ?

11
Practical scientific

• View as engineer or mathematician
• Build things vs. describe and abstract things
• Pick area according to strengths and interests
• Best research combines
• new idea (application, algorithm, )
• solid analysis
• implementation to test ideas (and to use problems
  as source of new ideas)

12
Finding research topics

• Build on existing expertise
• in CS, hard to compete as amateur in new area
• Look at NSF workshop reports for various areas
• See if resource cost equation has changed
• e.g., power suddenly matters for mobile nodes
• New environmental problems
• spam
• DOS attacks
• Look at other sub-fields or disciplines to see if
  problems or approaches apply

13
Finding research problems

• Well-known problems
• competitions
• classic algorithm problem
• Problem definition part of the problem
• must have convincing example application
• Talk to industry
• Observe own behavior web search
• Devils advocate can be solved with existing
  tools?

14
Publication process
workshop paper
conference paper
journal paper
conference paper (related topic)
technical report
15
Publication process - reminders

• Never, ever submit the same paper to different
  conferences, workshops or journals at the same
  time
• i.e., dont submit to conference and journal
• can submit accepted papers in workshop to
  conference, but
• 80/20 rule new paper should be 80 new, no more
  than 20 the same as earlier paper
• self-plagiarism

16
Patents and publication

• Patenting after publication is possible, but
  restricted
• Publication public presentation or printing,
  not submission to publication for review

17
Other ways for research impact

• Distribute software
• sourceforge
• particularly useful if platform for other
  research
• Distribute measurement data
• but worry about anonymity (see AOL search data)
• Survey pages on particular topic
• collection of references by self and others

18
Finding related work

• ACM Digital Library
• IEEE Xplore
• Google
• CiteSeer
• netbib

19
Finding conferences

• Conference calendars
• IEEE and Comsoc (http//www.comsoc.org)
• ACM
• Usenix
• SPIE
• EDAS ?
• TCCC mailing list
• http//www.comsoc.org/tccc

20
Major networking conferences

• ACM Sigcomm (deadline January)
• IEEE Infocom (July)
• IEEE ICNP
• ACM Sigmetrics
• Usenix
• ACM Mobicom
• ICC and Globecom
• Milcom
• NOSSDAV