CME212 Introduction to LargeScale Computing in Engineering

1
Programming Tools

• Compilers and Linking, make, Debuggers, Source
  Code Management, man

2
Man Pages

• Almost all UNIX-like OS are documented in
  so-called manual pages
• Or man pages, for short
• Looked up using the man tool
• Try man man
• Path to man pages is given by the shell variable
  MANPATH

3
Man Pages, Sections
4
Man Pages

• Man pages are divided into chapters
• NAME (search phrase)
• SYNOPSIS (short description, function prototypes)
• DESCRIPTION (details)
• OPTIONS (command line options)
• ENVIRONMENT VARIABLES
• EXIT STATUS
• FILES (relevant configuration files)
• SEE ALSO (related commands)
• NOTES (release notes, bugs, quirks)

5
Man Tools

• By default man pages are shown using the more or
  less tool
• man less
• To trigger search press / and then a phrase
• / SEE ALSO
• Raw format of man pages is device independent
• You can generate ASCII, PS, DVI
• Format troff/nroff or groff (GNU variants)

6
Man Tools, contd

• You can search the man database using the
  commands whatis and apropos
• whatis malloc
• apropos compression
• apropos errno
• Some interesting pages/commands
• man intro (try all man sections)
• tkman (graphical man browser)

7
The Compilation Process

• The parser is language dependent, follows a
  grammar.
• Checks your syntax
• Called the front end of a compiler
• The compiler works with an intermediate form,
  built from the parser output, which it operates
  on to produce assembly code
• This form is language-independent
• This involves register allocations, data layouts
  and code optimization (compiler course)
• Back end

8
Some Compiler Options

• gcc -c
• Just compile, dont link
• gcc -E
• Just preprocess, print out source
• gcc -S
• Produce assembly code
• gcc -o
• Save result into
• gcc -I
• Add to search for include files
• gcc -D
• Define the macro symbol
• Equivalent to define

9
Documentation on gcc

• man gcc
• http//gcc.gnu.org/onlinedocs/gcc-4.0.3/gcc/
• Typical manual topics
• Introduction
• Programming Languages Supported by GCC
• Language Standards Supported by GCC
• GCC Command Options
• C Implementation-defined behavior
• Extensions to the C Language Family

10
The Link Editor (ld)

• The compiler produces machine code and a table of
  names (symbol table) of variables and procedures
• This is called an object file (.o)
• The link editor (or linker) takes several object
  files and merges them, by associating the symbols
  with machine code addresses
• Performs relocation
• Works with position-independent code (PIC)

11
Symbol Table Example

• include
• include
• int im_a_global
• int main(void)
• double pi
• pi 4.0atan(1.0)
• printf(Pi is lf\n,pi)
• return 0

• gcc c pi.c
• nm pi.o
• 00000004 C im_a_global
• 00000000 T main
• U printf
• gcc o pi pi.o
• ./pi
• Pi is 3.141593

Undefined symbol
Automatic variables are not exported to the linker
12
Executable Files

• The link editor produces an executable file which
  is compatible with the program loader
• Format ELF (Executable and Linking Format)
• When a program is executed, the program loader
  copies the sections of the executable file into
  the virtual address space and execution can begin

13
Executable Files, Example

• gcc o pi pi.c
• nm ./pi
• 08049508 D _DYNAMIC
• 080495dc D _GLOBAL_OFFSET_TABLE_
• 08048470 T __i686.get_pc_thunk.bx
• 080494f4 A __init_array_end
• 080494f4 A __init_array_start
• 08048420 T __libc_csu_fini
• 080483c0 T __libc_csu_init
• U __libc_start_main_at__at_GLIBC_2.0
• 080495fc A _edata
• 08049604 A _end
• 080484b4 T _fini
• 080484d0 R _fp_hw
• 08048284 T _init
• 080482d0 T _start
• 08049600 B im_a_global
• 08048374 T main
• 080495f8 d p.4385

• include
• include
• int im_a_global
• int main(void)
• double pi
• pi 4.0atan(1.0)
• printf(Pi is lf\n,pi)
• return 0

The compiler inserts some loading code into the
executable
Undefined symbols are left to the runtime linker
14
Libraries

• It is very common to use a library of object
  files instead of explicitly stating every object
  file to use when linking
• Libraries can be either static or dynamic
• In static libraries (.a), the object files are
  copied into the executable file
• In dynamic libraries (.so, .dll), only the
  symbols are associated and the linking takes
  place at runtime by the runtime linker
• Most dynamic libraries can also be shared between
  many programs to save memory

15
Creating Static Libraries

• A static library is put together by the ar
  command
• Static libraries are sometimes referred to as an
  archive (.a)
• When using libraries the lib part of the name
  is removed at link time
• gcc -c triangle.c
• ar cru libtriangle.a triangle.o
• gcc -c program.c
• gcc -o program program.o -L -ltriangle

16
Creating a Dynamic Library

• Dynamic shared libraries are created by the
  compiler where each object file should be
  compiled with -fPIC.
• gcc -c -fPIC triangle.c
• gcc -shared -fPIC -Wl,-soname,libtriangle.so -o
  libtriangle.so triangle.o
• gcc -c program.c
• gcc -o program program.o -L -ltriangle

17
Sun Compiler

• The Sun compiler cc takes the same flags as gcc
  except for shared libraries
• -fPIC is -KPIC
• -shared is -G
• Sun compiler more optimized for Suns hardware
• junior.stanford.edu, tree.stanford.edu, CEES
• More information
• http//docs.sun.com
• cc -flags
• /opt/SUNWspro/docs/index.html

18
Checking Dependencies

• include
• include
• int im_a_global
• int main(void)
• double pi
• pi 4.0atan(1.0)
• printf(Pi is lf\n,pi)
• return 0

• gcc o pi pi.c
• ldd ./pi.o
• libm.so.6 /lib/tls/libm.so.6 (0xb7f05000)
• libc.so.6 /lib/tls/libc.so.6 (0xb7dcf000)
• /lib/ld-linux.so.2 /lib/ld-linux.so.2
  (0xb7f2c000)

Lists libraries to be loaded at runtime.
ld-linux.so.2 is the runtime linker. man
ld man ld.so
19
Linking and Loading, Solaris

• ldd(1), lists dependencies of executable files or
  shared objects
• dump(1), elfdump(1) dumps parts of an object file
• nm(1), GNU program that lists symbols from object
  files
• truss(1), traces system calls
• sotruss(1), traces shared library calls
• pstack PID, dumps the stack frames of a running
  process, see proc(1)
• pldd PID, lists loaded dynamical libraries

20
Linking and Loading, Linux

• ldd(1), lists dependencies of executable files or
  shared objects
• objdump(1) dumps parts of an object file or
  executable
• readelf(1) dumps parts of executable files
• nm(1), GNU program that lists symbols from object
  files
• strace(1), traces system calls
• ltrace(1), traces shared library calls

21
strace ./pi

• execve("./pi", "./pi", / 29 vars /) 0
• uname(sys"Linux", node"cardinal2.Stanford.EDU",
  ...) 0
• old_mmap(NULL, 4096, PROT_READPROT_WRITE,
  MAP_PRIVATEMAP_ANONYMOUS, -1, 0) 0xb7fc2000
• open("/etc/ld.so.preload", O_RDONLY) -1
  ENOENT (No such file or directory)
• open("/etc/ld.so.cache", O_RDONLY) 3
• fstat64(3, st_modeS_IFREG0644, st_size15207,
  ...) 0
• old_mmap(NULL, 15207, PROT_READ, MAP_PRIVATE, 3,
  0) 0xb7fbe000
• close(3) 0
• open("/lib/tls/libm.so.6", O_RDONLY) 3
• read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1
  \0\0\00005\0\000"..., 512) 512
• fstat64(3, st_modeS_IFREG0644, st_size134496,
  ...) 0
• old_mmap(NULL, 136976, PROT_READPROT_EXEC,
  MAP_PRIVATE, 3, 0) 0xb7f9c000
• old_mmap(0xb7fbd000, 4096, PROT_READPROT_WRITE,
  MAP_PRIVATEMAP_FIXED, 3, 0x20000) 0xb7fbd000
• close(3) 0
• open("/lib/tls/libc.so.6", O_RDONLY) 3
• read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1
  \0\0\0Z\1\000"..., 512) 512
• fstat64(3, st_modeS_IFREG0755,
  st_size1254660, ...) 0
• old_mmap(NULL, 4096, PROT_READPROT_WRITE,
  MAP_PRIVATEMAP_ANONYMOUS, -1, 0) 0xb7f9b000
• old_mmap(NULL, 1264972, PROT_READPROT_EXEC,
  MAP_PRIVATE, 3, 0) 0xb7e66000

Math library is loaded
Stdio library
22
make, Maintenance Tool

• make is usually used to compile and link programs
  consisting of several files
• A program often uses, or depends on, several
  other modules or libraries
• A Makefile specifies the dependencies between the
  program and the modules
• If a file changes, make recompiles only those
  files which depend on the changed file

23
make, Concepts

• target A file that depends on other files
• dependencies A list of files that a target
  depends on
• rule How to build a target

prog module1.o prog.o gcc -o prog module1.o
prog.o prog.o prog.c module1.o gcc -c
prog.c module1.o module1.c module1.h gcc -c
module1.c

24
make, Advanced Usage

• - base name of current target (no extension)
• _at_ - full name of current target
• ? - the list of dependencies newer than target
• The makedepend command goes through source files
  and determines dependencies for you, see man
  makedepend
• depend
• makedepend prog.c

25
make, Example

• CFLAGS -g -Wall
• CC gcc
• LIBS -lm
• INCLUDES
• OBJS a.o b.o c.o
• SRCS a.c b.c c.c prog1.c prog2.c
• HDRS abc.h
• all prog1 prog2
• The variable _at_ has the value of the target. In
  this case _at_ prog1
• prog1 prog1.o OBJS
• CC CFLAGS INCLUDES -o _at_ prog1.o
  OBJS LIBS
• prog2 prog2.o OBJS
• CC CFLAGS -o _at_ prog2.o OBJS LIBS
• .c.o
• CC CFLAGS INCLUDES -c

MACROS
26
make, Example contd

• depend
• makedepend SRCS
• clean
• rm .o core
• tar
• tar cf code.tar Makefile .c .h testfile1
• print
• more Makefile (HDRS) (SRCS) enscript -2r -p
  listing.ps
• DO NOT DELETE THIS LINE -- make depend depends
  on it

27
Source Code Management

• A lot of people can work on the same code
• Store all your code in a safe repository
• Check out a version, which gives a local copy of
  the requested source code
• Check in alterations (commit), to submit your
  modified code
• Versioning is done automatically with comments
• Resolve inconsistensies at check in, by merging
  the codes (often done manually)
• Several tools
• CVS (most common), PRCS, RCS, SCCS, Subversion

28
Using RCS

• RCS puts the history files into a directory
  called RCS
• Local repository
• History files end with the suffix ,v
• man rcsintro
• co myprog.c
• ci myprog.c
• rcsdiff myprog.c
• The Tools menu in GNU emacs provides a nice
  interface to RCS

29
More on RCS

• RCS can annotate your source using the id tag
• RCS inputs version data
• More details are given using the Log variable
• You can go back using co with the r flag to
  specify a revision to checkout

30
Team Projects

• For projects involving multiple developers at
  different locations we need more sophisticated
  tools
• Access control
• Security/encryption
• Handling conflicts
• Two major tools used in the open source community
• CVS
• Subversion

31
Basic SVN Usage

• Create your filesystem layout locally
• Use svn import to copy this structure to the
  repository
• svn import https//wilkinson.stanford.edu/svn/cm
  e212/henlof/ -m Initial import
• Use svn checkout and svn commit to reflect
  changes in your local filesystem to the repository

32
Typical SVN Repository Layout

• https//wilkinson.stanford.edu/svn/cme212/SUNetID/

project1/
trunk/ tags/ branches/
trunk stores the current active
branch branches store experimental
releases tags are finished products
project2/
trunk/ tags/ branches/
project3/
trunk/ tags/ branches/
33
Debuggers

• A debugger helps you find causes of errors in
  your code
• Executes your code like an interpreter
• You can step through your program
• Set breakpoints
• Print and set variables
• Catch run-time exceptions
• Two standard debuggers
• gdb (GNU)
• dbx

34
Compiling for Debugging

• To be able to refer to names and symbols in your
  source code it must be stored in the executable
  file
• Standard flag -g
• Without -g you can only debug using virtual
  addresses

35
Debug Information with -g

• include
• include
• int im_a_global
• int main(void)
• double pi
• pi 4.0atan(1.0)
• printf(Pi is lf\n,pi)
• return 0

• gcc c g pi.c
• readelf -w pi.o
• Abbrev Number 4 (DW_TAG_subprogram)
• DW_AT_sibling
• DW_AT_external 1
• DW_AT_name main
• DW_AT_decl_file 1
• DW_AT_decl_line 6
• DW_AT_prototyped 1
• DW_AT_type
• DW_AT_low_pc 0
• DW_AT_high_pc 0x3f
• DW_AT_frame_base 1 byte block 55
  (DW_OP_reg5)
• Abbrev Number 5 (DW_TAG_variable)
• DW_AT_name pi
• DW_AT_decl_file 1
• DW_AT_decl_line 8
• DW_AT_type
• DW_AT_location 2 byte block 75 78
  (DW_OP_breg5 -8)

36
Using gdb
gdb ./pi (gdb) break main Breakpoint 1 at
0x8048380 file pi.c, line 10. (gdb) run Starting
program /afs/ir.stanford.edu/users/h/e/henlof/tes
t/trunk/pi Breakpoint 1, main (argc1 ,..) at
debug_me.c19 (gdb) print pi 2
4.8542713620543657e-270 (gdb) next 11
printf("Pi is lf\n",pi) (gdb) print pi 3
3.1415926535897931 (gdb) list 6 int
main(void) 7 8 double pi 9
10 pi 4.0atan(1.0) 11
printf("Pi is lf\n",pi) 12 13
return 0 14 (gdb) where 0 main () at
pi.c11
37
Some gdb Commands
38
More on gdb

• Type help in the gdb prompt
• help breakpoints, help running
• gdb can also debug core dumps and attach to
  already-running processes that you own
• There is a graphical interface to gdb called the
  Dynamic Data Debugger ddd
• ddd ./a.out
• Allows you to visualize data and breakpoints

39
Core Files

• The OS can be configured to dump the virtual
  address space of a process to a core file
• Typically this happens at a fatal error
• core dumped
• This file can be used to debug your progam
  post-mortem
• Especially useful if debug symbols are present