INDRA

1
INDRA

• Mithun Shanbhag
• mrshanbh_at_syr.edu

2
CONTENTS
You are here

• Code Analyzer
• PE - Disassembler
• Symbol Handler
• P-T-M Enumerator
• Debug Events
• Call Interceptor
• Hardware Breakpoints
• Virtual Address Space Viewer
• Memory Dumper
• Call Stack Viewer
• Locals Enumerator
• Single Stepping Mode
• Architecture Analyzer
• Processor Features
• Cache Configurations
• Hyperthreading Support
• Kernel Mode Device Driver

• INDRA Hardware Event Monitor
• Applications
• Compiler Optimizations
• Memory Allocators
• Future Directions
• References

3
PE - DISASSEMBLER

• Overview of the PE file format
• PE file format is derived from the earlier COFF
  (common object file format) described in VMS/VAX
  architectures.
• A Portable Executable File Format supported on
  all CPUs on which Windows OS runs.
• File Extensions include .exe, .dll, .cpl, .ocx
• New PE32 format is described for IA64
  architecture.
• Windows CE uses the PE format with support for
  new processors.
• .NET binaries are also in the PE format.

4
PE - DISASSEMBLER
Image
Module
Picture taken from Matt Pietreks excellent
article REF.
5
PE - DISASSEMBLER

• Overview of the PE file format
• Mapping of the PE file is consistent on disk and
  in memory.
• Offsets of some sections may differ on disk and
  in memory.
• Some parts of a PE file may be read but not
  mapped in memory. (e.g. relocation section)
• Some parts of a PE file may not be mapped in at
  all (e.g. debug section)
• All defines, definitions and enumerations are in
  the WINNT.H file.

6
PE - DISASSEMBLER

• PE file contents -
• DOS Header
• PE File Header
• PE Optional Header
• PE Section Headers
• PE Sections
• Code Sections (.code)
• Data Sections (.rdata, .idata, .pdata)
• Text Section (.text)
• Resource Section (.rsrc)
• Relocation Section (.reloc)
• API Import and Export tables
• Custom Sections
• pragma data_seg (CUSTOM_SECTION)

DataDirectory (Array of 16 Structures)
7
PE - DISASSEMBLER

• Memory Mapping -

8
PE - DISASSEMBLER

• DOS Header Contents
• A 2-byte magic field, set to 0x5A4D (ascii Z,
  M ).
• A linker created stub which prints out the
  message This program cannot be run in DOS mode
  when the image is run in MS-DOS. At offset 0x3C,
  the stub has the file offset to the PE signature.
  
• The above mentioned 4-byte PE signature PE\0\0.

9
PE - DISASSEMBLER

• PE (File) Header

May have multiple characteristics
Screenshot from the INDRA analyzer prototype
10
PE - DISASSEMBLER

• PE Supported CPU Architectures
• Always look in WINNT.H first !
• Commonly Supported architectures (look at
  Offset 0x0)

11
PE - DISASSEMBLER

• The PE Optional Header
• Loader reads the PE Optional Header.
• Size of the PE Optional Header is not fixed.
• At present, we are ignoring the PE32 format.

12
Screenshot from the INDRA analyzer prototype
13

• Who stole my (121)th slide ?

14
PE - DISASSEMBLER

• Data Directory -

15
PE - DISASSEMBLER

• Section Headers
• Each row of the Section Table, in effect, is a
  section header.
• This table immediately follows the optional
  header. Hence the PE File Header does not contain
  a direct pointer to the section table.

16
Screenshot from the INDRA analyzer prototype
17
PE - DISASSEMBLER

• Exports Section
• The Export Directory Table (described in WINNT.H
  by the structure IMAGE_EXPORT_DIRECTORY) is used
  to resolve references to the functions exported
  within from the image.

18
PE - DISASSEMBLER

• Exports Section
• On the next slide we will try to see all the
  functions exported by kernel32.dll

We are ignoring Export Forwarding for now.
Export Ordinal Table
Picture taken from Matt Pietreks excellent
article REF.
19
Cant get all exports in one screen Shot !
Screenshot from the INDRA analyzer prototype
20
PE - DISASSEMBLER

• Imports Section
• There is One Import Directory Table (described in
  WINNT.H by IMAGE_IMPORT_DESCRIPTOR) for each
  imported binary module (DLL).
• The Import Directory Table consists of an array
  of Import Directory Entries, one entry for each
  DLL the image references.

21
PE - DISASSEMBLER

• Imports Section -

Import Address Table
Picture taken from Matt Pietreks excellent
article REF.
22
CONTENTS
You are here

• Code Analyzer
• PE - Disassembler
• Symbol Handler
• P-T-M Enumerator
• Debug Events
• Call Interceptor
• Hardware Breakpoints
• Virtual Address Space Viewer
• Memory Dumper
• Call Stack Viewer
• Locals Enumerator
• Single Stepping Mode
• Architecture Analyzer
• Processor Features
• Cache Configurations
• Hyperthreading Support
• Kernel Mode Device Driver

• Hardware Events
• Applications
• Compiler Optimizations
• Memory Allocators
• Future Directions
• References

23
SYMBOL HANDLER

• COMPLETE THIS PART !

24
DEBUG EVENTS

• COMPLETE THIS PART !

25
P-T-M Enumerator

• P-T-M Enumeration
• A debugger must be able to enumerate processes
  and threads (and the modules loaded in their
  address spaces) are currently running on the
  system.
• If sufficient privileges available, the debugger
  can attach to processes and start debugging them.
• In that case, debug symbols will almost always be
  unavailable.but that is for later ?

26
Screenshot from the INDRA analyzer prototype
27
CALL INTERCEPTOR

• COMPLETE THIS PART !

28
HARDWARE BREAKPOINTS

• COMPLETE THIS PART !

29
VIRTUAL ADDRESS SPACE VIEWER

• Address Spaces for 32 bit processes
• Total 4 GB Virtual Address Space
• 0x00000000 to 0xFFFFFFFF
• Only lower 2 GB available to a process
• 0x00000000 to 0x7FFFFFFF
• Upper 2 GB reserved by Windows
• 0x80000000 to 0xFFFFFFFF
• 65,536 bytes Guard Region

30
VIRTUAL ADDRESS SPACE VIEWER

• State of an address FREE
• This is the initial State of an address.
• An address in FREE state can be
• RESERVED for future use
• COMMITTED to memory
• Any attempt to access an address in FREE state
  results in an Access Violation Exception

31
VIRTUAL ADDRESS SPACE VIEWER

• State of an address RESERVED
• Reserves a FREE address range until needed,
  protecting them from other allocation requests.
• Reserving is mainly done to prevent the Windows
  Loader from loading a DLLs or resources into the
  specified addresses.
• Any attempt to access an address in FREE state
  results in an Access Violation Exception.

32
VIRTUAL ADDRESS SPACE VIEWER

• State of an address RESERVED
• Reserving does not mean that these addresses are
  mapped to physical memory. No pages of physical
  memory are committed in this process.
• Reserving a range of addresses is no guarantee
  that at a later time there will be physical
  memory available to commit to those addresses.
• Reserving only makes an entry into the process's
  virtual address descriptor (VAD) tree.
• At present I have not idea how windows implements
  
• this !

33
VIRTUAL ADDRESS SPACE VIEWER

• State of an address COMMITTED
• To use any address, it must be in a COMMITTED
  state, i.e. it must be mapped to physical memory.
• For Committing Memory,
• Minimum Size -gt 1 Page Size (4 KB ?)
• Maximum Size -gt Biggest contiguous range
  available
• Committed memory can have,
• Page Read Only Access
• Page Read Write Access
• Page No Access

34
VIRTUAL ADDRESS SPACE VIEWER

• State of an address COMMITTED

35
VIRTUAL ADDRESS SPACE VIEWER

• Protection for Committed addresses-

36
VIRTUAL ADDRESS SPACE VIEWER
Page Frame

• Paging Mechanism

0
0
Page Table
Page Frame
1023
Page Directory
0
Page Table
Page Frame
1023
Page Frame
1023
1024 4 KB 4 MB
1024 4 MB 4 GB
4 KB
37
VIRTUAL ADDRESS SPACE VIEWER

• Translation of Virtual Addresses -
• Let us do an example address 0x186F103A.
• In Binary 00011000 01101111 00010000 00111010
• We need to split the address as follows
• So we get - 00011000 01101111 00010000 00111010
• All 10-bit offsets to be padded with lower order
  zeros
• 0001 1000 0100
• 1011 1100 0100
• 0000 0011 1010

Padded with zeros
38
VIRTUAL ADDRESS SPACE VIEWER

• Paging Example

Page Table
Location of Page Frame
Location of Page Table
Page Directory
Page Frame
0001 1000 0100
1011 1100 0100
0000 0011 1010
Offset into Page Directory
Offset into Page Table
Offset of referred byte in frame
Physical address not available to user !
Virtual address available to user !
39
Screenshot from the INDRA analyzer prototype
40
MEMORY DUMPER

• Hex Dump of Memory locations
• A Memory location can be read only if a virtual
  address in mapped to that physical location (i.e.
  COMMITTED).
• Also it must have a READ protection enabled
  (either
• PAGE_READWRITE, PAGE_EXECUTE_READWRITE,
  PAGE_EXECUTE_READ, PAGE_READONLY).
• Else an access violation exception will be
  generated !

41
Screenshot from the INDRA analyzer prototype
42
CALL STACK VIEWER

• Function Call Mechanism
• Consider the caller code ..
• .
• .
• .
• Callee ( a, b, c, d ) // calls the callee
• .
• .
• Consider the callee code
• void callee ( int i, double d, char c, int pi )
• // local variables
• int x
• Char y

43
CALL STACK VIEWER

• Function Call Mechanism
• Control is with the caller.

Return Address on stack
Arguments pushed on stack
Initially
ESP
ESP
ESP
44
CALL STACK VIEWER
PUSH EBP MOV EBP, ESP SUB ESP, 0x5

• Function Call Mechanism
• Control goes to callee. (the prologue begins)

Base (Frame) pointer goes on stack
ESP copied into EBP
Locals on stack
ESP
EBP
EBP
ESP
ESP
45
CALL STACK VIEWER
MOV ESP, EBP POP EBP

• Function Call Mechanism
• Control still with callee. (the epilogue begin)

ESP decremented.
EBP restored by popping stack
ESP
EBP
ESP
46
CALL STACK VIEWER

• Function Call Mechanism
• Control returns to caller.

EIP used to return, now popped
Done !
ESP
ESP
47
CALL STACK VIEWER

• Calling Conventions -

Modified from REF
48
CALL STACK VIEWER

• Frame Pointer Omission
• Methods to walk the stack ?
• Manually, using the call mechanism just described
• Using Debug Symbols (.pdb files)
• Why the latter is preferred ?
• Compiler optimizations can force frequently
  accessed variables onto registers instead of the
  stack.
• In some cases, the compiler might not generate
  the stack frame at all.
• Compiler generates FPO data during such
  optimizations
• Lookup WINNT.H for FPO structures.

49
Stack Trace in reverse order
Function calls
50
LOCALS ENUMERATOR

• COMPLETE THIS PART !

51
SINGLE STEPPING MODE

• COMPLETE THIS PART !

52
IA-32 PROCESSOR FEATURES

• Intel CPU Identification -
• Originally, Intel published small code snippets
  that could detect architectural implementation.
• Later, with the advent of the i386 processor,
  Intel implemented processor signature
  identification that upon reset provided the
  following
• processor family,
• Model encoding,
• stepping numbers.
• With the Pentium Family onwards the CPUID
  instruction provides not only the processor
  signature, but also provides information on
  processor supported features.

53
IA-32 PROCESSOR FEATURES

• Processor Signature
• 32 bit signature with 6 fields.
• Execute CPUID (opcodes 0x0F, 0xA2) with EAX
  0x1,
• The Processor Signature is returned in EDX

Processor Family
EAX
Revision Number of Processor Model
Model of Processor within Processor Family
54
IA-32 PROCESSOR FEATURES

• Brand ID
• Is an extension to the processor signature.
• Only supported P3 family upwards (check Manual
  REF).
• Execute CPUID with EAX 0x1, the Brand Id is
  returned in EBX70
• Brand String
• 96 bit ASCII String indicating ID Max Operating
  Freq.
• Only supported P4 family upwards (check Manual
  REF).

CPUID with
55
IA-32 PROCESSOR FEATURES
Brand ID and Brand String
Processor signature
Screenshot from the INDRA analyzer prototype
56
IA-32 PROCESSOR FEATURES

• Feature Flags Extended Feature Flags
• Complete list of processor supported features.
• Execute CPUID with EAX 0x1, the Feature Flags
  are dumped into EDX, ECX registers.
• If corresponding bit is set, feature is
  available.
• Similarly, executing CPUID with EAX 0x80000001,
  the Extended Feature Flags are placed in EDX.
• Some of the feature flags of interest are

Extended Feature Flag
57
Processor supported Features
Feature Flags Extended Feature Flags
Screenshot from the INDRA analyzer prototype
58
IA-32 CACHE CONFIGURATION

• Cache TLB Configuration
• When CPUID executed with EAX 0x2 ---gt
• EAX, EBX, ECX, EDX loaded with 8-bit Cache, TLB
  descriptors.
• Descriptor Decode Values are available in Intel
  Manual Ref

Move sequentially from MSB to LSB for all
descriptors
59
IA-32 CACHE CONFIGURATION

• TLB Buffers
• Translation Lookaside Buffer is nothing but a
  cache buffer.
• Frequently accessed Virtual addresses and their
  associated page frames numbers are stored in the
  TLB.
• TLB flushes on context switches.
• Trace Caches
• After instructions are retired from the execution
  pipeline, they are stored in a trace cache.
• If program counters access that instruction
  frequently,
• It is reloaded from the trace cache
• This action reduces the pre-fetch bandwidth.

60

• Get configuration of
• L1 Data Cache
• L1 Instruction Cache
• L2 Unified Cache
• L3 Unified Cache
• Trace Cache
• Data TLB
• Instruction TLB
• Prefetch Buffer

Screenshot from the INDRA analyzer prototype
61
HYPERTHREADING SUPPORT

• We have currently defined four status flags for
  Hyperthreading support
• Hyper Threading NOT SUPPORTED
• EDX28 i.e. hyperthreading bit is disabled. Host
  processor does not have the facility to support
  hyperthreading.
• Hyper Threading NOT DETECTED
• If (process affinity mask ! system affinity
  mask), then it means that some physical
  processors have been disabled. Hence
  hyperthreading cannot be detected.
• Hyper Threading DISABLED
• If (logical processor count 1), then
• Hyperthreading disabled by system BIOS.
• Hyper Threading ENABLED
• Hyper Threading Supported and Enabled.

62
On a Uniprocessor
System Process Affinity
Hyperthreading disabled
Only one physical processor
Screenshot from the INDRA analyzer prototype
63
On Hyperthreaded processor
System Process Affinity
Hyperthreading enabled
two logical processors per physical processor core
Screenshot from the INDRA analyzer prototype
64
KERNEL MODE DEVICE DRIVER

• COMPLETE THIS PART !

65
CONTENTS
You are here

• Code Analyzer
• Disassembler
• Symbol Handler
• P-T-M Enumerator
• Debug Events
• Call Interceptor
• Hardware Breakpoints
• Virtual Address Space Viewer
• Memory Dumper
• Call Stack Viewer
• Locals Enumerator
• Single Stepping Mode
• Architecture Analyzer
• Processor Features
• Cache Configurations
• Hyperthreading Support
• Kernel Mode Device Driver

• Hardware Events
• Applications
• Compiler Optimizations
• Memory Allocators
• Future Directions
• References

66
REFERENCE MATERIAL

• COMPLETE THI PART !