IA64 Cluster Satellite Boot

1
IA64 Cluster Satellite Boot

• Ruth Goldenberg
• OpenVMS Engineering

2
Topics

• Alpha satellite booting
• IA64 disk booting
• Memory disk booting
• Satellite booting with memory disk
• Boot server requirements and setup
• Satellite setup
• Things to look out for

3
Alpha Satellite Booting
gtgtgt b Excnnn
MOP Request
4
Alpha Satellite Booting (cont.)

• MOP request by station ID (MAC address)
• MOP server locates boot information keyed by MAC
  address in LANCP or DECNET database.
• MOP server appends information to the boot image
• Server SCSNODE SCSSYSTEMID
• One true name of the satellites system disk
• Cluster group number
• Cluster password
• Satellites NISCS_CONV_BOOT
• Satellites NISCS_PORT_SERV

5
Alpha Satellite Booting (cont.)

• Boot driver
• Primitive device driver used to boot
• Until runtime driver is loaded
• Used to write crash dump
• Provided by either OpenVMS or console firmware
• Architected interface to boot driver for use by
  OpenVMS
• Boot drivers in APB include
• LAN boot drivers
• NISCA_BTDRIVER
• NISCA_BTDRIVER establishes a primitive connection
  to the cluster and locates the system disk.
• APB reads the other files required to boot from
  the served disk until the system is far enough
  along to switch to the real system disk device.

6
IA64 EFI Console

• Extensible Firmware Interface
• Designed without VMS input
• Provides rich environment for EFI applications
• TCP/IP
• Loadable drivers
• File system
• CLI and GUI
• Provides boot drivers but no OS interface to them

7
IA64 Disk Boot Sequence

• VMS_LOADER.EFI runs initially
• EFI application that starts the boot sequence
• Sets up a more VMS-like environment
• Reads in IPB.EXE from FAT partition on system
  disk and transfers control to it
• IPB.EXE - Itanium Primary Bootstrap
• Cannot access EFI firmware boot drivers
• Linked with VMS boot drivers for each boot device
  and primitive file system
• Loads SYSBOOT from system disk and transfers
  control
• SYSBOOT.EXE loads many OpenVMS execlets from
  system disk using boot driver and primitive file
  system

8
Difficulties Porting the Alpha Satellite Boot
Model

• Console speaks PXE, not MOP
• PXE is essentially a BOOTP/DHCP/TFTP sequence
• We could implement MOP, but
• Unless we did it all in EFI, we still need
  initial PXE boot and then we have multiple load
  images and multiple databases to hold the images.
• MOP would make us different
• BOOTP/DHCP/TFTP is handled by the IP products
• Many customers dont run the HP IP stack
• We dont own the boot loader
• Need parameters that the MOP loader provides
• Wed like to get out of the network boot driver
  business

9
Solution - Memory Disk Booting
VMS_LOADER allocates the memory disk and then
uses the firmware-supplied EFI drivers to read
the required files.
Memory Disk
120 files required to boot VMS
(SYSMEMORYDISK.DAT)
The boot driver is a simple memory disk driver
that reads files from the memory disk. When
were up and running with the appropriate
run-time drivers, we switch to the real system
device and its run-time drivers.
VMS system
10
Memory Disk Booting (cont.)

• In V8.3, used for
• USB DVD booting
• Satellite booting
• Relies on EFI drivers supplied with firmware or
  hardware
• VMS_LOADER reads and parses three system disk
  files telling it what other files to read from
  system disk and write to memory disk
• SYSMEMORYDISK.DAT
• VMSSYSTEM_IMAGES.DATA
• SYSUSER_MEMORYDISK.DAT
• VMS_LOADER reads IPB.EXE from system disk
• IPB uses memory disk boot driver to load SYSBOOT
  and other files from memory disk

11
Memory Disk Booting (cont.)

• SYSMEMORY_DISK.DAT
• Created by the build
• Text file listing files required for booting
• Non-image files must be flagged , text
• Optional files are flagged , optional to cut
  down on scary error messages
• SYSUSER_MEMORYDISK.DAT can be edited to add
  additional files to the memory disk, for example
• Phase V adds images to VMSSYSTEM_IMAGES
• but also requires the file NETCONFIG.DAT

12
Memory Disk Boot Flag

• APB distinguishes InfoServer boot from satellite
  boot by the presence of the FILE flag
• gtgtgt b fi APB_xxx Excn InfoServer boot
• gtgtgt b Excn satellite boot
• VMS_LOADER distinguishes by the presence of the
  memory disk boot flag. (0x200000)
• Not set InfoServer boot
• Set satellite boot
• When you add an entry for a network device to the
  boot menu using BOOT_OPTIONS.COM, it asks if you
  intend it for satellite boot and will set the
  memory disk flag

13
Satellite Booting with Memory Disk
PXE Request
full-file-path VMS_LOADER.EFI
VMS_LOADER parses the device and system root info
from the full path info provided when it was sent
from the server. It uses that to request
SYSMEMORYDISK, then parses each line, requesting
the files by TFTP. Each file is requested first
in SYSSPECIFIC, then SYSCOMMON. VMSSYSTEM_IMAGE
S.DATA is parsed and files listed there are
loaded via TFTP. SYSUSER_MEMORYDISK is parsed
and any additional files are requested. Last,
VMS_LOADER requests IPB via TFTP.
14
Satellite Booting (cont.)

• Boot device is the memory disk, used with the
  memory disk boot driver
• A BTADP structure is created for the remote
  system disk and marked hardware device
• IPB notices that were booted from a network
  adapter and memory disk is set and declares
  satellite boot
• System disk name and root are filled in from boot
  path
• System disk name must be fully qualified node
  or allocation class
• Boot path to disk is fixed up to say that its
  remote so the correct drivers will get loaded.
• Forces SCS to get loaded.
• System boot continues from the memory disk

15
Still Booting

• Execlets and device drivers are loaded from the
  memory disk
• SYSGEN parameters are read in from memory disk.
• NISCS_CONV_BOOT
• NISCS_PORT_SERV
• Driver initialization notes that the system
  device is not local and creates an anonymous
  system block and hangs the system disk off of it
• Some driver init is stalled until the device is
  actually located

16
Finding the System Disk

• Swapper waits for the system disk to be marked
  local valid
• SWAPPER-I-WAITDISK,
• PEDRIVER solicits other members with the same
  cluster number and password looking for the
  system disk
• When a node that can serve the device is found,
  the info is sent back to PEDRIVER on the
  satellite
• Anonymous system block becomes server nodes SB
• Device data is fixed up
• Stalled driver init continues
• Swapper proceeds
• SWAPPER-I-WAITDONE,

17
Boot Server

• Only V8.3 IA64 boot servers supported at this
  time
• Satellites system disk must be mounted
• Requires HP TCPIP V5.6
• 3rd-party stack support in the future
• Enable BOOTP serving via TCPIPCONFIG
• In theory, DHCP works as well, but there were
  some issues when we tested it
• BOOTP supports cluster-common BOOTP database
• Enable TFTP serving via TCPIPCONFIG
• CLUSTER_CONFIG_LAN enable boot server option
  points you to TCPIPCONFIG

18
Adding a Satellite

• CLUSTER_CONFIG_LAN
• CLUSTER_CONFIG calls _LAN for IA64 systems
• Usual questions
• DECnet-related information
• System device root
• MAC address
• New Questions
• TCP/IP address (default from bind database)
• Gateway address (optional)
• Network mask

19
A Satellite Needs a Local Disk

• Because memory disk is populated by EFI drivers,
  there is no boot driver for crashing and shutdown
• DOSD required for crash dumps
• DOSD also used for error log buffer dump
• ERRFMT was enhanced to find the error log dump
  buffers and place them into memory in proper
  order
• Therefore satellite node should have a local disk
• The local disk should be mounted via SYCONFIG.COM
• Edit SYCONFIG.COM on satellites system disk to
  mount the DOSD device
• Also strongly recommended for page and swap files

20
Pre-Boot Setup For the Satellite System

• If the system has a local disk and can boot from
  it, use SYSMANAGERBOOT_OPTIONS.COM to
• Add an entry to the boot menu. When you specify
  a network device, the procedure will ask you if
  its for satellite boot and ensure the memory
  disk boot flag (0x200000) is set
• Set up DOSD. BOOT_OPTIONS can help manage the
  DOSD disk list.
• You can also run BOOT_OPTIONS booted from the
  installation DVD. Choose the DCL option.
• Obtain the MAC address for the boot NIC
• Must be visible to EFI

21
Additional Setup For the Satellite System

• When you run CLUSTER_CONFIG, if you say yes to
  the question about local page and swap files, it
  will boot the satellite
• If booting by hand, dont forget the memory
  disk flag
• After the first AUTOGEN/reboot cycle, if you
  havent done so already
• Add an entry to the boot menu (BOOT_OPTIONS)
• Set up DOSD for dump and error log buffers
• Edit SYCONFIG.COM to mount the DOSD disk
• Required for error log buffers
• Required for a dump file

22
Things to Look Out For

• Satellite system disk name
• Disk name must be fully qualified cluster name
  nodeDxcnnn or nDxcnnn (unless its a
  shadowset member)
• CLUSTER_CONFIG_LAN handles this correctly, but at
  the cost of exposing the true device name behind
  logicals
• Protection
• TFTP is a user-mode application
• Execlets are wre by default. Be careful if you
  copy side images into SYSSPECIFIC
• IA64VMSSYS.PAR and CLUSTER_AUTHORIZE.DAT require
  the VMSSATELLITE_ACCESS identifier
• CLUSTER_CONFIG_LAN will handle initial protection

23
Things to Look Out For (cont.)

• IP-related pitfalls
• Server must be able to see boot request
• Network mask
• Gateway

24
Questions