HP XC System Software: XC Installation Guide Version 3.2.1

HP Part Number: A-XCINS-321B

Published: November 2007

Abstract

This document describes how to install and configure HP XC System Software Version 3.2.1 on HP Cluster Platforms 3000, 4000, and 6000.

Table of Contents

About This Document

Intended Audience
How to Use This Document
Naming Conventions Used in This Document
New and Changed Information in This Edition
Typographic Conventions
HP XC and Related HP Products Information
Related Information
Manpages
HP Encourages Your Comments

1 Preparing for a New Installation

Task 1: Read Related Documentation

Task 2: Plan for Future HP XC Releases

Task 3: Prepare Existing HP XC Systems

Task 4: Prepare the Hardware

Task 5: Verify Firmware Versions

Task 6: Arrange for IP Address Assignments and Host Names

Task 7: Have the License Key File Ready

Task 8: Purchase Additional Software from HP and Third-Party Vendors

Task 9: Plan a Service Availability Strategy

What Is Improved Availability?
How to Configure Improved Availability
Choosing an Availability Tool
Writing Translator and Other Supporting Scripts
Choosing Nodes as Members of Availability Sets
Assigning Node Roles For Improved Availability
Using the Improved Availability Planning Worksheet

You Are Done

2 Installing Software on the Head Node

Software Installation Overview

Kickstart Installation Process
HP XC Software Stack
Kickstart Installation File
Default File System Layout and Disk Partition Sizes
4 GB /hptc_cluster Partition Size Limit Might Be Too Small For Some Hardware Configurations

Task 1: Gather Information Required for the Installation

Task 2: Start the Installation Process

Installation Procedure For A Non-Blade Server Head Node
Installation Procedure For A Server Blade Head Node

Task 3: Install Additional RPMs from the HP XC DVD

Enable HP XC To Run Serviceguard
Run the SVA Installation Script to Install SVA
Install Optional Linux RPMs

You Are Done

3 Configuring and Imaging the System

System Configuration and Imaging Overview

System Configuration Process
Internal Node Naming
System Imaging Process
System Configuration and Imaging Log Files

Task 1: Prepare for the System Configuration

Task 2: Change the Default IP Address Base (Optional)

Task 3: Run the cluster_prep Command to Prepare the System

Task 4: Install Patches or RPM Updates

Download and Install Patches

Task 5: Run the discover Command to Discover Hardware Components

Discovering Non-Blade Hardware Configurations
Discovering Hardware Configurations With Server Blades and More Than One Enclosure
Discovering Hardware Configurations With Server Blades And A Single Enclosure
Discovering Hardware Configurations To Plan For Future Compute Node Expansion
Modify the Default Password for HP ProLiant DL140 and DL145 Hardware Models

Task 6: Set Up the System Environment

Put the License Key File in the Correct Location (Required)
Configure Interconnect Switch Monitoring Line Cards (Required)
Configure sendmail (Required)
Customize the Nagios Environment (Required)
Configure Access to the Console Port on the Head Node (Required)
Set the BMC Password on HP Integrity Systems (Required)
Install Additional Software From HP or Third-Party Vendors
Create the /hptc_cluster File System
Modify Workstation Model Names in the Database
Enable Software RAID-0 or RAID-1 on Client Nodes
Create Local User Accounts
Override Default User and Group Account IDs
Customize Client Node Disk Partitioning
Create the HP Modular Cooling System Configuration File
Mount Network File Systems
Update initrd Files With Required Hardware

Task 7: Run the cluster_config Utility to Configure the System

Task 8: Configure Availability Sets

Task 9: Modify and Assign Node Roles

Task 10: Respond to Configuration Questions

Task 11: Edit the /etc/dhcpd.conf File

Task 12: Run the startsys Utility to Start the System and Propagate the Golden Image

Task 13: Perform Postconfiguration Tasks for the InfiniBand Interconnect

Task 14: Create a Lock LUN Device File

Task 15: Start Availability Tools

Task 16: Configure SNMP Trap Destination for Enclosures

Task 17: Configure SNMP Trap Destination for Modular Cooling System Devices

Task 18: Finalize the Configuration of Compute Resources

Perform SLURM Postconfiguration Tasks
Perform LSF Postconfiguration Tasks

Task 19: Generate an SVA Site Configuration File

You Are Done

4 Verifying the System and Creating a Baseline Record of the Configuration

Task 1: Verify the LSF Configuration

Verify LSF-HPC with SLURM
Verify Standard LSF

Task 2: Verify Availability Tools

Task 3: Run the OVP to Verify Software and Hardware Components

Task 4: Run the SVA OVP Utility

Task 5: View System Health

Nagios Web Interface
The nrg Command

Task 6: Create a Baseline Copy of the Database

Task 7: Create a Baseline Report of the System Configuration

You Are Done

5 Upgrading Your HP XC System To This Release

Software Upgrade Overview

Supported Upgrade Paths
Is Upgrading Appropriate for Your System Configuration?
Upgrade Characteristics
Upgrade Commands

Task 1: Prepare for the Upgrade

Task 2: Prepare the System State

Task 3: Install the Upgrade RPM and Prepare the System

Task 4: Upgrade Linux and HP XC RPMs

Task 5: Reboot the Head Node and Verify the New Kernel

Task 6: Install Patches and Reinstall Additional Software

Task 7: Manually Merge File Customizations

Task 8: Configure the System and Propagate the Golden Image

Task 9: Image and Boot the System and Start Compute Resources

Task 10: Start Availability Tools After the Upgrade

Task 11: Verify the Upgrade

6 Reinstalling HP XC System Software Version 3.2.1

Reinstalling Systems with HP ProLiant Hardware Models

Reinstalling the Entire System
Reinstalling One or More Nodes

Reinstalling Systems with HP Integrity Hardware Models

Reinstalling the Entire System
Reinstalling One or More Nodes

7 Installing and Using PBS Professional

PBS Professional Overview

Before You Begin

Plan the Installation

Perform Installation Actions Specific to HP XC

Configure PBS Professional under HP XC

Configure the OpenSSH scp Utility
Remove Nodes from the SLURM or LSF Configuration
Add Nodes to the PBS Professional Configuration

Replicate Execution Nodes

Enter License Information

Start the Service Daemons

Set Up PBS Professional at the User Level

Run HP MPI Tasks

8 Installing the Maui Scheduler

Maui Scheduler Overview

Readiness Criteria

Preparing for the Installation

Installing the Maui Scheduler

Task 1: Download the Maui Scheduler Kit
Task 2: Compile the Maui Scheduler from Its Source Distribution
Task 3: Update the Maui Scheduler Configuration File
Task 4: Edit the SLURM Configuration File
Task 5: Configure the Maui Scheduler

Verifying the Successful Installation of the Maui Scheduler

9 Adding Visualization Nodes to An Existing HP XC System

Prerequisites

Installation Scenarios

New Visualization Nodes Exceed the Maximum Number of Nodes Supplied to the cluster_prep Command
New Visualization Nodes Do Not Exceed the Maximum Number of Nodes Supplied to the cluster_prep Command
Graphics Cards Have Been Added to Existing Nodes

10 Troubleshooting

Troubleshooting the Discovery Process

Discovery Process Hangs While Discovering Console Ports
ProCurve Switches Do Not Obtain Their IP Addresses
ProCurve Switches Can Take Time to Get IP Addresses
Not All Console Ports Are Discovered
Some Console Ports Have Not Obtained Their IP Addresses
Not All Nodes Are Discovered

Troubleshooting the Cluster Configuration Process

lsadmin limrestart Command Fails
Cannot Connect to Database During Configuration

Troubleshooting the Imaging Process

/hptc_cluster File System Does Not Mount
Client Node or Nodes Do Not Network Boot
How To Monitor An Imaging Session

Troubleshooting LSF and Licensing

Troubleshooting the OVP

OVP network_bidirectional Test Might Report False Error on HP Server Blades
OVP Reports Benign Nagios Warnings
OVP qsnet_database Test May Fail Due to Benign Errors Returned By the qsctrl Utility

Troubleshooting SLURM

SLURM Reconfiguration Errors

Troubleshooting the Software Upgrade Procedure

The upgradesys Utility Might Fail While Backing Up the Database
The hptc-ire-serverlog Service Might Not Start
External Ethernet Connection Fails To Come Up

A Installation and Configuration Checklist

B Host Name and Password Guidelines

Host Name Guidelines
Password Guidelines

C Enabling telnet on iLO and iLO2 Devices

iLO Devices
iLO2 Devices

D Configuring Interconnect Switch Monitoring Cards

Configure Quadrics Switch Controller Cards

Configure Myrinet Switch Monitoring Line Cards

Configure InfiniBand Switch Controller Cards

Configure Voltaire InfiniBand Switches
Configure Cisco InfiniBand Switches

E Customizing Client Node Disks

Overview of Client Node Disk Imaging

Dynamically Configuring Client Node Disks

Component Files Required for Dynamic Configuration of Client Node Disks
Example 1: Modifying Partitions Using Fixed Sizes and Defining an Additional Partition
Example 2: Changing Default Partition Sizes and Swap Space for All Client Nodes
Example 3: Customizing Partition Sizes on a Group of Client Nodes
Example 4: Customizing Partition Sizes to Maximize File System Performance

Statically Configuring Client Node Disks

Enable Static Disk Configuration
Customize Client Disk Configuration

F Description of Node Roles, Services, and the Default Configuration

Default Node Role Assignments

Special Considerations for Modifying Node Role Assignments

General Considerations
Special Considerations for Hardware Configurations with 63 or Fewer Nodes
Special Considerations for Hardware Configurations with 64 or More Nodes
Special Considerations for Improved Availability

Role Definitions

Availability Role
Avail_node_management Role
Cisco_hsm Role
Common Role
Compute Role
Console_network Role
Disk_io Role
External Role
Login Role
Management Hub Role
Management Server Role
NIS Server Role
Node Management Role
Resource Management Role
Voltaire_hsm Role

G Using the cluster_config Command-Line Menu

Overview of the cluster_config Command-Line Menu

Displaying Node Configuration Information

Modifying a Node

Configuring an Ethernet Connection

Modifying Node Role Assignments

Analyzing Current Role Assignments Against HP Recommendations

Customize Service and Client Configurations

Services Configuration Commands

H Determining the Network Type

I LSF and SLURM Environment Variables

J Customizing the SLURM Configuration

Assigning Features
Creating Additional SLURM Partitions
Required Customizations for SVA

K OVP Command Output

L upgraderpms Command Output

Glossary

Index

List of Figures

10-1 Discovery Flowchart

List of Tables

1 Installation Types
2 Naming Conventions
1-1 Improved Availability Summary
1-2 Role and Service Placement for Improved Availability
1-3 Availability Sets Worksheet
2-1 HP XC Software Stack
2-2 Default Values in the ks.cfg File
2-3 Default Disk Partition Layout on the Head Node
2-4 Criteria for 4 GB /hptc_cluster Partition
2-5 Chip Architecture by Cluster Platform
2-6 Information Required for the Kickstart Installation Session
2-7 Kickstart Boot Command Line
2-8 Additional Boot Command Line Options For Specific Server Blade Hardware Models
3-1 Information Required by the cluster_prep Command
3-2 Information Required by the discover Command
3-3 Information Required by the cluster_config Utility
3-4 discover Command Options
3-5 System Environment Setup Tasks
3-6 HP XC Default User and Group Account IDs
3-7 Default Client Node Partition Layout
3-8 Number of NFS Daemons Based on System Size
3-9 LSF-HPC with SLURM and Standard LSF Features
3-10 startsys Command-Line Options Based on Hardware Configuration
5-1 Supported Upgrade Paths
5-2 Upgrade Characteristics
5-3 Commands Used During the Upgrade Process
5-4 Upgrade Readiness Criteria
5-5 Files Containing User Customizations
5-6 Upgrade Options for the cluster_config Utility
5-7 Responding to cluster_config Prompts During an Upgrade
5-8 Upgrade startsys Command-Line Options Based on Hardware Configuration
8-1 Maui Scheduler Readiness Criteria
8-2 Maui Scheduler Diagnostic Commands
10-1 Diagnosing System Imaging Problems
10-2 Software Upgrade Log Files
A-1 Installation and Configuration Checklist
D-1 Quadrics Switch Controller Card Naming Conventions and IP Addresses for Reduced Bandwidth
D-2 Quadrics Switch Controller Card Naming Conventions and IP Addresses for Full Bandwidth
D-3 Myrinet Switch Controller Card Naming Conventions and IP Addresses
D-4 InfiniBand Switch Controller Card Naming Conventions and IP Addresses
D-5 InfiniBand Switch Controller Card Naming Conventions and IP Addresses For Hardware Configurations With HP Server Blades and Enclosures
F-1 Default Role Assignments Based on Number of Total Nodes
G-1 Description of cluster_config Command-Line Menu Options
G-2 First Portion of cluster_config Analysis Option
G-3 Second Portion of cluster_config Analysis Option
G-4 Service Configuration Command Descriptions
H-1 Network Type Based on System Topology
I-1 Default Installation Values for LSF and SLURM

List of Examples

1-1 Sample XC.lic File
3-1 Default Content Of The base_addr.ini File
3-2 Default Content Of The base_addrV2.ini File
3-3 discover Command Output On A Small Non-Blade Configuration
3-4 discover Command Output For Large-Scale Systems
3-5 Sample mcs.ini File
3-6 Script To Update initrd Files With Hardware Required For Booting
5-1 Resolved Dependencies in yum.upgrade.log File
G-1 First Portion of cluster_config Analysis
G-2 Second Portion of cluster_config Analysis

HP XC System Software: XC Installation Guide
Version 3.2.1

Technical documentation

» Table of Contents

» Glossary

» Index

HP XC System Software: XC Installation GuideVersion 3.2.1

Technical documentation

» Table of Contents

» Glossary

» Index

HP XC System Software: XC Installation Guide
Version 3.2.1