Troubleshooting Kernel, Patch, and Filesystem Issues in HP-UX

Details: Category: Operating Systems; By Mindful Chase; 08.Aug; Hits: 231

HP-UX, Hewlett-Packard's UNIX-based operating system, is a staple in many legacy enterprise data centers—particularly in banking, telecom, and manufacturing sectors. While it offers rock-solid stability, HP-UX systems are often opaque when it comes to troubleshooting complex issues like kernel panics, system hangs, patch conflicts, or file system corruption. These problems are further exacerbated by proprietary tools and aging hardware dependencies. This article addresses advanced troubleshooting methods for HP-UX environments, helping senior administrators and architects diagnose root causes and implement long-term solutions.

Mindful Chase

Writing Code, Writing Stories

tbd

Experience

tbd

More to Explore

Understanding HP-UX Architecture

Key System Components

PA-RISC or Itanium-based hardware architecture
HP-UX kernel with dynamically loadable modules
System Administration Manager (SAM) for legacy GUI-based config
Logical Volume Manager (LVM) for disk and partition management

Common Use Cases

HP-UX is widely used in environments requiring:

High availability with Serviceguard clusters
Proprietary application support (Oracle, SAP)
Backward compatibility with legacy hardware and software stacks

Typical Troubleshooting Scenarios

1. System Hangs or Kernel Panics

These often arise from faulty drivers, patch regressions, or kernel parameter misconfiguration. The /var/adm/crash directory and crashinfo tool provide core insights.

2. Performance Degradation

Common in I/O-bound workloads, especially with older HBA or SCSI drivers. Misconfigured kernel parameters like dbc_max_pct and maxdsiz also cause memory thrashing.

3. Filesystem Corruption

VxFS (Veritas File System) corruption can occur after unclean shutdowns or hardware faults. HP-UX systems often use fsck or vxdiskadm for recovery.

4. Patch Management Issues

Incorrect or incomplete patch application via swinstall can leave systems in inconsistent states, especially with kernel patch sets or Java dependencies.

5. LVM Volume Failures

HP-UX LVM can fail to activate volumes at boot due to stale disk metadata or lost quorum. This often manifests as vgchange -a y failures.

Root Cause Diagnostics

Collecting Crash Dumps

Use crashconf to ensure dump devices are configured, and crashinfo to analyze logs post-reboot.

crashconf -v
crashinfo -v /var/adm/crash/crash.0

Checking System Health

Use top, glance, or sar to monitor CPU, memory, and disk I/O bottlenecks.

Filesystem Integrity

For VxFS:

/sbin/fsck -F vxfs /dev/vg00/lvol3

Patch Verification

swlist -l patch | grep -i PHCO

Check for required patch dependencies and rollback if needed using swremove.

Step-by-Step Fix Strategy

1. Investigate Kernel Issues

Examine /var/adm/syslog/syslog.log for last kernel messages
Use kctune to review modified tunables

2. Restore Corrupted Volumes

Attempt volume activation manually: vgchange -a y vg00
Rebuild LVM headers if required using vgcfgrestore

3. Fix Patch Inconsistencies

Use swverify to validate installed patches
Rollback failed patches using swremove with the proper session ID

4. Tune Kernel Parameters

kctune maxdsiz=134217728
kctune dbc_max_pct=10

Restart is not required for most tunables in newer HP-UX 11i versions.

5. Automate System Recovery

Enable automatic crash dump capture and reboot:

crashconf -s ON
kctune dump_on_panic=1

Best Practices for HP-UX Operations

Regularly backup /etc/lvmtab, /stand, and kernel tunables
Use Ignite-UX to image and clone systems
Pin patch levels and test updates in staging before production
Use EMS (Event Monitoring System) to proactively monitor hardware faults
Harden SSH, disable legacy services (telnet, rexec), and apply latest security patches

Conclusion

HP-UX remains deeply embedded in many mission-critical systems, but its proprietary nature and aging ecosystem demand careful maintenance. Kernel panics, LVM issues, and patch mismatches are common pain points in large deployments. By understanding HP-UX's unique toolsets, system layout, and diagnostic mechanisms, administrators can efficiently troubleshoot and extend the life of their infrastructure while planning migrations or hybrid integrations with modern platforms.

FAQs

1. How do I check if crash dumps are enabled on HP-UX?

Use crashconf -v to verify dump device configuration. Also, ensure dump_on_panic is enabled via kctune.

2. What is the safest way to update patches?

Use swinstall with a test depot and always run swverify post-install. Avoid kernel patches unless fully validated in a QA system.

3. Why won't my volume group activate at boot?

This may indicate quorum failure or stale metadata. Try vgcfgrestore followed by vgchange -a y.

4. How do I monitor real-time performance?

Use glance for an interactive view or sar for historical performance data. Both are native to HP-UX.

5. Can I virtualize HP-UX?

Yes, using HP Integrity Virtual Machines or hardware partitioning (nPars/vPars), but compatibility is limited to specific PA-RISC and Itanium platforms.

Contact Us