Running Xi-Batch on Solaris, AIX and HP-UX

Xi-BatchXi-Batchconfigurationnetworkingplatformsbtcichangebtcilistbtstart

You are installing Xi-Batch on a platform you have not run it on before, or moving a scheduler from one platform to another. The commands, options and file layout are the same everywhere; what differs is how the operating system starts services at boot, how it accounts for users, and where it keeps the shells and interpreters that jobs are run by. This article covers those differences and the settings a platform change makes you revisit.

The product is installed as a binary built for one platform, and a configuration moves between platforms as text produced by the conversion tools, never as a copy of the spool directory's state files.

The platforms a build is issued for

These are the platforms a system may be registered against on the customer portal, and therefore the platforms for which a build can be issued:

Linux
Debian 12 and 13, Rocky Linux 8 and 9 (and other Red Hat rebuilds), SUSE Linux Enterprise 15 / openSUSE Leap, Ubuntu 22.04 and 24.04, on x86_64; Debian 13 and Rocky Linux 9 also on aarch64

Solaris
Solaris 10 and 11 on SPARC, Solaris 11 on x86_64

AIX
AIX 7.1, 7.2 and 7.3 on POWER

HP-UX
HP-UX 11.23 and 11.31 on Itanium, HP-UX 11.23 on PA-RISC

Older releases of these operating systems, and platforms retired years ago, are served from the archive of historical builds. Where a platform is not offered when you register a system on the portal, there is no current build for it - ask Xi Software before planning a migration onto it.

Obtaining the right build

Builds are downloaded from the customer portal. Register the machine as a system, choosing its architecture, operating system and version; the portal then lists the builds that match it. Once a system is registered its platform fields are fixed, so a machine that changes platform is registered afresh rather than edited.

On Linux the product is supplied as an RPM or a Debian package, according to the distribution family. On Solaris, AIX and HP-UX it is supplied as a tar.gz containing an interactive installer.

Directory layout

The default directories

The compiled-in defaults are the same on every platform:

• Spool directory: /var/spool/xi/batch
• Internal programs: /usr/libexec/xi
• User commands: /usr/local/bin

Installations that use /usr/spool were placed there when /usr/spool was the convention, on Linux as much as on the traditional Unix platforms. The location is a property of the installation. Read the installation's own master configuration file, /etc/xi/batchconfig.

Choosing different locations

The packaged installations use the defaults above. The tar.gz installer asks whether you want to change the location of the main directories, and then whether to base them all on a common directory. There is no command-line option for this - the installer takes no arguments and asks its questions interactively.

Where the user commands go

The user commands are installed in /usr/local/bin. Ensure that directory is on the PATH of everyone who uses the product, and of any startup script that calls btstart or btquit - several platforms omit it from root's default PATH.

Accounts

The batch account

The installation creates the system user. The packaged installations do it in their pre-install step, and the tar.gz installer prints "I need to install user ..." and asks to continue, then creates it with the first free user id from 51. The Xi-Batch user is batch.

Because the id is chosen from whatever is free on each host, the same account usually carries different numeric ids on different machines. The generated restore scripts name users and groups by name, so what matters on a target machine is that the account exists.

If you must create the account yourself before installing - on a host where accounts come from a directory service, for example - create it with the name the product expects and a home directory under the spool area, then run the installation.

Names come from the local files

Xi-Batch builds its own table of user names and ids by reading /etc/passwd and /etc/group directly, line by line, without the system's name-service library. One consequence follows on a platform with centralised accounts:

NIS, LDAP and other directory services are not consulted. A user who exists only in a directory service is not found in that table, and is displayed and exported by number instead of by name, as u1234. Nothing fails; the number appears everywhere the name would, including in the scripts the conversion tools generate.

For names to appear, the accounts that use the product must be present in the local /etc/passwd on the machine running it.

Shells and command interpreters

A job is run by an interpreter, and the interpreters live at different paths on different platforms.

Default shells by platform

Linux
/bin/bash typically default

Solaris
/bin/sh (POSIX shell) or /bin/bash

HP-UX
/bin/sh (POSIX), /bin/ksh available

AIX
/bin/ksh (Korn shell) typically default

The standard shell

Xi-Batch uses a standard shell of its own, fixed when the build is made. Override it for the installation by setting SHELL in the master configuration file, /etc/xi/batchconfig - the first setting to check where jobs fail with a "not found" error after a platform change.

The command interpreter table

An Xi-Batch job names a command interpreter, and the interpreter's path is held in the interpreter table, not in the job. When the platform changes and an interpreter has moved - /bin/ksh on one system, /usr/bin/ksh on another - correct the table with btcichange and every job that names that interpreter is corrected with it. List the current table with btcilist:

btcilist

The job is handed to the interpreter the table names for it, and a "#!" line inside the job script takes no part in that. Keep the interpreter names the same on every platform and let the paths differ, so that the same job runs everywhere and only the table changes.

Restore or correct the interpreter table before starting the scheduler. btcichange edits the interpreter file directly and a running scheduler picks up a change without a restart, so the requirement is not about the scheduler noticing. It is about the job file: the scheduler loads the saved jobs straight after the interpreter list, and any saved job naming an interpreter that is missing at that moment is silently reset to the first entry in the table. Once that has happened, correcting the table afterwards does not put the jobs back.

The generated scripts

The scripts produced by the conversion tools begin "#! /bin/sh" and use POSIX constructs only, so they replay on every supported platform. Run them with sh; bash is not present by default on stock AIX or HP-UX.

IPC facilities

What a running scheduler uses

A running Xi-Batch uses:

• System V shared memory - for the job queue and the variable list. A further segment appears briefly while a segment is being grown.
• File locks - advisory record locks taken with fcntl() on lock files in the spool directory. No System V semaphores are used, except one set when a networked licence is in force.
• A Unix-domain socket for requests to the scheduler. No System V message queue is used.

All four platforms provide these. What differs is the limit each kernel places on shared memory.

Shared memory sizing

The size of the segments is decided by the job and variable counts given when the product is started, as in btstart -j 2000 -v 500. Asking for a large queue on a system with a low shared-memory limit is what causes a failure to start, and it is reported as a message naming the segment and the system error.

Where that happens, either reduce the requested counts or raise the kernel's shared-memory limit using the platform's own tuning mechanism. The default limits on current Linux kernels cover any realistic queue; the older Unix platforms are the ones where tuning is more often needed. Consult the platform's documentation for the current mechanism, which has changed between releases of every one of these operating systems.

Networking

The service entries

All platforms use /etc/services for service-to-port mapping. The installation adds its own entries if they are not already present. Xi-Batch adds:

xibatch         2050/tcp        # Connection port
xibatch         2050/udp        # Probe port
btq             2150/tcp        # Feeder port
xbnetsrv        2250/tcp        # External job submission
xbnetsrv        2250/udp        # Client access
xbapi           2260/tcp        # Xi-Batch API
xbapi           2260/udp        # Xi-Batch API

A site may change these, and the product takes the port from /etc/services, so every machine in a networked installation must agree. Copy the entries rather than retyping them, and open the firewall for the ones you use.

The hosts file

The network configuration lives in /etc/xi/batch-hosts. It names the other machines in the installation, and each name is resolved through the system resolver, so /etc/hosts, DNS or a mixture all work provided the answer is consistent on every machine.

Two constraints apply when this file is moved between machines:

• Host addresses are resolved as IPv4. A name that resolves only to an IPv6 address will not be usable.
• The file may begin with a localaddress line giving the machine's own address, and if present it must be the first non-comment line in the file. A localaddress line further down is ignored and the file is flagged as having errors.

Keep one master list of the other hosts and prepend each machine's own localaddress line, so the files differ by that one line from machine to machine.

Terminal handling

The character-mode screen, btq, is a curses program and uses whichever terminal database the platform's curses library was built against, terminfo on current systems and termcap on older ones. Ensure a definition exists for the terminal types in use, and set the TERM environment variable appropriately - TERM=vt100 or TERM=wyse60 for a physical terminal, TERM=xterm or TERM=screen for an emulator, and whatever the client sends over SSH.

A TERM value the target platform does not have a definition for causes display corruption in btq. Terminal databases differ between platforms, so a TERM setting that works on Linux may have no entry on AIX or HP-UX.

File system limits

Maximum path length

The product falls back to a limit of 1024 characters where the platform does not define one. Spool file names and job work directories are all built within that limit, so a deeply nested base directory reduces the room left for the rest.

Maximum job size

Job sizes and file positions are held as 32-bit quantities in the saved formats, so a single job is limited to just under 2 GB. This is a property of the format and applies equally on a 64-bit build and a 64-bit platform.

File locking

Locking uses fcntl() advisory record locks, which every supported platform provides. BSD-style flock() is not used.

NFS considerations
Advisory record locking over NFS depends on a working lock manager on both sides and behaves inconsistently in practice. Keep spool directories on local file systems.

File system types

Local file systems (ext4, xfs, zfs, jfs2, vxfs)
Suitable for spool directories

Network file systems (NFS, SMB/CIFS)
Unsuitable, because of the locking behaviour above

Temporary file systems (tmpfs, ramfs)
Unsuitable - the queue would be lost at every reboot

Locale

The scheduler and its command-line tools do not set a locale, so they run in the C locale whatever LANG or LC_ALL are set to. A job's own environment is a separate matter: what a job's script does with LANG is between the script and the programs it runs, and the scheduler takes no part in it.

Starting at boot

What the installation sets up

The RPM and Debian packages install a systemd unit and enable it. The unit runs the product's own start and stop commands:

ExecStart=/usr/local/bin/btstart -j 2000 -v 500 -l 10000
ExecStop=/usr/local/bin/btquit -y

Adjust the numbers on the ExecStart line to size the queue for the site, then reload systemd.

The tar.gz installer detects the platform's startup mechanism and writes the appropriate file:

systemd (current Linux)
a unit file, which the installer also enables

Upstart (older Ubuntu)
a job file under /etc/init

LSB and System V init (older Linux, Solaris)
/etc/init.d/xibatch plus rc links, registered with update-rc.d or chkconfig where available

AIX
a line appended to /etc/inittab, naming the start command and the run level

HP-UX
the installer prints a message that it does not know how to handle HP-UX startups and installs nothing. Add the startup by hand as below.

Verify after installation that the scheduler starts on boot, by rebooting a test machine.

Adding startup by hand

On a System V style system:

# Create startup script
cat > /etc/init.d/xibatch << 'EOF'
#!/bin/sh
case "$1" in
start)
    /usr/local/bin/btstart -j 2000 -v 500 -l 10000
    ;;
stop)
    /usr/local/bin/btquit -y
    ;;
esac
EOF

chmod 755 /etc/init.d/xibatch
ln -s /etc/init.d/xibatch /etc/rc3.d/S80xibatch
ln -s /etc/init.d/xibatch /etc/rc0.d/K01xibatch

Use the run level the system boots to - the initdefault entry in /etc/inittab - and start the product late (S80 above) so that the network is already up.

Those paths are the Solaris and older-Linux ones. On HP-UX the startup directories are under /sbin: write the script as /sbin/init.d/xibatch and link it from /sbin/rc3.d and /sbin/rc0.d. HP-UX is the platform the installer leaves alone, so this is the case most often done by hand.

On a systemd system, if you are adding a unit rather than taking the packaged one:

# Create unit file
cat > /etc/systemd/system/xibatch.service << 'EOF'
[Unit]
Description=Xi-Batch Job Scheduler
After=network.target

[Service]
Type=forking
ExecStart=/usr/local/bin/btstart -j 2000 -v 500 -l 10000
ExecStop=/usr/local/bin/btquit -y
Restart=on-failure
TimeoutStopSec=330
KillMode=mixed

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl enable xibatch
systemctl start xibatch

Type=forking is required: btstart returns once the daemon is running. TimeoutStopSec is set high because a graceful stop syncs the spool before releasing the shared memory, which takes longer than systemd's default allows; a scheduler killed part way through leaves its shared memory segments behind.

Performance

Throughput is dominated by the storage holding the spool directory and by the work the jobs themselves do. Two things follow:

• Local storage always outperforms network storage for spool directories, and network storage brings the locking problems described above as well.
• Shared memory is sized once at startup, from the job and variable counts on the start command. Growing a segment while running is supported and involves a copy, so size it for the site's busiest day.

Where a platform needs tuning, it shows up as a failure to allocate shared memory at startup.

What a platform change requires

Two things are properties of the machine and stay with it, whatever the platform.

The licence. It is keyed to the machine's hardware signature and to the word size of the build. Renaming a machine leaves a working licence; a move to different hardware needs new codes, as does replacing a 32-bit installation with a 64-bit one on the same machine. Register the new system on the portal and obtain codes before the cut-over - Xi Software confirms the registration as a separate step, and a trial licence covers the interval.

The saved state. The job, variable and user files in the spool directory are raw memory images in the machine's own byte order and word size. The scheduler reads such a file back as it stands, so a copy taken to a machine of a different byte order or word size loads without an error and produces an unusable queue. Use the conversion tools on the source machine and carry their text output; they apply plausibility checks and usually refuse a file from a foreign machine with "I am confused about the format of your job file".

Stop the product before exporting. The saved files are rewritten only every few minutes, so an export taken from a running system can be several minutes behind the queue. Replay the generated scripts on a non-production machine of the target platform before the cut-over.

The model behind both is set out in the concept article on what travels between machines; the export and restore procedure is in the Xi-Batch migration article.

Related articles

All articles · Release notes · Contact support