prcmd(3N)

NAME

prcmd(), prcmd_init() — return streams to parallel remote commands

SYNOPSIS

#include <prcmd.h> 

void prcmd_init ( 
     struct prc_host *hostp, 
     int num_hosts, 
     int caller_status, 
     char *command, 
     time_t timeout 
); 

int prcmd ( 
     struct prc_host *hostp, 
     int num_hosts 
); 

Remarks

These functions reside in libdc, and are linked using the -ldc option to the ld or cc command.

DESCRIPTION

prcmd() is an interface similar to rcmd() that allows a client-side calling program to use an existing rcmd() server side daemon. prcmd(), however, performs in parallel across multiple systems, the time consuming steps associated with executing commands remotely (connection, command invocation, and command execution/interaction). With rcmd(), only command execution and interaction is under the control of the calling program, and it requires the calling program to perform its own select() calls in order to manage multiple (parallel) connections.

Features of the existing server-side remote-shell daemon (remshd) include: well known service, access control, and shell processing of the command line.

The calling program passes prcmd() a pointer (hostp) to the first node in a list of hosts for which to initiate, or continue, remote command processing via Internet socket connections. It also supplies the number of hosts in the list (num_hosts).

The calling program makes repeated calls to prcmd() to advance the status of the host connections. Each call to prcmd() causes a single call of select() to check the status of all connections that require checking (if there are any). Once a connection (to a remote command) is ready for reading, the calling program can read data from it, optionally write data to it, and then wait for more data to be ready to read back from it.

Header File

The <prcmd.h> header file defines a prc_host structure containing the following fields of interest:

char     *prc_hostid;         /* host name or IP address      */
int      prc_prev_status;     /* previous connection status   */
int      prc_conn_status;     /* connection status            */
time_t   prc_conn_time;       /* time of connection           */
int      prc_errno;           /* for failed connections       */
char     *prc_errmsg;         /* string from remshd           */
FILE     *prc_fp;             /* file ptr to stdin/stdout     */
FILE     *prc_fp2;            /* file pointer to stderr       */
int      prc_conn_close;      /* flag: close connection       */
int      prc_caller_status;   /* caller's info about conn     */

The calling program should change only the prc_hostid, prc_conn_close, prc_caller_status, and possibly prc_conn_time fields, as explained below. The other fields should be considered read-only.

The header file also defines the following macros of interest:

PRC_OK               /* function succeeded, check entries           */
PRC_ERR_NETWORKING   /* no networking on calling program's system   */
PRC_ERR_NOFILE       /* cannot get even one file descriptor         */
PRC_ERR_RCMD         /* cannot get "shell" service num/etc.         */
PRC_ERR_SELECT       /* select() failed                             */

PRC_CSBIT_ERR        /* connection has errored out                  */

PRC_CONN_NONE        /* needs connection                            */
PRC_CONN1_WAIT       /* waiting for stdio connect()                 */
PRC_CONN2_WAIT       /* waiting for stderr connect()                */
PRC_CONN3_WAIT       /* waiting for remshd reply                    */
PRC_READ_WAIT        /* waiting for data                            */
PRC_READ_READY       /* data is ready to read                       */
PRC_CONN_DONE        /* connection closed                           */
PRC_CONN_NO_IPS      /* can't get IP addresses                      */
PRC_CONN_FAILED      /* various causes                              */
PRC_CONN_REFUSED     /* by remote system                            */
PRC_CONN_TIMEOUT     /* during connection attempt                   */

typedef struct prc_host *prcp_t;

#define PRC_NULLP   ((prcp_t) 0)
#define PRC_SIZE    (sizeof (struct prc_host))

Data Initialization

The program calling prcmd() should create an array of prc_host structures with one node for each target system. It should then call prcmd_init() using that array (hostp) and its size (num_hosts), a desired initial value (caller_status) for each host's prc_caller_status field, a command to be executed when initiating a remote connection to a host, and a timeout value to apply to each host during setup of its connection. This call sets the values of most fields for each host. The calling program can change each host's value for prc_caller_status, but normally they are all initialized to the same value.

Finally, initialize the following field in each array element before the first call to prcmd():

prc_hostid

Either a system name (with or without domain suffix) or an Internet address in dot notation. This value differs for each host; it's easier for the calling program to set the values directly than to pass a list into prcmd_init().

This value is a pointer to memory allocated by the calling program. Avoid destroying the underlying data until prcmd() establishes a connection to the host. After that, it no longer needs prc_hostid.

Like rcmd(), prcmd() does name lookup on host names to get internet addresses; but unlike rcmd(), it does not modify the calling program's prc_hostid values.

Common Error Handling

Each time prcmd() is called, it scans the host list looking for actions to take based on each host's prc_conn_close field and its prc_conn_status field, as described below. Errors are generally handled as follows:

Failure: If any system call fails while working on a given host's connection, except as noted below, prcmd() sets that host's prc_conn_status field to PRC_CONN_FAILED, sets its prc_errno to the returned errno value from the failed call, and closes all file (socket) descriptors, and stream pointers (if any), that are associated with the host.
Refusal: If any failure occurs on the remote (remshd) side of a connection, as described below, or if an invalid attempt is made to connect back to a host's standard error port, prcmd() sets prc_conn_status to PRC_CONN_REFUSED and closes the files specified by the host's prc_fp and prc_fp2 fields. If any message is available from the remote remshd (up to a limit of BUFSIZ-1 characters) and malloc() succeeds, prcmd() saves the message text in a location pointed to by the prc_errmsg field; otherwise that field remains a null pointer. If instead the refusal occurs for "local" reasons and an errno value is available, prcmd() sets prc_errno to that value, otherwise to 0.
Timeout: A connection times out when it is in PRC_CONN1_WAIT, PRC_CONN2_WAIT, or PRC_CONN3_WAIT state (see below), select() reports that the host is not ready for I/O, and at least timeout seconds have elapsed since the host entered that state. The test is low-precision, to the second, using time(). In this case, prcmd() sets the prc_conn_status field for the host to PRC_CONN_TIMEOUT, and closes the files specified by the host's prc_fp and prc_fp2 fields.

A host specified by name can be listed with multiple IP addresses in the hosts database. In that case, when a connection to one of that host's IP addresses fails, is refused, or times out before reaching the PRC_READ_WAIT state, prcmd() attempts to open a new connection on the host's next IP address (up to a limit of five IP addresses per host). A failure is not returned for a given host until all of its IP addresses have been tried.

Host/Connection Status

See Usage Notes for additional information on how calling programs should handle the following connection status conditions.

PRC_CONN_NONE

prcmd() maps the prc_hostid value from a name to one or more IP addresses (if it does not appear to already be an address), maps the IP addresses to binary, gets an Internet socket bound to a reserved port number for communicating with the host, maps the socket descriptor to a stream pointer using fdopen(), and performs a non-blocking connect() of the socket descriptor to the remote host's "shell" service. The socket's send and receive buffer sizes are the default; the calling program can change the sizes with setsockopt (fileno (hostp → prc_fp)) at any time, subject to the restrictions documented in setsockopt(2).

If gethostbyname() (see gethostbyname(3N)) or inet_addr() (see inet_addr(3N)) fails, prcmd() sets the prc_conn_status field to PRC_CONN_NO_IPS.

Information needed for an rcmd() connection includes the port number for the "shell" service and the user name of the effective user ID as the local and remote user names. prcmd() looks up this information the first time it makes a connection. If prcmd() cannot obtain this information it returns PRC_ERR_RCMD (and tries again each time it's called).

If socket() fails with any of the following local-host errors: EHOSTDOWN, EAFNOSUPPORT, ESOCKTNOSUPPORT, EPROTONOSUPPORT, EPROTOTYPE, or EINVAL, prcmd() returns PRC_ERR_NETWORKING and the status of the host list is unspecified.

If socket() fails with EMFILE or ENFILE, which means the process or the system is out of file descriptors, or if there are no available reserved ports, the prc_conn_status field is left as PRC_CONN_NONE for later retries. After checking all host entries, if prcmd() has no open files (sockets) for any host, and it didn't close any files during this call, it returns PRC_ERR_NOFILE — that is, all file descriptors are in use for this process, but none by prcmd().

If connect() succeeds, or if it fails with EINPROGRESS, which is normal for a non-blocking connect request, prcmd() sets prc_conn_status to PRC_CONN1_WAIT, prc_conn_time to the current system clock in seconds, and prc_fp to the stream pointer for this socket. This host is then included in the select() list per PRC_CONN1_WAIT (see below).

PRC_CONN1_WAIT

prcmd() includes this host in the select() list for write-readiness. If the connection is not ready for writing, the prc_conn_status field is left unchanged.

When the connection is ready for writing, prcmd() returns the socket to "blocking", starts a connection for standard error back from the remote command, sets prc_fp2 to the file pointer for this connection, writes standard remsh connection information (see remshd(1M) and also above) to the remote process with SIGPIPE temporarily ignored, changes prc_conn_status to PRC_CONN2_WAIT, and sets prc_conn_time to the current system clock in seconds.

If a connection is refused, prcmd() gets ECONNREFUSED or EPIPE on attempt to write remsh connection information, and handles the error as previously described in the section called Common Error Handling.

PRC_CONN2_WAIT

prcmd() includes this host in the select() list for read-readiness — for both prc_fp (in case of error) and prc_fp2 (in case of successful connection back to the standard error port). When neither connection is ready for reading, the prc_conn_status field is left unchanged.

When the standard error connection is ready for reading, prcmd() does an accept() on it, closes the old prc_fp2 and revises the value to a new file pointer for the new socket/fd, changes prc_conn_status to PRC_CONN3_WAIT, and sets prc_conn_time to the current system clock in seconds.

If only the stdout connection is ready for reading, this indicates a remshd failure. prcmd() treats this as a refused connection, as described earlier under Common Error Handling.

Note that a host is "held" in PRC_CONN1_WAIT or PRC_CONN2_WAIT status if there are no available file descriptors or reserved ports when one is needed.

PRC_CONN3_WAIT

prcmd() includes this host in the select() list for read-readiness. If the connection is not ready for reading, the prc_conn_status field is left unchanged.

When the connection is ready for reading, prcmd() consumes the null byte on stdout from remshd, sets prc_conn_status to PRC_READ_WAIT, and resets prc_conn_time to the current system clock in seconds for use by the calling program during the ensuing conversation. If anything other than a null byte arrives, this indicates a remshd failure. prcmd() treats this as a refused connection, as described previously.

PRC_READ_WAIT

prcmd() includes this host in the select() list for read-readiness. When the connection is not ready for reading, the prc_conn_status field is left unchanged. Note, the remote command might be waiting for the calling program to write data to it.

When the connection is ready for reading, prcmd() sets prc_conn_status to PRC_READ_READY.

PRC_READ_READY

prcmd() treats a host in this status the same as PRC_READ_WAIT. If the connection is not ready for reading, prcmd() sets prc_conn_status back to PRC_READ_WAIT.

PRC_CONN_DONE

prcmd() ignores this host's entry.

prcmd() enters this state from any other state when the prc_conn_close field (set by the calling program) is non-zero. It closes the files specified by prc_fp and prc_fp2 (if not already null), sets those fields to null, and resets the prc_conn_close field to 0.

PRC_CONN_NO_IPS

PRC_CONN_FAILED

PRC_CONN_REFUSED

PRC_CONN_TIMEOUT

With any of these failure status values, prcmd() ignores this host's entry, except if prc_conn_close is set, the entry's prc_conn_status field is set to PRC_CONN_DONE.

Changes in Host Status

After each prcmd() call, the prc_prev_status field for each host's entry is set to the previous value of its prc_conn_status field. The calling program can use (prc_prev_status != prc_conn_status) to quickly check for a new status for any host. This is useful for once-only logging or display of status values except PRC_READ_WAIT and PRC_READ_READY — each of these states can be entered many times during a conversation.

Note that a host can leave and then re-enter PRC_CONN_NONE status if it has multiple IP addresses, a connection to one IP fails, and there are no files or ports available to try the host's next IP address.

The calling program can freely use the prc_caller_status field to record additional information about the status of each connection.

Usage Notes

Once a connection is made, and a host is in one of the PRC_READ_* states, control of the "conversation" between the calling program and remote host belongs to the calling program. Conversations are of two types:

1.: calling program communicates first
2.: remote command communicates first

If the calling program communicates first, it must use the prc_caller_status field, or other means, to remember whether or not it has sent initial data to the command; and it must send that data upon the host entering the PRC_READ_WAIT state the first time. If the command communicates first, or in any conversation after the calling program sends initial data, the calling program must "track" the conversation to know when to send more data, if any, and when to close the conversation by setting prc_conn_close.

Before the calling program can send the next input, or close the connection, it might have to wait through a series of PRC_READ_WAIT and PRC_READ_READY states (until all bytes of a single response from a command are received).

After each prcmd() call, the calling program should check the return value for a serious error. If the return value is PRC_OK, the calling program should next scan the hostp array and check prc_prev_status (if desired) and prc_conn_status fields for each host. The calling program only needs to take action for certain status values (marked with • below).

PRC_CONN_NONE: After the first prcmd() call: Indicates the process ran out of socket (file) descriptors, or reserved ports, after making at least one (still open) connection (in this call, or in a previous call). Connection to this host is attempted again on the next call. The calling program can increase the number of available file descriptors (by closing files or calling rlimit()), or wait for existing connections to finish their communications.
PRC_CONN1_WAIT
PRC_CONN2_WAIT
PRC_CONN3_WAIT: These are internal states of little interest to the calling program. No action is needed except to call prcmd() again later. If (prc_prev_status != prc_conn_status), the calling program might log/display the host's new status.
PRC_READ_WAIT •: Command is running but has not yet produced output. If the calling program communicates first, meaning the command needs some initial input, write it to the file specified by prc_fp and remember that this was done, probably by using the prc_caller_status field. Otherwise no action is required, except maybe to log/display the connection's status.
PRC_READ_READY •: Command produced output that is ready for reading. After reading the output, if the command needs more input, write it now to the file specified by prc_fp. If instead the conversation is done, set the prc_conn_close field non-zero.
PRC_CONN_DONE: Command is done; connection was closed in response to the calling program setting the prc_conn_close field. No action is required.

The calling program can use (prc_conn_status & PRC_CSBIT_ERR) to quickly detect any error status. It can use (prc_prev_status != prc_conn_status) to detect that any failure condition is new so it is only handled once, or use prc_caller_status to record that it was processed the first time it appears in a return from prcmd(). In each case the calling program should handle or report the error as desired.

PRC_CONN_NO_IPS •: The prc_hostid field appears to be a name, not an address, and gethostbyname() fails for it, or inet_addr() fails on a numeric prc_hostid value.

The remaining conditions only refer to the cause of failure on the last of the host's IP addresses, if it has more than one.

PRC_CONN_FAILED •

Connection could not be initiated for some reason (on any of the host's IP addresses) due to failure on the local system. prc_errno is set to the errno value from a failed system call.

PRC_CONN_REFUSED •

Connection refused. Per errno(2): "This usually results from trying to connect to a service that is inactive on the foreign host." It can also be due to any other remote (remshd) failure, including access denial; and to an improper connection attempt on the standard error port.

The calling program should check the host's prc_errmsg field. If it is not a null pointer, message text is available from the remote remshd command; and the calling program must free() the pointer when done using the value.

PRC_CONN_TIMEOUT •

Connection timed out. The connection has remained in one of the PRC_CONN1_WAIT, PRC_CONN2_WAIT, or PRC_CONN3_WAIT states for more than timeout seconds without becoming ready for I/O.

If the calling program performs unbuffered I/O using socket (file) descriptors instead of streams, it can refer to fileno(prc_fp) (see fileno(3S)).

If the calling program needs to send a signal to a remote command, it can write the signal number to the file specified by prc_fp2 (see rcmd(3N)).

There is no way to tell prcmd() to close down all connections, except by setting prc_conn_close for all hosts, and calling prcmd() once more. This causes prc_conn_status to be set to PRC_CONN_DONE.

Regarding Timeouts

prcmd() is designed to maximize the number of parallel connections, and to maximize control by the calling program. Therefore prcmd() always calls select() with a timeout value of zero (immediate polling). It is the responsibility of the calling program to avoid calling prcmd() more frequently than is necessary (therefore consuming excessive CPU time). For example, the calling program might call sleep(1) (see sleep(3C)) between prcmd() calls. Note: This is unnecessary if the calling program is itself periodically invoked by timer interrupts, or if it performs other (time-consuming) tasks between prcmd() calls.

A timeout will occur if the host's connection remains in one of the wait states, PRC_CONN1_WAIT, PRC_CONN2_WAIT, or PRC_CONN3_WAIT, for more than timeout seconds without becoming ready for I/O, and no untried IP addresses for this host remain. When a host's connection reaches PRC_READ_WAIT state the first time, prcmd() resets the host's prc_conn_time field but does not check the host again for timeout. The calling program can do this if desired. The calling program is also permitted to reset (update) the prc_conn_time field as the communications proceed (for example, each time it writes data to the remote command).

RETURN VALUE

PRC_OK: Call succeeded; check the host list for prc_prev_status and prc_conn_status field values. If select() fails with EINTR (due to a signal arriving), prcmd() returns control to the calling program with PRC_OK. In this case some connections might have been started (entered the PRC_CONN1_WAIT state) or stopped (entered the PRC_CONN_DONE state), under the calling program's control, but none has advanced due to I/O status; call prcmd() again as usual.
PRC_ERR_NETWORKING: A socket() call failed due to any of a number of serious networking problems (see earlier list), indicating the local host's networking is not enabled, and prcmd() is useless; errno is set on return from prcmd().
PRC_ERR_NOFILE: A socket() call failed with EMFILE or ENFILE, or there were no more available reserved ports, at a time when prcmd() had no open connections (sockets). This means further attempts to open connections are futile until the calling program, or the system, frees some file descriptors, or reserved ports.
PRC_ERR_RCMD: Unable to get the "shell" service port number or the username for the effective user ID. It's not useful to call prcmd() again unless this condition is corrected.
PRC_ERR_SELECT: A select() call failed (other than with EINTR); errno is set on return from prcmd(). The data in the host list is valid but no hosts are read-ready or write-ready, even if marked PRC_READ_READY from a previous successful call.

DIAGNOSTICS

Unlike rcmd(), prcmd() does not copy messages from remshd to the local standard error when remshd fails. It just puts the host in PRC_CONN_REFUSED state.

EXAMPLES

The following code fragment illustrates how to allocate and initialize an array of hosts, and call prcmd() once for the entire list of hosts:

int index;
struct prc_host prc_host [MAXHOSTS];

prcmd_init (prc_host, argc, 0, REMCMD, TIMEOUT);

for (index = 0; index < argc; ++index)
   (prc_host[index].prc_hostid) = argv [index];

if ((rc = prcmd (prc_host, argc)) != PRC_OK)
   ...

WARNINGS

prcmd() is unsafe in multi-thread applications.

Like rcmd(), prcmd() should only be called by a privileged process. Otherwise it puts every host in PRC_CONN_FAILED status with errno = 13 (EACCES) because it cannot bind() to any reserved port.

At this time select() calls are done on files specified by prc_fp only, not prc_fp2. prcmd() assumes that the remote command will never write to standard error and then block, but will instead terminate or otherwise close stdout. When stdout is read-ready, the calling program can safely do a blocking read() on prc_fp, and upon EOF or error, check prc_fp2 with a blocking read().

(Exception: For status PRC_CONN2_WAIT both files are selected since either might come ready depending on success or failure, as described earlier. However, this exception is handled internal to prcmd() and does not affect the calling program.)

Note that prcmd_init() saves a pointer to the value of command, not the value itself, so the calling program must preserve the value through all calls of prcmd().

connect() times out after about two minutes, even with non-blocking I/O. If this happens to a host's connection, the result is indistinguishable from PRC_CONN_REFUSED since write() returns EPIPE in either case. This is not a problem if the calling program allows less than two minutes for a connection, that is, sends a timeout value less than 120, and calls prcmd() frequently enough that the latter catches the timeout itself before connect() does.

Be careful when using buffered I/O. Avoid calls to fgets(), fread(), and similar functions in circumstances where they might block indefinitely if less data is available than requested. Also, be sure to read all available data from a host via the file specified by prc_fp before waiting again for a connection to be read-ready. The connection might appear not-ready even though buffered, pending data was already read by the stdio library, and is available. Avoiding these problems might require the use of non-blocking I/O; see fcntl(2).

Be careful about writing to a socket that might be closed if the remote command terminates, or if the connection is lost. This can cause the calling process to receive SIGPIPE. The calling program must handle this signal if appropriate.

When handling a host whose connection failed with PRC_CONN_REFUSED, remember to free() the host's prc_errmsg pointer if it is non-null. This can be skipped if regaining malloc'd memory is not required.

Performance Issues

It is faster to call prcmd() with host IDs specified as internet addresses rather than names, because prcmd() will not need to look up each host in the hosts database. However, when doing so with hosts that have multiple internet addresses, only the specified internet address is tried.

There is a limit of about 512 reserved ports on each system, and a soft limit for the number of available file descriptors for each process. Each specified host requires two reserved ports and two file descriptors for the connection to check its status. prcmd() defers (serializes) additional connections once all ports or file descriptors are in use, so additional hosts are not ignored, but the time required for all connections to complete increases accordingly. To improve performance, it might be necessary to increase the limit on open files if there are a lot of hosts; see setrlimit(2).

AUTHOR

prcmd() was developed by Hewlett-Packard.