NAME
numa_policy — physical memory allocation policy on cell-based HP-UX servers
VALUES
Allowed values
Minimum:
0
Maximum:
2
DESCRIPTION
Large servers are built from one or more component blocks called cells.
Each cell has at least 1 CPU and normally some amount of memory.
These cell-based servers use Non-Uniform Memory Access (NUMA) protocols.
This means that an application thread can experience widely variable
memory latency depending on whether it is accessing memory from the same
cell upon which it is executing or from a different cell.
HP-UX implements a number of memory allocation features to optimize performance.
Users can divide the memory on the system into two classes of memory:
Interleave
and
Cell-local.
Interleave memory delivers uniform latency.
It makes NUMA hardware look like a single bank of uniform memory, delivering
the same average latency for all applications across all CPUs.
This is very
useful in spreading the memory load across the entire memory fabric.
By default, all the memory in an HP-UX server is configured as interleaved.
Cell-local memory delivers low latency (high performance) to a process
running in the same cell as the memory it accesses.
Conversely, references from a remote cell will have a much higher latency.
Users can configure cell-local memory using the HP-UX command
parmodify(1M).
The
numa_policy
dynamic kernel tunable allows users to define how
interleaved and cell-local memory are utilized.
There are three possible values:
- numa_policy = 0
This is the default policy.
In a machine configured with a mix of cell-local memory and interleave memory,
HP-UX makes intelligent decisions on how to use the memory by distinguishing
between private memory and shared memory.
Memory allocated to private objects (stack, heap, private files, etc.) is
likely to come from cell-local memory.
Memory allocated to shared objects
(shared files, System V shared memory) is allocated from interleave memory
unless the user specifies otherwise using
fadvise(2)
for files or uses one of the memory options in
shmget(2).
- numa_policy = 1
This policy directs HP-UX to use cell-local memory for all memory allocations.
It will select the cell from which the locality is to be allocated by first
honoring any cell-locality policies specified in
fadvise()
or
shmget().
If no cell-locality policies were specified, then it allocate all memory
from the local cell.
If there is
not sufficient memory in the local cell, then memory from the closest cell is
used, and so on and so forth.
Even the interleave is used, but only if the
cell-local memory available is too remote.
When using this option, care needs
to be exercised to ensure that applications are scheduled on the same
cell where the memory has been allocated.
See
mpsched(1)
for more information.
- numa_policy = 2
This policy directs HP-UX to over-ride any locality policies supplied via
fadvise()
or
shmget()
and allocate all memory from the interleave.
If there is not
sufficient memory in the interleave, then cell-local memory is used.
Restrictions on Changing
Changes to this tunable take effect on all memory allocated subsequent to the
change.
Any memory created before the change will not be affected.
When Should the Value of This Tunable Be Changed to 1?
The tunable value should be set to 1 only if all performance-sensitive
applications on the system have a high-degree of cell
locality.
That is, if the great majority of references to shared memory
are from processes running on the same cell where the memory has been
allocated, then setting
numa_policy = 2
will have a positive effect on performance.
The value should normally be changed back to 0 after the memory allocations have
been completed.
Changing the value to 1 should be done after sufficient memory
has been designated as cell-local using
parmodify(1M)
and the machine has been rebooted.
When Should the Value of This Tunable Be Changed to 2?
The tunable value can be set to 2 if the workload has a low-degree of locality
to a cell and it is not possible to change the source code.
The value should
be changed to 0 after the memory allocations have been completed.
Changing the
value to 2 should be done after sufficient memory has been designated as
interleave using
parmodify(1M)
and the machine has been rebooted.
Note, setting the
numa_policy
tunable to 2 only makes sense in a mixed environment in which some applications
can benefit from cell-local memory, and others cannot.
WARNINGS
All HP-UX kernel tunable parameters are release specific.
This parameter may be removed or have its meaning changed in
future releases of HP-UX.
Installation of optional kernel software, from HP or other vendors,
may cause changes to tunable parameter values. After installation,
some tunable parameters may no longer be at the default or
recommended values. For information about the effects of
installation on tunable values, consult the documentation
for the kernel software being installed.
For information about optional kernel software that was
factory installed on your system, see
HP-UX Release Notes
at
http://docs.hp.com.
AUTHOR
numa_policy
was developed by HP.
Printable version
