Technote (FAQ)


Question

How do I move local data such as configuration files, log files, historical data, etc. from CANDLEHOME to a different file system?

Cause

Since its inception, IBM Tivoli Monitoring (ITM) has stored local data such as configuration files, log files, and historical data together with the installation files in CANDLEHOME. For example, ITM normally installs in a directory named /opt/IBM/ITM. Within that directory, it stores programs in subdirectory names like /opt/IBM/ITM/bin and /opt/IBM/ITM/classes. It also stores local data in subdirectory names like /opt/IBM/ITM/config and /opt/IBM/ITM/logs. This homogeneous design is a problem for installations that require an approach that keeps code in /opt and stores data in the /var file system.

Answer

Introduction

Under APAR IV67523, ITM added a new mechanism to move local data to a different file system. For example, data typically stored in /opt/IBM/ITM/config and /opt/IBM/ITM/logs can be redirected to /var/IBM/ITM/config and /var/IBM/ITM/logs. This new file system location is designated by the user at installation time, and it is referred to as CANDLEDATA. A conversion utility is provided to implement CANDLEDATA for existing installations.

The CANDLEDATA mechanism is available with ITM 6.30.05.00.


How CANDLEDATA Works

The CANDLEDATA mechanism is implemented primarily using symbolic links. This means most of the underlying code in ITM is unaware of CANDLEDATA. This also means the CANDLEDATA mechanism is limited to Linux and UNIX platforms.

The rest of this section shows examples of links where CANDLEHOME=/opt/IBM/ITM and CANDLEDATA=/var/IBM/ITM. You can define different locations for your own installation.

Base Directory Links

The following local data directories are always linked from CANDLEHOME to CANDLEDATA:

    /opt/IBM/ITM/InstallITM         -> /var/IBM/ITM/InstallITM
    /opt/IBM/ITM/config             -> /var/IBM/ITM/config
    /opt/IBM/ITM/kt1v3depot         -> /var/IBM/ITM/kt1v3depot
    /opt/IBM/ITM/localconfig        -> /var/IBM/ITM/localconfig
    /opt/IBM/ITM/logs               -> /var/IBM/ITM/logs
    /opt/IBM/ITM/patchlogs          -> /var/IBM/ITM/patchlogs
    /opt/IBM/ITM/tables             -> /var/IBM/ITM/tables
    /opt/IBM/ITM/tmaitm6/agentdepot -> /var/IBM/ITM/tmaitm6/agentdepot
    /opt/IBM/ITM/tmp                -> /var/IBM/ITM/tmp

These directories and links are automatically generated by ITM when the system is installed or converted to use CANDLEDATA.

General Agent Directory Links

In addition to the base set of directory links, most agents have general links for TEMA data:

    /opt/IBM/ITM/ATTRLIB                        -> /var/IBM/ITM/ATTRLIB
    /opt/IBM/ITM/<platform>/<pc>/tables/ATTRLIB -> /var/IBM/ITM/ATTRLIB/<pc>
    /opt/IBM/ITM/EIFLIB                         -> /var/IBM/ITM/EIFLIB
    /opt/IBM/ITM/<platform>/<pc>/tables/EIFLIB  -> /var/IBM/ITM/EIFLIB/<pc>
    /opt/IBM/ITM/hist                           -> /var/IBM/ITM/hist
    /opt/IBM/ITM/<platform>/<pc>/hist           -> /var/IBM/ITM/hist/<pc>
    /opt/IBM/ITM/psit                           -> /var/IBM/ITM/psit
    /opt/IBM/ITM/<platform>/<pc>/psit           -> /var/IBM/ITM/psit/<pc>

These directories and links are automatically generated by ITM when the agent is converted to use CANDLEDATA.

Special Agent Directory Links

Some agents have links that are specific to that agent. For example, the LZ and UX agents create the following link for the Watchdog component to store local data:

    /opt/IBM/ITM/kca                     -> /var/IBM/ITM/kca

These directories and links are automatically generated by ITM when the agent is converted to use CANDLEDATA.

CANDLEDATA <-> CANDLEHOME Directory Links

The CANDLEHOME and CANDLEDATA directories also have cross references to each other:

    /opt/IBM/IBM/registry/CANDLEDATA -> /var/IBM/ITM
    /opt/IBM/ITM                     <- /var/IBM/ITM/CANDLEHOME

These links are all automatically generated by ITM when the system is installed or converted to use CANDLEDATA.

Configuration Variables exploited by CANDLEDATA

In addition to symbolic links, some individual components have special configuration settings to redirect agent-specific data to CANDLEDATA. ITM redirects data using the following configuration variables:

    CTIRA_SIT_PATH  - redirects situation data to CANDLEHOME/ <platform>/<pc>/psit.
    KCA_IP_DIR      - redirects watchdog data to CANDLEHOME/kca.
    KCAWD_WRITE_DIR - redirects watchdog data to CANDLEHOME/kca.
Notes:
  1. While the variables point to CANDLEHOME/xxx, the directory links cause the local data to be written to the corresponding CANDLEDATA/xxx.
  2. If you use configuration overrides that set these configuration variables, you'll need to review how those overrides work in an environment configured to use CANDLEDATA. In most cases, you'll want to remove your overrides.

Configuration Variables affected by CANDLEDATA
The symbolic links established by ITM for CANDLEDATA assume standard agent configuration. If you've modified any of the following, data will be continue to be redirected to the location you specified instead of the CANDLEDATA directory:
.

    ATTRLIB
    CTIRA_HIST_DIR
    EIFLIB
    .
When converting an existing CANDLEHOME, ITM migrates any data in the original directories before linking them to CANDLEDATA. If you've modified any of the following, you must consider whether you need to change the setting and migrate the data manually:
.
    CTIRA_SIT_FILE
    .
By default, all of the following configuration variables direct data to /opt/IBM/ITM/localconfig. ITM migrates data from /opt/IBM/ITM/localconfig to /var/IBM/ITM/localconfig and then links the two. If you've modified any of these, you must consider whether you need to change the setting and migrate the data manually:
.
    CTIRA_THRESHOLDS
    IRA_EVENT_EXPORT_SNMP_TRAP_CONFIG  
    IRA_LOCALCONFIG_DIR  
    IRA_PRIVATE_SITUATION_CONFIG  
    IRA_SERVICE_INTERFACE_DEFAULT_PAGE  
    IRA_SERVICE_INTERFACE_DIR  
    KXX_FCP_SCRIPT_DEFINITIONS


Known Restrictions

  1. The CANDLEDATA mechanism is available only for ITM on Linux and UNIX. ITM on Windows continues to keep local data under CANDLEHOME.
    .
  2. No facility is provided switch from a CANDLEHOME+CANDLEDATA implementation back to a unified CANDLEHOME implementation. To do this, you must uninstall and reinstall ITM.
    .
  3. No facility is provided to move CANDLEDATA from one directory to another. To do this, you must uninstall and reinstall ITM.
    .
  4. The CANDLEHOME and CANDLEDATA arrangement is a 1:1 relationship.
    • Only one CANDLEHOME can use a given CANDLEDATA. This means a given CANDLEDATA cannot be shared with multiple CANDLEHOMEs.
    • A given CANDLEHOME can only use one CANDLEDATA. This means components cannot be split amongst multiple CANDLEDATAs.
      .
  5. The monitoring server and portal server cannot exploit CANDLEDATA.
    • The conversion utility will not convert an existing CANDLEHOME if it contains the monitoring server or portal server.
    • The installer warns if CANDLEDATA is specified and the install image contains the monitoring server or portal server. However, the installer does not prevent monitoring server or portal server from being installed.
    • ITM does not allow the monitoring server to start if CANDLEDATA is implemented.
    • ITM does not allow the portal server to start if CANDLEDATA is implemented.
      .
  6. Not all agents have been verified to work with CANDLEDATA. Before using CANDLEDATA, check with the support group for each of the agents you intend to install.
    .
  7. System Monitor Agents (SMA) can exploit CANDLEDATA. However, the SMA installer does not create CANDLEDATA. These agents must be converted to use CANDLEDATA after they are installed.
    .
  8. The CANDLEDATA mechanism is intended only for current agents. If the agent media contains installer 6.30.00.00 or above, it should be ok to use with CANDLEDATA.

    Some older or discontinued agents such as the Universal Agent (UM) and UNIX Logs agent (UL) cannot use CANDLEDATA reliably. ITM does not prevent older or discontinued agents from being installed into a new CANDLEHOME with CANDLEDATA implemented.
    .
  9. When CANDLEDATA is in use, ITM implements internal overrides for the following configuration variables: CTIRA_SIT_PATH, KCA_IP_DIR, and KCAWD_WRITE_DIR.
    • If you've already overridden any of these configuration variables, you must remove the overrides before converting a given component.
    • If you've overridden these with CANDLEHOME/config/xx.environment, the conversion utility will detect and warn about them. Your overrides will win, so you must remove them.
    • If you've overridden these with the old method of sourcing a script into CANDLEHOME/config/xx.ini, the conversion utility cannot detect them. However, the CANDLEDATA overrides will win.
      .
  10. The prerequisite scanner does not know about CANDLEDATA. It continues to assume everything is stored in CANDLEHOME. This means:
    • No space checking is performed for CANDLEDATA.
    • Results are somewhat inflated for CANDLEHOME.
      .
  11. The conversion utility does not check for space when converting an existing new CANDLEHOME to use CANDLEDATA. Make sure there is enough space in CANDLEDATA before starting the conversion process.


Agents Verified with CANDLEDATA

The following agents have been verified to run properly when CANDLEDATA is implemented:

    LO - Tivoli Log File Agent
    LZ - Monitoring Agent for Linux OS
    UD - IBM Tivoli Composite Application Manager Agent for DB2
    UX - Monitoring Agent for UNIX OS
For other agents, contact the corresponding support team.


Setting Up a New Local Installation with CANDLEDATA

For new installations, you can define the location for local data at installation time.

Local install (interactive):

    The interactive installer asks for the location of CANDLEDATA after is asks for CANDLEHOME. Here is an example of the prompt:

    # ./install.sh
    INSTALL

    Enter the name of the IBM Tivoli Monitoring installation directory.
    [ default = /opt/IBM/ITM ]:
    "/opt/IBM/ITM" does not exist
    Try to create it [ 1-yes, 2-no; "1" is default ]?
    Creating directory "/opt/IBM/ITM"...

    Enter the name of the IBM Tivoli Monitoring local data directory.
    [ default = /opt/IBM/ITM ]: /var/IBM/ITM
    "/var/IBM/ITM" does not exist
    Try to create it [ 1-yes, 2-no; "1" is default ]?
    Creating directory "/var/IBM/ITM"...
    itmCANDLEDATA.sh: Installation directory "/opt/IBM/ITM" successfully converted to use data directory "/var/IBM/ITM".

    This prompt is presented only for new installations. For existing installations, you must use the conversion utility to implement CANDLEDATA. After this prompt, the rest of the installation process is unchanged.

    If you set the CANDLEDATA environment variable before launching the installer, then that value is used as though it had been prompted. (This behavior is the same for CANDLEHOME.) Otherwise, the default for CANDLEDATA is the value of CANDLEHOME, meaning the installer creates a unified CANDLEHOME unless specifically directed otherwise.

Local install (silent):
    The silent installer provides for a new command option for the location of CANDLEDATA: This command option is only effective for new installations. For existing installations, you must use the conversion utility to implement CANDLEDATA.

    install.sh
      [-l
    CANDLEDATA ]

      Where:
        -l specifies where ITM's local data is to be stored. This is a fully qualified directory name. If the directory does not exist, it will be created. If the option is not specified, local data is stored in CANDLEHOME.

    This command option is only effective for new installations. For existing installations, you must use the conversion utility to implement CANDLEDATA. The rest of the install.sh command options are unchanged.

    If you set the CANDLEDATA environment variable before launching the installer, then that value is used as though it had been passed with the new command option. (This behavior is the same for CANDLEHOME.) If both the command option is used and the CANDLEDATA environment variable is defined before install.sh is started, the command option takes precedence.


Starting an agent for the first time with CANDLEDATA

When you specify CANDLEDATA at install time, ITM establishes the base directory links. All agent-specific conversion is handled the first time a given agent is started. When this happens, an extra message is included in the agent start output: Here is an example:

# itmcmd agent start ux
Processing. Please wait...
itmCANDLEDATA.sh: Component ux successfully converted to use data directory "/var/IBM/ITM".
Starting Monitoring Agent for UNIX OS ...
Monitoring Agent for UNIX OS started


Converting an existing installation or new agents to use CANDLEDATA

For existing installations, ITM provides a utility to convert a CANDLEHOME to use CANDLEDATA. You can also use this utility to convert a newly installed agent before it is started. Under the covers, the installer runs this same utility.

itmCANDLEDATA.sh
   [-b yes|no]
   [-h CANDLEHOME]
   [-l CANDLEDATA]
   [-f no|yes]
   [-p PC|*]
   [-r no|yes]

Where:


    -b indicates build mode. If yes (default), the conversion utility builds the CANDLEDATA directory, moves data to it from CANDLEHOME, and links CANDLEHOME to it. If no, the conversion utility assumes the CANDLEDATA build is complete.

    -h specifies the ITM installation directory.

    -l specifies the ITM local data directory. For an existing CANDLEHOME where CANDLEDATA is already implemented, you can omit this command option. The conversion utility derives it from existing directory links.

    -f indicates force mode. If no (default), conversion is skipped for a given component if it has already been done for that component. If yes, conversion is performed even if it has already been done for that component.

    -p indicates conversion for a specific product code. Multiple product codes may be specified by separating them with a blank space. An asterisk ( *) indicates all product components are to be converted. Note that this is intended for optimization. It is not for picking and choosing which components are converted. For example, if a new component has just been installed to an existing environment, this command option can be used to tell the conversion utility to process just that new one vs. all of them.

    -r indicates restart mode, which controls whether components are automatically stopped and restarted. If no (default), then affected components must be stopped before the conversion utility is run. If yes, the conversion utility automatically stops and restarts components as needed. You must either run the conversion utility as root or as the user that started all of the components that are running. Otherwise, the conversion utility is not authorized to stop and restart the components.

If you set the CANDLEHOME or CANDLEDATA environment variables before launching the installer, then those values are used as though they had been passed with the command options. If both the command option is used and the environment variable is defined before the conversion utility is started, the command option takes precedence.

Example 1: Conversion of a typical CANDLEHOME can be performed with an incantation similar to this:

    itmCANDLEDATA.sh -l /var/IBM/ITM -b yes -p "*" -r yes

Example 2: Conversion of a newly installed agent to a CANDLEHOME where CANDLEDATA is already implemented can be performed with an incantation similar to this:

    itmCANDLEDATA.sh -b no -p "ux" -r yes


Using CANDLEDATA with Remote Deploy

You can define the location for local data remotely:

tacmd createnode (new installations):

    There are no specific changes to remote deploy to enable the CANDLEDATA mechanism. However, under the covers, remote deploy simply uses the silent local install. This means you can specify CANDLEDATA using remote deploy's mechanism for passing generic environment variables:

    tacmd createnode ... -o ENV_CANDLEDATA= data_directory  ...

    Just as with a silent local installation, install.sh uses the CANDLEDATA environment variable to create and populate a directory for local data.
tacmd executecommand + itmCANDLEDATA.sh (existing installations):
    Just as the CANDLEDATA environment variable is not used for existing local installations, it is not used with tacmd updateagent. Instead, use the conversion utility remotely to implement CANDLEDATA. You can launch the conversion utility remotely with tacmd executecommand, like this:

    tacmd executecommand ... -c "/opt/IBM/ITM/bin/itmCANDLEDATA.sh -l /var/IBM/ITM -p \"*\" -b yes -r yes > /tmp/itmCANDLEDATA_remote.log 2>&1 &"

    This example does the following:
    • Launches the conversion utility on the agent (/opt/IBM/ITM/bin/itmCANDLEDATA.sh).
    • Specifies the local data directory (-l /var/IBM/ITM).
    • Specifies all agents are to be converted (-p \"*\"). Note that the quotes are escaped because the entire command string is quoted.
    • Specifies the CANDLEDATA directory is to be built (-b yes).
    • Specifies the agents are to be restarted (-r yes).
    • Redirects STDOUT and STDERR to a log file (> /tmp/itmCANDLEDATA_remote.log 2>&1). Note that the standard -o/-e/-d/-v options for tacmd executecommand don't function properly because of the agent restart. Note also that /tmp is used instead of /opt/IBM/ITM/logs; the latter is moved during the conversion process and therefore must be inactive.
    • Launches the command in the background (trailing &). tacmd executecommand waits for the commands to complete, and this is signaled by a communication from the agent. However, since the agent is restarted by the command, it will never reply to tacmd executecommand. Running the command in the background allows tacmd executecommand to complete normally.


Other Implications

The following general ITM utilities have been updated to run properly when CANDLEDATA is implemented:

  • cinfo - The output heading indicates the location of CANDLEDATA if it implemented.
  • itmcmd agent start - Agent configurations are automatically updated if CANDLEDATA is implemented. Also, the portal server is prevented from starting if CANDLEDATA is implemented.
  • itmcmd server start - The monitoring server is prevented from starting if CANDLEDATA is implemented.
  • secureMain - Permissions and ownership in CANDLEDATA are maintained just as in CANDLEHOME. If secureMain was run before CANDLEDATA is implemented, the conversion utility automatically runs secureMain again as part of the conversion process. However, if you run the conversion utility as a nonroot user (not recommended), you must run secureMain manually after conversion is complete.
  • tacmd pdcollect and pdcollect - Data is gathered from CANDLEDATA just as from CANDLEHOME.
  • uninstall.sh - Relevant data is removed from in CANDLEDATA just as from CANDLEHOME. For uninstall.sh REMOVE EVERYTHING, both the CANDLEHOME and CANDLEDATA directories are removed.