IBM InfoSphere DataStage and QualityStage Operations Console and operations database log files

-

Question

Where are the log files for the IBM InfoSphere DataStage and QualityStage Operations Console and operations database (DSODB) located? How do I enable extra tracing for the log files?

Answer

The Operations Console and operations database is split into three sections; service management, populating DSODB, and Operations Console.

(Note that the paths provided in this note use InformationServerRoot as the path of the Information Server installation. The default path is C:\IBM\InformationServer for Windows or /opt/IBM/InformationServer for Linux and UNIX. WebSphereRoot is used as the path of the WebSphere installation. The default path is C:\IBM\WebSphere for Windows or /opt/IBM/WebSphere for Linux and UNIX.).

The relevant log files are as follows:

Service management

1) When the DSAppWatcher.sh file is run (or the DataStage AppWatcher Windows service), details of the request is logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\DSODB\logs\AppWatcher_Cmd.log

Unix and Linux: InformationServerRoot/Server/DSODB/logs/AppWatcher_Cmd.log

To increase the amount of information logged in this file, change the APW_DEBUG entry from 0 to 1 in the DSAppWatcher.sh file (in InformationServerRoot\Server\DSODB\bin):
APW_DEBUG=1

2) When the DSAppWatcher process is running, details are logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\DSODB\logs\AppWatcher_Svc-<processStartTimestamp>.log

Unix and Linux: InformationServerRoot/Server/DSODB/logs/AppWatcher_Svc-<processStartTimestamp>.log

To increase the amount of information logged in this file, change the APW_DEBUG entry from 0 to 1 in the DSAppWatcher.sh file (in InformationServerRoot\Server\DSODB\bin) and restart the DSAppWatcher service:
APW_DEBUG=1

Populating DSODB

3) When a job is run it creates event files that are used to populate the DSODB. If a problem occurs during creation of these files at runtime, an error is logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\DSODB\logs\JobRuntime.err

Unix and Linux: InformationServerRoot/Server/DSODB/logs/JobRuntime.err

To increase the amount of information logged at job run time, uncomment the TraceMax entry in the DSODBConfig.cfg file (in InformationServerRoot\Server\DSODB\) :
TraceMax=1000.

The TraceMax entry causes detailed information to be logged in the following file on the engine tier:

Windows: InformationServerRoot\Server\DSODB\logs\JobRuntime.log

Unix & Linux: InformationServerRoot/Server/DSODB/logs/JobRuntime.log

4) Independent of job runtime, a service called EngMonApp reads any event files created in #3 above and inserts the relevant rows into the DSODB. If a problem occurs during this operation an error is logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\DSODB\logs\EngMonApp-<processStartTimestamp>.log

Unix and Linux: InformationServerRoot/Server/DSODB/logs/EngMonApp-<processStartTimestamp>.log

To increase the amount of information logged in this file, change the following entries from INFO to DEBUG in the DSODBTuning.cfg file (in InformationServerRoot\Server\DSODB\) and restart the EngMonApp service:
log4j.logger.com.ibm.datastage.runtime.engmonapp=DEBUG
log4j.logger.com.ibm.datastage.runtime.common=DEBUG

5) A service called ResTrackApp periodically examines the system state (for example CPU load, free memory, free disk space etc). The service runs on the engine tier and all related remote PXEngine systems. If a problem occurs during this operation an error is logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\PXEngine\DSResourceTracker\Tracker.<hostName>.log

Unix and Linux: InformationServerRoot/Server/PXEngine/DSResourceTracker/Tracker.<hostName>.log

To increase the amount of information logged in this file, change the following entries from INFO to DEBUG in the DSODBTuning.cfg file (in InformationServerRoot\Server\DSODB\) and restart the ResMonApp service (this setting affects both ResMonApp and ResTrackApp):
log4j.logger.com.ibm.datastage.runtime.resmonapp=DEBUG

6) A service called ResMonApp periodically connects to each instance of ResTrackApp running in #5 above and inserts the relevant rows into the DSODB. If a problem occurs during this operation an error is logged in the following file on the engine tier computer:

Windows: InformationServerRoot\Server\DSODB\logs\ResMonApp-<processStartTimestamp>.log

Unix and Linux: InformationServerRoot/Server/DSODB/logs/ResMonApp-<processStartTimestamp>.log

To increase the amount of information logged in this file, change the following entries from INFO to DEBUG in the DSODBTuning.cfg file (in InformationServerRoot\Server\DSODB\) and restart the ResMonApp service (this setting affects both ResMonApp and ResTrackApp):
log4j.logger.com.ibm.datastage.runtime.resmonapp=DEBUG
log4j.logger.com.ibm.datastage.runtime.common=DEBUG

Operations Console

7) When a user logs into the Operations Console details about authentication is logged in the following file on the services tier computer:

Windows: WebSphereRoot\AppServer\profiles\InfoSphere\logs\datastage_console_web.log

Unix and Linux: WebSphereRoot/AppServer/profiles/InfoSphere/logs/datastage_console_web.log

8) All requests for data by the Operations Console are made to a REST interface in the application server. If a problem occurs during this operation an error is logged in one or more of the following files on the services tier computer:

Windows:

WebSphereRoot\AppServer\profiles\InfoSphere\logs\datastage_api_restservices.log

WebSphereRoot\AppServer\profiles\InfoSphere\logs\datastage_api_services-server1.log

WebSphereRoot\AppServer\profiles\InfoSphere\logs\server1\SystemOut.log

WebSphereRoot\AppServer\profiles\InfoSphere\logs\server1\SystemErr.log

Unix and Linux:

WebSphereRoot/AppServer/profiles/InfoSphere/logs/datastage_api_restservices.log

WebSphereRoot/AppServer/profiles/InfoSphere/logs/datastage_api_services-server1.log

WebSphereRoot/AppServer/profiles/InfoSphere/logs/server1/SystemOut.log

WebSphereRoot/AppServer/profiles/InfoSphere/logs/server1/SystemErr.log

9) If a request for data by the Operations Console requires data to be read from the DSODB the application server connects to the engine tier in order to read the data from DSODB (specifically this connection is to the ASBAgent and ODBQueryApp services on the engine tier). If a problem occurs during this operation an error is logged in one or more of the following files on the engine tier computer:

Windows:

InformationServerRoot\Server\DSODB\logs\handler-<processStartTimestamp>.log

InformationServerRoot\Server\DSODB\logs\odbqueryapp-<processStartTimestamp>.log

InformationServerRoot\ASBNode\asbagent_startup[.out|.err]

InformationServerRoot\ASBNode\bin\Agent.out

InformationServerRoot\ASBNode\bin\LoggingAgent.out

Unix and Linux:

InformationServerRoot/Server/DSODB/logs/handler-<processStartTimestamp>.log

InformationServerRoot/Server/DSODB/logs/odbqueryapp-<processStartTimestamp>.log

InformationServerRoot/ASBNode/asbagent_startup[.out|.err]

InformationServerRoot/ASBNode/bin/Agent.out

InformationServerRoot/ASBNode/bin/LoggingAgent.out

To increase the amount of information logged in the odbqueryapp-<processStartTimestamp>.log file, change the following entries from INFO to DEBUG in the DSODBTuning.cfg file (in InformationServerRoot\Server\DSODB\) and restart the ODBQueryApp service:
log4j.logger.com.ibm.datastage.runtime.odbqapp.server=DEBUG
log4j.logger.com.ibm.datastage.runtime.common=DEBUG

To increase the amount of information logged in the handler-<processStartTimestamp>.log file, change the following entries from INFO to DEBUG in the DSODBTuning.cfg file (in InformationServerRoot\Server\DSODB\) and restart the ASBAgent service:
log4j.logger.com.ibm.datastage.asb.handler=DEBUG
log4j.logger.com.ibm.datastage.runtime.manager=DEBUG






Popular posts from this blog

Shrink you container size up to 95%.

-
BLAFS is a bloat-aware filesystem for container debloating. The design principles of BLAFS are effective, efficient, and easy to use. It detects the files used by the container, and then debloats the container by removing the unused files. The debloated containers are still functional and can run the same workload as the original containers, but with a much smaller size and faster deployment. https://github.com/negativa-ai/BLAFS?tab=readme-ov-file
Read more

alma linux: dnf Module yaml error: Unexpected key in data

-
While applying recent updates to a server running Rocky Linux 8, an error presented itself. Module yaml error: Unexpected key in data: static_context [line 9 col 3] Module yaml error: Unexpected key in data: static_context [line 9 col 3] The solution turned out to be a simple one. Update the libmodulemd first to correct the problem then perform the dnf updates as usual. dnf update libmodulemd For a detailed explanation as to the cause of this problem.
Read more