How to verify if DataStage project has any corrupted files

-

Question

If DataStage project or temp directory runs out of disk space, some of the buffered write requests might not get correctly completed and the project might ended up having corrupted hash files. There is one approach to check the project for corrupted hash files.

Answer

Run the uvbackup process and redirect the backup to null devices. The uvbackup produces an output file that can be used to identify the corrupted files.
Here are the steps (please note that you must be logged in as DataStage Administrator):

  1. Source your dsenv file in $DSHOME (. ./dsenv)
  2. Go to your project directory (../InformationServer/Project/<project name>)
  3. List all files and direct them to a file (ls > myfiles.txt) - this is used to list of files for the uvbackup
  4. Run the uvbackup and redirect output to null with this command: "$DSHOME/bin/uvbackup -V -f -cmdfil myfiles.txt -s uvbackupout.txt -t /dev/null  2>&1 > testing123.txt"
  5. grep "WARNING:" uvbackupout.txt


The output file: uvbackupout.txt will help to identify if there are any corrupted files in the project..

Here is an example what you might see in the uvbackupout.txt file (output of the grep command in step 5)

WARNING: Unable to open file 'RT_STATUS3' for reading. File not saved!^G

The uvbackup verifies the integrity of the files and will not backup any files that are corrupted.





Popular posts from this blog

Shrink you container size up to 95%.

-
BLAFS is a bloat-aware filesystem for container debloating. The design principles of BLAFS are effective, efficient, and easy to use. It detects the files used by the container, and then debloats the container by removing the unused files. The debloated containers are still functional and can run the same workload as the original containers, but with a much smaller size and faster deployment. https://github.com/negativa-ai/BLAFS?tab=readme-ov-file
Read more

alma linux: dnf Module yaml error: Unexpected key in data

-
While applying recent updates to a server running Rocky Linux 8, an error presented itself. Module yaml error: Unexpected key in data: static_context [line 9 col 3] Module yaml error: Unexpected key in data: static_context [line 9 col 3] The solution turned out to be a simple one. Update the libmodulemd first to correct the problem then perform the dnf updates as usual. dnf update libmodulemd For a detailed explanation as to the cause of this problem.
Read more