· Solaris · Unix

Compare Solaris Directories with dircmp and Show Only Differences

dircmp -s compares two Solaris directory trees and lists only the files that differ or exist on one side, for checking a copy, a restore or a migration.

Compare Solaris Directories with dircmp and Show Only Differences

Purpose

After a tar pipe copies /var to /var1, you need to check the copy before the old tree goes away. Paging through every same line to find the files that did not match is slow. dircmp -s removes those lines.

dircmp walks both directory trees. It always lists the files found in only one of them , and for each name the two trees share, it reports whether the contents match. The -s flag suppresses messages about identical files , so what is left is the list of differences. This command was used on Solaris 7 through 10 systems. On Solaris 10 it ships in the SUNWesu package .

It is one of the commands collected on the Solaris commands and scripts page , under the entry "Directory compare (don't show files that are the same)".

Code

1dircmp -s /var /var1

/var is dir1 and /var1 is dir2 in the synopsis dircmp [ -ds ] [ -w n ] dir1 dir2.

Code Breakdown

dircmp

On illumos , dircmp is a shell script with a 2009 Sun Microsystems copyright. Script details below are from the illumos copy. The race condition under Key Points was fixed by patching this script, so an unpatched Solaris 8, 9 or 10 system can handle temp files differently.

The illumos script sets PATH=/usr/bin at the top, so it runs the system's own find, sort and cmp, not whatever the caller has in their path. It first makes a scratch directory with mktemp -d /var/tmp/dir.XXXXXX and removes it on exit. Then it changes into each tree in turn and runs find . -print | sort. That makes the comparison recursive: every file and subdirectory at every depth is included. comm then splits the two sorted lists into names common to both trees and names unique to each one.

-s

On Solaris 10 and illumos , -s "Suppresses messages about identical files." The illumos script hides more than that, though. Each of these lines is gated on if (( Sflag != 1 )) in dircmp.sh :

  • same <name>: common files with matching contents.
  • directory <name>: subdirectories present in both trees.
  • special <name>: common entries that are not regular files or directories.

Two things are still printed with -s. The script always prints the different <name> line for a common regular file whose size or contents differ. The listing of files unique to each directory is generated for all the options .

/var and /var1 (dir1 and dir2)

Operand order sets the headings. In the illumos script, the unique-files listing goes out under the header $D1 only and $D2 only (dircmp.sh ). Here that reads /var only and /var1 only, with the two lists merged side by side by pr -m. Swap the operands and the columns swap too.

With the illumos script, these operands have a side effect. The scratch directory is created under /var/tmp before find runs in /var. So the /var column can show an entry under ./tmp/dir. with a random suffix. That entry is the scratch directory dircmp made for itself, not a missing file in the copy.

Reading the output

The illumos script prints two sections:

  1. The unique-files listing, headed /var only and /var1 only.
  2. The comparison listing, headed Comparison of /var /var1. With -s, this holds the different lines.

For each common name, the illumos script collects sizes with ls -lLgnd in both trees. The -lgn flags stop long user or group names from breaking the field layout the script parses. When two sizes differ, the file is marked different and cmp is skipped. cmp runs only when the sizes match, to decide between same and different.

Options not used here

Key Points

  • The synopsis is the same on Solaris 10 and Solaris 11.4 (dircmp [-ds] [-w n] dir1 dir2), and -s means the same thing on illumos .
  • The package changed. It is SUNWesu on Solaris 10 and system/core-os on Solaris 11.4.
  • The exit status does not report differences: "Differences in directory contents are not considered errors."
  • Large files: on Solaris 10, a large file is 2 GB or more, and dircmp is large file aware . That means it processes large files the same way it processes small ones. illumos keeps the largefile reference, now as largefile(7). The Solaris 11.4 man page has no largefile reference. Its See Also stops at cmp(1), diff(1), attributes(7) and environ(7).
  • A race condition in the dircmp script (bug 6633566, Sun Alert 1020168.1 ) lets a local unprivileged user overwrite or create arbitrary files, with the privileges of whoever runs dircmp. The fixes are patches 140837-01, 138896-01 and 141014-01 for Solaris 8, 9 and 10 on SPARC, and 140838-01, 138897-01 and 141015-01 on x86. OpenSolaris is fixed in build snv_112 and later.

Insights and Best Practices

Safe usage

The race condition has no workaround and no predictable symptoms . On an unpatched Solaris 8, 9 or 10 box, every dircmp run as root gives a local user the chance to write files as root. Apply the patch for your release and platform first. Then run the comparison as the least-privileged account that can read both trees.

dircmp vs diff -r

On Solaris 10, diff takes two directories and -r "Applies diff recursively to common subdirectories encountered." Its output looks like Common subdirectories: dir1/x and dir2/x and Only in dir2/x: y. Two differences matter:

  • diff exits 0 when it finds no differences, 1 when it finds some, and above 1 on an error. dircmp does not use its exit status this way.
  • diff -s does the opposite of dircmp -s: it "Reports files that are identical."

On large files, diff is only large file safe . That means it cannot process a large file properly but returns an error. dircmp is large file aware.

Scripting the command

Because the exit status ignores differences, an if dircmp ... test passes whether the trees match or not. Use the text instead. Count the lines that contain different in the comparison section, and check whether either column of the only listing has entries. If a script needs a pass/fail exit code, diff -r provides one.

What to log

Save the output to a file that has the host name and the date in its name. That way a later run can be compared with the one before it. Pass a larger -w when paths are long, so the side-by-side only columns fit. The locale variables LC_COLLATE, LC_CTYPE, LC_MESSAGES and NLSPATH affect dircmp , and the script sorts its file lists. Run both sides of a logged comparison under the same locale.

When to Run This

  • After a tar pipe or cpio copy from one filesystem to another, before you delete the source.
  • Before and after a patch run, against a saved copy of a configuration tree.
  • After a restore from backup, against the tree that is still live.
  • During a filesystem migration, when /var moves to new disk as /var1.
  • When you compare a configuration directory on two hosts with both trees mounted on one machine.

Troubleshooting Common Issues

The messages and exit codes below come from the illumos dircmp.sh . An unpatched Solaris 8, 9 or 10 system runs the older script, whose temp-file handling differs.

  • /var1 not a directory !: an operand was given but is not a directory, and the script exits 2. Check it with ls -ld /var1.
  • The usage line usage: dircmp [-d] [-s] [-wn] dir1 dir2: fewer than two operands were given (exit 1) or an unknown option was passed (exit 2).
  • dircmp: numeric argument required: the value after -w is not a number. Pass an integer, for example -w 132.
  • Exit 1 before any listing: mktemp -d /var/tmp/dir.XXXXXX returned nothing, and the script exits 1 before it checks its arguments. Check that /var/tmp exists and is writable with ls -ld /var/tmp.
  • Exit status 0 although files differ: this is documented behaviour. Read the different lines and the only listing.
  • A ./tmp/dir. entry in the /var column: this is the scratch directory the illumos script creates under /var/tmp, not a file lost in the copy.
  • is an object file under -d: the file is not text, so no diff is printed. The comparison section still marks it same or different.
  • Unpatched system: compare the installed patches with the IDs in Sun Alert 1020168.1 before you run dircmp as root.

References