NAME

robsd-regress — run OpenBSD regression tests

SYNOPSIS

DESCRIPTION

The robsd-regress utility runs OpenBSD regression tests. The process is divided into the steps as follows:

env

Dump the current environment. This is a no operation only used to collect data useful while debugging.

pkg-add

Install all packages(7) associated with the configured regression tests. If a package already is installed, it is ignored by the pkg-del step. Failing to install a package is not considered fatal.

cvs

Update or checkout the bsd-srcdir source directory. This step is optional and requires presence of certain configuration, see robsd-regress.conf(5).

patch

Apply patches including sticky ones, see robsd-regress.conf(5).

obj

Create object directories for all configured regression tests.

mount

Optionally mount bsd-srcdir read-only if rdonly is enabled, see robsd-regress.conf(5).

regress

Run all configured regression tests, by default in parallel; see robsd-regress.conf(5).

umount

Inverse of mount step.

revert

Inverse of patch step.

pkg-del

Inverse of pkg-add step.

dmesg

Take a copy of dmesg(8).

end

Auxillary step used to generate report.

Each invocation is identified by a directory created by robsd-regress rooted in robsddir and is named after the current date. The directory contains the entries as follows. In addition, each regression test has a corresponding log file in the same directory.

comment

File used to annotate the invocation which is included in report.

dmesg

Copy of dmesg(8).

report

A summary delivered as a mail to root while running in the background once robsd-regress exits.

robsd.log

Standard output and error from robsd-regress while running in the background.

stat.csv

Collected statistics, see robsd-stat(8).

step.csv

Internal representation of the executed steps, see robsd-step(8).

tags

Tags included in report.

robsd-regress is configured using robsd-regress.conf(5) and must run as root. Some steps can however operate on behalf of another user. By default, robsd-regress runs in the background and must be killed using robsd-kill(8) to ensure a clean exit.

The options are as follows:

-d

Do not detach robsd-regress into the background.

-S src-patch

Patch to apply to bsd-srcdir. The patch path names can be relative to any directory in bsd-srcdir. The same patch will be reverted by the revert step, see above. In addition, a copy of the patch is saved to a file named src.diff.N in the invocation directory where N is incremented for every given patch.

-c comment

Path to comment included in report.

-r path

Resume an invocation located at path. If the last regression test failed or aborted, it will be executed again. Otherwise, the next regression test will be executed.

-t tag

Arbitrary tag used to annotate the invocation which is included in report. May be given multiple times.

FILES

/etc/robsd-regress.conf

The default configuration file.

DIAGNOSTICS

The robsd-regress utility exits 0 on success, and >0 if an error occurs.

SEE ALSO

bsd.regress.mk(5), robsd-regress.conf(5), robsd(8), robsd-regress-html(8), robsd-regress-log(8), robsd-stat(8)

AUTHORS

Anton Lindqvist <anton@basename.se>