The >& form is only for file descriptors, passing a file name there is
a bash extension.
$ /bin/dash -c 'echo foo >&/dev/null'
/bin/dash: 1: Syntax error: Bad fd number
I think this aligns better with how redo works. Otherwise, if a.do
creates a as a symlink, then changes to the symlink's *target* will
change a's stat/stamp information without re-running a.do, which looks
to redo like you modified a by hand, which causes it to stop running
a.do altogether.
With this change, modifications to a's target are okay, but they don't
trigger any redo dependency changes. If you want that, then a.do
should redo-ifchange on its symlink target explicitly.
If you run something like
blah_function || return 1
then everything even *inside* blah_function is *not* subject to the "set -e"
that would otherwise be in effect. That's true even for ". subfile" inside
blah_function - which is exactly how minimal/do runs .do files.
Instead, rewrite it as
blah_function
[ "$?" = "0" ] || return 1
And add a bit to the unit tests to ensure that "set -e" behaviour is enabled
in .do files as we expect, and crash loudly otherwise.
(This weird behaviour may only happen in some shells and not others.)
Also, we had a "helpful" alias of redo() defined at the bottom of the file.
Combined with the way we use '.' to source the .do files, this would make it
not start a new shell just to run a recursive 'redo' command. It almost
works, but this stupid "set -e" bug could cause a nested .do file to not
honour "set -e" if someone ran "redo foo || exit 1" from inside a .do
script. The performance optimization is clearly not worth it here, so
rename it to _redo(); that causes it to actually re-exec the redo program
(which is a symlink to minimal/do).
If you use "redo --old-args", it will switch back to the old
(apenwarr-style) arguments for now, to give you time to update your .do
scripts. This option will go away eventually.
Note: minimal/do doesn't understand the --old-args option. If you're using
minimal/do in your project, keep using the old one until you update your use
of $1/$2, and then update to the new one.
apenwarr-style default.o.do:
$1 foo
$2 .o
$3 whatever.tmp
djb-style default.o.do:
$1 foo.o
$2 foo
$3 whatever.tmp
apenwarr-style foo.o.do:
$1 foo.o
$2 ""
$3 whatever.tmp
djb-style foo.o.do:
$1 foo.o
$2 foo.o (I think?)
$3 whatever.tmp
If the error message only triggered a shelltest warning instead of a
failure, then they're harmless, so we might as well silence them when
running 'redo test' (which runs t/shelltest.do, which enables
SHELLTEST_QUIET). We still want to print them when actually testing out
shelltest.od, though, to help with debugging the script.
If y contains a * character, it's a case-esac style wildcard unless it's in
"" or '', which tells it to match a literal *. But in dash/busybox, the ''
quoting doesn't actually work, only "" does.
Since there's a workaround - just always use "" quoting - it's not that
fatal. If the "" quoting doesn't work, then we'll make it fail.
We already did this in minimal/do, and redo-ifchange already did this, but
plain redo didn't. This made constructs like:
for d in *.x; do
echo "${d%.x}"
done | xargs redo
dangerous, because if there were no matching files, we'd try to 'redo all'.
It should just build nothing. Because sometimes you want to do something
like:
redo-ifchange $(find -name '*.c')
And the find doesn't return any results. This is consistent with what real
redo does.
Added a test to confirm that it works.
It was working fine, but the style wasn't exactly the way I like it, because
I'm unnecessarily picky. :)
Also, removed the file extension since we should probably learn from the
fact that it's already been rewritten once from one language to another.
Who knows, maybe it will be again someday.
Turns out we don't need sed to process the output of gcc -MD. We can just
do this:
read DEPS <filename.deps
The 'read' command in sh actually handles backslashes correctly, so we don't
have to sed them out after all. And then a simple ${DEPS#*:} removes the
"target:" prefix from the dependency line, and we're done!
By default, the database redo uses to store file state returns filenames
as Unicode strings, and if redo tries to run a build-script whose
fully-qualified path contains non-ASCII characters then redo crashes
when trying to promote the path to a Unicode string.
This patch ensures that the database always returns byte-strings, not
Unicode strings. That way, the fully-qualified path and the target name
are both byte-strings and can be joined without issue.
(Fixes a bug reported by Zoran Zaric.)
It seems 'set -e' doesn't quite work, in some shells, in that case.
Eric Kow wrote:
> I saw this warning, by the way:
> redo t/shelltest
> warning: 94
> cd: 271: can't cd to /opt/redo-0.05/t/space
> rmdir: failed to remove pace home dir': No such file or directory
> warning: 111
If we're using a .do file from a parent directory, we should set $3 using
the same path prefix as $1. We were previously using just the basename,
which mostly works (since we would rename it to $1$2 eventually anyway) but
is not quite right, and you can't safely rename files across filesystems, so
it could theoretically cause problems.
Also improved t/defaults-nested to test for this behaviour.
Reported by Eric Kow.
Supposedly it's not POSIX, but every shell I have seems to support it, so
let's just reject any that don't. And like magic, anybody using redo can
now count on the 'local' builtin working.
This fails on ash, dash, and busybox sh (for now). But it's kind of
important since $HOME often has spaces on Win32. I hope dash will be fixed
relatively soon.
I downgraded it to a warning since on Unix, this probably isn't a problem.
The result was that t/deps/dirtest was actually failing in some cases. But
it wasn't failing quite reliably enough, because the failing test was
dirtest/dir1/all, which has the same name as some other 'all' files,
confusing the issue. Renamed dirtest/dir1/all.do to dirtest/dir1/go.do instead.
Reported by Prakhar Goel and Berke Durak.
* master:
Fixed markdown errors in README - code samples now correctly formatted.
Fix use of config.sh in example
log.py, minimal/do: don't use ansi colour codes if $TERM is blank or 'dumb'
Use named constants for terminal control codes.
redo-sh: keep testing even after finding a 'good' shell.
redo-sh.do: hide warning output from 'which' in some shells.
redo-sh.do: wrap long lines.
Handle .do files that start with "#!/" to specify an explicit interpreter.
minimal/do: don't print an error on exit if we don't build anything.
bash completions: also mark 'do' as a completable command.
bash completions: work correctly when $cur is an empty string.
bash completions: call redo-targets for a more complete list.
bash completions: work correctly with subdirs, ie. 'redo t/<tab>'
Sample bash completion rules for redo targets.
minimal/do: faster deletion of stamp files.
minimal/do: delete .tmp files if a build fails.
minimal/do: use ".did" stamp files instead of empty target files.
minimal/do: use posix shell features instead of dirname/basename.
Automatically select a good shell instead of relying on /bin/sh.
Conflicts:
t/clean.do
This includes a fairly detailed test of various known shell bugs from the
autoconf docs.
The idea here is that if redo works on your system, you should be able to
rely on a *good* shell to run your .do files; you shouldn't have to work
around zillions of bugs like autoconf does.
Previously, we would only search for default*.do in the same directory in
the target; now we search parent directories as well.
Let's say we're in a/b/ and trying to build foo.o. If we find
../../default.o.do, then we'll run
cd ../..; sh default.o.do a/b/foo .o $TMPNAME
In other words, we still always chdir to the same directory as the .do file.
But now $1 might have a path in it, not just a basename.
The previous method, using fcntl byterange locks, was very efficient and
avoided unnecessarily filesystem metadata churn (ie. creating/deleting
inodes). Unfortunately, MacOS X (at least version 10.6.5) apparently has a
race condition in its fcntl locking that makes it unusably unreliable
(http://apenwarr.ca/log/?m=201012#13).
My tests indicate that if you only ever lock a *single* byterange on a file,
the race condition doesn't cause a problem. So let's just use one lockfile
per target. Now "redo -j20 test" passes for me on both MacOS and Linux.
This doesn't measurably affect the speed on Linux, at least, in my tests.
The bad news: it's hard to safely *delete* those lockfiles when we're done
with them, so they tend to accumulate in the .redo dir.
This comes down to the lack of a 'seq' command (what?!) and the fact that
BSD "wc -l" returns extra whitespace, while the GNU version doesn't. We
should be using numeric comparisons instead of string comparisons, and then
it's ok.
