We need to create the File object to get its f.id, then lock that id.
During that gap, another instance of redo may have modified the file or
its state data, so we have to refresh it.
This fixes 'redo -j10 t/stress'.
It looks like we're updating the stamp for t/countall while another
task is replacing the file, which suggests a race condition in our
state management database.
flush-cache can cause files affected by redo-stamp to get rebuilt
unnecessarily, which the test is specifically trying to validate.
Since other tests run flush-cache at random times when using -j, this
would cause random test failures.
Because the two programs use separate state databases, it helps if we
clean up some temp files between runs. Otherwise they might think you
created some targets "by hand" and refuse to rebuild them.
I'm not quite sure what I was thinking, using redo-ifchange there, but
the result was that some tests wouldn't run if you run 'redo test'
repeatedly, even after modifying redo itself.
Also tweaked t/950-curse so that it always runs, not just the first
time.
I feel a little dirty doing this, but the way the code was before, redo
almost always picked bash as the shell. bash is way too overpowered
and this led to bashisms in do scripts unnecessarily. The two failures
in dash are things that I would really like to have, but they haven't
materialized after 6 years, so I guess we should be realistic.
To appropriately penalize bash for asking for trouble, I added a
warning about [ 1 == 1 ] syntax being valid (as opposed to the POSIX
correct [ 1 = 1 ]). This allows dash to be selected ahead of bash.
I also moved 'sh' to the end of the list, because although it's the
weakest shell on some systems, on other systems it's just bash. And I
put zsh in front of bash, because fewer people have zsh and we want
them to test zsh.
If a depends on b which depends on a, redo would just freeze. Now it
aborts with a somewhat helpful error message.
[Updated by apenwarr for coding style and to add a test.]
The >& form is only for file descriptors, passing a file name there is
a bash extension.
$ /bin/dash -c 'echo foo >&/dev/null'
/bin/dash: 1: Syntax error: Bad fd number
I think this aligns better with how redo works. Otherwise, if a.do
creates a as a symlink, then changes to the symlink's *target* will
change a's stat/stamp information without re-running a.do, which looks
to redo like you modified a by hand, which causes it to stop running
a.do altogether.
With this change, modifications to a's target are okay, but they don't
trigger any redo dependency changes. If you want that, then a.do
should redo-ifchange on its symlink target explicitly.
If you run something like
blah_function || return 1
then everything even *inside* blah_function is *not* subject to the "set -e"
that would otherwise be in effect. That's true even for ". subfile" inside
blah_function - which is exactly how minimal/do runs .do files.
Instead, rewrite it as
blah_function
[ "$?" = "0" ] || return 1
And add a bit to the unit tests to ensure that "set -e" behaviour is enabled
in .do files as we expect, and crash loudly otherwise.
(This weird behaviour may only happen in some shells and not others.)
Also, we had a "helpful" alias of redo() defined at the bottom of the file.
Combined with the way we use '.' to source the .do files, this would make it
not start a new shell just to run a recursive 'redo' command. It almost
works, but this stupid "set -e" bug could cause a nested .do file to not
honour "set -e" if someone ran "redo foo || exit 1" from inside a .do
script. The performance optimization is clearly not worth it here, so
rename it to _redo(); that causes it to actually re-exec the redo program
(which is a symlink to minimal/do).
If you use "redo --old-args", it will switch back to the old
(apenwarr-style) arguments for now, to give you time to update your .do
scripts. This option will go away eventually.
Note: minimal/do doesn't understand the --old-args option. If you're using
minimal/do in your project, keep using the old one until you update your use
of $1/$2, and then update to the new one.
apenwarr-style default.o.do:
$1 foo
$2 .o
$3 whatever.tmp
djb-style default.o.do:
$1 foo.o
$2 foo
$3 whatever.tmp
apenwarr-style foo.o.do:
$1 foo.o
$2 ""
$3 whatever.tmp
djb-style foo.o.do:
$1 foo.o
$2 foo.o (I think?)
$3 whatever.tmp
If the error message only triggered a shelltest warning instead of a
failure, then they're harmless, so we might as well silence them when
running 'redo test' (which runs t/shelltest.do, which enables
SHELLTEST_QUIET). We still want to print them when actually testing out
shelltest.od, though, to help with debugging the script.
If y contains a * character, it's a case-esac style wildcard unless it's in
"" or '', which tells it to match a literal *. But in dash/busybox, the ''
quoting doesn't actually work, only "" does.
Since there's a workaround - just always use "" quoting - it's not that
fatal. If the "" quoting doesn't work, then we'll make it fail.
We already did this in minimal/do, and redo-ifchange already did this, but
plain redo didn't. This made constructs like:
for d in *.x; do
echo "${d%.x}"
done | xargs redo
dangerous, because if there were no matching files, we'd try to 'redo all'.
It should just build nothing. Because sometimes you want to do something
like:
redo-ifchange $(find -name '*.c')
And the find doesn't return any results. This is consistent with what real
redo does.
Added a test to confirm that it works.
It was working fine, but the style wasn't exactly the way I like it, because
I'm unnecessarily picky. :)
Also, removed the file extension since we should probably learn from the
fact that it's already been rewritten once from one language to another.
Who knows, maybe it will be again someday.
Turns out we don't need sed to process the output of gcc -MD. We can just
do this:
read DEPS <filename.deps
The 'read' command in sh actually handles backslashes correctly, so we don't
have to sed them out after all. And then a simple ${DEPS#*:} removes the
"target:" prefix from the dependency line, and we're done!
By default, the database redo uses to store file state returns filenames
as Unicode strings, and if redo tries to run a build-script whose
fully-qualified path contains non-ASCII characters then redo crashes
when trying to promote the path to a Unicode string.
This patch ensures that the database always returns byte-strings, not
Unicode strings. That way, the fully-qualified path and the target name
are both byte-strings and can be joined without issue.
(Fixes a bug reported by Zoran Zaric.)
It seems 'set -e' doesn't quite work, in some shells, in that case.
Eric Kow wrote:
> I saw this warning, by the way:
> redo t/shelltest
> warning: 94
> cd: 271: can't cd to /opt/redo-0.05/t/space
> rmdir: failed to remove pace home dir': No such file or directory
> warning: 111
If we're using a .do file from a parent directory, we should set $3 using
the same path prefix as $1. We were previously using just the basename,
which mostly works (since we would rename it to $1$2 eventually anyway) but
is not quite right, and you can't safely rename files across filesystems, so
it could theoretically cause problems.
Also improved t/defaults-nested to test for this behaviour.
Reported by Eric Kow.
Supposedly it's not POSIX, but every shell I have seems to support it, so
let's just reject any that don't. And like magic, anybody using redo can
now count on the 'local' builtin working.
This fails on ash, dash, and busybox sh (for now). But it's kind of
important since $HOME often has spaces on Win32. I hope dash will be fixed
relatively soon.
I downgraded it to a warning since on Unix, this probably isn't a problem.
