If you use "redo --old-args", it will switch back to the old
(apenwarr-style) arguments for now, to give you time to update your .do
scripts. This option will go away eventually.
Note: minimal/do doesn't understand the --old-args option. If you're using
minimal/do in your project, keep using the old one until you update your use
of $1/$2, and then update to the new one.
apenwarr-style default.o.do:
$1 foo
$2 .o
$3 whatever.tmp
djb-style default.o.do:
$1 foo.o
$2 foo
$3 whatever.tmp
apenwarr-style foo.o.do:
$1 foo.o
$2 ""
$3 whatever.tmp
djb-style foo.o.do:
$1 foo.o
$2 foo.o (I think?)
$3 whatever.tmp
If the error message only triggered a shelltest warning instead of a
failure, then they're harmless, so we might as well silence them when
running 'redo test' (which runs t/shelltest.do, which enables
SHELLTEST_QUIET). We still want to print them when actually testing out
shelltest.od, though, to help with debugging the script.
If y contains a * character, it's a case-esac style wildcard unless it's in
"" or '', which tells it to match a literal *. But in dash/busybox, the ''
quoting doesn't actually work, only "" does.
Since there's a workaround - just always use "" quoting - it's not that
fatal. If the "" quoting doesn't work, then we'll make it fail.
We already did this in minimal/do, and redo-ifchange already did this, but
plain redo didn't. This made constructs like:
for d in *.x; do
echo "${d%.x}"
done | xargs redo
dangerous, because if there were no matching files, we'd try to 'redo all'.
It should just build nothing. Because sometimes you want to do something
like:
redo-ifchange $(find -name '*.c')
And the find doesn't return any results. This is consistent with what real
redo does.
Added a test to confirm that it works.
It was working fine, but the style wasn't exactly the way I like it, because
I'm unnecessarily picky. :)
Also, removed the file extension since we should probably learn from the
fact that it's already been rewritten once from one language to another.
Who knows, maybe it will be again someday.
Turns out we don't need sed to process the output of gcc -MD. We can just
do this:
read DEPS <filename.deps
The 'read' command in sh actually handles backslashes correctly, so we don't
have to sed them out after all. And then a simple ${DEPS#*:} removes the
"target:" prefix from the dependency line, and we're done!
By default, the database redo uses to store file state returns filenames
as Unicode strings, and if redo tries to run a build-script whose
fully-qualified path contains non-ASCII characters then redo crashes
when trying to promote the path to a Unicode string.
This patch ensures that the database always returns byte-strings, not
Unicode strings. That way, the fully-qualified path and the target name
are both byte-strings and can be joined without issue.
(Fixes a bug reported by Zoran Zaric.)
It seems 'set -e' doesn't quite work, in some shells, in that case.
Eric Kow wrote:
> I saw this warning, by the way:
> redo t/shelltest
> warning: 94
> cd: 271: can't cd to /opt/redo-0.05/t/space
> rmdir: failed to remove pace home dir': No such file or directory
> warning: 111
If we're using a .do file from a parent directory, we should set $3 using
the same path prefix as $1. We were previously using just the basename,
which mostly works (since we would rename it to $1$2 eventually anyway) but
is not quite right, and you can't safely rename files across filesystems, so
it could theoretically cause problems.
Also improved t/defaults-nested to test for this behaviour.
Reported by Eric Kow.
Supposedly it's not POSIX, but every shell I have seems to support it, so
let's just reject any that don't. And like magic, anybody using redo can
now count on the 'local' builtin working.
This fails on ash, dash, and busybox sh (for now). But it's kind of
important since $HOME often has spaces on Win32. I hope dash will be fixed
relatively soon.
I downgraded it to a warning since on Unix, this probably isn't a problem.
The result was that t/deps/dirtest was actually failing in some cases. But
it wasn't failing quite reliably enough, because the failing test was
dirtest/dir1/all, which has the same name as some other 'all' files,
confusing the issue. Renamed dirtest/dir1/all.do to dirtest/dir1/go.do instead.
Reported by Prakhar Goel and Berke Durak.
* master:
Fixed markdown errors in README - code samples now correctly formatted.
Fix use of config.sh in example
log.py, minimal/do: don't use ansi colour codes if $TERM is blank or 'dumb'
Use named constants for terminal control codes.
redo-sh: keep testing even after finding a 'good' shell.
redo-sh.do: hide warning output from 'which' in some shells.
redo-sh.do: wrap long lines.
Handle .do files that start with "#!/" to specify an explicit interpreter.
minimal/do: don't print an error on exit if we don't build anything.
bash completions: also mark 'do' as a completable command.
bash completions: work correctly when $cur is an empty string.
bash completions: call redo-targets for a more complete list.
bash completions: work correctly with subdirs, ie. 'redo t/<tab>'
Sample bash completion rules for redo targets.
minimal/do: faster deletion of stamp files.
minimal/do: delete .tmp files if a build fails.
minimal/do: use ".did" stamp files instead of empty target files.
minimal/do: use posix shell features instead of dirname/basename.
Automatically select a good shell instead of relying on /bin/sh.
Conflicts:
t/clean.do
This includes a fairly detailed test of various known shell bugs from the
autoconf docs.
The idea here is that if redo works on your system, you should be able to
rely on a *good* shell to run your .do files; you shouldn't have to work
around zillions of bugs like autoconf does.
Previously, we would only search for default*.do in the same directory in
the target; now we search parent directories as well.
Let's say we're in a/b/ and trying to build foo.o. If we find
../../default.o.do, then we'll run
cd ../..; sh default.o.do a/b/foo .o $TMPNAME
In other words, we still always chdir to the same directory as the .do file.
But now $1 might have a path in it, not just a basename.
The previous method, using fcntl byterange locks, was very efficient and
avoided unnecessarily filesystem metadata churn (ie. creating/deleting
inodes). Unfortunately, MacOS X (at least version 10.6.5) apparently has a
race condition in its fcntl locking that makes it unusably unreliable
(http://apenwarr.ca/log/?m=201012#13).
My tests indicate that if you only ever lock a *single* byterange on a file,
the race condition doesn't cause a problem. So let's just use one lockfile
per target. Now "redo -j20 test" passes for me on both MacOS and Linux.
This doesn't measurably affect the speed on Linux, at least, in my tests.
The bad news: it's hard to safely *delete* those lockfiles when we're done
with them, so they tend to accumulate in the .redo dir.
This comes down to the lack of a 'seq' command (what?!) and the fact that
BSD "wc -l" returns extra whitespace, while the GNU version doesn't. We
should be using numeric comparisons instead of string comparisons, and then
it's ok.
...only when running under minimal/do, of course.
The tests in question mostly fail because they're testing particular
dependency-related behaviour, and minimal/do doesn't support dependencies,
so naturally it doesn't work.
If a checksummed target A used to exist but is now missing, and we tried to
redo-ifchange that exact file, we would unnecessarily run 'redo-oob A A';
that is, we have to build A in order to determine if A needs to be built.
The sub-targets of redo-oob aren't run with REDO_UNLOCKED, so this would
deadlock instantly.
Add an assertion to redo-oob to ensure we never try to redo-ifchange the
primary target (thus converting the deadlock into an exception). And skip
doing redo-oob when the target is already the same as the thing we have to
check.
We were giving up and rebuilding the toplevel object, which did eventually
rebuild our checksummed file, but then the file turned out to be identical
to what it was before, so that nobody *else* who depended on it ended up
getting rebuilt. So the results were indeterminate.
Now we treat it as if its dirtiness is unknown, so we build it using
redo-oob before building any of its dependencies.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
