Commit graph

292 commits

Author SHA1 Message Date
Avery Pennarun
93ab32d0f9 jwack: has not been actually executable since the earlier rewrite. 2010-11-16 00:28:01 -08:00
Avery Pennarun
8aea96cc90 Oops, don't use the SHELL environment variable after all.
Reading the docs for GNU make more closely, it seems they *don't* use the
one from the environment, because the user's interactive shell preferences
shouldn't affect how the Makefile runs.  Good point.

In a Makefile, you can define SHELL explicitly, and that works.  But let's
worry about that some other time.
2010-11-16 00:28:01 -08:00
Avery Pennarun
4243f31e1b Add minimal/do, a stripped-down redo implementation in 977 bytes of sh.
This could be good for distributing with your packages, so that people who
don't have redo installed can at least build it.  Also, we could use it for
building redo itself.

Will surely need to get slightly bigger as I inevitably discover I've
forgotten a critical feature.
2010-11-16 00:27:52 -08:00
Avery Pennarun
3803da525c Notice sooner when make has "helpfully" closed its job control file descriptors. 2010-11-13 05:05:48 -08:00
Avery Pennarun
662f53896a _try_read: hacky fix to make GNU make less angry.
We'll have to stop using nonblocking reads, unfortunately.  But this seems
to work better than nothing.  There's still a race condition that could
theoretically make GNU make angry, unfortunately, since we briefly set the
socket to nonblocking.
2010-11-13 04:50:53 -08:00
Avery Pennarun
9d5afd67f0 redo.py: shuffle the list of targets.
This encourages greater randomness in builds, which both decreases lock
contention and should help with testing the consistency of parallel builds.
2010-11-13 04:42:13 -08:00
Avery Pennarun
bd5daf9754 Totally disgusting support for jobservers.
It needs some heavy rethinking and cleanup.  But it seems to work!  And it's
even compatible with GNU make, apparently!
2010-11-13 04:36:44 -08:00
Avery Pennarun
ae3b6ee363 Basic locking infrastructure.
So that more than one redo doesn't try to build the same thing at the same
time.  Kind of dumb, though, since it currently wipes out all the locks at
the toplevel, so running more than one at a time won't give accurate
results, but the -j option doesn't do anything yet.
2010-11-13 02:50:39 -08:00
Avery Pennarun
534dd9813f t/*.do: add some time delays.
These only take effect if you export SLEEP=1.  Useful for testing
parallelism.
2010-11-13 02:17:22 -08:00
Avery Pennarun
3de384497d Handle KeyboardInterrupt with less annoying spew to stderr. 2010-11-13 02:09:42 -08:00
Avery Pennarun
9c349cf652 Fix log() output so you can cut-and-paste it verbatim into a command line.
It used to say:
	redo:   t/all
	redo:     hello
and now it says:
	redo    t/all
	redo      t/hello

ie. there's no colon, and the path is intact.  That means if the build
fails, you can cut-and-paste 'redo   t/hello', add a -v, and try to debug
what went wrong.
2010-11-13 01:55:07 -08:00
Avery Pennarun
3c42645cdf Given a directory name as a target, try building dirname/all.
...if dirname/all.do exists.  This is a convenient command-line shortcut.
2010-11-13 01:45:16 -08:00
Avery Pennarun
edd8382a52 Change the default rule name from 'it' to 'all'.
This is a departure from how djb seems to have it set up, but I just like it
better.  It's more like the reasonably-common Makefile standard.  (Although
what make *actually* does is just use the first target declared in the
file.)
2010-11-13 01:40:37 -08:00
Avery Pennarun
7505048093 redo-if*.py: import fewer things from helpers.
Just to keep track of how many helper functions we actually are using.  In
case I get brave and try to convert to C sometime.
2010-11-13 01:40:01 -08:00
Avery Pennarun
7a8b7340a3 Use the SHELL variable instead of assuming 'sh'.
This is what GNU make does.  If SHELL isn't defined, we still fall back to
calling sh.

Rumour has it that Google has some kind of build system that can be
massively distributed if you just set SHELL to the right program; maybe
it'll work with redo now.  (Of course it won't do you any good until we
implement parallel builds...)
2010-11-13 01:30:31 -08:00
Avery Pennarun
b9b1d18c3f log: print a newline before each log() if VERBOSE is set.
This gives relatively nice separation of subsections, I think.
2010-11-13 01:28:42 -08:00
Avery Pennarun
5417d0165a redo-ifcreate: barf if the file already exists.
I'm pretty sure this must be the intended behaviour.  It's kind of
meaningless to use this to declare a dependency on a file that might start
to exist later, if the file already exists.
2010-11-13 01:22:11 -08:00
Avery Pennarun
0bd6d07155 Prettier printing of errors. 2010-11-13 01:21:59 -08:00
Avery Pennarun
91cc82ff4a Add redo-ifcreate to match djb's design.
I'm not quite sure why this needs to be a separate program, however, so I'm
probably not testing it correctly.
2010-11-13 01:10:43 -08:00
Avery Pennarun
89449b909c redo.py: don't add dependencies on files we rebuild.
redo-ifchange already does this, and then calls redo.py, so it's redundant.

The only weird case is if a .do file calls 'redo' instead of
'redo-ifchange'.  I'm not sure what exactly is supposed to happen here.
test.do does this, for example, and it works, but only because we always do
'redo test' and nobody depends on 'redo-ifchange test'; if they did, the
dependencies would be off, I guess.  But maybe it's meaningless to do that
anyway.
2010-11-13 01:10:29 -08:00
Avery Pennarun
2c3df2c80a Don't throw a weird exception if REDO_BASE isn't set. 2010-11-13 01:08:28 -08:00
Avery Pennarun
fc27b19108 Merge libdo.py and log.py into helpers.py. 2010-11-13 00:53:55 -08:00
Avery Pennarun
c57de820fb Move 'redo --ifchange' into 'redo-ifchange' to match djb's style.
It does simplify the logic of both redo.py and redo-ifchange.py, I suppose.
2010-11-13 00:47:49 -08:00
Avery Pennarun
63c596ac61 Move log stuff into a separate log.py. 2010-11-13 00:11:34 -08:00
Avery Pennarun
440af04821 Move creation of .redo directory.
This fixes "./redo clean test", since clean.do deletes .redo.
2010-11-12 23:58:39 -08:00
Avery Pennarun
903793ecd9 Colourize redo's output. 2010-11-12 23:58:32 -08:00
Avery Pennarun
2e71e20ce2 When auto-choosing a .redo dir, prefer ones uphill that already exist.
So if we're in redo/t/ and running 'redo hello', we'll detect redo/.redo and
use that one rather than creating a new redo/t/.redo directory.

The downside of this is we get slightly different behaviour if the *first*
thing you build isn't from the root.  Probably that's bad, but it should
hopefully be rare.
2010-11-12 23:23:06 -08:00
Avery Pennarun
690d1dd7f0 Makefile: use a .PHONY rule to force everything to forward to redo. 2010-11-12 23:19:27 -08:00
Avery Pennarun
3c16056c21 When running x/y/z/whatever.do, first cd to x/y/z.
That way it's exactly equivalent to
	cd x/y/z; redo whatever
2010-11-12 23:14:58 -08:00
Avery Pennarun
45c6aad649 use os.environ[] instead of os.getenv/os.putenv.
Oddly, in python, os.putenv() does not make changes that os.getenv() can see.
2010-11-12 23:14:02 -08:00
Avery Pennarun
f43187b73a 'make clean' can now safely remove .redo dirs. 2010-11-12 22:50:23 -08:00
Avery Pennarun
425a295d6f Use a single .redo dir for the entire build. 2010-11-12 22:47:03 -08:00
Avery Pennarun
3c2fc49c63 Complicate the test build a bit to show unnecessary dependency re-checking.
'./redo -d test' reveals that we check and re-check things like CC and LD
even when they're children of objects that don't need to be rebuilt, and who
we've already checked earlier in the build cycle.  We need to be a little
smarter about pruning if we want dependency checking to be fast.
2010-11-12 22:15:34 -08:00
Avery Pennarun
2ed0511094 Cleaner environment variable management in redo.py. 2010-11-12 22:14:39 -08:00
Avery Pennarun
7cbf39d52a Make jwack... mostly... work with GNU make.
But it seems to be pretty unsolvable in the current form; the problem is
that when you're nesting one jwack inside the other and the jobserver is GNU
make, there's no way to tell the parent jwack not to use up a token.  Thus,
if you nest too deeply, it just deadlocks.

So this approach isn't really going to work the way it is.
2010-11-12 21:09:29 -08:00
Avery Pennarun
f77e4b5c91 Add jwack, a GNU make-like jobserver.
Theoretically compatible with GNU make's jobserver pipes.  Haven't tested
that yet.
2010-11-12 20:10:21 -08:00
Avery Pennarun
07d3e3b13c Extremely basic integration of wvtest for testing.
And move test scripts into t/ subdir to keep things clean.

As part of that, make sure redo adds itself to the PATH on startup so that
subscripts can find it.
2010-11-12 17:19:42 -08:00
Avery Pennarun
ae5b71fbc1 Add a -v (verbose) option. 2010-11-12 07:10:55 -08:00
Avery Pennarun
9a45f066f8 Add actual dependency checking. 2010-11-12 07:03:06 -08:00
Avery Pennarun
ee8a3c8c6e More comprehensible output as we produce stuff. 2010-11-12 05:32:38 -08:00
Avery Pennarun
7c14f21613 A Makefile that just forwards everything to redo. 2010-11-12 05:32:35 -08:00
Avery Pennarun
a51764c907 Extremely basic first crack at implementing djb's redo.
And a test program.
2010-11-12 05:25:03 -08:00