The second half of redo-stamp: out-of-order building.

If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty.  Eek!

Luckily, there's a safe solution.  If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.

If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.

Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed().  Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not.  Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.

...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.

builder.py

@@ -65,14 +65,23 @@ class BuildJob:
 
     def start(self):
         assert(self.lock.owned)
-        t = self.t
-        sf = self.sf
         try:
-            if not self.shouldbuildfunc(t):
+            dirty = self.shouldbuildfunc(self.t)
+            if not dirty:
                 # target doesn't need to be built; skip the whole task
                 return self._after2(0)
         except ImmediateReturn, e:
             return self._after2(e.rv)
+
+        if dirty == True:
+            self._start_do()
+        else:
+            self._start_oob(dirty)
+
+    def _start_do(self):
+        assert(self.lock.owned)
+        t = self.t
+        sf = self.sf
         newstamp = sf.read_stamp()
         if (sf.is_generated and
             not sf.failed_runid and
@@ -132,6 +141,27 @@ class BuildJob:
         state.commit()
         jwack.start_job(t, self._do_subproc, self._after)
 
+    def _start_oob(self, dirty):
+        # out-of-band redo of some sub-objects.  This happens when we're not
+        # quite sure if t needs to be built or not (because some children look
+        # dirty, but might turn out to be clean thanks to checksums).  We have
+        # to call redo-oob to figure it all out.
+        #
+        # Note: redo-oob will handle all the updating of sf, so we don't have
+        # to do it here, nor call _after1.
+        argv = ['redo-oob', self.sf.name] + [d.name for d in dirty]
+        log('(%s)\n' % _nice(self.t))
+        state.commit()
+        def run():
+            os.chdir(vars.BASE)
+            os.environ['REDO_DEPTH'] = vars.DEPTH + '  '
+            os.execvp(argv[0], argv)
+            assert(0)
+            # returns only if there's an exception
+        def after(t, rv):
+            return self._after2(rv)
+        jwack.start_job(self.t, run, after)
+
     def _do_subproc(self):
         # careful: REDO_PWD was the PWD relative to the STARTPATH at the time
         # we *started* building the current target; but that target ran
@@ -250,7 +280,10 @@ def main(targets, shouldbuildfunc):
             break
         f = state.File(name=t)
         lock = state.Lock(f.id)
-        lock.trylock()
+        if vars.UNLOCKED:
+            lock.owned = True
+        else:
+            lock.trylock()
         if not lock.owned:
             if vars.DEBUG_LOCKS:
                 log('%s (locked...)\n' % _nice(t))