2010-11-13 00:45:49 -08:00
|
|
|
#!/usr/bin/python
|
2010-12-05 03:58:20 -08:00
|
|
|
import sys, os, errno, stat
|
2010-11-21 21:15:24 -08:00
|
|
|
import vars, state, builder, jwack
|
2010-12-09 03:01:26 -08:00
|
|
|
from helpers import debug, debug2, err, unlink
|
2010-11-13 00:45:49 -08:00
|
|
|
|
2010-12-11 02:13:46 -08:00
|
|
|
def _nice(t):
|
|
|
|
|
return state.relpath(os.path.join(vars.BASE, t), vars.STARTDIR)
|
2010-11-13 00:45:49 -08:00
|
|
|
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
CLEAN = 0
|
|
|
|
|
DIRTY = 1
|
2010-12-07 02:17:22 -08:00
|
|
|
def dirty_deps(f, depth, max_changed):
|
2010-12-11 02:13:46 -08:00
|
|
|
if vars.DEBUG >= 1:
|
|
|
|
|
debug('%s?%s\n' % (depth, _nice(f.name)))
|
2010-12-07 02:17:22 -08:00
|
|
|
|
2010-12-10 22:42:33 -08:00
|
|
|
if f.failed_runid:
|
|
|
|
|
debug('%s-- DIRTY (failed last time)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
return DIRTY
|
2010-12-07 02:17:22 -08:00
|
|
|
if f.changed_runid == None:
|
|
|
|
|
debug('%s-- DIRTY (never built)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
return DIRTY
|
2010-12-07 02:17:22 -08:00
|
|
|
if f.changed_runid > max_changed:
|
2010-11-21 04:14:52 -08:00
|
|
|
debug('%s-- DIRTY (built)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
return DIRTY # has been built more recently than parent
|
2010-12-07 02:17:22 -08:00
|
|
|
if f.is_checked():
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
if vars.DEBUG >= 1:
|
|
|
|
|
debug('%s-- CLEAN (checked)\n' % depth)
|
|
|
|
|
return CLEAN # has already been checked during this session
|
2010-12-07 02:17:22 -08:00
|
|
|
if not f.stamp:
|
2010-11-19 03:03:05 -08:00
|
|
|
debug('%s-- DIRTY (no stamp)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
return DIRTY
|
2010-12-11 05:43:02 -08:00
|
|
|
newstamp = f.read_stamp()
|
|
|
|
|
if f.stamp != newstamp:
|
|
|
|
|
if newstamp == state.STAMP_MISSING:
|
|
|
|
|
debug('%s-- DIRTY (missing)\n' % depth)
|
|
|
|
|
else:
|
|
|
|
|
debug('%s-- DIRTY (mtime)\n' % depth)
|
|
|
|
|
if f.csum:
|
|
|
|
|
return [f]
|
|
|
|
|
else:
|
|
|
|
|
return DIRTY
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
|
|
|
|
|
must_build = []
|
2010-12-09 02:13:36 -08:00
|
|
|
for mode,f2 in f.deps():
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
dirty = CLEAN
|
2010-11-13 00:45:49 -08:00
|
|
|
if mode == 'c':
|
2010-12-09 02:13:36 -08:00
|
|
|
if os.path.exists(os.path.join(vars.BASE, f2.name)):
|
2010-11-13 00:45:49 -08:00
|
|
|
debug('%s-- DIRTY (created)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
dirty = DIRTY
|
2010-11-13 00:45:49 -08:00
|
|
|
elif mode == 'm':
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
sub = dirty_deps(f2, depth = depth + ' ',
|
|
|
|
|
max_changed = max(f.changed_runid,
|
|
|
|
|
f.checked_runid))
|
|
|
|
|
if sub:
|
2010-11-21 04:14:52 -08:00
|
|
|
debug('%s-- DIRTY (sub)\n' % depth)
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
dirty = sub
|
|
|
|
|
else:
|
|
|
|
|
assert(mode in ('c','m'))
|
|
|
|
|
if not f.csum:
|
|
|
|
|
# f is a "normal" target: dirty f2 means f is instantly dirty
|
|
|
|
|
if dirty:
|
|
|
|
|
# if dirty==DIRTY, this means f is definitely dirty.
|
|
|
|
|
# if dirty==[...], it's a list of the uncertain children.
|
|
|
|
|
return dirty
|
|
|
|
|
else:
|
|
|
|
|
# f is "checksummable": dirty f2 means f needs to redo,
|
|
|
|
|
# but f might turn out to be clean after that (ie. our parent
|
|
|
|
|
# might not be dirty).
|
|
|
|
|
if dirty == DIRTY:
|
|
|
|
|
# f2 is definitely dirty, so f definitely needs to
|
|
|
|
|
# redo. However, after that, f might turn out to be
|
|
|
|
|
# unchanged.
|
|
|
|
|
return [f]
|
|
|
|
|
elif isinstance(dirty,list):
|
|
|
|
|
# our child f2 might be dirty, but it's not sure yet. It's
|
|
|
|
|
# given us a list of targets we have to redo in order to
|
|
|
|
|
# be sure.
|
|
|
|
|
must_build += dirty
|
|
|
|
|
|
|
|
|
|
if must_build:
|
|
|
|
|
# f is *maybe* dirty because at least one of its children is maybe
|
|
|
|
|
# dirty. must_build has accumulated a list of "topmost" uncertain
|
|
|
|
|
# objects in the tree. If we build all those, we can then
|
|
|
|
|
# redo-ifchange f and it won't have any uncertainty next time.
|
|
|
|
|
return must_build
|
|
|
|
|
|
|
|
|
|
# if we get here, it's because the target is clean
|
2010-12-10 22:42:33 -08:00
|
|
|
if f.is_override:
|
|
|
|
|
builder.warn_override(f.name)
|
2010-12-07 02:17:22 -08:00
|
|
|
f.set_checked()
|
|
|
|
|
f.save()
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
return CLEAN
|
2010-11-13 00:45:49 -08:00
|
|
|
|
|
|
|
|
|
2010-11-21 04:14:52 -08:00
|
|
|
def should_build(t):
|
2010-12-07 02:17:22 -08:00
|
|
|
f = state.File(name=t)
|
2010-12-10 20:53:31 -08:00
|
|
|
if f.is_failed():
|
|
|
|
|
raise builder.ImmediateReturn(32)
|
2010-12-07 02:17:22 -08:00
|
|
|
return dirty_deps(f, depth = '', max_changed = vars.RUNID)
|
2010-11-21 04:14:52 -08:00
|
|
|
|
|
|
|
|
|
2010-11-19 07:32:16 -08:00
|
|
|
rv = 202
|
2010-11-13 02:09:42 -08:00
|
|
|
try:
|
2010-11-25 06:35:22 -08:00
|
|
|
me = os.path.join(vars.STARTDIR,
|
|
|
|
|
os.path.join(vars.PWD, vars.TARGET))
|
2010-12-07 02:17:22 -08:00
|
|
|
f = state.File(name=me)
|
2010-11-25 06:35:22 -08:00
|
|
|
debug2('TARGET: %r %r %r\n' % (vars.STARTDIR, vars.PWD, vars.TARGET))
|
2010-11-21 21:15:24 -08:00
|
|
|
try:
|
2010-11-21 23:33:11 -08:00
|
|
|
targets = sys.argv[1:]
|
The second half of redo-stamp: out-of-order building.
If a depends on b depends on c, and c is dirty but b uses redo-stamp
checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need
to run a.do unless we first build b, but the script that *normally* runs
'redo-ifchange b' is a.do, and we don't want to run that yet, because we
don't know for sure if b is dirty, and we shouldn't build a unless one of
its dependencies is dirty. Eek!
Luckily, there's a safe solution. If we *know* a is dirty - eg. because
a.do or one of its children has definitely changed - then we can just run
a.do immediately and there's no problem, even if b is indeterminate, because
we were going to run a.do anyhow.
If a's dependencies are *not* definitely dirty, and all we have is
indeterminate ones like b, then that means a's build process *hasn't
changed*, which means its tree of dependencies still includes b, which means
we can deduce that if we *did* run a.do, it would end up running b.do.
Since we know that anyhow, we can safely just run b.do, which will either
b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's
dependencies and this time conclusively tell if it needs to be redone or
not. Even if it does, b is already up-to-date, so the 'redo-ifchange b'
line in a.do will be fast.
...now take all the above and do it recursively to handle nested
dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
|
|
|
if not vars.UNLOCKED:
|
|
|
|
|
for t in targets:
|
|
|
|
|
f.add_dep('m', t)
|
|
|
|
|
f.save()
|
2010-11-21 23:33:11 -08:00
|
|
|
rv = builder.main(targets, should_build)
|
2010-11-21 21:15:24 -08:00
|
|
|
finally:
|
|
|
|
|
jwack.force_return_tokens()
|
2010-11-13 02:09:42 -08:00
|
|
|
except KeyboardInterrupt:
|
|
|
|
|
sys.exit(200)
|
2010-11-19 07:32:16 -08:00
|
|
|
sys.exit(rv)
|