forks/apenwarr-redo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
apenwarr-redo/redo/env.py

35 lines
1.4 KiB
Python
Raw Normal View History

Use a single .redo dir for the entire build.
2010-11-12 22:47:03 -08:00
import os
The only thing in helpers.py that needed vars.py was the log stuff. So put it in its own file. Now it's safer to import and use helpers even if you can't safely touch vars.
2010-12-11 18:32:40 -08:00
from atoi import atoi
Use a single .redo dir for the entire build.
2010-11-12 22:47:03 -08:00
redo-if{change,create}: print a useful message if REDO isn't set. Again, I forgot to make vars.py not crash if the variables aren't set, so we can print a useful error message. But this time I have the right solution: vars.py will do the checking for itself, and abort with a nice message.
2010-11-22 02:45:00 -08:00
if not os.environ.get('REDO'):
import sys
sys.stderr.write('%s: error: must be run from inside a .do\n'
% sys.argv[0])
sys.exit(100)
Remove the need for relpath (and thus abspath) in builder.py.
2010-11-21 03:34:32 -08:00
PWD = os.environ.get('REDO_PWD', '')
Move log stuff into a separate log.py.
2010-11-13 00:11:34 -08:00
TARGET = os.environ.get('REDO_TARGET', '')
DEPTH = os.environ.get('REDO_DEPTH', '')
The only thing in helpers.py that needed vars.py was the log stuff. So put it in its own file. Now it's safer to import and use helpers even if you can't safely touch vars.
2010-12-11 18:32:40 -08:00
DEBUG = atoi(os.environ.get('REDO_DEBUG', ''))
Add a --debug-locks option. Get rid of the "locked..." and "...unlocked!" messages by default, since they're not usually interesting. But add a new option to bring them back in case we end up with trouble debugging the locking stuff. (I don't really 100% trust it yet, although I haven't had a problem for a while now.)
2010-11-21 06:23:41 -08:00
DEBUG_LOCKS = os.environ.get('REDO_DEBUG_LOCKS', '') and 1 or 0
Add a new --debug-pids option. This makes the helpers.* log functions prepend getpid() to each line, so you can see which pid did what.
2010-11-22 01:50:46 -08:00
DEBUG_PIDS = os.environ.get('REDO_DEBUG_PIDS', '') and 1 or 0
Move log stuff into a separate log.py.
2010-11-13 00:11:34 -08:00
VERBOSE = os.environ.get('REDO_VERBOSE', '') and 1 or 0
Add a -x option that just passes -x to the subshell. This is often more useful than -v, since it prints the actual commands being executed, not just the lines being input from the script.
2010-11-21 06:35:52 -08:00
XTRACE = os.environ.get('REDO_XTRACE', '') and 1 or 0
Add a new -k (--keep-going) option, like make has. Previously, the default was to *always* keep going, which is actually not usually what you want. Now we actually exit correctly after an error. Of course you still might have multiple errors before existing if you were building in parallel.
2010-11-21 07:10:48 -08:00
KEEP_GOING = os.environ.get('REDO_KEEP_GOING', '') and 1 or 0
Split --raw-logs into --no-pretty and --no-log options. --no-log: don't capture logs or run redo-log (same as pre-redo-log redo) --no-pretty: don't pretty-print logs, print @@REDO lines. The latter is an option to both redo and redo-log.
2018-11-19 10:55:56 -05:00
LOG = atoi(os.environ.get('REDO_LOG', '1')) # defaults on
redo-log: fix stdout vs stderr; don't recapture if .do script redirects stderr. redo-log should log to stdout, because when you ask for the specific logs from a run, the logs are the output you requested. redo-log's stderr should be about any errors retrieving that output. On the other hand, when you run redo, the logs are literally the stderr of the build steps, which are incidental to the main job (building things). So that should be send to stderr. Previously, we were sending to stderr when --no-log, but stdout when --log, which is totally wrong. Also, adding redo-log had the unexpected result that if a .do script redirected the stderr of a sub-redo or redo-ifchange to a file or pipe, the output would be eaten by redo-log instead of the intended output. So a test runner like this: self.test: redo self.runtest 2>&1 | grep ERROR would not work; the self.runtest output would be sent to redo's log buffer (and from there, probably printed to the toplevel redo's stderr) rather than passed along to grep.
2018-11-19 16:27:41 -05:00
LOG_INODE = os.environ.get('REDO_LOG_INODE', '')
Add --color and --no-color options. By default, we auto-detect color mode (if it's a tty, color default on, else off). But you can force it either way.
2018-11-19 11:22:53 -05:00
COLOR = atoi(os.environ.get('REDO_COLOR', '1')) # defaults on
Split --raw-logs into --no-pretty and --no-log options. --no-log: don't capture logs or run redo-log (same as pre-redo-log redo) --no-pretty: don't pretty-print logs, print @@REDO lines. The latter is an option to both redo and redo-log.
2018-11-19 10:55:56 -05:00
# subprocesses mustn't pretty-print if a parent is running redo-log
PRETTY = (not LOG) and atoi(os.environ.get('REDO_PRETTY', '1'))
Add a --shuffle option to let you enable dependency randomization. Previously, for testing, we were *always* randomizing the build order of dependencies. That's annoying since it'll make build logs differ randomly from one run to the next, which could make comparisons harder. However, the feature is still useful for uncovering hidden dependencies between objects.
2010-11-16 00:14:57 -08:00
SHUFFLE = os.environ.get('REDO_SHUFFLE', '') and 1 or 0
redo-ifchange can now be run even if there's no parent redo.
2010-12-11 19:08:53 -08:00
STARTDIR = os.environ.get('REDO_STARTDIR', '')
The only thing in helpers.py that needed vars.py was the log stuff. So put it in its own file. Now it's safer to import and use helpers even if you can't safely touch vars.
2010-12-11 18:32:40 -08:00
RUNID = atoi(os.environ.get('REDO_RUNID')) or None
vars.py: it's safe to assume REDO_BASE and REDO_STARTDIR are set correctly. This lets us remove an extra call to os.abspath().
2010-11-21 03:09:21 -08:00
BASE = os.environ['REDO_BASE']
while BASE and BASE.endswith('/'):
BASE = BASE[:-1]
The second half of redo-stamp: out-of-order building. If a depends on b depends on c, and c is dirty but b uses redo-stamp checksums, then 'redo-ifchange a' is indeterminate: we won't know if we need to run a.do unless we first build b, but the script that *normally* runs 'redo-ifchange b' is a.do, and we don't want to run that yet, because we don't know for sure if b is dirty, and we shouldn't build a unless one of its dependencies is dirty. Eek! Luckily, there's a safe solution. If we *know* a is dirty - eg. because a.do or one of its children has definitely changed - then we can just run a.do immediately and there's no problem, even if b is indeterminate, because we were going to run a.do anyhow. If a's dependencies are *not* definitely dirty, and all we have is indeterminate ones like b, then that means a's build process *hasn't changed*, which means its tree of dependencies still includes b, which means we can deduce that if we *did* run a.do, it would end up running b.do. Since we know that anyhow, we can safely just run b.do, which will either b.set_checked() or b.set_changed(). Once that's done, we can re-parse a's dependencies and this time conclusively tell if it needs to be redone or not. Even if it does, b is already up-to-date, so the 'redo-ifchange b' line in a.do will be fast. ...now take all the above and do it recursively to handle nested dependencies, etc, and you're done.
2010-12-11 04:40:05 -08:00
UNLOCKED = os.environ.get('REDO_UNLOCKED', '') and 1 or 0
os.environ['REDO_UNLOCKED'] = '' # not inheritable by subprocesses
Whoops, redo-oob was slightly wrong when used with -j. We called 'redo' instead of 'redo-ifchange' on our indeterminate objects. Since other instances of redo-oob might be running at the same time, this could cause the same object to get rebuilt more than once unnecessarily. The unit tests caught this, I just didn't notice earlier.
2010-12-11 05:50:29 -08:00
NO_OOB = os.environ.get('REDO_NO_OOB', '') and 1 or 0
os.environ['REDO_NO_OOB'] = '' # not inheritable by subprocesses
Reference in a new issue Copy permalink