forks/apenwarr-redo
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Make apenwarr/redo installable on windows and work with uv tool install
445 commits 1 branch 0 tags 1.1 MiB
Find a file
Avery Pennarun 61f3e4672e Workaround for completely broken file locking on Windows 10 WSL. 
WSL (Windows Services for Linux) provides a Linux-kernel-compatible ABI
for userspace processes, but the current version doesn't not implement
fcntl() locks at all; it just always returns success.  See
https://github.com/Microsoft/WSL/issues/1927.

This causes us three kinds of problem:
  1. sqlite3 in WAL mode gives "OperationalError: locking protocol".
     1b. Other sqlite3 journal modes also don't work when used by
         multiple processes.
  2. redo parallelism doesn't work, because we can't prevent the same
     target from being build several times simultaneously.
  3. "redo-log -f" doesn't work, since it can't tell whether the log
     file it's tailing is "done" or not.

To fix #1, we switch the sqlite3 journal back to PERSIST instead of
WAL.  We originally changed to WAL in commit 5156feae9d to reduce
deadlocks on MacOS.  That was never adequately explained, but PERSIST
still acts weird on MacOS, so we'll only switch to PERSIST when we
detect that locking is definitely broken.  Sigh.

To (mostly) fix #2, we disable any -j value > 1 when locking is broken.
This prevents basic forms of parallelism, but doesn't stop you from
re-entrantly starting other instances of redo.  To fix that properly,
we need to switch to a different locking mechanism entirely, which is
tough in python.  flock() locks probably work, for example, but
python's locks lie and just use fcntl locks for those.

To fix #3, we always force --no-log mode when we find that locking is
broken.
2019-01-02 14:49:33 -05:00
bin Move setproctitle() stuff into title.py. 2018-12-05 02:28:34 -05:00
contrib/bash_completion.d MacOS: "-e filename/." returns true even for non-directories. 2011-03-05 19:03:30 -08:00
docs Move _all.do -> all.do and slightly update docs. 2018-12-31 15:07:18 -05:00
minimal minimal/do: use 'pwd -P' instead of '/bin/pwd'. 2019-01-01 19:24:07 -05:00
redo Workaround for completely broken file locking on Windows 10 WSL. 2019-01-02 14:49:33 -05:00
t Workaround for completely broken file locking on Windows 10 WSL. 2019-01-02 14:49:33 -05:00
.gitignore Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
.pylintrc Add a bunch of missing python docstrings. 2018-12-14 09:03:53 +00:00
all.do Move _all.do -> all.do and slightly update docs. 2018-12-31 15:07:18 -05:00
clean.do Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
do Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
install.do Move _all.do -> all.do and slightly update docs. 2018-12-31 15:07:18 -05:00
LICENSE Change license to Apache 2.0. 2018-11-26 17:04:31 -05:00
Makefile Directory reorg: move code into redo/, generate binaries in bin/. 2018-12-04 02:53:40 -05:00
MANIFEST.in setup.py: add a python setuptools package. 2018-12-31 21:12:39 -05:00
mkdocs.yml Directory reorg: move code into redo/, generate binaries in bin/. 2018-12-04 02:53:40 -05:00
README.md Move _all.do -> all.do and slightly update docs. 2018-12-31 15:07:18 -05:00
setup.py setup.py: add a python setuptools package. 2018-12-31 21:12:39 -05:00
test.do Move _all.do -> all.do and slightly update docs. 2018-12-31 15:07:18 -05:00

README.md

redo - a recursive build system

Smaller, easier, more powerful, and more reliable than make.

This is an implementation of Daniel J. Bernstein's redo build system. He never released his version, so other people have implemented different variants based on his published specification.

This version, sometimes called apenwarr/redo, is probably the most advanced one, including parallel builds, improved logging, extensive automated tests, and helpful debugging features.

To build and test redo, run ./do -j10 test. To install it, run DESTDIR=/tmp/testinstall PREFIX=/usr/local ./do -j10 install.

  • View the documentation via readthedocs.org
  • Visit the source code on github
  • Discussions and support via the mailing list (archives). You can subscribe by sending any email message to redo-list+subscribe@googlegroups.com (note the plus sign).