apenwarr-redo/t
Avery Pennarun 61f3e4672e Workaround for completely broken file locking on Windows 10 WSL.
WSL (Windows Services for Linux) provides a Linux-kernel-compatible ABI
for userspace processes, but the current version doesn't not implement
fcntl() locks at all; it just always returns success.  See
https://github.com/Microsoft/WSL/issues/1927.

This causes us three kinds of problem:
  1. sqlite3 in WAL mode gives "OperationalError: locking protocol".
     1b. Other sqlite3 journal modes also don't work when used by
         multiple processes.
  2. redo parallelism doesn't work, because we can't prevent the same
     target from being build several times simultaneously.
  3. "redo-log -f" doesn't work, since it can't tell whether the log
     file it's tailing is "done" or not.

To fix #1, we switch the sqlite3 journal back to PERSIST instead of
WAL.  We originally changed to WAL in commit 5156feae9d to reduce
deadlocks on MacOS.  That was never adequately explained, but PERSIST
still acts weird on MacOS, so we'll only switch to PERSIST when we
detect that locking is definitely broken.  Sigh.

To (mostly) fix #2, we disable any -j value > 1 when locking is broken.
This prevents basic forms of parallelism, but doesn't stop you from
re-entrantly starting other instances of redo.  To fix that properly,
we need to switch to a different locking mechanism entirely, which is
tough in python.  flock() locks probably work, for example, but
python's locks lie and just use fcntl locks for those.

To fix #3, we always force --no-log mode when we find that locking is
broken.
2019-01-02 14:49:33 -05:00
..
000-set-minus-e shelltest.od: add more "set -e" tests and add a 'skip' return code. 2018-12-17 16:17:37 +00:00
010-jobserver Workaround for completely broken file locking on Windows 10 WSL. 2019-01-02 14:49:33 -05:00
100-args Remove deprecated old-args feature. 2018-10-11 03:28:05 -04:00
101-atime Rename redo/python -> redo/py. 2018-12-05 02:27:04 -05:00
102-empty {ood,sources,targets}: fix relative paths; turn missing targets into sources. 2018-11-23 19:18:43 -05:00
103-unicode t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
104-space t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
105-sympath Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
110-compile t/110-compile and t/111-example: skip if no C compiler installed. 2018-11-19 15:46:22 -05:00
111-example t/110-compile and t/111-example: skip if no C compiler installed. 2018-11-19 15:46:22 -05:00
120-defaults-flat t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
121-defaults-nested t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
130-mode Rename redo/python -> redo/py. 2018-12-05 02:27:04 -05:00
140-shuffle Add a test for --shuffle option. 2012-02-09 00:42:41 -05:00
141-keep-going Avoid bashism >&file 2018-10-11 03:28:05 -04:00
200-shell shelltest.od: we accidentally treated some fails as mere warnings. 2018-12-17 16:17:37 +00:00
201-fail minimal/do: don't create a .did file until after a file is actually built. 2018-11-02 04:25:35 -04:00
202-del Use mkstemp() to create the stdout temp file, and simplify $3 path. 2018-12-13 13:28:44 +00:00
203-make t/010-jobserver: add serial/parallel override tests. 2018-12-31 19:24:27 -05:00
204-readonly Ensure correct operation with read-only target dirs and .do file dirs. 2018-12-13 13:28:44 +00:00
220-ifcreate t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
250-makedir minimal/do: didn't work correctly with multi-level auto dir creation. 2018-11-02 04:27:28 -04:00
260-whichdo minimal/do whichdo: should print ../default.do if ../$1.do exists. 2018-11-02 03:13:45 -04:00
350-deps t/*: some cleanups so switching between redo and minimal/do works. 2018-10-12 05:20:27 -04:00
351-deps-forget t/351-deps-forget: remove a test that occasionally flakes. 2018-12-31 19:35:56 -05:00
355-deps-cyclic t/*: some cleanups so switching between redo and minimal/do works. 2018-10-12 05:20:27 -04:00
360-symlinks Fix minimal/do and tests when built in a path containing spaces. 2018-12-11 01:22:29 +00:00
370-logs redo-log: add automated tests, and fix some path bugs revealed by them. 2018-11-19 18:58:36 -05:00
550-chdir t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
640-always t/*: rearrange tests into numbered directories. 2012-02-08 01:56:50 -05:00
950-curse t/stress: add a test that usually triggers a bug using 950-curse. 2018-10-12 05:48:56 -04:00
999-installer Fix minimal/do and tests when built in a path containing spaces. 2018-12-11 01:22:29 +00:00
s60-stamp t/660-stamp: don't run at the same time as other tests in redo -j. 2018-10-12 05:20:27 -04:00
.gitignore Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
all.do Directory reorg: move code into redo/, generate binaries in bin/. 2018-12-04 02:53:40 -05:00
clean.do Fix more inconsistent behaviour with symlinks in paths. 2018-12-17 16:17:37 +00:00
dotparams.od redo-sh: downgrade failures that affected dash; add a bash warning. 2018-10-12 05:18:25 -04:00
flush-cache.do Directory reorg: move code into redo/, generate binaries in bin/. 2018-12-04 02:53:40 -05:00
flush-cache.in Directory reorg: move code into redo/, generate binaries in bin/. 2018-12-04 02:53:40 -05:00
nothing.od shelltest: detect the ". of empty script" shell problem. 2011-02-28 22:06:57 -08:00
shelltest.od minimal/do: use 'pwd -P' instead of '/bin/pwd'. 2019-01-01 19:24:07 -05:00
skip-if-minimal-do.sh t/351-deps-forget: forgot skip-if-minimal-do. 2018-12-04 02:43:58 -05:00
sleep
stress.do t/stress: add a test that usually triggers a bug using 950-curse. 2018-10-12 05:48:56 -04:00