78 lines
2.1 KiB
Markdown
78 lines
2.1 KiB
Markdown
|
# Compression Integration Tests
|
||
|
|
|
||
|
|
Manual integration tests for compressed archive functionality.
|
||
|
|
|
||
|
|
These scripts exercise the tool's ability to:
|
||
|
|
1. Read archives that were compressed by external programs (gzip, brotli, zstd)
|
||
|
|
2. Append new observations to compressed archives
|
||
|
|
3. Produce correct results whether reading compressed or uncompressed
|
||
|
|
|
||
|
|
## Scripts
|
||
|
|
|
||
|
|
### `generate_state.py <n>`
|
||
|
|
Generates a JSON state file with `n` items in each array. Output goes to stdout.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./generate_state.py 3
|
||
|
|
# Output: {"colors":["color_1","color_2","color_3"],"numbers":["number_1","number_2","number_3"],"animals":["animal_1","animal_2","animal_3"]}
|
||
|
|
```
|
||
|
|
|
||
|
|
### `generate_state_files.py <count> <output_dir>`
|
||
|
|
Generates a series of state files (state_1.json through state_N.json) with progressively more items.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./generate_state_files.py 9 ./data
|
||
|
|
# Creates: data/state_1.json, data/state_2.json, ... data/state_9.json
|
||
|
|
```
|
||
|
|
|
||
|
|
### `run_gzip_test.sh`
|
||
|
|
Tests the gzip compression workflow:
|
||
|
|
1. Create archive from first state file
|
||
|
|
2. Compress with gzip
|
||
|
|
3. Append remaining 8 state files to the compressed archive
|
||
|
|
4. Decompress and inspect
|
||
|
|
|
||
|
|
### `run_brotli_test.sh`
|
||
|
|
Same workflow but with brotli compression.
|
||
|
|
|
||
|
|
### `run_zstd_test.sh`
|
||
|
|
Same workflow but with zstd compression.
|
||
|
|
|
||
|
|
### `run_all.sh`
|
||
|
|
Runs all compression tests in sequence.
|
||
|
|
|
||
|
|
### `validate.sh` (optional)
|
||
|
|
Smoke test to verify the final state matches expectations.
|
||
|
|
|
||
|
|
## Usage
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cd tests/compression-integration
|
||
|
|
|
||
|
|
# Run all tests (generates data, builds, runs all compression formats)
|
||
|
|
./run_all.sh
|
||
|
|
|
||
|
|
# Or run individual steps:
|
||
|
|
./generate_state_files.py 9 ./data
|
||
|
|
./run_gzip_test.sh
|
||
|
|
./run_brotli_test.sh
|
||
|
|
./run_zstd_test.sh
|
||
|
|
|
||
|
|
# Optional: validate outputs match
|
||
|
|
./validate.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
## What to look for
|
||
|
|
|
||
|
|
After running the tests, you can manually verify:
|
||
|
|
|
||
|
|
1. The compressed archives were created
|
||
|
|
2. Appending to compressed archives worked (check file sizes grew)
|
||
|
|
3. The `info` command shows the same observation count for compressed and decompressed versions
|
||
|
|
4. The `state` command returns the same final state
|
||
|
|
|
||
|
|
## Dependencies
|
||
|
|
|
||
|
|
- gzip (usually pre-installed)
|
||
|
|
- brotli (`brew install brotli`)
|
||
|
|
- zstd (`brew install zstd`)
|