# Compression Integration Tests

Manual integration tests for compressed archive functionality.

These scripts exercise the tool's ability to:
1. Read archives that were compressed by external programs (gzip, brotli, zstd)
2. Append new observations to compressed archives
3. Produce correct results whether reading compressed or uncompressed

## Scripts

### `generate_state.py <n>`
Generates a JSON state file with `n` items in each array. Output goes to stdout.

```bash
./generate_state.py 3
# Output: {"colors":["color_1","color_2","color_3"],"numbers":["number_1","number_2","number_3"],"animals":["animal_1","animal_2","animal_3"]}
```

### `generate_state_files.py <count> <output_dir>`
Generates a series of state files (state_1.json through state_N.json) with progressively more items.

```bash
./generate_state_files.py 9 ./data
# Creates: data/state_1.json, data/state_2.json, ... data/state_9.json
```

### `run_gzip_test.sh`
Tests the gzip compression workflow:
1. Create archive from first state file
2. Compress with gzip
3. Append remaining 8 state files to the compressed archive
4. Decompress and inspect

### `run_brotli_test.sh`
Same workflow but with brotli compression.

### `run_zstd_test.sh`
Same workflow but with zstd compression.

### `run_all.sh`
Runs all compression tests in sequence.

### `validate.sh` (optional)
Smoke test to verify the final state matches expectations.

## Usage

```bash
cd tests/compression-integration

# Run all tests (generates data, builds, runs all compression formats)
./run_all.sh

# Or run individual steps:
./generate_state_files.py 9 ./data
./run_gzip_test.sh
./run_brotli_test.sh
./run_zstd_test.sh

# Optional: validate outputs match
./validate.sh
```

## What to look for

After running the tests, you can manually verify:

1. The compressed archives were created
2. Appending to compressed archives worked (check file sizes grew)
3. The `info` command shows the same observation count for compressed and decompressed versions
4. The `state` command returns the same final state

## Dependencies

- gzip (usually pre-installed)
- brotli (`brew install brotli`)
- zstd (`brew install zstd`)