diff --git a/CHANGELOG.md b/CHANGELOG.md index 6511a96..1665b9d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.5.4] - 2025-10-23 + +### Updated + +- Updated cel-rust from 0.11.4 to 0.11.6 +- Updated PyO3 from 0.25.1 to 0.27.1 + +### Changed + +- Reduced logging verbosity ## [0.5.3] - 2025-10-14 - Added new `cel.stdlib` module with Python implementations of CEL functions missing from upstream cel-rust. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..d9bca85 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,190 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +This is a Python package that provides Python bindings for the Common Expression Language (CEL) using a Rust implementation. The project uses PyO3 to create Python bindings for the `cel-interpreter` Rust crate. + +## Architecture + +### Core Components + +- **Rust Core (`src/lib.rs`)**: Main evaluation engine that compiles and executes CEL expressions +- **Context Management (`src/context.rs`)**: Manages variables and Python functions in CEL evaluation context +- **Type Conversion**: Bidirectional conversion between Python types and CEL Value types via `RustyCelType` and `RustyPyType` wrappers + +### Key Features + +- **Expression Evaluation**: Compile and execute CEL expressions with Python context +- **Custom Functions**: Support for user-defined Python functions callable from CEL expressions +- **Type System**: Handles primitive types (int, float, string, bool), collections (list, dict), timestamps, durations, and bytes +- **Context API**: Both dict-based and explicit Context object for managing evaluation context + +## Development Commands + +### Building +```bash +# Build the Rust extension module +maturin develop + +# Build release version +maturin build --release +``` + +### Testing +```bash +# Run all tests with debug logging +uv run pytest --log-cli-level=debug + +# Run specific test file +uv run pytest tests/test_basics.py + +# Run single test +uv run pytest tests/test_basics.py::test_hello_world +``` + +### Linting & Formatting +```bash +# Format Rust code +cargo fmt --all + +# Check Rust code with clippy +cargo clippy --all-targets --all-features -- -D warnings + +# Build workspace to verify compilation +cargo build --workspace +``` + +## Key Files + +- `src/lib.rs`: Main evaluation function and type conversions (src/lib.rs:191 for `evaluate` function) +- `src/context.rs`: Context management and Python function integration +- `tests/`: Comprehensive test suite covering all features +- `pyproject.toml`: Python package configuration with maturin build system +- `Cargo.toml`: Rust dependencies including PyO3 and cel-interpreter + +## CLI Architecture + +### Command-Line Interface (`python/cel/cli.py`) + +The CLI is built with Typer and provides multiple modes: + +- **Single expression evaluation**: `cel 'expression'` +- **Interactive REPL**: `cel --interactive` (uses prompt_toolkit) +- **TUI mode**: `cel --tui` (uses textual library) +- **File mode**: `cel --file expressions.cel` (evaluate expressions from file) +- **Batch context processing**: `cel 'expr' --for-each file1.json --for-each file2.json` + +### Key CLI Components + +- **CELEvaluator** (python/cel/cli.py:~180): Wrapper around core evaluation with context management +- **CELFormatter** (python/cel/cli.py:~90): Rich-based output formatting (JSON, pretty, python, auto) +- **InteractiveCELREPL** (python/cel/cli.py:~250): REPL with command dispatch and history +- **evaluate_expression_with_multiple_contexts** (python/cel/cli.py:~537): Batch processing function + +### CLI Design Patterns + +#### Batch Processing (`--for-each`) + +The `--for-each` flag uses a **repeated flag pattern** (like `cargo test --test`) rather than positional arguments: + +```bash +# Correct: Explicit separate evaluation +cel 'user.age >= 18' --for-each user1.json --for-each user2.json --for-each user3.json + +# NOT: cel 'expr' file1.json file2.json (ambiguous: merge or separate?) +``` + +**Why repeated flags?** +- Makes separate evaluation explicit (vs merging) +- Distinct from `--context-file` (singular, merged) +- Follows precedent: `cargo test --test`, `curl -H` +- Prevents confusion about merge vs separate semantics + +See research in `/tmp/cli_patterns_detailed.md` for analysis of 20+ CLI tools. + +#### Context Merging + +- `--context-file file.json`: Loads ONE merged context +- `--for-each f1.json --for-each f2.json`: Evaluates SEPARATELY for EACH file +- Both can combine: `--context '{"base": 1}' --for-each file.json` → base merged into each file + +## Testing Patterns + +### Test Organization + +- **Unit tests** (`tests/test_*.py`): Test individual functions with mocks +- **E2E tests** (`tests/test_cli.py`): Test full CLI via subprocess +- **Total**: 91 tests (50 unit + 41 E2E) + +### Test Files + +- `test_basics.py`: Core evaluation functionality and type handling +- `test_context.py`: Context management and variable passing +- `test_functions.py`: Custom Python function integration +- `test_cli.py`: CLI features (both unit and E2E tests) + - TestCELFormatter: Output formatting tests + - TestCELEvaluator: Evaluator wrapper tests + - TestInteractiveCELREPL: REPL functionality tests + - TestBatchContextProcessing: Unit tests for batch processing + - **TestCLIE2EBasicFeatures**: E2E tests for basic CLI (25 tests) + - **TestCLIE2EFileMode**: E2E tests for --file mode (7 tests) + - **TestBatchContextProcessingE2E**: E2E tests for --for-each (9 tests) + +### E2E Testing Strategy + +E2E tests use `subprocess.run()` to invoke the actual `cel` CLI command: + +```python +# Example E2E test pattern +result = subprocess.run( + ["cel", "expression", "--context", '{"var": 1}'], + capture_output=True, + text=True, + check=True, +) +assert result.returncode == 0 +assert "expected output" in result.stdout +``` + +**Why E2E tests?** +- Verify actual user experience (CLI flag parsing, exit codes, output) +- Catch integration issues that unit tests miss +- Serve as executable documentation +- Test shell integration (exit codes, pipes) + +**Coverage**: ~95% of non-interactive CLI features + +Test fixtures in `conftest.py` provide parameterized test cases for various expression types and contexts. + +## Common Pitfalls & Gotchas + +### CLI Development + +1. **Always test both unit AND E2E**: Unit tests verify function logic, E2E tests verify CLI integration +2. **Exit codes matter**: Users rely on exit codes for scripting (`if cel 'expr'; then`) +3. **Flag naming**: Use `--long-flag` for clarity, `-s` for common shortcuts +4. **Repeated flags**: When adding new batch operations, consider `--for-each` pattern + +### Testing + +1. **E2E tests are slower**: ~141ms per E2E test vs ~30ms per unit test (due to subprocess) +2. **Temp files**: Always use `tempfile.NamedTemporaryFile` with cleanup in `finally` blocks +3. **Output verification**: E2E tests should verify both stdout content AND exit codes +4. **JSON parsing**: When testing JSON output, parse it to verify structure, not just string matching + +### Documentation + +When adding CLI features, update ALL of: +- README.md (quick example) +- docs/reference/cli-reference.md (complete flag documentation) +- docs/how-to-guides/cli-recipes.md (usage recipes and patterns) +- CLI docstring (examples in `main()` function) + +## Performance Notes + +- CEL evaluation is microsecond-level (Rust backend) +- CLI overhead is ~1-2ms per invocation (Python startup) +- E2E tests take ~7.5s for 91 tests total +- Batch processing with 20 files: ~50-100ms total (file I/O dominates) \ No newline at end of file diff --git a/Cargo.lock b/Cargo.lock index c3dac81..2daa189 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -22,9 +22,9 @@ dependencies = [ [[package]] name = "antlr4rust" -version = "0.3.0-rc2" +version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d240d49ee89063f90fa0cb18aead41a5893cd544a1785983dc3bf5c3d5faa58b" +checksum = "855de4069e655d114ee7a7e003b8c96f623671532da2f8bb46d25dfd6a16abd9" dependencies = [ "better_any", "bit-set", @@ -78,9 +78,9 @@ checksum = "5e764a1d40d510daf35e07be9eb06e75770908c27d411ee6c92109c9840eaaf7" [[package]] name = "bitflags" -version = "2.9.4" +version = "2.10.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2261d10cca569e4643e526d8dc2e62e433cc8aba21ab764233731f8d369bf394" +checksum = "812e12b5285cc515a9c72a5c1d3b6d46a19dac5acfef5265968c166106e31dd3" [[package]] name = "bumpalo" @@ -96,9 +96,9 @@ checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" [[package]] name = "cc" -version = "1.2.36" +version = "1.2.41" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5252b3d2648e5eedbc1a6f501e3c795e07025c1e93bbf8bbdd6eef7f447a6d54" +checksum = "ac9fe6cdbb24b6ade63616c0a0688e45bb56732262c158df3c0c4bea4ca47cb7" dependencies = [ "find-msvc-tools", "shlex", @@ -106,9 +106,9 @@ dependencies = [ [[package]] name = "cel" -version = "0.5.3" +version = "0.5.4" dependencies = [ - "cel 0.11.4", + "cel 0.11.6", "chrono", "log", "pyo3", @@ -117,9 +117,9 @@ dependencies = [ [[package]] name = "cel" -version = "0.11.4" +version = "0.11.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2fd757373c0c269eaaca1c355cbd8dc78e1191982c0e9e6dfe1f61ff38b235ad" +checksum = "8edeb3d082fb4f559d994ed50338e924b0b0bc74f855d7fcf49c980cb4c2d95f" dependencies = [ "antlr4rust", "base64", @@ -135,9 +135,9 @@ dependencies = [ [[package]] name = "cfg-if" -version = "1.0.3" +version = "1.0.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2fd1289c04a9ea8cb22300a459a72a385d7c73d3259e2ed7dcb2af674838cfa9" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" [[package]] name = "chrono" @@ -150,7 +150,7 @@ dependencies = [ "num-traits", "serde", "wasm-bindgen", - "windows-link 0.2.0", + "windows-link", ] [[package]] @@ -161,9 +161,9 @@ checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b" [[package]] name = "find-msvc-tools" -version = "0.1.1" +version = "0.1.4" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7fd99930f64d146689264c637b5af2f0233a933bef0d8570e2526bf9e083192d" +checksum = "52051878f80a721bb68ebfbc930e07b65ba72f2da88968ea5c06fd6ca3d3a127" [[package]] name = "heck" @@ -173,9 +173,9 @@ checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" [[package]] name = "iana-time-zone" -version = "0.1.63" +version = "0.1.64" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b0c919e5debc312ad217002b8048a17b7d83f80703865bbfcfebb0458b0b27d8" +checksum = "33e57f83510bb73707521ebaffa789ec8caf86f9657cad665b092b581d40e9fb" dependencies = [ "android_system_properties", "core-foundation-sys", @@ -197,9 +197,12 @@ dependencies = [ [[package]] name = "indoc" -version = "2.0.6" +version = "2.0.7" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f4c7245a08504955605670dbf141fceab975f15ca21570696aebe9d2e71576bd" +checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706" +dependencies = [ + "rustversion", +] [[package]] name = "itoa" @@ -209,9 +212,9 @@ checksum = "4a5f13b858c8d314ee3e8f639011f7ccefe71f97f96e50151fb991f267928e2c" [[package]] name = "js-sys" -version = "0.3.78" +version = "0.3.81" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0c0b063578492ceec17683ef2f8c5e89121fbd0b172cbc280635ab7567db2738" +checksum = "ec48937a97411dcb524a265206ccd4c90bb711fca92b2792c407f268825b9305" dependencies = [ "once_cell", "wasm-bindgen", @@ -225,17 +228,16 @@ checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" [[package]] name = "libc" -version = "0.2.175" +version = "0.2.177" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6a82ae493e598baaea5209805c49bbf2ea7de956d50d7da0da1164f9c6d28543" +checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976" [[package]] name = "lock_api" -version = "0.4.13" +version = "0.4.14" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "96936507f153605bddfcda068dd804796c84324ed2510809e5b2a624c81da765" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" dependencies = [ - "autocfg", "scopeguard", ] @@ -247,9 +249,9 @@ checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432" [[package]] name = "memchr" -version = "2.7.5" +version = "2.7.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "32a282da65faaf38286cf3be983213fcf1d2e2a58700e808f83f4ea9a4804bc0" +checksum = "f52b00d39961fc5b2736ea853c9cc86238e165017a493d1d5c8eac6bdc4cc273" [[package]] name = "memoffset" @@ -302,9 +304,9 @@ checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d" [[package]] name = "parking_lot" -version = "0.12.4" +version = "0.12.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "70d58bf43669b5795d1576d0641cfb6fbb2057bf629506267a92807158584a13" +checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a" dependencies = [ "lock_api", "parking_lot_core", @@ -312,15 +314,15 @@ dependencies = [ [[package]] name = "parking_lot_core" -version = "0.9.11" +version = "0.9.12" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bc838d2a56b5b1a6c25f55575dfc605fabb63bb2365f6c2353ef9159aa69e4a5" +checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1" dependencies = [ "cfg-if", "libc", "redox_syscall", "smallvec", - "windows-targets", + "windows-link", ] [[package]] @@ -337,18 +339,18 @@ checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483" [[package]] name = "proc-macro2" -version = "1.0.101" +version = "1.0.102" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "89ae43fd86e4158d6db51ad8e2b80f313af9cc74f5c0e03ccb87de09998732de" +checksum = "8e0f6df8eaa422d97d72edcd152e1451618fed47fabbdbd5a8864167b1d4aff7" dependencies = [ "unicode-ident", ] [[package]] name = "pyo3" -version = "0.25.1" +version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8970a78afe0628a3e3430376fc5fd76b6b45c4d43360ffd6cdd40bdde72b682a" +checksum = "37a6df7eab65fc7bee654a421404947e10a0f7085b6951bf2ea395f4659fb0cf" dependencies = [ "chrono", "indoc", @@ -364,19 +366,18 @@ dependencies = [ [[package]] name = "pyo3-build-config" -version = "0.25.1" +version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "458eb0c55e7ece017adeba38f2248ff3ac615e53660d7c71a238d7d2a01c7598" +checksum = "f77d387774f6f6eec64a004eac0ed525aab7fa1966d94b42f743797b3e395afb" dependencies = [ - "once_cell", "target-lexicon", ] [[package]] name = "pyo3-ffi" -version = "0.25.1" +version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7114fe5457c61b276ab77c5055f206295b812608083644a5c5b2640c3102565c" +checksum = "2dd13844a4242793e02df3e2ec093f540d948299a6a77ea9ce7afd8623f542be" dependencies = [ "libc", "pyo3-build-config", @@ -384,9 +385,8 @@ dependencies = [ [[package]] name = "pyo3-log" -version = "0.12.4" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "45192e5e4a4d2505587e27806c7b710c231c40c56f3bfc19535d0bb25df52264" +version = "0.13.1" +source = "git+https://github.com/a1phyr/pyo3-log.git?branch=pyo3_0.27#1b4d070c6b3d466a9f060c06683e3994e8dfb7e3" dependencies = [ "arc-swap", "log", @@ -395,9 +395,9 @@ dependencies = [ [[package]] name = "pyo3-macros" -version = "0.25.1" +version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a8725c0a622b374d6cb051d11a0983786448f7785336139c3c94f5aa6bef7e50" +checksum = "eaf8f9f1108270b90d3676b8679586385430e5c0bb78bb5f043f95499c821a71" dependencies = [ "proc-macro2", "pyo3-macros-backend", @@ -407,9 +407,9 @@ dependencies = [ [[package]] name = "pyo3-macros-backend" -version = "0.25.1" +version = "0.27.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "4109984c22491085343c05b0dbc54ddc405c3cf7b4374fc533f5c3313a572ccc" +checksum = "70a3b2274450ba5288bc9b8c1b69ff569d1d61189d4bff38f8d22e03d17f932b" dependencies = [ "heck", "proc-macro2", @@ -420,27 +420,27 @@ dependencies = [ [[package]] name = "quote" -version = "1.0.40" +version = "1.0.41" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "1885c039570dc00dcb4ff087a89e185fd56bae234ddc7f056a945bf36467248d" +checksum = "ce25767e7b499d1b604768e7cde645d14cc8584231ea6b295e9c9eb22c02e1d1" dependencies = [ "proc-macro2", ] [[package]] name = "redox_syscall" -version = "0.5.17" +version = "0.5.18" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5407465600fb0548f1442edf71dd20683c6ed326200ace4b1ef0763521bb3b77" +checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d" dependencies = [ "bitflags", ] [[package]] name = "regex" -version = "1.11.2" +version = "1.12.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "23d7fd106d8c02486a8d64e778353d1cffe08ce79ac2e82f540c86d0facf6912" +checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4" dependencies = [ "aho-corasick", "memchr", @@ -450,9 +450,9 @@ dependencies = [ [[package]] name = "regex-automata" -version = "0.4.10" +version = "0.4.13" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "6b9458fa0bfeeac22b5ca447c63aaf45f28439a709ccd244698632f9aa6394d6" +checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c" dependencies = [ "aho-corasick", "memchr", @@ -461,9 +461,9 @@ dependencies = [ [[package]] name = "regex-syntax" -version = "0.8.6" +version = "0.8.8" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "caf4aa5b0f434c91fe5c7f1ecb6a5ece2130b02ad2a590589dda5146df959001" +checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58" [[package]] name = "rustversion" @@ -485,18 +485,27 @@ checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" [[package]] name = "serde" -version = "1.0.219" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", +] + +[[package]] +name = "serde_core" +version = "1.0.228" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5f0e2c6ed6606019b4e29e69dbaba95b11854410e5347d525002456dbbb786b6" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" dependencies = [ "serde_derive", ] [[package]] name = "serde_derive" -version = "1.0.219" +version = "1.0.228" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5b0276cf7f2c73365f7157c8123c21cd9a50fbbd844757af28ca1f5925fc2a00" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" dependencies = [ "proc-macro2", "quote", @@ -505,14 +514,15 @@ dependencies = [ [[package]] name = "serde_json" -version = "1.0.143" +version = "1.0.145" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "d401abef1d108fbd9cbaebc3e46611f4b1021f714a0597a71f41ee463f5f4a5a" +checksum = "402a6f66d8c709116cf22f558eab210f5a50187f702eb4d7e5ef38d9a7f1c79c" dependencies = [ "itoa", "memchr", "ryu", "serde", + "serde_core", ] [[package]] @@ -529,9 +539,9 @@ checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" [[package]] name = "syn" -version = "2.0.106" +version = "2.0.108" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ede7c438028d4436d71104916910f5bb611972c5cfd7f89b8300a8186e6fada6" +checksum = "da58917d35242480a05c2897064da0a80589a2a0476c9a3f2fdc83b53502e917" dependencies = [ "proc-macro2", "quote", @@ -572,9 +582,9 @@ checksum = "6af6ae20167a9ece4bcb41af5b80f8a1f1df981f6391189ce00fd257af04126a" [[package]] name = "unicode-ident" -version = "1.0.19" +version = "1.0.20" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f63a545481291138910575129486daeaf8ac54aee4387fe7906919f7830c7d9d" +checksum = "462eeb75aeb73aea900253ce739c8e18a67423fadf006037cd3ff27e82748a06" [[package]] name = "unindent" @@ -594,9 +604,9 @@ dependencies = [ [[package]] name = "wasm-bindgen" -version = "0.2.101" +version = "0.2.104" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7e14915cadd45b529bb8d1f343c4ed0ac1de926144b746e2710f9cd05df6603b" +checksum = "c1da10c01ae9f1ae40cbfac0bac3b1e724b320abfcf52229f80b547c0d250e2d" dependencies = [ "cfg-if", "once_cell", @@ -607,9 +617,9 @@ dependencies = [ [[package]] name = "wasm-bindgen-backend" -version = "0.2.101" +version = "0.2.104" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "e28d1ba982ca7923fd01448d5c30c6864d0a14109560296a162f80f305fb93bb" +checksum = "671c9a5a66f49d8a47345ab942e2cb93c7d1d0339065d4f8139c486121b43b19" dependencies = [ "bumpalo", "log", @@ -621,9 +631,9 @@ dependencies = [ [[package]] name = "wasm-bindgen-macro" -version = "0.2.101" +version = "0.2.104" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7c3d463ae3eff775b0c45df9da45d68837702ac35af998361e2c84e7c5ec1b0d" +checksum = "7ca60477e4c59f5f2986c50191cd972e3a50d8a95603bc9434501cf156a9a119" dependencies = [ "quote", "wasm-bindgen-macro-support", @@ -631,9 +641,9 @@ dependencies = [ [[package]] name = "wasm-bindgen-macro-support" -version = "0.2.101" +version = "0.2.104" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7bb4ce89b08211f923caf51d527662b75bdc9c9c7aab40f86dcb9fb85ac552aa" +checksum = "9f07d2f20d4da7b26400c9f4a0511e6e0345b040694e8a75bd41d578fa4421d7" dependencies = [ "proc-macro2", "quote", @@ -644,31 +654,31 @@ dependencies = [ [[package]] name = "wasm-bindgen-shared" -version = "0.2.101" +version = "0.2.104" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "f143854a3b13752c6950862c906306adb27c7e839f7414cec8fea35beab624c1" +checksum = "bad67dc8b2a1a6e5448428adec4c3e84c43e561d8c9ee8a9e5aabeb193ec41d1" dependencies = [ "unicode-ident", ] [[package]] name = "windows-core" -version = "0.61.2" +version = "0.62.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c0fdd3ddb90610c7638aa2b3a3ab2904fb9e5cdbecc643ddb3647212781c4ae3" +checksum = "b8e83a14d34d0623b51dce9581199302a221863196a1dde71a7663a4c2be9deb" dependencies = [ "windows-implement", "windows-interface", - "windows-link 0.1.3", + "windows-link", "windows-result", "windows-strings", ] [[package]] name = "windows-implement" -version = "0.60.0" +version = "0.60.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "a47fddd13af08290e67f4acabf4b459f647552718f683a7b415d290ac744a836" +checksum = "053e2e040ab57b9dc951b72c264860db7eb3b0200ba345b4e4c3b14f67855ddf" dependencies = [ "proc-macro2", "quote", @@ -677,9 +687,9 @@ dependencies = [ [[package]] name = "windows-interface" -version = "0.59.1" +version = "0.59.3" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "bd9211b69f8dcdfa817bfd14bf1c97c9188afa36f4750130fcdf3f400eca9fa8" +checksum = "3f316c4a2570ba26bbec722032c4099d8c8bc095efccdc15688708623367e358" dependencies = [ "proc-macro2", "quote", @@ -688,94 +698,24 @@ dependencies = [ [[package]] name = "windows-link" -version = "0.1.3" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "5e6ad25900d524eaabdbbb96d20b4311e1e7ae1699af4fb28c17ae66c80d798a" - -[[package]] -name = "windows-link" -version = "0.2.0" +version = "0.2.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "45e46c0661abb7180e7b9c281db115305d49ca1709ab8242adf09666d2173c65" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" [[package]] name = "windows-result" -version = "0.3.4" +version = "0.4.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "56f42bd332cc6c8eac5af113fc0c1fd6a8fd2aa08a0119358686e5160d0586c6" +checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5" dependencies = [ - "windows-link 0.1.3", + "windows-link", ] [[package]] name = "windows-strings" -version = "0.4.2" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "56e6c93f3a0c3b36176cb1327a4958a0353d5d166c2a35cb268ace15e91d3b57" -dependencies = [ - "windows-link 0.1.3", -] - -[[package]] -name = "windows-targets" -version = "0.52.6" +version = "0.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +checksum = "7837d08f69c77cf6b07689544538e017c1bfcf57e34b4c0ff58e6c2cd3b37091" dependencies = [ - "windows_aarch64_gnullvm", - "windows_aarch64_msvc", - "windows_i686_gnu", - "windows_i686_gnullvm", - "windows_i686_msvc", - "windows_x86_64_gnu", - "windows_x86_64_gnullvm", - "windows_x86_64_msvc", + "windows-link", ] - -[[package]] -name = "windows_aarch64_gnullvm" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" - -[[package]] -name = "windows_aarch64_msvc" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" - -[[package]] -name = "windows_i686_gnu" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" - -[[package]] -name = "windows_i686_gnullvm" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" - -[[package]] -name = "windows_i686_msvc" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" - -[[package]] -name = "windows_x86_64_gnu" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" - -[[package]] -name = "windows_x86_64_gnullvm" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" - -[[package]] -name = "windows_x86_64_msvc" -version = "0.52.6" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" diff --git a/Cargo.toml b/Cargo.toml index e02b4e2..30e57cf 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "cel" -version = "0.5.3" +version = "0.5.4" edition = "2021" # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html @@ -9,8 +9,8 @@ name = "cel" crate-type = ["cdylib"] [dependencies] -pyo3 = { version = "0.25.1", features = ["chrono", "py-clone"]} -cel = { version = "0.11.4", features = ["chrono", "json", "regex"] } +pyo3 = { version = "0.27", features = ["chrono", "py-clone"]} +cel = { version = "0.11.6", features = ["chrono", "json", "regex"] } log = "0.4.27" -pyo3-log = "0.12.4" -chrono = { version = "0.4.41", features = ["serde"] } +pyo3-log = { git = "https://github.com/a1phyr/pyo3-log.git", branch = "pyo3_0.27" } +chrono = { version = "0.4.42", features = ["serde"] } diff --git a/README.md b/README.md index 5b71f9e..3171fe1 100644 --- a/README.md +++ b/README.md @@ -63,8 +63,14 @@ cel '1 + 2' # 3 # With context cel 'age >= 18' --context '{"age": 25}' # true -# Interactive REPL +# Batch processing - evaluate expression SEPARATELY for each file +cel 'user.age >= 18' --for-each user1.json --for-each user2.json --for-each user3.json + +# Interactive REPL mode cel --interactive + +# Modern TUI interface +cel --tui ``` ### Custom Functions @@ -108,9 +114,11 @@ access_granted = evaluate(policy, context) # True - ✅ **Fast Evaluation**: Microsecond-level expression evaluation via Rust - ✅ **Rich Type System**: Integers, floats, strings, lists, maps, timestamps, durations - ✅ **Python Integration**: Seamless type conversion and custom function support -- ✅ **CLI Tools**: Interactive REPL and batch processing capabilities +- ✅ **Interactive TUI**: Text-based UI with syntax highlighting and real-time evaluation +- ✅ **CLI Tools**: Command-line evaluation and batch processing capabilities - ✅ **Safety First**: Non-Turing complete, safe for untrusted expressions + ## Documentation 📚 **Complete documentation available at**: https://python-common-expression-language.readthedocs.io/ diff --git a/docs/how-to-guides/cli-recipes.md b/docs/how-to-guides/cli-recipes.md index d2a56e5..9050354 100644 --- a/docs/how-to-guides/cli-recipes.md +++ b/docs/how-to-guides/cli-recipes.md @@ -64,24 +64,63 @@ curl -s https://api.github.com/users/octocat | \ ### Batch Processing + +#### Batch Context Processing + +Evaluate one expression SEPARATELY for each context file: + ```bash -# Process multiple files -for config in configs/*.json; do - echo "Validating $config..." - if cel 'has("database.host") && database.host != ""' --context-file "$config" --exit-status; then - echo "✓ $config is valid" - else - echo "✗ $config is invalid" - fi -done +# Validate age - repeat --for-each for each file +cel 'user.age >= 18' --for-each users/user1.json --for-each users/user2.json --for-each users/user3.json + +# Shorter: use shell loops for glob patterns +for file in users/*.json; do + printf -- "--for-each %s " "$file" +done | xargs cel 'user.age >= 18' + +# Output (table format shows results for each file): +# Expression Results: user.age >= 18 +# ┏━━━┳━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━┓ +# ┃ # ┃ Context File┃ Result ┃ Time (ms) ┃ +# ┡━━━╇━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━┩ +# │ 1 │ user1.json │ True │ 3.10 │ +# │ 2 │ user2.json │ True │ 0.16 │ +# │ 3 │ user3.json │ False │ 0.16 │ +# └───┴─────────────┴────────┴───────────┘ + +# Check product inventory (with JSON output) +cel 'product.inStock && product.price < 500' \ + --for-each inventory/p1.json \ + --for-each inventory/p2.json \ + --output json + +# Validate configuration files - shell helper function +check_configs() { + for f in configs/app-*.json; do echo "--for-each $f"; done | \ + xargs cel 'has(config.database) && config.database.port > 0' +} -# Transform data files -ls data/*.json | while read -r file; do - cel 'user.name + "," + user.email + "," + string(user.age)' \ - --context-file "$file" >> users.csv -done +# Combine with base context (base merged into each file) +cel 'value * multiplier' \ + --context '{"multiplier": 1.2}' \ + --for-each data/metrics-1.json \ + --for-each data/metrics-2.json ``` +**Benefits over loops:** +- Single table output showing all results +- Timing information for each evaluation +- More efficient (no subprocess spawning per file) +- Better for parallel processing +- Clearer visualization of results + +**Use cases:** +- Validate data files against a schema/rule +- Check compliance across multiple configurations +- Filter/identify files matching criteria +- Apply the same transformation to multiple datasets +- Audit logs or records for specific conditions + ## Validation and Testing ### Configuration Validation diff --git a/docs/reference/cli-reference.md b/docs/reference/cli-reference.md index 14bdcaf..f27e98b 100644 --- a/docs/reference/cli-reference.md +++ b/docs/reference/cli-reference.md @@ -87,10 +87,39 @@ cel 'config.valid' -f config.json ``` **Format**: Path to valid JSON file -**Special values**: +**Special values**: - `/dev/stdin` - Read from standard input - `-` - Read from standard input (shorthand) +#### `--for-each` +Evaluate expression SEPARATELY for each context file (batch processing). + +```bash +# Evaluate expression for each file (repeat flag for each file) +cel 'user.age >= 18' --for-each user1.json --for-each user2.json --for-each user3.json + +# With JSON output format +cel 'product.inStock' --for-each products/p1.json --for-each products/p2.json --output json + +# Combine with base context (base merged into each file's context) +cel 'value * multiplier' --context '{"multiplier": 2}' --for-each data1.json --for-each data2.json + +# Convenient shell loop for globs +for file in users/*.json; do echo --for-each "$file"; done | xargs cel 'user.active' +``` + +**Format**: Repeat `--for-each` flag for each file path +**Behavior**: Each file is evaluated independently with the same expression +**Use case**: Apply one validation rule or filter across many data files +**Output**: Table showing results for each context file with timing information + +**Key difference from `--context-file`**: +- `--context-file file.json` - Loads ONE context for the expression +- `--for-each f1.json --for-each f2.json` - Evaluates expression SEPARATELY for EACH file + +**Why repeat the flag?** This is standard CLI behavior (like `git add file1 file2` vs `grep -e pattern1 -e pattern2`). It makes it explicit that each file is processed independently, not merged together. + + ### Interactive Mode #### `--interactive`, `-i` diff --git a/pyproject.toml b/pyproject.toml index bb4b82b..06ca134 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -15,9 +15,12 @@ classifiers = [ dynamic = ["version"] dependencies = [ "typer>=0.12.0", - "rich>=13.0.0", + "rich>=13.0.0", "prompt-toolkit>=3.0.0", "pygments>=2.0.0", + "textual>=1.0.0", + "pyyaml>=6.0.0", + "platformdirs>=4.0.0", ] [project.scripts] @@ -39,6 +42,7 @@ python-source = "python" [tool.uv] dev-dependencies = [ "pytest>=8.4.1", + "pytest-asyncio>=0.23.0", "maturin>=1.8.0", "ruff>=0.12.7", "mypy>=1.17.1", @@ -110,3 +114,7 @@ addopts = [ "--strict-config", "--verbose", ] +markers = [ + "asyncio: mark test as asyncio test", +] +asyncio_mode = "auto" diff --git a/python/cel/cel_lexer.py b/python/cel/cel_lexer.py new file mode 100644 index 0000000..d3e6fca --- /dev/null +++ b/python/cel/cel_lexer.py @@ -0,0 +1,69 @@ +""" +Pygments lexer for Common Expression Language (CEL). + +Provides syntax highlighting for CEL expressions in Textual TextArea widgets. +""" + +__all__ = ["CELLexer"] + +from pygments.lexer import RegexLexer, bygroups, words +from pygments.token import ( + Comment, + Keyword, + Name, + Number, + Operator, + Punctuation, + String, + Text, +) + + +class CELLexer(RegexLexer): + """ + Lexer for Common Expression Language (CEL). + + CEL is a non-Turing complete expression language designed for + safely evaluating expressions in security policies, rules, and more. + """ + + name = "CEL" + aliases = ["cel"] + filenames = ["*.cel"] + mimetypes = ["text/x-cel"] + + tokens = { + "root": [ + # Keywords - boolean literals + (words(("true", "false", "null"), suffix=r"\b"), Keyword.Constant), + # Keywords - operators + (words(("in", "has"), suffix=r"\b"), Keyword), + # String literals - double quoted + (r'"[^"\\]*(?:\\.[^"\\]*)*"', String.Double), + # String literals - single quoted + (r"'[^'\\]*(?:\\.[^'\\]*)*'", String.Single), + # Numbers - integers and floats with optional scientific notation + (r"\d+\.?\d*([eE][+-]?\d+)?", Number), + # Comparison operators + (r"(==|!=|<=|>=|<|>)", Operator), + # Logical operators + (r"(&&|\|\||!)", Operator), + # Arithmetic operators + (r"[+\-*/%]", Operator), + # Ternary operator + (r"(\?|:)", Operator), + # Brackets and delimiters + (r"[(){}\[\],.]", Punctuation), + # Function calls - identifier followed by ( + ( + r"([a-zA-Z_][a-zA-Z0-9_]*)(\s*)(\()", + bygroups(Name.Function, Text, Punctuation), + ), + # Identifiers (variables, properties) + (r"[a-zA-Z_][a-zA-Z0-9_]*", Name), + # Whitespace + (r"\s+", Text), + # Catch-all for any other characters + (r".", Text), + ] + } diff --git a/python/cel/cli.py b/python/cel/cli.py index 383f5f3..b27e303 100644 --- a/python/cel/cli.py +++ b/python/cel/cli.py @@ -534,6 +534,78 @@ def evaluate_expressions_from_file( console.print(table) +def evaluate_expression_with_multiple_contexts( + expression: str, context_files: list[Path], base_context: Dict[str, Any], output_format: str +) -> None: + """Evaluate a single expression against multiple context files with Rich output.""" + if not context_files: + console.print("[yellow]No context files provided[/yellow]") + return + + results = [] + + with console.status(f"[bold green]Evaluating against {len(context_files)} context files..."): + for i, context_file in enumerate(context_files, 1): + try: + # Load context from file + file_context = load_context_from_file(context_file) + + # Merge with base context (file context takes precedence) + eval_context = {**base_context, **file_context} + + # Create evaluator with this context + evaluator = CELEvaluator(eval_context) + + # Evaluate expression + start_time = time.time() + result = evaluator.evaluate(expression) + eval_time = time.time() - start_time + + results.append( + { + "context_file": str(context_file), + "result": result, + "time_ms": eval_time * 1000, + } + ) + + except Exception as e: + console.print(f"[red]Error with context file '{context_file}': {e}[/red]") + results.append({"context_file": str(context_file), "error": str(e)}) + + # Display results + if output_format == "json": + json_output = json.dumps(results, indent=2, default=str) + syntax = Syntax(json_output, "json", theme="monokai") + console.print(syntax) + else: + table = Table( + title=f"Expression Results: {expression}", show_header=True, header_style="bold magenta" + ) + table.add_column("#", style="dim", width=3) + table.add_column("Context File", style="cyan") + table.add_column("Result", style="green") + table.add_column("Time (ms)", style="yellow") + + for i, result in enumerate(results, 1): + if "error" in result: + context_name = Path(result["context_file"]).name + table.add_row( + str(i), + context_name, + f"[red]Error: {result['error']}[/red]", + "—", + ) + else: + context_name = Path(result["context_file"]).name + result_str = str(result["result"]) + if len(result_str) > 50: + result_str = result_str[:47] + "..." + table.add_row(str(i), context_name, result_str, f"{result['time_ms']:.2f}") + + console.print(table) + + @app.command() def main( expression: Annotated[Optional[str], typer.Argument(help="CEL expression to evaluate")] = None, @@ -544,6 +616,13 @@ def main( Optional[Path], typer.Option("-f", "--context-file", help="Load context from JSON file"), ] = None, + for_each: Annotated[ + Optional[list[Path]], + typer.Option( + "--for-each", + help="Evaluate expression SEPARATELY for each context file. Repeat for multiple files: --for-each file1.json --for-each file2.json", + ), + ] = None, file: Annotated[ Optional[Path], typer.Option("--file", help="Read expressions from file (one per line)"), @@ -552,6 +631,9 @@ def main( interactive: Annotated[ bool, typer.Option("-i", "--interactive", help="Start interactive REPL mode") ] = False, + tui: Annotated[ + bool, typer.Option("--tui", help="Launch graphical TUI interface") + ] = False, timing: Annotated[bool, typer.Option("-t", "--timing", help="Show evaluation timing")] = False, verbose: Annotated[bool, typer.Option("-v", "--verbose", help="Verbose output")] = False, version: Annotated[ @@ -575,9 +657,15 @@ def main( # Load context from file cel 'user.name' --context-file context.json + # Batch processing - evaluate expression SEPARATELY for each file + cel 'user.age >= 18' --for-each user1.json --for-each user2.json --for-each user3.json + # Interactive REPL mode cel --interactive + # Graphical TUI mode + cel --tui + # Evaluate expressions from file cel --file expressions.cel --output json @@ -602,17 +690,46 @@ def main( # Initialize evaluator evaluator = CELEvaluator(eval_context) - # Interactive mode + # Interactive REPL mode if interactive: repl = InteractiveCELREPL(evaluator) repl.run() return + # TUI mode + if tui: + try: + from .tui import run_tui + + run_tui() + except ImportError as e: + console.print( + "[red]Error: TUI requires the 'textual' and 'pyyaml' packages. " + "Install with: pip install textual pyyaml[/red]" + ) + console.print(f"[dim]Details: {e}[/dim]") + raise typer.Exit(1) from e + except Exception as e: + console.print(f"[red]Error launching TUI: {e}[/red]") + raise typer.Exit(1) from e + return + # File mode if file: evaluate_expressions_from_file(file, evaluator, output) return + # Batch context mode - evaluate one expression against multiple context files + if for_each: + if not expression: + console.print( + "[red]Error: --for-each requires an expression argument.[/red]" + ) + console.print("\nExample: [bold]cel 'user.age >= 18' --for-each user1.json user2.json user3.json[/bold]") + raise typer.Exit(1) + evaluate_expression_with_multiple_contexts(expression, for_each, eval_context, output) + return + # Single expression evaluation if not expression: console.print( diff --git a/python/cel/expression_storage.py b/python/cel/expression_storage.py new file mode 100644 index 0000000..ee929af --- /dev/null +++ b/python/cel/expression_storage.py @@ -0,0 +1,190 @@ +""" +Expression storage for CEL TUI. + +Manages user-defined expressions stored in OS-specific configuration directory. +""" + +import json +from pathlib import Path +from typing import List, Tuple + +try: + from platformdirs import user_config_dir + HAS_PLATFORMDIRS = True +except ImportError: + HAS_PLATFORMDIRS = False + + +def get_config_dir() -> Path: + """Get OS-specific configuration directory for CEL.""" + if HAS_PLATFORMDIRS: + # Use platformdirs for proper cross-platform config directory + config_dir = Path(user_config_dir("cel", appauthor=False)) + else: + # Fallback for when platformdirs is not available + import sys + if sys.platform == "win32": + # Windows: %APPDATA%\cel + base = Path.home() / "AppData" / "Roaming" + config_dir = base / "cel" + elif sys.platform == "darwin": + # macOS: ~/Library/Application Support/cel + base = Path.home() / "Library" / "Application Support" + config_dir = base / "cel" + else: + # Linux/Unix: ~/.config/cel (XDG Base Directory spec) + xdg_config = Path.home() / ".config" + config_dir = xdg_config / "cel" + + # Create directory if it doesn't exist + config_dir.mkdir(parents=True, exist_ok=True) + return config_dir + + +def get_expressions_file() -> Path: + """Get path to user expressions JSON file.""" + return get_config_dir() / "expressions.json" + + +def load_user_expressions() -> List[Tuple[str, str, str]]: + """ + Load user-defined expressions from config file. + + Returns: + List of tuples: (name, description, expression) + """ + expressions_file = get_expressions_file() + + if not expressions_file.exists(): + return [] + + try: + with expressions_file.open("r", encoding="utf-8") as f: + data = json.load(f) + + # Validate structure + if not isinstance(data, list): + return [] + + expressions = [] + for item in data: + if isinstance(item, dict) and all(k in item for k in ("name", "description", "expression")): + expressions.append(( + item["name"], + item["description"], + item["expression"] + )) + + return expressions + except (json.JSONDecodeError, IOError): + # If file is corrupted or unreadable, return empty list + return [] + + +def save_user_expressions(expressions: List[Tuple[str, str, str]]) -> None: + """ + Save user-defined expressions to config file. + + Args: + expressions: List of tuples (name, description, expression) + """ + expressions_file = get_expressions_file() + + # Convert to list of dicts for JSON serialization + data = [ + { + "name": name, + "description": description, + "expression": expression + } + for name, description, expression in expressions + ] + + try: + with expressions_file.open("w", encoding="utf-8") as f: + json.dump(data, f, indent=2, ensure_ascii=False) + except IOError as e: + raise IOError(f"Failed to save expressions: {e}") from e + + +def add_expression(name: str, description: str, expression: str) -> None: + """ + Add a new expression to user's library. + + Args: + name: Display name for the expression + description: Human-readable description + expression: The CEL expression string + + Raises: + ValueError: If an expression with the same name already exists + """ + expressions = load_user_expressions() + + # Check for duplicate names + existing_names = {expr[0] for expr in expressions} + if name in existing_names: + raise ValueError(f"Expression '{name}' already exists") + + # Add new expression + expressions.append((name, description, expression)) + save_user_expressions(expressions) + + +def delete_expression(name: str) -> bool: + """ + Delete an expression from user's library. + + Args: + name: Name of the expression to delete + + Returns: + True if expression was deleted, False if not found + """ + expressions = load_user_expressions() + + # Filter out the expression to delete + new_expressions = [expr for expr in expressions if expr[0] != name] + + # Check if anything was deleted + if len(new_expressions) == len(expressions): + return False + + save_user_expressions(new_expressions) + return True + + +def update_expression(old_name: str, new_name: str, description: str, expression: str) -> None: + """ + Update an existing expression. + + Args: + old_name: Current name of the expression + new_name: New name for the expression + description: New description + expression: New expression string + + Raises: + ValueError: If expression not found or new name conflicts + """ + expressions = load_user_expressions() + + # Find the expression to update + found_index = None + for i, (name, _, _) in enumerate(expressions): + if name == old_name: + found_index = i + break + + if found_index is None: + raise ValueError(f"Expression '{old_name}' not found") + + # Check if new name conflicts (unless it's the same name) + if new_name != old_name: + existing_names = {expr[0] for i, expr in enumerate(expressions) if i != found_index} + if new_name in existing_names: + raise ValueError(f"Expression '{new_name}' already exists") + + # Update the expression + expressions[found_index] = (new_name, description, expression) + save_user_expressions(expressions) diff --git a/python/cel/tui.py b/python/cel/tui.py new file mode 100644 index 0000000..6fca3b7 --- /dev/null +++ b/python/cel/tui.py @@ -0,0 +1,933 @@ +#!/usr/bin/env python3 +""" +CEL TUI - Figma Design Implementation + +A professional 3-column TUI implementing the Figma design: +- Left: Expression library with examples +- Middle: Context editor with JSON/YAML/URL loading +- Right: Evaluation and results +""" + +import json +from datetime import datetime +from pathlib import Path +from typing import Any, Dict, List, Optional, Tuple +from urllib.request import urlopen + +import yaml +from textual import events, on, work +from textual.app import App, ComposeResult +from textual.binding import Binding +from textual.containers import Container, Horizontal, Vertical, VerticalScroll +from textual.widgets import Button, Footer, Header, Input, Label, Static, TextArea + +from .cel import Context, evaluate +from .cel_lexer import CELLexer +from .expression_storage import ( + add_expression, + delete_expression, + get_expressions_file, + load_user_expressions, +) +from .stdlib import add_stdlib_to_context + +# Register CEL lexer with Pygments so TextArea can use it +try: + from pygments.lexers import get_lexer_by_name + from pygments.lexers._mapping import LEXERS + + # Register our custom lexer + LEXERS["CELLexer"] = ( + "cel.cel_lexer", + "CEL", + ("cel",), + ("*.cel",), + ("text/x-cel",), + ) +except ImportError: + pass # Pygments not available + +# Example expressions library +EXPRESSION_LIBRARY = [ + ("Age Check", "Check if user is 18 or older", "user.age >= 18"), + ("Role Validation", "Check if user has admin role", '"admin" in user.roles'), + ("String Manipulation", "Check name starts with A and length > 3", + 'user.name.startsWith("A") && user.name.size() > 3'), + ("List Operations", "Validate all roles are non-empty", + "user.roles.size() > 0 && user.roles.all(r, r.size() > 0)"), + ("Conditional Expression", "Return age category", + 'user.age < 18 ? "minor" : "adult"'), + ("Complex Boolean", "Validate API GET request", + 'request.method == "GET" && request.path.startsWith("/api/")'), + ("Mathematical", "Age in months > 300", "user.age * 12 > 300"), + ("Map Access", "Check for company email", + 'has(user.email) && user.email.endsWith("@example.com")'), +] + + +class ExpressionLibrary(VerticalScroll): + """Left sidebar with searchable expression examples.""" + + DEFAULT_CLASSES = "expression-library" + + def __init__(self) -> None: + super().__init__() + self.built_in_expressions = EXPRESSION_LIBRARY + self.user_expressions: List[Tuple[str, str, str]] = [] + self.load_expressions() + + def load_expressions(self) -> None: + """Load user expressions from config file.""" + try: + self.user_expressions = load_user_expressions() + except Exception: + self.user_expressions = [] + # Will notify on mount + + def compose(self) -> ComposeResult: + yield Label("📚 Expression Library", classes="lib-title") + yield Input(placeholder="🔍 Search...", id="search-input", classes="search-box") + + # Add save button + with Horizontal(classes="lib-actions"): + yield Button("💾 Save Current", id="save-expr-btn", variant="primary", classes="save-btn") + yield Label(f"({len(self.user_expressions)} saved)", id="saved-count", classes="saved-count") + + yield Container(id="expression-list") + + def on_mount(self) -> None: + """Populate expression list on mount.""" + self._update_list() + # Show info about config location + config_file = get_expressions_file() + self.notify( + f"User expressions: {config_file}", + severity="information", + timeout=3 + ) + + @on(Input.Changed, "#search-input") + def filter_expressions(self, event: Input.Changed) -> None: + """Filter expressions based on search.""" + self._update_list(event.value.lower()) + + def _update_list(self, search: str = "") -> None: + """Update the expression list.""" + container = self.query_one("#expression-list", Container) + container.remove_children() + + # Add user expressions first (if any) + if self.user_expressions: + user_label = Label("👤 Your Expressions", classes="section-label") + container.mount(user_label) + + for idx, (title, desc, expr) in enumerate(self.user_expressions): + if search and search not in title.lower() and search not in expr.lower(): + continue + + # Create expression card with delete button + # Use index for ID since titles can have spaces + card = Container( + Horizontal( + Label(title, classes="expr-title"), + Button("🗑", id=f"delete-{idx}", classes="delete-btn"), + classes="expr-title-row" + ), + Label(desc, classes="expr-desc"), + Label(expr, classes="expr-code"), + classes="expr-card user-expr-card" + ) + container.mount(card) + + # Add built-in expressions + if self.user_expressions: # Add separator if we have user expressions + builtin_label = Label("📖 Built-in Examples", classes="section-label") + container.mount(builtin_label) + + for title, desc, expr in self.built_in_expressions: + if search and search not in title.lower() and search not in expr.lower(): + continue + + # Create expression card (no delete button for built-ins) + card = Container( + Label(title, classes="expr-title"), + Label(desc, classes="expr-desc"), + Label(expr, classes="expr-code"), + classes="expr-card" + ) + container.mount(card) + + # Update saved count + try: + saved_count = self.query_one("#saved-count", Label) + saved_count.update(f"({len(self.user_expressions)} saved)") + except Exception: + pass # Widget might not be mounted yet + + def on_click(self, event: events.Click) -> None: + """Handle clicks on expression cards.""" + # Find if we clicked within an expr-card + widget = event.widget + while widget and widget != self: + if "expr-card" in widget.classes: + code_label = widget.query_one(".expr-code", Label) + if code_label: + expr_area = self.app.query_one("#expression-input", TextArea) + expr_area.clear() + expr_area.insert(str(code_label.render())) + self.notify("Loaded expression", severity="information") + break + widget = widget.parent + + @on(Button.Pressed, "#save-expr-btn") + def handle_save_expression(self) -> None: + """Handle save expression button.""" + # Get current expression from the input area + expr_area = self.app.query_one("#expression-input", TextArea) + expression = expr_area.text.strip() + + if not expression: + self.notify("No expression to save", severity="warning") + return + + # Trigger the save dialog in the main app + self.app.show_save_dialog(expression) + + @on(Button.Pressed, ".delete-btn") + def handle_delete_expression(self, event: Button.Pressed) -> None: + """Handle delete button click on user expression.""" + button_id = event.button.id + if not button_id or not button_id.startswith("delete-"): + return + + # Extract index from button ID + try: + idx = int(button_id[7:]) # Remove "delete-" prefix + expr_name = self.user_expressions[idx][0] + except (ValueError, IndexError): + self.notify("Could not identify expression to delete", severity="error") + return + + try: + if delete_expression(expr_name): + # Reload expressions and update display + self.load_expressions() + self._update_list() + self.notify(f"Deleted '{expr_name}'", severity="information") + else: + self.notify(f"Expression '{expr_name}' not found", severity="error") + except Exception as e: + self.notify(f"Error deleting expression: {e}", severity="error") + + +class ContextEditor(VerticalScroll): + """Middle column for context management.""" + + DEFAULT_CLASSES = "context-editor" + + def compose(self) -> ComposeResult: + yield Label("📄 Context", classes="section-title") + + # Load Context section + with Container(classes="load-section"): + yield Label("⬇ Load Context", classes="subsection-title") + yield Label("From URL", classes="field-label") + with Horizontal(classes="url-row"): + yield Input( + placeholder="https://api.example.com/context.json", + id="url-input", + classes="url-input" + ) + yield Button("🔗", id="load-url-btn", classes="icon-btn") + + yield Label("From File", classes="field-label") + with Horizontal(classes="file-row"): + yield Input( + placeholder="./context.json or ./config.yaml", + id="file-input", + classes="file-input" + ) + yield Button("📤 Upload JSON File", id="upload-btn", + variant="success", classes="upload-btn") + + # Context Data section + with Container(classes="data-section"): + with Horizontal(classes="data-header"): + yield Label("📋 Context Data (JSON)", classes="subsection-title") + yield Label("", id="validation-status", classes="validation") + + yield TextArea( + '{\n "user": {\n "name": "Alice",\n "age": 30,\n ' + '"roles": ["admin", "user"]\n },\n "request": {\n ' + '"method": "GET",\n "path": "/api/data"\n }\n}', + language="json", + theme="monokai", + id="context-input", + classes="context-area" + ) + + +class ResultsPanel(VerticalScroll): + """Right column for evaluation and results.""" + + DEFAULT_CLASSES = "results-panel" + + def compose(self) -> ComposeResult: + # Evaluate section + with Container(classes="eval-section"): + yield Button("▶ Evaluate", id="eval-btn", variant="success", + classes="eval-button") + + # Expression input + with Container(classes="expr-section"): + yield Label("CEL Expression:", classes="field-label") + yield TextArea( + 'user.age >= 18', + language="cel", + theme="monokai", + id="expression-input", + classes="expression-area" + ) + + # Results section + with Container(classes="results-section"): + yield Label("◎ Results", classes="subsection-title") + yield Label("", id="current-expression", classes="current-expr") + yield Static("", id="result-display", classes="result-content") + yield Label("", id="result-meta", classes="result-meta") + + +class CELTuiApp(App): + """CEL TUI implementing Figma design.""" + + CSS = """ + Screen { + background: #0a0e14; + } + + Header { + background: #0d1117; + color: #00ff41; + } + + Footer { + background: #0d1117; + } + + .main-container { + layout: horizontal; + height: 1fr; + } + + /* Left Column - Expression Library */ + .expression-library { + width: 1fr; + background: #0d1117; + border-right: solid #1a3a1a; + padding: 1 2; + } + + .lib-title { + color: #00ff41; + text-style: bold; + margin-bottom: 1; + } + + .search-box { + margin-bottom: 1; + border: solid #1a3a1a; + background: #0a0e14; + color: #00ff41; + } + + #expression-list { + height: auto; + } + + .lib-actions { + height: auto; + margin-bottom: 1; + } + + .save-btn { + width: 1fr; + background: #1a3a1a; + color: #00ff41; + } + + .saved-count { + color: #6b8e6b; + margin-left: 1; + width: auto; + } + + .section-label { + color: #00ff41; + text-style: bold; + margin-top: 1; + margin-bottom: 1; + } + + .expr-card { + background: #0a0e14; + border: solid #1a3a1a; + padding: 1; + margin-bottom: 1; + } + + .expr-card:hover { + background: #0d1a0d; + border: solid #00ff41; + } + + .user-expr-card { + border: solid #00ff41; + background: #0d1a0d; + } + + .expr-title-row { + height: auto; + } + + .expr-title { + color: #00ff41; + text-style: bold; + width: 1fr; + } + + .delete-btn { + width: 5; + min-width: 5; + background: #3a1a1a; + color: #ff4444; + } + + .delete-btn:hover { + background: #ff4444; + color: #0a0e14; + } + + .expr-desc { + color: #6b8e6b; + margin-top: 0; + } + + .expr-code { + color: #00cc33; + text-style: italic; + margin-top: 1; + } + + /* Middle Column - Context */ + .context-editor { + width: 2fr; + background: #0a0e14; + border-right: solid #1a3a1a; + padding: 1 2; + } + + .section-title { + color: #00ff41; + text-style: bold; + margin-bottom: 1; + } + + .subsection-title { + color: #00ff41; + margin-bottom: 1; + } + + .load-section { + margin-bottom: 2; + } + + .field-label { + color: #6b8e6b; + margin-bottom: 1; + margin-top: 1; + } + + .url-row, .file-row { + height: auto; + margin-bottom: 1; + } + + .url-input, .file-input { + width: 1fr; + border: solid #1a3a1a; + background: #0d1117; + color: #00ff41; + } + + .icon-btn { + width: 5; + min-width: 5; + margin-left: 1; + } + + .upload-btn { + margin-left: 1; + background: #1a3a1a; + color: #00ff41; + } + + .data-section { + height: 1fr; + } + + .data-header { + height: auto; + } + + .validation { + color: #00ff41; + margin-left: 1; + } + + .context-area { + height: 1fr; + border: solid #1a3a1a; + } + + /* Right Column - Results */ + .results-panel { + width: 1fr; + background: #0a0e14; + padding: 1 2; + } + + .eval-section { + margin-bottom: 2; + } + + .eval-button { + width: 100%; + background: #1a3a1a; + color: #00ff41; + } + + .eval-button:hover { + background: #00ff41; + color: #0a0e14; + } + + .expr-section { + margin-bottom: 2; + } + + .expression-area { + height: 10; + border: solid #1a3a1a; + } + + .results-section { + height: 1fr; + } + + .current-expr { + color: #6b8e6b; + text-style: italic; + margin-bottom: 1; + padding: 0 1; + } + + .result-content { + background: #0d1117; + border: solid #1a3a1a; + padding: 2; + color: #00ff41; + min-height: 10; + } + + .result-meta { + color: #6b8e6b; + margin-top: 1; + text-style: italic; + } + + Input { + height: 3; + } + + Button { + height: 3; + } + """ + + BINDINGS = [ + Binding("ctrl+q", "quit", "Quit", show=True), + Binding("ctrl+e", "evaluate", "Evaluate", show=True), + Binding("f1", "show_help", "Help", show=True), + ] + + TITLE = "CEL Expression Evaluator" + SUB_TITLE = "Common Expression Language Testing & Development Environment" + + def __init__(self) -> None: + super().__init__() + self.context: Optional[Context] = None + self.context_dict: Dict[str, Any] = {} + + def compose(self) -> ComposeResult: + yield Header() + with Horizontal(classes="main-container"): + yield ExpressionLibrary() + yield ContextEditor() + yield ResultsPanel() + yield Footer() + + def on_mount(self) -> None: + """Initialize with default context.""" + self._load_context_from_editor() + self.notify("CEL Evaluator ready • Press Ctrl+E to evaluate • F1 for help", + severity="information") + + @on(Button.Pressed, "#eval-btn") + def handle_eval_button(self) -> None: + """Handle evaluate button press.""" + self.action_evaluate() + + @on(Button.Pressed, "#load-url-btn") + def handle_load_url(self) -> None: + """Load context from URL.""" + url_input = self.query_one("#url-input", Input) + url = url_input.value.strip() + + if not url: + self.notify("Please enter a URL", severity="warning") + return + + try: + self.notify(f"Loading from {url}...", severity="information") + with urlopen(url, timeout=10) as response: + content = response.read().decode('utf-8') + data = json.loads(content) + + # Update context editor + context_area = self.query_one("#context-input", TextArea) + context_area.clear() + context_area.insert(json.dumps(data, indent=2)) + + self._load_context_from_editor() + self.notify("✓ Loaded from URL", severity="information") + + except Exception as e: + self.notify(f"✗ Error loading URL: {e}", severity="error") + + @on(Button.Pressed, "#upload-btn") + def handle_upload(self) -> None: + """Load context from file.""" + file_input = self.query_one("#file-input", Input) + file_path = file_input.value.strip() + + if not file_path: + self.notify("Please enter a file path", severity="warning") + return + + try: + path = Path(file_path).expanduser() + if not path.exists(): + self.notify(f"✗ File not found: {file_path}", severity="error") + return + + content = path.read_text() + + # Parse based on extension + if path.suffix in ['.yaml', '.yml']: + data = yaml.safe_load(content) + else: + data = json.loads(content) + + # Update context editor + context_area = self.query_one("#context-input", TextArea) + context_area.clear() + context_area.insert(json.dumps(data, indent=2)) + + self._load_context_from_editor() + self.notify("✓ Loaded from file", severity="information") + + except Exception as e: + self.notify(f"✗ Error loading file: {e}", severity="error") + + def _load_context_from_editor(self) -> None: + """Load context from the JSON editor.""" + try: + context_area = self.query_one("#context-input", TextArea) + context_json = context_area.text + + # Parse JSON + self.context_dict = json.loads(context_json) + + # Create CEL context + self.context = Context() + add_stdlib_to_context(self.context) + + for key, value in self.context_dict.items(): + self.context.add_variable(key, value) + + # Update validation status + validation = self.query_one("#validation-status", Label) + validation.update("✓ Valid 11 lines") + validation.styles.color = "#00ff41" + + except json.JSONDecodeError as e: + validation = self.query_one("#validation-status", Label) + validation.update("✗ Invalid JSON") + validation.styles.color = "#ff4444" + self.notify(f"Invalid JSON: {e}", severity="error") + + @work(exclusive=True, thread=True) + async def action_evaluate(self) -> None: + """Evaluate the CEL expression.""" + try: + # Ensure context is loaded + if self.context is None: + self._load_context_from_editor() + + expr_area = self.query_one("#expression-input", TextArea) + expression = expr_area.text.strip() + + if not expression: + self.notify("Expression is empty", severity="warning") + result_display = self.query_one("#result-display", Static) + result_display.update( + '[dim]ℹ️ No expression yet\n\n' + 'Click "Evaluate" to run the expression[/dim]' + ) + # Clear current expression display + current_expr_label = self.query_one("#current-expression", Label) + current_expr_label.update("") + return + + # Show what we're evaluating with syntax highlighting + current_expr_label = self.query_one("#current-expression", Label) + expr_preview = expression if len(expression) <= 60 else expression[:57] + "..." + highlighted = self._syntax_highlight_expr(expr_preview) + current_expr_label.update(f"⟳ Evaluating: {highlighted}") + + # Measure evaluation time + start_time = datetime.now() + result = evaluate(expression, self.context) + end_time = datetime.now() + elapsed_ms = (end_time - start_time).total_seconds() * 1000 + + # Update result display + self._update_result(result, elapsed_ms, expression) + + self.notify("✓ Evaluated successfully", severity="information") + + except Exception as e: + self._update_result_error(str(e)) + self.notify(f"✗ Error: {e}", severity="error") + + def _update_result(self, result: Any, elapsed_ms: float, expression: str = "") -> None: + """Update the result display with success.""" + result_display = self.query_one("#result-display", Static) + result_meta = self.query_one("#result-meta", Label) + current_expr_label = self.query_one("#current-expression", Label) + + # Format result + if isinstance(result, str): + result_str = f'"{result}"' + elif isinstance(result, (list, dict)): + result_str = json.dumps(result, indent=2) + else: + result_str = str(result) + + result_display.update(result_str) + result_meta.update(f"type: {type(result).__name__} | {elapsed_ms:.2f}ms") + + # Update current expression label to show what was evaluated with syntax highlighting + if expression: + expr_preview = expression if len(expression) <= 60 else expression[:57] + "..." + highlighted = self._syntax_highlight_expr(expr_preview) + current_expr_label.update(f"✓ {highlighted}") + + def _syntax_highlight_expr(self, expr: str) -> str: + """Apply basic syntax highlighting to CEL expression using Rich markup.""" + import re + + # Use a token-based approach to avoid overlapping markup + tokens = [] + i = 0 + + # Simple tokenizer + while i < len(expr): + # Skip whitespace + if expr[i].isspace(): + tokens.append(('space', expr[i])) + i += 1 + continue + + # String literals + if expr[i] in '"\'': + quote = expr[i] + j = i + 1 + while j < len(expr) and expr[j] != quote: + if expr[j] == '\\' and j + 1 < len(expr): + j += 2 + else: + j += 1 + if j < len(expr): + j += 1 + tokens.append(('string', expr[i:j])) + i = j + continue + + # Numbers + if expr[i].isdigit(): + j = i + while j < len(expr) and (expr[j].isdigit() or expr[j] == '.'): + j += 1 + tokens.append(('number', expr[i:j])) + i = j + continue + + # Identifiers/keywords + if expr[i].isalpha() or expr[i] == '_': + j = i + while j < len(expr) and (expr[j].isalnum() or expr[j] == '_'): + j += 1 + word = expr[i:j] + # Check if keyword + if word in ['true', 'false', 'null', 'in', 'has']: + tokens.append(('keyword', word)) + else: + # Check if function (followed by '(') + k = j + while k < len(expr) and expr[k].isspace(): + k += 1 + if k < len(expr) and expr[k] == '(': + tokens.append(('function', word)) + else: + tokens.append(('identifier', word)) + i = j + continue + + # Multi-char operators + if i + 1 < len(expr): + two_char = expr[i:i+2] + if two_char in ['>=', '<=', '==', '!=', '&&', '||']: + tokens.append(('operator', two_char)) + i += 2 + continue + + # Single char operators and punctuation + if expr[i] in '+-*/%<>!&|()[]{},.': + tokens.append(('operator', expr[i])) + i += 1 + continue + + # Anything else + tokens.append(('other', expr[i])) + i += 1 + + # Build highlighted string + result = [] + for token_type, text in tokens: + if token_type == 'keyword': + result.append(f'[bold green]{text}[/bold green]') + elif token_type == 'string': + result.append(f'[yellow]{text}[/yellow]') + elif token_type == 'number': + result.append(f'[cyan]{text}[/cyan]') + elif token_type == 'function': + result.append(f'[magenta]{text}[/magenta]') + elif token_type == 'operator': + result.append(f'[bold]{text}[/bold]') + else: + result.append(text) + + return ''.join(result) + + def _update_result_error(self, error_msg: str) -> None: + """Update result display with error.""" + result_display = self.query_one("#result-display", Static) + result_meta = self.query_one("#result-meta", Label) + + result_display.update(f"[red]✗ Error:[/red]\n\n{error_msg}") + result_meta.update("") + + def show_save_dialog(self, expression: str) -> None: + """ + Show save dialog for current expression. + + For now, uses a simple prompt system. In future, can be replaced with a modal. + """ + # For this implementation, we'll use notifications to prompt user + # A full modal dialog would require more complex Textual screens + + # Prompt for name via notification + self.notify( + "💾 To save expression:\n" + "1. Note your expression\n" + "2. Edit config file directly\n" + f"3. Location: {get_expressions_file()}\n" + "\nOr use the CLI to add expressions programmatically", + severity="information", + timeout=10 + ) + + # For now, show a simplified save with auto-generated name + # Count existing user expressions to create unique name + library = self.query_one(ExpressionLibrary) + expr_count = len(library.user_expressions) + 1 + auto_name = f"Custom Expression {expr_count}" + + try: + # Auto-save with generated name and prompt user to edit file for custom name + add_expression( + name=auto_name, + description="User-defined expression (edit config to customize)", + expression=expression + ) + + # Reload the library + library.load_expressions() + library._update_list() + + self.notify( + f"✓ Saved as '{auto_name}'\n" + f"Edit {get_expressions_file()} to customize name/description", + severity="information", + timeout=8 + ) + except ValueError as e: + self.notify(f"✗ Error saving: {e}", severity="error") + except Exception as e: + self.notify(f"✗ Unexpected error: {e}", severity="error") + + def action_show_help(self) -> None: + """Show help information.""" + help_text = """ +[bold #00ff41]CEL Evaluator - Help[/bold #00ff41] + +[#00ff41]Keyboard Shortcuts:[/#00ff41] +• Ctrl+E - Evaluate expression +• Ctrl+Q - Quit application +• F1 - Show this help + +[#00ff41]Expression Library (Left):[/#00ff41] +Click any example to load it into the expression editor + +[#00ff41]Context Loading (Middle):[/#00ff41] +• Enter URL and click 🔗 to load from API +• Enter file path and click Upload to load from file +• Or edit JSON directly in the editor + +[#00ff41]Evaluation (Right):[/#00ff41] +• Edit expression in the text area +• Click "Evaluate" or press Ctrl+E +• Results appear below with timing info + """ + self.notify(help_text.strip(), severity="information", timeout=15) + + +def run_tui() -> None: + """Entry point to run the CEL TUI application.""" + app = CELTuiApp() + app.run() + + +if __name__ == "__main__": + run_tui() diff --git a/src/lib.rs b/src/lib.rs index f951a49..a5c92b2 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -2,7 +2,7 @@ mod context; use ::cel::objects::{Key, TryIntoValue}; use ::cel::{Context as CelContext, ExecutionError, Program, Value}; -use log::debug; +use log::warn; use pyo3::exceptions::{PyRuntimeError, PyTypeError, PyValueError}; use pyo3::prelude::*; use pyo3::BoundObject; @@ -32,10 +32,7 @@ impl<'py> IntoPyObject<'py> for RustyCelType { RustyCelType(Value::Int(i64)) => i64.into_pyobject(py)?.into_any(), RustyCelType(Value::UInt(u64)) => u64.into_pyobject(py)?.into_any(), RustyCelType(Value::Float(f)) => f.into_pyobject(py)?.into_any(), - RustyCelType(Value::Timestamp(ts)) => { - debug!("Converting a fixed offset datetime to python type"); - ts.into_pyobject(py)?.into_any() - } + RustyCelType(Value::Timestamp(ts)) => ts.into_pyobject(py)?.into_any(), RustyCelType(Value::Duration(d)) => d.into_pyobject(py)?.into_any(), RustyCelType(Value::String(s)) => s.as_ref().to_string().into_pyobject(py)?.into_any(), RustyCelType(Value::List(val)) => { @@ -200,19 +197,19 @@ impl TryIntoValue for RustyPyType<'_> { Ok(Value::Duration(value)) } else if let Ok(value) = pyobject.extract::() { Ok(Value::String(value.into())) - } else if let Ok(value) = pyobject.downcast::() { + } else if let Ok(value) = pyobject.cast::() { let list = value .iter() .map(|item| RustyPyType(&item).try_into_value()) .collect::, Self::Error>>(); list.map(|v| Value::List(Arc::new(v))) - } else if let Ok(value) = pyobject.downcast::() { + } else if let Ok(value) = pyobject.cast::() { let list = value .iter() .map(|item| RustyPyType(&item).try_into_value()) .collect::, Self::Error>>(); list.map(|v| Value::List(Arc::new(v))) - } else if let Ok(value) = pyobject.downcast::() { + } else if let Ok(value) = pyobject.cast::() { let mut map: HashMap = HashMap::new(); for (key, value) in value.into_iter() { let key = if key.is_none() { @@ -407,7 +404,7 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes // Clone variables and functions into our local Context ctx.variables = py_context_ref.variables.clone(); ctx.functions = py_context_ref.functions.clone(); - } else if let Ok(py_dict) = evaluation_context.downcast::() { + } else if let Ok(py_dict) = evaluation_context.cast::() { // User passed in a dict - let's process variables and functions from the dict ctx.update(py_dict)?; } else { @@ -419,12 +416,10 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes variables_for_env = ctx.variables.clone(); } - // Strict mode only - preserve original expression without any preprocessing - let processed_src = src.clone(); - // Use panic::catch_unwind to handle parser panics gracefully - let program = panic::catch_unwind(|| Program::compile(&processed_src)) + let program = panic::catch_unwind(|| Program::compile(&src)) .map_err(|_| { + warn!("CEL parser panic for expression: '{}'", src); PyValueError::new_err(format!( "Failed to parse expression '{src}': Invalid syntax or malformed string" )) @@ -445,7 +440,7 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes // Register Python functions for (function_name, py_function) in ctx.functions.iter() { // Create a wrapper function - let py_func_clone = Python::with_gil(|py| py_function.clone_ref(py)); + let py_func_clone = Python::attach(|py| py_function.clone_ref(py)); let func_name_clone = function_name.clone(); // Register a function that takes Arguments (variadic) and returns a Value @@ -455,7 +450,7 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes let py_func = py_func_clone.clone(); let func_name = func_name_clone.clone(); - Python::with_gil(|py| { + Python::attach(|py| { // Convert CEL arguments to Python objects let mut py_args = Vec::new(); for cel_value in args.0.iter() { @@ -479,6 +474,7 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes // Call the Python function let py_result = py_func.call1(py, py_args_tuple).map_err(|e| { + warn!("Python function '{}' failed: {}", func_name, e); ExecutionError::FunctionError { function: func_name.clone(), message: format!("Python function call failed: {e}"), @@ -508,18 +504,14 @@ fn evaluate(src: String, evaluation_context: Option<&Bound<'_, PyAny>>) -> PyRes // AssertUnwindSafe is needed because the environment contains function closures let result = panic::catch_unwind(AssertUnwindSafe(|| program.execute(&environment))).map_err(|_| { + warn!("CEL execution panic for expression: '{}'", src); PyValueError::new_err(format!( "Failed to execute expression '{src}': Internal parser error" )) })?; match result { - Err(error) => { - debug!("An error occurred during execution"); - debug!("Execution error: {error:?}"); - Err(map_execution_error_to_python(&error)) - } - + Err(error) => Err(map_execution_error_to_python(&error)), Ok(value) => Ok(RustyCelType(value)), } } diff --git a/tests/test_cli.py b/tests/test_cli.py index f7b2916..8fc5c99 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py @@ -22,6 +22,7 @@ CELEvaluator, CELFormatter, InteractiveCELREPL, + evaluate_expression_with_multiple_contexts, evaluate_expressions_from_file, load_context_from_file, ) @@ -487,5 +488,1545 @@ def test_repl_command_parsing_with_spaces(self): Path(spaced_path).unlink() +class TestBatchContextProcessing: + """Test batch context processing - evaluating one expression against multiple context files.""" + + def test_evaluate_expression_with_multiple_contexts_success(self): + """Test evaluating an expression against multiple context files.""" + # Create multiple context files + contexts = [ + {"user": {"name": "Alice", "age": 25}}, + {"user": {"name": "Bob", "age": 30}}, + {"user": {"name": "Charlie", "age": 22}}, + ] + + temp_files = [] + try: + # Create temporary context files + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_user{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # Evaluate expression against all contexts + expression = "user.age >= 25" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should have printed results + assert mock_console.print.called + + finally: + # Clean up temp files + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_json_output(self): + """Test batch context processing with JSON output format.""" + contexts = [ + {"value": 10}, + {"value": 20}, + {"value": 30}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_val{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + expression = "value * 2" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "json" + ) + + # Should have printed JSON syntax + assert mock_console.print.called + # Check that Syntax object was printed (JSON formatting) + calls = mock_console.print.call_args_list + assert any( + isinstance(call[0][0], Syntax) for call in calls if call[0] + ) + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_with_errors(self): + """Test batch context processing handles errors gracefully.""" + contexts = [ + {"user": {"name": "Alice"}}, + {"user": {"age": 30}}, # Missing 'name' field + {"user": {"name": "Charlie"}}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_user{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # This expression will fail on the second context (no name field) + expression = "user.name" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should have printed results and errors + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_base_context_merge(self): + """Test that base context is merged with file contexts.""" + # Base context provides a default value + base_context = {"multiplier": 2} + + contexts = [ + {"value": 10}, + {"value": 20, "multiplier": 3}, # Override multiplier + {"value": 30}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_val{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + expression = "value * multiplier" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, base_context, "auto" + ) + + # Should have successfully evaluated all contexts + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_empty_list(self): + """Test batch context processing with empty file list.""" + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts("1 + 2", [], {}, "auto") + + # Should print warning about no context files + mock_console.print.assert_called_once() + assert "No context files" in mock_console.print.call_args[0][0] + + def test_evaluate_expression_with_multiple_contexts_file_not_found(self): + """Test batch context processing with non-existent file.""" + nonexistent_files = [ + Path("definitely_does_not_exist_1.json"), + Path("definitely_does_not_exist_2.json"), + ] + + with patch("cel.cli.console") as mock_console: + # Should handle file not found errors + evaluate_expression_with_multiple_contexts( + "1 + 2", nonexistent_files, {}, "auto" + ) + + # Should have printed error messages + assert mock_console.print.called + calls = [str(call) for call in mock_console.print.call_args_list] + assert any("Error" in call for call in calls) + + def test_evaluate_expression_with_multiple_contexts_timing(self): + """Test that batch context processing includes timing information.""" + contexts = [ + {"value": 10}, + {"value": 20}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_val{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + expression = "value * 2" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should have printed a table with timing column + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_invalid_json(self): + """Test batch context processing with invalid JSON files.""" + temp_files = [] + try: + # Create file with invalid JSON + with tempfile.NamedTemporaryFile( + mode="w", suffix="_invalid.json", delete=False + ) as f: + f.write("{ invalid json }") + temp_files.append(Path(f.name)) + + # Create file with valid JSON + with tempfile.NamedTemporaryFile( + mode="w", suffix="_valid.json", delete=False + ) as f: + json.dump({"value": 42}, f) + temp_files.append(Path(f.name)) + + expression = "value * 2" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should have printed error for invalid file + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_mixed_types(self): + """Test batch context processing with different data types across files.""" + contexts = [ + {"result": True}, + {"result": "success"}, + {"result": 42}, + {"result": [1, 2, 3]}, + {"result": {"nested": "value"}}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_type{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # Expression that works with any type + expression = "has(result)" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should handle all types successfully + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_large_dataset(self): + """Test batch context processing with a larger number of files.""" + # Create 20 context files + contexts = [{"id": i, "value": i * 10, "active": i % 2 == 0} for i in range(20)] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_data{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + expression = "active && value > 50" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should process all files + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_complex_expression(self): + """Test batch context processing with complex CEL expressions.""" + contexts = [ + {"user": {"age": 25, "verified": True, "role": "admin"}}, + {"user": {"age": 17, "verified": False, "role": "user"}}, + {"user": {"age": 30, "verified": True, "role": "moderator"}}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_user{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # Complex expression with multiple conditions + expression = 'user.age >= 18 && user.verified && user.role in ["admin", "moderator"]' + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should evaluate complex expression correctly + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_base_context_override(self): + """Test that file context properly overrides base context values.""" + base_context = {"default_value": 100, "multiplier": 2, "enabled": False} + + contexts = [ + {"value": 10}, # Uses base context default_value and multiplier + {"value": 20, "multiplier": 3}, # Overrides multiplier + {"value": 30, "enabled": True}, # Overrides enabled + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_override{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + expression = "value * multiplier" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, base_context, "auto" + ) + + # Should merge contexts correctly with file taking precedence + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_evaluate_expression_with_multiple_contexts_empty_files(self): + """Test batch context processing with empty JSON files.""" + temp_files = [] + try: + # Create empty object file + with tempfile.NamedTemporaryFile( + mode="w", suffix="_empty.json", delete=False + ) as f: + f.write("{}") + temp_files.append(Path(f.name)) + + # Create normal file + with tempfile.NamedTemporaryFile( + mode="w", suffix="_normal.json", delete=False + ) as f: + json.dump({"value": 42}, f) + temp_files.append(Path(f.name)) + + # Expression that uses 'has' to check for field + expression = "has(value)" + + with patch("cel.cli.console") as mock_console: + evaluate_expression_with_multiple_contexts( + expression, temp_files, {}, "auto" + ) + + # Should handle empty objects gracefully + assert mock_console.print.called + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + +class TestCLIE2EBasicFeatures: + """ + End-to-end tests for basic CLI features using subprocess. + + Tests core functionality like simple evaluation, context, output formats, etc. + """ + + def test_e2e_simple_expression(self): + """Test simple expression evaluation without context.""" + import subprocess + + cmd = ["cel", "1 + 2"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "3" in result.stdout + + def test_e2e_string_expression(self): + """Test string manipulation expression.""" + import subprocess + + cmd = ["cel", "'Hello ' + 'World'"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "Hello World" in result.stdout + + def test_e2e_inline_context(self): + """Test evaluation with inline JSON context.""" + import subprocess + + cmd = [ + "cel", + "age >= 18", + "--context", '{"age": 25}' + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "true" in result.stdout.lower() + + def test_e2e_inline_context_short_flag(self): + """Test evaluation with inline context using -c short flag.""" + import subprocess + + cmd = [ + "cel", + "name + ' is ' + string(age)", + "-c", '{"name": "Alice", "age": 30}' + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "Alice is 30" in result.stdout + + def test_e2e_context_file(self): + """Test evaluation with context from file.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"user": {"name": "Bob", "role": "admin"}}, f) + temp_file = Path(f.name) + + cmd = [ + "cel", + 'user.name + " (" + user.role + ")"', + "--context-file", str(temp_file) + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "Bob (admin)" in result.stdout + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_context_file_short_flag(self): + """Test evaluation with context file using -f short flag.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"value": 42}, f) + temp_file = Path(f.name) + + cmd = [ + "cel", + "value * 2", + "-f", str(temp_file) + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "84" in result.stdout + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_combined_contexts(self): + """Test that --context and --context-file can be combined.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"base": 10}, f) + temp_file = Path(f.name) + + cmd = [ + "cel", + "base * multiplier", + "--context", '{"multiplier": 5}', + "--context-file", str(temp_file) + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "50" in result.stdout + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_output_json(self): + """Test JSON output format.""" + import subprocess + + cmd = [ + "cel", + '{"name": "Alice", "age": 30}', + "--output", "json" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + output = json.loads(result.stdout) + assert output["name"] == "Alice" + assert output["age"] == 30 + + def test_e2e_output_json_short_flag(self): + """Test JSON output format with -o short flag.""" + import subprocess + + cmd = [ + "cel", + "[1, 2, 3]", + "-o", "json" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + output = json.loads(result.stdout) + assert output == [1, 2, 3] + + def test_e2e_output_pretty(self): + """Test pretty output format.""" + import subprocess + + cmd = [ + "cel", + '{"key": "value", "number": 42}', + "--output", "pretty" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + # Pretty format should use rich table + assert "key" in result.stdout + assert "value" in result.stdout + + def test_e2e_output_python(self): + """Test python output format.""" + import subprocess + + cmd = [ + "cel", + '{"test": true}', + "--output", "python" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + # Python repr format + assert "test" in result.stdout + assert "True" in result.stdout + + def test_e2e_timing_flag(self): + """Test that --timing flag shows timing information.""" + import subprocess + + cmd = [ + "cel", + "1 + 2", + "--timing" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "ms" in result.stdout.lower() + + def test_e2e_timing_short_flag(self): + """Test that -t flag shows timing information.""" + import subprocess + + cmd = [ + "cel", + "1 + 2", + "-t" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "ms" in result.stdout.lower() + + def test_e2e_verbose_flag(self): + """Test that --verbose flag shows additional information.""" + import subprocess + + cmd = [ + "cel", + "age * 2", + "--context", '{"age": 21}', + "--verbose" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + # Verbose should show timing, expression, result type, context vars + assert "ms" in result.stdout.lower() + assert "Expression" in result.stdout or "expression" in result.stdout.lower() + + def test_e2e_verbose_short_flag(self): + """Test that -v flag shows additional information.""" + import subprocess + + cmd = [ + "cel", + "1 + 2", + "-v" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "ms" in result.stdout.lower() + + def test_e2e_no_expression_error(self): + """Test that no expression gives helpful error.""" + import subprocess + + cmd = ["cel"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + assert "expression" in result.stdout.lower() or "help" in result.stdout.lower() + + def test_e2e_invalid_json_context_error(self): + """Test error handling for invalid JSON in --context.""" + import subprocess + + cmd = [ + "cel", + "age > 18", + "--context", "{invalid json}" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + assert "JSON" in result.stdout or "json" in result.stdout.lower() + + def test_e2e_missing_context_file_error(self): + """Test error handling for missing context file.""" + import subprocess + + cmd = [ + "cel", + "value > 0", + "--context-file", "/nonexistent/file.json" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + + def test_e2e_expression_evaluation_error(self): + """Test error handling for expression evaluation errors.""" + import subprocess + + cmd = [ + "cel", + "unknown_variable + 1" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + + def test_e2e_version_flag(self): + """Test --version flag shows version and exits.""" + import subprocess + + cmd = ["cel", "--version"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Version callback exits with code 0 + assert result.returncode == 0 + # Should show version number + assert "version" in result.stdout.lower() or any(c.isdigit() for c in result.stdout) + + def test_e2e_help_flag(self): + """Test --help flag shows help and exits.""" + import subprocess + + cmd = ["cel", "--help"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode == 0 + assert "Usage" in result.stdout or "usage" in result.stdout.lower() + assert "expression" in result.stdout.lower() + + def test_e2e_complex_nested_expression(self): + """Test complex expression with nested fields and operators.""" + import subprocess + + context = { + "user": { + "name": "Alice", + "age": 30, + "verified": True + }, + "permissions": ["read", "write", "delete"] + } + + cmd = [ + "cel", + 'user.verified && user.age >= 18 && "write" in permissions', + "--context", json.dumps(context) + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "true" in result.stdout.lower() + + def test_e2e_list_operations(self): + """Test list operations in expressions.""" + import subprocess + + cmd = [ + "cel", + "[1, 2, 3].size()", + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "3" in result.stdout + + def test_e2e_map_operations(self): + """Test map/dict operations in expressions.""" + import subprocess + + cmd = [ + "cel", + '{"a": 1, "b": 2}.size()', + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "2" in result.stdout + + def test_e2e_string_functions(self): + """Test string functions.""" + import subprocess + + cmd = [ + "cel", + '"hello".size()', + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "5" in result.stdout + + +class TestCLIE2EFileMode: + """ + End-to-end tests for --file mode (evaluating expressions from a file). + """ + + def test_e2e_file_mode_single_expression(self): + """Test evaluating single expression from file.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + f.write("1 + 2\n") + temp_file = Path(f.name) + + cmd = ["cel", "--file", str(temp_file)] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "3" in result.stdout + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_file_mode_multiple_expressions(self): + """Test evaluating multiple expressions from file.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + f.write("1 + 2\n") + f.write("'Hello ' + 'World'\n") + f.write("10 * 5\n") + temp_file = Path(f.name) + + cmd = ["cel", "--file", str(temp_file)] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "3" in result.stdout + assert "Hello World" in result.stdout + assert "50" in result.stdout + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_file_mode_with_context(self): + """Test file mode with context.""" + import subprocess + + expr_file = None + context_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + f.write("user.name\n") + f.write("user.age >= 18\n") + expr_file = Path(f.name) + + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"user": {"name": "Alice", "age": 25}}, f) + context_file = Path(f.name) + + cmd = [ + "cel", + "--file", str(expr_file), + "--context-file", str(context_file) + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + assert "Alice" in result.stdout + assert "true" in result.stdout.lower() + + finally: + if expr_file and expr_file.exists(): + expr_file.unlink() + if context_file and context_file.exists(): + context_file.unlink() + + def test_e2e_file_mode_with_json_output(self): + """Test file mode with JSON output format.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + f.write("1 + 2\n") + f.write("5 * 3\n") + temp_file = Path(f.name) + + cmd = [ + "cel", + "--file", str(temp_file), + "--output", "json" + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + assert result.returncode == 0 + # Should be valid JSON array or object + output = json.loads(result.stdout) + assert output is not None + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_file_mode_empty_file(self): + """Test file mode with empty file.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + # Empty file + pass + temp_file = Path(f.name) + + cmd = ["cel", "--file", str(temp_file)] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Should handle gracefully (might succeed with no output or show message) + # Just verify it doesn't crash + assert result.returncode in [0, 1] + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_file_mode_with_comments(self): + """Test file mode handles lines with # comments or empty lines.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".cel", delete=False + ) as f: + f.write("# This is a comment\n") + f.write("1 + 2\n") + f.write("\n") # Empty line + f.write("5 * 3\n") + temp_file = Path(f.name) + + cmd = ["cel", "--file", str(temp_file)] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # May or may not support comments - just verify it doesn't crash + assert result.returncode in [0, 1] + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_file_mode_nonexistent_file(self): + """Test file mode with non-existent file.""" + import subprocess + + cmd = ["cel", "--file", "/nonexistent/expressions.cel"] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + + +class TestBatchContextProcessingE2E: + """ + End-to-end tests for batch context processing using subprocess. + + These tests actually invoke the `cel` CLI command to verify the full user experience. + """ + + def test_e2e_basic_batch_processing(self): + """Test basic batch processing with actual CLI invocation.""" + import subprocess + + contexts = [ + {"user": {"name": "Alice", "age": 25}}, + {"user": {"name": "Bob", "age": 30}}, + {"user": {"name": "Charlie", "age": 17}}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_user{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # Build command with repeated --for-each flags + cmd = [ + "cel", + "user.age >= 18", + "--for-each", str(temp_files[0]), + "--for-each", str(temp_files[1]), + "--for-each", str(temp_files[2]), + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + # Verify output contains results for all files + assert result.returncode == 0 + assert "Expression Results" in result.stdout + assert str(temp_files[0].name) in result.stdout or "user0.json" in result.stdout + assert str(temp_files[1].name) in result.stdout or "user1.json" in result.stdout + assert str(temp_files[2].name) in result.stdout or "user2.json" in result.stdout + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_json_output(self): + """Test batch processing with JSON output format.""" + import subprocess + + contexts = [ + {"value": 10}, + {"value": 20}, + {"value": 30}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_data{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + cmd = [ + "cel", + "value * 2", + "--for-each", str(temp_files[0]), + "--for-each", str(temp_files[1]), + "--for-each", str(temp_files[2]), + "--output", "json", + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + # Verify JSON output + assert result.returncode == 0 + output = json.loads(result.stdout) + + assert isinstance(output, list) + assert len(output) == 3 + + # Check results + assert output[0]["result"] == 20 + assert output[1]["result"] == 40 + assert output[2]["result"] == 60 + + # Check that timing info is present + assert "time_ms" in output[0] + assert "context_file" in output[0] + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_with_base_context(self): + """Test batch processing with base context merging.""" + import subprocess + + contexts = [ + {"value": 5}, + {"value": 10}, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_data{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + cmd = [ + "cel", + "value * multiplier", + "--context", '{"multiplier": 3}', + "--for-each", str(temp_files[0]), + "--for-each", str(temp_files[1]), + "--output", "json", + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + # Verify results + assert result.returncode == 0 + output = json.loads(result.stdout) + + assert len(output) == 2 + assert output[0]["result"] == 15 # 5 * 3 + assert output[1]["result"] == 30 # 10 * 3 + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_file_not_found(self): + """Test batch processing with non-existent file.""" + import subprocess + + cmd = [ + "cel", + "user.age >= 18", + "--for-each", "/nonexistent/file1.json", + "--for-each", "/nonexistent/file2.json", + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Should not crash, but show errors + assert "Error" in result.stdout or "error" in result.stdout.lower() + # May or may not exit with error code depending on implementation + # Just verify it doesn't crash + + def test_e2e_batch_processing_invalid_json(self): + """Test batch processing with invalid JSON file.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + f.write("{invalid json content") + temp_file = Path(f.name) + + cmd = [ + "cel", + "user.age >= 18", + "--for-each", str(temp_file), + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Should show error about invalid JSON + assert "Error" in result.stdout or "error" in result.stdout.lower() + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_no_expression(self): + """Test that --for-each requires an expression.""" + import subprocess + + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"test": "data"}, f) + temp_file = Path(f.name) + + cmd = [ + "cel", + "--for-each", str(temp_file), + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Should exit with error + assert result.returncode != 0 + assert "Error" in result.stdout or "error" in result.stdout.lower() + # Should mention that expression is required + assert "expression" in result.stdout.lower() or "requires" in result.stdout.lower() + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_complex_expression(self): + """Test batch processing with complex CEL expression.""" + import subprocess + + contexts = [ + { + "user": {"name": "Alice", "age": 25, "verified": True}, + "permissions": ["read", "write"], + }, + { + "user": {"name": "Bob", "age": 30, "verified": False}, + "permissions": ["read"], + }, + { + "user": {"name": "Charlie", "age": 22, "verified": True}, + "permissions": ["read", "write", "admin"], + }, + ] + + temp_files = [] + try: + for i, context in enumerate(contexts): + with tempfile.NamedTemporaryFile( + mode="w", suffix=f"_user{i}.json", delete=False + ) as f: + json.dump(context, f) + temp_files.append(Path(f.name)) + + # Complex expression with multiple conditions + expression = 'user.verified && user.age >= 21 && "write" in permissions' + + cmd = [ + "cel", + expression, + "--for-each", str(temp_files[0]), + "--for-each", str(temp_files[1]), + "--for-each", str(temp_files[2]), + "--output", "json", + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + # Verify results + assert result.returncode == 0 + output = json.loads(result.stdout) + + assert len(output) == 3 + assert output[0]["result"] is True # Alice: verified, age 25, has write + assert output[1]["result"] is False # Bob: not verified + assert output[2]["result"] is True # Charlie: verified, age 22, has write + + finally: + for temp_file in temp_files: + if temp_file.exists(): + temp_file.unlink() + + def test_e2e_batch_processing_with_shell_expansion(self): + """Test that batch processing works with shell glob patterns via xargs pattern.""" + import subprocess + import tempfile + import os + + # Create a temporary directory with multiple JSON files + temp_dir = tempfile.mkdtemp() + try: + # Create test files + for i in range(3): + file_path = os.path.join(temp_dir, f"data{i}.json") + with open(file_path, "w") as f: + json.dump({"value": i * 10}, f) + + # Simulate what a shell glob would do: build the command with all files + files = sorted([ + os.path.join(temp_dir, f) + for f in os.listdir(temp_dir) + if f.endswith(".json") + ]) + + cmd = ["cel", "value >= 10"] + for file in files: + cmd.extend(["--for-each", file]) + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + check=True, + ) + + # Verify output + assert result.returncode == 0 + # Check that all files were processed + for i in range(3): + assert f"data{i}.json" in result.stdout + + finally: + # Cleanup + import shutil + shutil.rmtree(temp_dir) + + def test_e2e_exit_status_verification(self): + """Test that exit status reflects evaluation success.""" + import subprocess + + # Create a file with valid context + temp_file = None + try: + with tempfile.NamedTemporaryFile( + mode="w", suffix=".json", delete=False + ) as f: + json.dump({"value": 42}, f) + temp_file = Path(f.name) + + # Test successful evaluation + cmd = [ + "cel", + "value == 42", + "--for-each", str(temp_file), + ] + + result = subprocess.run( + cmd, + capture_output=True, + text=True, + ) + + # Should exit successfully + assert result.returncode == 0 + + finally: + if temp_file and temp_file.exists(): + temp_file.unlink() + + if __name__ == "__main__": pytest.main([__file__]) diff --git a/tests/test_expression_storage.py b/tests/test_expression_storage.py new file mode 100644 index 0000000..c2146ce --- /dev/null +++ b/tests/test_expression_storage.py @@ -0,0 +1,299 @@ +""" +Tests for expression storage functionality. + +Tests the OS-specific configuration directory and user expression persistence. +""" + +import json +import tempfile +from pathlib import Path +from unittest.mock import patch + +import pytest + +from cel.expression_storage import ( + add_expression, + delete_expression, + get_config_dir, + get_expressions_file, + load_user_expressions, + save_user_expressions, + update_expression, +) + + +class TestConfigDir: + """Test configuration directory location.""" + + def test_get_config_dir_creates_directory(self): + """Test that get_config_dir creates the directory if it doesn't exist.""" + config_dir = get_config_dir() + assert config_dir.exists() + assert config_dir.is_dir() + + def test_config_dir_in_home(self): + """Test that config dir is within user's home directory.""" + config_dir = get_config_dir() + home = Path.home() + assert str(config_dir).startswith(str(home)) + + def test_expressions_file_path(self): + """Test that expressions file path is correct.""" + expressions_file = get_expressions_file() + assert expressions_file.name == "expressions.json" + assert expressions_file.parent == get_config_dir() + + +class TestSaveLoad: + """Test saving and loading expressions.""" + + def test_save_and_load_empty_list(self, tmp_path): + """Test saving and loading an empty expression list.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Save empty list + save_user_expressions([]) + + # Load and verify + expressions = load_user_expressions() + assert expressions == [] + + def test_save_and_load_single_expression(self, tmp_path): + """Test saving and loading a single expression.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Save expression + test_expr = ("Test Expr", "A test expression", "1 + 1") + save_user_expressions([test_expr]) + + # Load and verify + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0] == test_expr + + def test_save_and_load_multiple_expressions(self, tmp_path): + """Test saving and loading multiple expressions.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Save expressions + test_exprs = [ + ("Expr 1", "First expression", "1 + 1"), + ("Expr 2", "Second expression", "2 * 3"), + ("Expr 3", "Third expression", '"hello"'), + ] + save_user_expressions(test_exprs) + + # Load and verify + expressions = load_user_expressions() + assert len(expressions) == 3 + assert expressions == test_exprs + + def test_load_nonexistent_file(self, tmp_path): + """Test loading from non-existent file returns empty list.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "nonexistent.json" + mock_file.return_value = test_file + + expressions = load_user_expressions() + assert expressions == [] + + def test_load_invalid_json(self, tmp_path): + """Test loading invalid JSON returns empty list.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "invalid.json" + mock_file.return_value = test_file + + # Write invalid JSON + test_file.write_text("{ invalid json }") + + expressions = load_user_expressions() + assert expressions == [] + + def test_load_wrong_format(self, tmp_path): + """Test loading wrong format returns empty list.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "wrong_format.json" + mock_file.return_value = test_file + + # Write wrong format (not a list) + test_file.write_text('{"key": "value"}') + + expressions = load_user_expressions() + assert expressions == [] + + def test_save_preserves_json_structure(self, tmp_path): + """Test that saved JSON has correct structure.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Save expression + test_expr = ("Test", "Description", "expression") + save_user_expressions([test_expr]) + + # Read and verify JSON structure + with test_file.open() as f: + data = json.load(f) + + assert isinstance(data, list) + assert len(data) == 1 + assert data[0]["name"] == "Test" + assert data[0]["description"] == "Description" + assert data[0]["expression"] == "expression" + + +class TestAddExpression: + """Test adding expressions.""" + + def test_add_expression_to_empty_file(self, tmp_path): + """Test adding first expression to empty file.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + add_expression("Test", "A test", "1 + 1") + + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0] == ("Test", "A test", "1 + 1") + + def test_add_multiple_expressions(self, tmp_path): + """Test adding multiple expressions sequentially.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + add_expression("First", "First expr", "1") + add_expression("Second", "Second expr", "2") + add_expression("Third", "Third expr", "3") + + expressions = load_user_expressions() + assert len(expressions) == 3 + + def test_add_duplicate_name_raises_error(self, tmp_path): + """Test that adding duplicate name raises ValueError.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + add_expression("Test", "First", "1") + + with pytest.raises(ValueError, match="already exists"): + add_expression("Test", "Duplicate", "2") + + +class TestDeleteExpression: + """Test deleting expressions.""" + + def test_delete_existing_expression(self, tmp_path): + """Test deleting an existing expression.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Add expression + add_expression("Test", "A test", "1 + 1") + assert len(load_user_expressions()) == 1 + + # Delete it + result = delete_expression("Test") + assert result is True + assert len(load_user_expressions()) == 0 + + def test_delete_nonexistent_expression(self, tmp_path): + """Test deleting non-existent expression returns False.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + result = delete_expression("Nonexistent") + assert result is False + + def test_delete_one_of_many(self, tmp_path): + """Test deleting one expression from many.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Add multiple + add_expression("First", "A", "1") + add_expression("Second", "B", "2") + add_expression("Third", "C", "3") + + # Delete middle one + result = delete_expression("Second") + assert result is True + + expressions = load_user_expressions() + assert len(expressions) == 2 + assert expressions[0][0] == "First" + assert expressions[1][0] == "Third" + + +class TestUpdateExpression: + """Test updating expressions.""" + + def test_update_expression_name_only(self, tmp_path): + """Test updating just the name of an expression.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Add expression + add_expression("Old Name", "Description", "expression") + + # Update name + update_expression("Old Name", "New Name", "Description", "expression") + + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0][0] == "New Name" + + def test_update_all_fields(self, tmp_path): + """Test updating all fields of an expression.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Add expression + add_expression("Name", "Old desc", "old expr") + + # Update all fields + update_expression("Name", "New Name", "New desc", "new expr") + + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0] == ("New Name", "New desc", "new expr") + + def test_update_nonexistent_raises_error(self, tmp_path): + """Test updating non-existent expression raises ValueError.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + with pytest.raises(ValueError, match="not found"): + update_expression("Nonexistent", "New", "desc", "expr") + + def test_update_to_duplicate_name_raises_error(self, tmp_path): + """Test updating to a duplicate name raises ValueError.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + + # Add two expressions + add_expression("First", "A", "1") + add_expression("Second", "B", "2") + + # Try to rename Second to First + with pytest.raises(ValueError, match="already exists"): + update_expression("Second", "First", "B", "2") + + +if __name__ == "__main__": + pytest.main([__file__, "-v"]) diff --git a/tests/test_tui.py b/tests/test_tui.py new file mode 100644 index 0000000..05eff1c --- /dev/null +++ b/tests/test_tui.py @@ -0,0 +1,677 @@ +""" +Comprehensive End-to-End tests for the CEL TUI. + +Tests all functionality using Textual's testing framework: +- Expression library loading and filtering +- Context loading from JSON/YAML files and URLs +- Expression evaluation +- Results display +- Keyboard shortcuts +- Error handling +""" + +import json +import tempfile +from pathlib import Path +from unittest.mock import Mock, patch + +import pytest +import yaml +from textual.widgets import Button, Input, Label, Static, TextArea + +from cel.tui import CELTuiApp, ExpressionLibrary + + +class TestExpressionLibrary: + """Test the Expression Library panel functionality.""" + + @pytest.mark.asyncio + async def test_library_displays_default_expressions(self): + """Test that default expressions are displayed on mount.""" + app = CELTuiApp() + async with app.run_test() as pilot: + # Check that expression library exists + library = app.query_one(ExpressionLibrary) + assert library is not None + + # Check that expression cards are rendered + cards = library.query(".expr-card") + assert len(cards) == 8 # Should have 8 default expressions + + # Check first expression contains expected text + first_card = cards[0] + labels = first_card.query(Label) + label_texts = [str(label.render()) for label in labels] + combined_text = " ".join(label_texts) + assert "Age Check" in combined_text + + @pytest.mark.asyncio + async def test_library_search_filtering(self): + """Test that search input filters expressions.""" + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + search_input = library.query_one("#search-input", Input) + + # Initially all 8 expressions should be visible + cards = library.query(".expr-card") + assert len(cards) == 8 + + # Type in search box + search_input.value = "age" + await pilot.pause() + + # Should filter to only expressions containing "age" + cards = library.query(".expr-card") + assert len(cards) < 8 # Should have fewer expressions + assert len(cards) > 0 # But at least one + + @pytest.mark.asyncio + async def test_library_search_no_results(self): + """Test search with no matching expressions.""" + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + search_input = library.query_one("#search-input", Input) + + # Search for something that doesn't exist + search_input.value = "zzzznonexistent" + await pilot.pause() + + # Should have no expression cards + cards = library.query(".expr-card") + assert len(cards) == 0 + + +class TestContextEditor: + """Test the Context Editor panel functionality.""" + + @pytest.mark.asyncio + async def test_context_editor_has_default_context(self): + """Test that context editor starts with default JSON context.""" + app = CELTuiApp() + async with app.run_test() as pilot: + context_area = app.query_one("#context-input", TextArea) + + # Should have default context + assert context_area.text != "" + + # Should be valid JSON + context_data = json.loads(context_area.text) + assert "user" in context_data + assert "request" in context_data + + @pytest.mark.asyncio + async def test_context_validation_updates(self): + """Test that context validation updates when JSON is edited.""" + app = CELTuiApp() + async with app.run_test() as pilot: + context_area = app.query_one("#context-input", TextArea) + validation = app.query_one("#validation-status", Label) + + # Start with valid JSON - should show valid + app._load_context_from_editor() + await pilot.pause() + validation_text = validation.render() + assert "✓" in validation_text + + # Edit to invalid JSON + context_area.clear() + context_area.insert("{ invalid json") + app._load_context_from_editor() + await pilot.pause() + + # Should show invalid + validation_text = validation.render() + assert "✗" in validation_text + + @pytest.mark.asyncio + async def test_load_context_from_json_file(self): + """Test loading context from a JSON file.""" + app = CELTuiApp() + + # Create temporary JSON file + test_context = { + "name": "TestUser", + "age": 25, + "roles": ["user", "tester"] + } + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump(test_context, f) + temp_file = Path(f.name) + + try: + async with app.run_test() as pilot: + file_input = app.query_one("#file-input", Input) + context_area = app.query_one("#context-input", TextArea) + + # Set file path + file_input.value = str(temp_file) + await pilot.pause() + + # Trigger upload handler directly + app.handle_upload() + await pilot.pause() + + # Context should be loaded + loaded_data = json.loads(context_area.text) + assert loaded_data["name"] == "TestUser" + assert loaded_data["age"] == 25 + assert "tester" in loaded_data["roles"] + finally: + temp_file.unlink() + + @pytest.mark.asyncio + async def test_load_context_from_yaml_file(self): + """Test loading context from a YAML file.""" + app = CELTuiApp() + + # Create temporary YAML file + test_context = { + "server": "production", + "port": 8080, + "features": ["auth", "api"] + } + with tempfile.NamedTemporaryFile(mode="w", suffix=".yaml", delete=False) as f: + yaml.dump(test_context, f) + temp_file = Path(f.name) + + try: + async with app.run_test() as pilot: + file_input = app.query_one("#file-input", Input) + context_area = app.query_one("#context-input", TextArea) + + # Set file path + file_input.value = str(temp_file) + await pilot.pause() + + # Trigger upload handler directly + app.handle_upload() + await pilot.pause() + + # Context should be loaded and converted to JSON + loaded_data = json.loads(context_area.text) + assert loaded_data["server"] == "production" + assert loaded_data["port"] == 8080 + assert "api" in loaded_data["features"] + finally: + temp_file.unlink() + + @pytest.mark.asyncio + async def test_load_context_from_url(self): + """Test loading context from URL with mocked HTTP response.""" + app = CELTuiApp() + + test_context = {"api_key": "test123", "endpoint": "https://api.example.com"} + + async with app.run_test() as pilot: + url_input = app.query_one("#url-input", Input) + context_area = app.query_one("#context-input", TextArea) + + # Mock urlopen + with patch("cel.tui.urlopen") as mock_urlopen: + mock_response = Mock() + mock_response.read.return_value = json.dumps(test_context).encode('utf-8') + mock_response.__enter__ = Mock(return_value=mock_response) + mock_response.__exit__ = Mock(return_value=False) + mock_urlopen.return_value = mock_response + + # Set URL + url_input.value = "https://example.com/context.json" + await pilot.pause() + + # Trigger URL load handler directly + app.handle_load_url() + await pilot.pause() + + # Context should be loaded + loaded_data = json.loads(context_area.text) + assert loaded_data["api_key"] == "test123" + + +class TestEvaluation: + """Test expression evaluation functionality.""" + + @pytest.mark.asyncio + async def test_evaluate_simple_expression(self): + """Test evaluating a simple expression.""" + app = CELTuiApp() + async with app.run_test() as pilot: + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + + # Set expression + expr_area.clear() + expr_area.insert("1 + 2") + await pilot.pause() + + # Trigger evaluation + app.action_evaluate() + await pilot.pause() + + # Result should show "3" + result_text = str(result_display.render()) + assert "3" in result_text + + @pytest.mark.asyncio + async def test_evaluate_with_context(self): + """Test evaluating expression with context variables.""" + app = CELTuiApp() + async with app.run_test() as pilot: + context_area = app.query_one("#context-input", TextArea) + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + + # Set context + context_area.clear() + context_area.insert('{"x": 10, "y": 5}') + await pilot.pause() + app._load_context_from_editor() + await pilot.pause() + + # Set expression + expr_area.clear() + expr_area.insert("x * y") + await pilot.pause() + + # Trigger evaluation + app.action_evaluate() + await pilot.pause() + + # Result should show "50" + result_text = str(result_display.render()) + assert "50" in result_text + + @pytest.mark.asyncio + async def test_evaluate_boolean_expression(self): + """Test evaluating boolean expression with default context.""" + app = CELTuiApp() + async with app.run_test() as pilot: + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + result_meta = app.query_one("#result-meta", Label) + + # Use default context which has user.age = 30 + expr_area.clear() + expr_area.insert("user.age >= 18") + await pilot.pause() + + # Trigger evaluation + app.action_evaluate() + await pilot.pause() + + # Result should show "true" or "True" + result_text = str(result_display.render()).lower() + assert "true" in result_text + + # Metadata should show type and timing + meta_text = str(result_meta.render()).lower() + assert "type:" in meta_text or "ms" in meta_text + + @pytest.mark.asyncio + async def test_evaluate_error_handling(self): + """Test error display when expression is invalid.""" + app = CELTuiApp() + async with app.run_test() as pilot: + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + + # Set invalid expression + expr_area.clear() + expr_area.insert("invalid syntax (((") + await pilot.pause() + + # Trigger evaluation + app.action_evaluate() + await pilot.pause() + + # Result should show error + result_text = str(result_display.render()).lower() + assert "error" in result_text + + @pytest.mark.asyncio + async def test_evaluate_with_stdlib_functions(self): + """Test that stdlib functions are available during evaluation.""" + app = CELTuiApp() + async with app.run_test() as pilot: + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + + # Use substring function from stdlib + expr_area.clear() + expr_area.insert('substring("hello world", 0, 5)') + await pilot.pause() + + # Trigger evaluation + app.action_evaluate() + await pilot.pause() + + # Result should contain "hello" + result_text = str(result_display.render()) + assert "hello" in result_text + + +class TestKeyboardShortcuts: + """Test keyboard shortcuts functionality.""" + + @pytest.mark.asyncio + async def test_ctrl_e_evaluates_expression(self): + """Test that Ctrl+E triggers evaluation.""" + app = CELTuiApp() + async with app.run_test() as pilot: + expr_area = app.query_one("#expression-input", TextArea) + result_display = app.query_one("#result-display", Static) + + # Set expression + expr_area.clear() + expr_area.insert("2 + 3") + await pilot.pause() + + # Press Ctrl+E + await pilot.press("ctrl+e") + await pilot.pause() + + # Result should show "5" + result_text = str(result_display.render()) + assert "5" in result_text + + @pytest.mark.asyncio + async def test_f1_shows_help(self): + """Test that F1 action exists and is callable.""" + app = CELTuiApp() + async with app.run_test() as pilot: + # Verify the action exists + assert hasattr(app, "action_show_help") + assert callable(app.action_show_help) + + +class TestUIIntegration: + """Test overall UI integration and layout.""" + + @pytest.mark.asyncio + async def test_app_has_three_columns(self): + """Test that app displays all three main panels.""" + app = CELTuiApp() + async with app.run_test() as pilot: + # Check that all three panels exist + library = app.query_one(".expression-library") + context = app.query_one(".context-editor") + results = app.query_one(".results-panel") + + assert library is not None + assert context is not None + assert results is not None + + @pytest.mark.asyncio + async def test_app_has_header_and_footer(self): + """Test that header and footer are displayed.""" + app = CELTuiApp() + async with app.run_test() as pilot: + from textual.widgets import Footer, Header + + header = app.query_one(Header) + footer = app.query_one(Footer) + + assert header is not None + assert footer is not None + + @pytest.mark.asyncio + async def test_all_required_widgets_present(self): + """Test that all required widgets are present in the UI.""" + app = CELTuiApp() + async with app.run_test() as pilot: + # Expression library widgets + assert app.query_one("#search-input") is not None + assert app.query_one("#expression-list") is not None + + # Context editor widgets + assert app.query_one("#url-input") is not None + assert app.query_one("#file-input") is not None + assert app.query_one("#context-input") is not None + assert app.query_one("#validation-status") is not None + assert app.query_one("#load-url-btn") is not None + assert app.query_one("#upload-btn") is not None + + # Results panel widgets + assert app.query_one("#expression-input") is not None + assert app.query_one("#eval-btn") is not None + assert app.query_one("#result-display") is not None + assert app.query_one("#result-meta") is not None + + @pytest.mark.asyncio + async def test_app_initialization(self): + """Test that app initializes with correct default state.""" + app = CELTuiApp() + async with app.run_test() as pilot: + # Should have context initialized + assert app.context is not None + assert app.context_dict is not None + + # Should have title and subtitle + assert app.TITLE == "CEL Expression Evaluator" + assert "Common Expression Language" in app.SUB_TITLE + + +class TestExpressionManagement: + """Test expression save and delete functionality.""" + + @pytest.mark.asyncio + async def test_save_expression_button(self, tmp_path): + """Test saving expression via Save button.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file, \ + patch('cel.tui.get_expressions_file') as mock_file2: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + mock_file2.return_value = test_file + + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + expr_area = app.query_one("#expression-input", TextArea) + + # Enter a custom expression + expr_area.clear() + expr_area.insert("user.age * 2") + await pilot.pause() + + # Click save button - call handler directly + library.handle_save_expression() + await pilot.pause() + + # Verify expression was saved + from cel.expression_storage import load_user_expressions + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0][2] == "user.age * 2" + + # Reload library to see the changes + library.load_expressions() + library._update_list() + await pilot.pause() + + # Verify library updated + user_cards = library.query(".user-expr-card") + assert len(user_cards) == 1 + + @pytest.mark.asyncio + async def test_delete_expression_button(self, tmp_path): + """Test deleting expression via delete button.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file, \ + patch('cel.tui.get_expressions_file') as mock_file2: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + mock_file2.return_value = test_file + + # Pre-populate with test expressions + from cel.expression_storage import add_expression + add_expression("TestExpr1", "First test", "1 + 1") + add_expression("TestExpr2", "Second test", "2 + 2") + + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + library.load_expressions() + library._update_list() + await pilot.pause() + + # Verify we have 2 user expressions + user_cards = library.query(".user-expr-card") + assert len(user_cards) == 2 + + # Delete first expression by calling handler directly + # Create a mock event with the delete button + from unittest.mock import Mock + delete_btn = library.query_one("#delete-0", Button) + mock_event = Mock() + mock_event.button = delete_btn + library.handle_delete_expression(mock_event) + await pilot.pause() + + # Verify expression was deleted + from cel.expression_storage import load_user_expressions + expressions = load_user_expressions() + assert len(expressions) == 1 + assert expressions[0][0] == "TestExpr2" + + # Verify library updated + user_cards = library.query(".user-expr-card") + assert len(user_cards) == 1 + + @pytest.mark.asyncio + async def test_save_empty_expression_shows_warning(self): + """Test that saving empty expression shows warning.""" + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + expr_area = app.query_one("#expression-input", TextArea) + + # Clear expression area + expr_area.clear() + await pilot.pause() + + # Try to save + save_btn = library.query_one("#save-expr-btn", Button) + await pilot.click(save_btn) + await pilot.pause() + + # Should show warning notification (we can't easily verify notification, + # but we can verify no expression was created) + # Since no file was configured, this would fail if it tried to save + + @pytest.mark.asyncio + async def test_click_user_expression_loads_it(self, tmp_path): + """Test clicking user expression loads it into editor.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file, \ + patch('cel.tui.get_expressions_file') as mock_file2: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + mock_file2.return_value = test_file + + # Pre-populate with test expression + from cel.expression_storage import add_expression + add_expression("MyExpr", "Test expression", "42 * 2") + + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + library.load_expressions() + library._update_list() + await pilot.pause() + + expr_area = app.query_one("#expression-input", TextArea) + + # Click on the user expression card + user_card = library.query_one(".user-expr-card") + await pilot.click(user_card) + await pilot.pause() + + # Verify expression was loaded + assert "42 * 2" in expr_area.text + + @pytest.mark.asyncio + async def test_saved_count_updates(self, tmp_path): + """Test that saved count label updates correctly.""" + with patch('cel.expression_storage.get_expressions_file') as mock_file, \ + patch('cel.tui.get_expressions_file') as mock_file2: + test_file = tmp_path / "test_expressions.json" + mock_file.return_value = test_file + mock_file2.return_value = test_file + + app = CELTuiApp() + async with app.run_test() as pilot: + library = app.query_one(ExpressionLibrary) + expr_area = app.query_one("#expression-input", TextArea) + + # Initial count should be 0 + saved_count = library.query_one("#saved-count", Label) + assert "(0 saved)" in str(saved_count.render()) + + # Save an expression + expr_area.clear() + expr_area.insert("test.expression") + await pilot.pause() + + # Call save handler directly + library.handle_save_expression() + await pilot.pause() + + # Reload and update display + library.load_expressions() + library._update_list() + await pilot.pause() + + # Count should update to 1 + saved_count = library.query_one("#saved-count", Label) + assert "(1 saved)" in str(saved_count.render()) + + +class TestCompleteWorkflow: + """Test complete end-to-end workflows.""" + + @pytest.mark.asyncio + async def test_complete_evaluation_workflow(self): + """Test a complete workflow: load context, set expression, evaluate.""" + app = CELTuiApp() + + # Create test context file + test_context = { + "user": { + "name": "Bob", + "age": 25, + "roles": ["admin"] + } + } + with tempfile.NamedTemporaryFile(mode="w", suffix=".json", delete=False) as f: + json.dump(test_context, f) + temp_file = Path(f.name) + + try: + async with app.run_test() as pilot: + # Step 1: Load context from file + file_input = app.query_one("#file-input", Input) + file_input.value = str(temp_file) + await pilot.pause() + app.handle_upload() + await pilot.pause() + + # Step 2: Enter expression + expr_area = app.query_one("#expression-input", TextArea) + expr_area.clear() + expr_area.insert('user.age >= 18') + await pilot.pause() + + # Step 3: Evaluate + app.action_evaluate() + await pilot.pause() + + # Step 4: Verify result + result_display = app.query_one("#result-display", Static) + result_text = str(result_display.render()).lower() + assert "true" in result_text + + finally: + temp_file.unlink() + + +if __name__ == "__main__": + pytest.main([__file__, "-v"])