diff --git a/docs/DataGrid Formatting System.md b/docs/DataGrid Formatting System.md new file mode 100644 index 0000000..07ec5a8 --- /dev/null +++ b/docs/DataGrid Formatting System.md @@ -0,0 +1,1437 @@ +# DataGrid Formatting System + +## Introduction + +The DataGrid Formatting System provides a comprehensive solution for applying conditional formatting and data transformation to DataGrid cells. It combines a powerful formatting engine with a user-friendly Domain Specific Language (DSL) and intelligent autocompletion. + +**Key features:** + +- Conditional formatting based on cell values and cross-column comparisons +- Rich styling with DaisyUI 5 theme presets +- Multiple data formatters (currency, dates, percentages, text transformations, enums) +- Text-based DSL for defining formatting rules +- Context-aware autocompletion with CodeMirror integration +- Five scope levels for precise targeting (cell, row, column, table, global) + +**Common use cases:** + +- Highlight negative amounts in red +- Format currency values with proper separators +- Apply different styles based on status columns +- Compare actual vs budget values +- Display dates in localized formats +- Transform enum values to readable labels + +--- + +## Architecture Overview + +### System Layers + +The formatting system is built in three layers, each with a specific responsibility: + +``` +┌─────────────────────────────────────────────────────────┐ +│ User Interface │ +│ DataGridFormattingEditor (DSL) │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ Layer 3: Autocompletion │ +│ FormattingCompletionEngine + Provider │ +│ (context detection, dynamic suggestions) │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ Layer 2: DSL Parser │ +│ Lark Grammar → Transformer → ScopedRule │ +│ (text → structured format rules) │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ Layer 1: Formatting Engine │ +│ FormattingEngine (ConditionEvaluator + │ +│ StyleResolver + FormatterResolver) │ +│ (apply rules → CSS + formatted value) │ +└─────────────────────────────────────────────────────────┘ +``` + +| Layer | Module | Responsibility | +|-------|--------|----------------| +| **1. Formatting Engine** | `core/formatting/` | Apply format rules to cell values, resolve conflicts | +| **2. DSL Parser** | `core/formatting/dsl/` | Parse text DSL into structured rules | +| **3. Autocompletion** | `core/formatting/dsl/completion/` | Provide intelligent suggestions while editing | +| **Infrastructure** | `core/dsl/` | Generic DSL framework (reusable) | + +### End-to-End Flow + +``` +User writes DSL in editor + │ + ▼ +┌────────────────────┐ +│ Lark Parser │ Parse text with indentation support +└────────────────────┘ + │ + ▼ +┌────────────────────┐ +│ DSLTransformer │ Convert AST → ScopedRule objects +└────────────────────┘ + │ + ▼ +┌────────────────────┐ +│ DataGrid │ Dispatch rules to column/row/cell formats +│ FormattingEditor │ +└────────────────────┘ + │ + ▼ +┌────────────────────┐ +│ FormattingEngine │ Apply rules to each cell +└────────────────────┘ + │ + ▼ + CSS + Formatted Value → Rendered Cell +``` + +### Module Responsibilities + +| Module | Location | Purpose | +|--------|----------|---------| +| `FormattingEngine` | `core/formatting/engine.py` | Main facade for applying format rules | +| `ConditionEvaluator` | `core/formatting/condition_evaluator.py` | Evaluate conditions (==, <, contains, etc.) | +| `StyleResolver` | `core/formatting/style_resolver.py` | Resolve styles to CSS strings | +| `FormatterResolver` | `core/formatting/formatter_resolver.py` | Format values for display | +| `parse_dsl()` | `core/formatting/dsl/__init__.py` | Public API for parsing DSL | +| `DSLTransformer` | `core/formatting/dsl/transformer.py` | Convert Lark AST to dataclasses | +| `FormattingCompletionEngine` | `core/formatting/dsl/completion/` | Autocompletion for DSL | +| `DataGridFormattingEditor` | `controls/DataGridFormattingEditor.py` | DSL editor integrated in DataGrid | + +--- + +## Fundamental Concepts + +### FormatRule + +A `FormatRule` is the core data structure combining three optional components: + +```python +@dataclass +class FormatRule: + condition: Condition = None # When to apply (optional) + style: Style = None # Visual formatting (optional) + formatter: Formatter = None # Value transformation (optional) +``` + +**Rules:** + +- At least one of `style` or `formatter` must be present +- `condition` cannot appear alone +- If `condition` is present, the rule applies only when the condition is met +- Multiple rules can be defined; they are evaluated in order + +**Conflict resolution:** + +When multiple rules match the same cell: + +1. **Specificity** = 1 if rule has condition, 0 otherwise +2. **Higher specificity wins** +3. **At equal specificity, last rule wins entirely** (no property merging) + +**Example:** + +```python +# Rule 1: Unconditional (specificity = 0) +FormatRule(style=Style(color="gray")) + +# Rule 2: Conditional (specificity = 1) +FormatRule( + condition=Condition(operator="<", value=0), + style=Style(color="red") +) + +# Rule 3: Conditional (specificity = 1) +FormatRule( + condition=Condition(operator="==", value=-5), + style=Style(color="black") +) + +# For value = -5: Rule 3 wins (same specificity as Rule 2, but last) +``` + +### Scopes + +Scopes define **where** a rule applies. Five scope levels provide hierarchical targeting: + +``` +┌─────────────────────────────────────────────────────────┐ +│ TablesScope │ +│ (all cells in all tables) │ +│ ┌───────────────────────────────────────────────────┐ │ +│ │ TableScope │ │ +│ │ (all cells in one table) │ │ +│ │ ┌─────────────────────────────────────────────┐ │ │ +│ │ │ ColumnScope │ │ │ +│ │ │ (all cells in column) │ │ │ +│ │ │ ┌───────────────────────────────────────┐ │ │ │ +│ │ │ │ RowScope │ │ │ │ +│ │ │ │ (all cells in row) │ │ │ │ +│ │ │ │ ┌─────────────────────────────────┐ │ │ │ │ +│ │ │ │ │ CellScope │ │ │ │ │ +│ │ │ │ │ (single cell) │ │ │ │ │ +│ │ │ │ └─────────────────────────────────┘ │ │ │ │ +│ │ │ └───────────────────────────────────────┘ │ │ │ +│ │ └─────────────────────────────────────────────┘ │ │ +│ └───────────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────────┘ +``` + +| Scope | Targets | Specificity | DSL Syntax | +|-------|---------|-------------|------------| +| **Cell** | 1 specific cell | Highest (1) | `cell (column, row):` or `cell tcell_id:` | +| **Row** | All cells in row | High (2) | `row index:` | +| **Column** | All cells in column | Medium (3) | `column name:` | +| **Table** | All cells in specific table | Low (4) | `table "name":` | +| **Tables** | All cells in all tables | Lowest (5) | `tables:` | + +**Usage:** + +- **Cell**: Override specific cell (e.g., total cell) +- **Row**: Style header row or specific data row +- **Column**: Format all values in a column (e.g., currency) +- **Table**: Apply defaults to all cells in one DataGrid +- **Tables**: Global defaults for the entire application + +### Presets + +Presets are named configurations that can be referenced by name instead of specifying all properties. + +**Style Presets (DaisyUI 5):** + +| Preset | Background | Text | Use Case | +|--------|------------|------|----------| +| `primary` | `var(--color-primary)` | `var(--color-primary-content)` | Important values | +| `secondary` | `var(--color-secondary)` | `var(--color-secondary-content)` | Secondary info | +| `accent` | `var(--color-accent)` | `var(--color-accent-content)` | Highlights | +| `neutral` | `var(--color-neutral)` | `var(--color-neutral-content)` | Headers, labels | +| `info` | `var(--color-info)` | `var(--color-info-content)` | Information | +| `success` | `var(--color-success)` | `var(--color-success-content)` | Positive values | +| `warning` | `var(--color-warning)` | `var(--color-warning-content)` | Warnings | +| `error` | `var(--color-error)` | `var(--color-error-content)` | Errors, negatives | + +**Formatter Presets:** + +| Preset | Type | Output Example | Use Case | +|--------|------|----------------|----------| +| `EUR` | number | `1 234,56 €` | Euro currency | +| `USD` | number | `$1,234.56` | US Dollar | +| `percentage` | number | `75.0%` | Percentages (×100) | +| `short_date` | date | `29/01/2026` | European dates | +| `iso_date` | date | `2026-01-29` | ISO dates | +| `yes_no` | boolean | `Yes` / `No` | Boolean values | + +**Customization:** + +Presets can be customized globally via `DataGridsManager`: + +```python +manager.add_style_preset("highlight", { + "background-color": "yellow", + "color": "black" +}) + +manager.add_formatter_preset("CHF", { + "type": "number", + "prefix": "CHF ", + "precision": 2, + "thousands_sep": "'" +}) +``` + +--- + +## Layer 1: Formatting Engine + +The formatting engine (`core/formatting/`) is the foundation that applies formatting rules to cell values. It is **independent of the DSL** and can be used programmatically. + +### Dataclasses + +#### Condition + +```python +@dataclass +class Condition: + operator: str # Comparison operator + value: Any = None # Value or reference ({"col": "..."}) + negate: bool = False # Invert result + case_sensitive: bool = False # Case-sensitive strings + col: str = None # Column for row-level conditions + row: int = None # Row for column-level conditions (not implemented) +``` + +**Supported operators:** + +| Operator | Description | Value Type | Example | +|----------|-------------|------------|---------| +| `==` | Equal | scalar | `value == 0` | +| `!=` | Not equal | scalar | `value != ""` | +| `<` | Less than | number | `value < 0` | +| `<=` | Less or equal | number | `value <= 100` | +| `>` | Greater than | number | `value > 1000` | +| `>=` | Greater or equal | number | `value >= 0` | +| `contains` | String contains | string | `value contains "error"` | +| `startswith` | String starts with | string | `value startswith "VIP"` | +| `endswith` | String ends with | string | `value endswith ".pdf"` | +| `in` | Value in list | list | `value in ["A", "B"]` | +| `between` | Value in range | `[min, max]` | `value between [0, 100]` | +| `isempty` | Is null/empty | - | `value isempty` | +| `isnotempty` | Not null/empty | - | `value isnotempty` | +| `isnan` | Is NaN | - | `value isnan` | + +**Column references:** + +Compare with another column in the same row: + +```python +Condition( + operator=">", + value={"col": "budget"} +) +# Evaluates: cell_value > row_data["budget"] +``` + +**Row-level conditions:** + +Evaluate a different column for all cells in a row: + +```python +Condition( + col="status", + operator="==", + value="error" +) +# For each cell in row: row_data["status"] == "error" +``` + +#### Style + +```python +@dataclass +class Style: + preset: str = None # Preset name + background_color: str = None # Background (hex, CSS, var()) + color: str = None # Text color + font_weight: str = None # "normal" or "bold" + font_style: str = None # "normal" or "italic" + font_size: str = None # "12px", "0.9em" + text_decoration: str = None # "none", "underline", "line-through" +``` + +**Resolution logic:** + +1. If `preset` is specified, apply all preset properties +2. Override with any explicit properties + +**Example:** + +```python +Style(preset="error", font_weight="bold") +# Result: error background + error text + bold +``` + +#### Formatter + +Base class with 6 specialized subclasses: + +**NumberFormatter:** + +```python +@dataclass +class NumberFormatter(Formatter): + prefix: str = "" # Text before value ("$") + suffix: str = "" # Text after value (" €") + thousands_sep: str = "" # Thousands separator (",", " ") + decimal_sep: str = "." # Decimal separator + precision: int = 0 # Decimal places + multiplier: float = 1.0 # Multiply before display (100 for %) +``` + +**DateFormatter:** + +```python +@dataclass +class DateFormatter(Formatter): + format: str = "%Y-%m-%d" # strftime pattern +``` + +**BooleanFormatter:** + +```python +@dataclass +class BooleanFormatter(Formatter): + true_value: str = "true" + false_value: str = "false" + null_value: str = "" +``` + +**TextFormatter:** + +```python +@dataclass +class TextFormatter(Formatter): + transform: str = None # "uppercase", "lowercase", "capitalize" + max_length: int = None # Truncate if exceeded + ellipsis: str = "..." # Suffix when truncated +``` + +**EnumFormatter:** + +```python +@dataclass +class EnumFormatter(Formatter): + source: dict = field(default_factory=dict) # Data source + default: str = "" # Label for unknown values + allow_empty: bool = True + empty_label: str = "-- Select --" + order_by: str = "source" # "source", "display", "value" +``` + +**Source types:** + +- **Mapping**: `{"type": "mapping", "value": {"draft": "Draft", "approved": "Approved"}}` +- **DataGrid**: `{"type": "datagrid", "value": "grid_id", "value_column": "id", "display_column": "name"}` + +**ConstantFormatter:** + +```python +@dataclass +class ConstantFormatter(Formatter): + value: str = None # Fixed string to display +``` + +### FormattingEngine + +Main facade that combines condition evaluation, style resolution, and formatting. + +```python +class FormattingEngine: + def __init__( + self, + style_presets: dict = None, + formatter_presets: dict = None, + lookup_resolver: Callable[[str, str, str], dict] = None + ): + """ + Initialize the engine. + + Args: + style_presets: Custom style presets (uses defaults if None) + formatter_presets: Custom formatter presets (uses defaults if None) + lookup_resolver: Function for resolving enum datagrid sources + """ +``` + +**Main method:** + +```python +def apply_format( + self, + rules: list[FormatRule], + cell_value: Any, + row_data: dict = None +) -> tuple[str | None, str | None]: + """ + Apply format rules to a cell value. + + Args: + rules: List of FormatRule to evaluate + cell_value: The cell value to format + row_data: Dict of {col_id: value} for column references + + Returns: + Tuple of (css_string, formatted_value): + - css_string: CSS inline style string, or None + - formatted_value: Formatted string, or None + """ +``` + +### Programmatic Usage + +```python +from myfasthtml.core.formatting.engine import FormattingEngine +from myfasthtml.core.formatting.dataclasses import FormatRule, Condition, Style, NumberFormatter + +# Create engine +engine = FormattingEngine() + +# Define rules +rules = [ + FormatRule( + formatter=NumberFormatter(suffix=" €", precision=2, thousands_sep=" ") + ), + FormatRule( + condition=Condition(operator="<", value=0), + style=Style(preset="error") + ), +] + +# Apply to cell +css, formatted = engine.apply_format(rules, -1234.56) +# css = "background-color: var(--color-error); color: var(--color-error-content);" +# formatted = "-1 234,56 €" +``` + +### Sub-components + +#### ConditionEvaluator + +Evaluates conditions against cell values with support for: + +- Column references: `{"col": "column_name"}` +- Row-level conditions: `condition.col = "status"` +- Case-sensitive/insensitive string comparisons +- Type-safe numeric comparisons (allows int/float mixing) +- Null-safe evaluation (returns False for None values) + +```python +evaluator = ConditionEvaluator() + +condition = Condition(operator=">", value={"col": "budget"}) +result = evaluator.evaluate(condition, cell_value=1000, row_data={"budget": 800}) +# result = True +``` + +#### StyleResolver + +Converts `Style` objects to CSS strings: + +```python +resolver = StyleResolver() + +style = Style(preset="error", font_weight="bold") +css_dict = resolver.resolve(style) +# {"background-color": "var(--color-error)", "color": "var(--color-error-content)", "font-weight": "bold"} + +css_string = resolver.to_css_string(style) +# "background-color: var(--color-error); color: var(--color-error-content); font-weight: bold;" +``` + +#### FormatterResolver + +Dispatches formatting to specialized resolvers based on formatter type: + +```python +resolver = FormatterResolver() + +formatter = NumberFormatter(prefix="$", precision=2, thousands_sep=",") +formatted = resolver.resolve(formatter, 1234.56) +# "$1,234.56" +``` + +**Error handling:** + +If formatting fails (e.g., non-numeric value for NumberFormatter), returns `"⚠"`. + +--- + +## Layer 2: DSL Parser + +The DSL provides a human-readable text format for defining formatting rules. It is parsed into structured `ScopedRule` objects that can be applied by the formatting engine. + +### DSL Syntax Overview + +```python +# Column scope +column amount: + format("EUR") + style("error") if value < 0 + style("success") if value > col.budget + +# Row scope +row 0: + style("neutral", bold=True) + +# Cell scope +cell (amount, 10): + style("accent", bold=True) + +# Table scope +table "orders": + style(font_size="14px") + +# Global scope +tables: + style(color="#333") +``` + +**Structure:** + +``` +scope_header: + rule + rule + ... + +scope_header: + rule + ... +``` + +- Rules are indented (Python-style) +- Comments start with `#` +- String values use double or single quotes + +### Scopes in Detail + +#### Column Scope + +Target all cells in a column by name (matches `col_id` first, then `title`): + +```python +# Simple name +column amount: + style("error") if value < 0 + +# Name with spaces (quoted) +column "total amount": + format("EUR") +``` + +#### Row Scope + +Target all cells in a row by index (0-based): + +```python +# Header row +row 0: + style("neutral", bold=True) + +# Specific row +row 5: + style("highlight") +``` + +#### Cell Scope + +Target a specific cell by coordinates or ID: + +```python +# By coordinates (column, row) +cell (amount, 3): + style("highlight") + +cell ("total amount", 0): + style("neutral", bold=True) + +# By cell ID +cell tcell_grid1-3-2: + style(background_color="yellow") +``` + +#### Table Scope + +Target all cells in a specific table (must match DataGrid `_settings.name`): + +```python +table "products": + style("neutral") + format.number(precision=2) +``` + +#### Tables Scope + +Global rules for all tables: + +```python +tables: + style(font_size="14px", color="#333") +``` + +### Rules Syntax + +A rule consists of optional `style`, optional `format`, and optional `condition`: + +``` +[style(...)] [format(...)] [if ] +``` + +At least one of `style()` or `format()` must be present. + +#### Style Expression + +```python +# Preset only +style("error") + +# Preset with overrides +style("error", bold=True) +style("success", italic=True, underline=True) + +# No preset, direct properties +style(color="red", bold=True) +style(background_color="#ffeeee", color="#cc0000") +``` + +**Parameters:** + +| Parameter | Type | Description | +|-----------|------|-------------| +| `preset` | string (positional) | Preset name | +| `background_color` | string | Background color | +| `color` | string | Text color | +| `bold` | boolean | Bold text | +| `italic` | boolean | Italic text | +| `underline` | boolean | Underlined text | +| `strikethrough` | boolean | Strikethrough text | +| `font_size` | string | Font size ("12px", "0.9em") | + +#### Format Expression + +**Using presets:** + +```python +# Preset only +format("EUR") + +# Preset with overrides +format("EUR", precision=3) +format("percentage", precision=0) +``` + +**Using explicit types:** + +```python +format.number(precision=2, suffix=" €", thousands_sep=" ") +format.date(format="%d/%m/%Y") +format.boolean(true_value="Yes", false_value="No") +format.text(max_length=50, ellipsis="...") +format.enum(source={"draft": "Draft", "approved": "Approved"}, default="Unknown") +format.constant(value="N/A") +``` + +**Type-specific parameters:** + +| Type | Parameters | +|------|------------| +| `number` | `prefix`, `suffix`, `thousands_sep`, `decimal_sep`, `precision`, `multiplier` | +| `date` | `format` (strftime pattern) | +| `boolean` | `true_value`, `false_value`, `null_value` | +| `text` | `transform`, `max_length`, `ellipsis` | +| `enum` | `source`, `default`, `allow_empty`, `empty_label`, `order_by` | +| `constant` | `value` | + +#### Condition Expression + +```python +if +if +``` + +**Operators:** + +| Operator | Example | +|----------|---------| +| `==`, `!=` | `value == 0` | +| `<`, `<=`, `>`, `>=` | `value < 0` | +| `contains` | `value contains "error"` | +| `startswith` | `value startswith "VIP"` | +| `endswith` | `value endswith ".pdf"` | +| `in` | `value in ["A", "B", "C"]` | +| `between` | `value between 0 and 100` | +| `isempty`, `isnotempty` | `value isempty` | +| `isnan` | `value isnan` | + +**Negation:** + +```python +style("error") if not value in ["valid", "approved"] +style("warning") if not value contains "OK" +``` + +**Case sensitivity:** + +String comparisons are case-insensitive by default. Use `(case)` modifier: + +```python +style("error") if value == "Error" (case) +style("warning") if value contains "WARN" (case) +``` + +**References:** + +| Reference | Description | Example | +|-----------|-------------|---------| +| `value` | Current cell value | `value < 0` | +| `col.` | Another column (same row) | `value > col.budget` | +| `col.""` | Column with spaces | `value > col."max amount"` | + +### Complete Examples + +**Basic formatting:** + +```python +column amount: + format("EUR") + style("error") if value < 0 +``` + +**Cross-column comparison:** + +```python +column actual: + format("EUR") + style("error") if value > col.budget + style("warning") if value > col.budget * 0.8 + style("success") if value <= col.budget * 0.8 +``` + +**Multiple conditions:** + +```python +column status: + style("success") if value == "approved" + style("warning") if value == "pending" + style("error") if value == "rejected" + style("neutral") if value isempty +``` + +**Complex example:** + +```python +# Global defaults +tables: + style(font_size="14px", color="#333") + +# Table-specific +table "financial_report": + format.number(precision=2) + +# Header row +row 0: + style("neutral", bold=True) + +# Amount column +column amount: + format.number(precision=2, suffix=" €", thousands_sep=" ") + style("error") if value < 0 + style("success") if value > col.target + +# Percentage column +column progress: + format("percentage") + style("error") if value < 0.5 + style("warning") if value between 0.5 and 0.8 + style("success") if value > 0.8 + +# Status column +column status: + format.enum(source={"draft": "Draft", "review": "In Review", "approved": "Approved"}) + style("neutral") if value == "draft" + style("info") if value == "review" + style("success") if value == "approved" + +# Highlight specific cell +cell (amount, 10): + style("accent", bold=True) +``` + +### Parser and Transformer + +#### Lark Grammar + +The DSL uses a Lark grammar (EBNF) with Python-style indentation: + +```python +from myfasthtml.core.formatting.dsl.grammar import GRAMMAR +# Contains the full Lark grammar string +``` + +Key features: + +- Indentation tracking with `_INDENT` / `_DEDENT` tokens +- Comments with `#` (ignored) +- Quoted strings for names with spaces +- Support for arithmetic expressions (for future use) + +#### Parsing API + +```python +from myfasthtml.core.formatting.dsl import parse_dsl, DSLSyntaxError + +try: + scoped_rules = parse_dsl(dsl_text) + # scoped_rules: list[ScopedRule] +except DSLSyntaxError as e: + print(f"Syntax error at line {e.line}, column {e.column}: {e.message}") +``` + +**Output structure:** + +```python +@dataclass +class ScopedRule: + scope: ColumnScope | RowScope | CellScope | TableScope | TablesScope + rule: FormatRule +``` + +Each `ScopedRule` contains: +- **scope**: Where the rule applies +- **rule**: What formatting to apply (condition + style + formatter) + +--- + +## Layer 3: Autocompletion System + +The autocompletion system provides intelligent, context-aware suggestions while editing the DSL. + +### Architecture + +``` +┌─────────────────────────────────────────────────────────┐ +│ CodeMirror Editor │ +│ User types: style("err| │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ FormattingCompletionEngine │ +│ │ +│ 1. Detect scope (which column?) │ +│ 2. Detect context (inside style()?) │ +│ 3. Generate suggestions (presets + metadata) │ +│ 4. Filter by prefix ("err") │ +└─────────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────────┐ +│ DatagridMetadataProvider │ +│ │ +│ - list_columns() → ["amount", "status", ...] │ +│ - list_column_values("status") → ["draft", "approved"]│ +│ - list_style_presets() → custom presets │ +└─────────────────────────────────────────────────────────┘ +``` + +### How It Works + +**Step 1: Scope Detection** + +Before suggesting completions, detect the current scope by scanning backwards: + +```python +column amount: # ← Scope: ColumnScope("amount") + style("error") if | # ← Cursor here +``` + +The engine knows we're in the `amount` column, so it can suggest column values from that column. + +**Step 2: Context Detection** + +Analyze the current line up to the cursor to determine what's expected: + +``` +User types Context Detected +────────────────── ───────────────────── +col SCOPE_KEYWORD +column a COLUMN_NAME +column amount: + st RULE_START + style( STYLE_ARGS + style(" STYLE_PRESET + style("err STYLE_PRESET (filtered) + style("error", STYLE_PARAM + style("error", bold= BOOLEAN_VALUE + style("error") if CONDITION_START + style("error") if value OPERATOR + style("error") if value == OPERATOR_VALUE +``` + +**Step 3: Suggestion Generation** + +Based on context, generate appropriate suggestions: + +| Context | Suggestions | +|---------|-------------| +| `SCOPE_KEYWORD` | `column`, `row`, `cell`, `table`, `tables` | +| `COLUMN_NAME` | Column names from DataGrid | +| `STYLE_PRESET` | Style presets ("error", "success", etc.) | +| `STYLE_PARAM` | `bold=`, `italic=`, `color=`, etc. | +| `FORMAT_TYPE` | `number`, `date`, `boolean`, `text`, `enum`, `constant` | +| `OPERATOR` | `==`, `<`, `contains`, `in`, etc. | +| `OPERATOR_VALUE` | `col.`, `True`, `False` + column values | + +**Step 4: Filtering** + +Filter suggestions by the current prefix (case-insensitive): + +```python +# User typed: style("err +# Prefix: "err" +# Suggestions: ["error"] (filtered from all style presets) +``` + +### DatagridMetadataProvider + +Protocol for providing DataGrid metadata: + +```python +class DatagridMetadataProvider(Protocol): + def list_tables(self) -> list[str]: + """List of available DataGrid names.""" + ... + + def list_columns(self, table_name: str) -> list[str]: + """Column names for a specific DataGrid.""" + ... + + def list_column_values(self, table_name: str, column_name: str) -> list[Any]: + """Distinct values for a column.""" + ... + + def get_row_count(self, table_name: str) -> int: + """Number of rows in a DataGrid.""" + ... + + def list_style_presets(self) -> list[str]: + """Available style preset names.""" + ... + + def list_format_presets(self) -> list[str]: + """Available format preset names.""" + ... +``` + +Implementations (like `DataGridsManager`) provide this interface to enable dynamic suggestions. + +### Completion Contexts + +The engine recognizes ~30 distinct contexts: + +| Category | Contexts | +|----------|----------| +| **Scope** | `SCOPE_KEYWORD`, `COLUMN_NAME`, `ROW_INDEX`, `CELL_START`, `CELL_COLUMN`, `CELL_ROW`, `TABLE_NAME`, `TABLES_SCOPE` | +| **Rule** | `RULE_START`, `AFTER_STYLE_OR_FORMAT` | +| **Style** | `STYLE_ARGS`, `STYLE_PRESET`, `STYLE_PARAM` | +| **Format** | `FORMAT_PRESET`, `FORMAT_TYPE`, `FORMAT_PARAM_DATE`, `FORMAT_PARAM_TEXT` | +| **Condition** | `CONDITION_START`, `CONDITION_AFTER_NOT`, `OPERATOR`, `OPERATOR_VALUE`, `BETWEEN_AND`, `BETWEEN_VALUE`, `IN_LIST_START`, `IN_LIST_VALUE` | +| **Values** | `BOOLEAN_VALUE`, `COLOR_VALUE`, `DATE_FORMAT_VALUE`, `TRANSFORM_VALUE`, `COLUMN_REF`, `COLUMN_REF_QUOTED` | + +--- + +## Generic DSL Infrastructure + +The `core/dsl/` module provides a **reusable framework** for creating custom DSLs with CodeMirror integration and autocompletion. + +### Purpose + +- Define custom DSLs for different domains +- Automatic syntax highlighting via CodeMirror Simple Mode +- Context-aware autocompletion +- Integration with DslEditor control + +### DSLDefinition + +Abstract base class for defining a DSL: + +```python +from myfasthtml.core.dsl.base import DSLDefinition + +class MyDSL(DSLDefinition): + name: str = "My Custom DSL" + + def get_grammar(self) -> str: + """Return the Lark grammar string.""" + return """ + start: statement+ + statement: NAME "=" VALUE + ... + """ +``` + +**Automatic features:** + +- `completions`: Extracted from grammar terminals +- `simple_mode_config`: CodeMirror 5 Simple Mode for syntax highlighting +- `get_editor_config()`: Configuration for DslEditor JavaScript + +### BaseCompletionEngine + +Abstract base class for completion engines: + +```python +from myfasthtml.core.dsl.base_completion import BaseCompletionEngine + +class MyCompletionEngine(BaseCompletionEngine): + def detect_scope(self, text: str, current_line: int): + """Detect current scope from previous lines.""" + ... + + def detect_context(self, text: str, cursor: Position, scope): + """Detect what kind of completion is expected.""" + ... + + def get_suggestions(self, context, scope, prefix) -> list[Suggestion]: + """Generate suggestions for the context.""" + ... +``` + +**Main entry point:** + +```python +engine = MyCompletionEngine(provider) +result = engine.get_completions(text, cursor) +# result.suggestions: list[Suggestion] +# result.from_pos / result.to_pos: replacement range +``` + +### BaseMetadataProvider + +Protocol for providing domain-specific metadata: + +```python +from myfasthtml.core.dsl.base_provider import BaseMetadataProvider + +class MyProvider(BaseMetadataProvider): + # Implement methods to provide metadata for suggestions + ... +``` + +### Utilities + +**Lark to Simple Mode conversion:** + +```python +from myfasthtml.core.dsl.lark_to_simple_mode import lark_to_simple_mode + +simple_mode_config = lark_to_simple_mode(grammar_string) +# Returns CodeMirror 5 Simple Mode configuration +``` + +**Extract completions from grammar:** + +```python +from myfasthtml.core.dsl.lark_to_simple_mode import extract_completions_from_grammar + +completions = extract_completions_from_grammar(grammar_string) +# Returns: {"keywords": [...], "operators": [...], ...} +``` + +--- + +## DataGrid Integration + +### DataGridFormattingEditor + +The `DataGridFormattingEditor` is a specialized `DslEditor` that integrates the formatting DSL into DataGrid: + +```python +class DataGridFormattingEditor(DslEditor): + def on_content_changed(self): + """Called when DSL content changes.""" + # 1. Parse DSL + scoped_rules = parse_dsl(self.get_content()) + + # 2. Group by scope + columns_rules = defaultdict(list) + rows_rules = defaultdict(list) + cells_rules = defaultdict(list) + table_rules = [] + tables_rules = [] + + # 3. Dispatch to DataGrid state + state.columns[i].format = rules + state.rows[i].format = rules + state.cell_formats[cell_id] = rules + state.table_format = rules + # tables rules stored in DataGridsManager +``` + +**Column name resolution:** + +The editor matches column names by `col_id` first, then `title`: + +```python +# DSL +column amount: # Matches col_id="amount" or title="amount" + ... +``` + +**Cell ID resolution:** + +```python +# By coordinates +cell (amount, 3): # Resolved to tcell_grid1-3-0 + +# By ID +cell tcell_grid1-3-0: # Used directly +``` + +### FormattingDSL + +Implementation of `DSLDefinition` for the formatting DSL: + +```python +from myfasthtml.core.formatting.dsl.definition import FormattingDSL + +dsl = FormattingDSL() +dsl.name # "Formatting DSL" +dsl.get_grammar() # Returns the Lark grammar +dsl.simple_mode_config # CodeMirror Simple Mode config +dsl.completions # Static completion items +``` + +Used by `DataGridFormattingEditor` to configure the CodeMirror editor. + +### Complete Flow in DataGrid + +``` +1. User opens formatting editor + │ + ▼ +2. DslEditor loads with FormattingDSL configuration + - Syntax highlighting enabled + - Autocompletion registered + │ + ▼ +3. User types DSL text + - Autocompletion triggered on keystrokes + - FormattingCompletionEngine provides suggestions + │ + ▼ +4. User saves or content changes + - DataGridFormattingEditor.on_content_changed() + │ + ▼ +5. Parse DSL → ScopedRule[] + │ + ▼ +6. Dispatch rules to state + - state.columns[i].format = [rule, ...] + - state.rows[i].format = [rule, ...] + - state.cell_formats[cell_id] = [rule, ...] + - state.table_format = [rule, ...] + │ + ▼ +7. DataGrid renders cells + - mk_body_cell_content() applies formatting + - FormattingEngine.apply_format(rules, cell_value, row_data) + │ + ▼ +8. CSS + formatted value rendered in cell +``` + +--- + +## Developer Reference + +### Extension Points + +#### Add Custom Style Presets + +```python +from myfasthtml.controls.DataGridsManager import DataGridsManager + +manager = DataGridsManager.get_instance(session) + +manager.add_style_preset("corporate", { + "background-color": "#003366", + "color": "#FFFFFF", + "font-weight": "bold" +}) +``` + +#### Add Custom Formatter Presets + +```python +manager.add_formatter_preset("GBP", { + "type": "number", + "prefix": "£", + "thousands_sep": ",", + "decimal_sep": ".", + "precision": 2 +}) +``` + +#### Implement Lookup Resolver for Enum + +When using `EnumFormatter` with `source.type = "datagrid"`, provide a lookup resolver: + +```python +def my_lookup_resolver(grid_id: str, value_col: str, display_col: str) -> dict: + """ + Resolve enum values from another DataGrid. + + Returns: + Dict mapping value_col values to display_col values + """ + # Fetch data from grid_id + # Build mapping {value: display} + return mapping + +engine = FormattingEngine(lookup_resolver=my_lookup_resolver) +``` + +#### Create Custom Formatter + +```python +from myfasthtml.core.formatting.dataclasses import Formatter +from myfasthtml.core.formatting.formatter_resolver import BaseFormatterResolver + +@dataclass +class CustomFormatter(Formatter): + custom_param: str = None + +class CustomFormatterResolver(BaseFormatterResolver): + def resolve(self, formatter: CustomFormatter, value: Any) -> str: + # Custom formatting logic + return str(value).upper() + + def apply_preset(self, formatter: Formatter, presets: dict) -> CustomFormatter: + # Preset application logic + return formatter + +# Register in FormatterResolver +FormatterResolver._resolvers[CustomFormatter] = CustomFormatterResolver() +``` + +#### Create a New DSL + +```python +from myfasthtml.core.dsl.base import DSLDefinition +from myfasthtml.core.dsl.base_completion import BaseCompletionEngine + +class MyDSL(DSLDefinition): + name = "My DSL" + + def get_grammar(self) -> str: + return """ + start: rule+ + rule: NAME ":" VALUE + ... + """ + +class MyCompletionEngine(BaseCompletionEngine): + def detect_scope(self, text, line): + # Scope detection logic + return scope + + def detect_context(self, text, cursor, scope): + # Context detection logic + return context + + def get_suggestions(self, context, scope, prefix): + # Suggestion generation + return suggestions +``` + +### Module Dependency Graph + +``` +┌─────────────────────────────────────────────────────────┐ +│ controls/DataGridFormattingEditor │ +└─────────────────────────────────────────────────────────┘ + │ + ┌────────────────┼────────────────┐ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ formatting/ │ │ formatting/ │ │ formatting/ │ +│ dsl/ │ │ dsl/ │ │ engine.py │ +│ __init__.py │ │ definition.py│ │ │ +└──────────────┘ └──────────────┘ └──────────────┘ + │ │ │ + ▼ ▼ ▼ +┌──────────────┐ ┌──────────────┐ ┌──────────────┐ +│ formatting/ │ │ dsl/ │ │ formatting/ │ +│ dsl/ │ │ base.py │ │ condition_ │ +│ parser.py │ │ │ │ evaluator.py │ +│ transformer │ │ │ │ style_ │ +│ .py │ │ │ │ resolver.py │ +│ scopes.py │ │ │ │ formatter_ │ +│ grammar.py │ │ │ │ resolver.py │ +└──────────────┘ └──────────────┘ └──────────────┘ + │ │ + ▼ ▼ +┌──────────────┐ ┌──────────────┐ +│ formatting/ │ │ formatting/ │ +│ dataclasses │◄─────────────────│ presets.py │ +│ .py │ │ │ +└──────────────┘ └──────────────┘ +``` + +**Dependency layers (bottom to top):** + +1. **Dataclasses + Presets**: Core data structures +2. **Resolvers**: Condition evaluation, style, formatting +3. **Engine**: Facade combining resolvers +4. **DSL Grammar + Parser + Transformer**: Text → structured rules +5. **Completion**: Intelligent suggestions +6. **Integration**: DataGrid editor + +### Public APIs by Module + +| Module | Public API | Description | +|--------|------------|-------------| +| `core/formatting/` | `FormattingEngine` | Main engine for applying rules | +| `core/formatting/dataclasses.py` | `Condition`, `Style`, `Formatter`, `FormatRule` | Core data structures | +| `core/formatting/presets.py` | `DEFAULT_STYLE_PRESETS`, `DEFAULT_FORMATTER_PRESETS` | Built-in presets | +| `core/formatting/dsl/` | `parse_dsl()` | Parse DSL text → `list[ScopedRule]` | +| `core/formatting/dsl/` | `ColumnScope`, `RowScope`, `CellScope`, `TableScope`, `TablesScope` | Scope classes | +| `core/formatting/dsl/` | `DSLSyntaxError`, `DSLValidationError` | Exceptions | +| `core/formatting/dsl/completion/` | `FormattingCompletionEngine` | Autocompletion engine | +| `core/formatting/dsl/completion/` | `DatagridMetadataProvider` | Metadata protocol | +| `core/dsl/` | `DSLDefinition` | Base for creating DSLs | +| `core/dsl/` | `BaseCompletionEngine` | Base for completion engines | + +### Key Classes and Responsibilities + +| Class | Location | Responsibility | +|-------|----------|----------------| +| `FormattingEngine` | `core/formatting/engine.py` | Apply format rules to cell values | +| `ConditionEvaluator` | `core/formatting/condition_evaluator.py` | Evaluate 13 operators + references | +| `StyleResolver` | `core/formatting/style_resolver.py` | Resolve styles to CSS | +| `FormatterResolver` | `core/formatting/formatter_resolver.py` | Dispatch to type-specific formatters | +| `DSLParser` (internal) | `core/formatting/dsl/parser.py` | Parse DSL text with Lark | +| `DSLTransformer` | `core/formatting/dsl/transformer.py` | Convert AST → dataclasses | +| `FormattingCompletionEngine` | `core/formatting/dsl/completion/FormattingCompletionEngine.py` | Context-aware suggestions | +| `DataGridFormattingEditor` | `controls/DataGridFormattingEditor.py` | Integrate DSL into DataGrid | +| `FormattingDSL` | `core/formatting/dsl/definition.py` | DSL definition for editor | + +### Known Limitations + +| Limitation | Status | Impact | +|------------|--------|--------| +| `row` parameter for column-level conditions | Not implemented | Cannot reference specific row in column scope | +| AND/OR operators | Not implemented | Use multiple rules or `in`/`between` | +| Format chaining | Not implemented | Only one formatter per rule | +| Cell references in conditions | Partial | Only column references (`col.x`), not arbitrary cells | +| Arithmetic in conditions | Parsed but not evaluated | Expressions like `col.budget * 0.9` not supported | + +--- + +## Summary + +The DataGrid Formatting System provides a comprehensive, three-layer architecture for conditional formatting: + +1. **Formatting Engine**: Core logic for applying rules (condition evaluation, style resolution, formatting) +2. **DSL Parser**: Human-readable text format for defining rules with scopes and conditions +3. **Autocompletion**: Intelligent suggestions based on context and DataGrid metadata + +**Key strengths:** + +- **Flexible**: 5 scope levels, 13 operators, 6 formatter types +- **DaisyUI integrated**: 8 style presets that adapt to theme +- **Developer-friendly**: Clean API, programmatic + DSL usage +- **Extensible**: Custom presets, formatters, and DSLs +- **Production-ready**: Robust parsing, error handling, conflict resolution + +**Typical workflow:** + +1. Write DSL in DataGridFormattingEditor +2. DSL parsed → ScopedRule objects +3. Rules dispatched to column/row/cell formats +4. FormattingEngine applies rules during rendering +5. Cells display with CSS styles + formatted values + +For most use cases, the DSL provides sufficient expressiveness without requiring programmatic interaction with the formatting engine.