# DataGrid Formatting System

## Introduction

The DataGrid Formatting System provides a comprehensive solution for applying conditional formatting and data transformation to DataGrid cells. It combines a powerful formatting engine with a user-friendly Domain Specific Language (DSL) and intelligent autocompletion.

**Key features:**

- Conditional formatting based on cell values and cross-column comparisons
- Rich styling with DaisyUI 5 theme presets
- Multiple data formatters (currency, dates, percentages, text transformations, enums)
- Text-based DSL for defining formatting rules
- Context-aware autocompletion with CodeMirror integration
- Five scope levels for precise targeting (cell, row, column, table, global)

**Common use cases:**

- Highlight negative amounts in red
- Format currency values with proper separators
- Apply different styles based on status columns
- Compare actual vs budget values
- Display dates in localized formats
- Transform enum values to readable labels

---

## Architecture Overview

### System Layers

The formatting system is built in three layers, each with a specific responsibility:

```
┌─────────────────────────────────────────────────────────┐
│                  User Interface                         │
│            DataGridFormattingEditor (DSL)               │
└─────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│               Layer 3: Autocompletion                   │
│         FormattingCompletionEngine + Provider           │
│    (context detection, dynamic suggestions)             │
└─────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│                 Layer 2: DSL Parser                     │
│         Lark Grammar → Transformer → ScopedRule         │
│    (text → structured format rules)                     │
└─────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│            Layer 1: Formatting Engine                   │
│    FormattingEngine (ConditionEvaluator +               │
│       StyleResolver + FormatterResolver)                │
│    (apply rules → CSS + formatted value)                │
└─────────────────────────────────────────────────────────┘
```

| Layer | Module | Responsibility |
|-------|--------|----------------|
| **1. Formatting Engine** | `core/formatting/` | Apply format rules to cell values, resolve conflicts |
| **2. DSL Parser** | `core/formatting/dsl/` | Parse text DSL into structured rules |
| **3. Autocompletion** | `core/formatting/dsl/completion/` | Provide intelligent suggestions while editing |
| **Infrastructure** | `core/dsl/` | Generic DSL framework (reusable) |

### End-to-End Flow

```
User writes DSL in editor
         │
         ▼
┌────────────────────┐
│  Lark Parser       │  Parse text with indentation support
└────────────────────┘
         │
         ▼
┌────────────────────┐
│  DSLTransformer    │  Convert AST → ScopedRule objects
└────────────────────┘
         │
         ▼
┌────────────────────┐
│  DataGrid          │  Dispatch rules to column/row/cell formats
│  FormattingEditor  │
└────────────────────┘
         │
         ▼
┌────────────────────┐
│  FormattingEngine  │  Apply rules to each cell
└────────────────────┘
         │
         ▼
    CSS + Formatted Value → Rendered Cell
```

### Module Responsibilities

| Module | Location | Purpose |
|--------|----------|---------|
| `FormattingEngine` | `core/formatting/engine.py` | Main facade for applying format rules |
| `ConditionEvaluator` | `core/formatting/condition_evaluator.py` | Evaluate conditions (==, <, contains, etc.) |
| `StyleResolver` | `core/formatting/style_resolver.py` | Resolve styles to CSS strings |
| `FormatterResolver` | `core/formatting/formatter_resolver.py` | Format values for display |
| `parse_dsl()` | `core/formatting/dsl/__init__.py` | Public API for parsing DSL |
| `DSLTransformer` | `core/formatting/dsl/transformer.py` | Convert Lark AST to dataclasses |
| `FormattingCompletionEngine` | `core/formatting/dsl/completion/` | Autocompletion for DSL |
| `DataGridFormattingEditor` | `controls/DataGridFormattingEditor.py` | DSL editor integrated in DataGrid |

---

## Fundamental Concepts

### FormatRule

A `FormatRule` is the core data structure combining three optional components:

```python
@dataclass
class FormatRule:
    condition: Condition = None      # When to apply (optional)
    style: Style = None              # Visual formatting (optional)
    formatter: Formatter = None      # Value transformation (optional)
```

**Rules:**

- At least one of `style` or `formatter` must be present
- `condition` cannot appear alone
- If `condition` is present, the rule applies only when the condition is met
- Multiple rules can be defined; they are evaluated in order

**Conflict resolution:**

When multiple rules match the same cell:

1. **Specificity** = 1 if rule has condition, 0 otherwise
2. **Higher specificity wins**
3. **At equal specificity, last rule wins entirely** (no property merging)

**Example:**

```python
# Rule 1: Unconditional (specificity = 0)
FormatRule(style=Style(color="gray"))

# Rule 2: Conditional (specificity = 1)
FormatRule(
    condition=Condition(operator="<", value=0),
    style=Style(color="red")
)

# Rule 3: Conditional (specificity = 1)
FormatRule(
    condition=Condition(operator="==", value=-5),
    style=Style(color="black")
)

# For value = -5: Rule 3 wins (same specificity as Rule 2, but last)
```

### Scopes

Scopes define **where** a rule applies. Five scope levels provide hierarchical targeting:

```
┌─────────────────────────────────────────────────────────┐
│                       TablesScope                       │
│                 (all cells in all tables)               │
│  ┌───────────────────────────────────────────────────┐  │
│  │                  TableScope                       │  │
│  │            (all cells in one table)               │  │
│  │  ┌─────────────────────────────────────────────┐  │  │
│  │  │            ColumnScope                      │  │  │
│  │  │       (all cells in column)                 │  │  │
│  │  │  ┌───────────────────────────────────────┐  │  │  │
│  │  │  │        RowScope                       │  │  │  │
│  │  │  │   (all cells in row)                  │  │  │  │
│  │  │  │  ┌─────────────────────────────────┐  │  │  │  │
│  │  │  │  │      CellScope                  │  │  │  │  │
│  │  │  │  │   (single cell)                 │  │  │  │  │
│  │  │  │  └─────────────────────────────────┘  │  │  │  │
│  │  │  └───────────────────────────────────────┘  │  │  │
│  │  └─────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────┘
```

| Scope | Targets | Specificity | DSL Syntax |
|-------|---------|-------------|------------|
| **Cell** | 1 specific cell | Highest (1) | `cell (column, row):` or `cell tcell_id:` |
| **Row** | All cells in row | High (2) | `row index:` |
| **Column** | All cells in column | Medium (3) | `column name:` |
| **Table** | All cells in specific table | Low (4) | `table "name":` |
| **Tables** | All cells in all tables | Lowest (5) | `tables:` |

**Usage:**

- **Cell**: Override specific cell (e.g., total cell)
- **Row**: Style header row or specific data row
- **Column**: Format all values in a column (e.g., currency)
- **Table**: Apply defaults to all cells in one DataGrid
- **Tables**: Global defaults for the entire application

### Presets

Presets are named configurations that can be referenced by name instead of specifying all properties.

**Style Presets (DaisyUI 5):**

| Preset | Background | Text | Use Case |
|--------|------------|------|----------|
| `primary` | `var(--color-primary)` | `var(--color-primary-content)` | Important values |
| `secondary` | `var(--color-secondary)` | `var(--color-secondary-content)` | Secondary info |
| `accent` | `var(--color-accent)` | `var(--color-accent-content)` | Highlights |
| `neutral` | `var(--color-neutral)` | `var(--color-neutral-content)` | Headers, labels |
| `info` | `var(--color-info)` | `var(--color-info-content)` | Information |
| `success` | `var(--color-success)` | `var(--color-success-content)` | Positive values |
| `warning` | `var(--color-warning)` | `var(--color-warning-content)` | Warnings |
| `error` | `var(--color-error)` | `var(--color-error-content)` | Errors, negatives |

**Formatter Presets:**

| Preset | Type | Output Example | Use Case |
|--------|------|----------------|----------|
| `EUR` | number | `1 234,56 €` | Euro currency |
| `USD` | number | `$1,234.56` | US Dollar |
| `percentage` | number | `75.0%` | Percentages (×100) |
| `short_date` | date | `29/01/2026` | European dates |
| `iso_date` | date | `2026-01-29` | ISO dates |
| `yes_no` | boolean | `Yes` / `No` | Boolean values |

**Customization:**

Presets can be customized globally via `DataGridsManager`:

```python
manager.add_style_preset("highlight", {
    "background-color": "yellow",
    "color": "black"
})

manager.add_formatter_preset("CHF", {
    "type": "number",
    "prefix": "CHF ",
    "precision": 2,
    "thousands_sep": "'"
})
```

---

## Layer 1: Formatting Engine

The formatting engine (`core/formatting/`) is the foundation that applies formatting rules to cell values. It is **independent of the DSL** and can be used programmatically.

### Dataclasses

#### Condition

```python
@dataclass
class Condition:
    operator: str              # Comparison operator
    value: Any = None          # Value or reference ({"col": "..."})
    negate: bool = False       # Invert result
    case_sensitive: bool = False  # Case-sensitive strings
    col: str = None            # Column for row-level conditions
    row: int = None            # Row for column-level conditions (not implemented)
```

**Supported operators:**

| Operator | Description | Value Type | Example |
|----------|-------------|------------|---------|
| `==` | Equal | scalar | `value == 0` |
| `!=` | Not equal | scalar | `value != ""` |
| `<` | Less than | number | `value < 0` |
| `<=` | Less or equal | number | `value <= 100` |
| `>` | Greater than | number | `value > 1000` |
| `>=` | Greater or equal | number | `value >= 0` |
| `contains` | String contains | string | `value contains "error"` |
| `startswith` | String starts with | string | `value startswith "VIP"` |
| `endswith` | String ends with | string | `value endswith ".pdf"` |
| `in` | Value in list | list | `value in ["A", "B"]` |
| `between` | Value in range | `[min, max]` | `value between [0, 100]` |
| `isempty` | Is null/empty | - | `value isempty` |
| `isnotempty` | Not null/empty | - | `value isnotempty` |
| `isnan` | Is NaN | - | `value isnan` |

**Column references:**

Compare with another column in the same row:

```python
Condition(
    operator=">",
    value={"col": "budget"}
)
# Evaluates: cell_value > row_data["budget"]
```

**Row-level conditions:**

Evaluate a different column for all cells in a row:

```python
Condition(
    col="status",
    operator="==",
    value="error"
)
# For each cell in row: row_data["status"] == "error"
```

#### Style

```python
@dataclass
class Style:
    preset: str = None              # Preset name
    background_color: str = None    # Background (hex, CSS, var())
    color: str = None               # Text color
    font_weight: str = None         # "normal" or "bold"
    font_style: str = None          # "normal" or "italic"
    font_size: str = None           # "12px", "0.9em"
    text_decoration: str = None     # "none", "underline", "line-through"
```

**Resolution logic:**

1. If `preset` is specified, apply all preset properties
2. Override with any explicit properties

**Example:**

```python
Style(preset="error", font_weight="bold")
# Result: error background + error text + bold
```

#### Formatter

Base class with 6 specialized subclasses:

**NumberFormatter:**

```python
@dataclass
class NumberFormatter(Formatter):
    prefix: str = ""           # Text before value ("$")
    suffix: str = ""           # Text after value (" €")
    thousands_sep: str = ""    # Thousands separator (",", " ")
    decimal_sep: str = "."     # Decimal separator
    precision: int = 0         # Decimal places
    multiplier: float = 1.0    # Multiply before display (100 for %)
```

**DateFormatter:**

```python
@dataclass
class DateFormatter(Formatter):
    format: str = "%Y-%m-%d"   # strftime pattern
```

**BooleanFormatter:**

```python
@dataclass
class BooleanFormatter(Formatter):
    true_value: str = "true"
    false_value: str = "false"
    null_value: str = ""
```

**TextFormatter:**

```python
@dataclass
class TextFormatter(Formatter):
    transform: str = None      # "uppercase", "lowercase", "capitalize"
    max_length: int = None     # Truncate if exceeded
    ellipsis: str = "..."      # Suffix when truncated
```

**EnumFormatter:**

```python
@dataclass
class EnumFormatter(Formatter):
    source: dict = field(default_factory=dict)  # Data source
    default: str = ""          # Label for unknown values
    allow_empty: bool = True
    empty_label: str = "-- Select --"
    order_by: str = "source"   # "source", "display", "value"
```

**Source types:**

- **Mapping**: `{"type": "mapping", "value": {"draft": "Draft", "approved": "Approved"}}`
- **DataGrid**: `{"type": "datagrid", "value": "grid_id", "value_column": "id", "display_column": "name"}`

**ConstantFormatter:**

```python
@dataclass
class ConstantFormatter(Formatter):
    value: str = None          # Fixed string to display
```

### FormattingEngine

Main facade that combines condition evaluation, style resolution, and formatting.

```python
class FormattingEngine:
    def __init__(
        self,
        style_presets: dict = None,
        formatter_presets: dict = None,
        lookup_resolver: Callable[[str, str, str], dict] = None
    ):
        """
        Initialize the engine.

        Args:
            style_presets: Custom style presets (uses defaults if None)
            formatter_presets: Custom formatter presets (uses defaults if None)
            lookup_resolver: Function for resolving enum datagrid sources
        """
```

**Main method:**

```python
def apply_format(
    self,
    rules: list[FormatRule],
    cell_value: Any,
    row_data: dict = None
) -> tuple[str | None, str | None]:
    """
    Apply format rules to a cell value.

    Args:
        rules: List of FormatRule to evaluate
        cell_value: The cell value to format
        row_data: Dict of {col_id: value} for column references

    Returns:
        Tuple of (css_string, formatted_value):
        - css_string: CSS inline style string, or None
        - formatted_value: Formatted string, or None
    """
```

### Programmatic Usage

```python
from myfasthtml.core.formatting.engine import FormattingEngine
from myfasthtml.core.formatting.dataclasses import FormatRule, Condition, Style, NumberFormatter

# Create engine
engine = FormattingEngine()

# Define rules
rules = [
    FormatRule(
        formatter=NumberFormatter(suffix=" €", precision=2, thousands_sep=" ")
    ),
    FormatRule(
        condition=Condition(operator="<", value=0),
        style=Style(preset="error")
    ),
]

# Apply to cell
css, formatted = engine.apply_format(rules, -1234.56)
# css = "background-color: var(--color-error); color: var(--color-error-content);"
# formatted = "-1 234,56 €"
```

### Sub-components

#### ConditionEvaluator

Evaluates conditions against cell values with support for:

- Column references: `{"col": "column_name"}`
- Row-level conditions: `condition.col = "status"`
- Case-sensitive/insensitive string comparisons
- Type-safe numeric comparisons (allows int/float mixing)
- Null-safe evaluation (returns False for None values)

```python
evaluator = ConditionEvaluator()

condition = Condition(operator=">", value={"col": "budget"})
result = evaluator.evaluate(condition, cell_value=1000, row_data={"budget": 800})
# result = True
```

#### StyleResolver

Converts `Style` objects to CSS strings:

```python
resolver = StyleResolver()

style = Style(preset="error", font_weight="bold")
css_dict = resolver.resolve(style)
# {"background-color": "var(--color-error)", "color": "var(--color-error-content)", "font-weight": "bold"}

css_string = resolver.to_css_string(style)
# "background-color: var(--color-error); color: var(--color-error-content); font-weight: bold;"
```

#### FormatterResolver

Dispatches formatting to specialized resolvers based on formatter type:

```python
resolver = FormatterResolver()

formatter = NumberFormatter(prefix="$", precision=2, thousands_sep=",")
formatted = resolver.resolve(formatter, 1234.56)
# "$1,234.56"
```

**Error handling:**

If formatting fails (e.g., non-numeric value for NumberFormatter), returns `"⚠"`.

---

## Layer 2: DSL Parser

The DSL provides a human-readable text format for defining formatting rules. It is parsed into structured `ScopedRule` objects that can be applied by the formatting engine.

### DSL Syntax Overview

```python
# Column scope
column amount:
    format("EUR")
    style("error") if value < 0
    style("success") if value > col.budget

# Row scope
row 0:
    style("neutral", bold=True)

# Cell scope
cell (amount, 10):
    style("accent", bold=True)

# Table scope
table "orders":
    style(font_size="14px")

# Global scope
tables:
    style(color="#333")
```

**Structure:**

```
scope_header:
    rule
    rule
    ...

scope_header:
    rule
    ...
```

- Rules are indented (Python-style)
- Comments start with `#`
- String values use double or single quotes

### Scopes in Detail

#### Column Scope

Target all cells in a column by name (matches `col_id` first, then `title`):

```python
# Simple name
column amount:
    style("error") if value < 0

# Name with spaces (quoted)
column "total amount":
    format("EUR")
```

#### Row Scope

Target all cells in a row by index (0-based):

```python
# Header row
row 0:
    style("neutral", bold=True)

# Specific row
row 5:
    style("highlight")
```

#### Cell Scope

Target a specific cell by coordinates or ID:

```python
# By coordinates (column, row)
cell (amount, 3):
    style("highlight")

cell ("total amount", 0):
    style("neutral", bold=True)

# By cell ID
cell tcell_grid1-3-2:
    style(background_color="yellow")
```

#### Table Scope

Target all cells in a specific table (must match DataGrid `_settings.name`):

```python
table "products":
    style("neutral")
    format.number(precision=2)
```

#### Tables Scope

Global rules for all tables:

```python
tables:
    style(font_size="14px", color="#333")
```

### Rules Syntax

A rule consists of optional `style`, optional `format`, and optional `condition`:

```
[style(...)] [format(...)] [if <condition>]
```

At least one of `style()` or `format()` must be present.

#### Style Expression

```python
# Preset only
style("error")

# Preset with overrides
style("error", bold=True)
style("success", italic=True, underline=True)

# No preset, direct properties
style(color="red", bold=True)
style(background_color="#ffeeee", color="#cc0000")
```

**Parameters:**

| Parameter | Type | Description |
|-----------|------|-------------|
| `preset` | string (positional) | Preset name |
| `background_color` | string | Background color |
| `color` | string | Text color |
| `bold` | boolean | Bold text |
| `italic` | boolean | Italic text |
| `underline` | boolean | Underlined text |
| `strikethrough` | boolean | Strikethrough text |
| `font_size` | string | Font size ("12px", "0.9em") |

#### Format Expression

**Using presets:**

```python
# Preset only
format("EUR")

# Preset with overrides
format("EUR", precision=3)
format("percentage", precision=0)
```

**Using explicit types:**

```python
format.number(precision=2, suffix=" €", thousands_sep=" ")
format.date(format="%d/%m/%Y")
format.boolean(true_value="Yes", false_value="No")
format.text(max_length=50, ellipsis="...")
format.enum(source={"draft": "Draft", "approved": "Approved"}, default="Unknown")
format.constant(value="N/A")
```

**Type-specific parameters:**

| Type | Parameters |
|------|------------|
| `number` | `prefix`, `suffix`, `thousands_sep`, `decimal_sep`, `precision`, `multiplier` |
| `date` | `format` (strftime pattern) |
| `boolean` | `true_value`, `false_value`, `null_value` |
| `text` | `transform`, `max_length`, `ellipsis` |
| `enum` | `source`, `default`, `allow_empty`, `empty_label`, `order_by` |
| `constant` | `value` |

#### Condition Expression

```python
if <left> <operator> <right>
if <operand> <unary_operator>
```

**Operators:**

| Operator | Example |
|----------|---------|
| `==`, `!=` | `value == 0` |
| `<`, `<=`, `>`, `>=` | `value < 0` |
| `contains` | `value contains "error"` |
| `startswith` | `value startswith "VIP"` |
| `endswith` | `value endswith ".pdf"` |
| `in` | `value in ["A", "B", "C"]` |
| `between` | `value between 0 and 100` |
| `isempty`, `isnotempty` | `value isempty` |
| `isnan` | `value isnan` |

**Negation:**

```python
style("error") if not value in ["valid", "approved"]
style("warning") if not value contains "OK"
```

**Case sensitivity:**

String comparisons are case-insensitive by default. Use `(case)` modifier:

```python
style("error") if value == "Error" (case)
style("warning") if value contains "WARN" (case)
```

**References:**

| Reference | Description | Example |
|-----------|-------------|---------|
| `value` | Current cell value | `value < 0` |
| `col.<name>` | Another column (same row) | `value > col.budget` |
| `col."<name>"` | Column with spaces | `value > col."max amount"` |

### Complete Examples

**Basic formatting:**

```python
column amount:
    format("EUR")
    style("error") if value < 0
```

**Cross-column comparison:**

```python
column actual:
    format("EUR")
    style("error") if value > col.budget
    style("warning") if value > col.budget * 0.8
    style("success") if value <= col.budget * 0.8
```

**Multiple conditions:**

```python
column status:
    style("success") if value == "approved"
    style("warning") if value == "pending"
    style("error") if value == "rejected"
    style("neutral") if value isempty
```

**Complex example:**

```python
# Global defaults
tables:
    style(font_size="14px", color="#333")

# Table-specific
table "financial_report":
    format.number(precision=2)

# Header row
row 0:
    style("neutral", bold=True)

# Amount column
column amount:
    format.number(precision=2, suffix=" €", thousands_sep=" ")
    style("error") if value < 0
    style("success") if value > col.target

# Percentage column
column progress:
    format("percentage")
    style("error") if value < 0.5
    style("warning") if value between 0.5 and 0.8
    style("success") if value > 0.8

# Status column
column status:
    format.enum(source={"draft": "Draft", "review": "In Review", "approved": "Approved"})
    style("neutral") if value == "draft"
    style("info") if value == "review"
    style("success") if value == "approved"

# Highlight specific cell
cell (amount, 10):
    style("accent", bold=True)
```

### Parser and Transformer

#### Lark Grammar

The DSL uses a Lark grammar (EBNF) with Python-style indentation:

```python
from myfasthtml.core.formatting.dsl.grammar import GRAMMAR
# Contains the full Lark grammar string
```

Key features:

- Indentation tracking with `_INDENT` / `_DEDENT` tokens
- Comments with `#` (ignored)
- Quoted strings for names with spaces
- Support for arithmetic expressions (for future use)

#### Parsing API

```python
from myfasthtml.core.formatting.dsl import parse_dsl, DSLSyntaxError

try:
    scoped_rules = parse_dsl(dsl_text)
    # scoped_rules: list[ScopedRule]
except DSLSyntaxError as e:
    print(f"Syntax error at line {e.line}, column {e.column}: {e.message}")
```

**Output structure:**

```python
@dataclass
class ScopedRule:
    scope: ColumnScope | RowScope | CellScope | TableScope | TablesScope
    rule: FormatRule
```

Each `ScopedRule` contains:
- **scope**: Where the rule applies
- **rule**: What formatting to apply (condition + style + formatter)

---

## Layer 3: Autocompletion System

The autocompletion system provides intelligent, context-aware suggestions while editing the DSL.

### Architecture

```
┌─────────────────────────────────────────────────────────┐
│                    CodeMirror Editor                    │
│  User types: style("err|                                │
└─────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│           FormattingCompletionEngine                    │
│                                                         │
│  1. Detect scope (which column?)                       │
│  2. Detect context (inside style()?)                   │
│  3. Generate suggestions (presets + metadata)          │
│  4. Filter by prefix ("err")                           │
└─────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────┐
│          DatagridMetadataProvider                       │
│                                                         │
│  - list_columns() → ["amount", "status", ...]          │
│  - list_column_values("status") → ["draft", "approved"]│
│  - list_style_presets() → custom presets              │
└─────────────────────────────────────────────────────────┘
```

### How It Works

**Step 1: Scope Detection**

Before suggesting completions, detect the current scope by scanning backwards:

```python
column amount:        # ← Scope: ColumnScope("amount")
    style("error") if |  # ← Cursor here
```

The engine knows we're in the `amount` column, so it can suggest column values from that column.

**Step 2: Context Detection**

Analyze the current line up to the cursor to determine what's expected:

```
User types          Context Detected
──────────────────  ─────────────────────
col                 SCOPE_KEYWORD
column a            COLUMN_NAME
column amount:
    st              RULE_START
    style(          STYLE_ARGS
    style("         STYLE_PRESET
    style("err      STYLE_PRESET (filtered)
    style("error",  STYLE_PARAM
    style("error", bold=  BOOLEAN_VALUE
    style("error") if     CONDITION_START
    style("error") if value  OPERATOR
    style("error") if value == OPERATOR_VALUE
```

**Step 3: Suggestion Generation**

Based on context, generate appropriate suggestions:

| Context | Suggestions |
|---------|-------------|
| `SCOPE_KEYWORD` | `column`, `row`, `cell`, `table`, `tables` |
| `COLUMN_NAME` | Column names from DataGrid |
| `STYLE_PRESET` | Style presets ("error", "success", etc.) |
| `STYLE_PARAM` | `bold=`, `italic=`, `color=`, etc. |
| `FORMAT_TYPE` | `number`, `date`, `boolean`, `text`, `enum`, `constant` |
| `OPERATOR` | `==`, `<`, `contains`, `in`, etc. |
| `OPERATOR_VALUE` | `col.`, `True`, `False` + column values |

**Step 4: Filtering**

Filter suggestions by the current prefix (case-insensitive):

```python
# User typed: style("err
# Prefix: "err"
# Suggestions: ["error"] (filtered from all style presets)
```

### DatagridMetadataProvider

Protocol for providing DataGrid metadata:

```python
class DatagridMetadataProvider(Protocol):
    def list_tables(self) -> list[str]:
        """List of available DataGrid names."""
        ...

    def list_columns(self, table_name: str) -> list[str]:
        """Column names for a specific DataGrid."""
        ...

    def list_column_values(self, table_name: str, column_name: str) -> list[Any]:
        """Distinct values for a column."""
        ...

    def get_row_count(self, table_name: str) -> int:
        """Number of rows in a DataGrid."""
        ...

    def list_style_presets(self) -> list[str]:
        """Available style preset names."""
        ...

    def list_format_presets(self) -> list[str]:
        """Available format preset names."""
        ...
```

Implementations (like `DataGridsManager`) provide this interface to enable dynamic suggestions.

### Completion Contexts

The engine recognizes ~30 distinct contexts:

| Category | Contexts |
|----------|----------|
| **Scope** | `SCOPE_KEYWORD`, `COLUMN_NAME`, `ROW_INDEX`, `CELL_START`, `CELL_COLUMN`, `CELL_ROW`, `TABLE_NAME`, `TABLES_SCOPE` |
| **Rule** | `RULE_START`, `AFTER_STYLE_OR_FORMAT` |
| **Style** | `STYLE_ARGS`, `STYLE_PRESET`, `STYLE_PARAM` |
| **Format** | `FORMAT_PRESET`, `FORMAT_TYPE`, `FORMAT_PARAM_DATE`, `FORMAT_PARAM_TEXT` |
| **Condition** | `CONDITION_START`, `CONDITION_AFTER_NOT`, `OPERATOR`, `OPERATOR_VALUE`, `BETWEEN_AND`, `BETWEEN_VALUE`, `IN_LIST_START`, `IN_LIST_VALUE` |
| **Values** | `BOOLEAN_VALUE`, `COLOR_VALUE`, `DATE_FORMAT_VALUE`, `TRANSFORM_VALUE`, `COLUMN_REF`, `COLUMN_REF_QUOTED` |

---

## Generic DSL Infrastructure

The `core/dsl/` module provides a **reusable framework** for creating custom DSLs with CodeMirror integration and autocompletion.

### Purpose

- Define custom DSLs for different domains
- Automatic syntax highlighting via CodeMirror Simple Mode
- Context-aware autocompletion
- Integration with DslEditor control

### DSLDefinition

Abstract base class for defining a DSL:

```python
from myfasthtml.core.dsl.base import DSLDefinition

class MyDSL(DSLDefinition):
    name: str = "My Custom DSL"

    def get_grammar(self) -> str:
        """Return the Lark grammar string."""
        return """
        start: statement+
        statement: NAME "=" VALUE
        ...
        """
```

**Automatic features:**

- `completions`: Extracted from grammar terminals
- `simple_mode_config`: CodeMirror 5 Simple Mode for syntax highlighting
- `get_editor_config()`: Configuration for DslEditor JavaScript

### BaseCompletionEngine

Abstract base class for completion engines:

```python
from myfasthtml.core.dsl.base_completion import BaseCompletionEngine

class MyCompletionEngine(BaseCompletionEngine):
    def detect_scope(self, text: str, current_line: int):
        """Detect current scope from previous lines."""
        ...

    def detect_context(self, text: str, cursor: Position, scope):
        """Detect what kind of completion is expected."""
        ...

    def get_suggestions(self, context, scope, prefix) -> list[Suggestion]:
        """Generate suggestions for the context."""
        ...
```

**Main entry point:**

```python
engine = MyCompletionEngine(provider)
result = engine.get_completions(text, cursor)
# result.suggestions: list[Suggestion]
# result.from_pos / result.to_pos: replacement range
```

### BaseMetadataProvider

Protocol for providing domain-specific metadata:

```python
from myfasthtml.core.dsl.base_provider import BaseMetadataProvider

class MyProvider(BaseMetadataProvider):
    # Implement methods to provide metadata for suggestions
    ...
```

### Utilities

**Lark to Simple Mode conversion:**

```python
from myfasthtml.core.dsl.lark_to_simple_mode import lark_to_simple_mode

simple_mode_config = lark_to_simple_mode(grammar_string)
# Returns CodeMirror 5 Simple Mode configuration
```

**Extract completions from grammar:**

```python
from myfasthtml.core.dsl.lark_to_simple_mode import extract_completions_from_grammar

completions = extract_completions_from_grammar(grammar_string)
# Returns: {"keywords": [...], "operators": [...], ...}
```

---

## DataGrid Integration

### DataGridFormattingEditor

The `DataGridFormattingEditor` is a specialized `DslEditor` that integrates the formatting DSL into DataGrid:

```python
class DataGridFormattingEditor(DslEditor):
    def on_content_changed(self):
        """Called when DSL content changes."""
        # 1. Parse DSL
        scoped_rules = parse_dsl(self.get_content())

        # 2. Group by scope
        columns_rules = defaultdict(list)
        rows_rules = defaultdict(list)
        cells_rules = defaultdict(list)
        table_rules = []
        tables_rules = []

        # 3. Dispatch to DataGrid state
        state.columns[i].format = rules
        state.rows[i].format = rules
        state.cell_formats[cell_id] = rules
        state.table_format = rules
        # tables rules stored in DataGridsManager
```

**Column name resolution:**

The editor matches column names by `col_id` first, then `title`:

```python
# DSL
column amount:  # Matches col_id="amount" or title="amount"
    ...
```

**Cell ID resolution:**

```python
# By coordinates
cell (amount, 3):  # Resolved to tcell_grid1-3-0

# By ID
cell tcell_grid1-3-0:  # Used directly
```

### FormattingDSL

Implementation of `DSLDefinition` for the formatting DSL:

```python
from myfasthtml.core.formatting.dsl.definition import FormattingDSL

dsl = FormattingDSL()
dsl.name  # "Formatting DSL"
dsl.get_grammar()  # Returns the Lark grammar
dsl.simple_mode_config  # CodeMirror Simple Mode config
dsl.completions  # Static completion items
```

Used by `DataGridFormattingEditor` to configure the CodeMirror editor.

### Complete Flow in DataGrid

```
1. User opens formatting editor
         │
         ▼
2. DslEditor loads with FormattingDSL configuration
   - Syntax highlighting enabled
   - Autocompletion registered
         │
         ▼
3. User types DSL text
   - Autocompletion triggered on keystrokes
   - FormattingCompletionEngine provides suggestions
         │
         ▼
4. User saves or content changes
   - DataGridFormattingEditor.on_content_changed()
         │
         ▼
5. Parse DSL → ScopedRule[]
         │
         ▼
6. Dispatch rules to state
   - state.columns[i].format = [rule, ...]
   - state.rows[i].format = [rule, ...]
   - state.cell_formats[cell_id] = [rule, ...]
   - state.table_format = [rule, ...]
         │
         ▼
7. DataGrid renders cells
   - mk_body_cell_content() applies formatting
   - FormattingEngine.apply_format(rules, cell_value, row_data)
         │
         ▼
8. CSS + formatted value rendered in cell
```

---

## Developer Reference

### Extension Points

#### Add Custom Style Presets

```python
from myfasthtml.controls.DataGridsManager import DataGridsManager

manager = DataGridsManager.get_instance(session)

manager.add_style_preset("corporate", {
    "background-color": "#003366",
    "color": "#FFFFFF",
    "font-weight": "bold"
})
```

#### Add Custom Formatter Presets

```python
manager.add_formatter_preset("GBP", {
    "type": "number",
    "prefix": "£",
    "thousands_sep": ",",
    "decimal_sep": ".",
    "precision": 2
})
```

#### Implement Lookup Resolver for Enum

When using `EnumFormatter` with `source.type = "datagrid"`, provide a lookup resolver:

```python
def my_lookup_resolver(grid_id: str, value_col: str, display_col: str) -> dict:
    """
    Resolve enum values from another DataGrid.

    Returns:
        Dict mapping value_col values to display_col values
    """
    # Fetch data from grid_id
    # Build mapping {value: display}
    return mapping

engine = FormattingEngine(lookup_resolver=my_lookup_resolver)
```

#### Create Custom Formatter

```python
from myfasthtml.core.formatting.dataclasses import Formatter
from myfasthtml.core.formatting.formatter_resolver import BaseFormatterResolver

@dataclass
class CustomFormatter(Formatter):
    custom_param: str = None

class CustomFormatterResolver(BaseFormatterResolver):
    def resolve(self, formatter: CustomFormatter, value: Any) -> str:
        # Custom formatting logic
        return str(value).upper()

    def apply_preset(self, formatter: Formatter, presets: dict) -> CustomFormatter:
        # Preset application logic
        return formatter

# Register in FormatterResolver
FormatterResolver._resolvers[CustomFormatter] = CustomFormatterResolver()
```

#### Create a New DSL

```python
from myfasthtml.core.dsl.base import DSLDefinition
from myfasthtml.core.dsl.base_completion import BaseCompletionEngine

class MyDSL(DSLDefinition):
    name = "My DSL"

    def get_grammar(self) -> str:
        return """
        start: rule+
        rule: NAME ":" VALUE
        ...
        """

class MyCompletionEngine(BaseCompletionEngine):
    def detect_scope(self, text, line):
        # Scope detection logic
        return scope

    def detect_context(self, text, cursor, scope):
        # Context detection logic
        return context

    def get_suggestions(self, context, scope, prefix):
        # Suggestion generation
        return suggestions
```

### Module Dependency Graph

```
┌─────────────────────────────────────────────────────────┐
│             controls/DataGridFormattingEditor           │
└─────────────────────────────────────────────────────────┘
                         │
        ┌────────────────┼────────────────┐
        ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ formatting/  │  │ formatting/  │  │ formatting/  │
│   dsl/       │  │   dsl/       │  │   engine.py  │
│ __init__.py  │  │ definition.py│  │              │
└──────────────┘  └──────────────┘  └──────────────┘
        │                │                │
        ▼                ▼                ▼
┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ formatting/  │  │ dsl/         │  │ formatting/  │
│   dsl/       │  │ base.py      │  │ condition_   │
│ parser.py    │  │              │  │ evaluator.py │
│ transformer  │  │              │  │ style_       │
│   .py        │  │              │  │ resolver.py  │
│ scopes.py    │  │              │  │ formatter_   │
│ grammar.py   │  │              │  │ resolver.py  │
└──────────────┘  └──────────────┘  └──────────────┘
        │                                   │
        ▼                                   ▼
┌──────────────┐                  ┌──────────────┐
│ formatting/  │                  │ formatting/  │
│ dataclasses  │◄─────────────────│ presets.py   │
│   .py        │                  │              │
└──────────────┘                  └──────────────┘
```

**Dependency layers (bottom to top):**

1. **Dataclasses + Presets**: Core data structures
2. **Resolvers**: Condition evaluation, style, formatting
3. **Engine**: Facade combining resolvers
4. **DSL Grammar + Parser + Transformer**: Text → structured rules
5. **Completion**: Intelligent suggestions
6. **Integration**: DataGrid editor

### Public APIs by Module

| Module | Public API | Description |
|--------|------------|-------------|
| `core/formatting/` | `FormattingEngine` | Main engine for applying rules |
| `core/formatting/dataclasses.py` | `Condition`, `Style`, `Formatter`, `FormatRule` | Core data structures |
| `core/formatting/presets.py` | `DEFAULT_STYLE_PRESETS`, `DEFAULT_FORMATTER_PRESETS` | Built-in presets |
| `core/formatting/dsl/` | `parse_dsl()` | Parse DSL text → `list[ScopedRule]` |
| `core/formatting/dsl/` | `ColumnScope`, `RowScope`, `CellScope`, `TableScope`, `TablesScope` | Scope classes |
| `core/formatting/dsl/` | `DSLSyntaxError`, `DSLValidationError` | Exceptions |
| `core/formatting/dsl/completion/` | `FormattingCompletionEngine` | Autocompletion engine |
| `core/formatting/dsl/completion/` | `DatagridMetadataProvider` | Metadata protocol |
| `core/dsl/` | `DSLDefinition` | Base for creating DSLs |
| `core/dsl/` | `BaseCompletionEngine` | Base for completion engines |

### Key Classes and Responsibilities

| Class | Location | Responsibility |
|-------|----------|----------------|
| `FormattingEngine` | `core/formatting/engine.py` | Apply format rules to cell values |
| `ConditionEvaluator` | `core/formatting/condition_evaluator.py` | Evaluate 13 operators + references |
| `StyleResolver` | `core/formatting/style_resolver.py` | Resolve styles to CSS |
| `FormatterResolver` | `core/formatting/formatter_resolver.py` | Dispatch to type-specific formatters |
| `DSLParser` (internal) | `core/formatting/dsl/parser.py` | Parse DSL text with Lark |
| `DSLTransformer` | `core/formatting/dsl/transformer.py` | Convert AST → dataclasses |
| `FormattingCompletionEngine` | `core/formatting/dsl/completion/FormattingCompletionEngine.py` | Context-aware suggestions |
| `DataGridFormattingEditor` | `controls/DataGridFormattingEditor.py` | Integrate DSL into DataGrid |
| `FormattingDSL` | `core/formatting/dsl/definition.py` | DSL definition for editor |

### Known Limitations

| Limitation | Status | Impact |
|------------|--------|--------|
| `row` parameter for column-level conditions | Not implemented | Cannot reference specific row in column scope |
| AND/OR operators | Not implemented | Use multiple rules or `in`/`between` |
| Format chaining | Not implemented | Only one formatter per rule |
| Cell references in conditions | Partial | Only column references (`col.x`), not arbitrary cells |
| Arithmetic in conditions | Parsed but not evaluated | Expressions like `col.budget * 0.9` not supported |

---

## Summary

The DataGrid Formatting System provides a comprehensive, three-layer architecture for conditional formatting:

1. **Formatting Engine**: Core logic for applying rules (condition evaluation, style resolution, formatting)
2. **DSL Parser**: Human-readable text format for defining rules with scopes and conditions
3. **Autocompletion**: Intelligent suggestions based on context and DataGrid metadata

**Key strengths:**

- **Flexible**: 5 scope levels, 13 operators, 6 formatter types
- **DaisyUI integrated**: 8 style presets that adapt to theme
- **Developer-friendly**: Clean API, programmatic + DSL usage
- **Extensible**: Custom presets, formatters, and DSLs
- **Production-ready**: Robust parsing, error handling, conflict resolution

**Typical workflow:**

1. Write DSL in DataGridFormattingEditor
2. DSL parsed → ScopedRule objects
3. Rules dispatched to column/row/cell formats
4. FormattingEngine applies rules during rendering
5. Cells display with CSS styles + formatted values

For most use cases, the DSL provides sufficient expressiveness without requiring programmatic interaction with the formatting engine.