kodjo/MyFastHtml

Fork 0

Files

Kodjo Sossouvi 0620cb678b I can validate formatting in editor

2026-02-01 21:49:46 +01:00

50 KiB

Raw Blame History

DataGrid Formatting DSL

Introduction

This document describes the Domain Specific Language (DSL) for defining formatting rules in the DataGrid component.

Purpose

The DSL provides a concise, readable way to define conditional formatting rules for cells, rows, and columns. It allows users to:

Apply styles (colors, fonts, decorations) based on cell values
Format values for display (currencies, dates, percentages)
Reference other cells for cross-column/row comparisons

Two Modes

The formatting system offers two interfaces:

Mode	Target Users	Description
Advanced (DSL)	Developers, power users	Text-based rules with Python-like syntax
Basic (GUI)	End users	Visual rule builder (80/20 coverage)

This document focuses on the Advanced DSL mode, which covers 100% of formatting capabilities.

Complete Syntax

Overview

A DSL document consists of one or more scopes, each containing one or more rules:

<scope>:
    <rule>
    <rule>
    ...

<scope>:
    <rule>
    ...

Rules are indented (Python-style) under their scope.

Scopes

Scopes define which cells a rule applies to:

Scope	Syntax	Applies To
Column	`column <name>:`	All cells in the column
Row	`row <index>:`	All cells in the row
Cell (coordinates)	`cell (<col>, <row>):`	Single cell by position
Cell (ID)	`cell <cell_id>:`	Single cell by ID

Column scope:

# Simple column name
column amount:
    style("error") if value < 0

# Column name with spaces (quoted)
column "total amount":
    format("EUR")

# Both syntaxes are valid
column status:
column "status":

Row scope:

# Row by index (0-based)
row 0:
    style("neutral", bold=True)

row 5:
    style("highlight")

Cell scope:

# By coordinates (column, row)
cell (amount, 3):
    style("highlight")

cell ("total amount", 0):
    style("neutral", bold=True)

# By cell ID
cell tcell_grid1-3-2:
    style(background_color="yellow")

Rules

A rule consists of optional style, optional format, and optional condition:

[style(...)] [format(...)] [if <condition>]

At least one of style() or format() must be present.

Examples:

# Style only
style("error")

# Format only
format("EUR")

# Style + Format
style("error") format("EUR")

# With condition
style("error") if value < 0

# All combined
style("error") format("EUR") if value < 0

Style

The style() function applies visual formatting to cells.

Syntax:

style(<preset>)
style(<preset>, <options>)
style(<options>)

Parameters:

Parameter	Type	Description
`preset`	string (positional)	Preset name (optional)
`background_color`	string	Background color
`color`	string	Text color
`bold`	boolean	Bold text
`italic`	boolean	Italic text
`underline`	boolean	Underlined text
`strikethrough`	boolean	Strikethrough text
`font_size`	string	Font size (e.g., "12px", "0.9em")

Available presets (DaisyUI 5):

Preset	Background	Text
`primary`	`var(--color-primary)`	`var(--color-primary-content)`
`secondary`	`var(--color-secondary)`	`var(--color-secondary-content)`
`accent`	`var(--color-accent)`	`var(--color-accent-content)`
`neutral`	`var(--color-neutral)`	`var(--color-neutral-content)`
`info`	`var(--color-info)`	`var(--color-info-content)`
`success`	`var(--color-success)`	`var(--color-success-content)`
`warning`	`var(--color-warning)`	`var(--color-warning-content)`
`error`	`var(--color-error)`	`var(--color-error-content)`

Examples:

# Preset only
style("error")

# Preset with overrides
style("error", bold=True)
style("success", italic=True, underline=True)

# No preset, direct properties
style(color="red", bold=True)
style(background_color="#ffeeee", color="#cc0000")

Format

The format() function transforms cell values for display.

Syntax:

format(<preset>)
format(<preset>, <options>)
format.<type>(<options>)

Using presets:

# Preset only
format("EUR")

# Preset with overrides
format("EUR", precision=3)
format("percentage", precision=0)

Available presets:

Preset	Type	Description
`EUR`	number	Euro currency (1 234,56 €)
`USD`	number	US Dollar ($1,234.56)
`percentage`	number	Percentage (×100, adds %)
`short_date`	date	DD/MM/YYYY
`iso_date`	date	YYYY-MM-DD
`yes_no`	boolean	Yes/No

Using explicit types:

When not using a preset, specify the type explicitly:

format.number(precision=2, suffix=" €", thousands_sep=" ")
format.date(format="%d/%m/%Y")
format.boolean(true_value="Oui", false_value="Non")
format.text(max_length=50, ellipsis="...")
format.enum(source={"draft": "Draft", "published": "Published"})

Type-specific parameters:

format.number:

Parameter	Type	Default	Description
`prefix`	string	`""`	Text before value
`suffix`	string	`""`	Text after value
`thousands_sep`	string	`""`	Thousands separator
`decimal_sep`	string	`"."`	Decimal separator
`precision`	int	`0`	Decimal places
`multiplier`	number	`1`	Multiply before display

format.date:

Parameter	Type	Default	Description
`format`	string	`"%Y-%m-%d"`	strftime pattern

format.boolean:

Parameter	Type	Default	Description
`true_value`	string	`"true"`	Display for true
`false_value`	string	`"false"`	Display for false
`null_value`	string	`""`	Display for null

format.text:

Parameter	Type	Default	Description
`transform`	string	-	`"uppercase"`, `"lowercase"`, `"capitalize"`
`max_length`	int	-	Truncate if exceeded
`ellipsis`	string	`"..."`	Suffix when truncated

format.enum:

Parameter	Type	Default	Description
`source`	object	-	Mapping or datagrid reference
`default`	string	`""`	Label for unknown values

Conditions

Conditions determine when a rule applies.

Syntax:

if <left> <operator> <right>
if <operand> <unary_operator>

Operators:

Operator	Description	Example
`==`	Equal	`value == 0`
`!=`	Not equal	`value != ""`
`<`	Less than	`value < 0`
`<=`	Less or equal	`value <= 100`
`>`	Greater than	`value > 1000`
`>=`	Greater or equal	`value >= 0`
`contains`	String contains	`value contains "error"`
`startswith`	String starts with	`value startswith "ERR"`
`endswith`	String ends with	`value endswith ".pdf"`
`in`	Value in list	`value in ["A", "B", "C"]`
`between`	Value in range	`value between 0 and 100`
`isempty`	Is null/empty	`value isempty`
`isnotempty`	Is not null/empty	`value isnotempty`

Negation:

Use not to negate any condition:

style("error") if not value in ["valid", "approved"]
style("warning") if not value contains "OK"

Case sensitivity:

String comparisons are case-insensitive by default. Use (case) modifier for case-sensitive:

style("error") if value == "Error" (case)
style("warning") if value contains "WARN" (case)

References

References allow comparing with values from other cells.

Syntax:

Reference	Description	Example
`value`	Current cell value	`value < 0`
`col.<name>`	Value from another column (same row)	`value > col.budget`
`col."<name>"`	Column with spaces	`value > col."max amount"`
`row.<index>`	Value from another row (same column)	`value != row.0`
`cell.<col>-<row>`	Specific cell by coordinates	`value == cell.status-0`

Examples:

# Compare with another column
column amount:
    style("error") if value > col.budget
    style("warning") if value > col.budget * 0.9

# Compare with header row
column total:
    style("highlight") if value == row.0

# Compare with specific cell
column status:
    style("success") if value == cell.status-0

Formal Grammar (EBNF)

// Top-level structure
program         : scope+

// Scopes
scope           : scope_header NEWLINE INDENT rule+ DEDENT
scope_header    : column_scope | row_scope | cell_scope
column_scope    : "column" column_name ":"
row_scope       : "row" INTEGER ":"
cell_scope      : "cell" cell_ref ":"
column_name     : NAME | QUOTED_STRING
cell_ref        : "(" column_name "," INTEGER ")" | CELL_ID

// Rules
rule            : (style_expr format_expr? | format_expr style_expr?) condition? NEWLINE
condition       : "if" comparison

// Comparisons
comparison      : "not"? (binary_comp | unary_comp) case_modifier?
binary_comp     : operand operator operand
                | operand "in" list
                | operand "between" operand "and" operand
unary_comp      : operand ("isempty" | "isnotempty")
case_modifier   : "(" "case" ")"

// Operators
operator        : "==" | "!=" | "<" | "<=" | ">" | ">="
                | "contains" | "startswith" | "endswith"

// Operands
operand         : value_ref | column_ref | row_ref | cell_ref_expr | literal | arithmetic
value_ref       : "value"
column_ref      : "col." (NAME | QUOTED_STRING)
row_ref         : "row." INTEGER
cell_ref_expr   : "cell." NAME "-" INTEGER
literal         : STRING | NUMBER | BOOLEAN
arithmetic      : operand ("*" | "/" | "+" | "-") operand
list            : "[" (literal ("," literal)*)? "]"

// Style expression
style_expr      : "style" "(" style_args ")"
style_args      : (QUOTED_STRING ("," style_kwargs)?) | style_kwargs
style_kwargs    : style_kwarg ("," style_kwarg)*
style_kwarg     : NAME "=" (QUOTED_STRING | BOOLEAN | NUMBER)

// Format expression
format_expr     : format_preset | format_typed
format_preset   : "format" "(" QUOTED_STRING ("," format_kwargs)? ")"
format_typed    : "format" "." FORMAT_TYPE "(" format_kwargs? ")"
format_kwargs   : format_kwarg ("," format_kwarg)*
format_kwarg    : NAME "=" (QUOTED_STRING | BOOLEAN | NUMBER | dict)
dict            : "{" (dict_entry ("," dict_entry)*)? "}"
dict_entry      : QUOTED_STRING ":" QUOTED_STRING

// Tokens
FORMAT_TYPE     : "number" | "date" | "boolean" | "text" | "enum"
NAME            : /[a-zA-Z_][a-zA-Z0-9_]*/
QUOTED_STRING   : /"[^"]*"/ | /'[^']*'/
INTEGER         : /[0-9]+/
NUMBER          : /[0-9]+(\.[0-9]+)?/
BOOLEAN         : "True" | "False" | "true" | "false"
CELL_ID         : /tcell_[a-zA-Z0-9_-]+/
NEWLINE         : /\n/
INDENT          : /^[ \t]+/
DEDENT          : // decrease in indentation

Examples

Basic Examples

Highlight negative values:

column amount:
    style("error") if value < 0

Format as currency:

column price:
    format("EUR")

Conditional formatting with multiple rules:

column status:
    style("success") if value == "approved"
    style("warning") if value == "pending"
    style("error") if value == "rejected"

Style header row:

row 0:
    style("neutral", bold=True)

Advanced Examples

Compare with another column:

column actual:
    style("error") if value > col.budget
    style("warning") if value > col.budget * 0.8
    style("success") if value <= col.budget * 0.8

Multiple formatting on same column:

column amount:
    format("EUR")
    style("error") if value < 0
    style("success", bold=True) if value > 10000

Complex conditions:

column score:
    style("error") if value between 0 and 30
    style("warning") if value between 31 and 70
    style("success") if value between 71 and 100

column category:
    style("primary") if value in ["A", "B", "C"]
    style("secondary") if not value in ["A", "B", "C"]

column name:
    style("info") if value startswith "VIP"
    style("neutral") if value isempty

Enum formatting with display mapping:

column status:
    format.enum(source={"draft": "Brouillon", "pending": "En attente", "approved": "Approuvé"}, default="Inconnu")

Complete example - Financial report:

# Header styling
row 0:
    style("neutral", bold=True)

# Amount column
column amount:
    format.number(precision=2, suffix=" €", thousands_sep=" ")
    style("error") if value < 0
    style("success") if value > col.target

# Percentage column
column progress:
    format("percentage")
    style("error") if value < 0.5
    style("warning") if value between 0.5 and 0.8
    style("success") if value > 0.8

# Status column
column status:
    format.enum(source={"draft": "Draft", "review": "In Review", "approved": "Approved", "rejected": "Rejected"})
    style("neutral") if value == "draft"
    style("info") if value == "review"
    style("success") if value == "approved"
    style("error") if value == "rejected"

# Date column
column created_at:
    format.date(format="%d %b %Y")

# Highlight specific cell
cell (amount, 10):
    style("accent", bold=True)

Autocompletion

The DSL editor provides context-aware autocompletion to help users write rules efficiently.

How It Works

┌─────────────────────────────────────────────────────────────┐
│                     CodeMirror Editor                        │
│                                                              │
│  User types: style("err|                                     │
│                        ▲                                     │
│                        │ cursor position                     │
└────────────────────────┼────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│              Autocompletion Request (HTMX)                   │
│  { "text": "style(\"err", "cursor": 11, "context": "..." }  │
└─────────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│                   Python Backend                             │
│                                                              │
│  1. Parse partial input                                      │
│  2. Determine context (inside style(), first arg)           │
│  3. Filter matching presets: ["error"]                      │
│  4. Return suggestions                                       │
└─────────────────────────────────────────────────────────────┘
                         │
                         ▼
┌─────────────────────────────────────────────────────────────┐
│              Suggestions Dropdown                            │
│  ┌─────────────────────────────────────────────────────┐    │
│  │ error     - Red background for errors               │    │
│  └─────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘

Completion Contexts

Context	Trigger	Suggestions
Scope keyword	Start of line	`column`, `row`, `cell`
Column name	After `column`	Column names from DataGrid
Style preset	Inside `style("`	Style presets
Style parameter	Inside `style(... ,`	`bold`, `italic`, `color`, etc.
Format preset	Inside `format("`	Format presets
Format type	After `format.`	`number`, `date`, `boolean`, `text`, `enum`
Format parameter	Inside `format.number(`	Type-specific params
Operator	After operand	`==`, `<`, `contains`, etc.
Column reference	After `col.`	Column names
Keyword	After condition	`if`, `not`, `and`, `or`

Example Completion Flow

User types     │ Suggestions
───────────────┼────────────────────────────────────
col            │ column
column         │ [column names from grid]
column amount: │ (new line, indent)
    st         │ style
    style(     │ "error", "warning", "success", ...
    style("e   │ "error"
    style("err │ "error" (filtered)
    style("error", │ bold=, italic=, color=, ...
    style("error", b │ bold=
    style("error", bold=│ True, False
    style("error", bold=True) │ format(, if
    style("error", bold=True) if │ value, col., row., not
    style("error", bold=True) if value │ ==, !=, <, >, in, ...
    style("error", bold=True) if value < │ [number input]

CodeMirror Integration

DaisyUI Theme

The editor uses DaisyUI CSS variables for consistent theming:

import { EditorView } from '@codemirror/view'
import { HighlightStyle, syntaxHighlighting } from '@codemirror/language'
import { tags } from '@lezer/highlight'

// Editor theme (container, cursor, selection)
const daisyEditorTheme = EditorView.theme({
  '&': {
    backgroundColor: 'var(--color-base-100)',
    color: 'var(--color-base-content)',
    fontSize: '14px',
  },
  '.cm-content': {
    fontFamily: 'var(--font-mono, ui-monospace, monospace)',
    padding: '8px 0',
  },
  '.cm-line': {
    padding: '0 8px',
  },
  '.cm-cursor': {
    borderLeftColor: 'var(--color-primary)',
    borderLeftWidth: '2px',
  },
  '.cm-selectionBackground': {
    backgroundColor: 'color-mix(in srgb, var(--color-primary) 20%, transparent)',
  },
  '&.cm-focused .cm-selectionBackground': {
    backgroundColor: 'color-mix(in srgb, var(--color-primary) 30%, transparent)',
  },
  '.cm-gutters': {
    backgroundColor: 'var(--color-base-200)',
    color: 'var(--color-base-content)',
    opacity: '0.5',
    border: 'none',
  },
  '.cm-activeLineGutter': {
    backgroundColor: 'var(--color-base-300)',
  },
  '.cm-activeLine': {
    backgroundColor: 'color-mix(in srgb, var(--color-base-content) 5%, transparent)',
  },
})

// Syntax highlighting
const daisyHighlightStyle = HighlightStyle.define([
  // Keywords: column, row, cell, if, not, and, or
  { tag: tags.keyword, color: 'var(--color-primary)', fontWeight: 'bold' },

  // Functions: style, format
  { tag: tags.function(tags.variableName), color: 'var(--color-secondary)' },

  // Strings: "error", "EUR", "approved"
  { tag: tags.string, color: 'var(--color-success)' },

  // Numbers: 0, 100, 3.14
  { tag: tags.number, color: 'var(--color-accent)' },

  // Operators: ==, <, >, contains, in
  { tag: tags.operator, color: 'var(--color-warning)' },
  { tag: tags.compareOperator, color: 'var(--color-warning)' },

  // Booleans: True, False
  { tag: tags.bool, color: 'var(--color-info)' },

  // Property names: bold=, precision=
  { tag: tags.propertyName, color: 'var(--color-base-content)', opacity: '0.8' },

  // Comments (if supported later)
  { tag: tags.comment, color: 'var(--color-base-content)', opacity: '0.5', fontStyle: 'italic' },

  // Invalid/errors
  { tag: tags.invalid, color: 'var(--color-error)', textDecoration: 'underline wavy' },
])

// Combined extension
export const daisyTheme = [
  daisyEditorTheme,
  syntaxHighlighting(daisyHighlightStyle),
]

Lezer Grammar (Translated from lark)

The lark grammar is translated to Lezer format for client-side parsing:

// formatrules.grammar (Lezer)

@top Program { scope+ }

@skip { space | newline }

scope {
  scopeHeader ":" indent rule+ dedent
}

scopeHeader {
  ColumnScope | RowScope | CellScope
}

ColumnScope { kw<"column"> columnName }
RowScope { kw<"row"> Integer }
CellScope { kw<"cell"> cellRef }

columnName { Name | QuotedString }
cellRef { "(" columnName "," Integer ")" | CellId }

rule {
  (styleExpr formatExpr? | formatExpr styleExpr?) condition?
}

condition { kw<"if"> comparison }

comparison {
  not? (binaryComp | unaryComp) caseModifier?
}

not { kw<"not"> }

binaryComp {
  operand compareOp operand |
  operand kw<"in"> list |
  operand kw<"between"> operand kw<"and"> operand
}

unaryComp {
  operand (kw<"isempty"> | kw<"isnotempty">)
}

caseModifier { "(" kw<"case"> ")" }

compareOp { "==" | "!=" | "<" | "<=" | ">" | ">=" | kw<"contains"> | kw<"startswith"> | kw<"endswith"> }

operand {
  ValueRef | ColumnRef | RowRef | CellRefExpr | literal | ArithmeticExpr
}

ValueRef { kw<"value"> }
ColumnRef { kw<"col"> "." (Name | QuotedString) }
RowRef { kw<"row"> "." Integer }
CellRefExpr { kw<"cell"> "." Name "-" Integer }

literal { QuotedString | Number | Boolean }

ArithmeticExpr { operand !arith arithmeticOp operand }
arithmeticOp { "*" | "/" | "+" | "-" }

list { "[" (literal ("," literal)*)? "]" }

styleExpr { kw<"style"> "(" styleArgs ")" }
styleArgs { (QuotedString ("," styleKwargs)?) | styleKwargs }
styleKwargs { styleKwarg ("," styleKwarg)* }
styleKwarg { Name "=" (QuotedString | Boolean | Number) }

formatExpr { formatPreset | formatTyped }
formatPreset { kw<"format"> "(" QuotedString ("," formatKwargs)? ")" }
formatTyped { kw<"format"> "." formatType "(" formatKwargs? ")" }
formatType { @specialize[@name=FormatType]<Name, "number" | "date" | "boolean" | "text" | "enum"> }
formatKwargs { formatKwarg ("," formatKwarg)* }
formatKwarg { Name "=" (QuotedString | Boolean | Number | Dict) }

Dict { "{" (dictEntry ("," dictEntry)*)? "}" }
dictEntry { QuotedString ":" QuotedString }

kw<term> { @specialize[@name={term}]<Name, term> }

@tokens {
  Name { @asciiLetter (@asciiLetter | @digit | "_")* }
  QuotedString { '"' (!["\\] | "\\" _)* '"' | "'" (!['\\] | "\\" _)* "'" }
  Integer { @digit+ }
  Number { @digit+ ("." @digit+)? }
  Boolean { "True" | "False" | "true" | "false" }
  CellId { "tcell_" ((@asciiLetter | @digit | "_" | "-"))+ }

  space { " " | "\t" }
  newline { "\n" | "\r\n" }

  @precedence { CellId, Name }
  @precedence { Number, Integer }
}

@precedence {
  arith @left
}

@detectDelim

Editor Setup

import { EditorState } from '@codemirror/state'
import { EditorView, keymap, lineNumbers, drawSelection } from '@codemirror/view'
import { defaultKeymap, indentWithTab } from '@codemirror/commands'
import { indentOnInput, bracketMatching } from '@codemirror/language'
import { autocompletion } from '@codemirror/autocomplete'
import { linter } from '@codemirror/lint'

import { formatRulesLanguage } from './formatrules-lang'  // Generated from grammar
import { daisyTheme } from './daisy-theme'
import { formatRulesCompletion } from './completion'
import { formatRulesLinter } from './linter'

function createFormatRulesEditor(container, initialValue, options = {}) {
  const { onChange, columns = [], stylePresets = [], formatPresets = [] } = options

  const state = EditorState.create({
    doc: initialValue,
    extensions: [
      lineNumbers(),
      drawSelection(),
      indentOnInput(),
      bracketMatching(),
      keymap.of([...defaultKeymap, indentWithTab]),

      // Language support
      formatRulesLanguage(),

      // Theme
      daisyTheme,

      // Autocompletion with context
      autocompletion({
        override: [formatRulesCompletion({ columns, stylePresets, formatPresets })],
      }),

      // Linting (validation)
      linter(formatRulesLinter()),

      // Change callback
      EditorView.updateListener.of((update) => {
        if (update.docChanged && onChange) {
          onChange(update.state.doc.toString())
        }
      }),
    ],
  })

  return new EditorView({
    state,
    parent: container,
  })
}

Technical Choices

Why lark (not pyparsing)

Both lark and pyparsing are mature Python parsing libraries. We chose lark for the following reasons:

Criterion	lark	pyparsing
Grammar definition	Declarative EBNF string	Python combinators
Portability to Lezer	Direct translation possible	Manual rewrite required
Grammar readability	Standard BNF-like notation	Embedded in Python code
Maintenance	Single grammar source	Two separate grammars to sync

The key factor: We need the same grammar for both:

Server-side (Python): Validation and execution
Client-side (JavaScript): Syntax highlighting and autocompletion

With lark's declarative EBNF grammar, we can translate it to Lezer (CodeMirror 6's parser system) with minimal effort. With pyparsing, the grammar is embedded in Python logic, making extraction and translation significantly harder.

Example comparison:

# lark - declarative grammar (easy to translate)
grammar = """
    scope: "column" NAME ":" NEWLINE INDENT rule+ DEDENT
    NAME: /[a-zA-Z_][a-zA-Z0-9_]*/
"""

// Lezer - similar structure
@top Program { scope+ }
scope { "column" Name ":" newline indent rule+ dedent }
Name { @asciiLetter (@asciiLetter | @digit | "_")* }

# pyparsing - grammar in Python code (hard to translate)
NAME = Word(alphas, alphanums + "_")
scope = Keyword("column") + NAME + ":" + LineEnd() + IndentedBlock(rule)

Why CodeMirror 6

For the web-based DSL editor, we chose CodeMirror 6:

Criterion	CodeMirror 6	Monaco	Ace
Bundle size	~150KB	~2MB	~300KB
Custom languages	Lezer parser	Monarch tokenizer	Custom modes
Theming	CSS variables	JSON themes	CSS
DaisyUI integration	Native (CSS vars)	Requires mapping	Partial
Architecture	Modern, modular	Monolithic	Legacy

CodeMirror 6's use of CSS variables makes it trivial to integrate with DaisyUI's theming system, ensuring visual consistency across the application.

Shared Grammar Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Grammar Source (EBNF)                     │
│                    lark format in Python                     │
└─────────────────────────────────────────────────────────────┘
                              │
              ┌───────────────┴───────────────┐
              ▼                               ▼
┌─────────────────────────┐     ┌─────────────────────────────┐
│     Python (Server)      │     │    JavaScript (Client)       │
│     lark parser          │     │    Lezer parser              │
│                          │     │    (translated from lark)    │
│  • Validation            │     │                              │
│  • Execution             │     │  • Syntax highlighting       │
│  • Error messages        │     │  • Autocompletion            │
│  • Preset resolution     │     │  • Error markers             │
└─────────────────────────┘     └─────────────────────────────┘

Autocompletion API

Overview

The autocompletion system provides context-aware suggestions for the DSL editor. Intelligence runs server-side via a REST API, while the editor (CodeMirror 5) handles display and user interaction.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    CodeMirror Editor                        │
│  User types: style("err|                                    │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│              REST API: /myfasthtml/completions           │
│  Request: { "text": "...", "cursor": {"line": 1, "ch": 15} }│
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│              Python: Autocompletion Engine                  │
│                                                             │
│  1. Detect SCOPE from previous lines                        │
│     - Find scope declaration (column/row/cell)              │
│     - Extract scope name (e.g., "amount" for column)        │
│                                                             │
│  2. Detect COMPLETION CONTEXT at cursor position            │
│     - Analyze current line up to cursor                     │
│     - Determine what kind of token is expected              │
│                                                             │
│  3. Generate suggestions                                    │
│     - Query DatagridMetadataProvider if needed              │
│     - Filter suggestions by prefix                          │
│                                                             │
│  4. Return CodeMirror-compatible response                   │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│  Response: {                                                │
│    "from": {"line": 1, "ch": 12},                           │
│    "to": {"line": 1, "ch": 15},                             │
│    "suggestions": [{"label": "error", "detail": "..."}]     │
│  }                                                          │
└─────────────────────────────────────────────────────────────┘

Scope Detection

Before determining completion context, the engine must identify the current scope by scanning backwards from the cursor position to find the most recent scope declaration.

Scope Types:

Scope Type	Pattern	Extracted Info
Column	`column <name>:` or `column "<name>":`	Column name
Row	`row <index>:`	Row index
Cell	`cell (<col>, <row>):`	Column name + row index

Algorithm:

Start from cursor line
Scan backwards for a non-indented line matching scope pattern
Extract scope type and parameters
If no scope found, cursor is at top level (suggest scope keywords)

Example:

column status:           # ← Scope: ColumnScope("status")
    style("error") if |  # ← Cursor here

When the engine needs column values for OPERATOR_VALUE context, it uses the detected scope to call provider.get_column_values("status").

DatagridMetadataProvider

A Protocol providing access to DataGrid metadata for context-aware suggestions:

class DatagridMetadataProvider(Protocol):
    """Provides DataGrid metadata for autocompletion."""

    def list_tables(self) -> list[str]:
        """List of available DataGrids (namespace.name format)."""
        ...

    def list_columns(self, table_name: str) -> list[str]:
        """Column names for a specific DataGrid."""
        ...

    def list_column_values(self, table_name: str, column_name: str) -> list[Any]:
        """Distinct values for a column in the current scope."""
        ...

    def get_row_count(self, table_name: str) -> int:
        """Number of rows in a DataGrid."""
        ...

    def list_style_presets(self) -> list[str]:
        """List of available style preset names."""
        ...

    def list_format_presets(self) -> list[str]:
        """List of available format preset names."""
        ...

Notes:

DatagridMetadataProvider is a Protocol (structural typing), not an abstract base class
DataGrid names follow the pattern namespace.name (multi-level namespaces supported)
The provider is passed at initialization, not with each API call
Column values are fetched lazily when the scope is detected
DataGridsManager implements this Protocol

API Interface

Request:

{
  "text": "column amount:\n    style(\"err",
  "cursor": {"line": 1, "ch": 15}
}

text: Full DSL document content
cursor.line: 0-based line number
cursor.ch: 0-based character position in line

Response:

{
  "from": {"line": 1, "ch": 12},
  "to": {"line": 1, "ch": 15},
  "suggestions": [
    {"label": "error", "detail": "Red background for errors"},
    {"label": "warning", "detail": "Yellow background for warnings"}
  ]
}

from/to: Range to replace (word boundaries using delimiters)
suggestions: List of completion items with label and optional detail

Completion Contexts

Context	Trigger	Suggestions
`SCOPE_KEYWORD`	Start of non-indented line	`column`, `row`, `cell`
`COLUMN_NAME`	After `column`	Column names from DataGrid
`ROW_INDEX`	After `row`	First 10 indices + last index
`CELL_START`	After `cell`	`(`
`CELL_COLUMN`	After `cell (`	Column names
`CELL_ROW`	After `cell (col,`	First 10 indices + last index
`RULE_START`	Start of indented line (after scope)	`style(`, `format(`, `format.`
`STYLE_ARGS`	After `style(` (no quote)	Presets with quotes + named params (`bold=`, `color=`, ...)
`STYLE_PRESET`	Inside `style("`	Style presets
`STYLE_PARAM`	After comma in `style(...)`	`bold=`, `italic=`, `underline=`, `strikethrough=`, `color=`, `background_color=`, `font_size=`
`FORMAT_PRESET`	Inside `format("`	Format presets
`FORMAT_TYPE`	After `format.`	`number`, `date`, `boolean`, `text`, `enum`
`FORMAT_PARAM_DATE`	Inside `format.date(`	`format=`
`FORMAT_PARAM_TEXT`	Inside `format.text(`	`transform=`, `max_length=`, `ellipsis=`
`AFTER_STYLE_OR_FORMAT`	After `)` of style/format	`style(`, `format(`, `format.`, `if`
`CONDITION_START`	After `if`	`value`, `col.`, `not`
`CONDITION_AFTER_NOT`	After `if not`	`value`, `col.`
`COLUMN_REF`	After `col.`	Column names
`COLUMN_REF_QUOTED`	After `col."`	Column names (with closing quote)
`OPERATOR`	After operand (`value`, literal, ref)	`==`, `!=`, `<`, `<=`, `>`, `>=`, `contains`, `startswith`, `endswith`, `in`, `between`, `isempty`, `isnotempty`
`OPERATOR_VALUE`	After comparison operator	`col.`, `True`, `False`, column values (from detected scope)
`BETWEEN_AND`	After `between X`	`and`
`BETWEEN_VALUE`	After `between X and`	Same as `OPERATOR_VALUE`
`IN_LIST_START`	After `in`	`[`
`IN_LIST_VALUE`	Inside `[` or after `,` in list	Column values (from detected scope)
`BOOLEAN_VALUE`	After `bold=`, `italic=`, etc.	`True`, `False`
`COLOR_VALUE`	After `color=`, `background_color=`	CSS colors + DaisyUI variables
`DATE_FORMAT_VALUE`	After `format=` in format.date	`"%Y-%m-%d"`, `"%d/%m/%Y"`, `"%m/%d/%Y"`, `"%d %b %Y"`
`TRANSFORM_VALUE`	After `transform=`	`"uppercase"`, `"lowercase"`, `"capitalize"`
`COMMENT`	After `#`	No suggestions

Context Detection Flow

column |           → SCOPE_KEYWORD: column, row, cell
column a|          → COLUMN_NAME: amount, active, ...
column amount:
    |              → RULE_START: style(, format(, format.
    style(|        → STYLE_ARGS: "error", "warning", ..., bold=, color=, ...
    style("|       → STYLE_PRESET: error, warning, success, ...
    style("e|      → STYLE_PRESET (filtered): error
    style("error", |  → STYLE_PARAM: bold=, italic=, color=, ...
    style("error", bold=|  → BOOLEAN_VALUE: True, False
    style("error", color=|  → COLOR_VALUE: red, blue, var(--color-primary), ...
    style("error")|  → AFTER_STYLE_OR_FORMAT: format(, format., if
    style("error") if |  → CONDITION_START: value, col., not
    style("error") if not |  → CONDITION_AFTER_NOT: value, col.
    style("error") if value |  → OPERATOR: ==, <, >, contains, in, ...
    style("error") if value == |  → OPERATOR_VALUE: col., True, False, "draft", "pending", ...
    style("error") if value in |  → IN_LIST_START: [
    style("error") if value in [|  → IN_LIST_VALUE: "draft", "pending", ...
    style("error") if value in ["draft", |  → IN_LIST_VALUE: "pending", ...
    style("error") if value between |  → OPERATOR_VALUE
    style("error") if value between 0 |  → BETWEEN_AND: and
    style("error") if value between 0 and |  → BETWEEN_VALUE

row |              → ROW_INDEX: 0, 1, 2, ..., 9, 149 (if 150 rows)
row 0:
    |              → RULE_START

cell |             → CELL_START: (
cell (|            → CELL_COLUMN: amount, status, ...
cell (amount, |    → CELL_ROW: 0, 1, 2, ..., 9, 149
cell (amount, 3):
    |              → RULE_START

    format.|       → FORMAT_TYPE: number, date, boolean, text, enum
    format.date(|  → FORMAT_PARAM_DATE: format=
    format.date(format=|  → DATE_FORMAT_VALUE: "%Y-%m-%d", "%d/%m/%Y", ...
    format.text(|  → FORMAT_PARAM_TEXT: transform=, max_length=, ellipsis=
    format.text(transform=|  → TRANSFORM_VALUE: "uppercase", "lowercase", "capitalize"

Suggestions Data

Style Presets (DaisyUI 5):

Label	Detail
`primary`	Primary theme color
`secondary`	Secondary theme color
`accent`	Accent theme color
`neutral`	Neutral theme color
`info`	Info (blue)
`success`	Success (green)
`warning`	Warning (yellow)
`error`	Error (red)

Format Presets:

Label	Detail
`EUR`	Euro currency (1 234,56 €)
`USD`	US Dollar ($1,234.56)
`percentage`	Percentage (×100, adds %)
`short_date`	DD/MM/YYYY
`iso_date`	YYYY-MM-DD
`yes_no`	Yes/No

CSS Colors (subset):

red, blue, green, yellow, orange, purple, pink, gray, black, white

DaisyUI Color Variables:

var(--color-primary), var(--color-secondary), var(--color-accent), var(--color-neutral), var(--color-info), var(--color-success), var(--color-warning), var(--color-error), var(--color-base-100), var(--color-base-200), var(--color-base-300), var(--color-base-content)

Date Format Patterns:

Label	Detail
`"%Y-%m-%d"`	ISO format (2026-01-29)
`"%d/%m/%Y"`	European (29/01/2026)
`"%m/%d/%Y"`	US format (01/29/2026)
`"%d %b %Y"`	Short month (29 Jan 2026)
`"%d %B %Y"`	Full month (29 January 2026)

Word Boundary Detection

To determine the range to replace (from/to), use delimiters:

Quotes: ", '
Parentheses: (, )
Brackets: [, ]
Braces: {, }
Operators: =, ,, :, <, >, !
Whitespace: space, tab, newline

Exclusions (Not Implemented)

The following features are excluded from autocompletion for simplicity:

(case) modifier for case-sensitive comparisons
Arithmetic expressions (col.budget * 0.9)
format.enum source configuration
format.number parameters (free input)
format.boolean parameters (free input)

Implementation Status

Component	Status	Location
DSL Parser
DSL Grammar (lark)	✅ Implemented	`src/myfasthtml/core/formatting/dsl/grammar.py`
DSL Parser	✅ Implemented	`src/myfasthtml/core/formatting/dsl/parser.py`
DSL Transformer	✅ Implemented	`src/myfasthtml/core/formatting/dsl/transformer.py`
Scope Dataclasses	✅ Implemented	`src/myfasthtml/core/formatting/dsl/scopes.py`
Exceptions	✅ Implemented	`src/myfasthtml/core/formatting/dsl/exceptions.py`
Public API (`parse_dsl()`)	✅ Implemented	`src/myfasthtml/core/formatting/dsl/__init__.py`
Unit Tests (Parser)	✅ ~35 tests	`tests/core/formatting/dsl/test_dsl_parser.py`
Autocompletion
DatagridMetadataProvider	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/provider.py`
Scope Detector	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/contexts.py`
Context Detector	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/contexts.py`
Suggestions Generator	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/suggestions.py`
Completion Engine	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/engine.py`
Presets	✅ Implemented	`src/myfasthtml/core/formatting/dsl/completion/presets.py`
Unit Tests (Completion)	✅ ~50 tests	`tests/core/formatting/dsl/test_completion.py`
REST Endpoint	✅ Implemented	`src/myfasthtml/core/utils.py` → `/myfasthtml/completions`
Client-side
Lark to Lezer converter	✅ Implemented	`src/myfasthtml/core/dsl/lark_to_lezer.py`
CodeMirror 5 assets	✅ Implemented	`assets/codemirror.min.js`, `show-hint.min.js`
DslEditor control	✅ Implemented	`src/myfasthtml/controls/DslEditor.py`
initDslEditor() JS	✅ Implemented	`assets/myfasthtml.js` (static completions only)
Dynamic completions (server calls)	✅ Implemented	`assets/myfasthtml.js` → `/myfasthtml/completions`
DaisyUI Theme	⭕ Deferred	-
Syntax Validation (Linting)
REST Endpoint	✅ Implemented	`src/myfasthtml/core/utils.py` → `/myfasthtml/validations`
CodeMirror lint integration	✅ Implemented	`assets/myfasthtml.js` → `dslLint()`
Lint CSS/JS assets	✅ Added	`assets/lint.min.js`, `assets/lint.css`
Warnings (semantic validation)	⭕ Future	Not yet implemented (see note below)

Implementation Notes

Dependencies

The DSL module requires:

lark - Python parsing library (added to pyproject.toml)

Key Implementation Details

Indentation handling (lark Indenter):

The DSL uses Python-style indentation. Lark's Indenter class requires the newline token (_NL) to include trailing whitespace so it can detect indentation levels:

_NL: /(\r?\n[\t ]*)+/

This means _NL captures the newline AND the following spaces/tabs. The Indenter uses token.rsplit('\n', 1)[1] to extract the indentation string.

Comment handling:

Comments (# ...) are pre-processed in parser.py before parsing. Comment lines are replaced with empty strings to preserve line numbers in error messages:

lines = ["" if line.strip().startswith("#") else line for line in lines]

The grammar's %ignore COMMENT directive handles inline comments (at end of lines).

Module Structure:

src/myfasthtml/core/formatting/dsl/
├── __init__.py          # Public API: parse_dsl()
├── grammar.py           # Lark EBNF grammar string
├── parser.py            # DSLParser class with Indenter
├── transformer.py       # AST → dataclass conversion
├── scopes.py            # ColumnScope, RowScope, CellScope, ScopedRule
├── exceptions.py        # DSLSyntaxError, DSLValidationError
├── definition.py        # FormattingDSL class for DslEditor
└── completion/          # Autocompletion module
    ├── __init__.py
    ├── contexts.py      # Context enum, detect_scope(), detect_context()
    ├── suggestions.py   # get_suggestions() for each context
    ├── engine.py        # FormattingCompletionEngine, get_completions()
    ├── presets.py       # Static suggestions (keywords, operators, colors)
    └── provider.py      # DatagridMetadataProvider protocol

Output structure:

parse_dsl() returns a list of ScopedRule objects:

@dataclass
class ScopedRule:
    scope: ColumnScope | RowScope | CellScope
    rule: FormatRule  # From core/formatting/dataclasses.py

Syntax Validation (Linting)

The DSL editor provides real-time syntax validation via CodeMirror's lint addon.

Architecture:

┌─────────────────────────────────────────────────────────────┐
│                    CodeMirror Editor                        │
│  User types invalid syntax                                  │
└─────────────────────────────────────────────────────────────┘
                              │ (debounced, 500ms)
                              ▼
┌─────────────────────────────────────────────────────────────┐
│              REST API: /myfasthtml/validations              │
│  Request: { e_id, text, line, ch }                          │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│              Python: parse_dsl(text)                        │
│  - If OK: return {"errors": []}                             │
│  - If DSLSyntaxError: return error with line/column/message │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│              CodeMirror Lint Display                        │
│  - Gutter marker (error icon)                               │
│  - Underline on error position                              │
│  - Tooltip with error message on hover                      │
└─────────────────────────────────────────────────────────────┘

API Response Format:

{
  "errors": [
    {
      "line": 5,
      "column": 12,
      "message": "Expected 'column', 'row' or 'cell'",
      "severity": "error"
    }
  ]
}

Note: Line and column are 1-based (from lark parser). The JavaScript client converts to 0-based for CodeMirror.

Future: Warnings Support

Currently, only syntax errors (parse failures) are reported. Semantic warnings are planned for a future release:

Warning Type	Example	Status
Unknown style preset	`style("unknown_preset")`	⭕ Future
Unknown format preset	`format("invalid")`	⭕ Future
Unknown parameter	`style(invalid_param=True)`	⭕ Future
Non-existent column reference	`col.nonexistent`	⭕ Future

These warnings would be non-blocking (DSL still parses) but displayed with a different severity in the editor.

Next Steps

Add lark to dependencies in pyproject.toml Done
~~Implement autocompletion API~~ Done
~~Translate lark grammar to Lezer~~ Done (lark_to_lezer.py)
~~Build CodeMirror extension~~ Done (DslEditor.py + initDslEditor())
Integrate with DataGrid - connect DSL output to formatting engine Done
~~Implement dynamic client-side completions~~ Done
- Engine ID: DSLDefinition.get_id() passed via completionEngineId in JS config
- Trigger: Hybrid (Ctrl+Space + auto after . ( " and space)
- Files modified:
  - src/myfasthtml/controls/DslEditor.py:134 - Added completionEngineId
  - src/myfasthtml/assets/myfasthtml.js:2138-2300 - Async fetch to /myfasthtml/completions

50 KiB Raw Blame History Unescape Escape

DataGrid Formatting DSL

Introduction

Purpose

Two Modes

Complete Syntax

Overview

Scopes

Rules

Style

Format

Conditions

References

Formal Grammar (EBNF)

Examples

Basic Examples

Advanced Examples

Autocompletion

How It Works

Completion Contexts

Example Completion Flow

CodeMirror Integration

DaisyUI Theme

Lezer Grammar (Translated from lark)

Editor Setup

Technical Choices

Why lark (not pyparsing)

Why CodeMirror 6

Shared Grammar Architecture

Autocompletion API

Overview

Architecture

Scope Detection

DatagridMetadataProvider

API Interface

Completion Contexts

Context Detection Flow

Suggestions Data

Word Boundary Detection

Exclusions (Not Implemented)

Implementation Status

Implementation Notes

Dependencies

Key Implementation Details

Syntax Validation (Linting)

Next Steps

50 KiB

Raw Blame History