diff --git a/docs/DataGrid Formatting DSL.md b/docs/DataGrid Formatting DSL.md index 3139164..67c384e 100644 --- a/docs/DataGrid Formatting DSL.md +++ b/docs/DataGrid Formatting DSL.md @@ -700,11 +700,99 @@ export const daisyTheme = [ ] ``` -### Lezer Grammar (Translated from lark) +### CodeMirror Simple Mode (Generated from lark) -The lark grammar is translated to Lezer format for client-side parsing: +The lark grammar terminals are extracted and converted to CodeMirror 5 Simple Mode format for syntax highlighting: ```javascript +// Generated from lark grammar terminals +CodeMirror.defineSimpleMode("formatting-dsl", { + start: [ + // Comments + {regex: /#.*/, token: "comment"}, + + // Keywords + {regex: /\b(?:column|row|cell|if|not|and|or|in|between|case)\b/, token: "keyword"}, + + // Built-in functions + {regex: /\b(?:style|format)\b/, token: "builtin"}, + + // Operators + {regex: /\b(?:contains|startswith|endswith|isempty|isnotempty)\b/, token: "operator"}, + {regex: /==|!=|<=|>=|<|>/, token: "operator"}, + + // References + {regex: /\b(?:value|col)\b/, token: "variable-2"}, + + // Booleans + {regex: /\b(?:True|False|true|false)\b/, token: "atom"}, + + // Numbers + {regex: /\b\d+(?:\.\d+)?\b/, token: "number"}, + + // Strings + {regex: /"(?:[^\\]|\\.)*?"/, token: "string"}, + {regex: /'(?:[^\\]|\\.)*?'/, token: "string"}, + + // Cell IDs + {regex: /\btcell_[a-zA-Z0-9_-]+\b/, token: "variable-3"}, + ] +}); +``` + +**Token classes**: +- `comment` - Comments (`#`) +- `keyword` - Keywords (`column`, `row`, `cell`, `if`, `not`, `and`, `or`, `in`, `between`, `case`) +- `builtin` - Built-in functions (`style`, `format`) +- `operator` - Comparison/string operators (`==`, `<`, `contains`, etc.) +- `variable-2` - Special variables (`value`, `col`) +- `atom` - Literals (`True`, `False`) +- `number` - Numeric literals +- `string` - String literals +- `variable-3` - Cell IDs (`tcell_*`) + +These classes are styled via CSS using DaisyUI color variables for automatic theme support. + +### Editor Setup (CodeMirror 5) + +```javascript +function initDslEditor(config) { + const editor = CodeMirror(container, { + value: initialValue, + mode: "formatting-dsl", // Use generated Simple Mode + lineNumbers: true, + extraKeys: { + "Ctrl-Space": "autocomplete" + }, + hintOptions: { + hint: dslHint, // Server-side completions + completeSingle: false + }, + gutters: ["CodeMirror-linenumbers", "CodeMirror-lint-markers"], + lint: { + getAnnotations: dslLint, // Server-side validation + async: true + } + }); + + return editor; +} +``` + +**Note**: The Simple Mode provides instant syntax highlighting (approximative, lexer-level). Server-side validation via `/myfasthtml/validations` provides accurate error reporting. + +--- + +## CodeMirror Integration (Deprecated - CodeMirror 6) + +The following sections describe the original approach using CodeMirror 6 + Lezer, which was abandoned due to bundler requirements incompatible with FastHTML. + +### ~~Lezer Grammar (Translated from lark)~~ (Deprecated) + +~~The lark grammar is translated to Lezer format for client-side parsing:~~ + +```javascript +// DEPRECATED: CodeMirror 6 + Lezer approach (requires bundler) // formatrules.grammar (Lezer) @top Program { scope+ } @@ -807,9 +895,10 @@ kw { @specialize[@name={term}] } @detectDelim ``` -### Editor Setup +### ~~Editor Setup~~ (Deprecated - CodeMirror 6) ```javascript +// DEPRECATED: CodeMirror 6 approach (requires bundler, incompatible with FastHTML) import { EditorState } from '@codemirror/state' import { EditorView, keymap, lineNumbers, drawSelection } from '@codemirror/view' import { defaultKeymap, indentWithTab } from '@codemirror/commands' @@ -875,54 +964,31 @@ Both `lark` and `pyparsing` are mature Python parsing libraries. We chose **lark | Criterion | lark | pyparsing | |-----------|------|-----------| | **Grammar definition** | Declarative EBNF string | Python combinators | -| **Portability to Lezer** | Direct translation possible | Manual rewrite required | +| **Regex extraction** | Easy to extract terminals | Embedded in Python logic | | **Grammar readability** | Standard BNF-like notation | Embedded in Python code | | **Maintenance** | Single grammar source | Two separate grammars to sync | **The key factor**: We need the same grammar for both: 1. **Server-side** (Python): Validation and execution -2. **Client-side** (JavaScript): Syntax highlighting and autocompletion +2. **Client-side** (JavaScript): Syntax highlighting -With lark's declarative EBNF grammar, we can translate it to Lezer (CodeMirror 6's parser system) with minimal effort. With pyparsing, the grammar is embedded in Python logic, making extraction and translation significantly harder. +With lark's declarative EBNF grammar, we can extract terminal regex patterns and translate them to CodeMirror Simple Mode for client-side syntax highlighting. With pyparsing, the grammar is embedded in Python logic, making extraction significantly harder. -**Example comparison:** +### Why CodeMirror 5 (not 6) -```python -# lark - declarative grammar (easy to translate) -grammar = """ - scope: "column" NAME ":" NEWLINE INDENT rule+ DEDENT - NAME: /[a-zA-Z_][a-zA-Z0-9_]*/ -""" -``` +For the web-based DSL editor, we chose **CodeMirror 5**: -```javascript -// Lezer - similar structure -@top Program { scope+ } -scope { "column" Name ":" newline indent rule+ dedent } -Name { @asciiLetter (@asciiLetter | @digit | "_")* } -``` +| Criterion | CodeMirror 5 | CodeMirror 6 | Monaco | Ace | +|-----------|--------------|--------------|--------|-----| +| **Bundle requirements** | CDN ready | Requires bundler | ~2MB | ~300KB | +| **FastHTML compatibility** | Direct `