Added DataServicesManager and DataService

docs/DataGrid Refactoring.md

@@ -0,0 +1,116 @@
+# DataGrid Refactoring
+
+## Objective
+
+Clearly separate data management and rendering responsibilities in the DataGrid system.
+The current architecture mixes data mutation, formula computation, and rendering in the
+same `DataGrid` class, which complicates cross-table formula management and code reasoning.
+
+## Guiding Principles
+
+- `DataService` can exist without rendering. The reverse is not true.
+- All data mutations go through `DataService`.
+- Columns have two facets: data semantics (`ColumnDefinition`) and UI presentation (`ColumnUiState`).
+- No more parent hierarchy where avoidable — access via `InstancesManager.get_by_type()`.
+- The persistence key is `grid_id` (stable), not `table_name` (can change over time).
+
+---
+
+## New Classes (`core/data/`)
+
+### `DataServicesManager` — SingleInstance
+
+- Owns the `FormulaEngine` (cross-table formula coordination)
+- Creates `DataService` instances on demand from `DataGridsManager`
+- Provides access to `DataService` instances by `grid_id`
+- Provides the resolver callback for `FormulaEngine`: `grid_id → DataStore`
+
+### `DataService` — companion to `DataGrid`
+
+- Owns `DataStore` and `list[ColumnDefinition]`
+- Holds a reference to `DataServicesManager` for `FormulaEngine` access
+- Methods: `load_dataframe(df)`, `add_row()`, `add_column()`, `set_data(col_id, row_index, value)`
+- Mutations call `mark_data_changed()` → set dirty flag
+- `ensure_ready()` → recalculates formulas if dirty (called by `mk_body_content_page()`)
+- Can exist without any rendering
+
+### `DataStore` — renamed from `DatagridStore`
+
+- Pure persistence: `ne_df`, `ns_fast_access`, `ns_row_data`, `ns_total_rows`
+- `DbObject` with no business logic
+
+### `ColumnDefinition`
+
+- Data semantics: `col_id`, `title`, `type`, `formula`, `col_index`
+
+---
+
+## Modified Classes
+
+### `DataGridsRegistry` — streamlined
+
+- Persistence only: `put()`, `remove()`, `get_all_entries()`
+- **Loses**: `get_columns()`, `get_column_type()`, `get_column_values()`, `get_row_count()`
+
+### `DatagridMetadataProvider` — becomes a concrete SingleInstance
+
+- No longer abstract / interface (only one concrete implementation exists)
+- Reads from `DataServicesManager` and `DataGridsRegistry`
+- Holds: `style_presets`, `formatter_presets`, `all_tables_formats`
+- Exposes: `list_tables()`, `list_columns()`, `list_column_values()`, `get_column_type()`,
+  `list_style_presets()`, `list_format_presets()`
+
+### `DataGridsManager` — pure UI
+
+- **Keeps**: `TreeView`, `TabsManager`, document state, `Commands`
+- **Loses**: `FormulaEngine`, presets, `DatagridMetadataProvider`, `_resolve_store_for_table()`
+
+### `DataGrid` — pure rendering
+
+- **Keeps**: `mk_*`, `render()`, `__ft__()`, `_state`, `_settings`
+- **Keeps**: `_apply_sort()`, `_apply_filter()`, `_get_filtered_df()`
+- **Loses**: `add_new_row()`, `add_new_column()`, `init_from_dataframe()`,
+  `_recalculate_formulas()`, `_register_existing_formulas()`, `_df_store`
+- Accesses its `DataService` via its `grid_id`:
+  `InstancesManager.get_by_type(DataServicesManager).get_service(grid_id)`
+- `mk_body_content_page()` calls `data_service.ensure_ready()` before rendering
+
+### `DatagridState`
+
+- `columns` changes from `list[DataGridColumnState]` → `list[ColumnUiState]`
+- Everything else remains unchanged
+
+### `DataGridColumnState` — split into two classes
+
+| Class | Belongs to | Fields |
+|---|---|---|
+| `ColumnDefinition` | `DataService` | `col_id`, `title`, `type`, `formula`, `col_index` |
+| `ColumnUiState` | `DatagridState` | `col_id`, `width`, `visible`, `format` |
+
+---
+
+## Structural Fix
+
+**Current bug**: `mark_data_changed()` is defined in `FormulaEngine` but is never called
+by `DataGrid`. Formulas are only recalculated defensively at render time.
+
+**After refactoring**:
+- Every mutation in `DataService` calls `mark_data_changed()` → dirty flag set
+- `mk_body_content_page()` calls `data_service.ensure_ready()` → recalculates if dirty
+- Multiple mutations before a render = a single recalculation
+
+---
+
+## Progress Tracking
+
+- [x] Create `DataStore` (rename `DatagridStore`)
+- [x] Create `ColumnDefinition`
+- [x] Create `DataService`
+- [x] Create `DataServicesManager`
+- [x] Refactor `DataGridsRegistry` (streamline)
+- [x] Refactor `DatagridMetadataProvider` (make concrete)
+- [x] Refactor `DataGridsManager` (pure UI)
+- [x] Refactor `DataGrid` (pure rendering, split `DataGridColumnState`)
+- [x] Update tests
+- [ ] Remove `init_from_dataframe` from `DataGrid` (kept temporarily for transition)
+- [ ] Full split of `DataGridColumnState` into `ColumnDefinition` + `ColumnUiState` in `DatagridState`