Refactored parsers to introduce priority

2020-01-08 19:45:54 +01:00
parent b4346b5af0
commit a62c1f0f13
13 changed files with 531 additions and 42 deletions
@@ -827,4 +827,67 @@ So I am going to introduce the keyword :code:`concept:name:` or :code:`c:name:`
 It will means that the concept is required.

 If the name is required, you can use :code:`"'name'"` or :code:`'"name"'`.
-It's already working. There is nothing to do for this one.
+It's already working. There is nothing to do for this one.
+
+2020-07-01
+**********
+
+How do we perform the parsing ?
+"""""""""""""""""""""""""""""""
+
+The basic flow of an execution is :
+
+* Parse the data -> Nodes
+* Evaluate the nodes -> Concepts
+* Display the results
+
+The theories says that there can exist as many parsers as necessary. Each one of them will
+be specialized to recognize a specific pattern. They will then send there information to
+the evaluators.
+
+As of now, I have implemented the following parsers:
+
+* EmptyStringParser
+    To recognize empty strings and react accordingly
+
+* PythonParser
+    To recognize Python source code
+
+* ExactConceptParser
+    To recognize simple form of concepts
+
+* DefaultParser (the name is not accurate)
+    To recognize builtin syntax (like 'def concept' or 'isa')
+
+* ConceptLexerParser
+    To recognize concept defined with BNF language
+
+All theses parsers are executed in the row (the order in not very important)
+
+The first observation is that there is lot of CPU waste. Most of the time (at least as of
+now, when a there is a match with one parser, the others fail). So there is no need to
+execute them.
+
+The second point is that there is now way for a parser to use the result of another.
+My idea is to have parsers that can be chained, each one of them will do the little thing
+it is capable of before leaving the rest to some more powerful parser.
+
+I don't want to bring out the big guns for every single user input. And I certainly
+don't want a massive and over complex parser that will be capable (in theory) of everything
+
+Why ?
+
+| First of all, monolithic code is bad :-)
+| Then I have to keep in mind that the process will be somehow distributed
+| And last, but not least. I don't have (and I certainly will never have) the full completion
+    of all possible parsing situation. So what I need is a plug and play system where I can add
+    and remove and chain parsers, depending of the input.
+
+So,
+
+* I'll give all parsers a priority
+* The parsers with the highest priority will be executed first
+* The parsers with the same priority will be executed at the same time (The order does matter)
+* If, for a given priority there is a match, the parser with a lower priority won't be executed
+* A parser has access to the output of the parsers of higher priorities (which were executed before it)
+