Updated blog
This commit is contained in:
+193
@@ -937,3 +937,196 @@ that was used for it.
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
2020-04-18
|
||||||
|
**********
|
||||||
|
|
||||||
|
Blog
|
||||||
|
""""""
|
||||||
|
|
||||||
|
It's been a (very) long time since I have written in this blog.
|
||||||
|
|
||||||
|
The main reason is that I found reStructured markup too complicated. I'm still not used to how directives are
|
||||||
|
supposed to work. There are so many way to do the same thing !
|
||||||
|
|
||||||
|
I guess that it's also because I don't have the proper tool to write this doc.
|
||||||
|
I use PyCharm and thought a have the basic rendering, I cannot easily navigate between
|
||||||
|
the articles
|
||||||
|
|
||||||
|
In need to install Sphinx. I want it in a docker. For sure it's not mandatory, but I'm must practice my
|
||||||
|
docker skill if I don't want to forget everything
|
||||||
|
|
||||||
|
Parsers
|
||||||
|
"""""""
|
||||||
|
As I keep repeating, parsing expression is a very big part of what I want to achieve (alongside with the
|
||||||
|
rule engine and the speech recognition)
|
||||||
|
It as to be very easy to expression a new concept
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept one as 1
|
||||||
|
def concept two as 2
|
||||||
|
|
||||||
|
That's it !
|
||||||
|
I should now can do
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
one + one
|
||||||
|
one + two
|
||||||
|
|
||||||
|
Now, I can decide that plus is also a concept
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept a plus b as a + b
|
||||||
|
|
||||||
|
So basically, every time Sheerka will parse something 'plus' something else, it will recognize the concept a plus b
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
one plus two
|
||||||
|
|
||||||
|
worked, but
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
one plus one
|
||||||
|
|
||||||
|
doesn't. Because 'a' and 'b' are two different letters, so it was looking for two different values. That was
|
||||||
|
an unexpected side effect of my first naive implementation.
|
||||||
|
|
||||||
|
Let's put that aside for the moment and keep on our exercise to model the world.
|
||||||
|
|
||||||
|
After an addition, it will be good to have the multiplication. Easy
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept a mult b as a * b
|
||||||
|
|
||||||
|
So I can try
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
one plus two mult three
|
||||||
|
|
||||||
|
Of course, this one does now work by magic. The precedence (priority ?) between addition and multiplication
|
||||||
|
was not respected.
|
||||||
|
|
||||||
|
.. _bnf: https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form
|
||||||
|
|
||||||
|
The first idea was the bnf_ parser in order to be able to write something like
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept plus from bnf mult ('plus' mult)*
|
||||||
|
def concept mult from bnf number ('mult' number)*
|
||||||
|
def concept number from bnf one|two
|
||||||
|
|
||||||
|
Expressing recursive concepts was simple. I was proud of this implementation :code:`one plus two mult three`
|
||||||
|
was understood in the correct way.
|
||||||
|
|
||||||
|
|
||||||
|
But it started to become complicated when I wanted to define the body. In ':code:`mult (plus mult)*`' where
|
||||||
|
is the left part? where is the right part ?
|
||||||
|
|
||||||
|
Ok, let's try something like
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept plus from bnf mult=a ('plus' mult)*=b
|
||||||
|
def concept mult from bnf number=a ('mult' number)*=b
|
||||||
|
def concept number from bnf one|two
|
||||||
|
|
||||||
|
We now have 'a' for the left part, and a potential list of 'b' for the right part.
|
||||||
|
The full definition of the concept :code:`plus` will look like
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
def concept plus from bnf mult=a ('plus' mult)*=b as:
|
||||||
|
res = a
|
||||||
|
for value in b:
|
||||||
|
res += value
|
||||||
|
return res
|
||||||
|
|
||||||
|
This should work fine. In my current implementation, 'a' is an instance of the concept 'mult', correctly
|
||||||
|
initialized with concept one or a concept two, and likewise 'b' is a list of concept 'mult'.
|
||||||
|
|
||||||
|
So it should work.
|
||||||
|
|
||||||
|
It's just that I have never been this far in the tests. I just couldn't. THIS IS WAY MORE TOO COMPLICATED
|
||||||
|
TO DEFINE A SIMPLE ADDITION !!!
|
||||||
|
|
||||||
|
**Note** that you must have quote surrounding the 'plus' in the definition, to make the difference between
|
||||||
|
the concept and the literal. It's necessary, but when you start to do that, you start to narrow the usage
|
||||||
|
of your system to developers only. So, even if there is no other way, I didn't really liked that.
|
||||||
|
|
||||||
|
.. _IronPython: https://ironpython.net/
|
||||||
|
.. _parsec: https://github.com/jparsec/jparsec
|
||||||
|
.. _Holy Grail: //www.youtube.com/watch?v=YxG5mDItkGU
|
||||||
|
.. _one: https://en.wikipedia.org/wiki/Shunting-yard_algorithm
|
||||||
|
|
||||||
|
So I am done ? Is this the end ? There should be another way to express the priority (precedence ?) between the concept.
|
||||||
|
|
||||||
|
Luckily for me, I remembered that I have once seen a implementation of the Python parser (IronPython_ I think) were they
|
||||||
|
used numbers to evaluate the precedence between additions and multiplications. And there were also something
|
||||||
|
like that when I used parsec_ parser.
|
||||||
|
|
||||||
|
So I went back on internet and found my `Holy Grail`_, well not this one, this one_.
|
||||||
|
|
||||||
|
**The Shunting Yard Algorithm**
|
||||||
|
|
||||||
|
I took me a few days to understand it and implement it in its basic form (which a already too long),
|
||||||
|
but it took me one entire month to adapt it to the concepts. I know, I am not quick :-)
|
||||||
|
|
||||||
|
As a matter of fact, the sya (Shunting Yard Algorithm) is designed for binary operators and functions where the number
|
||||||
|
of arguments is known. You can support unary operators, but there is nothing explained to ternary and more.
|
||||||
|
Dealing with concepts that can be expressed as :code:`'foo a b'` (suffixed concept) or :code:`'a b bar'`
|
||||||
|
(prefixed concept) was a interesting challenge!
|
||||||
|
|
||||||
|
Anyway, I am now in position where I can simply define my addition and my multiplication
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
> def concept a plus b as a + b
|
||||||
|
> def concept a mult b as a * b
|
||||||
|
> eval one plus two mult three
|
||||||
|
> 7
|
||||||
|
|
||||||
|
That's it !
|
||||||
|
|
||||||
|
At least in theory. The definition and the parsing of the concepts is done and fully tested when you
|
||||||
|
programmatically set the precedences, I now need a way to define/express the priorities
|
||||||
|
|
||||||
|
What I surely don't want is to write something like:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
plus.precedence = 1
|
||||||
|
mult.precedence = 2
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
set_precedence(plus, 1)
|
||||||
|
set_precedence(mult, 2)
|
||||||
|
|
||||||
|
Any solution where you have to give the actual value of the precedence is a bad solution. I would like to
|
||||||
|
have something like
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
precedence mult > precedence plus
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
mult.precedence > plus.precedence
|
||||||
|
|
||||||
|
|
||||||
|
It means that I now have to implement a partitioning algorithm with simple constraints (<, >). I think that I will
|
||||||
|
include <=, >=, = and != as well, once for all. Sorting things according to these constraints is something
|
||||||
|
human naturally do.
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user