Re: SAOL source tokenisation

From: Giorgio Zoia (Giorgio.Zoia@epfl.ch)
Date: Wed Jan 31 2001 - 03:59:18 EST


>
>Thanks Giorgio,
>
>So it is not possible to develop a program that generates the tokens for
>some given
>SAOL source without doing syntax/semantic analysis as well!?

IMHO, just go on with a context-free coding, if you can't easily keep track whether
a "table" or an "=" (or oparray) came last before arriving at buzz. I wonder where
it will not work...

In any case, probably a first issue for a second list ?

>In the SAOL source there is no difference between the two words, so why
>making a
>difference in the token codes based on the context? This makes things so
>complicated.

...only a bit complicated... :-)

Giorgio

__________________________________________________________________
Giorgio ZOIA

Integrated Systems Laboratory - DE/LSI - EPFL
CH-1015 Lausanne - SWITZERLAND

Phone: + 41 21 693 69 79 E-mail: Giorgio.Zoia@epfl.ch
Fax: +41 21 693 46 63
__________________________________________________________________



This archive was generated by hypermail 2b29 : Mon Jan 28 2002 - 12:11:06 EST