# *** [was Re: [stack] Digest Number 28]

## Mark van Gulik — 2000-06-07 03:01:26

    wtanksley@... said:

     > From: Mark van Gulik [mailto:ghoul6@...]

    >>Quotation should be as explicit as possible, as it represents a form of

    >>"escape" from a language.  If a language construct can be

    >>directly embedded,

    >>no quotation is necessary, but if quotation really is necessary then it

    >>should not be concealed under any circumstances.

    >

    > Forth's quotation is essentially a switch between compile-mode and run-mode.

    > This means that it can't be nested.  Joy's quotation actually forms a list

    > out of the words contained, and those lists may be nested.  Forth's

    > quotation is useless for forming other languages.  Joy's quotation is mostly

    > useless for that.

    >

    > Forth includes a different construct which is useful for forming other

    > languages: the ability for a word to parse ahead of itself in the source

    > where it occurs.  This ability is unique to concatenative languages (it's

    > completely unresonable in a language which has to be parsed before it's

    > executed).

     This may be true at the moment, but my short term plans (within the next

    year or so) for Avail involve something similar.  In order to allow

    arbitrary language syntaxes to be supported directly in Avail, I want to

    support an unusual implementation of macros.  Like the (multi)methods of

    Avail, these constructs will be strongly typed.  The difference is that

    instead of taking values as arguments at runtime, the macros take parse

    trees at compile time.  It will be horrendously difficult to get the parser

    to work this way (although the current parser was already horrendously

    difficult to write).

    As an example, consider adding syntactic sugar for iteration.  In the

    current syntax, one uses "_do_":

        <10,20,30> do [x : integer | Print x;];

    Say we wish to be able to write this as:

        For x in <10,20,30> do Print x;

    To do this we would implement the following:

    Macro "For_in_do_" [

            var : Variable Node,

            c : Parse Node of type tuple,

            s : Statement Node |

        /* ...Construct a call to "_do_", and a literal block... */

    ];

    Note that in this scheme the existence of a macro starting with the keyword

    "For" will allow the parser to attempt to parse a macro satisfying the

    'production rule' that the macro signature implies, in this case

    expression

        ::= "For" <VariableNode> "in" <ParseNode> "do" <StatementNode>

    As far as I can determine, a macro will not need to start with a keyword.

    Note that the current parser already requires backtracking, and that I use

    dynamic programming to make this tractable.  I intend to rewrite the (tiny)

    core syntax of Avail in terms of macros.  I could implement a macro "_:=_"

    for assignment, "_:_" for variable declaration, "[_]:_" and its variations

    for block literals, and maybe even "_;" for statements (but this one would

    be the most difficult).  These changes (and a modular scheme for lexical

    scanning) would allow an arbitrary syntax to be plugged into Avail.  That's

    one of my end goals.

     >>But what kinds of sublanguages are possible (and useful)?  Is

    >>it possible to create a useful subset of Joy that doesn't use

    >>metaprogramming? Probably not.

    >

    > Why not?  You don't even have to use the list operators if you don't want

    > to.

     Yes, but can one then *implement* some list-processing program without using

    a built-in notion of lists?  In Smalltalk I would say yes.  In Joy, I think

    the answer is no.

     >>Since "meta" in this context means "unusual" or

    >>"atypical" (often with

    >>circularity as the triggering criterion), is it "a good thing" for a

    >>language to rely on metaprogramming for the simplest of tasks?

    >

    > "meta" has never, to my knowledge, meant that.  "meta" means "above" or

    > "beyond".  When you metaprogram, you're writing programs which manipulate

    > and create other programs.

     Actually, the boundary of metaprogramming is very fuzzy.  Metaprogramming

    can mean writing programs that (1) create programs, (2) look at other

    programs as data, (3) look at types as data, (4) look at variables as

    objects, (5) look at running processes and coroutines as data, (6) allow

    extension of base syntax via macros, (7) provide tools to reason about

    programs.

    You may disagree that some of these uses of the term are truly

    metaprogramming, but I'm sure some of them take you a while to classify.

    That's an indication of a fuzzy boundary.  In your description of

    metaprogramming as "...writing programswhich manipulate...other programs",

    the circularity I mentioned is obvious, but sometimes it's more subtle.  In

    essence every compiler, interpreter, and even IDE is a metaprogram.

## Massimo Dentico — 2000-06-07 03:56:20

    Mark van Gulik wrote:

     >  [...]

    > > > Since "meta" in this context means "unusual" or "atypical" (often with

    > > > circularity as the triggering criterion), is it "a good thing" for a

    > > > language to rely on metaprogramming for the simplest of tasks?

    > >

    > > "meta" has never, to my knowledge, meant that.  "meta" means "above" or

    > > "beyond".  When you metaprogram, you're writing programs which manipulate

    > > and create other programs.

    >

    > Actually, the boundary of metaprogramming is very fuzzy.  Metaprogramming

    > can mean writing programs that (1) create programs, (2) look at other

    > programs as data, (3) look at types as data, (4) look at variables as

    > objects, (5) look at running processes and coroutines as data, (6) allow

    > extension of base syntax via macros, (7) provide tools to reason about

    > programs.

    >

    > You may disagree that some of these uses of the term are truly

    > metaprogramming, but I'm sure some of them take you a while to classify.

    > That's an indication of a fuzzy boundary.  In your description of

    > metaprogramming as "...writing programswhich manipulate...other programs",

    > the circularity I mentioned is obvious, but sometimes it's more subtle.  In

    > essence every compiler, interpreter, and even IDE is a metaprogram.

     Obviously! The problem are the artificial barriers that makes difficult

    or impedes meta-programming and that makes it a quite strange idea for

    the "average" programmer. For an informal introduction to meta-programming

    you can take a look to this article, from the Tunes project founder:

    François-René Rideau, "Metaprogramming and Free Availability of Sources"

    - http://fare.tunes.org/articles/ll99/index.en.html

    or directly in HTML format here:

    - http://fare.tunes.org/articles/ll99/mpfas.html

    --

    Massimo Dentico

## stevan apter — 2000-06-07 12:04:41

    >

    > >>Since "meta" in this context means "unusual" or

    > >>"atypical" (often with

    > >>circularity as the triggering criterion), is it "a good thing" for a

    > >>language to rely on metaprogramming for the simplest of tasks?

    > >

    > > "meta" has never, to my knowledge, meant that.  "meta" means "above" or

    > > "beyond".  When you metaprogram, you're writing programs which manipulate

    > > and create other programs.

    >

    > Actually, the boundary of metaprogramming is very fuzzy.  Metaprogramming

    > can mean writing programs that (1) create programs, (2) look at other

    > programs as data, (3) look at types as data, (4) look at variables as

    > objects, (5) look at running processes and coroutines as data, (6) allow

    > extension of base syntax via macros, (7) provide tools to reason about

    > programs.

     i would enlist the concept of firstclassness, to describe 1-6, and

    reserve 'metaprogramming' for 7.  to be is to to be assignable, to

    paraphrase quine.

     >

    > You may disagree that some of these uses of the term are truly

    > metaprogramming, but I'm sure some of them take you a while to classify.

    > That's an indication of a fuzzy boundary.  In your description of

    > metaprogramming as "...writing programswhich manipulate...other programs",

    > the circularity I mentioned is obvious, but sometimes it's more subtle.  In

    > essence every compiler, interpreter, and even IDE is a metaprogram.

    >

    > ------------------------------------------------------------------------

    > Remember four years of good friends, bad clothes, explosive chemistry

    > experiments.

    > http://click.egroups.com/1/4051/6/_/_/_/960347820/

    > ------------------------------------------------------------------------

    >

    > To unsubscribe from this group, send an email to:

    > concatenative-unsubscribe@egroups.com

    >

    >

    >

    >

## wtanksley@bigfoot.com — 2000-06-07 17:14:34

    From: Mark van Gulik [mailto:ghoul6@...]

     >wtanksley@... said:

    >> Forth includes a different construct which is useful for

    >> forming other

    >> languages: the ability for a word to parse ahead of itself

    >> in the source

    >> where it occurs.  This ability is unique to concatenative

    >> languages (it's

    >> completely unresonable in a language which has to be parsed

    >> before it's executed).

    >This may be true at the moment, but my short term plans

    >(within the next

    >year or so) for Avail involve something similar.  In order to allow

    >arbitrary language syntaxes to be supported directly in Avail,

    >I want to

    >support an unusual implementation of macros.  Like the

    >(multi)methods of

    >Avail, these constructs will be strongly typed.  The difference is that

    >instead of taking values as arguments at runtime, the macros take parse

    >trees at compile time.  It will be horrendously difficult to

    >get the parser

    >to work this way (although the current parser was already horrendously

    >difficult to write).

     This is a very respectable effort, and will result in a lot of usability.

    However, it still won't allow your words to completely parse their own

    input; no matter how clever you get, the parser is still going to be

    involved, and will still get things wrong for certain cases.

    The point about Forth's parsing ability is that it doesn't have any: the

    language makes absolutely no attempt to intervene in your source.  Imagine

    having to write comments in Avail macros, for example.  It's trivial in

    Forth; all you do is scan ahead for the comment delimiter, and throw away

    all of the source.

    Avail isn't unique in this sense, although it's better than any other

    solution.  Tcl also allows words to use its parser.

     >As far as I can determine, a macro will not need to start with

    >a keyword.

     No, you (and the programmer attempting to use it) will have to study parsing

    theory to design this, though.  May I recommend that you use a "ordered

    context-free grammar"?  It should be a _lot_ easier to manipulate than the

    traditional augmented LALR grammars, and it can express a lot more.

    There's some discussion at http://caesar.ics.uci.edu/laski/Bertha.html

     >>>But what kinds of sublanguages are possible (and useful)?  Is

    >>>it possible to create a useful subset of Joy that doesn't use

    >>>metaprogramming? Probably not.

    >> Why not?  You don't even have to use the list operators if

    >> you don't want to.

    >Yes, but can one then *implement* some list-processing program

    >without using

    >a built-in notion of lists?  In Smalltalk I would say yes.  In

    >Joy, I think the answer is no.

     I would agree -- this is a problem with Joy, and results from the fact that

    Joy isn't intended to be real programming language.

    However, this isn't what I intended to talk about; I wasn't claiming that

    you could write programs without lists, but rather that you can write

    programs without metaprogramming.

     >>>Since "meta" in this context means "unusual" or

    >>>"atypical" (often with

    >>>circularity as the triggering criterion), is it "a good thing" for a

    >>>language to rely on metaprogramming for the simplest of tasks?

    >> "meta" has never, to my knowledge, meant that.  "meta" means

    >> "above" or

    >> "beyond".  When you metaprogram, you're writing programs

    >> which manipulate and create other programs.

    >Actually, the boundary of metaprogramming is very fuzzy.

     Metaprogramming, fuzzy or not, still doesn't mean "abnormal programming."

     >Metaprogramming

    >can mean writing programs that (1) create programs, (2) look at other

    >programs as data, (3) look at types as data, (4) look at variables as

    >objects, (5) look at running processes and coroutines as data,

    >(6) allow

    >extension of base syntax via macros, (7) provide tools to reason about

    >programs.

     In all of those cases, you're doing metaprogramming if and only if you're

    writing programs that manipulate other programs.  Take #3 for example --

    it's only metaprogramming if the types you're playing with are somehow part

    of a program.  If they're not, you're only doing data processing.

     >In your description of

    >metaprogramming as "...writing programswhich

    >manipulate...other programs",

    >the circularity I mentioned is obvious, but sometimes it's

    >more subtle.  In

    >essence every compiler, interpreter, and even IDE is a metaprogram.

     No argument.

    -Billy