# RE: [stack] Inlining in Joy

## Billy Tanksley — 2001-04-27 20:30:30

    From: John Cowan [mailto:cowan@...]

     >(Manfred, you and I seem to be talking to ourselves, but what

    >the hell...)

     I assure you that you have an attentive, if otherwise occupied, audience.

     >Joy lends itself to extremely easy inlining:  when defining a

    >word, you can simply replace any user-defined word in the body

    >with *its* body, recursively, but watching for references to

    >words already being inlined (so that you do not get into an

    >infinite expansion).  The implementation should be extremely

    >easy.

    >However, I can think of at least three policies for deciding

    >whether to inline or not:

    >1) Mark inlineable words (e.g. by starting the definition

    >with INLINE instead of DEFINE/LIBRA).  Any word so marked

    >is always inlined when it is used in a definition, subject

    >to the caveat above.

     In other words, an INLINE word produces an error if it contains recursion.

     >Any thoughts, anyone?  Do Forth compilers inline?  What

    >about dependency tracking?

     The simplest fast Forth compilers inline only specific chosen words;

    however, they inline native code.

    I don't think inlining would be beneficial to Joy, unless it was in the form

    of a full-power macro system (as in Forth).

     >There is / one art             || John Cowan <jcowan@...>

     -Billy

## John Cowan — 2001-04-27 20:58:31

    Billy Tanksley wrote:

     > I assure you that you have an attentive, if otherwise occupied, audience.

     Good.  (I've been in classes like this:  I talk, professor talks,

    all others are asleep...)

     >> 1) Mark inlineable words (e.g. by starting the definition

    >> with INLINE instead of DEFINE/LIBRA).  Any word so marked

    >> is always inlined when it is used in a definition, subject

    >> to the caveat above.

    >

    > In other words, an INLINE word produces an error if it contains recursion.

     Not my intention, no: it simply would not be inlined recursively.

    So given: INLINE bar == foo bar foo.  DEFINE baz == zam bar zam.

    we would give baz an effective definition of "zam foo bar foo zam".

    The reference to bar within zam is inlined, but the reference

    to bar within itself is left alone.  This would be equally

    true of an indirect reference to any parent word.

     > I don't think inlining would be beneficial to Joy,

     Probably not, given the large interpretation overhead.  I was

    primarily thinking of the many short definitions in stdlib.joy,

    which would be more appealing to use if they were silently

    inlined.

     > unless it was in the form

    > of a full-power macro system (as in Forth).

     I understand now, I think, how Forth does its metasyntax.

    I am trying to see how, if at all, this can be usefully

    imported into the Joy context.

    I still do not grasp CREATE...DOES, though.

    --

    There is / one art             || John Cowan <jcowan@...>

    no more / no less              || http://www.reutershealth.com

    to do / all things             || http://www.ccil.org/~cowan

    with art- / lessness           \\ -- Piet Hein

## Billy Tanksley — 2001-04-27 21:29:45

    From: John Cowan [mailto:cowan@...]

     >Billy Tanksley wrote:

    >> I assure you that you have an attentive, if otherwise

    >> occupied, audience.

    >Good.  (I've been in classes like this:  I talk, professor talks,

    >all others are asleep...)

     Ah, yes.  I remember those as well.

     >>> 1) Mark inlineable words (e.g. by starting the definition

    >>> with INLINE instead of DEFINE/LIBRA).  Any word so marked

    >>> is always inlined when it is used in a definition, subject

    >>> to the caveat above.

    >> In other words, an INLINE word produces an error if it

    >> contains recursion.

    >Not my intention, no: it simply would not be inlined recursively.

     Ah.  It just seems kind of wasteful to leave multiple ways of calling the

    same word sitting around.

     >> I don't think inlining would be beneficial to Joy,

    >Probably not, given the large interpretation overhead.  I was

    >primarily thinking of the many short definitions in stdlib.joy,

    >which would be more appealing to use if they were silently

    >inlined.

     It would be pretty trivial to automatically inline short, nonrecursive

    words.  This goes double for single-word definitions.

     >> unless it was in the form

    >> of a full-power macro system (as in Forth).

    >I understand now, I think, how Forth does its metasyntax.

    >I am trying to see how, if at all, this can be usefully

    >imported into the Joy context.

     I don't see too much difficulty: add the concept of an IMMEDIATE word to

    Joy, and allow words to have access to the source-stream ahead of

    themselves.  Words should be able to consume source (like doing a 'car' to a

    list) and prepend new source (like doing a 'cons', but destructively).

     >I still do not grasp CREATE...DOES, though.

     It took me a couple of months to figure it out.  The main problem isn't

    CREATE or DOES>; the main problem is sorting out the time when various

    things are supposed to happen.  Compile-time, definition-time, run-time --

    augh!

    Fortunately, though, CREATE-DOES has no relationship with metasyntax.  Don't

    worry about it at the same time you're worrying about metasyntax.

     >There is / one art             || John Cowan <jcowan@...>

     -Billy

## John Cowan — 2001-04-27 21:51:51

    Billy Tanksley wrote:

     > Ah.  It just seems kind of wasteful to leave multiple ways of calling the

    > same word sitting around.

     It can't be avoided in the general case, unless we are to make it an

    error to have inlineables that recurse (and you cannot statically

    check for recursion, because another word's definition may change).

    There are various possible tactics, but the straightforward

    one is to keep both a "source" (not inlined) and an "object" (inlined)

    version for each word, which may be the same list.

     > It would be pretty trivial to automatically inline short, nonrecursive

    > words.  This goes double for single-word definitions.

     Exactly so.

     > I don't see too much difficulty: add the concept of an IMMEDIATE word to

    > Joy, and allow words to have access to the source-stream ahead of

    > themselves.  Words should be able to consume source (like doing a 'car' to a

    > list) and prepend new source (like doing a 'cons', but destructively).

     Does this source stream consist of characters or words?  I know it's

    characters in Forth (which can then be packaged as words or not), but I

    don't know that that would suit Joy, whose native syntax includes

    lists and sets and such.

    I think a new idea is needed, in the spirit of Joy.

    --

    There is / one art             || John Cowan <jcowan@...>

    no more / no less              || http://www.reutershealth.com

    to do / all things             || http://www.ccil.org/~cowan

    with art- / lessness           \\ -- Piet Hein

## Billy Tanksley — 2001-04-27 22:11:06

    From: John Cowan [mailto:cowan@...]

     >Billy Tanksley wrote:

    >> Ah.  It just seems kind of wasteful to leave multiple ways

    >> of calling the same word sitting around.

    >It can't be avoided in the general case, unless we are to make it an

    >error to have inlineables that recurse

     Which is exactly my point.  Disallowing recursion in inlinables is not a

    hardship; inlineables should be so short anyhow that it's more efficient to

    inline them than it is to compile a call to them.

     >(and you cannot statically

    >check for recursion, because another word's definition may change).

     In most modern functional language it's not possible to change the

    definition of a word after the fact.  One can define a new word with the

    same name, but not change the definition.  If you allow definition changes,

    then /ipso facto/, you utterly disallow inlining.  Take your choice: one or

    the other.  Never both.

     >> I don't see too much difficulty: add the concept of an

    >> IMMEDIATE word to

    >> Joy, and allow words to have access to the source-stream ahead of

    >> themselves.  Words should be able to consume source (like

    >> doing a 'car' to a

    >> list) and prepend new source (like doing a 'cons', but

    >> destructively).

    >Does this source stream consist of characters or words?

     I allowed that to remain ambiguous -- it's up to you.  I do recommend that

    you allow access on a string-by-string basis (like Forth), but provide

    higher-level access as well.

     >I know it's

    >characters in Forth (which can then be packaged as words or not), but I

    >don't know that that would suit Joy, whose native syntax includes

    >lists and sets and such.

     The purpose of a metasyntax is to allow you to build (or bypass) the syntax

    using it.  There should be certain limits, but by and large, you should be

    able to grab anything from the source, and thus render it invisible to the

    compiler.

    Now, what are those limits?

     + you can't grab anything the compiler's already processed: for this reason

    we want all sourcegrabs to be relative to the end of the current word.

     + you can't grab past the end of what's been read: for this reason Forth

    doesn't let you see the EOF.  I suspect that Joy wouldn't let you see the

    period at the end of the current definition (or, if you're not in a

    definition, the end of the file).

    Make sense?

     >I think a new idea is needed, in the spirit of Joy.

     I don't see Forth's solution as being antithetical to Joy.  Of course, Joy's

    solution should be easier to manipulate, and there should be higher-level

    ways of doing things in Joy, but none of that forbids including Forth's

    low-level access as well.

    Joy should have a word which parses a complete Joy-token; but that doesn't

    make it impossible to have a word which parses up to an arbitrary character.

     >There is / one art             || John Cowan <jcowan@...>

     -Billy

## John Cowan — 2001-04-27 22:26:27

    Billy Tanksley wrote:

     > If you allow definition changes,

    > then /ipso facto/, you utterly disallow inlining.  Take your choice: one or

    > the other.  Never both.

     Feh. :-)

    To really make this work, I would have to make it an error to redefine

    *anything*: once defined, always defined, unless you throw away the

    whole Joy interpreter and start over.  Not very interactive.

    No, I want to be able to redefine things as I go, correcting errors,

    and if the redefined word has been inlined by another word, the other

    word has to be fixed too.  This just involves keeping a bit of

    dependency graph, and a very simple one at that.

    The alternative is to allow people to make changes to a word, but

    then it turns out that some of the changes aren't effective everywhere:

    even after you have fixed a bug, there are places where the

    old buggy code is still running.  The C analogue is not using makefiles,

    and just hoping that you have remembered to recompile everything you need to.

     >  + you can't grab anything the compiler's already processed: for this reason

    > we want all sourcegrabs to be relative to the end of the current word.

    >  + you can't grab past the end of what's been read: for this reason Forth

    > doesn't let you see the EOF.  I suspect that Joy wouldn't let you see the

    > period at the end of the current definition (or, if you're not in a

    > definition, the end of the file).

    >

    > Make sense?

     It just isn't in the spirit of Joy, somehow.  It is, definitely, in

    the spirit of Forth, which is "Allow anything: if it cuts you, it's

    your fault."  Joy works quite differently, with run-time type checks

    and automatic polymorphism.  Joy metasyntax should have that flavor too.

     > I don't see Forth's solution as being antithetical to Joy.  Of course, Joy's

    > solution should be easier to manipulate, and there should be higher-level

    > ways of doing things in Joy, but none of that forbids including Forth's

    > low-level access as well.

     Not antithetical, just ... orthogonal.

    I must think.

    --

    There is / one art             || John Cowan <jcowan@...>

    no more / no less              || http://www.reutershealth.com

    to do / all things             || http://www.ccil.org/~cowan

    with art- / lessness           \\ -- Piet Hein

## Billy Tanksley — 2001-04-27 23:27:50

    From: John Cowan [mailto:cowan@...]

     >Billy Tanksley wrote:

    >> If you allow definition changes,

    >> then /ipso facto/, you utterly disallow inlining.  Take your

    >> choice: one or the other.  Never both.

    >Feh. :-)

     :-)

     >To really make this work, I would have to make it an error to redefine

    >*anything*: once defined, always defined, unless you throw away the

    >whole Joy interpreter and start over.  Not very interactive.

     Not at all!  I would make it possible to redefine things, but I would simply

    not make the redefinitions retroactive.

     >No, I want to be able to redefine things as I go, correcting errors,

    >and if the redefined word has been inlined by another word, the other

    >word has to be fixed too.  This just involves keeping a bit of

    >dependency graph, and a very simple one at that.

     Ah, an interesting point.  Yes, that's a solution.

    But a couple of problems: first, it adds a lot of overhead to the system;

    and second, it raises the alarming spectre of miscorrecting words.  What

    happens if you accidentally redefine a word which happens to be critical to

    the system?  In Forth it doesn't matter; the system's already compiled, so

    the system will still run.

    And finally, what if the buggy word has already been run, and its results

    are implicit in the system's state?  You (the compiler) have absolutely no

    way to know!  You're going to a lot of effort to solve a problem which you

    simply can't fix.

     >The alternative is to allow people to make changes to a word, but

    >then it turns out that some of the changes aren't effective everywhere:

    >even after you have fixed a bug, there are places where the

    >old buggy code is still running.  The C analogue is not using

    >makefiles, and just hoping that you have remembered to recompile

    >everything you need to.

     IF the thing you're redefining is a bugfix, the obvious solution is to erase

    from the dictionary all words which depend on the word being defined, and

    allow them to be recompiled at need.  I don't want to be _forced_ to do this

    as part of a redefinition, though; I may not even be _aware_ that I'm

    redefining something -- or my redefinition might not be a later version of

    the old definition, but rather just part of a different exploration.  As a

    programmer, I'd rather specify when I want to replace versus when I want to

    override.

    In the long run, though, why make it so /hard/ on yourself (the compiler

    writer)?  It's so easy to treat redefinitions the way almost every other

    language does: let them override the old definition (without erasing it).

    When the new definition falls out of scope, let the old definition be

    revealed.  If the programmer /wants/ to obsolete (delete) a word, he should

    be able to; that should cause all system changes after the word to be

    deleted (in Forth, this is modelled by deleting everything chronologically

    following the deleted word in the dictionary).

    It's handy to be able to bugfix words, but that's what source browsers are

    for.  Take a line from Smalltalk for this.  The source browser can maintain

    your dependancies, so that the compiler doesn't have to.  This way

    dependancies are optional, and don't affect people who don't use them.  You

    can also upgrade to a fancier dependancy checker when the time comes,

    without making your version of the language incompatible with earlier

    versions.

     >>  + you can't grab anything the compiler's already processed:

    >> for this reason

    >> we want all sourcegrabs to be relative to the end of the

    >> current word.

    >>  + you can't grab past the end of what's been read: for this

    >> reason Forth

    >> doesn't let you see the EOF.  I suspect that Joy wouldn't

    >> let you see the

    >> period at the end of the current definition (or, if you're not in a

    >> definition, the end of the file).

    >> Make sense?

    >It just isn't in the spirit of Joy, somehow.  It is, definitely, in

    >the spirit of Forth, which is "Allow anything: if it cuts you, it's

    >your fault."  Joy works quite differently, with run-time type checks

    >and automatic polymorphism.  Joy metasyntax should have that

    >flavor too.

     Almost everything you're quoting is non-optional (except for the "I

    suspect"); if it's not in the spirit of Joy, Joy is fundamentally broken.

    How can it be in the spirit of a language to be able to manipulate source

    which hasn't been read yet?  How can it be in the spirit of a language to

    manipulate source that's already been compiled and may possibly be

    executing?

     >> I don't see Forth's solution as being antithetical to Joy.

    >> Of course, Joy's

    >> solution should be easier to manipulate, and there should be

    >> higher-level

    >> ways of doing things in Joy, but none of that forbids

    >> including Forth's low-level access as well.

    >Not antithetical, just ... orthogonal.

    >I must think.

     The only difference I can see between us is that you don't want the

    programmer to have access to the source in string form, and I do.  I can't

    comprehend why you don't want the programmer to have that access.  We both

    agree that the programmer should have high-level access.

    With that said, though, I don't really mind giving the programmer ONLY

    high-level access.  I don't understand why, but I don't mind it.  IMO

    high-level languages should be powerful because they offer MORE than

    low-level ones, _not_ because they offer less.

    BTW, one comment on the new version: the inconsistent use of "." vs. "!" is

    distracting.  Would it be possible to make "30." be parsed as an integer

    followed by a dot -- if you want a floating point number with a value of

    thirty, you have to write it as "30.0"?

     >There is / one art             || John Cowan <jcowan@...>

     -Billy

## John Cowan — 2001-04-28 02:15:01

    Billy Tanksley scripsit:

     > Not at all!  I would make it possible to redefine things, but I would simply

    > not make the redefinitions retroactive.

     Since the existing semantics (no inlining) makes redefinitions retroactive,

    that would certainly eliminate universal (no programmer control) inlining

    as a strategy.

     > But a couple of problems: first, [a dependency graph]

    >  adds a lot of overhead to the system;

     Not really.  Basically, one extra pointer per word, chaining together

    the words that refer to this one.

     > and second, it raises the alarming spectre of miscorrecting words.  What

    > happens if you accidentally redefine a word which happens to be critical to

    > the system?  In Forth it doesn't matter; the system's already compiled, so

    > the system will still run.

     I'm not sure what "the system" means in this context.  The Joy interpreter

    will continue to run even if all user-defined words are cabbaged, and

    you always get a warning when you are redefining a system-defined word.

     > And finally, what if the buggy word has already been run, and its results

    > are implicit in the system's state?  You (the compiler) have absolutely no

    > way to know!  You're going to a lot of effort to solve a problem which you

    > simply can't fix.

     What state?  Joy's state is the stack and various file streams.

     > In the long run, though, why make it so /hard/ on yourself (the compiler

    > writer)?  It's so easy to treat redefinitions the way almost every other

    > language does: let them override the old definition (without erasing it).

     This certainly isn't how Lisp behaves.  Scheme can integrate

    the built-in functions at compile time if you tell it it's safe to.

     > If the programmer /wants/ to obsolete (delete) a word, he should

    > be able to; that should cause all system changes after the word to be

    > deleted (in Forth, this is modelled by deleting everything chronologically

    > following the deleted word in the dictionary).

     Seems way too drastic.

     > It's handy to be able to bugfix words, but that's what source browsers are

    > for.  Take a line from Smalltalk for this.  The source browser can maintain

    > your dependancies, so that the compiler doesn't have to.  This way

    > dependancies are optional, and don't affect people who don't use them.  You

    > can also upgrade to a fancier dependancy checker when the time comes,

    > without making your version of the language incompatible with earlier

    > versions.

     Good argument.

    More in the next message.

    --

    John Cowan                                   cowan@...

    One art/there is/no less/no more/All things/to do/with sparks/galore

    	--Douglas Hofstadter

## John Cowan — 2001-04-28 02:40:26

    Billy Tanksley scripsit:

     > >> I don't see Forth's solution as being antithetical to Joy.

    > >> Of course, Joy's

    > >> solution should be easier to manipulate, and there should be

    > >> higher-level

    > >> ways of doing things in Joy, but none of that forbids

    > >> including Forth's low-level access as well.

    >

    > >Not antithetical, just ... orthogonal.

    > >I must think.

     I have thought, and here's my current idea: immediacy without metasyntax.

    The Joy scanner remains fixed, but we get access to execution during

    definition time.

    Specifically, my notion is to allow "(" and ")" within definitions

    to indicate a program to be executed while the definition is in

    progress.  The Joy parser collects everything between "(" and ")"

    and executes it.  Whatever is on the stack when that is done

    is inserted into the definition.  Here are some examples:

    DEFINE seven == (3 4 +).	(* constant folding *)

    The program "3 4 +" executes, leaving 7 on the stack.  This is

    inserted into the definition, so it is equivalent to "DEFINE seven == 7."

    DEFINE inline == first body unstack.	(* inlining *)

    DEFINE foo == "1 2 +".

    DEFINE bar == 3 ([foo] inline) +.

    The parenthesized program puts the definition of foo as stack content.

    SO this is equivalent to "define bar == 3 1 2 + +."  Of course we are

    not maintaining the dependency graph.  :-)

    DEFINE cadr == ("(lambda (x) (car (cadr x)))" lisp-compile).	(* non-Joy *)

    Here the word "lisp-compile" takes a Lisp function definition as a string

    and puts appropriate words on the stack.  (You will excuse me for

    not exhibiting its definition.)  The use of a string allows

    arbitrary non-Joy syntax.

    Currently, "(" and ")" are tokens, but they aren't used for anything

    in Joy.  They mostly seem to exist as a byproduct of the comment

    syntax.  Parens outside definitions (including nested parens)

    would be illegal under this syntax, so perhaps it would be

    better to use some one character.

    What do you think?

    --

    John Cowan                                   cowan@...

    One art/there is/no less/no more/All things/to do/with sparks/galore

    	--Douglas Hofstadter

## John Cowan — 2001-04-28 02:23:13

    Billy Tanksley scripsit:

     > Almost everything you're quoting is non-optional (except for the "I

    > suspect"); if it's not in the spirit of Joy, Joy is fundamentally broken.

    > How can it be in the spirit of a language to be able to manipulate source

    > which hasn't been read yet?  How can it be in the spirit of a language to

    > manipulate source that's already been compiled and may possibly be

    > executing?

     Yes, the part I quoted wasn't to the purpose of my comment.

     > The only difference I can see between us is that you don't want the

    > programmer to have access to the source in string form, and I do.  I can't

    > comprehend why you don't want the programmer to have that access.  We both

    > agree that the programmer should have high-level access.

     Joy isn't a low-level language.  No pointers, no access to the return

    stack, no fine control of parsing.

     > With that said, though, I don't really mind giving the programmer ONLY

    > high-level access.  I don't understand why, but I don't mind it.  IMO

    > high-level languages should be powerful because they offer MORE than

    > low-level ones, _not_ because they offer less.

     That's the spirit of Forth.  I can't think of any other language

    that is both as low-level and as high-level, from the concatenative

    algebra to the built-in assembler.

     > BTW, one comment on the new version: the inconsistent use of "." vs. "!" is

    > distracting.  Would it be possible to make "30." be parsed as an integer

    > followed by a dot -- if you want a floating point number with a value of

    > thirty, you have to write it as "30.0"?

     It would be possible if I could figure out how!  It's a bug; the "!" is a

    workaround.

    More in the next message.

    --

    John Cowan                                   cowan@...

    One art/there is/no less/no more/All things/to do/with sparks/galore

    	--Douglas Hofstadter

## Billy Tanksley — 2001-04-28 03:55:02

    From: John Cowan [mailto:cowan@...]

     >Billy Tanksley scripsit:

    >> >> I don't see Forth's solution as being antithetical to Joy.

    >> >> Of course, Joy's

    >> >> solution should be easier to manipulate, and there should be

    >> >> higher-level

    >> >> ways of doing things in Joy, but none of that forbids

    >> >> including Forth's low-level access as well.

    >> >Not antithetical, just ... orthogonal.

    >> >I must think.

    >I have thought, and here's my current idea: immediacy without

    >metasyntax.

     A wonderful idea, as long as metasyntax remains open (just as a compromise

    to me, okay?).  I think metasyntax is one of the biggest advantages

    concatenative languages have over their more well-developed applicative

    brethren, and I think that Joy will eventually support it.

     >The Joy scanner remains fixed,

     Were you under the impression that I wanted to change the Joy scanner?  I

    don't.  The beautiful thing about concatenative metasyntax is that no

    scanner changes are needed.

     >but we get access to execution during definition time.

     Cool.

     >Specifically, my notion is to allow "(" and ")" within definitions

    >to indicate a program to be executed while the definition is in

    >progress.  The Joy parser collects everything between "(" and ")"

    >and executes it.  Whatever is on the stack when that is done

    >is inserted into the definition.  Here are some examples:

     That's just about perfect.  Another approach would be to use a single

    character to indicate immediacy, and if you want a group of words to be

    immediate, you [list] them and immediate the list.  But no matter; after

    pondering the consequences, I think I like parens (or curly braces).

    `[a list of immediate words]

    versus

    (a list of immediate words)

    versus

    `a `list `of `immediate `words

     >DEFINE seven == (3 4 +).	(* constant folding *)

     Good.  Makes sense.

     >DEFINE inline == first body unstack.	(* inlining *)

    >DEFINE foo == "1 2 +".

    >DEFINE bar == 3 ([foo] inline) +.

    >The parenthesized program puts the definition of foo as stack content.

    >SO this is equivalent to "define bar == 3 1 2 + +."  Of course we are

    >not maintaining the dependency graph.  :-)

     I have a question: what does

    DEFINE bar == (3 ([foo] inline) +).

    do?

    It seems like a reasonable thing to do -- is it?

     >DEFINE cadr == ("(lambda (x) (car (cadr x)))" lisp-compile).

    >(* non-Joy *)

     The whole concept of [lists in programs] is SO non-concatenative.  For a

    proof of the concept of concatenativity, Joy is remarkably shy about staying

    concatenative.  I know, I know; if I'm really unhappy I should define my own

    language, because Joy is already designed.  I'm not really unhappy, though;

    I'm just a bit regretful.

    I mention this because of how odd the above appears to me.

     >Currently, "(" and ")" are tokens, but they aren't used for anything

    >in Joy.  They mostly seem to exist as a byproduct of the comment

    >syntax.  Parens outside definitions (including nested parens)

    >would be illegal under this syntax, so perhaps it would be

    >better to use some one character.

    >What do you think?

     I like it as much as I like square braces as syntax -- it's a nifty thing,

    but it's not very concatenative.  (Why not?  Because it's syntax, and syntax

    created in the old applicative-language-parser way.)

     >John Cowan                                   cowan@...

     -Billy

## Louis Madon — 2001-04-28 04:22:58

    > -----Original Message-----

     > From: John Cowan [SMTP:cowan@...]

     > Sent: Saturday, April 28, 2001 12:40 PM

     > To: concatenative@yahoogroups.com

     > Subject: Re: [stack] Inlining in Joy

     >

     > Billy Tanksley scripsit:

     >

     > > >> I don't see Forth's solution as being antithetical to Joy.

     > > >> Of course, Joy's

     > > >> solution should be easier to manipulate, and there should be

     > > >> higher-level

     > > >> ways of doing things in Joy, but none of that forbids

     > > >> including Forth's low-level access as well.

     > >

     > > >Not antithetical, just ... orthogonal.

     > > >I must think.

     >

     > I have thought, and here's my current idea: immediacy without metasyntax.

     > The Joy scanner remains fixed, but we get access to execution during

     > definition time.

     >

     > Specifically, my notion is to allow "(" and ")" within definitions

     > to indicate a program to be executed while the definition is in

     > progress. The Joy parser collects everything between "(" and ")"

     > and executes it. Whatever is on the stack when that is done

     > is inserted into the definition. Here are some examples:

     >

     > DEFINE seven == (3 4 +). (* constant folding *)

     >

     > The program "3 4 +" executes, leaving 7 on the stack. This is

     > inserted into the definition, so it is equivalent to "DEFINE seven == 7."

     >

     > DEFINE inline == first body unstack. (* inlining *)

     > DEFINE foo == "1 2 +".

     > DEFINE bar == 3 ([foo] inline) +.

     >

     > The parenthesized program puts the definition of foo as stack content.

     > SO this is equivalent to "define bar == 3 1 2 + +." Of course we are

     > not maintaining the dependency graph. :-)

     >

     > DEFINE cadr == ("(lambda (x) (car (cadr x)))" lisp-compile). (*

     non-Joy *)

      >

     > Here the word "lisp-compile" takes a Lisp function definition as a string

     > and puts appropriate words on the stack. (You will excuse me for

     > not exhibiting its definition.) The use of a string allows

     > arbitrary non-Joy syntax.

     >

     > Currently, "(" and ")" are tokens, but they aren't used for anything

     > in Joy. They mostly seem to exist as a byproduct of the comment

     > syntax. Parens outside definitions (including nested parens)

     > would be illegal under this syntax, so perhaps it would be

     > better to use some one character.

     >

     > What do you think?

     I think it's a reasonable idea for an interpreter (where the goal is

    keeping the implementation simple) but not such a good idea for an

    optimising compiler. Such a compiler could infer the best inlining

    strategy without you having to annotate the source code at all. The more

    things that are inferred, the less work the programmer needs to do

    whenever changes need to be made.

    As a side note, I'm slowly continuing to make progress on my compiler.

    This automatically eliminates all invariant expressions (like the

    examples you give above) at compile time.

    [However, due to my work commitments, progress is much slower than I'd

    like - at this rate it still has months to go, unfortunately]

    Regards,

    Louis.

## Billy Tanksley — 2001-04-28 05:10:12

    From: Louis Madon [mailto:madonl@...]

     >I think it's a reasonable idea for an interpreter (where the goal is

    >keeping the implementation simple) but not such a good idea for an

    >optimising compiler. Such a compiler could infer the best inlining

    >strategy without you having to annotate the source code at

    >all. The more

    >things that are inferred, the less work the programmer needs to do

    >whenever changes need to be made.

     This is true -- but his example (inlining) was trivial.  What about

    immediate functions with side effects, such as the source manipulators I was

    discussing?  Such things would have to be implemented as true immediate

    functions, otherwise the smart compiler would wisely defer deciding them

    until runtime (since they have side effects).

     >As a side note, I'm slowly continuing to make progress on my compiler.

    >This automatically eliminates all invariant expressions (like the

    >examples you give above) at compile time.

     Yes, I remember that.  Very cool.

     >Louis.

     -Billy

## Louis Madon — 2001-04-28 05:41:03

    > From: Billy Tanksley [SMTP:wtanksley@...]

     >

     > From: Louis Madon [mailto:madonl@...]

     >

     > I think it's a reasonable idea for an interpreter (where the goal

     > is keeping the implementation simple) but not such a good idea for

     > an optimising compiler. Such a compiler could infer the best

     > inlining strategy without you having to annotate the source code at

     > all. The more things that are inferred, the less work the programmer

     > needs to do whenever changes need to be made.

     >

     > This is true - but his example (inlining) was trivial. What about

     > immediate functions with side effects, such as the source > manipulators

     > I was discussing? Such things would have to be implemented as

     > true immediate functions, otherwise the smart compiler would wisely

     > defer deciding them until runtime (since they have side effects).

     Indeed. My question is: is such a language feature (ie. source

    manipulation) really a good idea? Why not just put specialised-syntax

    inside strings, as in John's embedded lisp example?

    Overall, I can't really think of any good examples that need

    side-effecting functions to run at compile-time.

    Regards,

    Louis.

## Manfred von Thun — 2001-05-02 06:56:53

    On Fri, 27 Apr 2001, John Cowan wrote:

     > Billy Tanksley wrote:

    >

    > > I don't think inlining would be beneficial to Joy,

    >

    > Probably not, given the large interpretation overhead.  I was

    > primarily thinking of the many short definitions in stdlib.joy,

    > which would be more appealing to use if they were silently

    > inlined.

     Indeed. Those short definitions in stdlib.joy should eventually

    disappear by being implemented directly in the interpreter.

    So a simple implementation would use the definitions in a

    library, a more efficient implementation (your Joy 1.1, John),

    would just say, for example

        fold     :    as if implemented by   fold  ==  [swap] dip step

     - Manfred

## Manfred von Thun — 2001-05-02 07:23:40

    On Fri, 27 Apr 2001, John Cowan wrote:

     > Billy Tanksley scripsit:

     [lots of interesting discussion about inlining, redefinitions

    compilation, interaction between these and problems for the

    Joy implementor and the Joy programmer..]

    Those interactive languages that I have used (Basic when I was innocent,

    Prolog, Common Lisp) I have hardly ever used interactively in the

    way some users can and do. Even in Joy I program in my head first, then

    on paper, then I type it in. By this time it tends to be OK. That is no

    doubt a legacy of having done most of my programming in Pascal and

    (later) C.

    So here is my question: Do people get a lot out of trying this and that,

    modify after some test.. - really use the interactiveness of the

    language to its hilt? I know that some languages have their own

    online editor with an enormous repertoire of facilities. My fear is

    that a small language (Joy, so far) with a big editor would be

    quite lopsided. I get the impression that much of the interactive-

    ness and dynamism that some people ask of Joy might just amount

    to a powerful editor. Is it too early for that?

      - Manfred

## Manfred von Thun — 2001-05-02 08:01:18

    On Fri, 27 Apr 2001, John Cowan wrote:

     > I have thought, and here's my current idea: immediacy without metasyntax.

    > The Joy scanner remains fixed, but we get access to execution during

    > definition time.

    >

    > Specifically, my notion is to allow "(" and ")" within definitions

    > to indicate a program to be executed while the definition is in

    > progress.  The Joy parser collects everything between "(" and ")"

    > and executes it.  Whatever is on the stack when that is done

    > is inserted into the definition.

     [stuff deleted]

     > Currently, "(" and ")" are tokens, but they aren't used for anything

    > in Joy.  They mostly seem to exist as a byproduct of the comment

    > syntax.  Parens outside definitions (including nested parens)

    > would be illegal under this syntax, so perhaps it would be

    > better to use some one character.

     Quite early in the development of Joy (that's prehistory now)

    I contemplated a backtracking, relational version of Joy. That

    was most easily implemented in Prolog, and I did experiment

    with it for a while. Let me explain:

       Joy programs denote unary functions from stacks to stacks.

    Unary functions are just special cases of binary relations.

    So, think of a version of Prolog in which all predicates are

    binary relations between stacks. Binary relations can be

    composed just like unary functions. One can also have disjunctions

    of binary relations, just as in Prolog. That would give

    two binary operators: 1) composition, written without an explicit

    symbol, just by concatenation, and 2) disjunction, which now

    needs an explicit (infix) symbol. Now some sort of bracketing is

    needed for complex programs. Minimise bracketing by giving

    Composition a higher precedence - but you still need brackets.

    So, using "|" for disjunction, a program might look like this:

       (a b c | d e)  (e f | g)

    similar to regular expressions or grammars. This unfolds as

        a b c e f    a b c g    d e e f   d e g

    To be useful in programming one needs another primitive, "fail".

    (In a Lisp context, this sort of thing is discussed in Henderson's

    Lispkit book.)

    Anyhow, there may be other reasons why one might want to introduce

    another binary program constructor. Once there two or more

    some bracketing is needed to override precedences. In most

    languages parentheses are used for precisely that. So I would

    like to keep this option open for Joy.

    Parentheses are of course used for lots of other purposes.

    (When our Computer Science department was still using Pascal,

    I often asked my students for how many purposes Pascal used

    parentheses. They always missed out on a few. Can people

    answer ...?) Anyhow, I did not want to use parentheses for

    lists, but used square brackets instead, following Prolog,

    Miranda, Haskell (and ML, I think). It seems to be quite

    satisfactory. But it means that square brackets cannot be

    used to override precedences of binary program constructors.

    I don't shout often, but I would   U R G E   you not to use

    round parentheses "(" and ")" for the purpose you mentioned.

    I'm sure we can think of something else, if immediate

    (is this correct Forthese?) words are required in Joy.

      - Manfred

## Manfred von Thun — 2001-05-02 08:41:08

    On Fri, 27 Apr 2001, Billy Tanksley wrote:

    [replying to John Cowan]

    >

    > The only difference I can see between us is that you don't want the

    > programmer to have access to the source in string form, and I do.  I can't

    > comprehend why you don't want the programmer to have that access.  We both

    > agree that the programmer should have high-level access.

    >

    > With that said, though, I don't really mind giving the programmer ONLY

    > high-level access.  I don't understand why, but I don't mind it.  IMO

    > high-level languages should be powerful because they offer MORE than

    > low-level ones, _not_ because they offer less.

    What I said in an earlier posting about an inbuilt editor is relevant.

    One might (but it needs discussion whether it is worthwhile) do this:

      1. Introduce a new datatype "stringbuffer" (= a string + a movable

         integer pointer)

      2. Stealing good ideas from existing text editors, Joyify (that's

         a verb) the syntax of its commands:

            42 advance                             pointer movement

            7  delete                              chars at current pointer

            "text" insert                          at current pointer

            show                                   around pointer

            etc etc ...

      3. A command to pretend that the buffer is a file to be read in

         as if by

             "newfile" include.

    There. Is it worth it?  Such a datatype is not against the spirit of

    Joy, and it might be quite useful even without the command in 3.

    But different people like different kinds of editors (grown-ups like

    myself use TECO if they can get it to work in a new environment

    - never regretted it.) So maybe it is best to let Joy programmers

    use whatever editor they are familiar with. In my own  usrlib.joy

    there is just such a command which even does 3.

    So - do we need string-buffer-with-pointer as a full scale editor?

    Discussion welcome.

      - Manfred

## John Cowan — 2001-05-02 13:30:57

    Manfred von Thun wrote:

     > So, using "|" for disjunction, a program might look like this:

    >    (a b c | d e)  (e f | g)

    > similar to regular expressions or grammars. This unfolds as

    >     a b c e f    a b c g    d e e f   d e g

     I suppose this means a b c e f | a b c g | d e e f | d e g ?

     > To be useful in programming one needs another primitive, "fail".

     So when "fail" is executed, we backtrack to the last choice point?

    But what is the analogue of unbinding, just reverting the stack?

    I think this is a fascinating idea (and quite possibly easy to

    implement); can you explicate it a bit more?

     > Anyhow, there may be other reasons why one might want to introduce

    > another binary program constructor. Once there two or more

    > some bracketing is needed to override precedences. In most

    > languages parentheses are used for precisely that. So I would

    > like to keep this option open for Joy.

     Agreed.

     > I don't shout often, but I would   U R G E   you not to use

    > round parentheses "(" and ")" for the purpose you mentioned.

    > I'm sure we can think of something else, if immediate

    > (is this correct Forthese?) words are required in Joy.

     I think that `[foo bar baz] will be quite satisfactory.

    --

    There is / one art             || John Cowan <jcowan@...>

    no more / no less              || http://www.reutershealth.com

    to do / all things             || http://www.ccil.org/~cowan

    with art- / lessness           \\ -- Piet Hein

## Manfred von Thun — 2001-05-03 07:37:44

    On Wed, 2 May 2001, John Cowan wrote:

     > Manfred von Thun wrote:

    >

    > > So, using "|" for disjunction, a program might look like this:

    > >    (a b c | d e)  (e f | g)

    > > similar to regular expressions or grammars. This unfolds as

    > >     a b c e f    a b c g    d e e f   d e g

    >

    > I suppose this means a b c e f | a b c g | d e e f | d e g ?

     Well, yes. Your line means the same as my first line, it produces

    the same unfolding when executed. Any one may fail, if all do

    then the whole program fails.

     > > To be useful in programming one needs another primitive, "fail".

    >

    > So when "fail" is executed, we backtrack to the last choice point?

     Exactly right.

     > But what is the analogue of unbinding, just reverting the stack?

     There may or may not be any Prolog-like binding and unbinding.

    If there is not, then just revert to the stack before the last

    choice point, as you said.

     > I think this is a fascinating idea (and quite possibly easy to

    > implement); can you explicate it a bit more?

     For backtrack programming, the best is Prolog. For non-Prolog

    kinds of backtracking, implemented in Pascal, see from my

    home page the SYMBOLIC PROCESSING IN PASCAL page, or directly:

      http://www.latrobe.edu.au/www/philosophy/s00bok.html

    in particular Chapters 9 10 11 15 16 19 20 and Appendices 2 3 4.

    You need a proper Pascal which treats local procedures as

    parameters in recursive calls properly (at least the Turbo Prolog

    of 10 years ago won't do it). These programs are hard to

    translate into C. Most of my students could not do it, and

    I have done only those in Chapters 9 and 10 - it really is fiendish.

    If you are thinking of adding this to Joy, think twice. As

    an exercise, translate the programs from those Chapters into C.

    For backtracking in functional languages, try Henderson's

    book on Lispkit (don't have it here at the moment). Shows

    how backtracking primitives can be added to the Lisp (SECD-)

    machine. (shouting again:)  B U T  beware, referential

    transparency is lost, as Henderson shows. So, added to Joy

    it destroys the "purely" functional. Maybe some time in

    the future, yes, but not now. Other things are more urgent.

     > > Anyhow, there may be other reasons why one might want to introduce

    > > another binary program constructor. Once there two or more

    > > some bracketing is needed to override precedences. In most

    > > languages parentheses are used for precisely that. So I would

    > > like to keep this option open for Joy.

    >

    > Agreed.

    >

     Here is another one that you would like: "||" to mean: execute

    in parallel (unix fork, or on other processors)

        (a b c || c d || e || f) g h (i || j) k

                                 ^                    wait for all to finish

    I'm not sure whether the remote execution that you have

    in mind would allow for something like this. I haven't

    thought about this much, though. But there are bound to be

    other (infix) program constructors one might find handy.

      - Manfred

## John Cowan — 2001-05-03 14:04:30

    Manfred von Thun wrote:

     >> But what is the analogue of unbinding, just reverting the stack?

    >

    > There may or may not be any Prolog-like binding and unbinding.

     If there were, what would be bound and unbound?  There doesn't

    seem to be anything in concatenative programs that corresponds

    at all naturally to the Prolog variable.

     > For backtrack programming, the best is Prolog. For non-Prolog

    > kinds of backtracking, implemented in Pascal, see from my

    > home page the SYMBOLIC PROCESSING IN PASCAL page, or directly:

    >   http://www.latrobe.edu.au/www/philosophy/s00bok.html

    > in particular Chapters 9 10 11 15 16 19 20 and Appendices 2 3 4.

     I'll investigate this.

     > You need a proper Pascal which treats local procedures as

    > parameters in recursive calls properly (at least the Turbo Prolog

    > of 10 years ago won't do it). These programs are hard to

    > translate into C. Most of my students could not do it, and

    > I have done only those in Chapters 9 and 10 - it really is fiendish.

     There are now two excellent Pascal compilers, I'm told, though

    I haven't used them myself:  GNU Pascal and Free Pascal.

    So translation to C is a thing of the past.

     > If you are thinking of adding this to Joy, think twice. As

    > an exercise, translate the programs from those Chapters into C.

     Note that GNU C does have local functions and does treat them

    properly AFAIK.  This was done so that the back end, which

    has to be able to handle local functions, could be exercised.

     > I'm not sure whether the remote execution that you have

    > in mind would allow for something like this.

     That was not what I had in mind initially, only plain

    remote execution and "background" (like & in the shell),

    not real concurrency.

    But I realized that my sandboxing approach may be a little

    *too* thorough: the client has *no* access to the resources

    of the server computer except for its processor cycles, so

    in some sense there is no point in remote execution unless

    the originating machine is bogged down.  Even a Java applet

    gets access to the screen and keyboard of the local machine,

    though not to its filesystem nor to the rest of the network.

    There needs to be an agreed-upon way for the Joy server to

    push some things on the stack (like file streams for local

    files that can be manipulated) before it imports the

    client's code.  This could be arranged at the server

    and by including them in the server's usrlib.joy file,

    but how to communicate what is available to the client?

    --

    There is / one art             || John Cowan <jcowan@...>

    no more / no less              || http://www.reutershealth.com

    to do / all things             || http://www.ccil.org/~cowan

    with art- / lessness           \\ -- Piet Hein

## Manfred von Thun — 2001-05-08 02:20:04

    On Thu, 3 May 2001, John Cowan wrote:

     > Manfred von Thun wrote:

    >

    > >> But what is the analogue of unbinding, just reverting the stack?

    > >

    > > There may or may not be any Prolog-like binding and unbinding.

    >

    > If there were, what would be bound and unbound?  There doesn't

    > seem to be anything in concatenative programs that corresponds

    > at all naturally to the Prolog variable.

     In  Joy-as-she-is  there are no Prolog variables, yes. But

    as far as I can see there is nothing to stop us to introduce

    them, and keep Joy concatenative. For example

        2 3 + X unify

    would unify X with 5,

        f(X,3)  f(2,Y)  unify

    would unify X with 2 and Y with 3.

    In one of my earliest implementations of Joy-in-Prolog I did

    have such a feature.

    But for the backtracking that I had in mind there would be no

    Prolog style unifyable ("logical") variables. Instead there

    would be either the "fail" primitive, as in

        [..] [..] [fail] ifte

    or a "check" primitive, which fails if top of stack is

    a false Boolean. It might even be defined as

        check  ==  [] [] [fail] ifte

     > > For backtrack programming, the best is Prolog. For non-Prolog

    > > kinds of backtracking, implemented in Pascal, see from my

    > > home page the SYMBOLIC PROCESSING IN PASCAL page, or directly:

    > >   http://www.latrobe.edu.au/www/philosophy/s00bok.html

    > > in particular Chapters 9 10 11 15 16 19 20 and Appendices 2 3 4.

    >

    > I'll investigate this.

    >

    > > You need a proper Pascal which treats local procedures as

    > > parameters in recursive calls properly (at least the Turbo Prolog

    > > of 10 years ago won't do it). These programs are hard to

    > > translate into C. Most of my students could not do it, and

    > > I have done only those in Chapters 9 and 10 - it really is fiendish.

    >

    > There are now two excellent Pascal compilers, I'm told, though

    > I haven't used them myself:  GNU Pascal and Free Pascal.

    > So translation to C is a thing of the past.

     I know about GNU Pascal, I haven't heard of Free Pascal.

    They seem to make confident claims about GNU Pascal. Maybe

    there will be a Pascal revival. I have tried a free P2C,

    and it failed for my purposes.

     > > If you are thinking of adding this to Joy, think twice. As

    > > an exercise, translate the programs from those Chapters into C.

    >

    > Note that GNU C does have local functions and does treat them

    > properly AFAIK.  This was done so that the back end, which

    > has to be able to handle local functions, could be exercised.

     I would like to hear whether it really does work for the

    backtracking.

     > > I'm not sure whether the remote execution that you have

    > > in mind would allow for something like this.

    >

    > That was not what I had in mind initially, only plain

    > remote execution and "background" (like & in the shell),

    > not real concurrency.

    >

    > But I realized that my sandboxing approach may be a little

    > *too* thorough: the client has *no* access to the resources

    > of the server computer except for its processor cycles, so

    > in some sense there is no point in remote execution unless

    > the originating machine is bogged down.  Even a Java applet

    > gets access to the screen and keyboard of the local machine,

    > though not to its filesystem nor to the rest of the network.

    >

    > There needs to be an agreed-upon way for the Joy server to

    > push some things on the stack (like file streams for local

    > files that can be manipulated) before it imports the

    > client's code.  This could be arranged at the server

    > and by including them in the server's usrlib.joy file,

    > but how to communicate what is available to the client?

     Much of the detail that you describe I know nothing about,

    I have never had to deal with anything like it. So I cannot

    consider myself competent to judge whether any of this

    should really be made part of Joy. Do Lisp, Scheme, ML or

    Haskell have that sort of thing? Can anyone help, please?

    I also think it is important not to get too stuck inside

    unix.

      - Manfred