# Joy1 only with NOBDW

## Heiko.Kuhrt@t-online.de — 2004-01-13 13:07:43

    Is there a simple way for non C programmers to get Joy1 to run with

    BDW-garbage collection?

    The download from Joys homepage is only 75k (not 500) and does not

    contain any BDW garbage collector. The link to Johns homepage is

    broken.

    I downloaded BDW from its own homepage but was not able to get it to run

    with Joy. I don't know anything about C.

    Any hints?

    --Heiko

## Nick Forde — 2004-01-13 13:17:58

    Hi Heiko,

    Manfred needs to update his archive script to handle the 'gc' symbolic

    link correctly or include 'gc6.0' in the tarkit.

    If you have a copy of wget you could also try downloading it using:

      wget -r http://www.latrobe.edu.au/philosophy/phimvt/joy/gc6.0

    I've not tried this though.

    Nick.

    Heiko.Kuhrt@... writes:

      > Is there a simple way for non C programmers to get Joy1 to run with

     > BDW-garbage collection?

     >

     > The download from Joys homepage is only 75k (not 500) and does not

     > contain any BDW garbage collector. The link to Johns homepage is

     > broken.

     >

     > I downloaded BDW from its own homepage but was not able to get it to run

     > with Joy. I don't know anything about C.

     >

     > Any hints?

     > --Heiko

## Heiko.Kuhrt@t-online.de — 2004-01-13 20:11:55

    On Tuesday January 13 2004 14:17, Nick Forde wrote:

     > Hi Heiko,

    >

    > Manfred needs to update his archive script to handle the 'gc'

    > symbolic link correctly or include 'gc6.0' in the tarkit.

    >

    > If you have a copy of wget you could also try downloading it using:

    >

    >   wget -r http://www.latrobe.edu.au/philosophy/phimvt/joy/gc6.0

    >

    > I've not tried this though.

    >

    > Nick.

     I just tried your suggestion. What happend was a download of alltogether

    25MB from Latrobe University: infos of all sorts about a wonderfull

    place to study.

    Among all that is the whole Joy-directory. The current joy does not

    compile with BDW-gc (There is a joy-executable of 2MB but that one says

    NOBDW, too??).

    Hidden deeper in the directory tree is a tar.gz that compiles at once

    __with__ BDW -- but does not understand  #-comments or modules.

    I think I forget about it and go on with my working NOBDW-joy.

    Nevertheless

    thank you Nick

## Nick Forde — 2004-01-13 21:10:13

    Heiko.Kuhrt@... wrote:

     >

    > I think I forget about it and go on with my working NOBDW-joy.

    >

     I've downloaded the latest stable Hans Boehm distribution (6.2)

    and integrated it into the Joy source on Manfred's site. A few

    small updates to the make.gc file were all that was required.

    I'll send you the tarkit in a private mail.

     From my own experiments I've found that the Boehm collector is

    quite a bit slower than the basic stop-and-copy collector in the

    Joy source, although it does require more runtime memory.

    Regards,

    Nick.

## John Cowan — 2004-01-13 22:02:04

    Nick Forde scripsit:

     >  From my own experiments I've found that the Boehm collector is

    > quite a bit slower than the basic stop-and-copy collector in the

    > Joy source, although it does require more runtime memory.

     Slower and larger, yes; but it does GC strings, which the built in GC

    does not.

    --

    Henry S. Thompson said, / "Syntactic, structural,               John Cowan

    Value constraints we / Express on the fly."     jcowan@...

    Simon St. Laurent: "Your / Incomprehensible     http://www.reutershealth.com

    Abracadabralike / schemas must die!"            http://www.ccil.org/~cowan

## Nick Forde — 2004-01-14 13:52:15

    John Cowan writes:

      > Nick Forde scripsit:

     >

     > >  From my own experiments I've found that the Boehm collector is

     > > quite a bit slower than the basic stop-and-copy collector in the

     > > Joy source, although it does require more runtime memory.

     >

     > Slower and larger, yes; but it does GC strings, which the built in GC

     > does not.

     Good point. I forgot about that.

    While on the topic of GC algorithms can anyone suggest a good choice

    for concatenative languages? I've been experimenting with some GC

    implementations recently and couldn't help but feel that in Joy we

    should be able to take advantage of the pure functional nature of the

    language and properties of the evaluation stack.

    Doing some literature research I came across the paper 'Garbage

    Collection on a Stack' by Wolfgang Schreiner:

      http://citeseer.nj.nec.com/schreiner94garbage.html

    I think the algorithm described there could be adapted slightly to

    handle the multiple return values required for Joy. The thing I like

    most about this is that memory compaction is performed on the fly with

    no separate garbage collection phase.

    Billy, have you considered the GC scheme you'd like to use in your

    cKlike Forth language?

    Regards,

    Nick.

## Tanksley, William D. Jr. — 2004-01-14 17:04:48

    From: Nick Forde [mailto:nickf@...]

     >Doing some literature research I came across the paper

    >'Garbage Collection on a Stack' by Wolfgang Schreiner:

    >  http://citeseer.nj.nec.com/schreiner94garbage.html

     Cool! I'll be reading that!

     >Billy, have you considered the GC scheme you'd like to use in

    >your cKlike Forth language?

     Yes. My first version will have purely manual management; I'm in a quandry

    as to what to use after that. I may go for linear management (which means

    every item has exactly one pointer, so as soon as you're done using a

    pointer you delete the object); see Henry Baker's "Linear Logic: The Forth

    Shall Be First" paper. OTOH, I may decide that ref counting will work better

    for me.

    I'm not going to go with full GC, probably not ever (unless that paper looks

    sufficiently interesting). My focus is more on small size and efficiency.

     >Nick.

     -Billy

## Heiko.Kuhrt@t-online.de — 2004-01-14 19:03:53

    Thank you Nick, once more, it worked. I now have a joy that uses less

    that 1% of my system memory after a rabbit run. The old one needed more

    than 40% (400MB)(rabbit is all strings).

    Regards,

    Heiko

## Heiko.Kuhrt@t-online.de — 2004-01-15 10:36:00

    The following is more about OCaml than about Joy but as discussion is

    about gc management it may be interesting here, too. I found the

    results really surprising.

    The following program does lots of repetitions of a simple sequence of

    words. Depending on the relation of X:Y stack is growing/shrinking

    heavily or not.

    The time Joy1 needed to execute these programs is not effected by

    relation of X:Y. It allways needs 10 sec on my system.

    rjoy needs for the same programs from 0.71sec up to 74sec.

    rjoy is my own implementation of Joy, written in OCaml. The stack is

    implemented as a simple list.

    --Heiko

    (* ===================================== *)

    bench  ==

    	 25000       #see below: 1 ... 1,000,000  (X)

    	 [null]

    	 []

    	 [dup pred]

    	 [+]

    	 linrec  pop

    calcbench ==

            200                           #see below: 5,000,000 ... 5  (Y)

            [bench]times

    test ==  [calcbench]needed-time

    (* ------------------------------------- *)

    repetitions       Joy1        rjoy

    n/                n   sec         sec

    5000,000/   1   24.890    1.44

    500,000/   10   11.740    0.71

    50,000/   100   10.560    0.71

    5,000/   1000   11.040    1.27

     500/   10000   11.970    3.89

     200/   25000   10.500    5.28

     100/   50000    9.890    7.29

     50/   100000    9.570   11.09

     25/   200000    9.590   18.65

     20/   250000   10.910   21.78

     10/   500000    9.710   39.76

     5/   1000000   10.630   74.81

    (* ===================================== *)

    Joy1: with BDW-gc

    rjoy code sniplets

                Gc.space_overhead       = 200;

    ...

    let rec exec code stack =

            match code with

            |Func(first) ::rest  ->  exec rest (first stack)

            |Def (first) ::rest  ->  exec rest (exec first stack)

            |first       ::rest  ->  exec rest (first ::stack)

            |[]                  ->  stack

    ...

    ("dup", Func(function

            | x ::r -> x ::x ::r

            | others        -> rte "dup: stack is empty"

            ));

    ("+", Func(function

        | Int(x) ::Int(y) ::r -> Int(x + y) ::r

        | others        -> rte "+: stack is too small."

            ));

    ...

     On Wednesday January 14 2004 18:04, Tanksley, William D. Jr. wrote:

    > From: Nick Forde [mailto:nickf@...]

    >

    > >Doing some literature research I came across the paper

    > >'Garbage Collection on a Stack' by Wolfgang Schreiner:

    > >  http://citeseer.nj.nec.com/schreiner94garbage.html

    >

    > Cool! I'll be reading that!

    >

    > >Billy, have you considered the GC scheme you'd like to use in

    > >your cKlike Forth language?

    >

    > Yes. My first version will have purely manual management; I'm in a

    > quandry as to what to use after that. I may go for linear management

    > (which means every item has exactly one pointer, so as soon as you're

    > done using a pointer you delete the object); see Henry Baker's

    > "Linear Logic: The Forth Shall Be First" paper. OTOH, I may decide

    > that ref counting will work better for me.

    >

    > I'm not going to go with full GC, probably not ever (unless that

    > paper looks sufficiently interesting). My focus is more on small size

    > and efficiency.

    >

    > >Nick.

    >

    > -Billy

## Serguey Zefirov — 2004-01-16 23:25:16

    Hello Nick,

    Wednesday, January 14, 2004, 5:52:15 AM, you wrote:

    NF> While on the topic of GC algorithms can anyone suggest a good choice

    NF> for concatenative languages? I've been experimenting with some GC

    NF> implementations recently and couldn't help but feel that in Joy we

    NF> should be able to take advantage of the pure functional nature of the

    NF> language and properties of the evaluation stack.

    NF> Doing some literature research I came across the paper 'Garbage

    NF> Collection on a Stack' by Wolfgang Schreiner:

    NF>   http://citeseer.nj.nec.com/schreiner94garbage.html

    NF> I think the algorithm described there could be adapted slightly to

    NF> handle the multiple return values required for Joy. The thing I like

    NF> most about this is that memory compaction is performed on the fly with

    NF> no separate garbage collection phase.

    To put some constroversy and to change mode from "lurking" one:

    ----------------------------------------------------------------------

    http://citeseer.nj.nec.com/appel87garbage.html

    A very old and simple algorithm for garbage collection gives very

    good results when the physical memory is much larger than the number

    of reachable cells. In fact, the overhead associated with allocating

    and collecting cells from the heap can be reduced to less than one

    instruction per cell by increasing the size of physical memory.

    Special hardware, intricate garbage-collection algorithms, and fancy

    compiler analysis become unnecessary.

    ----------------------------------------------------------------------

    Some ML variants puts execution frames into garbage collected heap.

    --

    Best regards,

     Serguey                            mailto:sz@...

## Nick Forde — 2004-01-19 18:47:19

    Serguey Zefirov wrote:

     > To put some constroversy and to change mode from "lurking" one:

    > ----------------------------------------------------------------------

    > http://citeseer.nj.nec.com/appel87garbage.html

    > A very old and simple algorithm for garbage collection gives very

    > good results when the physical memory is much larger than the number

    > of reachable cells. In fact, the overhead associated with allocating

    > and collecting cells from the heap can be reduced to less than one

    > instruction per cell by increasing the size of physical memory.

    > Special hardware, intricate garbage-collection algorithms, and fancy

    > compiler analysis become unnecessary.

    > ----------------------------------------------------------------------

    >

    > Some ML variants puts execution frames into garbage collected heap.

     Hi Serguey,

    In Manfred and John's Joy implementation programs are lists, data types

    are atoms or lists, and the stack is a list. Each of these lists are

    allocated on the heap. The stop-and-copy collector implemented in Joy

    is actually the same as described in this paper.

    In theory this should mean that by changing the MEMORYMAX value in

    globals.h you could run the experiments described in this paper.

    Unfortunately in practice this isn't quite true as the Joy2

    implementation uses the C stack to implement a number of the

    recursive combinators and the main evaluation loop.

    Other concatenative languages described on this list use a code stack

    avoiding the need for the C stack. See Martin Young's posting:

    http://groups.yahoo.com/group/concatenative/message/1550

    Martin, can you comment on the GC algorithm you've used in Monkey?

    Regards,

    Nick.

## phimvt@lurac.latrobe.edu.au — 2004-01-20 05:01:00

    Looks like some work for me ... :

     On Tue, 13 Jan 2004, Nick Forde wrote:

    > Hi Heiko,

    >

    > Manfred needs to update his archive script to handle the 'gc' symbolic

    > link correctly or include 'gc6.0' in the tarkit.

    I'll have to do this soon, it seems. I might need your help with

    the symbolic link though. I'll be in touch.

    [..]

    > Heiko.Kuhrt@... writes:

    [..]

    >  > The download from Joys homepage is only 75k (not 500) and does

    not

    >  > contain any BDW garbage collector. The link to Johns homepage is

    >  > broken.

    Ooops, I should have noticed that earlier. Sorry.

      - Manfred

## phimvt@lurac.latrobe.edu.au — 2004-01-20 05:15:32

    On Tue, 13 Jan 2004, Nick Forde wrote:

     > I've downloaded the latest stable Hans Boehm distribution (6.2)

    > and integrated it into the Joy source on Manfred's site. A few

    > small updates to the make.gc file were all that was required.

    > I'll send you the tarkit in a private mail.

     I never had any problems getting the John's original package to work,

    although some change needed to be made to a makefile at some later stage

    (I had help with that). A year or two later I heard that a new version of

    the Boehm collector was available but that it had a bug. So I did not

    bother to make the update. Judging by what you say, it seems to be OK now,

    Would you email me the same tarkit please, Nick?

      - Manfred

## phimvt@lurac.latrobe.edu.au — 2004-01-20 06:06:17

    On Tue, 13 Jan 2004, John Cowan wrote:

     > Nick Forde scripsit:

    >

    > >  From my own experiments I've found that the Boehm collector is

    > > quite a bit slower than the basic stop-and-copy collector in the

    > > Joy source, although it does require more runtime memory.

    >

    > Slower and larger, yes; but it does GC strings, which the built in GC

    > does not.

     And I and other concatenators are very grateful to you for this

    work and all the other additions and corrections that you made to Joy.

    Modesty compells me to clarify one important matter about the

    relative speeds of my home-grown collector versus the Boehm collector.

    It is not the case that my collector is intrinsically better than

    Boehm's, even if the figures suggest that it is. Let me explain:

    In much of my previous programming I quite often used global

    variables for efficiency, but occasionally in some recursive

    procedures had to do this trick:

        save current value of global in a local;

        modify global; do something else (possibly recurse);

        restore saved value from local to global;

    Silly me, in the earliest versions of the current Joy I used

    the same trick to save nodes of various lists (program, stack,

    lists being traversed, and so on). When I started on the garbage

    collector all sorts of nasties happened (now I say "of course").

    Fortunately the DEC C compiler was extremely helpful, and it

    produced an excellent runtime error message. So it did not take

    me long to work out what was wrong: you can't save in a local

    if doing-the-something-else-part can trigger a garbage collection.

    So all the saving has to be done not in local variables but

    in globals accessible to the collector, and these globals need

    to be lists just for this purpose. They are the various "dumps"

    in the implementation. My collector now worked correctly - well,

    it took some time to get it right.

    But note that saving a global on one of the dumps now required

    creating a new node and consing that in front of the relevant

    dump. This itself might trigger a collection, but nothing bad

    happens then. After the restoring, that new node of the dump

    can be popped off. So a fair bit of work has to be done to

    allow such saving and restoring.

    This extra work is being done irrespective of which collector

    is actually in use. If the Boehm collector is used, then it

    may well be that it would have been better to doing the saving

    in a local variable, and not a linked node on a dump. So I

    would conjecture that the Boem collector will be a fair bit

    faster if locals are used. And quite likely, being written by

    professionals, it will be faster than mine (modesty again).

    So, why not use locals with the Boehm collector? Assuming that

    the smaller implementation with my collector is still to be

    available, there would need to be two versions for all functions

    that currently need the dumps. Unfortunately there are so many Joy

    primitives that need the dumps that having two versions will

    not be a trivial effort. There are indeed several ways of doing it,

    with lots (and I mean lots) of #ifdef Boehm .. #else .., or

    with having two distinct files, or probably other methods again.

    It might become a maintenance nightmare if it is not done well.

    So, for the time being, please put up with the slower Boehm version.

      - Manfred

## Nick Forde — 2004-01-20 09:36:21

    phimvt@... writes:

      > So, why not use locals with the Boehm collector? Assuming that

     > the smaller implementation with my collector is still to be

     > available, there would need to be two versions for all functions

     > that currently need the dumps. Unfortunately there are so many Joy

     > primitives that need the dumps that having two versions will

     > not be a trivial effort. There are indeed several ways of doing it,

     > with lots (and I mean lots) of #ifdef Boehm .. #else .., or

     > with having two distinct files, or probably other methods again.

     > It might become a maintenance nightmare if it is not done well.

     Although keeping track of reachable data for the GC implementation may

    have been the motivation for the dumps variables I think the resulting

    lack of C locals is probably a good thing. See the "Stackless Python"

    project for some reasons why:

      http://www.stackless.com/

      http://www.stackless.com/spcpaper.htm

    Nick.

## phimvt@lurac.latrobe.edu.au — 2004-01-30 05:05:48

    On Tue, 20 Jan 2004 phimvt@... wrote:

     > On Tue, 13 Jan 2004, Nick Forde wrote:

    >

    > > I've downloaded the latest stable Hans Boehm distribution (6.2)

    > > and integrated it into the Joy source on Manfred's site. A few

    > > small updates to the make.gc file were all that was required.

    > > I'll send you the tarkit in a private mail.

    >

    > I never had any problems getting the John's original package to work,

    > although some change needed to be made to a makefile at some later stage

    > (I had help with that). A year or two later I heard that a new version of

    > the Boehm collector was available but that it had a bug. So I did not

    > bother to make the update. Judging by what you say, it seems to be OK now,

    > Would you email me the same tarkit please, Nick?

    >

    >   - Manfred

     Thanks for the tarkit, Nick, it arrived safe and sound. BUT ...

    Using your unpacking instruction

        $  gtar  -zxvf  joy-040113.tgz

    I get the log of what has been unpacked, such as all the README's up to

           joy/gc6.2/doc/README.changes

    and then error messages:

           gtar: skipping to next header

           gtar: Archive contains obsolescent base-64 header

           gzip: stdin: invalid compressed data  --  crc error

           gtar: Child returned status 1

           gtar: Error exit delayed from previous errors

    I tried this on two different machines, both running linux.

    I also tried as an alternative gunzip, getting the same "base-64 header"

    error message.

    After googling about for base-64 and similar, I found this informative:

           google:  gtar obsolescent base-64 header     (6-8 hits)

    Apparently this has happened to people elsewhere.

    Heiko, did you have a similar problem? Can anyone help?

      - Manfred

## Heiko.Kuhrt@t-online.de — 2004-01-30 19:07:50

    > Heiko, did you have a similar problem? Can anyone help?

     No, Nicks tarkit worked at once.

    (I have no comand "gtar" on my system. Instead I say "tar xvzf *".)

     >

    >   - Manfred

     -Heiko

## Nick Forde — 2004-01-31 10:24:15

    Manfred,

    In case the tarkit got corrupted in the mail I've added it to the

    concatenative files area.

    If you are having trouble with "gtar" on your system I suggest trying

    "gunzip" first followed by "tar xvf".

    Nick.

    Heiko.Kuhrt@... wrote:

     >>Heiko, did you have a similar problem? Can anyone help?

    >

    >

    > No, Nicks tarkit worked at once.

    >

    > (I have no comand "gtar" on my system. Instead I say "tar xvzf *".)

    >

    >

    >>  - Manfred

    >

    > -Heiko