Joy1 only with NOBDW
Heiko.Kuhrt@t-online.de — 2004-01-13 13:07:43
Is there a simple way for non C programmers to get Joy1 to run with
BDW-garbage collection?
The download from Joys homepage is only 75k (not 500) and does not
contain any BDW garbage collector. The link to Johns homepage is
broken.
I downloaded BDW from its own homepage but was not able to get it to run
with Joy. I don't know anything about C.
Any hints?
--Heiko
Nick Forde — 2004-01-13 13:17:58
Hi Heiko,
Manfred needs to update his archive script to handle the 'gc' symbolic
link correctly or include 'gc6.0' in the tarkit.
If you have a copy of wget you could also try downloading it using:
wget -r
http://www.latrobe.edu.au/philosophy/phimvt/joy/gc6.0
I've not tried this though.
Nick.
Heiko.Kuhrt@... writes:
> Is there a simple way for non C programmers to get Joy1 to run with
> BDW-garbage collection?
>
> The download from Joys homepage is only 75k (not 500) and does not
> contain any BDW garbage collector. The link to Johns homepage is
> broken.
>
> I downloaded BDW from its own homepage but was not able to get it to run
> with Joy. I don't know anything about C.
>
> Any hints?
> --Heiko
Heiko.Kuhrt@t-online.de — 2004-01-13 20:11:55
On Tuesday January 13 2004 14:17, Nick Forde wrote:
> Hi Heiko,
>
> Manfred needs to update his archive script to handle the 'gc'
> symbolic link correctly or include 'gc6.0' in the tarkit.
>
> If you have a copy of wget you could also try downloading it using:
>
> wget -r http://www.latrobe.edu.au/philosophy/phimvt/joy/gc6.0
>
> I've not tried this though.
>
> Nick.
I just tried your suggestion. What happend was a download of alltogether
25MB from Latrobe University: infos of all sorts about a wonderfull
place to study.
Among all that is the whole Joy-directory. The current joy does not
compile with BDW-gc (There is a joy-executable of 2MB but that one says
NOBDW, too??).
Hidden deeper in the directory tree is a tar.gz that compiles at once
__with__ BDW -- but does not understand #-comments or modules.
I think I forget about it and go on with my working NOBDW-joy.
Nevertheless
thank you Nick
Nick Forde — 2004-01-13 21:10:13
Heiko.Kuhrt@... wrote:
>
> I think I forget about it and go on with my working NOBDW-joy.
>
I've downloaded the latest stable Hans Boehm distribution (6.2)
and integrated it into the Joy source on Manfred's site. A few
small updates to the make.gc file were all that was required.
I'll send you the tarkit in a private mail.
From my own experiments I've found that the Boehm collector is
quite a bit slower than the basic stop-and-copy collector in the
Joy source, although it does require more runtime memory.
Regards,
Nick.
John Cowan — 2004-01-13 22:02:04
Nick Forde scripsit:
> From my own experiments I've found that the Boehm collector is
> quite a bit slower than the basic stop-and-copy collector in the
> Joy source, although it does require more runtime memory.
Slower and larger, yes; but it does GC strings, which the built in GC
does not.
--
Henry S. Thompson said, / "Syntactic, structural, John Cowan
Value constraints we / Express on the fly."
jcowan@...
Simon St. Laurent: "Your / Incomprehensible
http://www.reutershealth.com
Abracadabralike / schemas must die!"
http://www.ccil.org/~cowan
Nick Forde — 2004-01-14 13:52:15
John Cowan writes:
> Nick Forde scripsit:
>
> > From my own experiments I've found that the Boehm collector is
> > quite a bit slower than the basic stop-and-copy collector in the
> > Joy source, although it does require more runtime memory.
>
> Slower and larger, yes; but it does GC strings, which the built in GC
> does not.
Good point. I forgot about that.
While on the topic of GC algorithms can anyone suggest a good choice
for concatenative languages? I've been experimenting with some GC
implementations recently and couldn't help but feel that in Joy we
should be able to take advantage of the pure functional nature of the
language and properties of the evaluation stack.
Doing some literature research I came across the paper 'Garbage
Collection on a Stack' by Wolfgang Schreiner:
http://citeseer.nj.nec.com/schreiner94garbage.html
I think the algorithm described there could be adapted slightly to
handle the multiple return values required for Joy. The thing I like
most about this is that memory compaction is performed on the fly with
no separate garbage collection phase.
Billy, have you considered the GC scheme you'd like to use in your
cKlike Forth language?
Regards,
Nick.
Tanksley, William D. Jr. — 2004-01-14 17:04:48
From: Nick Forde [mailto:
nickf@...]
>Doing some literature research I came across the paper
>'Garbage Collection on a Stack' by Wolfgang Schreiner:
> http://citeseer.nj.nec.com/schreiner94garbage.html
Cool! I'll be reading that!
>Billy, have you considered the GC scheme you'd like to use in
>your cKlike Forth language?
Yes. My first version will have purely manual management; I'm in a quandry
as to what to use after that. I may go for linear management (which means
every item has exactly one pointer, so as soon as you're done using a
pointer you delete the object); see Henry Baker's "Linear Logic: The Forth
Shall Be First" paper. OTOH, I may decide that ref counting will work better
for me.
I'm not going to go with full GC, probably not ever (unless that paper looks
sufficiently interesting). My focus is more on small size and efficiency.
>Nick.
-Billy
Heiko.Kuhrt@t-online.de — 2004-01-14 19:03:53
Thank you Nick, once more, it worked. I now have a joy that uses less
that 1% of my system memory after a rabbit run. The old one needed more
than 40% (400MB)(rabbit is all strings).
Regards,
Heiko
Heiko.Kuhrt@t-online.de — 2004-01-15 10:36:00
The following is more about OCaml than about Joy but as discussion is
about gc management it may be interesting here, too. I found the
results really surprising.
The following program does lots of repetitions of a simple sequence of
words. Depending on the relation of X:Y stack is growing/shrinking
heavily or not.
The time Joy1 needed to execute these programs is not effected by
relation of X:Y. It allways needs 10 sec on my system.
rjoy needs for the same programs from 0.71sec up to 74sec.
rjoy is my own implementation of Joy, written in OCaml. The stack is
implemented as a simple list.
--Heiko
(* ===================================== *)
bench ==
25000 #see below: 1 ... 1,000,000 (X)
[null]
[]
[dup pred]
[+]
linrec pop
calcbench ==
200 #see below: 5,000,000 ... 5 (Y)
[bench]times
test == [calcbench]needed-time
(* ------------------------------------- *)
repetitions Joy1 rjoy
n/ n sec sec
5000,000/ 1 24.890 1.44
500,000/ 10 11.740 0.71
50,000/ 100 10.560 0.71
5,000/ 1000 11.040 1.27
500/ 10000 11.970 3.89
200/ 25000 10.500 5.28
100/ 50000 9.890 7.29
50/ 100000 9.570 11.09
25/ 200000 9.590 18.65
20/ 250000 10.910 21.78
10/ 500000 9.710 39.76
5/ 1000000 10.630 74.81
(* ===================================== *)
Joy1: with BDW-gc
rjoy code sniplets
Gc.space_overhead = 200;
...
let rec exec code stack =
match code with
|Func(first) ::rest -> exec rest (first stack)
|Def (first) ::rest -> exec rest (exec first stack)
|first ::rest -> exec rest (first ::stack)
|[] -> stack
...
("dup", Func(function
| x ::r -> x ::x ::r
| others -> rte "dup: stack is empty"
));
("+", Func(function
| Int(x) ::Int(y) ::r -> Int(x + y) ::r
| others -> rte "+: stack is too small."
));
...
On Wednesday January 14 2004 18:04, Tanksley, William D. Jr. wrote:
> From: Nick Forde [mailto:nickf@...]
>
> >Doing some literature research I came across the paper
> >'Garbage Collection on a Stack' by Wolfgang Schreiner:
> > http://citeseer.nj.nec.com/schreiner94garbage.html
>
> Cool! I'll be reading that!
>
> >Billy, have you considered the GC scheme you'd like to use in
> >your cKlike Forth language?
>
> Yes. My first version will have purely manual management; I'm in a
> quandry as to what to use after that. I may go for linear management
> (which means every item has exactly one pointer, so as soon as you're
> done using a pointer you delete the object); see Henry Baker's
> "Linear Logic: The Forth Shall Be First" paper. OTOH, I may decide
> that ref counting will work better for me.
>
> I'm not going to go with full GC, probably not ever (unless that
> paper looks sufficiently interesting). My focus is more on small size
> and efficiency.
>
> >Nick.
>
> -Billy
Serguey Zefirov — 2004-01-16 23:25:16
Hello Nick,
Wednesday, January 14, 2004, 5:52:15 AM, you wrote:
NF> While on the topic of GC algorithms can anyone suggest a good choice
NF> for concatenative languages? I've been experimenting with some GC
NF> implementations recently and couldn't help but feel that in Joy we
NF> should be able to take advantage of the pure functional nature of the
NF> language and properties of the evaluation stack.
NF> Doing some literature research I came across the paper 'Garbage
NF> Collection on a Stack' by Wolfgang Schreiner:
NF>
http://citeseer.nj.nec.com/schreiner94garbage.html
NF> I think the algorithm described there could be adapted slightly to
NF> handle the multiple return values required for Joy. The thing I like
NF> most about this is that memory compaction is performed on the fly with
NF> no separate garbage collection phase.
To put some constroversy and to change mode from "lurking" one:
----------------------------------------------------------------------
http://citeseer.nj.nec.com/appel87garbage.html
A very old and simple algorithm for garbage collection gives very
good results when the physical memory is much larger than the number
of reachable cells. In fact, the overhead associated with allocating
and collecting cells from the heap can be reduced to less than one
instruction per cell by increasing the size of physical memory.
Special hardware, intricate garbage-collection algorithms, and fancy
compiler analysis become unnecessary.
----------------------------------------------------------------------
Some ML variants puts execution frames into garbage collected heap.
--
Best regards,
Serguey mailto:
sz@...
Nick Forde — 2004-01-19 18:47:19
Serguey Zefirov wrote:
> To put some constroversy and to change mode from "lurking" one:
> ----------------------------------------------------------------------
> http://citeseer.nj.nec.com/appel87garbage.html
> A very old and simple algorithm for garbage collection gives very
> good results when the physical memory is much larger than the number
> of reachable cells. In fact, the overhead associated with allocating
> and collecting cells from the heap can be reduced to less than one
> instruction per cell by increasing the size of physical memory.
> Special hardware, intricate garbage-collection algorithms, and fancy
> compiler analysis become unnecessary.
> ----------------------------------------------------------------------
>
> Some ML variants puts execution frames into garbage collected heap.
Hi Serguey,
In Manfred and John's Joy implementation programs are lists, data types
are atoms or lists, and the stack is a list. Each of these lists are
allocated on the heap. The stop-and-copy collector implemented in Joy
is actually the same as described in this paper.
In theory this should mean that by changing the MEMORYMAX value in
globals.h you could run the experiments described in this paper.
Unfortunately in practice this isn't quite true as the Joy2
implementation uses the C stack to implement a number of the
recursive combinators and the main evaluation loop.
Other concatenative languages described on this list use a code stack
avoiding the need for the C stack. See Martin Young's posting:
http://groups.yahoo.com/group/concatenative/message/1550
Martin, can you comment on the GC algorithm you've used in Monkey?
Regards,
Nick.
phimvt@lurac.latrobe.edu.au — 2004-01-20 05:01:00
Looks like some work for me ... :
On Tue, 13 Jan 2004, Nick Forde wrote:
> Hi Heiko,
>
> Manfred needs to update his archive script to handle the 'gc' symbolic
> link correctly or include 'gc6.0' in the tarkit.
I'll have to do this soon, it seems. I might need your help with
the symbolic link though. I'll be in touch.
[..]
> Heiko.Kuhrt@... writes:
[..]
> > The download from Joys homepage is only 75k (not 500) and does
not
> > contain any BDW garbage collector. The link to Johns homepage is
> > broken.
Ooops, I should have noticed that earlier. Sorry.
- Manfred
phimvt@lurac.latrobe.edu.au — 2004-01-20 05:15:32
On Tue, 13 Jan 2004, Nick Forde wrote:
> I've downloaded the latest stable Hans Boehm distribution (6.2)
> and integrated it into the Joy source on Manfred's site. A few
> small updates to the make.gc file were all that was required.
> I'll send you the tarkit in a private mail.
I never had any problems getting the John's original package to work,
although some change needed to be made to a makefile at some later stage
(I had help with that). A year or two later I heard that a new version of
the Boehm collector was available but that it had a bug. So I did not
bother to make the update. Judging by what you say, it seems to be OK now,
Would you email me the same tarkit please, Nick?
- Manfred
phimvt@lurac.latrobe.edu.au — 2004-01-20 06:06:17
On Tue, 13 Jan 2004, John Cowan wrote:
> Nick Forde scripsit:
>
> > From my own experiments I've found that the Boehm collector is
> > quite a bit slower than the basic stop-and-copy collector in the
> > Joy source, although it does require more runtime memory.
>
> Slower and larger, yes; but it does GC strings, which the built in GC
> does not.
And I and other concatenators are very grateful to you for this
work and all the other additions and corrections that you made to Joy.
Modesty compells me to clarify one important matter about the
relative speeds of my home-grown collector versus the Boehm collector.
It is not the case that my collector is intrinsically better than
Boehm's, even if the figures suggest that it is. Let me explain:
In much of my previous programming I quite often used global
variables for efficiency, but occasionally in some recursive
procedures had to do this trick:
save current value of global in a local;
modify global; do something else (possibly recurse);
restore saved value from local to global;
Silly me, in the earliest versions of the current Joy I used
the same trick to save nodes of various lists (program, stack,
lists being traversed, and so on). When I started on the garbage
collector all sorts of nasties happened (now I say "of course").
Fortunately the DEC C compiler was extremely helpful, and it
produced an excellent runtime error message. So it did not take
me long to work out what was wrong: you can't save in a local
if doing-the-something-else-part can trigger a garbage collection.
So all the saving has to be done not in local variables but
in globals accessible to the collector, and these globals need
to be lists just for this purpose. They are the various "dumps"
in the implementation. My collector now worked correctly - well,
it took some time to get it right.
But note that saving a global on one of the dumps now required
creating a new node and consing that in front of the relevant
dump. This itself might trigger a collection, but nothing bad
happens then. After the restoring, that new node of the dump
can be popped off. So a fair bit of work has to be done to
allow such saving and restoring.
This extra work is being done irrespective of which collector
is actually in use. If the Boehm collector is used, then it
may well be that it would have been better to doing the saving
in a local variable, and not a linked node on a dump. So I
would conjecture that the Boem collector will be a fair bit
faster if locals are used. And quite likely, being written by
professionals, it will be faster than mine (modesty again).
So, why not use locals with the Boehm collector? Assuming that
the smaller implementation with my collector is still to be
available, there would need to be two versions for all functions
that currently need the dumps. Unfortunately there are so many Joy
primitives that need the dumps that having two versions will
not be a trivial effort. There are indeed several ways of doing it,
with lots (and I mean lots) of #ifdef Boehm .. #else .., or
with having two distinct files, or probably other methods again.
It might become a maintenance nightmare if it is not done well.
So, for the time being, please put up with the slower Boehm version.
- Manfred
Nick Forde — 2004-01-20 09:36:21
phimvt@... writes:
> So, why not use locals with the Boehm collector? Assuming that
> the smaller implementation with my collector is still to be
> available, there would need to be two versions for all functions
> that currently need the dumps. Unfortunately there are so many Joy
> primitives that need the dumps that having two versions will
> not be a trivial effort. There are indeed several ways of doing it,
> with lots (and I mean lots) of #ifdef Boehm .. #else .., or
> with having two distinct files, or probably other methods again.
> It might become a maintenance nightmare if it is not done well.
Although keeping track of reachable data for the GC implementation may
have been the motivation for the dumps variables I think the resulting
lack of C locals is probably a good thing. See the "Stackless Python"
project for some reasons why:
http://www.stackless.com/
http://www.stackless.com/spcpaper.htm
Nick.
phimvt@lurac.latrobe.edu.au — 2004-01-30 05:05:48
On Tue, 20 Jan 2004
phimvt@... wrote:
> On Tue, 13 Jan 2004, Nick Forde wrote:
>
> > I've downloaded the latest stable Hans Boehm distribution (6.2)
> > and integrated it into the Joy source on Manfred's site. A few
> > small updates to the make.gc file were all that was required.
> > I'll send you the tarkit in a private mail.
>
> I never had any problems getting the John's original package to work,
> although some change needed to be made to a makefile at some later stage
> (I had help with that). A year or two later I heard that a new version of
> the Boehm collector was available but that it had a bug. So I did not
> bother to make the update. Judging by what you say, it seems to be OK now,
> Would you email me the same tarkit please, Nick?
>
> - Manfred
Thanks for the tarkit, Nick, it arrived safe and sound. BUT ...
Using your unpacking instruction
$ gtar -zxvf joy-040113.tgz
I get the log of what has been unpacked, such as all the README's up to
joy/gc6.2/doc/README.changes
and then error messages:
gtar: skipping to next header
gtar: Archive contains obsolescent base-64 header
gzip: stdin: invalid compressed data -- crc error
gtar: Child returned status 1
gtar: Error exit delayed from previous errors
I tried this on two different machines, both running linux.
I also tried as an alternative gunzip, getting the same "base-64 header"
error message.
After googling about for base-64 and similar, I found this informative:
google: gtar obsolescent base-64 header (6-8 hits)
Apparently this has happened to people elsewhere.
Heiko, did you have a similar problem? Can anyone help?
- Manfred
Heiko.Kuhrt@t-online.de — 2004-01-30 19:07:50
> Heiko, did you have a similar problem? Can anyone help?
No, Nicks tarkit worked at once.
(I have no comand "gtar" on my system. Instead I say "tar xvzf *".)
>
> - Manfred
-Heiko
Nick Forde — 2004-01-31 10:24:15
Manfred,
In case the tarkit got corrupted in the mail I've added it to the
concatenative files area.
If you are having trouble with "gtar" on your system I suggest trying
"gunzip" first followed by "tar xvf".
Nick.
Heiko.Kuhrt@... wrote:
>>Heiko, did you have a similar problem? Can anyone help?
>
>
> No, Nicks tarkit worked at once.
>
> (I have no comand "gtar" on my system. Instead I say "tar xvzf *".)
>
>
>> - Manfred
>
> -Heiko