Nick Forde — 2002-10-26 15:41:13

I've been experimenting a bit with matrix arithmetic in Joy and
noticed that mtrlib uses 'map2'. This appears to be no longer
implemented by the current interpreter. I used mapr2 instead but as
I'd encountered a similar problem before I thought it would be useful
to add a primitive 'undefs' to help identify other potential problems
in the libraries.

I've attached my C implementation of undefs as a diff file for the
current interp.c but thought my findings may be of some interest to
those on the list.

Firstly I loaded all of the libraries in the current tarkit:

inilib agglib seqlib fraclib numlib symlib grmlib
lazlib lsplib mtrlib plglib tutlib typlib usrlib

I then ran 'undefs' which listed the following symbols as
missing a definition:

toc-of-tutorial K & A
v E iff C
imp N map2 y
RECFOLDR tern l2 l1
FOLDR2 FOLDL bin ini
lis FOLDR LIST NULL
ATOM EQ CONS CDR
CAR DEFUN DEF IF
CLOSURE LAMBDA prs-trace epsilon
| $ ? QUOTE
pair unix

Running grep gave the following references (only the first are noted):

Symbol: Reference: Symbol: Reference:
--------------- -------------- --------------- --------------
toc-of-tutorial joytut.joy:7 K plglib.joy:10
& plglib.joy:10 A plglib.joy:10
v plglib.joy:10 E plglib.joy:11
iff plglib.joy:10 C plglib.joy:10
imp plglib.joy:10 N plglib.joy:9
map2 mtrlib.joy:29 y lsplib.joy:141
RECFOLDR lsplib.joy:140 tern lsplib.joy:136
l2 lsplib.joy:136 l1 lsplib.joy:136
FOLDR2 lsplib.joy:135 FOLDL lsplib.joy:129
bin lsplib.joy:127 ini lsplib.joy:125
lis lsplib.joy:125 FOLDR lsplib.joy:124
LIST lsplib.joy:112 NULL lsplib.joy:111
ATOM lsplib.joy:110 EQ lsplib.joy:109
CONS lsplib.joy:108 CDR lsplib.joy:107
CAR lsplib.joy:106 DEFUN lsplib.joy:46
DEF lsplib.joy:37 IF lsplib.joy:30
CLOSURE lsplib.joy:29 LAMBDA lsplib.joy:28
prs-trace grmlib.joy:143 epsilon grmlib.joy:93
| grmlib.joy:14 $ grmlib.joy:85
? grmlib.joy:13 QUOTE grmlib.joy:88
pair seqlib.joy:128 unix usrlib.joy:48

From a quick glance through the source it appears the majority of
these are simply due to obsolete code, old notes, and misplaced
comments. However, if you've ever used '0 setundeferror' (e.g. by
ignoring 'standard-setting') you may find this to be of some help. I'm
sure there's a lesson in this for me somewhere :-)

Regards,

Nick.

interp.c.diff
17a18

> PRIVATE void undefs_();

1386a1388,1405

> PRIVATE void undefs_(void)
> {
> int col = 0, len = 20;
> Entry *i = symtabindex;
>
> while (i != symtab) {
> if (((--i)->name[0] != 0) && (i->name[0] != '_') && (i->u.body == NULL)) {
> if(strlen(i->name) > len) { printf("\n"); col = 4; }
> printf("%-*s", len, i->name);
> if (++col > 3) {
> printf("\n");
> col = 0;
> }
> }
> }
> if (col != 0) printf("\n");
> }
>

2315a2335,2337

>
> {"undefs", undefs_, "->",
> "Prints all undefined symbols in the current symbol table."},

Manfred von Thun — 2002-10-28 04:33:51

On Sat, 26 Oct 2002, Nick Forde wrote:

> I've been experimenting a bit with matrix arithmetic in Joy and
> noticed that mtrlib uses 'map2'. This appears to be no longer
> implemented by the current interpreter.

Thank you. Ooops, Yes, Sorry. (It used to be in seqlib.joy, see below)

> I used mapr2 instead

Exactly right.
To make the naming of combinators more systematic, I had changed
the name from map2 to mapr2 in seqlib.joy, but I failed to update
seqlib.joy on the web page.
Things like map, step, fold and so on are implemented by going
from left to right in the aggregates. In some languages they would
be called mapl, stepl, foldl. The corresponding mapr stepr foldr
are more naturally expressed recursively, but at a performance
penalty. One day I shall get around to programming map2 step2 fold2
in interpret.c. In the meantime mapr2 will do, because for
associative binary operators it makes no difference in the
result whether the algorithm goes from left to right or the
other way around.

> but as
> I'd encountered a similar problem before I thought it would be useful
> to add a primitive 'undefs' to help identify other potential problems
> in the libraries.

Yes, it would be useful. It also highlights another problem with Joy
(or is it a "feature"?): symbols that are used solely as data are
stored in the same table as everything else. On the other hand,
doing it this way makes it very easy to use symbols in a library
which the user is supposed to define. If I understand C++ jargon
correctly, such a symbol would be called a virtual function of
an object. So - how should a hypothetical Joy2 handle this?

> I've attached my C implementation of undefs as a diff file for the
> current interp.c but thought my findings may be of some interest to
> those on the list.

Thank you very much. But I now have to confess my ignorance
(please don't moan): I know how to use diff, I know how to read
the output, I know how to use the output to patch manually,
I know there is an automatic facility - but I have no idea
on how to use it. Three lines from you would save me a lot of time.

>
> Firstly I loaded all of the libraries in the current tarkit:
>
> inilib agglib seqlib fraclib numlib symlib grmlib
> lazlib lsplib mtrlib plglib tutlib typlib usrlib
>
> I then ran 'undefs' which listed the following symbols as
> missing a definition:

[.. long output deleted]

> Running grep gave the following references (only the first are noted):
>
> Symbol: Reference: Symbol: Reference:
> --------------- -------------- --------------- --------------
> toc-of-tutorial joytut.joy:7 K plglib.joy:10
> & plglib.joy:10 A plglib.joy:10
> v plglib.joy:10 E plglib.joy:11
> iff plglib.joy:10 C plglib.joy:10
> imp plglib.joy:10 N plglib.joy:9

All the ones from plglib (propositional logic) are intentional,
they are never called by Joy but interpreted in some way in plglib.joy

> map2 mtrlib.joy:29 y lsplib.joy:141

Yes, map2 was my mistake.

> RECFOLDR lsplib.joy:140 tern lsplib.joy:136
> l2 lsplib.joy:136 l1 lsplib.joy:136
> FOLDR2 lsplib.joy:135 FOLDL lsplib.joy:129
> bin lsplib.joy:127 ini lsplib.joy:125
> lis lsplib.joy:125 FOLDR lsplib.joy:124
> LIST lsplib.joy:112 NULL lsplib.joy:111
> ATOM lsplib.joy:110 EQ lsplib.joy:109
> CONS lsplib.joy:108 CDR lsplib.joy:107
> CAR lsplib.joy:106 DEFUN lsplib.joy:46
> DEF lsplib.joy:37 IF lsplib.joy:30
> CLOSURE lsplib.joy:29 LAMBDA lsplib.joy:28

Same with lsplib.joy

> prs-trace grmlib.joy:143 epsilon grmlib.joy:93

prs-trace is an example of a "virtual Joy operator". I should
have documented that better.

> | grmlib.joy:14 $ grmlib.joy:85
> ? grmlib.joy:13 QUOTE grmlib.joy:88

Again, intentional. Never called, but interpreted in grmlib.joy

> pair seqlib.joy:128 unix usrlib.joy:48

Ooops, pair should be a call to pairlist (in the definition of cartesian
product). I have fixed that, too, because it gave a runtime error
when running mtrtst.joy

> From a quick glance through the source it appears the majority of
> these are simply due to obsolete code, old notes, and misplaced
> comments.

Actually, no, as I explained above. To summarise
1. There were 2 genuine errors:
a) in mtrlib.joy: map2 (several times) should be mapr2
(and also note the name change: matrtst.joy is now mtrtst.joy)
b) in seqlib.joy: pair (in cartesian product) should be pairlist.
2. Some symbols are not intended to be called by Joy
but are intended to be interpreted (by a case or something)
3. Some symbols are intended to be called after they have
been defined later - this is useful for
a) mutual recursion
b) "virtual" operators, defined by the user of a library.

> However, if you've ever used '0 setundeferror' (e.g. by
> ignoring 'standard-setting') you may find this to be of some help. I'm
> sure there's a lesson in this for me somewhere :-)

^^^^^^ for me, too.

> Regards,
>
> Nick.

> interp.c.diff
> 17a18
> > PRIVATE void undefs_();
> 1386a1388,1405
> > PRIVATE void undefs_(void)
> > {
> > int col = 0, len = 20;
> > Entry *i = symtabindex;
> >
> > while (i != symtab) {
> > if (((--i)->name[0] != 0) && (i->name[0] != '_') && (i->u.body == NULL)) {
> > if(strlen(i->name) > len) { printf("\n"); col = 4; }
> > printf("%-*s", len, i->name);
> > if (++col > 3) {
> > printf("\n");
> > col = 0;
> > }
> > }
> > }
> > if (col != 0) printf("\n");
> > }
> >
> 2315a2335,2337
> >
> > {"undefs", undefs_, "->",
> > "Prints all undefined symbols in the current symbol table."},

Thank you very much for that. Sorry about the mess.

- Manfred

John Cowan — 2002-10-28 04:52:09

Manfred von Thun scripsit:

> Thank you very much. But I now have to confess my ignorance
> (please don't moan): I know how to use diff, I know how to read
> the output, I know how to use the output to patch manually,
> I know there is an automatic facility - but I have no idea
> on how to use it. Three lines from you would save me a lot of time.

The "patch" command accepts diffs on the standard input and does the
Right Thing to the files mentioned.

--
Winter: MIT, John Cowan
Keio, INRIA, jcowan@...
Issue lots of Drafts. http://www.ccil.org/~cowan
So much more to understand! http://www.reutershealth.com
Might simplicity return? (A "tanka", or extended haiku)

Nick Forde — 2002-10-28 14:14:55

Manfred von Thun writes:

> > From a quick glance through the source it appears the majority of
> > these are simply due to obsolete code, old notes, and misplaced
> > comments.
> Actually, no, as I explained above. To summarise
> 1. There were 2 genuine errors:
> a) in mtrlib.joy: map2 (several times) should be mapr2
> (and also note the name change: matrtst.joy is now mtrtst.joy)
> b) in seqlib.joy: pair (in cartesian product) should be pairlist.
> 2. Some symbols are not intended to be called by Joy
> but are intended to be interpreted (by a case or something)
> 3. Some symbols are intended to be called after they have
> been defined later - this is useful for
> a) mutual recursion
> b) "virtual" operators, defined by the user of a library.

Manfred,

Many thanks for taking the time to look at this in detail (much more
than I did!). Case 3 was new to me. I think if a library contains
"virtual" operators either the language, or at least the interpreter,
should help identify them. This is kind of tricky today as libraries
may consist of multiple LIBRA or DEFINE statements, the symbol
namespace is global and definitions and executable code may be
interleaved.

One solution given the current interpreter would be to do something like:

DEFINE virtual == "error: Undefined virtual symbol encountered." quit.
DEFINE mysymbol == virtual.

Or alternatively define a new primitive to flag all symbols in a list
virtual if they are still undefined:

[mysymbol1 mysymbol2 mysymbol3] virtualdefs.

The undefs command could then explicitly list all symbols defined as
"virtual" in addition to those with no definition at all.

This approach is, however, only slightly more useful than code
comments and a convention that programmers must remember to
adopt. Personally I'd rather have libraries become more
object-oriented with a hierarchy of namespaces and symbols explicitly
defined as either public, private or virtual. This would also remove
the need for HIDE.

To enforce compile time checking of virtual symbols though I think the
language would have to prevent the interleaving of definitions with
executable code.

Regards,

Nick.

Manfred von Thun — 2002-11-04 06:15:00

On Sat, 26 Oct 2002, Nick Forde wrote:

> [...]
> I've attached my C implementation of undefs as a diff file for the
> current interp.c but thought my findings may be of some interest to
> those on the list.

Done. See the new interp.c on the Joy page. Thank you very much.
Thanks also to John Cowan and to Nick Forde for teaching me how to use
the unix patch utility (old dog, new tricks ..)

> [...]

- Manfred

Manfred von Thun — 2002-11-04 07:04:49

On Mon, 28 Oct 2002, Nick Forde wrote:

[...]
> One solution given the current interpreter would be to do something like:
>
> DEFINE virtual == "error: Undefined virtual symbol encountered." quit.
> DEFINE mysymbol == virtual.

That would be a simple and very useful convention.
It would not require any change to the interpreter.

> Or alternatively define a new primitive to flag all symbols in a list
> virtual if they are still undefined:
>
> [mysymbol1 mysymbol2 mysymbol3] virtualdefs.
>
> The undefs command could then explicitly list all symbols defined as
> "virtual" in addition to those with no definition at all.

Yes, maybe. It would only require a small change in the interpreter.
I'm not so sure whether I like it better than the earlier solution,
because that one puts the "virtual" about where "mysymbol" would
have been defined.

> This approach is, however, only slightly more useful than code
> comments and a convention that programmers must remember to
> adopt. Personally I'd rather have libraries become more
> object-oriented with a hierarchy of namespaces and symbols explicitly
> defined as either public, private or virtual.

Indeed. I have been thinking about this for a while. But I
have not yet seen any existing language which does this in a way
which I understand *and* know how to adapt to Joy.

Any thoughts anyone?

> This would also remove
> the need for HIDE.

I would reduce the need, certainly. But might there be cases
where a simple HIDE is all one needs?

> To enforce compile time checking of virtual symbols though I think the
> language would have to prevent the interleaving of definitions with
> executable code.

And that would be disastrous for interactive use. Apart from
the similarity with Forth, Joy resembles Lisp, Scheme and Prolog
in being weakly dynamically typed - this has disadvantages and
advantages of course. Maybe a future Joy2 will have all sorts
of checking at compile time. Any ideas?

- Manfred

Manfred von Thun — 2002-11-08 02:23:46

I have reasons to believe that our News server does not
handle postings properly. It seems that the outgoing wire
has come unstuck, and nobody here believes it. When I
post to a newsgroup, the posting almost immediately
appears in the group as seen from within the campus.
However, it does not appear even a day later on the
Google news archive, and does not provoke any reactions
from the outside world. (Maybe I should post something
about bombs and wait for the CIA?) [Oh dear, I just did it...]

My suspicion is that our news server is not configured
properly and that outgoings postings only remain within
the campus. So I am asking for a small favour:

Yesterday, Thursday 7-Nov, I posted:
comp.lang.forth "Readable forth.."
comp.lang.scheme "Small self-interpreting interpreter"

Would somebody kind in the concatenative group please
find out whether these one or the other (and hence presumably
both) articles are visible in the outside world ?
I would be most grateful.

- Manfred