# Some more thoughts on equivalence in Joy

## Nick Forde — 2002-12-06 13:42:29

    Whilst travelling to work today I had some more thoughts about

    equivalence in Joy. Rather than keep them to myself I've made some

    notes below in the hope that they will elicit some comments or

    alternative suggestions.

    Ideally equivalence atoms should provide a balanced tradeoff between

    performance, programmer convenience and value in identifying possible

    bugs. They should also offer a clear and consistent model for their

    use so that it isn't necessary to keep referring to a language

    reference manual.

    One model would be to split the atoms into those that test both type

    and value, from those that test only value (with type coercion if

    necessary). I suggest something like the following:

    Type + Value Tests

    ------------------

      _eq    : Internal convenience function that performs pointer comparison.

      eql    : First calls _eq, then compares types, followed by values.

      equal  : Recursively calls eql on aggregates. Same as eql for scalars.

      le     : Same types and less than or equal

      lt     : Same types and less than

      gt     : Same types and greater than

      ge     : Same types and greater than or equal

      ne     : Same types not equal to

      cmp    : Same types comparison, returns -1, 0, or 1

    Value Tests

    -----------

      =      : Numeric equality

      <=     : Numeric less than or equal

      <      : Numeric less than

       >      : Numeric greater than

      >=     : Numeric greater than or equal

       !=     : Numeric not equal to

    Joy currently has the following built-in types:

      definition, boolean, char, integer, set, string, list, float, file

    Not all equivalence tests make sense for all types so to help clarify

    where they are applicable we can group the types into classes as

    follows:

    Scalar Types    : definition, boolean, char, integer, float, file

    Aggregate Types : set, string, list

    Numeric Types   : boolean, char, integer, float

    Ordrered Types  : boolean, char, float, integer, string

    The above categorisation is pretty subjective and could be rehashed in

    a number of ways. For example, I've included both boolean and char as

    numeric types to make it easier to write polymorphic functions. At the

    same time I've tried not to expose too much of the underlying

    implementation details by including sets in the scalar or numeric

    types.

    These classes can also be used to conveniently categorise the valid

    types for the atoms above.

    Scalar Types    : eq_, eql, equal              (type and value tests)

    Aggregate Types : equal                        (type and value test)

    Numeric Types   : <=, <, =, >, >=, !=          (value tests)

    Ordrered Types  : eql, le, lt, gt, ge, ne, cmp (type and value tests)

    Note that the only types where coercion is allowed are Numeric

    Types where the following hierarchy may be used:

      float    ^

      integer  |

      char     |

      boolean  |

    For the numeric equivalence atoms given any two of the above types the

    lower can always be cast to match the higher before value testing.

    I've only considered the current Joy types in the above but this model

    seems to extend OK when considering other potential builtins such as

    arrays, hashes, bit-vectors, stacks, queues, tuples, etc.

    Given the above equality tests there are a number of convenience

    combinators that could be implemented. Two that have already been

    mentioned are 'in' and 'has'. I suggest that these should call 'eql'

    by default. I also like Manfred's suggestion of creating c-in and

    c-has. To be complete I think c-equal might also be useful, either as

    a library function or primitive.

    Perhaps given some free time over the weekend I'll be able to more

    closely compare the above with a number of other languages. Can anyone

    suggest alternative languages which may be interesting to look at

    because they tackle equality in a novel way?

    Regards,

    Nick.

## Manfred von Thun — 2002-12-09 05:56:09

    I also had the idea of revising the entire area afresh, with little

    or no regard to the current implementation. I started writing it up,

    but your schema is already more comprehensive than mine. So I'll

    just add my thoughts to what you have.

     On Fri, 6 Dec 2002, Nick Forde wrote:

    >      They should also offer a clear and consistent model for their

    > use so that it isn't necessary to keep referring to a language

    > reference manual.

    >

    > One model would be to split the atoms into those that test both type

    > and value, from those that test only value (with type coercion if

    > necessary). I suggest something like the following:

    >

    > Type + Value Tests

    > ------------------

    >   _eq    : Internal convenience function that performs pointer comparison.

    >   eql    : First calls _eq, then compares types, followed by values.

    In the current implementation the first call to _eq will only ever

    yield true for strings and lists (quotations), because for everything

    else a copy of the value is made:

        [1 2 3]  dup  _eq                ==>   true

        "abcde"  dup  _eq                ==>   true

        {1 2 3}  dup  _eq                ==>   false

        1234567  dup  _eq                ==>   false

    >   equal  : Recursively calls eql on aggregates. Same as eql for scalars.

                                          ^^^^^^^^^^

    Unless you have in mind some new future recursive aggregates, at

    least currently this would be just lists (or better, quotations).

    The other two aggregates, sets and strings, would not need recursive

    calls for traversal.

    >   le     : Same types and less than or equal

    >   lt     : Same types and less than

    >   gt     : Same types and greater than

    >   ge     : Same types and greater than or equal

    >   ne     : Same types not equal to

    >   cmp    : Same types comparison, returns -1, 0, or 1

    >

    > Value Tests

    > -----------

    >   =      : Numeric equality

    >   <=     : Numeric less than or equal

    >   <      : Numeric less than

    >   >      : Numeric greater than

    >   >=     : Numeric greater than or equal

    >   !=     : Numeric not equal to

    Sets, strings  and symbols need to be fitted somewhere into this scheme.

    For sets: The <= and < symbols look sufficiently like the usual

    subset and proper-subset symbols that it would plausible to

    use them for sets (implemented by a for-loop, 0..31, that does

    bitwise comparisons. Same for >= and >. Of course < and > are

    much less useful than <= and >=. [Incidentally, at the moment

    Joy gets it the wrong way around:  {1 3} {1 2 3 4 5}  <= should

    return true, but returns false. Similarly for >=  .] Also, note

    that sets (in Joy and in real life) are only partially ordered,

    whereas all the numerics are linearly ordered.

    Strings: There are two orderings one might consider:

    1. Lexicographical ordering, which is linear, and currently in Joy

    (implemented by strcmp if I remember right). 2. The less well known prefix

    ordering, which is only a partial ordering. ("slow" is a prefix of

    "slowly" and hence "slow" is also lexicographically before "slowly".

    By contrast, "slack" is lexicgraphically before slowly, but "slack"

    is not a prefix of "slowly".) I think that <= and friends should

    be used for lexicographic ordering, as they currently are. But the

    prefix ordering, with different symbols, should also be considered

    for strings and lists - and perhaps even for sets.

    Symbols: (such as dup map foo Paris), can occur in quotations, or

    unquoted on the stack:

        [dup] first [map] first  <=

    This sort of problem arises when one has to sort a list such as

        [Peter Paul Mary]  qsort

    I thinks it is best to treat them just like strings. However,

    comparison of symbols and strings is problematic. Currently Joy

    will happily compare them, by looking at the symbol table string

    for symbols. But maybe all symbols should be less than all strings,

    or maybe vice versa.

    > Joy currently has the following built-in types:

    >   definition, boolean, char, integer, set, string, list, float, file

    >

    > Not all equivalence tests make sense for all types so to help clarify

    > where they are applicable we can group the types into classes as

    > follows:

    >

    > Scalar Types    : definition, boolean, char, integer, float, file

    > Aggregate Types : set, string, list

    > Numeric Types   : boolean, char, integer, float

    > Ordrered Types  : boolean, char, float, integer, string

    There is a possibility of ambiguity here:

    1. "ordered" means that the members of the type are ordered

         a) integers, reals, strings are linearly ordered by < <= etc

         b) sets are partially ordered by inclusion

         c) strings under prefix are partially ordered

    2. "ordered" is a property of aggregates of some types

         a) strings and lists are ordered (I called them sequences)

         b) sets are not ordered

       The ordering of members in an aggregate has nothing to do with

       the ordering of those members in their type. Example:

         in  [5 3] the 5 comes before the 3, although in the type

         of integers 3 comes before 5.

    I am not sure what standard terminology is.

    > The above categorisation is pretty subjective and could be rehashed in

    > a number of ways. For example, I've included both boolean and char as

    > numeric types to make it easier to write polymorphic functions. At the

    > same time I've tried not to expose too much of the underlying

    > implementation details by including sets in the scalar or numeric

    > types.

    >

    > These classes can also be used to conveniently categorise the valid

    > types for the atoms above.

    >

    > Scalar Types    : eq_, eql, equal              (type and value tests)

    > Aggregate Types : equal                        (type and value test)

    > Numeric Types   : <=, <, =, >, >=, !=          (value tests)

    > Ordrered Types  : eql, le, lt, gt, ge, ne, cmp (type and value tests)

    >

    > Note that the only types where coercion is allowed are Numeric

    > Types where the following hierarchy may be used:

    >

    >   float    ^

    >   integer  |

    >   char     |

    >   boolean  |

    >

    > For the numeric equivalence atoms given any two of the above types the

    > lower can always be cast to match the higher before value testing.

    Entirely agree.

    > I've only considered the current Joy types in the above but this model

    > seems to extend OK when considering other potential builtins such as

    > arrays, hashes, bit-vectors, stacks, queues, tuples, etc.

    Future possibilities - it seems endless !

    > Given the above equality tests there are a number of convenience

    > combinators that could be implemented. Two that have already been

    > mentioned are 'in' and 'has'. I suggest that these should call 'eql'

    > by default. I also like Manfred's suggestion of creating c-in and

    > c-has. To be complete I think c-equal might also be useful, either as

    > a library function or primitive.

    I have been thinking about other combinators, too. There are several

    combinators which traverse one aggregate: step, map, filter, fold.

    There really is a need to have counterparts which traverse two

    aggregates: step2, map2, filter2, fold2 (which I mention already

    in agglib). They should travers the aggregates from left to right,

    but it is not possible to do that by defining them in a library.

    Two other combinators, all and some, could similarly be given

    counterparts all2 and some2 which traverse two aggregates but

    stop traversal when the (truth value) result cannot change.

    Examples:

    1.    [1 2 3]  [10 20 30]  [+]  map2           ==>   [11 22 33]

    2.    [1 2 3]  [10 20 30]  0 [+ +] fold2       ==>   66

    3.    [1 2 3]  [10 20 30]  true [< and] fold2  ==>   true

    4.    [1 2 3]  [10 2  30]  [<]  filter2        ==>   [1 3] [10 30]

    5.    [1 2 3]  [10 20 30]  [<]  all2           ==>   true

    As examples 3 and 5 show, some of the comparison relations need could

    also be implemented like this. Still, for efficiency I would be inclined

    to allow as _primitives_ the following:

          [1 2 3] [10 20 30]  <                    ==>   true

          "abc"   "xyz"       <                    ==>   true

    > Perhaps given some free time over the weekend I'll be able to more

    > closely compare the above with a number of other languages. Can anyone

    > suggest alternative languages which may be interesting to look at

    > because they tackle equality in a novel way?

    Nothing comes to mind.

    > Regards,

    >

    > Nick.

    Thank you for all this.

      - Manfred

## Nick Forde — 2002-12-09 14:21:54

    Manfred,

    Your reply has made me think a bit more about the choices I made in

    selecting equivalence atoms. Interestingly I think we are tackling the

    same problem from slightly different angles.

    Please correct me if I'm wrong, but your approach seems to be to

    overload the '<', '=', '>', '<=', and '>=' primitives to make them as

    general purpose, and powerful, as possible. This gives the programmer

    less atoms to learn and possibly also makes it easier to write some

    polymorphic functions.

    My approach was to try and keep the primitives as simple as possible

    by reducing their applicability, and hopefully also make them easier

    to understand. This requires the programmer to select the most

    appropriate atom for their data types, hopefully with some performance

    gain as a result. It also reduces the number of 'corner cases' which

    the programmer must learn. This does have the disadvantage of

    increasing the number of required primitives and putting more

    responsibility on the programmer.

    Perhaps your approach is truer to the spirit of Joy, I don't

    know.

    I've added a number of other comments below.

    Manfred von Thun writes:

      > > One model would be to split the atoms into those that test both type

     > > and value, from those that test only value (with type coercion if

     > > necessary). I suggest something like the following:

     > >

     > > Type + Value Tests

     > > ------------------

     > >   _eq    : Internal convenience function that performs pointer comparison.

     > >   eql    : First calls _eq, then compares types, followed by values.

     >

     > In the current implementation the first call to _eq will only ever

     > yield true for strings and lists (quotations), because for everything

     > else a copy of the value is made:

      [some examples...]

    Agreed. This is why I identified it for interpreter internal use

    only. As it is the quickest means of testing both type and value

    equality I think it is worth using to help define the other atoms.

      > >   equal  : Recursively calls eql on aggregates. Same as eql for scalars.

     >                                       ^^^^^^^^^^

     > Unless you have in mind some new future recursive aggregates, at

     > least currently this would be just lists (or better, quotations).

     > The other two aggregates, sets and strings, would not need recursive

     > calls for traversal.

     Sorry, I wasn't clear in what I meant here.

    I was trying to say that I was intending 'equal' to be the only test

    to allow comparison of hierarchies of aggregates, with 'eql' being

    used to test the leaf scalar nodes. Although in practice this wouldn't

    require a recursive implementation on sets and strings, and 'eql'

    wouldn't actually be called, I think it is useful to define the

    primitive in this way. It helps clarify the intended behaviour and

    makes it easier to remember the difference between equal and eql

    without having to refer to a language reference manual. It also helps

    hide the implementation details from the programmer.

      > > Type and Value Tests

     > > --------------------

     > >   le     : Same types and less than or equal

     > >   lt     : Same types and less than

     > >   gt     : Same types and greater than

     > >   ge     : Same types and greater than or equal

     > >   ne     : Same types not equal to

     > >   cmp    : Same types comparison, returns -1, 0, or 1

     > >

     > > Value Tests

     > > -----------

     > >   =      : Numeric equality

     > >   <=     : Numeric less than or equal

     > >   <      : Numeric less than

     > >   >      : Numeric greater than

     > >   >=     : Numeric greater than or equal

     > >   !=     : Numeric not equal to

     >

     > Sets, strings  and symbols need to be fitted somewhere into this scheme.

     >

     > For sets: The <= and < symbols look sufficiently like the usual

     > subset and proper-subset symbols that it would plausible to

     > use them for sets (implemented by a for-loop, 0..31, that does

     > bitwise comparisons. Same for >= and >. Of course < and > are

     > much less useful than <= and >=. [Incidentally, at the moment

     > Joy gets it the wrong way around:  {1 3} {1 2 3 4 5}  <= should

     > return true, but returns false. Similarly for >=  .] Also, note

     > that sets (in Joy and in real life) are only partially ordered,

     > whereas all the numerics are linearly ordered.

     Although I agree that subset and proper-subset tests are required

    personally I wouldn't overload the '<=' and '<' symbols for this. I

    say this firstly because this means that these tests now have a

    'special case' which doesn't fit well the the scalar numeric types. I

    find such special cases to be ugly and hard to remember. However, I

    can see that you may perhaps argue that the generality this introduces

    makes these tests more powerful. Secondly, ideally sets should be able

    to store non-numeric values. This means these tests would have to

    become applicable to non-numeric types. Thirdly, although perhaps

    least importantly, the numeric tests I described can be implemented

    relatively efficiently if their applicability is restricted to only

    types that can be coerced to match each other.

      > Strings: There are two orderings one might consider:

     >

     > 1. Lexicographical ordering, which is linear, and currently in Joy

     > (implemented by strcmp if I remember right). 2. The less well known prefix

     > ordering, which is only a partial ordering. ("slow" is a prefix of

     > "slowly" and hence "slow" is also lexicographically before "slowly".

     > By contrast, "slack" is lexicgraphically before slowly, but "slack"

     > is not a prefix of "slowly".) I think that <= and friends should

     > be used for lexicographic ordering, as they currently are. But the

     > prefix ordering, with different symbols, should also be considered

     > for strings and lists - and perhaps even for sets.

     I introduced the type and value tests (le, lt, gt, ge, ne, cmp)

    specifically to handle testing objects which really need their types

    to be the same for comparison. For strings I intended these to apply

    lexicographical tests although I see I accidentally omitted this from

    my previous email. Once again I don't like the overloading of '<=' for

    the reasons given above.

      > Symbols: (such as dup map foo Paris), can occur in quotations, or

     > unquoted on the stack:

     >     [dup] first [map] first  <=

     > This sort of problem arises when one has to sort a list such as

     >     [Peter Paul Mary]  qsort

     > I thinks it is best to treat them just like strings. However,

     > comparison of symbols and strings is problematic. Currently Joy

     > will happily compare them, by looking at the symbol table string

     > for symbols. But maybe all symbols should be less than all strings,

     > or maybe vice versa.

     Notice that in my model only 'eql' may be applied to symbols, i.e. I

    deliberately avoided allowing the comparison of definitions and

    strings. I did this because I thought that should this be required it

    would always be possible to use a combination of 'name' and 'intern'.

      > > Joy currently has the following built-in types:

     > >   definition, boolean, char, integer, set, string, list, float, file

     > >

     > > Not all equivalence tests make sense for all types so to help clarify

     > > where they are applicable we can group the types into classes as

     > > follows:

     > >

     > > Scalar Types    : definition, boolean, char, integer, float, file

     > > Aggregate Types : set, string, list

     > > Numeric Types   : boolean, char, integer, float

     > > Ordrered Types  : boolean, char, float, integer, string

     >

     > There is a possibility of ambiguity here:

     > 1. "ordered" means that the members of the type are ordered

     >      a) integers, reals, strings are linearly ordered by < <= etc

     >      b) sets are partially ordered by inclusion

     >      c) strings under prefix are partially ordered

     > 2. "ordered" is a property of aggregates of some types

     >      a) strings and lists are ordered (I called them sequences)

     >      b) sets are not ordered

     >    The ordering of members in an aggregate has nothing to do with

     >    the ordering of those members in their type. Example:

     >      in  [5 3] the 5 comes before the 3, although in the type

     >      of integers 3 comes before 5.

     > I am not sure what standard terminology is.

     You are right, I hadn't properly considered ordering of aggregates

    (although I did include string in this type group). The ordered types

    I listed were really meant to be "Ordered Scalar Types", although

    "string" doesn't fit well here in my model as I inconsistently treated

    them as aggregates and scalars.

      > > Scalar Types    : eq_, eql, equal              (type and value tests)

     > > Aggregate Types : equal                        (type and value test)

     > > Numeric Types   : <=, <, =, >, >=, !=          (value tests)

     > > Ordrered Types  : eql, le, lt, gt, ge, ne, cmp (type and value tests)

     Being able to create a table like this helps me to understand and

    remember which equality tests are appropriate for a particular

    situation. Your approach of making the equality atoms as general

    purpose as possible to some extent may reduce the need for such a

    table, however, the corner cases become a bit more difficult to

    remember. For example, if '<=' applied to sets means 'is subset of',

    and applied to strings it means 'lexicographically less than or equal

    to', what about when applied to lists? I think cases such as this are

    what often confuse newcomers to a language.

      > [...]

     > > I've only considered the current Joy types in the above but this model

     > > seems to extend OK when considering other potential builtins such as

     > > arrays, hashes, bit-vectors, stacks, queues, tuples, etc.

     >

     > Future possibilities - it seems endless !

     I think support for arrays would be top of my wish list. :-)

      > > Given the above equality tests there are a number of convenience

     > > combinators that could be implemented. Two that have already been

     > > mentioned are 'in' and 'has'. I suggest that these should call 'eql'

     > > by default. I also like Manfred's suggestion of creating c-in and

     > > c-has. To be complete I think c-equal might also be useful, either as

     > > a library function or primitive.

     >

     > I have been thinking about other combinators, too. There are several

     > combinators which traverse one aggregate: step, map, filter, fold.

     > There really is a need to have counterparts which traverse two

     > aggregates: step2, map2, filter2, fold2 (which I mention already

     > in agglib). They should travers the aggregates from left to right,

     > but it is not possible to do that by defining them in a library.

     > Two other combinators, all and some, could similarly be given

     > counterparts all2 and some2 which traverse two aggregates but

     > stop traversal when the (truth value) result cannot change.

     > Examples:

     > 1.    [1 2 3]  [10 20 30]  [+]  map2           ==>   [11 22 33]

     > 2.    [1 2 3]  [10 20 30]  0 [+ +] fold2       ==>   66

     > 3.    [1 2 3]  [10 20 30]  true [< and] fold2  ==>   true

     > 4.    [1 2 3]  [10 2  30]  [<]  filter2        ==>   [1 3] [10 30]

     > 5.    [1 2 3]  [10 20 30]  [<]  all2           ==>   true

     > As examples 3 and 5 show, some of the comparison relations need could

     > also be implemented like this.

     Good idea! I think these would be very useful.

      >  Still, for efficiency I would be inclined

     > to allow as _primitives_ the following:

     >       [1 2 3] [10 20 30]  <                    ==>   true

     I argue that if we know all types are numeric and make '<' as

    efficient as possible, i.e. limit it to numeric types, then there may

    not be much of a performance difference between:

    [1 2 3] [10 20 30] [<] all2

    and the above, where '<' also allows set and string comparison.

      >       "abc"   "xyz"       <                    ==>  true

              "abc"   "xyz"      lt                    ==>  true

    :-)

    Regards,

    Nick.

## Manfred von Thun — 2002-12-10 05:43:16

    On Mon, 9 Dec 2002, Nick Forde wrote:

    [..]

    > Please correct me if I'm wrong, but your approach seems to be to

    > overload the '<', '=', '>', '<=', and '>=' primitives to make them as

    > general purpose, and powerful, as possible.

    Yes, in analogy with + - * / and so on, which most languages

    overload, including Joy.

    >                                             This gives the programmer

    > less atoms to learn and possibly also makes it easier to write some

    > polymorphic functions.

    >

    > My approach was to try and keep the primitives as simple as possible

    > by reducing their applicability, and hopefully also make them easier

    > to understand. This requires the programmer to select the most

    > appropriate atom for their data types, hopefully with some performance

    > gain as a result.

    Yes, the type tests will reduce performance, unfortunately.

    On the other hand, a more efficient interpreter might be able to

    select the appropriate test-free operation on the basis of

    what the previous operation produced.

    >                  It also reduces the number of 'corner cases' which

    > the programmer must learn. This does have the disadvantage of

    > increasing the number of required primitives and putting more

    > responsibility on the programmer.

    >

    > Perhaps your approach is truer to the spirit of Joy, I don't

    > know.

    Well, apart from concatenative notation + quotations + combinators,

    the "spirit of Joy" really leaves this question quite open.

    One could imagine Joy having two daughters: max-poly-Joy which

    maximises polymorphism, and min-poly-Joy which minimises it.

    There are arguments on both sides.

    [..]

    >  > >   _eq    : Internal convenience function that performs pointer comparison.

    >  > >   eql    : First calls _eq, then compares types, followed by values.

    >  >

    >  > In the current implementation the first call to _eq will only ever

    >  > yield true for strings and lists (quotations), because for everything

    >  > else a copy of the value is made:

    >  [some examples...]

    >

    > Agreed. This is why I identified it for interpreter internal use

    > only.

    I see. Sorry, I misread the "internal".

    [..]

    >  > > Type and Value Tests

    >  > > --------------------

    >  > >   le     : Same types and less than or equal

    >  > >   lt     : Same types and less than

    >  > >   gt     : Same types and greater than

    >  > >   ge     : Same types and greater than or equal

    >  > >   ne     : Same types not equal to

    >  > >   cmp    : Same types comparison, returns -1, 0, or 1

    Note that these are predicates, returning a logical value.

    For consistency one would want to do the same for arithmetic

    operators which return a number:

             plus   : Same types, to be added

             minus  : Same types, to be added

             ...

    On the other hand, one could introduce a sametype-check operator

    which aborts when the two arguments are not of the same type. Then

             plus  ==  sametype-check +

    This would then also work for <= < and so on. And incidentally,

             sametype-check  ==  [sametype] [] [abort] ifte

    can be used to define it. If sametype-check is to be used frequently,

    then of course it should be made a new primitive.

    [..]

    >  > Symbols: (such as dup map foo Paris), can occur in quotations, or

    >  > unquoted on the stack:

    >  >     [dup] first [map] first  <=

    >  > This sort of problem arises when one has to sort a list such as

    >  >     [Peter Paul Mary]  qsort

    >  > I thinks it is best to treat them just like strings. However,

    >  > comparison of symbols and strings is problematic. Currently Joy

    >  > will happily compare them, by looking at the symbol table string

    >  > for symbols. But maybe all symbols should be less than all strings,

    >  > or maybe vice versa.

    >

    > Notice that in my model only 'eql' may be applied to symbols, i.e. I

    > deliberately avoided allowing the comparison of definitions and

    > strings. I did this because I thought that should this be required it

    > would always be possible to use a combination of 'name' and 'intern'.

    Well, yes. Of course sorting should be reasonably efficient, though.

    It's all debatable, though.

    [..]

    >  > There is a possibility of ambiguity here:

    >  > 1. "ordered" means that the members of the type are ordered

    >  >      a) integers, reals, strings are linearly ordered by < <= etc

    >  >      b) sets are partially ordered by inclusion

    >  >      c) strings under prefix are partially ordered

    >  > 2. "ordered" is a property of aggregates of some types

    >  >      a) strings and lists are ordered (I called them sequences)

    >  >      b) sets are not ordered

    >  >    The ordering of members in an aggregate has nothing to do with

    >  >    the ordering of those members in their type. Example:

    >  >      in  [5 3] the 5 comes before the 3, although in the type

    >  >      of integers 3 comes before 5.

    >  > I am not sure what standard terminology is.

    >

    > You are right, I hadn't properly considered ordering of aggregates

    > (although I did include string in this type group).

    It occurred to me that Smalltalk has perhaps the oldest systematic

    type system, and the terminology used there might well have become

    more or less widely used. I seem to remember:

         Collection                          (Joy's set, string, list ?)

           Unordered collection              (Joy's set ?)

           Ordered collection                (Joy's string and list ?)

    We should look at languages with well-developed type hierarchies.

    [..]

    > table, however, the corner cases become a bit more difficult to

    > remember. For example, if '<=' applied to sets means 'is subset of',

    > and applied to strings it means 'lexicographically less than or equal

    > to', what about when applied to lists?

    Thanks, I left that out. Flat lists (not containing other lists) which

    contain only truth values, chars or numbers are sufficiently like strings

    that they should be treated the same way. Even if they contain strings

    or sets that is not a problem, provided <= is defined for the base

    type of the flat list. What should be done if the list contains other

    lists I do not know at this stage.

    [..]

    >  > > I've only considered the current Joy types in the above but this model

    >  > > seems to extend OK when considering other potential builtins such as

    >  > > arrays, hashes, bit-vectors, stacks, queues, tuples, etc.

    >  >

    >  > Future possibilities - it seems endless !

    >

    > I think support for arrays would be top of my wish list. :-)

    Well, a survey of the literature on purely functional languages

    (Haskell .., but not Scheme, Lisp or ML) would be the first step.

    I seem to remember that there is a problem about arrays in a purely

    functional setting. On the other hand, this might be an argument

    to make non-pure concessions.

    [..]

    So, will it be minpoly-Joy or maxpoly-Joy ?

    At any rate, thanks for your thoughts.

      - Manfred

## Nick Forde — 2002-12-11 10:45:49

    Manfred von Thun writes:

      > >  > > I've only considered the current Joy types in the above but this model

     > >  > > seems to extend OK when considering other potential builtins such as

     > >  > > arrays, hashes, bit-vectors, stacks, queues, tuples, etc.

     > >  >

     > >  > Future possibilities - it seems endless !

     > >

     > > I think support for arrays would be top of my wish list. :-)

     >

     > Well, a survey of the literature on purely functional languages

     > (Haskell .., but not Scheme, Lisp or ML) would be the first step.

     > I seem to remember that there is a problem about arrays in a purely

     > functional setting. On the other hand, this might be an argument

     > to make non-pure concessions.

     Details of the Haskell array library can be found here:

    http://www.haskell.org/onlinelibrary/array.html

    I don't know the theory behind this library and so unfortunately can't

    comment on it's implications.

      > So, will it be minpoly-Joy or maxpoly-Joy ?

     Either is really fine with me as long as the equality operators are

    consistent and easy to remember.

    On a related point I'd actually love to see a definition of the core

    joy primitives from which all other functions may be derived (min-Joy

    :-)). I think it would be useful to categorise primitives into those

    which must be implemented in the interpreter from those which are

    provided for performance or convenience and could appear in a

    library. Brent's examination of combinators should provide an

    excellent starting point.

    Browsing through the "Haskell 98 Report" I notice that the Haskell

    standard type classes are actually quite close to what I was trying to

    achieve by categorising primitives into type groups. I wish I'd come

    across this before.

    http://www.haskell.org/onlinereport/basic.html

    http://www.haskell.org/onlinereport/basic.html#standard-classes

    I also notice that the Haskell '==' works for all (or at least most)

    types, but is only defined on pairs of objects of the same

    type. i.e. it is much the same as the 'equal' primitive I proposed.

    Regards,

    Nick.

## Manfred von Thun — 2002-12-12 03:03:07

    On Wed, 11 Dec 2002, Nick Forde wrote:

    [..]

    >  > So, will it be minpoly-Joy or maxpoly-Joy ?

    >

    > Either is really fine with me as long as the equality operators are

    > consistent and easy to remember.

    Well, after some further thought on this I think that Joy is

    already locked into maxpoly. Just consider null, cons, uncons,

    concat, step, map, fold, filter .. . They really continue the

    tradition of polymorphism for +, -, *, <, <= .. for the numeric

    types (which it so happens for these, might use type coercion).

    > On a related point I'd actually love to see a definition of the core

    > joy primitives from which all other functions may be derived (min-Joy

    > :-)). I think it would be useful to categorise primitives into those

    > which must be implemented in the interpreter from those which are

    > provided for performance or convenience and could appear in a

    > library.

    I often wondered that myself. The question can be taken at several

    levels.

    At the lowest level (probably not the one you have in mind) for the

    arithmetic operators on natural numbers (0 1 2 ..) we can do with

    just the successor operation together with recursion to define

    +, -, * .. . Recursive function theory is full of this. An older

    book by Zohar Manna does the same sort of thing for strings, I seem

    to remember. And of course the same for (Lisp-like) lists is

    in the literature, too.

    At much the same level, an interesting minumum is that given by

    the question: what is the minimum number of primitives needed for

    writing a mini-Joy interpreter which can interpret itself (interpreting

    itself .. interpreting itself). That is in the paper "A Joy interpreter

    written in Joy", where the last few pages describe such a beast

    of 12 lines or so. There are also some test runs.

    At a more reasonable level, I have at times thought to have what

    for now I will call a "fake library": it looks as though this is

    being read by the interpreter on start-up, when in fact everything

    in it is implemented as a primitive for efficiency. I think this

    is the sort of level you have in mind.

    But a final thought is this: such a minimal set of primitives will

    not be unique. Just remember truth functional logic or Boolean

    algebra: 'and' with 'not' is sufficient, 'or' with 'not' is

    sufficient, 'nand' and 'nor' are  sufficient just on their own.

    In Joy one could have either ot the definitions

        swoncat   ==   swap concat

        concat    ==   swap swoncat

    > Brent's examination of combinators should provide an

    > excellent starting point.

    Indeed. It is a major piece of theoretical work. I hope he

    gets all the kudos for it that he deserves, and not just from

    the friends of Joy.

    [..]

    > http://www.haskell.org/onlinereport/basic.html

    > http://www.haskell.org/onlinereport/basic.html#standard-classes

    Yes, moan moan. A masterpiece of language design, implementation

    and documentation. A whole army of researchers worked on Haskell

    and its predecessors. Really a "masterspiece". Something to

    aspire to, moan moan...

    You are quite right in suggesting it as a source of inspiration

    for Joy. It seems that the object oriented languages do have

    a lot to offer even for a language like Joy.

    Any definite proposals from you I shall take very seriously,

    provided of course that they do not break too much of

    existing code. As far as I am concerned, that means the

    libraries - but there may be others in the group who would

    want a say, too.

    I'll probably take off for my Christmas vacation very soon,

    so for the time being from me it is

                Happy  Christmas  to  all

     - Manfred

## Manfred von Thun — 2002-12-12 03:44:21

    I just happened to look at the Gnu Smalltalk manual table of contents:

        http://www.gnu.org/software/smalltalk/gst-manual/gst_toc.html

     On Mon, 9 Dec 2002, Nick Forde wrote:

    [..]

    > Please correct me if I'm wrong, but your approach seems to be to

    > overload the '<', '=', '>', '<=', and '>=' primitives to make them as

    > general purpose, and powerful, as possible. This gives the programmer

    > less atoms to learn and possibly also makes it easier to write some

    > polymorphic functions.

    I noticed that Smalltalk also uses these same symbols for the

    set relations of subset, proper-subset.. . So Joy is really

    not too unconventional in this respect.

    [..]

     - Manfred

## Brian T Rice — 2002-12-12 14:18:38

    On Thu, 12 Dec 2002, Manfred von Thun wrote:

     > I just happened to look at the Gnu Smalltalk manual table of contents:

    >     http://www.gnu.org/software/smalltalk/gst-manual/gst_toc.html

    >

    > On Mon, 9 Dec 2002, Nick Forde wrote:

    > > Please correct me if I'm wrong, but your approach seems to be to

    > > overload the '<', '=', '>', '<=', and '>=' primitives to make them as

    > > general purpose, and powerful, as possible. This gives the programmer

    > > less atoms to learn and possibly also makes it easier to write some

    > > polymorphic functions.

    >

    > I noticed that Smalltalk also uses these same symbols for the

    > set relations of subset, proper-subset.. . So Joy is really

    > not too unconventional in this respect.

     No, it doesn't!

     > - Manfred

     --

    Brian T. Rice

    LOGOS Research and Development

    mailto:water@...

    http://tunes.org/~water/

## Brian T Rice — 2002-12-12 14:19:55

    Except very anomalously in GNU Smalltalk.

    --

    Brian T. Rice

    LOGOS Research and Development

    mailto:water@...

    http://tunes.org/~water/

## Nick Forde — 2002-12-12 16:58:56

    Manfred von Thun writes:

      >

     > On Wed, 11 Dec 2002, Nick Forde wrote:

     > [..]

     > >  > So, will it be minpoly-Joy or maxpoly-Joy ?

     > >

     > > Either is really fine with me as long as the equality operators are

     > > consistent and easy to remember.

     >

     > Well, after some further thought on this I think that Joy is

     > already locked into maxpoly. Just consider null, cons, uncons,

     > concat, step, map, fold, filter .. . They really continue the

     > tradition of polymorphism for +, -, *, <, <= .. for the numeric

     > types (which it so happens for these, might use type coercion).

     Don't get me wrong, I'm all for polymorphic functions, I was just

    questioning whether they made sense for the lowest level equality

    primitives. I think you've convinced me that they do.

      > > On a related point I'd actually love to see a definition of the core

     > > joy primitives from which all other functions may be derived (min-Joy

     > > :-)). I think it would be useful to categorise primitives into those

     > > which must be implemented in the interpreter from those which are

     > > provided for performance or convenience and could appear in a

     > > library.

     >

     > I often wondered that myself. The question can be taken at several

     > levels.

     >

     > At the lowest level (probably not the one you have in mind) for the

     > arithmetic operators on natural numbers (0 1 2 ..) we can do with

     > just the successor operation together with recursion to define

     > +, -, * .. . Recursive function theory is full of this. An older

     > book by Zohar Manna does the same sort of thing for strings, I seem

     > to remember. And of course the same for (Lisp-like) lists is

     > in the literature, too.

     This is actually close to what I had in mind, with some restriction as

    to what may be considered useful. For example, as you point out

    recursive implementations of +, -, *, etc. using pred and succ may

    give some nice library definitions on integers but not quite so nice

    on floats, etc. In this case I think it is therefore resonable to

    define these as primitives. Other functions such as those for file/io

    couldn't be reasonably written as library functions and so would also

    have to be identified as primitives.

    Given such a base set of primitives it then becomes very easy to fully

    spec. out the language since all library definitions may be built from

    the ground up.

      > At much the same level, an interesting minumum is that given by

     > the question: what is the minimum number of primitives needed for

     > writing a mini-Joy interpreter which can interpret itself (interpreting

     > itself .. interpreting itself). That is in the paper "A Joy interpreter

     > written in Joy", where the last few pages describe such a beast

     > of 12 lines or so. There are also some test runs.

     >

     > At a more reasonable level, I have at times thought to have what

     > for now I will call a "fake library": it looks as though this is

     > being read by the interpreter on start-up, when in fact everything

     > in it is implemented as a primitive for efficiency. I think this

     > is the sort of level you have in mind.

     For practical purposes yes, this is what I'd like to have.

      > But a final thought is this: such a minimal set of primitives will

     > not be unique. Just remember truth functional logic or Boolean

     > algebra: 'and' with 'not' is sufficient, 'or' with 'not' is

     > sufficient, 'nand' and 'nor' are  sufficient just on their own.

     > In Joy one could have either ot the definitions

     >     swoncat   ==   swap concat

     >     concat    ==   swap swoncat

     I think Brent's work has also helped illustrate that the "best" choice

    of primitives isn't always obvious. Though it may be fun to give it a

    try!

     [...]

      > I'll probably take off for my Christmas vacation very soon,

     > so for the time being from me it is

     >

     >             Happy  Christmas  to  all

     Thanks for your feedback. Have a good holiday.

    Regards,

    Nick.

## Manfred von Thun — 2002-12-16 07:20:07

    On Thu, 12 Dec 2002, Brian T Rice wrote:

     > On Thu, 12 Dec 2002, Manfred von Thun wrote:

    >

    > > I just happened to look at the Gnu Smalltalk manual table of contents:

    > >     http://www.gnu.org/software/smalltalk/gst-manual/gst_toc.html

     Having looked at it a bit more, I think a future extension of Joy

    will have a lot to learn from this.

     > > On Mon, 9 Dec 2002, Nick Forde wrote:

    > > > Please correct me if I'm wrong, but your approach seems to be to

    > > > overload the '<', '=', '>', '<=', and '>=' primitives to make them as

    > > > general purpose, and powerful, as possible. This gives the programmer

    > > > less atoms to learn and possibly also makes it easier to write some

    > > > polymorphic functions.

    > >

    > > I noticed that Smalltalk also uses these same symbols for the

    > > set relations of subset, proper-subset.. . So Joy is really

    > > not too unconventional in this respect.

    >

    > No, it doesn't!

     If by "it" you meant Smalltalk, maybe Gnu Smalltalk is actually

    an extension - it doesn't really matter for the point I was making.

    The reference to the exact section is:

     http://www.gnu.org/software/smalltalk/gst-manual/gst_168.html-SEC801

    Here is the intro to the section and the relevant subsection:

    BEGIN QUOTE

    [...]

    6.126 Set

       Defined in namespace Smalltalk

       Category: Collections-Unordered

              I am the typical set object; I also known how to do arithmetic

              on my instances.

         6.126.1 Set: arithmetic    (instance)

         6.126.2 Set: awful ST-80 compatibility hacks    (instance)

         6.126.3 Set: comparing    (instance)

    [...]

      6.126.3 Set: comparing

       < aSet

              Answer whether the receiver is a strict subset of aSet

       <= aSet

              Answer whether the receiver is a subset of aSet

        > aSet

               Answer whether the receiver is a strict superset of aSet

        >= aSet

               Answer whether the receiver is a superset of aSet

    [...]

    END QUOTE

     - Manfred

## Eberhard Lutz — 2002-12-16 11:35:37

    I posted a short monography of the Joy language on my web page:

    http://www.E7L3.com/joylang.html

## Manfred von Thun — 2003-01-17 03:46:10

    I have no teaching duties for the first half of the year, so I'll

    have some time to spend on possibly several Joy projects. One of

    these will be updating documentation. John Cowan's extensions

    are only visible from the inbuilt online manual, and some of Nick

    Forde's extensions do not fit into the online manual. The most

    urgent need seems to be for these papers/docs:

       1. The "Tutorial"

       2. The "Manual"

       3. The "Further Frequently Asked Questions"

    If you know any details that need changing or should be added,

    please let me know - this does not need to be restricted to the

    three items above.

    Also, Billy: As far as I can tell the term "concatenative notation"

    or "concatenative language" first occurred in the name of this

    mailing group. So am I right in believing that I should attribute

    it to you ?

    Best of 2003 to all.

      - Manfred

## Manfred von Thun — 2003-01-17 05:40:12

    Another of my projects for this first half year will be to give

    some thought to a module system for Joy. Currently Joy has next

    to nothing for structuring larger programs, with the HIDE facility

    being a very weak exception. There is really only one large name

    space, so everything (every function) is global. With all libraries

    loaded the danger of name conflict is getting closer. For the time

    being I am not thinking of anything sophisticated, nothing like

    the various object systems in the more developed languages. But

    a good starting point would be a module system which allows defi-

    nitions of modules in addition to the definition of functions.

    Inside a module would be definitions of functions (and possibly

    further modules). An example might look like this:

    DEFINE

      foo  ==  ... ;                      # ordinary function defined

      MODULE  m  IS                       # module m defined

        a  ==  ... ;                      # function a inside m defined

        b  ==  ...                        # function b inside m defined

        END;

      bar  ==  .. m.a .. m.b ..           # using a and b from m

      baz  ==  .. a .. b ..               # when  baz is called,

                                          # run time error for a and b,

                                          # unless they are defined

                                          # in the enclosing name space

      .                                   # END of these definitions

    ..  foo ..  m.a  .. m.b .. bar ..     # all OK

    ..  baz ..                            # the above run time error

    So that is the general idea. Modules are pretty much the same as

    records in Pascal or structures in C - except of course that

    what they contain are (normally) functions (but perhaps also

    other modules or whatever else nameable might be added to Joy).

    To access such a function the full qualified name can be used:

    "m.a" or "m.b"  -  and for the time being that will be the only

    way to access these functions. Too simple? Well, it is only

    meant as a start.

    Most developed languages have something far more sophisticated,

    partly because they are strongly typed, partly because they

    are procedural, partly because they are compiled. So, trying

    to find something comparable, I have looked at the various

    module systems for Scheme, and even these are much more

    elaborate than I have in mind for Joy right now.

    Just about all module systems allow some functions to be hidden

    and some to be exported. There are two ways of doing that:

      1. Explicitly list or annotate the funtions that are exported

         (made available outside the module). Any functions that

         are not so listed or annotated are hidden by default.

      2. Explicitly list or annotate the functions that are hidden,

         any others are exported. (This is the opposite of 1.)

    Apparently there are arguments for or against both views.

    Which one should Joy use ?

    I favour the second solution, because that would make MODULE

    and HIDE two orthogonal concepts. Either on its own is useful,

    and together they are "more useful than the sum of their parts",

    as they say. Here is what a combination would look like;

    the module exports e1 e2 e3 e4 and hides h1 h2 :

    DEFINE

        ...                               # some definitions here

        MODULE m IS

            e1  ==   ... ;

            HIDE

               h1  ==  ... ;

               h2  ==  ...

            IN

               e2  ==  .. h1 .. h2 .. ;

               e3  ==  .. h1 .. h2 ..

            END                           # end of HIDE

            e4  ==  .. e1 ..

        END                               # end of MODULE

        ...                               # more definitions here

    .                                     # end of DEFINE

    The four exported functions e1 .. e4 can now be used with

    ordinary qualified names:

           m.e1   m.e2   m.e3   m.e4

    Each of the following produce a compile time error:

           m.h1   m.h2

    Each of the following produce a run time error:

           h1  h2  e1  e2  e3  e4

    unless they have been defined globally.

    So, for the time being I favour the notation  m.e1  m.e2 ..

    unless anybody can think of a notation which in some way

    or another is more suited for a concatenative language.

    If the module name and its function names are very long

    (for whatever reason), one can introduce abbreviations outside

    the module:

    DEFINE

       a  ==  module-with-long-name.function-alpha-with-long-name

    This does not require a new mechanism at all.

    Only much later would I want to consider modules which

    selectively import functions from other modules, or something

    resembling the WITH statement of PASCAL to "open up" a module:

        WITH m DO  .. e1  .. e2  ..  e3  .. e4  END

    I have some thoughts on how the module system should be implemented.

    It will not affect the long file interp.c, but small changes will

    be needed in each of the other four files. I'll discuss my ideas

    for this implementation after we have had some discussion about

    the user's perspective of the module system. If anybody is itching

    to write the C-code, you will be most welcome. But please wait till

    after the discussion.

    Any thoughts?

      - Manfred

## Nick Forde — 2003-01-17 17:40:08

    Hi Manfred,

    Happy New Year to you too.

    Manfred von Thun writes:

      > I have no teaching duties for the first half of the year, so I'll

     > have some time to spend on possibly several Joy projects. One of

     > these will be updating documentation. John Cowan's extensions

     > are only visible from the inbuilt online manual, and some of Nick

     > Forde's extensions do not fit into the online manual. The most

     > urgent need seems to be for these papers/docs:

     >

     >    1. The "Tutorial"

     >    2. The "Manual"

     >    3. The "Further Frequently Asked Questions"

     >

     > If you know any details that need changing or should be added,

     > please let me know - this does not need to be restricted to the

     > three items above.

     Fantastic news! I believe a good Language Reference Manual would be

    most valuable. As Joy is a relatively easy language to write an

    interpreter for I wouldn't be surprised if this helped spawn some new

    "miniature" implementations and updates to the existing interpreter.

    I can supply a lex definition of the Joy tokens if it would help. Your

    description at

    http://www.latrobe.edu.au/philosophy/phimvt/joy/j09imp.html is now

    incomplete as it doesn't contain floating point constants, etc.  For

    example, the current interpreter treats '2.' as a float. I'd prefer if

    it was treated as integer-constant followed by dot.

    Another thing you might consider is making it easier for others to

    contribute to the interpreter implementation by having released

    versions and possibly also putting the code into a revision control

    system such as CVS (http://www.cvshome.org/).

    I only wish I too had the luxury of a few months to work on a project

    such as this :-)

    Regards,

    Nick.

## Martin Young — 2003-01-17 17:42:43

    This is something I posted some time ago - it's a brief overview of how

    I did modules in my Joy variant...

    For example:

    ---- snip -- snip ----

    << The interpreter always runs word "main" from module "main", with an

       empty stack, on startup. >>

    module main provides main .

                uses io control .

    main ==

    {{

    	[ 45 emit ] 10 repeat nl

    	10 factorial . nl

    	[ 45 emit ] 10 repeat nl

    }}

    ---- snip -- snip ----

    The module definition denotes which word are exported from the current

    module, and which modules' exported words are imported.  Unlike C,

    modules are not "chained" - in the above, module "io" might include a

    module "foo" but the words exported by "foo" would not be visible in

    "main".

    In the spirit of Joy a word can be lexically replaced by its

    definition.  A word actually consists of the ascii string the programmer

    sees and modules where that instance of the word originated.  When a

    user defined word is encountered, the definition is taken from the

    originating module *not* the module of the word currently being

    executed.

    For example:

    module main provides main foo .

    uses an_external_module

    foo ==

    {{

    	42

    }}

    main ==

    {{

    	[ foo ] run

    }}

    module an_external_module provides run .

    foo ==

    {{

    	84

    }}

    run ==

    {{

    	i . nl

    }}

    Here, the output will be 42 because the instance of "foo" executed in

    "run" originates in module "main" so that definition is used.  As the

    user is not allowed to construct new word names at runtime, every word

    is forced to have an originating module.

    The upshot of this is that lexical substitution is retained and so ease

    reasoning about a program is also retained.  Most importantly, the

    meaning of a particular instance of a word is constant throughout the

    execution of a program.

    Once the main program has been loaded, the interpreted tries to import

    modules which are used by looking in the current directory then in a

    directory specified by an environment variable.  The ordering of modules

    is not significant which implies that there is no problem with forward

    references.

    --

    Martin Young, at home.

## Nick Forde — 2003-01-22 10:41:33

    I've included some comments below and added a few more suggestions

    of my own.

    Manfred von Thun writes:

      > Just about all module systems allow some functions to be hidden

     > and some to be exported. There are two ways of doing that:

     >   1. Explicitly list or annotate the funtions that are exported

     >      (made available outside the module). Any functions that

     >      are not so listed or annotated are hidden by default.

     >   2. Explicitly list or annotate the functions that are hidden,

     >      any others are exported. (This is the opposite of 1.)

     > Apparently there are arguments for or against both views.

     > Which one should Joy use ?

     >

     > I favour the second solution, because that would make MODULE

     > and HIDE two orthogonal concepts. Either on its own is useful,

     > and together they are "more useful than the sum of their parts",

     > as they say. Here is what a combination would look like;

     > the module exports e1 e2 e3 e4 and hides h1 h2 :

     With the existing definition of HIDE I agree that the second solution

    is more consistent. However, like Billy my preference would be to make

    module definitions private by default, possibly removing HIDE, and

    adding some syntax to declare definitions public.

    I say this because I find when developing modular code generally the

    publically exported functions are in the minority and less likely to

    change over time. Explicit declaration of the public functions make me

    less likely to accidentally expose something I didn't intend to be

    public. As a module user I also find it easier to quickly determine

    what functions are available to me (i.e. those that are public).

      > DEFINE

     >     ...                               # some definitions here

     >     MODULE m IS

     >         e1  ==   ... ;

     >         HIDE

     >            h1  ==  ... ;

     >            h2  ==  ...

     >         IN

     >            e2  ==  .. h1 .. h2 .. ;

     >            e3  ==  .. h1 .. h2 ..

     >         END                           # end of HIDE

     >         e4  ==  .. e1 ..

     >     END                               # end of MODULE

     >     ...                               # more definitions here

     > .                                     # end of DEFINE

     To me although the above is consistent I don't find it easy enough to

    determine the public functions of the module. I think I'd prefer

    something like:

    DEFINE

      ...

      MODULE m IS

        EXPORT

          e1 == ... ;

          e2 == ... ;

          e3 == ... ;

          e4 == ...

        END

        h1 == ... ;

        h2 == ...

      END

      ...

    .

    In this example as a module user I can ignore everything below the

    first END. As a module developer I might start with something like:

    DEFINE

      ...

      MODULE m IS

        h1 == ... ;

        h2 == ... ;

        e1 == ... ;

        e2 == ... ;

        e3 == ... ;

        e4 == ...

      END

    .

    i.e. everything private, and then gradually start making the public

    functions available once I'm happy they do as I intend.

    Another option would be to use an explicit list of public function

    names in the module definition:

    MODULE m [e1 e2 e3 e4] WHERE

      e1 == ... ;

      e2 == ... ;

      h1 == ... ;

      h2 == ... ;

      e3 == ... ;

      e4 == ...

    END

    .

    I like this alternative most because it allows you to intermix public

    and private function definitions to make the code easier to read. It

    also makes it easy to publically expose new functions without the need

    to re-order code. Note that I've also removed the enclosing DEFINE.

      > ...

     > Only much later would I want to consider modules which

     > selectively import functions from other modules, or something

     > resembling the WITH statement of PASCAL to "open up" a module:

     >     WITH m DO  .. e1  .. e2  ..  e3  .. e4  END

     Even if you don't intent to implement this just now I think it is

    worth defining the syntax for doing this to make sure it is

    consistent. For example, if you decide to use an EXPORT statement it

    would seem appropriate to use an associated IMPORT statement for the

    above.

     >So, for the time being I favour the notation  m.e1  m.e2 ..

    >unless anybody can think of a notation which in some way

    >or another is more suited for a concatenative language.

     I think '.' is a good suggestion.

    Regards,

    Nick.

## phimvt@lurac.latrobe.edu.au — 2003-02-03 04:28:14

    On Wed, 22 Jan 2003, Nick Forde wrote:

     > I've included some comments below and added a few more suggestions

    > of my own.

    >

    > Manfred von Thun writes:

    >

    >  > Just about all module systems allow some functions to be hidden

    >  > and some to be exported. There are two ways of doing that:

    >

    >  > 1. Explicitly list or annotate the funtions that are exported

    >  > (made available outside the module). Any functions that

    >  > are not so listed or annotated are hidden by default.

    >  > 2. Explicitly list or annotate the functions that are hidden,

    >  > any others are exported. (This is the opposite of 1.)

    >  > Apparently there are arguments for or against both views.

    >  > Which one should Joy use ?

    >

    >  > I favour the second solution, because that would make MODULE

    >  > and HIDE two orthogonal concepts. Either on its own is useful,

    >  > and together they are "more useful than the sum of their parts",

    >  > as they say. Here is what a combination would look like;

    >  > the module exports e1 e2 e3 e4 and hides h1 h2 :

    >

    > With the existing definition of HIDE I agree that the second solution

    > is more consistent. However, like Billy my preference would be to make

    > module definitions private by default, possibly removing HIDE, and

    > adding some syntax to declare definitions public.

    >

    > I say this because I find when developing modular code generally the

    > publically exported functions are in the minority and less likely to

    > change over time. Explicit declaration of the public functions make me

    > less likely to accidentally expose something I didn't intend to be

    > public. As a module user I also find it easier to quickly determine

    > what functions are available to me (i.e. those that are public).

    >

    >  > DEFINE

    >  >     ...                               # some definitions here

    >  >     MODULE m IS

    >  >         e1  ==   ... ;

    >  >         HIDE

    >  >            h1  ==  ... ;

    >  >            h2  ==  ...

    >  >         IN

    >  >            e2  ==  .. h1 .. h2 .. ;

    >  >            e3  ==  .. h1 .. h2 ..

    >  >         END                           # end of HIDE

    >  >         e4  ==  .. e1 ..

    >  >     END                               # end of MODULE

    >  >     ...                               # more definitions here

    >  > .                                     # end of DEFINE

    >

    > To me although the above is consistent I don't find it easy enough to

    > determine the public functions of the module.

     I chose the above example to illustrate how MODULE and HIDE can

    work together: the above notation makes it clear that e2 and e3

    use h1 and h2 and that e1 and e4 do not. You rewrite below

    does not preserve this.

     >                                                I think I'd prefer

    > something like:

    >

    > DEFINE

    >   ...

    >   MODULE m IS

    >     EXPORT

    >       e1 == ... ;

    >       e2 == ... ;

    >       e3 == ... ;

    >       e4 == ...

    >     END

    >     h1 == ... ;

    >     h2 == ...

    >   END

    >   ...

    > .

     If one wanted to have all the four e's together, ignoring that

    only some use the h's, then one might write your version as

      MODULE m IS

             HIDE

               h1 == ... ;

               h2 == ...

             IN

          e1 == ... ;

          e2 == ... ;

          e3 == ... ;

          e4 == ...

      END

     > In this example as a module user I can ignore everything below the

    > first END.

     In my variant you ignore everything above the IN, or everything

    that has a double indent.

     >            As a module developer I might start with something like:

     [... good advice about developing]

    Yes, as you might have guessed, I have no experience in actually using

    modules, even less in writing modules.

     > Another option would be to use an explicit list of public function

    > names in the module definition:

    >

    > MODULE m [e1 e2 e3 e4] WHERE

     [...]

     > I like this alternative most because it allows you to intermix public

    > and private function definitions to make the code easier to read. It

    > also makes it easy to publically expose new functions without the need

    > to re-order code.

     Umm. Yes, that is a major consideration about program development.

    I'll have to mull over that.

     >                     Note that I've also removed the enclosing DEFINE.

     I almost missed that. My reason for this was that I wanted to have

    the possibility of full block structure for modules, i.e. modules

    within modules .. within modules. Pascal allows that with procedures,

    and I have always followed the techniques used by the Pascal compilers

    (written in Pascal) when writing Pascal. But again, I do not know

    whether any module system has any use for nested modules.

     > >So, for the time being I favour the notation  m.e1  m.e2 ..

    > >unless anybody can think of a notation which in some way

    > >or another is more suited for a concatenative language.

    >

    > I think '.' is a good suggestion.

     I do take John Cowan's suggestion very seriously - to use ':'

    instead of '.' . The period is already somewhat overused, as

    indeed it is in most languages. But as far as I can see,

    at least for recursive descent parsing as currently used by

    Joy, '.' does not create a problem. I do not know whether

    there would be problems for a bottom up parser (e.g. yacc).

    Notations that I have considered were:

       m.e

       m:e                # Common Lisp

       e@m                # reverse the order

       m e                # no explicit symbol at all

    The last would even allow things like

      MODULE module-with-long-name IS

      ...

      END

    and then

       DEFINE

         m == module-with-long-name ;

         foo == m e1 ;

         bar == m e2 ;

    It would also allow various other tricks. But I don't know

    whether they have any use.

    For now, as always, thanks for the careful discussion.

      - Manfred

## Nick Forde — 2003-02-03 10:41:49

    phimvt@... writes:

      > On Wed, 22 Jan 2003, Nick Forde wrote:

     > >  > DEFINE

     > >  >     ...                               # some definitions here

     > >  >     MODULE m IS

     > >  >         e1  ==   ... ;

     > >  >         HIDE

     > >  >            h1  ==  ... ;

     > >  >            h2  ==  ...

     > >  >         IN

     > >  >            e2  ==  .. h1 .. h2 .. ;

     > >  >            e3  ==  .. h1 .. h2 ..

     > >  >         END                           # end of HIDE

     > >  >         e4  ==  .. e1 ..

     > >  >     END                               # end of MODULE

     > >  >     ...                               # more definitions here

     > >  > .                                     # end of DEFINE

     > >

     > > To me although the above is consistent I don't find it easy enough to

     > > determine the public functions of the module.

     >

     > I chose the above example to illustrate how MODULE and HIDE can

     > work together: the above notation makes it clear that e2 and e3

     > use h1 and h2 and that e1 and e4 do not. You rewrite below

     > does not preserve this.

     You are correct. However, such "HIDE" granularity is not something I

    think I'd ever use in practice given a module system with private by

    default definitions, particularly if modules may be nested.

      > [...]

     > > Another option would be to use an explicit list of public function

     > > names in the module definition:

     > >

     > > MODULE m [e1 e2 e3 e4] WHERE

     > [...]

     > > I like this alternative most because it allows you to intermix public

     > > and private function definitions to make the code easier to read. It

     > > also makes it easy to publically expose new functions without the need

     > > to re-order code.

     >

     > Umm. Yes, that is a major consideration about program development.

     > I'll have to mull over that.

     >

     > >                     Note that I've also removed the enclosing DEFINE.

     >

     > I almost missed that. My reason for this was that I wanted to have

     > the possibility of full block structure for modules, i.e. modules

     > within modules .. within modules. Pascal allows that with procedures,

     > and I have always followed the techniques used by the Pascal compilers

     > (written in Pascal) when writing Pascal. But again, I do not know

     > whether any module system has any use for nested modules.

     I think it is worth making nested modules possible.

    MODULE m [e1 e2] WHERE

      e1 == ... ;

      e2 == ... ;

      h1 == ... ;

      h2 == n.e3 ;

      MODULE n [ e3 e4 ] WHERE

        e3 == ... ;

        e4 == ... ;

        h3 == ...

      END

    END

    From the global scope I could then call m.n.e3. One advantage of the

    above syntax is that if at a later date I find module n to be

    generally useful I can easily cut and paste it into a global scope

    without the need to change module m.

      > > >So, for the time being I favour the notation  m.e1  m.e2 ..

     > > >unless anybody can think of a notation which in some way

     > > >or another is more suited for a concatenative language.

     > >

     > > I think '.' is a good suggestion.

     >

     > I do take John Cowan's suggestion very seriously - to use ':'

     > instead of '.' . The period is already somewhat overused, as

     > indeed it is in most languages. But as far as I can see,

     > at least for recursive descent parsing as currently used by

     > Joy, '.' does not create a problem. I do not know whether

     > there would be problems for a bottom up parser (e.g. yacc).

     No problem.

      > Notations that I have considered were:

     >

     >    m.e

     >    m:e                # Common Lisp

     >    e@m                # reverse the order

     >    m e                # no explicit symbol at all

     >

     > The last would even allow things like

     >

     >   MODULE module-with-long-name IS

     >   ...

     >   END

     >

     > and then

     >

     >    DEFINE

     >      m == module-with-long-name ;

     >      foo == m e1 ;

     >      bar == m e2 ;

     >

     > It would also allow various other tricks. But I don't know

     > whether they have any use.

     I think it would be better to allow the above through a "USING" type

    statement. e.g.

    MODULE m [e1 e2] WHERE

      USING [module-with-long-name m2 m3]

      e1 == e5 e6 ;

      e2 == ... ;

      h1 == ... ;

      h2 == ... ;

    END

    MODULE module-with-long-name [e5 e6] WHERE

      e5 == ... ;

      e6 == ... ;

    END

    Obviously in the above having USING in the MODULE statement is

    optional allowing it to be implemented at a later date if desirable

    without breaking the existing syntax.

    Regards,

    Nick.

## phimvt@lurac.latrobe.edu.au — 2003-02-28 03:52:32

    Thank you all for the discussion on modules so far. I am now persuaded

    that hiding the private part of a module should be not rely on the

    HIDE mechanism, but have a symtax in its own right. But recently

    a possibility occurred to me which is at least worth discussing a bit:

    combine 1) the LIBRA/DEFINE, and 2) the HIDE, and 3) the MODULE

    (with or without private parts) all into one. Yes.

    Currently the syntax for definitions looks like this, with at least

    a skeleton for modules:

    definition ::=

            identifier "==" term

         |  "HIDE" definition-sequence "IN" definition-sequence ("END" |".")

         |  "MODULE" identifier ...more-here... ("END" | ".")

         |  null

    We also have:

    definition-sequence ::=

            definition  { ";" definition }

    cycle ::=

    	{     ("LIBRA" | "DEFINE" ) definition-sequence ("END" | ".")

               |  term "." }

    On the proposal we would have definition-sequence as before, but

    definition and cycle like this:

    definition ::=

            identifier "==" term

          | compound-definition

    cycle ::=

    	{  compound-definition

               |  term "." }

    And this is new:

    compound-definition ::=

          [ "MODULE" identifier ]                    # this is optional

          [ "PRIVATE" definition-sequence]           # this is optional

          "PUBLIC" defsequence " ("END" | ".")       # this is mandatory

    There are two optional parts, so there are four possibilities:

    1) neither optional part is present

    2) MODULE is not present, but PRIVATE is

    3) MODULE is present, PRIVATE is not

    4) both optional parts are present.

    Each of these has a clear meaning and use. I'll spell them

    out with examples:

    1)    PUBLIC

              p == ..

              q == ..

          END

    This does exacly what LIBRA/DEFINE does

    2)    PRIVATE

              h1 == ..

              h2 == ..

          PUBLIC

              p == .. h1 .. h2 ..

              q == .. h1 .. h2 ..

          END

    This is the same as a HIDE previously

    3)    MODULE m

          PUBLIC

              p == ..

              q == ..

          END

    A module with only public definitions. These must now

    be accessed by  m.p and so on (or by other mechanisms).

    4)    MODULE m

          PRIVATE

              h1 == ..

              h2 == ..

          PUBLIC

              p == .. h1 .. h2 ..

              q == .. h1 .. h2 ..

          END

    This is a module with a private part. Only p and q are exported.

    Remember that the BNF definitions are (mutually) recursive.

    That means, for example, that wherever there is a simple definition

    like

               p == ...

    there could instead be a compound definition of any of the four sorts.

    It also means that modules can be nested.

    To my ears only 1) grates a little. Also, in normal use 1) would

    actually only be called from within cycle - but there is no point

    in prohibiting it elsewhere any more than we prohibit  x := y + 0

    as an statement.

    assignment.

    Disadvantage: the current LIBRA/DEFINE and also HIDE..IN become

    obsolescent.

    Advantage: economy of concepts, orthogonality.

    Any views, my friends ?

      - Manfred

## Nick Forde — 2003-03-11 13:21:56

    phimvt@... writes:

      > On the proposal we would have definition-sequence as before, but

     > definition and cycle like this:

     >

     > definition ::=

     >         identifier "==" term

     >       | compound-definition

     >

     > cycle ::=

     > 	{  compound-definition

     >            |  term "." }

     >

     > And this is new:

     >

     > compound-definition ::=

     >       [ "MODULE" identifier ]                    # this is optional

     >       [ "PRIVATE" definition-sequence]           # this is optional

     >       "PUBLIC" defsequence " ("END" | ".")       # this is mandatory

     >

     > There are two optional parts, so there are four possibilities:

     > 1) neither optional part is present

     > 2) MODULE is not present, but PRIVATE is

     > 3) MODULE is present, PRIVATE is not

     > 4) both optional parts are present.

     As I mentioned in one of my previous posts I'd prefer to have the

    PUBLIC definitions at the top of a file. Could you perhaps make the

    order of 'PRIVATE defsequence PUBLIC defsequence END' interchangeable

    with 'PUBLIC defsequence PRIVATE defsequence END'?

      > Each of these has a clear meaning and use. I'll spell them

     > out with examples:

     >

     > 1)    PUBLIC

     >           p == ..

     >           q == ..

     >       END

     > This does exacly what LIBRA/DEFINE does

     The only problem I have with this is in interactive use where

    'DEFINE p == mydef.' is more convenient than "PUBLIC p == mydef END".

    I also find the DEFINE alternative a little more intuitive for short

    definitions which don't really need a module or PUBLIC/PRIVATE

    distinction.

    How about making DEFINE synonymous with PUBLIC in the same way

    as END and "."?

      > [...]

     > To my ears only 1) grates a little. Also, in normal use 1) would

     > actually only be called from within cycle - but there is no point

     > in prohibiting it elsewhere any more than we prohibit  x := y + 0

     > as an statement.

     > assignment.

     >

     > Disadvantage: the current LIBRA/DEFINE and also HIDE..IN become

     > obsolescent.

     Other than my comment above it sounds like a good idea to me.

    Regards,

    Nick.

## phimvt@lurac.latrobe.edu.au — 2003-03-12 06:00:39

    On Tue, 11 Mar 2003, Nick Forde wrote:

     > phimvt@... writes:

     [..stuff about syntax of MODULEs with PRIVATE and PUBLIC parts..]

     > As I mentioned in one of my previous posts I'd prefer to have the

    > PUBLIC definitions at the top of a file.

                                          ^^^^

    I had not planned to make a strong connection between modules

    and files (in the way C does). Of course, even if Joy does

    not enforce it, one could adopt such a connection as a matter

    of style. But if one wants to, one can have several modules

    in one file, or a module split over several files (see below).

     >                                         Could you perhaps make the

    > order of 'PRIVATE defsequence PUBLIC defsequence END' interchangeable

    > with 'PUBLIC defsequence PRIVATE defsequence END'?

     That is actually much harder than it would seem. The reason has

    to do with "declaration before use", which is familiar in C

    and even more so in Pascal which allows definitions within

    definitions. Let's distinguish between declarations as in

    	void foo(int x; char y);	/* no body yet

    and definitions

            void foo(int x; char y)

    	{

    	 ...

    	}

    Here foo can be used any time after its definition, but still before

    its declaration. Or the declaration can be omitted and  then the

    definition also serves as the declaration.

    Now, for modules in Joy the PRIVATE parts need to be declared

    before the PUBLIC parts can use them - otherwise the PUBLIC parts

    would have to assume that any functions they use will at least

    be defined by the time they are run. The only other possibility

    would be a hideous backpatching of the PUBLIC parts as soon as the

    PRIVATE parts have been defined. No,no.

    However, here is a possible style which might satisfy your

    preference:

    In file  foo-pub.joy we have

    	MODULE foo

    	PRIVATE "foo-priv.joy" include  # this file might be long

    	PUBLIC

    	    p1 == ..   # all the public definitions clearly seen

    	    p2 == ..

    	    ...

    	    pn == ..

    	END

    And in file  foo-priv.joy we have many private hidden definitions:

                 h1 == ..

    	     h2 == ..

    	     ...

    	     hn

    And incidentally this is an argument for decoupling files and modules.

     > The only problem I have with this is in interactive use where

    > 'DEFINE p == mydef.' is more convenient than "PUBLIC p == mydef END".

     Quite agree. "END" and "." will be interchangeable in this context.

     > I also find the DEFINE alternative a little more intuitive for short

    > definitions which don't really need a module or PUBLIC/PRIVATE

    > distinction.

     I came to the same conclusion, too.

     > How about making DEFINE synonymous with PUBLIC in the same way

    > as END and "."?

     Exactly. The only odd man out is "HIDE .. IN .. END", which

    needs to be retained for a while at least.

    I have now implemented a trial version of all this, but

    because I wanted arbitrary nesting to be possible it all

    took longer than I had hoped. I need to test a bit more

    before I put the new version on the web page. In the meantime

    we can think about a useful module to have as a new library.

      - Manfred

## phimvt@lurac.latrobe.edu.au — 2003-03-17 06:16:52

    On Wed, 12 Mar 2003 phimvt@... wrote:

     > [..stuff about syntax of MODULEs with PRIVATE and PUBLIC parts..]

    > I have now implemented a trial version of all this, but

    > because I wanted arbitrary nesting to be possible it all

    > took longer than I had hoped. I need to test a bit more

    > before I put the new version on the web page. In the meantime

    > we can think about a useful module to have as a new library.

     I have put the new version of the sources and a demo/sample

    with output on the Joy page. See the NEW section right at the start.

      - Manfred

## Nick Forde — 2003-03-29 19:33:06

    Manfred,

    Sorry for taking so long to respond to this, I've been very busy

    lately but have just grabbed a little time to play with your updates

    to the interpreter.

    phimvt@... writes:

      >

     > On Tue, 11 Mar 2003, Nick Forde wrote:

     >

     > > phimvt@... writes:

     >

     > [..stuff about syntax of MODULEs with PRIVATE and PUBLIC parts..]

     >

     > > As I mentioned in one of my previous posts I'd prefer to have the

     > > PUBLIC definitions at the top of a file.

     >                                      ^^^^

     > I had not planned to make a strong connection between modules

     > and files (in the way C does). Of course, even if Joy does

     > not enforce it, one could adopt such a connection as a matter

     > of style. But if one wants to, one can have several modules

     > in one file, or a module split over several files (see below).

     I would also prefer that a single module per file wasn't enforced but

    I suspect many modules will end up being like this, e.g. as with the

    libraries in the Joy distribution.

      > >                                         Could you perhaps make the

     > > order of 'PRIVATE defsequence PUBLIC defsequence END' interchangeable

     > > with 'PUBLIC defsequence PRIVATE defsequence END'?

     >

     > That is actually much harder than it would seem. The reason has

     > to do with "declaration before use", which is familiar in C

     > and even more so in Pascal which allows definitions within

     > definitions. Let's distinguish between declarations as in

     > 	void foo(int x; char y);	/* no body yet

     > and definitions

     >         void foo(int x; char y)

     > 	{

     > 	 ...

     > 	}

     > Here foo can be used any time after its definition, but still before

     > its declaration. Or the declaration can be omitted and  then the

     > definition also serves as the declaration.

     This is one reason I proposed something like:

    MODULE mymod [ func1 func2 func3 ]

      pfunc1 == ...

      func1  == ...

      func2  == ...

      pfunc2 == ...

    END

    The list in the module definition was intended to declare the public

    functions. All others would be private by default. As I mentioned

    before I think the main advantage here is that private and public

    functions can be intermixed and the module user can immediately see

    the module's working interface.

    Without looking at the existing interpreter implementation in detail I

    imagine this must be fairly easy to implement since func1, func2 &

    func3 need only be initially created as empty function

    definitions with a public flag.

      > Now, for modules in Joy the PRIVATE parts need to be declared

     > before the PUBLIC parts can use them - otherwise the PUBLIC parts

     > would have to assume that any functions they use will at least

     > be defined by the time they are run. The only other possibility

     > would be a hideous backpatching of the PUBLIC parts as soon as the

     > PRIVATE parts have been defined. No,no.

     Is this restriction of define before use not inconsistent with the

    current implementation of public function definitions?

      DEFINE c == a b + .

      DEFINE b == 2 .

      DEFINE a == 4 .

      c.

      6

      > However, here is a possible style which might satisfy your

     > preference:

     > In file  foo-pub.joy we have

     > 	MODULE foo

     > 	PRIVATE "foo-priv.joy" include  # this file might be long

     > 	PUBLIC

     > 	    p1 == ..   # all the public definitions clearly seen

     > 	    p2 == ..

     > 	    ...

     > 	    pn == ..

     > 	END

     > And in file  foo-priv.joy we have many private hidden definitions:

     >              h1 == ..

     > 	     h2 == ..

     > 	     ...

     > 	     hn

     > And incidentally this is an argument for decoupling files and modules.

     This approach does make it easier to publish the public interface of a

    module but personally I would find the inconvenience of having to use

    multiple files annoying.

      > > The only problem I have with this is in interactive use where

     > > 'DEFINE p == mydef.' is more convenient than "PUBLIC p == mydef END".

     >

     > Quite agree. "END" and "." will be interchangeable in this context.

     Great.

      > > I also find the DEFINE alternative a little more intuitive for short

     > > definitions which don't really need a module or PUBLIC/PRIVATE

     > > distinction.

     >

     > I came to the same conclusion, too.

     >

     > > How about making DEFINE synonymous with PUBLIC in the same way

     > > as END and "."?

     >

     > Exactly. The only odd man out is "HIDE .. IN .. END", which

     > needs to be retained for a while at least.

     Why would you like to keep this? Is it simply to allow the libraries

    in the interpreter distribution to be updated over time? Or are there

    other libraries you'd rather not break?

    I can help with the migration if you'd like.

      > I have now implemented a trial version of all this, but

     > because I wanted arbitrary nesting to be possible it all

     > took longer than I had hoped. I need to test a bit more

     > before I put the new version on the web page. In the meantime

     > we can think about a useful module to have as a new library.

     I've experimented a bit with your latest updates and they seem to work

    OK. I think this new functionality is a big step towards making Joy

    more "industrial strength".

    Have you thought about creating a "std" module for the core library

    functions distributed with the interpreter?

    Regards,

    Nick.

## Heiko.Kuhrt@t-online.de — 2003-03-30 04:38:36

    On Sat, Mar 29, 2003 at 07:33:06PM +0000, Nick Forde wrote:

     > phimvt@... writes:

    >  >

    >  > On Tue, 11 Mar 2003, Nick Forde wrote:

    >  >

    >  > > phimvt@... writes:

      ... a lot deleted ....

     >  > Exactly. The only odd man out is "HIDE .. IN .. END", which

    >  > needs to be retained for a while at least.

    >

    > Why would you like to keep this? Is it simply to allow the libraries

    > in the interpreter distribution to be updated over time? Or are there

    > other libraries you'd rather not break?

    >

    > I can help with the migration if you'd like.

    >

     Do I get that right? You are going to replace HIDE IN END by the new

    module system, so HIDE IN END could be removed from Joy?

    I just got warm with HIDE. I'm using it like "where" in some other

    functional languages, that is in order to factorize complicated

    definitions into a set of smaller, better readable ones.

    HIDE makes the relations pretty clear and gives any freedom in choosing

    definition names.

    Using modules for a purpose like that is not appropriate.

    (Of course I don't insist on HIDE IN END but on something like

    that: where end, let in, ... )

    Heiko

## Nick Forde — 2003-03-31 19:16:11

    Hi Heiko,

    Heiko.Kuhrt@... writes:

      > Do I get that right? You are going to replace HIDE IN END by the new

     > module system, so HIDE IN END could be removed from Joy?

     >

     > I just got warm with HIDE. I'm using it like "where" in some other

     > functional languages, that is in order to factorize complicated

     > definitions into a set of smaller, better readable ones.

     > HIDE makes the relations pretty clear and gives any freedom in choosing

     > definition names.

     What language's implementation of "where" do you mean?

      > Using modules for a purpose like that is not appropriate.

     I'm not sure I understand why. Is it because a module name must be

    supplied? Is HIDE IN END not just a top level (default) module

    definition?

    Regards,

    Nick.

## phimvt@lurac.latrobe.edu.au — 2003-04-01 05:59:34

    On Sat, 29 Mar 2003, Nick Forde wrote:

    [...]

    > This is one reason I proposed something like:

                           ^^^^^^^^

    Yes, not forgotten, just not done yet. The more basic stuff

    was harder than anticipated.

    > MODULE mymod [ func1 func2 func3 ]

    >   pfunc1 == ...

    >   func1  == ...

    >   func2  == ...

    >   pfunc2 == ...

    > END

    >

    > The list in the module definition was intended to declare the public

    > functions. All others would be private by default. As I mentioned

    > before I think the main advantage here is that private and public

    > functions can be intermixed and the module user can immediately see

    > the module's working interface.

    Yes. But to be consistent, the same should be allowed for libraries:

    > LIBRA [ func1 func2 func3 ]

    >   pfunc1 == ...

    >   func1  == ...

    >   func2  == ...

    >   pfunc2 == ...

    > END

    The point is this: a library is really just a special case -- it is

    an anonymous module whose public functions can be called from the

    outside even without the name of the module (and without the ".").

    > Without looking at the existing interpreter implementation in detail I

    > imagine this must be fairly easy to implement since func1, func2 &

    > func3 need only be initially created as empty function

    > definitions with a public flag.

    Yes, this looks about right.

    >  > Now, for modules in Joy the PRIVATE parts need to be declared

    >  > before the PUBLIC parts can use them - otherwise the PUBLIC parts

    >  > would have to assume that any functions they use will at least

    >  > be defined by the time they are run. The only other possibility

    >  > would be a hideous backpatching of the PUBLIC parts as soon as the

    >  > PRIVATE parts have been defined. No,no.

    >

    > Is this restriction of define before use not inconsistent with the

    > current implementation of public function definitions?

    I now think you are right here - I suppose I was unconsciously finding

    logical justifications for avoiding what I still think would be

    some quite difficult backpatching.

    [..a propos module spread over public and private files..]

    > This approach does make it easier to publish the public interface of a

    > module but personally I would find the inconvenience of having to use

    > multiple files annoying.

    Yes, that is a consideration.

    [...]

    >  > Exactly. The only odd man out is "HIDE .. IN .. END", which

    >  > needs to be retained for a while at least.

    >

    > Why would you like to keep this? Is it simply to allow the libraries

    > in the interpreter distribution to be updated over time? Or are there

    > other libraries you'd rather not break?

    See next posting - reply to your reply to Heiko's posting.

    [...]

    > Have you thought about creating a "std" module for the core library

    > functions distributed with the interpreter?

    Yes, that has been the prime motivation for modules in the first place.

    With many libraries loaded, the threat of an unnoticed name-clash

    is coming closer all the time.

    But it is not all that easy to decide which Joy functions should

    be made a member of some module and which should not. Three

    possibilities:

       1. Turn all current libraries into modules.

       2. Leave inilib.joy and the current three "standard libraries"

          agglib, seqlib and numlib as they are, but turn all other

          "special libraries" into modules.

       3. For the time being at least, leave all current libraries

          unchanged, but write some new module libraries -- for

          example the ancient but now defunct typlib.joy contained

          stuff for some good candidates:

             stacks  queues  trees  big-sets  dictionaries

       4. Don't put any modules into standard distribution - leave

          modules entirely to be written by users.

    My current thinking is for (3), and then after a while and after

    some experience, (2) - but in a way tht does not interfere with

    the original special libraries.

    In the meantime, I'll continue rewriting Chapter 9, j09imp.html,

    the principal documentation. So I won't touch the C-sources

    for a while. So ... if anybody wants to start on the implementation

    of the additions

        MODULE m [func1 func2 func3]

     or    LIBRA [func1 func2 func3]

    ourwork would not interfere.

      - Manfred

## phimvt@lurac.latrobe.edu.au — 2003-04-01 06:53:48

    On Mon, 31 Mar 2003, Nick Forde wrote:

     > Hi Heiko,

    >

    > Heiko.Kuhrt@... writes:

    >  > Do I get that right? You are going to replace HIDE IN END by the new

    >  > module system, so HIDE IN END could be removed from Joy?

     Only the words, and only in the distant future. The function will

    remain, but now using PRIVATE PUBLIC END as the preferred words.

     >  > I just got warm with HIDE. I'm using it like "where" in some other

    >  > functional languages, that is in order to factorize complicated

    >  > definitions into a set of smaller, better readable ones.

    >  > HIDE makes the relations pretty clear and gives any freedom in choosing

    >  > definition names.

    >

    > What language's implementation of "where" do you mean?

     I know "where" from Haskell and ML, in which an expression can be

    of the form   "E1 where w = E2", in other words a local definition

    of "w" that can be used inside E1. It is equivalent to "LET w = E2

    IN E1" which is equivalent to "(lambda w E1) @ E2" in the lambda

    calculus. Something similar in Joy might be

        DEFINE   foo   ==   ...w..w... where w = ...

    That would be a definition of w that is local to just the RHS

    of the definition of foo.                        ^^^^

    It must be distinguished from

        DEFINE

            foo == ..w..w..  (global, public, exported)

            bar == ..w..w..  (global, public, exported)

        where

            w == ...         (local, private, hidden)

        END

    Here the definition of w extends over the two definitions, of

    foo and bar.

    I never had the first form in mind, I think the second is more

    versatile because the scope of w is wider.

    So, note the difference:

        LET w1 = E1, w2 == E2 IN E3.          (E3 is one expression)

       HIDE w1 = E1, w2 == E3 IN D1 D2 D3.    (several definitions)

     >  > Using modules for a purpose like that is not appropriate.

    >

    > I'm not sure I understand why. Is it because a module name must be

    > supplied? Is HIDE IN END not just a top level (default) module

    > definition?

     I think we have run into a terminological problem: what exactly is

    a module? (1) Just a collection of definitions of new functions? or

    (2) a name (say, "m") together with a collection of definitions?

    I always had the second in mind, where the defined functions

    are then accessed from the ouside by "m.f" and so on.

    In the following, the second column might have been written

    as HIDE IN END and the third column as LIBRA END. Also, I am

    not at all adverse to replacing the PUBLIC by a list [publ1 publ2]

    of exported functions, provided the whole remains consistent.

    D E F I N I T I O N S

    MODULE m

      PRIVATE             PRIVATE

        priv1 == ..         priv1 == ..

        priv2 == ..         priv2 == ..

      PUBLIC              PUBLIC              PUBLIC

        publ1 == ..priv..   publ1 == ..priv..    publ1 == ..

        publ2 == ..priv..   publ1 == ..priv..    publ2 == ..

      END                 END                 END

    A C C E S S E D   A S

     m.publ1  m.publ2      publ1  publ2          publ1  publ2

      - Manfred

## Heiko.Kuhrt@t-online.de — 2003-04-01 07:54:57

    > What language's implementation of "where" do you mean?

     Clean, Haskell

     >

    >  > Using modules for a purpose like that is not appropriate.

    >

    > I'm not sure I understand why. Is it because a module name must be

    > supplied? Is HIDE IN END not just a top level (default) module

    > definition?

     The following is an example out of my lib:

    (*-------------------------------------------------------------------------*)

    intersect ==	## A I:Index -> A A

    	dup2   drop  [take]dip;

    HIDE

    	unconsX ==	## [hd:x:tl] I -> [hd] [tl] X

    		intersect unswons;

    	appP	==	## X [P] -> X

    		i;

    IN

    onitem	== ##  A [P] I:Index -> A

    	swap [unconsX]dip

    	appP

    	rollup enconcat;

    END;

    (*-------------------------------------------------------------------------*)

    intersect and onitem are intended to be used from inside or outside the lib.

    unconsX and appP are defined only in order to increase readability.

    (I really don't know if they do, but the intention is that way.)

    In this case HIDE IN END makes clear, the two definitions unconsX and appP

    belong to onitem and nowhere else.

    In addition I don't have to fear to override other names.

    Of course it is possible to add the extra information using comments, too:

    onitem	== ##  A [P] I:Index -> A

    	swap [intersect unswons]dip	#uncons X

    	i 				#apply P to X

    	rollup enconcat;

    In my experience this works pretty good as long as things are simple. But

    often I ended up with long definitions, where all these comments didn't

    help mutch.

    I don't know. I played a little with Clean, found the "where" implementation

    pretty intuitive and started to use HIDE IN END that way. I wouldn't mind if

    HIDE IN END would be changed in some way or replaced by something else.

    Adapting my code would be no problem to me. But I think by removeing it entirely

    we would give away functionality without need.

    Heiko

## phimvt@lurac.latrobe.edu.au — 2003-04-17 06:48:19

    On Tue, 1 Apr 2003 Heiko.Kuhrt@... wrote:

    [.. on the usefulness of HIDE..IN.. ]

    HIDE..IN.. won't go away. Promise.

    Sorry, must fly.

      - Manfred