Command line processing in Joy
Nick Forde — 2002-11-08 21:17:51
To make scripting in Joy more convenient I'd like to propose 2 new
primitives: argv and argc. If we define the interpreter usage as:
$ joy [JOYARGS] [INFILE [ARGS ...]]
Since there currently aren't any JOYARGS argv and argc may therefore
behave as follows:
$ joy
argv == ["joy"]; argc == 1
$ joy test.joy
argv == ["test.joy"]; argc == 1
$ joy test.joy -h
argv == ["test.joy" "-h"]; argc == 2
$ chmod +x test.joy
$ test.joy
argv == ["test.joy"]; argc == 1
$ test.joy -a -b
argv == ["test.joy" "-a" "-b"]; argc == 3
I have found the above to be very useful for writing parameterised
programs such as:
DEFINE seed == [argc 1 >] [argv second strtod] [time] ifte.
...
or
(*
* Attempt to open the file specified on the command line.
* If it doesn't exist then use stdin instead.
*)
DEFINE ifopen == (* -> F *)
[argc 1 >]
[argv second "r" fopen
[null] ["error: failed to open input file.\n" putchars quit] [] ifte]
[stdin]
ifte.
...
One additional benefit of my interpreter changes for this is that
stdin isn't hogged if joy is run as 'joy source.joy' and hence it is
much easier to use fgets, etc. from a file containing multiple joy
programs. This does, however, mean that if you really want to combine
source code and data in the same file you must pipe it into the
interpreter using 'joy < source.joy' or 'cat source.joy | joy'. On
Unix systems at least this is much more conventional behaviour.
What do you think?
Manfred & John,
This only requires a few updates to the existing interpreter. I'll
forward them on to you via email for your consideration.
Regards,
Nick.
Manfred von Thun — 2002-11-13 02:18:28
On Fri, 8 Nov 2002, Nick Forde wrote:
> To make scripting in Joy more convenient I'd like to propose 2 new
> primitives: argv and argc.
[...]
>
> One additional benefit of my interpreter changes for this is that
> stdin isn't hogged if joy is run as 'joy source.joy' and hence it is
> much easier to use fgets, etc. from a file containing multiple joy
> programs. This does, however, mean that if you really want to combine
> source code and data in the same file you must pipe it into the
^^^^^^^^^^^^^^^^^^^^
I take it this means data to be read by fgets etc., and not by
the ordinary get, which uses data on the original (stack of) input
files. (I noticed you had to change "stdin" to "srcfil" somewhere.)
> interpreter using 'joy < source.joy' or 'cat source.joy | joy'. On
> Unix systems at least this is much more conventional behaviour.
I am happy to accept your judgement here.
>
> What do you think?
So far no thoughts. I have never written a C program with argv and
argc, and I have no experience with them except with standard
unix utilities.
>
> Manfred & John,
>
> This only requires a few updates to the existing interpreter. I'll
> forward them on to you via email for your consideration.
Thanks for that, I have compiled them only on our VMS machine so far,
but I got the argv and argc to work. But so far that has been only
a very superficial test.
For the next two weeks I shall be very busy with exam marking and
administration, and I won't have any time to do justice to your
work. You might consider mailing your changes to the group for
earlier feedback.
Thank you very much for your contribution.
- Manfred
[...]
Nick Forde — 2002-11-13 15:52:51
Manfred von Thun writes:
> On Fri, 8 Nov 2002, Nick Forde wrote:
> > To make scripting in Joy more convenient I'd like to propose 2 new
> > primitives: argv and argc.
> [...]
> >
> > One additional benefit of my interpreter changes for this is that
> > stdin isn't hogged if joy is run as 'joy source.joy' and hence it is
> > much easier to use fgets, etc. from a file containing multiple joy
> > programs. This does, however, mean that if you really want to combine
> > source code and data in the same file you must pipe it into the
> ^^^^^^^^^^^^^^^^^^^^
> I take it this means data to be read by fgets etc., and not by
> the ordinary get, which uses data on the original (stack of) input
> files. (I noticed you had to change "stdin" to "srcfil" somewhere.)
Yes.
With the current implementation the following are equivalent:
$ joy < source.joy
$ joy source.joy
They both read the program text from stdin.
The srcfile change I made means that now only the first usage above
adopts stdin. The other reads from a newly created filehandle, leaving
stdin free, and making the following more practical:
$ joy source.joy < data.in
or if you make source.joy an executable script:
$ source.joy < data.in
When running source.joy as above 'get' will read from source.joy (or the
current file on the include stack) whilst 'stdin fgets' will read from
the file data.in.
> For the next two weeks I shall be very busy with exam marking and
> administration, and I won't have any time to do justice to your
> work. You might consider mailing your changes to the group for
> earlier feedback.
No problem, I'm in no rush - it works for me! If anyone on the list
would like these changes feel free to drop me an email.
Nick.
Manfred von Thun — 2002-11-29 09:31:07
On Wed, 13 Nov 2002, Manfred von Thun wrote:
>
> On Fri, 8 Nov 2002, Nick Forde wrote:
[... about his additions to several C files for Joy]
> For the next two weeks I shall be very busy with exam marking and
> administration, and I won't have any time to do justice to your
> work.
I am now ready to devote some time to this. Tonight I tested your
changes with the joytut.joy library, which probably needs the
messiest file handling of all libraries. Your modification works
fine. Next week I intend to do some more testing, and then I'll
transfer your new implementation to the Joy page.
Thanks again
- Manfred
Manfred von Thun — 2002-12-02 03:34:50
I have now done all my testing under VMS, and all old things
work fine. I was even able to test your argv and argc under VMS
even though I had never used that before. So I copied it all to
the Joy web page.
Trouble:
1. using just $ make
I get the usual long stuff from the BDW collector,
then a compiled executable joy
then $ joy
produces the two line header (with (BDW)) and then a core dump.
2. using $ make -f make.nogc
I get the compiled executable joy
then $ joy
produces the two line header (with (NOBDW)) and runs fine.
3. using $ make -f make.gc
I get a much shorter blurb from the BDW collector,
and then the same problem as in (1).
I am lost.
I did notice each of those compilations gave
main.c:141
warning: unused variable 'f' (* that is: FILE *f *)
But I cannot see how that could produce the crash.
There was no such warning under VMS.
For the moment I have left the *.c and *.h as you sent them
on the web page, for you or somebody else to look at. I did
not update the tars. But I should not leave it all inconsistent
for too long.
- Manfred
John Cowan — 2002-12-02 06:27:58
Manfred von Thun scripsit:
> 2. using $ make -f make.nogc
> I get the compiled executable joy
> then $ joy
> produces the two line header (with (NOBDW)) and runs fine.
On VMS you should always be using make.nogc. The automated logic
that decides in the main Makefile whether to use the BDW garbage collector
or not will not work on VMS.
--
A poetical purist named Cowan [that's me:
jcowan@...]
Once put the rest of us dowan. [on xml-dev]
"Your verse would be sweeter
http://www.ccil.org/~cowan
If it only had metre
http://www.reutershealth.com
And rhymes that didn't force me to frowan." [overpacked line!] --Michael Kay
Manfred von Thun — 2002-12-02 06:44:12
On Mon, 2 Dec 2002, John Cowan wrote:
> Manfred von Thun scripsit:
>
> > 2. using $ make -f make.nogc
> > I get the compiled executable joy
> > then $ joy
> > produces the two line header (with (NOBDW)) and runs fine.
>
> On VMS you should always be using make.nogc. The automated logic
> that decides in the main Makefile whether to use the BDW garbage collector
> or not will not work on VMS.
Sorry, I expressed myself badly. I have not had any problems at all
with VMS because I just compile and link without reference to the
BDW garbage collector.
The problem that I reported occurred on our web machine which
runs Unix (well, Linux). In the past I have had no problems
there, everything went like a charm - at least after I had
learned to ignore the various messages whizzing past. You
did a splendid job there. After copying from VMS to the web
machine I have always run all the tests with BDW and without
exceptions everything went well straight away.
But I am baffled by what I reported.
Thanks, John
Nick - did you compile with or without BDW ?
- Manfred
Nick Forde — 2002-12-02 13:11:40
Manfred von Thun writes:
>
> I have now done all my testing under VMS, and all old things
> work fine. I was even able to test your argv and argc under VMS
> even though I had never used that before. So I copied it all to
> the Joy web page.
>
> Trouble:
> 1. using just $ make
> I get the usual long stuff from the BDW collector,
> then a compiled executable joy
> then $ joy
> produces the two line header (with (BDW)) and then a core dump.
I have tested the changes on Linux, Solaris and HP-UX both with and
without the BDW collector. I can't recreate your crash. Perhaps you
could try running in a debugger to see where it crashes, or re-compile
with -DDEBUG to print some more info. You could also try running from
a directory without any libraries to see if it is the library loading
or include file handling causing the problem.
I suspected this could have been due to the GC trying to free the argv
strings but from a quick look at the code I don't think this is a
problem.
I have also done some runtime memory profiling using Purify
(
http://www.rational.com/products/purify_unix/index.jsp) and in my
tests it only identified leaks due to objects left on the Joy
stack. If I get some free time I'll use this to do some more analysis.
> 2. using $ make -f make.nogc
> I get the compiled executable joy
> then $ joy
> produces the two line header (with (NOBDW)) and runs fine.
> 3. using $ make -f make.gc
> I get a much shorter blurb from the BDW collector,
> and then the same problem as in (1).
>
> I am lost.
> I did notice each of those compilations gave
> main.c:141
> warning: unused variable 'f' (* that is: FILE *f *)
Sorry, that was my fault. You can remove that line.
Regards,
Nick.
Manfred von Thun — 2002-12-03 04:58:07
On Mon, 2 Dec 2002, Nick Forde wrote:
> Manfred von Thun writes:
> > 1. using just $ make
> > I get the usual long stuff from the BDW collector,
> > then a compiled executable joy
> > then $ joy
> > produces the two line header (with (BDW)) and then a core dump.
>
> I have tested the changes on Linux, Solaris and HP-UX both with and
> without the BDW collector. I can't recreate your crash.
Thank you very much for being so thorough, this will save me
a lot of time. Obviously the fault is at my end somewhere.
[... good advice about debugging]
> > main.c:141
> > warning: unused variable 'f' (* that is: FILE *f *)
>
> Sorry, that was my fault. You can remove that line.
Easy.
Thanks.
- Manfred
Manfred von Thun — 2002-12-03 08:49:24
On Tue, 3 Dec 2002, Manfred von Thun wrote:
> Thank you very much for being so thorough, this will save me
> a lot of time. Obviously the fault is at my end somewhere.
Indeed, and it seems it is not even my fault - see below
> [... good advice about debugging]
Yes, very good advice it was, too. Just the -DDEBUG helped a lot.
Empty lines were OK, comments were OK, LIBRA and == were OK,
but anything involving readfactor was not. (And now I know why:
readfactor needs GC_malloc_something, and that wasn't working,
see below.)
Tearing out lots of hair, I decided to recompile the previous
version, without the command line addition. Same crash result.
So: clearly the fault is not with Nick's additions. Sorry
Nick, if I caused you unnecessary work.
Next hypothesis: I might have accidentally damaged something
in the precious gc directory. Created new directory for the
joy/gc/*.o files just in case. Deleted joy/gc.success. Deleted
joy/*.o and joy/joy. Ready for the fireworks:
$ make should start from scratch
.... watch the fireworks whizzing past
./setjmptest DBW internal test
"This appears to be a I386 running LINUX"
APPEARS ? Are things as bad as that?
...
./gctest DBW internal test
Segfault at 0x1c
Unexpected bus error or segmentation fault
make[1] [KandRtest] Aborted Core dumped
...
gcc all Joy *.c files similar messages:
undefined GC_malloc_atomic, GC_malloc_collect ...
Thus the output from just the $ make command.
So, even when compiling the gc something is wrong.
Now, my understanding is that a bus error is a hardware error,
and there is nothing I can do about that. But how come the
C-compiler did not hit that soft spot itself? Should I report
this to the chaps with the soldering irons?
Too tired to go on ...
- Manfred
Nick Forde — 2002-12-03 09:52:13
Manfred von Thun writes:
[...]
> Tearing out lots of hair, I decided to recompile the previous
> version, without the command line addition. Same crash result.
> So: clearly the fault is not with Nick's additions. Sorry
> Nick, if I caused you unnecessary work.
No, problem. I really didn't spend much time on it. I already had the
test machines at my fingertips.
> Next hypothesis: I might have accidentally damaged something
> in the precious gc directory. Created new directory for the
> joy/gc/*.o files just in case. Deleted joy/gc.success. Deleted
> joy/*.o and joy/joy. Ready for the fireworks:
>
> $ make should start from scratch
> .... watch the fireworks whizzing past
> ./setjmptest DBW internal test
> "This appears to be a I386 running LINUX"
> APPEARS ? Are things as bad as that?
> ...
> ./gctest DBW internal test
> Segfault at 0x1c
> Unexpected bus error or segmentation fault
> make[1] [KandRtest] Aborted Core dumped
> ...
> gcc all Joy *.c files similar messages:
> undefined GC_malloc_atomic, GC_malloc_collect ...
>
> Thus the output from just the $ make command.
> So, even when compiling the gc something is wrong.
This sounds like a compatibility problem between your Linux install
and the GC library used in Joy as I haven't had problems on my RedHat
7.2 Linux box. I suggest updating the current GC library from version
5.3 to the latest release (6.0). You can download it from:
http://www.hpl.hp.com/personal/Hans_Boehm/gc/gc_source/gc.tar.gz
This version uses GNU autoconf to configure the library for your
target system so it should make a better guess with system
dependencies. It doesn't look like there is any VMS support yet
though.
To use the new version of the GC drop it in place of the old version,
run configure, copy the main header file to the gc directory and
re-build:
$ mv gc gc-5.3
$ gtar -zxvf gc.tar.gz
$ ln -s gc6.0 gc
$ cd gc
$ ./configure
$ cp include/gc.h .
$ cd ..
$ gmake -f make.gc
This seems to work fine for me. I also note from the README.QUICK file
that the old build mechanism still exists so other than copying the
gc.h file the existing Joy Makefile scripts should work OK.
Regards,
Nick.
John Cowan — 2002-12-03 11:53:31
Manfred von Thun scripsit:
> Now, my understanding is that a bus error is a hardware error,
> and there is nothing I can do about that. But how come the
> C-compiler did not hit that soft spot itself? Should I report
> this to the chaps with the soldering irons?
By no means. Historically, the PDP-11 reported a "bus error" when you
tried to address something on the bus and nobody answered; i.e. a reference
to nonexistent memory. Modern kernels report SIGBUS (or sometimes
SIGSEGV; the difference has become unmeaning) when a reference to unallocated
virtual memory is made.
I think the advice about upgrading to a more recent version of the
gc is probably sound.
--
I am expressing my opinion. When my John Cowan
honorable and gallant friend is called,
jcowan@...
he will express his opinion. This is
http://www.ccil.org/~cowan
the process which we call Debate. --Winston Churchill
Martin Young — 2002-12-03 12:08:08
On Tue, 2002-12-03 at 11:53, John Cowan wrote:
> Modern kernels report SIGBUS (or sometimes
> SIGSEGV; the difference has become unmeaning) when a reference to unallocated
> virtual memory is made.
I'd suggest that this is generally not the case.
A reference to a virtual memory address which has no physical memory
backing it will cause SIGSEGV to be raised. OSes usually refuse to
allocate memory at address 0 so that dereferencing a NULL pointer always
causes a segmentation fault.
SIGBUS will be raised in one of two circumstances.
The first is when an illegal virtual address is used. The most common
cause is a mis-aligned pointer dereference (although on x86 the CPU will
patch this up for you in hardware, Sparc, Alpha and company will not).
Natural alignment is the most common restriction: every type must be
aligned by its own size e.g. ints on four byte boundaries, shorts on two
bytes boundaries etc.
The second cause of bus errors is an invalid physical address i.e. if
the machine attempts to access physical memory which does not exist. On
modern operating systems (e.g. Windows post-95, Solaris, Linux) this can
only happen as a symptom of a kernel bug.
--
Martin Young, working for: | Phone: +44(0) 1454 615151
Siroyan Limited, Bristol Design Centre, | Mobile: +44(0) 7855 758771
West Point Court, Great Park Road, | web: www.siroyan.com
Bradley Stoke, Bristol BS32 4QG. UK | email:
my@...
**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.
This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.
www.mimesweeper.com
**********************************************************************
John Cowan — 2002-12-03 12:29:56
Martin Young scripsit:
> I'd suggest that this is generally not the case.
I bow to your obviously superior knowledge.
--
John Cowan
jcowan@... www.reutershealth.com www.ccil.org/~cowan
Consider the matter of Analytic Philosophy. Dennett and Bennett are well-known.
Dennett rarely or never cites Bennett, so Bennett rarely or never cites Dennett.
There is also one Dummett. By their works shall ye know them. However, just as
no trinities have fourth persons (Zeppo Marx notwithstanding), Bummett is hardly
known by his works. Indeed, Bummett does not exist. It is part of the function
of this and other e-mail messages, therefore, to do what they can to create him.
Manfred von Thun — 2002-12-04 08:28:26
I downloaded the new GC, and it compiled fine.
Joy-with-gc compiled, too, but it crashed when running as before.
Luckily a very clever person here heard my moans.
He changed a line in make.gc in the line as marked ^^^^^ below.
All is well now.
# makefile for Joy
HDRS = globals.h
SRCS = interp.c scan.c utils.c main.c
OBJS = interp.o scan.o utils.o main.o
CC = gcc -g -ansi -pedantic -Wall -D_C_SOURCE=1 -DGC_BDW
joy: $(OBJS) gc/gc.a
# $(CC) $(OBJS) gc/gc.a -lm -o joy
$(CC) -static $(OBJS) -L gc/.libs -lgc -lm -o joy
^^^^^^^ ^^^^^^^^^^^^^^^^
$(OBJS): $(HDRS)
gc/gc.a:
cd gc; $(MAKE)
Apparently the GC by default uses shared runtime libraries,
and the way this web machine is set up this is a no-no here.
The flag -static enforces non-shared libraries.
Joy-with-gc is working fine now on our web machine under Linux.
Thank you all.
- Manfred