New release of Factor

Slava Pestov — 2004-05-27 19:52:06

Hi everybody,

There is a new release of Factor. For now its hosted at
http://www.jedit.org/factor/, but soon it will move.

Since the last release, there were a large number of bug fixes,
optimizations, cleanups, and so on. The big new language feature is
multiple vocabularies, for incrased code modularity and freedom in
naming words without fear of clashes.

There is one new addition to the very small set of available documentation:

http://www.jedit.org/factor/intro.txt

The compiler is disabled by default right now -- this will change in the
next release. In the meantime, enter this as soon as the interpreter has
started, for improved performance:

compile-all

The interpreter takes a while to load -- especially if the time to
compile the library is taken into account. To solve this, I'm working on
a "workspace" feature. Its basically a persistent store. Its in the very
early stages right now and disabled by default. The idea is instead of
re-parsing and re-compiling the entire Factor library on startup --
which takes time -- one dumps a "workspace". What makes this interesting
is that global variables and word definitions entered a the interactive
interpreter are automatically saved. Most of the code is done, it just
needs debugging and testing. Expect it to be more or less functional in
the next release.

Slava

Chris Double — 2004-05-28 02:19:37

You mention on the Factor website that the language supports
continuations. Are there examples of usage? Are these continuations in
the Scheme call/cc sense?

Chris.
--
Chris Double
chris.double@...

Slava Pestov — 2004-05-28 03:20:52

Hi,

First of all, thanks for the questions and interest! I never thought
anybody would care about this.

Yes, Factor's continuations are modelled after Scheme. Two words are
used to capture continuations:

callcc0
callcc1

Both have stack effect ( quotation -- ). They capture a continuation
right after the callcc call, push it on the stack, and call the
quotation. The quotation is then supposed to store the continuation
somewhere.

The capture continuation is in fact a quotation as well, so you can call
it with the 'call' primitive to restore execution.

The difference between callcc0 and callcc1 is the nature of the captured
continuation.

callcc0 simply restores execution to the point of continuation capture.
callcc1 restores execution and 'transfers' one parameter from the
current data stack to the restored data stack.

So callcc0 is like task switching, and callcc1 is like co-routines.

A non-trivial example can be found in the listener implementation (the
listener is the GUI interpreter). Continuations are used here because
GUI is event-driven ("push"), while the top-level interpreter is "pull".
I could just have written a new top-level interpreter with a "push"
model, but reusing the exact same code is much nicer.

When the top-level interpreter loop decides to read a line of input, it
calls a number of words which eventually delegate to the listener-readln
word.

The two key words in the listener implementation are:

: listener-readln* ( continuation -- line )
"listener" get
[ "factor.Cons" ]
"factor.listener.FactorListener"
"readLine" jinvoke ;

: listener-readln ( -- line )
reset-attrs [ listener-readln* suspend ] callcc1 ;

There is a lot of low level noise in these definitions, but they key
idea is that listener-readln captures the current continuation and calls
listener-readln*, which passes it to the GUI code, which is really a
thin layer over the Swing text pane. The GUI code is event-driven; so it
simply stores the continuation in an instance variable.

The "suspend" word clears all three stacks, effectively aborting the
interpreter.

Then later on when the ENTER event is received, the GUI pushes the input
line on the stack and calls the continuation. Since the continuation was
captured using callcc1, execution resumes at listener-readln, with the
new line of text on the restored stack. The top level interpreter
continues execution, completely unaware of what took place.

The game I'm working on uses continuations extensively for various
in-game events llike character interactions, waiting for user input, and
even as a multitasking engine in the implementation of big explosions,
that consist of multiple consecutive blasts.

Note that continuations cannot be compiled, and probably never will be,
due to limitations in the JVM. So they're not really suitable for use in
tight loops and such.

Chris Double wrote:

>You mention on the Factor website that the language supports
>continuations. Are there examples of usage? Are these continuations in
>the Scheme call/cc sense?
>
>Chris.
>
>

Chris Double — 2004-05-28 03:35:18

On Thu, 27 May 2004 23:20:52 -0400, "Slava Pestov" <slava@...>
said:
> So callcc0 is like task switching, and callcc1 is like co-routines.

Nice! So the following Scheme:

(define test #f)
(define (do-test)
(call/cc (lambda (x) (set! test x)))
(display "here") (newline))

(do-test)
=> "here"
(test #f)
=> "here"
(test #f)
=> "here"

Is equivalent to the following Factor:

: do-test [ "test" swap put ] callcc0 "here" print ;
do-test
=> "here"
"test" get call
=> "here"
"test" get call
=> "here"

With such multi-shot continuations it would be possible to do a
continuation based web server in Factor.

Chris.
--
Chris Double
chris.double@...

Slava Pestov — 2004-05-28 03:48:54

Chris Double wrote:

>Is equivalent to the following Factor:
>
>: do-test [ "test" swap put ] callcc0 "here" print ;
>do-test
>=> "here"
>"test" get call
>=> "here"
>"test" get call
>=> "here"
>
>
Exactly! A bit off topic, but there is a word:

: set swap put ;

'set' is used more often actually, because the value/name ordering comes
up more often than name/value.

>With such multi-shot continuations it would be possible to do a
>continuation based web server in Factor.
>
>
How does that work?

Slava

Chris Double — 2004-05-28 04:10:59

On Thu, 27 May 2004 23:48:54 -0400, "Slava Pestov" <slava@...>
said:
> >With such multi-shot continuations it would be possible to do a
> >continuation based web server in Factor.
> >
> >
> How does that work?

If you are fluent with Scheme you can take a look at:

http://www.double.co.nz/scheme/modal-web-server.html

It is a way of writing a web application where you can code the
application logic in a procedural manner and have the system
automatically break it up into the HTTP request/response mechanism.

For example, see:

http://radio.weblogs.com/0102385/2003/10/08.html

Here I define a simple application in an imaginary programming language:

define method rate-app()
let rate = get-rate();
if(rate <= 0)
show-error("Rate must be > 0");
else
let amount = get-amount();
if(amount <= 0)
show-error("Amount must be > 0");
else
show-result(rate, amount);
end if;
end if;
/* Redisplay from the beginning */
rate-app();
end method;

This is run by the web server when the user requests the URL associated
with it. When the 'get-rate' or 'show-' functions are called the web
server captures the current continuation and associates it with a
randomly generated URL. The HTML page is shown topthe user and it has a
link with that URL in it. When the user clicks that link the continuation
associated with that URL is called which resumes the original function at
the point past the request until it hits another 'show-page' type
request.

This makes web applications quite easy to develop as the code to do it
looks very much like normal application code. The most well known
framework of this type is the Smalltalk Seaside framework:
http://www.beta4.com/seaside2/

I work on a Scheme based framework using Sisc (sisc.sourceforge.net).
Sisc enables continuations to be serialized so I serialize them to a
database and re-load them on user requests. It works quite well.

Links to other systems and information at:

http://radio.weblogs.com/0102385/2003/08/30.html

Chris.








--
Chris Double
chris.double@...

Slava Pestov — 2004-05-28 04:31:18

Chris Double wrote:
> This is run by the web server when the user requests the URL associated
> with it. When the 'get-rate' or 'show-' functions are called the web
> server captures the current continuation and associates it with a
> randomly generated URL. The HTML page is shown topthe user and it has a
> link with that URL in it. When the user clicks that link the continuation
> associated with that URL is called which resumes the original function at
> the point past the request until it hits another 'show-page' type
> request.

Ah, nice! I'll take a shot at implementing this in Factor.

> I work on a Scheme based framework using Sisc (sisc.sourceforge.net).
> Sisc enables continuations to be serialized so I serialize them to a
> database and re-load them on user requests. It works quite well.

How exactly do serializable continuations work? What if one of the
environments in the callstack references an unserializable object? What
if I save a continuation and restore it in a different SISC instance,
with different libraries loaded, etc?

Slava

Chris Double — 2004-05-28 04:36:59

On Fri, 28 May 2004 00:31:18 -0400, "Slava Pestov" <slava@...>
said:
> How exactly do serializable continuations work?

Everything reachable by the current call stack, except for the 'top
level' is serialized.

> What if one of the environments in the callstack references an unserializable object?

The serialization fails with an exception. In practice this is not common
as only non-serializable Java objects have this problem. In the web
server I use that includes the servlet request and response so I copy
this to top level 'dynamic' variables (ie. thread local variables) and
set the local variable to 'false' so the call stack can be serialized.

> What if I save a continuation and restore it in a different SISC instance,
> with different libraries loaded, etc?

Loading the continuation from a difference instance works fine. Even on a
different machine. As long as the top level definitions are loaded and
accessible. So in practice you need to ensure that all the libraries that
that continuation uses are loaded in the SISC instance.

Here's an example that I did that serializes the current continuation,
sends it to another machine over a socket, and it resumes it. It allows
'migrating' threads or mobile agents:

http://radio.weblogs.com/0102385/2004/03/24.html

For a small example like that the serialized continuation is about 4Kb.
In my web server examples most continuations are less than 50Kb when
serialized.

Chris.
--
Chris Double
chris.double@...