Deferred Words proposal

Deferred Words

Poll standings

See below for voting instructions.

Systems

[ ] conforms to ANS Forth.
 iForth
 Gforth
 bigFORTH
 Carbon MacForth
 SwiftForth, SwiftX
 Quartus Forth
 PFE (Guido Draheim)
[ ] already implements the proposal in full since release [ ]:
 Gforth 0.7.0 (Anton Ertl)
 4th (Hans Bezemer, partially ANS compliant)
[ ] implements the proposal in full in a development version:
 iForth (Marcel Hendrix)
 Quartus Forth 1.5.3b (Neal Bridges)
 PFE
[ ] will implement the proposal in full in release [ ].
 iForth 2.01
 Gforth 0.7
 bigFORTH 2.0.12 (Bernd Paysan)
 Carbon MacForth 6.05 (Ward McFarland)
 PFE 33.67
[ ] will implement the proposal in full in some future release.
There are no plans to implement the proposal in full in [ ].
 SwiftForth, SwiftX (Elizabeth Rather)
[ ] will never implement the proposal in full:

Programmers

[ ] I have used (parts of) this proposal in my programs:
  Hans Bezemer
  Brad Eckert
  Marcel Hendrix
  Elko Tchernev
  David N. Williams
  Anton Ertl
  Bernd Paysan
  Graham Smith
  Elizabeth Rather
  Richard Borrell
  Neal Bridges
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it:
  (same as "have-used", plus)
  Bruce McFarling (interesting systems: Gforh, Win32Forth)
[ ] I would use (parts of) this proposal in my programs if this
    proposal was in the Forth standard:
  (same as "have-used", plus)
  Bruce McFarling
[ ] I would not use (parts of) this proposal in my programs.

Informal results

4th (Hans Bezemer, partially ANS compliant) supports DEFER and IS since V3.4e, and will support DEFER@ and DEFER! in V3.4e-pre10. Hans Bezemer writes that he will not add ACTION-OF, because it can be easily replaced with ['] word defer@. He wrote in 2010 that ACTION-OF is supported by 4th now.

Brad Eckert writes that ACTION-OF is redundant.

Elko Tchernev writes: "Actually, I am using (and will be using) the proposal in its entirety, regardless of whether it is standard or implemented (I just define the words in my utilities file). When (if) they become standard, I'll stop defining and will use the system's definitions instead."

Elizabeth Rather writes that she uses and implements DEFER and IS, but has no interest in DEFER@ and DEFER!.

Guido Draheim writes: DEFER and IS were defined since PFE 30.38 (Dec. 2000).

Problem

How do we program a hook into which we insert functionality later? How do we program an indirect recursion or other forward references to words?

Proposal

DEFER ( "<spaces>name" -- ) CORE-EXT

Skip leading space delimiters. Parse name delimited by a space. Create a definition for name with the execution semantics defined below.

name Execution: ( i*x -- j*x )

Execute xt associated with name. An ambiguous condition exists if name has not been associated with an xt yet.

IS CORE-EXT

Interpretation: ( xt "<spaces>name" -- ): Skip leading spaces and parse name delimited by a space. Set name to execute xt. An ambiguous condition exists if name was not defined by DEFER.
Compilation: ( "<spaces>name" -- ): Skip leading spaces and parse name delimited by a space. Append the run-time semantics given below to the current definition. An ambiguous condition exists if name was not defined by DEFER.
Run-time: ( xt -- ): Set name to execute xt.

An ambiguous condition exists if POSTPONE, [COMPILE], ['] or ' is applied to IS.

DEFER@ ( xt1 -- xt2 ) CORE-EXT

xt2 is the xt associated with the deferred word corresponding to xt1. An ambiguous condition exists if xt1 is not for a word defined via DEFER, or if the deferred word has not been associated with an xt yet.

DEFER! ( xt2 xt1 -- ) CORE-EXT

Set the word xt1 to execute xt2. An ambiguous condition exists if xt1 is not for a word defined via DEFER.

ACTION-OF CORE-EXT

Interpretation: ( "<spaces>name" -- xt ): Skip leading spaces and parse name delimited by a space. xt is the xt associated with name. An ambiguous condition exists if name was not defined by DEFER, or if the name has not been associated with an xt yet.
Compilation: ( "<spaces>name" -- ): Skip leading spaces and parse name delimited by a space. Append the run-time semantics given below to the current definition. An ambiguous condition exists if name was not defined by DEFER.
Run-time: ( -- xt ): xt is the execution token associated with name when the run-time semantics is performed. An ambiguous condition exists if name has not been associated with an xt yet.

An ambiguous condition exists if POSTPONE, [COMPILE], ['] or ' is applied to ACTION-OF.

Typical Use

DEFER plus
' + IS plus
1 2 plus .
1 2 ' plus DEFER@ execute .
' + ' plus defer! \ same as ' + IS plus
1 2 ACTION-OF plus execute .

Remarks

Why standardize this?

These words can be defined in ANS Forth, e.g., like this:

: defer ( "name" -- )
  create ['] abort ,
does> ( ... -- ... )
  @ execute ;

: defer@ ( xt1 -- xt2 )
  >body @ ;

: defer! ( xt2 xt1 -- )
  >body ! ;

: is
  state @ if
    POSTPONE ['] POSTPONE defer!
  else
    ' defer!
  then ; immediate

: action-of
 state @ if
   POSTPONE ['] POSTPONE defer@
 else
   ' defer@
 then ; immediate

example implementation and test cases

So why standardize these words? For the following reasons:

Many systems define DEFER and IS already.
DEFER and IS are used in many programs.
They will be implemented more efficiently on many systems. Deferred words are used with relatively high frequency (0.6%-1% of the primitives in the peep benchmark traces), so their efficiency is important.

Why DEFER@ and DEFER! ?

These words are not commonly present, but the functionality of these words is available in various systems through knowledge of the internal data structures of the implementation. Such words are necessary to implement the functionality of words that some Forth systems have (e.g., WHAT'S and DEFERS in Gforth can be implemented with DEFER@). Moreover, IS parses and has non-default compilation semantics, so DEFER! should be present for situations where IS is cumbersome or error-prone. Open Firmware has BEHAVIOR (equivalent to DEFER@), MacForth has WHATIS>> and IS>>. Michael Gassanenko uses BEHAVIOR and BEHAVIOR! (for DEFER!). Several people have announced their preference for the names DEFER@ and DEFER!.

Why ACTION-OF ?

Many systems have a parsing word that provides access to the xt associated with a deferred word: IS? in iForth, WHAT'S in Gforth, WHATIS in MacForth, ACTION-OF in VFX Forth. Stephen Pelc is a very vocal advocate of having such a word. Several people (e.g., Ward McFarland) indicate that such a word has quite a bit of usage in some environments.

>BODY vs. DEFER@/DEFER!

Instead of having DEFER@ and DEFER!, one could also extend >BODY to be applicable to deferred words, and to use >BODY @ instead of DEFER@ and >BODY ! instead of DEFER!. This change would eliminate two words, but also eliminate a number of implementation options. Two systems have been named where >BODY @/! would not work for deferred words (as implemented on these systems).

IS vs. TO

Some people prefer to use TO in place of IS. This proposal allows a system to implement such functionality in TO; however, it standardizes on IS, since many systems implement IS and because IS is easier to implement. It is also very easy to let systems that implement TO for deferred words to also implement IS.

The disadvantage of this approach is that it would be more work to make programs standard that have used TO with deferred words. However, the systems running these programs could easily help the conversion by emitting an optional warning when they see TO used with a deferred word.

In the Rfd phase, several people have expressed a preference for IS, none a preference for TO.

STATE-smartness etc.

IS is defined as a word with interpretation and non-default compilation semantics, with restrictions that allow it to be implemented as a STATE-smart word, much like TO. An alternative would be to define two words, e.g., IS and [IS], like ' and [']. The definition above reflects the more common practice (AFAIK). Similar reasoning applies to ACTION-OF. If you need to do some things where this definition of these words is in the way, use DEFER@ and DEFER!.

Default action

Many systems define the default action of a deferred word to be noop. This proposal does not define a default action, and thus requires programs to set the action before using the deferred word. Systems could still use noop as a default action, but programs relying on that behaviour would continue their dependency on the system. Systems could also use other default actions, e.g., reporting the non-initialization of the deferred word as warning (which would help find such uninitialized uses, which may indicate bugs).

Multi-tasking

Like ANS Forth '94, the present wordset does not discuss multi-tasking at all. If a system implements multi-tasking, it could make deferred words shared or task-local, or it could have a mechanism that lets the user decide between these alternatives.

Experience

DEFER and IS have been used in many systems and programs for a long time. ACTION-OF also has been used, often under a different name (WHAT'S, WHATIS, IS?). DEFER@ and DEFER! also have been present in some systems (under different names); on many systems >BODY @/! were used instead.

Change history

2004-11-04: Revision incorporating feedback until now.
2004-10-28: Corrected information about BEHAVIOR.
2004-10-26: Added sections "Questions", "Parsing words for DEFER@" and ">BODY vs. DEFER@/DEFER!". Fixed use of ENDIF in example implementation. Revised the efficiency argument for standardization.
2004-10-25: fixed defered->deferred. Changed the names of DEFERRED@/! to DEFER@/!.

Comments

Several people expressed a preference for defer@ defer! over alternative names.

Only Stephen Pelc expressed an opinion on the parsing variant of defer@. He favours the name action-of.

Gary Chanson and Stephen Pelc gave examples for systems where >body @/! does not work as defer@/!.

Marcel Hendrix reports the following statistics:

Searching for: IS?        Found   60 occurrence(s)
Searching for: IS         Found 2137 occurrence(s)
Searching for: DEFER      Found  716 occurrence(s)
Searching for: ' >BODY !  Found   10 occurrence(s)
Searching for: ' >BODY @  Found   20 occurrence(s)

Ward McFarland writes that MacForth has IS>> for DEFER!, WHATIS>> for DEFER@, and WHATIS for a parsing DEFER@. IS and WHATIS are STATE-smart.

Thomas Pornin suggests adding a word IS-FOREVER that freezes the contents of the deferred word. That's for another proposal IMO. Bernd Paysan suggests having another word (FORWARD) for forward references instead.

Stephen Pelc suggests that we should not have DEFER! and DEFER@, just IS (and maybe [IS]) and BEHAVIOR (or ACTION/[ACTION]).

Bernd Paysan suggests that we should use >BODY @ and >BODY ! instead of DEFER@ and DEFER!.

Voting instructions

Fill out the appropriate ballot(s) below and mail it/them to me <anton@mips.complang.tuwien.ac.at>. Your vote will be published (including your name (without email address) and/or the name of your system) here. You can vote (or change your vote) at any time by mailing to me, and the results will be published here.

Note that you can be both a system implementor and a programmer, so you can submit both kinds of ballots.

Ballot for systems

If you maintain several systems, please mention the systems separately in the ballot. Insert the system name or version between the brackets. Multiple hits for the same system are possible (if they do not conflict).

[ ] conforms to ANS Forth.
[ ] already implements the proposal in full since release [ ].
[ ] implements the proposal in full in a development version.
[ ] will implement the proposal in full in release [ ].
[ ] will implement the proposal in full in some future release.
There are no plans to implement the proposal in full in [ ].
[ ] will never implement the proposal in full.

If you want to provide information on partial implementation, please do so informally, and I will aggregate this information in some way.

Ballot for programmers

Just mark the statements that are correct for you (e.g., by putting an "x" between the brackets). If some statements are true for some of your programs, but not others, please mark the statements for the dominating class of programs you write.

[ ] I have used (parts of) this proposal in my programs.
[ ] I would use (parts of) this proposal in my programs if the systems
    I am interested in implemented it.
[ ] I would use (parts of) this proposal in my programs if this
    proposal was in the Forth standard.
[ ] I would not use (parts of) this proposal in my programs.

If you feel that there is closely related functionality missing from the proposal (especially if you have used that in your programs), make an informal comment, and I will collect these, too. Note that the best time to voice such issues is the RfD stage.

Anton Ertl