Last updated: 3 August 2014


ANS Forth Test Programs


Introduction

Checking that an ANS Forth system is compliant with the ANS Forth standard requires a set of test programs. This download provides some test programs that go some way towards meeting this requirement. The tests are based on the widely known tester developed by John Hayes and include his original core tests. The tests provided in the download are:

  • Core words (by John Hayes) plus some additional tests
  • Core extension words
  • The Double Number word set and extensions
  • The Exception word set and extensions
  • The File-Access word set and extensions
  • The Memory-Allocation word set
  • The Programming-Tools word set and extensions (incomplete - a few words in each)
  • The Search-Order word set and extensions
  • The String word set

The download also includes this file as readme.html

The tests are not comprehensive, no claim is made about their being correct and no warranty is provided. However they have been run successfully with such widely used systems as GForth, VFX Forth, SwiftForth and Win32 Forth, any failing tests being due to the Forth system rather than the tests themselves.

Some provisional floating point wordset tests are also available.

Downloads

Version 0.10, 3 August 2014, anstests0.10.zip

See the file changes.txt for a summary of the changes to the tests.

Floating point test programs, fpanstests0.1.zip, for more details see below

Running the tests

  1. Unzip the files into a suitable directory
  2. Start the Forth system and set the working directory/file path to that directory
  3. Possibly edit the file paths in the file runtests.fth
  4. Select either tester.fr or ttester.fs by commenting out the unwanted file. Note that tester.fr is better for developing a new Forth system whereas ttester.fs is more suited for regression testing of an existing system.
  5. Include the file runtests.fth to run all the tests

There is a flag, called VERBOSE, in file tester.fr that can be set to obtain more output from the test programs. When a test fails an error message is displayed and the tests carry on. To stop on the first failure uncomment the line marked *** in tester.fr

Error Messages

There are two possible error messages when a test fails:

  1. "WRONG NUMBER OF RESULTS:" when the stack depth is wrong
  2. "INCORRECT RESULT:" when the stack contents are incorrect

Both are followed by the offending line of source code.

Other Failures

A test may fail for some system not recognising a standard word, crashing, refusing to compile some test code or other system fault. These failures stop further tests from running.

Sample Output

The output from running the tests under GForth 0.7.0 using tester.fr with the VERBOSE flag clear are:

Gforth 0.7.0, Copyright (C) 1995-2008 Free Software Foundation, Inc.
Gforth comes with ABSOLUTELY NO WARRANTY; for details type `license'
Type `bye' to exit
s" gwj\tests\runtests.fth" included
Running ANS Forth test programs, version 0.10

*********************YOU SHOULD SEE THE STANDARD GRAPHIC CHARACTERS:
 !"#$%&'()*+,-./0123456789:;<=>?@
ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`
abcdefghijklmnopqrstuvwxyz{|}~
YOU SHOULD SEE 0-9 SEPARATED BY A SPACE:
0 1 2 3 4 5 6 7 8 9
YOU SHOULD SEE 0-9 (WITH NO SPACES):
0123456789
YOU SHOULD SEE A-G SEPARATED BY A SPACE:
A B C D E F G
YOU SHOULD SEE 0-5 SEPARATED BY TWO SPACES:
0  1  2  3  4  5
YOU SHOULD SEE TWO SEPARATE LINES:
LINE 1
LINE 2
YOU SHOULD SEE THE NUMBER RANGES OF SIGNED AND UNSIGNED NUMBERS:
  SIGNED: -80000000 7FFFFFFF
UNSIGNED: 0 FFFFFFFF
*
PLEASE TYPE UP TO 80 CHARACTERS:
This is a test run on 3 August 2014
RECEIVED: "This is a test run on 3 August 2014"
*redefined GDX
End of Core word set tests
*********
You should see 2345: 2345

End of additional Core tests
***********redefined ma1  ************

Output from .(
You should see -9876: -9876
and again: -9876


On the next 2 lines you should see First then Second messages:
First message via .(
Second message via ."

*

Output from .R and U.R
You should see lines duplicated:
indented by 0 spaces
1984383623
1984383623
-2088648480
-2088648480
1984383623
1984383623
2206318816
2206318816

indented by 0 spaces
1984383623
1984383623
-2088648480
-2088648480
1984383623
1984383623
2206318816
2206318816

indented by 5 spaces
     1984383623
     1984383623
     -2088648480
     -2088648480
     1984383623
     1984383623
     2206318816
     2206318816

***
End of Core Extension word tests
*****************
You should see lines duplicated:
     8970676912557384689
     8970676912557384689
        8970676912557384689
        8970676912557384689
     -8522862768232894102
     -8522862768232894102
          -8522862768232894102
          -8522862768232894102
*
End of Double-Number word tests
***
End of Exception word tests
***********
End of File-Access word tests
***INCORRECT RESULT: T{ addr1 @ -1 RESIZE 0= -> addr1 @ FALSE }T

End of Memory-Allocation word tests
*****
End of Programming Tools word tests
**********
ONLY FORTH DEFINITIONS search order and compilation wordlist
Forth Root     Forth
Plus another unnamed wordlist at the head of the search order
??? Forth Root     ???
End of Search Order word tests
********
End of String word tests


Forth tests completed
 ok

As can be seen GForth failed one test.

The output will differ for different Forth systems, e.g. warnings of redefinitions, error messages and display of the search order.

Floating Point Tests

Some floating point test programs from various sources have been collected together and made available in this download. These tests are unproven and provided in the hope that implementers of ANS Forth/Forth 200X systems will try them and report back on any perceived deficiencies which can then be resolved. When they are generally agreed to be correct they can be incorporated into the above set of test programs. For more details see the file readme.txt in the download.

Additional tests

As indicated above some of the word set tests are incomplete, others are absent. If anyone wishes to contribute test programs to cover these gaps, please contact me via the email address below.

Contact

Gerry Jackson