Last updated: 3 August 2014
ANS Forth Test Programs
Introduction
Checking that an ANS Forth system is compliant with the ANS Forth standard requires a set of test programs. This download provides some test programs that go some way towards meeting this requirement. The tests are based on the widely known tester developed by John Hayes and include his original core tests. The tests provided in the download are:
- Core words (by John Hayes) plus some additional tests
- Core extension words
- The Double Number word set and extensions
- The Exception word set and extensions
- The File-Access word set and extensions
- The Memory-Allocation word set
- The Programming-Tools word set and extensions (incomplete - a few words in each)
- The Search-Order word set and extensions
- The String word set
The download also includes this file as readme.html
The tests are not comprehensive, no claim is made about their being correct and no warranty is provided. However they have been run successfully with such widely used systems as GForth, VFX Forth, SwiftForth and Win32 Forth, any failing tests being due to the Forth system rather than the tests themselves.
Some provisional floating point wordset tests are also available.
Downloads
Version 0.10, 3 August 2014, anstests0.10.zip
See the file changes.txt for a summary of the changes to the tests.
Floating point test programs, fpanstests0.1.zip, for more details see below
Running the tests
- Unzip the files into a suitable directory
- Start the Forth system and set the working directory/file path to that directory
- Possibly edit the file paths in the file runtests.fth
- Select either tester.fr or ttester.fs by commenting out the unwanted file. Note that tester.fr is better for developing a new Forth system whereas ttester.fs is more suited for regression testing of an existing system.
- Include the file runtests.fth to run all the tests
There is a flag, called VERBOSE, in file tester.fr that can be set to obtain more output from the test programs. When a test fails an error message is displayed and the tests carry on. To stop on the first failure uncomment the line marked *** in tester.fr
Error Messages
There are two possible error messages when a test fails:
- "WRONG NUMBER OF RESULTS:" when the stack depth is wrong
- "INCORRECT RESULT:" when the stack contents are incorrect
Both are followed by the offending line of source code.
Other Failures
A test may fail for some system not recognising a standard word, crashing, refusing to compile some test code or other system fault. These failures stop further tests from running.
Sample Output
The output from running the tests under GForth 0.7.0 using tester.fr with the VERBOSE flag clear are:
|
As can be seen GForth failed one test.
The output will differ for different Forth systems, e.g. warnings of redefinitions, error messages and display of the search order.
Floating Point Tests
Some floating point test programs from various sources have been collected together and made available in this download. These tests are unproven and provided in the hope that implementers of ANS Forth/Forth 200X systems will try them and report back on any perceived deficiencies which can then be resolved. When they are generally agreed to be correct they can be incorporated into the above set of test programs. For more details see the file readme.txt in the download.
Additional tests
As indicated above some of the word set tests are incomplete, others are absent. If anyone wishes to contribute test programs to cover these gaps, please contact me via the email address below.