TM(1)

NAME
tm - Multi-language test runner for embedded development

SYNOPSIS
tm [OPTIONS] [PATTERNS...]

DESCRIPTION
tm is a multi-language test runner built with Bun that discovers, compiles, and executes tests across shell, C, JavaScript, TypeScript, and Ejscript with configurable patterns and parallel execution. It is designed for embedded development environments and provides a simple, consistent interface for running tests. TestMe discovers test files with specific extensions (.tst.sh, .tst.c, .tst.js, .tst.ts, .tst.es) and executes them according to their type. C tests are automatically compiled before execution, while script tests are run directly. By default, TestMe shows test names as they execute and displays a summary of results. Use --quiet for silent operation with only exit codes, or --verbose for detailed output with additional information.

OPTIONS

--chdir DIR
    Change to directory before running tests. Useful for running tests from different locations.

--clean
    Clean all .testme artifact directories. Removes compilation outputs and temporary files.

-c, --config FILE
    Use specific configuration file instead of searching for testme.json5.

-d, --debug
    Launch debugger for C tests. Uses GDB on Linux and Xcode on macOS.

--depth NUMBER
    Run tests with depth requirement <= NUMBER (default: 0). Tests with higher depth requirements in their configuration will be skipped. Sets TESTME_DEPTH environment variable for tests.

-h, --help
    Show help message with usage information and examples.

--init
    Create testme.json5 configuration file in the current directory with sensible defaults. Exits with error if file already exists.

-k, --keep
    Keep .testme artifact directories after running tests. Useful for debugging compilation issues.

-l, --list
    List discovered tests without running them. Shows all test files that would be executed.

--new NAME
    Create new test file from template. Auto-detects test type from extension (e.g., --new math.c creates math.tst.c). Supports C, Shell, JavaScript, and TypeScript templates.

-p, --profile NAME
    Set build profile (overrides configuration and PROFILE environment variable). Used in ${PROFILE} variable expansion for platform-specific build paths.

-q, --quiet
    Run silently with no output, only exit codes. Useful for scripting and automation.

-s, --show
    Display the C compile command used. Shows compiler flags, libraries, and paths for debugging.

--step
    Run tests one at a time with prompts. Forces serial mode and prompts before each test execution.

-v, --verbose
    Enable verbose mode with detailed output. Sets TESTME_VERBOSE environment variable for tests.

-V, --version
    Show version information.

-w, --workers NUMBER
    Number of parallel workers (overrides configuration). Must be a positive integer.

PATTERNS
Test patterns are glob-style expressions used to filter which tests to run:

Full filenames
    "math.tst.c", "*.tst.c" - Match specific files or all files of a type

Base names
    "math", "test*" - Match base names across all test types (math.tst.c, math.tst.js, etc.)

Directory names
    "integration", "unit/api" - Match all tests in the specified directory or subdirectory

Path patterns
    "**/math*", "tests/*.tst.c" - Match files in subdirectories or specific paths If no patterns are provided, all discoverable tests are run.

TEST TYPES
TestMe supports five types of test files:

.tst.sh
    Shell script tests. Must be executable or have a shebang line. Exit code 0 indicates success.

.tst.c
    C program tests. Automatically compiled with gcc/clang using configuration flags and libraries. Linked against specified libraries and run as executables.

.tst.js
    JavaScript tests. Executed directly with the Bun runtime.

.tst.ts
    TypeScript tests. Executed directly with Bun's TypeScript support.

.tst.es
    Ejscript tests. Executed with the ejs runtime. Can use require configuration to preload modules.

TESTING UTILITIES
TestMe provides built-in testing helper functions for C, JavaScript, and TypeScript tests.

  C Testing Functions (testme.h)
Include testme.h in C tests to access these assertion and utility functions:

teq(a, b, msg)
    Assert that two values are equal. Prints success message or fails with detailed output.

tneq(a, b, msg)
    Assert that two values are not equal.

ttrue(expr, msg)
    Assert that expression evaluates to true.

tfalse(expr, msg)
    Assert that expression evaluates to false.

tmatch(str, pattern, msg)
    Assert that string matches the given pattern.

tcontains(str, substr, msg)
    Assert that string contains the given substring.

tfail(msg)
    Immediately fail the test with the given message.

tget(key, default)
    Get environment variable value with fallback to default.

tgeti(key, default)
    Get environment variable as integer with fallback to default.

thas(key)
    Check if environment variable exists (returns 1 if exists, 0 otherwise).

tdepth()
    Get current test execution depth from TESTME_DEPTH environment variable.

tinfo(...), tdebug(...)
    Print informational messages (printf-style formatting).

  JavaScript/TypeScript Testing Functions (testme.js)
Import functions from testme.js for JavaScript and TypeScript tests:

teq(received, expected, msg)
    Assert that received value equals expected value.

tneq(received, expected, msg)
    Assert that received value does not equal expected value.

ttrue(expr, msg)
    Assert that expression is truthy.

tfalse(expr, msg)
    Assert that expression is falsy.

tmatch(str, pattern, msg)
    Assert that string matches regex pattern.

tcontains(str, substr, msg)
    Assert that string contains substring.

tfail(msg)
    Immediately fail the test with message.

tget(key, default)
    Get environment variable with default fallback.

thas(key)
    Check if environment variable exists (returns numeric value).

tverbose()
    Check if verbose mode is enabled (returns boolean).

tdepth()
    Get current test execution depth.

tinfo(...), tdebug(...)
    Print informational messages.

tassert(expr, msg)
    Alias for ttrue() function. All testing functions automatically handle test failure by printing descriptive error messages with file locations and exiting with non-zero status codes.

WORKING DIRECTORY
All tests execute with their working directory (CWD) set to the directory containing the test file. This ensures consistent behavior across all test types and allows tests to access relative files reliably.

C Tests
    Compiled in the .testme artifact directory but executed from the test file's directory. Xcode debugging projects also set the working directory to the test directory.

Script Tests
    Shell, JavaScript, and TypeScript tests execute directly from the test file's directory.

Relative File Access
    Tests can reliably access configuration files, data files, and other resources using relative paths from their location.

CONFIGURATION
TestMe supports hierarchical configuration using nested testme.json5 files throughout your project structure.

  Configuration Discovery
TestMe discovers configuration files using the following priority order (highest to lowest):

1.  CLI arguments (highest priority)

2.  Test-specific testme.json5 (nearest to test file)

3.  Project testme.json5 (walking up directory tree)

4.  Built-in defaults (lowest priority)

  Nested Configuration Behavior
Each test file gets its own configuration resolution by walking up from the test file's directory to find the nearest testme.json5 file. This enables:

•   Project-wide defaults at the repository root

•   Module-specific overrides in subdirectories

•   Test-specific configuration closest to individual tests

•   Automatic merging with CLI arguments preserved For example, a project structure like:
    project/
    ├── testme.json5          # Project defaults
    ├── module-a/
    │   ├── testme.json5      # Module-specific settings
    │   └── test.tst.c
    └── module-b/
        └── test.tst.c        # Uses project defaults
    Configuration files support:

  Compiler Settings
Configure C compilation with custom compilers, flags, and libraries, and Ejscript module requirements:
{
    compiler: {
        c: {
            compiler: "gcc",
            flags: ["-std=c99", "-Wall", "-Wextra"],
            libraries: ["m", "pthread", "mylib"]
        },
        es: {
            require: "testme"  // Modules to preload with --require
        }
    }
}

  Execution Settings
Control test execution behavior:
{
    execution: {
        timeout: 30000,        // Timeout per test (ms)
        parallel: true,        // Run tests in parallel
        workers: 4,            // Number of parallel workers
    }
}

  Output Settings
Control output formatting:
{
    output: {
        verbose: false,        // Show detailed output
        format: "simple",      // simple, detailed, json
        colors: true          // Enable colored output
    }
}

  Pattern Settings
Configure test discovery:
{
    patterns: {
        include: ["**/*.tst.c", "**/*.tst.sh"],
        exclude: ["**/node_modules/**", "**/.*/**"]
    }
}

  Test Control Settings
Configure whether tests are enabled, minimum depth requirements, and setup delays:
{
    enable: true,              // Enable, disable, or require explicit naming
    depth: 0,                  // Minimum depth required to run tests (default: 0)
}
The enable setting accepts three values:

•   true (default): Tests run normally when discovered by pattern matching

•   false: Tests are completely disabled and skipped

•   'manual': Tests only run when explicitly named by full path or base name (excludes directory-level or wildcard patterns) Set enable: false to disable all tests in a directory. Disabled directories are skipped during execution and excluded from --list output. In verbose mode, disabled directories show a "🚫 Tests disabled" message. Set enable: 'manual' to require explicit test naming. Manual tests are excluded when using directory-level patterns (e.g., running tm in a directory) or wildcard patterns (e.g., *.tst.c), but will run when named explicitly (e.g., tm math or tm test/slow.tst.c). This is useful for slow tests, destructive tests, or tests requiring special setup that should not run automatically. Set depth: N to require --depth N or higher to run tests in this directory. This is useful for marking integration or resource-intensive tests that should only run when explicitly requested. Tests with higher depth requirements than the current --depth value are skipped.

  Service Settings
Configure skip, prep, setup and cleanup commands:
{
    services: {
        skip: "check-requirements",  // Check if tests should run (0=run, non-zero=skip)
        prep: "make build",
        setup: "docker-compose up -d",
        cleanup: "docker-compose down",
        skipTimeout: 30000,
        prepTimeout: 30000,
        setupTimeout: 30000,
        cleanupTimeout: 10000,
        delay: 3000                // Wait 3 seconds after setup before tests
    }
}
The skip command runs first to determine if tests should be executed. Exit code 0 enables tests, non-zero skips them. The skip script can output a message (stdout or stderr) explaining why tests are skipped, displayed in verbose mode. The prep command runs once before all tests begin and waits for completion. The setup command starts a background service that runs during test execution. The delay field specifies milliseconds to wait after the setup service starts before beginning test execution, allowing time for service initialization. The cleanup command runs after all tests complete to clean up resources.

  Environment Variables
Configure environment variables available to all tests during execution:
{
    env: {
        BIN: "${../build/*/bin}",           // Expands to build directory
        LIB_PATH: "${../lib}",              // Points to library directory
        TEST_DATA: "${./test-data/*.json}", // Expands to test data files
        API_URL: "http://localhost:8080"    // Static values work too
    }
}
Environment variable values support ${...} expansion using glob patterns. Paths are resolved relative to the configuration file's directory. This is useful for providing dynamic paths to build artifacts, libraries, and test data.

  Special Variables
TestMe provides special variables that can be used in compiler flags, library paths, and environment variables:

${TESTDIR}
    Relative path from compiled executable to test file directory (e.g., "../..")

${CONFIGDIR}
    Relative path from compiled executable to testme.json5 directory

${OS}
    Operating system: "macosx", "linux", "windows"

${ARCH}
    CPU architecture: "arm64", "x64", "x86"

${PLATFORM}
    Combined OS-ARCH: "macosx-arm64", "linux-x64", "windows-x64"

${CC}
    Compiler name: "gcc", "clang", "msvc"

${PROFILE}
    Build profile from --profile option, config file, PROFILE environment variable, or default "dev" Example usage in compiler configuration:
    {
        profile: "dev",  // Can be overridden by --profile or env.PROFILE
        compiler: {
            c: {
                clang: {
                    flags: [
                        "-I${CONFIGDIR}/../build/${PLATFORM}-${PROFILE}/inc",
                        "-L${CONFIGDIR}/../build/${PLATFORM}-${PROFILE}/bin",
                        "-Wl,-rpath,@executable_path/${CONFIGDIR}/../build/${PLATFORM}-${PROFILE}/bin"
                    ]
                }
            }
        }
    }
    These variables ensure correct paths regardless of test nesting level and support platform-specific build configurations. Tests can access these variables using standard environment variable mechanisms:

•   C tests: getenv("BIN")

•   Shell tests: $BIN or ${BIN}

•   JavaScript/TypeScript: process.env.BIN Environment variables are automatically included in Xcode debugging projects when using --debug mode.

ARTIFACTS
C tests create build artifacts in .testme directories co-located with test files:

Compiled binaries
    C source files are compiled to executables with names matching the test base name.

Compilation logs
    compile.log files contain compiler output for debugging compilation issues.

Debug symbols
    Debug builds include .dSYM directories on macOS for debugging support.

Xcode projects
    Debug mode creates Xcode project files for integrated debugging on macOS.

PARALLEL EXECUTION
TestMe executes tests in parallel by default with configurable concurrency:

Batched processing
    Tests are processed in batches to prevent system overload.

Fresh handler instances
    Each test gets isolated handler instances to prevent race conditions.

Artifact isolation
    Each test compiles in its own directory to avoid conflicts.

Configurable concurrency
    Use workers setting to tune based on system resources.

OUTPUT MODES
TestMe provides three levels of output verbosity:

Default Mode
    Shows test names as they execute with pass/fail status and execution time, followed by a summary.

Verbose Mode (--verbose)
    Includes all default output plus detailed error information, compilation commands, and sets TESTME_VERBOSE=1 for tests.

Quiet Mode (--quiet)
    Produces no output at all, only returns exit codes. Ideal for scripts and automated systems.

ENVIRONMENT VARIABLES
TestMe sets and respects several environment variables:

TESTME_VERBOSE
    Set to "1" when verbose mode is enabled. Tests can check this for detailed output.

TESTME_DEPTH
    Set to the value provided by --depth option. Used for nested test execution control.

PROFILE
    Read as the default build profile if not specified in config or via --profile. Used in ${PROFILE} variable expansion.

TMPDIR
    Set to /tmp/claude/ for temporary file operations in sandboxed environments.

EXIT STATUS

0   All tests passed successfully.

1   One or more tests failed, had errors, or compilation failed.

2   Invalid command line arguments or configuration errors.

EXAMPLES

  Getting Started

Create testme.json5 configuration file:
    tm --init

Create a C test file from template:
    tm --new math.c

Create a JavaScript test file:
    tm --new api.js

Create a Shell test file:
    tm --new test.sh

  Running Tests

Run all tests (shows test names as they execute):
    tm

Run only C tests:
    tm *.tst.c

Run specific test file:
    tm math.tst.c

Run tests matching pattern:
    tm **/math*

List all discoverable tests:
    tm --list

Clean all test artifacts:
    tm --clean

Run with verbose output:
    tm -v integration*

Keep build artifacts for debugging:
    tm --keep *.tst.c

Run tests one at a time with prompts:
    tm --step

Set custom test depth:
    tm --depth 5

Debug a specific C test:
    tm --debug math.tst.c

Show compilation commands:
    tm --show *.tst.c

Run silently with no output (for scripts):
    tm --quiet

Use custom configuration:
    tm -c /path/to/testme.json5

Change directory before running:
    tm --chdir /path/to/tests

Run with specific build profile:
    tm --profile release

Run with profile from environment:
    PROFILE=prod tm

FILES

testme.json5
    Configuration file searched from current directory upward.

.testme/
    Artifact directories created alongside test files for build outputs.

*.tst.sh, *.tst.c, *.tst.js, *.tst.ts, *.tst.es
    Test files with recognized extensions.

testme.h
    C testing utility header file with assertion and helper functions.

testme.js
    JavaScript/TypeScript testing utility module with assertion and helper functions.

DEBUGGING
For C tests, TestMe provides integrated debugging support:

  macOS (Xcode)
Use --debug to create and open an Xcode project with proper build settings, include paths, and library linking. The project includes: - All compiler flags from configuration - Expanded library and include paths - Direct dylib linking for runtime libraries - Proper rpath configuration

  Linux (GDB)
Use --debug to launch GDB with the compiled test binary. Provides command-line debugging with full symbol information.

TROUBLESHOOTING

Compilation failures
    Use --show to see exact compiler commands and --keep to examine compilation logs in .testme directories.

Library linking issues
    Check library paths in configuration and ensure dylib files exist in specified locations.

Parallel execution issues
    Reduce workers setting if tests fail due to resource contention.

Permission errors
    Ensure test files are readable and script files are executable.

SEE ALSO
gcc(1), clang(1), bun(1), gdb(1), xcodegen(1)

AUTHOR
TestMe was written for embedded development environments requiring multi-language test execution with consistent tooling across C, shell scripts, JavaScript, and TypeScript.

COPYRIGHT
This is free software; see the source for copying conditions.