Lotte
\n\nLotte is a headless, automated testing framework built on top of PhantomJS and inspired by Ghostbuster.\nIt adds jQuery-like methods and chaining, more assertion logic and an extensible core.\nTests can be written in either JavaScript or CoffeeScript.
\n\nLotte comes with tools for accessing the DOM, evaluating arbitrary code, simulating mouse and keyboard input.
\n\nTests are sandboxed and run asynchronously. Blocking methods are available to simulate dependencies and to control the flow of execution.
\n\nThis project is still highly experimental. Using it may cause your computer to blow up. Seriously.
\n\nPrerequisites
\n\n\n\nOptional Dependencies
\n\n- \n
- CoffeeScript \n
Installation
\n$ npm -g install lotte\n\n-global is preferred so you can run lotte from any directory.
Usage
\n\nCreate a new file lotte_github.js (preferably in an empty directory) and copy the following code:
this.open('https://github.com', function() {\n this.describe('Sign Up button', function() {\n this.assert.ok(this.$('.signup-button').length, 'expects button to be in the DOM');\n this.success();\n });\n});\n\nRun lotte from within the directory and you should see the following output:
/tmp/lotte_github.js\n @ https://github.com\n ✓ Sign Up button\n\nCommand-Line Options
\n\nYou can customise many aspects of Lotte's behaviour either on the command-line on through Lottefiles. The following options are available:
$ lotte --help\nUsage: lotte [OPTION...] PATH\n\nOptions:\n --help, -h give this help page\n --version, -v print program version\n --concurrent, -c limit of files to run in parallel [default: 4]\n --timeout, -t timeout for individual files (in milliseconds) [default: 30000]\n --include, -I glob pattern to match files in PATH [string] [default: "**/lotte_*.js"]\n --exclude, -E glob pattern to remove included files [string]\n --lottefile, -f look for 'lottefile' in PATH [string] [default: "Lottefile"]\n --verify verify PhantomJS version (expected ~1.3.0) [boolean] [default: true]\n --phantom executable for PhantomJS [string] [default: "phantomjs"]\n --coffee executable for CofeeScript [string] [default: "coffee"]\n\nThere are four key options you would want to customise while the rest should work with their defaults.
\n\n- \n
\n\n--concurrent, -cIf you have more than one test file in a directory, Lotte will attempt to run them in parallel (asynchronously).\nYou can specify how many tests can be running at any given time through this option.
\n\nIf you want to run tests synchronously, specify a value of
1. \n
\n\n--timeout, -tEach test is expected to finish within a given period of time. If a test takes longer, it is interruped and recorded as failed.
\n\nThe default value is
30seconds, but you should consider reducing it. \n
\n\n--include, -I
\n--exclude, -EWhen you run
\n\nlottefrom any directory the script collects a list of all files in the current directory and all sub-directories.\nThe list is reduced by running theincludeglob pattern and dropping any files that did not match.\nThe list is then reduced further by running theexcludeglob pattern and dropping any files that did match.\nThe remaining list is sorted and considered final.You can specify these arguments more than once to create an array of include/exclude patterns.
\n
Lottefile
\n\nIn order to avoid having to type the full lotte command-line each time, you can use Lottefiles to store your settings per project.
Lottefiles are regular JavaScript files where each global variable maps to a command-line option. For example, the following command:
$ lotte --include '**/*.coffee' --include '**/*.js' --concurrent 1 tests\n\ncan be stored in a Lottefile as this:
path = 'tests'\ninclude = ['**/*.coffee', '**/*.js']\nconcurrent = 1\n\nRunning lotte from the project directory will then read the Lottefile and scan the tests directory for all files matching **/*.{coffee,js}.
Writing Tests
\n\nTests can be written in either JavaScript or CoffeeScript.\nIn the sections below substitute @ with this. if you are using JavaScript.\nArguments wrapped in square brackets [ and ] are optional.\nArguments ending in ... can be used more than once.
At the top-level, the following functions are available:
\n\n- \n
\n\n@title([name])Gets or sets the test title.\nThis is useful for giving meaningful names to your tests.
\n\nWhen called with zero arguments, returns the current title or
\n\nundefined.
\nWhen called with one argument, sets the title.If you don't explicitly specify a title, the filename will be used instead.
\n
\n\n@base([uri])Gets or sets the absolute URI for all relative URIs in the test.\nYou can use this to specify the root URI for your project.
\n\nWhen called with zero arguments, returns the current URI or
\n\nundefined.
\nWhen called with one argument, sets the URI.If you don't explicitly specify an absolute URI, all calls to
@openwill expect an absolute URI instead. \n
\n\n@open(uri, [message], [options], block)Creates a new test.
\n\n
\n\nurican be either an absolute or relative URI (see above).
\nmessageis an optional description for the URI. If you don't specify it, Lotte will print theuriin the output instead.
\noptionsis an object hash to pass to PhantomJS. See settings (object).
\nblockis a function which is executed if the server returns a valid response (2xx or 3xx).If the server returns a 4xx or 5xx HTTP code instead, the test is recorded as failed.
\n\nIf you have more than one
@opencall at the top-level, they will be executed asynchronously. \n
Putting it all together, a test file could look like this:
\n@title 'Github'\n@base 'https://github.com'\n\n@open '/', 'the homepage', ->\n # ...body of test...\n\nCases & Grouping
\n\nOnce you have successfully requested an URI, you can start writing test cases against the page.
\n\nThe following functions are available:
\n\n- \n
\n\n@group(name, block)Groups the nested test cases. This is mainly for structuring the output Lotte prints.
\n\nnameis the name of the group.
\nblockis a function which contains the nested test cases. \n
\n\n@describe(name, block)Starts a new test case.
\n\nnameis the name of the test case.
\nblockis a function which is executed and is expected to contain assertion logic. \n
Putting it all together a test file could now look like this:
\n@title 'Github'\n@base 'https://github.com'\n\n@open '/', 'the homepage', ->\n @describe 'counter shows number of repositories', ->\n # ...assertion logic...\n @group 'Sign Up button', ->\n @describe 'is in place', ->\n # ...assertion logic...\n @describe 'takes you to /plans', ->\n # ...assertion logic...\n\nFlow of Execution
\n\nEach test case is executed in the order in which it is defined:
\n@describe 'I run first', ->\n@describe 'I run second', ->\n# etc.\n\nIf a test case contains an asynchronous function call, the next test case is executed without waiting for the function to finish:
\n@describe 'I run first', ->\n@describe 'I run second', ->\n setTimeout ( -> ), 2500\n@describe 'I run third in parallel with second still running', ->\n\nBe extremely careful when dealing with asynchronous function. For example, using .click() to follow an anchor could change the page while another test case is running.
If a test case fails, any remaining test cases are skipped:
\n@describe 'I run first', ->\n throw 'Whoops!'\n@describe 'I should run second, but I never will', ->\n\nTo simulate dependencies and control the flow of execution, you can use the following functions:
\n\n- \n
\n\n@wait(name..., block)Blocks the current test case until all dependencies have finished (either passed or failed).
\n
The earlier example can now be rewritten as follows to make it synchronous again:
\n@describe 'I run first', ->\n@describe 'I run second', ->\n setTimeout ( -> ), 2500\n@describe 'I run third', ->\n @wait 'I run second', ->\n # ...assertion logic...\n\nEnvironments
\n\nLotte uses PhantomJS to execute tests. While you may be writing tests in JavaScript and expect to be able to access the DOM of a page directly, this is not the case.
\n\nEach test file runs in its own sandboxed environment. Each page you request also runs in a sandbox.\nYou cannot access variables across environments, i.e., you cannot define a variable in your test file and access it within the page you have just requested:
\n@open 'https://github.com', ->\n @describe 'Sandbox', ->\n val = 'value'\n # following line throws an exception\n console.log(@page.evaluate( -> return val))\n throw 'exit'\n\nIn the above code snippet @page.evaluate runs the function as if it were defined on the page you just requested, i.e, github.com.\nIn order to do so, PhantomJS serializes the function, but it does not include the context in which it was defined.\nWhen the function is executed, val is missing in the new context causing it to throw an exception.
Another limitation of PhantomJS is the fact you cannot return complex types from the page.\nObjects are serialized before they leave the page sandbox and unserialized back in the parent (test case) environment:
\n@open 'https://github.com', ->\n @describe 'serialize/unserialize', ->\n h1 = @page.evaluate( -> return document.querySelector('h1'))\n # prints 'H1' correctly\n console.log h1.tagName\n # prints 'undefined' as functions cannot be serialized/unserialized\n console.log h1.focus\n throw 'exit'\n\nLotte comes with a workaround which allows you to pass variables to the PhantomJS environment.\nThis hack has the same limitations as outlined above (it uses JSON.stringify internally):
- \n
\n\n@using(hash, block)
\n\nhashis akey: valueobject where:- \n
keymust be a valid identifier and defines the name of the variable within the PhantomJS environment \nvaluemust be serializable and contains the value of the variable \n
\n\nblockis either a function or a string.Returns a
Functionstring with allkey: valuepairs fromhashas arguments. \n
@open 'https://github.com', ->\n @describe '@using(..)', ->\n expected = 'git'\n @$('h2').first @using { expected }, (element) -> element.innerHTML.indexOf(expected) is 0\n throw 'exit'\n\nDocument Queries
\n\nThere is a lot of boilerplate code required to access the DOM of a page.\nLotte comes with a jQuery-like query function to abstract some of the most common operations:
\n\n- \n
\n\n@$(selector)
\n\nselectoris a string containing a selector expression.Returns a
DocumentQueryobject. \n
The earlier example can now be rewritten as follows:
\n@open 'https://github.com', ->\n @describe 'Document Queries', ->\n # prints 'H1' correctly\n console.log @$('h1').tagName\n # prints 'undefined'\n console.log @$('h1').focus\n throw 'exit'\n\nAdditional Methods
\n\nA DocumentQuery object has the following methods to deal with the DOM:
- \n
\n\nDocumentQuery.prototype.attr([index], property)Gets the value of
\n\npropertyfor the element atindex.The following code snippets are equivalent:
\n\n\n@$('h1').attr('tagName')\n@$('h1').tagName\n
\nThe direct property access always returns the property value for the first matched element.
\n\nWhile you can use
attr(..)to access any property, the direct access will only work with the following pre-defined list of properties:action,alt,checked,className,clientHeight,clientLeft,clientTop,clientWidth,disabled,enctype,height,href,id,innerHTML,length,maxLength,media,method,name,nodeName,nodeValue,offsetHeight,offsetLeft,offsetTop,offsetWidth,options,outerHTML,outerText,readOnly,rel,scrollHeight,scrollLeft,scrollTop,scrollWidth,selectedIndex,size,src,style,tagName,target,textContent,title,type,value,width. \n
\n\nDocumentQuery.prototype.click([index], [message], [block])Clicks on the element at
\n\nindexand returns execution to the test case (i.e., asynchronous). This method can be used to simulate mouse input.
\nindexis the position of the element in the matched selector collection. If you omit this argument, the first element is assumed.
\nmessageis an optional message to display if something goes wrong, i.e., if no elements were matched.
\nblockis an optional function to execute when the new page has loaded, i.e., when you follow an anchor.@open 'https://github.com', ->\n @describe 'click(..)', ->\n # prints https://github.com/\n console.log('I am now on: ' + @page.evaluate( -> location.href))\n @$('.signup-button').click ->\n # prints https://github.com/plans\n console.log('I am now on: ' + @page.evaluate( -> location.href))\n throw 'exit'\n
\n \n
\n\nDocumentQuery.prototype.input(value, [index], [message])Inputs
\n\nvaluein the element atindex. This method can be used to simulate keyboard input.
\nvalueis a string to input as if it were keyboard input.
\nindexis the position of the element in the matched selector collection. If you omit this argument, the first element is assumed.
\nmessageis an optional message to display if something goes wrong, i.e., if no elements were matched.@open 'http://www.google.com', ->\n @describe 'input(..)', ->\n @$('[name="q"]').input 'meaning of life'\n @$('input[type="submit"]').click ->\n console.log @$('#res').attr('innerText')\n throw 'exit'\n
\n \n
See DOM Assertions below for additional methods.
\n\nAssertions
\n\nLotte comes with two types of assertion logic:
\n\n- \n
- Generic assertions \n
- DOM assertions \n
Generic Assertions
\n\nIf you have used Node's built-in assert module, these functions will be familiar:
\n\n- \n
\n\n@assert.fail(actual, expected, message, operator)Throws an exception that displays the values for
actualandexpectedseparated by the providedoperator. \n
\n\n@assert.ok(value, message)Tests if
valueis a true value, it is equivalent to@assert.equal(true, value, message). \n
\n\n@assert.equal(actual, expected, message)Tests shallow, coercive equality with the equal comparison operator
==. \n
\n\n@assert.notEqual(actual, expected, message)Tests shallow, coercive non-equality with the not equal comparison operator
!=. \n
\n\n@assert.deepEqual(actual, expected, message)Tests for deep equality.
\n
\n\n@assert.notDeepEqual(actual, expected, message)Tests for any deep inequality.
\n
\n\n@assert.strictEqual(actual, expected, message)Tests strict equality, as determined by the strict equality operator
===. \n
\n\n@assert.notStrictEqua(actual, expected, message)Tests strict non-equality, as determined by the strict not equal operator
!==. \n
\n\n@assert.throws(block, error, message)Expects
blockto throw an error.errorcan be constructor, RegExp or validation function. \n
\n\n@assert.doesNotThrow(block, error, message)Expects
blocknot to throw an error, see@assert.throwsfor details. \n
\n\n@assert.contains(actual, expected, message)Expects
actualto containexpected.expectedcan be a string or a RegExp. \n
DOM Assertions
\n\nDocumentQuery (see above) comes with additional methods to deal with assertions:
- \n
\n\nDocumentQuery.prototype.contains([message], pattern)Expects at least one of the matched elements to contain
pattern.patterncan be a string or a RegExp. \n
\n\nDocumentQuery.prototype.each([message], block)Tests if calling
blockon each matched element as an argument returns a true value. \n
\n\nDocumentQuery.prototype.first([message], block)Tests if calling
blockwith the first matched element as an argument returns a true value. \n
\n\nDocumentQuery.prototype.last([message], block)Tests if calling
blockwith the last matched element as an argument returns a true value. \n
\n\nDocumentQuery.prototype.nth(index, [message], block)Tests if calling
blockwith the element atindexas an argument returns a true value. \n
A general note which applies to all functions above: you cannot access variables from scope within block (see Environments).
An example test file that performs DOM assertions could look like this:
\n@open 'https://github.com', ->\n @describe 'Navigation and children have classes', ->\n @$('.nav').first (element) -> element.classList.contains('logged_out')\n @$('.nav li').each (element) -> !! element.className\n throw 'exit'\n\nPassing Tests
\n\nSo far we have ended tests with throw 'exit'. To pass a test case, use the following functions:
- \n
\n\n@success()Marks the test case as passed. You must call this once within each
@describeblock. \n
If you don't end a test case either by failing any of the asserts or calling @success(), the entire test file will hang until timeout is reached at which point it is recorded as failed.
FAQs
\n\n- \n
Q: How can I log in before requesting a page?
\n\nA: You can nest
\n@open(..)functions to achieve this:@base 'http://local.dev'\n@open '/login', ->\n @describe 'Log in', ->\n @$('.username').input 'admin'\n @$('.password').input 'password'\n @$('input[type="submit"]').click ->\n @success()\n @open '/account', ->\n @describe 'My Account', ->\n # ...assertion logic...\n @success()\n
\n \nQ: Can I pass arguments to my tests?
\n\nA: Yes. Use
\n--on the command-line followed by the arguments:\n\n$ lotte -- arg1 arg2\n
\nArguments will be available in your test files as
phantom.args[..]. \nQ: How can I see what PhantomJS 'sees'?
\n\nA: Within the context of a test case (
\n@describe), you can refer to@pagewhich is the PhantomJS WebPage object.@describe 'snapshot', ->\n @page.render 'home.png'\n @success()\n
\n \nQ: I don't have
\n\nphantomjsand/orcoffeeon$PATH.A: See
lotte --helpfor information on how to specify a path to the missing binary. \n
Contributing
\n\nThe goal of this project is to provide an awesome tool for developers to test their websites or apps in their favourite language with minimum effort.
\n\nCommit and code reviews, ideas and documentation improvements are welcomed.
\n\nChangelog
\n\n0.2
\n\n- \n
- 74aee9c - Drop 'findit' and use 'walkdir', support Windows. \n
- 4ab5679 - Bump dependencies versions in package.json and address deprecations. \n
- 8a49077 - Address deprecation “
path.existsis now calledfs.exists.” \n - 1189ae6 - Add (verified) support for PhantomJS 1.6.x. \n
- 4c9d441 - Emit 'compile' to allow hooks to modify the code before it's compiled. \n
Copyright
\n\n\n\n\nCopyright © 2011 Stan Angeloff. See LICENSE.md for details.
\n