Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
JavaScript test-runners benchmark
The benchmark for running the same tests on the most popular JavaScript test-runners and measuring execution time. Tests are divided onto unit and functional.
Unit tests
Runners
Benchmark conditions
This benchmark measures execution time for every combination of following conditions:
1. Test types
- Synchronous empty test:
function () {}
- Synchronous heavy test:
function () { for (let i = 0; i < 10000; i++) { new Date(); } }
- Asynchronous empty test with zero delay:
function (done) { setTimeout(done, 0); }
- Asynchronous empty test with random delay:
function (done) { setTimeout(done, Math.round(Math.random() * 10)); }
2. Tests structure
- With nested suites
- Without nested suites
3. Run types
- With Babel transpiling
- Without Babel transpiling
Latest results
https://vitalets.github.io/js-testrunners-bench/index.html
Run yourself
-
Clone the repo:
git clone https://github.com/vitalets/js-testrunners-bench.git
-
Install dependencies:
cd js-testrunners-bench npm install -
Generate tests:
node unit gen
After run check that
/testsdirectory is created and filled with test-files. -
Run benchmark:
node unit run [testsType] [runType]
Examples:
# Synchronous empty tests without nested suites and without Babel node unit run test=syncEmptyFn_nestedSuites=false babel=false # Asynchronous tests with random delay 0-10ms with nested suites and Babel node unit run test=asyncEmptyFnRandomDelay_nestedSuites=true babel=true etc..
Example output:
> node unit run test=syncEmptyFn_nestedSuites=false babel=false
JavaScript test-runners benchmark
System: darwin x64 4 cpu(s) node v7.2.0
Date: Wed Jul 26 2017
RUNNER VERSION
mocha 3.4.2
jasmine 2.6.0
mocha.parallel 0.15.2
mocha-parallel-tests 1.2.9
qunit 2.3.3
tape 4.6.3
tap 10.3.3
lab 13.1.0
ava 0.19.1
jest 20.0.4
Bench type: test=syncEmptyFn, nestedSuites=false, babel=false
Tests count: 250 (50 files)
Running: mocha, cmd: mocha tests/unit/test=syncEmptyFn/nestedSuites=false/mocha
Running: jasmine, cmd: jasmine JASMINE_CONFIG_PATH=temp/jasmine.json
Running: mocha.parallel, cmd: mocha tests/unit/test=syncEmptyFn/nestedSuites=false/mocha.parallel
Running: mocha-parallel-tests, cmd: mocha-parallel-tests tests/unit/test=syncEmptyFn/nestedSuites=false/mocha-parallel-tests
Running: qunit, cmd: qunit tests/unit/test=syncEmptyFn/nestedSuites=false/qunit
Running: tape, cmd: tape tests/unit/test=syncEmptyFn/nestedSuites=false/tape/*.js
Running: tap, cmd: tap tests/unit/test=syncEmptyFn/nestedSuites=false/tap --jobs-auto
Running: lab, cmd: lab --parallel tests/unit/test=syncEmptyFn/nestedSuites=false/lab
Running: ava, cmd: ava tests/unit/test=syncEmptyFn/nestedSuites=false/ava --concurrency=4
Running: jest (jsdom), cmd: jest --env=jsdom tests/unit/test=syncEmptyFn/nestedSuites=false/jest
Running: jest (node), cmd: jest --env=node tests/unit/test=syncEmptyFn/nestedSuites=false/jest
Result:
LABEL TIME
jasmine 0.205
tape 0.273
qunit 0.332
mocha 0.346
mocha.parallel 0.420
lab 0.429
mocha-parallel-tests 0.471
jest (node) 1.84
jest (jsdom) 3.78
tap 6.32
ava 8.34
Done.
Functional tests
Not ready yet.
Runners
- todo Webdriverio
- todo Mocha + selenium-webdriver
- todo CucumberJS
- todo Nightwatch
- todo TestCafe
- todo Nemo
Related links
- JavaScript Test-Runners Benchmark (Part 1, The Unit Testing)
- JavaScript unit testing tools
- An Overview of JavaScript Testing in 2017
- Picking Jest over Mocha – testing tools comparison
License
MIT @ Vitaliy Potapov