➔ tox
python installed: EasyProcess==0.2.3,html5lib==0.999999999,py==1.4.34,pytest==3.1.3,PyVirtualDisplay==0.2.1,selenium==3.4.3,six==1.10.0,webencodings==0.5.1
python runtests: PYTHONHASHSEED='1297519254'
python runtests: commands[0] | pytest -vv tests
===================================================================== test session starts =====================================================================
platform darwin -- Python 2.7.13, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 -- /Users/maj/src/web-platform-tests/resources/test/.tox/python/bin/python2.7
cachedir: .cache
rootdir: /Users/maj/src/web-platform-tests/resources/test, inifile:
collected 36 items
::Test#add_cleanup <- . PASSED
::Test#add_cleanup reported count <- . PASSED
::Test#add_cleanup: error <- . PASSED
::Test#add_cleanup: multiple functions with one in error <- . PASSED
::Sample HTML5 API Tests <- . PASSED
::Sample HTML5 API Tests <- . PASSED
::Sample HTML5 API Tests <- . PASSED
::Sample for using generate_tests to create a series of tests that share the same callback. <- . PASSED
::Example with iframe that notifies containing document via callbacks <- . PASSED
::Example with iframe that consolidates errors via fetch_tests_from_window <- . PASSED
::Example with iframe that consolidates tests via fetch_tests_from_window <- . FAILED
::Example with iframe that notifies containing document via cross document messaging <- . PASSED
::Ordering <- . PASSED
::Async Tests and Promises <- . FAILED
::Promise Tests <- . FAILED
::Example with file_is_test (should fail) <- . PASSED
::Example single page test with no asserts <- . PASSED
::Example single page test with no body <- . PASSED
::Example with file_is_test <- . PASSED
::Harness Handling Uncaught Exception <- . PASSED
::Harness Ignoring Uncaught Exception <- . PASSED
::Dedicated Worker Tests <- . PASSED
::Example with a service worker <- . PASSED
::Example with a shared worker <- . PASSED
::idlharness basic <- . PASSED
::IdlArray.prototype.is_json_type() <- . FAILED
::IdlDictionary.prototype.get_inheritance_stack() <- . PASSED
::IdlDictionary.prototype.default_to_json_operation() <- . PASSED
::IdlInterface.prototype.get_inheritance_stack() <- . PASSED
::IdlInterface.prototype.has_default_to_json_regular_operation() <- . PASSED
::IdlInterface.prototype.has_to_json_regular_operation() <- . PASSED
::idlharness: Immutable prototypes <- . PASSED
::idlharness: Primary interface <- . PASSED
::IdlInterface.prototype.test_to_json_operation() <- . PASSED
::IdlDictionary.prototype.traverse_inherited_and_consequential_interfaces() <- . PASSED
::IdlInterfaceMember.prototype.is_to_json_regular_operation() <- . PASSED
========================================================================== FAILURES ===========================================================================
____________________________________ /Users/maj/src/web-platform-tests/resources/test/tests/iframe-consolidate-tests.html _____________________________________
conftest.py:83: in runtest
assert summarized == self.expected
E AssertionError: assert {'summarized_...': 'complete'} == {'summarized_s...': 'complete'}
E Common items:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'type': u'complete'}
E Differing items:
E {'summarized_tests': [{'message': None, 'name': 'Promise rejection', 'properties': {}, 'stack': None, ...}, {'message'... None, ...}, {'message': None, 'name': 'Test executing in parent context', 'properties': {}, 'stack': None, ...}, ...]} != {'summarized_tests': [{'message': None, 'name': 'Promise rejection', 'properties': {}, 'stack': None, ...}, {'message'... None, ...}, {'message': None, 'name': 'Test executing in parent context', 'properties': {}, 'stack': None, ...}, ...]}
E Full diff:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'summarized_tests': [{u'message': None,
E u'name': u'Promise rejection',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promise resolution',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': u'assert_true: This failure is expected expected true got false',
E u'name': u'Promises and test assertion failures (should fail)',
E u'properties': {},
E u'stack': u'(implementation-defined)',
E u'status_string': u'FAIL'},
E {u'message': None,
E u'name': u'Promises are supported in your browser',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promises resolution chaining',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Test executing in parent context',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Use of step_func with Promises',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': u'assert_unreached: This failure is expected Reached unreachable code',
E u'name': u'Use of unreached_func with Promises (should fail)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'}],
E u'type': u'complete'}
__________________________________________ /Users/maj/src/web-platform-tests/resources/test/tests/promise-async.html __________________________________________
conftest.py:83: in runtest
assert summarized == self.expected
E AssertionError: assert {'summarized_...': 'complete'} == {'summarized_s...': 'complete'}
E Common items:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'type': u'complete'}
E Differing items:
E {'summarized_tests': [{'message': None, 'name': 'Promise rejection', 'properties': {}, 'stack': None, ...}, {'message'...': None, ...}, {'message': None, 'name': 'Use of step_func with Promises', 'properties': {}, 'stack': None, ...}, ...]} != {'summarized_tests': [{'message': None, 'name': 'Promise rejection', 'properties': {}, 'stack': None, ...}, {'message'...': None, ...}, {'message': None, 'name': 'Use of step_func with Promises', 'properties': {}, 'stack': None, ...}, ...]}
E Full diff:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'summarized_tests': [{u'message': None,
E u'name': u'Promise rejection',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promise resolution',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': u'assert_true: This failure is expected expected true got false',
E u'name': u'Promises and test assertion failures (should fail)',
E u'properties': {},
E u'stack': u'(implementation-defined)',
E u'status_string': u'FAIL'},
E {u'message': None,
E u'name': u'Promises are supported in your browser',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promises resolution chaining',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Use of step_func with Promises',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': u'assert_unreached: This failure is expected Reached unreachable code',
E u'name': u'Use of unreached_func with Promises (should fail)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'}],
E u'type': u'complete'}
_____________________________________________ /Users/maj/src/web-platform-tests/resources/test/tests/promise.html _____________________________________________
conftest.py:83: in runtest
assert summarized == self.expected
E assert {'summarized_...': 'complete'} == {'summarized_s...': 'complete'}
E Common items:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'type': u'complete'}
E Differing items:
E {'summarized_tests': [{'message': 'assert_true: Expected failure. expected true got false', 'name': 'Assertion failure... None, 'name': "promise_test with function that doesn't return a Promise", 'properties': {}, 'stack': None, ...}, ...]} != {'summarized_tests': [{'message': 'assert_true: Expected failure. expected true got false', 'name': 'Assertion failure... None, 'name': "promise_test with function that doesn't return a Promise", 'properties': {}, 'stack': None, ...}, ...]}
E Full diff:
E {u'summarized_status': {u'message': None,
E u'stack': None,
E u'status_string': u'OK'},
E u'summarized_tests': [{u'message': u'assert_true: Expected failure. expected true got false',
E u'name': u'Assertion failure in a fulfill reaction (should FAIL with an expected failure)',
E u'properties': {},
E u'stack': u'(implementation-defined)',
E u'status_string': u'FAIL'},
E {u'message': None,
E u'name': u'Chain of promise resolutions',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promise fulfillment with result',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promise rejection with result',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u'Promises are supported in your browser',
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': None,
E u'name': u"promise_test with function that doesn't return a Promise",
E u'properties': {},
E u'stack': None,
E u'status_string': u'PASS'},
E {u'message': u'assert_not_equals: got disallowed value undefined',
E u'name': u"promise_test with function that doesn't return anything",
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'},
E {u'message': u'promise_test: Unhandled rejection with value: object "Error: Expected exception."',
E u'name': u'promise_test with unhandled exception in fulfill reaction (should FAIL)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'},
E {u'message': u'promise_test: Unhandled rejection with value: object "Error: Expected exception."',
E u'name': u'promise_test with unhandled exception in reject reaction (should FAIL)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'},
E {u'message': u'promise_test: Unhandled rejection with value: "Expected rejection"',
E u'name': u'promise_test with unhandled rejection (should FAIL)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'},
E {u'message': u'assert_unreached: Expected failure. Reached unreachable code',
E u'name': u'unreached_func as reactor (should FAIL with an expected failure)',
E u'properties': {},
E - u'stack': None,
E ? ^
E + u'stack': u'(implementation-defined)',
E ? ^^^^^^^^^^^^^^^ ++ +++++++
E u'status_string': u'FAIL'}],
E u'type': u'complete'}
________________________________ /Users/maj/src/web-platform-tests/resources/test/tests/idlharness/IdlArray/is_json_type.html _________________________________
conftest.py:81: in runtest
assert test[u'status_string'] == u'PASS', msg
E AssertionError: should return true for interfaces which mixin an interface which declare a toJSON operation
E this is undefined:
E (implementation-defined)
E assert 'FAIL' == 'PASS'
E - FAIL
E + PASS
============================================================ 4 failed, 32 passed in 56.85 seconds =============================================================
ERROR: InvocationError: '/Users/maj/src/web-platform-tests/resources/test/.tox/python/bin/pytest -vv tests'
___________________________________________________________________________ summary ___________________________________________________________________________
ERROR: python: commands failed