web-platform-tests — Firefox Source Docs documentation

<!doctype html> <html class="writer-html5" lang="en" data-content_root="../"> <head> <base href="https://firefox-source-docs.mozilla.org/web-platform/index.html"> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>web-platform-tests — Firefox Source Docs documentation</title> <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50"> <link rel="stylesheet" type="text/css" href="../_static/css/theme.css?v=19f00094"> <link rel="stylesheet" type="text/css" href="../_static/graphviz.css?v=fd3f3429"> <link rel="stylesheet" type="text/css" href="../_static/copybutton.css?v=76b2166b"> <link rel="stylesheet" type="text/css" href="../_static/custom_theme.css?v=dc5ed815"> <link rel="stylesheet" type="text/css" href="../_static/design-style.1e8bd061cd6da7fc9cf755528e8ffc24.min.css?v=0a3b3ea7"> <link rel="shortcut icon" href="../_static/firefox.ico"> <script src="../_static/jquery.js?v=5d32c60e"></script> <script src="../_static/_sphinx_javascript_frameworks_compat.js?v=2cd50e6c"></script> <script src="../_static/documentation_options.js?v=5929fcd5"></script> <script src="../_static/doctools.js?v=9a2dae69"></script> <script src="../_static/sphinx_highlight.js?v=dc90522c"></script> <script src="../_static/clipboard.min.js?v=a7894cd8"></script> <script src="../_static/copybutton.js?v=30646c52"></script> <script src="../_static/design-tabs.js?v=36754332"></script> <script src="../_static/js/theme.js"></script> <link rel="index" title="Index" href="../genindex.html"> <link rel="search" title="Search" href="../search.html"> <link rel="next" title="GTest" href="../gtest/index.html"> <link rel="prev" title="TPS" href="../testing/tps/index.html"> <meta http-equiv="X-Translated-By" content="Google"> <meta http-equiv="X-Translated-To" content="iw"> <script type="text/javascript" src="https://www.gstatic.com/_/translate_http/_/js/k=translate_http.tr.en_GB.tKc6KWkFf-8.O/am=gAE/d=1/rs=AN8SPfrf36LIV3DkhtRBGWFnLWWzaykPyw/m=corsproxy" data-sourceurl="https://firefox-source-docs.mozilla.org/web-platform/index.html"></script> <link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20..48,100..700,0..1,-50..200" rel="stylesheet"> <script type="text/javascript" src="https://www.gstatic.com/_/translate_http/_/js/k=translate_http.tr.en_GB.tKc6KWkFf-8.O/am=gAE/d=1/exm=corsproxy/ed=1/rs=AN8SPfrf36LIV3DkhtRBGWFnLWWzaykPyw/m=phishing_protection" data-phishing-protection-enabled="false" data-forms-warning-enabled="true" data-source-url="https://firefox-source-docs.mozilla.org/web-platform/index.html"></script> <meta name="robots" content="none"> </head> <body class="wy-body-for-nav"> <script type="text/javascript" src="https://www.gstatic.com/_/translate_http/_/js/k=translate_http.tr.en_GB.tKc6KWkFf-8.O/am=gAE/d=1/exm=corsproxy,phishing_protection/ed=1/rs=AN8SPfrf36LIV3DkhtRBGWFnLWWzaykPyw/m=navigationui" data-environment="prod" data-proxy-url="https://firefox--source--docs-mozilla-org.translate.goog" data-proxy-full-url="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" data-source-url="https://firefox-source-docs.mozilla.org/web-platform/index.html" data-source-language="pl" data-target-language="iw" data-display-language="en-GB" data-detected-source-language="" data-is-source-untranslated="false" data-source-untranslated-url="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://firefox-source-docs.mozilla.org/web-platform/index.html&anno=2" data-client="tr"></script> <div class="wy-grid-for-nav"> <nav data-toggle="wy-nav-shift" class="wy-nav-side"> <div class="wy-side-scroll"> <div class="wy-side-nav-search"><a href="https://firefox--source--docs-mozilla-org.translate.goog/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" class="icon icon-home"> Firefox Source Docs <img src="../_static/firefox-wordmark.svg" class="logo" alt="Logo"> </a> <div> <h3>Quick search</h3> <script> (function () { var cx = "dd12886298f75dbef"; var gcse = document.createElement("script"); gcse.async = true; gcse.src = "https://cse.google.com/cse.js?cx=" + cx; var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(gcse, s); })(); </script><gcse:search></gcse:search> </div> </div> <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu"> <p class="caption" role="heading"><span class="caption-text">Overview</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/glossary/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">A Glossary of Common Terms</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/overview/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">A Quick Guide to Mozilla Applications</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Getting Started</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/setup/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Getting Set Up To Work On The Firefox Codebase</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Working On Firefox</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/contributing/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Working on Firefox</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/bug-mgmt/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Bug Handling</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Firefox User Guide</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/devtools-user/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox DevTools User Docs</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Source Code Documentation</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mots/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Governance</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/browser/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox Front-end</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/dom/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">DOM</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/editor/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Editor</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/layout/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Style system (CSS) & Layout</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/gfx/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Graphics</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/ipc/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Processes, Threads and IPC</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/devtools/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox DevTools Contributor Docs</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/toolkit/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Toolkit</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/js/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">SpiderMonkey</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/jsloader/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">JS Loader</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mobile/android/geckoview/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">GeckoView</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mobile/android/fenix/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Fenix</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mobile/android/focus-android/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Focus for Android</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/dom/bindings/webidl/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">WebIDL</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/modules/libpref/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">libpref</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/networking/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Networking</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/remote/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Remote Protocols</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/services/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Services</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/permissions/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Permissions</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/uriloader/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">File Handling</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/widget/cocoa/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox on macOS</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/widget/windows/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox on Windows</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/toolkit/components/ml/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox AI Runtime</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/accessible/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Accessibility</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/media/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Media Playback</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/code-quality/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Code quality</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/writing-rust-code/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Writing Rust Code</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/rust-components/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Rust Components</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/profiler/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Gecko Profiler</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/performance/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Performance</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/storage/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Database bindings (SQLite, KV, …)</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/xpcom/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">XPCOM</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/nspr/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">NSPR</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/security/nss/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Network Security Services (NSS)</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/content-security/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Web Security Checks in Gecko</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">The Firefox Build System</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mach/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Mach</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/try/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Pushing to Try</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/build/buildsystem/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Build System</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/taskcluster/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox CI and Taskgraph</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/moztreedocs/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Managing Documentation</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mozbuild/vendor/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Vendoring Third Party Components</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Testing & Test Infrastructure</span></p> <ul class="current"> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/automated-testing/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Automated Testing</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/treeherder-try/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Understanding Treeherder Results</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/sheriffed-intermittents/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Sheriffed intermittent failures</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/tests-for-new-config/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Turning on Firefox tests for a new configuration</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/intermittent/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Avoiding intermittent tests</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/testing-policy/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Testing Policy</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/ci-configs/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Configuration Changes</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/browser-chrome/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Browser chrome mochitests</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/chrome-tests/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Chrome Tests</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/marionette/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Marionette</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/geckodriver/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">geckodriver</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/test-verification/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Test Verification</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/webrender/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">WebRender Tests</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/mochitest-plain/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Mochitest</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/xpcshell/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">XPCShell tests</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/tps/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">TPS</a></li> <li class="toctree-l1 current"><a class="current reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#">web-platform-tests</a> <ul> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#upstream-documentation">Upstream Documentation</a></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#running-tests">Running Tests</a> <ul> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#running-tests-in-other-browsers">Running Tests In Other Browsers</a></li> </ul></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#directories">Directories</a></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#metadata">Metadata</a> <ul> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#expectation-data">Expectation Data</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#auto-generating-expectation-data">Auto-generating Expectation Data</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#disabling-tests">Disabling Tests</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#fuzzy-reftests">Fuzzy Reftests</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#enabling-prefs">Enabling Prefs</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#disabling-leak-checks">Disabling Leak Checks</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#per-directory-metadata">Per-Directory Metadata</a></li> <li class="toctree-l3"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#setting-metadata-for-release-branches">Setting Metadata for Release Branches</a></li> </ul></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#test-manifest">Test Manifest</a></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#synchronization-with-upstream">Synchronization with Upstream</a></li> <li class="toctree-l2"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#wpt-serve">wpt-serve</a></li> </ul></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/gtest/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">GTest</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/fuzzing/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Fuzzing</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/sanitizer/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Sanitizer</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing/perfdocs/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Performance Testing</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/code-coverage/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Code coverage</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/testing-rust-code/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Testing & Debugging Rust Code</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Releases & Updates</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/update-infrastructure/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Mozilla Update Infrastructure</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/update-infrastructure/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#watershed-updates">Watershed Updates</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/update-infrastructure/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#desupport-updates">Desupport Updates</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/tools/update-verify/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Update Verify</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Localization & Internationalization</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/intl/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Internationalization</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/l10n/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Localization</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Firefox and Python</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/mozbase/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">mozbase</a></li> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/python/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Using third-party Python packages</a></li> </ul> <p class="caption" role="heading"><span class="caption-text">Metrics Collected in Firefox</span></p> <ul> <li class="toctree-l1"><a class="reference internal" href="https://firefox--source--docs-mozilla-org.translate.goog/metrics/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Metrics</a></li> </ul> </div> </div> </nav> <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"> <nav class="wy-nav-top" aria-label="Mobile navigation menu"><i data-toggle="wy-nav-top" class="fa fa-bars"></i> <a href="https://firefox--source--docs-mozilla-org.translate.goog/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">Firefox Source Docs</a> </nav> <div class="wy-nav-content"> <div class="rst-content"> <div role="navigation" aria-label="Page navigation"> <ul class="wy-breadcrumbs"> <li><a href="https://firefox--source--docs-mozilla-org.translate.goog/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" class="icon icon-home" aria-label="Home"></a></li> <li class="breadcrumb-item active">web-platform-tests</li> <li class="wy-breadcrumbs-aside"><a href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://bugzilla.mozilla.org/enter_bug.cgi?product%3DDeveloper%2BInfrastructure%26component%3DFirefox%2BSource%2BDocs%253A%2BContent%26short_desc%3DDocumentation%2Bissue%2Bon%2Bweb-platform/index%26comment%3DURL%2B%3D%2Bhttps://firefox-source-docs.mozilla.org/web-platform/index.html%26bug_file_loc%3Dhttps://firefox-source-docs.mozilla.org/web-platform/index.html" rel="nofollow">Report an issue</a> / <a href="https://firefox--source--docs-mozilla-org.translate.goog/_sources/web-platform/index.rst.txt?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" rel="nofollow"> View page source</a></li> </ul> <hr> </div> <div role="main" class="document" itemscope itemtype="http://schema.org/Article"> <div itemprop="articleBody"> <section id="web-platform-tests"> <h1>web-platform-tests<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#web-platform-tests" title="Link to this heading"></a></h1> <p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=http://web-platform-tests.org">web-platform-tests</a> is a cross-browser testsuite. Writing tests as web-platform-tests helps ensure that browsers all implement the same behaviour for web-platform features.</p> <section id="upstream-documentation"> <h2>Upstream Documentation<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#upstream-documentation" title="Link to this heading"></a></h2> <p>This documentation covers the integration of web-platform-tests into the Firefox tree. For documentation on writing tests, see <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=http://web-platform-tests.org">web-platform-tests.org</a>. In particular the following documents cover common test-writing needs:</p> <ul class="simple"> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/testharness.html">Javascript Tests (testharness.js)</a></p> <ul> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/testharness-api.html">testharness.js API</a></p></li> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/testdriver.html">testdriver.js API</a> - features for writing tests that require special privileges e.g. user-initiated input events.</p></li> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/channels.html">Message Channels</a> - features for communicating between different globals (including those in different browsing context groups / processes).</p></li> </ul></li> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/reftests.html">Reftests</a></p></li> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/crashtest.html">Crashtests</a></p></li> <li><p><a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/server-features.html">Server features</a> - e.g. multiple origins, substitutions, server-side Python scripts.</p></li> </ul> </section> <section id="running-tests"> <h2>Running Tests<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#running-tests" title="Link to this heading"></a></h2> <p>Tests can be run using <code class="docutils literal notranslate"><span class="pre">mach</span></code>:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span> </pre> </div> </div> <p>To run only certain tests, pass these as additional arguments to the command. For example to include all tests in the dom directory:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span> <span class="n">testing</span><span class="o">/</span><span class="n">web</span><span class="o">-</span><span class="n">platform</span><span class="o">/</span><span class="n">tests</span><span class="o">/</span><span class="n">dom</span> </pre> </div> </div> <p>Tests may also be passed by id; this is the path plus any query or fragment part of a URL and is suitable for copying directly from logs e.g. on treeherder:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span> <span class="o">/</span><span class="n">web</span><span class="o">-</span><span class="n">nfc</span><span class="o">/</span><span class="n">idlharness</span><span class="o">.</span><span class="n">https</span><span class="o">.</span><span class="n">window</span><span class="o">.</span><span class="n">html</span> </pre> </div> </div> <p>A single file can produce multiple tests, so passing test ids rather than paths is sometimes necessary to run exactly one test.</p> <p>The testsuite contains a mix of various test types including Javascript (<code class="docutils literal notranslate"><span class="pre">testharness</span></code>) tests, reftests and wdspec tests. To limit the type of tests that get run, use <code class="docutils literal notranslate"><span class="pre">--test-type=<type></span></code> e.g. <code class="docutils literal notranslate"><span class="pre">--test-type=reftest</span></code> for reftests.</p> <p>Note that if only a single testharness test is run the browser will stay open by default (matching the behaviour of mochitest). To prevent this pass <code class="docutils literal notranslate"><span class="pre">--no-pause-after-test</span></code> to <code class="docutils literal notranslate"><span class="pre">mach</span> <span class="pre">wpt</span></code>.</p> <p>When the source tree is configured for building android, tests will also be run on Android, by default using a local emulator.</p> <section id="running-tests-in-other-browsers"> <h3>Running Tests In Other Browsers<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#running-tests-in-other-browsers" title="Link to this heading"></a></h3> <p>web-platform-tests is cross browser, and the runner is compatible with multiple browsers. Therefore it’s possible to check the behaviour of tests in other browsers. By default Chrome, Edge and Servo are supported. In order to run the tests in these browsers use the <code class="docutils literal notranslate"><span class="pre">--product</span></code> argument to wptrunner:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span> <span class="o">--</span><span class="n">product</span> <span class="n">chrome</span> <span class="n">dom</span><span class="o">/</span><span class="n">historical</span><span class="o">.</span><span class="n">html</span> </pre> </div> </div> <p>By default these browsers run without expectation metadata, but it can be added in the <code class="docutils literal notranslate"><span class="pre">testing/web-platform/products/<product></span></code> directory. To run with the same metadata as for Firefox (so that differences are reported as unexpected results), pass <code class="docutils literal notranslate"><span class="pre">--meta</span> <span class="pre">testing/web-platform/meta</span></code> to the mach command.</p> <p>Results from the upstream CI for many browser, including Chrome and Safari, are available on <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://wpt.fyi">wpt.fyi</a>. There is also a <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://jgraham.github.io/wptdash/">gecko dashboard</a> which by default shows tests that are failing in Gecko but not in Chrome and Safari, organised by bug component, based on the wpt.fyi data.</p> </section> </section> <section id="directories"> <h2>Directories<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#directories" title="Link to this heading"></a></h2> <p>Under <code class="docutils literal notranslate"><span class="pre">testing/web-platform</span></code> are the following directories:</p> <dl class="simple"> <dt> <code class="docutils literal notranslate"><span class="pre">tests/</span></code> </dt> <dd> <p>An automatically-updated import of the web-platform-tests repository. Any changes to this directory are automatically converted into pull requests against the upstream repository, and merged if they pass CI.</p> </dd> <dt> <code class="docutils literal notranslate"><span class="pre">meta/</span></code> </dt> <dd> <p>Gecko-specific metadata including expected test results and configuration e.g. prefs that are set when running the test. This is explained in the following section.</p> </dd> <dt> <code class="docutils literal notranslate"><span class="pre">mozilla/tests</span></code> </dt> <dd> <p>Tests that will not be upstreamed and may make use of Mozilla-specific features. They can access the <code class="docutils literal notranslate"><span class="pre">SpecialPowers</span></code> APIs.</p> </dd> <dt> <code class="docutils literal notranslate"><span class="pre">mozilla/meta</span></code> </dt> <dd> <p>Metadata for the Mozilla-specific tests.</p> </dd> </dl> </section> <section id="metadata"> <h2>Metadata<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#metadata" title="Link to this heading"></a></h2> <p>In order to separate out the shared testsuite from Firefox-specific metadata about the tests, all the metadata is stored in separate ini-like files in the <code class="docutils literal notranslate"><span class="pre">meta/</span></code> sub-directory.</p> <p>There is one metadata file per test file with associated gecko-specific data. The metadata file of a test has the same relative path as the test file and has the the suffix <code class="docutils literal notranslate"><span class="pre">.ini</span></code> e.g. for the test in <code class="docutils literal notranslate"><span class="pre">testing/web-platform/tests/example/example.html</span></code>, the corresponding metadata file is <code class="docutils literal notranslate"><span class="pre">testing/web-platform/meta/example/example.html.ini</span></code>.</p> <p>The format of these files is similar to <cite>ini</cite> files, but with a couple of important differences; sections can be nested using indentation, and only <cite>:</cite> is permitted as a key-value separator. For example:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="p">[</span><span class="n">Subtest</span> <span class="mi">1</span> <span class="n">name</span><span class="p">]</span> <span class="n">key</span><span class="p">:</span> <span class="n">value</span> <span class="p">[</span><span class="n">Subtest</span> <span class="mi">2</span> <span class="n">name</span><span class="p">]</span> <span class="n">key</span><span class="p">:</span> <span class="p">[</span><span class="nb">list</span><span class="p">,</span> <span class="n">value</span><span class="p">]</span> </pre> </div> </div> <p>For cases where a single file generates multiple tests (e.g. variants or <code class="docutils literal notranslate"><span class="pre">.any.js</span></code> tests), the metadata file has one top-level section for each test, for example:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">test</span><span class="o">.</span><span class="n">any</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="p">[</span><span class="n">Subtest</span> <span class="n">name</span><span class="p">]</span> <span class="n">key</span><span class="p">:</span> <span class="n">value</span> <span class="p">[</span><span class="n">test</span><span class="o">.</span><span class="n">any</span><span class="o">.</span><span class="n">worker</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="p">[</span><span class="n">Subtest</span> <span class="n">name</span><span class="p">]</span> <span class="n">key</span><span class="p">:</span> <span class="n">other</span><span class="o">-</span><span class="n">value</span> </pre> </div> </div> <p>Values can be made conditional using a Python-like conditional syntax:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">key</span><span class="p">:</span> <span class="k">if</span> <span class="n">os</span> <span class="o">==</span> <span class="s2">"linux"</span><span class="p">:</span> <span class="n">linux</span><span class="o">-</span><span class="n">value</span> <span class="n">default</span><span class="o">-</span><span class="n">value</span> </pre> </div> </div> <p>The available variables for the conditions are those provided by <a class="reference external" href="https://firefox--source--docs-mozilla-org.translate.goog/mozbase/mozinfo.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB">mozinfo</a>, plus some additional <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://searchfox.org/mozilla-central/search?q%3Ddef%2520run_info_extras%26path%3Dtesting%252Fweb-platform%252Ftests%252Ftools%252Fwptrunner%252Fwptrunner%252Fbrowsers%252Ffirefox.py%26case%3Dfalse%26regexp%3Dfalse">wpt-specific values</a>.</p> <p>For more information on manifest files, see the <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/tools/wptrunner/docs/expectation.html">wptrunner documentation</a></p> <section id="expectation-data"> <h3>Expectation Data<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#expectation-data" title="Link to this heading"></a></h3> <p>All tests that don’t pass in our CI have expectation data stored in the metadata file, under the key <code class="docutils literal notranslate"><span class="pre">expected</span></code>. For example the expectation file for a test with one failing subtest and one erroring subtest might look like:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="p">[</span><span class="n">Subtest</span> <span class="n">name</span> <span class="k">for</span> <span class="n">failing</span> <span class="n">test</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="n">FAIL</span> <span class="p">[</span><span class="n">Subtest</span> <span class="n">name</span> <span class="k">for</span> <span class="n">erroring</span> <span class="n">test</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="n">ERROR</span> </pre> </div> </div> <p>Expectations can be made configuration-specific using the conditional syntax:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="k">if</span> <span class="n">os</span> <span class="o">==</span> <span class="s2">"linux"</span> <span class="ow">and</span> <span class="n">bits</span> <span class="o">==</span> <span class="mi">32</span><span class="p">:</span> <span class="n">TIMEOUT</span> <span class="k">if</span> <span class="n">os</span> <span class="o">==</span> <span class="s2">"win"</span><span class="p">:</span> <span class="n">ERROR</span> <span class="n">FAIL</span> </pre> </div> </div> <p>Tests that are intermittent may be marked with multiple statuses using a list of possibilities e.g. for a test that usually passes, but intermittently fails:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="p">[</span><span class="n">Subtest</span> <span class="n">name</span> <span class="k">for</span> <span class="n">intermittent</span> <span class="n">test</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="p">[</span><span class="n">PASS</span><span class="p">,</span> <span class="n">FAIL</span><span class="p">]</span> </pre> </div> </div> </section> <section id="auto-generating-expectation-data"> <h3>Auto-generating Expectation Data<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#auto-generating-expectation-data" title="Link to this heading"></a></h3> <p>After changing some code it may be necessary to update the expectation data for the relevant tests. This can of course be done manually, but tools are available to automate much of the process.</p> <p>First one must run the tests that have changed status, and save the raw log output to a file:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span> <span class="o">/</span><span class="n">url</span><span class="o">/</span><span class="n">of</span><span class="o">/</span><span class="n">test</span><span class="o">.</span><span class="n">html</span> <span class="o">--</span><span class="n">log</span><span class="o">-</span><span class="n">wptreport</span> <span class="n">wptreport</span><span class="o">.</span><span class="n">json</span> </pre> </div> </div> <p>Then the <code class="docutils literal notranslate"><span class="pre">wpt-update</span></code> command may be run using this log data to update the expectation files:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span><span class="o">-</span><span class="n">update</span> <span class="n">wptreport</span><span class="o">.</span><span class="n">json</span> </pre> </div> </div> <p>CI runs also produce <code class="docutils literal notranslate"><span class="pre">wptreport.json</span></code> files that can be downloaded as artifacts. When tests are run across multiple platforms, and all the wptreport files are processed together, the tooling will set the appropriate conditions for any platform-specific results:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">mach</span> <span class="n">wpt</span><span class="o">-</span><span class="n">update</span> <span class="n">logs</span><span class="o">/*.</span><span class="n">json</span> </pre> </div> </div> <p>For complete runs the <code class="docutils literal notranslate"><span class="pre">--full</span></code> flag will cause metadata to be removed when a) the test was updated and b) there is a condition that didn’t match any of the configuration in the input files.</p> <p>When tests are run more than once <code class="docutils literal notranslate"><span class="pre">--update-intermittent</span></code> flag will cause conflicting results to be marked as intermittents (otherwise the data is not updated in the case of conflicts).</p> </section> <section id="disabling-tests"> <h3>Disabling Tests<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#disabling-tests" title="Link to this heading"></a></h3> <p>Tests are disabled using the same manifest files used to set expectation values. For example, if a test is unstable on Windows, it can be disabled using an ini file with the contents:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span>[filename.html] disabled: if os == "win": https://bugzilla.mozilla.org/show_bug.cgi?id=1234567 </pre> </div> </div> <p>For intermittents it’s generally preferable to give the test multiple expectations rather than disable it.</p> </section> <section id="fuzzy-reftests"> <h3>Fuzzy Reftests<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#fuzzy-reftests" title="Link to this heading"></a></h3> <p>Reftests where the test doesn’t exactly match the reference can be marked as fuzzy. If the difference is inherent to the test, it should be encoded in a <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://web-platform-tests.org/writing-tests/reftests.html%23fuzzy-matching">meta element</a>, but where it’s a Gecko-specific difference it can be added to the metadata file, using the same syntax:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">fuzzy</span><span class="p">:</span> <span class="n">maxDifference</span><span class="o">=</span><span class="mi">10</span><span class="o">-</span><span class="mi">15</span><span class="p">;</span><span class="n">totalPixels</span><span class="o">=</span><span class="mi">200</span><span class="o">-</span><span class="mi">300</span> </pre> </div> </div> <p>In this case we expect between 200 and 300 pixels, inclusive, to be different, and the maximum difference in any RGB colour channel to be between 10 and 15.</p> </section> <section id="enabling-prefs"> <h3>Enabling Prefs<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#enabling-prefs" title="Link to this heading"></a></h3> <p>Some tests require specific prefs to be enabled before running. These prefs can be set in the expectation data using a <code class="docutils literal notranslate"><span class="pre">prefs</span></code> key with a comma-separated list of <code class="docutils literal notranslate"><span class="pre">pref.name:value</span></code> items:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">prefs</span><span class="p">:</span> <span class="p">[</span><span class="n">dom</span><span class="o">.</span><span class="n">serviceWorkers</span><span class="o">.</span><span class="n">enabled</span><span class="p">:</span><span class="n">true</span><span class="p">,</span> <span class="n">dom</span><span class="o">.</span><span class="n">serviceWorkers</span><span class="o">.</span><span class="n">exemptFromPerDomainMax</span><span class="p">:</span><span class="n">true</span><span class="p">,</span> <span class="n">dom</span><span class="o">.</span><span class="n">caches</span><span class="o">.</span><span class="n">enabled</span><span class="p">:</span><span class="n">true</span><span class="p">]</span> </pre> </div> </div> </section> <section id="disabling-leak-checks"> <h3>Disabling Leak Checks<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#disabling-leak-checks" title="Link to this heading"></a></h3> <p>When a test is imported that leaks, it may be necessary to temporarily disable leak checking for that test in order to allow the import to proceed. This works in basically the same way as disabling a test, but with the key <code class="docutils literal notranslate"><span class="pre">leaks</span></code>:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span>[filename.html] leaks: if os == "linux": https://bugzilla.mozilla.org/show_bug.cgi?id=1234567 </pre> </div> </div> </section> <section id="per-directory-metadata"> <h3>Per-Directory Metadata<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#per-directory-metadata" title="Link to this heading"></a></h3> <p>Occasionally it is useful to set metadata for an entire directory of tests e.g. to disable then all, or to enable prefs for every test. In that case it is possible to create a <code class="docutils literal notranslate"><span class="pre">__dir__.ini</span></code> file in the metadata directory corresponding to the tests for which you want to set this metadata e.g. to disable all the tests in <code class="docutils literal notranslate"><span class="pre">tests/feature/unsupported/</span></code>, one might create <code class="docutils literal notranslate"><span class="pre">meta/feature/unsupported/__dir__.ini</span></code> with the contents:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">disabled</span><span class="p">:</span> <span class="n">Feature</span> <span class="ow">is</span> <span class="n">unsupported</span> </pre> </div> </div> <p>Settings set in this way are inherited into sub-directories. It is possible to unset a value that has been set in a parent using the special token <code class="docutils literal notranslate"><span class="pre">@Reset</span></code> (usually used with prefs), or to force a value to true or false using <code class="docutils literal notranslate"><span class="pre">@True</span></code> and <code class="docutils literal notranslate"><span class="pre">@False</span></code>. For example to enable the tests in <code class="docutils literal notranslate"><span class="pre">meta/feature/unsupported/subfeature-supported</span></code> one might create an ini file <code class="docutils literal notranslate"><span class="pre">meta/feature/unsupported/subfeature-supported/__dir__.ini</span></code> like:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="n">disabled</span><span class="p">:</span> <span class="nd">@False</span> </pre> </div> </div> </section> <section id="setting-metadata-for-release-branches"> <h3>Setting Metadata for Release Branches<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#setting-metadata-for-release-branches" title="Link to this heading"></a></h3> <p>Run info properties can be used to set metadata for release branches that differs from nightly (e.g. for when a feature depends on prefs that are only set on nightly), for example:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="k">if</span> <span class="n">release_or_beta</span><span class="p">:</span> <span class="n">FAIL</span> </pre> </div> </div> <p>Note that in general the automatic metadata update will work better if the nonstandard configuration is used explicitly in the conditional, and placed at the top of the set of conditions, i.e. the following would cause problems later:</p> <div class="highlight-default notranslate"> <div class="highlight"> <pre><span></span><span class="p">[</span><span class="n">filename</span><span class="o">.</span><span class="n">html</span><span class="p">]</span> <span class="n">expected</span><span class="p">:</span> <span class="k">if</span> <span class="n">nightly_build</span><span class="p">:</span> <span class="n">PASS</span> <span class="n">FAIL</span> </pre> </div> </div> <p>This is because on import the automatic metadata updates are run against the results of nightly builds, and we remove any existing conditions that match all the input runs to avoid building up stale configuration options.</p> </section> </section> <section id="test-manifest"> <h2>Test Manifest<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#test-manifest" title="Link to this heading"></a></h2> <p>web-platform-tests use a large auto-generated JSON file as their manifest. This stores data about the type of tests, their references, if any, and their timeout, gathered by inspecting the filenames and the contents of the test files. It is not necessary to manually add new tests to the manifest; it is automatically kept up to date when running <cite>mach wpt</cite>.</p> </section> <section id="synchronization-with-upstream"> <h2>Synchronization with Upstream<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#synchronization-with-upstream" title="Link to this heading"></a></h2> <p>Tests are automatically synchronized with upstream using the <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://github.com/mozilla/wpt-sync">wpt-sync bot</a>. This performs the following tasks:</p> <ul class="simple"> <li><p>Creates upstream PRs for changes in <code class="docutils literal notranslate"><span class="pre">testing/web-platform/tests</span></code> once they land on autoland, and automatically merges them after they reach mozilla-central.</p></li> <li><p>Runs merged upstream PRs through gecko CI to generate updated expectation metadata.</p></li> <li><p>Updates the copy of web-platform-tests in the gecko tree with changes from upstream, and the expectation metadata required to make CI jobs pass.</p></li> </ul> <p>The nature of a two-way sync means that occasional merge conflicts and other problems. If something isn’t in sync with upstream in the way you expect, please ask on <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://chat.mozilla.org/%23/room/%23interop:mozilla.org">#interop</a> on matritx.</p> </section> <section id="wpt-serve"> <h2>wpt-serve<a class="headerlink" href="https://firefox--source--docs-mozilla-org.translate.goog/web-platform/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB#wpt-serve" title="Link to this heading"></a></h2> <p>Sometimes, it’s preferable to run the WPT’s web server on its own, and point different browsers to the test files.</p> <blockquote> <div> <p>./mach wpt-serve</p> </div> </blockquote> <p>can be used for this, after a short setup:</p> <p>On Unix, one can run:</p> <blockquote> <div> <p>./wpt make-hosts-file | sudo tee -a /etc/hosts</p> </div> </blockquote> <p>from the root of the WPT checkout, present at <code class="docutils literal notranslate"><span class="pre">testing/web-platform/tests/</span></code>.</p> <p>On Windows, from an administrator <code class="docutils literal notranslate"><span class="pre">mozilla-build</span></code> shell, one can run:</p> <blockquote> <div> <p>./wpt make-hosts-file >> /c/Windows/System32/drivers/etc/host</p> </div> </blockquote> <p>from the WPT checkout.</p> <p>Most of the time, browsing to <a class="reference external" href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=http://localhost:8000">http://localhost:8000</a> will allow running the test, although some tests have special requirements, such as running on a specific domain or running using https.</p> </section> </section> </div> </div> <footer> <div class="rst-footer-buttons" role="navigation" aria-label="Footer"><a href="https://firefox--source--docs-mozilla-org.translate.goog/testing/tps/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" class="btn btn-neutral float-left" title="TPS" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a> <a href="https://firefox--source--docs-mozilla-org.translate.goog/gtest/index.html?_x_tr_sl=pl&_x_tr_tl=iw&_x_tr_hl=en-GB" class="btn btn-neutral float-right" title="GTest" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a> </div> <hr> <div role="contentinfo"> <p></p> </div> Built with <a href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://www.sphinx-doc.org/">Sphinx</a> using a <a href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://translate.google.com/website?sl=pl&tl=iw&hl=en-GB&u=https://readthedocs.org">Read the Docs</a>. </footer> </div> </div> </section> </div> <script> jQuery(function () { SphinxRtdTheme.Navigation.enable(true); }); </script> <script>function gtElInit() {var lib = new google.translate.TranslateService();lib.translatePage('pl', 'iw', function () {});}</script> <script src="https://translate.google.com/translate_a/element.js?cb=gtElInit&hl=en-GB&client=wt" type="text/javascript"></script> </body> </html>

CINXE.COM

web-platform-tests — Firefox Source Docs documentation