Media Capture and Streams

<!DOCTYPE html> <html lang="en-us"> <head> <link href="getusermedia.css" rel="stylesheet" type="text/css"> <title>Media Capture and Streams</title> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"> <script class="remove" src="respec-w3c.js" type="text/javascript"></script> <script class="remove" src="getusermedia.js" type="text/javascript"></script> </head> <body> <p class='copyright'>Initial Author of this Specification was Ian Hickson, Google Inc., with the following copyright statement:<br /> © Copyright 2004-2011 Apple Computer, Inc., Mozilla Foundation, and Opera Software ASA. You are granted a license to use, reproduce and create derivative works of this document.</p> <p class='copyright'>All subsequent changes since 26 July 2011 done by the W3C WebRTC Working Group (and previously the Device APIs Working Group) are under the following <a href="https://www.w3.org/policies/#copyright">Copyright</a> © 2011-2023 <a href="https://www.w3.org/">World Wide Web Consortium</a>. <abbr title="World Wide Web Consortium">W3C</abbr><sup>®</sup> <a href="https://www.w3.org/policies/#Legal_Disclaimer">liability</a>, <a href="https://www.w3.org/policies/#W3C_Trademarks">trademark</a> and <a href="https://www.w3.org/copyright/software-license/">permissive document license</a> rules apply. <section id="abstract"> <p>This document defines a set of JavaScript APIs that allow local media, including audio and video, to be requested from a platform.</p> </section> <section id="sotd"> <p>This document is not complete. The API is based on preliminary work done in the WHATWG.</p> <p>Before this document proceeds to Proposed Recommendation, the WebRTC Working Group intends to address <a href="https://www.w3.org/PM/horizontal/review.html?shortname=mediacapture-streams">issues that emerged from wide review</a>.</p> </section> <section class="informative" id="intro"> <h2>Introduction</h2> <p>This document defines APIs for requesting access to local multimedia devices, such as microphones or video cameras.</p> <p>This document also defines the MediaStream API, which provides the means to control where multimedia stream data is consumed, and provides some control over the devices that produce the media. It also exposes information about devices able to capture and render media.</p> </section> <section id="conformance"> <p>This specification defines conformance criteria that apply to a single product: the <dfn>User Agent</dfn> that implements the interfaces that it contains.</p> <p>Conformance requirements phrased as algorithms or specific steps may be implemented in any manner, so long as the end result is equivalent. (In particular, the algorithms defined in this specification are intended to be easy to follow, and not intended to be performant.)</p> <p>Implementations that use ECMAScript [[ECMA-262]] to implement the APIs defined in this specification must implement them in a manner consistent with the ECMAScript Bindings defined in the Web IDL specification [[WEBIDL]], as this specification uses that specification and terminology.</p> </section> <section> <h2>Terminology</h2> <dl> <dt><dfn data-export>source</dfn></dt> <dd> <p>A source is the "thing" providing the source of a media stream track. The source is the broadcaster of the media itself. A source can be a physical webcam, microphone, local video or audio file from the user's hard drive, network resource, or static image. Note that this document describes the use of microphone and camera type sources only, the use of other source types is described in other documents.</p> <p>An application that has no prior authorization regarding sources is only given the number of available sources, their type and any relationship to other devices. Additional information about sources can become available when applications are authorized to use a source (see <a href="#access-control-model"></a>).</p> <p>Sources <strong>do not</strong> have constraints — tracks have constraints. When a source is connected to a track, it must produce media that conforms to the constraints present on that track, to that track. Multiple tracks can be attached to the same source. [=User Agent=] processing, such as downsampling, MAY be used to ensure that all tracks have appropriate media.</p> <p>Sources have constrainable properties which have [=capabilities=] and [=settings=] exposed on tracks. While the constrainable properties are "owned" by the source, sources MAY be able to accommodate different demands at once. For this reason, capabilities are common to any (multiple) tracks that happen to be using the same source, whereas settings MAY differ per track (e.g., if two different track objects bound to the same source query capability and settings information, they will get back the same capabilities, but may get different settings that are tailored to satisfy their individual constraints).</p> </dd> <dt><dfn data-lt=settings>Setting</dfn> (Source Setting)</dt> <dd> <p>A setting refers to the immediate, current value of the source's constrainable properties. Settings are always read-only.</p> <p>A source conditions may dynamically change, such as when a camera switches to a lower frame rate due to low light conditions. In these cases the tracks related to the affected source might not satisfy the set constraints any longer. The platform SHOULD try to minimize such excursions as far as possible, but will continue to deliver media even when a temporary or permanent condition exists that prevents satisfying the constraints.</p> <p>Although settings are a property of the source, they are only exposed to the application through the tracks attached to the source. This is exposed via the <a>ConstrainablePattern</a> interface.</p> </dd> <dt><dfn data-lt=capabilities>Capability</dfn></dt> <dd> <p>For each constrainable property, there is a capability that describes whether it is supported by the source and if so, the range of supported values. As with settings, capabilities are exposed to the application via the <a>ConstrainablePattern</a> interface.</p> <p>The values of the supported capabilities must be normalized to the ranges and enumerated types defined in this specification.</p> <p>A {{MediaStreamTrack/getCapabilities()}} call on a track returns the same underlying per-source capabilities for all tracks connected to the source.</p> <p>Source capabilities are effectively constant. Applications should be able to depend on a specific source having the same capabilities for any browsing session.</p> <p>This API is intentionally simplified. Capabilities are not capable of describing interactions between different values. For instance, it is not possible to accurately describe the capabilities of a camera that can produce a high resolution video stream at a low frame rate and lower resolutions at a higher frame rate. Capabilities describe the complete range of each value. Interactions between constraints are exposed by attempting to apply constraints.</p> </dd> <dt><dfn>Constraint</dfn>s</dt> <dd> <p>Constraints provide a general control surface that allows applications to both select an appropriate source for a track and, once selected, to influence how a source operates.</p> <p>Constraints limit the range of operating modes that a source can use when providing media for a track. Without provided track constraints, implementations are free to select a source's settings from the full ranges of its supported capabilities. Implementations may also adjust source settings at any time within the bounds imposed by all applied constraints.</p> <p>{{MediaDevices/getUserMedia()}} uses constraints to help select an appropriate source for a track and configure it. Additionally, the <a>ConstrainablePattern</a> interface on tracks includes an API for dynamically changing the track's constraints at any later time.</p> <p>A track will not be connected to a source using {{MediaDevices/getUserMedia()}} if its initial constraints cannot be satisfied. However, the ability to meet the constraints on a track can change over time, and constraints can be changed. If circumstances change such that constraints cannot be met, the <a>ConstrainablePattern</a> interface defines an appropriate error to inform the application. [[[#the-model-sources-sinks-constraints-and-settings]]] explains how constraints interact in more detail.</p> <p>For each constrainable property, a constraint exists whose name corresponds with the relevant source setting name and capability name.</p> <p>A constraint falls into one of three groups, depending on its place in the constraints structure. The groups are:</p> <ul> <li><dfn>required constraints</dfn> are all <a data-lt= "advanced constraints">non-advanced</a> constraints that are <a>required</a>.</li> <li><dfn>optional basic constraints</dfn> are the remaining <a data-lt="advanced constraints">non-advanced</a> constraints.</li> <li><dfn>advanced constraints</dfn> are all constraints specified using the <code><a href="#dom-constraints-advanced">advanced</a></code> keyword.</li> </ul> <p>In general, [=User Agents=] will have more flexibility to optimize the media streaming experience the fewer constraints are applied, so application authors are strongly encouraged to use <a>required constraints</a> sparingly.</p> </dd> </dl> </section> <section id="stream-api"> <h2>MediaStream API</h2> <section> <h2>Introduction</h2> <p>The two main components in the MediaStream API are the {{MediaStreamTrack}} and {{MediaStream}} interfaces. The {{MediaStreamTrack}} object represents media of a single type that originates from one media source in the [=User Agent=], e.g. video produced by a web camera. A {{MediaStream}} is used to group several {{MediaStreamTrack}} objects into one unit that can be recorded or rendered in a media element.</p> <p>Each {{MediaStream}} can contain zero or more {{MediaStreamTrack}} objects. All tracks in a {{MediaStream}} are intended to be synchronized when rendered. This is not a hard requirement, since it might not be possible to synchronize tracks from sources that have different clocks. Different {{MediaStream}} objects do not need to be synchronized.</p> <p class="note">While the intent is to synchronize tracks, it could be better in some circumstances to permit tracks to lose synchronization. In particular, when tracks are remotely sourced and real-time [[?WEBRTC]], it can be better to allow loss of synchronization than to accumulate delays or risk glitches and other artifacts. Implementations are expected to understand the implications of choices regarding synchronization of playback and the effect that these have on user perception.</p> <p>A single {{MediaStreamTrack}} can represent multi-channel content, such as stereo or 5.1 audio or stereoscopic video, where the channels have a well defined relationship to each other. Information about channels might be exposed through other APIs, such as [[?WEBAUDIO]], but this specification provides no direct access to channels.</p> <p>A {{MediaStream}} object has an input and an output that represent the combined input and output of all the object's tracks. The output of the {{MediaStream}} controls how the object is rendered, e.g., what is saved if the object is recorded to a file or what is displayed if the object is used in a [^video^] element. A single {{MediaStream}} object can be attached to multiple different outputs at the same time.</p> <p>A new {{MediaStream}} object can be created from existing media streams or tracks using the {{MediaStream/MediaStream()}} constructor. The constructor argument can either be an existing {{MediaStream}} object, in which case all the tracks of the given stream are added to the new {{MediaStream}} object, or an array of {{MediaStreamTrack}} objects. The latter form makes it possible to compose a stream from different source streams.</p> <p>Both {{MediaStream}} and {{MediaStreamTrack}} objects can be cloned. A cloned {{MediaStream}} contains clones of all member tracks from the original stream. A cloned {{MediaStreamTrack}} has a <a href="#constrainable-interface">set of constraints</a> that is independent of the instance it is cloned from, which allows media from the same source to have different constraints applied for different <a>consumer</a>s. The {{MediaStream}} object is also used in contexts outside {{MediaDevices/getUserMedia}}, such as [[?WEBRTC]].</p> </section> <section> <h2>{{MediaStream}}</h2> <p id= "mediastream-constructor">The MediaStream <dfn data-idl data-dfn-for=MediaStream>constructor</dfn> composes a new stream out of existing tracks. It takes an optional argument of type {{MediaStream}} or an array of {{MediaStreamTrack}} objects. When the constructor is invoked, the User Agent must run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>stream</var> be a newly constructed {{MediaStream}} object.</p> </li> <li> <p>Initialize <var>stream</var>.{{MediaStream/id}} attribute to a newly generated value.</p> </li> <li> <p>If the constructor's argument is present, run the following steps:</p> <ol> <li> <p>Construct a set of tracks <var>tracks</var> based on the type of argument:</p> <ul> <li> <p>A {{MediaStream}} object:</p> <p>Let <var>tracks</var> be a set containing all the {{MediaStreamTrack}} objects in the {{MediaStream}} <a>track set</a>.</p> </li> <li> <p>A sequence of {{MediaStreamTrack}} objects:</p> <p>Let <var>tracks</var> be a set containing all the {{MediaStreamTrack}} objects in the provided sequence.</p> </li> </ul> </li> <li> <p>For each {{MediaStreamTrack}}, <var>track</var> , in <var>tracks</var>, run the following steps:</p> <ol> <li> <p>If <var>track</var> is already in <var>stream</var>'s [=track set=], skip <var>track</var>.</p> </li> <li> <p>Otherwise, add <var>track</var> to <var>stream</var>'s [=track set=].</p> </li> </ol> </li> </ol> </li> <li> <p>Return <var>stream</var>.</p> </li> </ol> <p>The tracks of a {{MediaStream}} are stored in a <dfn class="export">track set</dfn>. The track set MUST contain the {{MediaStreamTrack}} objects that correspond to the tracks of the stream. The relative order of the tracks in the set is User Agent defined and the API will never put any requirements on the order. The proper way to find a specific {{MediaStreamTrack}} object in the set is to look it up by its {{MediaStreamTrack/id}}.</p> <p>An object that reads data from the output of a {{MediaStream}} is referred to as a {{MediaStream}} <dfn data-export data-for=MediaStream>consumer</dfn>. The list of {{MediaStream}} consumers currently include media elements (such as [^video^] and [^audio^]) [[HTML]], Web Real-Time Communications (WebRTC; {{RTCPeerConnection}}) [[?WEBRTC]], media recording (<code class=fixme>MediaRecorder</code>) [[?mediastream-recording]], image capture (<code class=fixme>ImageCapture</code>) [[?image-capture]], and web audio ({{MediaStreamAudioSourceNode}}) [[?WEBAUDIO]].</p> <p class="note">{{MediaStream}} consumers must be able to handle tracks being added and removed. This behavior is specified per consumer.</p> <p>A {{MediaStream}} object is said to be <dfn data-dfn-for="stream" id= "stream-active">active</dfn> when it has at least one {{MediaStreamTrack}} that has not [=MediaStreamTrack/ended=]. A {{MediaStream}} that does not have any tracks or only has tracks that are [= MediaStreamTrack/ended =] is <dfn data-dfn-for="stream" id="stream-inactive">inactive</dfn>.</p> <p>A {{MediaStream}} object is said to be <dfn data-dfn-for=stream id= "stream-audible" data-dfn-for=stream>audible</dfn> when it has at least one {{MediaStreamTrack}} whose {{MediaStreamTrack/[[Kind]]}} is <a>"audio"</a> that has not [=MediaStreamTrack/ended=]. A {{MediaStream}} that does not have any audio tracks or only has audio tracks that are [=MediaStreamTrack/ended=] is <dfn id="stream-inaudible" data-dfn-for=stream>inaudible</dfn>.</p> <p>The [=User Agent=] may update a {{MediaStream}}'s [=track set=] in response to, for example, an external event. This specification does not specify any such cases, but other specifications using the MediaStream API may. One such example is the WebRTC 1.0 [[?WEBRTC]] specification where the [=track set=] of a {{MediaStream}}, received from another peer, can be updated as a result of changes to the media session.</p> <p>To <dfn class="abstract-op" data-dfn-for="MediaStream">add a track</dfn> <var>track</var> to a {{MediaStream}} <var>stream</var>, the [=User Agent=] MUST run the following steps:</p> <ol class=algorithm> <li> <p>If <var>track</var> is already in <var>stream's</var> [=track set=], then abort these steps.</p> </li> <li> <p>Add <var>track</var> to <var>stream</var>'s [=track set=].</p> </li> <li> <p>[= Fire a track event=] named {{addtrack}} with <var>track</var> at <var>stream</var>.</p> </li> </ol> <p>To <dfn class="abstract-op" data-dfn-for="MediaStream">remove a track</dfn> <var>track</var> from a {{MediaStream}} <var>stream</var>, the [=User Agent=] MUST run the following steps:</p> <ol class=algorithm> <li> <p>If <var>track</var> is not in <var>stream's</var> [=track set=], then abort these steps.</p> </li> <li> <p>[=MediaStream/Remove a track|Remove=] <var>track</var> from <var>stream</var>'s [=track set=].</p> </li> <li> <p>[= Fire a track event =] named {{removetrack}} with <var>track</var> at <var>stream</var>.</p> </li> </ol> <div> <pre class="idl" >[Exposed=Window] interface MediaStream : EventTarget { constructor(); constructor(MediaStream stream); constructor(sequence<MediaStreamTrack> tracks); readonly attribute DOMString id; sequence<MediaStreamTrack> getAudioTracks(); sequence<MediaStreamTrack> getVideoTracks(); sequence<MediaStreamTrack> getTracks(); MediaStreamTrack? getTrackById(DOMString trackId); undefined addTrack(MediaStreamTrack track); undefined removeTrack(MediaStreamTrack track); MediaStream clone(); readonly attribute boolean active; attribute EventHandler onaddtrack; attribute EventHandler onremovetrack; };</pre> <section> <h2>Constructors</h2> <dl data-link-for="MediaStream" data-dfn-for="MediaStream" class= "constructors"> <dt>{{MediaStream}}</dt> <dd> <p>See the <a href="#mediastream-constructor">MediaStream constructor algorithm</a></p> <div> <em>No parameters.</em> </div> </dd> <dt>{{MediaStream}}</dt> <dd> <p>See the <a href="#mediastream-constructor">MediaStream constructor algorithm</a></p> </dd> <dt>{{MediaStream}}</dt> <dd> <p>See the <a href="#mediastream-constructor">MediaStream constructor algorithm</a></p> </dd> </dl> </section> <section> <h2>Attributes</h2> <dl data-link-for="MediaStream" data-dfn-for="MediaStream" class= "attributes"> <dt>{{id}} of type {{DOMString}}, readonly</dt> <dd> <p>The <dfn data-idl>id</dfn> attribute MUST return the value to which it was initialized when the object was created.</p> <p>When a {{MediaStream}} is created, the User Agent MUST generate an identifier string, and MUST initialize the object's {{id}} attribute to that string, unless the object is created as part of a special purpose algorithm that specifies how the stream id must be initialized. A good practice is to use a UUID [[rfc4122]], which is 36 characters long in its canonical form. To avoid fingerprinting, implementations SHOULD use the forms in section 4.4 or 4.5 of RFC 4122 when generating UUIDs.</p> <p>An example of an algorithm that specifies how the stream id must be initialized is the algorithm to associate an incoming network component with a {{MediaStream}} object. [[?WEBRTC]]</p> </dd> <dt><dfn>active</dfn> of type {{boolean}}, readonly</dt> <dd> <p>The {{active}} attribute MUST return <code>true</code> if this {{MediaStream}} is [= stream/active =] and <code>false</code> otherwise.</p> </dd> <dt><dfn>onaddtrack</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is {{addtrack}}.</p> </dd> <dt><dfn>onremovetrack</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is {{removetrack}}.</p> </dd> </dl> </section> <section> <h2>Methods</h2> <dl data-link-for="MediaStream" data-dfn-for="MediaStream" class= "methods"> <dt><dfn>getAudioTracks()</dfn></dt> <dd> <p>Returns a sequence of {{MediaStreamTrack}} objects representing the audio tracks in this stream.</p> <p>The {{getAudioTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=] whose {{MediaStreamTrack/[[Kind]]}} is equal to <a>"audio"</a>. The conversion from the [=track set=] to the sequence is [=User Agent=] defined and the order does not have to be stable between calls.</p> </dd> <dt><dfn>getVideoTracks()</dfn></dt> <dd> <p>Returns a sequence of {{MediaStreamTrack}} objects representing the video tracks in this stream.</p> <p>The {{getVideoTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=] whose {{MediaStreamTrack/[[Kind]]}} is equal to <a>"video"</a>. The conversion from the [=track set=] to the sequence is [=User Agent=] defined and the order does not have to be stable between calls.</p> </dd> <dt><dfn>getTracks()</dfn></dt> <dd> <p>Returns a sequence of {{MediaStreamTrack}} objects representing all the tracks in this stream.</p> <p>The {{getTracks}} method MUST return a sequence that represents a snapshot of all the {{MediaStreamTrack}} objects in this stream's [=track set=], regardless of {{MediaStreamTrack/[[Kind]]}}. The conversion from the [=track set=] to the sequence is User Agent defined and the order does not have to be stable between calls.</p> </dd> <dt><dfn>getTrackById()</dfn></dt> <dd> <p>The {{getTrackById}} method MUST return either a {{MediaStreamTrack}} object from this stream's [=track set=] whose {{MediaStreamTrack/[[Id]]}} is equal to <var>trackId</var>, or <code>null</code>, if no such track exists.</p> </dd> <dt><dfn>addTrack()</dfn></dt> <dd> <p>Adds the given {{MediaStreamTrack}} to this {{MediaStream}}.</p> <p>When the {{addTrack}} method is invoked, the [=User Agent=] MUST run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>track</var> be the methods argument and <var>stream</var> the {{MediaStream}} object on which the method was called.</p> </li> <li> <p>If <var>track</var> is already in <var>stream</var>'s [=track set=], then abort these steps.</p> </li> <li> <p>[=MediaStream/Add a track|Add=] <var>track</var> to <var>stream</var>'s [=track set=].</p> </li> </ol> </dd> <dt><dfn>removeTrack()</dfn></dt> <dd> <p>Removes the given {{MediaStreamTrack}} object from this {{MediaStream}}.</p> <p>When the {{removeTrack}} method is invoked, the [=User Agent=] MUST run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>track</var> be the methods argument and <var>stream</var> the {{MediaStream}} object on which the method was called.</p> </li> <li> <p>If <var>track</var> is not in <var>stream's</var> [=track set=], then abort these steps.</p> </li> <li> <p>[=MediaStream/Remove a track|Remove=] <var>track</var> from <var>stream</var>'s [=track set=].</p> </li> </ol> </dd> <dt><dfn>clone()</dfn></dt> <dd> <p>Clones the given {{MediaStream}} and all its tracks.</p> <p>When the {{clone()}} method is invoked, the User Agent MUST run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>streamClone</var> be a newly constructed {{MediaStream}} object.</p> </li> <li> <p>Initialize <var>streamClone</var>.{{MediaStream.id}} to a newly generated value.</p> </li> <li> <p><a href="#track-clone">Clone each track</a> in this {{MediaStream}} object and add the result to <var>streamClone</var>'s <a>track set</a>.</p> </li> <li>Return <var>streamClone</var>.</li> </ol> </dd> </dl> </section> </div> </section> <section> <h2>{{MediaStreamTrack}}</h2> <p>A {{MediaStreamTrack}} object represents a media source in the [=User Agent=]. An example source is a device connected to the [=User Agent=]. Other specifications may define sources for {{MediaStreamTrack}} that override the behavior specified here. Several {{MediaStreamTrack}} objects can represent the same media source, e.g., when the user chooses the same camera in the UI shown by two consecutive calls to {{MediaDevices/getUserMedia()}}.</p> <p>A {{MediaStreamTrack}} source defines the following properties: <ol> <li>A source has a <dfn data-export>MediaStreamTrack source type</dfn>. It is set to either {{MediaStreamTrack}} or a subtype of {{MediaStreamTrack}}. By default, it is set to {{MediaStreamTrack}}. </li> <li>A source has <dfn data-export>MediaStreamTrack source-specific construction steps</dfn> that are executed when creating a {{MediaStreamTrack}} from a source. The steps take a newly created {{MediaStreamTrack}} as input. By default, the steps are empty.</li> <li>A source has <dfn data-export>MediaStreamTrack source-specific clone steps</dfn> that are executed when cloning a {{MediaStreamTrack}} of the given source. The steps take the source and destination {{MediaStreamTrack}}s as input. By default, the steps are empty.</li> </ol> </p> <p>The data from a {{MediaStreamTrack}} object does not necessarily have a canonical binary form; for example, it could just be "the video currently coming from the user's video camera". This allows [=User Agents=] to manipulate media in whatever fashion is most suitable on the user's platform.</p> <p>A script can indicate that a {{MediaStreamTrack}} object no longer needs its source with the {{MediaStreamTrack/stop()}} method. When all tracks using a source have been stopped or ended by some other means, the source is <dfn id="source-stopped" data-lt="source stopped state">stopped</dfn>. If the source is a device exposed by {{MediaDevices/getUserMedia()}}, then when the source is stopped, the [=User Agent=] MUST run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>mediaDevices</var> be the {{MediaDevices}} object in question.</p> <li> <p>Let <var>deviceId</var> be the source device's {{MediaDeviceInfo/deviceId}}.</p> </li> <li> <p>Set <var>mediaDevices</var>.{{MediaDevices/[[devicesLiveMap]]}}[<var>deviceId</var>] to <code>false</code>.</p> </li> <li> <p>If the [=permission state=] of the permission associated with the device's kind and <var>deviceId</var> for <var>mediaDevices</var>'s [=relevant settings object=], is not {{PermissionState/"granted"}}, then set <var>mediaDevices</var>.{{MediaDevices/[[devicesAccessibleMap]]}}[<var>deviceId</var>] to <code>false</code>.</p> </li> </ol> <p>To <dfn class="abstract-op">create a MediaStreamTrack</dfn> with an underlying <var>source</var>, and a <var>mediaDevicesToTieSourceTo</var>, run the following steps:</p> <ol class=algorithm> <li> <p>Let <var>track</var> be a new object of type <var>source</var>'s [=MediaStreamTrack source type=].</p> <p>Initialize track with the following internal slots:</p> <ul data-dfn-for="MediaStreamTrack"> <li> <p><dfn>[[\Source]]</dfn>, initialized to <var>source</var>.</p> </li> <li> <p><dfn>[[\Id]]</dfn>, initialized to a newly generated unique identifier string. See {{MediaStream.id}} attribute for guidelines on how to generate such an identifier.</p> </li> <li> <p><dfn>[[\Kind]]</dfn>, initialized to <dfn><code>"audio"</code></dfn> if <var>source</var> is an audio source, or <dfn><code>"video"</code></dfn> if <var>source</var> is a video source.</p> </li> <li> <p><dfn>[[\Label]]</dfn>, initialized to <var>source</var>'s label, if provided by the User Agent, or <code>""</code> otherwise. [=User Agents=] MAY label audio and video sources (e.g., "Internal microphone" or "External USB Webcam"). </p> </li> <li> <p><dfn>[[\ReadyState]]</dfn>, initialized to {{MediaStreamTrackState/"live"}}.</p> </li> <li> <p><dfn>[[\Enabled]]</dfn>, initialized to <code>true</code>.</p> </li> <li> <p><dfn>[[\Muted]]</dfn>, initialized to <code>true</code> if <var>source</var> is [= source/muted =], and <code>false</code> otherwise.</p> </li> <li> <a data-link-for="constrainable object" data-link-type="attribute">[[\Capabilities]]</a>, <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a>, and <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a>, all initialized as specified in the {{ConstrainablePattern}}.</p> </li> <li> <p><dfn class="export">[[\Restrictable]]</dfn>, initialized to <code>false</code>.</p> </li> </ul> </li> <li> <p>If <var>mediaDevicesToTieSourceTo</var> is not <code>null</code>, [=tie track source to `MediaDevices`=] with <var>source</var> and <var>mediaDevicesToTieSourceTo</var>.</p> </li> <li><p>Run <var>source</var>'s [=MediaStreamTrack source-specific construction steps=] with <var>track</var> as parameter.</p></li> <li><p>Return <var>track</var>.</p></li> </ol> <p>To <dfn class="abstract-op">initialize the underlying source</dfn> of <var>track</var> to <var>source</var>, run the following steps:</p> <ol class="algorithm"> <li><p>Initialize <var>track</var>.{{MediaStreamTrack/[[Source]]}} to <var>source</var>.</p></li> <li><p>Initialize <var>track</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Capabilities]]</a>, <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a>, and <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a>, as specified in the {{ConstrainablePattern}}.</p></li> </ol> <p>To <dfn class="abstract-op">tie track source to `MediaDevices`</dfn>, given <var>source</var> and <var>mediaDevices</var>, run the following steps: <ol class="algorithm"> <li> <p>Add <var>source</var> to <var>mediaDevices</var>.{{MediaDevices/[[mediaStreamTrackSources]]}}.</p> </li> </ol> <p>To <dfn class="abstract-op">stop all sources</dfn> of a [=global object=], named <var>globalObject</var>, the [=User Agent=] MUST run the following steps:</p> <ol class="algorithm"> <li><p>For each {{MediaStreamTrack}} object <var>track</var> whose <a data-cite="!HTML/#concept-relevant-global">relevant global object</a> is <var>globalObject</var>, set <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.</p></li> <li><p>If <var>globalObject</var> is a {{Window}}, then for each <var>source</var> in <var>globalObject</var>'s [=associated `MediaDevices`=].{{MediaDevices/[[mediaStreamTrackSources]]}}, [= source/stopped | stop =] <var>source</var>.</p></li> </ol> <p>The [=User Agent=] MUST [=stop all sources=] of a <var>globalObject</var> in the following conditions:</p> <ol> <li><p>If <var>globalObject</var> is a {{Window}} object and the [=unloading document cleanup steps=] are executed for its [=associated document=].</p></li> <li><p>If <var>globalObject</var> is a {{WorkerGlobalScope}} object and its <a data-cite="!HTML/workers.html#dom-workerglobalscope-closing">closing</a> flag is set to true.</p></li> </ol> <p>An implementation may use a per-source reference count to keep track of source usage, but the specifics are out of scope for this specification.</p> <p>To <dfn class="abstract-op" id="track-clone">clone a track</dfn> the [=User Agent=] MUST run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>track</var> be the {{MediaStreamTrack}} object to be cloned.</p> </li> <li> <p>Let <var>source</var> be <var>track</var>'s {{MediaStreamTrack/[[Source]]}}.</p> </li> <li> <p>Let <var>trackClone</var> be the result of [=create a MediaStreamTrack | creating a MediaStreamTrack=] with <var>source</var> and <code>null</code>. </p> </li> <li> <p>Set <var>trackClone</var>'s {{MediaStreamTrack/[[ReadyState]]}} to <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} value.</p> </li> <li> <p>Set <var>trackClone</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Capabilities]]</a> to a clone of <var>track</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Capabilities]]</a>.</p> </li> <li> <p>Set <var>trackClone</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a> to a clone of <var>track</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a>.</p> </li> <li> <p>Set <var>trackClone</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a> to a clone of <var>track</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a>.</p> </li> <li> <p>Run <var>source</var> [=MediaStreamTrack source-specific clone steps=] with <var>track</var> and <var>trackClone</var> as parameters.</p> </li> <li> <p>Return <var>trackClone</var>.</p> </li> </ol> <section> <h3>Media Flow and Life-cycle</h3> <section> <h4>Media Flow</h4> <p>There are two dimensions related to the media flow for a {{MediaStreamTrackState/"live"}} {{MediaStreamTrack}} : muted / not muted, and enabled / disabled.</p> <p><dfn class="export" data-dfn-for="MediaStreamTrack" data-dfn-type="dfn" id= "track-muted">Muted</dfn> refers to the input to the {{MediaStreamTrack}}. A {{MediaStreamTrack}} is [= MediaStreamTrack/muted =] when its source is <dfn data-dfn-for="source">muted</dfn>, i.e. temporarily unable to provide the track with data. Live samples MUST NOT be made available to a {{MediaStreamTrack}} while it is [=MediaStreamTrack/muted=].</p> <p>The [=MediaStreamTrack/muted=] state is outside the control of web applications, but can be observed by the application by reading the {{MediaStreamTrack/muted}} attribute and listening to the associated events {{mute}} and {{unmute}}. The reasons for a {{MediaStreamTrack}} to be muted are defined by its <a>source</a>.</p> <p>For camera and microphone sources, the reasons to [=source/muted|mute=] are [=implementation-defined=]. This allows user agents to implement privacy mitigations in situations like: the user pushing a physical mute button on the microphone, the user closing a laptop lid with an embedded camera, the user toggling a control in the operating system, the user clicking a mute button in the [=User Agent=] chrome, the [=User Agent=] (on behalf of the user) mutes, etc.</p> <p>On some operating systems, microphone access may get stolen from the [=User Agent=] when another application with higher-audio priority gets access to it, for instance in case of an incoming phone call on mobile OS. The [=User Agent=] SHOULD provide this information to the web application through {{MediaStreamTrack/muted}} and its associated events.</p> <p>Whenever the [=User Agent=] initiates such an [= implementation-defined=] change for camera or microphone sources, it MUST queue a task, using the user interaction task source, to [=MediaStreamTrack/set a track's muted state=] to the state desired by the user.</p> <div class="note">This does not apply to [=source|sources=] defined in other specifications. Other specifications need to define their own steps to [=MediaStreamTrack/set a track's muted state=] if desired.</div> <p>To <dfn class="export abstract-op" data-dfn-for="MediaStreamTrack" id="set-track-muted">set a track's muted state</dfn> to <var>newState</var>, the [=User Agent=] MUST run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>track</var> be the {{MediaStreamTrack}} in question.</p> </li> <li> <p>If <var>track</var>.{{MediaStreamTrack/[[Muted]]}} is already <var>newState</var>, then abort these steps.</p> </li> <li> <p>Set <var>track</var>.{{MediaStreamTrack/[[Muted]]}} to <var>newState</var>.</p> </li> <li> <p>If <var>newState</var> is <code>true</code> let <var>eventName</var> be {{mute}}, otherwise {{unmute}}.</p> </li> <li> <p>[=Fire an event=] named <var>eventName</var> on <var>track</var>.</p> </li> </ol> <p><dfn data-export id="track-enabled" data-dfn-for="MediaStreamTrack" data-dfn-type="dfn" data-lt="track enabled state|enabled" data-lt-noDefault>Enabled/disabled</dfn> on the other hand is available to the application to control (and observe) via the {{MediaStreamTrack/enabled}} attribute.</p> <p>The result for the consumer is the same in the sense that whenever {{MediaStreamTrack}} is muted or disabled (or both) the consumer gets zero-information-content, which means silence for audio and black frames for video. In other words, media from the source only flows when a {{MediaStreamTrack}} object is both unmuted and enabled. For example, a video element sourced by a {{MediaStream}} containing only muted or disabled {{MediaStreamTrack}}s for audio and video, is playing but rendering black video frames in silence.</p> <p>For a newly created {{MediaStreamTrack}} object, the following applies: the track is always enabled unless stated otherwise (for example when cloned) and the muted state reflects the state of the source at the time the track is created.</p> </section> <section> <h4>Life-cycle</h4> <p>A {{MediaStreamTrack}} has two states in its life-cycle: live and ended. A newly created {{MediaStreamTrack}} can be in either state depending on how it was created. For example, cloning an ended track results in a new ended track. The current state is reflected by the object's {{MediaStreamTrack/readyState}} attribute.</p> <p>In the live state, the track is active and media (or zero-information-content if the {{MediaStreamTrack}} is [= MediaStreamTrack/muted =] or [= MediaStreamTrack/enabled | disabled =]) is available for use by consumers.</p> <p>If the source is a device exposed by `navigator.mediaDevices.`{{MediaDevices/getUserMedia()}}, then when a track becomes either muted or disabled, and this brings all tracks connected to the device to be either muted, disabled, or stopped, then the UA MAY, using the device's {{MediaDeviceInfo/deviceId}}, <var>deviceId</var>, set `navigator.mediaDevices.`{{MediaDevices/[[devicesLiveMap]]}}[<var>deviceId</var>] to <code>false</code>, provided the UA sets it back to <code>true</code> as soon as any unstopped track connected to this device becomes un-muted or enabled again.</p> <p>When a {{MediaStreamTrackState/"live"}}, [= MediaStreamTrack/muted | unmuted =], and [= MediaStreamTrack/enabled =] track sourced by a device exposed by {{MediaDevices/getUserMedia()}} becomes either [= MediaStreamTrack/muted =] or [= MediaStreamTrack/enabled | disabled =], and this brings <em>all</em> tracks connected to the device (across all [=navigables=] the user agent operates) to be either muted, disabled, or stopped, then the UA SHOULD relinquish the device within 3 seconds while allowing time for a reasonably-observant user to become aware of the transition. The UA SHOULD attempt to reacquire the device as soon as any live track sourced by the device becomes both [= MediaStreamTrack/muted | unmuted =] and [= MediaStreamTrack/enabled =] again, provided that track's [=relevant global object=]'s [=associated `Document`=] [=Document/is in view=] at that time. If the document is not [=Document/is in view|in view=] at that time, the UA SHOULD instead queue a task to [=MediaStreamTrack/muted|mute=] the track, and not queue a task to [=MediaStreamTrack/muted|unmute=] it until the document comes [=Document/is in view|into view=]. If reacquiring the device fails, the UA MUST [= track ended by the User agent | end the track =] (The UA MAY end it earlier should it detect a device problem, like the device being physically removed).</p> <div class="note"> <p>The intent is to give users the assurance of privacy that having physical camera (and microphone) hardware lights off brings, by aligning physical and logical “privacy indicators”, at least while the current document is the sole user of a device.</p> <p>While other applications and documents using the device simultaneously may interfere with this intent at times, they do not interfere with the rules laid forth.</p> </div> <p>A {{MediaStreamTrack}} object is said to <em>end</em> when the source of the track is disconnected or exhausted.</p> <p>If all {{MediaStreamTrack}}s that are using the same source are [= MediaStreamTrack/ended =], the source will be [= source/stopped =].</p> <p>After the application has invoked the {{MediaStreamTrack/stop()}} method on a {{MediaStreamTrack}} object, or once the [=source=] of a {{MediaStreamTrack}} permanently ends production of live samples to its tracks, whichever is sooner, a {{MediaStreamTrack}} is said to be <dfn id="track-ended" data-dfn-for="MediaStreamTrack" data-dfn-type="dfn" data-export>ended</dfn>.</p> <p>For camera and microphone sources, the reasons for a source to [=MediaStreamTrack/ended|end=] besides {{MediaStreamTrack/stop()}} are [=implementation-defined=] (e.g., because the user rescinds the permission for the page to use the local camera, or because the User Agent has instructed the track to end for any reason).</p> <p>When a {{MediaStreamTrack}} <var>track</var> <dfn data-lt="track ended by the User agent" data-for=MediaStreamTrack data-export id="ends-nostop">ends for any reason other than the {{MediaStreamTrack/stop()}} method being invoked</dfn>, the [=User Agent=] MUST queue a task that runs the following steps:</p> <ol class="algorithm"> <li> <p>If <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} has the value {{MediaStreamTrackState/"ended"}} already, then abort these steps.</p> </li> <li> <p>Set <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.</p> </li> <li> <p>Notify <var>track</var>'s {{MediaStreamTrack/[[Source]]}} that <var>track</var> is [= MediaStreamTrack/ended =] so that the source may be [= source/stopped =], unless other {{MediaStreamTrack}} objects depend on it.</p> </li> <li> <p>[=Fire an event=] named <a data-link-type=event>ended</a> at the object.</p> </li> </ol> <p>If the end of the track was reached due to a user request, the event source for this event is the user interaction event source.</p> <p>To invoke the <dfn class="abstract-op">device permission revocation algorithm</dfn> with <var>permissionName</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>tracks</var> be the set of all currently {{MediaStreamTrackState/"live"}} <code>MediaStreamTrack</code>s whose permission associated with this kind of track (<a>"camera"</a> or <a>"microphone"</a>) matches <var>permissionName</var>.</p> </li> <li> <p>For each <var>track</var> in <var>tracks</var>, <a href="#ends-nostop">end</a> the track.</p> </li> </ol> </section> </section> <section> <h3>Tracks and Constraints</h3> <p>{{MediaStreamTrack}} is a <a>constrainable object</a> as defined in the <a href= "#constrainable-interface">Constrainable Pattern</a> section. Constraints are set on tracks and may affect sources.</p> <p>Whether <code><a>Constraints</a></code> were provided at track initialization time or need to be established later at runtime, the APIs defined in the <a>ConstrainablePattern</a> Interface allow the retrieval and manipulation of the constraints currently established on a track.</p> <p>Once ended, a track will continue exposing a <dfn id="list-of-inherent-constrainable-track-properties"> list of inherent constrainable track properties</dfn>. This list contains <code><a href="#def-constraint-deviceId">deviceId</a></code>, <code><a href="#def-constraint-facingMode">facingMode</a></code> and <code><a href="#def-constraint-groupId">groupId</a></code>. </p> </section> <section id="media-stream-track-interface-definition"> <h3>Interface Definition</h3> <div> <pre class="idl" >[Exposed=Window] interface MediaStreamTrack : EventTarget { readonly attribute DOMString kind; readonly attribute DOMString id; readonly attribute DOMString label; attribute boolean enabled; readonly attribute boolean muted; attribute EventHandler onmute; attribute EventHandler onunmute; readonly attribute MediaStreamTrackState readyState; attribute EventHandler onended; MediaStreamTrack clone(); undefined stop(); MediaTrackCapabilities getCapabilities(); MediaTrackConstraints getConstraints(); MediaTrackSettings getSettings(); Promise<undefined> applyConstraints(optional MediaTrackConstraints constraints = {}); };</pre> <section> <h2>Attributes</h2> <dl data-link-for="MediaStreamTrack" data-dfn-for= "MediaStreamTrack" class="attributes"> <dt>{{kind}} of type {{DOMString}}, readonly</dt> <dd> <p>The <dfn>kind</dfn> attribute MUST return [=this=].{{MediaStreamTrack/[[Kind]]}}.</p> </dd> <dt>{{id}} of type {{DOMString}}, readonly</dt> <dd> <p>The <dfn>id</dfn> attribute MUST return [=this=].{{MediaStreamTrack/[[Id]]}}.</p> </dd> <dt>{{label}} of type {{DOMString}}, readonly</dt> <dd> <p>The <dfn>label</dfn> attribute MUST return [=this=].{{MediaStreamTrack/[[Label]]}}.</p> </dd> <dt>{{enabled}} of type {{boolean}}</dt> <dd> <p>The <dfn id= "dom-mediastreamtrack-enabled">enabled</dfn> attribute controls the [= MediaStreamTrack/enabled =] state for the object.</p> <p>On getting, [=this=].{{MediaStreamTrack/[[Enabled]]}} MUST be returned. On setting, [=this=].{{MediaStreamTrack/[[Enabled]]}} MUST be set to the new value.</p> <p class="note">Thus, after a {{MediaStreamTrack}} has [= MediaStreamTrack/ended =], its {{MediaStreamTrack/enabled}} attribute still changes value when set; it just doesn't do anything with that new value.</p> </dd> <dt>{{muted}} of type {{boolean}}, readonly</dt> <dd> <p>The <dfn data-idl>muted</dfn> attribute reflects whether the track is [= MediaStreamTrack/muted =]. It MUST return [=this=].{{MediaStreamTrack/[[Muted]]}}.</p> </dd> <dt><dfn>onmute</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is <a data-link-type=event>mute</a>.</p> </dd> <dt><dfn>onunmute</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is <a data-link-type=event>unmute</a>.</p> </dd> <dt>{{readyState}} of type {{MediaStreamTrackState}}, readonly</dt> <dd> <p>On getting, the <dfn>readyState</dfn> attribute MUST return [=this=].{{MediaStreamTrack/[[ReadyState]]}}.</p> </dd> <dt><dfn>onended</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is <a data-link-type=event>ended</a>.</p> </dd> </dl> </section> <section> <h2>Methods</h2> <dl data-link-for="MediaStreamTrack" data-dfn-for= "MediaStreamTrack" class="methods"> <dt><dfn>clone</dfn></dt> <dd> <p>When the {{clone()}} method is invoked, the [=User Agent=] MUST return the result of [=clone a track=] with [=this=].</p> </dd> <dt><dfn>stop</dfn></dt> <dd> <p>When a {{MediaStreamTrack}} object's {{stop()}} method is invoked, the User Agent MUST run following steps:</p> <ol class="algorithm"> <li> <p>Let <var>track</var> be the current {{MediaStreamTrack}} object.</p> </li> <li> <p>If <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, then abort these steps.</p> </li> <li> <p>Notify <var>track</var>'s source that <var>track</var> is [= MediaStreamTrack/ended =].</p> <p>A source that is notified of a track ending will be [= source/stopped =], unless other {{MediaStreamTrack}} objects depend on it.</p> </li> <li> <p>Set <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} to {{MediaStreamTrackState/"ended"}}.</p> </li> </ol> </dd> <dt><dfn>getCapabilities</dfn></dt> <dd> <p>Returns the capabilites of the source that this {{MediaStreamTrack}}, the <a>constrainable object</a>, represents.</p> <p>See <a href="#constrainable-interface">ConstrainablePattern Interface</a> for the definition of this method.</p> <p class="fingerprint">Since this method gives likely persistent, cross-origin information about the underlying device, it adds to the fingerprint surface of the device.</p> </dd> <dt><dfn>getConstraints</dfn></dt> <dd> <p>See <a href="#constrainable-interface">ConstrainablePattern Interface</a> for the definition of this method.</p> </dd> <dt><dfn>getSettings</dfn></dt> <dd> <p>When a {{MediaStreamTrack}} object's {{MediaStreamTrack.getSettings()}} method is invoked, the [=User Agent=] MUST run following steps:</p> <ol class="algorithm"> <li> <p>Let <var>track</var> be the current {{MediaStreamTrack}} object.</p> </li> <li> <p>If <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, run the following sub steps:</p> <ol> <li><p>Let <var>settings</var> be a new {{MediaTrackSettings}} dictionary.</p></li> <li><p> For each <var>property</var> of the <a>list of inherent constrainable track properties</a>, add a corresponding property to <var>settings</var> if <var>track</var> had such property at the time it was ended, with the value at the time <var>track</var> was ended. </p></li> <li><p>Return <var>settings</var>.</p></li> </ol> </li> <li> <p>Return the current settings of the track as defined in <a href="#constrainable-interface">ConstrainablePattern Interface</a>.</p> </li> </ol> </dd> <dt><dfn>applyConstraints</dfn></dt> <dd> <p>When a {{MediaStreamTrack}} object's {{applyConstraints()}} method is invoked, the User Agent MUST run following steps:</p> <ol class="algorithm"> <li> <p>Let <var>track</var> be the current {{MediaStreamTrack}} object.</p> </li> <li> <p>If <var>track</var>'s {{MediaStreamTrack/[[ReadyState]]}} is {{MediaStreamTrackState/"ended"}}, run the following sub steps:</p> <ol> <li><p>Let <var>p</var> be a new promise.</p></li> <li><p>[= resolve =] <var>p</var> with <code>undefined</code>.</p></li> <li><p>Return <var>p</var>.</p></li> </ol> </li> <li> <p>Invoke and return the result of the <a>applyConstraints template method</a> where:</p> <ul> <li>In the <a>SelectSettings</a> algorithm, <ul> <li> <var>object</var> is the {{MediaStreamTrack}} on which this method was called, and </li> <li> <a>settings dictionary</a> refers to a possible instance of the {{MediaTrackSettings}} dictionary. The [=User Agent=] MUST NOT include inherent unchangeable device properties as members unless they are in the <a>list of inherent constrainable track properties</a>, or otherwise include device properties that <dfn>must not be exposed</dfn>. <p class="note">Other specifications may define constrainable properties that at times <a>must not be exposed</a>.</p> </li> </ul> </li> <li>In step 3 of the <a>ApplyConstraints algorithm</a>, all changes listed are to be made to <var>object</var>, and </li> <li>In step 4 of the <a>ApplyConstraints algorithm</a>, the requirement on getConstraints() applies to the getConstraints() method of <var>object</var>.</li> </ul> </li> </ol> </dd> </dl> </section> </div> <div> <pre class="idl" >enum MediaStreamTrackState { "live", "ended" };</pre> <table data-link-for="MediaStreamTrackState" data-dfn-for= "MediaStreamTrackState" class="simple"> <caption><dfn>MediaStreamTrackState</dfn> Enumeration description</caption> <thead> <tr> <th>Enum value</th><th>Description</th> </tr> </thead> <tbody> <tr> <td><dfn id= "idl-def-MediaStreamTrackState.live">live</dfn></td> <td> <p>The track is active (the track's underlying media source is making a best-effort attempt to provide data in real time).</p> <p>The output of a track in the {{MediaStreamTrackState/"live"}} state can be switched on and off with the {{MediaStreamTrack/enabled}} attribute.</p> </td> </tr> <tr> <td><dfn id= "idl-def-MediaStreamTrackState.ended">ended</dfn></td> <td> <p>The track has [= MediaStreamTrack/ended =] (the track's underlying media source is no longer providing data, and will never provide more data for this track). Once a track enters this state, it never exits it.</p> <p>For example, a video track in a {{MediaStream}} ends when the user unplugs the USB web camera that acts as the track's media source.</p> </td> </tr> </tbody> </table> </div> </section> <section id="media-track-supported-constraints"> <h2><dfn>MediaTrackSupportedConstraints</dfn></h2> <p>{{MediaTrackSupportedConstraints}} represents the list of constraints recognized by a [=User Agent=] for controlling the <a>Capabilities</a> of a {{MediaStreamTrack}} object. This dictionary is used as a function return value, and never as an operation argument.</p> <p>Future specifications can extend the {{MediaTrackSupportedConstraints}} dictionary by defining a partial dictionary with dictionary members of type {{boolean}}.</p> <p class="note">The constraints specified in this specification apply only to instances of {{MediaStreamTrack}} generated by {{MediaDevices.getUserMedia()}}, unless stated otherwise in other specifications.</p> <div> <pre class="idl" >dictionary MediaTrackSupportedConstraints { boolean width = true; boolean height = true; boolean aspectRatio = true; boolean frameRate = true; boolean facingMode = true; boolean resizeMode = true; boolean sampleRate = true; boolean sampleSize = true; boolean echoCancellation = true; boolean autoGainControl = true; boolean noiseSuppression = true; boolean latency = true; boolean channelCount = true; boolean deviceId = true; boolean groupId = true; boolean backgroundBlur = true; };</pre> <section> <h2>Dictionary {{MediaTrackSupportedConstraints}} Members</h2> <dl data-link-for="MediaTrackSupportedConstraints" data-dfn-for= "MediaTrackSupportedConstraints" class="dictionary-members"> <dt><dfn>width</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href="#def-constraint-width">width</a> for details.</dd> <dt><dfn>height</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href="#def-constraint-height">height</a> for details.</dd> <dt><dfn>aspectRatio</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-aspect">aspectRatio</a> for details.</dd> <dt><dfn>frameRate</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-frameRate">frameRate</a> for details.</dd> <dt><dfn>facingMode</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-facingMode">facingMode</a> for details.</dd> <dt><dfn>resizeMode</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-resizeMode">resizeMode</a> for details.</dd> <dt><dfn>sampleRate</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-sampleRate">sampleRate</a> for details.</dd> <dt><dfn>sampleSize</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-sampleSize">sampleSize</a> for details.</dd> <dt><dfn>echoCancellation</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-echoCancellation">echoCancellation</a> for details.</dd> <dt><dfn>autoGainControl</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-autoGainControl">autoGainControl</a> for details.</dd> <dt><dfn>noiseSuppression</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-noiseSuppression">noiseSuppression</a> for details.</dd> <dt><dfn>latency</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-latency">latency</a> for details.</dd> <dt><dfn>channelCount</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-channelCount">channelCount</a> for details.</dd> <dt><dfn>deviceId</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-deviceId">deviceId</a> for details.</dd> <dt><dfn>groupId</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-groupId">groupId</a> for details.</dd> <dt><dfn>backgroundBlur</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-backgroundBlur">backgroundBlur</a> for details.</dd> </dl> </section> </div> </section> <section id="media-track-capabilities"> <h2><dfn>MediaTrackCapabilities</dfn></h2> <p>{{MediaTrackCapabilities}} represents the <a>Capabilities</a> of a {{MediaStreamTrack}} object.</p> <p>Future specifications can extend the MediaTrackCapabilities dictionary by defining a partial dictionary with dictionary members of appropriate type.</p> <div> <pre class="idl" >dictionary MediaTrackCapabilities { ULongRange width; ULongRange height; DoubleRange aspectRatio; DoubleRange frameRate; sequence<DOMString> facingMode; sequence<DOMString> resizeMode; ULongRange sampleRate; ULongRange sampleSize; sequence<boolean> echoCancellation; sequence<boolean> autoGainControl; sequence<boolean> noiseSuppression; DoubleRange latency; ULongRange channelCount; DOMString deviceId; DOMString groupId; sequence<boolean> backgroundBlur; };</pre> <div class="note"> <p>For historical reasons, {{MediaTrackCapabilities/deviceId}} and {{MediaTrackCapabilities/groupId}} are {{DOMString}} instead of the `sequence<DOMString>` expected by {{Capabilities}} in the <code><a>ConstrainablePattern</a></code>.</p> </div> <section> <h2>Dictionary {{MediaTrackCapabilities}} Members</h2> <dl data-link-for="MediaTrackCapabilities" data-dfn-for= "MediaTrackCapabilities" class="dictionary-members"> <dt><dfn>width</dfn> of type {{ULongRange}}</dt> <dd>See <a href="#def-constraint-width">width</a> for details.</dd> <dt><dfn>height</dfn> of type {{ULongRange}}</dt> <dd>See <a href="#def-constraint-height">height</a> for details.</dd> <dt><dfn>aspectRatio</dfn> of type {{DoubleRange}}</dt> <dd>See <a href= "#def-constraint-aspect">aspectRatio</a> for details.</dd> <dt><dfn>frameRate</dfn> of type {{DoubleRange}}</dt> <dd>See <a href= "#def-constraint-frameRate">frameRate</a> for details.</dd> <dt><dfn>facingMode</dfn> of type <code>sequence<{{DOMString}}></code></dt> <dd> <p>A camera can report multiple facing modes. For example, in a high-end telepresence solution with several cameras facing the user, a camera to the left of the user can report both {{VideoFacingModeEnum/"left"}} and {{VideoFacingModeEnum/"user"}}. See <a href= "#def-constraint-facingMode">facingMode</a> for additional details.</p> </dd> <dt><dfn>resizeMode</dfn> of type <code>sequence<{{DOMString}}></code></dt> <dd> <p>The [=User Agent=] MAY use cropping and downscaling to offer more resolution choices than this camera naturally produces. The reported sequence MUST list all the means the UA may employ to derive resolution choices for this camera. The value {{VideoResizeModeEnum/"none"}} MUST be present, indicating the ability to constrain the UA from cropping and downscaling. See <a href= "#def-constraint-resizeMode">resizeMode</a> for additional details.</p> </dd> <dt><dfn>sampleRate</dfn> of type {{ULongRange}}</dt> <dd>See <a href= "#def-constraint-sampleRate">sampleRate</a> for details.</dd> <dt><dfn>sampleSize</dfn> of type {{ULongRange}}</dt> <dd>See <a href= "#def-constraint-sampleSize">sampleSize</a> for details.</dd> <dt><dfn>echoCancellation</dfn> of type <code>sequence<{{boolean}}></code></dt> <dd> <p>If the source cannot do echo cancellation a single <code>false</code> is reported. If echo cancellation cannot be turned off, a single <code>true</code> is reported. If the script can control the feature, the source reports a list with both <code>true</code> and <code>false</code> as possible values. See <a href= "#def-constraint-echoCancellation">echoCancellation</a> for additional details.</p> </dd> <dt><dfn>autoGainControl</dfn> of type <code>sequence<{{boolean}}></code></dt> <dd> <p>If the source cannot do auto gain control a single <code>false</code> is reported. If auto gain control cannot be turned off, a single <code>true</code> is reported. If the script can control the feature, the source reports a list with both <code>true</code> and <code>false</code> as possible values. See <a href= "#def-constraint-autoGainControl">autoGainControl</a> for additional details.</p> </dd> <dt><dfn>noiseSuppression</dfn> of type <code>sequence<{{boolean}}></code></dt> <dd> <p>If the source cannot do noise suppression a single <code>false</code> is reported. If noise suppression cannot be turned off, a single <code>true</code> is reported. If the script can control the feature, the source reports a list with both <code>true</code> and <code>false</code> as possible values. See <a href= "#def-constraint-noiseSuppression">noiseSuppression</a> for additional details.</p> </dd> <dt><dfn>latency</dfn> of type {{DoubleRange}}</dt> <dd>See <a href= "#def-constraint-latency">latency</a> for details.</dd> <dt><dfn>channelCount</dfn> of type {{ULongRange}}</dt> <dd>See <a href= "#def-constraint-channelCount">channelCount</a> for details.</dd> <dt><dfn>deviceId</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-deviceId">deviceId</a> for details.</dd> <dt><dfn>groupId</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-groupId">groupId</a> for details.</dd> <dt><dfn>backgroundBlur</dfn> of type <code>sequence<{{boolean}}></code></dt> <dd>If the source does not have built-in background blurring, a single <code>false</code> is reported. If background blurring cannot be turned off, a single <code>true</code> is reported. If the script can control the feature, the source reports a list with both true and false as possible values. See <a href= "#def-constraint-backgroundBlur">backgroundBlur</a> for details.</dd> </dl> </section> </div> </section> <section id="media-track-constraints"> <h2><dfn>MediaTrackConstraints</dfn></h2> <div> <pre class="idl" >dictionary MediaTrackConstraints : MediaTrackConstraintSet { sequence<MediaTrackConstraintSet> advanced; };</pre> <section> <h2>Dictionary {{MediaTrackConstraints}} Members</h2> <dl data-link-for="MediaTrackConstraints" data-dfn-for= "MediaTrackConstraints" class="dictionary-members"> <dt><dfn>advanced</dfn> of type <code>sequence<{{MediaTrackConstraintSet}}></code></dt> <dd> <p>See <a href="#constraints">Constraints and ConstraintSet</a> for the definition of this element.</p> </dd> </dl> </section> </div> <p>Future specifications can extend the <dfn>MediaTrackConstraintSet</dfn> dictionary by defining a partial dictionary with dictionary members of appropriate type.</p> <div> <pre class="idl" >dictionary MediaTrackConstraintSet { ConstrainULong width; ConstrainULong height; ConstrainDouble aspectRatio; ConstrainDouble frameRate; ConstrainDOMString facingMode; ConstrainDOMString resizeMode; ConstrainULong sampleRate; ConstrainULong sampleSize; ConstrainBoolean echoCancellation; ConstrainBoolean autoGainControl; ConstrainBoolean noiseSuppression; ConstrainDouble latency; ConstrainULong channelCount; ConstrainDOMString deviceId; ConstrainDOMString groupId; ConstrainBoolean backgroundBlur; };</pre> <section> <h2>Dictionary {{MediaTrackConstraintSet}} Members</h2> <dl data-link-for="MediaTrackConstraintSet" data-dfn-for= "MediaTrackConstraintSet" class="dictionary-members"> <dt><dfn>width</dfn> of type {{ConstrainULong}}</dt> <dd>See <a href="#def-constraint-width">width</a> for details.</dd> <dt><dfn>height</dfn> of type {{ConstrainULong}}</dt> <dd>See <a href="#def-constraint-height">height</a> for details.</dd> <dt><dfn>aspectRatio</dfn> of type {{ConstrainDouble}}</dt> <dd>See <a href= "#def-constraint-aspect">aspectRatio</a> for details.</dd> <dt><dfn>frameRate</dfn> of type {{ConstrainDouble}}</dt> <dd>See <a href= "#def-constraint-frameRate">frameRate</a> for details.</dd> <dt><dfn>facingMode</dfn> of type {{ConstrainDOMString}}</dt> <dd>See <a href= "#def-constraint-facingMode">facingMode</a> for details.</dd> <dt><dfn>resizeMode</dfn> of type {{ConstrainDOMString}}</dt> <dd>See <a href= "#def-constraint-resizeMode">resizeMode</a> for details.</dd> <dt><dfn>sampleRate</dfn> of type {{ConstrainULong}}</dt> <dd>See <a href= "#def-constraint-sampleRate">sampleRate</a> for details.</dd> <dt><dfn>sampleSize</dfn> of type {{ConstrainULong}}</dt> <dd>See <a href= "#def-constraint-sampleSize">sampleSize</a> for details.</dd> <dt><dfn>echoCancellation</dfn> of type {{ConstrainBoolean}}</dt> <dd>See <a href= "#def-constraint-echoCancellation">echoCancellation</a> for details.</dd> <dt><dfn>autoGainControl</dfn> of type {{ConstrainBoolean}}</dt> <dd>See <a href= "#def-constraint-autoGainControl">autoGainControl</a> for details.</dd> <dt><dfn>noiseSuppression</dfn> of type {{ConstrainBoolean}}</dt> <dd>See <a href= "#def-constraint-noiseSuppression">noiseSuppression</a> for details.</dd> <dt><dfn>latency</dfn> of type {{ConstrainDouble}}</dt> <dd>See <a href= "#def-constraint-latency">latency</a> for details.</dd> <dt><dfn>channelCount</dfn> of type {{ConstrainULong}}</dt> <dd>See <a href= "#def-constraint-channelCount">channelCount</a> for details.</dd> <dt><dfn>deviceId</dfn> of type {{ConstrainDOMString}}</dt> <dd>See <a href= "#def-constraint-deviceId">deviceId</a> for details.</dd> <dt><dfn>groupId</dfn> of type {{ConstrainDOMString}}</dt> <dd>See <a href= "#def-constraint-groupId">groupId</a> for details.</dd> <dt><dfn>backgroundBlur</dfn> of type {{ConstrainBoolean}}</dt> <dd>See <a href= "#def-constraint-backgroundBlur">backgroundBlur</a> for details.</dd> </dl> </section> </div> </section> <section id="media-track-settings"> <h2><dfn>MediaTrackSettings</dfn></h2> <p>{{MediaTrackSettings}} represents the <a>Settings</a> of a {{MediaStreamTrack}} object.</p> <p>Future specifications can extend the MediaTrackSettings dictionary by defining a partial dictionary with dictionary members of appropriate type.</p> <div> <pre class="idl" >dictionary MediaTrackSettings { unsigned long width; unsigned long height; double aspectRatio; double frameRate; DOMString facingMode; DOMString resizeMode; unsigned long sampleRate; unsigned long sampleSize; boolean echoCancellation; boolean autoGainControl; boolean noiseSuppression; double latency; unsigned long channelCount; DOMString deviceId; DOMString groupId; boolean backgroundBlur; };</pre> <section> <h2>Dictionary {{MediaTrackSettings}} Members</h2> <dl data-link-for="MediaTrackSettings" data-dfn-for= "MediaTrackSettings" class="dictionary-members"> <dt><dfn>width</dfn> of type {{unsigned long}}</dt> <dd>See <a href="#def-constraint-width">width</a> for details.</dd> <dt><dfn>height</dfn> of type {{unsigned long}}</dt> <dd>See <a href="#def-constraint-height">height</a> for details.</dd> <dt><dfn>aspectRatio</dfn> of type {{double}}</dt> <dd>See <a href= "#def-constraint-aspect">aspectRatio</a> for details.</dd> <dt><dfn>frameRate</dfn> of type {{double}}</dt> <dd>See <a href= "#def-constraint-frameRate">frameRate</a> for details.</dd> <dt><dfn>facingMode</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-facingMode">facingMode</a> for details.</dd> <dt><dfn>resizeMode</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-resizeMode">resizeMode</a> for details.</dd> <dt><dfn>sampleRate</dfn> of type {{unsigned long}}</dt> <dd>See <a href= "#def-constraint-sampleRate">sampleRate</a> for details.</dd> <dt><dfn>sampleSize</dfn> of type {{unsigned long}}</dt> <dd>See <a href= "#def-constraint-sampleSize">sampleSize</a> for details.</dd> <dt><dfn>echoCancellation</dfn> of type {{boolean}}</dt> <dd>See <a href= "#def-constraint-echoCancellation">echoCancellation</a> for details.</dd> <dt><dfn>autoGainControl</dfn> of type {{boolean}}</dt> <dd>See <a href= "#def-constraint-autoGainControl">autoGainControl</a> for details.</dd> <dt><dfn>noiseSuppression</dfn> of type {{boolean}}</dt> <dd>See <a href= "#def-constraint-noiseSuppression">noiseSuppression</a> for details.</dd> <dt><dfn>latency</dfn> of type {{double}}</dt> <dd>See <a href= "#def-constraint-latency">latency</a> for details.</dd> <dt><dfn>channelCount</dfn> of type {{unsigned long}}</dt> <dd>See <a href= "#def-constraint-channelCount">channelCount</a> for details.</dd> <dt><dfn>deviceId</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-deviceId">deviceId</a> for details.</dd> <dt><dfn>groupId</dfn> of type {{DOMString}}</dt> <dd>See <a href= "#def-constraint-groupId">groupId</a> for details.</dd> <dt><dfn>backgroundBlur</dfn> of type {{boolean}}, defaulting to <code>true</code></dt> <dd>See <a href= "#def-constraint-backgroundBlur">backgroundBlur</a> for details.</dd> </dl> </section> </div> </section> <section> <h2>Constrainable Properties</h2> <p>The names of the initial set of constrainable properties for MediaStreamTrack are defined below.</p> <p>The following constrainable properties are defined to apply to both video and audio {{MediaStreamTrack}} objects:</p> <table class="simple"> <thead> <tr> <th>Property Name</th> <th>Type</th> <th>Notes</th> </tr> </thead> <tbody> <tr id="def-constraint-deviceId"> <td><dfn>deviceId</dfn></td> <td>{{DOMString}}</td> <td> The identifier of the device generating the content of the {{MediaStreamTrack}}. It conforms with the definition of {{MediaDeviceInfo.deviceId}}. Note that the setting of this property is uniquely determined by the source that is attached to the {{MediaStreamTrack}}. In particular, {{MediaStreamTrack/getCapabilities()}} will return only a single value for deviceId. This property can therefore be used for initial media selection with {{MediaDevices/getUserMedia()}}. However, it is not useful for subsequent media control with {{MediaStreamTrack/applyConstraints()}}, since any attempt to set a different value will result in an unsatisfiable <a>ConstraintSet</a>. If a string of length 0 is used as a deviceId value constraint with {{MediaDevices/getUserMedia()}}, it MAY be interpreted as if the constraint is not specified. </td> </tr> <tr id="def-constraint-groupId"> <td><dfn>groupId</dfn></td> <td>{{DOMString}}</td> <td> The [=document=]-unique group identifier for the device generating the content of the {{MediaStreamTrack}}. It conforms with the definition of {{MediaDeviceInfo.groupId}}. Note that the setting of this property is uniquely determined by the source that is attached to the {{MediaStreamTrack}}. In particular, {{MediaStreamTrack/getCapabilities()}} will return only a single value for groupId. Since this property is not stable between browsing sessions, its usefulness for initial media selection with {{MediaDevices/getUserMedia()}} is limited. It is not useful for subsequent media control with {{MediaStreamTrack/applyConstraints()}}, since any attempt to set a different value will result in an unsatisfiable <a>ConstraintSet</a>. </td> </tr> </tbody> </table> <p>The following constrainable properties are defined to apply only to video {{MediaStreamTrack}} objects:</p> <table class="simple"> <thead> <tr> <th>Property Name</th> <th>Type</th> <th>Notes</th> </tr> </thead> <tbody> <tr id="def-constraint-width"> <td><dfn>width</dfn></td> <td>{{unsigned long}}</td> <td>The width, in pixels. As a capability, its valid range should span the video source's pre-set width values with min being equal to 1 and max being the largest width. The [=User Agent=] MUST support downsampling to any value between the min width range value and the native resolution width.</td> </tr> <tr id="def-constraint-height"> <td><dfn>height</dfn></td> <td>{{unsigned long}}</td> <td>The height, in pixels. As a capability, its valid range should span the video source's pre-set height values with min being equal to 1 and max being the largest height. The [=User Agent=] MUST support downsampling to any value between the min height range value and the native resolution height.</td> </tr> <tr id="def-constraint-frameRate"> <td><dfn>frameRate</dfn></td> <td>{{double}}</td> <td><p>The frame rate (frames per second). If video source's pre-set can determine frame rates, then, as a capability, its valid range should span the video source's pre-set frame rate values with min being equal to 0 and max being the largest frame rate. The [=User Agent=] MUST support frame rates obtained from integral decimation of the native resolution frame rate. If frame rate cannot be determined (e.g. the source does not natively provide a frame rate, or the frame rate cannot be determined from the source stream), then the capability values MUST refer to the [=User Agent=]'s vsync display rate.</p> <p>As a setting, this value represents the configured frame rate. If decimation is used, this is that value rather than the native frame rate. For example, if the setting is 25 frames per second via decimation, the native frame rate of the camera is 30 frames per second but due to lighting conditions only 20 frames per second is achieved, {{frameRate}} reports the setting: 25 frames per second.</p></td> </tr> <tr id="def-constraint-aspect"> <td><dfn>aspectRatio</dfn></td> <td>{{double}}</td> <td>The exact aspect ratio (width in pixels divided by height in pixels, represented as a double rounded to the tenth decimal place) or aspect ratio range.</td> </tr> <tr id="def-constraint-facingMode"> <td><dfn>facingMode</dfn></td> <td>{{DOMString}}</td> <td>This string is one of the members of {{VideoFacingModeEnum}}. The members describe the directions that the camera can face, as seen from the user's perspective. Note that <code><a data-link-for= "ConstrainablePattern">getConstraints</a></code> may not return exactly the same string for strings not in this enum. This preserves the possibility of using a future version of WebIDL enum for this property.</td> </tr> <tr id="def-constraint-resizeMode"> <td><dfn>resizeMode</dfn></td> <td>{{DOMString}}</td> <td> This string is one of the members of {{VideoResizeModeEnum}}. The members describe the means by which the resolution can be derived by the UA. In other words, whether the UA is allowed to use cropping and downscaling on the camera output. <p class="fingerprint">The UA MAY disguise concurrent use of the camera, by downscaling, upscaling, and/or cropping to mimic native resolutions when "none" is used, but only when the camera is in use in another application outside the [=User Agent=].</p>Note that <code><a data-link-for= "ConstrainablePattern">getConstraints</a></code> may not return exactly the same string for strings not in this enum. This preserves the possibility of using a future version of WebIDL enum for this property. </td> </tr> <tr> <td><dfn id="def-constraint-backgroundBlur">backgroundBlur</dfn></td> <td>{{boolean}}</td> <td> Some platforms or User Agents may provide built-in support for background blurring of video frames, in particular for camera video streams. Web applications may either want to control or at least be aware that background blur is applied at the source level. This may for instance allow the web application to update its UI or to not apply background blur on its own. </td> </tr> </tbody> </table> <p>On systems where it's desirable to sometimes automatically flip the X and Y axis of the resulting captured video in response to ongoing environmental factors, the {{width}}, {{height}} and {{aspectRatio}} constraints and capabilities MUST remain unaffected in all algorithms and be considered in the <a>primary orientation</a> only, except for the {{MediaStreamTrack/getSettings()}} algorithm where settings for these constrainable properties MUST be flipped if necessary to match the returned dimensions of the captured video at any point in time.</p> <p>The <dfn>primary orientation</dfn> of a system that supports flipping the X and Y axis of resulting captured video is defined by the User Agent for the particular system.</p> <div class="note"> <p>On systems that support automatic switching between landscape and portrait mode, [=User Agents=] are encouraged to make landscape mode the <a>primary orientation</a>.</p> </div> <div> <pre class="idl" >enum VideoFacingModeEnum { "user", "environment", "left", "right" };</pre> <table data-link-for="VideoFacingModeEnum" data-dfn-for= "VideoFacingModeEnum" class="simple"> <caption><dfn>VideoFacingModeEnum</dfn> Enumeration description</caption> <thead> <tr> <th>Enum value</th><th>Description</th> </tr> </thead> <tbody> <tr> <td><dfn><code id= "idl-def-VideoFacingModeEnum.user">user</code></dfn></td> <td> <p>The source is facing toward the user (a self-view camera).</p> </td> </tr> <tr> <td><dfn><code id= "idl-def-VideoFacingModeEnum.environment">environment</code></dfn></td> <td> <p>The source is facing away from the user (viewing the environment).</p> </td> </tr> <tr> <td><dfn><code id= "idl-def-VideoFacingModeEnum.left">left</code></dfn></td> <td> <p>The source is facing to the left of the user.</p> </td> </tr> <tr> <td><dfn><code id= "idl-def-VideoFacingModeEnum.right">right</code></dfn></td> <td> <p>The source is facing to the right of the user.</p> </td> </tr> </tbody> </table> </div> <p>Below is an illustration of the video facing modes in relation to the user.<br> <img alt="Illustration of video facing modes in relation to user" src= "images/camera-names-exp.svg" style="width:40%"></p> <div> <pre class="idl" >enum VideoResizeModeEnum { "none", "crop-and-scale" };</pre> <table data-link-for="VideoResizeModeEnum" data-dfn-for= "VideoResizeModeEnum" class="simple"> <caption><dfn>VideoResizeModeEnum</dfn> Enumeration description</caption> <thead> <tr> <th>Enum value</th><th>Description</th> </tr> </thead> <tbody> <tr> <td><dfn id= "idl-def-VideoResizeModeEnum.user">none</dfn></td> <td> <p>This resolution and frame rate is offered by the camera, its driver, or the OS.</p> <p class="fingerprint">Note: The UA MAY report this value to disguise concurrent use, but only when the camera is in use in another [=navigable=].</p> </td> </tr> <tr> <td><dfn id= "idl-def-VideoResizeModeEnum.right">crop-and-scale</dfn></td> <td> <p>This resolution is downscaled and/or cropped from a higher camera resolution by the [=User Agent=], or its frame rate is decimated by the [=User Agent=]. The media MUST NOT be upscaled, stretched or have fake data created that did not occur in the input source, except as noted below.</p> <p class="fingerprint">Note: The UA MAY upscale to disguise concurrent use, but only when the camera is in use in another application outside the [=User Agent=].</p> </td> </tr> </tbody> </table> </div> <p>The following constrainable properties are defined to apply only to audio {{MediaStreamTrack}} objects:</p> <table class="simple"> <thead> <tr> <th>Property Name</th> <th>Values</th> <th>Notes</th> </tr> </thead> <tbody> <tr id="def-constraint-sampleRate"> <td><dfn>sampleRate</dfn></td> <td>{{unsigned long}}</td> <td>The sample rate in samples per second for the audio data.</td> </tr> <tr id="def-constraint-sampleSize"> <td><dfn>sampleSize</dfn></td> <td>{{unsigned long}}</td> <td>The linear sample size in bits. As a constraint, it can only be satisfied for audio devices that produce linear samples.</td> </tr> <tr id="def-constraint-echoCancellation"> <td><dfn>echoCancellation</dfn></td> <td>{{boolean}}</td> <td>When one or more audio streams is being played in the processes of various microphones, it is often desirable to attempt to remove all the sound being played from the input signals recorded by the microphones. This is referred to as echo cancellation. There are cases where it is not needed and it is desirable to turn it off so that no audio artifacts are introduced. This allows applications to control this behavior.</td> </tr> <tr id="def-constraint-autoGainControl"> <td><dfn>autoGainControl</dfn></td> <td>{{boolean}}</td> <td>Automatic gain control is often desirable on the input signal recorded by the microphone. There are cases where it is not needed and it is desirable to turn it off so that the audio is not altered. This allows applications to control this behavior.</td> </tr> <tr id="def-constraint-noiseSuppression"> <td><dfn>noiseSuppression</dfn></td> <td>{{boolean}}</td> <td>Noise suppression is often desirable on the input signal recorded by the microphone. There are cases where it is not needed and it is desirable to turn it off so that the audio is not altered. This allows applications to control this behavior.</td> </tr> <tr id="def-constraint-latency"> <td><dfn>latency</dfn></td> <td>{{double}}</td> <td>The latency or latency range, in seconds. The latency is the time between start of processing (for instance, when sound occurs in the real world) to the data being available to the next step in the process. Low latency is critical for some applications; high latency may be acceptable for other applications because it helps with power constraints. The number is expected to be the target latency of the configuration; the actual latency may show some variation from that.</td> </tr> <tr id="def-constraint-channelCount"> <td><dfn>channelCount</dfn></td> <td>{{unsigned long}}</td> <td>The number of independent channels of sound that the audio data contains, i.e. the number of audio samples per sample frame.</td> </tr> </tbody> </table> </section> </section> <section> <h3>{{MediaStreamTrackEvent}}</h3> <p>The {{addtrack}} and {{removetrack}} events use the {{MediaStreamTrackEvent}} interface.</p> <p>The {{addtrack}} and {{removetrack}} events notify the script that the [=track set=] of a {{MediaStream}} has been updated by the [=User Agent=].</p> <p><dfn data-lt="Fire a track event">Firing a track event named <var>e</var></dfn> with a {{MediaStreamTrack}} <var>track</var> means that an event with the name <var>e</var>, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the {{MediaStreamTrackEvent}} interface with the {{MediaStreamTrackEvent/track}} attribute set to <var>track</var>, MUST be created and dispatched at the given target.</p> <div> <pre class="idl" >[Exposed=Window] interface MediaStreamTrackEvent : Event { constructor(DOMString type, MediaStreamTrackEventInit eventInitDict); [SameObject] readonly attribute MediaStreamTrack track; };</pre> <section> <h2>Constructors</h2> <dl data-link-for="MediaStreamTrackEvent" data-dfn-for= "MediaStreamTrackEvent" class="constructors"> <dt><dfn>constructor()</dfn></dt> <dd> <p>Constructs a new {{MediaStreamTrackEvent}}.</p> </dd> </dl> </section> <section> <h2>Attributes</h2> <dl data-link-for="MediaStreamTrackEvent" data-dfn-for= "MediaStreamTrackEvent" class="attributes"> <dt>{{track}} of type {{MediaStreamTrack}}, readonly</dt> <dd> <p>The <dfn id= "dom-mediastreamtrackevent-track">track</dfn> attribute represents the {{MediaStreamTrack}} object associated with the event.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary MediaStreamTrackEventInit : EventInit { required MediaStreamTrack track; };</pre> <section> <h2>Dictionary <dfn>MediaStreamTrackEventInit</dfn> Members</h2> <dl data-link-for="MediaStreamTrackEventInit" data-dfn-for= "MediaStreamTrackEventInit" class="dictionary-members"> <dt><dfn>track</dfn> of type {{MediaStreamTrack}}, required</dt> <dd></dd> </dl> </section> </div> </section> </section> <section class="informative" id= "the-model-sources-sinks-constraints-and-settings" data-cite="?webrtc"> <h2>The model: sources, sinks, constraints, and settings</h2> <p>[=User Agents=] provide a media pipeline from sources to sinks. In a [=User Agent=], sinks are the <[^img^]>, <[^video^]>, and <[^audio^]> tags. Traditional sources include streamed content, files, and web resources. The media produced by these sources typically does not change over time - these sources can be considered to be static.</p> <p>The sinks that display these sources to the user (the actual tags themselves) have a variety of controls for manipulating the source content. For example, an <[^img^]> tag scales down a huge source image of 1600x1200 pixels to fit in a rectangle defined with <code>width="400"</code> and <code>height="300"</code>.</p> <p>Sources have a lifetime. By default, a source lifetime is tied to the context that created it. For instance, sources created by {{MediaDevices.getUserMedia()}} are considered as created by its navigator.{{mediaDevices}} context. Similarly, sources of {{RTCRtpReceiver}} objects are bound to the {{RTCPeerConnection}} itself, which is bound to its creation context. Except if stated explicitly in the definition of specific sources, a source is always [= source/stopped =] when its creation context goes away. It should be noted that two sources of different contexts may use the same capture device at the same time. One source can be stopped independently of the other one. </p> <p>The getUserMedia API adds dynamic sources such as microphones and cameras - the characteristics of these sources can change in response to application needs. These sources can be considered to be dynamic in nature. A <[^video^]> element that displays media from a dynamic source can either perform scaling or it can feed back information along the media pipeline and have the source produce content more suitable for display.</p> <div class="note"> <p><strong>Note:</strong> This sort of feedback loop is obviously just enabling an "optimization", but it's a non-trivial gain. This optimization can save battery, allow for less network congestion, etc...</p> </div> <p>Note that {{MediaStream}} sinks (such as <[^video^]>, <[^audio^]>, and even {{RTCPeerConnection}}) will continue to have mechanisms to further transform the source stream beyond that which the <a>Settings</a>, <a>Capabilities</a>, and <a>Constraints</a> described in this specification offer. (The sink transformation options, including those of {{RTCPeerConnection}}, are outside the scope of this specification.)</p> <p>The act of changing or applying a track constraint may affect the [=settings=] of all tracks sharing that source and consequently all down-level sinks that are using that source. Many sinks may be able to take these changes in stride, such as the <code><[^video^]></code> element or {{RTCPeerConnection}}. Others like the Recorder API may fail as a result of a source setting change.</p> <p>The {{RTCPeerConnection}} is an interesting object because it acts simultaneously as both a sink <strong>and</strong> a source for over-the-network streams. As a sink, it has source transformational capabilities (e.g., lowering bit-rates, scaling-up / down resolutions, and adjusting frame-rates), and as a source it could have its own settings changed by a track source.</p> <p>To illustrate how changes to a given source impact various sinks, consider the following example. This example only uses width and height, but the same principles apply to all of the <a>Settings</a> exposed in this specification. In the first figure a home client has obtained a video source from its local video camera. The source's width and height settings are 800 pixels and 600 pixels, respectively. Three {{MediaStream}} objects on the home client contain tracks that use this same <{{MediaStreamTrack/deviceId}}. The three media streams are connected to three different sinks: a <code><[^video^]></code> element (A), another <code><[^video^]></code> element (B), and a peer connection (C). The peer connection is streaming the source video to a remote client. On the remote client there are two media streams with tracks that use the peer connection as a source. These two media streams are connected to two <code><[^video^]></code> element sinks (Y and Z).</p><img alt= "Changing media stream source effects: before the requested change" src= "images/change_states_before.svg"> <p>Note that at this moment, all of the sinks on the home client must apply a transformation to the original source's provided dimension settings. B is scaling the video down, A is scaling the video up (resulting in loss of quality), and C is also scaling the video up slightly for sending over the network. On the remote client, sink Y is scaling the video <em>way</em> down, while sink Z is not applying any scaling.</p> <p>In response to {{MediaStreamTrack/applyConstraints()}} being called, one of the tracks wants a higher resolution (1920 by 1200 pixels) from the home client's video source.</p><img alt= "Changing media stream source effects: after the requested change" src= "images/change_states_after.svg"> <p>Note that the source change immediately affects all of the tracks and sinks on the home client, but does not impact any of the sinks (or sources) on the remote client. With the increase in the home client source video's dimensions, sink A no longer has to perform any scaling, while sink B must scale down even further than before. Sink C (the peer connection) must now scale down the video in order to keep the transmission constant to the remote client.</p> <p>While not shown, an equally valid settings change request could be made on the remote client's side. In addition to impacting sink Y and Z in the same manner as A, B and C were impacted earlier, it could lead to re-negotiation with the peer connection on the home client in order to alter the transformation that it is applying to the home client's video source. Such a change is NOT REQUIRED to change anything related to sink A or B or the home client's video source.</p> <p>Note that this specification does not define a mechanism by which a change to the remote client's video source could automatically trigger a change to the home client's video source. Implementations may choose to make such source-to-sink optimizations as long as they only do so within the constraints established by the application, as the next example demonstrates.</p> <p>It is fairly obvious that changes to a given source will impact sink consumers. However, in some situations changes to a given sink may also cause implementations to adjust a source's settings. This is illustrated in the following figures. In the first figure below, the home client's video source is sending a video stream sized at 1920 by 1200 pixels. The video source is also unconstrained, such that the exact source dimensions are flexible as far as the application is concerned. Two {{MediaStream}} objects contain tracks with the same {{MediaStreamTrack/deviceId}}, and those {{MediaStream}}s are connected to two different <code><[^video^]></code> element sinks A and B. Sink A has been sized to <code>width="1920"</code> and <code>height="1200"</code> and is displaying the source's video content without any transformations. Sink B has been sized smaller and, as a result, is scaling the video down to fit its rectangle of 320 pixels across by 200 pixels down.</p><img alt= "Changing media stream sinks may affect sources: before the requested change" src="images/change_states_before2.svg"> <p>When the application changes sink A to a smaller dimension (from 1920 to 1024 pixels wide and from 1200 to 768 pixels tall), the [=User Agent=]'s media pipeline may recognize that none of its sinks require the higher source resolution, and needless work is being done both on the part of the source and sink A. In such a case and without any other constraints forcing the source to continue producing the higher resolution video, the media pipeline MAY change the source resolution:</p><img alt= "Changing media stream sinks may affect sources: after the requested change" src="images/change_states_after2.svg"> <p>In the above figure, the home client's video source resolution was changed to the greater of that from sink A and B in order to optimize playback. While not shown above, the same behavior could apply to peer connections and other sinks.</p> <p>It is possible that <a>constraints</a> can be applied to a track which a source is unable to satisfy, either because the source itself cannot satisfy the constraint or because the source is already satisfying a conflicting constraint. When this happens, the promise returned from {{MediaStreamTrack/applyConstraints()}} will be <a>rejected</a>, without applying any of the new constraints. Since no change in constraints occurs in this case, there is also no required change to the source itself as a result of this condition. Here is an example of this behavior.</p> <p>In this example, two media streams each have a video track that share the same source. The first track initially has no constraints applied. It is connected to sink N. Sink N has a resolution of 800 by 600 pixels and is scaling down the source's resolution of 1024 by 768 to fit. The other track has a <a data-lt="required constraints">required constraint</a> forcing off the source's fill light; it is connected to sink P. Sink P has a width and height equal to that of the source.</p> <p><img alt="Overconstrained application" src= "images/overconstrained_apply.svg"></p> <p>Now, the first track adds a <a data-lt="required constraints">required constraint</a> that the fill light should be forced on. At this point, both <a>required constraints</a> cannot be satisfied by the source (the fill light cannot be simultaneously on and off at the same time). Since this state was caused by the first track's attempt to apply a conflicting constraint, the constraint application fails and there is no change in the source's settings nor to the constraints on either track.</p> </section> <section> <h2>MediaStreams in Media Elements</h2> <p>A {{MediaStream}} may be assigned to media elements. A {{MediaStream}} is not preloadable or seekable and represents a simple, potentially infinite, linear <a data-cite="!HTML/#media-timeline">media timeline</a>. The timeline starts at 0 and increments linearly in real time as long as the media element is <a data-cite="!HTML/#potentially-playing"> potentially playing</a>. The timeline does not increment when the playout of the {{MediaStream}} is paused.</p> <p>[=User Agents=] that support this specification MUST support the {{HTMLMediaElement/srcObject}} attribute of the {{HTMLMediaElement}} interface defined in [[HTML]], which includes support for playing {{MediaStream}} objects.</p> <p>The [[HTML]] document outlines how the {{HTMLMediaElement}} works with a <em>media provider object</em>. The following applies when the <em>media provider object</em> is a {{MediaStream}}:</p> <ul> <li> <p>Whenever an {{AudioTrack}} or a {{VideoTrack}} is created, the <code>id</code> and <code>label</code> attributes must be initialized to the corresponding attributes of the {{MediaStreamTrack}}, the <code>kind</code> attribute must be initialized to <code>"main"</code> and the <code>language</code> attribute to the empty string</p> </li> <li>The [=User Agent=] MUST always play the current data from the {{MediaStream}} and MUST NOT buffer.</li> <li> <p>Since the order in the {{MediaStream}} 's [=track set=] is undefined, no requirements are put on how the {{AudioTrackList}} and {{VideoTrackList}} is ordered</p> </li> <li> <p>If the element is an {{HTMLVideoElement}}, then it is said to have <a data-cite="!HTML/#ended-playback">ended playback</a> when it has ended video playback, which is when:</p> <ol> <li> <p>The element's {{HTMLMediaElement/readyState}} is {{HTMLMediaElement/HAVE_METADATA}} or greater, and</p> <ol> <li> <p>The {{MediaStream}} state is [= stream/inactive =] after having been [= stream/active =], or</p> </li> <li> <p>The {{MediaStream}} state is [= stream/active =] after having been [= stream/inactive =] after having been [= stream/active =] after {{HTMLMediaElement/play()}} was last called, and {{HTMLMediaElement/autoplay}} is <code>false</code>.</p> </li> </ol> <div class="note"> <p>Once playback has ended, it won't resume if new {{MediaStreamTrack}}s are added to the {{MediaStream}} unless {{HTMLMediaElement/autoplay}} is <code>true</code> or the element is restarted, e.g., by the web application calling {{HTMLMediaElement/play()}}. </p> </div> </li> </ol> </li> <li> <p>If the element is an {{HTMLAudioElement}}, then it is said to have <a data-cite="!HTML/#ended-playback">ended playback</a> when it has ended audio playback, which is when:</p> <ol> <li> <p>The element's {{HTMLMediaElement/readyState}} is {{HTMLMediaElement/HAVE_METADATA}} or greater, and</p> <ol> <li> <p>The {{MediaStream}} state is [= stream/inaudible =] after having been [= stream/audible =], or</p> </li> <li> <p>The {{MediaStream}} state is [= stream/audible =] after having been [= stream/inaudible =] after having been [= stream/audible =] after {{HTMLMediaElement/play()}} was last called, and {{HTMLMediaElement/autoplay}} is <code>false</code>.</p> </li> </ol> <div class="note"> <p>Once playback has ended, it won't resume if new audio {{MediaStreamTrack}}s are added to the {{MediaStream}} unless {{HTMLMediaElement/autoplay}} is <code>true</code> or the element is restarted, e.g., by the web application calling {{HTMLMediaElement/play()}}. </p> </div> </li> </ol> </li> <li> <p>Any calls to the {{HTMLMediaElement/fastSeek()}} method on a {{HTMLMediaElement}} must be ignored</p> </li> </ul> <p>The nature of the {{MediaStream}} places certain restrictions on the behavior of attributes of the associated {{HTMLMediaElement}} and on the operations that can be performed on it, as shown below:</p> <table class="simple"> <thead> <tr> <th scope="col">Attribute Name</th> <th scope="col">Attribute Type</th> <th scope="col">Setter/Getter Behavior When Provider is a MediaStream</th> <th scope="col">Additional considerations</th> </tr> </thead> <tbody> <tr> <td> {{HTMLMediaElement/preload}} </td> <td>{{DOMString}}</td> <td>On getting: <code>none</code>. On setting: ignored.</td> <td>A {{MediaStream}} cannot be preloaded.</td> </tr> <tr> <td> {{HTMLMediaElement/buffered}} </td> <td> {{TimeRanges}} </td> <td><code>buffered.length</code> MUST return <code>0</code>.</td> <td>A {{MediaStream}} cannot be preloaded. Therefore, the amount buffered is always an empty time range.</td> </tr> <tr> <td> {{HTMLMediaElement/currentTime}} </td> <td>{{double}}</td> <td>Any non-negative integer. The initial value is <code>0</code> and the values increments linearly in real time whenever the stream is playing.</td> <td> The value is the <a data-cite= "!HTML/#official-playback-position"> official playback position</a>, in seconds. Any attempt to alter it MUST be ignored. </td> </tr> <tr> <td> {{HTMLMediaElement/seeking}} </td> <td>{{boolean}}</td> <td><code>false</code></td> <td>A {{MediaStream}} is not seekable. Therefore, this attribute MUST always return the value <code>false</code>.</td> </tr> <tr> <td> {{HTMLMediaElement/defaultPlaybackRate}} </td> <td>{{double}}</td> <td>On getting: <code>1.0</code>. On setting: ignored.</td> <td>A {{MediaStream}} is not seekable. Therefore, this attribute MUST always return the value <code>1.0</code> and any attempt to alter it MUST be ignored. Note that this also means that the <code><a data-cite= "!HTML/#event-media-ratechange">ratechange</a></code> event will not fire.</td> </tr> <tr> <td> {{HTMLMediaElement/playbackRate}} </td> <td>{{double}}</td> <td>On getting: <code>1.0</code>. On setting: ignored.</td> <td>A {{MediaStream}} is not seekable. Therefore, this attribute MUST always return the value <code>1.0</code> and any attempt to alter it MUST be ignored. Note that this also means that the <code><a data-cite= "!HTML/#event-media-ratechange">ratechange</a></code> event will not fire.</td> </tr> <tr> <td> {{HTMLMediaElement/played}} </td> <td> {{TimeRanges}} </td> <td> <code>played.length</code> MUST return <code>1</code>.<br> <code>played.start(0)</code> MUST return <code>0</code>.<br> <code>played.end(0)</code> MUST return the last known {{HTMLMediaElement/currentTime}}. </td> <td>A {{MediaStream}}'s timeline always consists of a single range, starting at 0 and extending up to the currentTime.</td> </tr> <tr> <td> {{HTMLMediaElement/seekable}} </td> <td> {{TimeRanges}} </td> <td><code>seekable.length</code> MUST return <code>0</code>.</td> <td>A {{MediaStream}} is not seekable.</td> </tr> <tr> <td> {{HTMLMediaElement/loop}} </td> <td>{{boolean}}</td> <td><code>true</code>, <code>false</code></td> <td>Setting the {{HTMLMediaElement/loop}} attribute has no effect since a {{MediaStream}} has no defined end and therefore cannot be looped.</td> </tr> </tbody> </table> <p>Since none of the setters listed above alter internal state of the {{HTMLMediaElement}}, once a {{MediaStream}} is no longer the element's <a data-cite="HTML#assigned-media-provider-object" >assigned media provider object</a>, the attributes listed will appear to resume the values they had before a stream was assigned to the element.</p> <div class="note"> <p>A {{MediaStream}} stops being the element's <a data-cite="HTML#assigned-media-provider-object">assigned media provider object</a> when {{HTMLMediaElement/srcObject}} is assigned <code>null</code> or a non-stream object, just ahead of the <a data-cite="HTML#media-element-load-algorithm">media element load algorithm</a>. As a result, the <code><a data-cite= "!HTML/#event-media-ratechange">ratechange</a></code> event may fire (from step 7) if {{HTMLMediaElement/playbackRate}} and {{HTMLMediaElement/defaultPlaybackRate}} were different from before a {{MediaStream}} was assigned.</p> </div> </section> <section> <h2>Error Handling</h2> <p>Some operations throw or fire <code>OverconstrainedError</code>. This is an extension of {{DOMException}} that carries additional information related to constraints failure.</p> <section> <h2>OverconstrainedError Interface</h2> <div> <pre class="idl" data-tests >[Exposed=Window] interface OverconstrainedError : DOMException { constructor(DOMString constraint, optional DOMString message = ""); readonly attribute DOMString constraint; };</pre> </div> <section> <h2>Constructors</h2> <dl data-link-for="OverconstrainedError" data-dfn-for="OverconstrainedError" class="constructors"> <dt><dfn data-idl><code>OverconstrainedError</code></dfn></dt> <dd> <p>Run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>constraint</var> be the constructor's first argument.</p> </li> <li> <p>Let <var>message</var> be the constructor's second argument. </p> </li> <li> <p>Let <var>e</var> be a new {{OverconstrainedError}} object.</p> </li> <li> <p>Invoke the {{DOMException}} constructor of <var>e</var> with the <code>message</code> argument set to <var>message</var> and the <code>name</code> argument set to <code>"OverconstrainedError"</code>.</p> <p class="note">This name does not have a mapping to a legacy code so <var>e</var>'s <code>code</code> attribute will return 0.</p> </li> <li> <p>Set <var>e.constraint</var> to <var>constraint</var>.</p> </li> <li> <p>Return <var>e</var>.</p> </li> </ol> </dd> </dl> </section> <section> <h2>Attributes</h2> <dl data-link-for="OverconstrainedError" data-dfn-for="OverconstrainedError" class="attributes"> <dt><dfn data-idl><code>constraint</code></dfn> of type <span class="idlAttrType">{{DOMString}}</span>, readonly</dt> <dd> <p>The name of a constraint associated with this error, or <code>""</code> if no specific constraint name is revealed.</p> </dd> </dl> </section> </section> </section> <section class="informative"> <h2>Event summary</h2> <p>The following events fire on {{MediaStream}} objects:</p> <table> <thead> <tr> <th>Event name</th> <th>Interface</th> <th>Fired when...</th> </tr> </thead> <tbody> <tr> <th scope=row><dfn id= "event-mediastream-addtrack" data-dfn-type=event>addtrack</dfn></td> <td>{{MediaStreamTrackEvent}}</td> <td>A new {{MediaStreamTrack}} has been added to this stream. Note that this event is not fired when the script directly modifies the tracks of a {{MediaStream}}.</td> </tr> <tr> <th scope=row><dfn id= "event-mediastream-removetrack" data-dfn-type=event>removetrack</dfn></td> <td>{{MediaStreamTrackEvent}}</td> <td>A {{MediaStreamTrack}} has been removed from this stream. Note that this event is not fired when the script directly modifies the tracks of a {{MediaStream}}.</td> </tr> </tbody> </table> <p>The following events fire on {{MediaStreamTrack}} objects:</p> <table> <thead> <tr> <th>Event name</th> <th>Interface</th> <th>Fired when...</th> </tr> </thead> <tbody> <tr> <th scope=row><dfn id= "event-mediastreamtrack-mute" data-dfn-type=event>mute</dfn></td> <td>{{Event}}</td> <td>The {{MediaStreamTrack}} object's source is temporarily unable to provide data.</td> </tr> <tr> <th scope=row><dfn id= "event-mediastreamtrack-unmute" data-dfn-type=event>unmute</dfn></td> <td>{{Event}}</td> <td>The {{MediaStreamTrack}} object's source is live again after having been temporarily unable to provide data.</td> </tr> <tr> <th scope=row><dfn data-dfn-type=event id="event-mediastreamtrack-ended">ended</dfn></td> <td>{{Event}}</td> <td> <p>The {{MediaStreamTrack}} object's source will no longer provide any data, either because the user revoked the permissions, or because the source device has been ejected, or because the remote peer permanently stopped sending data.</p> </td> </tr> </tbody> </table> <p>The following events fire on {{MediaDevices}} objects:</p> <table> <thead> <tr> <th>Event name</th> <th>Interface</th> <th>Fired when...</th> </tr> </thead> <tbody> <tr> <th scope=row><dfn id= "event-mediadevices-devicechange" data-dfn-type=event>devicechange</dfn></td> <td>{{DeviceChangeEvent}}</td> <td>The set of media devices, available to the [=User Agent=], has changed. The current list of devices is available in the {{DeviceChangeEvent/devices}} attribute.</td> </tr> </tbody> </table> </section> <section id="enumerating-devices"> <h2>Enumerating Local Media Devices</h2> <p>This section describes an API that the script can use to query the User Agent about connected media input and output devices (for example a web camera or a headset).</p> <section> <h3>`Navigator` Interface Extensions</h3> <p>Each {{Window}} has an <dfn>associated `MediaDevices`</dfn>, which is a {{MediaDevices}} object. Upon creation of the {{Window}} object, its [=associated `MediaDevices`=] MUST be set to a newly [=create a MediaDevices | created MediaDevices=] object with the {{Window}} object's [=relevant realm=].</p> <div> <pre class="idl" >partial interface Navigator { [SameObject, SecureContext] readonly attribute MediaDevices mediaDevices; };</pre> <section> <h2>Attributes</h2> <dl data-link-for="Navigator" data-dfn-for="Navigator" class= "attributes"> <dt><dfn>mediaDevices</dfn> of type {{MediaDevices}}, readonly</dt> <dd> <p>Return [=this=]'s [=relevant global object=]'s [=associated `MediaDevices`=].</p> </dd> </dl> </section> </div> </section> <section> <h3>{{MediaDevices}}</h3> <p>The <dfn>MediaDevices</dfn> object is the entry point to the API used to examine and get access to media devices available to the [=User Agent=].</p> <p>To <dfn class="abstract-op">create a MediaDevices</dfn> object, given <var>realm</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>mediaDevices</var> be a new {{MediaDevices}} object in <var>realm</var>, initalized with the following internal slots:</p> <ul data-dfn-for="MediaDevices"> <li> <p><dfn>[[\devicesLiveMap]]</dfn>, initialized to an empty [=ordered map | map=].</p> </li> <li> <p><dfn>[[\devicesAccessibleMap]]</dfn>, initialized to an empty [=ordered map | map=].</p> </li> <li> <p><dfn>[[\kindsAccessibleMap]]</dfn>, initialized to an empty [=ordered map | map=].</p> </li> <li> <p><dfn data-dfn-for="MediaDevices">[[\storedDeviceList]]</dfn>, initialized to a [=list=] of all media input and output devices available to the [=User Agent=].</p> </li> <li> <p><dfn data-dfn-for="MediaDevices">[[\canExposeCameraInfo]]</dfn>, initialized to <code>false</code>.</p> </li> <li> <p><dfn data-dfn-for="MediaDevices">[[\canExposeMicrophoneInfo]]</dfn>, initialized to <code>false</code>.</p> </li> <li> <p><dfn data-dfn-for="MediaDevices">[[\mediaStreamTrackSources]]</dfn>, initialized to an empty [=set=].</p> </li> </ul> </li> <li> <p>Let <var>settings</var> be <var>mediaDevices</var>'s [=relevant settings object=].</p> </li> <li> <p>For each kind of device, <var>kind</var>, that {{MediaDevices.getUserMedia()}} exposes, run the following step:<p> <ol> <li> <p>Set <var>mediaDevices</var>.{{MediaDevices/[[kindsAccessibleMap]]}}<var>[kind]</var> to either <code>true</code> if the [=permission state=] of the permission associated with <var>kind</var> (e.g. <a>"camera"</a>, <a>"microphone"</a>) for <var>settings</var> is {{PermissionState/"granted"}}, or to <code>false</code> otherwise.</p> </li> </ol> </li> <li> <p>For each individual device that {{MediaDevices.getUserMedia()}} exposes, using the device's deviceId, <var>deviceId</var>, run the following step:<p> <ol> <li> <p>Set <var>mediaDevices</var>.{{MediaDevices/[[devicesLiveMap]]}}<var>[deviceId]</var> to <code>false</code>, and set <var>mediaDevices</var>.{{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> to either <code>true</code> if the [=permission state=] of the permission associated with the device’s kind and <var>deviceId</var> for <var>settings</var>, is {{PermissionState/"granted"}}, or to <code>false</code> otherwise.</p> </li> </ol> </li> <li> <p>Return <var>mediaDevices</var>.</p> </li> </ol> <p>For each kind of device, <var>kind</var>, that {{MediaDevices/getUserMedia()}} exposes, [=permission state|whenever a transition occurs of the permission state=] of the permission associated with <var>kind</var> for <var>mediaDevices</var>'s [=relevant settings object=], run the following steps:</p> <ol class=algorithm> <li> <p>If the transition is to {{PermissionState/"granted"}} from another value, then set <var>mediaDevices</var>.{{MediaDevices/[[kindsAccessibleMap]]}}<var>[kind]</var> to <code>true</code>.</p> </li> <li> <p>If the transition is from {{PermissionState/"granted"}} to another value, then set <var>mediaDevices</var>.{{MediaDevices/[[kindsAccessibleMap]]}}<var>[kind]</var> to <code>false</code>.</p> </li> </ol> <p>For each device that {{MediaDevices/getUserMedia()}} exposes, whenever a transition occurs of the [=permission state=] of the permission associated with the device's kind and the device's deviceId, <var>deviceId</var>, for <var>mediaDevices</var>'s [=relevant settings object=], run the following steps:</p> <ol class=algorithm> <li> <p>If the transition is to {{PermissionState/"granted"}} from another value, then set <var>mediaDevices</var>.{{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> to <code>true</code>, if it isn’t already <code>true</code>.</p> </li> <li> <p>If the transition is from {{PermissionState/"granted"}} to another value, and the device is currently [= source/stopped =], then set <var>mediaDevices</var>.{{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> to <code>false</code>.</p> </li> </ol> <p>When new media input and/or output devices are made available to the [=User Agent=], or any available input and/or output device becomes unavailable, or the system default for input and/or output devices of a {{MediaDeviceKind}} changed, the [=User Agent=] MUST run the following <dfn class="abstract-op">device change notification steps</dfn> for each {{MediaDevices}} object, <var>mediaDevices</var>, for which [=device enumeration can proceed=] is <code>true</code>, but for no other {{MediaDevices}} object:</p> <ol class=algorithm> <li> <p>Let <var>lastExposedDevices</var> be the result of [=creating a list of device info objects=] with <var>mediaDevices</var> and <var>mediaDevices</var>.{{MediaDevices/[[storedDeviceList]]}}.</p> </li> <li> <p>Let <var>deviceList</var> be the list of all media input and/or output devices available to the [=User Agent=].</p> </li> <li> <p>Let <var>newExposedDevices</var> be the result of [=creating a list of device info objects=] with <var>mediaDevices</var> and <var>deviceList</var>.</p> </li> <li> <p>If the {{MediaDeviceInfo}} objects in <var>newExposedDevices</var> match those in <var>lastExposedDevices</var> and have the same order, then abort these steps.</p> <div class="note"> <p>Due to the {{MediaDevices/enumerateDevices}} algorithm, the above step limits firing the <a data-link-type=event>devicechange</a> event to documents [=allowed to use=] {{MediaDevices/enumerateDevices}} to enumerate devices of a particular {{MediaDeviceKind}}.</p> </div> </li> <li> <p>Set <var>mediaDevices</var>.{{MediaDevices/[[storedDeviceList]]}} to <var>deviceList</var>.</p> </li> <li> <p>Queue a task that [= fire an event | fires an event=] named {{devicechange}}, using the {{DeviceChangeEvent}} constructor with {{DeviceChangeEventInit/devices}} initialized to <var>newExposedDevices</var>, at <var>mediaDevices</var>.</p> <p>The [=User Agent=] MAY combine firing multiple events into firing one event when several events are due or when multiple devices are added or removed at the same time, e.g. a camera with a microphone.</p> </li> </ol> <p>Additionally, if a {{MediaDevices}} object that was traversed comes to meet the [=device enumeration can proceed=] criteria later (e.g. [=Document/is in view | comes into view=]), the [=User Agent=] MUST execute the [=device change notification steps=] on the {{MediaDevices}} object at that time.</p> <div class="note"> <p class="fingerprint">These events are potentially triggered simultaneously on documents of different origins. [=User Agents=] MAY add fuzzing on the timing of events to avoid cross-origin activity correlation.</p> </div> <div> <pre class="idl" >[Exposed=Window, SecureContext] interface MediaDevices : EventTarget { attribute EventHandler ondevicechange; Promise<sequence<MediaDeviceInfo>> enumerateDevices(); };</pre> <section> <h2>Attributes</h2> <dl data-link-for="MediaDevices" data-dfn-for="MediaDevices" class= "attributes"> <dt><dfn>ondevicechange</dfn> of type {{EventHandler}}</dt> <dd> <p>The event type of this event handler is <a data-link-type=event>devicechange</a>.</p> </dd> </dl> </section> <section> <h2>Methods</h2> <dl data-link-for="MediaDevices" data-dfn-for="MediaDevices" class= "methods"> <dt><dfn>enumerateDevices</dfn></dt> <dd> <p>Collects information about the [=User Agent=]'s available media input and output devices.</p> <p>This method returns a promise. The promise will be [=upon fulfillment|fulfilled=] with a sequence of {{MediaDeviceInfo}} objects representing the [=User Agent=]'s available media input and output devices if enumeration is successful.</p> <p>Elements of this sequence that represent input devices will be of type {{InputDeviceInfo}} which extends {{MediaDeviceInfo}}.</p> <p>Camera and microphone sources SHOULD be enumerable. Specifications that add additional types of source will provide recommendations about whether the source type should be enumerable.</p> <p>When the {{MediaDevices/enumerateDevices()}} method is called, the [=User Agent=] must run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>p</var> be a new promise.</p> </li> <li> <p>Let <var>proceed</var> be the result of [=device enumeration can proceed=] with [=this=].</p> </li> <li> <p>Let <var>mediaDevices</var> be [=this=].</p> </li> <li> <p>Run the following steps in parallel:</p> <ol> <li> <p>While <var>proceed</var> is `false`, the [=User Agent=] MUST wait to proceed to the next step until a task queued to set <var>proceed</var> to the result of [=device enumeration can proceed=] with <var>mediaDevices</var>, would set <var>proceed</var> to `true`.</p> </li> <li> <p>Let <var>resultList</var> be the result of [=creating a list of device info objects=] with <var>mediaDevices</var> and <var>mediaDevices</var>.{{MediaDevices/[[storedDeviceList]]}}.</p></li> <li> <p>[= resolve =] <var>p</var> with <var>resultList</var>.</p> </li> </ol> </li> <li> <p>Return <var>p</var>.</p> </li> </ol> <p>To perform <dfn class="abstract-op" id= "creating-a-list-of-device-info-objects"> creating a list of device info objects</dfn>, given <var>mediaDevices</var> and <var>deviceList</var>, run the following steps:</p> <ol class="algorithm"> <li><p>Let <var>resultList</var> be an empty list.</p></li> <li> <p>Let <var>microphoneList</var>, <var>cameraList</var> and <var>otherDeviceList</var> be empty lists.</p> </li> <li> <p>Let <var>document</var> be <var>mediaDevices</var>'s [=relevant global object=]'s [=associated `Document`=].</p> </li> <li> <p>Run the following sub steps for each discovered device in <var>deviceList</var>, <var>device</var>:</p> <ol> <li> <p>If <var>device</var> is not a microphone, or <var>document</var> is not [=allowed to use=] the feature identified by <a>"microphone"</a>, abort these sub steps and continue with the next device (if any).</p> </li> <li> <p>Let <var>deviceInfo</var> be the result of [=creating a device info object=] to represent <var>device</var>, with <var>mediaDevices</var>.</p> </li> <li> <p>If <var>device</var> is the system default microphone, prepend <var>deviceInfo</var> to <var>microphoneList</var>. Otherwise, append <var>deviceInfo</var> to <var>microphoneList</var>.</p> </li> </ol> </li> <li> <p>Run the following sub steps for each discovered device in <var>deviceList</var>, <var>device</var>:</p> <ol> <li> <p>If <var>device</var> is not a camera, or <var>document</var> is not [=allowed to use=] the feature identified by <a>"camera"</a>, abort these sub steps and continue with the next device (if any).</p> </li> <li> <p>Let <var>deviceInfo</var> be the result of [=creating a device info object=] to represent <var>device</var>, with <var>mediaDevices</var>.</p> </li> <li> <p>If <var>device</var> is the system default camera, prepend <var>deviceInfo</var> to <var>cameraList</var>. Otherwise, append <var>deviceInfo</var> to <var>cameraList</var>.</p> </li> </ol> </li> <li> <p>If [=microphone information can be exposed=] on <var>mediaDevices</var> is <code>false</code>, truncate <var>microphoneList</var> to its first item.</p> </li> <li> <p>If [=camera information can be exposed=] on <var>mediaDevices</var> is <code>false</code>, truncate <var>cameraList</var> to its first item.</p> </li> <li> <p>Run the following sub steps for each discovered device in <var>deviceList</var>, <var>device</var>:</p> <ol> <li> <p>If <var>device</var> is a microphone or <var>device</var> is a camera, abort these sub steps and continue with the next device (if any).</p> </li> <li> <p>Run the [=exposure decision algorithm for devices other than camera and microphone=], with <var>device</var>, <var>microphoneList</var>, <var>cameraList</var> and <var>mediaDevices</var> as input. If the result of this algorithm is <code>false</code>, abort these sub steps and continue with the next device (if any).</p> </li> <li> <p>Let <var>deviceInfo</var> be the result of [=creating a device info object=] to represent <var>device</var>, with <var>mediaDevices</var>.</p> </li> <li> <p>If <var>device</var> is the system default audio output, prepend <var>deviceInfo</var> to <var>otherDeviceList</var>. Otherwise, append <var>deviceInfo</var> to <var>otherDeviceList</var>.</p> </li> </ol> </li> <li> <p>Append to <var>resultList</var> all devices of <var>microphoneList</var> in order.</p> </li> <li> <p>Append to <var>resultList</var> all devices of <var>cameraList</var> in order.</p> </li> <li> <p>Append to <var>resultList</var> all devices of <var>otherDeviceList</var> in order.</p> </li> <li> <p>Return <var>resultList</var>.</p> </li> </ol> <p class="fingerprint">Since this method returns persistent information across browsing sessions and origins via the availability of media capture devices, it adds to the fingerprinting surface exposed by the [=User Agent=].</p> <p class="fingerprint">As long as the [=relevant global object=]'s [=associated `Document`=] did not capture, this method will limit exposure to two bits of information: whether there is a camera and whether there is a microphone. A [=User Agent=] may mitigate this by pretending the system has a camera and a microphone, for instance until the [=relevant global object=]'s [=associated `Document`=] calls {{MediaDevices/getUserMedia()}} with constraints deemed reasonable.</p> <p class="fingerprint">After the [=relevant global object=]'s [=associated `Document`=] started capture, it provides additional persistent cross-origin information via the list of all media capture devices, including their grouping and human readable labels associated with the capture devices, which further adds to the fingerprinting surface.</p> <p class="fingerprint" id="sanitize-device-labels"> A [=User Agent=] may limit exposure by sanitizing device labels. This could for instance mean removing user names found in labels, but keeping device manufacturer or model information. It is important that the sanitized labels allow users to identify the corresponding devices.</p> </dd> </dl> </section> </div> <section> <h2>Access control model</h2> <p>The algorithm described above means that the access to media device information depends on whether or not the [=relevant global object=]'s [=associated `Document`=] did capture.</p> <p>For camera and microphone devices, if the [=relevant global object=]'s [=associated `Document`=] did not capture (i.e. {{MediaDevices/getUserMedia()}} was not called or never resolved successfully), the {{MediaDeviceInfo}} object will contain a valid value for {{MediaDeviceInfo/kind}} but empty strings for {{MediaDeviceInfo/deviceId}}, {{MediaDeviceInfo/label}}, and {{MediaDeviceInfo/groupId}}. Additionally, at most one device of each {{MediaDeviceInfo/kind}} will be listed in {{MediaDevices/enumerateDevices()}} result.</p> <p>Otherwise, the <dfn>MediaDeviceInfo</dfn> object will contain meaningful values for {{MediaDeviceInfo/deviceId}}, {{MediaDeviceInfo/kind}}, {{MediaDeviceInfo/label}}, and {{MediaDeviceInfo/groupId}}. All available devices are listed in {{MediaDevices/enumerateDevices()}} result.</p> <p>To perform <dfn class="abstract-op" id= "creating-a-device-info-object">creating a device info object</dfn> to represent a discovered device, <var>device</var>, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>deviceInfo</var> be a new {{MediaDeviceInfo}} object to represent <var>device</var>.</p> </li> <li> <p>Initialize <var>deviceInfo</var>.{{MediaDeviceInfo/kind}} for <var>device</var>.</p> </li> <li> <p>If <var>deviceInfo</var>.{{MediaDeviceInfo/kind}} is equal to "videoinput" and [=camera information can be exposed=] on <var>mediaDevices</var> is <code>false</code>, return <var>deviceInfo</var>.</p> </li> <li> <p>If <var>deviceInfo</var>.{{MediaDeviceInfo/kind}} is equal to "audioinput" and [=microphone information can be exposed=] on <var>mediaDevices</var> is <code>false</code>, return <var>deviceInfo</var>.</p> </li> <li> <p>Initialize <var>deviceInfo</var>.{{MediaDeviceInfo/label}} for <var>device</var>.</p> </li> <li> <p>If a stored {{MediaDeviceInfo/deviceId}} exists for <var>device</var>, initialize <var>deviceInfo</var>.{{MediaDeviceInfo/deviceId}} to that value. Otherwise, let <var>deviceInfo</var>.{{MediaDeviceInfo/deviceId}} be a newly generated unique identifier as described under {{MediaDeviceInfo/deviceId}}.</p> </li> <li> <p>If <var>device</var> belongs to the same physical device as a device already represented for <var>document</var>, initialize <var>deviceInfo</var>.{{MediaDeviceInfo/groupId}} to the {{MediaDeviceInfo/groupId}} value of the existing {{MediaDeviceInfo}} object. Otherwise, let <var>deviceInfo</var>.{{MediaDeviceInfo/groupId}} be a newly generated unique identifier as described under {{MediaDeviceInfo/groupId}}.</p> </li> <li> <p>Return <var>deviceInfo</var></p> </li> </ol> </section> <section> <h2>Device information exposure</h2> <p>To perform a <dfn class="abstract-op" id= "device-enumeration-can-proceed">device enumeration can proceed</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>The [=User Agent=] MAY return <code>true</code> if [=device information can be exposed=] on <var>mediaDevices</var>. </li> <li> <p>Return the result of [=Document/is in view=] with <var>mediaDevices</var>.</p> </li> </ol> <p>To perform a <dfn class="abstract-op" id= "device-information-can-be-exposed">device information can be exposed</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If [=camera information can be exposed=] on <var>mediaDevices</var>, return <code>true</code>.</p> </li> <li> <p>If [=microphone information can be exposed=] on <var>mediaDevices</var>, return <code>true</code>.</p> </li> <li> <p>Return <code>false</code>.</p> </li> </ol> <p>To perform a <dfn class="abstract-op" id= "camera-information-can-be-exposed">camera information can be exposed</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If any of the local devices of kind "videoinput" are attached to a live {{MediaStreamTrack}} in <var>mediaDevices</var>'s [=relevant global object=]'s [=associated `Document`=], return <code>true</code>.</p> </li> <li> <p>Return <var>mediaDevices</var>.{{MediaDevices/[[canExposeCameraInfo]]}}.</p> </li> </ol> <p>To perform a <dfn class="abstract-op" id= "microphone-information-can-be-exposed">microphone information can be exposed</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If any of the local devices of kind "audioinput" are attached to a live {{MediaStreamTrack}} in the [=relevant global object=]'s [=associated `Document`=], return <code>true</code>.</p> </li> <li> <p>Return <var>mediaDevices</var>.{{MediaDevices/[[canExposeMicrophoneInfo]]}}.</p> </li> </ol> <p>To perform an <dfn data-dfn-for="Document" class="abstract-op">is in view</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If <var>mediaDevices</var>'s [=relevant global object=]'s [=associated `Document`=] is [=Document/fully active=] and its [=Document/visibility state=] is `"visible"`, return `true`. Otherwise, return `false`.</p> </li> </ol> <p>To perform a <dfn>has system focus</dfn> check, given <var>mediaDevices</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If <var>mediaDevices</var>'s [=relevant global object=]'s [=navigable=]'s [=top-level traversable=] has <a data-cite="!HTML/#tlbc-system-focus">system focus</a>, return `true`. Otherwise, return `false`.</p> </li> </ol> </section> <section> <h2>Set device information exposure</h2> <p>To <dfn class="abstract-op" id= "set-device-information-exposure">set the device information exposure</dfn> on <var>mediaDevices</var>, given a <var>requestedTypes</var> [=set=], and a boolean <var>value</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If <a>"video"</a> is in <var>requestedTypes</var>, then set <var>mediaDevices</var>.{{MediaDevices/[[canExposeCameraInfo]]}} to <var>value</var>.</p> </li> <li> <p>If <a>"audio"</a> is in <var>requestedTypes</var>, then set <var>mediaDevices</var>.{{MediaDevices/[[canExposeMicrophoneInfo]]}} to <var>value</var>.</p> </li> </ol> <div class="note"> <p>A [=User Agent=] MAY at any point set the device information exposure back to <code>false</code>, for instance if the [=User Agent=] decides to revoke device access on a given {{Document}}.</p> </div> </section> <section> <h2>Exposure decision algorithm for devices other than camera and microphone</h2> <p>The <dfn data-export id= "device-exposure-decision-non-camera-microphone">exposure decision algorithm for devices other than camera and microphone</dfn> takes a <var>device</var>, <var>microphoneList</var>, <var>cameraList</var> and <var>mediaDevices</var> as input and returns a boolean to decide whether to expose information about <var>device</var> to the web page or not.</p> <p>By default, it returns <code>false</code>.</p> <p>Other specifications can define the algorithm for specific device types. </section> <section> <h2>Context capturing state</h2> <p>To perform a <dfn class="abstract-op" id="context-is-capturing">context is capturing</dfn> check for <var>globalObject</var>, run the following steps:</p> <ol class="algorithm"> <li> <p>If <var>globalObject</var> is not a {{Window}}, then return false.</p> </li> <li> <p>Let <var>mediaDevices</var> be <var>globalObject</var>'s [=associated `MediaDevices`=].</p> </li> <li> <p>For each <var>source</var> in <var>mediaDevices</var>.{{MediaDevices/[[mediaStreamTrackSources]]}}, run the following sub steps:</p> <ol> <li> <p>If <var>source</var> is [=source/stopped=] or [=source/muted=], abort these steps.</p> </li> <li> <p>Let <var>deviceId</var> be <var>source</var>'s device's deviceId.</p> </li> <li> <p>If <var>mediaDevices</var>.{{MediaDevices/[[devicesLiveMap]]}}[<var>deviceId</var>] is <code>true</code>, return <code>true</code>.</p> </li> </ol> </li> <li> <p>Return <code>false</code>.</p> </li> </ol> <p class="note">This algorithm covers all capture tracks, including microphone, camera and display.</p> </section> </section> <section> <h2>Device Info</h2> <div> <pre class="idl" >[Exposed=Window, SecureContext] interface MediaDeviceInfo { readonly attribute DOMString deviceId; readonly attribute MediaDeviceKind kind; readonly attribute DOMString label; readonly attribute DOMString groupId; [Default] object toJSON(); };</pre> <section> <h2>Attributes</h2> <dl data-link-for="MediaDeviceInfo" data-dfn-for="MediaDeviceInfo" class="attributes"> <dt id="def-mediadeviceinfo-deviceId"><dfn>deviceId</dfn> of type {{DOMString}}, readonly</dt> <dd> <p>The identifier of the represented device. The device MUST be uniquely identified by its identifier and its {{MediaDeviceInfo/kind}}.</p> <p>To ensure stored identifiers are recognized, the identifier MUST be the same in {{Document}}s of the [=same origin=] in [=top-level traversables=]. In [=child navigables=], the decision of whether or not the identifier is the same across documents, MUST follow the [=User Agent=]'s partitioning rules for storage (such as {{WindowLocalStorage/localStorage}}), if any, to not interfere with mitigations for cross-site correlation. If the identifier can uniquely identify the user, then it MUST be un-guessable in documents from other origins to prevent the identifier from being used to correlate the same user across different origins. An identifier can be reused across origins as long as it is not tied to the user and can be guessed by other means, like the User-Agent string.</p> <p>If any local devices have been attached to a live {{MediaStreamTrack}} in a page from this origin, or [=stored permission=] to access local devices has been granted to this origin, then this identifier MUST be persisted, except as detailed below. Unique and stable identifiers let the application save, identify the availability of, and directly request specific sources, across multiple visits.</p> <p>However, as long as no local device has been attached to a live MediaStreamTrack in a page from this origin, and no [=stored permission=] to access local devices has been granted to this origin, then the [=User Agent=] MAY clear this identifier once the last browsing session from this origin has been closed. If the [=User Agent=] chooses not to clear the identifier in this condition, then it MUST provide for the user to visibly inspect and delete the identifier, like a cookie.</p> <p class="fingerprint">Since {{deviceId}} may persist across browsing sessions and to reduce its potential as a fingerprinting mechanism, {{deviceId}} is to be treated as other persistent storage mechanisms such as cookies [[COOKIES]], in that [=User Agents=] MUST NOT persist device identifiers for sites that are blocked from using cookies, and [=User Agents=] MUST rotate per-origin device identifiers when other persistent storage are cleared.</p> </dd> <dt id="def-mediadeviceinfo-kind"><dfn>kind</dfn> of type {{MediaDeviceKind}}, readonly</dt> <dd> <p>The kind of the represented device.</p> </dd> <dt><dfn>label</dfn> of type {{DOMString}}, readonly</dt> <dd> <p>A label describing this device (for example "External USB Webcam"). This label is intended to allow the end user to tell the difference between devices. Applications can’t assume that the label contains any specific information, such as the device type or model. If the device has no associated label, then this attribute MUST return the empty string.</p> </dd> <dt id="def-mediadeviceinfo-groupId"><dfn>groupId</dfn> of type {{DOMString}}, readonly</dt> <dd> <p>The group identifier of the represented device. Two devices have the same group identifier if they belong to the same physical device. For example, the audio input and output devices representing the speaker and microphone of the same headset have the same groupId.</p> <p>The group identifier MUST be uniquely generated for each <a>document</a>.</p> </dd> </dl> </section> <section> <h2>Methods</h2> <dl data-link-for="MediaDeviceInfo" data-dfn-for="MediaDeviceInfo" class="methods"> <dt><dfn>toJSON</dfn></dt> <dd> When called, run [[WEBIDL]]'s <a>default toJSON steps</a>. </dd> </dl> </section> </div> <div> <pre class="idl" >enum MediaDeviceKind { "audioinput", "audiooutput", "videoinput" };</pre> <table data-link-for="MediaDeviceKind" data-dfn-for="MediaDeviceKind" class="simple"> <tbody> <tr> <th colspan="2"><dfn>MediaDeviceKind</dfn> Enumeration description</th> </tr> <tr> <td><dfn id= "idl-def-MediaDeviceKind.audioinput">audioinput</dfn></td> <td> <p>Represents an audio input device; for example a microphone.</p> </td> </tr> <tr> <td><dfn id= "idl-def-MediaDeviceKind.audiooutput">audiooutput</dfn></td> <td> <p>Represents an audio output device; for example a pair of headphones.</p> </td> </tr> <tr> <td><dfn id= "idl-def-MediaDeviceKind.videoinput">videoinput</dfn></td> <td> <p>Represents a video input device; for example a webcam.</p> </td> </tr> </tbody> </table> </div> </section> <section> <h2>Input-specific Device Info</h2> <p>The <dfn>InputDeviceInfo</dfn> interface gives access to the capabilities of the input device it represents.</p> <div> <pre class="idl" >[Exposed=Window, SecureContext] interface InputDeviceInfo : MediaDeviceInfo { MediaTrackCapabilities getCapabilities(); };</pre> <section> <h2>Methods</h2> <dl data-link-for="InputDeviceInfo" data-dfn-for="InputDeviceInfo" class="methods"> <dt><dfn>getCapabilities</dfn></dt> <dd> <p>Returns a {{MediaTrackCapabilities}} object describing the primary audio or video track of a device's {{MediaStream}} (according to its {{MediaStreamTrack/kind}} value), in the absence of any user-supplied constraints. These capabilities MUST be identical to those that would have been obtained by calling {{MediaStreamTrack/getCapabilities()}} on the first {{MediaStreamTrack}} of this type in a {{MediaStream}} returned by <code>getUserMedia({deviceId: <i>id</i>})</code> where <var>id</var> is the value of the {{MediaDeviceInfo/deviceId}} attribute of this {{MediaDeviceInfo}}.</p> <p>If no access has been granted to any local devices and this {{InputDeviceInfo}} has been filtered with respect to unique identifying information (see above description of {{MediaDevices/enumerateDevices()}} result), then this method returns an empty dictionary.</p> </dd> </dl> </section> </div> </section> <section> <h3>{{DeviceChangeEvent}}</h3> <p>The {{devicechange}} event uses the {{DeviceChangeEvent}} interface.</p> <div> <pre class="idl" >[Exposed=Window] interface DeviceChangeEvent : Event { constructor(DOMString type, optional DeviceChangeEventInit eventInitDict = {}); [SameObject] readonly attribute FrozenArray<MediaDeviceInfo> devices; [SameObject] readonly attribute FrozenArray<MediaDeviceInfo> userInsertedDevices; };</pre> <section> <h2>Constructors</h2> <dl data-link-for="DeviceChangeEvent" data-dfn-for= "DeviceChangeEvent" class="constructors"> <dt><dfn>constructor()</dfn></dt> <dd> <p>Initialize [=this=].{{DeviceChangeEvent/devices}} to the result of [=creating a frozen array=] from <var>eventInitDict</var>.{{DeviceChangeEventInit/devices}}.</p> </dd> </dl> </section> <section> <h2>Attributes</h2> <dl data-link-for="DeviceChangeEvent" data-dfn-for= "DeviceChangeEvent" class="attributes"> <dt> <dfn data-idl="">devices</dfn> of type <span class= "idlAttrType">FrozenArray<{{MediaDeviceInfo}}></span>, readonly </dt> <dd> <p>The {{devices}} attribute returns an array of {{MediaDeviceInfo}} objects representing the list of available devices at this time. </p> </dd> <dt> <dfn data-idl="">userInsertedDevices</dfn> of type <span class= "idlAttrType">FrozenArray<{{MediaDeviceInfo}}></span>, readonly </dt> <dd> <p>The {{userInsertedDevices}} attribute returns an array containing only those {{MediaDeviceInfo}} objects from {{devices}} that the user physically inserted or activated recently and are newly exposed with this event as a result. Otherwise, an empty list is returned.</p> <p>The [=User Agent=] MAY include devices the user inserted or activated before {{MediaDevices/getUserMedia()}} was called, provided this event marks their first exposure, and the user did not choose devices in {{MediaDevices/getUserMedia()}}. </p> <p>The {{MediaDeviceInfo}} objects, if any, MUST also exist in {{devices}}.</p> <div class="note"> <p>A user inserting a device during (or immediately ahead of) a call can be a strong signal that they wish to use the device immediately.</p> <p>Applications are encouraged to rely on this attribute to disambiguate this signal from differences in {{devices}} that might happen from changes in device information exposure.</p> </div> <div class="note"> If the attribute is missing, it means that this UA has not been upgraded to implement this version of the specification. </div> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary DeviceChangeEventInit : EventInit { sequence<MediaDeviceInfo> devices = []; };</pre> <section> <h2>Dictionary <dfn>DeviceChangeEventInit</dfn> Members</h2> <dl data-link-for="DeviceChangeEventInit" data-dfn-for= "DeviceChangeEventInit" class="dictionary-members"> <dt> <dfn data-idl="">devices</dfn> of type <span class= "idlMemberType">sequence<{{MediaDeviceInfo}}></span>, defaulting to <code>[]</code> </dt> <dd> <p> The {{devices}} member is an array of {{MediaDeviceInfo}} objects representing the available devices. </p> </dd> </dl> </section> </div> </section> </section> <section id="local-content"> <h2>Obtaining local multimedia content</h2> <p>This section extends {{Navigator}} and {{MediaDevices}} with APIs to request permission to access media input devices available to the [=User Agent=].</p> <p>Alternatively, a local {{MediaStream}} can be captured from certain types of DOM elements, such as the video element [[?mediacapture-fromelement]]. This can be useful for automated testing.</p> <section> <h3>{{MediaDevices}} Interface Extensions</h3> <div class="note"> The definition of {{Navigator/getUserMedia()}} in this section reflects two major changes from the method definition that has existed under {{Navigator}} for many months. <p>First, the official definition for the {{MediaDevices/getUserMedia()}} method, and the one which developers are encouraged to use, is now the one defined here under {{MediaDevices}}. This decision reflected consensus as long as the original API remained available at <a>Navigator.getUserMedia</a> under the {{Navigator}} object for backwards compatibility reasons, since the working group acknowledges that early users of these APIs have been encouraged to define getUserMedia as "var getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;" in order for their code to be functional both before and after official implementations of getUserMedia() in popular [=User Agents=]. To ensure functional equivalence, the getUserMedia() method under {{Navigator}} is defined in terms of the method here.</p> <p>Second, the method defined here is Promises-based, while the one defined under {{Navigator}} is currently still callback-based. Developers expecting to find getUserMedia() defined under Navigator are strongly encouraged to read the detailed Note given there.</p> </div> <p>The {{MediaDevices/getSupportedConstraints}} method is provided to allow the application to determine which constraints the [=User Agent=] recognizes. Applications may need this information to use <a>required constraints</a> reliably or get predictable results from combinatory logic in <a>advanced constraints</a>.</p> <div> <pre class="idl" >partial interface MediaDevices { MediaTrackSupportedConstraints getSupportedConstraints(); Promise<MediaStream> getUserMedia(optional MediaStreamConstraints constraints = {}); };</pre> <section> <h2>Methods</h2> <dl data-link-for="MediaDevices" data-dfn-for="MediaDevices" class= "methods"> <dt><dfn>getSupportedConstraints</dfn></dt> <dd> <p>Returns a dictionary whose members are the constrainable properties known to the [=User Agent=]. A supported constrainable property MUST be represented and any constrainable properties not supported by the [=User Agent=] MUST NOT be present in the returned dictionary. The values returned represent what the [=User Agent=] implements and will not change during a browsing session.</p> </dd> <dt>getUserMedia</dt> <dd> <p>Prompts the user for permission to use their Web cam or other video or audio input.</p> <p>The <var>constraints</var> argument is a dictionary of type {{MediaStreamConstraints}}.</p> <p>This method returns a promise. The promise will be [=upon fulfillment|fulfilled=] with a suitable {{MediaStream}} object if the user accepts valid tracks as described below.</p> <p>The promise will be <a>rejected</a> if there is a failure in finding valid tracks or if the user denies permission, as described below.</p> <p>When the <dfn>getUserMedia()</dfn> method is called, the [=User Agent=] MUST run the following steps:</p> <ol> <li> <p>Let <var>constraints</var> be the method's first argument.</p> </li> <li> <p>Let <var>requestedMediaTypes</var> be the set of media types in <var>constraints</var> with either a dictionary value or a value of <code>true</code>.</p> </li> <li> <p>If <var>requestedMediaTypes</var> is the empty set, return a promise <a>rejected</a> with a {{TypeError}}. The word "optional" occurs in the WebIDL due to WebIDL rules, but the argument MUST be supplied in order for the call to succeed.</p> </li> <li> <p>Let <var>document</var> be the [=relevant global object=]'s [=associated `Document`=].</p> </li> <li> <p>If <var>document</var> is NOT [=Document/fully active=], return a promise <a>rejected</a> with a {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"InvalidStateError"}}.</p> </li> <li> <p>If <var>requestedMediaTypes</var> contains "audio" and <var>document</var> is not [=allowed to use=] the feature identified by the "microphone" permission name, jump to the step labeled <em>Permission Failure</em> below.</p> </li> <li> <p>If <var>requestedMediaTypes</var> contains "video" and <var>document</var> is not [=allowed to use=] the feature identified by the "camera" permission name, jump to the step labeled <em>Permission Failure</em> below.</p> </li> <li> <p>Let <var>mediaDevices</var> be [=this=].</p> </li> <li> <p>Let <var>isInView</var> be the result of the [= Document/is in view =] algorithm.</p> </li> <li> <p>Let <var>p</var> be a new promise.</p> </li> <li> <p>Run the following steps in parallel:</p> <ol> <li> <p>While <var>isInView</var> is `false`, the [=User Agent=] MUST wait to proceed to the next step until a task queued to set <var>isInView</var> to the result of the [=Document/is in view=]</a> algorithm, would set <var>isInView</var> to `true`.</p> </li> <li> <p>Let <var>finalSet</var> be an (initially) empty set.</p> </li> <li> <p>For each media type <var>kind</var> in <var>requestedMediaTypes</var>, run the following steps:</p> <ol> <li> <p>For each possible configuration of each possible source device of media of type <var>kind</var>, conceive a <dfn>candidate</dfn> as a placeholder for an eventual {{MediaStreamTrack}} holding a source device and configured with a <a>settings dictionary</a> comprised of its specific settings.</p> <p>Call this set of candidates the <var>candidateSet</var>.</p> <p>If <var>candidateSet</var> is the empty set, jump to the step labeled <em>NotFound Failure</em> below.</p> </li> <li>If the value of the <var>kind</var> entry of <var>constraints</var> is <code>true</code>, set <var>CS</var> to the empty constraint set (no constraint). Otherwise, continue with <var>CS</var> set to the value of the <var>kind</var> entry of <var>constraints</var>.</li> <li>Remove any constrainable property inside of <var>CS</var> that are not defined for {{MediaStreamTrack}} objects of type <var>kind</var>. This means that audio-only constraints inside of <a>"video"</a> and video-only constraints inside of <a>"audio"</a> are simply ignored rather than causing <code>OverconstrainedError</code>.</li> <li><p>If <var>CS</var> contains a member that is a <a>required constraint</a> and whose name is not in the list of <a>allowed required constraints for device selection</a>, then [= reject =] <var>p</var> with a {{TypeError}}, and abort these steps.</p></li> <li> <p>Run the <a>SelectSettings</a> algorithm on each candidate in <var>candidateSet</var> with <var>CS</var> as the constraint set. If the algorithm returns <code>undefined</code>, remove the candidate from <var>candidateSet</var>. This eliminates devices unable to satisfy the constraints, by verifying that at least one settings dictionary exists that satisfies the constraints.</p> <p>If <var>candidateSet</var> is the empty set, let <var>failedConstraint</var> be any <a data-lt="required constraints">required constraint</a> whose fitness distance was infinity for all settings dictionaries examined while executing the <a>SelectSettings</a> algorithm, or <code>""</code> if there isn't one, and jump to the step labeled <em>Constraint Failure</em> below.</p> <p class="fingerprint">This error gives information about what the underlying device is not capable of producing, before the user has given any authorization to any device, and can thus be used as a fingerprinting surface.</p> </li> <li> <p>Read the current [=permission state=] for all candidate devices in <var>candidateSet</var> that are not attached to a live {{MediaStreamTrack}} in the current {{Document}}. Remove from <var>candidateSet</var> any candidate whose device's permission state is {{PermissionState/"denied"}}.</p> <p>If <var>candidateSet</var> is now empty, indicating that all devices of this type are in state {{PermissionState/"denied"}}, jump to the step labeled <em>PermissionFailure</em> below.</p> </li> <li> <p>Optionally, e.g., based on a previously-established user preference, for security reasons, or due to platform limitations, jump to the step labeled <em>Permission Failure</em> below.</p> </li> <li> <p>Add all candidates from <var>candidateSet</var> to <var>finalSet</var>.</p> </li> </ol> </li> <li> <p>Let <var>stream</var> be a new and empty {{MediaStream}} object.</p> </li> <li> <p>For each media type <var>kind</var> in <var>requestedMediaTypes</var>, run the following sub steps, preferably at the same time:</p> <div class="note"> <p>[=User Agents=] are encouraged to bundle concurrent requests for different kinds of media into a single user-facing permission prompt.</p> </div> <ol> <li> <p>[=Request permission to use=] a {{PermissionDescriptor}} with its {{PermissionDescriptor/name}} member set to the permission name associated with <var>kind</var> (e.g. <a>"camera"</a> for <a>"video"</a>, <a>"microphone"</a> for <a>"audio"</a>), while considering all devices attached to a live and <a>same-permission</a> {{MediaStreamTrack}} in the current {{Document}} to have permission status {{PermissionState/"granted"}}, resulting in a set of provided media. <dfn>Same-permission</dfn> in this context means a {{MediaStreamTrack}} that required the same level of permission to obtain as what is being requested (e.g. not isolated).</p> <p>When asking the user’s permission, the [=User Agent=] MUST disclose whether permission will be granted only to the device chosen, or to all devices of that <var>kind</var>.</p> <div class="note"> <p>If the user never responds, this algorithm stalls on this step.</p> </div> </li> <li> <p>If the result of the request is {{PermissionState/"denied"}}, jump to the step labeled <em>Permission Failure</em> below.</p> </li> <li> <p>Let <var>hasSystemFocus</var> be `false`.</p> </li> <li> <p>While <var>hasSystemFocus</var> is `false`, the [=User Agent=] MUST wait to proceed to the next step until a task queued to set <var>hasSystemFocus</var> to the result of the [=has system focus=]</a> algorithm, would set <var>hasSystemFocus</var> to `true`.</p> </li> <li> <p>Let <var>finalCandidate</var> be the provided media, which MUST be precisely one <a>candidate</a> of type <var>kind</var> from <var>finalSet</var>. The decision of which candidate to choose from the <var>finalSet</var> is completely up to the [=User Agent=] and may be determined by asking the user. </p> <p>The [=User Agent=] SHOULD use the value of the computed <a>fitness distance</a> from the <a>SelectSettings</a> algorithm as an input to the selection algorithm. However, it MAY also use other internally-available information about the devices, such as user preference.</p> <p>[=User Agents=] are encouraged to default to using the user's primary or system default device for <var>kind</var> (when possible). [=User Agents=] MAY allow users to use any media source, including pre-recorded media files.</p> </li> <li> <p>The result of the request is {{PermissionState/"granted"}}. If a hardware error such as an OS/program/webpage lock prevents access, remove the corresponding candidate from <var>finalSet</var>. If <var>finalSet</var> has no candidates of type <var>kind</var>, [= reject =] <var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"NotReadableError"}} and abort these steps. Otherwise, restart these sub steps with the updated <var>finalSet</var>.</p> <p>If device access fails for any reason other than those listed above, remove the corresponding candidate from <var>finalSet</var>. If <var>finalSet</var> has no candidates of type <var>kind</var>, [= reject =] <var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"AbortError"}} and abort these steps. Otherwise, restart these sub steps with the updated <var>finalSet</var>.</p> </li> <li> <p>Let <var>grantedDevice</var> be <var>finalCandidate</var>'s source device.</p> <li> <p>Using <var>grantedDevice</var>'s deviceId, <var>deviceId</var>, set <var>mediaDevices</var>.{{MediaDevices/[[devicesLiveMap]]}}[<var>deviceId</var>] to <code>true</code>, if it isn’t already <code>true</code>, and set <var>mediaDevices</var>.{{MediaDevices/[[devicesAccessibleMap]]}}[<var>deviceId</var>] to <code>true</code>, if it isn’t already <code>true</code>.</p> </li> <li> <p>Let <var>track</var> be the result of [=create a MediaStreamTrack|creating a MediaStreamTrack=] with <var>grantedDevice</var> and <var>mediaDevices</var>. The source of the {{MediaStreamTrack}} MUST NOT change.</p> </li> <li> <p>Add <var>track</var> to <var>stream</var>'s track set.</p> </li> </ol> </li> <li> <p>Run the <a>ApplyConstraints algorithm</a> on all tracks in <var>stream</var> with the appropriate constraints. If any of them returns something other than <code>undefined</code>, let <var>failedConstraint</var> be that result and jump to the step labeled <em>Constraint Failure</em> below.</p> </li> <li> <p>For each <var>track</var> in <var>stream</var>, [=tie track source to `MediaDevices`=] with <var>track</var>.{{MediaStreamTrack/[[Source]]}} and <var>mediaDevices</var>.</p> </li> <li> <p>[=Set the device information exposure=] on <var>mediaDevices</var> with <code>requestedMediaTypes</code> and <code>true</code>.</p> </li> <li> <p>[= Resolve =] <var>p</var> with <var>stream</var> and abort these steps.</p> </li> <li> <p><em>NotFound Failure</em>:</p> <ol> <li><p> If [=getUserMedia specific failure is allowed=] given <var>requestedMediaTypes</var> returns <code>false</code>, jump to the step labeled <em>Permission Failure</em> below. </p></li> <li><p> [=Reject=] <var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"NotFoundError"}}.</p> </p></li> </ol> </li> <li> <p><em>Constraint Failure</em>:</p> <ol> <li><p> If [=getUserMedia specific failure is allowed=] given <var>requestedMediaTypes</var> returns <code>false</code>, jump to the step labeled <em>Permission Failure</em> below. </p></li> <li><p> Let <var>message</var> be either <code>undefined</code> or an informative human-readable message, let <var>constraint</var> be <var>failedConstraint</var> if [=device information can be exposed=] is <code>true</code>, or <code>""</code> otherwise. </p></li> <li><p> [=Reject=] <var>p</var> with a new <code>OverconstrainedError</code> created by calling <code>OverconstrainedError(<var>constraint</var>, <var>message</var>)</code>.</p> </p></li> </ol> </li> <li> <p><em>Permission Failure</em>: [= Reject =] <var>p</var> with a new {{DOMException}} object whose {{DOMException/name}} attribute has the value {{"NotAllowedError"}}.</p> </li> </ol> </li> <li> <p>Return <var>p</var>.</p> </li> </ol> </dd> </dl> </section> </div> <div> <p>To check whether <dfn class="abstract-op" id= "getUserMedia-specific-failure-is-allowed">getUserMedia specific failure is allowed</dfn>, given <var>requestedMediaTypes</var>, run the following steps:</p> <ol class="algorithm"> <li><p>If <var>requestedMediaTypes</var> contains "audio", read the [=permission state=] for the descriptor whose name is "microphone". If the result of the request is {{PermissionState/"denied"}}, return <code>false</code>.</p></li> <li><p>If <var>requestedMediaTypes</var> contains "video", read the [=permission state=] for the descriptor whose name is "camera". If the result of the request is {{PermissionState/"denied"}}, return <code>false</code>.</p></li> <li><p>Return <code>true</code>.</p></li> </ol> </div> <div class="note"> <p>In the algorithm above, constraints are checked twice - once at device selection, and once after access approval. Time may have passed between those checks, so it is conceivable that the selected device is no longer suitable. In this case, a NotReadableError will result.</p> </div> <div> <p>The <dfn class="export">allowed required constraints for device selection</dfn> contains the following constraint names: <a>width</a>, <a>height</a>, <a>aspectRatio</a>, <a>frameRate</a>, <a>facingMode</a>, <a>resizeMode</a>, <a>sampleRate</a>, <a>sampleSize</a>, <a>echoCancellation</a>, <a>autoGainControl</a>, <a>noiseSuppression</a>, <a>latency</a>, <a>channelCount</a>, <a>deviceId</a>, <a>groupId</a>. </p> </div> </section> <section> <h2>{{MediaStreamConstraints}}</h2> <p>The <dfn>MediaStreamConstraints</dfn> dictionary is used to instruct the [=User Agent=] what sort of {{MediaStreamTrack}}s to include in the <a>MediaStream</a> returned by {{MediaDevices/getUserMedia()}}.</p> <div> <pre class="idl" >dictionary MediaStreamConstraints { (boolean or MediaTrackConstraints) video = false; (boolean or MediaTrackConstraints) audio = false; };</pre> <section> <h2>Dictionary <a class="idlType">MediaStreamConstraints</a> Members</h2> <dl data-link-for="MediaStreamConstraints" data-dfn-for= "MediaStreamConstraints" class="dictionary-members"> <dt><dfn>video</dfn> of type <code>({{boolean}} or {{MediaTrackConstraints}})</code>, defaulting to <code>false</code></dt> <dd> <p>If <code>true</code>, it requests that the returned <a>MediaStream</a> contain a video track. If a <a>Constraints</a> structure is provided, it further specifies the nature and settings of the video Track. If <code>false</code>, the {{MediaStream}} MUST NOT contain a video Track.</p> </dd> <dt><dfn>audio</dfn> of type <code>({{boolean}} or {{MediaTrackConstraints}})</code>, defaulting to <code>false</code></dt> <dd> <p>If <code>true</code>, it requests that the returned <a>MediaStream</a> contain an audio track. If a <a>Constraints</a> structure is provided, it further specifies the nature and settings of the audio Track. If <code>false</code>, the <a>MediaStream</a> MUST NOT contain an audio Track.</p> </dd> </dl> </section> </div> </section> <section class="informative"> <h3>Legacy GetUserMedia interface</h3> <div class="note"> <p> The definition of getUserMedia() in this section reflects the call format that was originally proposed; it is only documented here for browsers that wish to retain backwards compatibility. It differs from the recommended interface in two important ways. </p> <p>First, the official definition for the getUserMedia() method, and the one which developers are encouraged to use, is now at {{MediaDevices}}. This decision reflected consensus as long as the original API remained available here under the Navigator object for backwards compatibility reasons, since the working group acknowledges that early users of these APIs have been encouraged to define getUserMedia as "var getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia;" in order for their code to be functional both before and after official implementations of getUserMedia() in popular browsers. To ensure functional equivalence, the getUserMedia() method here is defined in terms of the method under MediaDevices.</p> <p>Second, the decision to change all other callback-based methods in the specification to be based on Promises instead required that the navigator.getUserMedia() definition reflect this in its use of navigator.mediaDevices.getUserMedia(). Because navigator.getUserMedia() is now the only callback-based method remaining in the specification, there is ongoing discussion as to a) whether it still belongs in the specification, and b) if it does, whether its syntax should remain callback-based or change in some way to use Promises. Input on these questions is encouraged, particularly from developers actively using today's implementations of this functionality.</p> <p>Note that the other methods that changed from a callback-based syntax to a Promises-based syntax were not considered to have been implemented widely enough in any form to have to consider legacy usage.</p> <p>Implementations do not need to implement this interface in order to be considered compliant.</p> </div> <section> <h3>Interface definition</h3> <div> <pre class="idl" >partial interface Navigator { [SecureContext] undefined getUserMedia(MediaStreamConstraints constraints, NavigatorUserMediaSuccessCallback successCallback, NavigatorUserMediaErrorCallback errorCallback); };</pre> <section> <h2>Methods</h2> <dl data-link-for="Navigator" data-dfn-for="Navigator" class= "methods"> <dt><dfn>getUserMedia</dfn></dt> <dd> <p>Prompts the user for permission to use their Web cam or other video or audio input.</p> <p>The <var>constraints</var> argument is a dictionary of type {{MediaStreamConstraints}}.</p> <p>The <var>successCallback</var> will be invoked with a suitable {{MediaStream}} object as its argument if the user accepts valid tracks as described in {{MediaDevices/getUserMedia()}} on {{MediaDevices}}.</p> <p>The <var>errorCallback</var> will be invoked if there is a failure in finding valid tracks or if the user denies permission, as described in {{MediaDevices/getUserMedia()}} on {{MediaDevices}}.</p> <p>When the {{getUserMedia()}} method is called, the User Agent MUST run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>constraints</var> be the method's first argument.</p> </li> <li> <p>Let <var>successCallback</var> be the callback indicated by the method's second argument.</p> </li> <li> <p>Let <var>errorCallback</var> be the callback indicated by the method's third argument.</p> </li> <li> <p>Run the steps specified by the <a href= "#dom-mediadevices-getusermedia">getUserMedia() algorithm</a> with <var>constraints</var> as the argument, and let <var>p</var> be the resulting promise.</p> </li> <li> <p>[=Upon fulfillment=] of <var>p</var> with value <var>stream</var>, run the following step:</p> <ol> <li> <p>Invoke <var>successCallback</var> with <var>stream</var> as the argument.</p> </li> </ol> </li> <li> <p>[=Upon rejection=] of <var>p</var> with reason <var>r</var>, run the following step:</p> <ol> <li> <p>Invoke <var>errorCallback</var> with <var>r</var> as the argument.</p> </li> </ol> </li> </ol> </dd> </dl> </section> </div> </section> <section> <h2><dfn>NavigatorUserMediaSuccessCallback</dfn></h2> <div> <pre class="idl" >callback NavigatorUserMediaSuccessCallback = undefined (MediaStream stream);</pre> <section> <h2>Callback <a class="idlType">NavigatorUserMediaSuccessCallback</a> Parameters</h2> <dl data-link-for="NavigatorUserMediaSuccessCallback" data-dfn-for= "NavigatorUserMediaSuccessCallback" class="callback-members"> <dt>stream of type {{MediaStream}}</dt> <dd> {{MediaStream}} object representing the stream to which the user granted permission as described in the {{Navigator.getUserMedia}} algorithm. </dd> </dl> </section> </div> </section> <section> <h2><dfn>NavigatorUserMediaErrorCallback</dfn></h2> <div> <pre class="idl" >callback NavigatorUserMediaErrorCallback = undefined (DOMException error);</pre> <section> <h2>Callback <a class="idlType">NavigatorUserMediaErrorCallback</a> Parameters</h2> <dl data-link-for="NavigatorUserMediaErrorCallback" data-dfn-for= "NavigatorUserMediaErrorCallback" class="callback-members"> <dt>error of type {{DOMException}}</dt> <dd> Error in obtaining a {{MediaStream}} as described in the failure steps of the {{Navigator.getUserMedia}} algorithm. </dd> </dl> </section> </div> </section> </section> <section class="informative"> <h2>Implementation Suggestions</h2> <div class="practice"> <span class="practicelab" id="resource-reservation">Resource reservation</span> <p class="practicedesc">The [=User Agent=] is encouraged to reserve resources when it has determined that a given call to {{MediaDevices/getUserMedia()}} will be successful. It is preferable to reserve the resource prior to resolving the returned promise. Subsequent calls to {{MediaDevices/getUserMedia()}} (in this page or any other) should treat the resource that was previously allocated, as well as resources held by other applications, as busy. Resources marked as busy should not be provided as sources to the current web page, unless specified by the user. Optionally, the [=User Agent=] may choose to provide a stream sourced from a busy source but only to a page whose origin matches the owner of the original stream that is keeping the source busy.</p> <p class="practicedesc">This document recommends that in the permission grant dialog or device selection interface (if one is present), the user be allowed to select any available hardware as a source for the stream requested by the page (provided the resource is able to fulfill any specified <a>required constraints</a>). Although not specifically recommended as best practice, note that some [=User Agents=] may support the ability to substitute a video or audio source with local files and other media. A file picker may be used to provide this functionality to the user.</p> <p class="practicedesc">This document also recommends that the user be shown all resources that are currently busy as a result of prior calls to <a href="#dom-mediadevices-getusermedia">getUserMedia()</a> (in this page or any other page that is still alive) and be allowed to terminate that stream and utilize the resource for the current page instead. If possible in the current operating environment, it is also suggested that resources currently held by other applications be presented and treated in the same manner. If the user chooses this option, the track corresponding to the resource that was provided to the page whose stream was affected must be removed.</p> </div> <div class="practice"> <dfn id="stored-permissions" class="practicelab">Stored Permission</dfn>s <p class="practicedesc">When permission is requested for a device, the [=User Agent=] may choose to store this permission for later use by the same origin, so that the user does not need to grant permission again at a later time. It is a [=User Agent=] choice whether it offers functionality to store permission to each device separately, all devices of a given class, or all devices; the choice needs to be apparent to the user, and permission must have been granted for the entire set whose permission is being stored, e.g., to store permission to use all cameras the user must have given permission to use all cameras and not just one.</p> <p class="practicedesc">As described, this specification does not dictate whether or not granting permission results in a stored permission. When permission is not stored, permission will last only until such time as all MediaStreamTracks sourced from that device have been stopped.</p> </div> <div class="practice"> <span class="practicelab" id="handling-devices">Handling multiple devices</span> <p class="practicedesc">A <a>MediaStream</a> may contain more than one video and audio track. This makes it possible to include video from two or more webcams in a single stream object, for example. However, the current API does not allow a page to express a need for multiple video streams from independent sources.</p> <p class="practicedesc">It is recommended for multiple calls to <a href="#dom-mediadevices-getusermedia">getUserMedia()</a> from the same page to be allowed as a way for pages to request multiple discrete video and/or audio streams.</p> <p class="practicedesc">Note also that if multiple <a href= "#dom-mediadevices-getusermedia">getUserMedia()</a> calls are done by a page, the order in which they request resources, and the order in which they complete, is not constrained by this specification.</p> <p class="practicedesc">A single call to <a href= "#dom-mediadevices-getusermedia">getUserMedia()</a> will always return a stream with either zero or one audio tracks, and either zero or one video tracks. If a script calls <a href= "#dom-mediadevices-getusermedia">getUserMedia()</a> multiple times before reaching a stable state, this document advises the UI designer that the permission dialogs should be merged, so that the user can give permission for the use of multiple cameras and/or media sources in one dialog interaction. The constraints on each getUserMedia call can be used to decide which stream gets which media sources.</p> </div> <div class="practice"> <span class="practicelab" id="generating-deviceids">Generating deviceIds</span> <p class="practicedesc">An efficient practice for generating a {{MediaDeviceInfo/deviceId}} is to generate a cryptographic hash from a private key + (origin or origin + top-level origin, based on the user agents' partitioning rules) + salt + device's underlying (hardware) id in the driver, and present the resulting hash as an alphanumeric string. Using 32 bits or fewer for the hash is recommended, but not much lower, to avoid risk of collision. </p> <p class="practicedesc">A lower-entropy alternative, at the cost of storage, is to assign the numbers 0 through 255 randomly to each new device encountered for each origin or origin + top-level origin, based on the [=User Agent=]'s partitioning rules, retiring the number that hasn't been seen the longest if numbers run out.</p> </div> <div class="practice"> <span class="practicelab" id="muting-devices">Device muting initiated by [=User Agent=]</span> <p class="practicedesc">A track sourced by a camera or microphone may be forcibly [= MediaStreamTrack/muted =] by a [=User Agent=] at any time, in order to manage a user's privacy. However, doing so may create web compatibility issues, as well as leak information about user activity, so caution is advised. </p> <p class="practicedesc">Best practice is to <a data-lt=muted>mute</a> a camera or microphone track in the following instances:</p> <ul> <li> <p>An OS-level event for which the [=User Agent=] already suspends media playback globally, but JavaScript is not suspended. The rationale is users may otherwise be surprised if capture were to continue in this situation (unless they've intentionally configured it this way). If the OS-level event already causes frames to stop coming in on the track, then no new information of user activity is revealed by this. Even when this is not the case, revealing that capture is ending seems like a reasonable privacy tradeoff compared to continuing capture in situations that may surprise users.</p> </li> <li> <p>A web page not [=Document/is in view|in view=] [=MediaStreamTrack/enabled|re-enables=] a track when all tracks from that source are [=MediaStreamTrack/enabled|disabled=], in order to delay resumption of capture until the page [=Document/is in view=]. </p> </li> </ul> <p class="practicedesc">Best practice is to [= MediaStreamTrack/muted | unmute =] a camera or microphone track it previously [= MediaStreamTrack/muted =], in the following instances:</p> <ul> <li> <p>An OS-level event for which the [=User Agent=] already resumes media playback globally, <em>and</em> the page is visible to the user (e.g. not during a lock screen). [=User Agents=] may defer such action if it determines significant time has passed that may jeopardize a user's awareness of the earlier capture session.</p> </li> <li> <p>A web page comes [=Document/is in view|into view=] and has one or more [=MediaStreamTrack/enabled=] tracks that are also [= MediaStreamTrack/muted =]. </p> </li> </ul> </div> </section> </section> <section id="constrainable-interface"> <h2>Constrainable Pattern</h2> <p>The Constrainable pattern allows applications to inspect and adjust the properties of objects implementing it (the <dfn>constrainable object</dfn>). It is broken out as a separate set of definitions so that it can be referred to by other specifications. The core concept is the Capability, which consists of a constrainable property of an object and the set of its possible values, which may be specified either as a range or as an enumeration. For example, a camera might be capable of framerates (a property) between 20 and 50 frames per second (a range) and may be able to be positioned (a property) facing towards the user, away from the user, or to the left or right of the user (an enumerated set). The application can examine a constrainable property's supported Capabilities via the <code>getCapabilities()</code> accessor.</p> <p>The application can select the (range of) values it wants for an object's Capabilities by means of basic and/or advanced ConstraintSets and the <code>applyConstraints()</code> method. A ConstraintSet consists of the names of one or more properties of the object plus the desired value (or a range of desired values) for each property. Each of those property/value pairs can be considered to be an individual constraint. For example, the application may set a ConstraintSet containing two constraints, the first stating that the framerate of a camera be between 30 and 40 frames per second (a range) and the second that the camera should be facing the user (a specific value). How the individual constraints interact depends on whether and how they are given in the basic Constraint structure, which is a ConstraintSet with an additional 'advanced' property, or whether they are in a ConstraintSet in the advanced list. The behavior is as follows: all 'min', 'max', and 'exact' constraints in the basic Constraint structure are together treated as the <a>required constraints</a>, and if it is not possible to satisfy simultaneously all of those individual constraints for the indicated property names, the [=User Agent=] MUST [= reject =] the returned promise. Otherwise, it must apply the required constraints. Next, it will consider any ConstraintSets given in the <code><a href="#dom-constraints-advanced">advanced</a></code> list, in the order in which they are specified, and will try to satisfy/apply each complete ConstraintSet (i.e., all constraints in the ConstraintSet together), but will skip a ConstraintSet if and only if it cannot satisfy/apply it in its entirety. Next, the [=User Agent=] MUST attempt to apply, individually, any 'ideal' constraints or a constraint given as a bare value for the property (referred to as <a>optional basic constraints</a>). Of these properties, it MUST satisfy the largest number that it can, in any order. Finally, the [=User Agent=] MUST [= resolve =] the returned promise.</p> <div class="note"> Any constraint provided via this API will only be considered if the given constrainable property is supported by the [=User Agent=]. JavaScript application code is expected to first check, via <code>getSupportedConstraints()</code>, that all the named properties that are used are supported by the [=User Agent=]. The reason for this is that WebIDL drops any unsupported names from the dictionary holding the constraints, so the [=User Agent=] does not see them and the unsupported names end up being silently ignored. This will cause confusing programming errors as the JavaScript code will be setting constraints but the [=User Agent=] will be ignoring them. [=User Agents=] that support (recognize) the name of a required constraint but cannot satisfy it will generate an error, while [=User Agents=] that do not support the constrainable property will not generate an error. </div> <p>The following examples may help to understand how constraints work. The first shows a basic Constraint structure. Three constraints are given, each of which the [=User Agent=] will attempt to satisfy individually. Depending upon the resolutions available for this camera, it is possible that not all three constraints can be satisfied at the same time. If so, the [=User Agent=] will satisfy two if it can, or only one if not even two constraints can be satisfied together. Note that if not all three can be satisfied simultaneously, it is possible that there is more than one combination of two constraints that could be satisfied. If so, the [=User Agent=] will choose.</p> <pre class="example"> const stream = await navigator.mediaDevices.getUserMedia({ video: { width: 1280, height: 720, aspectRatio: 3/2 } }); </pre> <p>This next example adds a small bit of complexity. The ideal values are still given for width and height, but this time with minimum requirements on each as well as a minimum frameRate that must be satisfied. If it cannot satisfy the frameRate, width or height minimum it will [= reject =] the promise. Otherwise, it will try to satisfy the width, height, and aspectRatio target values as well and then [= resolve =] the promise.</p> <pre class="example"> try { const stream = await navigator.mediaDevices.getUserMedia({ video: { width: {min: 640, ideal: 1280}, height: {min: 480, ideal: 720}, aspectRatio: 3/2, frameRate: {min: 20} } }); } catch (error) { if (error.name != "OverconstrainedError") { throw error; } // Overconstrained. Try again with a different combination (no prompt was shown) } </pre> <p>This example illustrates the full control possible with the Constraints structure by adding the 'advanced' property. In this case, the [=User Agent=] behaves the same way with respect to the required constraints, but before attempting to satisfy the ideal values it will process the 'advanced' list. In this example the 'advanced' list contains two ConstraintSets. The first specifies width and height constraints, and the second specifies an aspectRatio constraint. Note that in the advanced list, these bare values are treated as 'exact' values. This example represents the following: "I need my video to be at least 640 pixels wide and at least 480 pixels high. My preference is for precisely 1920x1280, but if you can't give me that, give me an aspectRatio of 4x3 if at all possible. If even that is not possible, give me a resolution as close to 1280x720 as possible."</p> <pre class="example"> try { const stream = await navigator.mediaDevices.getUserMedia({ video: { width: {min: 640, ideal: 1280}, height: {min: 480, ideal: 720}, frameRate: {min: 30}, advanced: [ {width: 1920, height: 1280}, {aspectRatio: 4/3}, {frameRate: {min: 50}}, {frameRate: {min: 40}} ] } }); } catch (error) { if (error.name != "OverconstrainedError") { throw error; } // Overconstrained. Try again with a different combination (no prompt was shown) } </pre> <p>The ordering of advanced ConstraintSets is significant. In the preceding example it is impossible to satisfy both the 1920x1280 ConstraintSet and the 4x3 aspect ratio ConstraintSet at the same time. Since the 1920x1280 occurs first in the list, the [=User Agent=] will attempt to satisfy it first. Application authors can therefore implement a backoff strategy by specifying multiple advanced ConstraintSets for the same property. The application also specifies two more advanced ConstraintSets, the first asking for a frame rate greater than 50, the second asking for a frame rate greater than 40. If the [=User Agent=] is capable of setting a frame rate greater than 50, it will (and the subsequent ConstraintSet will be trivially satisfied). However, if the [=User Agent=] cannot set the frame rate above 50, it will skip that ConstraintSet and attempt to set the frame rate above 40. In case the [=User Agent=] cannot satisfy either of the two ConstraintSets, the 'min' value in the basic ConstraintSet insists on 30 as a lower bound. In other words, the [=User Agent=] would fail altogether if it couldn't get a value over 30, but would choose a value over 50 if possible, then try for a value over 40.</p> <p>Note that, unlike basic constraints, the constraints within a ConstraintSet in the advanced list must be satisfied together or skipped together. Thus, {width: 1920, height: 1280} is a request for that specific resolution, not a request for that width or that height. One can think of the basic constraints as requesting an 'or' (non-exclusive) of the individual constraints, while each advanced ConstraintSet is requesting an 'and' of the individual constraints in the ConstraintSet. An application may inspect the full set of Constraints currently in effect via the <code>getConstraints()</code> accessor.</p> <p>The specific value that the [=User Agent=] chooses for a constrainable property is referred to as a Setting. For example, if the application applies a ConstraintSet specifying that the frameRate must be at least 30 frames per second, and no greater than 40, the Setting can be any intermediate value, e.g., 32, 35, or 37 frames per second. The application can query the current settings of the object's constrainable properties via the {{MediaStreamTrack/getSettings()}} accessor.</p> <section> <h2>Interface Definition</h2> <p>Although this specification formally defines <dfn>ConstrainablePattern</dfn> as a WebIDL interface, it is actually a template or pattern for other interfaces and cannot be inherited directly since the return values of the methods need to be extended, something WebIDL cannot do. Thus, each interface that wishes to make use of the functionality defined here will have to provide its own copy of the WebIDL for the functions and interfaces given here. However it can refer to the semantics defined here, which will not change. See <a href= "#media-stream-track-interface-definition">MediaStreamTrack Interface Definition</a> for an example of this.</p> <p>This pattern relies on the <a>constrainable object</a> defining three internal slots:</p> <ol> <li> <p>A <dfn data-dfn-for="constrainable object">[[\Capabilities]]</dfn> internal slot, initialized to a <code>Capabilities</code> dictionary describing the aggregate allowable values for each constrainable property exposed, as explained under <a href="#dom-capabilities">Capabilities</a>, or an empty dictionary if it has none.</p> </li> <li> <p>A <dfn data-dfn-for="constrainable object">[[\Constraints]]</dfn> internal slot, initialized to an empty <code>Constraints</code> dictionary.</p> </li> <li> <p>A <dfn data-dfn-for="constrainable object">[[\Settings]]</dfn> internal slot, initialized to a <code>Settings</code> dictionary describing the currently active settings values for each constrainable property exposed, as explained under <a href="#dom-settings">Settings</a>, or an empty dictionary if it has none.</p> </li> </ol> <div> <div class="example">Template: <pre class="idl" >[Exposed=Window] interface ConstrainablePattern { Capabilities getCapabilities(); Constraints getConstraints(); Settings getSettings(); Promise<undefined> applyConstraints(optional Constraints constraints = {}); };</pre></div> <section> <h2>Methods</h2> <dl data-link-for="ConstrainablePattern" data-dfn-for= "ConstrainablePattern" class="methods"> <dt>getCapabilities</dt> <dd> <p>The <dfn data-lt= "ConstrainablePattern.getCapabilities">getCapabilities()</dfn> method returns the dictionary of the names of the constrainable properties that the object supports. When invoked, the [=User Agent=] MUST return the value of the <a data-link-for="constrainable object" data-link-type="attribute">[[\Capabilities]]</a> internal slot.</p> <div class="note"> <p>It is possible that the underlying hardware may not exactly map to the range defined for the constrainable property. Where this is possible, the entry SHOULD define how to translate and scale the hardware's setting onto the values defined for the property. For example, suppose that a hypothetical fluxCapacitance property ranges from -10 (min) to 10 (max), but there are common hardware devices that support only values of "off" "medium" and "full". The constrainable property definition might specify that for such hardware, the [=User Agent=] should map the range value of -10 to "off", 10 to "full", and 0 to "medium". It might also indicate that given a ConstraintSet imposing a strict value of 3, the [=User Agent=] should attempt to set the value of "medium" on the hardware, and that {{getSettings()}} should return a fluxCapacitance of 0, since that is the value defined as corresponding to "medium".</p> </div> </dd> <dt>{{getConstraints}}</dt> <dd> <p>The <dfn>getConstraints()</dfn> method returns the Constraints that were the argument to the most recent successful invocation of the <a>ApplyConstraints algorithm</a> on the object, maintaining the order in which they were specified. Note that some of the advanced ConstraintSets returned may not be currently satisfied. To check which ConstraintSets are currently in effect, the application should use {{getSettings}}. Instead of returning the exact constraints as described above, the UA MAY return a constraint set that has the identical effect in all situations as the applied constraints. When invoked, the [=User Agent=] MUST return the value of the <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a> internal slot. </p> </dd> <dt>getSettings</dt> <dd> <p>The <dfn>getSettings()</dfn> method returns the current settings of all the constrainable properties of the object, whether they are platform defaults or have been set by the <a>ApplyConstraints algorithm</a>. Note that a setting is a target value that complies with constraints, and therefore may differ from measured performance at times. When invoked, the User Agent MUST return the value of the <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a> internal slot.</p> </dd> <dt>applyConstraints</dt> <dd> <p>When the <dfn>applyConstraints template method</dfn> is invoked, the [=User Agent=] MUST run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>object</var> be the object on which this method was invoked.</p> </li> <li> <p>Let <var>newConstraints</var> be the argument to this method.</p> </li> <li> <p>Let <var>p</var> be a new promise.</p> </li> <li> <p>Run the following steps in parallel, maintaining the order of invocations if this method is called multiple times:</p> <ol> <li> <p>Let <var>failedConstraint</var> be the result of running the <a>ApplyConstraints algorithm</a> with <var>newConstraints</var> as the argument.</p> </li> <li> <p>Let <var>successfulSettings</var> be the <var>object</var>'s current settings after the algorithm in the above step has finished.</p> </li> <li> <p>Queue a task that runs the following steps:</p> <ol> <li> <p>If <var>failedConstraint</var> is not <code>undefined</code>, let <var>message</var> be either <code>undefined</code> or an informative human-readable message, [= reject =] <var>p</var> with a new <code>OverconstrainedError</code> created by calling <code>OverconstrainedError(<var>failedConstraint</var>, <var>message</var>)</code>, and abort these steps. The existing constraints remain in effect in this case.</p> </li> <li> <p>Set <var>object</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Constraints]]</a> internal slot to <var>newConstraints</var> or a <code>Constraints</code> dictionary that has the identical effect in all situations as <var>newConstraints</var>.</p> </li> <li> <p>Set <var>object</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a> internal slot to <var>successfulSettings</var>.</p> </li> <li> <p>[= resolve =] <var>p</var> with <code>undefined</code>.</p> </li> </ol> </li> </ol> </li> <li> <p>Return <var>p</var>.</p> </li> </ol> <p>The [=ApplyConstraints algorithm=] for applying constraints is stated below. Here are some preliminary definitions that are used in the statement of the algorithm:</p> <p>We use the term <dfn data-export>settings dictionary</dfn> for the set of values that might be applied as settings to the object.</p> <p>For string valued constraints, we define "==" below to be true if one of the values in the sequence is exactly the same as the value being compared against.</p> <p>We define the <dfn data-export>fitness distance</dfn> between a <a>settings dictionary</a> and a constraint set <var>CS</var> as the sum, for each member (represented by a <var>constraintName</var> and <var>constraintValue</var> pair) which [= map/exist =]s in <var>CS</var>, of the following values:</p> <ol> <li> <p>If <var>constraintName</var> is not supported by the [=User Agent=], the fitness distance is 0.</p> </li> <li> <p>If the constraint is <dfn>required</dfn> (<var>constraintValue</var> either contains one or more members named 'min', 'max', or 'exact', or is itself a bare value and bare values are to be treated as 'exact'), and the <a>settings dictionary</a>'s <var>constraintName</var> member's value does not satisfy the constraint or doesn't [= map/exist =], the fitness distance is positive infinity.</p> </li> <li> <p>If the constraint does not apply for this type of object, the fitness distance is 0 (that is, the constraint does not influence the fitness distance).</p> </li> <li> <p>If <var>constraintValue</var> is a boolean, but the constrainable property is not, then the fitness distance is based on whether the <a>settings dictionary</a>'s <var>constraintName</var> member [= map/exist | exists =] or not, from the formula <pre>(constraintValue == exists) ? 0 : 1</pre> </li> <li> <p>If the <a>settings dictionary</a>'s <var>constraintName</var> member does [= map/exist | not exist=], the fitness distance is 1.</p> </li> <li>If no ideal value is specified (<var>constraintValue</var> either contains no member named 'ideal', or, if bare values are to be treated as 'ideal', isn't a bare value), the fitness distance is 0.</li> <li>For all positive numeric constraints (such as height, width, frameRate, aspectRatio, sampleRate and sampleSize), the fitness distance is the result of the formula <pre> (actual == ideal) ? 0 : |actual - ideal| / max(|actual|, |ideal|)</pre> </li> <li>For all string, enum and boolean constraints (e.g. deviceId, groupId, facingMode, resizeMode, echoCancellation), the fitness distance is the result of the formula <pre>(actual == ideal) ? 0 : 1</pre> </li> </ol> <p>More definitions:</p> <ul> <li>We refer to each element of a ConstraintSet (other than the special term 'advanced') as a 'constraint' since it is intended to constrain the acceptable settings for the given property from the full list or range given in the corresponding Capability of the ConstrainablePattern object to a value that is within the range or list of values it specifies.</li> <li>We refer to the "effective Capability" C of an object O as the possibly proper subset of the possible values of C (as returned by getCapabilities) taking into consideration environmental limitations and/or restrictions placed by other constraints. For example given a ConstraintSet that constrains the aspectRatio, height, and width properties, the values assigned to any two of the properties limit the effective Capability of the third. The set of effective Capabilities may be platform dependent. For example, on a resource-limited device it may not be possible to set properties P1 and P2 both to 'high', while on another less limited device, this may be possible.</li> <li>A settings dictionary, which is a set of values for the constrainable properties of an object O, satisfies ConstraintSet CS if the <a>fitness distance</a> between the set and CS is less than infinity. </li> <li>A set of ConstraintSets CS1...CSn (n >= 1) can be satisfied by an object O if it is possible to find a settings dictionary of O that satisfies CS1...CSn simultaneously.</li> <li>To apply a set of ConstraintSets CS1...CSn to object O is to choose such a sequence of values that satisfy CS1...CSn and assign them as the settings for the properties of O.</li> </ul> <p>We define the <dfn class="abstract-op">SelectSettings</dfn> algorithm as follows:</p> <ol class="algorithm"> <li>Each <a>constraint</a> specifies one or more values (or a range of values) for its property. A property MAY appear more than once in the list of 'advanced' ConstraintSets. If an empty list has been given as the value for a constraint, it MUST be interpreted as if the constraint were not specified (in other words, an empty constraint == no constraint). <p>Note that unknown properties are discarded by WebIDL, which means that unknown/unsupported required constraints will silently disappear. To avoid this being a surprise, application authors are expected to first use the {{MediaDevices/getSupportedConstraints()}} method as shown in the Examples below.</p> </li> <li>Let <var>object</var> be the <code><a>ConstrainablePattern</a></code> object on which this algorithm is applied. Let <var>copy</var> be an unconstrained copy of <var>object</var> (i.e., <var>copy</var> should behave as if it were <var>object</var> with all ConstraintSets removed.)</li> <li> <p>For every possible <a>settings dictionary</a> of <var>copy</var> compute its <a>fitness distance</a>, treating bare values of properties as ideal values. Let <var>candidates</var> be the set of <a data-lt= "settings dictionary">settings dictionaries</a> for which the fitness distance is finite.</p> </li> <li> <p>If <var>candidates</var> is empty, return <code>undefined</code> as the result of the <a>SelectSettings</a> algorithm.</p> </li> <li>Iterate over the 'advanced' ConstraintSets in <var>newConstraints</var> in the order in which they were specified. For each ConstraintSet: <ol> <li> <p>compute the <a>fitness distance</a> between it and each settings dictionary in <var>candidates</var>, treating bare values of properties as exact.</p> </li> <li> <p>If the fitness distance is finite for one or more settings dictionaries in <var>candidates</var>, keep those settings dictionaries in <var>candidates</var>, discarding others.</p> <p>If the fitness distance is infinite for all settings dictionaries in <var>candidates</var>, ignore this ConstraintSet.</p> </li> </ol> </li> <li> <p>Select one settings dictionary from <var>candidates</var>, and return it as the result of the <a>SelectSettings</a> algorithm. The [=User Agent=] MUST use one with the smallest <a>fitness distance</a>, as calculated in step 3. If more than one settings dictionary have the smallest fitness distance, the [=User Agent=] chooses one of them based on system default property values and [=User Agent=] default property values.</p> </li> </ol> <p>For any property with a system default value for the selected device, the system default value SHOULD be used if compatible with the above algorithm. This is usually the case for properties like <a href="#def-constraint-sampleRate">sampleRate</a> or <a href="#def-constraint-sampleSize">sampleSize</a>. Other properties, like <a href="#def-constraint-echoCancellation">echoCancellation</a> or <a href="#def-constraint-resizeMode">resizeMode</a> do not usually have system default values. The [=User Agent=] defines its own default values for these properties. Implementors need to be cautious to select good default values since they will often have an impact on how media content is generated.</p> <div class="note"> <p>It is recommended to look at existing implementations to select meaningful default values. Note that default values may differ based on the system, for instance desktop vs. mobile. At time of writing, [=User Agent=] implementations tend to use the following default values, which were chosen for their suitability for using RTCPeerConnection as a sink:</p> <ol> <li><p><a href="#def-constraint-width">width</a> set to 640.</p></li> <li><p><a href="#def-constraint-height">height</a> set to 480.</p></li> <li><p><a href="#def-constraint-frameRate">frameRate</a> set to 30.</p></li> <li><p><a href="#def-constraint-echoCancellation">echoCancellation</a> set to <code>true</code>.</p></li> </ol> </div> <p>To apply the <dfn class="abstract-op">ApplyConstraints algorithm</dfn> to an <var>object</var>, given <var>newConstraints</var> as an argument, the [=User Agent=] MUST run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>successfulSettings</var> be the result of running the <a>SelectSettings</a> algorithm with <var>newConstraints</var> as the constraint set.</p> </li> <li> <p>If <var>successfulSettings</var> is <code>undefined</code>, let <var>failedConstraint</var> be any <a data-lt="required constraints">required constraint</a> whose fitness distance was infinity for all settings dictionaries examined while executing the <a>SelectSettings</a> algorithm, or <code>""</code> if there isn't one, and then return <var>failedConstraint</var> and abort these steps.</p> </li> <li>In a single operation, remove the existing constraints from <var>object</var>, apply <var>newConstraints</var>, and apply <var>successfulSettings</var> as the current settings.</li> <li>Return <code>undefined</code>.</li> </ol> <div class="note"> <p>Any implementation that has the same result as the algorithm above is an allowed implementation. For instance, the implementation may choose to keep track of the maximum and minimum values for a setting that are OK under the constraints considered, rather than keeping track of all possible values for the setting.</p> </div> <div class="note"> <p>When picking a settings dictionary, the UA can use any information available to it. Examples of such information may be whether the selection is done as part of device selection in getUserMedia, whether the energy usage of the camera varies between the settings dictionaries, or whether using a settings dictionary will cause the device driver to apply resampling.</p> </div> <p>The [=User Agent=] MAY choose new settings for the constrainable properties of the object at any time. When it does so it MUST attempt to satisfy all current Constraints, in the manner described in the algorithm above, let <var>successfulSettings</var> be the resulting new settings, and queue a task to run the following steps:</p> <ol class="algorithm"> <li> <p>Let <var>object</var> be the <code><a>ConstrainablePattern</a></code> object on which new settings for one or more constrainable properties have changed. </p> </li> <li> <p>Set <var>object</var>'s <a data-link-for="constrainable object" data-link-type="attribute">[[\Settings]]</a> internal slot to <var>successfulSettings</var>.</p> </li> </ol> </dd> </dl> </section> </div> <p>An example of Constraints that could be passed into {{MediaStreamTrack/applyConstraints()}} or returned as a value of <code><a>constraints</a></code> is below. It uses the <a href="#constrainable-properties">constrainable properties</a> defined for camera-sourced {{MediaStreamTrack}}s. In this example, all constraints are ideal values, which means results are "best effort" based on the user's specific camera:</p> <pre class="example"> await track.applyConstraints({ width: 1920, height: 1080, frameRate: 30, }); const {width, height, frameRate} = track.getSettings(); console.log(`${width}x${height}x${frameRate}`); // 1920x1080x30, or it might be e.g. // 1280x720x30 as best effort </pre> <p>For finer control, an application can insist on an exact match, provided it's prepared to handle failure: <pre class="example"> try { await track.applyConstraints({ width: {exact: 1920}, height: {exact: 1080}, frameRate: {min: 25, ideal: 30, max: 30}, }); const {width, height, frameRate} = track.getSettings(); console.log(`${width}x${height}x${frameRate}`); // 1920x1080x25-30! } catch (error) { if (error.name != "OverconstrainedError") { throw error; } console.log(`This camera cannot produce the requested ${error.constraint}.`); } </pre> </p> <p>Constraints can also be passed into {{MediaDevices/getUserMedia}}, not just as an initialization convenience, but to influence device selection. In this case, [= list of inherent constrainable track properties | inherent constraints =] are also available. </p> <p>Here's an example of using constraints to prefer a specific camera and microphone from a previous visit, with requirements on dimensions and a preference for stereo, to be applied once granted, and to help find suitable replacements in case the requested devices are no longer available (or in some user agents, overriden by the user).</p> <pre class="example"> try { const stream = await navigator.mediaDevices.getUserMedia({ video: { deviceId: localStorage.camId, width: {min: 800, ideal: 1024, max: 1280}, height: {min: 600} }, audio: { deviceId: localStorage.micId, channelCount: 2 } }); // Granted. Store deviceIds for next time localStorage.camId = stream.getVideoTracks()[0].getSettings().deviceId; localStorage.micId = stream.getAudioTracks()[0].getSettings().deviceId; } catch (error) { if (error.name != "OverconstrainedError") { throw error; } // Overconstrained. No suitable replacements found } </pre> <div class="note"> <p>The above example avoids using <code>{exact: deviceId}</code>, so that browsers can immediately offer a choice between different cameras if your preferred device is not available.</p> <p>The example also stores the <code>deviceId</code>s on every grant, in case they represent a new choice.</p> </div> <p>In contrast, here's an example of using constraints to implement an in-content camera picker. In this case, we use <code>exact</code> and rely solely on a <code>deviceId</code> that comes from the user picking from a list of choices:</p> <pre class="example"> async function switchCameraTrack(freshlyChosenDeviceId, oldTrack) { if (isMobile) { oldTrack.stop(); // Some platforms can only open one camera at a time. } const stream = await navigator.mediaDevices.getUserMedia({ video: { deviceId: {exact: freshlyChosenDeviceId} } }); const [track] = stream.getVideoTracks(); localStorage.camId = track.getSettings().deviceId; return track; } </pre> <p>Here's an example asking for a back camera on a phone, ideally in 720p, but accepting anything close to that. Note how constraints on dimensions are specified in landscape mode:</p> <pre class="example"> async function getBackCamera() { return await navigator.mediaDevices.getUserMedia({ video: { facingMode: {exact: 'environment'}, width: 1280, height: 720 } }); } </pre> <p>Here's an example of "I want a native 16:9 resolution near 720p, but with an exact frame rate of 10 even if not natively available". This needs to be done in two steps: One to discover the native mode, and a second step to apply the custom frame rate. This also shows how to derive constraints from current settings, which may be rotated:</p> <pre class="example"> async function nativeResolutionButDecimatedFrameRate() { const stream = await navigator.mediaDevices.getUserMedia({ video: { resizeMode: 'none', // means native resolution and frame rate width: 1280, height: 720, aspectRatio: 16 / 9 // aspect ratios may not be exactly accurate } }); const [track] = stream.getVideoTracks(); const {width, height, aspectRatio} = track.getSettings(); // Constraints are in landscape, while settings may be rotated (portrait) if (width < height) { [width, height] = [height, width]; aspectRatio = 1 / aspectRatio; } await track.applyConstraints({ resizeMode: 'crop-and-scale', width: {exact: width}, height: {exact: height}, frameRate: {exact: 10}, aspectRatio, }); return stream; } </pre> <div class="note">The above example assumes the <a>primary orientation</a> is landscape.</div> <p>Here's an example showing how to use {{MediaDevices/getSupportedConstraints}}, for cases where a constraint being ignored due to lack of support in a user agent is not tolerated by the application: <pre class="example"> async function getFrontCameraRes() { const supports = navigator.mediaDevices.getSupportedConstraints(); for (const constraint of ["facingMode", "aspectRatio", "resizeMode"]) { if (!(constraint in supports) { throw new OverconstrainedError(constraint, "Not supported"); } } return await navigator.mediaDevices.getUserMedia({ video: { facingMode: {exact: 'user'}, advanced: [ {aspectRatio: 16/9, height: 1080, resizeMode: "none"}, {aspectRatio: 4/3, width: 1280, resizeMode: "none"} ] } }); } </pre> </section> <section> <h2>Constraint Types</h2> <p>The syntax for the specification of the set of valid inputs depends on the type of the values. In addition to the standard atomic types (boolean, long, double, DOMString), valid values include lists of any of the atomic types, plus min-max ranges, as defined below.</p> <p>List values MUST be interpreted as disjunctions. For example, if a property 'facingMode' for a camera is defined as having valid values ["left", "right", "user", "environment"], this means that 'facingMode' can have the values "left", "right", "environment", and "user". Similarly <a>Constraints</a> restricting 'facingMode' to ["user", "left", "right"] would mean that the [=User Agent=] should select a camera (or point the camera, if that is possible) so that "facingMode" is either "user", "left", or "right". This Constraint would thus request that the camera not be facing away from the user, but would allow the [=User Agent=] to allow the user to choose other directions.</p> <div> <pre class="idl" >dictionary DoubleRange { double max; double min; };</pre> <section> <h2>Dictionary <dfn>DoubleRange</dfn> Members</h2> <dl data-link-for="DoubleRange" data-dfn-for="DoubleRange" class= "dictionary-members"> <dt><dfn>max</dfn> of type {{double}}</dt> <dd> <p>The maximum valid value of this property.</p> </dd> <dt><dfn>min</dfn> of type {{double}}</dt> <dd> <p>The minimum value of this Property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary ConstrainDoubleRange : DoubleRange { double exact; double ideal; };</pre> <section> <h2>Dictionary <dfn>ConstrainDoubleRange</dfn> Members</h2> <dl data-link-for="ConstrainDoubleRange" data-dfn-for= "ConstrainDoubleRange" class="dictionary-members"> <dt><dfn>exact</dfn> of type {{double}}</dt> <dd> <p>The exact required value for this property.</p> </dd> <dt><dfn>ideal</dfn> of type {{double}}</dt> <dd> <p>The ideal (target) value for this property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary ULongRange { [Clamp] unsigned long max; [Clamp] unsigned long min; };</pre> <section> <h2>Dictionary <dfn>ULongRange</dfn> Members</h2> <dl data-link-for="ULongRange" data-dfn-for="ULongRange" class= "dictionary-members"> <dt><dfn>max</dfn> of type {{unsigned long}}</dt> <dd> <p>The maximum valid value of this property.</p> </dd> <dt><dfn>min</dfn> of type {{unsigned long}}</dt> <dd> <p>The minimum value of this property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary ConstrainULongRange : ULongRange { [Clamp] unsigned long exact; [Clamp] unsigned long ideal; };</pre> <section> <h2>Dictionary <dfn>ConstrainULongRange</dfn> Members</h2> <dl data-link-for="ConstrainULongRange" data-dfn-for= "ConstrainULongRange" class="dictionary-members"> <dt><dfn>exact</dfn> of type {{unsigned long}}</dt> <dd> <p>The exact required value for this property.</p> </dd> <dt><dfn>ideal</dfn> of type {{unsigned long}}</dt> <dd> <p>The ideal (target) value for this property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary ConstrainBooleanParameters { boolean exact; boolean ideal; };</pre> <section> <h2>Dictionary <dfn>ConstrainBooleanParameters</dfn> Members</h2> <dl data-link-for="ConstrainBooleanParameters" data-dfn-for= "ConstrainBooleanParameters" class="dictionary-members"> <dt><dfn>exact</dfn> of type {{boolean}}</dt> <dd> <p>The exact required value for this property.</p> </dd> <dt><dfn>ideal</dfn> of type {{boolean}}</dt> <dd> <p>The ideal (target) value for this property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >dictionary ConstrainDOMStringParameters { (DOMString or sequence<DOMString>) exact; (DOMString or sequence<DOMString>) ideal; };</pre> <section> <h2>Dictionary <dfn>ConstrainDOMStringParameters</dfn> Members</h2> <dl data-link-for="ConstrainDOMStringParameters" data-dfn-for= "ConstrainDOMStringParameters" class="dictionary-members"> <dt><dfn>exact</dfn> of type <code>({{DOMString}} or sequence<{{DOMString}}>)</code></dt> <dd> <p>The exact required value for this property.</p> </dd> <dt><dfn>ideal</dfn> of type <code>({{DOMString}} or sequence<{{DOMString}}>)</code></dt> <dd> <p>The ideal (target) value for this property.</p> </dd> </dl> </section> </div> <div> <pre class="idl" >typedef ([Clamp] unsigned long or ConstrainULongRange) ConstrainULong;</pre> <div class="idlTypedefDesc"> Throughout this specification, the identifier <dfn>ConstrainULong</dfn> is used to refer to the <span class= "idlTypedefType">([Clamp] unsigned long or ConstrainULongRange)</span> type. </div> </div> <div> <pre class="idl" >typedef (double or ConstrainDoubleRange) ConstrainDouble;</pre> <div class="idlTypedefDesc"> Throughout this specification, the identifier <dfn>ConstrainDouble</dfn> is used to refer to the <span class= "idlTypedefType">(double or ConstrainDoubleRange)</span> type. </div> </div> <div> <pre class="idl" >typedef (boolean or ConstrainBooleanParameters) ConstrainBoolean;</pre> <div class="idlTypedefDesc"> Throughout this specification, the identifier <dfn>ConstrainBoolean</dfn> is used to refer to the <span class= "idlTypedefType">(boolean or ConstrainBooleanParameters)</span> type. </div> </div> <div> <pre class="idl" >typedef (DOMString or sequence<DOMString> or ConstrainDOMStringParameters) ConstrainDOMString;</pre> <div class="idlTypedefDesc"> Throughout this specification, the identifier <dfn>ConstrainDOMString</dfn> is used to refer to the <span class= "idlTypedefType">(DOMString or sequence<DOMString> or ConstrainDOMStringParameters)</span> type. </div> </div> </section> <section id="capabilities"> <h3>Capabilities</h3> <p><dfn data-dfn-type=dictionary>Capabilities</dfn> is a dictionary containing one or more key-value pairs, where each key MUST be a constrainable property, and each value MUST be a subset of the set of values allowed for that property. The exact syntax of the value expression depends on the type of the property. The Capabilities dictionary specifies which constrainable properties that can be applied, as constraints, to the <a>constrainable object</a>. Note that the Capabilities of a <a>constrainable object</a> MAY be a subset of the properties defined in the Web platform, with a subset of the set values for those properties. Note that Capabilities are returned from the [=User Agent=] to the application, and cannot be specified by the application. However, the application can control the Settings that the [=User Agent=] chooses for constrainable properties by means of Constraints.</p> <p>An example of a Capabilities dictionary is shown below. In this case, the <a>constrainable object</a> is a video source with a very limited set of Capabilities.</p> <pre class="example"> { frameRate: {min: 1.0, max: 60.0}, facingMode: ['user', 'left'] } </pre> <p>The next example below points out that capabilities for range values provide ranges for individual constrainable properties, not combinations. This is particularly relevant for video width and height, since the ranges for width and height are reported separately. In the example, if the <a>constrainable object</a> can only provide 640x480 and 800x600 resolutions the relevant capabilities returned would be:</p> <pre class="example"> { width: {min: 640, max: 800}, height: {min: 480, max: 600}, aspectRatio: {min: 4/3, max: 4/3} } </pre> <p>Note in the example above that the aspectRatio would make clear that arbitrary combination of widths and heights are not possible, although it would still suggest that more than two resolutions were available.</p>A specification using the Constrainable Pattern should not subclass the below dictionary, but instead provide its own definition. See {{MediaTrackCapabilities}} for an example. <div class=example>Template: <pre class="idl">dictionary Capabilities {};</pre> </div> </section> <section> <h3>Settings</h3> <p><dfn>Settings</dfn> is a dictionary containing one or more key-value pairs. It MUST contain each key returned in <code>getCapabilities()</code> for which the property is defined on the object type it's returned on; for instance, an audio {{MediaStreamTrack}} has no "width" property. There MUST be a single value for each key and the value MUST be a member of the set defined for that property by <code>getCapabilities()</code>. The <code>Settings</code> dictionary contains the actual values that the User Agent has chosen for the object's constrainable properties. The exact syntax of the value depends on the type of the property.</p> <p>A conforming [=User Agent=] MUST support all the constrainable properties defined in this specification.</p> <p>An example of a Settings dictionary is shown below. This example is not very realistic in that a [=User Agent=] would actually be required to support more constrainable properties than just these.</p> <pre class="example"> { frameRate: 30.0, facingMode: 'user' } </pre>A specification using the Constrainable Pattern should not subclass the below dictionary, but instead provide its own definition. See {{MediaTrackSettings}} for an example. <div class=example>Template: <pre class="idl">dictionary Settings {};</pre> </div> </section> <section id="constraints"> <h3>Constraints and ConstraintSet</h3> <p>Due to the limitations of WebIDL, interfaces implementing the Constrainable Pattern cannot simply subclass Constraints and ConstraintSet as they are defined here. Instead they must provide their own definitions that follow this pattern. See <a href= "#media-track-constraints">MediaTrackConstraints</a> for an example of this.</p> <div class=example>Template: <pre class="idl">dictionary ConstraintSet {};</pre> </div> <p>Each member of a <dfn>ConstraintSet</dfn> corresponds to a constrainable property and specifies a subset of the property's valid Capability values. Applying a ConstraintSet instructs the [=User Agent=] to restrict the settings of the corresponding constrainable properties to the specified values or ranges of values. A given property MAY occur both in the basic Constraints set and in the advanced ConstraintSets list, and MAY occur at most once in each ConstraintSet in the advanced list.</p> <div class=example>Template: <pre class="idl" >dictionary Constraints : ConstraintSet { sequence<ConstraintSet> advanced; };</pre> </div> <section> <h2>Dictionary <dfn>Constraints</dfn> Members</h2> <dl data-link-for="Constraints" data-dfn-for="Constraints" class= "dictionary-members"> <dt><dfn><code>advanced</code></dfn> of type <span class= "idlMemberType">sequence<{{ConstraintSet}}></span></dt> <dd> <p>This is the list of ConstraintSets that the [=User Agent=] MUST attempt to satisfy, in order, skipping only those that cannot be satisfied. The order of these ConstraintSets is significant. In particular, when they are passed as an argument to <code>applyConstraints</code>, the [=User Agent=] MUST try to satisfy them in the order that is specified. Thus if advanced ConstraintSets C1 and C2 can be satisfied individually, but not together, then whichever of C1 and C2 is first in this list will be satisfied, and the other will not. The [=User Agent=] MUST attempt to satisfy all ConstraintSets in the list, even if some cannot be satisfied. Thus, in the preceding example, if constraint C3 is specified after C1 and C2, the [=User Agent=] will attempt to satisfy C3 even though C2 cannot be satisfied. Note that a given property name may occur only once in each ConstraintSet but may occur in more than one ConstraintSet.</p> </dd> </dl> </section> </div> </section> </section> <section> <h2>Examples</h2> <div> <p>This sample code exposes a button. When clicked, the button is disabled and the user is prompted to offer a stream. The user can cause the button to be re-enabled by providing a stream (e.g., giving the page access to the local camera) and then disabling the stream (e.g., revoking that access).</p> <pre class="example"> <button id="startBtn">Start</button> <script> const startBtn = document.getElementById('startBtn'); startBtn.onclick = async () => { try { startBtn.disabled = true; const constraints = { audio: true, video: true }; const stream = await navigator.mediaDevices.getUserMedia(constraints); for (const track of stream.getTracks()) { track.onended = () => { startBtn.disabled = stream.getTracks().some((t) => t.readyState == 'live'); }; } } catch (err) { console.error(err); } }; </script> </pre> </div> <div> <p>This example allows people to take photos of themselves from the local video camera. Note that the Image Capture specification [[?image-capture]] provides a simpler way to accomplish this.</p> <pre class="example"> <script> window.onload = async () => { const video = document.getElementById('monitor'); const canvas = document.getElementById('photo'); const shutter = document.getElementById('shutter'); try { video.srcObject = await navigator.mediaDevices.getUserMedia({video: true}); await new Promise(resolve => video.onloadedmetadata = resolve); canvas.width = video.videoWidth; canvas.height = video.videoHeight; document.getElementById('splash').hidden = true; document.getElementById('app').hidden = false; shutter.onclick = () => canvas.getContext('2d').drawImage(video, 0, 0); } catch (err) { console.error(err); } }; </script> <h1>Snapshot Kiosk</h1> <section id="splash"> <p id="errorMessage">Loading...</p> </section> <section id="app" hidden> <video id="monitor" autoplay></video> <button id="shutter">&#x1F4F7;</button> <canvas id="photo"></canvas> </section> </pre> </div> </section> <section> <h1 id="permissions-integration">Permissions Integration</h1> <p>This specification defines two [=powerful features=] identified by the [=powerful feature/names=] <code><dfn class="permission">"camera"</dfn></code> and <code><dfn class="permission">"microphone"</dfn></code>.</p> <p>It defines the following types and algorithms:</p> <dl> <dt> [=powerful feature/permission descriptor type=] </dt> <dd> <pre class="idl"> dictionary CameraDevicePermissionDescriptor : PermissionDescriptor { boolean panTiltZoom = false; }; </pre> <p> A permission covers access to at least one device of a kind. </p> <p> The semantics of the descriptor is that it queries for access to any device of that kind. Thus, if a query for the <a>"camera"</a> permission returns {{PermissionState/"granted"}}, the client knows that it will get access to one camera without a permission prompt, and if {{PermissionState/"denied"}} is returned, it knows that no getUserMedia request for a camera will succeed. </p> <p> If the User Agent considers permission given to some, but not all, devices of a kind, a query will return {{PermissionState/"granted"}}. </p> <p> If the User Agent considers permission denied to all devices of a kind, a query will return {{PermissionState/"denied"}}. </p> <p> `{name: "camera", panTiltZoom: true}` is [=PermissionDescriptor/stronger than=] `{name: "camera", panTiltZoom: false}`. </p> <p class="note"> A {{PermissionState/"granted"}} permission is no guarantee that getUserMedia will succeed. It only indicates that the user will not be prompted for permission. There are many other things (such as constraints or the camera being in use) that can cause getUserMedia to fail. </p> </dd> <dt> [=powerful feature/permission revocation algorithm=] </dt> <dd> This is the result of calling the [=device permission revocation algorithm=] passing {{PermissionDescriptor/name}} as argument. </dd> </dl> </section> <section> <h1 id=permissions-policy-integration>Permissions Policy Integration</h1> <p>This specification defines two [=policy-controlled features=] identified by the strings <a>"camera"</a> and <a>"microphone"</a>. Both have a [=policy-controlled feature/default allowlist=] of <code class=permissionspolicy>"self"</code>. <div class="note"> <p>A [=document=]'s [=Document/permissions policy=] determines whether any content in that document is allowed to use {{MediaDevices/getUserMedia}} to request camera or microphone respectively. If disabled in any document, no content in the document will be [=allowed to use=] {{MediaDevices/getUserMedia}} to request the camera or microphone respectively. This is enforced by the [=request permission to use=] algorithm. </p> <p>Additionally, {{MediaDevices/enumerateDevices}} will only enumerate devices the document is [=allowed to use=].</p> </div> </section> <section> <h1>Privacy Indicator Requirements</h1> <p class="note"> This specification expresses privacy indicator requirements using algorithms from the viewpoint of a single {{MediaDevices}} object. Implementers are encouraged to extrapolate these principles to unify presentation of indicators to cover multiple {{MediaDevices}} objects that can co-exist on a page due to iframes. </p> <p>For each <var>kind</var> of device that {{MediaDevices/getUserMedia()}} exposes,</p> <ul> <li>Define <var>any<kind>Accessible</var> (e.g. <var>anyAudioAccessible</var>, <var>anyVideoAccessible</var>) as the logical OR of the {{MediaDevices/[[kindsAccessibleMap]]}}<var>[kind]</var> value and all the {{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> values for devices of that kind.</li> <li>Define any<var><kind>Live</var> (e.g. <var>anyAudioLive</var>, <var>anyVideoLive</var>) to be the logical OR of the {{MediaDevices/[[kindsAccessibleMap]]}}<var>[kind]</var> value and all the {{MediaDevices/[[devicesLiveMap]]}}<var>[deviceId]</var> values for devices of that kind.</li> </ul> <p>Define <var>anyAccessible</var> to be the logical OR of all <var>any<kind>Accessible</var> values.</p> <p>Define <var>anyLive</var> to be the logical OR of all <var>any<kind>Live</var> values.</p> <p>Then the following are requirements on the [=User Agent=]:</p> <ul> <li>The [=User Agent=] MUST indicate to the user when the value of <var>anyAccessible</var> changes.</li> <li>The [=User Agent=] MUST indicate to the user when the value of <var>anyLive</var> changes.</li> <li>If the [=User Agent=] provides indication to the user per <var>kind</var>, then for each <var>any<kind>Accessible</var> value and <var>any<kind>Live</var> value, it MUST at minimum indicate when the value changes.</li> <li>If the [=User Agent=] provides indication to the user per <var>device</var>, then for each {{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> value and {{MediaDevices/[[devicesLiveMap]]}}<var>[deviceId]</var> value, it MUST at minimum indicate when the value changes.</li> <li>Any false-to-true transition indicated MUST remain observable for a sufficient time that a reasonably-observant user could become aware of it. This SHOULD be at least 3 seconds.</li> <li>Any of the above transition indications MAY be combined as long as the combined indication cannot transition to false if any of its component indications are still true.</li> </ul> <p>and the following are encouraged behaviors for the [=User Agent=]:</p> <ul> <li>The [=User Agent=] is encouraged to provide ongoing indication of the current state of <var>anyAccessible</var>.</li> <li>The [=User Agent=] is encouraged to provide ongoing indication of the current state of <var>anyLive</var> and to make any generic hardware device indicator light match.</li> <li>If the [=User Agent=] provides indication to the user per <var>kind</var>, then for each any<var><kind>Accessible</var> value and any<var><kind>Live</var> value, it is encouraged to provide ongoing indication of the current state of the value. It is also encouraged to make any device-type-specific hardware indicator light match the corresponding any<var><kind>Live</var> value.</li> <li>If the [=User Agent=] provides indication to the user per <var>device</var>, then for each {{MediaDevices/[[devicesAccessibleMap]]}}<var>[deviceId]</var> value and {{MediaDevices/[[devicesLiveMap]]}}<var>[deviceId]</var> value, it is encouraged to provide ongoing indication of the current state of the value. It is also encouraged to make any device-specific hardware indicator light match the corresponding {{MediaDevices/[[devicesLiveMap]]}}<var>[deviceId]</var> value.</li> <li>Any of the above ongoing indications MAY be used instead of the corresponding required transition indication provided the false-to-true transition requirement is met.</li> </ul> </section> <section> <h1>Privacy and Security Considerations</h1> <p>This section is non-normative; it specifies no new behavior, but instead summarizes information already present in other parts of the specification.</p> <p>This specification extends the Web platform with the ability to manage input devices for media - specifically microphones, and cameras. It also potentially allows exposure of information about other media devices, such as audio output devices (speakers and headphones), but the details of such exposure is relegated to other specifications. Capturing audio and video from the user's microphone and camera exposes personally-identifiable information to applications, and this specification requires obtaining explicit user consent before sharing it.</p> <p>Ahead of camera or microphone capture, an application (the "drive-by web") is only offered the ability to tell whether the user has a camera or a microphone (but not how many). The identifiers for the devices are designed to not be useful for a fingerprint that can track the user between origins, but the presence of camera or microphone ability adds two bits to the fingerprint surface. It recommends to treat the per-origin persistent identifier {{MediaDeviceInfo/deviceId}} as other persistent storage (e.g. cookies) are treated.</p> <p>Once camera or microphone capture has begun, this specification describes how to get access to, and use, media data from the devices mentioned. This data may be sensitive; advice is given that indicators should be supplied to indicate that devices are in use, but both the nature of permission and the indicators of in-use devices are platform decisions.</p> <p>Permission to begin capture may be given on a case-by-case basis, or be persistent. In the case of a case-by-case permission, it is important that the user be able to say "no" in a way that prevents the UI from blocking user interaction until permission is given - either by offering a way to say a "persistent NO" or by not using a modal permissions dialog.</p> <p>Once capture of camera or microphone has begun, the web document gains the ability to list all available media capture devices and their labels. This ability lasts until the web document is closed, and cannot be persisted. In most cases, the labels are stable across origins, and thus potentially provide a way to track a given device across time and origins.</p> <p class="note"> This specification exposes device information of devices other than those in use. This is for backwards compatibility and legacy reasons. Future specifications are advised to not use this model and instead follow best practices as described in the <a href="https://w3ctag.github.io/design-principles/#device-enumeration"> device enumeration design principles</a>. </p> <p>For open web documents where capture has begun or has taken place, or for web documents that [=Document/is in view|are in view=], the <a data-link-type=event>devicechange</a> event can end up being fired at the same time across [=navigables=] and origins each time a new media device is added or removed; user agents can mitigate the risk of correlation of browsing activity across origins by fuzzing the timing of these events, or by deferring their firing until those web documents [=Document/is in view | come into view=].</p> <p>Once a web document gains access to a media stream from a capture device, it also gains access to detailed information about the device, including its range of operating capabilities (e.g. available resolutions for a camera). These operating capabilities are for the most part persistent across browsing sessions and origins, and thus provide a way to track a given device across time and origins.</p> <p>Once access to a video stream from a capture device is obtained, that stream can most likely be used to fingerprint uniquely the said device (e.g. via dead pixel detection). Similarly, once access to an audio stream is obtained, that stream can most likely be used to fingerprint user location down to the level of a room or even simultaneous occupation of a room by disparate users (e.g. via analysis of ambient audio or of unique audio purposely played out of the device speaker). User-level mitigation for both audio and video consists of covering up the camera and/or microphone or revoking permission via [=User Agent=] chrome controls.</p> <p>It is possible to use constraints so that the failure of a getUserMedia call will return information about devices on the system without prompting the user, which increases the surface available for fingerprinting. The [=User Agent=] should consider limiting the rate at which failed getUserMedia calls are allowed in order to limit this additional surface.</p> <p>In the case of a stored persistent permission to begin capture, it is important that it is easy to find the list of granted permissions and revoke permissions that the user wishes to revoke.</p> <p>Once permission has been granted, the [=User Agent=] should make two things readily apparent to the user:</p> <ul> <li>That the page has access to the devices for which permission is given</li> <li>Whether or not any of the devices are presently recording ("on air") indicator</li> </ul> <div class="note"> <p>Developers of sites with stored permissions should be careful that these permissions not be abused. These permissions can be revoked using the [[permissions]] API.</p> <p>In particular, they should not make it possible to automatically send audio or video streams from authorized media devices to an end point that a third party can select.</p> <p>Indeed, if a site offered URLs such as <code>https://webrtc.example.org/?call=<var>user</var></code> that would automatically set up calls and transmit audio/video to <code><var>user</var></code>, it would be open for instance to the following abuse:</p> <p>Users who have granted stored permissions to <code>https://webrtc.example.org/</code> could be tricked to send their audio/video streams to an attacker <code>EvilSpy</code> by following a link or being redirected to <code>https://webrtc.example.org/?user=EvilSpy</code>.</p> </div> </section> <section class="informative"> <h2>Extensibility</h2> <p>Although new versions of this specification may be produced in the future, it is also expected that other standards will need to define new capabilities that build upon those in this specification. The purpose of this section is to provide guidance to creators of such extensions.</p> <p>Any WebIDL-defined interfaces, methods, or attributes in the specification may be extended. Two likely extension points are defining a new media type and defining a new constrainable property.</p> <section> <h2>Defining a new {{MediaStreamTrack/kind}} of media (beyond audio and video)</h2> <p>At a minimum, defining a new media type would require</p> <ul> <li>adding a new getXXXXTracks() method for the type to the {{MediaStream}} interface,</li> <li>describing what a muted or disabled track of that type will render (see [[[#media-flow-and-life-cycle]]]), </li> <li>adding the new type as an additional valid value for the {{MediaStreamTrack/kind}} attribute on the {{MediaStreamTrack}} interface,</li> <li>defining any constrainable properties (see <a href= "#constrainable-properties"></a>) that are applicable to the media type for each <a>source</a>, </li> <li>updating how the {{HTMLMediaElement}} works with a {{MediaStream}} containing a track of the new media type (see <a href="#mediastreams-in-media-elements"></a>), including adding a corollary to [= stream/audible =]/inaudible for the new media type, </li> <li>updating {{MediaDeviceKind}} if the new type has enumerable devices,</li> <li>updating the {{MediaStreamTrack/getCapabilities()}} and {{MediaDevices/getUserMedia()}} descriptions,</li> <li>adding the new type to the {{MediaStreamConstraints}} dictionary,</li> <li>describing any new security and/or privacy considerations (see <a href="#privacy-and-security-considerations"></a>) introduced by the new type, and </li> <li>if the new type requires user authorization, defining new permissions for it, including a new PermissionDescriptor name associated with the new {{MediaStreamTrack/kind}}, and defining how these permissions, along with access starting and ending, as well as muting/disabling, affect any new and/or existing "on-air" and "device accessible" indicator states (see <a href="#mediadevices">MediaDevices</a>). </li> </ul> <p>Additionally, it should include updating</p> <ul> <li>the <a>source</a> definition, </li> <li>the list of media stream <a>consumer</a>s, </li> <li>the description of the {{MediaStreamTrack/label}} attribute on the {{MediaStreamTrack}} interface,</li> <li>the list of sinks (see [[[#the-model-sources-sinks-constraints-and-settings]]]), and </li> <li>the best practice statements referring to video and audio (see <a href="#implementation-suggestions"></a>). </li> </ul> <p>It might also include</p> <ul> <li>explaining how the media is expected to be used by potential <a>consumer</a>s, and </li> <li>giving examples in {{MediaStreamTrackState}} of how such a track might become ended.</li> </ul> </section> <section> <h2>Defining a new constrainable property</h2> <p>This will require thinking through and defining how Constraints, Capabilities, and Settings for the property (see <a href= "#terminology"></a>) will work. The relevant text in {{MediaTrackSupportedConstraints}}, {{MediaTrackCapabilities}}, {{MediaTrackConstraints}}, {{MediaTrackSettings}}, <a href= "#constrainable-properties"></a>, and {{MediaStreamConstraints}} are the model to use.</p> <p>Creators of extension specifications are strongly encouraged to notify the specification maintainers on <a href="https://github.com/w3c/mediacapture-main/issues">the specification repository</a>.<br> Future versions of this specification and others created by the WebRTC Working Group will take into consideration all extensions they are aware of in an attempt to reduce potential usage conflicts.</p> </section> <section> <h2>Defining a new sink for {{MediaStreamTrack}} and {{MediaStream}}</h2> <p>Other specs can define new sinks for {{MediaStream}} and/or {{MediaStreamTrack}}. At a minimum, a new consumer of a {{MediaStreamTrack}} will need to define:</p> <ul> <li>how a {{MediaStreamTrack}} will be consumed in the various states in which it can be, including muted and disabled (see [[[#media-flow-and-life-cycle]]]). </li> </ul> </section> <section> <h2>Defining a new <a>source</a> of {{MediaStreamTrack}}</h2> <p>Other specs can define new sources of {{MediaStreamTrack}}. At a minimum, a new source of {{MediaStreamTrack}} will need to</p> <ul> <li>define a new API to [=create a MediaStreamTrack=] of the relevant {{MediaStreamTrack/kind}}s from this new <a>source</a> ({{MediaDevices/getUserMedia()}} is dedicated to camera and microphone sources), </li> <li>declare which constrainable properties (see <a href= "#constrainable-properties"></a>), if any, are applicable to each {{MediaStreamTrack/kind}} of media this new <a>source</a> produces, and how they work with this source, </li> <li>describe how and when to [=MediaStreamTrack/set a track's muted state=] for this <a>source</a>, </li> <li>describe how and when to <a href="#ends-nostop">end</a> tracks from this <a>source</a>, </li> <li>if capture of the source is a [=powerful feature=] requiring [=express permission=], describe its <a href="#permissions-integration">permissions integration</a> and <a href="#permissions-policy-integration">permissions policy integration</a>, </li> <li>if capture of the source poses a privacy concern, describe its <a href="#privacy-indicator-requirements">privacy indicator requirements</a>. </li> </ul> </section> </section> <section class="appendix"> <h2>Acknowledgements</h2> <p>The editors wish to thank the Working Group chairs and Team Contact, Harald Alvestrand, Stefan Håkansson, Erik Lagerway and Dominique Hazaël-Massieux, for their support. Substantial text in this specification was provided by many people including  Jim Barnett,  Harald Alvestrand,  Travis Leithead,  Josh Soref,  Martin Thomson,  Jan-Ivar Bruaroey,  Peter Thatcher,  Dominique Hazaël-Massieux,  and Stefan Håkansson. Dan Burnett would like to acknowledge the significant support received from Voxeo and Aspect during the development of this specification.</p> </section> </body> </html>

CINXE.COM

Media Capture and Streams