CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="description" content="Keras documentation"> <meta name="author" content="Keras Team"> <link rel="shortcut icon" href="https://keras.io/img/favicon.ico"> <link rel="canonical" href="https://keras.io/2.18/api/losses/probabilistic_losses/" />  <meta property="og:title" content="Keras documentation: Probabilistic losses"> <meta property="og:image" content="https://keras.io/img/logo-k-keras-wb.png"> <meta name="twitter:title" content="Keras documentation: Probabilistic losses"> <meta name="twitter:image" content="https://keras.io/img/k-keras-social.png"> <meta name="twitter:card" content="summary"> <title>Probabilistic losses</title>  <link href="/css/bootstrap.min.css" rel="stylesheet">  <link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;600;700;800&display=swap" rel="stylesheet">  <link href="/css/docs.css" rel="stylesheet"> <link href="/css/monokai.css" rel="stylesheet">  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-5DNGF4N'); </script> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-175165319-128', 'auto'); ga('send', 'pageview'); </script>  <script async defer src="https://buttons.github.io/buttons.js"></script> </head> <body>  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5DNGF4N" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <div class='k-page'> <div class="k-nav" id="nav-menu"> <a href='/'><img src='/img/logo-small.png' class='logo-small' /></a> <div class="nav flex-column nav-pills" role="tablist" aria-orientation="vertical"> <a class="nav-link" href="/about/" role="tab" aria-selected="">About Keras</a> <a class="nav-link" href="/getting_started/" role="tab" aria-selected="">Getting started</a> <a class="nav-link" href="/guides/" role="tab" aria-selected="">Developer guides</a> <a class="nav-link" href="/examples/" role="tab" aria-selected="">Code examples</a> <a class="nav-link" href="/api/" role="tab" aria-selected="">Keras 3 API documentation</a> <a class="nav-link active" href="/2.18/api/" role="tab" aria-selected="">Keras 2 API documentation</a> <a class="nav-sublink" href="/2.18/api/models/">Models API</a> <a class="nav-sublink" href="/2.18/api/layers/">Layers API</a> <a class="nav-sublink" href="/2.18/api/callbacks/">Callbacks API</a> <a class="nav-sublink" href="/2.18/api/optimizers/">Optimizers</a> <a class="nav-sublink" href="/2.18/api/metrics/">Metrics</a> <a class="nav-sublink active" href="/2.18/api/losses/">Losses</a> <a class="nav-sublink2 active" href="/2.18/api/losses/probabilistic_losses/">Probabilistic losses</a> <a class="nav-sublink2" href="/2.18/api/losses/regression_losses/">Regression losses</a> <a class="nav-sublink2" href="/2.18/api/losses/hinge_losses/">Hinge losses for "maximum-margin" classification</a> <a class="nav-sublink" href="/2.18/api/data_loading/">Data loading</a> <a class="nav-sublink" href="/2.18/api/datasets/">Built-in small datasets</a> <a class="nav-sublink" href="/2.18/api/applications/">Keras Applications</a> <a class="nav-sublink" href="/2.18/api/mixed_precision/">Mixed precision</a> <a class="nav-sublink" href="/2.18/api/utils/">Utilities</a> <a class="nav-link" href="/keras_tuner/" role="tab" aria-selected="">KerasTuner: Hyperparam Tuning</a> <a class="nav-link" href="/keras_hub/" role="tab" aria-selected="">KerasHub: Pretrained Models</a> </div> </div> <div class='k-main'> <div class='k-main-top'> <script> function displayDropdownMenu() { e = document.getElementById("nav-menu"); if (e.style.display == "block") { e.style.display = "none"; } else { e.style.display = "block"; document.getElementById("dropdown-nav").style.display = "block"; } } function resetMobileUI() { if (window.innerWidth <= 840) { document.getElementById("nav-menu").style.display = "none"; document.getElementById("dropdown-nav").style.display = "block"; } else { document.getElementById("nav-menu").style.display = "block"; document.getElementById("dropdown-nav").style.display = "none"; } var navmenu = document.getElementById("nav-menu"); var menuheight = navmenu.clientHeight; var kmain = document.getElementById("k-main-id"); kmain.style.minHeight = (menuheight + 100) + 'px'; } window.onresize = resetMobileUI; window.addEventListener("load", (event) => { resetMobileUI() }); </script> <div id='dropdown-nav' onclick="displayDropdownMenu();"> <svg viewBox="-20 -20 120 120" width="60" height="60"> <rect width="100" height="20"></rect> <rect y="30" width="100" height="20"></rect> <rect y="60" width="100" height="20"></rect> </svg> </div> <form class="bd-search d-flex align-items-center k-search-form" id="search-form"> <input type="search" class="k-search-input" id="search-input" placeholder="Search Keras documentation..." aria-label="Search Keras documentation..." autocomplete="off"> <button class="k-search-btn"> <svg width="13" height="13" viewBox="0 0 13 13"><title>search</title><path d="m4.8495 7.8226c0.82666 0 1.5262-0.29146 2.0985-0.87438 0.57232-0.58292 0.86378-1.2877 0.87438-2.1144 0.010599-0.82666-0.28086-1.5262-0.87438-2.0985-0.59352-0.57232-1.293-0.86378-2.0985-0.87438-0.8055-0.010599-1.5103 0.28086-2.1144 0.87438-0.60414 0.59352-0.8956 1.293-0.87438 2.0985 0.021197 0.8055 0.31266 1.5103 0.87438 2.1144 0.56172 0.60414 1.2665 0.8956 2.1144 0.87438zm4.4695 0.2115 3.681 3.6819-1.259 1.284-3.6817-3.7 0.0019784-0.69479-0.090043-0.098846c-0.87973 0.76087-1.92 1.1413-3.1207 1.1413-1.3553 0-2.5025-0.46363-3.4417-1.3909s-1.4088-2.0686-1.4088-3.4239c0-1.3553 0.4696-2.4966 1.4088-3.4239 0.9392-0.92727 2.0864-1.3969 3.4417-1.4088 1.3553-0.011889 2.4906 0.45771 3.406 1.4088 0.9154 0.95107 1.379 2.0924 1.3909 3.4239 0 1.2126-0.38043 2.2588-1.1413 3.1385l0.098834 0.090049z"></path></svg> </button> </form> <script> var form = document.getElementById('search-form'); form.onsubmit = function(e) { e.preventDefault(); var query = document.getElementById('search-input').value; window.location.href = '/search.html?query=' + query; return False } </script> </div> <div class='k-main-inner' id='k-main-id'> <div class='k-location-slug'> ► <a href='/2.18/api/'>Keras 2 API documentation</a> / <a href='/2.18/api/losses/'>Losses</a> / Probabilistic losses </div> <div class='k-content'> <h1 id="probabilistic-losses">Probabilistic losses</h1> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L561">[source]</a> <h3 id="binarycrossentropy-class"><code>BinaryCrossentropy</code> class</h3> <div class="codehilite"><pre><code>tf_keras.losses.BinaryCrossentropy( from_logits=False, label_smoothing=0.0, axis=-1, reduction="auto", name="binary_crossentropy", ) </code></pre></div> Computes the cross-entropy loss between true labels and predicted labels. Use this cross-entropy loss for binary (0 or 1) classification applications. The loss function requires the following inputs: <ul> <li><code>y_true</code> (true label): This is either 0 or 1.</li> <li><code>y_pred</code> (predicted value): This is the model's prediction, i.e, a single floating-point value which either represents a <a href="https://en.wikipedia.org/wiki/Logit">logit</a>, (i.e, value in [-inf, inf] when <code>from_logits=True</code>) or a probability (i.e, value in [0., 1.] when <code>from_logits=False</code>).</li> </ul> Recommended Usage: (set <code>from_logits=True</code>) With <a href="https://www.tensorflow.org/api_docs/python/tf/keras"><code>tf.keras</code></a> API: <div class="codehilite"><pre><code>model.compile( loss=tf.keras.losses.BinaryCrossentropy(from_logits=True), .... ) </code></pre></div> As a standalone function: <div class="codehilite"><pre><code>>>> # Example 1: (batch_size = 1, number of samples = 4) >>> y_true = [0, 1, 0, 0] >>> y_pred = [-18.6, 0.51, 2.94, -12.8] >>> bce = tf.keras.losses.BinaryCrossentropy(from_logits=True) >>> bce(y_true, y_pred).numpy() 0.865 </code></pre></div> <div class="codehilite"><pre><code>>>> # Example 2: (batch_size = 2, number of samples = 4) >>> y_true = [[0, 1], [0, 0]] >>> y_pred = [[-18.6, 0.51], [2.94, -12.8]] >>> # Using default 'auto'/'sum_over_batch_size' reduction type. >>> bce = tf.keras.losses.BinaryCrossentropy(from_logits=True) >>> bce(y_true, y_pred).numpy() 0.865 >>> # Using 'sample_weight' attribute >>> bce(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy() 0.243 >>> # Using 'sum' reduction` type. >>> bce = tf.keras.losses.BinaryCrossentropy(from_logits=True, ... reduction=tf.keras.losses.Reduction.SUM) >>> bce(y_true, y_pred).numpy() 1.730 >>> # Using 'none' reduction type. >>> bce = tf.keras.losses.BinaryCrossentropy(from_logits=True, ... reduction=tf.keras.losses.Reduction.NONE) >>> bce(y_true, y_pred).numpy() array([0.235, 1.496], dtype=float32) </code></pre></div> Default Usage: (set <code>from_logits=False</code>) <div class="codehilite"><pre><code>>>> # Make the following updates to the above "Recommended Usage" section >>> # 1. Set `from_logits=False` >>> tf.keras.losses.BinaryCrossentropy() # OR ...('from_logits=False') >>> # 2. Update `y_pred` to use probabilities instead of logits >>> y_pred = [0.6, 0.3, 0.2, 0.8] # OR [[0.6, 0.3], [0.2, 0.8]] </code></pre></div> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L844">[source]</a> <h3 id="categoricalcrossentropy-class"><code>CategoricalCrossentropy</code> class</h3> <div class="codehilite"><pre><code>tf_keras.losses.CategoricalCrossentropy( from_logits=False, label_smoothing=0.0, axis=-1, reduction="auto", name="categorical_crossentropy", ) </code></pre></div> Computes the crossentropy loss between the labels and predictions. Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided in a <code>one_hot</code> representation. If you want to provide labels as integers, please use <code>SparseCategoricalCrossentropy</code> loss. There should be <code># classes</code> floating point values per feature. In the snippet below, there is <code># classes</code> floating pointing values per example. The shape of both <code>y_pred</code> and <code>y_true</code> are <code>[batch_size, num_classes]</code>. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [[0, 1, 0], [0, 0, 1]] >>> y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] >>> # Using 'auto'/'sum_over_batch_size' reduction type. >>> cce = tf.keras.losses.CategoricalCrossentropy() >>> cce(y_true, y_pred).numpy() 1.177 </code></pre></div> <div class="codehilite"><pre><code>>>> # Calling with 'sample_weight'. >>> cce(y_true, y_pred, sample_weight=tf.constant([0.3, 0.7])).numpy() 0.814 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'sum' reduction type. >>> cce = tf.keras.losses.CategoricalCrossentropy( ... reduction=tf.keras.losses.Reduction.SUM) >>> cce(y_true, y_pred).numpy() 2.354 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'none' reduction type. >>> cce = tf.keras.losses.CategoricalCrossentropy( ... reduction=tf.keras.losses.Reduction.NONE) >>> cce(y_true, y_pred).numpy() array([0.0513, 2.303], dtype=float32) </code></pre></div> Usage with the <code>compile()</code> API: <div class="codehilite"><pre><code>model.compile(optimizer='sgd', loss=tf.keras.losses.CategoricalCrossentropy()) </code></pre></div> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L1073">[source]</a> <h3 id="sparsecategoricalcrossentropy-class"><code>SparseCategoricalCrossentropy</code> class</h3> <div class="codehilite"><pre><code>tf_keras.losses.SparseCategoricalCrossentropy( from_logits=False, ignore_class=None, reduction="auto", name="sparse_categorical_crossentropy", ) </code></pre></div> Computes the crossentropy loss between the labels and predictions. Use this crossentropy loss function when there are two or more label classes. We expect labels to be provided as integers. If you want to provide labels using <code>one-hot</code> representation, please use <code>CategoricalCrossentropy</code> loss. There should be <code># classes</code> floating point values per feature for <code>y_pred</code> and a single floating point value per feature for <code>y_true</code>. In the snippet below, there is a single floating point value per example for <code>y_true</code> and <code># classes</code> floating pointing values per example for <code>y_pred</code>. The shape of <code>y_true</code> is <code>[batch_size]</code> and the shape of <code>y_pred</code> is <code>[batch_size, num_classes]</code>. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [1, 2] >>> y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] >>> # Using 'auto'/'sum_over_batch_size' reduction type. >>> scce = tf.keras.losses.SparseCategoricalCrossentropy() >>> scce(y_true, y_pred).numpy() 1.177 </code></pre></div> <div class="codehilite"><pre><code>>>> # Calling with 'sample_weight'. >>> scce(y_true, y_pred, sample_weight=tf.constant([0.3, 0.7])).numpy() 0.814 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'sum' reduction type. >>> scce = tf.keras.losses.SparseCategoricalCrossentropy( ... reduction=tf.keras.losses.Reduction.SUM) >>> scce(y_true, y_pred).numpy() 2.354 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'none' reduction type. >>> scce = tf.keras.losses.SparseCategoricalCrossentropy( ... reduction=tf.keras.losses.Reduction.NONE) >>> scce(y_true, y_pred).numpy() array([0.0513, 2.303], dtype=float32) </code></pre></div> Usage with the <code>compile()</code> API: <div class="codehilite"><pre><code>model.compile(optimizer='sgd', loss=tf.keras.losses.SparseCategoricalCrossentropy()) </code></pre></div> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L1422">[source]</a> <h3 id="poisson-class"><code>Poisson</code> class</h3> <div class="codehilite"><pre><code>tf_keras.losses.Poisson(reduction="auto", name="poisson") </code></pre></div> Computes the Poisson loss between <code>y_true</code> & <code>y_pred</code>. <code>loss = y_pred - y_true * log(y_pred)</code> Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [[0., 1.], [0., 0.]] >>> y_pred = [[1., 1.], [0., 0.]] >>> # Using 'auto'/'sum_over_batch_size' reduction type. >>> p = tf.keras.losses.Poisson() >>> p(y_true, y_pred).numpy() 0.5 </code></pre></div> <div class="codehilite"><pre><code>>>> # Calling with 'sample_weight'. >>> p(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy() 0.4 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'sum' reduction type. >>> p = tf.keras.losses.Poisson( ... reduction=tf.keras.losses.Reduction.SUM) >>> p(y_true, y_pred).numpy() 0.999 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'none' reduction type. >>> p = tf.keras.losses.Poisson( ... reduction=tf.keras.losses.Reduction.NONE) >>> p(y_true, y_pred).numpy() array([0.999, 0.], dtype=float32) </code></pre></div> Usage with the <code>compile()</code> API: <div class="codehilite"><pre><code>model.compile(optimizer='sgd', loss=tf.keras.losses.Poisson()) </code></pre></div> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L2488">[source]</a> <h3 id="binarycrossentropy-function"><code>binary_crossentropy</code> function</h3> <div class="codehilite"><pre><code>tf_keras.losses.binary_crossentropy( y_true, y_pred, from_logits=False, label_smoothing=0.0, axis=-1 ) </code></pre></div> Computes the binary crossentropy loss. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [[0, 1], [0, 0]] >>> y_pred = [[0.6, 0.4], [0.4, 0.6]] >>> loss = tf.keras.losses.binary_crossentropy(y_true, y_pred) >>> assert loss.shape == (2,) >>> loss.numpy() array([0.916 , 0.714], dtype=float32) </code></pre></div> Arguments <ul> <li>y_true: Ground truth values. shape = <code>[batch_size, d0, .. dN]</code>.</li> <li>y_pred: The predicted values. shape = <code>[batch_size, d0, .. dN]</code>.</li> <li>from_logits: Whether <code>y_pred</code> is expected to be a logits tensor. By default, we assume that <code>y_pred</code> encodes a probability distribution.</li> <li>label_smoothing: Float in [0, 1]. If > <code>0</code> then smooth the labels by squeezing them towards 0.5 That is, using <code>1. - 0.5 * label_smoothing</code> for the target class and <code>0.5 * label_smoothing</code> for the non-target class.</li> <li>axis: The axis along which the mean is computed. Defaults to -1.</li> </ul> Returns Binary crossentropy loss value. shape = <code>[batch_size, d0, .. dN-1]</code>. <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L2159">[source]</a> <h3 id="categoricalcrossentropy-function"><code>categorical_crossentropy</code> function</h3> <div class="codehilite"><pre><code>tf_keras.losses.categorical_crossentropy( y_true, y_pred, from_logits=False, label_smoothing=0.0, axis=-1 ) </code></pre></div> Computes the categorical crossentropy loss. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [[0, 1, 0], [0, 0, 1]] >>> y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] >>> loss = tf.keras.losses.categorical_crossentropy(y_true, y_pred) >>> assert loss.shape == (2,) >>> loss.numpy() array([0.0513, 2.303], dtype=float32) </code></pre></div> Arguments <ul> <li>y_true: Tensor of one-hot true targets.</li> <li>y_pred: Tensor of predicted targets.</li> <li>from_logits: Whether <code>y_pred</code> is expected to be a logits tensor. By default, we assume that <code>y_pred</code> encodes a probability distribution.</li> <li>label_smoothing: Float in [0, 1]. If > <code>0</code> then smooth the labels. For example, if <code>0.1</code>, use <code>0.1 / num_classes</code> for non-target labels and <code>0.9 + 0.1 / num_classes</code> for target labels.</li> <li>axis: Defaults to -1. The dimension along which the entropy is computed.</li> </ul> Returns Categorical crossentropy loss value. <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L2403">[source]</a> <h3 id="sparsecategoricalcrossentropy-function"><code>sparse_categorical_crossentropy</code> function</h3> <div class="codehilite"><pre><code>tf_keras.losses.sparse_categorical_crossentropy( y_true, y_pred, from_logits=False, axis=-1, ignore_class=None ) </code></pre></div> Computes the sparse categorical crossentropy loss. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [1, 2] >>> y_pred = [[0.05, 0.95, 0], [0.1, 0.8, 0.1]] >>> loss = tf.keras.losses.sparse_categorical_crossentropy(y_true, y_pred) >>> assert loss.shape == (2,) >>> loss.numpy() array([0.0513, 2.303], dtype=float32) </code></pre></div> <div class="codehilite"><pre><code>>>> y_true = [[[ 0, 2], ... [-1, -1]], ... [[ 0, 2], ... [-1, -1]]] >>> y_pred = [[[[1.0, 0.0, 0.0], [0.0, 0.0, 1.0]], ... [[0.2, 0.5, 0.3], [0.0, 1.0, 0.0]]], ... [[[1.0, 0.0, 0.0], [0.0, 0.5, 0.5]], ... [[0.2, 0.5, 0.3], [0.0, 1.0, 0.0]]]] >>> loss = tf.keras.losses.sparse_categorical_crossentropy( ... y_true, y_pred, ignore_class=-1) >>> loss.numpy() array([[[2.3841855e-07, 2.3841855e-07], [0.0000000e+00, 0.0000000e+00]], [[2.3841855e-07, 6.9314730e-01], [0.0000000e+00, 0.0000000e+00]]], dtype=float32) </code></pre></div> Arguments <ul> <li>y_true: Ground truth values.</li> <li>y_pred: The predicted values.</li> <li>from_logits: Whether <code>y_pred</code> is expected to be a logits tensor. By default, we assume that <code>y_pred</code> encodes a probability distribution.</li> <li>axis: Defaults to -1. The dimension along which the entropy is computed.</li> <li>ignore_class: Optional integer. The ID of a class to be ignored during loss computation. This is useful, for example, in segmentation problems featuring a "void" class (commonly -1 or 255) in segmentation maps. By default (<code>ignore_class=None</code>), all classes are considered.</li> </ul> Returns Sparse categorical crossentropy loss value. <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L2761">[source]</a> <h3 id="poisson-function"><code>poisson</code> function</h3> <div class="codehilite"><pre><code>tf_keras.losses.poisson(y_true, y_pred) </code></pre></div> Computes the Poisson loss between y_true and y_pred. The Poisson loss is the mean of the elements of the <code>Tensor</code> <code>y_pred - y_true * log(y_pred)</code>. Standalone usage: <div class="codehilite"><pre><code>>>> y_true = np.random.randint(0, 2, size=(2, 3)) >>> y_pred = np.random.random(size=(2, 3)) >>> loss = tf.keras.losses.poisson(y_true, y_pred) >>> assert loss.shape == (2,) >>> y_pred = y_pred + 1e-7 >>> assert np.allclose( ... loss.numpy(), np.mean(y_pred - y_true * np.log(y_pred), axis=-1), ... atol=1e-5) </code></pre></div> Arguments <ul> <li>y_true: Ground truth values. shape = <code>[batch_size, d0, .. dN]</code>.</li> <li>y_pred: The predicted values. shape = <code>[batch_size, d0, .. dN]</code>.</li> </ul> Returns Poisson loss value. shape = <code>[batch_size, d0, .. dN-1]</code>. Raises <ul> <li>InvalidArgumentError: If <code>y_true</code> and <code>y_pred</code> have incompatible shapes.</li> </ul> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L1539">[source]</a> <h3 id="kldivergence-class"><code>KLDivergence</code> class</h3> <div class="codehilite"><pre><code>tf_keras.losses.KLDivergence(reduction="auto", name="kl_divergence") </code></pre></div> Computes Kullback-Leibler divergence loss between <code>y_true</code> & <code>y_pred</code>. <code>loss = y_true * log(y_true / y_pred)</code> See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence Standalone usage: <div class="codehilite"><pre><code>>>> y_true = [[0, 1], [0, 0]] >>> y_pred = [[0.6, 0.4], [0.4, 0.6]] >>> # Using 'auto'/'sum_over_batch_size' reduction type. >>> kl = tf.keras.losses.KLDivergence() >>> kl(y_true, y_pred).numpy() 0.458 </code></pre></div> <div class="codehilite"><pre><code>>>> # Calling with 'sample_weight'. >>> kl(y_true, y_pred, sample_weight=[0.8, 0.2]).numpy() 0.366 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'sum' reduction type. >>> kl = tf.keras.losses.KLDivergence( ... reduction=tf.keras.losses.Reduction.SUM) >>> kl(y_true, y_pred).numpy() 0.916 </code></pre></div> <div class="codehilite"><pre><code>>>> # Using 'none' reduction type. >>> kl = tf.keras.losses.KLDivergence( ... reduction=tf.keras.losses.Reduction.NONE) >>> kl(y_true, y_pred).numpy() array([0.916, -3.08e-06], dtype=float32) </code></pre></div> Usage with the <code>compile()</code> API: <div class="codehilite"><pre><code>model.compile(optimizer='sgd', loss=tf.keras.losses.KLDivergence()) </code></pre></div> <hr /> <a href="https://github.com/keras-team/tf-keras/tree/v2.18.0/tf_keras/losses.py#L2715">[source]</a> <h3 id="kldivergence-function"><code>kl_divergence</code> function</h3> <div class="codehilite"><pre><code>tf_keras.losses.kl_divergence(y_true, y_pred) </code></pre></div> Computes Kullback-Leibler divergence loss between <code>y_true</code> & <code>y_pred</code>. <code>loss = y_true * log(y_true / y_pred)</code> See: https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence Standalone usage: <div class="codehilite"><pre><code>>>> y_true = np.random.randint(0, 2, size=(2, 3)).astype(np.float64) >>> y_pred = np.random.random(size=(2, 3)) >>> loss = tf.keras.losses.kullback_leibler_divergence(y_true, y_pred) >>> assert loss.shape == (2,) >>> y_true = tf.keras.backend.clip(y_true, 1e-7, 1) >>> y_pred = tf.keras.backend.clip(y_pred, 1e-7, 1) >>> assert np.array_equal( ... loss.numpy(), np.sum(y_true * np.log(y_true / y_pred), axis=-1)) </code></pre></div> Arguments <ul> <li>y_true: Tensor of true targets.</li> <li>y_pred: Tensor of predicted targets.</li> </ul> Returns A <code>Tensor</code> with loss. Raises <ul> <li>TypeError: If <code>y_true</code> cannot be cast to the <code>y_pred.dtype</code>.</li> </ul> <hr /> </div> <div class='k-outline'> <div class='k-outline-depth-1'> <a href='#probabilistic-losses'>Probabilistic losses</a> </div> <div class='k-outline-depth-3'> <a href='#binarycrossentropy-class'><code>BinaryCrossentropy</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#categoricalcrossentropy-class'><code>CategoricalCrossentropy</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#sparsecategoricalcrossentropy-class'><code>SparseCategoricalCrossentropy</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#poisson-class'><code>Poisson</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#binarycrossentropy-function'><code>binary_crossentropy</code> function</a> </div> <div class='k-outline-depth-3'> <a href='#categoricalcrossentropy-function'><code>categorical_crossentropy</code> function</a> </div> <div class='k-outline-depth-3'> <a href='#sparsecategoricalcrossentropy-function'><code>sparse_categorical_crossentropy</code> function</a> </div> <div class='k-outline-depth-3'> <a href='#poisson-function'><code>poisson</code> function</a> </div> <div class='k-outline-depth-3'> <a href='#kldivergence-class'><code>KLDivergence</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#kldivergence-function'><code>kl_divergence</code> function</a> </div> </div> </div> </div> </div> </body> <footer style="float: left; width: 100%; padding: 1em; border-top: solid 1px #bbb;"> <a href="https://policies.google.com/terms">Terms</a> | <a href="https://policies.google.com/privacy">Privacy</a> </footer> </html>