The base Layer class

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="description" content="Keras documentation"> <meta name="author" content="Keras Team"> <link rel="shortcut icon" href="https://keras.io/img/favicon.ico"> <link rel="canonical" href="https://keras.io/api/layers/base_layer/" />  <meta property="og:title" content="Keras documentation: The base Layer class"> <meta property="og:image" content="https://keras.io/img/logo-k-keras-wb.png"> <meta name="twitter:title" content="Keras documentation: The base Layer class"> <meta name="twitter:image" content="https://keras.io/img/k-keras-social.png"> <meta name="twitter:card" content="summary"> <title>The base Layer class</title>  <link href="/css/bootstrap.min.css" rel="stylesheet">  <link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;600;700;800&display=swap" rel="stylesheet">  <link href="/css/docs.css" rel="stylesheet"> <link href="/css/monokai.css" rel="stylesheet">  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-5DNGF4N'); </script> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-175165319-128', 'auto'); ga('send', 'pageview'); </script>  <script async defer src="https://buttons.github.io/buttons.js"></script> </head> <body>  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5DNGF4N" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <div class='k-page'> <div class="k-nav" id="nav-menu"> <a href='/'><img src='/img/logo-small.png' class='logo-small' /></a> <div class="nav flex-column nav-pills" role="tablist" aria-orientation="vertical"> <a class="nav-link" href="/about/" role="tab" aria-selected="">About Keras</a> <a class="nav-link" href="/getting_started/" role="tab" aria-selected="">Getting started</a> <a class="nav-link" href="/guides/" role="tab" aria-selected="">Developer guides</a> <a class="nav-link active" href="/api/" role="tab" aria-selected="">Keras 3 API documentation</a> <a class="nav-sublink" href="/api/models/">Models API</a> <a class="nav-sublink active" href="/api/layers/">Layers API</a> <a class="nav-sublink2 active" href="/api/layers/base_layer/">The base Layer class</a> <a class="nav-sublink2" href="/api/layers/activations/">Layer activations</a> <a class="nav-sublink2" href="/api/layers/initializers/">Layer weight initializers</a> <a class="nav-sublink2" href="/api/layers/regularizers/">Layer weight regularizers</a> <a class="nav-sublink2" href="/api/layers/constraints/">Layer weight constraints</a> <a class="nav-sublink2" href="/api/layers/core_layers/">Core layers</a> <a class="nav-sublink2" href="/api/layers/convolution_layers/">Convolution layers</a> <a class="nav-sublink2" href="/api/layers/pooling_layers/">Pooling layers</a> <a class="nav-sublink2" href="/api/layers/recurrent_layers/">Recurrent layers</a> <a class="nav-sublink2" href="/api/layers/preprocessing_layers/">Preprocessing layers</a> <a class="nav-sublink2" href="/api/layers/normalization_layers/">Normalization layers</a> <a class="nav-sublink2" href="/api/layers/regularization_layers/">Regularization layers</a> <a class="nav-sublink2" href="/api/layers/attention_layers/">Attention layers</a> <a class="nav-sublink2" href="/api/layers/reshaping_layers/">Reshaping layers</a> <a class="nav-sublink2" href="/api/layers/merging_layers/">Merging layers</a> <a class="nav-sublink2" href="/api/layers/activation_layers/">Activation layers</a> <a class="nav-sublink2" href="/api/layers/backend_specific_layers/">Backend-specific layers</a> <a class="nav-sublink" href="/api/callbacks/">Callbacks API</a> <a class="nav-sublink" href="/api/ops/">Ops API</a> <a class="nav-sublink" href="/api/optimizers/">Optimizers</a> <a class="nav-sublink" href="/api/metrics/">Metrics</a> <a class="nav-sublink" href="/api/losses/">Losses</a> <a class="nav-sublink" href="/api/data_loading/">Data loading</a> <a class="nav-sublink" href="/api/datasets/">Built-in small datasets</a> <a class="nav-sublink" href="/api/applications/">Keras Applications</a> <a class="nav-sublink" href="/api/mixed_precision/">Mixed precision</a> <a class="nav-sublink" href="/api/distribution/">Multi-device distribution</a> <a class="nav-sublink" href="/api/random/">RNG API</a> <a class="nav-sublink" href="/api/utils/">Utilities</a> <a class="nav-sublink" href="/api/keras_tuner/">KerasTuner</a> <a class="nav-sublink" href="/api/keras_cv/">KerasCV</a> <a class="nav-sublink" href="/api/keras_nlp/">KerasNLP</a> <a class="nav-sublink" href="/api/keras_hub/">KerasHub</a> <a class="nav-link" href="/2.18/api/" role="tab" aria-selected="">Keras 2 API documentation</a> <a class="nav-link" href="/examples/" role="tab" aria-selected="">Code examples</a> <a class="nav-link" href="/keras_tuner/" role="tab" aria-selected="">KerasTuner: Hyperparameter Tuning</a> <a class="nav-link" href="/keras_hub/" role="tab" aria-selected="">KerasHub: Pretrained Models</a> <a class="nav-link" href="/keras_cv/" role="tab" aria-selected="">KerasCV: Computer Vision Workflows</a> <a class="nav-link" href="/keras_nlp/" role="tab" aria-selected="">KerasNLP: Natural Language Workflows</a> </div> </div> <div class='k-main'> <div class='k-main-top'> <script> function displayDropdownMenu() { e = document.getElementById("nav-menu"); if (e.style.display == "block") { e.style.display = "none"; } else { e.style.display = "block"; document.getElementById("dropdown-nav").style.display = "block"; } } function resetMobileUI() { if (window.innerWidth <= 840) { document.getElementById("nav-menu").style.display = "none"; document.getElementById("dropdown-nav").style.display = "block"; } else { document.getElementById("nav-menu").style.display = "block"; document.getElementById("dropdown-nav").style.display = "none"; } var navmenu = document.getElementById("nav-menu"); var menuheight = navmenu.clientHeight; var kmain = document.getElementById("k-main-id"); kmain.style.minHeight = (menuheight + 100) + 'px'; } window.onresize = resetMobileUI; window.addEventListener("load", (event) => { resetMobileUI() }); </script> <div id='dropdown-nav' onclick="displayDropdownMenu();"> <svg viewBox="-20 -20 120 120" width="60" height="60"> <rect width="100" height="20"></rect> <rect y="30" width="100" height="20"></rect> <rect y="60" width="100" height="20"></rect> </svg> </div> <form class="bd-search d-flex align-items-center k-search-form" id="search-form"> <input type="search" class="k-search-input" id="search-input" placeholder="Search Keras documentation..." aria-label="Search Keras documentation..." autocomplete="off"> <button class="k-search-btn"> <svg width="13" height="13" viewBox="0 0 13 13"><title>search</title><path d="m4.8495 7.8226c0.82666 0 1.5262-0.29146 2.0985-0.87438 0.57232-0.58292 0.86378-1.2877 0.87438-2.1144 0.010599-0.82666-0.28086-1.5262-0.87438-2.0985-0.59352-0.57232-1.293-0.86378-2.0985-0.87438-0.8055-0.010599-1.5103 0.28086-2.1144 0.87438-0.60414 0.59352-0.8956 1.293-0.87438 2.0985 0.021197 0.8055 0.31266 1.5103 0.87438 2.1144 0.56172 0.60414 1.2665 0.8956 2.1144 0.87438zm4.4695 0.2115 3.681 3.6819-1.259 1.284-3.6817-3.7 0.0019784-0.69479-0.090043-0.098846c-0.87973 0.76087-1.92 1.1413-3.1207 1.1413-1.3553 0-2.5025-0.46363-3.4417-1.3909s-1.4088-2.0686-1.4088-3.4239c0-1.3553 0.4696-2.4966 1.4088-3.4239 0.9392-0.92727 2.0864-1.3969 3.4417-1.4088 1.3553-0.011889 2.4906 0.45771 3.406 1.4088 0.9154 0.95107 1.379 2.0924 1.3909 3.4239 0 1.2126-0.38043 2.2588-1.1413 3.1385l0.098834 0.090049z"></path></svg> </button> </form> <script> var form = document.getElementById('search-form'); form.onsubmit = function(e) { e.preventDefault(); var query = document.getElementById('search-input').value; window.location.href = '/search.html?query=' + query; return False } </script> </div> <div class='k-main-inner' id='k-main-id'> <div class='k-location-slug'> ► <a href='/api/'>Keras 3 API documentation</a> / <a href='/api/layers/'>Layers API</a> / The base Layer class </div> <div class='k-content'> <h1 id="the-base-layer-class">The base Layer class</h1> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L61">[source]</a> <h3 id="layer-class"><code>Layer</code> class</h3> <div class="codehilite"><pre><code>keras.layers.Layer( activity_regularizer=None, trainable=True, dtype=None, autocast=True, name=None, **kwargs ) </code></pre></div> This is the class from which all layers inherit. A layer is a callable object that takes as input one or more tensors and that outputs one or more tensors. It involves computation, defined in the <code>call()</code> method, and a state (weight variables). State can be created: <ul> <li>in <code>__init__()</code>, for instance via <code>self.add_weight()</code>;</li> <li>in the optional <code>build()</code> method, which is invoked by the first <code>__call__()</code> to the layer, and supplies the shape(s) of the input(s), which may not have been known at initialization time.</li> </ul> Layers are recursively composable: If you assign a Layer instance as an attribute of another Layer, the outer layer will start tracking the weights created by the inner layer. Nested layers should be instantiated in the <code>__init__()</code> method or <code>build()</code> method. Users will just instantiate a layer and then treat it as a callable. Arguments <ul> <li>trainable: Boolean, whether the layer's variables should be trainable.</li> <li>name: String name of the layer.</li> <li>dtype: The dtype of the layer's computations and weights. Can also be a <code>keras.DTypePolicy</code>, which allows the computation and weight dtype to differ. Defaults to <code>None</code>. <code>None</code> means to use <code>keras.config.dtype_policy()</code>, which is a <code>float32</code> policy unless set to different value (via <code>keras.config.set_dtype_policy()</code>).</li> </ul> Attributes <ul> <li>name: The name of the layer (string).</li> <li>dtype: Dtype of the layer's weights. Alias of <code>layer.variable_dtype</code>.</li> <li>variable_dtype: Dtype of the layer's weights.</li> <li>compute_dtype: The dtype of the layer's computations. Layers automatically cast inputs to this dtype, which causes the computations and output to also be in this dtype. When mixed precision is used with a <code>keras.DTypePolicy</code>, this will be different than <code>variable_dtype</code>.</li> <li>trainable_weights: List of variables to be included in backprop.</li> <li>non_trainable_weights: List of variables that should not be included in backprop.</li> <li>weights: The concatenation of the lists trainable_weights and non_trainable_weights (in this order).</li> <li>trainable: Whether the layer should be trained (boolean), i.e. whether its potentially-trainable weights should be returned as part of <code>layer.trainable_weights</code>.</li> <li>input_spec: Optional (list of) <code>InputSpec</code> object(s) specifying the constraints on inputs that can be accepted by the layer.</li> </ul> We recommend that descendants of <code>Layer</code> implement the following methods: <ul> <li><code>__init__()</code>: Defines custom layer attributes, and creates layer weights that do not depend on input shapes, using <code>add_weight()</code>, or other state.</li> <li><code>build(self, input_shape)</code>: This method can be used to create weights that depend on the shape(s) of the input(s), using <code>add_weight()</code>, or other state. <code>__call__()</code> will automatically build the layer (if it has not been built yet) by calling <code>build()</code>.</li> <li><code>call(self, *args, **kwargs)</code>: Called in <code>__call__</code> after making sure <code>build()</code> has been called. <code>call()</code> performs the logic of applying the layer to the input arguments. Two reserved keyword arguments you can optionally use in <code>call()</code> are: 1. <code>training</code> (boolean, whether the call is in inference mode or training mode). 2. <code>mask</code> (boolean tensor encoding masked timesteps in the input, used e.g. in RNN layers). A typical signature for this method is <code>call(self, inputs)</code>, and user could optionally add <code>training</code> and <code>mask</code> if the layer need them.</li> <li><code>get_config(self)</code>: Returns a dictionary containing the configuration used to initialize this layer. If the keys differ from the arguments in <code>__init__()</code>, then override <code>from_config(self)</code> as well. This method is used when saving the layer or a model that contains this layer.</li> </ul> Examples Here's a basic example: a layer with two variables, <code>w</code> and <code>b</code>, that returns <code>y = w . x + b</code>. It shows how to implement <code>build()</code> and <code>call()</code>. Variables set as attributes of a layer are tracked as weights of the layers (in <code>layer.weights</code>). <div class="codehilite"><pre><code>class SimpleDense(Layer): def __init__(self, units=32): super().__init__() self.units = units # Create the state of the layer (weights) def build(self, input_shape): self.kernel = self.add_weight( shape=(input_shape[-1], self.units), initializer="glorot_uniform", trainable=True, name="kernel", ) self.bias = self.add_weight( shape=(self.units,), initializer="zeros", trainable=True, name="bias", ) # Defines the computation def call(self, inputs): return ops.matmul(inputs, self.kernel) + self.bias # Instantiates the layer. linear_layer = SimpleDense(4) # This will also call `build(input_shape)` and create the weights. y = linear_layer(ops.ones((2, 2))) assert len(linear_layer.weights) == 2 # These weights are trainable, so they're listed in `trainable_weights`: assert len(linear_layer.trainable_weights) == 2 </code></pre></div> Besides trainable weights, updated via backpropagation during training, layers can also have non-trainable weights. These weights are meant to be updated manually during <code>call()</code>. Here's a example layer that computes the running sum of its inputs: <div class="codehilite"><pre><code>class ComputeSum(Layer): def __init__(self, input_dim): super(ComputeSum, self).__init__() # Create a non-trainable weight. self.total = self.add_weight( shape=(), initializer="zeros", trainable=False, name="total", ) def call(self, inputs): self.total.assign(self.total + ops.sum(inputs)) return self.total my_sum = ComputeSum(2) x = ops.ones((2, 2)) y = my_sum(x) assert my_sum.weights == [my_sum.total] assert my_sum.non_trainable_weights == [my_sum.total] assert my_sum.trainable_weights == [] </code></pre></div> <hr /> <h3 id="weights-property"><code>weights</code> property</h3> <div class="codehilite"><pre><code>keras.layers.Layer.weights </code></pre></div> List of all weight variables of the layer. Unlike, <code>layer.variables</code> this excludes metric state and random seeds. <hr /> <h3 id="trainableweights-property"><code>trainable_weights</code> property</h3> <div class="codehilite"><pre><code>keras.layers.Layer.trainable_weights </code></pre></div> List of all trainable weight variables of the layer. These are the weights that get updated by the optimizer during training. <hr /> <h3 id="nontrainableweights-property"><code>non_trainable_weights</code> property</h3> <div class="codehilite"><pre><code>keras.layers.Layer.non_trainable_weights </code></pre></div> List of all non-trainable weight variables of the layer. These are the weights that should not be updated by the optimizer during training. Unlike, <code>layer.non_trainable_variables</code> this excludes metric state and random seeds. <hr /> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L485">[source]</a> <h3 id="addweight-method"><code>add_weight</code> method</h3> <div class="codehilite"><pre><code>Layer.add_weight( shape=None, initializer=None, dtype=None, trainable=True, autocast=True, regularizer=None, constraint=None, aggregation="mean", name=None, ) </code></pre></div> Add a weight variable to the layer. Arguments <ul> <li>shape: Shape tuple for the variable. Must be fully-defined (no <code>None</code> entries). Defaults to <code>()</code> (scalar) if unspecified.</li> <li>initializer: Initializer object to use to populate the initial variable value, or string name of a built-in initializer (e.g. <code>"random_normal"</code>). If unspecified, defaults to <code>"glorot_uniform"</code> for floating-point variables and to <code>"zeros"</code> for all other types (e.g. int, bool).</li> <li>dtype: Dtype of the variable to create, e.g. <code>"float32"</code>. If unspecified, defaults to the layer's variable dtype (which itself defaults to <code>"float32"</code> if unspecified).</li> <li>trainable: Boolean, whether the variable should be trainable via backprop or whether its updates are managed manually. Defaults to <code>True</code>.</li> <li>autocast: Boolean, whether to autocast layers variables when accessing them. Defaults to <code>True</code>.</li> <li>regularizer: Regularizer object to call to apply penalty on the weight. These penalties are summed into the loss function during optimization. Defaults to <code>None</code>.</li> <li>constraint: Contrainst object to call on the variable after any optimizer update, or string name of a built-in constraint. Defaults to <code>None</code>.</li> <li>aggregation: String, one of <code>'mean'</code>, <code>'sum'</code>, <code>'only_first_replica'</code>. Annotates the variable with the type of multi-replica aggregation to be used for this variable when writing custom data parallel training loops.</li> <li>name: String name of the variable. Useful for debugging purposes.</li> </ul> <hr /> <h3 id="trainable-property"><code>trainable</code> property</h3> <div class="codehilite"><pre><code>keras.layers.Layer.trainable </code></pre></div> Settable boolean, whether this layer should be trainable or not. <hr /> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L688">[source]</a> <h3 id="getweights-method"><code>get_weights</code> method</h3> <div class="codehilite"><pre><code>Layer.get_weights() </code></pre></div> Return the values of <code>layer.weights</code> as a list of NumPy arrays. <hr /> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L692">[source]</a> <h3 id="setweights-method"><code>set_weights</code> method</h3> <div class="codehilite"><pre><code>Layer.set_weights(weights) </code></pre></div> Sets the values of <code>layer.weights</code> from a list of NumPy arrays. <hr /> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L1531">[source]</a> <h3 id="getconfig-method"><code>get_config</code> method</h3> <div class="codehilite"><pre><code>Model.get_config() </code></pre></div> Returns the config of the object. An object config is a Python dictionary (serializable) containing the information needed to re-instantiate it. <hr /> <a href="https://github.com/keras-team/keras/tree/v3.6.0/keras/src/layers/layer.py#L1104">[source]</a> <h3 id="addloss-method"><code>add_loss</code> method</h3> <div class="codehilite"><pre><code>Layer.add_loss(loss) </code></pre></div> Can be called inside of the <code>call()</code> method to add a scalar loss. Example <div class="codehilite"><pre><code>class MyLayer(Layer): ... def call(self, x): self.add_loss(ops.sum(x)) return x </code></pre></div> <hr /> <h3 id="losses-property"><code>losses</code> property</h3> <div class="codehilite"><pre><code>keras.layers.Layer.losses </code></pre></div> List of scalar losses from <code>add_loss</code>, regularizers and sublayers. <hr /> </div> <div class='k-outline'> <div class='k-outline-depth-1'> <a href='#the-base-layer-class'>The base Layer class</a> </div> <div class='k-outline-depth-3'> <a href='#layer-class'><code>Layer</code> class</a> </div> <div class='k-outline-depth-3'> <a href='#weights-property'><code>weights</code> property</a> </div> <div class='k-outline-depth-3'> <a href='#trainableweights-property'><code>trainable_weights</code> property</a> </div> <div class='k-outline-depth-3'> <a href='#nontrainableweights-property'><code>non_trainable_weights</code> property</a> </div> <div class='k-outline-depth-3'> <a href='#addweight-method'><code>add_weight</code> method</a> </div> <div class='k-outline-depth-3'> <a href='#trainable-property'><code>trainable</code> property</a> </div> <div class='k-outline-depth-3'> <a href='#getweights-method'><code>get_weights</code> method</a> </div> <div class='k-outline-depth-3'> <a href='#setweights-method'><code>set_weights</code> method</a> </div> <div class='k-outline-depth-3'> <a href='#getconfig-method'><code>get_config</code> method</a> </div> <div class='k-outline-depth-3'> <a href='#addloss-method'><code>add_loss</code> method</a> </div> <div class='k-outline-depth-3'> <a href='#losses-property'><code>losses</code> property</a> </div> </div> </div> </div> </div> </body> <footer style="float: left; width: 100%; padding: 1em; border-top: solid 1px #bbb;"> <a href="https://policies.google.com/terms">Terms</a> | <a href="https://policies.google.com/privacy">Privacy</a> </footer> </html>

CINXE.COM

The base Layer class