CINXE.COM

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <meta name="description" content="Keras documentation"> <meta name="author" content="Keras Team"> <link rel="shortcut icon" href="https://keras.io/img/favicon.ico"> <link rel="canonical" href="https://keras.io/guides/serialization_and_saving/" />  <meta property="og:title" content="Keras documentation: Save, serialize, and export models"> <meta property="og:image" content="https://keras.io/img/logo-k-keras-wb.png"> <meta name="twitter:title" content="Keras documentation: Save, serialize, and export models"> <meta name="twitter:image" content="https://keras.io/img/k-keras-social.png"> <meta name="twitter:card" content="summary"> <title>Save, serialize, and export models</title>  <link href="/css/bootstrap.min.css" rel="stylesheet">  <link href="https://fonts.googleapis.com/css2?family=Open+Sans:wght@400;600;700;800&display=swap" rel="stylesheet">  <link href="/css/docs.css" rel="stylesheet"> <link href="/css/monokai.css" rel="stylesheet">  <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= 'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-5DNGF4N'); </script> <script> (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){ (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o), m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m) })(window,document,'script','https://www.google-analytics.com/analytics.js','ga'); ga('create', 'UA-175165319-128', 'auto'); ga('send', 'pageview'); </script>  <script async defer src="https://buttons.github.io/buttons.js"></script> </head> <body>  <noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-5DNGF4N" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>  <div class='k-page'> <div class="k-nav" id="nav-menu"> <a href='/'><img src='/img/logo-small.png' class='logo-small' /></a> <div class="nav flex-column nav-pills" role="tablist" aria-orientation="vertical"> <a class="nav-link" href="/about/" role="tab" aria-selected="">About Keras</a> <a class="nav-link" href="/getting_started/" role="tab" aria-selected="">Getting started</a> <a class="nav-link active" href="/guides/" role="tab" aria-selected="">Developer guides</a> <a class="nav-sublink" href="/guides/functional_api/">The Functional API</a> <a class="nav-sublink" href="/guides/sequential_model/">The Sequential model</a> <a class="nav-sublink" href="/guides/making_new_layers_and_models_via_subclassing/">Making new layers & models via subclassing</a> <a class="nav-sublink" href="/guides/training_with_built_in_methods/">Training & evaluation with the built-in methods</a> <a class="nav-sublink" href="/guides/custom_train_step_in_jax/">Customizing `fit()` with JAX</a> <a class="nav-sublink" href="/guides/custom_train_step_in_tensorflow/">Customizing `fit()` with TensorFlow</a> <a class="nav-sublink" href="/guides/custom_train_step_in_torch/">Customizing `fit()` with PyTorch</a> <a class="nav-sublink" href="/guides/writing_a_custom_training_loop_in_jax/">Writing a custom training loop in JAX</a> <a class="nav-sublink" href="/guides/writing_a_custom_training_loop_in_tensorflow/">Writing a custom training loop in TensorFlow</a> <a class="nav-sublink" href="/guides/writing_a_custom_training_loop_in_torch/">Writing a custom training loop in PyTorch</a> <a class="nav-sublink active" href="/guides/serialization_and_saving/">Serialization & saving</a> <a class="nav-sublink" href="/guides/customizing_saving_and_serialization/">Customizing saving & serialization</a> <a class="nav-sublink" href="/guides/writing_your_own_callbacks/">Writing your own callbacks</a> <a class="nav-sublink" href="/guides/transfer_learning/">Transfer learning & fine-tuning</a> <a class="nav-sublink" href="/guides/distributed_training_with_jax/">Distributed training with JAX</a> <a class="nav-sublink" href="/guides/distributed_training_with_tensorflow/">Distributed training with TensorFlow</a> <a class="nav-sublink" href="/guides/distributed_training_with_torch/">Distributed training with PyTorch</a> <a class="nav-sublink" href="/guides/distribution/">Distributed training with Keras 3</a> <a class="nav-sublink" href="/guides/migrating_to_keras_3/">Migrating Keras 2 code to Keras 3</a> <a class="nav-sublink" href="/guides/keras_tuner/">Hyperparameter Tuning</a> <a class="nav-sublink" href="/guides/keras_cv/">KerasCV</a> <a class="nav-sublink" href="/guides/keras_nlp/">KerasNLP</a> <a class="nav-sublink" href="/guides/keras_hub/">KerasHub</a> <a class="nav-link" href="/api/" role="tab" aria-selected="">Keras 3 API documentation</a> <a class="nav-link" href="/2.18/api/" role="tab" aria-selected="">Keras 2 API documentation</a> <a class="nav-link" href="/examples/" role="tab" aria-selected="">Code examples</a> <a class="nav-link" href="/keras_tuner/" role="tab" aria-selected="">KerasTuner: Hyperparameter Tuning</a> <a class="nav-link" href="/keras_hub/" role="tab" aria-selected="">KerasHub: Pretrained Models</a> <a class="nav-link" href="/keras_cv/" role="tab" aria-selected="">KerasCV: Computer Vision Workflows</a> <a class="nav-link" href="/keras_nlp/" role="tab" aria-selected="">KerasNLP: Natural Language Workflows</a> </div> </div> <div class='k-main'> <div class='k-main-top'> <script> function displayDropdownMenu() { e = document.getElementById("nav-menu"); if (e.style.display == "block") { e.style.display = "none"; } else { e.style.display = "block"; document.getElementById("dropdown-nav").style.display = "block"; } } function resetMobileUI() { if (window.innerWidth <= 840) { document.getElementById("nav-menu").style.display = "none"; document.getElementById("dropdown-nav").style.display = "block"; } else { document.getElementById("nav-menu").style.display = "block"; document.getElementById("dropdown-nav").style.display = "none"; } var navmenu = document.getElementById("nav-menu"); var menuheight = navmenu.clientHeight; var kmain = document.getElementById("k-main-id"); kmain.style.minHeight = (menuheight + 100) + 'px'; } window.onresize = resetMobileUI; window.addEventListener("load", (event) => { resetMobileUI() }); </script> <div id='dropdown-nav' onclick="displayDropdownMenu();"> <svg viewBox="-20 -20 120 120" width="60" height="60"> <rect width="100" height="20"></rect> <rect y="30" width="100" height="20"></rect> <rect y="60" width="100" height="20"></rect> </svg> </div> <form class="bd-search d-flex align-items-center k-search-form" id="search-form"> <input type="search" class="k-search-input" id="search-input" placeholder="Search Keras documentation..." aria-label="Search Keras documentation..." autocomplete="off"> <button class="k-search-btn"> <svg width="13" height="13" viewBox="0 0 13 13"><title>search</title><path d="m4.8495 7.8226c0.82666 0 1.5262-0.29146 2.0985-0.87438 0.57232-0.58292 0.86378-1.2877 0.87438-2.1144 0.010599-0.82666-0.28086-1.5262-0.87438-2.0985-0.59352-0.57232-1.293-0.86378-2.0985-0.87438-0.8055-0.010599-1.5103 0.28086-2.1144 0.87438-0.60414 0.59352-0.8956 1.293-0.87438 2.0985 0.021197 0.8055 0.31266 1.5103 0.87438 2.1144 0.56172 0.60414 1.2665 0.8956 2.1144 0.87438zm4.4695 0.2115 3.681 3.6819-1.259 1.284-3.6817-3.7 0.0019784-0.69479-0.090043-0.098846c-0.87973 0.76087-1.92 1.1413-3.1207 1.1413-1.3553 0-2.5025-0.46363-3.4417-1.3909s-1.4088-2.0686-1.4088-3.4239c0-1.3553 0.4696-2.4966 1.4088-3.4239 0.9392-0.92727 2.0864-1.3969 3.4417-1.4088 1.3553-0.011889 2.4906 0.45771 3.406 1.4088 0.9154 0.95107 1.379 2.0924 1.3909 3.4239 0 1.2126-0.38043 2.2588-1.1413 3.1385l0.098834 0.090049z"></path></svg> </button> </form> <script> var form = document.getElementById('search-form'); form.onsubmit = function(e) { e.preventDefault(); var query = document.getElementById('search-input').value; window.location.href = '/search.html?query=' + query; return False } </script> </div> <div class='k-main-inner' id='k-main-id'> <div class='k-location-slug'> ► <a href='/guides/'>Developer guides</a> / Save, serialize, and export models </div> <div class='k-content'> <h1 id="save-serialize-and-export-models">Save, serialize, and export models</h1> Authors: Neel Kovelamudi, Francois Chollet Date created: 2023/06/14 Last modified: 2023/06/30 Description: Complete guide to saving, serializing, and exporting models. <img class="k-inline-icon" src="https://colab.research.google.com/img/colab_favicon.ico"/> <a href="https://colab.research.google.com/github/keras-team/keras-io/blob/master/guides/ipynb/serialization_and_saving.ipynb">View in Colab</a> •<img class="k-inline-icon" src="https://github.com/favicon.ico"/> <a href="https://github.com/keras-team/keras-io/blob/master/guides/serialization_and_saving.py">GitHub source</a> <hr /> <h2 id="introduction">Introduction</h2> A Keras model consists of multiple components: <ul> <li>The architecture, or configuration, which specifies what layers the model contain, and how they're connected.</li> <li>A set of weights values (the "state of the model").</li> <li>An optimizer (defined by compiling the model).</li> <li>A set of losses and metrics (defined by compiling the model).</li> </ul> The Keras API saves all of these pieces together in a unified format, marked by the <code>.keras</code> extension. This is a zip archive consisting of the following: <ul> <li>A JSON-based configuration file (config.json): Records of model, layer, and other trackables' configuration.</li> <li>A H5-based state file, such as <code>model.weights.h5</code> (for the whole model), with directory keys for layers and their weights.</li> <li>A metadata file in JSON, storing things such as the current Keras version.</li> </ul> Let's take a look at how this works. <hr /> <h2 id="how-to-save-and-load-a-model">How to save and load a model</h2> If you only have 10 seconds to read this guide, here's what you need to know. Saving a Keras model: <div class="codehilite"><pre><code>model = ... # Get model (Sequential, Functional Model, or Model subclass) model.save('path/to/location.keras') # The file needs to end with the .keras extension </code></pre></div> Loading the model back: <div class="codehilite"><pre><code>model = keras.models.load_model('path/to/location.keras') </code></pre></div> Now, let's look at the details. <hr /> <h2 id="setup">Setup</h2> <div class="codehilite"><pre><code>import numpy as np import keras from keras import ops </code></pre></div> <hr /> <h2 id="saving">Saving</h2> This section is about saving an entire model to a single file. The file will include: <ul> <li>The model's architecture/config</li> <li>The model's weight values (which were learned during training)</li> <li>The model's compilation information (if <code>compile()</code> was called)</li> <li>The optimizer and its state, if any (this enables you to restart training where you left)</li> </ul> <h4 id="apis">APIs</h4> You can save a model with <code>model.save()</code> or <code>keras.models.save_model()</code> (which is equivalent). You can load it back with <code>keras.models.load_model()</code>. The only supported format in Keras 3 is the "Keras v3" format, which uses the <code>.keras</code> extension. Example: <div class="codehilite"><pre><code>def get_model(): # Create a simple model. inputs = keras.Input(shape=(32,)) outputs = keras.layers.Dense(1)(inputs) model = keras.Model(inputs, outputs) model.compile(optimizer=keras.optimizers.Adam(), loss="mean_squared_error") return model model = get_model() # Train the model. test_input = np.random.random((128, 32)) test_target = np.random.random((128, 1)) model.fit(test_input, test_target) # Calling `save('my_model.keras')` creates a zip archive `my_model.keras`. model.save("my_model.keras") # It can be used to reconstruct the model identically. reconstructed_model = keras.models.load_model("my_model.keras") # Let's check: np.testing.assert_allclose( model.predict(test_input), reconstructed_model.predict(test_input) ) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code> 4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 8ms/step - loss: 0.4232 4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 281us/step 4/4 ━━━━━━━━━━━━━━━━━━━━ 0s 373us/step </code></pre></div> </div> <h3 id="custom-objects">Custom objects</h3> This section covers the basic workflows for handling custom layers, functions, and models in Keras saving and reloading. When saving a model that includes custom objects, such as a subclassed Layer, you must define a <code>get_config()</code> method on the object class. If the arguments passed to the constructor (<code>__init__()</code> method) of the custom object aren't Python objects (anything other than base types like ints, strings, etc.), then you must also explicitly deserialize these arguments in the <code>from_config()</code> class method. Like this: <div class="codehilite"><pre><code>class CustomLayer(keras.layers.Layer): def __init__(self, sublayer, **kwargs): super().__init__(**kwargs) self.sublayer = sublayer def call(self, x): return self.sublayer(x) def get_config(self): base_config = super().get_config() config = { "sublayer": keras.saving.serialize_keras_object(self.sublayer), } return {**base_config, **config} @classmethod def from_config(cls, config): sublayer_config = config.pop("sublayer") sublayer = keras.saving.deserialize_keras_object(sublayer_config) return cls(sublayer, **config) </code></pre></div> Please see the <a href="#config_methods">Defining the config methods section</a> for more details and examples. The saved <code>.keras</code> file is lightweight and does not store the Python code for custom objects. Therefore, to reload the model, <code>load_model</code> requires access to the definition of any custom objects used through one of the following methods: <ol> <li>Registering custom objects (preferred),</li> <li>Passing custom objects directly when loading, or</li> <li>Using a custom object scope</li> </ol> Below are examples of each workflow: <h4 id="preferred">Registering custom objects (preferred)</h4> This is the preferred method, as custom object registration greatly simplifies saving and loading code. Adding the <code>@keras.saving.register_keras_serializable</code> decorator to the class definition of a custom object registers the object globally in a master list, allowing Keras to recognize the object when loading the model. Let's create a custom model involving both a custom layer and a custom activation function to demonstrate this. Example: <div class="codehilite"><pre><code># Clear all previously registered custom objects keras.saving.get_custom_objects().clear() # Upon registration, you can optionally specify a package or a name. # If left blank, the package defaults to `Custom` and the name defaults to # the class name. @keras.saving.register_keras_serializable(package="MyLayers") class CustomLayer(keras.layers.Layer): def __init__(self, factor): super().__init__() self.factor = factor def call(self, x): return x * self.factor def get_config(self): return {"factor": self.factor} @keras.saving.register_keras_serializable(package="my_package", name="custom_fn") def custom_fn(x): return x**2 # Create the model. def get_model(): inputs = keras.Input(shape=(4,)) mid = CustomLayer(0.5)(inputs) outputs = keras.layers.Dense(1, activation=custom_fn)(mid) model = keras.Model(inputs, outputs) model.compile(optimizer="rmsprop", loss="mean_squared_error") return model # Train the model. def train_model(model): input = np.random.random((4, 4)) target = np.random.random((4, 1)) model.fit(input, target) return model test_input = np.random.random((4, 4)) test_target = np.random.random((4, 1)) model = get_model() model = train_model(model) model.save("custom_model.keras") # Now, we can simply load without worrying about our custom objects. reconstructed_model = keras.models.load_model("custom_model.keras") # Let's check: np.testing.assert_allclose( model.predict(test_input), reconstructed_model.predict(test_input) ) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code> 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step - loss: 0.2571 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step </code></pre></div> </div> <h4 id="passing-custom-objects-to-loadmodel">Passing custom objects to <code>load_model()</code></h4> <div class="codehilite"><pre><code>model = get_model() model = train_model(model) # Calling `save('my_model.keras')` creates a zip archive `my_model.keras`. model.save("custom_model.keras") # Upon loading, pass a dict containing the custom objects used in the # `custom_objects` argument of `keras.models.load_model()`. reconstructed_model = keras.models.load_model( "custom_model.keras", custom_objects={"CustomLayer": CustomLayer, "custom_fn": custom_fn}, ) # Let's check: np.testing.assert_allclose( model.predict(test_input), reconstructed_model.predict(test_input) ) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code> 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 37ms/step - loss: 0.0535 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 12ms/step </code></pre></div> </div> <h4 id="using-a-custom-object-scope">Using a custom object scope</h4> Any code within the custom object scope will be able to recognize the custom objects passed to the scope argument. Therefore, loading the model within the scope will allow the loading of our custom objects. Example: <div class="codehilite"><pre><code>model = get_model() model = train_model(model) model.save("custom_model.keras") # Pass the custom objects dictionary to a custom object scope and place # the `keras.models.load_model()` call within the scope. custom_objects = {"CustomLayer": CustomLayer, "custom_fn": custom_fn} with keras.saving.custom_object_scope(custom_objects): reconstructed_model = keras.models.load_model("custom_model.keras") # Let's check: np.testing.assert_allclose( model.predict(test_input), reconstructed_model.predict(test_input) ) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code> 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 40ms/step - loss: 0.0868 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step 1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 11ms/step </code></pre></div> </div> <h3 id="model-serialization">Model serialization</h3> This section is about saving only the model's configuration, without its state. The model's configuration (or architecture) specifies what layers the model contains, and how these layers are connected. If you have the configuration of a model, then the model can be created with a freshly initialized state (no weights or compilation information). <h4 id="apis">APIs</h4> The following serialization APIs are available: <ul> <li><code>keras.models.clone_model(model)</code>: make a (randomly initialized) copy of a model.</li> <li><code>get_config()</code> and <code>cls.from_config()</code>: retrieve the configuration of a layer or model, and recreate a model instance from its config, respectively.</li> <li><code>keras.models.model_to_json()</code> and <code>keras.models.model_from_json()</code>: similar, but as JSON strings.</li> <li><code>keras.saving.serialize_keras_object()</code>: retrieve the configuration any arbitrary Keras object.</li> <li><code>keras.saving.deserialize_keras_object()</code>: recreate an object instance from its configuration.</li> </ul> <h4 id="inmemory-model-cloning">In-memory model cloning</h4> You can do in-memory cloning of a model via <code>keras.models.clone_model()</code>. This is equivalent to getting the config then recreating the model from its config (so it does not preserve compilation information or layer weights values). Example: <div class="codehilite"><pre><code>new_model = keras.models.clone_model(model) </code></pre></div> <h4 id="getconfig-and-fromconfig"><code>get_config()</code> and <code>from_config()</code></h4> Calling <code>model.get_config()</code> or <code>layer.get_config()</code> will return a Python dict containing the configuration of the model or layer, respectively. You should define <code>get_config()</code> to contain arguments needed for the <code>__init__()</code> method of the model or layer. At loading time, the <code>from_config(config)</code> method will then call <code>__init__()</code> with these arguments to reconstruct the model or layer. Layer example: <div class="codehilite"><pre><code>layer = keras.layers.Dense(3, activation="relu") layer_config = layer.get_config() print(layer_config) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>{'name': 'dense_4', 'trainable': True, 'dtype': 'float32', 'units': 3, 'activation': 'relu', 'use_bias': True, 'kernel_initializer': {'module': 'keras.src.initializers.random_initializers', 'class_name': 'GlorotUniform', 'config': {'seed': None}, 'registered_name': 'GlorotUniform'}, 'bias_initializer': {'module': 'keras.src.initializers.constant_initializers', 'class_name': 'Zeros', 'config': {}, 'registered_name': 'Zeros'}, 'kernel_regularizer': None, 'bias_regularizer': None, 'kernel_constraint': None, 'bias_constraint': None} </code></pre></div> </div> Now let's reconstruct the layer using the <code>from_config()</code> method: <div class="codehilite"><pre><code>new_layer = keras.layers.Dense.from_config(layer_config) </code></pre></div> Sequential model example: <div class="codehilite"><pre><code>model = keras.Sequential([keras.Input((32,)), keras.layers.Dense(1)]) config = model.get_config() new_model = keras.Sequential.from_config(config) </code></pre></div> Functional model example: <div class="codehilite"><pre><code>inputs = keras.Input((32,)) outputs = keras.layers.Dense(1)(inputs) model = keras.Model(inputs, outputs) config = model.get_config() new_model = keras.Model.from_config(config) </code></pre></div> <h4 id="tojson-and-kerasmodelsmodelfromjson"><code>to_json()</code> and <code>keras.models.model_from_json()</code></h4> This is similar to <code>get_config</code> / <code>from_config</code>, except it turns the model into a JSON string, which can then be loaded without the original model class. It is also specific to models, it isn't meant for layers. Example: <div class="codehilite"><pre><code>model = keras.Sequential([keras.Input((32,)), keras.layers.Dense(1)]) json_config = model.to_json() new_model = keras.models.model_from_json(json_config) </code></pre></div> <h4 id="arbitrary-object-serialization-and-deserialization">Arbitrary object serialization and deserialization</h4> The <code>keras.saving.serialize_keras_object()</code> and <code>keras.saving.deserialize_keras_object()</code> APIs are general-purpose APIs that can be used to serialize or deserialize any Keras object and any custom object. It is at the foundation of saving model architecture and is behind all <code>serialize()</code>/<code>deserialize()</code> calls in keras. Example: <div class="codehilite"><pre><code>my_reg = keras.regularizers.L1(0.005) config = keras.saving.serialize_keras_object(my_reg) print(config) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>{'module': 'keras.src.regularizers.regularizers', 'class_name': 'L1', 'config': {'l1': 0.004999999888241291}, 'registered_name': 'L1'} </code></pre></div> </div> Note the serialization format containing all the necessary information for proper reconstruction: <ul> <li><code>module</code> containing the name of the Keras module or other identifying module the object comes from</li> <li><code>class_name</code> containing the name of the object's class.</li> <li><code>config</code> with all the information needed to reconstruct the object</li> <li><code>registered_name</code> for custom objects. See <a href="#custom_object_serialization">here</a>.</li> </ul> Now we can reconstruct the regularizer. <div class="codehilite"><pre><code>new_reg = keras.saving.deserialize_keras_object(config) </code></pre></div> <h3 id="model-weights-saving">Model weights saving</h3> You can choose to only save & load a model's weights. This can be useful if: <ul> <li>You only need the model for inference: in this case you won't need to restart training, so you don't need the compilation information or optimizer state.</li> <li>You are doing transfer learning: in this case you will be training a new model reusing the state of a prior model, so you don't need the compilation information of the prior model.</li> </ul> <h4 id="apis-for-inmemory-weight-transfer">APIs for in-memory weight transfer</h4> Weights can be copied between different objects by using <code>get_weights()</code> and <code>set_weights()</code>: <ul> <li><code>keras.layers.Layer.get_weights()</code>: Returns a list of NumPy arrays of weight values.</li> <li><code>keras.layers.Layer.set_weights(weights)</code>: Sets the model weights to the values provided (as NumPy arrays).</li> </ul> Examples: Transferring weights from one layer to another, in memory <div class="codehilite"><pre><code>def create_layer(): layer = keras.layers.Dense(64, activation="relu", name="dense_2") layer.build((None, 784)) return layer layer_1 = create_layer() layer_2 = create_layer() # Copy weights from layer 1 to layer 2 layer_2.set_weights(layer_1.get_weights()) </code></pre></div> Transferring weights from one model to another model with a compatible architecture, in memory <div class="codehilite"><pre><code># Create a simple functional model inputs = keras.Input(shape=(784,), name="digits") x = keras.layers.Dense(64, activation="relu", name="dense_1")(inputs) x = keras.layers.Dense(64, activation="relu", name="dense_2")(x) outputs = keras.layers.Dense(10, name="predictions")(x) functional_model = keras.Model(inputs=inputs, outputs=outputs, name="3_layer_mlp") # Define a subclassed model with the same architecture class SubclassedModel(keras.Model): def __init__(self, output_dim, name=None): super().__init__(name=name) self.output_dim = output_dim self.dense_1 = keras.layers.Dense(64, activation="relu", name="dense_1") self.dense_2 = keras.layers.Dense(64, activation="relu", name="dense_2") self.dense_3 = keras.layers.Dense(output_dim, name="predictions") def call(self, inputs): x = self.dense_1(inputs) x = self.dense_2(x) x = self.dense_3(x) return x def get_config(self): return {"output_dim": self.output_dim, "name": self.name} subclassed_model = SubclassedModel(10) # Call the subclassed model once to create the weights. subclassed_model(np.ones((1, 784))) # Copy weights from functional_model to subclassed_model. subclassed_model.set_weights(functional_model.get_weights()) assert len(functional_model.weights) == len(subclassed_model.weights) for a, b in zip(functional_model.weights, subclassed_model.weights): np.testing.assert_allclose(a.numpy(), b.numpy()) </code></pre></div> The case of stateless layers Because stateless layers do not change the order or number of weights, models can have compatible architectures even if there are extra/missing stateless layers. <div class="codehilite"><pre><code>inputs = keras.Input(shape=(784,), name="digits") x = keras.layers.Dense(64, activation="relu", name="dense_1")(inputs) x = keras.layers.Dense(64, activation="relu", name="dense_2")(x) outputs = keras.layers.Dense(10, name="predictions")(x) functional_model = keras.Model(inputs=inputs, outputs=outputs, name="3_layer_mlp") inputs = keras.Input(shape=(784,), name="digits") x = keras.layers.Dense(64, activation="relu", name="dense_1")(inputs) x = keras.layers.Dense(64, activation="relu", name="dense_2")(x) # Add a dropout layer, which does not contain any weights. x = keras.layers.Dropout(0.5)(x) outputs = keras.layers.Dense(10, name="predictions")(x) functional_model_with_dropout = keras.Model( inputs=inputs, outputs=outputs, name="3_layer_mlp" ) functional_model_with_dropout.set_weights(functional_model.get_weights()) </code></pre></div> <h4 id="apis-for-saving-weights-to-disk-amp-loading-them-back">APIs for saving weights to disk & loading them back</h4> Weights can be saved to disk by calling <code>model.save_weights(filepath)</code>. The filename should end in <code>.weights.h5</code>. Example: <div class="codehilite"><pre><code># Runnable example sequential_model = keras.Sequential( [ keras.Input(shape=(784,), name="digits"), keras.layers.Dense(64, activation="relu", name="dense_1"), keras.layers.Dense(64, activation="relu", name="dense_2"), keras.layers.Dense(10, name="predictions"), ] ) sequential_model.save_weights("my_model.weights.h5") sequential_model.load_weights("my_model.weights.h5") </code></pre></div> Note that changing <code>layer.trainable</code> may result in a different <code>layer.weights</code> ordering when the model contains nested layers. <div class="codehilite"><pre><code>class NestedDenseLayer(keras.layers.Layer): def __init__(self, units, name=None): super().__init__(name=name) self.dense_1 = keras.layers.Dense(units, name="dense_1") self.dense_2 = keras.layers.Dense(units, name="dense_2") def call(self, inputs): return self.dense_2(self.dense_1(inputs)) nested_model = keras.Sequential([keras.Input((784,)), NestedDenseLayer(10, "nested")]) variable_names = [v.name for v in nested_model.weights] print("variables: {}".format(variable_names)) print("\nChanging trainable status of one of the nested layers...") nested_model.get_layer("nested").dense_1.trainable = False variable_names_2 = [v.name for v in nested_model.weights] print("\nvariables: {}".format(variable_names_2)) print("variable ordering changed:", variable_names != variable_names_2) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>variables: ['kernel', 'bias', 'kernel', 'bias'] </code></pre></div> </div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>Changing trainable status of one of the nested layers... </code></pre></div> </div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>variables: ['kernel', 'bias', 'kernel', 'bias'] variable ordering changed: False </code></pre></div> </div> <h5>Transfer learning example</h5> When loading pretrained weights from a weights file, it is recommended to load the weights into the original checkpointed model, and then extract the desired weights/layers into a new model. Example: <div class="codehilite"><pre><code>def create_functional_model(): inputs = keras.Input(shape=(784,), name="digits") x = keras.layers.Dense(64, activation="relu", name="dense_1")(inputs) x = keras.layers.Dense(64, activation="relu", name="dense_2")(x) outputs = keras.layers.Dense(10, name="predictions")(x) return keras.Model(inputs=inputs, outputs=outputs, name="3_layer_mlp") functional_model = create_functional_model() functional_model.save_weights("pretrained.weights.h5") # In a separate program: pretrained_model = create_functional_model() pretrained_model.load_weights("pretrained.weights.h5") # Create a new model by extracting layers from the original model: extracted_layers = pretrained_model.layers[:-1] extracted_layers.append(keras.layers.Dense(5, name="dense_3")) model = keras.Sequential(extracted_layers) model.summary() </code></pre></div> <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">Model: "sequential_4" </pre> <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ dense_1 (Dense) │ (None, 64) │ 50,240 │ ├─────────────────────────────────┼───────────────────────────┼────────────┤ │ dense_2 (Dense) │ (None, 64) │ 4,160 │ ├─────────────────────────────────┼───────────────────────────┼────────────┤ │ dense_3 (Dense) │ (None, 5) │ 325 │ └─────────────────────────────────┴───────────────────────────┴────────────┘ </pre> <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Total params: 54,725 (213.77 KB) </pre> <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Trainable params: 54,725 (213.77 KB) </pre> <pre style="white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace"> Non-trainable params: 0 (0.00 B) </pre> <h3 id="appendix-handling-custom-objects">Appendix: Handling custom objects</h3> <a name="config_methods"></a> <h4 id="defining-the-config-methods">Defining the config methods</h4> Specifications: <ul> <li><code>get_config()</code> should return a JSON-serializable dictionary in order to be compatible with the Keras architecture- and model-saving APIs.</li> <li><code>from_config(config)</code> (a <code>classmethod</code>) should return a new layer or model object that is created from the config. The default implementation returns <code>cls(**config)</code>.</li> </ul> NOTE: If all your constructor arguments are already serializable, e.g. strings and ints, or non-custom Keras objects, overriding <code>from_config</code> is not necessary. However, for more complex objects such as layers or models passed to <code>__init__</code>, deserialization must be handled explicitly either in <code>__init__</code> itself or overriding the <code>from_config()</code> method. Example: <div class="codehilite"><pre><code>@keras.saving.register_keras_serializable(package="MyLayers", name="KernelMult") class MyDense(keras.layers.Layer): def __init__( self, units, *, kernel_regularizer=None, kernel_initializer=None, nested_model=None, **kwargs ): super().__init__(**kwargs) self.hidden_units = units self.kernel_regularizer = kernel_regularizer self.kernel_initializer = kernel_initializer self.nested_model = nested_model def get_config(self): config = super().get_config() # Update the config with the custom layer's parameters config.update( { "units": self.hidden_units, "kernel_regularizer": self.kernel_regularizer, "kernel_initializer": self.kernel_initializer, "nested_model": self.nested_model, } ) return config def build(self, input_shape): input_units = input_shape[-1] self.kernel = self.add_weight( name="kernel", shape=(input_units, self.hidden_units), regularizer=self.kernel_regularizer, initializer=self.kernel_initializer, ) def call(self, inputs): return ops.matmul(inputs, self.kernel) layer = MyDense(units=16, kernel_regularizer="l1", kernel_initializer="ones") layer3 = MyDense(units=64, nested_model=layer) config = keras.layers.serialize(layer3) print(config) new_layer = keras.layers.deserialize(config) print(new_layer) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>{'module': None, 'class_name': 'MyDense', 'config': {'name': 'my_dense_1', 'trainable': True, 'dtype': 'float32', 'units': 64, 'kernel_regularizer': None, 'kernel_initializer': None, 'nested_model': {'module': None, 'class_name': 'MyDense', 'config': {'name': 'my_dense', 'trainable': True, 'dtype': 'float32', 'units': 16, 'kernel_regularizer': 'l1', 'kernel_initializer': 'ones', 'nested_model': None}, 'registered_name': 'MyLayers>KernelMult'}}, 'registered_name': 'MyLayers>KernelMult'} <MyDense name=my_dense_1, built=False> </code></pre></div> </div> Note that overriding <code>from_config</code> is unnecessary above for <code>MyDense</code> because <code>hidden_units</code>, <code>kernel_initializer</code>, and <code>kernel_regularizer</code> are ints, strings, and a built-in Keras object, respectively. This means that the default <code>from_config</code> implementation of <code>cls(**config)</code> will work as intended. For more complex objects, such as layers and models passed to <code>__init__</code>, for example, you must explicitly deserialize these objects. Let's take a look at an example of a model where a <code>from_config</code> override is necessary. Example: <a name="registration_example"></a> <div class="codehilite"><pre><code>@keras.saving.register_keras_serializable(package="ComplexModels") class CustomModel(keras.layers.Layer): def __init__(self, first_layer, second_layer=None, **kwargs): super().__init__(**kwargs) self.first_layer = first_layer if second_layer is not None: self.second_layer = second_layer else: self.second_layer = keras.layers.Dense(8) def get_config(self): config = super().get_config() config.update( { "first_layer": self.first_layer, "second_layer": self.second_layer, } ) return config @classmethod def from_config(cls, config): # Note that you can also use [`keras.saving.deserialize_keras_object`](/api/models/model_saving_apis/serialization_utils#deserializekerasobject-function) here config["first_layer"] = keras.layers.deserialize(config["first_layer"]) config["second_layer"] = keras.layers.deserialize(config["second_layer"]) return cls(**config) def call(self, inputs): return self.first_layer(self.second_layer(inputs)) # Let's make our first layer the custom layer from the previous example (MyDense) inputs = keras.Input((32,)) outputs = CustomModel(first_layer=layer)(inputs) model = keras.Model(inputs, outputs) config = model.get_config() new_model = keras.Model.from_config(config) </code></pre></div> <a name="custom_object_serialization"></a> <h4 id="how-custom-objects-are-serialized">How custom objects are serialized</h4> The serialization format has a special key for custom objects registered via <code>@keras.saving.register_keras_serializable</code>. This <code>registered_name</code> key allows for easy retrieval at loading/deserialization time while also allowing users to add custom naming. Let's take a look at the config from serializing the custom layer <code>MyDense</code> we defined above. Example: <div class="codehilite"><pre><code>layer = MyDense( units=16, kernel_regularizer=keras.regularizers.L1L2(l1=1e-5, l2=1e-4), kernel_initializer="ones", ) config = keras.layers.serialize(layer) print(config) </code></pre></div> <div class="k-default-codeblock"> <div class="codehilite"><pre><code>{'module': None, 'class_name': 'MyDense', 'config': {'name': 'my_dense_2', 'trainable': True, 'dtype': 'float32', 'units': 16, 'kernel_regularizer': {'module': 'keras.src.regularizers.regularizers', 'class_name': 'L1L2', 'config': {'l1': 1e-05, 'l2': 0.0001}, 'registered_name': 'L1L2'}, 'kernel_initializer': 'ones', 'nested_model': None}, 'registered_name': 'MyLayers>KernelMult'} </code></pre></div> </div> As shown, the <code>registered_name</code> key contains the lookup information for the Keras master list, including the package <code>MyLayers</code> and the custom name <code>KernelMult</code> that we gave in the <code>@keras.saving.register_keras_serializable</code> decorator. Take a look again at the custom class definition/registration <a href="#registration_example">here</a>. Note that the <code>class_name</code> key contains the original name of the class, allowing for proper re-initialization in <code>from_config</code>. Additionally, note that the <code>module</code> key is <code>None</code> since this is a custom object. </div> <div class='k-outline'> <div class='k-outline-depth-1'> <a href='#save-serialize-and-export-models'>Save, serialize, and export models</a> </div> <div class='k-outline-depth-2'> ◆ <a href='#introduction'>Introduction</a> </div> <div class='k-outline-depth-2'> ◆ <a href='#how-to-save-and-load-a-model'>How to save and load a model</a> </div> <div class='k-outline-depth-2'> ◆ <a href='#setup'>Setup</a> </div> <div class='k-outline-depth-2'> ◆ <a href='#saving'>Saving</a> </div> <div class='k-outline-depth-3'> <a href='#custom-objects'>Custom objects</a> </div> <div class='k-outline-depth-3'> <a href='#model-serialization'>Model serialization</a> </div> <div class='k-outline-depth-3'> <a href='#model-weights-saving'>Model weights saving</a> </div> <div class='k-outline-depth-3'> <a href='#appendix-handling-custom-objects'>Appendix: Handling custom objects</a> </div> </div> </div> </div> </div> </body> <footer style="float: left; width: 100%; padding: 1em; border-top: solid 1px #bbb;"> <a href="https://policies.google.com/terms">Terms</a> | <a href="https://policies.google.com/privacy">Privacy</a> </footer> </html>