How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)? - TensorFlow

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <title>How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)? - TensorFlow - Build with Google AI</title> <meta name="description" content="Hi, everyone here, I was trying to do TFLite post training quantization practice as breizhn/DTLN. The author did default quantization(float32) and separate as 2 stage of TFlite model as TF2.3 not support complex value &hellip;"> <meta name="generator" content="Discourse 3.4.0.beta3-dev - https://github.com/discourse/discourse version d880db3b7b748901366d0c94f58dada9096f967f"> <link rel="icon" type="image/png" href="https://canada1.discourse-cdn.com/googleaitrial/optimized/2X/8/8bfe64fc593eb7baba8df149ebba4457c16ca1f9_2_32x32.png"> <link rel="apple-touch-icon" type="image/png" href="https://canada1.discourse-cdn.com/googleaitrial/optimized/2X/8/8bfe64fc593eb7baba8df149ebba4457c16ca1f9_2_180x180.png"> <meta name="theme-color" media="all" content="#1f1f1f"> <meta name="viewport" content="width=device-width, initial-scale=1.0, minimum-scale=1.0, user-scalable=yes, viewport-fit=cover"> <link rel="canonical" href="https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277" /> <link rel="search" type="application/opensearchdescription+xml" href="https://discuss.ai.google.dev/opensearch.xml" title="Build with Google AI Search"> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/color_definitions_google-ai-for-developers_8_2_9664c403088a1d589ded99d7cdbbd8f338343719.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" class="light-scheme"/> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/checklist_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="checklist" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-ai_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-ai" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-akismet_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-akismet" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-cakeday_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-cakeday" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-data-explorer_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-data-explorer" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-details_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-details" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-lazy-videos_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-lazy-videos" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-local-dates_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-local-dates" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-narrative-bot_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-narrative-bot" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-policy_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-policy" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-presence_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-presence" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-reactions_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-reactions" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-solved_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-solved" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-templates_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-templates" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-topic-voting_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-topic-voting" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/footnote_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="footnote" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/hosted-site_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="hosted-site" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/poll_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="poll" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/spoiler-alert_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="spoiler-alert" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-ai_desktop_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-ai_desktop" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-reactions_desktop_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-reactions_desktop" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/discourse-topic-voting_desktop_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="discourse-topic-voting_desktop" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/poll_desktop_5a2434b361468bc0f0245e49d5dee554eebac4e3.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="poll_desktop" /> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_theme_4_7896a10e11b78b3457576dae80aa6a4f54f8c407.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="4" data-theme-name="discourse header search"/> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_theme_3_8fbb151033f3941d27908950c0e87f8fb39a38a6.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="3" data-theme-name="discourse-material-icons"/> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_theme_9_f0575d150ab7e99c44052a5015695044e12c5b03.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="9" data-theme-name="global notice custom css"/> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_theme_8_afb04a60a9c25410f7b14d8fa0f3ae4ed9169b51.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="8" data-theme-name="temp nav fixes"/> <link href="https://yyz1.discourse-cdn.com/googleaitrial/stylesheets/desktop_theme_2_6b14ef109bd30716a9d1b973997ba1349dda629b.css?__ws=discuss.ai.google.dev" media="all" rel="stylesheet" data-target="desktop_theme" data-theme-id="2" data-theme-name="google ai"/> <link href="https://fonts.googleapis.com/css2?family=Material+Symbols+Outlined:opsz,wght,FILL,GRAD@20..48,100..400,0..1,0&family=Roboto+Mono:ital,wght@0,100..700;1,100..700&family=Roboto:ital,wght@0,100;0,300;0,400;0,500;0,700;0,900;1,100;1,300;1,400;1,500;1,700;1,900&display=swap" rel="stylesheet"> <link rel="alternate nofollow" type="application/rss+xml" title="RSS feed of 'How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)?'" href="https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277.rss" /> <meta property="og:site_name" content="Build with Google AI" /> <meta property="og:type" content="website" /> <meta name="twitter:card" content="summary" /> <meta name="twitter:image" content="https://canada1.discourse-cdn.com/googleaitrial/optimized/3X/8/1/81d2aa624025034929191720e82f8009da8b9262_2_1024x222.png" /> <meta property="og:image" content="https://canada1.discourse-cdn.com/googleaitrial/optimized/3X/8/1/81d2aa624025034929191720e82f8009da8b9262_2_1024x222.png" /> <meta property="og:url" content="https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277" /> <meta name="twitter:url" content="https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277" /> <meta property="og:title" content="How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)?" /> <meta name="twitter:title" content="How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)?" /> <meta property="og:description" content="Hi, everyone here, I was trying to do TFLite post training quantization practice as breizhn/DTLN. The author did default quantization(float32) and separate as 2 stage of TFlite model as TF2.3 not support complex value well. Then I try to use TF2.15 and load his saved model from ‘DTLN/pretrained_model/dtln_saved_model’ and try to do Full Integer quantization. The error shows as below I am not sure how to fix. During searching for help, I found there’s someone did the work and shares the Mo..." /> <meta name="twitter:description" content="Hi, everyone here, I was trying to do TFLite post training quantization practice as breizhn/DTLN. The author did default quantization(float32) and separate as 2 stage of TFlite model as TF2.3 not support complex value well. Then I try to use TF2.15 and load his saved model from ‘DTLN/pretrained_model/dtln_saved_model’ and try to do Full Integer quantization. The error shows as below I am not sure how to fix. During searching for help, I found there’s someone did the work and shares the Mo..." /> <meta property="og:article:section" content="TensorFlow" /> <meta property="og:article:section:color" content="f66f00" /> <meta property="og:article:tag" content="tflite" /> <meta property="og:article:tag" content="models" /> <meta property="article:published_time" content="2024-08-05T02:19:59+00:00" /> <meta property="og:ignore_canonical" content="true" /> </head> <body class="crawler browser-update"> <header> <a href="/"> Build with Google AI </a> </header> <div id="main-outlet" class="wrap" role="main"> <div id="topic-title"> <h1> <a href="/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277">How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)?</a> </h1> <div class="topic-category" itemscope itemtype="http://schema.org/BreadcrumbList"> <span itemprop="itemListElement" itemscope itemtype="http://schema.org/ListItem"> <a href="/c/tensorflow/21" class="badge-wrapper bullet" itemprop="item"> <span class='badge-category-bg' style='background-color: #f66f00'></span> <span class='badge-category clear-badge'> <span class='category-name' itemprop='name'>TensorFlow</span> </span> </a> <meta itemprop="position" content="1" /> </span> </div> <div class="topic-category"> <div class='discourse-tags list-tags'> <a href='https://discuss.ai.google.dev/tag/tflite' class='discourse-tag' rel="tag">tflite</a>, <a href='https://discuss.ai.google.dev/tag/models' class='discourse-tag' rel="tag">models</a> </div> </div> </div> <div itemscope itemtype='http://schema.org/DiscussionForumPosting'> <meta itemprop='headline' content='How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)?'> <link itemprop='url' href='https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277'> <meta itemprop='datePublished' content='2024-08-05T02:19:59Z'> <meta itemprop='articleSection' content='TensorFlow'> <meta itemprop='keywords' content='tflite, models'> <div itemprop='publisher' itemscope itemtype="http://schema.org/Organization"> <meta itemprop='name' content='Build with Google AI'> <div itemprop='logo' itemscope itemtype="http://schema.org/ImageObject"> <meta itemprop='url' content='https://canada1.discourse-cdn.com/googleaitrial/original/1X/9c9b743566b5ef0d0701ebba72bf83038f42ea0b.png'> </div> </div> <div id='post_1' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/saraphinesER'><span itemprop='name'>saraphinesER</span></a> </span> <link itemprop="mainEntityOfPage" href="https://discuss.ai.google.dev/t/how-to-correctly-do-tflite-fully-integer-quantization-of-dtln-dual-signal-transformation-lstm-network/35277"> <link itemprop="image" href="https://canada1.discourse-cdn.com/googleaitrial/original/3X/8/1/81d2aa624025034929191720e82f8009da8b9262.png"> <span class="crawler-post-infos"> <time datetime='2024-08-05T02:19:59Z' class='post-time'> August 5, 2024, 2:19am </time> <meta itemprop='dateModified' content='2024-08-05T07:37:46Z'> <span itemprop='position'>1</span> </span> </div> <div class='post' itemprop='text'> <p>Hi, everyone here,<br> I was trying to do TFLite post training quantization practice as <a href="https://github.com/breizhn/DTLN" rel="noopener nofollow ugc">breizhn/DTLN</a>.<br> The author did default quantization(float32) and separate as 2 stage of TFlite model as TF2.3 not support complex value well.<br> Then I try to use TF2.15 and load his saved model from ‘DTLN/pretrained_model/dtln_saved_model’ and try to do Full Integer quantization. The error shows as below I am not sure how to fix.</p> <p><div class="lightbox-wrapper"><a class="lightbox" href="https://canada1.discourse-cdn.com/googleaitrial/original/3X/8/1/81d2aa624025034929191720e82f8009da8b9262.png" data-download-href="/uploads/short-url/iwsWrfm6aZpSQzKnpthsWHf6cCu.png?dl=1" title="breizhn_DTLN_PTQ_needs Flex-ops" rel="noopener nofollow ugc"><img src="https://canada1.discourse-cdn.com/googleaitrial/optimized/3X/8/1/81d2aa624025034929191720e82f8009da8b9262_2_690x150.png" alt="breizhn_DTLN_PTQ_needs Flex-ops" data-base62-sha1="iwsWrfm6aZpSQzKnpthsWHf6cCu" width="690" height="150" srcset="https://canada1.discourse-cdn.com/googleaitrial/optimized/3X/8/1/81d2aa624025034929191720e82f8009da8b9262_2_690x150.png, https://canada1.discourse-cdn.com/googleaitrial/optimized/3X/8/1/81d2aa624025034929191720e82f8009da8b9262_2_1035x225.png 1.5x, https://canada1.discourse-cdn.com/googleaitrial/original/3X/8/1/81d2aa624025034929191720e82f8009da8b9262.png 2x" data-dominant-color="272226"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">breizhn_DTLN_PTQ_needs Flex-ops</span><span class="informations">1314×286 35.4 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p> <p>During searching for help, I found there’s someone did the work and shares the <em>Modified by moderator</em><a href="https://github.com" rel="noopener nofollow ugc">https://github.com</a> , <a href="https://github.com/nyadla-sys/whisper.tflite/blob/main/models/dtln_quantized.tflite" rel="noopener nofollow ugc">TFLite model</a>.<br> The model graph is more simple and concise compare to the original 2 stage TFLite models, while the auther nyadla-sys didn’t talk about how to do this or share python script.</p> <p>I am wondering how he made it, I am not sure if this work needs other skill since I only knew the basic converter steps. Perhaps it’s simple just I have no idea how to make it.<br> Hope someone can help here, give more clear steps or even script to help me understand how to merge the 2 stage into 1 model.<br> Thanks.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_3' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni'><span itemprop='name'>Kiran_Sai_Ramineni</span></a> </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-08-05T08:21:25Z' class='post-time'> August 5, 2024, 8:21am </time> <meta itemprop='dateModified' content='2024-08-05T08:21:25Z'> <span itemprop='position'>3</span> </span> </div> <div class='post' itemprop='text'> <p>Hi <a class="mention" href="/u/saraphineser">@saraphinesER</a>, Could you please share the standalone code to reproduce the issue. Thank You.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_5' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/saraphinesER'><span itemprop='name'>saraphinesER</span></a> </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-08-27T05:18:57Z' class='post-time'> August 27, 2024, 5:18am </time> <meta itemprop='dateModified' content='2024-08-27T05:18:57Z'> <span itemprop='position'>5</span> </span> </div> <div class='post' itemprop='text'> <p>Hi, <a class="mention" href="/u/kiran_sai_ramineni">@Kiran_Sai_Ramineni</a>, really sorry for the late reply, I was busy on something else for 2 weeks and almost have no chance to check news here. Hope this wouldn’t make you thought I am just a question shooter or hit-and-run poster.<br> Here is my script to do the post training quantization with full Integer, <a href="https://drive.google.com/file/d/1m4VoPJt8VYdK7qeatLy0F7jtNICrMW_h/view?usp=drive_link" rel="noopener nofollow ugc">breizhn_DTLN_PTQ_modify.py</a><br> using the model ‘<a href="https://github.com/breizhn/DTLN/blob/master/pretrained_model/dtln_saved_model/saved_model.pb" rel="noopener nofollow ugc">saved_model.pb</a>’ as input. Then will get the error as the one in my first post.</p> <p>Please give advise or comments in any.<br> Thanks.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_7' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni'><span itemprop='name'>Kiran_Sai_Ramineni</span></a> </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-09-09T06:24:10Z' class='post-time'> September 9, 2024, 6:24am </time> <meta itemprop='dateModified' content='2024-09-09T06:24:10Z'> <span itemprop='position'>7</span> </span> </div> <div class='post' itemprop='text'> <p>Hi <a class="mention" href="/u/saraphineser">@saraphinesER</a>, Could you please confirm if you are trying to quantize the model that was saved as a savaed_model.pd file. If yes could you please let me know the Tensorflow version you are using to save the model. Because I am not able to load that model in 2.17. Thank You.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_9' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/saraphinesER'><span itemprop='name'>saraphinesER</span></a> </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-09-09T10:16:14Z' class='post-time'> September 9, 2024, 10:16am </time> <meta itemprop='dateModified' content='2024-09-09T10:16:14Z'> <span itemprop='position'>9</span> </span> </div> <div class='post' itemprop='text'> <p>Hi, <a class="mention" href="/u/kiran_sai_ramineni">@Kiran_Sai_Ramineni</a> ,<br> Yes, I am trying to quantize the model from a “saved_model.pb” file by the<br> “tf.lite.TFLiteConverter.from_saved_model( … )” API.<br> I was using TF 2.15.0 in both local and Colab and no error when loading the model.<br> I don’t have a chance to try 2.17.0 after Colab upgrade to 2.17.0 on 2024-08-20.<br> As I knew TF has a big change on 2.16, which is taking Keras 3 as default from Keras 2, I am not sure if this has something to do with the saved_model…</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_11' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/Kiran_Sai_Ramineni'><span itemprop='name'>Kiran_Sai_Ramineni</span></a> </span> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-09-24T08:02:03Z' class='post-time'> September 24, 2024, 8:02am </time> <meta itemprop='dateModified' content='2024-09-24T08:02:03Z'> <span itemprop='position'>11</span> </span> </div> <div class='post' itemprop='text'> <p>Hi <a class="mention" href="/u/saraphineser">@saraphinesER</a>, I am not able to load the model from the saved_model.pd file, because when you save the model using model.save() there are also other files created which have model weights and meta data to load the model. Could please provide all the files to load the model. Thank You.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> <div id='post_13' itemprop='comment' itemscope itemtype='http://schema.org/Comment' class='topic-body crawler-post'> <div class='crawler-post-meta'> <span class="creator" itemprop="author" itemscope itemtype="http://schema.org/Person"> <a itemprop="url" href='https://discuss.ai.google.dev/u/saraphinesER'><span itemprop='name'>saraphinesER</span></a> </span> <link itemprop="image" href="https://canada1.discourse-cdn.com/googleaitrial/original/3X/a/f/af9f0f23c10608508640b2abd94460db3579ea02.png"> <span class="crawler-post-infos"> <time itemprop='datePublished' datetime='2024-09-25T07:36:05Z' class='post-time'> September 25, 2024, 7:36am </time> <meta itemprop='dateModified' content='2024-09-25T07:36:05Z'> <span itemprop='position'>13</span> </span> </div> <div class='post' itemprop='text'> <p>Hi, <a class="mention" href="/u/kiran_sai_ramineni">@Kiran_Sai_Ramineni</a>,<br> Sorry for not describe well, actually other information are saved in same layer of the pb model as “variables” folder.<br> <img src="https://canada1.discourse-cdn.com/googleaitrial/original/3X/a/f/af9f0f23c10608508640b2abd94460db3579ea02.png" alt="image" data-base62-sha1="p3Cl4OFSn4yRT9yT0UbmRk0E16a" width="277" height="207"><br> So, to load the ‘<a href="https://github.com/breizhn/DTLN/blob/master/pretrained_model/dtln_saved_model/saved_model.pb" rel="noopener nofollow ugc">saved_model.pb </a>’ it should write as below</p> <blockquote> <p>model = load_model(“<a href="https://github.com/breizhn/DTLN/tree/master/pretrained_model/dtln_saved_model" rel="noopener nofollow ugc">.\breizhn_DTLN\pretrained_model\dtln_saved_model</a>”, compile=False)</p> </blockquote> <p>I also load the model in the same way in my script.<br> <a href="https://drive.google.com/file/d/1m4VoPJt8VYdK7qeatLy0F7jtNICrMW_h/view?usp=drive_link" rel="noopener nofollow ugc">breizhn_DTLN_PTQ_modify.py</a></p> <p>Thanks for your patient.</p> </div> <div itemprop="interactionStatistic" itemscope itemtype="http://schema.org/InteractionCounter"> <meta itemprop="interactionType" content="http://schema.org/LikeAction"/> <meta itemprop="userInteractionCount" content="0" /> <span class='post-likes'></span> </div> </div> </div> <div id="related-topics" class="more-topics__list " role="complementary" aria-labelledby="related-topics-title"> <h3 id="related-topics-title" class="more-topics__list-title"> Related topics </h3> <div class="topic-list-container" itemscope itemtype='http://schema.org/ItemList'> <meta itemprop='itemListOrder' content='http://schema.org/ItemListOrderDescending'> <table class='topic-list'> <thead> <tr> <th>Topic</th> <th></th> <th class="replies">Replies</th> <th class="views">Views</th> <th>Activity</th> </tr> </thead> <tbody> <tr class="topic-list-item" id="topic-list-item-28893"> <td class="main-link" itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <meta itemprop='position' content='1'> <span class="link-top-line"> <a itemprop='url' href='https://discuss.ai.google.dev/t/integer-quantization-of-lstm-model/28893' class='title raw-link raw-topic-link'>Integer Quantization of LSTM model</a> </span> <div class="link-bottom-line"> <a href='/c/tensorflow/general-discussion-6/37' class='badge-wrapper bullet'> <span class='badge-category-bg' style='background-color: #f2a346'></span> <span class='badge-category clear-badge'> <span class='category-name'>General Discussion</span> </span> </a> <div class="discourse-tags"> <a href='https://discuss.ai.google.dev/tag/models' class='discourse-tag'>models</a> ,  <a href='https://discuss.ai.google.dev/tag/keras' class='discourse-tag'>keras</a> ,  <a href='https://discuss.ai.google.dev/tag/tflite' class='discourse-tag'>tflite</a> ,  <a href='https://discuss.ai.google.dev/tag/model_optimization' class='discourse-tag'>model_optimization</a> ,  <a href='https://discuss.ai.google.dev/tag/help_request' class='discourse-tag'>help_request</a> </div> </div> </td> <td class="replies"> <span class='posts' title='posts'>3</span> </td> <td class="views"> <span class='views' title='views'>3034</span> </td> <td> June 30, 2021 </td> </tr> <tr class="topic-list-item" id="topic-list-item-29854"> <td class="main-link" itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <meta itemprop='position' content='2'> <span class="link-top-line"> <a itemprop='url' href='https://discuss.ai.google.dev/t/problem-with-quantization-of-gru-model/29854' class='title raw-link raw-topic-link'>Problem with quantization of GRU model</a> </span> <div class="link-bottom-line"> <a href='/c/tensorflow/general-discussion-6/37' class='badge-wrapper bullet'> <span class='badge-category-bg' style='background-color: #f2a346'></span> <span class='badge-category clear-badge'> <span class='category-name'>General Discussion</span> </span> </a> <div class="discourse-tags"> <a href='https://discuss.ai.google.dev/tag/models' class='discourse-tag'>models</a> ,  <a href='https://discuss.ai.google.dev/tag/tflite' class='discourse-tag'>tflite</a> ,  <a href='https://discuss.ai.google.dev/tag/help_request' class='discourse-tag'>help_request</a> </div> </div> </td> <td class="replies"> <span class='posts' title='posts'>1</span> </td> <td class="views"> <span class='views' title='views'>1137</span> </td> <td> August 23, 2024 </td> </tr> <tr class="topic-list-item" id="topic-list-item-25252"> <td class="main-link" itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <meta itemprop='position' content='3'> <span class="link-top-line"> <a itemprop='url' href='https://discuss.ai.google.dev/t/quantizing-the-models-using-tflite/25252' class='title raw-link raw-topic-link'>Quantizing the models using tflite</a> </span> <div class="link-bottom-line"> <a href='/c/tensorflow/general-discussion-6/37' class='badge-wrapper bullet'> <span class='badge-category-bg' style='background-color: #f2a346'></span> <span class='badge-category clear-badge'> <span class='category-name'>General Discussion</span> </span> </a> <div class="discourse-tags"> <a href='https://discuss.ai.google.dev/tag/models' class='discourse-tag'>models</a> ,  <a href='https://discuss.ai.google.dev/tag/keras' class='discourse-tag'>keras</a> ,  <a href='https://discuss.ai.google.dev/tag/tflite' class='discourse-tag'>tflite</a> ,  <a href='https://discuss.ai.google.dev/tag/post-training' class='discourse-tag'>post-training</a> </div> </div> </td> <td class="replies"> <span class='posts' title='posts'>2</span> </td> <td class="views"> <span class='views' title='views'>320</span> </td> <td> January 4, 2024 </td> </tr> <tr class="topic-list-item" id="topic-list-item-32148"> <td class="main-link" itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <meta itemprop='position' content='4'> <span class="link-top-line"> <a itemprop='url' href='https://discuss.ai.google.dev/t/quantization-aware-training-in-output-still-float32/32148' class='title raw-link raw-topic-link'>Quantization aware training -> In/Output still float32?</a> </span> <div class="link-bottom-line"> <a href='/c/tensorflow/general-discussion-6/37' class='badge-wrapper bullet'> <span class='badge-category-bg' style='background-color: #f2a346'></span> <span class='badge-category clear-badge'> <span class='category-name'>General Discussion</span> </span> </a> <div class="discourse-tags"> <a href='https://discuss.ai.google.dev/tag/models' class='discourse-tag'>models</a> ,  <a href='https://discuss.ai.google.dev/tag/keras' class='discourse-tag'>keras</a> ,  <a href='https://discuss.ai.google.dev/tag/model_optimization' class='discourse-tag'>model_optimization</a> ,  <a href='https://discuss.ai.google.dev/tag/help_request' class='discourse-tag'>help_request</a> </div> </div> </td> <td class="replies"> <span class='posts' title='posts'>1</span> </td> <td class="views"> <span class='views' title='views'>1586</span> </td> <td> May 13, 2022 </td> </tr> <tr class="topic-list-item" id="topic-list-item-29057"> <td class="main-link" itemprop='itemListElement' itemscope itemtype='http://schema.org/ListItem'> <meta itemprop='position' content='5'> <span class="link-top-line"> <a itemprop='url' href='https://discuss.ai.google.dev/t/tfliteconverter-adds-de-quantization-blocks-before-and-after-operations-on-a-weight-variable/29057' class='title raw-link raw-topic-link'>TFLiteConverter adds (de)quantization blocks before(and after) operations on a weight variable</a> </span> <div class="link-bottom-line"> <a href='/c/tensorflow/general-discussion-6/37' class='badge-wrapper bullet'> <span class='badge-category-bg' style='background-color: #f2a346'></span> <span class='badge-category clear-badge'> <span class='category-name'>General Discussion</span> </span> </a> <div class="discourse-tags"> <a href='https://discuss.ai.google.dev/tag/tflite' class='discourse-tag'>tflite</a> ,  <a href='https://discuss.ai.google.dev/tag/help_request' class='discourse-tag'>help_request</a> </div> </div> </td> <td class="replies"> <span class='posts' title='posts'>2</span> </td> <td class="views"> <span class='views' title='views'>595</span> </td> <td> February 18, 2023 </td> </tr> </tbody> </table> </div> </div> </div> <footer class="container wrap"> <nav class='crawler-nav'> <ul> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/' itemprop="url">Home </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/categories' itemprop="url">Categories </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='/guidelines' itemprop="url">Guidelines </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='https://policies.google.com/terms' itemprop="url">Terms of Service </a> </span> </li> <li itemscope itemtype='http://schema.org/SiteNavigationElement'> <span itemprop='name'> <a href='https://policies.google.com/privacy' itemprop="url">Privacy Policy </a> </span> </li> </ul> </nav> <p class='powered-by-link'>Powered by <a href="https://www.discourse.org">Discourse</a>, best viewed with JavaScript enabled</p> </footer> <div class="buorg"><div>Unfortunately, <a href="https://www.discourse.org/faq/#browser">your browser is unsupported</a>. Please <a href="https://browsehappy.com">switch to a supported browser</a> to view rich content, log in and reply.</div></div> </body> </html>

CINXE.COM

How to correctly do TFLite fully integer quantization of DTLN (Dual-signal Transformation LSTM Network)? - TensorFlow - Build with Google AI