The NSynth Dataset

<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"> <meta http-equiv="X-UA-Compatible" content="IE=edge"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>The NSynth Dataset</title> <meta name="description" content="A large-scale and high-quality dataset of annotated musical notes.">  <meta property="og:image" content="https://magenta.tensorflow.org/assets/magenta-logo-card.jpg"> <meta property="og:title" content="The NSynth Dataset"> <meta property="og:description" content="A large-scale and high-quality dataset of annotated musical notes."> <meta property="og:url" content="https://magenta.tensorflow.org/datasets/nsynth"> <meta property="og:site_name" content="Magenta">  <meta name="twitter:card" content="summary"> <meta name="twitter:title" content="The NSynth Dataset"> <meta name="twitter:description" content="A large-scale and high-quality dataset of annotated musical notes."> <meta name="twitter:image" content="https://magenta.tensorflow.org/assets/magenta-logo-card.jpg"> <link rel="stylesheet" href="/css/main.css"> <link rel="canonical" href="https://magenta.tensorflow.org/datasets/nsynth"> <link rel="alternate" type="application/rss+xml" title="Magenta" href="https://magenta.tensorflow.org/feed.xml"> <link href="https://fonts.googleapis.com/css?family=Google+Sans:+400,500,700" media="all" rel="stylesheet"> <script src="//www.google.com/js/gweb/analytics/autotrack.js"></script> <script> new gweb.analytics.AutoTrack({ profile: 'UA-80107903-1' }); </script> </head> <body> <div class="scrim" onclick="document.body.classList.toggle('drawer-opened', false)"></div> <header> <div class="top-bar background"> <div class="top-bar-content"> <div class="logo"> <a href="/"><img src="/assets/magenta-logo.png" height="70" alt="magenta logo"></a> </div> <nav> <button class="menu-button" onclick="document.body.classList.toggle('drawer-opened', true)" aria-label="open nav menu"> <svg viewBox="0 0 18 15"> <path fill="#424242" d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.031C17.335,0,18,0.665,18,1.484L18,1.484z"/> <path fill="#424242" d="M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0c0-0.82,0.665-1.484,1.484-1.484 h15.031C17.335,6.031,18,6.696,18,7.516L18,7.516z"/> <path fill="#424242" d="M18,13.516C18,14.335,17.335,15,16.516,15H1.484C0.665,15,0,14.335,0,13.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.031C17.335,12.031,18,12.696,18,13.516L18,13.516z"/> </svg> </button> <div class="links"> <a href="/get-started">Get Started</a> <a href="/studio">Studio</a> <a href="/ddsp-vst">DDSP-VST</a> <a href="/demos">Demos</a> <a href="/blog">Blog</a> <a href="/research">Research</a> <a href="/talks">Talks</a> <a href="/community">Community</a> </div> </nav> </div> </div> </header> <div class="drawer"> <div class="drawer-content"> <a href="/get-started">Get Started</a> <a href="/studio">Studio</a> <a href="/ddsp-vst">DDSP-VST</a> <a href="/demos">Demos</a> <a href="/blog">Blog</a> <a href="/research">Research</a> <a href="/talks">Talks</a> <a href="/community">Community</a> </div> </div> <div class="main"> <section class="white"> <article class="content single" itemscope itemtype="http://schema.org/BlogPosting"> <h1 class="post-title" itemprop="name headline">The NSynth Dataset</h1> <time datetime="2017-04-05T11:00:00-07:00" itemprop="datePublished">Apr 5, 2017</time> NSynth <div class="article-body" itemprop="articleBody"> A large-scale and high-quality dataset of annotated musical notes.  <h1 class="no_toc" id="download"><a href="#files">Download</a></h1> <h1 class="no_toc" id="contents">Contents</h1> <ul id="markdown-toc"> <li><a href="#motivation" id="markdown-toc-motivation">Motivation</a></li> <li><a href="#description" id="markdown-toc-description">Description</a></li> <li><a href="#format" id="markdown-toc-format">Format</a> <ul> <li><a href="#files" id="markdown-toc-files">Files</a></li> <li><a href="#example-features" id="markdown-toc-example-features">Example Features</a></li> <li><a href="#feature-encodings" id="markdown-toc-feature-encodings">Feature Encodings</a> <ul> <li><a href="#instrument-sources" id="markdown-toc-instrument-sources">Instrument Sources</a></li> <li><a href="#instrument-families" id="markdown-toc-instrument-families">Instrument Families</a></li> <li><a href="#note-qualities" id="markdown-toc-note-qualities">Note Qualities</a></li> </ul> </li> <li><a href="#example" id="markdown-toc-example">Example</a></li> </ul> </li> <li><a href="#statistics" id="markdown-toc-statistics">Statistics</a> <ul> <li><a href="#instrument-classes" id="markdown-toc-instrument-classes">Instrument Classes</a></li> <li><a href="#qualitiy-co-occurrences" id="markdown-toc-qualitiy-co-occurrences">Qualitiy Co-occurrences</a></li> </ul> </li> <li><a href="#license" id="markdown-toc-license">License</a></li> <li><a href="#how-to-cite" id="markdown-toc-how-to-cite">How to Cite</a></li> <li><a href="#updates" id="markdown-toc-updates">Updates</a></li> </ul> <h1 id="motivation">Motivation</h1> Recent breakthroughs in generative modeling of images have been predicated on the availability of high-quality and large-scale datasebts such as MNIST, CIFAR and ImageNet. We recognized the need for an audio dataset that was as approachable as those in the image domain. Audio signals found in the wild contain multi-scale dependencies that prove particularly difficult to model, leading many previous efforts at data-driven audio synthesis to focus on more constrained domains such as texture synthesis or training small parametric models. We encourage the broader community to use NSynth as a benchmark and entry point into audio machine learning. We also view NSynth as a building block for future datasets and envision a high-quality multi-note dataset for tasks like generation and transcription that involve learning complex language-like dependencies. <h1 id="description">Description</h1> NSynth is an audio dataset containing 305,979 musical notes, each with a unique pitch, timbre, and envelope. For 1,006 instruments from commercial sample libraries, we generated four second, monophonic 16kHz audio snippets, referred to as notes, by ranging over every pitch of a standard MIDI pian o (21-108) as well as five different velocities (25, 50, 75, 100, 127). The note was held for the first three seconds and allowed to decay for the final second. Some instruments are not capable of producing all 88 pitches in this range, resulting in an average of 65.4 pitches per instrument. Furthermore, the commercial sample packs occasionally contain duplicate sounds across multiple velocities, leaving an average of 4.75 unique velocities per pitch. We also annotated each of the notes with three additional pieces of information based on a combination of human evaluation and heuristic algorithms: <ul> <li> Source: The method of sound production for the note’s instrument. This can be one of <code class="language-plaintext highlighter-rouge">acoustic</code> or <code class="language-plaintext highlighter-rouge">electronic</code> for instruments that were recorded from acoustic or electronic instruments, respectively, or <code class="language-plaintext highlighter-rouge">synthetic</code> for synthesized instruments. See their <a href="#instrument-classes">frequencies</a> below. </li> <li> Family: The high-level family of which the note’s instrument is a member. Each instrument is a member of exactly one family. See <a href="#instrument-families">the complete list</a> and their <a href="#instrument-classes">frequencies</a> below. </li> <li> Qualities: Sonic qualities of the note. See the quality <a href="#note-qualities">descriptions</a> and their <a href="#quality-co-occurrences">co-occurrences</a> below. Each note is annotated with zero or more qualities. </li> </ul> <h1 id="format">Format</h1> <h2 id="files">Files</h2> The NSynth dataset can be download in two formats: <ul> <li><a href="https://www.tensorflow.org/api_guides/python/python_io#tfrecords_format_details">TFRecord</a> files of serialized <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/example/example.proto">TensorFlow Example protocol buffers</a> with one Example proto per note.</li> <li>JSON files containing non-audio features alongside 16-bit PCM WAV audio files.</li> </ul> The full dataset is split into three sets: <ul> <li>Train [<a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-train.tfrecord">tfrecord</a> | <a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-train.jsonwav.tar.gz">json/wav</a>]: A training set with 289,205 examples. Instruments do not overlap with valid or test.</li> <li>Valid [<a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-valid.tfrecord">tfrecord</a> | <a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-valid.jsonwav.tar.gz">json/wav</a>]: A validation set with 12,678 examples. Instruments do not overlap with train.</li> <li>Test [<a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-test.tfrecord">tfrecord</a> | <a href="http://download.magenta.tensorflow.org/datasets/nsynth/nsynth-test.jsonwav.tar.gz">json/wav</a>]: A test set with 4,096 examples. Instruments do not overlap with train.</li> </ul> Below we detail how the note features are encoded in the Example protocol buffers and JSON files. <h2 id="example-features">Example Features</h2> Each Example contains the following features. <table> <thead> <tr> <th>Feature</th> <th>Type</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td>note</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>A unique integer identifier for the note.</td> </tr> <tr> <td>note_str</td> <td><code class="language-plaintext highlighter-rouge">bytes</code></td> <td>A unique string identifier for the note in the format <code class="language-plaintext highlighter-rouge"><instrument_str>-<pitch>-<velocity></code>.</td> </tr> <tr> <td>instrument</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>A unique, sequential identifier for the instrument the note was synthesized from.</td> </tr> <tr> <td>instrument_str</td> <td><code class="language-plaintext highlighter-rouge">bytes</code></td> <td>A unique string identifier for the instrument this note was synthesized from in the format <code class="language-plaintext highlighter-rouge"><instrument_family_str>-<instrument_production_str>-<instrument_name></code>.</td> </tr> <tr> <td>pitch</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>The 0-based MIDI pitch in the range [0, 127].</td> </tr> <tr> <td>velocity</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>The 0-based MIDI velocity in the range [0, 127].</td> </tr> <tr> <td>sample_rate</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>The samples per second for the <code class="language-plaintext highlighter-rouge">audio</code> feature.</td> </tr> <tr> <td>audio*</td> <td><code class="language-plaintext highlighter-rouge">[float]</code></td> <td>A list of audio samples represented as floating point values in the range [-1,1].</td> </tr> <tr> <td>qualities</td> <td><code class="language-plaintext highlighter-rouge">[int64]</code></td> <td>A binary vector representing which <a href="#note-qualities">sonic qualities</a> are present in this note.</td> </tr> <tr> <td>qualities_str</td> <td><code class="language-plaintext highlighter-rouge">[bytes]</code></td> <td>A list IDs of which qualities are present in this note selected from the <a href="#note-qualities">sonic qualities list</a>.</td> </tr> <tr> <td>instrument_family</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>The index of the <a href="#instrument-families">instrument family</a> this instrument is a member of.</td> </tr> <tr> <td>instrument_family_str</td> <td><code class="language-plaintext highlighter-rouge">bytes</code></td> <td>The ID of the <a href="#instrument-families">instrument family</a> this instrument is a member of.</td> </tr> <tr> <td>instrument_source</td> <td><code class="language-plaintext highlighter-rouge">int64</code></td> <td>The index of the <a href="#instrument-sources">sonic source</a> for this instrument.</td> </tr> <tr> <td>instrument_source_str</td> <td><code class="language-plaintext highlighter-rouge">bytes</code></td> <td>The ID of the <a href="#instrument-sources">sonic source</a> for this instrument.</td> </tr> </tbody> </table> * Note: the “audio” feature is ommited from the JSON-encoded examples since the audio data is stored separately in WAV files keyed by the “note_str”. <h2 id="feature-encodings">Feature Encodings</h2> This section includes tables specifying the feature names and indicies used in the Example protos. <h3 id="instrument-sources">Instrument Sources</h3> The method of sound production for the note’s instrument. Each instrument (and all of its notes) are labeled with exactly one. <table> <thead> <tr> <th>Index</th> <th>ID</th> </tr> </thead> <tbody> <tr> <td>0</td> <td>acoustic</td> </tr> <tr> <td>1</td> <td>electronic</td> </tr> <tr> <td>2</td> <td>synthetic</td> </tr> </tbody> </table> <h3 id="instrument-families">Instrument Families</h3> The high-level family of which the note’s instrument is a member. Each instrument (and all of its notes) are labeled with exactly one. <table> <thead> <tr> <th>Index</th> <th>ID</th> </tr> </thead> <tbody> <tr> <td>0</td> <td>bass</td> </tr> <tr> <td>1</td> <td>brass</td> </tr> <tr> <td>2</td> <td>flute</td> </tr> <tr> <td>3</td> <td>guitar</td> </tr> <tr> <td>4</td> <td>keyboard</td> </tr> <tr> <td>5</td> <td>mallet</td> </tr> <tr> <td>6</td> <td>organ</td> </tr> <tr> <td>7</td> <td>reed</td> </tr> <tr> <td>8</td> <td>string</td> </tr> <tr> <td>9</td> <td>synth_lead</td> </tr> <tr> <td>10</td> <td>vocal</td> </tr> </tbody> </table> <h3 id="note-qualities">Note Qualities</h3> We provide quality annotations for the 10 different note qualities described below. None of the tags are mutually exclusive by definition except for “bright” and “dark”. However, it is possible for a note to be neither “bright” nor “dark”. <table> <thead> <tr> <th>Index</th> <th>ID</th> <th>Description</th> </tr> </thead> <tbody> <tr> <td>0</td> <td>bright</td> <td>A large amount of high frequency content and strong upper harmonics.</td> </tr> <tr> <td>1</td> <td>dark</td> <td>A distinct lack of high frequency content, giving a muted and bassy sound. Also sometimes described as ‘Warm’.</td> </tr> <tr> <td>2</td> <td>distortion</td> <td>Waveshaping that produces a distinctive crunchy sound and presence of many harmonics. Sometimes paired with non-harmonic noise.</td> </tr> <tr> <td>3</td> <td>fast_decay</td> <td>Amplitude envelope of all harmonics decays substantially before the ‘note-off’ point at 3 seconds.</td> </tr> <tr> <td>4</td> <td>long_release</td> <td>Amplitude envelope decays slowly after the ‘note-off’ point, sometimes still present at the end of the sample 4 seconds.</td> </tr> <tr> <td>5</td> <td>multiphonic</td> <td>Presence of overtone frequencies related to more than one fundamental frequency.</td> </tr> <tr> <td>6</td> <td>nonlinear_env</td> <td>Modulation of the sound with a distinct envelope behavior different than the monotonic decrease of the note. Can also include filter envelopes as well as dynamic envelopes.</td> </tr> <tr> <td>7</td> <td>percussive</td> <td>A loud non-harmonic sound at note onset.</td> </tr> <tr> <td>8</td> <td>reverb</td> <td>Room acoustics that were not able to be removed from the original sample.</td> </tr> <tr> <td>9</td> <td>tempo-synced</td> <td>Rhythmic modulation of the sound to a fixed tempo.</td> </tr> </tbody> </table> <h2 id="example">Example</h2> Below is a string view of the Example protocol buffer for an individual note in the dataset, with the audio portion suppressed: <div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{ # (tensorflow.Example) size=250.4K features: { # (tensorflow.Features) size=250.4K feature: { # (tensorflow.Features.FeatureEntry) size=21B key : "sample_rate" # size=11 value: { # (tensorflow.Feature) size=6B int64_list: { # (tensorflow.Int64List) size=4B value: [ 16000 ] } # features.feature[0].value.int64_list } # features.feature[0].value } # features.feature[0] feature: { # (tensorflow.Features.FeatureEntry) size=25B key : "qualities_str" # size=13 value: { # (tensorflow.Feature) size=8B bytes_list: { # (tensorflow.BytesList) size=6B value: [ "dark" ] } # features.feature[1].value.bytes_list } # features.feature[1].value } # features.feature[1] feature: { # (tensorflow.Features.FeatureEntry) size=42B key : "note_str" # size=8 value: { # (tensorflow.Feature) size=30B bytes_list: { # (tensorflow.BytesList) size=28B value: [ "bass_synthetic_033-022-050" ] # size=26 } # features.feature[2].value.bytes_list } # features.feature[2].value } # features.feature[2] feature: { # (tensorflow.Features.FeatureEntry) size=27B key : "qualities" # size=9 value: { # (tensorflow.Feature) size=14B int64_list: { # (tensorflow.Int64List) size=12B value: [ 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 ] } # features.feature[3].value.int64_list } # features.feature[3].value } # features.feature[3] feature: { # (tensorflow.Features.FeatureEntry) size=250.0K key : "audio" # size=5 value: { # (tensorflow.Feature) size=250.0K float_list: { # (tensorflow.FloatList) size=250.0K value: [ -1.3311218e-07, ..., 1.3244664e-07 ] } # features.feature[4].value.float_list } # features.feature[4].value } # features.feature[4] feature: { # (tensorflow.Features.FeatureEntry) size=26B key : "instrument_family" # size=17 value: { # (tensorflow.Feature) size=5B int64_list: { # (tensorflow.Int64List) size=3B value: [ 0 ] } # features.feature[5].value.int64_list } # features.feature[5].value } # features.feature[5] feature: { # (tensorflow.Features.FeatureEntry) size=14B key : "pitch" # size=5 value: { # (tensorflow.Feature) size=5B int64_list: { # (tensorflow.Int64List) size=3B value: [ 22 ] } # features.feature[6].value.int64_list } # features.feature[6].value } # features.feature[6] feature: { # (tensorflow.Features.FeatureEntry) size=26B key : "instrument_source" # size=17 value: { # (tensorflow.Feature) size=5B int64_list: { # (tensorflow.Int64List) size=3B value: [ 2 ] } # features.feature[7].value.int64_list } # features.feature[7].value } # features.feature[7] feature: { # (tensorflow.Features.FeatureEntry) size=40B key : "instrument_str" # size=14 value: { # (tensorflow.Feature) size=22B bytes_list: { # (tensorflow.BytesList) size=20B value: [ "bass_synthetic_033" ] # size=18 } # features.feature[8].value.bytes_list } # features.feature[8].value } # features.feature[8] feature: { # (tensorflow.Features.FeatureEntry) size=38B key : "instrument_source_str" # size=21 value: { # (tensorflow.Feature) size=13B bytes_list: { # (tensorflow.BytesList) size=11B value: [ "synthetic" ] # size=9 } # features.feature[9].value.bytes_list } # features.feature[9].value } # features.feature[9] feature: { # (tensorflow.Features.FeatureEntry) size=15B key : "note" value: { # (tensorflow.Feature) size=7B int64_list: { # (tensorflow.Int64List) size=5B value: [ 201034 ] # 196.32Ki; [if seconds]: 2 days 7 hours } # features.feature[10].value.int64_list } # features.feature[10].value } # features.feature[10] feature: { # (tensorflow.Features.FeatureEntry) size=20B key : "instrument" # size=10 value: { # (tensorflow.Feature) size=6B int64_list: { # (tensorflow.Int64List) size=4B value: [ 417 ] } # features.feature[11].value.int64_list } # features.feature[11].value } # features.feature[11] feature: { # (tensorflow.Features.FeatureEntry) size=33B key : "instrument_family_str" # size=21 value: { # (tensorflow.Feature) size=8B bytes_list: { # (tensorflow.BytesList) size=6B value: [ "bass" ] } # features.feature[12].value.bytes_list } # features.feature[12].value } # features.feature[12] feature: { # (tensorflow.Features.FeatureEntry) size=17B key : "velocity" # size=8 value: { # (tensorflow.Feature) size=5B int64_list: { # (tensorflow.Int64List) size=3B value: [ 50 ] } # features.feature[13].value.int64_list } # features.feature[13].value } # features.feature[13] } # features } </code></pre></div></div> This is a view of the same Example in JSON format: <div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code>"bass_synthetic_033-022-050": { "note": 201034, "sample_rate": 16000, "instrument_family": 0, "qualities": [ 0, 1, 0, 0, 0, 0, 0, 0, 0, 0 ], "instrument_source_str": "synthetic", "note_str": "bass_synthetic_033-022-050", "instrument_family_str": "bass", "instrument_str": "bass_synthetic_033", "pitch": 22, "instrument": 417, "velocity": 50, "instrument_source": 2, "qualities_str": [ "dark" ] } </code></pre></div></div> <h1 id="statistics">Statistics</h1> <h2 id="instrument-classes">Instrument Classes</h2> Frequency counts for instrument classes with Sources as columns an Families as rows. <table> <thead> <tr> <th>Family</th> <th style="text-align: right">Acoustic</th> <th style="text-align: right">Electronic</th> <th style="text-align: right">Synthetic</th> <th style="text-align: right">Total</th> </tr> </thead> <tbody> <tr> <td>Bass</td> <td style="text-align: right">200</td> <td style="text-align: right">8,387</td> <td style="text-align: right">60,368</td> <td style="text-align: right">68,955</td> </tr> <tr> <td>Brass</td> <td style="text-align: right">13,760</td> <td style="text-align: right">70</td> <td style="text-align: right">0</td> <td style="text-align: right">13,830</td> </tr> <tr> <td>Flute</td> <td style="text-align: right">6,572</td> <td style="text-align: right">35</td> <td style="text-align: right">2,816</td> <td style="text-align: right">9,423</td> </tr> <tr> <td>Guitar</td> <td style="text-align: right">13,343</td> <td style="text-align: right">16,805</td> <td style="text-align: right">5,275</td> <td style="text-align: right">35,423</td> </tr> <tr> <td>Keyboard</td> <td style="text-align: right">8,508</td> <td style="text-align: right">42,645</td> <td style="text-align: right">3,838</td> <td style="text-align: right">54,991</td> </tr> <tr> <td>Mallet</td> <td style="text-align: right">27,722</td> <td style="text-align: right">5,581</td> <td style="text-align: right">1,763</td> <td style="text-align: right">35,066</td> </tr> <tr> <td>Organ</td> <td style="text-align: right">176</td> <td style="text-align: right">36,401</td> <td style="text-align: right">0</td> <td style="text-align: right">36,577</td> </tr> <tr> <td>Reed</td> <td style="text-align: right">14,262</td> <td style="text-align: right">76</td> <td style="text-align: right">528</td> <td style="text-align: right">14,866</td> </tr> <tr> <td>String</td> <td style="text-align: right">20,510</td> <td style="text-align: right">84</td> <td style="text-align: right">0</td> <td style="text-align: right">20,594</td> </tr> <tr> <td>Synth Lead</td> <td style="text-align: right">0</td> <td style="text-align: right">0</td> <td style="text-align: right">5,501</td> <td style="text-align: right">5,501</td> </tr> <tr> <td>Vocal</td> <td style="text-align: right">3,925</td> <td style="text-align: right">140</td> <td style="text-align: right">6,688</td> <td style="text-align: right">10,753</td> </tr> <tr> <td>Total</td> <td style="text-align: right">108,978</td> <td style="text-align: right">110,224</td> <td style="text-align: right">86,777</td> <td style="text-align: right">305,979</td> </tr> </tbody> </table> <h2 id="qualitiy-co-occurrences">Qualitiy Co-occurrences</h2> Co-occurrence probabilities and marginal frequencies of quality annotations. Both are presented as percentages. <table> <thead> <tr> <th>Quality</th> <th style="text-align: right">Bright</th> <th style="text-align: right">Dark</th> <th style="text-align: right">Distortion</th> <th style="text-align: right">Fast Decay</th> <th style="text-align: right">Long Release</th> <th style="text-align: right">Multiphonic</th> <th style="text-align: right">Nonlinear Envelope</th> <th style="text-align: right">Percussive</th> <th style="text-align: right">Reverb</th> <th style="text-align: right">Tempo-Synced</th> </tr> </thead> <tbody> <tr> <td>Dark</td> <td style="text-align: right">0.0</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Distortion</td> <td style="text-align: right">25.9</td> <td style="text-align: right">2.5</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Fast Decay</td> <td style="text-align: right">10.0</td> <td style="text-align: right">7.5</td> <td style="text-align: right">8.1</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Long Release</td> <td style="text-align: right">9.0</td> <td style="text-align: right">5.2</td> <td style="text-align: right">9.8</td> <td style="text-align: right">0.0</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Multiphonic</td> <td style="text-align: right">6.0</td> <td style="text-align: right">1.5</td> <td style="text-align: right">5.4</td> <td style="text-align: right">2.8</td> <td style="text-align: right">6.9</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Nonlinear Envelope</td> <td style="text-align: right">8.5</td> <td style="text-align: right">1.4</td> <td style="text-align: right">6.6</td> <td style="text-align: right">2.1</td> <td style="text-align: right">6.7</td> <td style="text-align: right">8.6</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Percussive</td> <td style="text-align: right">6.2</td> <td style="text-align: right">5.1</td> <td style="text-align: right">3.0</td> <td style="text-align: right">52.0</td> <td style="text-align: right">0.8</td> <td style="text-align: right">2.4</td> <td style="text-align: right">0.9</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Reverb</td> <td style="text-align: right">6.6</td> <td style="text-align: right">8.9</td> <td style="text-align: right">0.3</td> <td style="text-align: right">13.0</td> <td style="text-align: right">13.7</td> <td style="text-align: right">0.7</td> <td style="text-align: right">3.5</td> <td style="text-align: right">12.4</td> <td style="text-align: right"> </td> <td style="text-align: right"> </td> </tr> <tr> <td>Tempo-Synced</td> <td style="text-align: right">2.4</td> <td style="text-align: right">1.8</td> <td style="text-align: right">5.2</td> <td style="text-align: right">0.4</td> <td style="text-align: right">6.4</td> <td style="text-align: right">9.3</td> <td style="text-align: right">2.3</td> <td style="text-align: right">1.5</td> <td style="text-align: right">0.0</td> <td style="text-align: right"> </td> </tr> <tr> <td>Frequency</td> <td style="text-align: right">13.5</td> <td style="text-align: right">11.0</td> <td style="text-align: right">17.0</td> <td style="text-align: right">14.7</td> <td style="text-align: right">8.5</td> <td style="text-align: right">3.4</td> <td style="text-align: right">3.2</td> <td style="text-align: right">10.2</td> <td style="text-align: right">16.8</td> <td style="text-align: right">1.8</td> </tr> </tbody> </table> <h1 id="license">License</h1> The dataset is made available by Google Inc. under a <a href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International (CC BY 4.0) license</a>. <h1 id="how-to-cite">How to Cite</h1> If you use the NSynth dataset in your work, please cite the <a href="https://arxiv.org/abs/1704.01279">paper</a> where it was introduced: <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Jesse Engel, Cinjon Resnick, Adam Roberts, Sander Dieleman, Douglas Eck, Karen Simonyan, and Mohammad Norouzi. "Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders." 2017. </code></pre></div></div> You can also use the following BibTeX entry: <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@misc{nsynth2017, Author = {Jesse Engel and Cinjon Resnick and Adam Roberts and Sander Dieleman and Douglas Eck and Karen Simonyan and Mohammad Norouzi}, Title = {Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders}, Year = {2017}, Eprint = {arXiv:1704.01279}, } </code></pre></div></div> <h1 id="updates">Updates</h1> <ul> <li>04-10-2017: Removed 64 duplicate notes from train set.</li> </ul>  </div> </article> </section> </div> <footer> <div class="footer-content"> <div class="logo"> <a href="https://ai.google/" target="_blank" rel="noopener" title="Google AI"> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 157.2 40.88"><defs><style>.cls-1{fill:none;}.cls-2{fill:#80868b;}.cls-3{fill:#80868b;}.cls-4{fill:#80868b;}.cls-5{fill:#80868b;}.cls-6{fill:#80868b;}</style></defs><g id="Running_copy" data-name="Running copy"><path class="cls-1" d="M82.91,16.35A4.8,4.8,0,0,0,79.29,18a5.66,5.66,0,0,0-1.49,4,5.53,5.53,0,0,0,1.49,3.94,4.78,4.78,0,0,0,3.62,1.58,4.47,4.47,0,0,0,3.49-1.58A5.7,5.7,0,0,0,87.81,22a5.84,5.84,0,0,0-1.41-4A4.48,4.48,0,0,0,82.91,16.35Z"></path><path class="cls-1" d="M42.8,16.35a4.92,4.92,0,0,0-3.66,1.57,5.49,5.49,0,0,0-1.51,4,5.52,5.52,0,0,0,1.52,4,5,5,0,0,0,7.3,0,5.48,5.48,0,0,0,1.53-4,5.49,5.49,0,0,0-1.51-4A4.93,4.93,0,0,0,42.8,16.35Z"></path><path class="cls-1" d="M62.89,16.35a4.93,4.93,0,0,0-3.67,1.57,5.53,5.53,0,0,0-1.51,4,5.48,5.48,0,0,0,1.53,4,5,5,0,0,0,7.3,0,5.48,5.48,0,0,0,1.53-4,5.49,5.49,0,0,0-1.51-4A4.93,4.93,0,0,0,62.89,16.35Z"></path><path class="cls-1" d="M111,16.82a4.15,4.15,0,0,0-2.12-.54,4.79,4.79,0,0,0-3.32,1.46,4.9,4.9,0,0,0-1.47,3.9l8.2-3.41A2.82,2.82,0,0,0,111,16.82Z"></path><rect class="cls-2" x="94.13" y="3.56" width="4.03" height="26.97"></rect><path class="cls-3" d="M42.8,12.74a9,9,0,0,0-6.53,2.62,8.83,8.83,0,0,0-2.68,6.55,8.84,8.84,0,0,0,2.68,6.56,9.46,9.46,0,0,0,13.07,0A8.83,8.83,0,0,0,52,21.91a8.82,8.82,0,0,0-2.67-6.55A9,9,0,0,0,42.8,12.74Zm3.65,13.15a5,5,0,0,1-7.3,0,5.52,5.52,0,0,1-1.52-4,5.49,5.49,0,0,1,1.51-4,5.06,5.06,0,0,1,7.33,0,5.49,5.49,0,0,1,1.51,4A5.48,5.48,0,0,1,46.45,25.89Z"></path><path class="cls-4" d="M18.89,15.55v3.9h9.32a8.27,8.27,0,0,1-2.12,4.9,9.76,9.76,0,0,1-7.2,2.85,9.75,9.75,0,0,1-7.24-3,10.07,10.07,0,0,1-3-7.33,10.07,10.07,0,0,1,3-7.33,9.75,9.75,0,0,1,7.24-3,9.89,9.89,0,0,1,7,2.78l2.75-2.74a13.63,13.63,0,0,0-9.77-3.93A14.07,14.07,0,0,0,8.71,6.78,13.58,13.58,0,0,0,4.44,16.84,13.56,13.56,0,0,0,8.71,26.9a14.07,14.07,0,0,0,10.18,4.19,13.12,13.12,0,0,0,9.94-4q3.38-3.36,3.37-9.1a12.59,12.59,0,0,0-.2-2.44Z"></path><path class="cls-4" d="M87.53,14.79h-.14a5.64,5.64,0,0,0-2-1.46,6.66,6.66,0,0,0-2.83-.59,8.37,8.37,0,0,0-6.15,2.69A9,9,0,0,0,73.77,22a8.86,8.86,0,0,0,2.64,6.46,8.36,8.36,0,0,0,6.15,2.68A5.87,5.87,0,0,0,87.39,29h.14v1.32a5.63,5.63,0,0,1-1.3,4,4.69,4.69,0,0,1-3.6,1.39,4.34,4.34,0,0,1-2.88-1A5.94,5.94,0,0,1,78,32.44L74.5,33.9a9.43,9.43,0,0,0,3,3.79,8.07,8.07,0,0,0,5.14,1.64,8.61,8.61,0,0,0,6.27-2.39c1.64-1.58,2.45-4,2.45-7.17V13.3H87.53ZM86.4,25.89a4.47,4.47,0,0,1-3.49,1.58,4.78,4.78,0,0,1-3.62-1.58A5.53,5.53,0,0,1,77.8,22a5.66,5.66,0,0,1,1.49-4,4.8,4.8,0,0,1,3.62-1.6A4.48,4.48,0,0,1,86.4,18a5.84,5.84,0,0,1,1.41,4A5.7,5.7,0,0,1,86.4,25.89Z"></path><path class="cls-5" d="M62.89,12.74a9,9,0,0,0-6.53,2.62,8.79,8.79,0,0,0-2.68,6.55,8.8,8.8,0,0,0,2.68,6.56,9.45,9.45,0,0,0,13.06,0,8.8,8.8,0,0,0,2.68-6.56,8.79,8.79,0,0,0-2.68-6.55A9,9,0,0,0,62.89,12.74Zm3.65,13.15a5,5,0,0,1-7.3,0,5.48,5.48,0,0,1-1.53-4,5.53,5.53,0,0,1,1.51-4,5.07,5.07,0,0,1,7.34,0,5.49,5.49,0,0,1,1.51,4A5.48,5.48,0,0,1,66.54,25.89Z"></path><path class="cls-3" d="M109.22,27.47a4.68,4.68,0,0,1-4.45-2.78L117,19.62l-.42-1a11,11,0,0,0-.91-1.81,10.64,10.64,0,0,0-1.49-1.86,7.14,7.14,0,0,0-2.36-1.56,7.73,7.73,0,0,0-3.1-.61,8.27,8.27,0,0,0-6.13,2.57,9.05,9.05,0,0,0-2.52,6.6,8.93,8.93,0,0,0,2.61,6.54,8.74,8.74,0,0,0,6.5,2.64,8.43,8.43,0,0,0,4.69-1.25,10.13,10.13,0,0,0,3-2.82l-3.13-2.08A5.26,5.26,0,0,1,109.22,27.47Zm-3.64-9.73a4.79,4.79,0,0,1,3.32-1.46,4.15,4.15,0,0,1,2.12.54,2.82,2.82,0,0,1,1.29,1.41l-8.2,3.41A4.9,4.9,0,0,1,105.58,17.74Z"></path><path class="cls-6" d="M127.47,30.54h-3.55l9.39-24.9h3.62l9.39,24.9h-3.55l-2.4-6.75H129.9Zm7.58-21L131,20.8h8.28L135.19,9.57Z"></path><path class="cls-6" d="M152.44,30.54h-3.2V5.64h3.2Z"></path></g></svg> </a> </div> <ul> <li> <a href="https://twitter.com/search?q=%23madewithmagenta" target="_blank" rel="noopener"> Twitter </a> </li> <li> <a href="/blog" target="_blank" rel="noopener"> Blog </a> </li> <li> <a href="https://github.com/tensorflow/magenta" target="_blank" rel="noopener"> GitHub </a> </li> <li> <a href="https://www.google.com/policies/privacy/" target="_blank" rel="noopener"> Privacy </a> </li> <li> <a href="https://www.google.com/policies/terms/" target="_blank" rel="noopener"> Terms </a> </li> </ul> </div> </footer> </body> <script src="/js/main.js"></script> </html>

CINXE.COM

The NSynth Dataset