CINXE.COM

<!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <link rel="stylesheet" href="/css/layout.css?1718759055581"> <link rel="stylesheet" href="/css/nav.css?1718759055581"> <link rel="stylesheet" href="/css/style.css?1718759055581"> <link rel="stylesheet" href="/css/prismjs@1.24-themes-prism.css"> <title>IPLD ♦ Glossary</title> </head> <body> <header> <div class="sidebar-button" onclick="document.getElementById('sidebar').classList.toggle('sidebar-open')"> <svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"> <path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path> </svg> </div> <a href="/" class="logo">IPLD</a> <aside id=breadcrumbs> <ul> <li><a href="/glossary/">glossary</a></li> </ul> </aside> </header> <aside id=sidebar> <nav> <ul> <li> <a href="/docs/">Docs</a><ul> <li> <a href="/docs/intro/">Intro</a><ul> <li> <a href="/docs/intro/hello-world/">Hello, World</a></li> <li> <a href="/docs/intro/primer/">The Brief Primer</a></li> <li> <a href="/docs/intro/ecosystem/">InterPlanetary Ecosystem Overview</a></li> <li> <a href="/docs/intro/community/">Finding Community</a></li></ul></li> <li> <a href="/docs/motivation/">Motivation</a><ul> <li> <a href="/docs/motivation/benefits-of-content-addressing/">Benefits of Content Addressing</a></li> <li> <a href="/docs/motivation/data-to-data-structures/">From Data to Data Structures</a></li></ul></li> <li> <a href="/docs/codecs/">Codecs</a><ul> <li> <a href="/docs/codecs/known/">Known Codecs</a><ul> <li> <a href="/docs/codecs/known/dag-cbor/">DAG-CBOR</a></li> <li> <a href="/docs/codecs/known/dag-json/">DAG-JSON</a></li> <li> <a href="/docs/codecs/known/dag-pb/">DAG-PB</a></li></ul></li></ul></li> <li> <a href="/docs/data-model/">Data Model</a><ul> <li> <a href="/docs/data-model/node/">Nodes</a></li> <li> <a href="/docs/data-model/kinds/">Kinds</a></li> <li> <a href="/docs/data-model/pathing/">Pathing</a></li> <li> <a href="/docs/data-model/traversal/">Traversal</a></li></ul></li> <li> <a href="/docs/advanced-data-layouts/">Advanced Data Layouts</a><ul> <li> <a href="/docs/advanced-data-layouts/intro/">Intro to ADLs</a></li> <li> <a href="/docs/advanced-data-layouts/naming/">ADL Naming</a></li> <li> <a href="/docs/advanced-data-layouts/signalling/">Signalling ADLs</a></li> <li> <a href="/docs/advanced-data-layouts/dynamic-loading/">Dynamic Loading</a></li> <li> <a href="/docs/advanced-data-layouts/known/">Known ADLs</a></li></ul></li> <li> <a href="/docs/schemas/">Schemas</a><ul> <li> <a href="/docs/schemas/intro/">Introduction</a><ul> <li> <a href="/docs/schemas/intro/compare/">compare</a></li> <li> <a href="/docs/schemas/intro/goals/">Goals</a></li> <li> <a href="/docs/schemas/intro/feature-summary/">Feature Summary</a></li></ul></li> <li> <a href="/docs/schemas/features/">Features</a><ul> <li> <a href="/docs/schemas/features/typekinds/">Type Kinds</a></li> <li> <a href="/docs/schemas/features/representation-strategies/">Representation Strategies</a></li> <li> <a href="/docs/schemas/features/links/">Links</a></li> <li> <a href="/docs/schemas/features/indicating-adls/">Using ADLs in Schemas</a></li></ul></li> <li> <a href="/docs/schemas/using/">Using Wisely</a><ul> <li> <a href="/docs/schemas/using/authoring-guide/">Authoring Guide</a></li> <li> <a href="/docs/schemas/using/migrations/">Migrations</a></li></ul></li></ul></li> <li> <a href="/docs/synthesis/">Synthesis</a><ul> <li> <a href="/docs/synthesis/gtd/">Getting Things Done</a></li> <li> <a href="/docs/synthesis/building-in-alignment/">Building in Alignment</a></li> <li> <a href="/docs/synthesis/how-ipfs-web-gateways-work/">How IPFS Web Gateways Work</a></li> <li> <a href="/docs/synthesis/encryption/">Working With Encryption</a></li></ul></li></ul></li> <li> <a href="/specs/">Specs</a><ul> <li> <a href="/specs/about/">About the Specifications</a></li> <li> <a href="/specs/codecs/">Codecs</a><ul> <li> <a href="/specs/codecs/dag-cbor/">DAG-CBOR</a><ul> <li> <a href="/specs/codecs/dag-cbor/fixtures/">DAG-CBOR Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-cbor/fixtures/cross-codec/">cross-codec</a></li></ul></li> <li> <a href="/specs/codecs/dag-cbor/spec/">Spec</a></li></ul></li> <li> <a href="/specs/codecs/dag-cosmos/">DAG-COSMOS</a><ul> <li> <a href="/specs/codecs/dag-cosmos/basic_types/">basic_types</a></li> <li> <a href="/specs/codecs/dag-cosmos/cosmos_state/">cosmos_state</a></li> <li> <a href="/specs/codecs/dag-cosmos/crypto_types/">crypto_types</a></li> <li> <a href="/specs/codecs/dag-cosmos/tendermint_chain/">tendermint_chain</a></li> <li> <a href="/specs/codecs/dag-cosmos/typed_protobuf/">typed_protobuf</a></li></ul></li> <li> <a href="/specs/codecs/dag-eth/">DAG-ETH</a><ul> <li> <a href="/specs/codecs/dag-eth/basic_types/">basic_types</a></li> <li> <a href="/specs/codecs/dag-eth/chain/">chain</a></li> <li> <a href="/specs/codecs/dag-eth/convenience_types/">convenience_types</a></li> <li> <a href="/specs/codecs/dag-eth/state/">state</a></li></ul></li> <li> <a href="/specs/codecs/dag-jose/">DAG-JOSE</a><ul> <li> <a href="/specs/codecs/dag-jose/fixtures/">fixtures</a></li> <li> <a href="/specs/codecs/dag-jose/spec/">Spec</a></li></ul></li> <li> <a href="/specs/codecs/dag-json/">DAG-JSON</a><ul> <li> <a href="/specs/codecs/dag-json/fixtures/">DAG-JSON Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-json/fixtures/cross-codec/">cross-codec</a></li></ul></li> <li> <a href="/specs/codecs/dag-json/spec/">Spec</a></li></ul></li> <li> <a href="/specs/codecs/dag-pb/">DAG-PB</a><ul> <li> <a href="/specs/codecs/dag-pb/fixtures/">DAG-PB Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-pb/fixtures/cross-codec/">cross-codec</a></li></ul></li> <li> <a href="/specs/codecs/dag-pb/spec/">Spec</a></li></ul></li></ul></li> <li> <a href="/specs/advanced-data-layouts/">Advanced Data Layouts</a><ul> <li> <a href="/specs/advanced-data-layouts/fbl/">FBL ADL</a><ul> <li> <a href="/specs/advanced-data-layouts/fbl/spec/">spec</a></li></ul></li> <li> <a href="/specs/advanced-data-layouts/hamt/">HAMT ADL</a><ul> <li> <a href="/specs/advanced-data-layouts/hamt/fixture/">HashMap (HAMT) Test Fixtures</a><ul> <li> <a href="/specs/advanced-data-layouts/hamt/fixture/alice-words/">alice-words</a></li></ul></li> <li> <a href="/specs/advanced-data-layouts/hamt/spec/">spec</a></li></ul></li></ul></li> <li> <a href="/specs/schemas/">Schemas</a><ul> <li> <a href="/specs/schemas/prelude/">prelude</a></li></ul></li> <li> <a href="/specs/transport/">Transports</a><ul> <li> <a href="/specs/transport/car/">CAR</a><ul> <li> <a href="/specs/transport/car/carv1/">CARv1 Specification</a></li> <li> <a href="/specs/transport/car/carv2/">CARv2 Specification</a></li> <li> <a href="/specs/transport/car/fixture/">CAR Test Fixtures</a><ul> <li> <a href="/specs/transport/car/fixture/carv1-basic/">carv1-basic</a></li> <li> <a href="/specs/transport/car/fixture/carv2-basic/">carv2-basic</a></li></ul></li></ul></li> <li> <a href="/specs/transport/graphsync/">Graphsync</a><ul> <li> <a href="/specs/transport/graphsync/known_extensions/">known_extensions</a></li></ul></li> <li> <a href="/specs/transport/trustless-pathing/">Trustless Pathing</a><ul> <li> <a href="/specs/transport/trustless-pathing/fixtures/">Trustless Pathing Fixtures</a><ul> <li> <a href="/specs/transport/trustless-pathing/fixtures/unixfs_20m_variety/">unixfs_20m_variety</a></li></ul></li></ul></li></ul></li> <li> <a href="/specs/selectors/">Selectors</a><ul> <li> <a href="/specs/selectors/fixtures/">fixtures</a><ul> <li> <a href="/specs/selectors/fixtures/selector-fixtures-1/">selector-fixtures-1</a></li> <li> <a href="/specs/selectors/fixtures/selector-fixtures-adl/">selector-fixtures-adl</a></li> <li> <a href="/specs/selectors/fixtures/selector-fixtures-recursion/">selector-fixtures-recursion</a></li></ul></li></ul></li> <li> <a href="/specs/patch/">Patch</a><ul> <li> <a href="/specs/patch/fixtures/">IPLD Patch Test Fixtures</a><ul> <li> <a href="/specs/patch/fixtures/fixtures-1/">fixtures-1</a></li></ul></li></ul></li></ul></li> <li> <a href="/libraries/">Libraries</a><ul> <li> <a href="/libraries/golang/">Golang</a></li> <li> <a href="/libraries/javascript/">JavaScript</a></li> <li> <a href="/libraries/python/">Python</a></li> <li> <a href="/libraries/rust/">Rust</a></li></ul></li> <li> <a href="/design/">Design</a><ul> <li> <a href="/design/objectives/">Objectives</a></li> <li> <a href="/design/concepts/">Concepts</a><ul> <li> <a href="/design/concepts/type-theory-glossary/">type-theory-glossary</a></li></ul></li> <li> <a href="/design/libraries/">Libraries</a><ul> <li> <a href="/design/libraries/nodes-and-kinds/">nodes-and-kinds</a></li></ul></li> <li> <a href="/design/tricky-choices/">Tricky Choices</a><ul> <li> <a href="/design/tricky-choices/dag-pb-forms-impl-and-use/">dag-pb-forms-impl-and-use</a></li> <li> <a href="/design/tricky-choices/map-key-domain/">map-key-domain</a></li> <li> <a href="/design/tricky-choices/numeric-domain/">numeric-domain</a></li> <li> <a href="/design/tricky-choices/ordering/">ordering</a></li> <li> <a href="/design/tricky-choices/string-domain/">string-domain</a></li></ul></li> <li> <a href="/design/open-research/">Open Research</a><ul> <li> <a href="/design/open-research/ADL-autoexecution/">ADL autoexecution</a></li></ul></li></ul></li> <li> <a href="/tools/">Tools</a></li> <li class="active-page"> <a href="/glossary/">Glossary</a></li> <li> <a href="/media/">Media</a></li> <li> <a href="/FAQ/">FAQ</a></li></ul> </nav> </aside> <main> <div class=content> <h1>Glossary</h1> <h4 id="adl" tabindex="-1"><a class="header-anchor" href="#adl">ADL</a></h4> <p>ADL is short for <strong>A</strong>dvanced <strong>D</strong>ata <strong>L</strong>ayout. See <a href="/docs/advanced-data-layouts/">docs for Advanced Data Layouts</a>.</p> <p>In brief: ADLs a concept in IPLD which describes when pluggable code is used to create data structures which act like typical <a href="#data-model">Data Model</a> <a href="#node">Nodes</a> (meaning regular generic code can work over them just like over plain <a href="#node">Nodes</a>), while having internal structure which is managed by the plugin code.</p> <p>ADLs are typically used to do things like create large sharded data structures which can be accessed in the same way as simpler structures, but the definition is fairly open-ended.</p> <h4 id="block" tabindex="-1"><a class="header-anchor" href="#block">Block</a></h4> <p>The term "block" refers to to a chunk of serialized binary data.</p> <p>Most users don't work directly with blocks. Instead, block binary data is encoded and decoded to and from the IPLD <a href="#data-model">Data Model</a> using a <a href="#codec">codec</a>, and users work with the data via the <a href="#data-model">Data Model</a>. It's only if writing a storage system or a data transport system that you will be likely to work directly with blocks.</p> <h4 id="cid" tabindex="-1"><a class="header-anchor" href="#cid">CID</a></h4> <p>CID stands for <strong>C</strong>ontent <strong>ID</strong>entifier. It's a self-describing data structure identifier. In other words, it's a hash that says what kind of hash it is and how to decode the binary data identified by the hash.</p> <p>See the <a href="https://github.com/multiformats/cid">CID specification</a> for further details.</p> <p>CIDs are for <a href="#content-addressing">Content Addressing</a>.</p> <h4 id="codec" tabindex="-1"><a class="header-anchor" href="#codec">Codec</a></h4> <p>The term "codec" refers to a function which decodes serial data into the <a href="#data-model">Data Model</a>, and encodes <a href="#data-model">Data Model</a> content into serial data.</p> <p>Sometimes we refer to the serialized data from one <a href="#data-model">Data Model</a> tree (i.e., not including crossing any <a href="#link">links</a>) as forming a block "<a href="#block">block</a>".</p> <h4 id="content-addressing" tabindex="-1"><a class="header-anchor" href="#content-addressing">Content Addressing</a></h4> <p>"Content addressability" refers to the ability to refer to content by a trustless identifier.</p> <p>Rather than referring to content by an imprecise name or a location-oriented concept like URL, content addressable systems refer to content by a cryptographic hash of the content itself. This allows complete decentralization of the content, as the identifier does not specify the retrieval method nor locations, and does provides a secure way to verify the content (regardless of wherever it may be found from).</p> <p><a href="https://proto.school/content-addressing">See more about Content-Addressing on ProtoSchool</a>!</p> <h4 id="dag" tabindex="-1"><a class="header-anchor" href="#dag">DAG</a></h4> <p>DAG is short for <strong>D</strong>irected <strong>A</strong>cyclic <strong>G</strong>raph. It describes some data where more than one path can lead to the same point, but there are only a finite number of paths, because the data does not include cycles.</p> <p>We talk about DAGs often in IPLD, because IPLD data is naturally a DAG. <a href="#link">Links</a> mean we can connect graphs of data, but because <a href="#link">links</a> are <a href="#content-addressing">content-addressable</a>, they can't form cycles.</p> <p>Read more about <a href="https://en.wikipedia.org/wiki/Directed_acyclic_graph">DAGs on Wikipedia</a>.</p> <h4 id="data-model" tabindex="-1"><a class="header-anchor" href="#data-model">Data Model</a></h4> <p>The IPLD Data Model describes common base types that we call <strong>kinds</strong>. ("types" is a term that we prefer to reserve for data structures described by IPLD Schemas.)</p> <p>These <em>kinds</em> allow IPLD to create data structures using simple types accessible across many programming languages and encoding formats.</p> <p>Using the Data Model we can implement file systems, databases, and custom application data structures in a format agnostic way and even link between these structures and formats using common toolchains.</p> <p>The Data Model kinds are:</p> <ul> <li><code>boolean</code></li> <li><code>integer</code></li> <li><code>float</code></li> <li><code>map</code></li> <li><code>list</code></li> <li><code>string</code></li> <li><code>null</code></li> <li><code>bytes</code></li> <li><code>link</code></li> </ul> <p>(You may notice the Data Model kinds are essentially what you're familiar with from JSON -- we've just added <code>bytes</code> for binary, and this <code>link</code> kind, which gives IPLD a lot of its magic.)</p> <p>There is a <code>link</code> kind is implemented by <a href="#cid">CID</a>s.</p> <h4 id="dmt" tabindex="-1"><a class="header-anchor" href="#dmt">DMT</a></h4> <p>DMT is short for "<strong>D</strong>ata <strong>M</strong>odel <strong>T</strong>ree". It is a term we coined in IPLD to describe data that's -- well -- in the <a href="#data-model">Data Model</a> form.</p> <p>The term "DMT" is usually only used when necessary for differentiation: for example, we might say: "this data isn't in JSON form anymore; we've parsed it into DMT form". Another example sentence might be: "this data was created with a library DSL, but really, its true form is a standard DMT". And so on.</p> <p>The concept of DMT could also be compared to the computer science concept of an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a>, but again, sometimes a unique term is useful for disambiguation. For example: "the DSL AST is somewhat richer than the DMT; the DMT only describes the logical elements of the document rather than the whole syntax used to specify them". (We also find the term "AST" somewhat of a bad match for what we mean by DMTs in IPLD, because an IPLD DMT is explicitly codec agnostic (in other words, syntax agnostic!), which doesn't line up well with the "S" in "AST".)</p> <h4 id="kind" tabindex="-1"><a class="header-anchor" href="#kind">Kind</a></h4> <p>In IPLD, we use the word "kind" to refer to one of the handful of basic sorts of data we can recognize and know how to operate on. For example, "string" and "boolean" and "map" are all examples of kinds.</p> <p>Depending on context, it might be referring to <a href="https://ipld.io/docs/data-model/kinds/">data model kind</a>, or to <a href="https://ipld.io/docs/schemas/features/typekinds/">typekinds</a>.</p> <p>All data in IPLD has a <a href="https://ipld.io/docs/data-model/kinds/">data model kind</a>. Data might also have a <a href="https://ipld.io/docs/schemas/features/typekinds/">typekind</a>, but only if it was processed with IPLD Schemas.</p> <h4 id="link" tabindex="-1"><a class="header-anchor" href="#link">Link</a></h4> <p>A link is just another name for a <a href="#cid">CID</a>. When we talk about linking in IPLD, we always mean this -- linking in IPLD is always immutable, and uses hashes, and therefore when we talk about linking, we always mean <a href="#cid">CID</a>s.</p> <h4 id="node" tabindex="-1"><a class="header-anchor" href="#node">Node</a></h4> <p>A "node" in IPLD is a point in a graph -- an element of the Data Model in an instantiated data structure. Every node has a "kind" property, which is one of the <a href="#data-model">Data Model kinds</a>.</p> <p>If a node is a <code>map</code> or <code>list</code> <em>kind</em>, then it will have children. The other node <em>kinds</em>, like <code>string</code>, are just values (they have no children).</p> <p>A "<a href="#block">block</a>" will typically contain many nodes.</p> <h4 id="reify" tabindex="-1"><a class="header-anchor" href="#reify">Reify</a></h4> <p>To "reify" an IPLD <a href="#node">"node"</a> is to attempt to wrap over it with an <a href="#adl">ADL</a> and return a <code>Node</code> which can be used in the rest of the model. For example, you can reify an IPLD node which represents a HAMT into a regular node so you can get keys from it without needing to manually traverse the nodes making up the HAMT within your application. Reifying is most useful when you have already parsed data into a raw tree and want to wrap it in an ADL.</p> <h4 id="schemas" tabindex="-1"><a class="header-anchor" href="#schemas">Schemas</a></h4> <p>IPLD Schemas are a system for describing data with structural types. That means Schemas can describe data, and the data either matches, or, doesn't -- and if it doesn't, you can just try a different schema.</p> <p>Schemas are a high-level feature in IPLD. You can apply them on top of data that's already legible as <a href="#data-model">Data Model</a> content. That means IPLD Schemas are agnostic of <a href="#codec">codecs</a>. It also means they're entirely optional -- you can parse data with or without them -- and you can use Schemas to describe and help process data even if that data <em>predates the Schema</em>.</p> <h4 id="selectors" tabindex="-1"><a class="header-anchor" href="#selectors">Selectors</a></h4> <p>IPLD Selectors are a form of graph query (or, a way to specify a traversal, if you prefer that mental model) over IPLD data.</p> <p>Selectors are a declarative format for specifying a walk over a <a href="#data-model">Data Model</a> graph. Selectors can specify which nodes to walk over (or not), which order to visit in, and it can mark certain positions as "matched" (in addition to just visited). You can think of Selectors as being roughly like "regexps for graphs". Libraries may yield iterators over matched nodes, or iterators over all visited nodes, or callback oriented interfaces.</p> <p>Selectors are natively implemented in most IPLD libraries (for performance reasons), but the format itself is standardized. The format is described in IPLD (using <a href="#schemas">IPLD Schemas</a>), so it's possible to serialize Selectors in any <a href="#codec">Codec</a> you want, and it's also possible to inspect (and transform!) Selector documents using standard <a href="#data-model">Data Model</a> tools.</p> <h4 id="signalling" tabindex="-1"><a class="header-anchor" href="#signalling">Signalling</a></h4> <p>See the <a href="/docs/advanced-data-layouts/signalling/">ADL Signalling</a> page, and especially the <a href="/docs/advanced-data-layouts/signalling/#signalling-mechanisms">Signalling Mechanisms</a>.</p> <p>The concept of signalling could be more generalized, as well (multicodec indicators can be seen as a signalling mechanism for codec selection)!</p> <h4 id="substrate" tabindex="-1"><a class="header-anchor" href="#substrate">Substrate</a></h4> <p>"Substrate" is a vocabulary term relating to <a href="#adl">ADLs</a> -- it refers to the data "inside" them, as contrasted with the "synthesized view" of the data, which is the node that the ADL presents itself as. (The substrate may also sometimes colloquially be called "content data" or "encoded form" or other terms.)</p> <p>For example, in a HAMT, the forest of <a href="#node">nodes</a> and <a href="#block">blocks</a> that make up the sharding structure are all considered the "substrate" data; while the map node the HAMT presents as is the "synthesized view".</p> <h4 id="traversal" tabindex="-1"><a class="header-anchor" href="#traversal">Traversal</a></h4> <p>Traversal is the act of walking across the <a href="#data-model">Data Model</a>.</p> <p>It is useful to consider the Data Model as being formed of "scalar" and "recursive" kinds when considering nodes and possible traversals.</p> <p>"Scalar" kinds are terminal nodes in the Data Model: null, boolean, integer, float, string, bytes</p> <p>"Recursive" kinds can contain other kinds within them and therefore allow deeper traversal: map and list.</p> <p>The link kind is scalar, but is typically treated as a transparent node for the purpose of traversal such that data spanning many blocks can be addressed as a single graph of nodes (so, sometimes, contextually, it can be seen as a sort of a "recursive" kind).</p> </div> </main> </body> </html>