CINXE.COM

<!doctype html> <html lang="en"> <head> <meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <link rel="stylesheet" href="/css/layout.css?1738557326388"> <link rel="stylesheet" href="/css/nav.css?1738557326388"> <link rel="stylesheet" href="/css/style.css?1738557326388"> <link rel="stylesheet" href="/css/prismjs@1.24-themes-prism.css"> <title>IPLD ♦ Schema Specs</title> </head> <body> <header> <div class="sidebar-button" onclick="document.getElementById('sidebar').classList.toggle('sidebar-open')"> <svg xmlns="http://www.w3.org/2000/svg" aria-hidden="true" role="img" viewBox="0 0 448 512" class="icon"> <path fill="currentColor" d="M436 124H12c-6.627 0-12-5.373-12-12V80c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12zm0 160H12c-6.627 0-12-5.373-12-12v-32c0-6.627 5.373-12 12-12h424c6.627 0 12 5.373 12 12v32c0 6.627-5.373 12-12 12z"></path> </svg> </div> <a href="/" class="logo">IPLD</a> <aside id=breadcrumbs> <ul> <li><a href="/specs">specs</a></li> <li><a href="/specs/schemas/">schemas</a></li> </ul> </aside> </header> <aside id=sidebar> <nav> <ul> <li> <a href="/docs/">Docs</a><ul> <li> <a href="/docs/intro/">Intro</a><ul> <li> <a href="/docs/intro/hello-world/">Hello, World</a></li> <li> <a href="/docs/intro/primer/">The Brief Primer</a></li> <li> <a href="/docs/intro/ecosystem/">InterPlanetary Ecosystem Overview</a></li> <li> <a href="/docs/intro/community/">Finding Community</a></li></ul></li> <li> <a href="/docs/motivation/">Motivation</a><ul> <li> <a href="/docs/motivation/benefits-of-content-addressing/">Benefits of Content Addressing</a></li> <li> <a href="/docs/motivation/data-to-data-structures/">From Data to Data Structures</a></li></ul></li> <li> <a href="/docs/codecs/">Codecs</a><ul> <li> <a href="/docs/codecs/known/">Known Codecs</a><ul> <li> <a href="/docs/codecs/known/dag-cbor/">DAG-CBOR</a></li> <li> <a href="/docs/codecs/known/dag-json/">DAG-JSON</a></li> <li> <a href="/docs/codecs/known/dag-pb/">DAG-PB</a></li></ul></li></ul></li> <li> <a href="/docs/data-model/">Data Model</a><ul> <li> <a href="/docs/data-model/node/">Nodes</a></li> <li> <a href="/docs/data-model/kinds/">Kinds</a></li> <li> <a href="/docs/data-model/pathing/">Pathing</a></li> <li> <a href="/docs/data-model/traversal/">Traversal</a></li></ul></li> <li> <a href="/docs/advanced-data-layouts/">Advanced Data Layouts</a><ul> <li> <a href="/docs/advanced-data-layouts/intro/">Intro to ADLs</a></li> <li> <a href="/docs/advanced-data-layouts/naming/">ADL Naming</a></li> <li> <a href="/docs/advanced-data-layouts/signalling/">Signalling ADLs</a></li> <li> <a href="/docs/advanced-data-layouts/dynamic-loading/">Dynamic Loading</a></li> <li> <a href="/docs/advanced-data-layouts/known/">Known ADLs</a></li></ul></li> <li> <a href="/docs/schemas/">Schemas</a><ul> <li> <a href="/docs/schemas/intro/">Introduction</a><ul> <li> <a href="/docs/schemas/intro/compare/">compare</a></li> <li> <a href="/docs/schemas/intro/goals/">Goals</a></li> <li> <a href="/docs/schemas/intro/feature-summary/">Feature Summary</a></li></ul></li> <li> <a href="/docs/schemas/features/">Features</a><ul> <li> <a href="/docs/schemas/features/typekinds/">Type Kinds</a></li> <li> <a href="/docs/schemas/features/representation-strategies/">Representation Strategies</a></li> <li> <a href="/docs/schemas/features/links/">Links</a></li> <li> <a href="/docs/schemas/features/indicating-adls/">Using ADLs in Schemas</a></li></ul></li> <li> <a href="/docs/schemas/using/">Using Wisely</a><ul> <li> <a href="/docs/schemas/using/authoring-guide/">Authoring Guide</a></li> <li> <a href="/docs/schemas/using/migrations/">Migrations</a></li></ul></li></ul></li> <li> <a href="/docs/synthesis/">Synthesis</a><ul> <li> <a href="/docs/synthesis/gtd/">Getting Things Done</a></li> <li> <a href="/docs/synthesis/building-in-alignment/">Building in Alignment</a></li> <li> <a href="/docs/synthesis/how-ipfs-web-gateways-work/">How IPFS Web Gateways Work</a></li> <li> <a href="/docs/synthesis/encryption/">Working With Encryption</a></li></ul></li></ul></li> <li> <a href="/specs/">Specs</a><ul> <li> <a href="/specs/about/">About the Specifications</a></li> <li> <a href="/specs/codecs/">Codecs</a><ul> <li> <a href="/specs/codecs/dag-cbor/">DAG-CBOR</a><ul> <li> <a href="/specs/codecs/dag-cbor/spec/">Spec</a></li> <li> <a href="/specs/codecs/dag-cbor/fixtures/">DAG-CBOR Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-cbor/fixtures/cross-codec/">cross-codec</a></li></ul></li></ul></li> <li> <a href="/specs/codecs/dag-cosmos/">DAG-COSMOS</a><ul> <li> <a href="/specs/codecs/dag-cosmos/basic_types/">basic_types</a></li> <li> <a href="/specs/codecs/dag-cosmos/cosmos_state/">cosmos_state</a></li> <li> <a href="/specs/codecs/dag-cosmos/crypto_types/">crypto_types</a></li> <li> <a href="/specs/codecs/dag-cosmos/tendermint_chain/">tendermint_chain</a></li> <li> <a href="/specs/codecs/dag-cosmos/typed_protobuf/">typed_protobuf</a></li></ul></li> <li> <a href="/specs/codecs/dag-eth/">DAG-ETH</a><ul> <li> <a href="/specs/codecs/dag-eth/basic_types/">basic_types</a></li> <li> <a href="/specs/codecs/dag-eth/chain/">chain</a></li> <li> <a href="/specs/codecs/dag-eth/convenience_types/">convenience_types</a></li> <li> <a href="/specs/codecs/dag-eth/state/">state</a></li></ul></li> <li> <a href="/specs/codecs/dag-jose/">DAG-JOSE</a><ul> <li> <a href="/specs/codecs/dag-jose/spec/">Spec</a></li> <li> <a href="/specs/codecs/dag-jose/fixtures/">fixtures</a></li></ul></li> <li> <a href="/specs/codecs/dag-json/">DAG-JSON</a><ul> <li> <a href="/specs/codecs/dag-json/spec/">Spec</a></li> <li> <a href="/specs/codecs/dag-json/fixtures/">DAG-JSON Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-json/fixtures/cross-codec/">cross-codec</a></li></ul></li></ul></li> <li> <a href="/specs/codecs/dag-pb/">DAG-PB</a><ul> <li> <a href="/specs/codecs/dag-pb/spec/">Spec</a></li> <li> <a href="/specs/codecs/dag-pb/fixtures/">DAG-PB Test Fixtures</a><ul> <li> <a href="/specs/codecs/dag-pb/fixtures/cross-codec/">cross-codec</a></li></ul></li></ul></li></ul></li> <li> <a href="/specs/advanced-data-layouts/">Advanced Data Layouts</a><ul> <li> <a href="/specs/advanced-data-layouts/fbl/">FBL ADL</a><ul> <li> <a href="/specs/advanced-data-layouts/fbl/spec/">spec</a></li></ul></li> <li> <a href="/specs/advanced-data-layouts/hamt/">HAMT ADL</a><ul> <li> <a href="/specs/advanced-data-layouts/hamt/spec/">spec</a></li> <li> <a href="/specs/advanced-data-layouts/hamt/fixture/">HashMap (HAMT) Test Fixtures</a><ul> <li> <a href="/specs/advanced-data-layouts/hamt/fixture/alice-words/">alice-words</a></li></ul></li></ul></li></ul></li> <li class="active-page"> <a href="/specs/schemas/">Schemas</a><ul> <li> <a href="/specs/schemas/prelude/">prelude</a></li></ul></li> <li> <a href="/specs/transport/">Transports</a><ul> <li> <a href="/specs/transport/car/">CAR</a><ul> <li> <a href="/specs/transport/car/carv1/">CARv1 Specification</a></li> <li> <a href="/specs/transport/car/carv2/">CARv2 Specification</a></li> <li> <a href="/specs/transport/car/fixture/">CAR Test Fixtures</a><ul> <li> <a href="/specs/transport/car/fixture/carv1-basic/">carv1-basic</a></li> <li> <a href="/specs/transport/car/fixture/carv2-basic/">carv2-basic</a></li></ul></li></ul></li> <li> <a href="/specs/transport/graphsync/">Graphsync</a><ul> <li> <a href="/specs/transport/graphsync/known_extensions/">known_extensions</a></li></ul></li> <li> <a href="/specs/transport/trustless-pathing/">Trustless Pathing</a><ul> <li> <a href="/specs/transport/trustless-pathing/fixtures/">Trustless Pathing Fixtures</a><ul> <li> <a href="/specs/transport/trustless-pathing/fixtures/unixfs_20m_variety/">unixfs_20m_variety</a></li></ul></li></ul></li></ul></li> <li> <a href="/specs/selectors/">Selectors</a><ul> <li> <a href="/specs/selectors/fixtures/">fixtures</a><ul> <li> <a href="/specs/selectors/fixtures/selector-fixtures-1/">selector-fixtures-1</a></li> <li> <a href="/specs/selectors/fixtures/selector-fixtures-adl/">selector-fixtures-adl</a></li> <li> <a href="/specs/selectors/fixtures/selector-fixtures-recursion/">selector-fixtures-recursion</a></li></ul></li></ul></li> <li> <a href="/specs/patch/">Patch</a><ul> <li> <a href="/specs/patch/fixtures/">IPLD Patch Test Fixtures</a><ul> <li> <a href="/specs/patch/fixtures/fixtures-1/">fixtures-1</a></li></ul></li></ul></li></ul></li> <li> <a href="/libraries/">Libraries</a><ul> <li> <a href="/libraries/golang/">Golang</a></li> <li> <a href="/libraries/java/">Java</a></li> <li> <a href="/libraries/javascript/">JavaScript</a></li> <li> <a href="/libraries/python/">Python</a></li> <li> <a href="/libraries/rust/">Rust</a></li></ul></li> <li> <a href="/design/">Design</a><ul> <li> <a href="/design/objectives/">Objectives</a></li> <li> <a href="/design/concepts/">Concepts</a><ul> <li> <a href="/design/concepts/type-theory-glossary/">type-theory-glossary</a></li></ul></li> <li> <a href="/design/libraries/">Libraries</a><ul> <li> <a href="/design/libraries/nodes-and-kinds/">nodes-and-kinds</a></li></ul></li> <li> <a href="/design/tricky-choices/">Tricky Choices</a><ul> <li> <a href="/design/tricky-choices/dag-pb-forms-impl-and-use/">dag-pb-forms-impl-and-use</a></li> <li> <a href="/design/tricky-choices/map-key-domain/">map-key-domain</a></li> <li> <a href="/design/tricky-choices/numeric-domain/">numeric-domain</a></li> <li> <a href="/design/tricky-choices/ordering/">ordering</a></li> <li> <a href="/design/tricky-choices/string-domain/">string-domain</a></li></ul></li> <li> <a href="/design/open-research/">Open Research</a><ul> <li> <a href="/design/open-research/ADL-autoexecution/">ADL autoexecution</a></li></ul></li></ul></li> <li> <a href="/tools/">Tools</a></li> <li> <a href="/glossary/">Glossary</a></li> <li> <a href="/media/">Media</a></li> <li> <a href="/FAQ/">FAQ</a></li></ul> </nav> </aside> <main> <div class=content> <h1>IPLD Schema Specifications</h1> <p>Here are collected the specifications and fixtures for IPLD Schemas.</p> <div class="callout callout-info"> <p>This is a <em>specs</em> chapter, rather than a <em>docs</em> chapter. It's geared for in-depth details.</p> <p>For more introductory and human-friendly material on Schemas -- what they are, what they do, and how to use them -- see <a href="/docs/schemas/">Schemas in the docs chapters</a> instead.)</p> </div> <h2 id="the-schema-schema" tabindex="-1"><a class="header-anchor" href="#the-schema-schema">The Schema-Schema</a></h2> <p>The IPLD Schema specification is self-hosting: it is primarily described by the "schema schema".</p> <p>The schema-schema can be seen in two forms: in DSL form, or as the equivalent parsed Schema DMT in JSON encoding:</p> <ul> <li><a href="./schema-schema.ipldsch">schema-schema.ipldsch</a></li> <li><a href="./schema-schema.ipldsch.json">schema-schema.ipldsch.json</a></li> </ul> <h3 id="dsl-vs-dmt" tabindex="-1"><a class="header-anchor" href="#dsl-vs-dmt">DSL vs DMT</a></h3> <p>We have something called the "DSL" and something called the "DMT". What's that stand for?</p> <ul> <li>"DSL" is short for "<a href="https://en.wikipedia.org/wiki/Domain-specific_language">Domain Specific Language</a>", a common term of art in computer science. We mean it in the usual way: the IPLD Schema DSL is a custom, fit-for-purpose syntax for describing IPLD Schemas.</li> <li>"DMT" is short for "<a href="/glossary/#dmt">Data Model Tree</a>", and that's a term we've coined in IPLD. The IPLD Schema DMT is the normalized essential form of IPLD Schemas, and use this as the interchange format and source of truth for reasoning about them.</li> </ul> <p>The IPLD Schema DSL can be compiled down to DMT form, and then passed around (typically, as JSON). (This makes it easy to build IPLD Schema tools and libraries in new languages, without having to necessarily build a DSL parser!)</p> <p>Note that the Schema DMT is not exactly an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> for the Schema DSL. The DMT is meant for mechanical, logical use; whereas the DSL is made for human readability and writability. As a result of those different priorities, they structure information in somewhat different ways. (The DMT is more redundant, because it insists on having type-level information be completely defined even if representation information were to be stripped, whereas the DSL AST avoids making the human repeat the same information twice for any reason; the DSL AST would probably see some representation information as adjacent to type information (e.g. next to struct fields, which is where syntactically the annotations go in that part of the DSL), whereas the DMT consistently specifies it all in one place under one object of representation information per type; etc.) As a user, you probably don't need to think about this, but as a library developer, keeping in mind that these are different may be a useful clarification.</p> <h3 id="the-prelude" tabindex="-1"><a class="header-anchor" href="#the-prelude">The Prelude</a></h3> <p>The IPLD Schema "prelude" is a series of type declarations that are implicitly present at the top of all schemas. It provides common types like <code>String</code> and <code>Map</code> and <code>List</code> as named types, without you needing to define them yourself.</p> <p><a href="./prelude/">More information about the Prelude</a></p> <h2 id="examples-and-fixtures" tabindex="-1"><a class="header-anchor" href="#examples-and-fixtures">Examples and Fixtures</a></h2> <ul> <li>Examples: <ul> <li>An example Schema, in DSL form, with matching parsed Schema DMT in JSON encoding: <ul> <li><a href="./examples.ipldsch">examples.ipldsch</a></li> <li><a href="./examples.ipldsch.json">examples.ipldsch.json</a></li> </ul> </li> </ul> </li> <li>Fixtures: <ul> <li>Aside from the schema-schema and examples above, more fixtures for this system are currently lacking. If you know of some and can link them here, or are willing to write some, please send an issue or PR.</li> </ul> </li> </ul> <h2 id="bootstrapping-a-schema-implementation" tabindex="-1"><a class="header-anchor" href="#bootstrapping-a-schema-implementation">Bootstrapping a Schema implementation</a></h2> <p>The schema-schema is intentionally designed to be relatively easy to parse. We chose to use features when defining the schema-schema that are the clearest to implement, and the most likely to be early selections for a high priority in implementing anyway; and we in some cases carefully avoided using features that would've been nice-to-have but technically nonessential.</p> <p>For example: the schema-schema only uses keyed and kinded unions, which are some of the most clear to implement.</p> <p>As another example: the schema schema in several places uses <code>type X struct {}</code> rather than using a <code>unit</code> typekind (a la <code>type X unit representation emptymap</code>). We chose to do this, even though use of a unit type would be more semantically explicit, because we figure that one less thing to implement before being able to successfully wrangle the schema-schema is probably going to be well-received by anyone implementing schemas in a new language.</p> <p>It's still probably wise to start with smaller goals than biting off the whole schema-schema at once as the first target for bootstrapping a new schema implementation in a new language. Still, we hope these choices in the design <em>help</em>.</p> </div> </main> </body> </html>