evestera/json_typegen

json_typegen - Rust types from JSON samples

json_typegen is a collection of tools for generating types from

json_typegen is a collection of tools for generating types from JSON samples for Rust, Kotlin and TypeScript. I.e. you give it some JSON, and it gives you the type definitions necessary to use that JSON in a program.

There are three interfaces to this code generation logic:

  • Rust procedural macro
  • Command line interface
  • Web interface

Procedural macro

In Rust the code generation can be used straight from the program you are making, with a procedural macro.

For those familiar with F#, the procedural macro json_typegen! works as a type provider for JSON in Rust. It was inspired by and uses the same kind of inference algorithm as F# Data.

As an example, the below code generates the type Point based on an inline sample.

use json_typegen::json_typegen;

json_typegen!("Point", r#"{ "x": 1, "y": 2 }"#);

fn main() {
    let mut p: Point = serde_json::from_str(r#"{ "x": 3, "y": 5 }"#).unwrap();
    println!("deserialized = {:?}", p);
    p.x = 4;
    let serialized = serde_json::to_string(&p).unwrap();
    println!("serialized = {}", serialized);
}

The following crate dependencies are necessary for this example to work:

[dependencies]
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
json_typegen = "0.6"

The sample json can also come from local or remote files:

json_typegen!("Point", "json_samples/point.json");
json_typegen!("Point", "http://example.com/someapi/point.json");

The code generation can also be customized:

json_typegen!("Point", "http://example.com/someapi/point.json", {
    use_default_for_missing_fields,
    "/foo/bar": {
        use_type: "map"
    }
});

For the details on configuration, see the relevant documentation.

Conditional compilation

To avoid doing a HTTP request per sample used for every build you can use conditional compilation to only check against remote samples when desired:

#[cfg(not(feature = "online-samples"))]
json_typegen!("pub Point", r#"{ "x": 1, "y": 2 }"#);
#[cfg(feature = "online-samples")]
json_typegen!("pub Point", "https://typegen.vestera.as/examples/point.json");

And in Cargo.toml:

[features]
online-samples = []

You can then verify that remote samples match your expectations in e.g. CI builds as follows:

cargo check --features "online-samples"

Command line interface

The crate json_typegen_cli provides a CLI to the same code generation as the procedural macro uses internally. This provides a useful migration path if you at some point need to customize the generated code beyond what is practical through macro arguments.

For details on installation and usage see its readme.

Web interface

For simple testing and one-time use there is also a WebAssembly-powered web interface hosted at https://typegen.vestera.as/. Source code in json_typegen_web.

Creating your own type provider crate

Both procedural macros and the shape inference algorithm are actually very simple. To learn/copy the algorithm you can look at this stripped-down version(< 200 lines).

License

This project is dual licensed, under either the Apache 2.0 or the MIT license, at your option.

Issues

Collection of the latest Issues

evestera

evestera

Comment Icon0

When reading sample from file using the procedural macro (e.g. json_typegen!("Point", "point.json")), if the sample file changes the compiler will not be aware that anything relevant has changed and so will not recompile.

There is a hack we can do to let the compiler know we are using a file. Output code like this in addition to the actual code:

Tracking issue for better way to do this (from which the snippet above was taken): https://github.com/rust-lang/rust/issues/73921

evestera

evestera

Comment Icon0

Just add a minimal Gradle project to the CI folder, use the CLI to generate a Kotlin file that is written to that folder and run ./gradlew test testing roundtripping and such.

evestera

evestera

Comment Icon0

E.g.

or

For typescript/typealias this is already possible with just adding <T> to the type name and using

evestera

evestera

Comment Icon0

Currently tuples are only used when all source arrays have the same length and combining the shapes of the array would lose type information. E.g. you can end up with [["a", 1], ["b", 2]] is inferred to Array<[string, number]>, but [[1, 2], [3, 4]] is inferred as number[][].

We already have the hint use_type: "map". We should similarly have use_type: "tuple" and with that make it possible to end up with types like [string, string] and [number, number, number?].

Versions

Find the latest versions by id

v0.7.0 - Sep 16, 2021

New features

  • Output imports/use statements for Rust and Kotlin
  • Collect additional/optional properties (Rust and Kotlin/Jackson)
  • Various improvements to the web UI
  • Various changes to better support use of json_typegen_shared as a library

Fixes

  • Kotlin:
    • Don't output empty data classes for empty objects
    • Only restrict use of hard keywords (use less rename annotations)
    • Output Any? instead of Any to include null for values of unknown type
    • Use correct Kotlin type alias syntax

Breaking changes

  • Option parsing is now an optional feature. This is a breaking change for anyone using default-features = false and option parsing. Enable by adding "option-parsing" to the list of enabled features.
  • Some Kotlin fields that would previously be renamed because they are soft keywords are no longer renamed.

v0.6.0 - Mar 14, 2021

New features

  • New output format: Kotlin w/ kotlinx.serialization
  • New output format: Typescript with a single nested type alias
  • New option unwrap, for "unwrapping" wrapped JSON nodes before generating types. Combined with inference hints specifying an opaque type this allows creating types for wrappers and actual content separately
  • Lots of improvements to the web interface (typegen.vestera.as)
    • Button to download output as file
    • Button to load input from file
    • Code generation moved to out of UI thread using Web Worker API. Combined with loading from file, the web interface should now be much more useable with large files. For very large files fixing #15 will still be needed.
    • Input form is now stored in localStorage
    • You can now use the mentioned unwrap option and inference hint blocks in the web interface
    • Several other general improvements...

Fixes

  • Kotlin: Cleaner output. JsonNaming was using Java syntax, should be correct now.
  • JSON Schema: Output is now targeting a specific version (draft-07)
  • Fixed a bug that caused wildcards for targeting inference hints to not be applied to arrays correctly.

v0.5.0 - Dec 25, 2020

There will now always be a type (or type alias) generated with the requested name. This used to not always be the case. While it is an obvious choice in hindsight, making this correction is unfortunately a potentially breaking change:

Breaking changes 💥

  • Requesting the name Foo for a list of objects would previously use the name Foo for the type of the elements. It will now use Foo for the list (the root type). If the codegeneration is able to "singularize" the name (e.g. "Points" to "Point") the object type name will be the same as before ("Point") and the plural will be used for the list.

Crates:

v0.4.1 - Sep 08, 2020

Changes:

  • Only some necessary updates to dependencies.

The lack of updates to this project is partly due to the fact that I'm working on a complete rewrite to fix some of the larger issues related to how the goals of the project has changed over time. The new version is a lot better with regards to performance on large files and WASM bundle size, but there is still some work left to do before it is ready for release. 🤷‍♂️

Crates:

v0.4.0 - Nov 12, 2019

Changes:

  • Better defaults for generated code. data class for Kotlin, export interface for TS, pub struct for Rust
  • Add support for specifying JSON property name format (option property_name_format) to get nicer code with rename_all with Rust/Serde and PropertyNamingStrategy for Kotlin/Jackson. Try it out in the web interface.

As before, precompiled binaries attached below!

Crates:

v0.3.3 - Jul 24, 2019

Changes:

  • A bugfix for Typescript generation (#10)
  • There is a shiny new wasm-based web interface at typegen.vestera.as

As before, precompiled binaries attached below!

Crates:

v0.3.2 - Feb 15, 2019

Changes:

  • Precompiled binaries! Download them below.
  • Bugfix for Kotlin generation
  • Output now includes trailing newline

Crates:

v0.3.1 - Feb 04, 2019

Bugfix release. Rust code generation had to be made aware of new Rust 2018 keywords: async, await and try.

Crates:

v0.3.0 - Feb 04, 2019

Highlights:

  • Additional output formats (Kotlin, Typescript, JSON Schema)
  • Rust 2018 (simpler importing and derive crate no longer necessary)

v0.2.0 - Jul 23, 2018

Long overdue release, so quite a few changes. In particular, have a look at CONFIGURATION.md.

Coming up: Multiple output formats!

v0.1.0 - Apr 25, 2017

Information - Updated Jun 20, 2022

Stars: 140
Forks: 10
Issues: 6

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically

Rust Greatest JSON weapon is Serde with over 4.4K stars on github and a massive developer community. This is considered a core Rust library for every developer to learn in BRC's opinion

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically

Rust 버전 JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

Rust 버전 JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

SIMD JSON for Rust  

Rust port of extremely fast serde compatibility

SIMD JSON for Rust  

JSON-E Rust data-struct paramter crate for lightweight embedded content with objects and much more

What makes JSON-e unique is that it extensive documentation and ease of use

JSON-E Rust data-struct paramter crate for lightweight embedded content with objects and much more
JSON

111

A Rust JSON5 serializer and deserializer which speaks Serde

Deserialize a JSON5 string with from_str

A Rust JSON5 serializer and deserializer which speaks Serde

Rust JSON Parser Benchmark

Download and Generate JSON Data

Rust JSON Parser Benchmark

Read JSON values quickly - Rust JSON Parser

AJSON get json value with specified path, such as project

Read JSON values quickly - Rust JSON Parser

Rust actix json request example

Send a json request to actix, and parse it

Rust actix json request example

Rust JSON parsing benchmarks

This project aims to provide benchmarks to show how various JSON-parsing libraries in the Rust programming language perform at various JSON-parsing tasks

Rust JSON parsing benchmarks

A tiny command line tool written in rust to print json data as a formatted...

A tiny command line tool written in rust to print json data as a formatted table

A tiny command line tool written in rust to print json data as a formatted...

A Rust implementation of the JSON-RPC 2

A Rust implementation of the serde) for JSON-RPC

A Rust implementation of the JSON-RPC 2
Facebook Instagram Twitter GitHub Dribbble
Privacy