jonasbb/serde_with

Custom de/serialization functions for Rust's serde

Apply a prefix to each field name of a struct, without changing the de/serialize implementations of the struct using serde_with::rust::StringWithSeparator::<CommaSeparator>

Custom de/serialization functions for Rust's serde


This crate provides custom de/serialization helpers to use in combination with serde's with-annotation and with the improved serde_as-annotation. Some common use cases are:

  • De/Serializing a type using the Display and FromStr traits, e.g., for u8, url::Url, or mime::Mime. Check DisplayFromStr or serde_with::rust::display_fromstr for details.
  • Support for arrays larger than 32 elements or using const generics. With serde_as large arrays are supported, even if they are nested in other types. [bool; 64], Option<[u8; M]>, and Box<[[u8; 64]; N]> are all supported, as this examples shows.
  • Skip serializing all empty Option types with #[skip_serializing_none].
  • Apply a prefix to each field name of a struct, without changing the de/serialize implementations of the struct using with_prefix!.
  • Deserialize a comma separated list like #hash,#tags,#are,#great into a Vec<String>. Check the documentation for serde_with::rust::StringWithSeparator::<CommaSeparator>.

Getting Help

Check out the user guide to find out more tips and tricks about this crate.

For further help using this crate you can open a new discussion or ask on users.rust-lang.org. For bugs, please open a new issue on GitHub.

Use serde_with in your Project

Add this to your Cargo.toml:

The crate contains different features for integration with other common crates. Check the feature flags section for information about all available features.

Examples

Annotate your struct or enum to enable the custom de/serializer. The #[serde_as] attribute must be place before the #[derive].

DisplayFromStr

Large and const-generic arrays

serde does not support arrays with more than 32 elements or using const-generics. The serde_as attribute allows to circumvent this restriction, even for nested types and nested arrays.

skip_serializing_none

This situation often occurs with JSON, but other formats also support optional fields. If many fields are optional, putting the annotations on the structs can become tedious. The #[skip_serializing_none] attribute must be place before the #[derive].

Advanced serde_as usage

This example is mainly supposed to highlight the flexibility of the serde_as-annotation compared to serde's with-annotation. More details about serde_as can be found in the user guide.

License

Licensed under either of

  • Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
  • MIT license (LICENSE-MIT or #404)

at your option.

Contribution

For detailed contribution instructions please read CONTRIBUTING.md.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

Issues

Collection of the latest Issues

tobiasvl

tobiasvl

enhancement
Comment Icon1

Perhaps not such a common use case as stripping prefixes, but it would be nice if there was a symmetrical with_suffix macro that could strip suffixes.

CAD97

CAD97

enhancement
Comment Icon3

This is one way to e.g. successfully roundtrip things like Option<Option<u32>> into formats that have unwrap_or_null behavior by default (as opposed to e.g. double_option or unwrap_or_skip).

Example implementation:

Output, showing behavior (manually reformatted):

jonasbb

jonasbb

enhancement
Comment Icon0

https://github.com/rust-lang/rust/blob/afaf33dcafe9c7068b63eb997df221aa08db7c29/library/core/src/array/mod.rs#L563-L628

This is not a public function, so the code needs to be copied. This is very likely be better than the existing code. The function uses too many unstable features to be useful right now.

These issues track adding such a function to the standard library. https://github.com/rust-lang/rust/pull/69985 https://github.com/rust-lang/rust/issues/81615

core::array::try_from_fn: Currently blocked on try trait v2. See tracking issue https://github.com/rust-lang/rust/issues/89379

jonasbb

jonasbb

enhancement
Comment Icon0

As seen in #201 it might be confusing why this code doesn't work:

The reason is that a blanket implementation for DeserializeAs/SerializeAs is not possible as it would overlap with other implementations. It is possible to special case the impl SerializeAs<String> for String case. This has to be done for every type and would bless the standard library types compared to types from crates.

Another possibility might be to move this logic into the serde_as macro. If it detects two identical types it replaces the type in the as part with _/Same. In the map case the BTreeMap<..., ...> compares not equal, but the String subtype does and would be replaced. In the map2 case already the BTreeMap<..., ...> type is equal and it would convert it to as = "_".

This could technically change how the serialization is performed as it then uses Serialize instead of SerializeAs. So this behavior might require an opt-out which disables the type equality check. Another option might be to limit this to leaf types, i.e., ones without any generic parameters, since I assume it might be more likely there.

A problem with this approach is how to generalize this to different transformations, for example the BTreeMap<K, V> to Vec<(K, V)> transformation. The first one has two generic arguments while the Vec only has one (the tuple). So the fear here is that it simplifies it for the trivial cases but as soon as somebody want to use other transformations the help would fail and that might cause even more confusion.

jonasbb

jonasbb

enhancement
Comment Icon3

Serialization code is often repetitive. One helper to improve on this already exists in this crate, the serde_with::skip_serializing_none attribute.

However, this could be generalized to work with any attribute on a user-specified type (list). This could look like

This would apply the attribute on all fields of type Vec and HashMap. On the right hand side, the placeholder $TYPE would be replaced by the type of the field. This would make it easier to skip serializing all empty container types.

serde_with::skip_serializing_none could in the end be implemented on top of this more general proc-macro.

Versions

Find the latest versions by id

v2.0.0 - Jul 17, 2022

Added

  • Make JsonString<T> smarter by allowing nesting serde_as definitions. This allows applying custom serialization logic, before the value gets converted into a JSON string.

Changed

  • Make #[serde_as] behave more intuitive on Option<T> fields.

    The #[serde_as] macro now detects if a #[serde_as(as = "Option<S>")] is used on a field of type Option<T> and applies #[serde(default)] to the field. This restores the ability to deserialize with missing fields and fixes a common annoyance (#183, #185, #311, #417). This is a breaking change, since now deserialization will pass where it did not before and this might be undesired.

    The Option field and transformation are detected by directly matching on the type name. These variants are detected as Option.

    • Option
    • std::option::Option, with or without leading ::
    • core::option::Option, with or without leading ::

    If an existing default attribute is detected, the attribute is not applied again. This behavior can be suppressed by using #[serde_as(no_default)] or #[serde_as(as = "Option<S>", no_default)].

  • NoneAsEmptyString and string_empty_as_none use a different serialization bound (#388).

    Both types used AsRef<str> as the serialization bound. This is limiting for non-string types like Option<i32>. The deserialization often was already more flexible, due to the FromStr bound.

    For most std types this should have little impact, as the types implementing AsRef<str> mostly implement Display, too, such as String, Cow<str>, or Rc<str>.

  • Bump MSRV to 1.60. This is required for the optional dependency feature syntax in cargo.

Removed

  • Remove old module based conversions.

    The newer serde_as based conversions are preferred.

    • seq_display_fromstr: Use DisplayFromStr in combination with your container type:

    • tuple_list_as_map: Use BTreeMap on a Vec of tuples:

    • map_as_tuple_list can be replaced with #[serde_as(as = "Vec<(_, _)>")].

    • display_fromstr can be replaced with #[serde_as(as = "DisplayFromStr")].

    • bytes_or_string can be replaced with #[serde_as(as = "BytesOrString")].

    • default_on_error can be replaced with #[serde_as(as = "DefaultOnError")].

    • default_on_null can be replaced with #[serde_as(as = "DefaultOnNull")].

    • string_empty_as_none can be replaced with #[serde_as(as = "NoneAsEmptyString")].

    • StringWithSeparator can now only be used in serde_as. The definition of the Separator trait and its implementations have been moved to the formats module.

    • json::nested can be replaced with #[serde_as(as = "json::JsonString")].

  • Remove previously deprecated modules.

    • sets_first_value_wins
    • btreemap_as_tuple_list and hashmap_as_tuple_list can be replaced with #[serde_as(as = "Vec<(_, _)>")].

macros-v2.0.0 - Jul 17, 2022

No changes compared to v2.0.0-rc.0.

Changed

  • Make #[serde_as] behave more intuitive on Option<T> fields.

    The #[serde_as] macro now detects if a #[serde_as(as = "Option<S>")] is used on a field of type Option<T> and applies #[serde(default)] to the field. This restores the ability to deserialize with missing fields and fixes a common annoyance (#183, #185, #311, #417). This is a breaking change, since now deserialization will pass where it did not before and this might be undesired.

    The Option field and transformation are detected by directly matching on the type name. These variants are detected as Option.

    • Option
    • std::option::Option, with or without leading ::
    • core::option::Option, with or without leading ::

    If an existing default attribute is detected, the attribute is not applied again. This behavior can be supressed by using #[serde_as(no_default)] or #[serde_as(as = "Option<S>", no_default)].

Fixed

  • Make the documentation clearer by stating that the #[serde_as] and #[skip_serializing_none] attributes must always be placed before #[derive].

v2.0.0-rc.0 - Jun 29, 2022

Changed

  • Make #[serde_as] behave more intuitive on Option<T> fields.

    The #[serde_as] macro now detects if a #[serde_as(as = "Option<S>")] is used on a field of type Option<T> and applies #[serde(default)] to the field. This restores the ability to deserialize with missing fields and fixes a common annoyance (#183, #185, #311, #417). This is a breaking change, since now deserialization will pass where it did not before and this might be undesired.

    The Option field and transformation are detected by directly matching on the type name. These variants are detected as Option.

    • Option
    • std::option::Option, with or without leading ::
    • core::option::Option, with or without leading ::

    If an existing default attribute is detected, the attribute is not applied again. This behavior can be suppressed by using #[serde_as(no_default)] or #[serde_as(as = "Option<S>", no_default)].

  • NoneAsEmptyString and string_empty_as_none use a different serialization bound (#388).

    Both types used AsRef<str> as the serialization bound. This is limiting for non-string types like Option<i32>. The deserialization often was already more flexible, due to the FromStr bound.

    For most std types this should have little impact, as the types implementing AsRef<str> mostly implement Display, too, such as String, Cow<str>, or Rc<str>.

  • Bump MSRV to 1.60. This is required for the optional dependency feature syntax in cargo.

Removed

  • Remove old module based conversions.

    The newer serde_as based conversions are preferred.

    • seq_display_fromstr: Use DisplayFromStr in combination with your container type:

    • tuple_list_as_map: Use BTreeMap on a Vec of tuples:

    • map_as_tuple_list can be replaced with #[serde_as(as = "Vec<(_, _)>")].

    • display_fromstr can be replaced with #[serde_as(as = "DisplayFromStr")].

    • bytes_or_string can be replaced with #[serde_as(as = "BytesOrString")].

    • default_on_error can be replaced with #[serde_as(as = "DefaultOnError")].

    • default_on_null can be replaced with #[serde_as(as = "DefaultOnNull")].

    • string_empty_as_none can be replaced with #[serde_as(as = "NoneAsEmptyString")].

    • StringWithSeparator can now only be used in serde_as. The definition of the Separator trait and its implementations have been moved to the formats module.

    • json::nested can be replaced with #[serde_as(as = "json::JsonString")].

  • Remove previously deprecated modules.

    • sets_first_value_wins
    • btreemap_as_tuple_list and hashmap_as_tuple_list can be replaced with #[serde_as(as = "Vec<(_, _)>")].

macros-v2.0.0-rc.0 - Jun 29, 2022

Changed

  • Make #[serde_as] behave more intuitive on Option<T> fields.

    The #[serde_as] macro now detects if a #[serde_as(as = "Option<S>")] is used on a field of type Option<T> and applies #[serde(default)] to the field. This restores the ability to deserialize with missing fields and fixes a common annoyance (#183, #185, #311, #417). This is a breaking change, since now deserialization will pass where it did not before and this might be undesired.

    The Option field and transformation are detected by directly matching on the type name. These variants are detected as Option.

    • Option
    • std::option::Option, with or without leading ::
    • core::option::Option, with or without leading ::

    If an existing default attribute is detected, the attribute is not applied again. This behavior can be suppressed by using #[serde_as(no_default)] or #[serde_as(as = "Option<S>", no_default)].

v1.14.0 - May 29, 2022

Added

  • Add support for time crate v0.3 #450

    time::Duration can now be serialized with the DurationSeconds and related converters.

    time::OffsetDateTime and time::PrimitiveDateTime can now be serialized with the TimestampSeconds and related converters.

    time::OffsetDateTime can be serialized in string format in different well-known formats. Two formats are supported, time::format_description::well_known::Rfc2822 and time::format_description::well_known::Rfc3339.

  • Deserialize bool from integers #456 462

    Deserialize an integer and convert it into a bool. BoolFromInt<Strict> (default) deserializes 0 to false and 1 to true, other numbers are errors. BoolFromInt<Flexible> deserializes any non-zero as true. Serialization only emits 0/1.

Changed

  • Bump MSRV to 1.53, since the new dependency time requires that version.

Fixed

  • Make the documentation clearer by stating that the #[serde_as] and #[skip_serializing_none] attributes must always be places before #[derive].

v1.13.0 - Apr 23, 2022

Added

  • Added support for indexmap::IndexMap and indexmap::IndexSet types. #431, #436

    Both types are now compatible with these functions: maps_duplicate_key_is_error, maps_first_key_wins, sets_duplicate_value_is_error, sets_last_value_wins. serde_as integration is provided by implementing both SerializeAs and DeserializeAs for both types. IndexMaps can also be serialized as a list of types via the serde_as(as = "Vec<(_, _)>") annotation.

    All implementations are gated behind the indexmap feature.

    Thanks to @jgrund for providing parts of the implementation.

v1.12.1 - Apr 07, 2022

Fixed

  • Depend on a newer serde_with_macros version to pull in some fixes.
    • Account for generics when deriving implementations with SerializeDisplay and DeserializeFromStr #413
    • Provide better error messages when parsing types fails #423

macros-v1.5.2 - Apr 07, 2022

Fixed

  • Account for generics when deriving implementations with SerializeDisplay and DeserializeFromStr #413
  • Provide better error messages when parsing types fails #423

v1.12.0 - Feb 07, 2022

Added

  • Deserialize a Vec and skip all elements failing to deserialize #383

    VecSkipError acts like a Vec, but elements which fail to deserialize, like the "Yellow" are ignored.

    Thanks to @hdhoang for creating the PR.

  • Transform between maps and Vec<Enum> #375

    The new EnumMap type converts Vec of enums into a single map. The key is the enum variant name, and the value is the variant value.

Changed

  • The Timestamp*Seconds and Timestamp*SecondsWithFrac types can now be used with chrono::NaiveDateTime. #389

v1.11.0 - Oct 18, 2021

Added

Fixed

  • Fixed RUSTSEC-2020-0071 in the time v0.1 dependency, but changing the feature flags of the chrono dependency. This should not change anything. Crates requiring the oldtime feature of chrono can enable it separately.

macros-v1.5.1 - Oct 18, 2021

Added

v1.10.0 - Sep 04, 2021

Added

  • Add BorrowCow which instructs serde to borrow data during deserialization of Cow<'_, str>, Cow<'_, [u8]>, or Cow<'_, [u8; N]>. (#347) The implementation is for serde#2072 and serde#2016, about #[serde(borrow)] not working for Option<Cow<'a, str>>.

    The #[serde(borrow)] annotation is automatically added by the #[serde_as] attribute.

Changed

  • Bump MSRV to 1.46, since the dev-dependency bitflags requires that version now.
  • flattened_maybe! no longer requires the serde_with crate to be available with a specific name. This allows renaming the crate or using flattened_maybe! through a re-export without any complications.

macros-v1.5.0 - Sep 04, 2021

Added

  • Add the attribute #[serde(borrow)] on a field if serde_as is used in combination with the BorrowCow type.

v1.9.4 - Jun 18, 2021

Fixed

  • with_prefix! now supports an optional visibility modifier. (#327, #328)
    If not specified pub(self) is assumed.

    Thanks to @elpiel for raising and fixing the issue.

v1.9.3 - Jun 14, 2021

Added

  • The Bytes type now supports borrowed and Cow arrays of fixed size (requires Rust 1.51+)

    Note: For borrowed arrays the used Deserializer needs to support Serde's 0-copy deserialization.

v1.9.2 - Jun 07, 2021

Fixed

  • Suppress clippy warnings, which can occur while using serde_conv (#320) Thanks to @mkroening for reporting and fixing the issue.

macros-v1.4.2 - Jun 07, 2021

Fixed

  • Describe how the serde_as macro works on a high level.
  • The derive macros SerializeDisplay and DeserializeFromStr were relying on the prelude where they were used. Properly name all types and traits required for the expanded code to work. The tests were improved to be better able to catch such problems.

v1.9.1 - May 15, 2021

Changed

  • NoneAsEmptyString: Deserialize using FromStr instead of using for<'a> From<&'a str> (#316) This will not change any behavior when applied to a field of type Option<String> as used in the documentation. Thanks to @mkroening for finding and fixing the issue.

v1.9.0 - May 09, 2021

Added

  • Added FromInto and TryFromInto adapters, which enable serialization by converting into a proxy type.

  • New serde_conv! macro to create conversion types with reduced boilerplate. The generated types can be used with #[serde_as] or serde's with-attribute.

v1.8.1 - Apr 19, 2021

Added

  • The hex::Hex type also works for u8-arrays on Rust 1.48. Thanks to @TheAlgorythm for raising and fixing the issue.

v1.8.0 - Mar 30, 2021

Added

  • Added PickFirst adapter for serde_as. #291 It allows to deserialize from multiple different forms. Deserializing a number from either a number or string can be implemented like:

  • Implement SerializeAs/DeserializeAs for more wrapper types. #288, #293 This now supports:

    • Arc, sync::Weak
    • Rc, rc::Weak
    • Cell, RefCell
    • Mutex, RwLock
    • Result

Changed

  • Add a new serde_with::rust::map_as_tuple_list module as a replacement for serde_with::rust::btreemap_as_tuple_list and serde_with::rust::hashmap_as_tuple_list. The new module uses IntoIterator and FromIterator as trait bound making it usable in more sitations. The old names continue to exist but are marked as deprecated.

Deprecated

  • Deprecated the module names serde_with::rust::btreemap_as_tuple_list and serde_with::rust::hashmap_as_tuple_list. You can use serde_with::rust::map_as_tuple_list as a replacement.

Fixed

  • Implement Timestamp*Seconds and Duration*Seconds also for chrono types. This closes #194. This was incompletely implemented in #199.

v1.7.0 - Mar 24, 2021

Added

  • Add support for arrays of arbitrary size. (#272) This feature requires Rust 1.51+.

    Mapping of arrays was available before, but limited to arrays of length 32. All conversion methods are available for the array elements.

    This is similar to the existing serde-big-array crate with three important improvements:

    1. Support for the serde_as annotation.
    2. Supports non-copy elements (see serde-big-array#6).
    3. Supports arbitrary nestings of arrays (see serde-big-array#7).
  • Arrays with tuple elements can now be deserialized from a map. (#272) This feature requires Rust 1.51+.

  • The Bytes type is heavily inspired by serde_bytes and ports it to the serde_as system. (#277)

    Compared to serde_bytes these improvements are available

    1. Integration with the serde_as annotation (see serde-bytes#14).
    2. Implementation for arrays of arbitrary size (Rust 1.51+) (see serde-bytes#26).
  • The OneOrMany allows to deserialize a Vec from either a single element or a sequence. (#281)

    This allows to deserialize from either cities: "Berlin" or cities: ["Berlin", "Paris"]. The serialization can be configured to always emit a list with PreferMany or emit a single element with PreferOne.

v1.6.4 - Feb 16, 2021

Fixed

  • Fix compiling when having a struct field without the serde_as annotation by updating serde_with_macros. This broke in 1.4.0 of serde_with_macros. #267

macros-v1.4.1 - Feb 16, 2021

Fixed

  • Fix compiling when having a struct field without the serde_as annotation. This broke in 1.4.0 #267

v1.6.3 - Feb 15, 2021

Changed

  • Bump macro crate dependency (serde_with_macros) to 1.4.0 to pull in those improvements.

macros-v1.4.0 - Feb 15, 2021

Changed

  • Improve error messages when #[serde_as(..)] is misused as a field attribute. Thanks to @Lehona for reporting the bug in #233.

  • Internal cleanup for assembling and parsing attributes during serde_as processing.

  • Change processing on #[serde_as(...)] attributes on fields.

    The attributes will no longer be stripped during proc-macro processing. Instead, a private derive macro is applied to the struct/enum which captures them and makes them inert, thus allowing compilation.

    This should have no effect on the generated code and on the runtime behavior. It eases integration of third-party crates with serde_with, since they can now process the #[serde_as(...)] field attributes reliably. Before this was impossible for derive macros and lead to akward ordering constraints on the attribute macros.

    Thanks to @Lehona for reporting this problem and to @dtolnay for suggesting the dummy derive macro.

v1.6.2 - Jan 30, 2021

Added

  • New function serde_with::rust::deserialize_ignore_any. This function allows deserializing any data and returns the default value of the type. This can be used in conjunction with #[serde(other)] to allow deserialization of unknown data carrying enum variants.

    Thanks to @lovasoa for suggesting and implementing it.

v1.6.1 - Jan 24, 2021

Added

  • Add new types similar to DurationSeconds and TimestampSeconds but for base units of milliseconds, microseconds, and nanoseconds. The *WithFrac variants also exist.
  • Add SerializeAs implementation for references.

Changed

  • Release Sized trait bound from As, Same, SerializeAs, and SerializeAsWrap. Only the serialize part is relaxed.

Information - Updated Aug 22, 2022

Stars: 300
Forks: 38
Issues: 9

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically

Rust Greatest JSON weapon is Serde with over 4.4K stars on github and a massive developer community. This is considered a core Rust library for every developer to learn in BRC's opinion

Serde is a framework for serializing and deserializing Rust data structures efficiently and generically

Rust 버전 JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

Rust 버전 JsonPath 구현으로 Webassembly와 Javascript에서도 유사한 API 인터페이스를 제공 한다

SIMD JSON for Rust  

Rust port of extremely fast serde compatibility

SIMD JSON for Rust  

JSON-E Rust data-struct paramter crate for lightweight embedded content with objects and much more

What makes JSON-e unique is that it extensive documentation and ease of use

JSON-E Rust data-struct paramter crate for lightweight embedded content with objects and much more
JSON

111

A Rust JSON5 serializer and deserializer which speaks Serde

Deserialize a JSON5 string with from_str

A Rust JSON5 serializer and deserializer which speaks Serde

Rust JSON Parser Benchmark

Download and Generate JSON Data

Rust JSON Parser Benchmark

Read JSON values quickly - Rust JSON Parser

AJSON get json value with specified path, such as project

Read JSON values quickly - Rust JSON Parser

Rust actix json request example

Send a json request to actix, and parse it

Rust actix json request example
JSON

140

json_typegen - Rust types from JSON samples

json_typegen is a collection of tools for generating types from

json_typegen - Rust types from JSON samples

Rust JSON parsing benchmarks

This project aims to provide benchmarks to show how various JSON-parsing libraries in the Rust programming language perform at various JSON-parsing tasks

Rust JSON parsing benchmarks

A tiny command line tool written in rust to print json data as a formatted...

A tiny command line tool written in rust to print json data as a formatted table

A tiny command line tool written in rust to print json data as a formatted...

Rust RPC client for Bitcoin Core JSON-RPC

rust-jsonrpc and makes it easier to talk to the Bitcoin JSON-RPC interface

Rust RPC client for Bitcoin Core JSON-RPC
Facebook Instagram Twitter GitHub Dribbble
Privacy