Proposal: Add mechanism to handle circular references when serializing

See initial proposal with extended comments here:
https://github.com/dotnet/runtime/pull/354/files

See proposal extension for `ReferenceResolver` on https://github.com/dotnet/runtime/issues/30820#issuecomment-623752644.

# Rationale and Usage

Currently there is no mechanism to prevent infinite looping in circular objects nor to preserve references when using System.Text.Json.

Community is heavily requesting this feature since is consider by many as a very common scenario, specially when serializing POCOs that came from an ORM Framework, such as Entity Framework; even though JSON specifiacation does not support reference loops by default. Therefore this will be shipped as an opt-in feature.

The current solution to deal with reference loops is to rely in MaxDepth and throw a `JsonException` after it is exceeded. Now, this is a decent and cheap solution but we will also offer other not-so-cheap options to deal with this problem while keeping the current one in order to not affect the out-of-the-box performance.

# Proposed API
```cs
namespace System.Text.Json
{
    public class JsonSerializerOptions
    {
        public ReferenceHandling ReferenceHandling { get; set; } = ReferenceHandling.Default;
    }
}

namespace System.Text.Json.Serialization
{
    /// <summary>
    /// This enumeration defines the various ways the <see cref="JsonSerializer"/> 
    /// can deal with references on serialization and deserialization.
    /// </summary>
    public enum ReferenceHandling
    {
        Default,
        Preserve
    }
}
```

## EDIT: 
We considered having `ReferenceHandlig.Ignore` but it was cut out of the final design due the lack of scenarios where you would really *need* `Ignore` over `Preserve`.

Although is not part of the shipping API, the samples and definitions of `Ignore` remain in this description for their informative value.

## In depth
* **Default**: 
   * **On Serialize**: Throw a JsonException when MaxDepth is exceeded, this may occur by either a Reference Loop or by passing a very deep object. This option will not affect the performance of the serializer.
   * **On Deserialize**: No effect.

* **Ignore**:
  * **On Serialize**: Ignores (skips writing) the property/element where the reference loop is detected.
  * **On Deserialize**: No effect.

* **Preserve**:
  * **On Serialize**: When writing complex types, the serializer also writes them metadata ($id, $values and $ref) properties in order re-use them by writing a reference to the object or array.
  * **On Deserialize**: While the other options show no effect on Deserialization, `Preserve` does affect its behavior with the following: Metadata will be expected (although is not mandatory) and the deserializer will try to understand it. 


# Feature Parity (Examples of System.Text.Json vs Newtonsoft's Json.Net)
Having the following class:
```cs
class Employee 
{ 
    string Name { get; set; }
    Employee Manager { get; set; }
    List<Employee> Subordinates { get; set; }
}
```

## Using Ignore on Serialize
On System.Text.Json:
```cs
public static void WriteIgnoringReferenceLoops()
{
    var bob = new Employee { Name = "Bob" };
    var angela = new Employee { Name = "Angela" };

    angela.Manager = bob;
    bob.Subordinates = new List<Employee>{ angela };

    var options = new JsonSerializerOptions
    {
        ReferenceHandling = ReferenceHandling.Ignore
        WriteIndented = true,
    };

    string json = JsonSerializer.Serialize(angela, options);
    Console.Write(json);
}
```

On Newtonsoft's Json.Net:
```cs
public static void WriteIgnoringReferenceLoops()
{
    var bob = new Employee { Name = "Bob" };
    var angela = new Employee { Name = "Angela" };

    angela.Manager = bob;
    bob.Subordinates = new List<Employee>{ angela };

    var settings = new JsonSerializerSettings
    {
        ReferenceLoopHandling = ReferenceLoopHandling.Ignore
        Formatting = Formatting.Indented 
    };

    string json = JsonConvert.SerializeObject(angela, settings);
    Console.Write(json);
}
```

Output:
```jsonc
{
    "Name": "Angela",
    "Manager": {
        "Name": "Bob",
        "Subordinates": [] //Note how subordinates is empty due Angela is being ignored.
    }
}
```

## Using Preserve on Serialize
On System.Text.Json:
```cs
public static void WritePreservingReference()
{
    var bob = new Employee { Name = "Bob" };
    var angela = new Employee { Name = "Angela" };

    angela.Manager = bob;
    bob.Subordinates = new List<Employee>{ angela };

    var options = new JsonSerializerOptions
    {
        ReferenceHandling = ReferenceHandling.Preserve
        WriteIndented = true,
    };

    string json = JsonSerializer.Serialize(angela, options);
    Console.Write(json);
}
```

On Newtonsoft's Json.Net:
```cs
public static void WritePreservingReference()
{
    var bob = new Employee { Name = "Bob" };
    var angela = new Employee { Name = "Angela" };

    angela.Manager = bob;
    bob.Subordinates = new List<Employee>{ angela };

    var settings = new JsonSerializerSettings
    {
        PreserveReferencesHandling = PreserveReferencesHandling.All
        Formatting = Formatting.Indented 
    };

    string json = JsonConvert.SerializeObject(angela, settings);
    Console.Write(json);
}
```

Output:
```jsonc
{
    "$id": "1",
    "Name": "Angela",
    "Manager": {
        "$id": "2",
        "Name": "Bob",
        "Subordinates": { //Note how the Subordinates' square braces are replaced with curly braces in order to include $id and $values properties, $values will now hold whatever value was meant for the Subordinates list.
            "$id": "3",
            "$values": [
                {  //Note how this object denotes reference to Angela that was previously serialized.
                    "$ref": "1"
                }
            ]
        }            
    }
}
```

## Using Preserve on Deserialize
On System.Text.Json:
```cs
public static void ReadJsonWithPreservedReferences(){
    string json = 
    @"{
        ""$id"": ""1"",
        ""Name"": ""Angela"",
        ""Manager"": {
            ""$id"": ""2"",
            ""Name"": ""Bob"",
            ""Subordinates"": {
                ""$id"": ""3"",
                ""$values"": [
                    { 
                        ""$ref"": ""1"" 
                    }
                ]
            }            
        }
    }";

    var options = new JsonSerializerOptions
    {
        ReferenceHandling = ReferenceHandling.Preserve
    };

    Employee angela = JsonSerializer.Deserialize<Employee>(json, options);
    Console.WriteLine(object.ReferenceEquals(angela, angela.Manager.Subordinates[0])); //prints: true.
}
```

On Newtonsoft's Json.Net:
```cs
public static void ReadJsonWithPreservedReferences(){
    string json = 
    @"{
        ""$id"": ""1"",
        ""Name"": ""Angela"",
        ""Manager"": {
            ""$id"": ""2"",
            ""Name"": ""Bob"",
            ""Subordinates"": {
                ""$id"": ""3"",
                ""$values"": [
                    { 
                        ""$ref"": ""1"" 
                    }
                ]
            }            
        }
    }";

    var options = new JsonSerializerSettings
    {
        MetadataPropertyHanding = MetadataPropertyHandling.Default //Json.Net reads metadata by default, just setting the option for ilustrative purposes.
    };

    Employee angela = JsonConvert.DeserializeObject<Employee>(json, settings);
    Console.WriteLine(object.ReferenceEquals(angela, angela.Manager.Subordinates[0])); //prints: true.
}
```

# Notes:
1. MaxDepth validation will not be affected by `ReferenceHandling.Ignore` or `ReferenceHandling.Preserve`.
2. We are merging the Json.Net types [`ReferenceLoopHandling`]("https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_ReferenceLoopHandling.htm") and [`PreserveReferencesHandling`]("https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_PreserveReferencesHandling.htm") (we are also not including the granularity on this one) into one single enum; `ReferenceHandling`.
3. While Immutable types and `System.Array`s can be Serialized with Preserve semantics, they will not be supported when trying to Deserialize them as a reference.
4. Value types, such as structs, will not be supported when Deserializing as well.
5. Additional features, such as Converter support, `ReferenceResolver`, `JsonPropertyAttribute.IsReference` and `JsonPropertyAttribute.ReferenceLoopHandling`,  that build on top of `ReferenceLoopHandling` and `PreserveReferencesHandling` were considered but they will not be included in this first effort.
6. We are still looking for evidence that backs up supporting `ReferenceHandling.Ignore`.

## Issues related:
* https://github.com/dotnet/corefx/issues/38579
* https://github.com/dotnet/corefx/issues/37786
* https://github.com/dotnet/corefx/issues/40045
* https://github.com/dotnet/corefx/issues/41139
* https://github.com/dotnet/corefx/issues/41288
* https://github.com/aspnet/AspNetCore/issues/14497


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Proposal: Add mechanism to handle circular references when serializing #30820

Rationale and Usage

Proposed API

EDIT:

In depth

Feature Parity (Examples of System.Text.Json vs Newtonsoft's Json.Net)

Using Ignore on Serialize

Using Preserve on Serialize

Using Preserve on Deserialize

Notes:

Issues related:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Proposal: Add mechanism to handle circular references when serializing #30820

Description

Rationale and Usage

Proposed API

EDIT:

In depth

Feature Parity (Examples of System.Text.Json vs Newtonsoft's Json.Net)

Using Ignore on Serialize

Using Preserve on Serialize

Using Preserve on Deserialize

Notes:

Issues related:

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions