This article is the 20th day post for the Unreal Engine 4 (UE4) Advent Calendar 2019.
Unreal Engine 4 (UE4) Advent Calendar 2019 - Qiita
I have a plugin called ObjectDeliverer published on the Marketplace.
This article is about the “function to serialize objects to Json” implemented in this plugin.
For example, if you have a Blueprint (inheriting from Object) with Variables defined like this,
It will be converted into a Json string like this.
{
"IntValue":0,
"BoolValue":false,
"StringValue":"ABC"
}
Since Json is represented as a string, it can be saved directly to a text file. Also, being a string, its content is easily understandable by humans.
By also creating a function to restore objects from this Json, saving and restoring objects via Json becomes possible.
In ObjectDeliverer, by sending and receiving this Json string over communication, objects can be delivered to another location (another application).
Handling Json in UE4
UE4 comes standard with modules named Json and JsonUtilities.
Json Module
A module that supports reading and writing Json. However, it cannot be used in Blueprints and is only supported in C++.
It can be used like this.
Json Object -> Json String
// Create FJsonObject (container for Json data)
TSharedPtr<FJsonObject> jsonObject = MakeShareable(new FJsonObject());
// Add properties to jsonObject
// Create a Writer to write Json to FString
FString OutputString;
TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&OutputString);
// Write Json to FString
FJsonSerializer::Serialize(jsonObject.ToSharedRef(), Writer);
// Json string is in OutputString, do something with it
Json String -> Json Object
// FString containing the json string
FString jsonString;
// Create a Reader to read Json from FString
TSharedRef<TJsonReader<TCHAR>> JsonReader = TJsonReaderFactory<TCHAR>::Create(jsonString);
// Create FJsonObject (container for Json data)
TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());
// Read from Json string into Json object
FJsonSerializer::Deserialize(JsonReader, JsonObject);
// Extract properties from jsonObject and do something
Get, Set Json Object Values
TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());
// Set values
JsonObject->SetNumberField("IntValue", 1);
JsonObject->SetBoolField("BoolValue", true);
JsonObject->SetStringField("StringValue", "ABCDEFG");
// Get values
int32 intValue = JsonObject->GetIntegerField("IntValue");
bool boolValue = JsonObject->GetBoolField("BoolValue");
FString stringValue = JsonObject->GetStringField("StringValue");
// Array and Object properties can also be Get, Set
Knowing the above patterns should cover most cases.
JsonUtilities Module
Separate from the Json module, there is a JsonUtilities module. This is a support module that uses the Json module.
Inside it, there is a class called FJsonObjectConverter, which sounds like it could convert objects to Json based on the name.
However, looking inside, it was a class that provides mutual conversion between UStruct and JSON. What I want to achieve is mutual conversion between UObject and JSON, so it was slightly different. Disappointing.
By the way, this functionality can be used by calling the following two static methods.
- UStructToJsonObject
- JsonObjectToUStruct
Creating UObject and JSON Mutual Conversion Yourself
First, although I wrote creating yourself, it’s almost the same as the implementation of the FJsonObjectConverter class.
https://github.com/ayumax/ObjectDeliverer/blob/master/Plugins/ObjectDeliverer/Source/ObjectDeliverer/Private/Utils/ObjectJsonSerializer.cpp
If you compare it side-by-side with the FJsonObjectConverter class, you’ll see they are almost identical.
I only added the UObject handling process to the implementation of the FJsonObjectConverter class.
The complete code is at the link above, but I will explain only the representative parts for realizing the functionality.
UObject -> JsonObject
This is the process of converting UObject to Json.
First, use TFieldIterator to get a list of UObject properties (including variables defined in Blueprints) and process them sequentially.
In a pure C++ program not using Unreal Engine, this kind of thing (reflection) is not possible. (Although I’ve seen articles discussing it for future C++ plans)
In UE4 C++, reflection is apparently called the property system.
Details are written here. Unreal Property System (Reflection)
Once property information (UProperty) is obtained, the property name and the actual property value can also be retrieved using it.
After that, Json properties can be created in the same way as setting properties on a Json object described earlier.
However, since nested structures where a UObject contains a UObject property also need to be considered, the processing here is done recursively.
TSharedPtr<FJsonObject> UObjectJsonSerializer::CreateJsonObject(const UObject* Obj)
{
TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());
if (!Obj) return JsonObject;
// Process the list of UObject properties sequentially
for (TFieldIterator<UProperty> PropIt(Obj->GetClass(), EFieldIteratorFlags::IncludeSuper); PropIt; ++PropIt)
{
UProperty* Property = *PropIt;
// Get property name
FString PropertyName = Property->GetName();
// Get property value
uint8* CurrentPropAddr = PropIt->ContainerPtrToValuePtr<uint8>((UObject*)Obj);
FJsonObjectConverter::CustomExportCallback CustomCB;
CustomCB.BindStatic(UObjectJsonSerializer::ObjectJsonCallback);
// Set to Json property
// FJsonObjectConverter::UPropertyToJsonValue is processed recursively
JsonObject->SetField(PropertyName, FJsonObjectConverter::UPropertyToJsonValue(Property, CurrentPropAddr, 0, 0, &CustomCB));
}
return JsonObject;
}
JsonObject -> UObject
This is the process of restoring UObject from Json.
Similar to the process of converting UObject to Json, this also uses the UE4 property system to dynamically obtain the property list and process it sequentially.
The flow is to obtain UProperty, get the property name from it, then get the Json property from JsonObject and set it to the UObject’s property.
This function is also called recursively to support nested structures.
bool UObjectJsonSerializer::JsonObjectToUObject(const TSharedPtr<FJsonObject>& JsonObject, UObject* OutObject)
{
auto& JsonAttributes = JsonObject->Values;
// Get the property list of the UObject to restore
for (TFieldIterator<UProperty> PropIt(OutObject->GetClass()); PropIt; ++PropIt)
{
UProperty* Property = *PropIt;
// Get Json property from JsonObject using property name as key
const TSharedPtr<FJsonValue>* JsonValue = JsonAttributes.Find(Property->GetName());
if (!JsonValue)
{
continue;
}
if (JsonValue->IsValid() && !(*JsonValue)->IsNull())
{
// Get pointer to UObject property
void* Value = Property->ContainerPtrToValuePtr<uint8>(OutObject);
// Read value from Json property and set to UObject property
// JsonValueToUProperty is processed recursively
if (!JsonValueToUProperty(*JsonValue, Property, Value))
{
return false;
}
}
}
return true;
}
Some Problems
In ObjectDeliverer, the above ObjectJsonSerializer is used within UObjectDeliveryBoxUsingJson to serialize UObjects to Json.
I also use this feature in my own projects, and while it generally works as intended, some points for improvement have emerged.
Need to Specify UObject Type During JsonObject -> UObject
As explained in the JsonObject -> UObject process description above, the UObject type must be predetermined for this process.
Looking at the Json format, you can see that Json has no mechanism to store the object type.
Therefore, just looking at the Json, it’s impossible to know which UObject to restore to, so the UObject type needs to be decided beforehand.
Because of this, communication using ObjectDeliverer’s UObjectDeliveryBoxUsingJson currently only supports sending and receiving a single UObject type. It cannot handle multiple types with normal usage.
Potential Information Loss When Having UObject Type Properties
This problem also arises because Json cannot store type information, similar to the previous issue.
For example, consider an environment where ClassB inherits from ClassA.
In that environment, the UObject saved to Json has a property of type ClassA.
An instance of ClassB can be put into this ClassA type property.
When saving to and restoring from Json at this time, properties that exist only in ClassB but not in ClassA will not be restored…
This is because the UProperty indicating ClassA type is used during property restoration.
This can be avoided if understood and used correctly, but it might lead to hard-to-notice bugs.
Json Property Name May Not Match the Specified String
For example, when converting the Blueprint shown in the example at the beginning to Json, it might end up like this.
{
"Intvalue":0,
"boolValue":false,
"Stringvalue":"ABC"
}
At first glance, it seems to be working correctly, but the case of the property names is wrong.
This is because UE4 properties use the FName type.
As mentioned on the page above, FName is case-insensitive, which causes this phenomenon.
No problem occurs if both saving to and restoring from Json use ObjectDeliverer within the same UE4 project, but problems (property name mismatch) may arise when exchanging Json with applications created in other development environments.
Countermeasures for Problems
Actually, I wanted to release an update with countermeasures for the above problems by the time this article was published, but it’s not complete yet… (Testing isn’t finished)
The work is being done on the following branch, and the implementation is almost complete, so you can see what it looks like.
The Json serialization part has undergone significant changes.
https://github.com/ayumax/ObjectDeliverer/tree/Remodeling-ObjectJsonSerializer
The countermeasures taken are the following two.
Write UObject Type Information into Json Based on User Selection
The first and second problems can be resolved by writing the UObject type information into the Json.
However, doing so changes the Json format from before, breaking compatibility, and purely increases the Json string length, making the data size larger, which might be undesirable from the perspective of data transmission, ObjectDeliverer’s main purpose.
Therefore, a mechanism is introduced where the user can choose whether or not to write type information into the Json for each UObject type.
This allows creating the same Json as before under the condition that only one type is sent/received and UObject properties using class inheritance are not created.
Also, by choosing to write the actual instance’s type information, reliable restoration can be achieved when saving and restoring UObjects.
Specifically, the following differences arise in the created Json.
When not writing type information (implementation up to now)
{
"IntValue":0,
"BoolValue":false,
"StringValue":"ABC"
}
When writing type information
Example when serializing a UObject named NewBlueprint
{
"Type":"NewBlueprint",
"Body" :
{
"IntValue":0,
"BoolValue" : false,
"StringValue" : "ABC"
}
}
Definition of an Interface to Implement Property Name Conversion Function
Since conversion from FName type to FString type might not result in the intended string if left to automatic conversion, an interface for property name conversion was added so users can define it reliably themselves.
If this interface is implemented on the UObject to be serialized, the custom property name conversion function will be used for writing and reading Json properties.
If the interface is not implemented, automatic FName -> FString conversion will occur as before.
By using this feature, it becomes possible to create Json properties with completely different strings, so I believe it has uses beyond solving the current problem.
Finally
This time, I wrote about serializing UObjects to JsonObjects.
The source code of FJsonObjectConverter from the JsonUtilities module mentioned in this article contains implementations that are helpful not only for creating UObject serialization to formats other than Json but also for utilizing the UE4 property system, so please take a look.
Also, the ObjectDeliverer I have published aims to easily create communication using TCP/IP, UDP, etc., using only Blueprints. It’s available for Free, so please try it if you like.