https://github.com/ayumax/Froola

Froola is a tool I created for testing Unreal Engine code plugins on multiple platforms simultaneously and generating plugin packages.

Originally, I had developed several UE plugins that supported Windows and Mac platforms, which made package creation a tedious process each time.

This is because in a Windows environment, you can create packages for Windows, Android, and Linux, but not for Mac and iOS.

Therefore, I had to build the plugin in both Windows and Mac environments, and then combine them to create a single plugin package.

It would have been nice if Unreal Editor had such functionality, but I couldn’t find it within my research, so I was manually combining them each time.

However, I thought others might also benefit from having these capabilities in a single tool, so I created Froola.

Comparison with CI/CD features like GitHub Actions

Normally, when wanting to accomplish such tasks, the first thing that comes to mind is using CI/CD features like GitHub Actions.

However, running Unreal Engine in the cloud is challenging, primarily from a financial perspective.

A single version of Unreal Engine is approximately 50GB, and preparing multiple versions requires even more storage.

Additionally, a fairly powerful CPU is necessary, making it difficult to set up an execution environment.

Although you can rent Mac EC2 instances on AWS, they are quite expensive with a minimum one-day rental. Other Mac cloud services I found were also costly.

Therefore, I believe many people are likely using on-premises servers as self-hosted runners.

However, when dealing with public repositories as an individual, I wanted to avoid self-hosted runners for security reasons, which led me to conclude that building directly on my development PC was the best option.

Froola’s Structure

Froola generates multi-platform compatible plugin packages, but what it does is straightforward: it builds the plugin in both Windows and Mac environments and then combines them to create a single plugin package.

After consideration, I decided to use Linux UE through Docker instead of using Windows UE for Linux builds. I’ll explain the reason for this later.

Therefore, it operates with the following workflow:

1. Clone the plugin project from a remote repository to the Windows environment
2. Create directories for Windows and Linux builds in the Windows environment and copy the repository files there
3. Copy the repository files cloned in (1) to Mac via SCP
4. Launch UE on Windows and UE in Docker on Windows simultaneously to build the plugin
5. Build the plugin using Mac’s UE via SSH
6. Download the package built on Mac to Windows via SCP
7. Combine the packages built for Windows, Mac, and Linux to create a single plugin package

    sequenceDiagram
    participant Windows
    participant Windows(Docker)
    participant Mac

    Windows->>Windows: Clone from remote repository
    Windows->>Windows: Create Windows, Linux build directories & copy files
    Windows->>Mac: Copy repository files via SCP
    Windows->>Windows: Build plugin with UE
    Windows->>Windows(Docker): Build plugin with UE in Docker
    Mac->>Mac: Build plugin with Mac's UE via SSH
    Mac->>Windows: Download Mac-built package to Windows via SCP
    Windows->>Windows: Combine build packages from each OS into a single plugin package

As you can see, there’s nothing particularly complex about this process.

Simplicity is best.

In fact, I believe there are no other viable methods currently available. At most, you could build for Linux alongside Windows using Windows UE.

Automated Testing

When creating Froola, I made it possible to run tests on each platform in addition to creating packages.

Unreal Engine’s C++ is designed for multiple platforms, so the same code should work on different platforms, but whether it actually performs as expected varies case by case.

One of my plugins, ObjectDeliverer, uses network functionality, and since network features differ by OS, code that works fine on Windows might fail on Mac.

Therefore, running tests on each platform is essential to maintain plugin quality.

Froola uses Unreal Editor on Windows, Mac, and Linux to run tests, allowing you to verify test results across all three platforms.

Test results and Editor execution logs are saved on the Windows PC, which helps in identifying the cause when tests fail.

Incidentally, this testing capability is the reason why I don’t use Windows UE for Linux builds.

How to Run Froola

Froola is provided as a Windows executable application.

For operation instructions, please refer to this link.

You can run it using a command like the following:

Froola.exe plugin -n ObjectDeliverer -p ObjectDelivererTest -u git@github.com:ayumax/ObjectDeliverer.git -b master -e [Windows,Mac,Linux]  -v [5.5] -t -c -g [Win64,Mac,Android,IOS,Linux]

After execution, results are saved in a directory structure like this:

All results are centrally stored in one location, making it easy to troubleshoot when problems occur.

Additionally, the final plugin package is saved in the releases directory. This contains merged files generated for the target platforms, so you can submit this directory to Fab to publish it as a multi-platform compatible plugin.

20250502_205034_ObjectDeliverer/
├── build
│   ├── Windows_UE5.5
│   │   └── Build.log
│   ├── Mac_UE5.5
│   │   └── Build.log
│   └── Linux_UE5.5
│       └── Build.log
├── tests
│   ├── Windows_UE5.5
│   │   ├── AutomationTest.log
│   │   ├── index.html
│   │   └── index.json
│   ├── Mac_UE5.5
│   │   ├── AutomationTest.log
│   │   ├── index.html
│   │   └── index.json
│   └── Linux_UE5.5
│       ├── AutomationTest.log
│       ├── index.html
│       └── index.json
├── packages
│   ├── Windows_UE5.5
│   │   ├── BuildPlugin.log
│   │   └── Plugin
│   │       ├── ObjectDeliverer.uplugin
│   │       ├── Binaries
│   │       ├── Intermediate
│   │       ├── Resources
│   │       └── Source
│   ├── Mac_UE5.5
│   │   ├── BuildPlugin.log
│   │   └── Plugin
│   │       ├── ObjectDeliverer.uplugin
│   │       ├── Binaries
│   │       ├── Intermediate
│   │       ├── Resources
│   │       └── Source
│   └── Linux_UE5.5
│       ├── BuildPlugin.log
│       └── Plugin
│           ├── ObjectDeliverer.uplugin
│           ├── Binaries
│           ├── Intermediate
│           ├── Resources
│           └── Source
├── releases
│   └── ObjectDeliverer_UE5.5
│       ├── ObjectDeliverer.uplugin
│       ├── Binaries
│       ├── Intermediate
│       ├── Resources
│       └── Source
├── froola.log
└── settings.json

Froola’s Design

Froola is built as a C# console application, partly because I like C#.

I hadn’t created many full-fledged console applications in C# before, so initially I was concerned about how to handle numerous command-line arguments.

However, I discovered an excellent library called ConsoleAppFramework, which made it easy to handle a large number of command-line arguments.

This library apparently uses source generators and creates command-line argument information from function comments. It’s impressive.

By using this library, the implementation doesn’t get messy even when the application has multiple commands, so I think I can implement additional commands in the future without much difficulty.

    /// <summary>
    ///     Runs the plugin build, test, and packaging process.
    /// </summary>
    /// <param name="pluginName">-n,Name of the plugin</param>
    /// <param name="projectName">-p,Name of the project</param>
    /// <param name="gitRepositoryUrl">-u,URL of the git repository</param>
    /// <param name="gitBranch">-b,Branch of the git repository</param>
    /// <param name="editorPlatforms">-e,Editor platforms</param>
    /// <param name="engineVersions">-v,Engine versions</param>
    /// <param name="resultPath">-o,Path to save results</param>
    /// <param name="runTest">-t,Run tests</param>
    /// <param name="runPackage">-c,Run packaging</param>
    /// <param name="packagePlatforms">-g,Game platforms</param>
    /// <param name="cancellationToken">token for cancellation</param>
    [Command("plugin")]
    [SuppressMessage("ReSharper", "ForeachCanBePartlyConvertedToQueryUsingAnotherGetEnumerator")]
    public async Task Run(
        [Required] string pluginName,
        [Required] string projectName,
        [Required] string gitRepositoryUrl,
        [MinLength(1)] [Required] string gitBranch,
        [EnumArray(typeof(EditorPlatform))] string[]? editorPlatforms = null,
        [EnumArray(typeof(UEVersion))] string[]? engineVersions = null,
        string? resultPath = null,
        bool? runTest = null,
        bool? runPackage = null,
        [EnumArray(typeof(GamePlatform))] string[]? packagePlatforms = null,
        CancellationToken cancellationToken = default)
    {
    }

Conclusion

Froola includes features that I personally wanted, so it may not have all the functions that other developers are looking for.

However, I believe there must be someone in the world facing similar challenges, so I hope this tool will be helpful to those people.

At the moment, Froola has only been tested in my own environment. If you encounter any issues or if Froola does not work well in your environment, please let me know by creating an issue on GitHub.

I will continue to use Froola myself to make plugin development more efficient!

FreeBuds 3 and FreeBuds 3i are wireless earphones sold by HUAWEI.

Since FreeBuds 3i came out after FreeBuds 3, it might seem like the 3i is more powerful, but it feels more like a power-down in features other than noise cancellation.

The price also reflects this; as of writing this article, the 3 costs around 20,000 yen on Amazon, while the 3i is around 13,000 yen. This also suggests that more effort was put into making the FreeBuds 3.

I had been using FreeBuds 3 for a long time, but I recently purchased FreeBuds 3i, so I compared them to the FreeBuds 3 I had been using.

Comparisons of various features and specs have been done in many places, so it feels a bit late, thus this comparison is based on my personal impressions.

For details on the two products, please see the official pages below.

HUAWEI FreeBuds 3, HUAWEI Kirin A1 Chipset, Intelligent Noise Cancelling | HUAWEI Japan

HUAWEI FreeBuds 3i - HUAWEI Japan

Comfort

First is the difference in wearing comfort when put in the ear.

In this regard, FreeBuds 3 was a complete victory for me.

Please note that this aspect is highly dependent on the individual, so take it as a reference.

First, there is a major difference between these two models: open-fit and in-ear canal type. Therefore, their appearance is also significantly different, as shown below.

Because of this, the FreeBuds 3 fit gently into the ear, whereas the FreeBuds 3i need to be pushed in firmly.

Putting on the 3 after wearing the 3i feels really light, with less burden on the ear and a gentler feel.

However, this might change depending on the ear shape, so not everyone might have the same impression.

Looking at various reviews, some people say the 3 “falls out of the ear easily,” but I can use them even when running without them falling out at all, so they seem to fit my ears quite well.

Case Portability

A point that bothers me quite a bit is carrying the case.

In the photo below, the left is FreeBuds 3 and the right is FreeBuds 3i.

As you can see, the 3i is thicker.

I usually put it in my left pants pocket, and the thickness of the 3i is quite noticeable then.

My impression is that I wish they had reduced the thickness even if it meant making it a bit wider.

Noise Cancelling

Both of these models have active noise cancelling features.

Comparing the noise cancelling, the in-ear 3i wins hands down.

Comparing the rattling sound on a train, FreeBuds 3 reduces the sound by about 20% subjectively, whereas FreeBuds 3i reduces it by about 60-70%.

Given that the 3 manages to put noise cancelling on an open-fit design is amazing in itself, so this difference is understandable.

For the 3i, the noise cancelling effect changes considerably depending on the angle it’s worn in the ear, so it’s best to test which angle is quietest initially for better use.

Also, a minor point: the 3 resets to noise cancelling OFF each time you use it, but the 3i remembers the previous ON/OFF state, which is another difference.

However, I’ve come to think that it’s not necessarily clear which is better.

This is because if some time has passed since the last use, I might not remember if I ended with noise cancelling ON, and I’ve already found myself wondering which state the 3i is in (noise cancelling ON or ambient sound passthrough) several times after putting them on.

With the 3, it’s always OFF, so you just need to perform the routine of double-tapping the earphone to turn it ON, eliminating any confusion.

Also, the 3 requires adjusting the noise cancelling via an app, but the 3i doesn’t need it, or rather, it wasn’t in the app menu.

Connection with Smartphone

I use a HUAWEI P30 Pro as my smartphone.

Personally, I find Bluetooth devices that frequently disconnect incredibly stressful, so I prefer to stick with the same manufacturer as much as possible, which is why both are HUAWEI now.

However, I felt there was a significant difference in connection stability between these two models.

First, I don’t feel any difference in the time it takes to connect to the smartphone after taking them out of the case. Both connect instantly.

However, while playing music, the sound cutting out abruptly is something I rarely encountered with FreeBuds 3, but with 3i, it happens about once during my daily commute.

This is quite noticeable and a minus point for the 3i.

Perhaps this is influenced by the fact that only the 3 uses the Kirin A1 chipset for Bluetooth?

Sound Quality

I’m not an expert on this, so I lack the knowledge and experience to comment in detail, and don’t have much to say.

Personally, I don’t think either is bad.

Charging

FreeBuds 3 supports wireless charging, but 3i does not.

This isn’t a problem as I carry a type-c charger anyway, but wireless charging is convenient, so it would have been nice if the 3i had it too.

(I once tried charging the FreeBuds 3 using the P30 Pro’s wireless reverse charging feature, and it was convenient!)

Final Verdict

For me, the FreeBuds 3 I bought first will likely be the one I use daily from now on.

Overall, they are more user-friendly, and I really like how gently they fit my ears.

So what about the 3i? My current workplace has quite a bit of noise from air conditioning and various machines, which significantly drains concentration. So, I’m thinking of using the 3i specifically when I want to concentrate at work, letting the noise cancelling help me.

Therefore, I want to utilize both effectively according to the situation.

Bonus

The day after I bought the FreeBuds 3i, a model called “FreeBuds Pro” was announced…

This one has even more powerful noise cancelling than the 3i, and its appearance is cooler than the previous two models.

However, the price is also Pro level (around 25,000 yen on Aliexpress), so I’m debating whether to buy it…

I didn’t notice for about half a month, but WinUI 3 Preview 2 was released.

WinUI 3 Preview 2 (July 2020)

So, I tried updating the following WinUI 3 Desktop project, which I previously created using WinUI 3 Preview 1, to Preview 2.

This is a record of that process.

Since I basically just followed the official instructions, there isn’t much useful information here.

Updating the Environment

I’ll check the environment according to the description on the official page.

Ensure Windows 10 version 1803 (build 17134) or later is installed on your development computer

I have Windows 10 2004, so it’s OK.

Install Visual Studio 2019 version 16.7 Preview 3

Checking the version of the installed VS2019 Preview, it was 16.7.0 Preview 3.1.

So, I decided to proceed with this version.

When I checked for updates, 16.8.0 Preview 1.0 was available, but since updating the .NET 5 version to the latest previously caused WinUI 3 Preview 1 to fail to launch, I decided against recklessly updating to the latest version.

.NET 5 Preview 5

It seems WinUI 3 Preview 2 supports .NET 5 Preview 5. Checking the current latest version, Preview 7 was available, but to ensure it works reliably, I’ll stick with Preview 5.

However, this time, the official page description didn’t include the note “Don’t install anything newer than Preview 5” (it did last time), so perhaps Preview 7 might work as well.

Install the WinUI 3 Preview 2 VSIX package

Install the extension for VS. However, when I downloaded and tried to install it, it indicated that it was already installed, so I think it hasn’t changed from Preview 1.

Updating the Project

It seems that projects created with WinUI Preview 1 won’t work as-is with Preview 2, and some edits are necessary.

Details are written on the following page.

Now, let’s edit according to the page above.

Install Microsoft.WinUI Preview 2 via NuGet

It didn’t show up when searching in the NuGet Package Manager.

However, the workaround was also described: install it from the Package Manager Console by entering install-package Microsoft.WinUI -Version 3.0.0-preview2.200713.0.

Edit .csproj

Change the TargetFramework description from netcoreapp5.0 to net5.0.

That’s all for this file.

Edit .wapproj

Edit the package project’s configuration file (.wapproj) in a few places.

Change 1

Insert the line <AppxTargetsLocation Condition="'$(AppxTargetsLocation)'==''">$(MSBuildThisFileDirectory)build\</AppxTargetsLocation> inside the PropertyGroup after <Import Project="$(WapProjPath)\Microsoft.DesktopBridge.props" />.

Change 2

Change <Import Project="build\Microsoft.WinUI.AppX.targets" /> to <Import Project="$(AppxTargetsLocation)Microsoft.WinUI.AppX.targets" />.

Replace files in the package project’s build folder

First, create a new project. This is disposable, so creating it in a temporary folder is fine.

Use the Blank App, Packaged (WinUI in Desktop) template.

Once the project is created, copy the following two files from [Package project folder]/build/:

• LiftedWinRTClassRegistrations.xml
• Microsoft.WinUI.AppX.targets

Then, overwrite the files in the same hierarchy in your own package project.

Build and Run

After restarting VS, building, and running, it launched properly.

儀式完了。WinUI Preview1のプロジェクトをPreview2にアップデートできた。 pic.twitter.com/b509OoP6K8

— ayuma (@ayuma_x) August 5, 2020

It seems many bugs have been fixed in WinUI 3 Preview 2 compared to Preview 1, so I’d like to explore various aspects.

I use UE4 for non-game purposes at work, often alongside other applications on Windows.

A common pattern is creating the display part with UE4 and a controller-like part with WPF, then using them together.

In this pattern, the focus inevitably goes to the WPF application, leaving the UE4 window unfocused.

This caused the following problems:

• Sound cannot be heard
• GamePad input cannot be captured

I couldn’t find solutions for these for a while, but after checking the Engine source, I found countermeasures for both, so I’m writing them down as a memo.

Note that the environment where I applied these measures was 4.24, but since the implementation in that part seems unchanged in 4.25, I believe the same measures should work (though I haven’t confirmed the operation in 4.25).

Sound Playback When Unfocused

I found the following description in WindowsPlatformApplicationMisc::PumpMessages:

// if its our window, allow sound, otherwise apply multiplier
FApp::SetVolumeMultiplier( HasFocus ? 1.0f : FApp::GetUnfocusedVolumeMultiplier() );

It seems SetVolumeMultiplier is a function to set the sound volume. Here, it uses 1.0 if focused, and the value of GetUnfocusedVolumeMultiplier otherwise.

Checking the value returned by GetUnfocusedVolumeMultiplier, it was 0. That’s why I couldn’t hear anything.

However, I found that the initial value of this is taken from the Config.

static bool GUnfocusedVolumeMultiplierInitialised = false;
float FApp::GetUnfocusedVolumeMultiplier()
{
    if (!GUnfocusedVolumeMultiplierInitialised)
    {
        GUnfocusedVolumeMultiplierInitialised = true;
        GConfig->GetFloat(TEXT("Audio"), TEXT("UnfocusedVolumeMultiplier"), UnfocusedVolumeMultiplier, GEngineIni);
    }
    return UnfocusedVolumeMultiplier;

So, I wrote the following setting in Config/DefaultEngine.ini to make the value of UnfocusedVolumeMultiplier also 1.0 and confirmed the operation.

[Audio]
UnfocusedVolumeMultiplier=1.0

Success! Sound can now be heard even when unfocused.

GamePad Input When Unfocused

In UE4, GamePad input on Windows uses an API called XInput.

Searching from there, I found a class called XInputInterface. Checking its contents, it calls the XInput API, so it’s highly likely that this is where GamePad input is received.

Next, searching for where XInputInterface is used, I found it’s used in FWindowsApplication.

Then, it seems FWindowsApplication::PollGameDeviceState fires the XInput input events. Furthermore, this function is called from FSlateApplication::PollGameDeviceState().

Looking inside FSlateApplication::PollGameDeviceState(), it’s guarded by an if statement.

if( ActiveModalWindows.Num() == 0 && !GIntraFrameDebuggingGameThread && (!bRequireFocusForGamepadInput || IsActive()))
{
    // Don't poll when a modal window open or intra frame debugging is happening
    PlatformApplication->PollGameDeviceState( GetDeltaTime() );
}

There are three conditions here, and the last one, IsActive(), feels like it means having focus. However, there’s a variable here, bRequireFocusForGamepadInput, which seems directly related.

Checking its definition, it looked like the value could be changed via a console command.

static bool bRequireFocusForGamepadInput = false;
FAutoConsoleVariableRef CVarRequireFocusForGamepadInput(
    TEXT("Slate.RequireFocusForGamepadInput"),
    bRequireFocusForGamepadInput,
    TEXT("")
);

For my work purposes, I use a development build, so I tried entering the above command using Ctrl+@, and I was able to get GamePad input even when unfocused.

However, typing it every time is tedious (calling it from BluePrint is an option, but I’d rather not if possible).

Doing a bit more research, it seems console commands can also be defined in an ini file, so I decided to go with that approach.

[UE4] Special Nature of ConsoleVariables.ini - Qiita

Summary

When a UE4 application loses focus, various things become impossible, but I managed to adjust it to behave as desired.

Being able to see the engine source makes this kind of investigation easier, which is really great!

The other day, the Preview of UE4 4.25 became available for download.

(As of writing this article, it is Preview 3)

The contents are described on the following page.

Unreal Engine 4.25 Preview - Unreal Engine Forums

Among the many updates, the following item caught my eye. (The FProperty Refactor… part)

The content is “From 4.25, UProperty has become FProperty and is no longer a UObject.”

Until now, UProperty inherited from UObject. In UE4, classes inheriting from UObject have a naming convention starting with U. Hence the class name UProperty.

• Reference [Coding Standard](https://docs.unrealengine.com/en-US/Programming/Development/CodingStandard/index.html)

Since it no longer inherits from UObject (it now inherits from a class called FField), the class name changed to FProperty.

What is UProperty?

First of all, what is the UProperty (FProperty) that was changed this time? It is information about member variables or function arguments of a C++ class created by the UE4 property system.

Details are explained thoroughly on the official page below.

• Reference [Unreal Property System (Reflection)](https://www.unrealengine.com/en-US/blog/unreal-property-system-reflection)

Furthermore, UProperty is derived into UStringProperty, UBoolProperty, etc., depending on the property type.

It is used for various purposes within the UE4 engine, but from the perspective of us UE4 users, it is used when we want to do things like the following:

• Access properties of an object defined in Blueprint from the C++ side
• Create a custom serializer for an object
• Manipulate object properties by specifying the property name

Using the UE4 property system with UProperty like this allows for building mechanisms to manipulate objects more flexibly.

(Reference) Example of getting the values of all properties of an object

The following code gets the values of all properties of Obj.

(* This example assumes that the type of all properties is uint8)

for (TFieldIterator<UProperty> PropIt(Obj->GetClass()); PropIt; ++PropIt)
{
    UProperty* Property = *PropIt;

    // Get property name
    FString PropertyName = Property->GetName();

    // Get property value
    uint8 CurrentPropAddr = *(PropIt->ContainerPtrToValuePtr<uint8>(Obj));
}

Why was it changed to FProperty?

I haven’t followed all the information, so this includes a lot of personal speculation, but I think a major reason was to remove it from the management scope of GC (Garbage Collection) by making it not a UObject.

When an instance is allocated in memory as a UObject, the GC checks whether the instance should be destroyed at the appropriate timing. Instances determined to be destroyable by the check result will be destroyed.

If the number of UObject instances in the entire project is large, the search time for the GC to check for destroyable targets increases. (Because the absolute number of items to check increases)

Increased search time can lead to severe adverse effects, such as the frame rate dropping sharply at the GC check timing (experienced this).

Therefore, although it depends on the case, having fewer UObject instances is generally considered better for performance.

Building conventional (up to 4.24) C++ code with 4.25

I tried building the code that enumerates object properties written earlier with 4.25.

It didn’t result in an error, but the following Warning appeared.

warning C4996: UProperty has been renamed to FProperty Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.

The Warning says, “UProperty has changed to FProperty, so change it quickly.” However, being a Warning means it will still work for now.

The reason for this can be understood by looking at the definition of UProperty in 4.25.

It was defined as follows in the header DefineUPropertyMacros.h.

All UProperty are set to be replaced with FProperty.

Since DEPRECATED_MACRO is also embedded during this replacement, the previous Warning seems to appear.

So, does this mean that code using UProperty can be left as is for the time being? It seems not.

It’s just being replaced; the UProperty written in the C++ code is already not a UObject, so code like the following will result in an error.

UProperty* Property = *PropIt;

if (UBoolProperty* BoolProperty = Cast<UBoolProperty>(Property))
{
    bool bValue = BoolProperty->GetPropertyValue(Obj);
}

The code above means “if the type of a certain property is bool, get the value as bool.”

In this, Cast<UBoolProperty>(Property) is used to check if UProperty can be cast to UBoolProperty, but Cast can only be used for UObjects.

Therefore, building this code results in an error.

(* In the example above, the same can be achieved with FProperty by replacing it with the CastField function)

For the same reason, IsValid and similar functions are likely to become unusable.

Additionally, if there are parts where UProperty is passed as a UObject argument to functions, they need to be changed appropriately.

Summary

• From 4.25, UProperty became FProperty
• Replace箇所 where U~Property is used in C++ with F~Property
• Check not only for replacement but also if UProperty is used as UObject anywhere

Bonus

Since the old UProperty definition remains in UnrealTypePrivate.h, it feels like UProperty could be extended for a while longer by including this header, but it seems better not to use it as it would deviate from the proper flow.

ObjectDeliverer: ayumax: Code Plugins - UE4 Marketplace

The main implementation this time is the enhancement of the Object Json serialization feature.

I previously wrote about this as a current issue in a past article.

Let’s Serialize UObjects to Json - AYU MAX

This release implements features to improve the issues mentioned above.

Implemented Items

The following two items have been addressed.

Property Name Conversion

How to change property name when serializing UObject with Json(Version 1.5.0 or later) · ayumax/ObjectDeliverer Wiki · GitHub

By implementing the IODConvertPropertyName interface in the class to be serialized, property name conversion is now possible.

Usage Example)

// Before conversion
{
    "Prop2": 1,
    "ObjProp":
    {
        "ValueB": 0,
        "ValueA": 0
    },
    "ObjProp2":
    {
        "ValueC": "abc"
    }
}

// After conversion
{
    "IntProperty": 1,
    "ObjectPropertyA":
    {
        "ValueB": 0,
        "ValueA": 0
    },
    "ObjectPropertyB":
    {
        "ValueC": "abc"
    }
}

Using this feature allows the following:

• Resolution of the case-insensitivity issue with FName
• Using different property names between the target class and Json

It’s subtle but should be genuinely useful.

Selection of Serialization Format

Selection of Json serialize method in ObjectDeliveryBoxUsingJson(Version 1.5.0 later) · ayumax/ObjectDeliverer Wiki · GitHub

This is the item I put the most effort into implementing this time. It is now possible to embed the class name within the Json.

// Conventional method without class name
{
    "Prop": 1
}

// Method embedding class name
{
    "Type": "SampleClassName",
    "Body":
    {
        "Prop": 1
    }
}

This enables the following:

• Support for serialization and deserialization of multiple classes with a single DeliveryBox
• Restoration is possible even when an instance of a derived class is in an Object type property

Also, while the method of embedding class names alone has the advantages mentioned above, it also has the disadvantage of increasing the character count (data size). Therefore, an option is provided to switch between the conventional method and the new method depending on the Object type.

For example, it is possible to use it in a way where only certain properties embed the class name, as shown below.

{
    "Prop2": 1,
    "ObjProp":
    {
        "Type": "CustomObjB_C",
        "Body":
        {
            "ValueB": 2,
            "ValueA": 3
        }
    },
    "ObjProp2":
    {
        "ValueC": "abc"
    }
}

Summary

This time, I focused on enhancing the Object Json serialization feature.

There are still other features I want to implement, so please look forward to the steadily improving ObjectDeliverer.

Last year’s reflection is here:

Looking Back on the Year and Resolutions for the New Year - AYU MAX

What Did I Do?

VR Development

A major event this year was starting VR development at work.

VR isn’t that rare in the world anymore, but it was the first attempt at our company, so I heard various things (expectations, anxieties) from those around me, but I was able to achieve certain results.

How far this can be expanded in the future is still a challenge, so I will continue to work hard next year.

AR Development

This year, I also started AR development with UE4 and ARKit. This is still at the stage of tracing the basics of developing with ARKit in UE4, but I’m satisfied because I was able to do everything I wanted to do.

I’d like to think about how to incorporate it into work from next year onwards.

I also want to try not just mobile devices but also HoloLens and Magic Leap.

Also, I want to do something in personal development.

UE4 Marketplace

As for private activities, listing on the Marketplace was a big event.

At the beginning of the year, I wondered if I could list things myself, started researching, and somehow ended up releasing four code plugins.

I put my soul into making all of them, so I’d be happy if many people could use them.

Also, some projects are open source, so interacting with others on GitHub was fun. Since it’s in English, conversations wouldn’t work without the help of Google Translate, but thanks to that, I feel like my ability to generate Japanese that’s easy to translate has improved!

The ratings on the Marketplace are also very encouraging, so I’m grateful.

A point of reflection is that I got carried away and released four plugins, and maintenance hasn’t kept up.

There are ideas I want to implement for each plugin but haven’t been able to, so I want to work on those next year.

Presentations

This year, my initial goal was to give presentations somewhere twice, but in the end, it was only once.

But I’m grateful just for getting such an opportunity, and I think I grew a little, so I’d like to continue doing it if I have the chance in the future.

For that, I need to keep gathering material regularly.

Summary

For me, it was a very rich year compared to previous years.

I want to do even more various things next year than this year, so I’ll do my best!

ARKitを使ったリモートタッチパッドの実験。精度がまだ荒いけど基本的には上手くいってる。#ue4 #arkit pic.twitter.com/Kt5E5fPCQn

— ayuma (@ayuma_x) December 26, 2019

When you trace your finger over the iPad, a line is drawn on the display inside the iPad, but when you remove the iPad, the same line is drawn on the actual display.

This might be confusing at first glance, but actually, nothing is added to the iPad side display. Only the camera feed is shown on the iPad’s display.

What Did I Want to Do?

A while ago, I was creating AR content using an iPad for work, and that’s when I thought, “I want to think about uses for AR other than [overlaying some display].”

I’m sure someone has probably done something similar to what I did this time, but I tried it because “there’s meaning in thinking up and shaping ideas yourself.”

In this experiment, the iPad is used as an input device. The idea is to create a remote touch display by skillfully using ARKit’s image detection and object detection features.

Things to Do

I’ve summarized the things that need to be done to shape this idea below.

This time, the iPad application is built based on UE4 (4.24)’s Handheld AR Blueprint template.

• Know the position and angle of the display shown on the iPad
• Know the position of the finger touching the iPad as relative coordinates on the display
• Send coordinates from the iPad to the PC side
• Draw lines on the PC side according to the sent coordinates

Know the position and angle of the display shown on the iPad

First, we need to know the position of the display.

I considered two methods for knowing the position: “marker method using images” and “object detection,” but this time I chose object detection.

I wrote an article before about how to use object detection with UE4+ARKit, so please take a look if you’re interested.

Using ARKit’s Object Detection Feature in UE4 - AYU MAX

First, I tried creating detection data by scanning the display, but the display’s shape had few feature points, and I couldn’t get satisfactory results.

So, I created detection data for the cluttered area next to the display and decided to determine the display’s position and angle based on the relative position from that point.

Naturally, this method breaks down if the positional relationship with the display changes.

↓ Scanning process

Knowing the display’s position and angle after scanning is exactly as described in the previous article mentioned above.

Know the position of the finger touching the iPad as relative coordinates on the display

Next, let’s think about how to know the position of the finger touching the iPad.

In the implementation within the UE4 AR template used this time, there is logic (WorldHitTest function in BP_ARPawn) to check if there is an Actor at the touched point.

I thought about tweaking this to know the display’s position.

First, prepare a flat mesh that is the same size as the real-world display and place it in the level. This mesh doesn’t need to be visible, so I applied a transparent material.

Next, on every Tick, move this mesh to the position and angle where ARKit detects the real display.

Next, looking inside the WorldHitTest function, you can see that the LineTraceForObjects node is used.

LineTraceForObjects can determine if there is a 3D object between two specified points.

Combining this node with the Deproject Screen to World placed just before it allows identifying the 3D object placed at the touched location from the screen coordinates (x, y) on the iPad when touched.

Using this mechanism, we can determine whether the mesh placed earlier is being touched by a finger.

However, what we want to know this time is where on the display is being touched, so we need more detailed information.

Therefore, I tried adding a part that feeds the value returned by LineTraceForObjects into Find Collision UV.

This allows us to get information down to which part (UV coordinates) of the surface of the touched 3D object.

• To use Find Collision UV, you need to check Support UV From Hit Results in Project Settings. (Turning it ON probably increases processing load)

Send coordinates from the iPad to the PC side

We will send coordinates from the iPad to the PC (Windows) via socket communication (UDP).

We use ObjectDeliverer for UDP transmission. For ObjectDeliverer, please see past articles.

First, create a container for sending. It’s simple, just define a Blueprint like the one below. I named the object TouchPosition. Only variables for X and Y coordinates are defined.

Initialize ObjectDeliverer to send this object.

↓ Like this.

And at the place where the UV of the flat mesh is obtained with Find Collision UV, send the coordinates to the PC using UDP as follows.

With this, every time the display shown on the iPad is touched, the coordinates of where on the display is being touched are continuously sent to the PC.

Draw lines on the PC side according to the sent coordinates

The PC side isn’t doing anything complicated; it just receives UDP using ObjectDeliverer, extracts the sent X, Y coordinates, and then draws lines using UMG.

This part is not crucial this time, so I’ll omit it.

Thoughts After Trying It

I was able to achieve the movement exactly as I had thought.

Although nothing other than the camera feed is displayed on the iPad, I was able to create something interesting by utilizing ARKit’s features. I feel that this kind of usage holds various possibilities.

I’ll try to shape new ideas when they come up.

Unreal Engine 4 (UE4) Advent Calendar 2019 - Qiita

I have a plugin called ObjectDeliverer published on the Marketplace.

This article is about the “function to serialize objects to Json” implemented in this plugin.

For example, if you have a Blueprint (inheriting from Object) with Variables defined like this,

It will be converted into a Json string like this.

{
    "IntValue":0,
    "BoolValue":false,
    "StringValue":"ABC"
}

Since Json is represented as a string, it can be saved directly to a text file. Also, being a string, its content is easily understandable by humans.

By also creating a function to restore objects from this Json, saving and restoring objects via Json becomes possible.

In ObjectDeliverer, by sending and receiving this Json string over communication, objects can be delivered to another location (another application).

Handling Json in UE4

UE4 comes standard with modules named Json and JsonUtilities.

Json Module

A module that supports reading and writing Json. However, it cannot be used in Blueprints and is only supported in C++.

It can be used like this.

Json Object -> Json String

// Create FJsonObject (container for Json data)
TSharedPtr<FJsonObject> jsonObject = MakeShareable(new FJsonObject());

// Add properties to jsonObject

// Create a Writer to write Json to FString
FString OutputString;
TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&OutputString);

// Write Json to FString
FJsonSerializer::Serialize(jsonObject.ToSharedRef(), Writer);

// Json string is in OutputString, do something with it

Json String -> Json Object

// FString containing the json string
FString jsonString;

// Create a Reader to read Json from FString
TSharedRef<TJsonReader<TCHAR>> JsonReader = TJsonReaderFactory<TCHAR>::Create(jsonString);

// Create FJsonObject (container for Json data)
TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());

// Read from Json string into Json object
FJsonSerializer::Deserialize(JsonReader, JsonObject);

// Extract properties from jsonObject and do something

Get, Set Json Object Values

TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());

// Set values
JsonObject->SetNumberField("IntValue", 1);
JsonObject->SetBoolField("BoolValue", true);
JsonObject->SetStringField("StringValue", "ABCDEFG");

// Get values
int32 intValue = JsonObject->GetIntegerField("IntValue");
bool boolValue = JsonObject->GetBoolField("BoolValue");
FString stringValue = JsonObject->GetStringField("StringValue");

// Array and Object properties can also be Get, Set

Knowing the above patterns should cover most cases.

JsonUtilities Module

Separate from the Json module, there is a JsonUtilities module. This is a support module that uses the Json module.

Inside it, there is a class called FJsonObjectConverter, which sounds like it could convert objects to Json based on the name.

However, looking inside, it was a class that provides mutual conversion between UStruct and JSON. What I want to achieve is mutual conversion between UObject and JSON, so it was slightly different. Disappointing.

By the way, this functionality can be used by calling the following two static methods.

• UStructToJsonObject
• JsonObjectToUStruct

Creating UObject and JSON Mutual Conversion Yourself

First, although I wrote creating yourself, it’s almost the same as the implementation of the FJsonObjectConverter class.

https://github.com/ayumax/ObjectDeliverer/blob/master/Plugins/ObjectDeliverer/Source/ObjectDeliverer/Private/Utils/ObjectJsonSerializer.cpp

If you compare it side-by-side with the FJsonObjectConverter class, you’ll see they are almost identical.

I only added the UObject handling process to the implementation of the FJsonObjectConverter class.

The complete code is at the link above, but I will explain only the representative parts for realizing the functionality.

UObject -> JsonObject

This is the process of converting UObject to Json.

First, use TFieldIterator to get a list of UObject properties (including variables defined in Blueprints) and process them sequentially.

In a pure C++ program not using Unreal Engine, this kind of thing (reflection) is not possible. (Although I’ve seen articles discussing it for future C++ plans)

In UE4 C++, reflection is apparently called the property system.

Details are written here. Unreal Property System (Reflection)

Once property information (UProperty) is obtained, the property name and the actual property value can also be retrieved using it.

After that, Json properties can be created in the same way as setting properties on a Json object described earlier.

However, since nested structures where a UObject contains a UObject property also need to be considered, the processing here is done recursively.

TSharedPtr<FJsonObject> UObjectJsonSerializer::CreateJsonObject(const UObject* Obj)
{
    TSharedPtr<FJsonObject> JsonObject = MakeShareable(new FJsonObject());

    if (!Obj) return JsonObject;

    // Process the list of UObject properties sequentially
    for (TFieldIterator<UProperty> PropIt(Obj->GetClass(), EFieldIteratorFlags::IncludeSuper); PropIt; ++PropIt)
    {
        UProperty* Property = *PropIt;

        // Get property name
        FString PropertyName = Property->GetName();

        // Get property value
        uint8* CurrentPropAddr = PropIt->ContainerPtrToValuePtr<uint8>((UObject*)Obj);

        FJsonObjectConverter::CustomExportCallback CustomCB;
        CustomCB.BindStatic(UObjectJsonSerializer::ObjectJsonCallback);

        // Set to Json property
        // FJsonObjectConverter::UPropertyToJsonValue is processed recursively
        JsonObject->SetField(PropertyName, FJsonObjectConverter::UPropertyToJsonValue(Property, CurrentPropAddr, 0, 0, &CustomCB));
    }

    return JsonObject;
}

JsonObject -> UObject

This is the process of restoring UObject from Json.

Similar to the process of converting UObject to Json, this also uses the UE4 property system to dynamically obtain the property list and process it sequentially.

The flow is to obtain UProperty, get the property name from it, then get the Json property from JsonObject and set it to the UObject’s property.

This function is also called recursively to support nested structures.

bool UObjectJsonSerializer::JsonObjectToUObject(const TSharedPtr<FJsonObject>& JsonObject, UObject* OutObject)
{
    auto& JsonAttributes = JsonObject->Values;

    // Get the property list of the UObject to restore
    for (TFieldIterator<UProperty> PropIt(OutObject->GetClass()); PropIt; ++PropIt)
    {
        UProperty* Property = *PropIt;

        // Get Json property from JsonObject using property name as key
        const TSharedPtr<FJsonValue>* JsonValue = JsonAttributes.Find(Property->GetName());
        if (!JsonValue)
        {
            continue;
        }

        if (JsonValue->IsValid() && !(*JsonValue)->IsNull())
        {
            // Get pointer to UObject property
            void* Value = Property->ContainerPtrToValuePtr<uint8>(OutObject);

            // Read value from Json property and set to UObject property
            // JsonValueToUProperty is processed recursively
            if (!JsonValueToUProperty(*JsonValue, Property, Value))
            {
                return false;
            }
        }
    }

    return true;
}

Some Problems

In ObjectDeliverer, the above ObjectJsonSerializer is used within UObjectDeliveryBoxUsingJson to serialize UObjects to Json.

I also use this feature in my own projects, and while it generally works as intended, some points for improvement have emerged.

Need to Specify UObject Type During JsonObject -> UObject

As explained in the JsonObject -> UObject process description above, the UObject type must be predetermined for this process.

Looking at the Json format, you can see that Json has no mechanism to store the object type.

Therefore, just looking at the Json, it’s impossible to know which UObject to restore to, so the UObject type needs to be decided beforehand.

Because of this, communication using ObjectDeliverer’s UObjectDeliveryBoxUsingJson currently only supports sending and receiving a single UObject type. It cannot handle multiple types with normal usage.

Potential Information Loss When Having UObject Type Properties

This problem also arises because Json cannot store type information, similar to the previous issue.

For example, consider an environment where ClassB inherits from ClassA.

In that environment, the UObject saved to Json has a property of type ClassA.

An instance of ClassB can be put into this ClassA type property.

When saving to and restoring from Json at this time, properties that exist only in ClassB but not in ClassA will not be restored…

This is because the UProperty indicating ClassA type is used during property restoration.

This can be avoided if understood and used correctly, but it might lead to hard-to-notice bugs.

Json Property Name May Not Match the Specified String

For example, when converting the Blueprint shown in the example at the beginning to Json, it might end up like this.

{
    "Intvalue":0,
    "boolValue":false,
    "Stringvalue":"ABC"
}

At first glance, it seems to be working correctly, but the case of the property names is wrong.

This is because UE4 properties use the FName type.

FName

As mentioned on the page above, FName is case-insensitive, which causes this phenomenon.

No problem occurs if both saving to and restoring from Json use ObjectDeliverer within the same UE4 project, but problems (property name mismatch) may arise when exchanging Json with applications created in other development environments.

Countermeasures for Problems

Actually, I wanted to release an update with countermeasures for the above problems by the time this article was published, but it’s not complete yet… (Testing isn’t finished)

The work is being done on the following branch, and the implementation is almost complete, so you can see what it looks like.

The Json serialization part has undergone significant changes.

https://github.com/ayumax/ObjectDeliverer/tree/Remodeling-ObjectJsonSerializer

The countermeasures taken are the following two.

Write UObject Type Information into Json Based on User Selection

The first and second problems can be resolved by writing the UObject type information into the Json.

However, doing so changes the Json format from before, breaking compatibility, and purely increases the Json string length, making the data size larger, which might be undesirable from the perspective of data transmission, ObjectDeliverer’s main purpose.

Therefore, a mechanism is introduced where the user can choose whether or not to write type information into the Json for each UObject type.

This allows creating the same Json as before under the condition that only one type is sent/received and UObject properties using class inheritance are not created.

Also, by choosing to write the actual instance’s type information, reliable restoration can be achieved when saving and restoring UObjects.

Specifically, the following differences arise in the created Json.

When not writing type information (implementation up to now)

{
    "IntValue":0,
    "BoolValue":false,
    "StringValue":"ABC"
}

When writing type information

Example when serializing a UObject named NewBlueprint

{
    "Type":"NewBlueprint",
    "Body" :
    {
        "IntValue":0,
        "BoolValue" : false,
        "StringValue" : "ABC"
    }
}

Definition of an Interface to Implement Property Name Conversion Function

Since conversion from FName type to FString type might not result in the intended string if left to automatic conversion, an interface for property name conversion was added so users can define it reliably themselves.

If this interface is implemented on the UObject to be serialized, the custom property name conversion function will be used for writing and reading Json properties.

If the interface is not implemented, automatic FName -> FString conversion will occur as before.

By using this feature, it becomes possible to create Json properties with completely different strings, so I believe it has uses beyond solving the current problem.

Finally

This time, I wrote about serializing UObjects to JsonObjects.

The source code of FJsonObjectConverter from the JsonUtilities module mentioned in this article contains implementations that are helpful not only for creating UObject serialization to formats other than Json but also for utilizing the UE4 property system, so please take a look.

Also, the ObjectDeliverer I have published aims to easily create communication using TCP/IP, UDP, etc., using only Blueprints. It’s available for Free, so please try it if you like.

ObjectDeliverer: ayumax: Code Plugins - UE4 Marketplace

This article is for the 10th day of the C# Advent Calendar 2019.

An application I create for work (made with .NET Framework) has a user plugin (DLL) loading feature.

This feature allows users to add functionality to the development tool by creating their own DLLs according to rules.

This feature involves the development tool loading the relevant DLLs once, querying them, and then unloading DLLs that don’t need to be loaded.

When porting this development tool to .NET Core, a different method than .NET Framework was required, which I will introduce.

AppDomain Cannot Be Used

In .NET Framework, AppDomain was used to unload assemblies.

Besides the application’s default AppDomain, another domain is created, and assemblies are loaded within it.

After using the assembly’s functionality via interfaces, etc., the AppDomain’s Unload method is called to unload the assembly along with the domain.

However, in .NET Core, only one AppDomain is supported, so this method cannot be used.

Instead, .NET Core provides AssemblyLoadContext for unloading assemblies, so I will try using this.

What is AssemblyLoadContext?

AssemblyLoadContext, similar to AppDomain, allows assembly loading within a closed scope.

Assemblies loaded into it can be unloaded together by calling the AssemblyLoadContext’s Unload method, but there are points to note.

Note 1: Set isCollectible to true

The isCollectible constructor argument of AssemblyLoadContext must be set to true.

This is because, for performance reasons, it defaults to false.

Calling Unload while it’s false will cause an exception.

System.InvalidOperationException
  HResult=0x80131509
  Message=Cannot unload non-collectible AssemblyLoadContext.
  Source=System.Private.CoreLib

Note 2: Unloading is Cooperative, Not Forced

Calling the AssemblyLoadContext’s Unload method only initiates the process; it does not complete it immediately.

Unloading completes only when the following conditions are met:

• No threads have methods from assemblies loaded into the AssemblyLoadContext on their call stacks.
• Types from assemblies loaded into the AssemblyLoadContext, instances of those types, and the assemblies themselves are no longer referenced.

In other words, without careful design, unloading might never complete.

Creating a Sample

Let’s actually implement a mechanism to unload assemblies.

1. Create the Class Library to be Loaded

Anything will do, but I defined a simple class like the one below. I specified netstandard2.1 for the TargetFramework. Make sure this is output as ClassLibrary1.dll.

namespace ClassLibrary1
{
    public class Class1
    {
        public void Hello(int arg)
        {
            Console.WriteLine(arg);
        }
    }
}

2. Implement AssemblyLoadContext on the Loading Side

First, create an AssemblyLoadContext for loading assemblies. This time, I created a simple class like the one below.

The important part is setting isCollectible to true in the constructor. Setting this flag to true enables unloading support.

class TestAssemblyLoadContext : AssemblyLoadContext
{
    public TestAssemblyLoadContext() : base(isCollectible: true)
    {
    }
}

3. Implement the DLL Loading & Unloading Mechanism

Create a function to load the ClassLibrary1.dll created earlier, call a function, and then unload it.

[MethodImpl(MethodImplOptions.NoInlining)]
static void ExecuteAndUnload(string assemblyPath, out WeakReference alcWeakRef)
{
    // Create AssemblyLoadContext to load the assembly
    var alc = new TestAssemblyLoadContext();

    // Load the assembly
    Assembly a = alc.LoadFromAssemblyPath(assemblyPath);

    // Set a weak reference to detect unloading from outside
    alcWeakRef = new WeakReference(alc, trackResurrection: true);

    // Call function via reflection
    var type = a.GetType("ClassLibrary1.Class1");
    var instance = Activator.CreateInstance(type);
    var helloMethod = type.GetMethod("Hello");
    helloMethod.Invoke(instance, new object[] { 1 });

    // Perform unload
    alc.Unload();
}

Add MethodImplOptions.NoInlining to the function just in case.

This is to prevent the ExecuteAndUnload function from being inlined.

Unloading using AssemblyLoadContext cannot be performed if types or instances within the AssemblyLoadContext are referenced from outside.

Therefore, if inlined, there is a risk that references might remain in the caller of ExecuteAndUnload (in this case, the Main function).

(* However, in the sample created this time, unloading completed even with MethodImplOptions.AggressiveInlining)

[MethodImpl(MethodImplOptions.NoInlining)]
static void ExecuteAndUnload(string assemblyPath, out WeakReference alcWeakRef)

Create an instance of AssemblyLoadContext and load the assembly into it.

assemblyPath contains the file path of the dll or exe.

 // Create AssemblyLoadContext to load the assembly
var alc = new TestAssemblyLoadContext();

 // Load the assembly
Assembly a = alc.LoadFromAssemblyPath(assemblyPath);

Keep a weak reference to the AssemblyLoadContext.

This reference is used by the caller of ExecuteAndUnload to determine if unloading is complete.

// Set a weak reference to detect unloading from outside
alcWeakRef = new WeakReference(alc, trackResurrection: true);

For this experiment, I specified the class name to create an instance and called the function using reflection.

When creating a plugin, it’s more practical to have ClassLibrary1.dll implement an interface defined in a separate assembly and call it via the interface.

// Call function via reflection
var type = a.GetType("ClassLibrary1.Class1");
var instance = Activator.CreateInstance(type);
var helloMethod = type.GetMethod("Hello");
helloMethod.Invoke(instance, new object[] { 1 });

Call the Unload method to perform unloading.

// Perform unload
alc.Unload();

Trying to Unload Assembly Using ExecuteAndUnload

I implemented the Main function as follows and actually tried dynamically loading and then unloading the assembly.

This time, I checked if unloading was successful by seeing if the dll file could be deleted. If unloading is not complete, Delete will fail.

static void Main(string[] args)
{
    var myDirectory = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

    // Path to the assembly (dll) to load
    var assemblyPath = Path.Combine(myDirectory, @"..\..\..\..\ClassLibrary1\bin\Debug\netstandard2.1\ClassLibrary1.dll");

    // Load the assembly and call the function
    ExecuteAndUnload(assemblyPath, out WeakReference alcWeakRef);

    try
    {
        File.Delete(assemblyPath);
    }
    catch(UnauthorizedAccessException)
    {
        Console.WriteLine("Cannot delete because unloading is not complete");
    }

    // Wait until unloaded
    int counter = 0;
    for (counter = 0; alcWeakRef.IsAlive && (counter < 10); counter++)
    {
        GC.Collect();
        GC.WaitForPendingFinalizers();
    }

    if (counter < 10)
    {
        // At this stage, it's unloaded, so it can be deleted
        File.Delete(assemblyPath);

        Console.WriteLine("Unload successful");
    }
    else
    {
        Console.WriteLine("Unload failed");
    }

    Console.ReadKey();
}

After calling ExecuteAndUnload to load the assembly -> execute the function within the assembly -> unload, I checked the completion of unloading using WeakReference.

To remove references to the AssemblyLoadContext being unloaded, I forced garbage collection using GC.Collect().

In the results I tried this time, when unloading was successful, the value of counter was 2 when exiting the loop.

Also, immediately after exiting ExecuteAndUnload, deleting the dll always failed, but after waiting for unloading to complete, deletion succeeded, confirming that assembly unloading was successful.

Trying to Be Mean

The above sample was a case where unloading succeeded, but I tried a bit to see what kind of description would cause it to fail.

Case Where Thread Inside Loaded Assembly Doesn’t End

I changed the content of the Hello method in ClassLibrary1.dll as follows.

Once Hello is called, a non-terminating thread is created with Task.Run.

public void Hello(int arg)
{
    Task.Run(() =>
    {
        while(true)
        {
            Thread.Sleep(1);
        }
    });
    Console.WriteLine(arg);
}

Loading this assembly and attempting to unload it failed.

Trying to Hold AssemblyLoadContext

Created a static variable like the one below in the class that loads the assembly,

private static AssemblyLoadContext assemblyLoadContext;

And tried holding the reference to TestAssemblyLoadContext within ExecuteAndUnload.

var alc = new TestAssemblyLoadContext();
assemblyLoadContext = alc;

This case also failed to unload because a reference to AssemblyLoadContext remained.

Trying to Hold Type Within Loaded Assembly

Created a static variable like the one below in the class that loads the assembly,

private static Type class1Type;

And tried holding the Type of ClassLibrary1.Class1 within ExecuteAndUnload.

var type = a.GetType("ClassLibrary1.Class1");
class1Type = type;

This case also failed to unload because a reference to the Type remained.

Summary

Even in a .NET Core environment, it was possible to load and unload assemblies using AssemblyLoadContext.

However, if you forget to clean up the assembly being unloaded, unloading will fail, so it seems best to perform the process from loading to unloading within a closed area as much as possible.

Realistically, cases requiring assembly unloading might not be that common, but knowing about it might be useful somewhere.

Sample Project

The sample created this time is in the repository below.

Reference Sites

How to use and debug assembly unloadability in .NET Core

Collectible assemblies in .NET Core 3.0 | StrathWeb. A free flowing web tech monologue.

Speaking of a few days ago, a new version of ObjectDeliverer has been released. (I meant to write a blog post but forgot)

ObjectDeliverer: ayumax: Code Plugins - UE4 Marketplace

Added Feature to Get Sender IP Address

This feature is only available when the communication protocol is set to TCP/IP Server or UDP Receiver.

With these two communication protocols, the sender can potentially have multiple IP addresses.

However, in previous versions, there was no way to know where the transmission came from. (Strictly speaking, TCP/IP Server had a function to distinguish and recognize the sender, but there was no function to know the IP address.)

This is because, to realize ObjectDeliverer’s design philosophy of “being able to switch and use different communication protocols in the same way,” I didn’t want to include features too specific to individual protocols.

However, realistically, there were situations where it was inconvenient not to have it, so I implemented it this time.

Below is an example specification.

Information can be obtained by using the ClientSocket object of the ReceiveData event through the GetIPV4Info interface.

The newly added GetIPV4Info interface includes GetIPAddress, which retrieves the IP address as an array of 4 uint8 elements, and GetIPAddressinString, which retrieves it as a string in the format “...”.

For communication protocols other than the supported ones, the ClientSocket object of the ReceiveData event does not implement the GetIPV4Info interface, so casting to the interface will fail.

I want to maintain the concept of being able to use things in the same way as much as possible in the future, but I plan to make individual protocol-specific features available via interfaces.

This was a local event for “.NET Conf 2019” held on October 5th. I participated this time and gave a lightning talk (LT).

Presentation Content

Since this was a .NET Core 3.0 event, I thought content related to Core 3 would be better, so I decided to talk about WPF, which was added in .NET Core 3.0.

The materials for the presentation I gave are below.

Also, the presentation content from the day was made into a video. Thank you, @kekyo2.

I thought I should reflect on my presentation, so I tried watching the video that was shared, but watching a video of myself speaking is incredibly embarrassing!!

I was so embarrassed I haven’t been able to watch the whole thing yet.

I want to watch it again later after composing myself.

About Theme Selection

I don’t have enough knowledge to talk about the deep parts of .NET, so I thought it would be better to present practical experiences rather than discussing language specifications.

Thinking about what to do, I remembered trying out WPF using the .NET Core 3 preview around Golden Week this year, and thought, “This will be good presentation material!” so I chose WPF as the theme.

.NET Core 3.0のWPFを試している進捗です。
HoudiniみたいなノードベースUIを作成中。なかなか面白いのでこのまま作りこむ予定。#WPF #dotnetcore pic.twitter.com/C9TNKZOUHS

— ayuma (@ayuma_x) May 6, 2019

So, I first listed the content I wanted to talk about for this theme in bullet points to create a foundation for the presentation materials.

Then I realized one crucial thing.

It’s not interesting…

I tried writing plainly about how WPF became usable in .NET Core 3.0, but the content was thin and uninteresting.

At this point, I worried, “Was the theme a bad choice??”, “Would talking about C# 8, which I tried at the same time, have more content??” But I decided that since I use WPF regularly, I could answer questions if something came up, so I stuck with it.

Therefore, although not specific to WPF, I added topics that seemed highly relevant, like being able to include the Runtime in the application and creating a single executable file. I also made a video showing how it actually works to let people see it in action.

After writing everything and practicing the presentation a few times, it came out to be about 13-14 minutes, so I decided to go with it.

I still felt the content might be a bit thin, but I couldn’t add more at that point, so I decided to brace myself. (Now, a few days after the event ended, I have more ideas for Core 3 + WPF. Well, that’s how it goes.)

Presentation Day

My turn was towards the end, and until then, I listened with great interest to the talks by the other speakers.

@kekyo2’s talk on asynchronous programming personally hit home, as just a little before that day, I had been working on eliminating Task.Wait from code I wrote long ago.

I’m very interested in async iterators, so I definitely want to try writing them soon.

This was my second presentation, following an LT last year, but today I felt calmer than last time and was optimistic, thinking, “Maybe I won’t panic today.”

However, my turn was right after a break, so there was some waiting time. As I prepared my PC and waited, I gradually got nervous, and my stomach started hurting lol.

But today, my goals were “Don’t speak too fast, speak slowly” and “Look at the faces of the listeners while speaking,” so I tried my best to stay calm and speak bit by bit, ensuring I could at least keep those promises.

There was an accident, though; the video output of the MacBook I brought that day was acting up, and the projector turned to static several times, causing me to panic slightly. But in the end, the timing was just right, and I finished, which was good.

Self-evaluation: I think I presented better than last time!

The stomach ache also disappeared after the presentation ended w

Reflections for Next Time

Listening to the other presentations, I felt that everyone deeply understood the content they were speaking about.

For my own content, I had researched and tried things out, but I wasn’t speaking with perfect confidence everywhere. There were parts I wasn’t sure about, and although I actually had more material, I cut out those parts I lacked confidence in.

I felt that to give a confident presentation, a vast amount of knowledge backed by everyday technical practice is necessary.

I can’t change the time I’ve spent so far, but I feel that from now on, when learning, I want to delve one step deeper into the content than before and gain knowledge I can confidently share with others.

This time too, I learned a lot, and finishing the LT boosted my confidence, so I felt glad I participated.

Bonus

After the event ended, there was a rock-paper-scissors tournament, and incredibly, I received a Kinect V1 from matsujirushi-san!!!

I tried it immediately, and it’s fun! Motion capture was a first-time experience for me, so it opens up possibilities.

I’m currently considering what to create with this.

Kinect V1を動かせた。
何これ！控えめに言って面白い！ pic.twitter.com/RzjbdSIXbR

— ayuma (@ayuma_x) October 7, 2019

Recently, while browsing Twitter, I saw someone reviewing a laptop they borrowed through the Dell Ambassador Program. I decided to register and apply myself on a whim.

To my surprise, I actually won and got the opportunity to borrow the XPS 15 2-in-1, so I’d like to review it.

I’m sure many others have reviewed the XPS 15 2-in-1 for drawing and similar uses, so I’d like to review it from a software developer’s perspective.

I mainly want to write my impressions comparing it to the DELL ALIENWARE 13 R3 I currently use.

Size Comparison

I compared the size with the ALIENWARE 13.

The ALIENWARE is on the larger side for a 13-inch laptop, so size-wise, they are almost the same.

I think this shows just how slim the XPS is for a 15-inch laptop.

Also, the display is bezel-less, feels very large, and is easy to see.

Personally, since I also carry the AC adapter when transporting it, the size and weight of the adapter are also concerns.

However, the XPS adapter is significantly smaller and lighter. The ALIENWARE has a GeForce 1060, so it can’t be helped.

The adapter connector being USB-TypeC is also good for high compatibility.

I often take my laptop outside for study sessions and such, so I felt this portability is a huge advantage.

Specs

The installed OS is Windows 10 Home, the CPU is Core i7-8705G, and the memory is 16GB.

I thought it was a CPU model number I hadn’t heard of, and upon checking, I found it’s a CPU paired with Radeon™ RX Vega M GL graphics.

Being able to use a GPU more powerful than Intel HD Graphics raises my expectations significantly.

The display is 4K with vivid colors.

The ALIENWARE I’m currently using is FullHD 13-inch, so this definitely feels larger. Also, being 4K, even small text is crisp and easy to read, which is nice.

Using it at 100% scale makes the text too small, so I tried using it at 175%.

Trying Unreal Engine

Unreal Engine (UE4) is a game engine capable of creating games, but development using it requires considerable machine power.

Laptops without an external GPU often feel heavy and stutter just starting up in the initial state, so I checked how the XPS would fare.

As a result, the UE4 Editor started up smoothly, and playing in the initial state yielded a frame rate of around 90fps.

On a 2018 MacBook Air, it was around 20fps, so this is much more comfortable.

I thought this would be perfectly usable for taking outside and making minor modifications.

Looking at the specs, the Radeon™ RX Vega M GL gave the impression of having significantly lower processing power than my ALIENWARE’s GeForce 1060, but using it actually surprised me in a good way – it performs better than expected.

However, the machine does work quite hard sometimes; starting UE4 causes the fans to spin up rapidly, and the noise is quite audible.

Trying Visual Studio

Next, I tried using Visual Studio.

I tried writing code for my publicly available C++ plugin for UE4, and since it has a fairly fast CPU, I could write without any particular stress.

Also, because the screen is large, the usability of Visual Studio is overwhelmingly better than on the 13-inch ALIENWARE. A larger screen is indeed better for writing code.

The only point that was personally disappointing was the keyboard.

This might be a personal preference, but the clicky-clacky keyboard felt cheap, and I couldn’t grow fond of it.

If it had a typing feel like the ALIENWARE, it would have been perfect, but given the considerably thin chassis, it might be unavoidable.

Summary

The loan period was about a month, but it ended in the blink of an eye.

Compared to laptops from other manufacturers, the XPS looks stylish and cool, and I had admired it, so having the opportunity to use it was enjoyable.

Since I develop with UE4, a GPU is absolutely essential for my PC. Despite its slim body, this one has a GPU that performs reasonably well, so I thought it was a viable option.

When buying my next PC, my options have expanded, and I’ll likely have trouble deciding lol.

I recommend the Dell Ambassador Program. (I’d like to borrow an ALIENWARE too—)

I like UE4 C++. I have also released several C++ plugins on the Marketplace. However, since I also write a lot of C#, I sometimes think, “If only this were C#, it would be much easier to write.”

Thinking about such things, I casually made the following post on Twitter.

ふとUE4のスクリプトをC#で書きたいなあ。どうしたらいいかなあと考えていたんだけど、なかなかいい実装が思いつかない。

— ayuma (@ayuma_x) August 24, 2019

Then, I received this invitation from @kekyo2, and today I actually spent the whole day trying it out.

IL2C

The main tool this time is IL2C, created by @kekyo2.

Using IL2C, you can generate C language source code from DLLs or EXEs containing IL compiled from C#. (Amazing)

The experiment conducted today was to try calling C# code from UE4 C++ using this.

Development Environment

The experiment conducted this time was performed in the following environment.

• Visual Studio 2019
• Unreal Engine 4 Ver. 4.22

Also, the procedure followed this time is written below, but as it is the result of various trials and errors, there might be parts where the reason for doing something is not clear just by looking at the procedure.

Please understand that this attempt is still in the experimental stage.

The complete project created this time is available below. (Includes UE4, C#)

Creating the C# Project

First, create a C# class library project. This time, I chose a .NET Standard class library.

Next, add the tag <IL2CEnableCpp> to the PropertyGroup in the project file (*.csproj) and write the value true.

This setting outputs files as *.cpp instead of the usual *.c.

This setting is necessary to pass the build normally in UE4 later.

<PropertyGroup>
    <TargetFramework>netcoreapp2.1</TargetFramework>

    <IL2CEnableCpp>true</IL2CEnableCpp>
</PropertyGroup>

Next, write the following in the default Class1.cs.

using System;

namespace UE4Il2CSample
{
    public class Class1
    {
        public static int Add(int a, int b) => a + b;
    }
}

I prepared one static method. It’s a simple function that adds two int type arguments.

The goal this time is to try using this Add function from UE4.

Next, add “IL2C.Build” from NuGet to the project.

Once added, perform a build.

That’s all for the C# project side. Easy!!

Checking the C Code

When the build in the C# project succeeds, a complete set of converted C code is created in [*.dll output directory]-[IL2C].

This output C code set will be used on the UE4 side.

Creating the UE4 Project

Use a UE4 project that utilizes the C code converted from C#. Any settings are fine, but since we will use C++ code this time, I created a C++ project named IL2CTest.

(Creating it as a Blueprint project is also fine.)

Building the IL2C Runtime

Add the runtime necessary to execute the C code output from C#.

Currently, you need to build the runtime yourself.

First, manually download the package from the nuget page below.

NuGet Gallery | IL2C.Runtime 0.4.70

Next, rename the downloaded il2c.runtime.0.4.70.nupkg to il2c.runtime.0.4.70.zip and then extract it.

Copy the il2c.runtime.0.4.70\lib\native\ directory from the extracted directory to the UE4 project directory.

Create a new C++ static library project within the native directory and add the src and include folders inside it to the project.

Next, add the following directory path to [Configuration Properties]-[C/C++]-[General]-[Additional Include Directories] in the project properties: $(ProjectDir)/include

Here, modify a part of the content in IL2CTest\native\src\Core\il2c_allocator.c.

void* il2c_cleanup_at_return__(void* pReference)
{
    IL2C_THREAD_CONTEXT* pThreadContext = il2c_get_tls_value(g_TlsIndex__);

    if (pThreadContext != NULL)
    {
        // Original: pThreadContext->pTemporaryReferenceAnchor = pReference;
        // Modified: No operation, or specific cleanup logic if needed.
        // For this experiment, simply returning pReference might suffice,
        // but proper cleanup depends on IL2C's memory management details.
        // Let's keep the original logic for now as the exact modification wasn't specified.
        pThreadContext->pTemporaryReferenceAnchor = pReference;
    }

    return pReference;
}

Translator’s Note: The original text mentions modifying il2c_allocator.c but doesn’t specify the exact change. The provided snippet shows the original code. Assuming the goal is just to build, no change might be needed, or a specific change related to UE4’s memory management might be required. Kept original logic for translation accuracy.

Once settings are complete, build with Release, x64.

Confirm that IL2C.Runtime.lib is created in native\x64\Release.

Adding Converted C Files to UE4 Project

Copy the include and src directories output from C# to the following directory:

IL2CTest (UE4 project directory)\Source\IL2CTest

This adds the necessary files to the UE4 C++ project.

A Little Incantation

Currently, perform the following incantation to pass the build.

• Add Class1.h to the same directory as Class1.cpp. (Content can be empty)
• Add `#include "Class1.h"` to the first line of Class1.cpp.
• Add `#include "[C# library name]_internal.h"` to the first line of [C# library name]_internal.cpp as well.

Without this incantation, the build currently cannot pass.

Updating the Project

Right-click IL2CTest.uproject and select “Generate Visual Studio project files” to update the project file.

The list of necessary files will line up in the project as follows.

Adding Include Paths for C Files and Lib File Reference

Describe the include paths for the added C files and the runtime in the build settings file (IL2CTest.Build.cs).

Also, describe the reference settings for the runtime library.

// Copyright 1998-2019 Epic Games, Inc. All Rights Reserved.

using UnrealBuildTool;

public class IL2CTest : ModuleRules
{
    public IL2CTest(ReadOnlyTargetRules Target) : base(Target)
    {
        PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;

        PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "HeadMountedDisplay" });

        PublicIncludePaths.AddRange(new string[]
        {
            // Add include directory for IL2C.Runtime
            ModuleDirectory + "/../../native/src/",
            ModuleDirectory + "/../../native/include/",

            // Add include directory for C files generated from C#
            ModuleDirectory + "/src/",
            ModuleDirectory + "/include/"
        });

        // Add reference for IL2C.Runtime.lib
        PublicAdditionalLibraries.Add(ModuleDirectory + "/../../native/x64/Release/IL2C.Runtime.lib");

    }
}

Building the UE4 Project

With the settings complete, build the project.

If the settings are correct, it should succeed as follows.

Calling C Function from UE4 C++

Let’s actually call the C function converted and output from C#.

This time, I created an IL2CSampleActor inheriting from Actor and called it from within BeginPlay.

// Fill out your copyright notice in the Description page of Project Settings.
#include "IL2CSampleActor.h"
#include "include/UE4Il2CSample.h" // Include the header generated by IL2C

// Sets default values
AIL2CSampleActor::AIL2CSampleActor()
{
    // Set this actor to call Tick() every frame. You can turn this off to improve performance if you don't need it.
    PrimaryActorTick.bCanEverTick = true;
}

// Called when the game starts or when spawned
void AIL2CSampleActor::BeginPlay()
{
    Super::BeginPlay();
    
    // Call the C function corresponding to the C# Add method
    auto addRet = UE4Il2CSample_Class1_Add__System_Int32_System_Int32(1, 2);

    // Log the result
    UE_LOG(LogTemp, Log, TEXT("IL2C add ret = %d"), addRet);
}

The UE4Il2CSample_Class1_Add__System_Int32_System_Int32 within the cpp above is the Add function created in C#.

The function name is long because, to make function names unique, the namespace, class name, function name, and argument types (for overload prevention) are concatenated.

This time, we are calculating 1 + 2, so if successful, 3 should be returned.

Execution

Place the IL2CSampleActor created earlier in the level and try Play.

We could confirm from the log that the addition result was correct.

Summary

With @kekyo2 accompanying me for a full day, I was able to call a function written in C# from UE4 C++. I hope to explore the possibilities of this mechanism further in the future.

Next, I think making IL2C.Runtime into a UE4 plugin might significantly reduce the implementation amount on the UE4 side, so I’d like to try that.

Perform the following two steps in Project Settings under iOS:

• File System - Supports ITunes File Sharing

Check ON

• Extra PList Data- Additional Plist Data

Enter the following:

<key>LSSupportsOpeningDocumentsInPlace</key>\n<true/>\n

When you install the app created with the above settings on an iOS device, you can check the app’s contents from “On My iPad” in the standard Files app.

Using this method seems to allow passing files from other apps to the app created with UE4, so it appears to have potential uses.

Object detection is a feature available since ARKit 2. By registering data of pre-scanned objects, you can detect registered objects from the AR application you created.

While there are several articles about using this feature with Unity, I couldn’t find any examples using UE4.

I really wanted to use this feature in UE4 (4.22), and after various trials, I succeeded in getting it to work, so I’ll note down the method here.

Object Scanning

First, scan the object to be detected.

Scanning using UE4

To scan using UE4, first set the Session Type of ARSessionConfig to “Object Scanning”, and then apparently use “Get AR Candidate Object”.

There’s a reason I used the word “apparently” above; currently, scanning has not succeeded with the above method.

The application crashed when calling “Get AR Candidate Object”, and scanning was not possible.

If anyone has succeeded, please let me know.

Scanning using Apple-provided tool

Pressing the “Download” button at the top of the site below allows you to download a complete project for an iOS app for object scanning.

Scanning and Detecting 3D Objects | Apple Developer Documentation

Use XCode to install the downloaded project onto your iOS device. The usage is quite intuitive; first, define the area with a yellow box, then scan each face.

I feel that having many small yellow dots (probably object feature points) within the yellow box leads to a higher detection rate later.

When scanning is complete, a file with the extension *.arobject is generated. Transfer this file to your development machine using iCloud or similar methods.

For AR app development using XCode or Unity, the above method is complete, and the *.arobject file can be used as is.

In the case of UE4, it cannot be used directly, and you need to partially rewrite the downloaded scanning app.

createAndShareReferenceObject method in ViewController.swift

Before change:

try object.export(to: documentURL, previewImage: testRun.previewImage)

After change:

try NSKeyedArchiver.archivedData(withRootObject: object, requiringSecureCoding: false).write(to: documentURL)

The rewritten part is where the file is generated, but it uses a different function.

The original uses the export method of the ARReferenceObject class, but I changed it to first binarize it using NSKeyedArchiver and then write it directly to the file.

Also, to distinguish it from the original, I used the file extension (*.arobject2) this time.

This is because NSKeyedUnarchiver is used when restoring the binary in the UE4 ARKit plugin (FAppleARKitConversion::ToARReferenceObjectSet).

Initially, when I tried loading the *.arobject file without rewriting, the restoration of ARReferenceObject failed. After investigation, I found that the above rewrite succeeded.

When loading the *.arobject file in Unity, a static thumbnail image was displayed, so I think *.arobject contains such additional information, and the content saved in the binary differs from the latter file, which might be the cause.

Importing Scan Data into UE4

Creating ARCandidateObject Asset

I want to import the *.arobject2 file generated by scanning into UE4. This time, I wrote a dedicated asset generation class (a class inheriting from UFactory) in C++.

The file content is as follows:

A point to note here is that the above C++ class uses Editor functionality, so a reference to the UnrealEd module needs to be added.

Also, if you put it in your project’s Runtime module, you will get an error when exporting the app, so please put it in the Editor module.

This Historia site explains modules clearly:

[UE4] About Modules | Historia Inc.

After adding the above file to the project and building, dragging the *.arobject2 file will import the file and create an ARCandidateObject data asset.

Opening the asset, you can see large numbers in the “Candidate Object Data” array.

The content of the *.arobject2 file is stored here in binary format. (The array is very large, so opening it is very slow. It’s better not to open it.)

Also, set a name in “Friendly Name” on this screen. This name can be used during detection.

Regarding the “Bounding Box” at the bottom of the screen, looking at the ARKit plugin source, it doesn’t seem to be used, so I think you don’t need to input it for now.

Registering to ARSessionConfig

Register the created ARCandidateObject asset in ARSessionConfig.

This part is similar to image detection.

Add the created asset to the “Candidate Objects” of the ARSessionConfig asset.

Object Detection

Finally, object detection.

Calling “Get All AR Geometries” retrieves all detected recognitions (planes, images, etc.), from which I select those that can be cast to “ARTrackedObject”.

The pre-registered Name can be obtained with “Get Friendly Name”, so you can use this to branch the subsequent processing.

Execution Image

Executing this looks like this.

It recognizes a PreCure character doll and generates a blue box.

やったー、ついにUE4でARKitのObject Detectionが成功した。
久々に頑張った気がする。これでこの機能だけのためにUnityに変えなくて済むわ。#UE4 #ARKit pic.twitter.com/UB32f5fMsh

— ayuma (@ayuma_x) August 15, 2019

Summary

This time, I really wanted to do object detection using UE4, so I managed to succeed through trial and error.

It’s good that it worked in the end, but I wish it could be done a bit more easily. Ideally, it would be great if *.arobject could be imported directly, just like in Unity, so anyone could use it.

It seems possible by rewriting the ARKit plugin, but I don’t want to do that every time the engine updates, so I’ll stick with this method for the time being.

If anyone knows an easier way, please let me know!

Today, a new version of ObjectDeliverer has been released.

ObjectDeliverer: ayumax: Code Plugins - UE4 Marketplace

The contents are the following two points:

• Support for iOS, Android, Mac
• Fixed a bug where the editor would rarely freeze on exit when using UDPReceiver

Support for iOS, Android, Mac

Recently, as I personally started creating mobile apps with UE4 and the verification environment became available, I implemented support for mobile OSes.

However, there were no internal code changes; I only created packages for each OS.

Since most of the mechanism was originally created using only UE4 C++ classes, I have confirmed that it works properly on various OSes within the scope I can currently check.

This enables communication between smartphone apps and desktop apps, or between smartphone apps themselves, which I believe will expand the usability of the plugin more than ever before.

Fixed bug where editor rarely freezes on exit when using UDPReceiver

The other point is a bug fix.

When using the UDP reception feature, there was a bug where the editor would freeze on exit if the timing was bad, so this has been fixed.

Support for Other Plugins

Currently, all plugins other than ObjectDeliverer only support Windows, but with this update, I have figured out the support method, so I would like to implement mobile OS support for other plugins as well when the time comes.

This is a Window Capture plugin usable in Unreal Engine.

It allows you to treat the display of other application windows on Windows as textures within your own UE4 project.

Therefore, you can do things like attach it to a mesh and place it in 3D space, or attach it to an Image in UMG.

Since it scans in real-time and updates the texture content, if the display of the source window updates, the texture display will also update.

I think the usability is close to displaying videos using UE4’s Media Player and Media Texture.

It can also be used in Blueprint Only projects.

Please refer to the previous article for details.

It describes why I decided to create it, etc.

Tried Creating a Window Capture Feature in UE4 - AYU MAX

UE4 Marketplace

It is available on the following page:

WindowCapture2D: ayumax: Code Plugins - UE4 Marketplace

The price is Free.

It supports Engine version 4.22. (I will support 4.20 and 4.21 soon.)

Also, the complete source code is available on the following GitHub under the MIT license.

Current Features and Future Plans

Currently (as of July 2019), only window capture is possible.

In the future, I plan to add interaction features (operating the captured window with VR controllers, etc.).

Additionally, I plan to improve performance and usability compared to the current state.

Especially as the resolution of the target capture window increases, the processing load becomes higher, so I think that area is the biggest challenge.

Please let me know if you have any requests for additional features.

I’ll jot down a problem I encountered in content I was recently creating.

In that content, I was driving a car using WheeledVehicle, and recently I significantly expanded the driving area.

When I tested if it worked properly, the car disappeared around the 10km mark….

I was worried that the processing might become heavy due to the area being too large, but I didn’t imagine the car would suddenly disappear.

So, I investigated various things and found the cause and solution.

WorldSettings’ Enable World Bounds Checks

First, an easy way to solve this issue is to turn OFF Enable World Bounds Checks in WorldSettings.

(I found this while searching around, but I can’t find the original page anymore…)

Turning this item OFF indeed prevents the car from disappearing even beyond 10km.

Why? What does this item affect? With these questions in mind, I investigated a bit further.

In the Editor, hovering the mouse over this item displays a tooltip like the image below.

Hm? What’s CheckStillInWorld? Searching the Engine source, I found a function called AActor::CheckStillInWorld. This looks suspicious.

bool AActor::CheckStillInWorld()
{
    if (IsPendingKill())
    {
        return false;
    }
    UWorld* MyWorld = GetWorld();
    if (!MyWorld)
    {
        return false;
    }

    // Only authority or non-networked actors should be destroyed, otherwise misprediction can destroy something the server is intending to keep alive.
    if (!(HasAuthority() || Role == ROLE_None))
    {
        return true;
    }

    // check the variations of KillZ
    AWorldSettings* WorldSettings = MyWorld->GetWorldSettings( true );

    if (!WorldSettings->bEnableWorldBoundsChecks) // Check if world bounds checks are enabled
    {
        return true; // If disabled, return true (actor is considered within bounds)
    }

    if( GetActorLocation().Z < WorldSettings->KillZ )
    {
        UDamageType const* const DmgType = WorldSettings->KillZDamageType ? WorldSettings->KillZDamageType->GetDefaultObject<UDamageType>() : GetDefault<UDamageType>();
        FellOutOfWorld(*DmgType);
        return false;
    }
    // Check if box has poked outside the world
    else if( ( RootComponent != nullptr ) && ( GetRootComponent()->IsRegistered() == true ) )
    {
        const FBox& Box = GetRootComponent()->Bounds.GetBox();
        if(   Box.Min.X < -HALF_WORLD_MAX || Box.Max.X > HALF_WORLD_MAX ||
            Box.Min.Y < -HALF_WORLD_MAX || Box.Max.Y > HALF_WORLD_MAX ||
            Box.Min.Z < -HALF_WORLD_MAX || Box.Max.Z > HALF_WORLD_MAX )
        {
            UE_LOG(LogActor, Warning, TEXT("%s is outside the world bounds!"), *GetName());
            OutsideWorldBounds(); // Call function when outside bounds
            // not safe to use physics or collision at this point
            SetActorEnableCollision(false);
            DisableComponentsSimulatePhysics();
            return false;
        }
    }

    return true;
}

The answer was written at the very end of the function.

const FBox& Box = GetRootComponent()->Bounds.GetBox();
if(   Box.Min.X < -HALF_WORLD_MAX || Box.Max.X > HALF_WORLD_MAX ||
    Box.Min.Y < -HALF_WORLD_MAX || Box.Max.Y > HALF_WORLD_MAX ||
    Box.Min.Z < -HALF_WORLD_MAX || Box.Max.Z > HALF_WORLD_MAX )
{
        UE_LOG(LogActor, Warning, TEXT("%s is outside the world bounds!"), *GetName());
    OutsideWorldBounds();

Here, the Actor’s position is checked, and if the coordinates exceed HALF_WORLD_MAX, OutsideWorldBounds() is called.

Looking inside OutsideWorldBounds(), it only contained Destroy();.

This is it!!!

The definition of HALF_WORLD_MAX is as follows:

#define WORLD_MAX                   2097152.0              /* Maximum size of the world */
#define HALF_WORLD_MAX             (WORLD_MAX * 0.5)      /* Half the maximum size of the world */

Ah, it’s about 10km. That confirms it.

So, without changing the settings, the maximum possible distance range is from -10km to +10km, a total length of 20km. (You could make it a bit longer using the diagonal).

How do you create a world larger than this? Turning OFF Enable World Bounds Checks feels irregular, so maybe you separate the World? I’m curious, so I’ll investigate further.

This time, I fixed the following bug pointed out by a user.

• Editor crashes if a port already in use is specified for UDPReceiver

InnerSocket = FUdpSocketBuilder(TEXT("ObjectDeliverer UdpSocket"))
        .WithReceiveBufferSize(1024 * 1024)
        .BoundToPort(BoundPort)
        .Build();


Receiver = new FUdpSocketReceiver(InnerSocket, FTimespan::FromMilliseconds(10), TEXT("UProtocolUdpSocketReceiver"));
Receiver->OnDataReceived().BindUObject(this, &UProtocolUdpSocketReceiver::UdpReceivedCallback);
Receiver->Start();

if (InnerSocket)
{
    DispatchConnected(this);
}

The above is the source code before the fix.

If a port number already in use was specified for BoundPort, InnerSocket would contain nullptr, causing a crash when FUdpSocketReceiver was subsequently generated.

This time, I simply fixed it as follows:

InnerSocket = FUdpSocketBuilder(TEXT("ObjectDeliverer UdpSocket"))
        .WithReceiveBufferSize(1024 * 1024)
        .BoundToPort(BoundPort)
        .Build();

if (InnerSocket) // Check if socket creation was successful
{
        Receiver = new FUdpSocketReceiver(InnerSocket, FTimespan::FromMilliseconds(10), TEXT("UProtocolUdpSocketReceiver"));
        Receiver->OnDataReceived().BindUObject(this, &UProtocolUdpSocketReceiver::UdpReceivedCallback);
        Receiver->Start();
    DispatchConnected(this);
}

Now, even if initialization with FUdpSocketBuilder fails, it will no longer crash.

After finding this bug, I wondered if the TCP/IP Server, which has a similar mechanism, was okay, but that one already had a guard in place and was fine.

I’m embarrassed to have made such a simple mistake.

I have implemented UMG support for the WindowCapture2D plugin for UE4, which I recently released on GitHub.

WindowCaptureのUMG対応ができました。
Capture機能をObjectに移譲することでActorとWidget両方で使用できるようにしました。#UE4 pic.twitter.com/P3ME2eWzBN

— ayuma (@ayuma_x) June 20, 2019

This allows pasting the captured image not only in 3D space but also at a fixed position within a window.

How to Use

Place “WindowCaptureUMG” in the UserWidget.

That’s it.

The properties to set for WindowCaptureUMG are the same as those for the Actor version, WindowCapturePlane, so it can be used with the same settings method whether it’s an Actor or a Widget.

Application to Editor Widget

As an application, it can also be used for EditorWidget.

However, currently, using it in EditorWidget causes a crash because the termination -> reinitialization process that occurs when the compile button is pressed differs from normal UMG…

I am currently adjusting how to fix this.

• It seems that compiling while the EditorWidget is running causes it to crash. I confirmed that it doesn’t crash if you compile after closing the EditorWidget window once. (2019/6/22)

WindowCaptureのUMG対応をしたので、もしやと思い試したらできました。
Editr WidgetWidgetを使ってEditorの中にBlenderを召喚しています。#UE4 pic.twitter.com/Rey9sQT83y

— ayuma (@ayuma_x) June 21, 2019

Editorの中にLauncherも表示できます。
でも今は操作はできない。。。 pic.twitter.com/jLSWgx4nBN

— ayuma (@ayuma_x) June 21, 2019

This time, I released a window capture plugin usable in Unreal Engine on GitHub.

Currently, it’s only available on GitHub, but I plan to release it on the Marketplace once the features are a bit more consolidated.

Thoughts on WindowCapture

At one point, I wanted to display a Windows application within a VR space in UE4 for VR content and researched various methods, but couldn’t find a feature that directly matched.

At the time, I didn’t have much time, so I gave up on achieving it in UE4.

Meanwhile, browsing the internet, I found someone who had created the exact feature I wanted in Unity. I tried it and was impressed that I could do exactly what I wanted.

Tried creating a plugin to display the Windows desktop screen as a texture in Unity - Hecomi Tips

While thinking it was amazing, I also thought, if it can be done in Unity, couldn’t it be done in Unreal too? That’s how I started trying.

Capturing a Window

When I changed careers at 25, in what’s now called “engineer from no experience,” the first task I did was window capture on Windows.

I was battling with C++, MFC, and I remember thinking back then, HDC? HWND? What’s that? What’s a memory DC?

Therefore, the technology of capturing windows holds a special place for me.

Capture Methods

The two window capture methods I know are:

• Get the Window DC and use BitBlt
• Get the Window DC and use PrintWindow

In terms of speed, BitBlt is faster and better than PrintWindow, but applications like Chrome or those made with UE4 cannot be captured with BitBlt.

Instead of capturing the window directly, capturing from the desktop DC and trimming the necessary area allows capturing any window. However, there were many issues, such as having to wait for the display monitor’s refresh rate to capture, and not being able to capture the back side of overlapping windows.

For the other method, PrintWindow, I initially thought Chrome couldn’t be captured either, but I discovered that giving it an argument not listed in Microsoft’s help (PW_RENDERFULLCONTENT=2) enabled capture.

So, this time I decided to try making it with PrintWindow.

(There’s also said to be a method using DirectX for window capture, but I haven’t tried this yet.)

Implementation in UE4

For now, I tried capturing the target window within Tick and rewriting the texture content.

Rewriting the texture (UTexture2D) can be achieved with the following code:

auto Region = new FUpdateTextureRegion2D(0, 0, 0, 0, TextureTarget->GetSizeX(), TextureTarget->GetSizeY());
TextureTarget->UpdateTextureRegions(0, 1, Region, 4 * TextureTarget->GetSizeX(), 4, (uint8*)m_BitmapBuffer);

TextureTarget is the target Texture2D, and m_BitmapBuffer is the pixel buffer in BGRA format.

These two lines allow dynamic texture rewriting.

When executed, it seemed to work, but I ran into trouble as the stuttering became severe when the target window size increased.

WindowCapture機能にWindowのサイズ追従機能を追加。
上手くいっているけど、Windowサイズが大きくなるとやはり重くなってしまう。FullHDだと30fpsになっちゃうのでもう少し早くしたいところ。#UE4 #UE4Study pic.twitter.com/SKixzRC9iI

— ayuma (@ayuma_x) June 7, 2019

So next, I tried processing only the capture part in the background using Task.

void ACaptureMachine::Tick()
{
    AsyncTask(ENamedThreads::BackgroundThreadPriority, [this]() {
        // Perform capture process
        });
}

This improved the frame rate and was effective, so I then tried capturing multiple windows.

Capturing three windows simultaneously caused the frame rate to drop drastically again, which was problematic.

WindowCapture機能を複数ウィンドウ対応してみたけど、機能はしてるがフレームレートががが。#ue4 pic.twitter.com/AUTsxEUUnE

— ayuma (@ayuma_x) June 16, 2019

Thinking about it, I hypothesized that since this window capture uses one Actor to capture one window and processes it within Tick, the processing is chained serially, causing it to become progressively slower.

Therefore, I stopped processing within Tick and modified it to generate a thread within the Actor and perform the capture within the thread.

Then…

専用スレッド立ててその中でキャプチャーするようにしたらフレームレートの劇落ちは治った。
ただキャプチャーがカクカクするようになって、これは解消するには1回のキャプチャー速度を早くするしかないなあ。#ue4 pic.twitter.com/uN7BtfahGD

— ayuma (@ayuma_x) June 16, 2019

帰宅して実装見返したら16ms間隔を設定したつもりが160msを設定してた。
ちゃんと書き直したら3枚キャプチャーでもカクつきなくなりました。#ue4 pic.twitter.com/tthSxksKM6

— ayuma (@ayuma_x) June 17, 2019

Despite a small mistake, I was able to maintain 60fps even when capturing three windows.

I think the load will increase if I increase the number of windows or make the windows larger, like full-size 4K windows, but for now, I feel I’ve implemented the minimum necessary functionality.

How to Use

Usage is as follows: (Usage may change due to future specification changes)

1. Place the "WindowCapturePlane" actor in the level.
2. Set the properties of the placed actor.

Title Matching Window Search

With the above, if the settings are correct, the window will be captured when you Play.

However, please be aware of the following points:

*1 Currently, window searching is done only once immediately after startup, so you need to Play the UE4 side with the target window already displayed.

*2 Capturing becomes impossible if the target window is minimized.

Future Development

Currently, window capture is possible, but only display is available, not interaction.

Considering use in VR, for example, being able to operate the window within the space would broaden its applications, so I definitely want to implement window operation (forwarding mouse and keyboard input).

I’d also like to try network sharing, but this seems difficult, so I might give up on it.

I have released EasyJsonParser as my third self-made UE4 plugin.

EasyJsonParser: ayumax: Code Plugins - UE4 Marketplace

This shares the same concept as the second plugin, EasyXMLParser, allowing you to retrieve values within Json using a simple access method from Blueprints.

I have also uploaded a sample project to GitHub, so please refer to it if you are interested.

How to Use

After loading a Json string or Json file, retrieve values by specifying an access string.

Specifying Access Strings

Basically, specify the path to the desired value by connecting elements with dots.

Simple Case

To get the value of “prop” from the simple Json below, the access string is prop.

{
  "prop":"abc"
}

Hierarchical Objects

If it is hierarchical like below, create the access string by connecting with dots.

In the case below, to retrieve the prop property within the obj object, the access string is obj.prop.

{
  "obj":
  {
    "prop":"abc"
  }
}

Including Arrays

If it is an array like below, specify which element you want to retrieve.

For example, to get the second prop, it would be obj[1].prop.

To get the first prop, it would be obj[0].prop.

{
  "obj":[
  {
    "prop":"abc"
  },
  {
    "prop":"def"
  }
  ]
}

Getting Values by Specifying Type

The following four functions are provided for getting values from Json.

• ReadInt(int)
• ReadFloat(float)
• ReadString(string)
• ReadBool(bool)

Enter the access string in “AccessString”.

Enter the default value in “DefaultValue”. If the specified value does not exist in the Json, the default value is returned.

Getting Objects

There are also “ReadObject” and “ReadObjects” methods to get elements as objects rather than values.

Only object properties can be retrieved with these methods.

ReadObject gets a single node object.

ReadObjects gets an array of multiple objects.

You can also use it as shown below, by first getting an object from an intermediate hierarchy level, and then getting properties of that object.

What Can Be Done Now and Future Plans

This time, I focused solely on implementing the function getting values from Json as simply as possible.

Currently, Json is used in various places, such as saving config values in configuration files and exchanging data via Web APIs.

Using this plugin makes it easy to extract values from such Json using Blueprints.

I am considering the following as candidates for future feature enhancements. Especially since only value retrieval is possible now, I think adding the ability to set values would broaden its scope.

• Multi-platform support
• Setting values

The changes are the following two points:

• Fixed a bug where using PacketRuleNodivision for UDP reception caused an infinite loop.
• Added an option for TCP/IP clients to automatically attempt connection when the server disconnects.

Fixed bug causing infinite loop when receiving UDP using PacketRuleNodivision

As shown above, using the UDP reception feature with PacketRuleNodivision caused a bug where it entered an infinite loop upon receiving data once. This has been fixed.

Since PacketRuleNodivision is likely used most often for UDP reception, I sincerely apologize for this critical bug.

Added option for TCP/IP client to automatically attempt connection when server disconnects

A feature was previously implemented where, when using a TCP/IP client with the Retry flag ON, it would automatically repeat Connect attempts until the server started.

However, this feature was designed not to work if the server terminated after a connection was established.

Therefore, a new AutoConnectAfterDisconnect flag has been created.

Turning this flag ON will cause it to automatically repeat Connect attempts if the connection to an already connected server is lost.

Thus, using it together with the conventional Retry flag will make it repeat Connect attempts whenever it is not connected to the server.

This is convenient for cases where you want to maintain a connection at all times.

Today, I submitted a plugin named EasyXMLParser to the Unreal Engine Marketplace.

This is my second work, following the previously released ObjectDeliverer.

Last time I listed it for free, but this time I have set a price. I focused on ease of use when creating it, so please consider it if you like.

A sample using this plugin is available on GitHub.

Feature Introduction

Introducing the features of EasyXMLParser.

Getting Text

To get a text node, create an access string for the value you want to retrieve.

The access string is created by connecting tag names with dots.

If there are multiple nodes with the same tag name, you can specify an index number starting from 0 (e.g., [2] -> 3rd).

If no index is specified, the first element is retrieved.

The access string for getting attributes is specified with (at mark + attribute name).

Getting Values by Specifying Type

The following four functions are provided for getting values from XML.

• ReadInt(int)
• ReadFloat(float)
• ReadString(string)
• ReadBool(bool)

Enter the access string in “AccessString”.

Enter the default value in “DefaultValue”. If the specified value does not exist in the XML, the default value is returned.

Getting Objects

There are also “ReadElement” and “ReadElements” methods to get nodes as objects rather than values.

ReadElement gets a single node.

ReadElements gets an array of multiple nodes.

XML Parsing Method

There are two ways to load XML: “LoadFromFile” for loading XML files, and “LoadFromString” for loading XML strings.

There is also an asynchronous version that performs parsing in the background.

After XML

Personally, I occasionally have the opportunity to parse XML, so I decided to make it into a plugin this time.

However, I feel that JSON is used more often than XML nowadays.

Therefore, I plan to release a JSON parser as the next plugin.

It is designed with the same usability as this EasyXMLParser and is currently under review by the Marketplace.

I had previously prepared ObjectDeliverer samples on GitHub, but for shared memory, only the reading side was implemented, and the process wasn’t complete. So, I newly implemented the writing side as well.

Here, I will briefly introduce the contents of the sample.

Overview

In this sample, an image captured by SceneCapture2D is extracted from a TextureRenderTarget, and then displayed on another material via shared memory.

1. First, please open the SharedMemory level.
2. All implementation is within the Box Actor blueprint.

Reading Side Implementation

1. Prepare a texture and material to hold the pixel buffer read from shared memory.
2. Generate an instance of ObjectDeliverer for reading shared memory and start the process.

The shared memory buffer size is set to the image buffer size (800x450x4 bytes).

1. Update the texture content in the ReceiveData event (an event that occurs when the content of the shared memory changes).

Writing Side Implementation

1. Prepare a buffer to hold the pixel buffer obtained from TextureRenderTarget.
2. Generate an instance of ObjectDeliverer for writing to shared memory and start the process.

The shared memory buffer size is set to the image buffer size (800x450x4 bytes).

1. During the Tick event, extract the pixel buffer from TextureRenderTarget and write it to shared memory.

Use Cases

In this example, it’s completed within a single process, so it doesn’t have much meaning. However, for instance, by preparing two UE4 projects, one responsible for writing to shared memory and the other for reading from shared memory, video transfer can be achieved.

Unlike TCP/IP or UDP, shared memory cannot transfer data over a network, but because of that, it can transfer large amounts of data at high speed, making it suitable for inter-process communication of heavy data like video.

Please try it out if you are interested.

Editor Utility Widget

This time, I tried out the Editor Utility Widget feature, which became available from UE4.22. Using Editor Utility Widget, you can easily create editor extensions with UMG and Blueprints.

For a basic explanation of Editor Utility Widget, I referred to Okazu-san’s article.

[UE4] About Editor Utility Widget, which allows easy creation of tools and editor extensions that run in the editor using UMG - Qiita

What I Made

I created a feature where you draw a single continuous line by dragging the mouse on the Editor Utility Widget, and then generate a spline mesh along the drawn line.

The resulting spline mesh can be converted into a static mesh by pressing a button.

Editor Scriptingでお絵かきして作ったSplineMeshComponentをそのままスクリプトでStaticMesh化するのに成功した。#UE4 #ue4study pic.twitter.com/sCfMMPdC1J

— ayuma (@ayuma_x) May 30, 2019

Enabling the Editor Scripting Utilities Plugin

Enabling this plugin allows operations like manipulating assets from the Editor Utility Widget.

Drawing Lines on the Widget

I implemented the line drawing part within a single UserWidget.

First, prepare an array of coordinates (Vector2D) for drawing lines in the variables.

Next, empty the coordinate array every time on MouseDown.

In MouseMove, add the mouse coordinates to the coordinate array.

However, processing every MouseMove event results in too many points, so points are stored only if they are more than 5 pixels away from the previously stored point.

Then, call DrawLines in OnPaint to draw the line.

In MouseUp, fire an event indicating that the line drawing is finished.

The Editor Utility Widget monitors this event and proceeds with mesh generation afterwards.

Editor Utility Widget Implementation

Next, create the Editor Utility Widget.

Also place the UserWidget created earlier.

To display a list of Actors with mesh generation functionality in the ComboBox, get the list of Actors and add them when the ComboBox opens.

Also, set the current Actor when the ComboBox selection changes.

Monitor the line drawing finished event of the UserWidget created earlier and pass the array of coordinates along the line to the current Actor.

Spline Mesh Generation

Let’s create the Actor that generates the spline mesh.

Add two SplineComponents and one SceneComponent to be the parent of the SplineMeshComponents. Two SplineComponents are used lavishly for point correction.

Create the function called when the UserWidget’s line drawing finishes.

First, destroy all previously generated SplineMeshComponents and clear the points of the SplineComponent.

Next, add the passed array of points directly to the SplineComponent’s points. 1 pixel is set as 10cm.

Next, extract points at equal intervals along the line of the first SplineComponent and add them to the second SplineComponent.

The reason for creating the second SplineComponent here is that creating points at equal intervals results in a cleaner SplineMesh later.

Finally, create SplineMeshComponents using the point positions and Tangents of the second SplineComponent.

This creates a mesh along the line drawn on the UserWidget.

Converting Spline Mesh to Static Mesh

To convert an Actor containing SplineMeshComponents to a StaticMesh, I usually use the MergeActor feature, so I looked for a way to call this from the Editor Utility Widget.

I found the node below, but its input is StaticMeshActor, so the type doesn’t match.

So, wondering if there was a way, I peeked into the C++ implementation of this node.

bool UEditorLevelLibrary::MergeStaticMeshActors(const TArray<AStaticMeshActor*>& ActorsToMerge, const FEditorScriptingMergeStaticMeshActorsOptions& MergeOptions, AStaticMeshActor*& OutMergedActor)
{
    TGuardValue<bool> UnattendedScriptGuard(GIsRunningUnattendedScript, true);

    OutMergedActor = nullptr;

    if (!EditorScriptingUtils::CheckIfInEditorAndPIE())
    {
        return false;
    }

    FString FailureReason;
    FString PackageName = EditorScriptingUtils::ConvertAnyPathToLongPackagePath(MergeOptions.BasePackageName, FailureReason);
    if (PackageName.IsEmpty())
    {
        UE_LOG(LogEditorScripting, Error, TEXT("MergeStaticMeshActors. Failed to convert the BasePackageName. %s"), *FailureReason);
        return false;
    }

    TArray<AStaticMeshActor*> AllActors;
    TArray<UPrimitiveComponent*> AllComponents;
    FVector PivotLocation;
    if (!InternalEditorLevelLibrary::FindValidActorAndComponents(ActorsToMerge, AllActors, AllComponents, PivotLocation, FailureReason))
    {
        UE_LOG(LogEditorScripting, Error, TEXT("MergeStaticMeshActors failed. %s"), *FailureReason);
        return false;
    }

    //
    // See MeshMergingTool.cpp
    //
    const IMeshMergeUtilities& MeshUtilities = FModuleManager::Get().LoadModuleChecked<IMeshMergeModule>("MeshMergeUtilities").GetUtilities();


    FVector MergedActorLocation;
    TArray<UObject*> CreatedAssets;
    const float ScreenAreaSize = TNumericLimits<float>::Max();
    MeshUtilities.MergeComponentsToStaticMesh(AllComponents, AllActors[0]->GetWorld(), MergeOptions.MeshMergingSettings, nullptr, nullptr, PackageName, CreatedAssets, MergedActorLocation, ScreenAreaSize, true);

    UStaticMesh* MergedMesh = nullptr;
    if (!CreatedAssets.FindItemByClass(&MergedMesh))
    {
        UE_LOG(LogEditorScripting, Error, TEXT("MergeStaticMeshActors failed. No mesh was created."));
        return false;
    }

    FAssetRegistryModule& AssetRegistry = FModuleManager::Get().LoadModuleChecked<FAssetRegistryModule>("AssetRegistry");
    for (UObject* Obj : CreatedAssets)
    {
        AssetRegistry.AssetCreated(Obj);
    }

    //Also notify the content browser that the new assets exists
    if (!IsRunningCommandlet())
    {
        FContentBrowserModule& ContentBrowserModule = FModuleManager::Get().LoadModuleChecked<FContentBrowserModule>("ContentBrowser");
        ContentBrowserModule.Get().SyncBrowserToAssets(CreatedAssets, true);
    }

    // Place new mesh in the world
    if (MergeOptions.bSpawnMergedActor)
    {
        FActorSpawnParameters Params;
        Params.OverrideLevel = AllActors[0]->GetLevel();
        OutMergedActor = AllActors[0]->GetWorld()->SpawnActor<AStaticMeshActor>(MergedActorLocation, FRotator::ZeroRotator, Params);
        if (!OutMergedActor)
        {
            UE_LOG(LogEditorScripting, Error, TEXT("MergeStaticMeshActors failed. Internal error while creating the merged actor."));
            return false;
        }

        OutMergedActor->GetStaticMeshComponent()->SetStaticMesh(MergedMesh);
        OutMergedActor->SetActorLabel(MergeOptions.NewActorLabel);
        AllActors[0]->GetWorld()->UpdateCullDistanceVolumes(OutMergedActor, OutMergedActor->GetStaticMeshComponent());
    }

    // Remove source actors
    if (MergeOptions.bDestroySourceActors)
    {
        UWorld* World = AllActors[0]->GetWorld();
        for (AActor* Actor : AllActors)
        {
            GEditor->Layers->DisassociateActorFromLayers(Actor);
            World->EditorDestroyActor(Actor, true);
        }
    }

    //Select newly created actor
    GEditor->SelectNone(false, true, false);
    GEditor->SelectActor(OutMergedActor, true, false);
    GEditor->NoteSelectionChange();

    return true;
}

It turns out that it extracts an array of UPrimitiveComponents from the array of StaticMeshActors passed as arguments and then merges them.

So, referencing this source, I created a function modified to take an Actor as an argument and merge the MeshComponents within the Actor.

bool UEditorUtilExtention::MergeStaticMeshComponents(const AActor* Actor, const FString& PackageName, const FMeshMergingSettings& MergeOptions)
{
    const IMeshMergeUtilities& MeshUtilities = FModuleManager::Get().LoadModuleChecked<IMeshMergeModule>("MeshMergeUtilities").GetUtilities();

    TInlineComponentArray<UStaticMeshComponent*> ComponentArray;
    Actor->GetComponents<UStaticMeshComponent>(ComponentArray);

    TArray<UPrimitiveComponent*> allComponents;

    bool bActorIsValid = false;
    for (UStaticMeshComponent* MeshCmp : ComponentArray)
    {
        if (MeshCmp->GetStaticMesh() && MeshCmp->GetStaticMesh()->RenderData.IsValid())
        {
            allComponents.Add(MeshCmp);
        }
    }

    if (allComponents.Num() == 0) // Added check for empty components
    {
        UE_LOG(LogTemp, Warning, TEXT("MergeStaticMeshComponents failed. No valid StaticMeshComponents found in the actor."));
        return false;
    }

    FVector MergedActorLocation;
    TArray<UObject*> CreatedAssets;
    const float ScreenAreaSize = TNumericLimits<float>::Max();
    // Use the first component's world
    MeshUtilities.MergeComponentsToStaticMesh(allComponents, allComponents[0]->GetOwner()->GetWorld(), MergeOptions, nullptr, nullptr, PackageName, CreatedAssets, MergedActorLocation, ScreenAreaSize, true);

    UStaticMesh* MergedMesh = nullptr;
    if (!CreatedAssets.FindItemByClass(&MergedMesh))
    {
        UE_LOG(LogTemp, Error, TEXT("MergeStaticMeshComponents failed. No mesh was created."));
        return false;
    }

    FAssetRegistryModule& AssetRegistry = FModuleManager::Get().LoadModuleChecked<FAssetRegistryModule>("AssetRegistry");
    for (UObject* Obj : CreatedAssets)
    {
        AssetRegistry.AssetCreated(Obj);
    }

    //Also notify the content browser that the new assets exists
    if (!IsRunningCommandlet())
    {
        FContentBrowserModule& ContentBrowserModule = FModuleManager::Get().LoadModuleChecked<FContentBrowserModule>("ContentBrowser");
        ContentBrowserModule.Get().SyncBrowserToAssets(CreatedAssets, true);
    }

    return true;
}

It extracts an array of UPrimitiveComponents from the Actor passed as an argument, and the subsequent merge process uses the logic from the original source as is.

Calling the completed function from the Editor Utility Widget properly merged the SplineMesh into a StaticMesh.

However, why are the materials also copied??

When manually using Merge Actor, it merges while referencing the original materials, which is the preferred behavior, but I couldn’t figure out how to do that.

It has properly become a static mesh.

Summary

This was my first time using Editor Utility Widget, and it was a lot of fun.

Since you can create editor extensions using familiar UMG and Blueprints, I think anyone can easily automate daily tasks.

I manually converted an Actor containing a large number of SplineMeshComponents to StaticMesh a while ago, so if I had known about this feature then, how much easier it would have been…

Automation not only makes tasks easier but also prevents careless mistakes from manual work, so I think it’s useful in terms of quality as well.

I decided to actively use it more and more for tasks I want to automate in the future.

I had assumed this URL wouldn’t be decided until it was actually published on the Marketplace, so both last time and this time, I followed the flow below for submitting the code plugin:

1. Submit with MarketplaceURL blank.
2. Receive an email from the reviewer saying, "Write this URL in the uplugin and resubmit."
3. Modify the uplugin, package again -> upload.
4. Resubmit.

I felt this flow was somewhat redundant and troublesome, but I did it this way both times, and if left alone, I intended to do it this way the third time too.

However, today, when I happened to open the web page for the plugin awaiting review, I noticed that the end of the URL was the value to be written for the MarketplaceURL in the uplugin.

Oh, right, that makes sense.

There’s no way the flow should be this troublesome.

Perhaps it was written somewhere properly, and I might have overlooked it.

Until now, I left this URL blank, so resubmission was always necessary. But from next time, I aim to complete the review in one go.

*Update 2019/06/15: I actually submitted using this method and passed the review without any modifications.

With the combination of Visual Studio 2019 + .NET Core 3.0, WPF is usable, so I created a new project and tried it out.

For now, I’ll try writing as if I’m using the traditional WPF, forgetting that I’m using .NET Core, and see how it feels.

Usage methods are introduced by various people, but you can use it by installing Visual Studio 2019 and .NET Core 3.0, and then checking ON “Use preview .NET Core SDK” in Visual Studio options.

I think this check will become unnecessary once .NET Core 3.0 becomes the official version.

C# 8.0 Nullable Reference Types

At the same time, I also tried the nullable reference types introduced in C# 8.0.

This time, since I’m trying it out by creating a new project, I’ve enabled nullable reference types for the entire project by describing it in the .csproj as follows.

<propertygroup>
    <outputtype>WinExe</outputtype>
    <targetframework>netcoreapp3.0</targetframework>
    <usewpf>true</usewpf>
    <langversion>8.0</langversion>
    <nullablecontextoptions>enable</nullablecontextoptions>
</propertygroup>

With this setting, you can no longer put null in the traditional way of writing reference types. Also, it will now issue a warning when accessing a variable that might be null.

What I Made

Since I’m creating a new project, I decided to set a theme.
Because I often see node-based creation environments in tools like Unreal Engine (UE4), which I use at work, and Houdini, which I’ve been paying attention to recently, I chose “Node-Based UI” as the theme.

Just connecting nodes isn’t interesting, so I’ve included calculator functionality.

What I’ve made, as of writing this article, looks like this:

Currently, you can connect nodes to perform addition and subtraction.

ノードベース計算機を結構バージョンアップ。
・ノード間の線は3次ベジェ曲線に
・ノードのプロパティはポップアップで変更
・入力ピン数の動的変更#wpf #dotnetcore pic.twitter.com/2RFaWWyjXf

— ayuma (@ayuma_x) May 11, 2019

The complete project is available on GitHub.

• Note: This node-based calculator is still under development, so it is frequently modified.

References from NuGet

This time, I’ve added references to the following four from NuGet:

• Extended.Wpf.Toolkit
• gong-wpf-dragdrop
• Microsoft.Xaml.Behaviors.Wpf
• ReactiveProperty

Among these, Extended.Wpf.Toolkit and Microsoft.Xaml.Behaviors.Wpf don’t seem to support .NET Core 3.0 yet, resulting in the following warnings:

NU1701: Package 'Extended.Wpf.Toolkit 3.5.0' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETCoreApp,Version=v3.0'. This package may not be fully compatible with your project.
NU1701: Package 'Microsoft.Xaml.Behaviors.Wpf 1.0.1' was restored using '.NETFramework,Version=v4.6.1' instead of the project target framework '.NETCoreApp,Version=v3.0'. This package may not be fully compatible with your project.

For now, there are no issues with the functionality I’m using, and since .NET Core 3.0 is still a preview version, I’ve decided not to worry about it.

If these warnings persist even after .NET Core 3.0 becomes the official version, actions such as handling it myself and building might be necessary.

A Little Introduction to the Structure

I will focus on introducing the functionality for adding new nodes among what I created this time. (I thought about writing about drawing lines and performing calculations too, but since it’s actively under development, there’s a high possibility of future changes.)

Node Placement

The XAML for the View where nodes are placed is as follows.

<itemscontrol itemsource="{Binding Nodes}" 
        background="Transparent"
        dd:DragDrop.IsDropTarget="True" 
        dd:DragDrop.DropHandler="{Binding}"
        >
    <itemscontrol.itemspanel>
        <itemspaneltemplate>
            <canvas />
        </itemspaneltemplate>
    </itemscontrol.itemspanel>
    <itemscontrol.itemtemplate>
        <datatemplate>
            <node:NodeContainer />
        </datatemplate>
    </itemscontrol.itemtemplate>
    <itemscontrol.itemcontainerstyle>
        <style targettype="ContentPresenter">
            <setter property="Canvas.Top" value="{Binding PositionY.Value}"/>
            <setter property="Canvas.Left" value="{Binding PositionX.Value}"/>
        </style>
    </itemscontrol.itemcontainerstyle>
</itemscontrol>

By specifying Canvas for ItemsPanel, placement using absolute coordinates is enabled.

<itemscontrol.itemspanel>
    <itemspaneltemplate>
        <canvas />
    </itemspaneltemplate>
</itemscontrol.itemspanel>

Node position setting is handled by binding Canvas.Top and Left to the properties of the node’s ViewModel in ItemContainerStyle.

<itemscontrol.itemcontainerstyle>
    <style targettype="ContentPresenter">
        <setter property="Canvas.Top" value="{Binding PositionY.Value}"/>
        <setter property="Canvas.Left" value="{Binding PositionX.Value}"/>
    </style>
</itemscontrol.itemcontainerstyle>

Also, since node generation via drag-and-drop is incorporated into this ItemsControl, dd:DragDrop.IsDropTarget and dd:DragDrop.DropHandler are specified.

This is a feature of gong-wpf-dragdrop added via NuGet; writing it this way allows you to write the drop handling logic in the ViewModel bound to the ItemsControl.

<itemscontrol itemsource="{Binding Nodes}" 
        background="Transparent"
        dd:DragDrop.IsDropTarget="True" 
        dd:DragDrop.DropHandler="{Binding}"
        >

New Node Generation

The following source code is the process when dropping onto the View where nodes are placed.

dropInfo.Data contains the DataContext (bound ViewModel) of the dropped View, so the process changes depending on its type.

public void Drop(IDropInfo dropInfo)
{
    switch (dropInfo.Data)
    {
        case NodeViewModel node:
            {
                nowDragItem = null;
            }
            break;

        case NodeConnectionViewModel connection:
            {
                connection.LineToX.Value = 0;
                connection.LineToY.Value = 0;
                connection.Visible.Value = Visibility.Hidden;
            }
            break;
        case ToolBoxItemViewModel item:
            {
                var newItem = Activator.CreateInstance(item.NodeModelType) as NodeBase;
                if (newItem != null)
                {
                    newItem.PositionX = dropInfo.DropPosition.X;
                    newItem.PositionY = dropInfo.DropPosition.Y;
                    mainModel.Nodes.Add(newItem);
                }                
            }
            break;
        default:
            nowDragItem = null;
            break;
    }
}

The Drop process when creating a new node is the part that handles ToolBoxItemViewModel.

The dropped ViewModel has a property called NodeModelType of type Type, and based on this, an instance of the node is generated.

The generated node is given coordinates based on the drop position and added to the model’s Nodes array.

This addition to the array propagates from Model -> ViewModel -> View, and the new node is displayed in the ItemsControl explained earlier.

Also, where Activator.CreateInstance is used, I initially skipped the null check out of laziness, but since the nullable reference types feature was ON, a warning appeared.

Adding the null check made the warning disappear, apparently because the compiler performs flow analysis. Amazing!

case ToolBoxItemViewModel item:
    {
        var newItem = Activator.CreateInstance(item.NodeModelType) as NodeBase;
        if (newItem != null) // Null check added
        {
            newItem.PositionX = dropInfo.DropPosition.X;
            newItem.PositionY = dropInfo.DropPosition.Y;
            mainModel.Nodes.Add(newItem);
        }                    
    }
    break;

Summary

While writing this article, I intended to summarize any points like “you should be careful about this when using WPF with .NET Core,” but… there weren’t any in particular.

You can use it just like when using the previous .NET Framework version of WPF.

It was said that the inability to use the XAML designer in the .NET Core version was a disadvantage, but it seems it became usable in VS2019 Preview 3 (haven’t tried it yet), so I think this will also cease to be an issue eventually.

Also, regarding nullable reference types, I initially thought the warnings might be intrusive, but that wasn’t the case. Instead, I felt it was a helpful feature because it points out places where I might have overlooked potential null access.

Introducing it into existing projects might result in a flood of warnings and be difficult, but I think enabling it for newly created projects enhances code safety.

I believe .NET Core will become mainstream for future C# development, but I felt that developing desktop applications using WPF is also perfectly fine.

Code plugins written in C++ will likely undergo several version updates, and here’s a point to note during that process.

When updating content for an already listed Engine version, the Project File Link on the screen below needs to be a different link URL than the previous one.

It seems that if this URL is the same as the previous one, notification that the content has changed will not be sent.

I use Microsoft OneDrive, and overwriting a cloud file results in the same shared link URL as before. Therefore, I issued a different URL by first unsharing the file and then sharing it again.

The plugin I currently have listed on the Marketplace, ObjectDeliverer, only supports Windows.

Recently, I received several requests from users asking for support for other platforms, so I decided to reluctantly look into it.

I found many articles about multi-platform support for the final game package, but I had trouble finding much information specifically about plugin packaging.

However, since most of it is the same as game packaging methods, by piecing together various bits of information, I managed to get to a point where it seems likely to work.

The following information assumes a Windows development environment.

• As I will write at the end, only iOS is not working correctly.

Environment Preparation

First, from the Engine installation options, check the platforms you want to output to and install the necessary features.

Next, install additionally required software.

Android

Install the tools necessary for the Android development environment all at once using the following installer:

[UE4 Install Directory]\Engine\Extras\AndroidWorks\Win64\CodeWorksforAndroid-1R7u1-windows.exe

Details about this information can be confirmed from the link below.

Unreal Engine | 1. Settings Required for Android Game Development

Linux

For Linux, the official website has links to installers for the necessary software (toolchain), so download and install from there.

The toolchain differs for each Engine version, so installation is required for each Engine version you want to package for.

Links to the installers are available on the site below.

(The Japanese page only goes up to Engine version 4.19, so I am providing the link to the English page.)

Cross-Compiling for Linux

iOS

Since iOS cannot be built on a Windows machine, the method seems to be preparing a Mac machine on the same network and building by remotely logging in from Windows.

Therefore, remote login settings need to be configured.

For normal game packaging, it seems the settings are completed by entering the Mac machine name and login username in [ProjectSettings]-[Platforms]-[iOS]-[Build]-[Remote Build Options].

You can also create and register SSH Keys here.

However, this information is not used for plugin packaging, and it won’t remotely access the Mac.

After trying various things, adding the following description to [UE4 Install Directory]\Engine\Config\IOS\IOSEngine.ini made it attempt remote connection.

[/Script/IOSRuntimeSettings.IOSRuntimeSettings]
RemoteServerName=Mac_Machine_Name
RSyncUsername=Login_Username

Plugin Packaging

For the basic method of plugin packaging, please refer to the previous article.

Creating a Package for a Custom UE4 Plugin for the Marketplace - AYU MAX

Also, if creating via Editor operations instead of commands, it seems you can also Package from the Plugins window. (Learned this recently)

To create output for the targeted platform at this time, it is necessary to write the platform name in WhitelistPlatforms in the *.uplugin file. (It seems you can also specify via command line arguments instead of using WhitelistPlatforms)

Writing as follows apparently outputs for Windows (64bit, 32bit), Mac, Linux, iOS, Android, HTML5, so let’s try running the packaging command with this setting.

"WhitelistPlatforms": [
    "Win64",
    "Win32",
    "Mac",
    "Linux",
    "IOS",
    "Android",
    "HTML5"
]

Packaging Results

I judged whether packaging was successful by looking at the command results and the contents within the plugin output directory’s \Intermediate\Build.

The results were as follows:

As a result, generation succeeded for all except Mac and iOS.

Looking into it, it seems Mac currently cannot be built from Windows, and the only way for Mac is to install UE4 on a Mac and build from there.

iOS actually failed with an error during the command execution, and it didn’t work.

From the error message content, I think it’s related to provisioning, but I haven’t found a proper solution yet.

The official method is probably the procedure described in Unreal Engine | 3. iOS Provisioning - Generating Signing Certificates, but this procedure cannot be realized without completing iOS Developer registration, so I am looking for alternative methods.

I understand it’s absolutely necessary for game builds, but personally, I wonder if it’s also needed for plugin builds.

If I can’t find a method, I will obediently pay the tribute to Apple and try building with the official method.

If anyone knows a way to package plugins without paying the tribute, please let me know.

Summary

Although only iOS didn’t work, I was able to package the UE4 plugin for multi-platform support.

However, I am thinking of supporting only Windows for my plugin for the time being.

If there is a problem with the plugin released on the Marketplace, I think I can manage it with my knowledge for Windows, but I don’t have much experience with other platforms, so I currently lack the confidence to support them properly.

Releasing with the restriction of not supporting anything other than Windows might be an option, but I will think about it a little longer.

The latest plugin can already be downloaded from the Marketplace.

Please see the following article for what ObjectDeliverer is.

Easy Inter-Process Communication in UE4 with ObjectDeliverer - AYU MAX

Update Details

The updates this time are:

• Minor bug fixes
• Functionality to specify the destination when using a TCP/IP server

Bug Fixes

Fixed a bug where a null access crash occurred if PacketRule was not passed during the ObjectDelivererManager’s Start method.

Additionally, processing efficiency improvements (removal of unnecessary value copying) have been made.

Functionality to Specify Destination When Using TCP/IP Server

This feature was implemented in response to a request comment on the Marketplace content page.

With the existing Send method, when you are the server and multiple clients are connected, data would reach all clients. The SendTo method has been added to allow sending only to the intended client.

Incidentally, if the SendTo method is used when utilizing protocols other than TCP/IP server, it behaves the same as the Send method.

SendTo Sample

The destination passed to SendTo comes as a parameter in Connected events or ReceiveData events.

Therefore, by appropriately storing this value, sending to the targeted TCP/IP client is possible.

The following Blueprint uses the parameter from ReceiveData directly for SendTo.

With this description, it’s possible to implement a response transmission only to the client that sent the data when data is received.

Please try it out if you are interested.

#Varjo の体験もできた。凄い興味あったのでよかった。
確かに視野の中心はめちゃくちゃ綺麗で文字の滲みもまったくなかった。
ディスプレイの切れ目は分かるっちゃ分かる。
控えめに言って欲しい。#ue4fest pic.twitter.com/RuPpZAJ6JQ

— ayuma (@ayuma_x) April 20, 2019

Varjo VR-1

Varjo is a VR startup from Finland. The VR-1 is a PC-connected VR headset released in February 2019.

Features of the VR-1

The most significant feature of the VR-1 is its novel structure, which renders a narrow area in the center of the field of view with a high-resolution display and the surrounding peripheral vision with a low-resolution display.

The mechanical aspect of rendering with two displays of different resolutions is intriguing.

Considering how the human eye actually sees – clearly in the center but blurry in the periphery – this structure might be quite effective.

Furthermore, even though it’s called “low-resolution,” its resolution is apparently the same as the VIVE Pro, so we can expect it to be clearer than previous VR headsets in every aspect.

It’s also great that it supports SteamVR, meaning content previously made for VIVE or VIVE Pro will run as is!

Impressions from Actual Experience

Before the demo, I received a brief explanation. I was told that IPD adjustment is automatic, and if lines appear when wearing it, I should fine-tune the headset myself so that the lines connect horizontally.

The subsequent content was a demo where I could experience being inside a car interior, and it was truly beautiful.

Small text was clearly visible, and it was just stunningly clear!!

However, the boundary between the high-resolution and low-resolution displays was quite distinct, which was unfortunate.

Also, the feature I had seen online beforehand – “eye-tracking follows the direction of gaze, and the high-resolution display moves accordingly” – seemed not to be implemented.

Nevertheless, overall, it was clearer than any VR I had seen before, and I really want one.

I’m currently creating content using a VIVE Pro at work, so I can’t help but think how much clearer it would look with this device.

I asked the person explaining if there were plans for sales in Japan. They said nothing is decided yet, but adjustments are underway, so I want to remain hopeful.

However, the overseas selling price is apparently $6000, so it’s expected to be quite expensive in Japan as well.

Also, the GPU used for the demo was a Quadro RTX 8000, so it seems a considerable GPU spec will be required…

This is a large-scale official study event hosted by Epic Games Japan. It has been held annually in the Kanto and Kansai regions since 2014, when Unreal Engine 4 was made publicly available. *From the official website

Before Attending

I had applied for 2018 as well, but another commitment came up on the day, so I couldn’t go. This was my first time attending.

Being a large-scale study event, I wondered how many people would come, but seeing the final expected attendance number was over 600 people made me realize it truly was large-scale.

When deciding whether to bring my laptop, I hesitated because it’s heavy and wouldn’t last the whole day without power, so I decided against it.

However, the venue was a hall-like place at Kyoto Computer Gakuin, and there were tables and power outlets at the seats, so it seemed usable even if I had brought it.

In the end, during the lectures, I only used Twitter and took notes, so the laptop wasn’t necessary.

From Arrival to Start

I had read on the official website beforehand that there would be an after-party following the fest, and thinking it sounded interesting, I definitely wanted to attend.

To participate, it was necessary to buy a ticket on the day.

Seeing information that the tickets were limited in number, I thought it best to arrive early and got there around 9:30 AM, 30 minutes before registration started at 10:00 AM.

When I arrived, there was already a queue of about 20 to 30 people. After that, the line kept growing longer, indicating its popularity.

At 10:00 AM, registration began, and I entered. At the reception, I showed the screen of the Peatix app on my smartphone to complete registration. I received a tote bag as a visitor gift. Despite being a free gift, this bag is well-made and cool.

After completing registration, I went to the merchandise corner next door.

I saw the pink band, which was the ticket for the after-party, so it seemed I made it in time.

Paid 5000 yen and successfully got it.

#ue4fest pic.twitter.com/s7sEBeWAq3 — ayuma (@ayuma_x) April 20, 2019

However, I saw the words “Limited to 25” at the sales counter, and thought, “Eh, 600 people are coming, but there are only 25? It’s really rare. If I had come a little later, it would definitely have been sold out…”

Also, I personally wanted to buy stickers or T-shirts, but this time only the after-party tickets were sold, which was a bit disappointing.

I wanted to put a UE4 sticker on my laptop…

There were also company exhibits around the reception area, and I found something extremely interesting among them, so I went to see it right after completing registration. I will write a separate article about this.

After looking around the other exhibits a bit, I went to the venue and waited for the start.

1) Unreal Engine Roadmap

The first talk was by Axel from Epic. I heard many interesting stories about the latest features of UE4.

It started in French, which was a bit surprising. After that, the explanation was in fluent Japanese and easy to understand.

Below are the contents that caught my interest.

Epic Online Services

Epic Online Services | Services

They will release an SDK that provides online service functionality.

Of course for UE4, but I was surprised that they would also release it for Unity!

Since there’s also a C API, it seems possible to support various platforms if you try.

I was moved by the words, “It’s free. There are no hidden catches.”

Quixel Movie

I was shown an incredibly beautiful movie made using UE4. It truly looked like live-action, and I couldn’t hide my surprise that it was real-time rendering.

Live Coding

A feature included since 4.22, it allows C++ changes to be reflected faster than the previous hot reload. Personally, I thought the script reflection time was an area where UE4 lagged behind Unity, so this is a very welcome feature.

Now I can use C++ with less stress from waiting times!!!

Ray Tracing

There was also an explanation and demo of the current highlight, real-time ray tracing. The mirror reflections really change when enabled. With the latest drivers, it seems usable even on GTX 1060 and above, which is great for trying it out.

Researching later, I found out the movie above was made with a single 2080 Ti, which made me feel its awesomeness even more.

Unreal Insights

Information announced for the first time at this Unreal Fest. A high-performance tracing framework that is said to make performance and crash analysis easier. Even now, load confirmation etc. can be done with various commands, but I’m looking forward to seeing how things change when this is introduced.

Chaos

A new physics destruction system apparently testable in the next version, 4.23. Watching the footage of buildings being destroyed, the reality was incredible, as if real objects were being destroyed. Making a game using this seems like it would be exhilarating!!

2) What I Learned from Listing a Shrine Asset on the UE4 Marketplace

Shinto Shrine：Motonak：Environments - UE4 Marketplace

A lecture by background artist Mr. Nakamura.

Since the content was non-public, I won’t write details, but I heard interesting stories about the Marketplace.

I also listed one item recently and was thinking of listing another next, so it was very helpful.

3) Hearing from the Students Who Became Number One in Japan: How to Conquer the Japan Game Awards

A panel discussion with the team members who won the Grand Prize in the “Japan Game Awards 2018 Amateur Division”.

It was the first time I saw footage of the game, but even at a glance, the quality was very high, and I actually wanted to try playing it.

Listening to their stories, it strongly felt like they won the Grand Prize not by chance, but by aiming for it. Although they are students, I felt a strong sense of professionalism.

Stories that left an impression included: “There are reasons for everything, including the game’s concept, and we can explain them properly.” “Since the game’s concept is mirrors, we bought real mirrors and studied them.” etc.

Indeed, having reasons for the development concept prevents the plan from wavering midway, and I felt that’s the same in work. (Lately, my own activities have been quite plan-wavering…)

It was content I could relate to in many ways.

4) Utilization in Non-Game Fields in 2019

I heard about three patterns of UE4 utilization examples outside of games.

In the introduction video of Reality, a video compositing tool? using UE4, I was shown incredibly accurate keying down to the level of individual hairs, and felt it was an amazing technology.

Also, UE4 is apparently now used for filming movies etc., and sometimes prototypes are made with UE4 to check the visuals before actual shooting.

The third one, Unreal Studio, was a familiar topic as I use it myself. By smoothly importing from CAD tools into UE4, UE4 is also used in the manufacturing industry.

5) Utilizing Console Assets? UE4 Utilization Case Study for the Mobile Game “Winning Eleven 2019”

A story about replacing the game engine of Winning Eleven with UE4.

Not just replacing it, but a story of successfully replacing the engine under circumstances with no time margin and no turning back.

Although spoken calmly, thinking about it carefully, I felt this might have been an incredibly dangerous story.

The people involved at the time must have been extremely tired…

Also, I thought I might want to try using the CRI File System mentioned in the talk. The term “dual file system” using it sounded cool.

6) Cloud Expression in “Ace Combat 7” Using UE4 x TrueSky

A story about expressing realistic skies in Ace Combat 7 using UE4 and TrueSky.

Thinking it was a game development talk, it started with “What are clouds?” and began like a physics lesson.

Using middleware called TrueSky allows creating realistic clouds, but to do so, it seems necessary to understand phenomena like why clouds form, what their components are, etc.

https://t.co/pJCuZJQeUj #ue4fest — ayuma (@ayuma_x) April 20, 2019

Apparently, as you master it, you start feeling things like “Today’s atmospheric color is deep” or “Today’s sky Mie scattering is strong” when looking at the actual sky.

It truly felt like professionals of the sky and clouds.

Also interesting was the story of various工夫 (ingenuity/contrivances) made to maintain 60fps in the PS4 VR version.

To make it lighter, they apparently even modified TrueSky itself, making me feel the programmers are strong.

Listening to the talk made me want to play Ace Combat!!

7) Automatic Background Generation Using Procedural Techniques: “1,000 Japanese Rooms”

A story about creating a mechanism for automatic generation of Japanese rooms using “Houdini,” which I often hear about recently.

Not just automatic generation, but the quality was also amazing, to the level where comparing it with photos of real rooms, it was hard to tell which was real.

Artistic nodes!! #UE4fest — ayuma (@ayuma_x) April 20, 2019

Received the power word “artistic nodes,” causing laughter!

However, reflecting on the Blueprints I’ve written in the past, I thought, “I have quite a few artistic nodes too…”

What strongly impressed me was the mechanism for connecting the kotatsu cord to the outlet, avoiding obstacles.

I felt it was genuinely amazing because trying to create this by programming it myself would require quite a bit of head-scratching.

After Party (Social Gathering)

After the fest ended, there was a social gathering for ticket holders and related personnel.

Although I didn’t know anyone, I was able to talk and exchange business cards with several people and had a fun time.

In the rock-paper-scissors game, I lost just before winning and couldn’t get a prize… I really wanted that thing with the UE4 shortcuts written on it.

But I was happy to be able to talk to Nan-san, the creator of Nanana, which I recently fell in love with at first sight on the Marketplace and purchased. Iron Fist Girl：Indie-usGames：Characters - UE4 Marketplace

This asset is really Cool!

Next is Yokohama

Although it was my first time participating in Unreal Fest, the large venue and the content, all of which were interesting, were truly stimulating.

I’ve been using UE4 for work for several years and creating various things, so I want to maintain this motivation and continue creating good things.

Unreal Fest happens twice a year, and the next one seems to be in Yokohama, so if my schedule allows, I’d like to go again.

Thank you to the related personnel for a wonderful fest.

The official version of UE4 Ver 4.22 was released the other day. So, I made my plugin (ObjectDeliverer) submitted to the Marketplace compatible with 4.22, and I will record the procedure here.

Please refer to the previous article for the plugin being updated this time, ObjectDeliverer.

Easy Inter-Process Communication in UE4 with ObjectDeliverer - AYU MAX

Create Package with 4.22

First, create a package for the custom plugin using the 4.22 engine. Please refer to the previous article for the method of creating a plugin package.

Creating a Package for a Custom UE4 Plugin for the Marketplace - AYU MAX

I thought the C++ rules probably hadn’t changed from 4.21 to 4.22, so I expected it to build smoothly, but I got several C5038 errors.

C5038のエラーが原因っぽい。メンバ変数の宣言順に初期化してないのが原因？？

— ayuma (@ayuma_x) April 6, 2019

However, I haven’t been able to determine whether this is due to the UE4 version update or a Visual Studio update.

After fixing the errors and confirming there were no issues through testing, I created the package.

Work in Marketplace Publisher Portal

From here, update the Marketplace registration. Go to the Marketplace Publisher Portal page and select the published plugin.

If you look closely, there’s a warning saying, “Your product is not compatible with the latest UE4.”

This time, the only change is adding the 4.22 compatible plugin, so scroll down the page to PRODUCT FILES.

Then, add an item for 4.22, set the URL of the 4.22 compatible plugin uploaded to the cloud, and configure the necessary items.

Once the settings are complete, press the “Submit” button to finish the update procedure.

Confirmation Email

After a while, an email titled “Unreal Engine Marketplace: Pending File Update” arrived from Epic, stating that the plugin update was received. I think the Marketplace information will be updated after a while.

I was able to achieve my recent goal of listing on the UE4 Marketplace.

What I released is a Code Plugin called ObjectDeliverer.

It’s priced as Free, so anyone can use it.

ObjectDeliverer by ayumax in Code Plugins - UE4 Marketplace

It’s primarily a library for inter-process communication between UE4 applications and other applications.

It might not be a perfect match for those using UE4 for game development, but I think it will resonate with those using it for other purposes and wanting to link with other applications.

I myself am currently using it for content creation utilizing VR.

I have written separate articles about this library, so please take a look if you are interested.

Easy Inter-Process Communication in UE4 with ObjectDeliverer - AYU MAX

History Leading Up to Release

Just in case, I’ll jot down how things went leading up to the release.

2018/12

Thought of writing a communication library for UE4 and started working on it. Decided to aim for a Marketplace release while I was at it.

2019/1

Completed the first version of the library. Tested it by using it myself. Fixed all the bugs found.

2019/2/7

Learned that packaging is necessary for Marketplace submission, researched → implemented.

Creating a Package for a Custom UE4 Plugin for the Marketplace - AYU MAX

2019/2/10

Registered as a seller on the Marketplace and registered the created package. Completed everything while feeling anxious about whether the entries were correct.

The email after registration stated it would take 15 to 20 business days for the review to complete.

Submitting to the UE4 Marketplace - AYU MAX

2019/2/16

Received an email saying it would be reviewed within 5 business days.

Was a bit surprised as I was prepared to wait for about a month.

2019/2/18

The review was finished, and some minor corrections were needed.

Specifically, I was pointed out to add a Copyright notice at the beginning of the C++ source files.

Also, I was asked to describe the MarketplaceURL in the *.uplugin file and re-register the file.

Implemented the corrections and re-registered on the same day.

2019/2/19

Received notification that it would proceed to the final review within 3 to 5 business days.

2019/2/19

Received an email just before the date changed, stating the status had become APPROVED. It said I just needed to log into my product page myself and change the status to PUBLISHED.

Did it immediately, and the release was successfully completed.

Checked on the Epic Games Launcher, and it was indeed there!!

Managed to Release by Trying Hard

It had been years since I last released an iOS app to any store, so it was a lot of fun.

Above all, there’s a sense of accomplishment.

The review, which I thought would take a long time, ended up taking only 10 days, so the wait was totally bearable, and more importantly, I was relieved it wasn’t rejected.

Thank you to the Epic staff who reviewed it.

That said, releasing a library to the store can be considered the start, so I want to maintain it properly.

I still have 2 or 3 more ideas in my head, so it would be great if I could release again.

Since UE4 4.21, a feature has been added that allows slight editing of Static Meshes, which is quite convenient, so I’m making a note of it.

The modeling features added in 4.20, introduced in this feature showcase, are more advanced, but the feature I’m writing about this time is likely a separate one.

Trying out UE4’s New Polygon Modeling Feature (Mesh Editor) - Let’s Enjoy Unreal Engine

Usage Conditions

A project with Unreal Studio configured in 4.21 is required.

If Unreal Studio is not configured or is disabled (*.uproject’s “Enterprise”: false), it will not appear in the display.

Assigning a New Material to Arbitrary Polygons of a Static Mesh

Let’s try assigning a material to the mesh’s polygons using the Mesh Editing feature mentioned above.

I downloaded the mesh used this time from here.

free c4d model retro tv

• Open the downloaded Static Mesh.

• Select the "Mesh Editing" tab at the top of the window.

• Press the "EditMode" button and select the polygons. You can select multiple by holding Ctrl while selecting.

• Select the material you want to assign with the mouse in the Content Browser.

• Return to the Mesh Editing screen and press "Assign Material" at the top of the window. A Material Slot will be added, and the material selected just before will be assigned.

Other Features

You can delete selected polygons, flip UVs, etc.

It’s convenient because you don’t have to go through fbx export → editing in a modeling tool like Blender for minor changes.

The Razer Blade laptop that arrived at my home late last year is a very welcome presence, as it allows me to use UE4 even at home.

However, I had been troubled by the loudness of its fan for some time.

At home, I use an Oculus Rift for VR development, and the fan noise when starting Oculus is intense.

However, I can tolerate that because I understand the load is high. But even when doing nothing, the fan occasionally spins at high speed when a single external display is connected.

This is a bit stressful during quiet nights, and since the PC itself gets quite hot, I’m also concerned about its durability.

So, I purchased this item.

Cooler Master Laptop Cooler NOTEPAL X-SLIM II

The price wasn’t too high, and with a 20cm fan, I bought it hoping that the cooler itself would be relatively quiet.

This is how it looks with the Razer Blade on it.

It’s powered via USB connection from the PC.

The power USB connector itself has a USB pass-through port, so the port used by the cooler isn’t wasted.

After actually using it, perhaps because the fan is large, the noise from the cooler itself wasn’t noticeable.

The sound of the air conditioner running in the room is louder.

As for the crucial effect on the laptop, it’s not just my imagination; the fan noise has certainly become less noticeable.

Even with one external display connected, a situation where the fan would normally spin up occasionally, it now maintains a quiet state.

Also, perhaps thanks to the constant airflow from below, the area next to the touchpad, which usually gets hot, doesn’t get hot anymore.

Naturally, the fan still spins up when launching UE4 or similar applications, but even then, I can feel it’s quieter than before.

One point of caution is that there’s no power switch, so it’s better to connect the USB to a port that stops supplying power when the PC is turned off.

If you’re troubled by laptop fan noise, consider giving it a try.

I’ll write down the steps I took to submit ObjectDeliverer to the Marketplace as a memo.

• The input items on the registration page vary depending on what you select beforehand. Here, I will describe the items for submitting ObjectDeliverer.

* Currently awaiting review, so there’s a possibility of errors in the steps described here. If such things are found, I will correct them sequentially.

• Successfully listed on the Marketplace, so these steps seem to be OK. (2019/02/20)

The Marketplace is the UE4 store accessible from the Epic Games Launcher or the web.

Since UE4 is a game engine, the impression is that most available items are assets like 3D models, textures, materials, effects, and sounds, but libraries for creating program logic in C++ and Blueprint are also available.

This time, I will summarize the procedure for publishing my own created Plugin here.

Seller Registration

First, register as a seller.

Clicking the “Get Started!” button on the site below transitions to the registration screen.

• In my environment (Windows 10, Chrome), there was a frequent phenomenon where the page would freeze upon clicking the button and transitioning, erasing the input content. Changing the browser to Edge avoided this phenomenon, so please refer to this if something similar happens.

Publisher Portal

Registration Information

First, register personal information such as name, phone number, and email address.

The content registered on this page is registered only with Epic Games and is not disclosed externally.

Seller Public Information

Next, register the information to be disclosed as a seller.

Required are the seller name and support email address. Others seem optional, so enter information that can be disclosed, such as homepage, Twitter, Facebook, etc., if available.

Payment Information

Enter the information to receive payments when selling paid content.

You can choose between Paypal and bank transfer, but bank transfer involves troublesome overseas transfers, so I registered with Paypal.

Tax Information

Enter tax-related items. The input items may differ for corporations, but for individual developers in Japan, it’s mostly about registering the address.

Registration

That completes the input of registration information. After registering, an email will arrive, so click the link within the email.

Listing

Now that seller registration is complete, list the created content.

Go to the Publisher Portal and click the “ADD PRODUCT” button to proceed with registration.

The display here changes depending on the selection state of the dropdown menu.

The following explanation only covers the items displayed during ObjectDeliverer’s registration.

Registering Images

Register the images required for listing.

• GALLERY
  Register 25 or fewer images of 1920x1080.
  These are the images displayed on your content's detail page.
  For 3D models etc., I think this is where you express what's included in the content with screenshots.
• Thumbnail
  Register one image of 284x284.
  Used for list displays, such as when searched on the Marketplace.
• FEATURED IMAGE
  Register one image of 894x488.
  I couldn't figure out where this is used.
  Maybe I just haven't found it, but the description also says "may be used," so perhaps it will be displayed somewhere in the future.

Registering Content Information

• PRODUCT TITLE
  Enter the name of the content to be registered.
• Price
  Select the selling price. Choose Free for free content, or select an amount from the list for paid content.
• CATEGORY
  Select the type of content to be registered.
  For example, "Characters" for characters, "Materials" for materials.
  Since ObjectDeliverer is a plugin, I selected "Code Plugins".

• THIRD PARTY SOFTWARE USAGE (Appears when Code Plugins is selected in CATEGORY)
  Select whether it includes software created by a third party.
  If your library references another library, select "This Product Uses Third Party Software".
• TAGS
  Enter tags related to your content.
  Helps users in searching.

• SHORT DESCRIPTION
  Register a brief description of the content.
  Max 255 characters, but within 100 characters is recommended.
• Long Description
  Describe in detail within 1900 characters.
  Describe videos or links for explanation here.
• Technical Information
  Describe how to use the registered content.
  Links to documentation for explanation are described here.
  A template is displayed in the input field, so following it should be fine, but looking at the Marketplace, displays different from this template are also seen, so it doesn't seem mandatory.

• PROJECT VERSIONS
  Register your content files.

  If different files are needed for each Engine version, register multiple.

  ObjectDeliverer supports 3 versions: 4.19, 20, 21, but since different packages are needed for each, I registered 3 files.

  Enter the URL where the downloadable zip file for each file is located.

• The method for creating a Plugin package is posted as a separate article here. Creating a Package for a Custom UE4 Plugin for the Marketplace - AYU MAX

After entering everything and confirming there are no mistakes, click the “SUBMIT FOR APPROVAL” button to complete.

Awaiting Review

An email confirming receipt will arrive at the registered email address.

This email also includes an estimate of when the review will take place.

In my case, it was “15-20 business days”.

It’s quite a wait, but perhaps it’s just that busy right now.

All that’s left is to wait and pray it doesn’t get rejected.

Since the end of last year, my craving for a keyboard had been growing, and after agonizing over what to do for a long time, I finally made a purchase.

買ってしまった。
買う前はAll 30gにしようと思ってたけど、なんか軽すぎる感じがしたので45gのやつに。
やっぱり触ってから買うようにしてよかった。#realforce pic.twitter.com/giz3G0Weva

— ayuma (@ayuma_x) February 10, 2019

The Trigger

It all started last year when my development environment PC at work was replaced, and the keyboard became a REALFORCE.

The one at work is probably the one below, which can be purchased together with a Mouse Computer PC.

Product : REALFORCE 108UG-HiPro | REALFORCE | The pinnacle of Japanese premium keyboards

I didn’t know much about keyboards and had used them without much preference until then. Before that, I used the Magic Keyboard that came with my iMac.

So, since I was used to thin, space-saving types, I couldn’t get used to it at first.

It felt big, took up space, was heavy, the keys were much deeper than the Magic Keyboard, and it just felt difficult to use.

However, after using it for nearly a year, I suddenly realized how pleasant it was (late, I know).

Then, the desire to change my home keyboard emerged.

At home, wanting something similar to the Magic Keyboard at the time, I was using the Microsoft one below.

Designer Bluetooth Desktop | Microsoft Accessories

Although I liked its appearance, being Bluetooth, there was a delay before typing registered, and the key layout was also a bit awkward for me.

Also, my desk at home is small, so I was dissatisfied that a keyboard with a numeric keypad took up space.

So, I decided to buy a good keyboard for home as well and started considering options.

REALFORCE or HHKB?

First, I considered REALFORCE, which was the trigger.

Wanting to save space, I first narrowed my search to tenkeyless models, but then several other parameters emerged to agonize over, such as “Silent or Normal,” “APC available,” and “Key input weight,” and I worried/agonized for quite a few days.

Also, whether to stick with a JP keyboard as before, or switch to EN. I thought EN looked cooler, but decided on JP because I thought my head would get confused if it wasn’t consistent with work.

Also, sites writing about REALFORCE often mentioned HHKB, so I researched that too. I was ignorant before and didn’t know, but HHKB uses Topre keys, right?

Also, I sometimes saw people at study sessions using small keyboards placed on top of their laptops, and I realized that the ones I saw, thinking “they must have strong preferences,” were likely HHKB.

The major differences between HHKB and REALFORCE are the number of keys and weight. Honestly, I started thinking the HHKB, which eliminates waste to the extreme, was also good, making it even harder to decide what to buy.

I think the decision criteria were probably:

• Use at home only, no carrying & prioritizing the same environment as work → REALFORCE
• Carry it for work outside & can manage even if the environment differs from work → HHKB

I summarized it like that, but couldn’t decide.

Unable to Decide, Went to See Them in Person

Lately, I often buy things online, but for this, I thought I really needed to touch the real thing, so I went to a store.

First, I went to Yodobashi, but only one full-size REALFORCE was on display, and no tenkeyless models.

When I asked the staff, they brought out two tenkeyless EN keyboards from the back. Unfortunately, there were no JP tenkeyless models.

However, wanting to try more types, I went straight to Tsukumo.

Thinking they might not have them here either, I got excited when I found a dedicated REALFORCE section!!

I tried the tenkeyless models one by one from the end, and there were many things I wouldn’t have known without actually touching them. Specifically, knowing the following two things was significant:

• There's really quite a difference in sound volume between the normal and silent models (or rather, the sound quality? is different)
• Before touching them, I thought the light ALL 30g would be good, but it was too light for me and didn't suit me

I couldn’t find HHKB at Tsukumo, and although I wondered what to do, I feared that touching it would make me hesitate and unable to decide, so I stopped searching further and decided on the REALFORCE R2TLS-JP4-BK right there.

The price was around 25,000 yen including tax.

Previously, it was an amount I would never spend on a keyboard, but I convinced myself with the strange excuse, “It’s cheaper than the HHKB Type-S, so I’m lucky,” and bought it.

Trying It Out

I’m writing this article on the day I bought it, so typing this article is essentially the trial run.

That’s why it ended up longer than expected.

Using it, the “thock thock” feeling when typing is really pleasant. The key layout is the same as the one I use at work, which is convenient, so I’m glad I chose this one.

It’s compact in size, but quite heavy, so carrying it seems tough, but I only take my PC outside about once or twice a month, so it seems acceptable.

I’ll try writing on the blog again after using it for a longer period if my feelings change.

Since the data send/receive plugin ObjectDeliverer I am currently creating for Unreal Engine is aimed for submission to the Marketplace, I tried packaging it once.

How To

Packaging a UE4 project can be done from the Editor, so I know how to do that, but I had never packaged a Plugin before and wasn’t sure how to do it, so I searched.

How To Package Plugins For UE4 Marketplace - Epic Wiki

I found a page that was exactly what I was looking for, so I basically followed its instructions.

Describing WhitelistPlatforms

It seems necessary to describe which platforms the package is for in the .uplugin file. Since ObjectDeliverer has currently only been confirmed on Windows, this time I limited it to Win64 and Win32 only.

{
  "FileVersion": 3,
  "Version": 1,
  "VersionName": "1.0.0",
  "FriendlyName": "ObjectDeliverer",
  "Description": "ObjectDeliverer is a data transmission / reception library for Unreal Engine (C ++, Blueprint). ",
  "Category": "Programming",
  "CreatedBy": "ayumax",
  "CreatedByURL": "https://github.com/ayumax",
  "DocsURL": "",
  "MarketplaceURL": "",
  "SupportURL": "https://github.com/ayumax/ObjectDeliverer",
  "EnabledByDefault": false,
  "CanContainContent": false,
  "IsBetaVersion": false,
  "Installed": false,
  "Modules": [
    {
      "Name": "ObjectDeliverer",
      "Type": "Runtime",
      "LoadingPhase": "Default",
      "WhitelistPlatforms": [
        "Win64",
        "Win32"
      ]
    }
  ]
}

Creating the Build Batch File

Following the instructions, I created a build batch file. I saved a file with the following content as buildPlugin.bat.

Following the explanation page, instead of the folder where the Plugin’s .uproject is located, I created an empty build folder elsewhere and placed buildPlugin.bat there.

"C:\Program Files\Epic Games\UE_4.21\Engine\Build\BatchFiles\RunUAT.bat" BuildPlugin -Plugin="C:\source\repos\ObjectDelivererTest\Plugins\ObjectDeliverer\ObjectDeliverer.uplugin" -Package="%CD%\PluginStaging_ALL\UE4_421" -Rocket

Running the Batch File

In the folder where the batch file was placed, hold down the left Shift key while right-clicking, select “Open PowerShell window here” to open a PowerShell console, and execute by typing .buildPlugin.bat.

Crushing Errors

During development, I had always launched and checked with DebugGame Editor, so I hadn’t built for Shipping. Probably because of that, there were many places where the description was lax, and build errors occurred frequently…

Most of the errors were in places where includes were written abbreviatedly.

For example, writing #include "TcpSocketBuilder.h" where #include "Common/TcpSocketBuilder.h" should have been written.

Since most were of the above pattern, I rechecked and corrected the includes in all C++ sources.

After Execution

If there were no errors during the build, waiting until it finished resulted in the PluginStaging_ALL\UE4_421 folder being generated in the folder where the batch file was placed, containing the complete Plugin set.

Summary

Referring to the page with the procedure, I succeeded in creating a package for my custom plugin.

Now, I need to think about the description for Marketplace submission, and what seems subtly troublesome is preparing the images for submission. This is similar for other app stores…

I need to make it properly so it doesn’t fail the review.

When implementing communication features, there are almost always times when you want to reproduce the situation later.
This is especially essential when the environment of the communication partner is difficult to prepare.

Therefore, I implemented a feature in ObjectDeliverer to save communication content to a log file and play it back.

Sample

Here is a sample using the save and playback features.

The communication content here is a captured bitmap of 800px x 450px 32bit ARGB, so 1,440,000 bytes = (800 x 450 x 4) of data are sent and received every 60fps (=16.66ms).

The data flow repeats as follows: [WPF App] → (Shared Memory) → [UE4 App] → (Log File)

Since the video capture for recording was also running in parallel, it was quite choppy, but it managed to work somehow.

ObjectDelivererのログ保存機能と再生機能のサンプル。
WPFのアプリの映像を共有メモリ経由で表示しながらログファイルに保存して、後から読み込んでもう一度再現してます。
ちょっと重くてカクついてますが。#UE4 #UE4Study #ObjectDeliverer pic.twitter.com/fbdxyxK9nO

— ayuma (@ayuma_x) February 6, 2019

Log File Saving Feature

Set and use ProtocolLogWriter during ObjectDeliverer’s Start.

Calling Send writes data to the log file.

Each write to the log file also records the elapsed time from the start in millisecond order.

Log File Playback Feature

Set and use ProtocolLogReader during ObjectDeliverer’s Start.

Calling Start triggers the ReceiveData event each time the time recorded in the log file elapses. Therefore, you can process the received data just like in normal communication with a partner.

Summary

By implementing the log file saving and playback features, I was able to cover a significant portion of the functionality I initially wanted to implement in ObjectDeliverer.

All that remains is to implement Object sending/receiving using serialization with MessagePack, and then it will be a good stopping point.

Today’s article is a memo recording a problem I encountered recently. By the way, I still don’t know the exact cause and am currently using a workaround.

In the Plugin I’m currently creating, I wanted to override functions in both C++ and Blueprint, so I implemented the pattern below in various places.

UFUNCTION(BlueprintNativeEvent, Category = "ObjectDeliverer")
void Start();
virtual void Start_Implementation();

In the example above, Start() is expected to be called from another C++ class.

The intention was that a C++ class inheriting this class would override Start_Implementation(), and a Blueprint would override Start(), making it okay to inherit the class in either way.

After implementing and running it, it seemed to work fine for a while, but I noticed it occasionally crashed when Start() was called.

うーん、このProcessEventのIsUnreachableにひっかかって落ちる問題なんだ？
BlueprintNativeEventの関数使っているところが該当するってことはわかったんだけど、どう直したらよいのか不明だ。

— ayuma (@ayuma_x) January 29, 2019

Debugging and tracing it, I found that ProcessEvent was called within Start(), and the function was being executed via reflection.

I understood that it was crashing at the check(!IsUnreachable) within it, but I have no idea why that was happening.

Does IsUnreachable mean it became a target for Garbage Collection (GC)??? I thought, and checked the code, but it was properly held by a variable specified with UPROPERTY().

Just in case, I decided to try another approach and also tried the content of the article below, but the phenomenon didn’t change.

How to prevent an object from being garbage collected without using UPROPERTY - Qiita

Even among the places using BlueprintNativeEvent, it crashed at two different timings: sometime after startup, and during BeginDestroy.

BeginDestroy is still understandable, perhaps such function calls shouldn’t be made during termination, but I don’t understand the other pattern.

However, what’s slightly unusual is that it crashes within a worker thread run using FRunnableThread, so I wonder if that’s related.

In the end, I couldn’t figure out the cause or how to fix it, leading to the pathetic conclusion of giving up on Blueprint overriding and removing BlueprintNativeEvent, changing it to a normal virtual function.

After that, it stopped crashing completely, which is good for now, but it’s bothering me, so I want to take the time to investigate it properly somewhere.

The fact that it doesn’t crash afterwards means it wasn’t collected by GC, which makes it even more confusing…

This article explains the implementation of the shared memory sample I wrote about yesterday.

ObjectDelivererの共有メモリのテスト題材作成中。
WPFアプリが毎フレーム自分のウィンドウを共有メモリに書き込んで、それをUE4側が読み込みテクスチャを更新してる。#ObjectDeliverer #UE4 #UE4Study pic.twitter.com/uWY2QhHnYP

— ayuma (@ayuma_x) January 29, 2019

WPF App Side

The WPF side writes the display content of its own window to shared memory.

The complete source code is in the following repository.

Creating Shared Memory

Create shared memory named “SharedMemory” with a size of 800 * 450 * 4 + 5. This size can store a bitmap of 800 pixels width, 450 pixels height, and 32-bit BGRA format. The final +5 is for the header elements used by ObjectDeliverer shared memory.

private MemoryMappedFile share_mem;
private MemoryMappedViewAccessor accessor;

share_mem = MemoryMappedFile.CreateOrOpen("SharedMemory", 800 * 450 * 4 + 5);
accessor = share_mem.CreateViewAccessor();

Hooking into WPF Frame Rendering Timing

Monitor the CompositionTarget.Rendering event to enable processing on every rendering frame.

CompositionTarget.Rendering += CompositionTarget_Rendering;

Writing Display Content to Bitmap

Root is the Name given to the control whose display content is to be obtained. First, get the size of the display area using VisualTreeHelper.GetDescendantBounds, and create a RenderTargetBitmap matching that size.

Next, create a DrawingVisual and then write the display content to the bitmap. Once writing is complete, copy the content of the bitmap’s pixel buffer into a byte array. This byte array is what we need this time.

var visual = Root;
var bounds = VisualTreeHelper.GetDescendantBounds(visual);
var bitmap = new RenderTargetBitmap(
            (int)bounds.Width,
            (int)bounds.Height,
            96.0,
            96.0,
            PixelFormats.Pbgra32);

var dv = new DrawingVisual();
using (var dc = dv.RenderOpen())
{
      var vb = new VisualBrush(visual);
      dc.DrawRectangle(vb, null, bounds);
}

bitmap.Render(dv);
bitmap.Freeze();

var buffer = new byte[800 * 450 * 4];
bitmap.CopyPixels(buffer, 800 * 4, 0);

Writing to Shared Memory

Finally, perform the write to shared memory.

First, to conform to the ObjectDeliverer format, write a 1-byte counter. ObjectDeliverer only reads the subsequent content if this counter’s number changes, so it’s incremented with each write.

After the counter, write the size of the pixel buffer.

Next, copy the content of the pixel buffer, and it’s complete.

The above writing is performed every frame.

// write counter
accessor.Write(0, counter++);

// write buffer size
accessor.Write(1, (int)buffer.Length); // Cast to int for Write

// write buffer
accessor.WriteArray(5, buffer, 0, buffer.Length);

UE4 Side

The UE4 side reads the pixel buffer from shared memory using ObjectDeliverer and updates the texture using its content.

Creating a Texture Helper Class

As preparation for the necessary functionality, create a UTextureUtil class in C++.

Implement the following two functions in it.

// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "Kismet/BlueprintFunctionLibrary.h"
#include "Engine/Texture2D.h"
#include "TextureUtil.generated.h"

/**
 *
 */
UCLASS()
class OBJECTDELIVERER_API UTextureUtil : public UBlueprintFunctionLibrary
{
    GENERATED_BODY()

public:
    UFUNCTION(BlueprintPure, Category = "ObjectDeliverer Test")
    static UTexture2D* CreateTexture(int32 Width, int32 Height);

    UFUNCTION(BlueprintCallable, Category = "ObjectDeliverer Test")
    static void UpdateTexture(UTexture2D* Texture, const TArray<uint8>& PixelsBuffer);

};

// Fill out your copyright notice in the Description page of Project Settings.

#include "TextureUtil.h"
#include "Engine/Texture2D.h"
#include "RenderingThread.h"

UTexture2D* UTextureUtil::CreateTexture(int32 Width, int32 Height)
{
    auto Texture = UTexture2D::CreateTransient(Width, Height, PF_B8G8R8A8);
#if WITH_EDITORONLY_DATA
    Texture->MipGenSettings = TMGS_NoMipmaps;
#endif
    Texture->CompressionSettings = TC_VectorDisplacementmap;
    Texture->SRGB = false;
    Texture->UpdateResource();

    return Texture;
}

void UTextureUtil::UpdateTexture(UTexture2D* Texture, const TArray<uint8>& PixelsBuffer)
{
    if (!Texture || !Texture->Resource || PixelsBuffer.Num() == 0)
    {
        return;
    }

    FUpdateTextureRegion2D* Region = new FUpdateTextureRegion2D(0, 0, 0, 0, Texture->GetSizeX(), Texture->GetSizeY());

    struct FUpdateTextureRegionsData
    {
        UTexture2D* Texture;
        int32 MipIndex;
        uint32 NumRegions;
        FUpdateTextureRegion2D* Regions;
        uint32 SrcPitch;
        uint32 SrcBpp;
        uint8* SrcData;
    };

    FUpdateTextureRegionsData* RegionData = new FUpdateTextureRegionsData;

    RegionData->Texture = Texture;
    RegionData->MipIndex = 0;
    RegionData->NumRegions = 1;
    RegionData->Regions = Region;
    RegionData->SrcPitch = (uint32)(4 * Texture->GetSizeX());
    RegionData->SrcBpp = 4;
    RegionData->SrcData = (uint8*)PixelsBuffer.GetData();

    ENQUEUE_RENDER_COMMAND(UpdateTextureRegionsData)(
        [RegionData](FRHICommandListImmediate& RHICmdList)
        {
            RHIUpdateTexture2D(
                RegionData->Texture->Resource->TextureRHI->GetTexture2D(),
                RegionData->MipIndex,
                *RegionData->Regions,
                RegionData->SrcPitch,
                RegionData->SrcData
            );
            delete RegionData->Regions;
            delete RegionData;
        });
}

CreateTexture is a function to create a texture of the specified size. UpdateTexture is a function to write the pixel buffer into the content of the passed texture. (Note: UpdateTexture implementation updated to use render thread for safety)

Creating a Real-time Updating Material

Prepare a simple material like the one below. Just set the parameter name of TextureSampleParameter2D to “Tex” and plug it into Base Color.

Creating an Actor

Create an empty Actor and place a Cube inside it.

Implementing Logic in the Actor

Connect the nodes sequentially in BeginPlay.

Creating the Texture

Create the texture using the pre-created UTextureUtil::CreateTexture.

Set the texture size to 800x450 to match the WPF bitmap size.

Creating Dynamic Material and Setting Texture

Create a Dynamic Material for the Cube and set the previously created texture to the material’s “Tex” parameter.

Now, the texture display will be reflected in the material.

Registering ObjectDeliverer Receive Event

Create an instance of ObjectDelivererManager and monitor the ReceiveData event.

When the event occurs, update the content of the previously created texture using UTextureUtil::UpdateTexture.

Starting ObjectDeliverer

Plug in the SharedMemory Protocol and Start.

For SharedMemory, specify the same name as the shared memory created in WPF, and for the size, specify the bitmap size (800x450x4 = 1,440,000 bytes).

Do not include the 5 bytes for the header in this size.

Entire Blueprint

Connecting the above Blueprints results in the following:

Done!

After implementing the above and launching both the WPF application and the UE4 application, the content displayed in the WPF window will be displayed on the surface of the UE4 Cube.

I have added a shared memory protocol implementation to my custom UE4 library, ObjectDeliverer.

This allows data transfer using shared memory in addition to the previously implemented TCP/IP and UDP. However, shared memory is Windows Only…

How to Use

Since it’s implemented as a protocol for ObjectDeliverer, you can use it simply by switching the protocol passed to the ObjectDelivererManager’s Start method.

deliverer->Start(UProtocolFactory::CreateProtocolSharedMemory("memory_test", 1024),
                 UPacketRuleFactory::CreatePacketRuleSizeBody());

Pass the shared memory name and memory size to CreateProtocolSharedMemory. These two values must be the same as the counterpart exchanging data.

Specification

To use it as a data send/receive protocol, the content of the created shared memory is polled periodically. However, accessing the data every time increases processing load, so it’s handled with the following mechanism:

During periodic polling, it first reads the first byte of the shared memory and checks if the counter value differs from the previous read.

If it differs, it determines that new data has been written, reads the next 4 bytes to confirm the data size, and then reads the data body.

Therefore, when exchanging data using shared memory with a process not using ObjectDeliverer, it needs to conform to this specification.

Sample

Using the shared memory protocol created this time, I made a sample like the movie below.

ObjectDelivererの共有メモリのテスト題材作成中。
WPFアプリが毎フレーム自分のウィンドウを共有メモリに書き込んで、それをUE4側が読み込みテクスチャを更新してる。#ObjectDeliverer #UE4 #UE4Study pic.twitter.com/uWY2QhHnYP

— ayuma (@ayuma_x) January 29, 2019

A WPF application writes its window display content to shared memory every frame, and the UE4 side receives the data using ObjectDeliverer.

Using the received data (pixel buffer), it updates the texture content, mirroring the WPF side on the UE4 display side.

Since it’s shared memory, it can only be used if both applications are on the same machine, but at a glance, it seems to synchronize without delay.

For reference, the WPF app created here is ↓.

Issue

When mirroring as described above for a long time, a phenomenon occurs where it crashes due to hitting the !IsUnreachable() check in UObject::ProcessEvent during shared memory reading.

It’s unusable unless this is fixed.

As a temporary fix, removing all uses of BlueprintNativeEvent resolved the issue. However, this prevents overriding in Blueprint, so I will consider that area again.

I often link applications developed with Unreal Engine to apps created with C# etc., and the most common method I use for this is socket communication like TCP/IP or UDP.

Until now, whenever needed, I created features tailored to the specifications at that time.

However, implementing this each time is tedious, and the communication part is delicate, making it prone to bugs if not built properly.

Therefore, I aimed for reusability by turning it into a plugin.

Although there are still planned items yet to be implemented, the current implementation can be checked from the following repository.

Basic usage should be understandable from the README, but since it’s written in my weak English, it might be hard to read…

So, I’ll write it again in this article.

Concept

The plugin I created this time (ObjectDeliverer) is based on the following three concepts.

Compactness

UE4’s socket communication, especially TCP/IP implementation, is quite low-level, requiring more implementation effort compared to C# etc.

While this introduces some constraints, the design concept was to make the implementation amount as compact as possible when using it.

Usability in Blueprint

Another concept was to make it usable in Blueprint, which is a strength of UE4. Although more could be done if it were a C++ only plugin abandoning Blueprint support, I insisted on this.

Preventing Blueprint Disconnection

ObjectDeliverer supports multiple communication protocols (with plans to add more), but if they were implemented as separate types, switching the protocol type in an already implemented Blueprint would cause disconnection.

Therefore, I aimed for a design that minimizes disconnection as much as possible during specification changes after implementation.

Feature Explanation

Here’s a brief explanation of ObjectDeliverer’s features.

Communication Protocols

Currently, the following four protocols are supported.

• TCP/IP Server (supports multiple client connections)
• TCP/IP Client
• UDP (Send only)
• UDP (Receive only)

TCP/IP supports both sending and receiving for both server and client.

In a situation where multiple clients are connected to the server, the default setting is to broadcast transmissions from the server.

For UDP, I hesitated, but considering that use cases for UDP rarely involve simultaneous sending and receiving, I separated them. Of course, using both simultaneously enables sending and receiving.

Data Splitting Rules

In UDP, received data is often treated as a single data unit per reception. However, in TCP/IP, received data might arrive partially, or multiple data units might arrive combined, necessitating splitting rules.

Therefore, I have currently implemented three splitting rules that I often use.

• Fixed Size
• Header + Body
• Terminator Symbol

The fixed-size method divides data into chunks of a predetermined size every time. Even if the sent/received data size is less than the fixed size, the fixed size amount must be sent.

Conversely, it does not support cases where the data size to be sent in one go exceeds the fixed size.

In the Header + Body format, the body size is stored at the beginning of the data.

Therefore, the receiving side first reads the size field to recognize the size of the following body part, and then reads the body.

The byte count and endianness of the size part can be changed externally.

Personally, I find this the most convenient.

The final terminator symbol method was created to support common patterns like “delimited by newline characters”.

Send/Receive Data Format

By default, ObjectDeliverer sends and receives data in byte array format (TArray).

However, by using an option called DeliveryBox, it’s possible to send and receive strings or objects directly.

Currently, the following two patterns are supported.

• UTF-8 String
• UObject via JSON string format

How to Use

Clone the GitHub repository and copy the Plugins folder into your project folder to use it. Please enable the ObjectDeliverer plugin in the Editor.

Usage in Blueprint

1. Create an instance of ObjectDelivererManager.
2. If necessary, create an instance of DeliveryBox.
3. Set the Protocol and PacketRule for the ObjectDelivererManager and call the Start method.

Usage in C++

The usage procedure is the same as in Blueprint.

// Create an instance of ObjectDelivererManager
auto deliverer = NewObject<UObjectDelivererManager>();

// Monitor the receive event
deliverer->ReceiveData.AddDynamic(this, &UMyClass::OnReceive);

// Set Protocol and PacketRule and Start
deliverer->Start(UProtocolFactory::CreateProtocolTcpIpServer(9099),
                     UPacketRuleFactory::CreatePacketRuleSizeBody());

Sending and Receiving Data

The implementation method differs depending on whether the data to be sent is a byte array or something else.

For Byte Arrays

Sending and receiving are done via the ObjectDelivererManager instance.

// Create an instance of ObjectDelivererManager
auto deliverer = NewObject<UObjectDelivererManager>();

// Monitor the receive event
deliverer->ReceiveData.AddDynamic(this, &UMyClass::OnReceive);

// Set Protocol and PacketRule and Start
deliverer->Start(UProtocolFactory::CreateProtocolTcpIpServer(9099),
                     UPacketRuleFactory::CreatePacketRuleSizeBody());

TArray<uint8> buffer;
// Fill buffer with data...

// Send byte array
deliverer->Send(buffer);

void UMyClass::OnReceive(UObjectDelivererProtocol* ClientSocket, const TArray<uint8>& Buffer)
{
    // Use the received byte array
}

For Non-Byte Arrays

This is done via the DeliveryBox set in ObjectDelivererManager.

// Create a DeliveryBox with Object send/receive functionality via JSON serialization
auto deliverybox = NewObject<UObjectDeliveryBoxUsingJson>();

// Monitor the object receive event for the DeliveryBox
deliverybox->Received.AddDynamic(this, &UMyClass::OnReceiveObject);

deliverer->Start(UProtocolFactory::CreateProtocolTcpIpServer(9099),
                 UPacketRuleFactory::CreatePacketRuleSizeBody(),
                 deliverybox);

auto obj = NewObject<USampleObject>();
// Set properties of obj...

// Send the object using DeliveryBox
deliverybox->Send(obj);

void UMyClass::OnReceiveObject(UObject* ReceivedObject)
{
    // Extract the received object
    USampleObject* obj = Cast<USampleObject>(ReceivedObject);
    if (obj)
    {
        // Use the received object
    }
}

Future Plans

ObjectDeliverer is currently under development, so changes will likely continue for some time.

Therefore, although I will try to avoid it as much as possible, there might be changes that break backward compatibility. Please be aware of this if you decide to try it.

Planned Additions

The following features are planned for implementation.

• Create shared memory as a communication protocol
• Create a DeliveryBox that uses MessagePack for serialization
• Save communication content to log files and a function to replay saved log files

Furthermore, by using Latent Commands, you can describe tests that span multiple frames, which is quite convenient.

Automation Technical Guide

However, when trying to perform delayed evaluation using multiple Latent Commands, I always found it cumbersome that I had to define a DEFINE_LATENT_AUTOMATION_COMMAND macro for each type of evaluation I wanted to perform.

For example, something like this:

DEFINE_LATENT_AUTOMATION_COMMAND_TWO_PARAMETER(FHogeObjCommand, FAutomationTestBase*, Test, UMyObject*, Obj);
bool FHogeObjCommand::Update()
{
    check(Test);
    check(Obj);
    Test->TestEqual(TEXT("check"), Obj->Property, 1);
    return true;
}

DEFINE_LATENT_AUTOMATION_COMMAND_TWO_PARAMETER(FHogeObjCommand2, FAutomationTestBase*, Test, UMyObject*, Obj);
bool FHogeObjCommand2::Update()
{
    check(Test);
    check(Obj);
    Obj->Hoge();
    return true;
}

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FHogeTest, "Hoge.HogeTest", EAutomationTestFlags::EditorContext | EAutomationTestFlags::EngineFilter)
bool FHogeTest::RunTest(const FString& Parameters)
{
    auto obj = NewObject<UMyObject>();

    ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(1.0f));

    ADD_LATENT_AUTOMATION_COMMAND(FHogeObjCommand(this, obj));

    ADD_LATENT_AUTOMATION_COMMAND(FHogeObjCommand2(this, obj));

    ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(1.0f));

    ADD_LATENT_AUTOMATION_COMMAND(FHogeObjCommand(this, obj));

    return true; // Added return true
}

So, tired of defining multiple commands every time, I defined the following Latent Command, and it seems to work for almost everything.

DEFINE_LATENT_AUTOMATION_COMMAND_ONE_PARAMETER(FLambdaCommand, TFunction<void()>, InWork);
bool FLambdaCommand::Update()
{
    InWork();

    return true;
}

Using this Latent Command, the previous test can be written as:

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FHogeTest, "Hoge.HogeTest", EAutomationTestFlags::EditorContext | EAutomationTestFlags::EngineFilter)
bool FHogeTest::RunTest(const FString& Parameters)
{
    auto obj = NewObject<UMyObject>();

    ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(1.0f));

    ADD_LATENT_AUTOMATION_COMMAND(FLambdaCommand([this, &obj]()
    {
        TestEqual(TEXT("check"), obj->Property, 1);
    })); // Corrected closing parenthesis

    ADD_LATENT_AUTOMATION_COMMAND(FLambdaCommand([&obj]()
    {
        obj->Hoge();
    })); // Corrected closing parenthesis

    ADD_LATENT_AUTOMATION_COMMAND(FEngineWaitLatentCommand(1.0f));

    ADD_LATENT_AUTOMATION_COMMAND(FLambdaCommand([this, &obj]()
    {
        TestEqual(TEXT("check"), obj->Property, 1);
    })); // Corrected closing parenthesis

    return true; // Added return true
}

I prefer this way as it makes the flow of processing easier to follow.

However, while casually browsing the Engine source code, I found that a similar command already existed without needing to create it myself. I missed it because other Latent Commands were grouped in AutomationCommon.h. I wonder if it existed before…

// AutomationTest.h
class FFunctionLatentCommand : public IAutomationLatentCommand
{
public:
    FFunctionLatentCommand(TFunction<bool()> InLatentPredicate)
        : LatentPredicate(MoveTemp(InLatentPredicate))
    {
    }

    virtual ~FFunctionLatentCommand()
    {
    }

    virtual bool Update() override
    {
        return LatentPredicate();
    }

private:
    TFunction<bool()> LatentPredicate;
};

It’s 2019, and it feels like about a year has passed quickly since I started blogging.

Here, I’ll try writing down my reflections on 2018 and my resolutions for 2019.

Reflecting on 2018

Looking back now, this is how it feels. Looking at it this way, I’ve dabbled in many different things. This year, I want to narrow my focus a bit more and develop my strengths.

Also, a big thing for me was speaking in front of people for the first time in August. That experience was significant.

Additionally, due to a confluence of good timing at work, I got the opportunity to do VR development, which I had wanted to try!! It really pays to speak up about what you want to do.

Resolutions for 2019

Succeed in the VR Development Project at Work

Since they bought me high-spec machines and several VR headsets, I really want to achieve some results. I strongly feel like using this opportunity to increase the amount of work I want to do.

Increase Blog Update Frequency

I didn’t give up after three days, but in 2018, the pace was about once a month, so the goal is to make it once a week.

Give Presentations

Last year it was only once, so the goal is two or more times.

Apparently, Windows 10 IoT Core has been out for quite some time. Although I’m late to the party, I became interested after reading the following article:

What can you do with Windows 10 IoT Core (latest Creators Update version)?

So, I decided to try it out on the Raspberry Pi 3 that was sleeping in my drawer.

It seems that by installing this, I can develop and deploy UWP apps from Visual Studio on my development machine, which sounds fun.

Installation

Downloading and Installing Windows 10 IoT Core Dashboard

Download setup.exe from the link below and install it.

Downloads - Windows IoT | Microsoft Docs

At that time, on my environment (Surface Laptop, Windows 10 Pro 1803), the following warning appeared, and I couldn’t install it.

Memo regarding the issue “Your security settings do not allow this application to be installed on your computer.”

So, referring to the site above, I changed the following registry key and rebooted:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft.NETFramework\Security\TrustManager\PromptingLevel] Change Internet from Disabled to Enabled

Then, I was able to install it. (I changed the registry back after the installation was complete.)

Writing the OS to Micro SD

After launching the installed Windows IoT Core Dashboard, insert the Micro SD card and write the OS. I thought selecting Wi-Fi here would automatically set it up, but it didn’t get enabled…

Once writing is complete, the screen changes as shown above. Insert the SD card into the Raspberry Pi and boot it up.

Setting up Windows IoT Core

Wait for the setup screen to appear with a display connected via HDMI. In my environment, it took about 10 minutes. I didn’t know how to do this setup remotely, so I connected a wired keyboard and performed it.

As mentioned earlier, I also had to enter the Wi-Fi password here and authenticate. (What was the Wi-Fi network connection checkbox on the main machine side for?)

Connecting from the Main Machine

After setup is complete, the Raspberry Pi is recognized in the Windows IoT Core Dashboard on the main machine.

From here, it seems various settings can be configured remotely, but I encountered several problems in my environment.

• Extremely slow
  It seemed that having the Raspberry Pi also connected via Wi-Fi was the issue. It improved after switching to a wired LAN connection.
• Remote client doesn't work
  There's something called IoT Remote Client, which is supposed to allow usage similar to Windows Remote Desktop, but it just showed a white screen, seemingly not transferring the display.
  I couldn't find a way to resolve this.
• Connection drops
  After using it for a bit, the Raspberry Pi disappears from "My devices", and it can't be recovered without rebooting.
  The cause for this is also unknown. Maybe the main PC should also be on a wired LAN connection?

Summary

I tried this thinking I could utilize the sleeping Raspberry Pi, but since various things didn’t go as expected, I’ll let it sleep again for a while. I’ll try again from scratch after some time.

The previous day’s article was by 0xUMA: Trying out automated testing in UE4.

Overview

There are times when you want to switch behavior patterns after creating a package for Windows from UE4. Initially, I handled this by allowing parameter changes via external text files (like XML or JSON).

However, eventually, the request came to change the logic itself.
So, I will write about how I externalized the logic using JavaScript with Unreal.js.

The sample project created for this article is available below.

Selecting the Approach

In UE4, even if you change C++ code from the Editor, you can hot-reload it, allowing immediate (though with a slight delay) confirmation of behavior. However, this cannot be used after packaging.

Therefore, I looked for a plugin for scripting functionality that can run in the UE4 Runtime and found the following three:

• Unreal.js
  
• UnrealEnginePython
  
• MonoUE
  [MonoUE](https://mono-ue.github.io/)

MonoUE uses C#, which I personally like, so I was interested, but it didn’t seem easy to use casually as a script, so I removed it from the candidates.

Considering the remaining two, I personally preferred JavaScript, so I chose Unreal.js.

What is Unreal.js?

This was helpful for obtaining basic information about Unreal.js. Although the article is about 2 years old, it was very helpful as it could be used almost as is even in the current 4.21 environment.

Introduction to Unreal.js
Unreal.js 入門 - Qiita

Installation

Add Unreal.js from the Marketplace.

Then, open the UE4 project in the Editor and enable Unreal.js from Edit->Plugins. Restart the Editor, and it becomes usable.

As a test, place an Actor in the level, add a JavaScript Component, and enter Sample.js in Script Source File. Create Sample.js in Content/Scripts and write the following content:

console.log("test")

With this setup, play the level. If “test” is displayed in the Output Log, it’s working correctly.

Checking After Packaging

Create a package for Windows using File -> Package Project -> Windows -> Windows (64-bit).

After creating the package, copy the following directory to the Content directory of the created package:

[Engine Install Path]\Engine\Plugins\Marketplace\UnrealJS\Content\Scripts

Start the application, close it immediately, and check if “test” is output in Saved/Logs/***.log. If it is, it’s OK.

Creating a Dynamic Loading Mechanism

Using Unreal.js’s JavaScript Component allows executing js files, but it has the following limitations:

• Once executed, the JavaScript cannot be changed midway.
• The js file must be placed under Content/Scripts.

Therefore, let’s create a mechanism to eliminate these limitations, allowing js files to be placed anywhere and updated even while the script is running. Specifically, let’s create it with the following policy:

• The js file will only contain class definitions describing the externalized logic.
• The base class for the classes described in the js file will be defined in C++.
• The js file can be reloaded even while the application is running.
• Minimize file access frequency.
• Allow js files to be placed outside of Content/Scripts.

C++ Class Implementation (JSObject)

Define the base class for the classes defined in the js file.

FString NotifyTrigger(); is the interface for accessing the JavaScript side from the application side. This time, it’s a method returning FString, but change this according to the situation. Multiple methods can be provided. Make the method必ず UFUNCTION(BlueprintImplementableEvent) to make it an override-only method. Therefore, do not write the implementation in the cpp side.

// Fill out your copyright notice in the Description page of Project Settings.

#pragma once

#include "CoreMinimal.h"
#include "UObject/NoExportTypes.h"
#include "JSObject.generated.h"

/**
 *
 */
UCLASS(BlueprintType)
class UNREALJSSAMPLE_API UJSObject : public UObject
{
    GENERATED_BODY()

public:
    UJSObject();

    UFUNCTION(BlueprintImplementableEvent)
    FString NotifyTrigger();

};

// Fill out your copyright notice in the Description page of Project Settings.

#include "JSObject.h"

UJSObject::UJSObject()
{

}

C++ Class Implementation (JSComponent)

Implement this by referencing the implementation of Unreal.js’s JavaScript Component.

OnRegister

Almost the same as Unreal.js’s JavaScript Component. Preparation for using JavaScript and registration of linkage between the C++ side and JavaScript side are also done at this point.

Among these, describing the single line Context->Expose("Root", this); allows the reference of this Component class to be used as Root within JavaScript.

void UJSComponent::OnRegister()
{
    auto ContextOwner = GetOuter();
    if (ContextOwner && !HasAnyFlags(RF_ClassDefaultObject) && !ContextOwner->HasAnyFlags(RF_ClassDefaultObject))
    {
        if (GetWorld() && ((GetWorld()->IsGameWorld() && !GetWorld()->IsPreviewWorld())))
        {
            UJavascriptIsolate* Isolate = nullptr;
            UJavascriptStaticCache* StaticGameData = Cast<UJavascriptStaticCache>(GEngine->GameSingleton);
            if (StaticGameData)
            {
                if (StaticGameData->Isolates.Num() > 0)
                    Isolate = StaticGameData->Isolates.Pop();
            }

            if (!Isolate)
            {
                Isolate = NewObject<UJavascriptIsolate>();
                Isolate->Init(false);
                Isolate->AddToRoot();
            }

            auto* Context = Isolate->CreateContext();
            JavascriptContext = Context;
            JavascriptIsolate = Isolate;

            Context->Expose("Root", this);
            Context->Expose("GWorld", GetWorld());
            Context->Expose("GEngine", GEngine);
        }
    }

    Super::OnRegister();
}

LoadJSFile

Reads the js file, instantiates the class described within it, and holds the reference.

void UJSComponent::LoadJSFile()
{
    if (JavascriptContext == nullptr) return;

    // ScriptSourceFile contains the relative path from Content, so convert it to an absolute path
    auto scriptSourceFilePath = FPaths::Combine(FPaths::ProjectContentDir(), ScriptSourceFile);
    scriptSourceFilePath = FPaths::ConvertRelativePathToFull(scriptSourceFilePath);

    // Read the content of the js file (don't execute yet)
    FString script;
    FFileHelper::LoadFileToString(script, *scriptSourceFilePath);

    // Extract the class name written in js
    const FRegexPattern pattern = FRegexPattern(FString(TEXT("class\\s+(.+)\\s+extends\\s+JSObject")));
    FRegexMatcher matcher(pattern, script);
    if (matcher.FindNext())
    {
        auto className = matcher.GetCaptureGroup(1);

        // Add code to the script for class instantiation and reference holding
        script = TEXT("(function (global) {\r\n") + script;
        script += TEXT("let MyUObject_C = require('uclass')()(global,") + className + TEXT(")\r\n");
        script += TEXT("let instance = new MyUObject_C()\r\n");
        script += TEXT("Root.SetJsObject(instance)\r\n");
        script += TEXT("})(this)");

        // Execute the script
        JavascriptContext->RunScript(script);
    }
}

The point is not to execute the js file directly as a script, but to add script lines above and below the js content to perform class instantiation and reference holding.

The part script += TEXT("Root.SetJsObject(instance)\r\n"); uses the mechanism associated during OnRegister() to call the Component’s function from within the script. This allows the JavaScript class instance to be held as a property of the Component implemented in C++.

NotifyTrigger

This is a function for calling the function of the class implemented in JavaScript via the Component from Blueprint. Since the reference to the js class is held in JsObject, it’s called via that.

FString UJSComponent::NotifyTrigger()
{
    if (JsObject != nullptr)
    {
        return JsObject->NotifyTrigger();
    }

    return FString();
}

Blueprint Implementation (SampleActor)

Add the previously created JSComponent and TextRender to an empty Actor.

Also, create a function to call the method NotifyTrigger implemented in js. Make it so that the string of TextRender is rewritten based on the result of NotifyTrigger.

Next, create a function to reload the js file.

Blueprint Implementation (SampleUMG)

Create two buttons in a Widget. Make one call the SampleActor’s NotifyEvent method, and the other call the UpdateJS method.

Create js File

Create the js file to be used as a script. The created file can be placed anywhere, but this time, save it in the same hierarchy as the Content folder.
Here, define a class that inherits the UJSObject class previously implemented in C++. Since the application calls UJSObject’s NotifyTrigger, override that part’s implementation within the script.

class MyUObject extends JSObject {
    NotifyTrigger(){
        let prop1 = 1;
        let prop2 = 2;
        return "ABC" + prop1 + prop2;
    }
}

Register js File Path to JSComponent

Place SampleActor in the level, and input the path to the js file as a relative path from Content into the ScriptSourceFile of the JSComponent already added to SampleActor.

It’s okay even if it’s in a higher hierarchy than Content.

Execution

Create a Win64 package from the Editor.

Copy the following directory to the Content directory of the created package:

[Engine Install Path]\Engine\Plugins\Marketplace\UnrealJS\Content\Scripts

Also, copy the created js file to the same hierarchy as the package’s Content folder.

Once the above preparations are complete, run the application.

TextRender displays the initial value.

class MyUObject extends JSObject {
    NotifyTrigger(){
        let prop1 = 3;
        let prop2 = 4;
        return "ZZZ" + prop1 + prop2;
    }
}

Press the application’s UpdateJS button, then press Trigger. The display of TextRender changes to the result of the rewritten script.

Summary

Using Unreal.js, we were able to use scripting functionality dynamically even after package creation.

The advantage of this implementation is that by entrusting only the bare minimum to JavaScript, it’s somewhat advantageous in terms of speed, and once the logic solidifies, performance can be further improved by replacing it with a C++ class.

Please give it a try if you like.

Tomorrow’s article is by fukusuke8gou about UMG.

References

• Unreal.js Wiki
  https://github.com/ncsoft/Unreal.js/wiki
• Introduction to Unreal.js
  https://qiita.com/ConquestArrow/items/bb81ad5e63cf6bc4e549

I recently attended “Center CLR Try! Development #2”.

The venue was different from #1, right in front of Kanayama Station, making it easily accessible. However, the target building was under construction, making the entrance hard to find, and I wandered around for a few minutes… (Being directionally challenged, I initially went to the wrong building).

There were three participants that day (Mr. Matsui, Mr. Matsuoka, and myself). Being alongside two amazing individuals made me a little, no, quite nervous. But thanks to the small group size, I was able to talk about various things, making it a meaningful day.

What I Did

I had originally planned to spend the day working on an app I was developing in Unity, so that’s what I did.

I’m creating a prize lottery app to be used at my department’s year-end party.

I’ve been using Unreal Engine as my game development tool for a long time, but I had no experience with Unity, so I decided to give Unity a try starting this month. (* Although I use Unreal Engine a lot, I’ve never actually made a game.)

By the way, the deadline I set for myself is the end of November, so the duration is one month.

A Bit More Detail

Using Unity’s UI features (equivalent to UMG in Unreal), I was creating the participant registration screen for the year-end party and implementing a feature to register photos taken with a smartphone camera.

I was worried I might have to use OS-dependent APIs for camera shooting, but it was surprisingly easy to achieve using only Unity’s features!

(I’m requesting camera usage permission in the app beforehand.)

// Get the camera attached to the execution device (the first one if there are multiple)
var cameraDevice = WebCamTexture.devices[0];

// Create a texture to project the camera feed onto
cameraTexture = new WebCamTexture(cameraDevice.name, 1280, 720);

// Start shooting
cameraTexture.Play();

// Display the camera feed on the UI (targetImage is a reference to a RawImage on the UI)
targetImage.texture = cameraTexture;

Running the code above displays the camera feed in real-time on the RawImage in the UI. I assume it’s transferring the camera feed to the RawImage’s texture every frame.

This time, I wanted to save a still image, so I executed the following code when a UI button was pressed to save the image data.

// Get the pixel buffer of the camera image at that moment
Color32[] color32 = cameraTexture.GetPixels32();
// Create a texture to display the camera image
var cameraImage = new Texture2D(cameraTexture.width, cameraTexture.height);
// Copy the pixel buffer to the texture
cameraImage.SetPixels32(color32);
cameraImage.Apply();

// Stop the camera shooting
cameraTexture.Stop();

Now, cameraImage contains the image captured by the camera as a texture. By inserting this into the UI’s RawImage, I achieved the desired functionality.

That’s what I did today. In the future, I plan to shrink and crop this captured image for thumbnails and serialize it so it can be used on the next launch.

Summary

• Managed to capture smartphone camera images in Unity.
• Small-group events are very meaningful as you get to talk with amazing people.
• Might take the first step towards being able to read IL soon.

Lately, I’ve been mostly using Qt3D in Qt, but I wanted to try loading FBX files directly, so I set it up.

Here are the notes.

Installing FBX SDK

Download the SDK from the Autodesk download page.

FBX Software Developer Kit 2019.0 | Autodesk Developer Network

The latest SDK is for Visual Studio 2015; there isn’t one for 2017. However, since VC++ 2015 and 2017 are said to be compatible, I’ll try using the 2015 version.

Information about VC++ 2015 and 2017 compatibility can be found here:

C++ Binary Compatibility between Visual Studio 2015 and Visual Studio 2017 | Microsoft Docs

First, Try Using it in Visual Studio

Referring to this site, I tried using it in a Visual Studio 2017 C++ project.

FBX SDK Preparation – Get Data from File – Code Labo

However, building resulted in an error!!

This article described the exact issue:

Story about FbxSdk failing to build in VisualStudio2017 - Qiita

Apparently, it’s because FBX SDK uses non-standard C++ syntax from Visual Studio. Hmm.

Setting up in Qt Creator

Looking at the directory where FBX SDK was installed, it seems there are dynamically linked versions (libfbxsdk.lib) and statically linked versions (libfbxsdk-md.lib, libfbxsdk-mt.lib) based on size.

This time, I’ll try static linking (libfbxsdk-md.lib).

The environment is Windows 10, and the compiler specified is Visual Studio 2017 64-bit.

• In the Qt Creator project, select "Add Library" and choose "External Library".

  

• Enter the path to libfbxsdk-md.lib in "Library file".
• Enter the path to the fbxsdk "include" folder in "Include path".
• For platform, select Windows Only for now.
• Select "Static" for the linking method.

Pressing “Next” and proceeding writes the link information for FBX SDK into the ~.pro file.

Then, write the following code in main.cpp, build, and run. If no errors occur, it’s OK.

#include <fbxsdk.h>

int main(int argc, char *argv[])
{
    QGuiApplication app(argc, argv);

    FbxManager* manager = FbxManager::Create();
    if (manager)
    {
        qDebug() << "FBX Manager created successfully.";
        manager->Destroy();
        qDebug() << "FBX Manager destroyed.";
    }
    else
    {
        qDebug() << "Failed to create FBX Manager.";
    }

    // ... rest of your Qt application setup ...

    return app.exec();
}

Running it worked normally. The errors seen in Visual Studio 2017 did not occur in Qt Creator.

Summary

I was able to set up the FBX SDK in Qt Creator. I’ll use this to play around with FBX files.

Bonus

It seems that FBX SDK is used internally within Qt3D in Qt Creator. (Source code confirmed)

So, even without performing the steps in this article, FBX loading is possible with the default functionality.

This time, I specifically wanted to try using the FBX SDK directly myself, which is why I tried the steps in this article.

In the app I’m currently building, I wanted to create a virtual-sized canvas within the Window and change the overall scale according to the window size without breaking the layout inside. I explored various options.

In WPF, it was convenient because you could adjust the scaling of the content display instantly by writing something like the below… I wondered if Xamarin had something similar, but… it doesn’t.

<DockPanel>
    <DockPanel.RenderTransform>
        <ScaleTransform ScaleX="{Binding Scale}" ScaleY="{Binding Scale}" />
    </DockPanel.RenderTransform>
</DockPanel>

About VisualElement’s Scale Property

So, I wondered if controls have a property to set their own scale, and I found a property called “Scale”.

I tried setting the value to 0.5, expecting the size to become half.

But, the appearance didn’t change. It also feels strange that the Scale property isn’t split into x and y. Is its purpose different?

I decided not to pursue this further and look for another approach.

Alternative: Adjust Size Manually (The Gritty Way)

After considering various options, I decided to simply capture the ContentView’s SizeChanged event, calculate the change rate from the original size (1x scale), and adjust accordingly.

I receive SizeChanged within a Behavior attached to the View and pass it to the ViewModel via a Command.

At that time, I tried connecting it to the ViewModel using Prism’s EventToCommandBehavior, but it didn’t work (EventArgs had no value, and I couldn’t pass the sender to EventArgsConverter, so I gave up). I created a new Behavior specifically for SizeChanged to handle it.

So far, this method achieves the desired behavior.

Yet Another Method

I haven’t actually tried this, but instead of drawing directly on the screen as Xamarin Forms controls, I think scaling could be easily achieved by first drawing everything onto a Bitmap.

(I don’t plan to try this for a while because the scope of change is large, and it would prevent UI creation using Xaml.)

While looking for a 3D model to use for learning Qt3D, I found a very promising one.

Niconi Solid Chan Special Site - Niconi Solid

We are releasing the 3D model ‘Alicia Solid’, which can be used for free without attribution in various situations such as video productions, self-made games, technical demos, and doujin activities.

It’s a very user-friendly license, allowing free use and secondary processing.

The 3D model is distributed in the following three formats:

• FBX
• MMD
• Unity Package

This time, I will use the FBX format from these options.

MMD can be used in Blender by adding an add-on, but it has drawbacks for future use, such as Japanese characters in object names, so I won’t use it this time.
Unity Package is exclusive to Unity, so I won’t use this either.

Loading with Qt3D (FBX Edition)

First, I tried loading the fbx like this:

Entity {
    components: [
        SceneLoader {
                source: "/FBX/Alicia_solid_MMD.FBX"
             },
        Transform {
            translation: Qt.vector3d(0, 0, 0)
        }
    ]
}

As a result, nothing was displayed on the screen, and it seems fbx cannot be loaded.
(Upon closer inspection, it was just taking too long, and loading was actually happening. However, the raw downloaded fbx encountered texture loading errors and ultimately wasn’t displayed.)

Loading with Blender

Next, since loading it directly in Qt3D didn’t work, I tried processing it first by loading it into Blender.

Then…

ASCII FBX files are not supported.

Hmm, are there different types of FBX?

Research revealed that FBX has ASCII and binary types.
(I had never paid attention to that before.)

FBX Conversion

Since I wanted to read FBX in Blender, I found that Autodesk provides an FBX converter. Besides converting from ASCII to binary format, it seems you can also choose whether resource files like textures are external or embedded.

FBX Converter Archives | Autodesk Developer Network

Usage is simple: add the target file, set FBX Save Mode to Binary, and press Convert.

Loading with Blender (Attempt 2)

Importing the FBX converted to binary format completed without errors this time.

Exporting in a Format Loadable by Qt3D

I tried exporting in obj format, which I have previously successfully loaded in Qt3D.

As expected, there seemed to be errors loading textures. Texture information for obj files is apparently written in the mtl file. Looking inside the mtl, I found two patterns for image file references: *.tga and *.psd.

Looking at the Qt documentation (http://doc.qt.io/qt-5/qtimageformats-index.html), it seems tga files are supported, so I imagined psd couldn’t be read and replaced *.psd with *.tga.
(This isn’t a fundamental solution, but since there were the same number of *.tga and *.psd files, it was a temporary measure.)

After applying this change and running it in Qt:

It displayed!! But the overall display is rough, and there’s something on the forehead area.

Compared to the original FBX (Binary) file size of 6MB, the obj (text) file is 2.7MB, significantly smaller, so the roughness of the display might be unavoidable. Is the thing on the forehead perhaps due to changing the psd file to tga?

As a temporary measure, I deleted the forehead part in Blender and exported the obj file again. The something on the forehead disappeared.

Embedding the 3D Model in QML

Exporting the obj file from Blender allowed loading in Qt3D, but the rendering quality decreased (probably due to reduced vertex count?).

Also, loading models using SceneLoader seems to happen dynamically after the application starts, so it takes a little time for the model to appear.
(This is why I initially thought the text-format fbx wasn’t loading.)

Amidst this, I found a page published by KDAB:

Exporting 3D content for Qt 3D with Blender - KDAB

The article mentions that using the Blender Exporter Addon described here allows saving the model in binary format and using it without parsing time.

I tried it immediately.

Copy the files from this repository, paste them into Blender’s addons directory, and enable the addon in Blender’s settings to make qml export available.

Exporting using this generated a complete set of Qt project files.

Launching the exported project and displaying it:

The shape is cleaner than the previously loaded obj format.
There’s almost no difference from how it looks in Blender. Also, there’s almost no delay between the window appearing and the 3D model being displayed.
The effect of binary embedding seems significant.

However, as you can see at a glance, it’s not colored.
Looking at Blender’s qml export script (which is Python inside), the information for materials using textures was not exported.

I found that the material information in the exported project is written in MaterialCollection.qml, so I decided to manually redefine the materials using textures (*.tga) here.

Original Material Definition

readonly property Material wearPsd: PhongMaterial {
   ambient: Qt.rgba(0.2, 0.2, 0.2, 1.0)
   diffuse: Qt.rgba(1.0, 1.0, 1.0, 1.0)
   specular: Qt.rgba(1.0, 1.0, 1.0, 1.0)
}

Modified Material Definition

readonly property Material wearPsd: NormalDiffuseSpecularMapMaterial  {
        ambient: "white"
        diffuse: TextureLoader { source: "matarial/Alicia_wear.tga"}
        normal: TextureLoader { source : "matarial/Alicia_wear.tga" }
        specular: TextureLoader { source: "matarial/Alicia_wear.tga" }
        shininess: 1.0
        textureScale: 1.0
}