Unity cluster#
Quickstart#
Note: You can easily test your cluster application on a single computer.
You must make sure your application is correctly synchronized on all cluster nodes. This means you must make sure its inputs/events (causes) or the results (consequences) of your scripts are correctly synchronized.
Typically this means using MiddleVR's input devices and time API which is automatically synchronized, and using MiddleVR's synchronization API to manually synchronize other data.
Let's take some examples in the order of difficulty:
- At the end of a frame, a GameObject's position/orientation has changed:
See Synchronize a GameObject' position and rotation.
- At the end of a frame, textures have been modified:
- My application is using keyboard, joystick, or VR controllers:
Those inputs are automatically synchronized if you use MiddleVR's input devices API. If your interactions don't have any random part, they should be correctly synchronized.
If your interactions use time, be sure to use MiddleVR's time API.
If your interactions use random, be sure to check Random values and Random objects positions.
- My user interacts with a GUI using a mouse:
See Synchronize a GUI.
- In my application, it is important that certain MonoBehaviours variables are synchronized:
See Synchronize MonoBehaviours' variables.
- I want to synchronize arbitrary data from the cluster server to the cluster nodes and back:
See MVR.Cluster.BroadcastMessage
.
Introduction#
We have already talked about the different levels of synchronization, so will focus here on what the Unity developer should know to create a properly synchronized cluster application.
What issues are introduced by using a cluster ?#
In a perfect world#
A cluster system should in the end display the same image as a non-cluster system. The main difficulty is to get everything correctly synchronized.
In a perfect world, Unity would synchronize each and every state of all scripts and scene graph. Unfortunately this is not the case and probably never will be: this would potentially mean thousands or millions of network synchronizations per frame!
So we must try to minimize both the amount of network synchronization and also minimize the work for developers to adapt their applications.
In reality#
The root issue is that MiddleVR will run a Unity instance (player) per computer that make up the graphics system.
Also note that on each cluster node, all your scripts will still be running by default, unless you disable them depending on the cluster state.
By default those player don't have any relation to each other, so they have no reason to be synchronized. More specifically they will probably have different inputs, resulting in different results:
-
user input: keyboard, mouse, joystick, trackers, GUI inputs,
-
time input: the elapsed time and time to render each frame (delta time) might be different on each cluster node.
-
even with the same inputs the result can be different: the application is non-deterministic. For example Unity's physics and particles are non-deterministic. If you run the same application twice you will not get the same result.
Cause or consequence ?#
Let's take the example where pressing "Space" on your keyboard opens a door.
There are two ways to think about cluster synchronization:
-
sharing the input (causes): this should give the same result in your application is deterministic. In our example, we have to synchronize the key press on all cluster nodes so the script will correctly open the door.
-
sharing the result (consequences): sometimes it is easier to just synchronize the result of a frame from the server. In our door example, it means that we would synchronize the door rotation.
Sharing causes#
The theory is the following: if your application is deterministic ("whose resulting behavior is entirely determined by its initial state and inputs, and which is not random"), its state at the end of each frame will be the same on all machines.
This means that:
-
the initial state must be the same: this is automatic since we run the same application on all cluster nodes (unless your application is doing random things at startup)
-
at each frame inputs must be the same: you need to make sure that your application uses inputs that are the same on cluster nodes by using MiddleVR's synchronized inputs or synchronization API.
Sharing consequences#
You can also decide that you don't want to synchronize the causes but only want to share the result of the frame from the server. This is fine as long as you don't forget any parameter.
For example if your application is only moving game objects around,
you can safely use MiddleVR's MVRClusterObject
script that will
automatically synchronize its position and orientation.
But if your application can also change the color of a material you also need to make sure the color change is correctly synchronized with a custom synchronization.
Note: MiddleVR automatically shares the position/rotation of physics objects because the physics engine is not always deterministic if the MVRManager "SimpleCluster" parameter is enabled.
How to choose ?#
Typically you choose to share causes or consequences based on two parameters:
-
ease of development: is it easier to share one or the other ?
-
resources: it might not be practical/fast to share the position of a million game objects if they are entirely determined by the inputs.
You can also safely mix the two methods:
-
share causes for deterministic aspects of your application,
-
share consequences for non-deterministic aspects.
How to synchronize ?#
Simple cluster#
If the MVRManager parameter Simple Cluster is enabled, the following will happen:
-
All
Rigidbody
physics objects that are notKinematic
will be automatically synchronized with theMVRClusterObject
script. -
All
Character Controllers
will be automatically synchronized with theMVRClusterObject
script. -
All
Animators
will have their culling mode changed toAlways Animate
to make sure that each animation runs even if it is not visible on a given cluster node. -
All
Timelines
will have their time synchronized. -
All
ParticleSystems
will be synchronized with theMVRParticleSystem
script.
Overview#
There are three ways to synchronize data:
-
Use MiddleVR's API for devices and time which is correctly synchronized.
-
Use
MVRClusterObject
which will automatically share a GameObject's position and rotation and MonoBehaviour's variables. -
Use
MVR.Cluster.BroadcastMessage
API to manually synchronize any type of data.
Input devices#
MiddleVR API#
MiddleVR will automatically synchronize the state of all its internal devices.
So instead of: Input.GetKeys(...)
, you need to use: MVR.DeviceMgr.IsKeyPressed
and other methods to manage the different devices.
See section Input devices.
Note: Make sure that your scripts always execute after the MVRManagerScript, which is the case by default unless you modify the Script Execution Order. See section When to synchronize ?
Unity new input system#
Unity has introduced recently a new Input system.
MiddleVR offers an experimental package in C:\Program Files\MiddleVR3\unity_packages\com.middlevr.inputsystem.tgz
to automatically synchronize all devices from the input system.
Make sure to add the prefab MVRInputSystem
to your scene.
Known limitations:
-
When using this experimental package there is one frame delay between MiddleVR's keyboard/mouse inputs and Unity's.
-
There can be up to two frames of delay between the server and client using other devices.
If you want zero frames of delay, make sure to use MiddleVR's API.
Time & Delta time#
Instead of using Time.time
you should use MVR.Kernel.GetTime()
which
gives the elapsed time since the server started and is correctly
synchronized on the cluster.
Instead of using Time.deltaTime
you should use MVR.Kernel.GetDeltaTime()
.
The time and delta time are only updated once per frame,
when MVRManagerScript
is updated.
See section When to synchronize ?
Time.timeScale
and Time.fixedDeltaTime
are automatically synchronized.
GameObject position and rotation#
All nodes from the MiddleVR's hierarchy nodes are automatically synchronized.
Adding the MVRClusterObject
to a GameObject will automatically
synchronize its position, rotation and scale.
Textures#
MiddleVR ships with a script that synchronizes textures:
Packages/MiddleVR/Runtime/MiddleVR.Unity.Scripts/Cluster/MVRClusterTexture.cs
.
It will compress, send and decompress the texture each frame.
You can choose the compression factor. A higher number will result in a lower quality texture but will also send less data on the network.
You can also choose a lossless compression which will preserve the original texture but have a higher impact on the data sent on the network.
You can modify the codec, compression factor and lossless option with the following command line arguments:
--mvr-codec <codec>
: H264, HEVC, AV1, RAW--mvr-compression-factor <float>
: compression factor value--mvr-lossless <bool>
: true, false
Note: There might be a few frames of delay between the server and the clients, but the texture is guaranteed to be synchronized between the clients.
Note: Compatible with the following pipelines: built-in, URP, HDRP.
Note: Make sure to have at least NVidia drivers version 531.61 or newer installed. Cuda 12.3 is required to be installed.
Currently only textures with a format of R8G8B8A8_UNORM are supported.
MiddleVR will use the mainTexture
of the material.
If you are using Unity's Video Player,
make sure to set Render Mode
to Render Texture
and assign a Target Texture
.
You can also disable the Video Player on the cluster clients using the script
MVRClusterDisableComponent
:
Videos#
The best way to synchronize a video is to use the MVRClusterTexture script.
If this is not possible, MiddleVR ships with a script that should help videos be correctly
synchronized by starting them at the same time:
Packages/MiddleVR/Runtime/MiddleVR.Unity.Scripts/Cluster/MVRClusterVideoSyncStart.cs
.
Place this script on a GameObject
with a Video Player
.
If PlayOnAwake
is enabled, it will prepare and start the video on all cluster nodes.
If not, call "PlayPauseVideo"
on this script to play/pause the video.
It will wait for all cluster nodes to finish preparing the video then start it.
It preloads the video on all nodes and synchronizes the play and pause.
Note: We recommend to play the videos from a local file on each cluter node to avoid network congestion and potential resulting synchronization problems.
MonoBehaviour variables#
Adding the MVRClusterObject
and enabling Synchronize Mono Behaviours
will automatically synchronize MonoBehaviours variables.
Note: This feature is experimental.
Physics objects#
If the MVRManager SimpleCluster
is enabled,
MiddleVR will automatically add the MVRClusterObject
script to physics objects.
This will automatically synchronize their position/orientation when MVRManagerPostFrameScript
updates.
See section When to synchronize ?
Note that this will only work if all your objects are already created and not dynamically added later on in the application.
If your objects are created after the first MVRManagerScript update,
you will need to manually add the MVRClusterObject
script.
Animations#
Make sure that the Culling Mode of your Animators are set to "Always Animate" and Update Mode is not "Animate Physics", otherwise cluster nodes might not update the animations at the same time.
This is automatically done since MiddleVR 2.1.1 for animators that are enabled on startup. For animations added dynamically, make sure to do it yourself.
Random values#
At startup MiddleVR automatically gets the seed of Unity for pseudo-random values and synchronizes it across the cluster.
You can safely use the Random functions of Unity after the first
update of MVRManagerScript
. The Random functions will
output the same sequences on all cluster nodes.
If a random value is not automatically synchronized, you can always
use Cluster.BroadcastMessage
to send the
random value from the server to the clients.
Random objects positions#
If as a result of your scripts, objects have random
positions/orientations, you can apply to them the MVRClusterObject
which will synchronize their position/orientation at each end of frame.
Particles#
If the MVRManager parameter SimpleCluster
is enabled, the random seed of the particles will be automatically synchronized.
Auto Random Seed
will also be disabled.
Simple particles should then correctly be synchronized.
More complex particles cannot at this time be correctly synchronized.
Disabling the following parameters helps reduce chances of non-deterministic particles:
-
Limit Velocity over Lifetime
-
Rotation By Speed
-
External Forces
-
Noise
-
Collision
-
Triggers
-
Sub Emitters
-
Trails
Cinemachine#
The component CinemachineBrain
will automatically be removed from all MiddleVR cameras.
GUI#
It is perfectly possible to use Unity's GUI on the cluster with a few limitations:
- A "Screen Space - Overlay" GUI is not compatible with the Window Mode Compositor.
This means that:
-
either you interact with a GUI using a VR controller by using either "Screen Space - Camera" or "World Space". This should automatically be synchronized because the Wand position/orientation and buttons are automatically synchronized.
-
or interact with your mouse and use the "Screen Space - Overlay" GUI only on the server in conjunction with the Server Native Window option. You then must manually synchronize the GUI events with
MVR.Cluster.BroadcastMessage
Without cluster:
With cluster:
Custom Data#
I want to synchronize arbitrary data from the cluster server to the cluster nodes and back:
See MVR.Cluster.BroadcastMessage
.
Limitations#
It is possible that some parts of the app won't be correctly synchronized by MiddleVR. This includes:
- Tree foliage,
- Particles,
- Random events,
- Some post-processing effects, see Unity URP & HDRP,
- Videos: See Videos
Contact support for more information.
When to synchronize ?#
See Cluster Advanced Topic - When to synchronize?
MVRClusterObject#
The MVRClusterObject
script can be found in
MiddleVR/MiddleVR.Unity.Scripts/Cluster/MVRClusterObject.cs
.
When added to a GameObject, it can synchronize automatically:
-
Transform: local position, orientation and scale
-
Mono Behaviours parameters: Experimental Automatically synchronizes all serializable fields of a MonoBehaviour:
- All public fields
- All private fields tagged [Serializable]
If Synchronize Children
is enabled, will also synchronize all the children of
the Game Object.
Note: The synchronization will only be effective after the MVRManagerPostFrameScript update. See section When to synchronize ?
Supported types and limits#
Supported types:
-
Any basic type
-
Any blittable/unmanaged struct
-
Lists/Dictionary
-
Structs and Classes marked with the [Serializable] attributes:
- Includes any public field of a type that is serializable,
- Includes non-public fields if they have the [UnityEngine.SerializeField] attribute,
- Public fields with the [NonSerialized] attribute are skipped.
Notably NOT Supported:
- Arrays (please use List
/Dictionary instead)
Additionally, the top-level type being serialized can be a derived
type of UnityEngine.MonoBehaviour
or UnityEngine.ScriptableObject
.
Limitations of this implementation:
-
It does not support arbitrary object graphs. Reference cycles will be ignored.
-
It uses dynamic code generation thus it is incompatible with any AOT (Ahead Of Time) compilation mode like IL2CPP.
Performance considerations:
- Prefer avoiding polymorphic references in serialized objects as it is considerably slower to serialize/deserialize.
Optimization#
Objects sync#
Try to synchronize the minimum number of objects with MVRClusterObject. The more objects you synchronize manually, or the more number of physic objects that you use (which will be automatically synchronized), the slower the network will be.
Master display#
It's better to display a small viewport on the master and to disable its VSync if it's not part of the actual display system so it runs faster and doesn't slow down the rest of the cluster.
Also if the master has a different refresh rate than the rest of the cluster, use the Server Unity Window option.
Logs#
Disable Unity's output logs (in the Player Settings, Use Player Log) otherwise Unity will write its logs through the network and slow the application down.
Use a low LogLevel in MiddleVR Config.
CPU Intensive tasks#
It's better to put all the CPU intensive tasks before the
MVRManagerScript::Update
executes: we use a thread at the end of a frame
which will handle the frame synchronization, but Unity can start to work
before the frame sync is actually over, so we get some parallelization.
Just be aware that MiddleVR will update the devices state a bit later, when the MVRManagerScript executes.
LoadScene#
MiddleVR supports the call to SceneManager.LoadScene
to change the current scene.
The only thing you have to do is make sure that the MVRManager
is in the first scene
you are running but not the other ones because the original MVRManager
will not be
destroyed when changing scene.
See sample script Packages\MiddleVR\Runtime\MiddleVR.Unity.Scripts\Samples\MVRLoadSceneSample.cs
.