What is LIV?

A company of gamers, creators, and all-round great humans; all passionate about building solutions for sharing VR.

We are dedicated to providing tools & solutions aimed at creators first, and believe that these tools are then univerally helpful for developers, educators, and the industry at large.

Our tools are primarily used by VoD creators, streamers, and studios. Our vision is to provide competent tools & standards for all hardware platforms in VR/AR.

What is the LIV SDK?

The LIV SDK provides a spectator view of your application. It contextualizes what the user feels & experiences by capturing their body directly inside your world! Thanks to our software, creators can film inside your app and have full control over the camera.

With the power of out-of-engine compositing, a creator can express themselves freely without limits; as a real person or an avatar!

How It Works

The LIV SDK spawns a camera inside your app which is controlled by LIV. This camera then renders your app into a background and foreground, to allow the user’s body to be composited in. The background & foreground are separated by clipping geometry, based on the user’s location within the scene. These textures are then submitted for composition!

The compositor takes in multiple timestamped sources, performs latency compensation, and composites them together. This output can then be recorded or streamed using software like OBS or Discord.

Doing this work out-of-engine comes with some significant benefits:

  • Optimised resource use - we only do the bare minimum work required in the SDK, allowing it to stay lightweight and easy to maintain.
  • Accurate latency compensation works without any additional effort from you, the developer.
  • Additional camera types, effects, and layers can be added in future updates to the LIV App, making your SDK integration last longer.

For this to work well, we have developed a minimum-latency, high-performance transport layer that also handles resource management. LIV’s footprint is almost entirely dependent on how well-optimised your application is!

Supported Platforms & Runtimes

LIV's SDK is made for PCVR only.

LIV works with applications that run using these runtimes:

  • SteamVR (OpenVR API)
  • OpenXR (When provided by SteamVR)

Supported HMDs:

  • Meta Quest 1 & 2 using Link or AirLink
  • Meta Quest 1 & 2 using Virtual Desktop
  • Valve Index
  • HTC Vive & HTC Vive Pro
  • Rift S

Essentially all SteamVR compatible headsets are supported, and games published on Steam or Viveport will be automatically detected by LIV.

Meta Quest Native & MRC

LIV supports MRC capture from Meta Quest devices using the official Oculus MRC implementations in both Unity and Unreal.

Support & Feedback

We want to hear your ideas, plans, and what you're going to end up doing with the SDK - if you've got an exotic idea that you might need a hand with - reach out. We're just as enthusiastic as you 💚

We're always available through our Discord Community, so please pop in if you need help. If you're not familiar with Discord, it's a popular gaming-oriented instant messenger & VOIP client.

Alternately, you can email us through techteam@liv.tv!

Support Links

LIV SDK v1.5 for Unity

Our Unity SDK can be downloaded directly through our Developer Portal.

Register your game and download the plugin from the portal. You will download the LIV SDK UnityPackage.

Overview

Importing & Setup

  1. Backup your project.
  2. Import the LIV SDK into your project. (Assets > Import > Custom Package)
  3. Add the LIV component anywhere in your project. Either spawn it every scene or use DontDestroyOnLoad.
  4. Set the LIV component's "HMD Camera" & "Stage" properties.

That's it - unless you're using URP. See the "Universal Render Pipeline" article for extra steps!

For more on rig hierarchy / customising your integration, read more below.

Testing

  • Download the LIV App.
  • Start LIV, and follow any prompts.
  • Launch the compositor.
  • You should now have an avatar on your screen. If you want to setup MR, refer to the online guide, linked in the app.
  • Go to Capture, Manual.
  • Run your project.
  • Target the Unity editor if testing there, otherwise target your app’s window!

If all has gone well, you'll see your game in LIV's output!

VR Rig Hierarchy

The LIV Unity SDK is interested in only two GameObjects, your HMD camera & "Stage".

The "Stage" object is where LIV's camera will be inserted in your hierarchy. It must be set to the GameObject that contains the player's hands. It must contain the entire player.

HMD Camera

Unity GameObject containing the player's HMD camera

This is the camera responsible for rendering the user’s HMD.

The LIV SDK, by default clones this object to match your application’s rendering setup. You can use your own camera prefab should you want to!

Stage

Unity GameObject containing the entire player, HMD & hands

This is the topmost transform of your VR rig.

When implementing VR locomotion (teleporting, joystick, etc), this is the GameObject that you should move around your scene. It represents the centre of the user’s playspace.

StageTransform

Extra Unity GameObject containing the stage GameObject

This transform is an additional wrapper to the user’s playspace, meant to allow for user-controlled transformations for special camera effects & transitions. This is optional.

If a creator is using a static camera, this transformation can give the illusion of camera movement. This video, for example, is filmed without any extra cameraman!

Customising Your Integration

Game-Driven Camera for Avatars

Call liv.render.SetPose(position, rotation, vfov) to control LIV's camera when avatars are being used!

Simply stop calling it to return control to the user's own camera.

Exclude MonoBehaviours

This is for removing unwanted scripts from the cloned MR camera when not using a camera prefab.

By default, we remove the AudioListener, Colliders and SteamVR scripts, as these are not necessary for rendering MR!

The excluded string must match the name of the MonoBehaviour.

Provide Your Own Camera Prefab

By default, LIV uses the HMD camera as a reference for the Mixed Reality camera.

It is cloned and set up as a Mixed Reality camera. This approach works for most apps.

However, some apps can experience issues because of custom MonoBehaviours attached to this camera.

You can use a custom camera prefab for those cases.

Spectator Layer Mask

This is a property on the LIV component.

The layer mask defines exactly which object layers should be rendered in MR.

You should use this to hide any in-game avatar that you’re using. LIV is meant to include the user’s body for you! Certain HMD-based effects should be disabled here too.

Also, this can be used to render special effects or additional UI only to the MR camera. Useful for showing the player’s health, or current score!

Shader Variants

LIV SDK adds MR-specific shader variants. That means that you can write a specific portion of existing shader to be used only for the mixed reality feed.

  • LIV_MR: Render into background and foreground texture
  • LIV_MR_BACKGROUND: Render only in the background texture
  • LIV_MR_FOREGROUND: Render only in foreground texture
#if defined(LIV_MR_BACKGROUND)
  col = fixed4(1, 1, 0, 1);
#elif defined(LIV_MR_FOREGROUND)
  col = fixed4(0, 1, 1, 1);
#else
  col = fixed4(0, 0, 1, 1);
#endif

Rendering Events

See the API reference for a list of Actions on the LIV component that can be used for adding custom behaviour to the LIV SDK.

Fix Post-Effects Alpha Channel

If your game is using any post-effects it is possible that the foreground texture alpha channel is corrupted.

By turning on this feature LIV tries to fix this issue by capturing the alpha channel before any post effects are applied and then simply overwriting the final alpha channel output with the captured one.

For Post-Effect Stack v2

  • On the Post-Process Layer, enable the Keep Alpha option.

Disable Standard Assets

This is a property on the LIV component.

Unity’s standard assets can interfere with the alpha channel that LIV needs to composite MR correctly.

This option disables all standard Unity assets for the Mixed Reality rendering.

Universal Render Pipeline

Make sure you've already setup the Unity SDK before following these steps:

  1. Find “File / Build Settings / Player Settings / Other Settings / Scripting Define Symbols”.
  2. Type: LIV_UNIVERSAL_RENDER
  3. Find your Forward Renderer Asset in the Unity project.
  4. Under Renderer Features, add "SDK Universal Render Feature".
  5. On your LIV component enable “Fix Post-Effects alpha channel”.

High Definition Render Pipeline

Currently unsupported.

Known Issues

Unity crashes during frame debugging

When LIV is attached to Unity editor and it's in play mode, using frame debugging, exiting play mode suddenly, the next time you connect the LIV SDK to Unity, it will crash. This can be avoided by disabling frame debugging after being done with debugging and unpause the player to let LIV SDK recover before exiting play mode.

Frequently Asked Questions

The LIV avatar's head doesn't move when my game is running

This likely means that your application is running using Oculus, or OpenXR through Oculus. Please make sure you're either using OpenXR through SteamVR, or SteamVR as your runtime.

How do I control the LIV camera?

You can fully control the camera by calling:

liv.render.SetPose(position, rotation, vertical field of view)

The camera is automatically released whenever you stop calling this function.

Note that only LIV avatar cameras are currently controllable, as MR cameras must remain aligned with their physical sensors 😉

I can't see the avatar / player in the LIV output

If your game is using post-processing it is possible that the foreground texture alpha channel is corrupted.

You can fix it by enabling the "Fix Post-Effects alpha channel" toggle on the LIV component. If you're using the Universal Render Pipeline, make sure to follow our steps to have it correctly work.

LIV doesn't activate when I capture a build of my game

Make sure your Unity project is targeting 64bit builds. LIV is only compatible with 64bit applications.

API Reference

LIV.cs

This is the main LIV component.

Properties:

TypeNameAccess
ActiononActivateread/write
ActiononDeactivateread/write
ActiononPreRenderread/write
ActiononPostRenderread/write
ActiononPreRenderForegroundread/write
ActiononPostRenderForegroundread/write
ActiononPreRenderBackgroundread/write
ActiononPostRenderBackgroundread/write
Transformstageread/write
Matrix4x4stageLocalToWorldMatrixreadonly
Matrix4x4stageWorldToLocalMatrixreadonly
TransformstageTransformread/write
CameraHMDCameraread/write
CameraMRCameraPrefabread/write
booldisableStandardAssetsread/write
LayerMaskspectatorLayerMaskread/write
string[]excludeBehavioursread/write
boolfixPostEffectsAlpharead/write
boolisValidreadonly
boolisActivereadonly
SDKRenderrenderreadonly

SDKRender.cs

Owner of rendering for the LIV SDK.

Constructors:

TypeNameNote
voidSDKRender(LIV liv)Not for public use.

Properties:

TypeNameAccess
SDKInputFrameinputFramereadonly
CameracameraInstancereadonly
CameracameraReferencereadonly
CamerahmdCamerareadonly
Transformstagereadonly
TransformstageTransformreadonly
Matrix4x4stageLocalToWorldMatrixreadonly
Matrix4x4localToWorldMatrixreadonly
intspectatorLayerMaskreadonly
booldisableStandardAssetsreadonly
boolcanSetPosereadonly
LIVlivreadonly

Public Methods:

TypeNameNote
voidRender()Called by LIV SDK
boolSetPose(Vector3 position, Quaternion rotation, float verticalFieldOfView, bool useLocalSpace = false)Called by the developer. Supports only avatars right now. Default space is world space, when using localSpace the space is relative to Stage/StageTransform.
voidDispose()Called by LIV SDK

Changelog

v1.5.3

  • LIV SDK bridge is excluded from non-windows platforms.
  • Moved all strings to editor only to prevent garbage collection.
  • Fixed undefined bridge state handling.
  • Universal render pipeline reporting.
  • Improved code readability and internal behavior.

v1.5.2

  • Added canSetPose for detecting if the game can change the pose during runtime.. It can be accessed from liv.render.canSetPose.

v1.5.1

  • Changed approach to extracting alpha information for the foreground layer of LIV; should fix foreground compositing behaviour for most titles, including those on deferred rendering pipelines.
  • Near and far frustum parameters for LIV’s camera are now driven by the game’s HMD camera, and can be manually controlled using LIV.render.cameraInstance.farClipPlane & nearClipPlane
  • Added universal render pipeline support
  • Modified SetPose to be world space by default.

Updating from SDKv1

If you've used our old SDK (check for "LIV_MR.dll" in your plugins), here's a quick guide on how to update.

  1. Backup your project.
  2. Remove the LIV folder from the project's assets.
  3. Re-import the LIV SDK, and follow the new SDK setup steps.

LIV SDK v1.5 for Unreal

Our Unreal SDK can be downloaded directly through our Developer Portal.

Register your game and download the plugin for your version of Unreal from the portal. You will download a zip containing the Unreal plugin.

Plugin Installation

  1. If not already present, create a directory called Plugins in your Unreal project's directory.

Ensure a directory named Plugins is present in your UE4 project directory

  1. Extract the LIV plugin content into the Plugins directory, it should be in a folder named 'LIV'.

Copy this directory into the Plugins directory

  1. Generate project files for your project (right click your [Project-Name].uproject file > Generate Visual Studio project files)

Right click your uproject file and generate project files

  1. Open the Visual Studio solution ([Project-Name].uproject) and make sure you can see the LIV plugin in the solution explorer.

Ensure the LIV plugin is present in Solution Explorer

  1. Build and run your project to launch the editor. You can just press F5 or click the button in the screenshot below.

Press F5 or click this button to build and launch UE4

  1. You can check the LIV plugin is installed correctly by navigating to Edit > Plugins.

Access plugins window via Edit > Plugins

  1. Check you see the LIV plugin which will typically be at the bottom of the lefthand list under Project.

Access plugins window via Edit > Plugins

Integrating LIV

Quick Integration

Just by installing the plugin your game is able to be captured by LIV, for more advanced customisation see "Advanced Integration".

Advanced Integration

The LIV plugin provides a variety of rendering methods for capturing your gameplay. To change the method used you can select a different class for "Capture Method" in the LIV Plugin Settings. You can access this from Edit > Project Settings, in the LIV section under the Plugins header.

Access Project Settings via Edit > Project Settings

The default capture method is ULivCaptureMeshClipPlaneNoPostProcess as it is one of the cheaper capture methods and will work without requiring you to enable the global clip plane.

If you want transparent objects in the foreground to render over the top of the player / avatar then then you must use a global clip plane based capture method.

Below we will outline some details of the capture methods LIV provides.

ULivCaptureGlobalClipPlaneNoPostProcess

This method captures your gameplay twice each frame for the foreground and the background. It requires support for the global clip plane in Project Settings. The captured images do not have post processing and this method requires 3 render targets.

ULivCaptureGlobalClipPlanePostProcess

This method requires capturing your gameplay three times each frame to provide post processed images with the correct masks to LIV. This method also requires support for the global clip plane. We require 4 render targets for this method.

ULivCaptureMeshClipPlaneNoPostProcess

This method uses a mesh instead of the global clip plane to separate foreground and background and requires capturing twice each frame. 4 render targets are used in this method and the images will not using post processing.

ULivCaptureMeshClipPlanePostProcess

This method also uses a mesh instead of the global clip plane and does support post processing but requires 3 captures each frame. Overally we also require 5 render targets for this method.

Global Clip Plane

Some of LIV's capture method require the Global Clip Plane to be enabled. Unless you are already utilising the Global Clip Plane then it is best to use the methods that do not require it as there is a performance impact from enabling it.

If you wish to enable support for the global clip plane it can be found in Project Settings and requires restarting the editor.

Enabling support for the Global Clip Plane in Project Settings

Show and Hide Actors and Components

By getting a reference to the Liv Local Player Subsystem via a Player Controller you can show and hide actors and components. A typical place to put the calls would be within the Begin Play and End Play events of your pawn. Make sure to check the validity of the subsystem!

Blueprint that hides an actor and a component.

Testing

Note: These instructions are the minimum you need to check if your integration worked.

We suggest testing with a packaged build of your application!

  • Download the LIV App.
  • Run it, install the Virtual Camera driver.
  • In LIV, launch the compositor.
  • Run your project.
  • Using "Manual" capture mode, pick your project from the list.

If all has gone well, you'll see your application in LIV's output!

Known Issues

We are aware of a performance bug (UE-86191) affecting UE4.23 onwards, if this issue affects you or your customers we recomend voting on the issue.

Adding the following to your DefaultEngine.ini may help mitigate the performance issues:

[/Script/Engine.RendererSettings]
r.SceneRenderTargetResizeMethod=2
r.SceneRenderTargetResizeMethodForceOverride=True