What is LIV?

A company of gamers, creators, and all-round great humans; all passionate about building solutions for sharing VR.

We are dedicated to providing tools & solutions aimed at creators first, and believe that these tools are then univerally helpful for developers, educators, and the industry at large.

Our tools are primarily used by VoD creators, streamers, and studios. Our vision is to provide competent tools & standards for all hardware platforms in VR/AR.

What is the LIV SDK?

The LIV SDK provides a spectator view of your application. It contextualizes what the user feels & experiences by capturing their body directly inside your world! Thanks to our software, creators can film inside your app and have full control over the camera.

With the power of out-of-engine compositing, a creator can express themselves freely without limits; as a real person or an avatar!

How It Works

The LIV SDK spawns a camera inside your app which is controlled by LIV. This camera then renders your app into a background and foreground, to allow the user’s body to be composited in. The background & foreground are separated by clipping geometry, based on the user’s location within the scene. These textures are then submitted for composition!

The compositor takes in multiple timestamped sources, performs latency compensation, and composites them together. This output can then be recorded or streamed using software like OBS or Discord.

Doing this work out-of-engine comes with some significant benefits:

  • Optimised resource use - we only do the bare minimum work required in the SDK, allowing it to stay lightweight and easy to maintain.
  • Accurate latency compensation works without any additional effort from you, the developer.
  • Additional camera types, effects, and layers can be added in future updates to the LIV App, making your SDK integration last longer.

For this to work well, we have developed a minimum-latency, high-performance transport layer that also handles resource management. LIV’s footprint is almost entirely dependent on how well-optimised your application is!

LIV on PCVR

You can download the LIV App on Steam for free!

When running VR applications on a PC, LIV works with these runtimes:

  • SteamVR (OpenVR API)
  • OpenXR (When provided by SteamVR)

Supported HMDs:

  • Meta Quest 1 & 2 using Link or AirLink
  • Meta Quest 1 & 2 using Virtual Desktop
  • Valve Index
  • HTC Vive & HTC Vive Pro
  • Rift S

Essentially all SteamVR compatible headsets are supported, and games published on Steam or Viveport will be automatically detected by LIV.

Support & Feedback

We want to hear your ideas, plans, and what you're going to end up doing with the SDK - if you've got an exotic idea that you might need a hand with - reach out. We're just as enthusiastic as you 💚

We're always available through our Discord Community, so please pop in if you need help. If you're not familiar with Discord, it's a popular gaming-oriented instant messenger & VOIP client.

Alternately, you can email us through techteam@liv.tv!

Support Links

LIV SDK v1.5 for Unity

Our Unity SDK can be downloaded directly through our Developer Portal.

Register your game and download the plugin from the portal. You will download the LIV SDK UnityPackage.

Overview

Importing & Setup

  1. Backup your project.
  2. Import the LIV SDK into your project. (Assets > Import > Custom Package)
  3. Add the LIV component anywhere in your project. Either spawn it every scene or use DontDestroyOnLoad.
  4. Set the LIV component's "HMD Camera" & "Stage" properties.

That's it - unless you're using URP. See the "Universal Render Pipeline" article for extra steps!

For more on rig hierarchy / customising your integration, read more below.

Testing

  • Download the LIV App.
  • Start LIV, and follow any prompts.
  • Launch the compositor.
  • You should now have an avatar on your screen. If you want to setup MR, refer to the online guide, linked in the app.
  • Go to Capture, Manual.
  • Run your project.
  • Target the Unity editor if testing there, otherwise target your app’s window!

If all has gone well, you'll see your game in LIV's output!

VR Rig Hierarchy

The LIV Unity SDK is interested in only two GameObjects, your HMD camera & "Stage".

The "Stage" object is where LIV's camera will be inserted in your hierarchy. It must be set to the GameObject that contains the player's hands. It must contain the entire player.

HMD Camera

Unity GameObject containing the player's HMD camera

This is the camera responsible for rendering the user’s HMD.

The LIV SDK, by default clones this object to match your application’s rendering setup. You can use your own camera prefab should you want to!

Stage

Unity GameObject containing the entire player, HMD & hands

This is the topmost transform of your VR rig.

When implementing VR locomotion (teleporting, joystick, etc), this is the GameObject that you should move around your scene. It represents the centre of the user’s playspace.

StageTransform

Extra Unity GameObject containing the stage GameObject

This transform is an additional wrapper to the user’s playspace, meant to allow for user-controlled transformations for special camera effects & transitions. This is optional.

If a creator is using a static camera, this transformation can give the illusion of camera movement. This video, for example, is filmed without any extra cameraman!

Customising Your Integration

Game-Driven Camera for Avatars

Call liv.render.SetPose(position, rotation, vfov) to control LIV's camera when avatars are being used!

Simply stop calling it to return control to the user's own camera.

Exclude MonoBehaviours

This is for removing unwanted scripts from the cloned MR camera when not using a camera prefab.

By default, we remove the AudioListener, Colliders and SteamVR scripts, as these are not necessary for rendering MR!

The excluded string must match the name of the MonoBehaviour.

Provide Your Own Camera Prefab

By default, LIV uses the HMD camera as a reference for the Mixed Reality camera.

It is cloned and set up as a Mixed Reality camera. This approach works for most apps.

However, some apps can experience issues because of custom MonoBehaviours attached to this camera.

You can use a custom camera prefab for those cases.

Spectator Layer Mask

This is a property on the LIV component.

The layer mask defines exactly which object layers should be rendered in MR.

You should use this to hide any in-game avatar that you’re using. LIV is meant to include the user’s body for you! Certain HMD-based effects should be disabled here too.

Also, this can be used to render special effects or additional UI only to the MR camera. Useful for showing the player’s health, or current score!

Shader Variants

LIV SDK adds MR-specific shader variants. That means that you can write a specific portion of existing shader to be used only for the mixed reality feed.

  • LIV_MR: Render into background and foreground texture
  • LIV_MR_BACKGROUND: Render only in the background texture
  • LIV_MR_FOREGROUND: Render only in foreground texture
#if defined(LIV_MR_BACKGROUND)
  col = fixed4(1, 1, 0, 1);
#elif defined(LIV_MR_FOREGROUND)
  col = fixed4(0, 1, 1, 1);
#else
  col = fixed4(0, 0, 1, 1);
#endif

Rendering Events

See the API reference for a list of Actions on the LIV component that can be used for adding custom behaviour to the LIV SDK.

Fix Post-Effects Alpha Channel

If your game is using any post-effects it is possible that the foreground texture alpha channel is corrupted.

By turning on this feature LIV tries to fix this issue by capturing the alpha channel before any post effects are applied and then simply overwriting the final alpha channel output with the captured one.

For Post-Effect Stack v2

  • On the Post-Process Layer, enable the Keep Alpha option.

Disable Standard Assets

This is a property on the LIV component.

Unity’s standard assets can interfere with the alpha channel that LIV needs to composite MR correctly.

This option disables all standard Unity assets for the Mixed Reality rendering.

Universal Render Pipeline

Make sure you've already setup the Unity SDK before following these steps:

  1. Find “File / Build Settings / Player Settings / Other Settings / Scripting Define Symbols”.
  2. Type: LIV_UNIVERSAL_RENDER
  3. Find your Forward Renderer Asset(s) in the Unity project. (Note: There can be multiple renderer assets in your project. Make sure you're doing this for all of the renderer assets that will be used with LIV)
  4. Under Renderer Features, add "SDK Universal Render Feature".
  5. On your LIV component enable “Fix Post-Effects alpha channel”.

High Definition Render Pipeline

Currently unsupported.

Known Issues

Unity crashes during frame debugging

When LIV is attached to Unity editor and it's in play mode, using frame debugging, exiting play mode suddenly, the next time you connect the LIV SDK to Unity, it will crash. This can be avoided by disabling frame debugging after being done with debugging and unpause the player to let LIV SDK recover before exiting play mode.

Frequently Asked Questions

The LIV avatar's head doesn't move when my game is running

This likely means that your application is running using Oculus, or OpenXR through Oculus. Please make sure you're either using OpenXR through SteamVR, or SteamVR as your runtime.

How do I control the LIV camera?

You can fully control the camera by calling:

liv.render.SetPose(position, rotation, vertical field of view)

The camera is automatically released whenever you stop calling this function.

Note that only LIV avatar cameras are currently controllable, as MR cameras must remain aligned with their physical sensors 😉

I can't see the avatar / player in the LIV output

If your game is using post-processing it is possible that the foreground texture alpha channel is corrupted.

You can fix it by enabling the "Fix Post-Effects alpha channel" toggle on the LIV component. If you're using the Universal Render Pipeline, make sure to follow our steps to have it correctly work.

LIV doesn't activate when I capture a build of my game

Make sure your Unity project is targeting 64bit builds. LIV is only compatible with 64bit applications.

API Reference

LIV.cs

This is the main LIV component.

Properties:

TypeNameAccess
ActiononActivateread/write
ActiononDeactivateread/write
ActiononPreRenderread/write
ActiononPostRenderread/write
ActiononPreRenderForegroundread/write
ActiononPostRenderForegroundread/write
ActiononPreRenderBackgroundread/write
ActiononPostRenderBackgroundread/write
Transformstageread/write
Matrix4x4stageLocalToWorldMatrixreadonly
Matrix4x4stageWorldToLocalMatrixreadonly
TransformstageTransformread/write
CameraHMDCameraread/write
CameraMRCameraPrefabread/write
booldisableStandardAssetsread/write
LayerMaskspectatorLayerMaskread/write
string[]excludeBehavioursread/write
boolfixPostEffectsAlpharead/write
boolisValidreadonly
boolisActivereadonly
SDKRenderrenderreadonly

SDKRender.cs

Owner of rendering for the LIV SDK.

Constructors:

TypeNameNote
voidSDKRender(LIV liv)Not for public use.

Properties:

TypeNameAccess
SDKInputFrameinputFramereadonly
CameracameraInstancereadonly
CameracameraReferencereadonly
CamerahmdCamerareadonly
Transformstagereadonly
TransformstageTransformreadonly
Matrix4x4stageLocalToWorldMatrixreadonly
Matrix4x4localToWorldMatrixreadonly
intspectatorLayerMaskreadonly
booldisableStandardAssetsreadonly
boolcanSetPosereadonly
LIVlivreadonly

Public Methods:

TypeNameNote
voidRender()Called by LIV SDK
boolSetPose(Vector3 position, Quaternion rotation, float verticalFieldOfView, bool useLocalSpace = false)Called by the developer. Supports only avatars right now. Default space is world space, when using localSpace the space is relative to Stage/StageTransform.
voidDispose()Called by LIV SDK

Changelog

v1.5.3

  • LIV SDK bridge is excluded from non-windows platforms.
  • Moved all strings to editor only to prevent garbage collection.
  • Fixed undefined bridge state handling.
  • Universal render pipeline reporting.
  • Improved code readability and internal behavior.

v1.5.2

  • Added canSetPose for detecting if the game can change the pose during runtime.. It can be accessed from liv.render.canSetPose.

v1.5.1

  • Changed approach to extracting alpha information for the foreground layer of LIV; should fix foreground compositing behaviour for most titles, including those on deferred rendering pipelines.
  • Near and far frustum parameters for LIV’s camera are now driven by the game’s HMD camera, and can be manually controlled using LIV.render.cameraInstance.farClipPlane & nearClipPlane
  • Added universal render pipeline support
  • Modified SetPose to be world space by default.

Updating from SDKv1

If you've used our old SDK (check for "LIV_MR.dll" in your plugins), here's a quick guide on how to update.

  1. Backup your project.
  2. Remove the LIV folder from the project's assets.
  3. Re-import the LIV SDK, and follow the new SDK setup steps.

Plugin Installation

  1. On the Developer Portal, register your game and download the plugin for the same Unreal version as your project.

Download the plugin version that matches your Unreal project

  1. Extract and copy the LIV plugin folder (contains LIV.uplugin) into the Plugins directory of your project. Create the directory if it doesn't already exist.

Copy the plugin into your project's Plugins folder

  1. If your project does not already support C++, add it by selecting "Tools" > "New C++ Class".

Create a new C++ class

  1. Regenerate the Visual Studio solution for your project. To do so, right-click your .uproject file, then select "Generate Visual Studio project files".

Right click your uproject file and generate project files

  1. Open the Visual Studio solution (.sln) and ensure LIV is loaded in the solution explorer.

Ensure the LIV plugin is present in Solution Explorer

  1. Run the Unreal Editor for your project within Visual Studio by selecting Build and run your project to launch the editor. You can just press F5 or click the button in the screenshot below.

Press F5 or click this button to build and launch UE4

  1. Ensure the LIV plugin is loaded in the Plugins window ("Edit" > "Plugins")

Access plugins window via Edit > Plugins

Integrating LIV

Quick Integration

Just by installing the plugin your game is able to be captured by LIV, for more advanced customisation see "Advanced Integration".

Advanced Integration

The LIV plugin provides a variety of rendering methods for capturing your gameplay. To change the method used you can select a different class for "Capture Method" in the LIV Plugin Settings. You can access this from Edit > Project Settings, in the LIV section under the Plugins header.

Access Project Settings via Edit > Project Settings

The fastest (and default) capture method is ULivCaptureSingle, although translucent objects will not render in the foreground.

If you want translucent objects to render on top of the player or avatar, then you must use ULivCaptureGlobalClipPlane(No)PostProcess.

Capture MethodDefaultSpeedPost-ProcessingForeground Transparency# of Captures
ULivCaptureSingleFastest1
ULivCaptureGlobalClipPlaneNoPostProcessSlower2
ULivCaptureGlobalClipPlanePostProcessSlowest3

Global Clip Plane

ULivCaptureGlobalClipPlane(No)PostProcess capture methods require the global clip plane to be manually enabled. Warning: this will incur a performance penalty.

If you wish to enable the global clip plane anyway, you can do so in the Project Settings > "Support for global clip plane...":

Enabling support for the Global Clip Plane in Project Settings

Show and Hide Actors and Components

By getting a reference to the Liv Local Player Subsystem via a Player Controller you can show and hide actors and components. A typical place to put the calls would be within the Begin Play and End Play events of your pawn. Make sure to check the validity of the subsystem!

Blueprint that hides an actor and a component.

Testing

Note: These instructions are the minimum you need to check if your integration worked.

We suggest testing with a packaged build of your application!

  • Download the LIV App.
  • Run the LIV app and launch the Avatar compositor
  • Run your project
  • Select your project under the "Manual" capture tab

If all has gone well, you'll see your application in LIV's output!

Known Issues

We are aware of a performance bug (UE-86191) affecting UE4.23 onwards, if this issue affects you or your customers we recommend voting on the issue.

Adding the following to your DefaultEngine.ini may help mitigate the performance issues:

[/Script/Engine.RendererSettings]
r.SceneRenderTargetResizeMethod=2
r.SceneRenderTargetResizeMethodForceOverride=True

LIV App API

This section of our documentation refers to the LIV App, available on Steam.

The LIV App currently has four published API surfaces:

  • Avatar Camera Plugins, used to animate and control LIV avatar cameras.
  • URI Scheme Commands, used to automate LIV's startup and management from code.
  • Registry, used to locate a LIV install or register external applications that support LIV.
  • Command Line, used to list all installed & supported LIV applications on the current PC.

All currently available APIs work for PCVR (aka, SteamVR) only.

We are working on better surfaces to interact with LIV from code, and want to provide as much control as possible - if you have a use-case not covered by our existing APIs, please reach out to Ruu#9999 on Discord and let him know! We have a lot in the works, and the more we learn early, the better the results will be.

What are URI Scheme Commands?

Long story short, they are URLs that are invokable as "shell commands" in Windows. As long as LIV is installed, and has been run once on a system, these commands will work when called from anywhere. Some commands will automatically start LIV, and boot LIV Capture. Some commands won't do anything if LIV is not already running and active.

You can call them manually, using the Windows "Run" prompt (Win+R).
You may also create shortcuts to these commands using Windows Explorer: New -> Shortcut, then enter your command. You can then create a hotkey for that shortcut by modifying its properties.

Behind the scenes, LIV registers itself as a URI Scheme handler for the liv-app "protocol". When Windows is asked to execute a URI as a shell command, it will look for a registered handler, and then start that program. When LIV is already running, a second instance of LIV is loaded - it then sends the command to the first instance of LIV before closing itself.
Naturally, this process takes some time. We don't recommend using this interface for high-frequency control of LIV.

AutoHotkey Example

Run liv.app://...

URI Commands - Cameras

These commands interact with LIV Capture for PCVR. These commands will not automatically launch LIV Capture if LIV is not already running.

liv-app://camera/set/<INDEX>

EG. liv-app://camera/set/2

Changes the currently active camera in LIV. The index is the number of the camera in the camera list, starting from 0. If you provide a camera index that is "out of bounds", the command will be ignored.
You may re-order cameras in LIV by modifying the configuration file in %localappdata%\LIV\App - modify the CompositorBasic.CameraProfiles array.

liv-app://camera/set/fovOverride/<NUMBER>

EG. liv-app://camera/set/fovOverride/43.5

Sets the current camera's FoV override value.
Set the FoV to 0 to clear the override.
Has no effect on avatar cameras; see avatar camera plugins.

liv-app://camera/add/fovOverride/<NUMBER>

EG. liv-app://camera/add/fovOverride/-1.5

Adds to the current camera's FoV override value.
If no override is set, it will use the camera's native FoV, then add to it as the override.
Has no effect on avatar cameras; see avatar camera plugins.

liv-app://camera/toggle/freeze

If you are using LIV with a tracked camera (IE. You have a SteamVR tracker attached to a camera) and you are not moving it, you can temporarily freeze it in place.
This command toggles the camera's frozen state.

URI Commands - Controlling LIV

All of the below commands will launch LIV if it is not running, then execute the command.

liv-app://launch/compositor

Starts PCVR Capture if not already running.

liv-app://launch/streamerkit

Starts LIV StreamerKit.

liv-app://focus/app

Brings LIV into the foreground.

liv-app://capture/manual/pid/<PROCESS ID>

EG. liv-app://capture/manual/pid/1337

Starts PCVR Capture if not already running, then switches to manual capture mode and attempts to capture the process ID given.

liv-app://launch/steam/<STEAM APPID>

EG. liv-app://launch/steam/1098100

Starts PCVR Capture if not already running, then switches to auto capture mode and launches the Steam game provided. Will not launch Steam games that are not detected by LIV. See the command line documentation for how to get a list of installed LIV games.
Once the game is loaded, LIV will automatically begin capturing it.

liv-app://launch/viveport/<VIVEPORT GAME ID>

EG. liv-app://launch/viveport/8fd7deb0-64d8-48b1-bd44-87db84f37de8

Starts PCVR Capture if not already running, then switches to auto capture mode and launches the Viveport game provided. Will not launch Viveport games that are not detected by LIV. See the command line documentation for how to get a list of installed LIV games.
Once the game is loaded, LIV will automatically begin capturing it.

Avatar Camera Plugins

Avatar camera plugins are currently managed through our community Discord, please join it and ask for access to the template project in the #camera-plugin-chat channel!

You will be able to publish plugins to the #camera-plugin-releases channel.

Avatar camera plugins are C# projects that compile a library that interfaces with LIV's Unity-based avatar renderer. You may control LIV's camera, or modify a user's avatar or scene.

Registry

LIV uses the registry to store a few helpful paths for locating and interacting with a LIV installation.

All registry keys are stored under the 64-bit Current User hive, in the location:

HKEY_CURRENT_USER\SOFTWARE\LIV.App

Stored here are the following keys:

KeyTypePurpose
BaseDirectoryStringPath to LIV's main directory.
ExecutableStringPath to LIV's main executable.
SDKRuntimePathStringPath to the LIV SDK's runtime directory.
ExternalApplicationsSub-keyA list of external applications compatible with LIV.

They are updated whenever LIV is opened. You can use the BaseDirectory path along with LIV.App.CommandLine.exe for easy access to the command-line API!

ExternalApplications

If you've integrated the LIV SDK, but are not distributing through Steam, Oculus, NetVios, or Viveport, you may add an entry to this sub-key and LIV will attempt to detect, launch, and automatically capture your app.

The format for these entries is described below, in JSON - this directly translates to the structure in the registry.

"ExternalApplications": {

  // An opaque string identifier for your application.
  "custom-app-id": {

    // User-visible name.
    "Name": "My Custom Application",

    // The app's install folder. Used to detect game engine & SDK version.
    "InstallPath": "C:\Path\To\App",

    "Executable": "C:\Path\To\App\Executable.exe",
    "Arguments": "-your -options -here",
    "WorkingDirectory": "C:\Path\To\App" // Optional.
  }
}

Screenshot of regedit.exe showing the values described above.

It is best practice, if you are adding your application to this list, to proactively update the values stored in the registry whenever your application is installed, run, or uninstalled.

If LIV cannot find the executable, or install path, it will not be added to LIV's capture list.
Additionally, if LIV cannot find the LIV SDK in the install path, it will not be added.

Changes to the registry are not live. You need to restart LIV for any changes to apply.

Command Line

You can find LIV's command line interface in the install folder, which you can navigate to by going through your Steam Library: Steam > Library > LIV > Properties > Local Files > Browse Local Files...
The executable is named LIV.App.CommandLine.exe.

All usable commands are documented in the executable itself, so refer to that for detailed and current documentation:
.\LIV.App.CommandLine.exe help
.\LIV.App.CommandLine.exe verb --help

version

EG. .\LIV.App.CommandLine.exe version

Outputs the current LIV Command Line version.

list-apps

EG. .\LIV.App.CommandLine.exe list-apps --human-readable
EG. .\LIV.App.CommandLine.exe list-apps --help

Lists all installed and supported LIV titles on the PC. You may filter for a minimum support level (No SDK, SDK, or SDKv1.5).
Refer to the command line documentation for more details.