> ## Documentation Index
> Fetch the complete documentation index at: https://docs.liv.tv/llms.txt
> Use this file to discover all available pages before exploring further.

# Vivox

> How to integrate Vivox voice chat audio capture with the LIV Camera Kit (LCK) SDK in Unreal Engine, including the FLCKVivoxSource implementation with TAtomic volume tracking, EVivoxAudioCallbackSource callback types, the required VivoxCore patch, channel mapping, and automatic audio source registration.

<Info>
  **Module:** LCKVivox | **Version:** 1.0 | **Platforms:** All
</Info>

## Overview

LCKVivox provides audio capture integration with Vivox, capturing both incoming voice chat audio and outgoing microphone audio for recording. It uses `TAtomic` volume tracking for thread-safe audio level monitoring and processes audio from multiple Vivox callback stages via `EVivoxAudioCallbackSource`.

## Supported Channels

| Channel      | Supported | Description                                          |
| ------------ | --------- | ---------------------------------------------------- |
| `Game`       | Yes       | Incoming voice chat audio (received before rendered) |
| `Microphone` | Yes       | Outgoing microphone audio (captured before sent)     |
| `VoiceChat`  | No        | Not used — Vivox audio mapped to Game/Microphone     |

<Note>
  LCKVivox maps Vivox audio to the standard `Game` and `Microphone` channels for unified audio handling with other sources.
</Note>

## Requirements

<Warning>
  **LCKVivox requires a patched version of VivoxCore.** The official VivoxCore plugin must be modified to add audio callback support. The LCKVivox plugin is disabled by default and will not compile without the patched VivoxCore installed.
</Warning>

* VivoxCore plugin for Unreal Engine (with LCK audio callbacks patch)
* Active Vivox account

## Source Implementation

### FLCKVivoxSource

The core audio source class that captures both incoming and outgoing Vivox audio:

```cpp theme={null}
class FLCKVivoxSource : public ILCKAudioSource
{
public:
    FLCKVivoxSource();
    virtual ~FLCKVivoxSource() override;

    bool StartCapture() noexcept override;
    bool StartCapture(TLCKAudioChannelsMask Channels) noexcept override;
    void StopCapture() noexcept override;
    const FString& GetSourceName() const noexcept override;
    float GetVolume() const noexcept override;

private:
    TAtomic<float> GameVolume{0.0f};
    TAtomic<float> MicVolume{0.0f};
};
```

| Member       | Type             | Description                                            |
| ------------ | ---------------- | ------------------------------------------------------ |
| `GameVolume` | `TAtomic<float>` | Thread-safe volume level for incoming voice chat audio |
| `MicVolume`  | `TAtomic<float>` | Thread-safe volume level for outgoing microphone audio |

### Thread-Safe Volume Tracking

`FLCKVivoxSource` uses `TAtomic<float>` for volume values because Vivox audio callbacks arrive on Vivox's internal audio thread, while `GetVolume()` may be called from the game thread. The atomic values ensure lock-free, thread-safe reads without blocking the audio pipeline.

### EVivoxAudioCallbackSource

The Vivox patch exposes multiple callback stages. `FLCKVivoxSource` uses specific stages for game and microphone audio:

```cpp theme={null}
enum class EVivoxAudioCallbackSource
{
    CaptureAfterRead,      // Raw mic data after capture
    CaptureBeforeSent,     // Mic data before network transmission
    RecvBeforeMixed,       // Per-participant received audio
    RecvBeforeRendered,    // Mixed received audio before playback
    FinalBeforeAEC,        // Final mix before echo cancellation
};
```

| Callback             | Used For                          | LCK Channel  |
| -------------------- | --------------------------------- | ------------ |
| `CaptureBeforeSent`  | Local microphone audio            | `Microphone` |
| `RecvBeforeRendered` | Incoming voice from other players | `Game`       |

<Note>
  The other callback stages (`CaptureAfterRead`, `RecvBeforeMixed`, `FinalBeforeAEC`) are available in the patched VivoxCore but are not used by the default `FLCKVivoxSource` implementation.
</Note>

## VivoxCore Patch

LCKVivox requires custom audio callbacks that are not available in the official VivoxCore plugin. You must apply the following modifications to VivoxCore before using LCKVivox.

### Files to Modify

<AccordionGroup>
  <Accordion title="1. VivoxConfig.h (+21 lines)">
    Add after the `media_codec_type` enum (around line 29):

    ```cpp theme={null}
    #include "Misc/Optional.h"  // Add at top with other includes

    enum class EVivoxAudioCallbackSource
    {
        CaptureAfterRead,
        CaptureBeforeSent,
        RecvBeforeMixed,
        RecvBeforeRendered,
        FinalBeforeAEC,
    };

    struct FVivoxAudioCallbackData
    {
        TArrayView<const float> PCM;
        int32 Channels = 0;
        int32 AudioFrameRate = 0;
        TOptional<bool> IsSpeaking;
        TOptional<bool> IsSilence;
    };

    DECLARE_MULTICAST_DELEGATE_TwoParams(FDelegateVivoxAudioData, EVivoxAudioCallbackSource, TArrayView<const FVivoxAudioCallbackData>)
    typedef FDelegateVivoxAudioData::FDelegate FOnVivoxAudioDataDelegate;
    VIVOXCORE_API FString EnumToString(EVivoxAudioCallbackSource Enum);
    ```
  </Accordion>

  <Accordion title="2. IClient.h (+2 lines)">
    Add at the end of the `IClient` class (before the closing `};`):

    ```cpp theme={null}
    virtual void SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataHandler) = 0;
    ```
  </Accordion>

  <Accordion title="3. ClientImpl.h (+14 lines)">
    Add in the `private:` section:

    ```cpp theme={null}
    FOnVivoxAudioDataDelegate OnAudioData;
    static void CaptureAfterReadCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame);
    static void CaptureBeforeSentCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_speaking);
    static void RecvBeforeMixedCallback(void* callback_handle, const char* session_group_handle, const char* session_uri,
        vx_before_recv_audio_mixed_participant_data_t* participants_data, size_t num_participants);
    static void RecvBeforeRenderedCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_silence);
    static void FinalBeforeAECCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame);
    ```

    Add in the `public:` section:

    ```cpp theme={null}
    void SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataHandler) override;
    ```
  </Accordion>

  <Accordion title="4. ClientImpl.cpp (+103 lines)">
    Add after `Cleanup()` function:

    ```cpp theme={null}
    void ClientImpl::SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataDelegate)
    {
        OnAudioData = AudioDataDelegate;
    }

    void ClientImpl::CaptureAfterReadCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame)
    {
        ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
        if (Config && Config->OnAudioData.IsBound())
        {
            TArray<float> PCM_FLT;
            const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
            Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
            Config->OnAudioData.Execute(EVivoxAudioCallbackSource::CaptureAfterRead, { {
                .PCM = PCM_FLT,
                .Channels = channels_per_frame,
                .AudioFrameRate = audio_frame_rate,
            } });
        }
    }

    void ClientImpl::CaptureBeforeSentCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_speaking)
    {
        ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
        if (Config && Config->OnAudioData.IsBound())
        {
            TArray<float> PCM_FLT;
            const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
            Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
            Config->OnAudioData.Execute(EVivoxAudioCallbackSource::CaptureBeforeSent, { {
                .PCM = PCM_FLT,
                .Channels = channels_per_frame,
                .AudioFrameRate = audio_frame_rate,
            } });
        }
    }

    void ClientImpl::RecvBeforeMixedCallback(void* callback_handle, const char* session_group_handle, const char* session_uri,
        vx_before_recv_audio_mixed_participant_data_t* participants_data, size_t num_participants)
    {
        ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
        if (Config && Config->OnAudioData.IsBound())
        {
            TArray<FVivoxAudioCallbackData> AudioData;
            TArray<TArray<float>> DataFloatArray;
            const auto Participants = MakeArrayView(participants_data, num_participants);
            for (auto Participant : Participants)
            {
                auto& PCM_FLT = DataFloatArray.Emplace_GetRef();
                const auto PCM = MakeArrayView(Participant.pcm_frames, Participant.pcm_frame_count * Participant.channels_per_frame);
                Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
                AudioData.Emplace(FVivoxAudioCallbackData{
                    .PCM = PCM_FLT,
                    .Channels = Participant.channels_per_frame,
                    .AudioFrameRate = Participant.audio_frame_rate,
                });
            }
            Config->OnAudioData.Execute(EVivoxAudioCallbackSource::RecvBeforeMixed, AudioData);
        }
    }

    void ClientImpl::RecvBeforeRenderedCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_silence)
    {
        ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
        if (Config && Config->OnAudioData.IsBound())
        {
            TArray<float> PCM_FLT;
            const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
            Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
            Config->OnAudioData.Execute(EVivoxAudioCallbackSource::RecvBeforeRendered, { {
                .PCM = PCM_FLT,
                .Channels = channels_per_frame,
                .AudioFrameRate = audio_frame_rate,
            } });
        }
    }

    void ClientImpl::FinalBeforeAECCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
        short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame)
    {
        ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
        if (Config && Config->OnAudioData.IsBound())
        {
            TArray<float> PCM_FLT;
            const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
            Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
            Config->OnAudioData.Execute(EVivoxAudioCallbackSource::FinalBeforeAEC, { {
                .PCM = PCM_FLT,
                .Channels = channels_per_frame,
                .AudioFrameRate = audio_frame_rate,
            } });
        }
    }
    ```

    In `Initialize()`, add before `vx_initialize3`:

    ```cpp theme={null}
    vxConfig.pf_on_audio_unit_after_capture_audio_read = CaptureAfterReadCallback;
    vxConfig.pf_on_audio_unit_before_capture_audio_sent = CaptureBeforeSentCallback;
    vxConfig.pf_on_audio_unit_before_recv_audio_mixed = RecvBeforeMixedCallback;
    vxConfig.pf_on_audio_unit_before_recv_audio_rendered = RecvBeforeRenderedCallback;
    vxConfig.pf_on_audio_unit_requesting_final_mix_for_echo_canceller_analysis = FinalBeforeAECCallback;
    vxConfig.callback_handle = this;
    ```
  </Accordion>

  <Accordion title="5. VivoxConfig.cpp (+12 lines)">
    Add at the end of the file:

    ```cpp theme={null}
    FString EnumToString(EVivoxAudioCallbackSource Enum)
    {
        switch (Enum)
        {
        case EVivoxAudioCallbackSource::CaptureAfterRead: return FString("CaptureAfterRead");
        case EVivoxAudioCallbackSource::CaptureBeforeSent: return FString("CaptureBeforeSent");
        case EVivoxAudioCallbackSource::RecvBeforeMixed: return FString("RecvBeforeMixed");
        case EVivoxAudioCallbackSource::RecvBeforeRendered: return FString("RecvBeforeRendered");
        case EVivoxAudioCallbackSource::FinalBeforeAEC: return FString("FinalBeforeAEC");
        default: return FString("UNKNOWN");
        }
    }
    ```
  </Accordion>
</AccordionGroup>

<Note>
  The patch adds audio callback hooks that allow LCKVivox to capture audio data at various stages of the Vivox audio pipeline without modifying Vivox's normal operation.
</Note>

## Dependencies

```csharp theme={null}
// Add to your .Build.cs
PublicDependencyModuleNames.AddRange(new string[] {
    "LCKVivox",
    "LCKAudio",
    "VivoxCore"
});
```

## Automatic Integration

The plugin automatically registers itself with the LCK audio system when the VivoxCore plugin is available. No manual initialization is required.

## Audio Flow

```
┌─────────────────────────────────────────────┐
│              Vivox SDK                       │
├─────────────────────────────────────────────┤
│  Incoming Voice    │    Outgoing Voice      │
│  (Other Players)   │    (Local Player)      │
│        ↓           │         ↓              │
│  RecvBefore        │  CaptureBefore         │
│   Rendered         │    Sent                │
└────────┬───────────┴────────────┬───────────┘
         │                        │
         ▼                        ▼
┌─────────────────┐    ┌─────────────────────┐
│  Game Channel   │    │ Microphone Channel  │
│  (Mixed audio)  │    │ (Pre-transmission)  │
└─────────────────┘    └─────────────────────┘
```

## Channel Mapping

| Vivox Source   | LCK Channel  | Description                                   |
| -------------- | ------------ | --------------------------------------------- |
| Incoming voice | `Game`       | Audio from other players in the voice channel |
| Outgoing voice | `Microphone` | Local player's microphone audio               |

This mapping allows:

* Voice chat to be recorded alongside game audio
* Microphone to be captured before network transmission
* Unified mixing with other audio sources

## Configuration

### Capture Channels Mask

By default, `FLCKVivoxSource` captures both Game and Microphone channels. You can customize which channels are captured by calling `StartCapture` with a `TLCKAudioChannelsMask`:

```cpp theme={null}
// Capture both channels (default behavior)
VivoxSource->StartCapture(ELCKAudioChannel::Game | ELCKAudioChannel::Microphone);

// Capture only incoming voice chat (other players)
VivoxSource->StartCapture(ELCKAudioChannel::Game);

// Capture only outgoing microphone audio (local player)
VivoxSource->StartCapture(ELCKAudioChannel::Microphone);
```

<Tip>
  Use `ELCKAudioChannel::Game` alone when you want to record other players' voice chat but exclude the local player's microphone (for example, if microphone audio is already captured by another source like LCKOboe or LCKUnrealAudio).
</Tip>

### Dual-Channel Behavior

LCKVivox captures audio at two distinct stages of the Vivox audio pipeline:

| LCK Channel  | Vivox Callback       | Description                                                                         |
| ------------ | -------------------- | ----------------------------------------------------------------------------------- |
| `Game`       | `RecvBeforeRendered` | Mixed audio from all remote participants, captured before speaker output            |
| `Microphone` | `CaptureBeforeSent`  | Local microphone audio after Vivox processing, captured before network transmission |

Each channel maintains its own `TAtomic<float>` volume level for thread-safe monitoring.

## Error Handling

If VivoxCore is not loaded when `StartCapture()` is called, it returns `false` and logs a warning to `LogLCKVivox`. The capture state remains inactive.

```cpp theme={null}
if (!VivoxSource->StartCapture())
{
    UE_LOG(LogTemp, Warning, TEXT("Vivox audio unavailable"));
}
```

<Warning>
  Call `StopCapture()` from the game thread for clean shutdown. The source uses a `FThreadSafeCounter` to track in-flight callbacks and waits up to 1 second for them to complete.
</Warning>

***

## Usage with Recording

When recording with LCKVivox enabled:

```cpp theme={null}
// Vivox audio automatically captured when recording starts
// No additional setup required beyond normal Vivox initialization

// Check if Vivox source is available
TArray<ILCKAudioSource*> Sources;
IModularFeatures::Get().GetModularFeatureImplementations<ILCKAudioSource>(
    ILCKAudioSource::GetModularFeatureName(), Sources);

for (ILCKAudioSource* Source : Sources)
{
    if (Source->GetSourceName() == TEXT("LCKVivox"))
    {
        UE_LOG(LogTemp, Log, TEXT("Vivox audio source available"));

        // Check supported channels
        auto Channels = Source->GetSupportedChannels();
        bool HasGame = (Channels & ELCKAudioChannel::Game) != 0;
        bool HasMic = (Channels & ELCKAudioChannel::Microphone) != 0;
    }
}
```

## Troubleshooting

<AccordionGroup>
  <Accordion title="No voice chat audio captured">
    1. Verify the VivoxCore patch has been applied correctly
    2. Check that players are connected to a Vivox channel
    3. Verify `SetAudioDataCallback` is being called during initialization
    4. Check LogLCKVivox for errors
  </Accordion>

  <Accordion title="Volume levels incorrect">
    1. Check `TAtomic` volume values via `GetVolume()`
    2. Verify Vivox channel volume settings in your game
    3. Ensure audio callback data contains non-zero PCM samples
  </Accordion>

  <Accordion title="Plugin not compiling">
    1. Verify all five VivoxCore patch files have been modified
    2. Check that `VivoxCore` module is listed in Build.cs dependencies
    3. Ensure `EVivoxAudioCallbackSource` enum is accessible
  </Accordion>
</AccordionGroup>

## Log Category

```cpp theme={null}
DECLARE_LOG_CATEGORY_EXTERN(LogLCKVivox, Log, All);
```

## See Also

<CardGroup cols={2}>
  <Card title="Audio Overview" icon="volume-high" href="/unreal/audio/overview">
    Audio system architecture
  </Card>

  <Card title="Unreal Audio" icon="gamepad" href="/unreal/audio/unreal-audio">
    Native audio capture
  </Card>
</CardGroup>