LCKVivox

Module: LCKVivox | Version: 0.9.2 | Platforms: All

Overview

LCKVivox provides audio capture integration with Vivox, capturing both incoming voice chat audio and outgoing microphone audio for recording.

Supported Channels

Channel	Supported	Description
`Game`	Yes	Incoming voice chat audio (received before rendered)
`Microphone`	Yes	Outgoing microphone audio (captured before sent)
`VoiceChat`	No	Not used - Vivox audio mapped to Game/Microphone

LCKVivox maps Vivox audio to the standard Game and Microphone channels for unified audio handling with other sources.

Requirements

LCKVivox requires a patched version of VivoxCore. The official VivoxCore plugin must be modified to add audio callback support. The LCKVivox plugin is disabled by default and will not compile without the patched VivoxCore installed.

VivoxCore plugin for Unreal Engine (with LCK audio callbacks patch)
Active Vivox account

VivoxCore Patch

LCKVivox requires custom audio callbacks that are not available in the official VivoxCore plugin. You must apply the following modifications to VivoxCore before using LCKVivox.

Files to Modify

1. VivoxConfig.h (+21 lines)

Add after the media_codec_type enum (around line 29):

#include "Misc/Optional.h"  // Add at top with other includes

enum class EVivoxAudioCallbackSource
{
    CaptureAfterRead,
    CaptureBeforeSent,
    RecvBeforeMixed,
    RecvBeforeRendered,
    FinalBeforeAEC,
};

struct FVivoxAudioCallbackData
{
    TArrayView<const float> PCM;
    int32 Channels = 0;
    int32 AudioFrameRate = 0;
    TOptional<bool> IsSpeaking;
    TOptional<bool> IsSilence;
};

DECLARE_MULTICAST_DELEGATE_TwoParams(FDelegateVivoxAudioData, EVivoxAudioCallbackSource, TArrayView<const FVivoxAudioCallbackData>)
typedef FDelegateVivoxAudioData::FDelegate FOnVivoxAudioDataDelegate;
VIVOXCORE_API FString EnumToString(EVivoxAudioCallbackSource Enum);

2. IClient.h (+2 lines)

Add at the end of the IClient class (before the closing };):

virtual void SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataHandler) = 0;

3. ClientImpl.h (+14 lines)

Add in the private: section:

FOnVivoxAudioDataDelegate OnAudioData;
static void CaptureAfterReadCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame);
static void CaptureBeforeSentCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_speaking);
static void RecvBeforeMixedCallback(void* callback_handle, const char* session_group_handle, const char* session_uri,
    vx_before_recv_audio_mixed_participant_data_t* participants_data, size_t num_participants);
static void RecvBeforeRenderedCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_silence);
static void FinalBeforeAECCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame);

Add in the public: section:

void SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataHandler) override;

4. ClientImpl.cpp (+103 lines)

Add after Cleanup() function:

void ClientImpl::SetAudioDataCallback(FOnVivoxAudioDataDelegate AudioDataDelegate)
{
    OnAudioData = AudioDataDelegate;
}

void ClientImpl::CaptureAfterReadCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame)
{
    ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
    if (Config && Config->OnAudioData.IsBound())
    {
        TArray<float> PCM_FLT;
        const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
        Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
        Config->OnAudioData.Execute(EVivoxAudioCallbackSource::CaptureAfterRead, { {
            .PCM = PCM_FLT,
            .Channels = channels_per_frame,
            .AudioFrameRate = audio_frame_rate,
        } });
    }
}

void ClientImpl::CaptureBeforeSentCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_speaking)
{
    ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
    if (Config && Config->OnAudioData.IsBound())
    {
        TArray<float> PCM_FLT;
        const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
        Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
        Config->OnAudioData.Execute(EVivoxAudioCallbackSource::CaptureBeforeSent, { {
            .PCM = PCM_FLT,
            .Channels = channels_per_frame,
            .AudioFrameRate = audio_frame_rate,
        } });
    }
}

void ClientImpl::RecvBeforeMixedCallback(void* callback_handle, const char* session_group_handle, const char* session_uri,
    vx_before_recv_audio_mixed_participant_data_t* participants_data, size_t num_participants)
{
    ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
    if (Config && Config->OnAudioData.IsBound())
    {
        TArray<FVivoxAudioCallbackData> AudioData;
        TArray<TArray<float>> DataFloatArray;
        const auto Participants = MakeArrayView(participants_data, num_participants);
        for (auto Participant : Participants)
        {
            auto& PCM_FLT = DataFloatArray.Emplace_GetRef();
            const auto PCM = MakeArrayView(Participant.pcm_frames, Participant.pcm_frame_count * Participant.channels_per_frame);
            Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
            AudioData.Emplace(FVivoxAudioCallbackData{
                .PCM = PCM_FLT,
                .Channels = Participant.channels_per_frame,
                .AudioFrameRate = Participant.audio_frame_rate,
            });
        }
        Config->OnAudioData.Execute(EVivoxAudioCallbackSource::RecvBeforeMixed, AudioData);
    }
}

void ClientImpl::RecvBeforeRenderedCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame, int is_silence)
{
    ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
    if (Config && Config->OnAudioData.IsBound())
    {
        TArray<float> PCM_FLT;
        const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
        Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
        Config->OnAudioData.Execute(EVivoxAudioCallbackSource::RecvBeforeRendered, { {
            .PCM = PCM_FLT,
            .Channels = channels_per_frame,
            .AudioFrameRate = audio_frame_rate,
        } });
    }
}

void ClientImpl::FinalBeforeAECCallback(void* callback_handle, const char* session_group_handle, const char* initial_target_uri,
    short* pcm_frames, int pcm_frame_count, int audio_frame_rate, int channels_per_frame)
{
    ClientImpl* Config = reinterpret_cast<ClientImpl*>(callback_handle);
    if (Config && Config->OnAudioData.IsBound())
    {
        TArray<float> PCM_FLT;
        const auto PCM = MakeArrayView(pcm_frames, pcm_frame_count * channels_per_frame);
        Algo::Transform(PCM, PCM_FLT, [](int32 V) { return static_cast<double>(V) / static_cast<double>(TNumericLimits<uint16>::Max()); });
        Config->OnAudioData.Execute(EVivoxAudioCallbackSource::FinalBeforeAEC, { {
            .PCM = PCM_FLT,
            .Channels = channels_per_frame,
            .AudioFrameRate = audio_frame_rate,
        } });
    }
}

In Initialize(), add before vx_initialize3:

vxConfig.pf_on_audio_unit_after_capture_audio_read = CaptureAfterReadCallback;
vxConfig.pf_on_audio_unit_before_capture_audio_sent = CaptureBeforeSentCallback;
vxConfig.pf_on_audio_unit_before_recv_audio_mixed = RecvBeforeMixedCallback;
vxConfig.pf_on_audio_unit_before_recv_audio_rendered = RecvBeforeRenderedCallback;
vxConfig.pf_on_audio_unit_requesting_final_mix_for_echo_canceller_analysis = FinalBeforeAECCallback;
vxConfig.callback_handle = this;

5. VivoxConfig.cpp (+12 lines)

Add at the end of the file:

FString EnumToString(EVivoxAudioCallbackSource Enum)
{
    switch (Enum)
    {
    case EVivoxAudioCallbackSource::CaptureAfterRead: return FString("CaptureAfterRead");
    case EVivoxAudioCallbackSource::CaptureBeforeSent: return FString("CaptureBeforeSent");
    case EVivoxAudioCallbackSource::RecvBeforeMixed: return FString("RecvBeforeMixed");
    case EVivoxAudioCallbackSource::RecvBeforeRendered: return FString("RecvBeforeRendered");
    case EVivoxAudioCallbackSource::FinalBeforeAEC: return FString("FinalBeforeAEC");
    default: return FString("UNKNOWN");
    }
}

The patch adds audio callback hooks that allow LCKVivox to capture audio data at various stages of the Vivox audio pipeline without modifying Vivox’s normal operation.

Dependencies

// Add to your .Build.cs
PublicDependencyModuleNames.AddRange(new string[] {
    "LCKVivox",
    "LCKAudio",
    "VivoxCore"
});

Automatic Integration

The plugin automatically registers itself with the LCK audio system when the VivoxCore plugin is available. No manual initialization is required.

Audio Flow

┌─────────────────────────────────────────────┐
│              Vivox SDK                       │
├─────────────────────────────────────────────┤
│  Incoming Voice    │    Outgoing Voice      │
│  (Other Players)   │    (Local Player)      │
└────────┬───────────┴────────────┬───────────┘
         │                        │
         ▼                        ▼
┌─────────────────┐    ┌─────────────────────┐
│  Game Channel   │    │ Microphone Channel  │
│  (Mixed audio)  │    │ (Pre-transmission)  │
└─────────────────┘    └─────────────────────┘

Channel Mapping

Vivox Source	LCK Channel	Description
Incoming voice	`Game`	Audio from other players in the voice channel
Outgoing voice	`Microphone`	Local player’s microphone audio

This mapping allows:

Voice chat to be recorded alongside game audio
Microphone to be captured before network transmission
Unified mixing with other audio sources

Usage with Recording

When recording with LCKVivox enabled:

// Vivox audio automatically captured when recording starts
// No additional setup required beyond normal Vivox initialization

// Check if Vivox source is available
TArray<ILCKAudioSource*> Sources;
IModularFeatures::Get().GetModularFeatureImplementations<ILCKAudioSource>(
    ILCKAudioSource::GetModularFeatureName(), Sources);

for (ILCKAudioSource* Source : Sources)
{
    if (Source->GetSourceName() == TEXT("LCKVivox"))
    {
        UE_LOG(LogTemp, Log, TEXT("Vivox audio source available"));

        // Check supported channels
        auto Channels = Source->GetSupportedChannels();
        bool HasGame = (Channels & ELCKAudioChannel::Game) != 0;
        bool HasMic = (Channels & ELCKAudioChannel::Microphone) != 0;
    }
}

Log Category

DECLARE_LOG_CATEGORY_EXTERN(LogLCKVivox, Log, All);

Audio Overview

Audio system architecture

Unreal Audio

Native audio capture

Unreal Customization

Overview

Supported Channels

Requirements

VivoxCore Patch

Files to Modify

Dependencies

Automatic Integration

Audio Flow

Channel Mapping

Usage with Recording

Log Category

See Also

Audio Overview

Unreal Audio

Unreal Customization

​Overview

​Supported Channels

​Requirements

​VivoxCore Patch

​Files to Modify

​Dependencies

​Automatic Integration

​Audio Flow

​Channel Mapping

​Usage with Recording

​Log Category

​See Also

Audio Overview

Unreal Audio

Overview

Supported Channels

Requirements

VivoxCore Patch

Files to Modify

Dependencies

Automatic Integration

Audio Flow

Channel Mapping

Usage with Recording

Log Category

See Also