Skip to main content
Version: Next

Stream Vision Camera to Fishjam Mobile

note

This tutorial is exclusively for Mobile (React Native) applications.

Don't need VisionCamera?

Fishjam manages the camera itself with useCamera, and can blur the background or run your own shaders on it. If you don't need VisionCamera for something else, follow the background blur tutorial or the WebGPU effects tutorial instead.

This tutorial walks you through publishing a VisionCamera feed to a Fishjam room, step by step. By the end, your camera frames will reach other participants as a custom video track, the same path you'd later use to run frame processors or draw your own effects into the published video.

What you'll build​

A React Native app that joins a room and publishes the device camera through VisionCamera, with a live self-view.

What you'll learn​

  • How to install and configure VisionCamera v5 alongside the Fishjam SDK
  • How to publish camera frames with useVisionCameraSource
  • How custom video tracks appear to other participants

Prerequisites​

  • The React Native Quick Start completed, or an existing app wrapped in FishjamProvider
  • A physical device, or the iOS Simulator with a virtual camera from SimCam
  • The New Architecture enabled; custom video tracks require it

Step 1: Install and configure​

Install the packages​

npm install @fishjam-cloud/react-native-vision-camera-source react-native-vision-camera react-native-vision-camera-worklets react-native-worklets

Configure the Babel plugin​

VisionCamera's frame outputs run as worklets, which need the react-native-worklets Babel plugin. Add it as the last plugin:

module.exports = {
  presets: ["babel-preset-expo"],
  plugins: ["react-native-worklets/plugin"],
};

Restart Metro with a cleared cache afterwards (npx expo start --clear).

Configure camera permission​

{
  "expo": {
    "android": {
      "permissions": ["android.permission.CAMERA"]
    },
    "ios": {
      "infoPlist": {
        "NSCameraUsageDescription": "Allow $(PRODUCT_NAME) to access your camera."
      }
    }
  }
}

Rebuild the app​

npx expo prebuild
npx expo run:ios # or run:android

Step 2: Join a room​

As in the quick start, join a room with a peer token. This time there is no need to initialize the camera through the Fishjam SDK, since VisionCamera will manage the camera:

import React from "react";
import { Button } from "react-native";
import { useConnection, useSandbox } from "@fishjam-cloud/react-native-client";

const SANDBOX_API_URL = "YOUR_SANDBOX_API_URL";

export function JoinRoomButton() {
  const { joinRoom } = useConnection();
  const { getSandboxPeerToken } = useSandbox({
    sandboxApiUrl: SANDBOX_API_URL,
  });

  const handleJoinRoom = async () => {
    // For testing with the Sandbox API, use getSandboxPeerToken
    // For production apps, get the peerToken from your own backend instead
    const peerToken = await getSandboxPeerToken("testRoom", "streamer");
    await joinRoom({ peerToken });
  };

  return <Button title="Join Room" onPress={handleJoinRoom} />;
}

Step 3: Publish the camera​

useVisionCameraSource creates the video track, publishes it under the given source ID, and cleans up on unmount. Plug its frameOutput into VisionCamera's useCamera:

import React, { useEffect } from "react";
import { Text } from "react-native";
import {
  useCamera as useVisionCamera,
  useCameraPermission,
} from "react-native-vision-camera";
import { RTCView } from "@fishjam-cloud/react-native-client";
import { useVisionCameraSource } from "@fishjam-cloud/react-native-vision-camera-source";

export function CameraPublisher() {
  const { hasPermission, canRequestPermission, requestPermission } =
    useCameraPermission();

  useEffect(() => {
    if (!hasPermission && canRequestPermission) {
      requestPermission();
    }
  }, [hasPermission, canRequestPermission, requestPermission]);

  const { frameOutput, stream } = useVisionCameraSource("vision-camera");

  useVisionCamera({
    device: "front",
    isActive: hasPermission,
    outputs: [frameOutput],
  });

  if (!stream) return <Text>Starting camera…</Text>;
  return (
    <RTCView
      mediaStream={stream}
      style={{ height: 300, width: 300 }}
      objectFit="cover"
      mirror={true}
    />
  );
}

The stream also serves as your self-view. Frames are handed to Fishjam without copying pixels, and rotation and timestamps are handled for you.

Step 4: Display it on the other side​

Because the feed is a custom source, other participants find it in customVideoTracks, not in cameraTrack:

import React from "react";
import { View } from "react-native";
import { usePeers, RTCView } from "@fishjam-cloud/react-native-client";

export function RemoteStreams() {
  const { remotePeers } = usePeers();

  return (
    <View>
      {remotePeers.map((peer) =>
        peer.customVideoTracks.map(
          (track) =>
            track.stream && (
              <RTCView
                key={track.trackId}
                mediaStream={track.stream}
                style={{ height: 300, width: 300 }}
                objectFit="cover"
              />
            ),
        ),
      )}
    </View>
  );
}

To verify end to end, join the same room from a second device or another client built in the quick start, and watch the published feed arrive.

Complete example​

function CameraPublisher() {
  const { hasPermission, canRequestPermission, requestPermission } =
    useCameraPermission();

  useEffect(() => {
    if (!hasPermission && canRequestPermission) {
      requestPermission();
    }
  }, [hasPermission, canRequestPermission, requestPermission]);

  const { frameOutput, stream } = useVisionCameraSource("vision-camera");

  useVisionCamera({
    device: "front",
    isActive: hasPermission,
    outputs: [frameOutput],
  });

  if (!stream) return <Text>Starting camera…</Text>;
  return (
    <RTCView
      mediaStream={stream}
      style={styles.video}
      objectFit="cover"
      mirror={true}
    />
  );
}

function Room() {
  const { joinRoom, peerStatus } = useConnection();
  const { remotePeers } = usePeers();
  const { getSandboxPeerToken } = useSandbox({
    sandboxApiUrl: SANDBOX_API_URL,
  });
  const [isJoined, setIsJoined] = useState(false);

  const handleJoin = async () => {
    // For testing with the Sandbox API, use getSandboxPeerToken
    // For production apps, get the peerToken from your own backend instead
    const peerToken = await getSandboxPeerToken("testRoom", "streamer");
    await joinRoom({ peerToken });
    setIsJoined(true);
  };

  return (
    <View style={styles.container}>
      <Text style={styles.title}>Vision Camera × Fishjam</Text>
      <Text style={styles.status}>Status: {peerStatus}</Text>

      {!isJoined && <Button title="Join Room" onPress={handleJoin} />}

      <Text style={styles.sectionTitle}>Your camera</Text>
      <CameraPublisher />

      <Text style={styles.sectionTitle}>Other participants</Text>
      {remotePeers.map((peer) =>
        peer.customVideoTracks.map(
          (track) =>
            track.stream && (
              <RTCView
                key={track.trackId}
                mediaStream={track.stream}
                style={styles.video}
                objectFit="cover"
              />
            ),
        ),
      )}
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    padding: 20,
  },
  title: {
    fontSize: 24,
    fontWeight: "bold",
    marginBottom: 10,
  },
  status: {
    fontSize: 16,
    marginBottom: 20,
  },
  sectionTitle: {
    fontSize: 18,
    fontWeight: "600",
    marginVertical: 10,
  },
  video: {
    height: 200,
    width: 200,
    borderRadius: 8,
  },
});

export default function App() {
  return (
    <FishjamProvider fishjamId={FISHJAM_ID}>
      <Room />
    </FishjamProvider>
  );
}

Next steps​