Logitech Hackathon Resources

This page contains resources for Logitech hackathon challenges. Have fun!

View on GitHub

LogiInsight Webcam — PC SDK API Reference

SDK version 1.5.2 · for the LogiInsight hackathon webcam

The AI features described here exist only on the LogiInsight webcam (the AI runs inside it). With another webcam (MX Brio, your own, a laptop’s built-in camera) only the plain video parts apply.

Quick start and installation: the quick start page, or README.md in the SDK zip.

Contents

  1. Architecture
  2. Video streams
  3. The near-infrared (IR) stream
  4. Coordinate systems
  5. Python API — LogiWebcamClient
  6. Python API — camera_utils
  7. AI output JSON
  8. AI configuration (INI)
  9. YUV snapshot
  10. Examples and limits
  11. Audio (USB microphone)
  12. Troubleshooting

1. Architecture

One USB-C cable carries three kinds of traffic:

Interface Protocol Carries
UVC (USB Video Class) standard webcam drivers on every OS video: the RGB stream and the near-infrared stream
USB NCM virtual Ethernet adapter → SSH/SFTP at 192.168.1.10:22 AI result JSON, AI configuration (INI), optional YUV snapshots
UAC (USB Audio Class) standard audio drivers on every OS the microphone input, 2 channels (§11)

Video and audio need no setup. AI data and configuration need the network link, which needs a manual address on your PC (192.168.1.20/24, no gateway; see the README).

 camera ──USB-C──┬── UVC  ──►  cv2.VideoCapture      (RGB, IR)
                 ├── UAC  ──►  sounddevice           (microphone)
                 └── NCM  ──►  LogiWebcamClient (SSH/SFTP): JSON · INI · YUV

The camera runs face detection and an optional 478-point face mesh on the device. It runs while the RGB camera is open, as an MJPG, YUY2 or NV12 stream (firmware 1.5.2 or newer; firmware 1.5.1 runs it on MJPG only); with no reader the result file simply stops changing.

Files visible over SFTP

The SSH user sees only this sandbox:

Path Content
/files/logi_conf/logiwebcam_ai.ini AI settings (read/write)
/files/logi_conf/logiwebcam_ai_output.json latest AI results
/files/logi_conf/logiwebcam_frame.yuv optional paired NV12 frame
/files/logi_conf/logiwebcam_frame.status one-line result of the last YUV write
/files/LogiApp.log the camera application’s log (read-only use; helpful when something fails)

The SSH user name and password are on the card in your kit. Never put them in code or commit them: export LOGIWEBCAM_USER and LOGIWEBCAM_PASSWORD (or pass --user / --password). python check_connection.py tests the whole link step by step.


2. Video streams

The camera appears to the host as UVC cameras (the name prefix is the product name, LVNext by default):

UVC camera Sensor Formats Use
LVNext Camera RGB MJPG, YUY2, NV12 RGB video and the AI
LVNext CPD Camera near-infrared (monochrome) MJPG only, portrait IR video — example_IR.py
LVNext Hello Camera Windows Hello stream 8-bit, 480×480 only not used by this SDK; camera apps cannot open it, ignore it

The camera is also a USB microphone (LVNext Microphone, §11), so camera apps list it next to the cameras.

OS camera indexes (the number you give to cv2.VideoCapture) are not fixed: they depend on the OS, the backend and what else is plugged in. On Linux the V4L2 card name (/sys/class/video4linux/video*/name) identifies each stream; elsewhere probe the indexes (camera_utils.list_cameras()) or pass --device. Each UVC camera also creates an extra metadata node on Linux that cannot deliver frames; ignore it.

RGB camera — advertised modes (maximum frame rate)

Rates depend on the USB link speed; every mode also offers lower rates down to 5 fps.

Format SuperSpeed (USB 3) High Speed (USB 2) Full Speed
MJPG 640×480@240 · 1280×720@240 · 1920×1080@120 · 3840×2160@60 640×480@60 · 1280×720@60 · 1920×1080@30 640×480@30
YUY2 640×480@240 · 1280×720@120 · 1920×1080@60 · 3840×2160@15 640×480@60 · 1280×720@60 · 1920×1080@15 640×480@15
NV12 640×480@60 · 1280×720@60 · 1920×1080@60 · 3840×2160@30 640×480@60 · 1280×720@60 · 1920×1080@15 640×480@15

These are upper bounds. Actual rates depend on exposure (low light lowers the frame rate), your USB controller and the OS backend. Use a USB 3 port and cable for 1080p and above; raw NV12 and YUY2 at 1080p need USB 3 for a usable frame rate (USB 2 offers 15 fps at most).

Opening the RGB stream correctly

import cv2

cap = cv2.VideoCapture(index, cv2.CAP_DSHOW)      # Windows; CAP_V4L2 Linux; CAP_AVFOUNDATION macOS
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1920)           # 1. size FIRST
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 1080)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*"MJPG"))   # 2. then the format

3. The near-infrared (IR) stream

What it is

The CPD stream (what this SDK uses)

LVNext CPD Camera delivers MJPG, so every OS and OpenCV backend can open it:

Link speed 256×320 512×640 1024×1280
SuperSpeed (USB 3) 240 fps max 240 fps max 120 fps max
High Speed (USB 2) 240 fps max 60 fps max 15 fps max
Full Speed 60 fps max 15 fps max —

256×320 is the default when no size is requested. As for RGB, these are advertised limits. The examples ask for 512×640 at 30 fps.

Reading it

import cv2
from camera_utils import find_camera, open_camera, backend_id

index, backend_name = find_camera("ir")           # by name on Linux, by probing elsewhere
cap, info = open_camera(index, backend_id(backend_name), 512, 640, "MJPG", 30)
ok, frame = cap.read()                            # numpy.ndarray, shape (640, 512, 3), uint8
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)    # single channel (640, 512), uint8
cap.release()

OpenCV decodes the MJPG into a three-channel BGR array whose channels are identical; convert to one channel with cv2.cvtColor(..., COLOR_BGR2GRAY).

Processing in example_IR.py

raw, stretch (1st–99th percentile contrast stretch), clahe (adaptive histogram equalisation), threshold (fixed), otsu (automatic). The functions to_gray, contrast_stretch, clahe_enhance, fixed_threshold, otsu_threshold and process are plain functions on uint8 arrays that you can import.

Good to know


4. Coordinate systems

Space Size Used for
Stream (preview) pixels the resolution of the RGB stream you opened, e.g. 1920×1080 every bbox and every landmark in the AI JSON; the YUV snapshot has this size
IR stream pixels 256×320, 512×640 or 1024×1280 (portrait) the IR image only; no AI data refers to it

Therefore:


5. Python API — LogiWebcamClient

from logiwebcam import LogiWebcamClient, LogiWebcamConnectionError, LogiWebcamDataError, __version__

Requires only paramiko to import. numpy and cv2 are imported lazily, inside get_device_snapshot().

Constructor

LogiWebcamClient(ip="192.168.1.10", username=None, password=None, port=22, timeout=10)

If username or password is None, the LOGIWEBCAM_USER / LOGIWEBCAM_PASSWORD environment variable is used; if either is still missing, ValueError is raised. Both values are on the card in your kit.

Connection

Member Description
connect() Opens SSH. Idempotent. Raises LogiWebcamConnectionError (wrong address/adapter, wrong password, timeout 10 s).
disconnect() Closes SFTP and SSH. Idempotent.
is_connected Property, True while the SSH transport is active.
with LogiWebcamClient() as cam: connect() on entry, disconnect() on exit.

There is no automatic reconnect: after a USB unplug, create a new client.

Configuration

cam.set_ai_config(
    enable_face_detection=True,
    enable_face_mesh=False,        # needs face detection on, else forced off
    enable_debug_timestamp=False,  # kept for compatibility; has no visible effect
    enable_yuv_snapshot=False,     # write a paired NV12 file each cycle
) -> bool

Writes the INI file. The camera re-reads it about twice a second while streaming (not instantly): wait ≈ 0.5–1 s after changing a flag. The returned True means the write command was issued, not that the camera has applied it; check the JSON.

Reading results

Call Returns
get_latest_ai_data() dict (see §7) or None if missing / empty / being rewritten
get_yuv_status() one-line status string of the last snapshot write, or None
get_device_snapshot(retries=3, verbose=False) (bgr_ndarray, data); (None, None) if no JSON; (None, data) if no YUV is being written
diagnose() prints and returns a dict of INI/JSON/YUV probes. Run this first when results stay None
get_firmware_version() first line of the camera’s version text (… hackathon-1.5.2 <variant> <commit>), or None

get_device_snapshot() fetches the JSON and the YUV file, checks that the timestamp in the JSON equals the 8-byte header of the YUV, retries on mismatch and converts NV12 → BGR with OpenCV.

Module helpers: firmware_matches_sdk(text) (True/False/None = unknown) compares the firmware release with the SDK’s major.minor version. AiStalenessMonitor(max_age_s=2.0) has one method, update(data): call it with every result of get_latest_ai_data(); it returns a warning text once when timestamp_ms stopped changing for max_age_s seconds (nobody reads the RGB stream, or on firmware 1.5.1 the stream is not MJPG), otherwise None. The RGB examples use both.

Exceptions: LogiWebcamConnectionError, LogiWebcamDataError, and ValueError for a missing password.


6. Python API — camera_utils

Video-side helpers (no SSH), used by example_IR.py.

Function Purpose
uvc_backends() OpenCV backends per OS: V4L2 (Linux); DSHOW then MSMF (Windows); AVFoundation (macOS)
open_camera(index, backend=None, width=None, height=None, fourcc="MJPG", fps=None) → (cap, info) Opens with size, then fps, then FOURCC set (the order DirectShow needs to keep MJPG); info holds the negotiated width, height, fourcc, fps, backend. Raises CameraNotFoundError
list_cameras(max_index=10, probe_size=None) Probes indexes; list of dicts (index, backend, name (Linux), width, height, fourcc, gray)
find_camera(kind) kind="rgb" or "ir" → (index, backend_name). Linux: by name. Windows/macOS: by probing (portrait or perfectly grey frame ⇒ IR). A laptop’s built-in webcam can be mistaken for the RGB camera: pass --device
backend_id(name) name → OpenCV backend constant
is_grayscale(frame) cheap “this is the IR picture” test
describe(info) one-line description of an open_camera result
install_stop_flag() → StopFlag Ctrl+C / SIGTERM set a flag instead of raising; a second Ctrl+C raises KeyboardInterrupt
camera_hint() OS-specific checklist used in error messages
CameraNotFoundError raised when no usable camera exists

Standard exit pattern used by every script:

stop = install_stop_flag()
cap, info = open_camera(...)
try:
    while not stop.is_set():
        ok, frame = cap.read()
        ...
except KeyboardInterrupt:
    pass
finally:
    cap.release()
    cv2.destroyAllWindows()

7. AI output JSON

File: /files/logi_conf/logiwebcam_ai_output.json, written atomically (temp file then rename) for every processed frame. The AI result rate is lower than the video frame rate, so consecutive video frames can share a result (with RGB alone at 1080p expect about 27 results per second; with the IR stream open next to it about 13–17).

{
  "api_version": 2,
  "plugin_loaded": true,
  "timestamp_ms": 1812345678,
  "face_timestamp_ms": 1812345678,
  "faces": [
    { "bbox": [x, y, w, h], "confidence": 0.97,
      "capture_timestamp_ms": 1812345678,
      "landmarks": [[x, y, z], "... 478 points when the mesh is on"] }
  ]
}

(The timestamp value above is illustrative.)

Field Meaning
api_version JSON schema version, the integer 2; always present, also when plugin_loaded is false
plugin_loaded true while the AI runs. false means empty faces: treat it as “no AI results”, not an error (stream not open, the AI restarted, or firmware 1.5.1 with a stream that is not MJPG)
timestamp_ms One value per processed frame, taken when the frame reaches the AI, before inference. It is the camera’s wall clock in milliseconds (the camera may have no real-time clock, so it can start at an arbitrary date and jump), stored in a 32-bit value, so it wraps. Use it only to compare with other values from the same camera, for example to see that results advance or to pair the YUV file, never as absolute time
face_timestamp_ms, faces[i].capture_timestamp_ms Always equal to timestamp_ms
faces[i].bbox [x, y, width, height], stream pixels (§4)
faces[i].confidence score 0…1
faces[i].landmarks empty unless enable_face_mesh. Even with the mesh on it is empty in some results (it was present in roughly half to most results, less while the head moves), so check its length before use. When present: 478 [x, y, z] in stream pixels (MediaPipe face-mesh topology, including 10 iris points at 468–477). z: relative depth in x/y units
yuv_* present only with the YUV snapshot (§9)

8. AI configuration (INI)

File /files/logi_conf/logiwebcam_ai.ini, section [logiwebcam_ai]. Created with defaults on first start; re-read about every 0.5 s while streaming, so changes apply without restarting the stream.

Key Default Meaning
enable_face_detection 1 face box and score
enable_face_mesh 0 478-point mesh; forced to 0 if face detection is off
enable_debug_timestamp 0 kept for compatibility; the camera no longer draws anything on the video
enable_yuv_snapshot 0 write the NV12 frame next to the JSON (§9)

set_ai_config() writes all four keys at once; keys you do not pass fall back to the defaults shown in its signature (so it turns enable_face_detection on unless you pass False).

To switch the built-in AI off write enable_face_detection=False and enable_face_mesh=False: the camera then runs no AI model; the JSON keeps updating but faces stays empty. This is the only way to switch the AI off: opening the RGB camera as YUY2 or NV12 does not stop it (the AI runs on MJPG, YUY2 and NV12; only firmware 1.5.1 is limited to MJPG).


9. YUV snapshot

Why the live picture and the JSON can differ

The video reaches you over USB (UVC); the AI JSON is a file you fetch over SSH/SFTP. They are two separate paths with no shared frame number, and the AI analyses a frame while newer ones are already streaming. If you draw the latest JSON on the frame you just read, the boxes and landmarks can therefore lag a moving face. This is expected. The camera draws nothing on the video, so every box you see comes from this JSON.

enable_yuv_snapshot=True makes the camera write the exact frame its models saw, so image and results are guaranteed to belong together. That picture is the frame the AI analysed, not necessarily the frame the live video showed at that moment, so the live image and the snapshot of the same instant can look different.


10. Examples and limits

Script What it does Notes
rgb_ai_capture.py Captures N RGB frames, fetches the JSON per frame (or snapshot pairs with --device-snapshot), draws face box and mesh, saves frame_NNNN.png + .json --frames, --interval, --width, --height, --format (MJPG, YUY2 or NV12; default MJPG), --face-mesh, --device-snapshot, --ip, --user, --password, --device
rgb_ai_live.py The same overlays live in a window lists the cameras and asks which to use if --device is missing; --format (MJPG, YUY2 or NV12; default MJPG); --no-overlay shows the raw picture (only an AI faces: N counter)
check_connection.py Checks the SSH/SFTP link step by step (address, Wi-Fi clash, port 22, login, AI results, firmware) and prints the fix for each failure --ip, --user, --password; exit code 1 if a step failed
example_IR.py Near-infrared live view with five processing modes no SSH; --device, --width/--height, --mode, --threshold, --flicker-filter, --frames, --no-display
example_audio.py Records the camera microphone (10 s by default) and saves a multi-channel .wav plus a per-channel power-spectrum .png; prints RMS, peak and strongest frequency per channel (§11) --duration, --device, --samplerate, --channels, --output-dir, --list; no SSH

Every script stops cleanly on Ctrl+C (a second press forces an exit), releases the camera in a finally block and closes all windows. Exit codes: 0 normal, 2 camera (or, for example_audio.py, microphone) not found / not openable, 3 (example_IR.py) the camera stopped delivering frames, 4 (example_audio.py) the audio stream failed, 1 error in rgb_ai_capture.py / rgb_ai_live.py (camera or connection), 130 interrupted with Ctrl+C (rgb_ai_capture.py, rgb_ai_live.py, example_audio.py).

Limits to keep in mind


11. Audio (USB microphone)

The camera has a microphone array. It shows up on your PC as an ordinary USB audio input device, named <product> Microphone (LVNext Microphone by default). No network link, login or configuration is needed: any program that can record from a microphone can use it. This SDK adds a small recorder and a spectrum plot as a starting point.

What the camera delivers

Property Value
USB class USB Audio Class 1, capture only (no speaker output)
Sample format signed 16-bit PCM, little endian
Sample rate 48 kHz (16 kHz is also offered)
Microphones 2, 2 cm apart (in a 6-channel listing: channels 1 and 4, counting from 1)
Processing none: raw microphone samples, with no noise suppression, echo cancellation, automatic gain control or filtering
Level a fixed gain is applied on the camera; your OS may also show an input-volume slider that changes the level

The camera sends 2 microphones, 2 cm apart. The USB description also lists 4- and 6-channel formats, and a PC driver may offer one of those. On Windows the microphone appears once per host API: WASAPI shows 2 channels, while MME, DirectSound and WDM-KS show 6. In a 6-channel recording keep channels 1 and 4 (counting from 1): they are the two microphones, 2 cm apart; ignore the others. A 2-channel entry gives you the two microphones directly (pass --device <index> to pick it). example_audio.py picks the entry with the most channels, records all of them and flags channels that are all zero or identical to another one.

Quick start

python example_audio.py              # 10 s from the camera microphone
python example_audio.py --list       # list the input devices (index, name, host API, channels)
python example_audio.py --duration 5 --device 3 --output-dir mycaptures

It saves audio_<time>.wav (one WAV channel per USB channel, 16-bit, lossless) and spectrum_<time>.png (power spectral density of every channel, in dBFS/Hz from 0 Hz to half the sample rate) into audio_captures/, and prints each channel’s RMS level, peak level and strongest frequency. Ctrl+C stops early and still saves what was recorded.

Python API (audio_utils)

The samples are never mixed, resampled or filtered.

Function / class Purpose
list_audio_devices() input devices: index, name, hostapi, max_input_channels, default_samplerate
find_audio_device(device=None, min_channels=1) the camera microphone (by name, and by USB id on Linux); device = index or name part; raises AudioDeviceNotFoundError listing what exists
record_audio(duration_s=10.0, device=None, samplerate=None, channels=None, stop_event=None) blocking; returns (audio, samplerate, stats), audio is an int16 array of shape (samples, channels); stats has xruns, duration_actual_s, channels, interrupted…
AudioRingRecorder(device_info, samplerate=None, channels=None, seconds=10) non-blocking recorder that always keeps the latest seconds; start(), snapshot(), stats(), stop()
compute_power_spectrum(audio, samplerate, nperseg=1024) (freqs_hz, psd_db), shapes (F,) and (F, channels), Welch method, dBFS/Hz
channel_stats(audio, samplerate) per channel: RMS and peak in dBFS, strongest frequency, silent, duplicate_of
save_power_spectrum_plot(freqs_hz, psd_db, out_path, title=...) PNG, no window needed
save_wav(path, audio, samplerate) multi-channel 16-bit WAV
from audio_utils import compute_power_spectrum, record_audio

audio, fs, stats = record_audio(duration_s=5)      # audio.shape == (5 * fs, channels)
freqs, psd_db = compute_power_spectrum(audio, fs)

dBFS here is relative to an amplitude of 1.0, so a full-scale sine reads -3 dB RMS and 0 dB peak.

Troubleshooting audio

Symptom Action
no camera microphone found plug the camera in; run python example_audio.py --list and pass --device <index>
6 channels but most are silent, or only 2 channels, on Windows the list shows the same microphone once per host API (WASAPI 2 channels, MME / DirectSound / WDM-KS 6 channels); with 6 channels keep channels 1 and 4 (counting from 1) and ignore the rest, or pick the 2-channel entry with --device
All channels are silent or every sample is zero Windows: Settings → Privacy & security → Microphone → allow desktop apps, and check the input is enabled and not muted; macOS: System Settings → Privacy & Security → Microphone → allow your terminal or editor, then restart it
PortAudio library not found Linux: sudo apt install libportaudio2
Linux: the device is busy PulseAudio or PipeWire may hold it; pick its entry in --list, or stop the other program
xruns above 0 close other USB-heavy programs, use a USB 3 port without a hub

12. Troubleshooting

Symptom Likely cause Action
Failed to connect to 192.168.1.10:22 the PC’s USB-Ethernet adapter has no 192.168.1.x address, or the cable is out set 192.168.1.20/24 (README §3); ping 192.168.1.10
Connects, but traffic goes elsewhere your Wi-Fi also uses 192.168.1.0/24 disconnect Wi-Fi for the test, or change the router’s subnet; the camera’s address is fixed
Authentication failed wrong login use the user name and password on your kit card; LOGIWEBCAM_USER, LOGIWEBCAM_PASSWORD
ValueError: SSH user name and password … required not supplied export LOGIWEBCAM_USER and LOGIWEBCAM_PASSWORD, then run python check_connection.py
get_latest_ai_data() returns None no RGB reader running, or AI not started open the RGB stream (MJPG, YUY2 or NV12) and keep reading; run cam.diagnose()
timestamp_ms frozen nobody reads the RGB stream, or the camera firmware is 1.5.1 and the stream is YUY2/NV12 keep a cap.read() loop running; on firmware 1.5.1 request MJPG (size before FOURCC; check cap.get(cv2.CAP_PROP_FOURCC)); on macOS/Windows try another backend
plugin_loaded: false the stream closed, the AI restarted, or firmware 1.5.1 with a stream that is not MJPG (re)open the RGB stream; on firmware 1.5.1 as MJPG
One stream does not start, the picture freezes or the frame rate drops when RGB and IR are open together RGB as MJPG slows the IR stream to about 1 fps; the opening order can matter depending on your system open RGB as NV12 or YUY2; close both streams and open them in the other order (RGB first is the usual choice)
Overlays offset the frame was resized before drawing scale the coordinates by the same factor (§4)
Wrong camera opens built-in webcam is index 0 --device <n>; on Linux check v4l2-ctl --list-devices
IR example: colour or landscape picture wrong camera choose another --device
IR picture flickers lit/unlit frames alternate --flicker-filter
IR example: no camera found cable, permissions, camera busy see the camera_hint() message
errno=28 in the YUV status storage full turn the snapshot off; use 640×480
The video freezes after the AI has run for about 10 minutes or more, or the camera is hot the camera overheated (continuous AI inference draws power and produces heat) unplug it, let it cool down, plug it in again; run experiments in shorter sessions and stop programs you are not using
macOS: no frames / black Camera permission missing for the terminal System Settings → Privacy & Security → Camera
Linux: cannot open /dev/video* not in the video group sudo usermod -aG video $USER, log out and in
PortAudio library not found (Linux) the system library is missing sudo apt install libportaudio2
example_audio.py: no microphone found, or only silence OS microphone permission, wrong input chosen, camera not listed as an audio device see §11 Troubleshooting audio; python example_audio.py --list
ImportError: libGL… (Linux) or a window cannot open headless OpenCV build installed pip uninstall opencv-python-headless; pip install opencv-python