How I bundled FLUX.1 + XTTS-v2 + Kokoro + llama.cpp into one Electron app that runs on a USB drive
After several months of building, Pocket Core AI ships with four local AI inference pipelines in a single desktop app. Here's how it's actually structured for anyone attempting something similar.
THE ARCHITECTURE PROBLEM
The core challenge: four completely different AI frameworks need to coexist in one app and run across Windows, macOS, and Linux without requiring the user to install Python, CUDA, or any other dependency.
Most guides cover one model or one framework. Nobody writes about shipping four together.
THE SOLUTION: ELECTRON + PYINSTALLER
The app is split into two processes:
Electron shell (Node.js)
Handles the UI (HTML/CSS/JS)
Spawns the Python backend as a child process
Communicates via HTTP to localhost
Python FastAPI backend
Bundled via PyInstaller into a single executable
Exposes REST endpoints for all AI operations
Manages model loading and inference
The PyInstaller build includes:
llama.cpp Python bindings (chat inference)
diffusers + torch (FLUX.1 image generation)
ONNX runtime (Kokoro TTS)
XTTS-v2 dependencies (voice cloning)
FastAPI + uvicorn (the server itself)
newspaper3k, beautifulsoup4 (web content)
All hidden imports specified in .spec file
The resulting backend executable is large (1.5-2.5GB depending on platform) but the user sees none of this complexity.
THE USB GHOST MODE IMPLEMENTATION
When the app detects it's running from removable media, it redirects all data paths:
Windows detection: import wmi c = wmi.WMI() for disk in c.Win32_LogicalDisk(): if disk.DeviceID == drive_letter: is_removable = (disk.DriveType == 2)
macOS detection: diskutil info [mount_point] | grep "Removable Media"
Linux detection: Parse /proc/mounts, check /sys/block/[device]/removable
Once detected, all data paths redirect to the USB drive. The host filesystem is never touched. Verified with Process Monitor — zero file operations on host drive during a full session.
THE CROSS-PLATFORM BUILD PROBLEM
PyInstaller cannot cross-compile. A Windows PyInstaller build produces a Windows binary. A Linux build produces a Linux binary.
Solution: GitHub Actions matrix with three jobs:
windows-latest runner builds the Windows installer
macos-latest runner builds the macOS DMG
ubuntu-latest runner builds the Linux AppImage
Each runner builds the PyInstaller backend for its platform, then packages it with electron-builder.
Challenge we hit: the ubuntu-latest runner has ~14GB free disk space. Our ML stack (torch, diffusers, TTS, etc.) plus PyInstaller output fills it.
Fix: add a disk cleanup step at the start of the Linux job:
sudo rm -rf /usr/share/dotnet sudo rm -rf /usr/local/lib/android sudo rm -rf /opt/ghc sudo apt-get clean docker system prune -af
This frees ~20GB and gives enough headroom.
THE MODEL BUNDLING STRATEGY
Not all models bundle the same way:
Kokoro TTS (~300MB): bundled with the app. Fast to download, small enough to ship.
FLUX.1-schnell (~8GB quantised): downloaded on first use via huggingface_hub.snapshot_download() with a progress UI. Too large to bundle.
XTTS-v2 (~1.8GB): downloaded on first use of voice cloning.
llama.cpp models (4-20GB): user selects on first launch. We ship a model download manager that fetches the appropriate GGUF file from Hugging Face based on the user's available RAM.
DEPENDENCY CONFLICTS WE HIT
The most painful: misaki version conflict.
requirements.txt had misaki==0.7.4 kokoro 0.7.16 requires misaki>=0.7.16
pip resolution fails silently in some environments and loudly in CI. Fix: update misaki to >=0.7.16 and pin kokoro to the minimum compatible version.
WHAT I WISH I KNEW EARLIER
Test PyInstaller on a completely clean VM before shipping. Hidden imports that work in your dev environment silently fail in a bundled executable.
XTTS-v2 has a memory leak on long inference sessions. Reinitialise the model every 50 generations as a workaround.
GPU detection should happen at startup, not at model load time. Users want to know immediately if their GPU will be used.
The Kokoro ONNX runtime and PyTorch (used by FLUX and XTTS) have conflicting CUDA library requirements on some systems. Run them in the same process but initialise ONNX before PyTorch.
Happy to answer technical questions about any of these. The implementation was messy in places — I'm sharing the real version, not a cleaned-up retrospective.
