# Rendering Clips From CS2 Demos This document explains what still needs to happen for Fragalytics to render playable video clips directly from uploaded CS2 `.dem` files. ## Current State What already works: - Demo uploads - Background parsing - Highlight scoring - Clip job creation with `start_tick` and `end_tick` - Manual clip upload and in-page playback for `.mp4` and `.webm` What does not exist yet: - Automatic demo-to-video rendering - A separate render worker process - A real CS2 capture backend - Camera automation - Encoding pipeline ## Important Constraint A `.dem` file is not video. It is replay data. To get a playable clip, the system must run Counter-Strike 2, load the demo, seek to the correct tick range, play the segment in-engine, capture the frames and audio, encode the output, and then attach the media file back to the clip job. ## Step-By-Step Plan ### 1. Define the render job contract Goal: Make clip jobs specific enough that an external worker can render them without guessing. Needed work: - Expand the clip job payload to include: - `demoId` - `matchId` - `highlightId` - `startTick` - `endTick` - `cameraMode` - `outputFormat` - `resolution` - `fps` - Decide which fields are required versus optional defaults - Version the payload format so future workers can evolve safely Recommended MVP default: - `cameraMode = killer_pov` - `outputFormat = mp4` - `resolution = 1920x1080` - `fps = 60` ### 2. Split rendering into a dedicated worker Goal: Keep the web app focused on uploads, scoring, storage, and playback while a separate process handles slow rendering work. Needed work: - Create a new render-worker project or service - Make it poll for clip jobs with `status = pending` - Add job state transitions: - `pending` - `rendering` - `rendered` - `failed` - Add failure details to the clip job payload or model - Ensure only one worker claims a job at a time Recommended MVP default: - Single worker process - One render at a time - Polling loop instead of a queue ### 3. Add a render backend abstraction Goal: Avoid hard-coding the first rendering strategy into the worker. Needed work: - Replace the current enqueue-only abstraction with a backend contract such as: - `RenderAsync(clipJob, outputPath, cancellationToken)` - Support backend selection by configuration - Keep output handling generic so different capture approaches can write the final file Recommended backend interface responsibilities: - Prepare commands/scripts - Launch capture - Wait for completion - Return output file metadata ### 4. Choose the real CS2 capture approach Goal: Pick one reliable rendering path instead of trying to support every option. Practical options: - Script CS2 directly with launch arguments and console commands - Use a capture toolchain that drives CS2 and records output - Use FFmpeg only for final encoding or post-processing, not for reading `.dem` files directly Reality: - FFmpeg alone cannot render a `.dem` - The worker needs a machine with CS2 installed - The worker usually needs desktop or GPU access to capture frames reliably Recommended MVP choice: - A dedicated Windows render machine - CS2 installed and configured for automated demo playback - A scripted capture path that outputs a raw or directly encoded video file ### 5. Implement deterministic demo playback control Goal: Make the worker reliably reproduce the requested clip boundaries. Needed work: - Open the correct demo file - Load the replay inside CS2 - Seek near the `startTick` - Stabilize playback before capture starts - Capture until `endTick` - Stop cleanly and close or recycle the game session Risks to handle: - Seeking may not land exactly on the intended frame - HUD and camera state may need warm-up time - Demo playback can behave differently across environments Recommended MVP default: - Add configurable pre-roll before `startTick` - Trim the output afterward if needed ### 6. Choose the first camera mode Goal: Start with one camera behavior that is simple and defensible. Possible camera modes: - `killer_pov` - `victim_pov` - `spec_follow` - `static_observer` - `freecam_cinematic` Recommended MVP default: - `killer_pov` Why: - It aligns directly with scored kill highlights - It is easier to reason about than cinematic camera logic - It avoids building a director system too early ### 7. Capture or encode audio and video Goal: Produce a playable media file in a standard format. Needed work: - Decide whether capture outputs: - directly encoded `.mp4` - intermediate lossless file plus final encode - Set consistent output settings: - codec - bitrate or CRF - resolution - framerate - Decide how audio will be handled: - game audio on - game audio off - no voice - no music Recommended MVP default: - H.264 `.mp4` - 1080p - 60 fps - gameplay audio only ### 8. Attach the rendered file back to Fragalytics Goal: Reuse the playback flow that already exists in the web app. What already exists: - Clip asset storage - Clip-job-to-media attachment - In-page HTML5 playback Needed work: - Let the render worker call the same clip-asset attach path used by manual uploads - Mark the clip job as `rendered` - Store file size, content type, and asset path Recommended implementation: - Expose an internal service or API endpoint that accepts: - `clipJobId` - rendered file stream - file name - content type ### 9. Add retries, timeouts, and failure logging Goal: Prevent broken jobs from blocking the worker forever. Needed work: - Add a render timeout per job - Record failure reason - Limit retries - Preserve logs for: - launched commands - process exit codes - capture duration - output path Recommended MVP default: - One retry - Clear failure message on the clip job - Log files per render attempt ### 10. Add machine-level prerequisites Goal: Document what the render machine must have installed and configured. Needed environment: - Counter-Strike 2 installed - Steam installed and logged in if required by the playback path - GPU drivers - Capture and encoding tooling - Enough disk space for temporary video files - A stable location for demo files and rendered output Recommended operations setup: - Separate render host from the web host - Shared storage or an explicit file transfer step - One dedicated account for rendering ## Suggested MVP Milestones ### Milestone 1: Worker skeleton Deliverables: - Render worker project - Pending job polling - Status transitions - Stub backend that writes logs but no video ### Milestone 2: Automatic file attachment Deliverables: - Worker can attach a pre-rendered local file to a clip job - No CS2 automation yet - End-to-end job completion path verified ### Milestone 3: First real CS2 render Deliverables: - One camera mode - One machine - One output format - One clip rendered automatically from a real `.dem` ### Milestone 4: Production hardening Deliverables: - Retry logic - Timeouts - Better logging - Configurable render presets - Concurrency controls ## Recommended First Implementation Order If you want this built pragmatically, the order should be: 1. Add a dedicated render worker project. 2. Expand the clip job payload to fully describe a render request. 3. Add an internal attachment API or shared service so the worker can save finished media. 4. Implement a stub backend that completes jobs using a known local video file. 5. Replace the stub backend with a real CS2 capture backend on one dedicated machine. 6. Add retries, logs, and operational safeguards after the first real render works. ## What Not To Do First Avoid these early mistakes: - Do not start with multiple camera modes. - Do not start with distributed workers. - Do not treat FFmpeg as if it can render demos directly. - Do not overbuild orchestration before one clip can render successfully end to end. ## Definition Of Done For Real Rendering This feature is truly working when: - A user uploads a `.dem` - The app creates highlight clip jobs - The render worker picks a pending job automatically - CS2 playback is driven to the requested tick range - A video file is produced without manual intervention - The file is attached back to the job - The clip becomes playable in the Fragalytics UI