fragalytics/docs/rendering-from-cs2-demo.md

Rendering Clips From CS2 Demos

This document explains what still needs to happen for Fragalytics to render playable video clips directly from uploaded CS2 .dem files.

Current State

What already works:

• Demo uploads
• Background parsing
• Highlight scoring
• Clip job creation with start_tick and end_tick
• Manual clip upload and in-page playback for .mp4 and .webm
• Render worker scaffold with a cs2-scripted backend that resolves demos, creates per-attempt scripts/logs, and invokes a Windows wrapper script

What does not exist yet:

• A separate render worker process
• Camera automation
• Encoding pipeline

What exists now for the first automation path:

• Worker backend: src/Fragalytics.RenderWorker/Services/Cs2ScriptedRenderBackend.cs
• Wrapper script entrypoint: src/Fragalytics.RenderWorker/scripts/render-cs2-clip.ps1
• Worker config: src/Fragalytics.RenderWorker/appsettings.json

Current behavior of that path:

• It resolves the source .dem from the clip job payload
• It creates per-attempt artifacts under data/render-artifacts
• It writes a launch script and log for each render attempt
• It invokes the configured wrapper script with resolved arguments
• It only marks success if the configured command produces the expected output file

This means the next implementation work is focused in one place:

• edit render-cs2-clip.ps1 to launch CS2, drive demo playback, capture the clip, and write the output file

Important Constraint

A .dem file is not video. It is replay data. To get a playable clip, the system must run Counter-Strike 2, load the demo, seek to the correct tick range, play the segment in-engine, capture the frames and audio, encode the output, and then attach the media file back to the clip job.

Step-By-Step Plan

1. Define the render job contract

Goal: Make clip jobs specific enough that an external worker can render them without guessing.

Needed work:

• Expand the clip job payload to include:
  • demoId
  • matchId
  • highlightId
  • startTick
  • endTick
  • cameraMode
  • outputFormat
  • resolution
  • fps
• Decide which fields are required versus optional defaults
• Version the payload format so future workers can evolve safely

Recommended MVP default:

• cameraMode = killer_pov
• outputFormat = mp4
• resolution = 1920x1080
• fps = 60

2. Split rendering into a dedicated worker

Goal: Keep the web app focused on uploads, scoring, storage, and playback while a separate process handles slow rendering work.

Needed work:

• Create a new render-worker project or service
• Make it poll for clip jobs with status = pending
• Add job state transitions:
  • pending
  • rendering
  • rendered
  • failed
• Add failure details to the clip job payload or model
• Ensure only one worker claims a job at a time

Recommended MVP default:

• Single worker process
• One render at a time
• Polling loop instead of a queue

3. Add a render backend abstraction

Goal: Avoid hard-coding the first rendering strategy into the worker.

Needed work:

• Replace the current enqueue-only abstraction with a backend contract such as:
  • RenderAsync(clipJob, outputPath, cancellationToken)
• Support backend selection by configuration
• Keep output handling generic so different capture approaches can write the final file

Recommended backend interface responsibilities:

• Prepare commands/scripts
• Launch capture
• Wait for completion
• Return output file metadata

4. Choose the real CS2 capture approach

Goal: Pick one reliable rendering path instead of trying to support every option.

Practical options:

• Script CS2 directly with launch arguments and console commands
• Use a capture toolchain that drives CS2 and records output
• Use FFmpeg only for final encoding or post-processing, not for reading .dem files directly

Reality:

• FFmpeg alone cannot render a .dem
• The worker needs a machine with CS2 installed
• The worker usually needs desktop or GPU access to capture frames reliably

Recommended MVP choice:

• A dedicated Windows render machine
• CS2 installed and configured for automated demo playback
• A scripted capture path that outputs a raw or directly encoded video file

5. Implement deterministic demo playback control

Goal: Make the worker reliably reproduce the requested clip boundaries.

Needed work:

• Open the correct demo file
• Load the replay inside CS2
• Seek near the startTick
• Stabilize playback before capture starts
• Capture until endTick
• Stop cleanly and close or recycle the game session

Risks to handle:

• Seeking may not land exactly on the intended frame
• HUD and camera state may need warm-up time
• Demo playback can behave differently across environments

Recommended MVP default:

• Add configurable pre-roll before startTick
• Trim the output afterward if needed

6. Choose the first camera mode

Goal: Start with one camera behavior that is simple and defensible.

Possible camera modes:

• killer_pov
• victim_pov
• spec_follow
• static_observer
• freecam_cinematic

Recommended MVP default:

• killer_pov

Why:

• It aligns directly with scored kill highlights
• It is easier to reason about than cinematic camera logic
• It avoids building a director system too early

7. Capture or encode audio and video

Goal: Produce a playable media file in a standard format.

Needed work:

• Decide whether capture outputs:
  • directly encoded .mp4
  • intermediate lossless file plus final encode
• Set consistent output settings:
  • codec
  • bitrate or CRF
  • resolution
  • framerate
• Decide how audio will be handled:
  • game audio on
  • game audio off
  • no voice
  • no music

Recommended MVP default:

• H.264 .mp4
• 1080p
• 60 fps
• gameplay audio only

8. Attach the rendered file back to Fragalytics

Goal: Reuse the playback flow that already exists in the web app.

What already exists:

• Clip asset storage
• Clip-job-to-media attachment
• In-page HTML5 playback

Needed work:

• Let the render worker call the same clip-asset attach path used by manual uploads
• Mark the clip job as rendered
• Store file size, content type, and asset path

Recommended implementation:

• Expose an internal service or API endpoint that accepts:
  • clipJobId
  • rendered file stream
  • file name
  • content type

9. Add retries, timeouts, and failure logging

Goal: Prevent broken jobs from blocking the worker forever.

Needed work:

• Add a render timeout per job
• Record failure reason
• Limit retries
• Preserve logs for:
  • launched commands
  • process exit codes
  • capture duration
  • output path

Recommended MVP default:

• One retry
• Clear failure message on the clip job
• Log files per render attempt

10. Add machine-level prerequisites

Goal: Document what the render machine must have installed and configured.

Needed environment:

• Counter-Strike 2 installed
• Steam installed and logged in if required by the playback path
• GPU drivers
• Capture and encoding tooling
• Enough disk space for temporary video files
• A stable location for demo files and rendered output

Recommended operations setup:

• Separate render host from the web host
• Shared storage or an explicit file transfer step
• One dedicated account for rendering

Suggested MVP Milestones

Milestone 1: Worker skeleton

Deliverables:

• Render worker project
• Pending job polling
• Status transitions
• Stub backend that writes logs but no video

Milestone 2: Automatic file attachment

Deliverables:

• Worker can attach a pre-rendered local file to a clip job
• No CS2 automation yet
• End-to-end job completion path verified

Milestone 3: First real CS2 render

Deliverables:

• One camera mode
• One machine
• One output format
• One clip rendered automatically from a real .dem

Milestone 4: Production hardening

Deliverables:

• Retry logic
• Timeouts
• Better logging
• Configurable render presets
• Concurrency controls

Recommended First Implementation Order

If you want this built pragmatically, the order should be:

1. Add a dedicated render worker project.
2. Expand the clip job payload to fully describe a render request.
3. Add an internal attachment API or shared service so the worker can save finished media.
4. Implement a stub backend that completes jobs using a known local video file.
5. Replace the stub backend with a real CS2 capture backend on one dedicated machine.
6. Add retries, logs, and operational safeguards after the first real render works.

What Not To Do First

Avoid these early mistakes:

• Do not start with multiple camera modes.
• Do not start with distributed workers.
• Do not treat FFmpeg as if it can render demos directly.
• Do not overbuild orchestration before one clip can render successfully end to end.

Definition Of Done For Real Rendering

This feature is truly working when:

• A user uploads a .dem
• The app creates highlight clip jobs
• The render worker picks a pending job automatically
• CS2 playback is driven to the requested tick range
• A video file is produced without manual intervention
• The file is attached back to the job
• The clip becomes playable in the Fragalytics UI