350 lines
9.0 KiB
Markdown
350 lines
9.0 KiB
Markdown
# Rendering Clips From CS2 Demos
|
|
|
|
This document explains what still needs to happen for Fragalytics to render playable video clips directly from uploaded CS2 `.dem` files.
|
|
|
|
## Current State
|
|
|
|
What already works:
|
|
|
|
- Demo uploads
|
|
- Background parsing
|
|
- Highlight scoring
|
|
- Clip job creation with `start_tick` and `end_tick`
|
|
- Manual clip upload and in-page playback for `.mp4` and `.webm`
|
|
- Render worker scaffold with a `cs2-scripted` backend that resolves demos, creates per-attempt scripts/logs, and invokes a Windows wrapper script
|
|
|
|
What does not exist yet:
|
|
|
|
- A separate render worker process
|
|
- Camera automation
|
|
- Encoding pipeline
|
|
|
|
What exists now for the first automation path:
|
|
|
|
- Worker backend: `src/Fragalytics.RenderWorker/Services/Cs2ScriptedRenderBackend.cs`
|
|
- Wrapper script entrypoint: `src/Fragalytics.RenderWorker/scripts/render-cs2-clip.ps1`
|
|
- Worker config: `src/Fragalytics.RenderWorker/appsettings.json`
|
|
|
|
Current behavior of that path:
|
|
|
|
- It resolves the source `.dem` from the clip job payload
|
|
- It creates per-attempt artifacts under `data/render-artifacts`
|
|
- It writes a launch script and log for each render attempt
|
|
- It invokes the configured wrapper script with resolved arguments
|
|
- It only marks success if the configured command produces the expected output file
|
|
|
|
This means the next implementation work is focused in one place:
|
|
|
|
- edit `render-cs2-clip.ps1` to launch CS2, drive demo playback, capture the clip, and write the output file
|
|
|
|
## Important Constraint
|
|
|
|
A `.dem` file is not video. It is replay data. To get a playable clip, the system must run Counter-Strike 2, load the demo, seek to the correct tick range, play the segment in-engine, capture the frames and audio, encode the output, and then attach the media file back to the clip job.
|
|
|
|
## Step-By-Step Plan
|
|
|
|
### 1. Define the render job contract
|
|
|
|
Goal:
|
|
Make clip jobs specific enough that an external worker can render them without guessing.
|
|
|
|
Needed work:
|
|
|
|
- Expand the clip job payload to include:
|
|
- `demoId`
|
|
- `matchId`
|
|
- `highlightId`
|
|
- `startTick`
|
|
- `endTick`
|
|
- `cameraMode`
|
|
- `outputFormat`
|
|
- `resolution`
|
|
- `fps`
|
|
- Decide which fields are required versus optional defaults
|
|
- Version the payload format so future workers can evolve safely
|
|
|
|
Recommended MVP default:
|
|
|
|
- `cameraMode = killer_pov`
|
|
- `outputFormat = mp4`
|
|
- `resolution = 1920x1080`
|
|
- `fps = 60`
|
|
|
|
### 2. Split rendering into a dedicated worker
|
|
|
|
Goal:
|
|
Keep the web app focused on uploads, scoring, storage, and playback while a separate process handles slow rendering work.
|
|
|
|
Needed work:
|
|
|
|
- Create a new render-worker project or service
|
|
- Make it poll for clip jobs with `status = pending`
|
|
- Add job state transitions:
|
|
- `pending`
|
|
- `rendering`
|
|
- `rendered`
|
|
- `failed`
|
|
- Add failure details to the clip job payload or model
|
|
- Ensure only one worker claims a job at a time
|
|
|
|
Recommended MVP default:
|
|
|
|
- Single worker process
|
|
- One render at a time
|
|
- Polling loop instead of a queue
|
|
|
|
### 3. Add a render backend abstraction
|
|
|
|
Goal:
|
|
Avoid hard-coding the first rendering strategy into the worker.
|
|
|
|
Needed work:
|
|
|
|
- Replace the current enqueue-only abstraction with a backend contract such as:
|
|
- `RenderAsync(clipJob, outputPath, cancellationToken)`
|
|
- Support backend selection by configuration
|
|
- Keep output handling generic so different capture approaches can write the final file
|
|
|
|
Recommended backend interface responsibilities:
|
|
|
|
- Prepare commands/scripts
|
|
- Launch capture
|
|
- Wait for completion
|
|
- Return output file metadata
|
|
|
|
### 4. Choose the real CS2 capture approach
|
|
|
|
Goal:
|
|
Pick one reliable rendering path instead of trying to support every option.
|
|
|
|
Practical options:
|
|
|
|
- Script CS2 directly with launch arguments and console commands
|
|
- Use a capture toolchain that drives CS2 and records output
|
|
- Use FFmpeg only for final encoding or post-processing, not for reading `.dem` files directly
|
|
|
|
Reality:
|
|
|
|
- FFmpeg alone cannot render a `.dem`
|
|
- The worker needs a machine with CS2 installed
|
|
- The worker usually needs desktop or GPU access to capture frames reliably
|
|
|
|
Recommended MVP choice:
|
|
|
|
- A dedicated Windows render machine
|
|
- CS2 installed and configured for automated demo playback
|
|
- A scripted capture path that outputs a raw or directly encoded video file
|
|
|
|
### 5. Implement deterministic demo playback control
|
|
|
|
Goal:
|
|
Make the worker reliably reproduce the requested clip boundaries.
|
|
|
|
Needed work:
|
|
|
|
- Open the correct demo file
|
|
- Load the replay inside CS2
|
|
- Seek near the `startTick`
|
|
- Stabilize playback before capture starts
|
|
- Capture until `endTick`
|
|
- Stop cleanly and close or recycle the game session
|
|
|
|
Risks to handle:
|
|
|
|
- Seeking may not land exactly on the intended frame
|
|
- HUD and camera state may need warm-up time
|
|
- Demo playback can behave differently across environments
|
|
|
|
Recommended MVP default:
|
|
|
|
- Add configurable pre-roll before `startTick`
|
|
- Trim the output afterward if needed
|
|
|
|
### 6. Choose the first camera mode
|
|
|
|
Goal:
|
|
Start with one camera behavior that is simple and defensible.
|
|
|
|
Possible camera modes:
|
|
|
|
- `killer_pov`
|
|
- `victim_pov`
|
|
- `spec_follow`
|
|
- `static_observer`
|
|
- `freecam_cinematic`
|
|
|
|
Recommended MVP default:
|
|
|
|
- `killer_pov`
|
|
|
|
Why:
|
|
|
|
- It aligns directly with scored kill highlights
|
|
- It is easier to reason about than cinematic camera logic
|
|
- It avoids building a director system too early
|
|
|
|
### 7. Capture or encode audio and video
|
|
|
|
Goal:
|
|
Produce a playable media file in a standard format.
|
|
|
|
Needed work:
|
|
|
|
- Decide whether capture outputs:
|
|
- directly encoded `.mp4`
|
|
- intermediate lossless file plus final encode
|
|
- Set consistent output settings:
|
|
- codec
|
|
- bitrate or CRF
|
|
- resolution
|
|
- framerate
|
|
- Decide how audio will be handled:
|
|
- game audio on
|
|
- game audio off
|
|
- no voice
|
|
- no music
|
|
|
|
Recommended MVP default:
|
|
|
|
- H.264 `.mp4`
|
|
- 1080p
|
|
- 60 fps
|
|
- gameplay audio only
|
|
|
|
### 8. Attach the rendered file back to Fragalytics
|
|
|
|
Goal:
|
|
Reuse the playback flow that already exists in the web app.
|
|
|
|
What already exists:
|
|
|
|
- Clip asset storage
|
|
- Clip-job-to-media attachment
|
|
- In-page HTML5 playback
|
|
|
|
Needed work:
|
|
|
|
- Let the render worker call the same clip-asset attach path used by manual uploads
|
|
- Mark the clip job as `rendered`
|
|
- Store file size, content type, and asset path
|
|
|
|
Recommended implementation:
|
|
|
|
- Expose an internal service or API endpoint that accepts:
|
|
- `clipJobId`
|
|
- rendered file stream
|
|
- file name
|
|
- content type
|
|
|
|
### 9. Add retries, timeouts, and failure logging
|
|
|
|
Goal:
|
|
Prevent broken jobs from blocking the worker forever.
|
|
|
|
Needed work:
|
|
|
|
- Add a render timeout per job
|
|
- Record failure reason
|
|
- Limit retries
|
|
- Preserve logs for:
|
|
- launched commands
|
|
- process exit codes
|
|
- capture duration
|
|
- output path
|
|
|
|
Recommended MVP default:
|
|
|
|
- One retry
|
|
- Clear failure message on the clip job
|
|
- Log files per render attempt
|
|
|
|
### 10. Add machine-level prerequisites
|
|
|
|
Goal:
|
|
Document what the render machine must have installed and configured.
|
|
|
|
Needed environment:
|
|
|
|
- Counter-Strike 2 installed
|
|
- Steam installed and logged in if required by the playback path
|
|
- GPU drivers
|
|
- Capture and encoding tooling
|
|
- Enough disk space for temporary video files
|
|
- A stable location for demo files and rendered output
|
|
|
|
Recommended operations setup:
|
|
|
|
- Separate render host from the web host
|
|
- Shared storage or an explicit file transfer step
|
|
- One dedicated account for rendering
|
|
|
|
## Suggested MVP Milestones
|
|
|
|
### Milestone 1: Worker skeleton
|
|
|
|
Deliverables:
|
|
|
|
- Render worker project
|
|
- Pending job polling
|
|
- Status transitions
|
|
- Stub backend that writes logs but no video
|
|
|
|
### Milestone 2: Automatic file attachment
|
|
|
|
Deliverables:
|
|
|
|
- Worker can attach a pre-rendered local file to a clip job
|
|
- No CS2 automation yet
|
|
- End-to-end job completion path verified
|
|
|
|
### Milestone 3: First real CS2 render
|
|
|
|
Deliverables:
|
|
|
|
- One camera mode
|
|
- One machine
|
|
- One output format
|
|
- One clip rendered automatically from a real `.dem`
|
|
|
|
### Milestone 4: Production hardening
|
|
|
|
Deliverables:
|
|
|
|
- Retry logic
|
|
- Timeouts
|
|
- Better logging
|
|
- Configurable render presets
|
|
- Concurrency controls
|
|
|
|
## Recommended First Implementation Order
|
|
|
|
If you want this built pragmatically, the order should be:
|
|
|
|
1. Add a dedicated render worker project.
|
|
2. Expand the clip job payload to fully describe a render request.
|
|
3. Add an internal attachment API or shared service so the worker can save finished media.
|
|
4. Implement a stub backend that completes jobs using a known local video file.
|
|
5. Replace the stub backend with a real CS2 capture backend on one dedicated machine.
|
|
6. Add retries, logs, and operational safeguards after the first real render works.
|
|
|
|
## What Not To Do First
|
|
|
|
Avoid these early mistakes:
|
|
|
|
- Do not start with multiple camera modes.
|
|
- Do not start with distributed workers.
|
|
- Do not treat FFmpeg as if it can render demos directly.
|
|
- Do not overbuild orchestration before one clip can render successfully end to end.
|
|
|
|
## Definition Of Done For Real Rendering
|
|
|
|
This feature is truly working when:
|
|
|
|
- A user uploads a `.dem`
|
|
- The app creates highlight clip jobs
|
|
- The render worker picks a pending job automatically
|
|
- CS2 playback is driven to the requested tick range
|
|
- A video file is produced without manual intervention
|
|
- The file is attached back to the job
|
|
- The clip becomes playable in the Fragalytics UI
|