first commit
This commit is contained in:
Zeeshaun
@@ -0,0 +1,332 @@
|
||||
# Rendering Clips From CS2 Demos
|
||||
|
||||
This document explains what still needs to happen for Fragalytics to render playable video clips directly from uploaded CS2 `.dem` files.
|
||||
|
||||
## Current State
|
||||
|
||||
What already works:
|
||||
|
||||
- Demo uploads
|
||||
- Background parsing
|
||||
- Highlight scoring
|
||||
- Clip job creation with `start_tick` and `end_tick`
|
||||
- Manual clip upload and in-page playback for `.mp4` and `.webm`
|
||||
|
||||
What does not exist yet:
|
||||
|
||||
- Automatic demo-to-video rendering
|
||||
- A separate render worker process
|
||||
- A real CS2 capture backend
|
||||
- Camera automation
|
||||
- Encoding pipeline
|
||||
|
||||
## Important Constraint
|
||||
|
||||
A `.dem` file is not video. It is replay data. To get a playable clip, the system must run Counter-Strike 2, load the demo, seek to the correct tick range, play the segment in-engine, capture the frames and audio, encode the output, and then attach the media file back to the clip job.
|
||||
|
||||
## Step-By-Step Plan
|
||||
|
||||
### 1. Define the render job contract
|
||||
|
||||
Goal:
|
||||
Make clip jobs specific enough that an external worker can render them without guessing.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Expand the clip job payload to include:
|
||||
- `demoId`
|
||||
- `matchId`
|
||||
- `highlightId`
|
||||
- `startTick`
|
||||
- `endTick`
|
||||
- `cameraMode`
|
||||
- `outputFormat`
|
||||
- `resolution`
|
||||
- `fps`
|
||||
- Decide which fields are required versus optional defaults
|
||||
- Version the payload format so future workers can evolve safely
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- `cameraMode = killer_pov`
|
||||
- `outputFormat = mp4`
|
||||
- `resolution = 1920x1080`
|
||||
- `fps = 60`
|
||||
|
||||
### 2. Split rendering into a dedicated worker
|
||||
|
||||
Goal:
|
||||
Keep the web app focused on uploads, scoring, storage, and playback while a separate process handles slow rendering work.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Create a new render-worker project or service
|
||||
- Make it poll for clip jobs with `status = pending`
|
||||
- Add job state transitions:
|
||||
- `pending`
|
||||
- `rendering`
|
||||
- `rendered`
|
||||
- `failed`
|
||||
- Add failure details to the clip job payload or model
|
||||
- Ensure only one worker claims a job at a time
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- Single worker process
|
||||
- One render at a time
|
||||
- Polling loop instead of a queue
|
||||
|
||||
### 3. Add a render backend abstraction
|
||||
|
||||
Goal:
|
||||
Avoid hard-coding the first rendering strategy into the worker.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Replace the current enqueue-only abstraction with a backend contract such as:
|
||||
- `RenderAsync(clipJob, outputPath, cancellationToken)`
|
||||
- Support backend selection by configuration
|
||||
- Keep output handling generic so different capture approaches can write the final file
|
||||
|
||||
Recommended backend interface responsibilities:
|
||||
|
||||
- Prepare commands/scripts
|
||||
- Launch capture
|
||||
- Wait for completion
|
||||
- Return output file metadata
|
||||
|
||||
### 4. Choose the real CS2 capture approach
|
||||
|
||||
Goal:
|
||||
Pick one reliable rendering path instead of trying to support every option.
|
||||
|
||||
Practical options:
|
||||
|
||||
- Script CS2 directly with launch arguments and console commands
|
||||
- Use a capture toolchain that drives CS2 and records output
|
||||
- Use FFmpeg only for final encoding or post-processing, not for reading `.dem` files directly
|
||||
|
||||
Reality:
|
||||
|
||||
- FFmpeg alone cannot render a `.dem`
|
||||
- The worker needs a machine with CS2 installed
|
||||
- The worker usually needs desktop or GPU access to capture frames reliably
|
||||
|
||||
Recommended MVP choice:
|
||||
|
||||
- A dedicated Windows render machine
|
||||
- CS2 installed and configured for automated demo playback
|
||||
- A scripted capture path that outputs a raw or directly encoded video file
|
||||
|
||||
### 5. Implement deterministic demo playback control
|
||||
|
||||
Goal:
|
||||
Make the worker reliably reproduce the requested clip boundaries.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Open the correct demo file
|
||||
- Load the replay inside CS2
|
||||
- Seek near the `startTick`
|
||||
- Stabilize playback before capture starts
|
||||
- Capture until `endTick`
|
||||
- Stop cleanly and close or recycle the game session
|
||||
|
||||
Risks to handle:
|
||||
|
||||
- Seeking may not land exactly on the intended frame
|
||||
- HUD and camera state may need warm-up time
|
||||
- Demo playback can behave differently across environments
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- Add configurable pre-roll before `startTick`
|
||||
- Trim the output afterward if needed
|
||||
|
||||
### 6. Choose the first camera mode
|
||||
|
||||
Goal:
|
||||
Start with one camera behavior that is simple and defensible.
|
||||
|
||||
Possible camera modes:
|
||||
|
||||
- `killer_pov`
|
||||
- `victim_pov`
|
||||
- `spec_follow`
|
||||
- `static_observer`
|
||||
- `freecam_cinematic`
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- `killer_pov`
|
||||
|
||||
Why:
|
||||
|
||||
- It aligns directly with scored kill highlights
|
||||
- It is easier to reason about than cinematic camera logic
|
||||
- It avoids building a director system too early
|
||||
|
||||
### 7. Capture or encode audio and video
|
||||
|
||||
Goal:
|
||||
Produce a playable media file in a standard format.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Decide whether capture outputs:
|
||||
- directly encoded `.mp4`
|
||||
- intermediate lossless file plus final encode
|
||||
- Set consistent output settings:
|
||||
- codec
|
||||
- bitrate or CRF
|
||||
- resolution
|
||||
- framerate
|
||||
- Decide how audio will be handled:
|
||||
- game audio on
|
||||
- game audio off
|
||||
- no voice
|
||||
- no music
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- H.264 `.mp4`
|
||||
- 1080p
|
||||
- 60 fps
|
||||
- gameplay audio only
|
||||
|
||||
### 8. Attach the rendered file back to Fragalytics
|
||||
|
||||
Goal:
|
||||
Reuse the playback flow that already exists in the web app.
|
||||
|
||||
What already exists:
|
||||
|
||||
- Clip asset storage
|
||||
- Clip-job-to-media attachment
|
||||
- In-page HTML5 playback
|
||||
|
||||
Needed work:
|
||||
|
||||
- Let the render worker call the same clip-asset attach path used by manual uploads
|
||||
- Mark the clip job as `rendered`
|
||||
- Store file size, content type, and asset path
|
||||
|
||||
Recommended implementation:
|
||||
|
||||
- Expose an internal service or API endpoint that accepts:
|
||||
- `clipJobId`
|
||||
- rendered file stream
|
||||
- file name
|
||||
- content type
|
||||
|
||||
### 9. Add retries, timeouts, and failure logging
|
||||
|
||||
Goal:
|
||||
Prevent broken jobs from blocking the worker forever.
|
||||
|
||||
Needed work:
|
||||
|
||||
- Add a render timeout per job
|
||||
- Record failure reason
|
||||
- Limit retries
|
||||
- Preserve logs for:
|
||||
- launched commands
|
||||
- process exit codes
|
||||
- capture duration
|
||||
- output path
|
||||
|
||||
Recommended MVP default:
|
||||
|
||||
- One retry
|
||||
- Clear failure message on the clip job
|
||||
- Log files per render attempt
|
||||
|
||||
### 10. Add machine-level prerequisites
|
||||
|
||||
Goal:
|
||||
Document what the render machine must have installed and configured.
|
||||
|
||||
Needed environment:
|
||||
|
||||
- Counter-Strike 2 installed
|
||||
- Steam installed and logged in if required by the playback path
|
||||
- GPU drivers
|
||||
- Capture and encoding tooling
|
||||
- Enough disk space for temporary video files
|
||||
- A stable location for demo files and rendered output
|
||||
|
||||
Recommended operations setup:
|
||||
|
||||
- Separate render host from the web host
|
||||
- Shared storage or an explicit file transfer step
|
||||
- One dedicated account for rendering
|
||||
|
||||
## Suggested MVP Milestones
|
||||
|
||||
### Milestone 1: Worker skeleton
|
||||
|
||||
Deliverables:
|
||||
|
||||
- Render worker project
|
||||
- Pending job polling
|
||||
- Status transitions
|
||||
- Stub backend that writes logs but no video
|
||||
|
||||
### Milestone 2: Automatic file attachment
|
||||
|
||||
Deliverables:
|
||||
|
||||
- Worker can attach a pre-rendered local file to a clip job
|
||||
- No CS2 automation yet
|
||||
- End-to-end job completion path verified
|
||||
|
||||
### Milestone 3: First real CS2 render
|
||||
|
||||
Deliverables:
|
||||
|
||||
- One camera mode
|
||||
- One machine
|
||||
- One output format
|
||||
- One clip rendered automatically from a real `.dem`
|
||||
|
||||
### Milestone 4: Production hardening
|
||||
|
||||
Deliverables:
|
||||
|
||||
- Retry logic
|
||||
- Timeouts
|
||||
- Better logging
|
||||
- Configurable render presets
|
||||
- Concurrency controls
|
||||
|
||||
## Recommended First Implementation Order
|
||||
|
||||
If you want this built pragmatically, the order should be:
|
||||
|
||||
1. Add a dedicated render worker project.
|
||||
2. Expand the clip job payload to fully describe a render request.
|
||||
3. Add an internal attachment API or shared service so the worker can save finished media.
|
||||
4. Implement a stub backend that completes jobs using a known local video file.
|
||||
5. Replace the stub backend with a real CS2 capture backend on one dedicated machine.
|
||||
6. Add retries, logs, and operational safeguards after the first real render works.
|
||||
|
||||
## What Not To Do First
|
||||
|
||||
Avoid these early mistakes:
|
||||
|
||||
- Do not start with multiple camera modes.
|
||||
- Do not start with distributed workers.
|
||||
- Do not treat FFmpeg as if it can render demos directly.
|
||||
- Do not overbuild orchestration before one clip can render successfully end to end.
|
||||
|
||||
## Definition Of Done For Real Rendering
|
||||
|
||||
This feature is truly working when:
|
||||
|
||||
- A user uploads a `.dem`
|
||||
- The app creates highlight clip jobs
|
||||
- The render worker picks a pending job automatically
|
||||
- CS2 playback is driven to the requested tick range
|
||||
- A video file is produced without manual intervention
|
||||
- The file is attached back to the job
|
||||
- The clip becomes playable in the Fragalytics UI
|
||||
Reference in New Issue
Block a user