Remaining issues:
Goal
In addition to our APIs for rendering on the server, we want to introduce APIs for rendering videos, images and audio on the client.
Timeline
Currently shipped as an experimental alpha:
https://remotion.dev/docs/client-side-rendering
Package
The package is named @remotion/web-renderer and it has just one dependency – remotion.
It will obviously just run in the browser.
Studio integration
In your remotion.config.ts:
Config.setExperimentalClientSideRenderingEnabled(true);
and restart your Studio.
Basic mechanic
https://www.remotion.dev/docs/client-side-rendering/how-it-works
API
We have 2 APIs:
http://remotion.dev/docs/web-renderer/render-media-on-web
http://remotion.dev/docs/web-renderer/render-still-on-web
Fixing previous mistakes
In renderMedia(), there was only “codec“ and „audioCodec“, which determined the container.
For this API, we have „container“, „videoCodec“ and „audioCodec“ to give full freedom.
Challenges
– Not everything is capturable. Support as much as possible, with as much precision as possible.
– Precision issues: Say an element is a child element and the non-capturable parent has a „transform“ applied to it. It might render differently due to this
– z-indexes: The stacking logic of the browser is very complicated. We probably cannot replicate it fully.
Isolating contexts
Isolating a render from the webpage: Remotion relies on a lot of contexts and uses a lot of global APIs like window.remotion_renderReady. There is a ton of scoping we need to do so that these APIs don’t conflict with e.g. the global variables in the Remotion Studio, or if multiple videos are rendered at once.
This also requires introduction of some new APIs (like delayRender() -> useDelayRender()) to allow scoping to the current render. Users will have to use these APIs in order to use browser rendering. We should have documentation, deprecations and ESLint rules to help them
Embedding videos and audio
We will only support @remotion/media for web rendering. Only for this one we have sufficient control over the processing, and our Rust binary cannot run in the browser, so OffthreadVideo is off the table.
Visual testing
We should not have to repeatedly test the rendering of images and videos manually.
I have set up visual regression testing that allows us to verify we compose the images correctly. It can be executed in packages/web-renderer -> bun run test.
Later we should check if we encode videos correctly as well using automated tests. Just letting some tests run and then read the metadata should be sufficient for launch.
Roadmap
✅ Scaffold package
✅ Proof of concept renderStillOnWeb() API - captures canvas and SVG elements and composes them
✅ Doing the best we can to handle transforms from parent elements
✅ Doing the best we can to handle z-indexing
✅ Audio capture
✅ Support calculateMetadata()
✅ Isolate contexts propely and remove dependency on window.remotion_* variables.
⬜ Create resources for users to ensure they use the correct APIs
✅ Capturing multiple frames and encoding the video
🚧 Studio render Dialog
⬜ Add browser rendering to Editor Starter
Remaining issues:
<span>s can get smaller #6220Goal
In addition to our APIs for rendering on the server, we want to introduce APIs for rendering videos, images and audio on the client.
Timeline
Currently shipped as an experimental alpha:
https://remotion.dev/docs/client-side-rendering
Package
The package is named
@remotion/web-rendererand it has just one dependency –remotion.It will obviously just run in the browser.
Studio integration
In your remotion.config.ts:
and restart your Studio.
Basic mechanic
https://www.remotion.dev/docs/client-side-rendering/how-it-works
API
We have 2 APIs:
http://remotion.dev/docs/web-renderer/render-media-on-web
http://remotion.dev/docs/web-renderer/render-still-on-web
Fixing previous mistakes
In renderMedia(), there was only “codec“ and „audioCodec“, which determined the container.
For this API, we have „container“, „videoCodec“ and „audioCodec“ to give full freedom.
Challenges
– Not everything is capturable. Support as much as possible, with as much precision as possible.
– Precision issues: Say an element is a child element and the non-capturable parent has a „transform“ applied to it. It might render differently due to this
– z-indexes: The stacking logic of the browser is very complicated. We probably cannot replicate it fully.
Isolating contexts
Isolating a render from the webpage: Remotion relies on a lot of contexts and uses a lot of global APIs like window.remotion_renderReady. There is a ton of scoping we need to do so that these APIs don’t conflict with e.g. the global variables in the Remotion Studio, or if multiple videos are rendered at once.
This also requires introduction of some new APIs (like delayRender() -> useDelayRender()) to allow scoping to the current render. Users will have to use these APIs in order to use browser rendering. We should have documentation, deprecations and ESLint rules to help them
Embedding videos and audio
We will only support @remotion/media for web rendering. Only for this one we have sufficient control over the processing, and our Rust binary cannot run in the browser, so OffthreadVideo is off the table.
Visual testing
We should not have to repeatedly test the rendering of images and videos manually.
I have set up visual regression testing that allows us to verify we compose the images correctly. It can be executed in packages/web-renderer -> bun run test.
Later we should check if we encode videos correctly as well using automated tests. Just letting some tests run and then read the metadata should be sufficient for launch.
Roadmap
✅ Scaffold package
✅ Proof of concept renderStillOnWeb() API - captures canvas and SVG elements and composes them
✅ Doing the best we can to handle transforms from parent elements
✅ Doing the best we can to handle z-indexing
✅ Audio capture
✅ Support calculateMetadata()
✅ Isolate contexts propely and remove dependency on
window.remotion_*variables.⬜ Create resources for users to ensure they use the correct APIs
✅ Capturing multiple frames and encoding the video
🚧 Studio render Dialog
⬜ Add browser rendering to Editor Starter