Videos drop frames when exporting

Known Limitation

Bug

elaina1704

I exoprted a few videos and a couple of them have lower frame rate - changed length on one of them to see if the video was too large (it was only around 10 seconds at first anyway) and it didn't make a difference.

Not sure if there's something I'm missing here to fix it!

I'm using chrome on a HP Envy x360

4 months ago

monokee

Fixed with last update (Merged beta with main)

Occasional frame drops may still occur for very large files, hard to decode codec variants or slow hardware, but this is probably as close as we're going to get with current browser standards.

3 months ago (edited)

monokee

Merged in a post:

when i export the footage it's frame dropping and gives a jlitter video

### Expected Behaviour ### Reproduction Steps 1. 2. 3. ### Your Browser and Device

lol1708 6 months ago Bug

3 months ago

monokee

Merged in a post:

After rendering videos are glitchy.

### Expected Behaviour ### Reproduction Steps 1. 2. 3. ### Your Browser and Device

christian1605 3 months ago Bug

3 months ago

monokee

~This is more or less expected as this point~

Video support is still pretty experimental, as stated in the app interface and also here:

Video Support

Ability to import and color directly on video files. Exporting video would also be nice but is not an immediate requirement. Video trimming, cutting, cropping and rotating would be awesome to have directly in the app down the line.

rogerio 1 year ago Suggestion

The reasons for this mostly come to down to fact that browsers are still incredibly limited in their capacity to both decode and encode video files and server-side processing is not something I am going to offer for Color.io.

Glitching / jittering can happen for two main reasons:

Browser have to native way to know what the original FPS of a video file is. I currently have to estimate the FPS based on deltas in frame presentation times which is a hack. It kind of works but can be skewed by CPU processing load and tends to be off by 1-2 frames on different hardware, not just because browsers drop frames to ensure smooth playback. Again, no way to control that right now.
Browsers have no native way to step through a video frame by frame without implementing decoders and demuxers in WebAssembly (which I don't have the capacity to develop right now). Accurate frame seeking would be required for the app to stream individual frames onto the GPU for color and texture processing, before reading them back on the CPU side for piping them into the WebCodecs Encoder (which also relies on accurate FPS - see (1)). Since frame seeking is not supported, I have to manually seek to frames using a similar hack that I use for determining the FPS. Again, there's no way to tell when the browser decides to drop frames.

All of this is why video processing is an unofficial and experimental feature and will remain in that state for the foreseeable future (which doesn't make me happy either 😢)

3 months ago (edited)

osoloso

dealing with this currently ! I thought I was the only one ! It was working just fine up until the other day

3 months ago

monokee

@osoloso please try the latest beta at https://beta.color.io - I just made some significant changes and things are looking much better on my end. Let me know if it's working better for you so I can merge the changes into the main app asap!

3 months ago

monokee

Potential Fix in Beta version

I just published an update to the beta version with some significant changes to the video encoding mechanism. This update improves things a lot.

I need your guy's feedback to merge this update into the main app! Thanks for your help!

Open Beta Version

3 months ago (edited)

christian1605

It works now!! Great!!

3 months ago

monokee

@christian1605 yay!!!

3 months ago

andrei242

@monokee Everything works smoothly! Video is smooth, audio is here and synced.
PS: I tested on iPhone non prores footage 4k 24, 25 and "high efficiency" one 4k60(59,99)

3 months ago

monokee

@andrei242 thanks for the feedback! Just to make sure: Are you saying you got audio working on iPhone or with iPhone footage but on desktop/chrome? Safari doesn't support AudioEncoding as far as I know and Safari is the only browser engine that's allowed on iOS so I'm really curious. :)

3 months ago

andrei242

@monokee oh, my bad. Yes, I meant that edited iphone footage in chrome on a laptop 😅. Btw just noticed that frame-rate changed just a tiny bit

3 months ago

monokee

@andrei242 yes, the frame rate is still a best guess based on frame time deltas during playback. I have some ideas to improve it further and I’ll probably also offer a way to set the frame rate manually to make things more robust 🙏

3 months ago

monokee

Fixed with last update (Merged beta with main)

Occasional frame drops may still occur for very large files, hard to decode codec variants or slow hardware, but this is probably as close as we're going to get with current browser standards.

3 months ago (edited)

ryan957

@monokee It was working great now its not. It seems well some days and not others.

2 months ago

What's new in app.color.io

Changelog

Feedback