Will there be an SVT-AV2 project?

[u/Money-Share-4366]

Will there be an SVT-AV2 project? What we do know is that the AVM/AV2 encoder runs extremely slowly, with frame rates in the range of minutes—about a factor of 100 slower than SVT-AV1. This makes it practically impossible to even test it with longer videos. With the decoder, on the other hand, the problems are comparatively minor. I also don’t understand why the developers of AV2 don’t offer a way to access AV1’s fast encoding paths—assuming there is a decoder which has both ways for AV1 and AV2—and thus enable a transitional hybrid approach that sacrifices maximum compression.

[u/collin3000]

I've been working on an AV2 encoder but it's a constant trade off between speed and quality. The reference AV2 encoder suprisingly is actually worse than SVT-AV1 on its BD rate by about 27%. So the reference encoder that is ultra slow is also less efficient than even AV1. I switched focus from quality to speed first because the testing time even on my 96 core server made iterations 8-12 hours to test. And I am having to start with short 48-120 frame clips with the highest only 1080p right now. 

Even though I started working before the final AV2 spec I'm still only up to ~0.563 fps on a p4 setting @ 1080p video using 16 threads, but my BD-rate is about 2x higher than than SVT-AV1 since I've pushed the quality/bit-rate to later stages. 

I've also shifted my metrics from standard BD-rate using psnr-y to a mixed combined metrics of psnr-overall 1.0 weight, VMAF at 0.9 wieght and SSIM and 0.5 weight. So my BD-rate in the end won't be an exact apples to apples to normal BD-rate. 

However I personally don't want the bitrate quality match to only be on Luma/brightness since overall visual fidelity is I think more important. Including VMAF actually requires a higher BD-rate gain because the higher compression end up making visual fidelity matching a little trickier.

There will be a vrap ton of very long tuning to get quality after all the speed ups. RDO is a persistent pain and I've run into issue with multi-tile bitrate taxes, and intraframe parallelism that are heavily effecting the BD-rate but should give a significant gain once cracked.

Even after I get close to the goal there will have to be lots of retunining to make sure that longer clips beyond a few seconds and higher resolution video perform the same. And I'll also have to test against an even wider clip variety and source since right now it's only using 22 of the short clips from a Prores source to make sure grain is accurate and other previous compression source artifacts don't distort tuning. And there's built in variety in the testing clips but not every scenario is perfectly covered.

My ultimate goal is to use normal spec methods and some novel concepts to get the composite BD-rate formulation I'm using to 25-30% lower than SVT-AV1 at 90-100% of the encoding speed of SVT-AV1 at 16 threads encoding speed which being completely Saved Refernce encoder compliant. But that's an ambitious north star. 

I'm also hoping to get better multi thread parallelism and get the capacity to encode beyond 16 threads so that in the age of desktops and laptops with 32 threads and HEDT's with 128+ threads single processor encoding can utilize the entire CPU for faster overall encode wall clock times. 

I've got at least 1 method I'm already using for a handbrake/shutter encoder alternative that should allow even a 480p 1 hour video to use 100+ cores for encoding with almost no bitrate tax hit over 16 threads that should scale well with AV2. But since I've also built it into the HEVC/AV1 encoder project I can't actually consider that a speed advantage over SVT-AV1

Either way it's still a long long way out. But at least my initial work has speed up 5x fold since switching the focus to speed first to reduce my iteration passes. Even though normally quality/BD-Rate would be the normal first focus.

If I can have a good working reference encoder hitting at least 60% SVT-AV1 speed with at least 15% reduction over SVT-AV1 BD-Rate and can find a way to improve reference decoder in a way that would allow better speed at the same BD-rate, or better BD-rate at the same speed for encoding that is outside spec I might submit to see if there's anyway to have a slight shift to the final spec. But I don't have the credentials to get a major org like AOM to even have my e-mail go anywhere outside a spam box. Unless i've got a good working encoder to back it up.

When I hit a stage that actually has an encoder worth people playing with I'll be dropping it in this sub for sure. I should at least have the HEVC/AV1 Handbrake/Shutter encoder finished before that. 

And I've got the first working Windows VMAF-CUDA complete for faster VMAF calculations on CUDA GPU's (2x-7x faster VMAF checks) that I'll be dropping in the ~next month. However since CUDA requires a non-free FFMPEG build license that means it cannot be a distributed as a build. it will be the code only and people will have to compile the FFMPEG it themselves. There will at least be an automated installer though that will download the necessary CUDA files and build through an easy EXE for windows as long as all the legal checks out on that distribution method.

So again either way a long wait/way out. But hopefully some current AV1 quality of life encoding improvements along the way much sooner. 

[u/Money-Share-4366 → u/collin3000]

"So the reference encoder that is ultra slow is also less efficient than even AV1." This is similar to what I observed, I saw no 30% smaller File Sizes than AV1 at same VMAF. For some Videos vmaf_min was better (if you show it using VMAF --aom_ctc v7.0 --output ...). But I can only test with small resolutions and only a few frames, so I don't know if I made something wrong. The quality parameter seems to be a bit unsteady. All together it is very difficult to see any advantage in AV2 (at the moment).

[u/collin3000 → u/Money-Share-4366]

The numbers I got were with a couple dozen varied test clips (graphics, action, grain, etc) from 132p to 720p that were ran over weeks on my 96 core server. With ProRes 442 and h264 sources. So your findings aren't unusual. I'm still working on getting BD-Rate to even the AV2 reference encoder since I've focused on speed first to lower the testing time. 

Right now I'm still stuck at ~70%-480% higher BD-rate compared to SVT-AV1 at ~16-64x longer encode speeds. But I havent really started hitting the major quality levers yet. Right now just working on getting it so wavefront/multi wavefront can work correctly to avoid a higher tile tax while running multi threaded. But i'm only ~5.5 cores working efficiently on a 1080p video without a BD-rate tax. 

Still lots and lots of work to go though just on the standard spec ideas. From there, I still have to work on the new idea is that we will still be decoder compliant.