feat: preview thumbnail when hovering over seek bar

[RonzyOnGIT]

What this does

• Shows little preview thumbnail when hovering over seekbar
• Thumbnails are cached in 4-second intervals
• Used refs to avoid unnecessary re-renders during hover

Notes | limitations

• Only for desktop version for this iteration.
• Tested on Chromium-based browsers (Chrome, Thorium)
• Tested on firefox as well but it does not work on firefox for some video files. For future iterations, I can just remove this feature entirely for unsupported browsers
• There is definitely room for optimization such as dynamic intervals depending on length of video to save space and different img formats for smaller sizes.

Found cached image and accurate frame

First cache miss, the whole cache is empty

When you hover along the seekbar fast, shows last found cached image. Since there is no re-render, will not show the correct thumbnail unless you re-hover after image is rendered

[killergerbah]

Thanks @RonzyOnGIT this is pretty cool
Just have two main concerns that I don't really have an answer for atm:

• Wonder how this will affect memory usage, since there are now two videos autoloading the same thing. Seeking also adds some IO and potential memory usage as well.
• I don't think the thumbnail should display above the subtitles. But if we let the subtitles float on top, it might not look good.

[RonzyOnGIT]

Thanks @RonzyOnGIT this is pretty cool Just have two main concerns that I don't really have an answer for atm:

* Wonder how this will affect memory usage, since there are now two videos autoloading the same thing. Seeking also adds some IO and potential memory usage as well.

* I don't think the thumbnail should display above the subtitles. But if we let the subtitles float on top, it might not look good.

Thanks for taking a look at it!
I also was unsure of how to handle subtitles and thumbnails overlapping. One potential solution would be to add a setting so that the user can choose how overlap is handled— for example: moving subtitles up, hiding the thumbnail, or allowing it to cover subtitles.

Regarding memory usage, one alternative I had in mind was creating a pre-generated sprite sheet, completely removing the need for a second video file. The issue with that is that there would be a need for a server to handle the video processing such as ffmpeg to generate frames.

I really love this project — thanks for all the work you’ve put into it. I’m excited to keep contributing where I can. I'll mark the pr as a draft for now.

[killergerbah]

I also was unsure of how to handle subtitles and thumbnails overlapping. One potential solution would be to add a setting so that the user can choose how overlap is handled— for example: moving subtitles up, hiding the thumbnail, or allowing it to cover subtitles.

Of these options, I think it might be best to either:

• No setting, thumbnail below subtitles
• Setting to control whether thumbnail is above or below

The other options look pretty hairy to implement, as you need to consider how to calculate the intersection between the thumbnail and the subtitles.

Regarding memory usage, one alternative I had in mind was creating a pre-generated sprite sheet, completely removing the need for a second video file. The issue with that is that there would be a need for a server to handle the video processing such as ffmpeg to generate frames.

I think I just need to profile on different browsers. The memory usage is probably not a big deal and maybe we could add a setting to disable the thumbnail entirely.

[RonzyOnGIT]

Sorry for the slow update 🙏😔.

Fixes

• Add the setting to allow user to change position of subtitles (behind the thumbnail or infront)

Things to work on

• Adjust resolution for videos with aspect ratios of 4:3
• Add setting to disable thumbnail entirely
• Stylize fetching thumbnail skeleton
• Ignore for mobile devices for now

I think I just need to profile on different browsers. The memory usage is probably not a big deal and maybe we could add a setting to disable the thumbnail entirely.

I’ll try to monitor memory usage and report on it in future updates — just wanted to share a quick progress update.

[killergerbah]

This feels more like a misc setting then a subtitle appearance setting. Subtitle appearance settings are track-specific and I'm not sure if it makes sense to offer per-track granularity for this new setting.

[killergerbah]

Curious why you decided to create another type instead of just using boolean?

[RonzyOnGIT]

Curious why you decided to create another type instead of just using boolean?

I was just trying to keep it consistent, since I implemented it in Layout and subtitleAlignment had its own type. I'll move it to misc and probably just keep it as a plain boolean type.

[killergerbah]

Wherever you put this you'll need to hide this behind insideApp

[RonzyOnGIT]

Could you clarify why it needs to be behind insideApp? Is it to like prevent any other clashes if its not in the app or something?

[killergerbah]

The preview only matters for the local video player available on the website, so displaying the setting from the extension settings might be confusing to the user

[RonzyOnGIT]

Fixes

• Moved the subtitle position setting to misc.
• Make the thumbnail size dynamic based on video aspect ratio
• Changed the interval for caches to 5s instead of 4s decreasing cache memory by ~21%

Memory report

No surprise, there is a noticeable memory increase.

On average I tracked an overall increase of ~50% in memory usage. I used various files ranging from 200mb - 1.1gb files.

I tested in chromium using the task manager to track memory usage. A large part of the memory increase is of course the second video element.

The cache to store the screenshots only makes up about 7% of this increase.

For reference, a 1.1gb video file without the feature peaks at around 270mb. With the preview feature it jumps up to 418mb, 55% increase.

Notes

• I'll work on adding a setting to disable the feature entirely and see if memory goes back to baseline
• As of now, if a user makes lots of request to get thumbnail, they will get dropped: if (isGeneratingRef.current) return;. A better way would be to create a queue or something, just something to keep in mind.
• I increased the cache interval to 5 seconds, which provided a solid performance improvement while still maintaining reasonably accurate thumbnails. Increasing the interval further produced diminishing returns imo.