Skip to content

Shader plugin support / Temporal processing support (Blur Busters has source code bounty (2K) for this) #35

Open
@mdrejhon

Description

@mdrejhon

I'm Mark, the creator of Blur Busters and TestUFO.

I noticed you have a virtual display driver capable of custom refresh rates -- maybe you want to make minor modifications to meet an existing Blur Busters source code bounty?

For those not familiar with me -- I'm an indie-turned-business, and the Internet's biggest advocate for high refresh rates, and I am in over 25 peer reviewed research papers and I have an easier Coles Notes research portal Area 51 on BlurBusters.

TestUFO is my free display-testing/display-demoing website used by over 500 content creators (reviewers, youtubers, bloggers) that tests displays. Two of the biggest content creators are RTINGS (which uses my invention) and LinusTechTips (which used the pursuit camera in some tests).

I've been looking for a good open source virtual display driver as a base to produce drivers that can implement algorithms such as testufo.com/blackframes (software BFI) and testufo.com/vrr (simulated VRR on non-VRR displays), as well as hundreds of other fantastic temporal filters.
Even if not all filters are possible, a virtualized display driver with temporal-processing capabilities, would unlock a lot of neat capabilities for a lot of communities.

It could also be used to add subpixel-awareness for OLED displays (odd pixel structures) by scaling things in a custom-subpixel-pattern way, it was also posted here in ClearType improvement thread, but I'm posting here since I've now discovered this project:

$2K Bounty for a Virtual Display Driver capable of Temporal Filters

Which I crosspost here:

**Jason-GitH ** commented May 11, 2023

Yes, that's another option as an alternative.

I mention this already at microsoft/PowerToys#25595 (comment)

A USD $2000.00 bounty already offered for this open source driver project

Blur Busters currently already offers a $2K bounty for this -- a subpixel-aware downscaler would allow everything to be "ClearType'd" including graphics, and not just text. This can even be done as a third-party Windows Indirect Display Driver, which I already offer a USD$2000.00 source code bounty for something under MIT or Apache. Contact me at mark[at]blurbusters.com

Basically, a Windows IDD that virtualizes a simulated monitor at a simulated resolution at a simulated refresh rate (can be higher or lower Hz than real display) -- we have some important uses cases for this), and runs all the frames through plug-in GPU shaders, to display on an existing physical monitor.

In addition to solving this problem -- plug-in GPU shaders can enables shockingly creative stuff such as:

  • subpixel-aware downscaling, etc.
  • custom developer refresh rate testing on monitors not owned (e.g. test refresh rates that your monitor does not support)
  • easier software developer testing with displays that aren't in the QA lab, by simulating a display
  • easier hardware prototyping and engineering of future displays, by simulating a display
  • simulated interpolated VRR (www.testufo.com/vrr algorithm) on displays that don't have VRR
  • software based BFI OLED motion blur eduction (www.testufo.com/blackframes#count=3)
  • streaming / headless use cases (without needing to be in a game nor open Remote Desktop)
  • software based overdrive algorithms
  • custom spatial filters (e.g. HLSL CRT filters for retro simulation / preservation).
  • custom temporal filters (e.g. simulate a CRT electron beam, complete with rolling scan and phosphor fade simulation, by using 8 digital refresh cycles per CRT Hz, on 480Hz display that windows thinks it "sees" as a 60Hz CRT tube), for even further improved retro simulation / preservation / education / history education.
  • add frame generation (e.g. reprojection https://www.youtube.com/watch?v=IvqrlgKuowE for 500fps UE5)
  • Other plug-in GPU shaders (that can modify, adjust, scale, process, merge, blend, etc history of buffered "refresh cycle" frames)

The $2K bounty doesn't have to include these custom shaders, but needs to include a plug-in shader system, and end users need the ability to share shaders/profiles with each other (e.g. a sharable modular file-based format, possibly JSON or XML) that loads settings such as shader module (compiled or uncompiled), input Hz, output Hz, constraints if any (outputHz = Formula(inputHz)). It could be a single-file format or a composite (.zip) that includes shaders and settings for a custom filter/processing. Such as shaders that require actual display output Hz to be integer multiples of the virtualized display Hz. The bounty requirements are negotiable (contact mark [at] blurbusters.com), but it needs to meet the needs of the open source community, the retro community, the blur busters fan community, and the commercial community (e.g. display prototyping), as I do work with display manufacturers. As you know, I am the inventor of free display tests used both by indies and manufacturers...

Other options include integration of SweetFX/Reshade/etc into the virtual display driver, including possibly open source or third party lagless frame generation algorithms (e.g. https://www.blurbusters.com/framegen ...) or whole-desktop SmoothVideoProject, etc. So this ideally should be supported too, although it cannot support the entire scope needed. However, the capability for output refresh rates higher AND/OR lower AND/OR variable than the refresh rate visible in Control Panel for the said virtualized display. In other words, the real Hz displayed on real display, is a processed version of the virtualized Hz visible to Windows,. e.g. 60Hz in Windows, but is outputting 240Hz+BFI with persistence adjustability (similar to https://www.testufo.com/blackframes#count=4&bonusufo=1 as one example). Or variable frametimes (I can help you understand how to control VRR).

I really would like to see a driver package capable of running custom shaders at refresh cycle granularity. And enough example dummy shaders to demonstrate successful virtualization of resolution AND refresh rate (both higher and lower than actual physical display(s) connected to computer). INCLUDING of course, variable refresh rate capability which is simply asynchronous timing of Present() that may not be related to timing interval of the refresh cycles. I am OK with the constraint of a fixed-Hz virtualization (just VRR output virtualization).

Although not needed for subpixel processing filters and input=output Hz filters (shader-process-only filters)..... for certain temporal refresh rate filtering tasks I'll need extremely accurate timing of output-frame Present()'s, which MAY require a separate CPU thread that busyloops right before Present() of the composited frame to the actual output display. So an ultra-high-precision presentation timing thread will be mandatory here, I can help assist you -- I have lots of experience via [url=https://forums.blurbusters.com/viewtopic.php?t=4213]Tearline Jedi[/url] raster-interrupt-style beam racing of VSYNC OFF tearlines).

Keep in mind that this is a refresh-cycle-granularity processor, not a frame-granularity processor. For many reasons, it has to process every refresh cycle independently of underlying framerates of the application running within the desktop on the virtualized display). Be noted, frame presenting timestamps do need to be made available to the shader run once every refresh-cycle (for certain shaders like VRR simulating shaders, which requires refresh cycle granularity processing while also separately knowing the frame presentation timestamps of the underlying application).

Except that it would work on everything in Windows, not just in a test. Reshade/SweetFX/SpecialK/etc does something similar in some ways, but not for many use cases. The other drivers have the problem of being frame-granularity (as frame Present() hooks) which means they are not refresh cycle granularity. Many algorithms need to process at Hz-granularity independently of the underlying frame rate, for successful operation. And being a Windows IDD, means desktop applications are included, not just games.

For qualifying for the bounty, some changes needed may include:

  • Support replacing an existing monitor with the virtualized monitor (or some compromise), ala Feature Request: Disable Other Monitors #19
  • Adding support for a loadable settings/shader system (to allow the temporal processing)
  • A relicensed fork (if that's an option) to Apache/MIT.
    (legal if you're the only developer, or all devs/contributers of 100% of the source code retroactively agree)
  • Bounty requirements can be negotiable (contact me privately), with requirements confirmed into stone before agreeing if necessary;

Example (of hundreds) of possible benefits....

  • End users who want to reduce display motion blur by adding software-based BFI
  • End users who don't have VRR that want to add software-based VRR
  • Adding software-based superior LCD overdrive to cheap monitors (that performs better than manufacturer 17x17 LCD OD LUTs), like superior versions of the old "ATI Radeon Overdrive" system from 15 years ago;
  • Game developers stuck at 60Hz, can finally QA-test on higher refresh rates (virtualized), with the extra Hz's being blended together one way or another (alphablend, or simul-vrr, or tiled, or other).
  • Manufacturers can prototype future-Hz displays like 1000Hz displays
  • Improved SDR/HDR adjustability (e.g. using HDR for SDR content as a brightness booster for BFI like Retrotink 5K).
  • Future CRT simulators for retro community (electron beam simulators in a shader, using 8 refresh cycles on upcoming 480Hz OLED to create 1 simulated 60Hz CRT refresh cycle)
  • etc.

Some possible catches

  • It may not be able to play DRM content (e.g. Netflix)

Question: Are you the only developer, @MolotovCherry?
Note: It could even be a fork of this project if this is too dramatically different. The original project can remain GPL3, just the fork qualifying for the bounty would need to be one of the permissive licenses to support both the indies and businesses, as per above. Can even backport the Apache/MIT version to your GPL3 fork (one-way compatibility), if you want -- all good.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions