Jump to content
sessionbeer

Bayer Dithering

Recommended Posts

Posted (edited)

I've tried to convert this to vex but stuck on the #define functions https://www.shadertoy.com/view/7sfXDn
I'm not sure vex is capable of looping through the functions like this?
 

float myscale = 0.4;
float Bayer2(vector2 a) {
    a = floor(a);
    return frac(a.x / 2.0 + a.y * a.y * .75);
}

#define Bayer4(a)   (Bayer2 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer8(a)   (Bayer4 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer16(a)  (Bayer8 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer32(a)  (Bayer16(.5 *(a)) * .25 + Bayer2(a))
#define Bayer64(a)  (Bayer32(.5 *(a)) * .25 + Bayer2(a))

float dithering = (Bayer8(@uv * 0.25) * 2.0 - 1.0) * 0.5;
vector2 uv = set(@P.x, @P.z) / myscale;
@uv.x += dithering; 
   
@Cd = @uv.x;

 

Edited by sessionbeer

Share this post


Link to post
Share on other sites
Posted (edited)

Huh, the result is pretty neat.
You're basically there, but with a few issues.
Sorry if I ramble a bit, I tend to be quite lengthy in my explanations.

The error message from your code says
"No matching function for float Bayer2(float). Candidates are: float Bayer2(vector2 &)"
This shows that VEX has an issue with a variable type.
If we comment everything and leave only this

float Bayer2(vector2 a) {
    a = floor(a);
    return frac(a.x / 2.0 + a.y * a.y * .75);
}
float dithering = (Bayer2(@uv * 0.25) * 2.0 - 1.0) * 0.5;

The same issue occurs. But that code doesn't do anything with #define and whatnot. So there's an issue with how we call Bayer2.
Basically, VEX doesn't like "@uv". Bayer2 takes a vector2. But how are we passing @uv ? Is it a vector3, or a float ? Either way that's a problem, we specifically need a vector2.
The fix is to tell VEX the type we want, with a "u" for vector2.

float Bayer2(vector2 a) {
    a = floor(a);
    return frac(a.x / 2.0 + a.y * a.y * .75);
}
float dithering = (Bayer2(u@uv * 0.25) * 2.0 - 1.0) * 0.5;

Great. Now we have a different error.
"Error: Read-only expression given for read/write parameter."
This is because we are trying to modify the variable "a" in the Bayer2 function. VEX doesn't like that for some reasons.
The fix is simple enough - make a temporary variable (and replace the subsequent references of a).

float Bayer2(vector2 a) {
    vector2 b = floor(a);
    return frac(b.x / 2.0 + b.y * b.y * .75);
}
float dithering = (Bayer2(u@uv * 0.25) * 2.0 - 1.0) * 0.5;

Cool, so we are now error-free. Let's bring back the #defines and use one of them instead of Bayer2.

float Bayer2(vector2 a) {
    vector2 b = floor(a);
    return frac(a.x / 2.0 + a.y * a.y * .75);
}

#define Bayer4(a)   (Bayer2 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer8(a)   (Bayer4 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer16(a)  (Bayer8 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer32(a)  (Bayer16(.5 *(a)) * .25 + Bayer2(a))
#define Bayer64(a)  (Bayer32(.5 *(a)) * .25 + Bayer2(a))

float dithering = (Bayer8(u@uv * 0.25) * 2.0 - 1.0) * 0.5;

Alright, no error, looking good. Now for the last part of your code..

float myscale = 0.4;
float Bayer2(vector2 a) {
    vector2 b = floor(a);
    return frac(a.x / 2.0 + a.y * a.y * .75);
}

#define Bayer4(a)   (Bayer2 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer8(a)   (Bayer4 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer16(a)  (Bayer8 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer32(a)  (Bayer16(.5 *(a)) * .25 + Bayer2(a))
#define Bayer64(a)  (Bayer32(.5 *(a)) * .25 + Bayer2(a))

float dithering = (Bayer8(u@uv * 0.25) * 2.0 - 1.0) * 0.5;
vector2 uv = set(@P.x, @P.z) / myscale;
@uv.x += dithering; 
   
@Cd = @uv.x;

Cool, no error again, and something happens in the viewport.. Something that is evidently not Bayer Dithering.
Hmm, alright, what's up with that.

Now that you are unblocked, if you want to figure it out by yourself, now's the time to do it.
Alright ? Cool. Here's the solution.

In your code, you assumed that "fragCoord" from the HLSL code was the same as UVs. That is slightly incorrect.
In shadertoy, if you map fragColor to fragCoord, you'll find out that fragCoord actually represents the canvas size in pixel.

In my case, max(fragCoord) == (1200, 675)
(those numbers were taken from the information at the bottom-left of the canvas)

To test that out, I wrote the following in shadertoy, and got the classic UV black, red, green, orange and yellow result.

fragColor = vec4(fragCoord.x / 1200.0, fragCoord.y / 675.0, 0, 0);

(actually I started with fragCoord.x only, saw that it was SUUUPER red, then tried dividing by 255, saw the gradient appearing in only a small portion of the canvas, then tried dividing with the X resolution, and got a full gradient in red, and thus I understood what I just mentioned)

SO ANYWAY
All of that is to say that you have to multiply the u@uv by the resolution of your grid.

Here's what I added

vector2 fragCoord = u@uv * set(ch("../grid1/rows"), ch("../grid1/cols"));
float dithering = (Bayer8(fragCoord * 0.25) * 2.0 - 1.0) * 0.5;

We now have a different result, but still a broken one. Hooray.
Let's focus on the last three lines, one after the other.

vector2 uv = set(@P.x, @P.z) / myscale;

That is wrong. The HLSL reference is

Quote

vec2 uv = fragCoord / iResolution.xy;

We know that fragCoord is u@uv * resolution
And that line is basically doing (u@uv * resolution) / resolution.

In our case, this line is simply

vector2 uv = u@uv;

But let's add the scale parameter you had

vector2 uv = u@uv / chf("scale");

 

Next line

@uv.x += dithering;

The fix is simple enough, we want to modify the uv variable, and not the uv attribute.

uv.x += dithering;

 

Final line

@uv.x += dithering;

Same thing, we want to use the uv variable. Also, the HLSL does a comparison

Quote

fragColor = vec4(uv.x < 0.5);

So let's do that

@Cd = uv.x < 0.5;

 

All put together, here's the final code

float Bayer2(vector2 a) {
    vector2 b = floor(a);
    return frac(b.x / 2.0 + b.y * b.y * 0.75);
}

#define Bayer4(a)   (Bayer2 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer8(a)   (Bayer4 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer16(a)  (Bayer8 (.5 *(a)) * .25 + Bayer2(a))
#define Bayer32(a)  (Bayer16(.5 *(a)) * .25 + Bayer2(a))
#define Bayer64(a)  (Bayer32(.5 *(a)) * .25 + Bayer2(a))

vector2 fragCoord = u@uv * set(ch("../grid1/rows"), ch("../grid1/cols"));
float dithering = (Bayer8(fragCoord * 0.25) * 2.0 - 1.0) * 0.5;
vector2 uv = u@uv / chf("scale");
uv.x += dithering; 
   
@Cd = uv.x < 0.5;

So yeah, sorry for the gigantic answer, it could have been only the last snippet, but I hope this is more useful to you than just the answer.

Cheers !

Edited by Alain2131
  • Thanks 2

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×