[next] Skybox HOM in CEZ2 indoors in software mode

In software mode, certain indoor locations with windows in CEZ2 show the skybox as a HOM. This does not happen when going outside. This does not happen in 2.2.8.

Here's a gif with homremoval turned on.

this seems to be the exact problem that stopped !763 (closed) from getting merged
to note is that katsy seemed to be using homremoval 2 (labelled "development homremoval" in the code) instead of the suggested homremoval 1 in the MR
which calls into question whether people just don't like homremoval

and no homremoval isnt laggy it's literally one drawfill i checked :,(

Here's another gif, with homremoval both off and on.

Also this does not happen in 2.2.8.

changed the description

confusing, since katsy experienced it previously ...

Yep..

Here's 2.2.8 with homremoval flash.

homremoval 2 was just for demonstration purposes, homremoval 1 makes the sky solid black instead

also, this does happen in 2.2.8

I just can't get that to show up. I tried the official .exe, and I tried compiling SRB2_release_2.2.8.

@katsy is that gif from the 2.2.8 release or is it from master?

added Bug Regression Rendering Software renderer labels

changed milestone to %2.2.9

mirroring my response in #hardcoding here for future reference

there was context i swear

This is a very interesting bug. I’m on the latest android version on srb2.org being 2.2.8 so here is another person who has indeed seen this bug. I’ve also seen this on my laptop which is also 2.2.8

Oh and to add to it. I myself cannot compile srb2 as both my android and windows build setups don’t work. So I HAVE to use the builds from srb2.org

removed Regression label

I have never seen this bug happen at any point during development, I can't reproduce it in 2.2.8 (or any prior release) at all, but it does happen in the pre-release for me.

added Regression label

This bug seems to start occurring since !1213 (merged). Additionally this only seems to happen with skyboxes on, and therefore it might be related to portal rendering.

If Yume is getting this problem in the Android version then I get the feeling there's always been a bug somewhere in the code but it was obscured/masked by the optimizations, and it became evident after !1213 (merged) although it was always there for Android due to not sharing the same compiler optimizations or something.

Quick pop in. I'm using a compilation of next myself now (switched back to linux for coding stuff) and I still see this same issue there.

Now I'm even more inclined to think there's been an underlying issue for who knows how long ago, which was obscured when compiling for Windows with the previous optimization flags.

After some investigation and an attempt to hide this bug by editing the map, I've noticed it happens in these specific areas:

(red lines are skybox geometry)

My assumption is that it's caused by the small skybox sectors between the outside and the player's positions inside the tower, which are used to slope the skybox tower's roof, and that it's trying to render the first linedef it encounters even though none of its textures should ever be visible (they're line slopes).

Weirdly enough, this bug seems to be somewhat dependent on the map's location on the grid: so far in testing, moving the entire map up roughly 3k to 13k fracunits eliminates the HOMs in the bottom area, while moving it up further than that brings them back but with reduced intensity. The HOMs on the left area seem to be unaffected by this. In addition, similar situations elsewhere in the map either do not have this issue at all, or have it but far less intense than the initially reported area.

I'm not sure what caused this regression, and it might just be a side-effect of something that's beneficial in other parts of rendering, but it does make skyboxes much less flexible...

I tried removing the front side lower textures (CESWCA02) from these lines (and also similarly from all the other pillars in the skybox) and that seems to fix this issue on my end.

Yep, that fixes the issue. I've removed every single unused texture in CEZ2's skybox, and this map update will be in the next release.

However, should this issue be left open? I'm still undecided whether this is just a bad mapping practice that accidentally worked until now, or if it's an actual regression in the rendering code...

After doing some testing and research, I think the problem part is the calculations here: https://git.do.srb2.org/STJr/SRB2/blob/next/src/r_segs.c#L1644

Occasionally something goes "wrong" with segleft and segright, resulting in worldtop, worldtopslope, worldbottom and worldbottomslope getting slightly different values, which then results in the bottom texture clipping test passing when it (probably?) shouldn't. (test located here https://git.do.srb2.org/STJr/SRB2/blob/next/src/r_segs.c#L1998 )

Since the SSE3 merge request triggers this issue, and it also happens on Android on versions before the merge request, my theory is that those calculations involving floating point values and conversions to integers and back get different results depending on the CPU architecture and compiler settings. Before the merge request, floating point calculations would actually use 80-bit intermediate values on x86 platforms. The merge request made x86 use real 32-bit and 64-bit floating point calculations instead.

If I try replacing 32-bit floats with 64-bit doubles in those calculations, (including also making FIXED_TO_DOUBLE and DOUBLE_TO_FIXED definitions,) the flickering walls don't appear anymore.

mentioned in merge request !1424 (merged)

closed via merge request !1424 (merged)

mentioned in commit d97de6d3