Here's an example. Say you have a fof behind a sprite, and the top of the fof is slightly raised above the floor the sprite is resting on. Which do you think will happen? Will the fof's plane be drawn behind the player or in front of the player? If you answered in front, that is correct.
Sprites sort against planes by checking if the bottom of the mobj is above the plane, if the plane is below the camera, or the top of the mobj is below the plane, if the plane is above the camera. This means that if the mobj is partially pushed into the plane, the plane gets sorted in front of the sprite. (There is an exception for MF_NOCLIPHEIGHT though that lets the mobj go half way in before the plane is sorted in front.)
What I've done here is check if the top of the sprite, not the mobj, is above the plane, or the bottom is below, for the respective cases relative to the camera. Meaning that the sprite will be drawn in front of the plane if any part of it would appear in front, not just if the whole mobj would.
I've also removed code that checks for plane heights in sprite thickseg sorting, so that sorting against thicksegs depends entirely on scales. This is to avoid a false positive with the new sprite plane sorting if the top of the sprite is above the plane but the sprite is horizontally behind the fof. This also fixes sorting issues with sides only fofs.
There is one caveat that is demonstrated by an intangible fof however:
Here you might think the sprite should be sorted in front of the fof's side and appear to be offset down into the fof. I believe this is a small sacrifice though since it looks less odd that the plane sorting issue.
