I messed up the multiplication order for texture scaling: it multiplied a floating point number with a fixed point number, instead of multiplying two floats and then converting the result into a fixed point number.