https://github.com/gopherjs/gopherjs/pull/1305 is one of the weirdest butterfly effect bugs I got to debug.

The effect that starts the chain is simple enough, the hash/maphash.TestHashHighBytes test introduced in Go 1.19 is flaky under GopherJS. What does the test do? It checks that the high 32 bits of the hash are actually sort of random.

The 64-bit hash is computed using a 64-bit seed and a 32-bit low-level hash function, here's an abbreviated version:

func rthash(b []byte, seed uint64) uint64 {
	lo := memhash(b, uint32(seed))
	hi := memhash(b, uint32(seed>>32))
	return uint64(hi)<<32 | uint64(lo)
}

Somehow, half the time the higher 32 bits of the seed are exactly FFFFFFFF. Why would that be?

Well, the seed is generated by this function, which looks innocent enough:

func fastrand64() uint64 {
	return uint64(fastrand())<<32 | uint64(fastrand())
}

Liberal use of print-debugging reveals that the high 32 bits of the right uint64(fastrand()) operand are FFFFFFFF. So when or-ed with the other side or basically stomps whatever randomness would have been there. What is fastrand then?

func fastrand() uint32 {
	return uint32(js.Global.Get("Math").Call("random").Float() * (1<<32 - 1))
}

Wat? How could upsizing a uint32 to uint64 possibly yield FFFFFFFF in the high bits?!

WARNING, we are now exiting vanilla Go lands and delving into JavaScript madness that makes GopherJS work.

Because JavaScript is what it is, GopherJS has little choice but use the same number type to represent all non-64-bit integers, throwing a modulo operation here and there to keep up the appearances. Similarly, float64 and float32 are actually also the same number under the hood. That said, JavaScript does give us a tool to pretend there are signed and unsigned integers: >> and >>> respectively. In JS, the following are true: (4294967295 >> 0) === -1 and (4294967295 >>> 0) === 4294967295 (yeah, the infamous === has a little >>> brother, don't ask what happened to <<<).

Going back to GopherJS, it uses this trick to maintain signed and unsigned integers. We almost have our smoking gun. The last piece of the puzzle has to do with the unit32 → uint64 conversion step. When GopherJS tries to interpret a negative number as uint64, it assumes the number is signed, and sets the high 32 bits to FFFFFFFF as it should. Let's look at fastrand again:

func fastrand() uint32 {
	return uint32(js.Global.Get("Math").Call("random").Float() * (1<<32 - 1))
}

The js.Global.Get("Math").Call("random").Float() * (1<<32 - 1) is not very interesting, it just gives us a float64 in the range of [0, 2^32). But then it gets converted to unit32. The correct way of doing it is using the >>> 0 trick, but the compiler was emitting >> 0. Which meant any value in the [2^31, 2^32) range would be converted to its two's complement negative number. Boom! The gun fires. When our negative supposedly uint32 value is converted to uint64 the high bits get set to FFFFFFFF; when it gets |'ed with another number, it overrides whatever higher bits the other side had. Bad things happen then.

#golang #gopherjs