https://github.com/gopherjs/gopherjs/pull/1305 is one of the weirdest butterfly effect bugs I got to debug.

The effect that starts the chain is simple enough, the hash/maphash.TestHashHighBytes test introduced in Go 1.19 is flaky under GopherJS. What does the test do? It checks that the high 32 bits of the hash are actually sort of random.

The 64-bit hash is computed using a 64-bit seed and a 32-bit low-level hash function, here's an abbreviated version:

func rthash(b []byte, seed uint64) uint64 {
	lo := memhash(b, uint32(seed))
	hi := memhash(b, uint32(seed>>32))
	return uint64(hi)<<32 | uint64(lo)
}

Somehow, half the time the higher 32 bits of the seed are exactly FFFFFFFF. Why would that be?

Well, the seed is generated by this function, which looks innocent enough:

func fastrand64() uint64 {
	return uint64(fastrand())<<32 | uint64(fastrand())
}

Liberal use of print-debugging reveals that the high 32 bits of the right uint64(fastrand()) operand are FFFFFFFF. So when or-ed with the other side or basically stomps whatever randomness would have been there. What is fastrand then?

func fastrand() uint32 {
	return uint32(js.Global.Get("Math").Call("random").Float() * (1<<32 - 1))
}

Wat? How could upsizing a uint32 to uint64 possibly yield FFFFFFFF in the high bits?!

WARNING, we are now exiting vanilla Go lands and delving into JavaScript madness that makes GopherJS work.

Because JavaScript is what it is, GopherJS has little choice but use the same number type to represent all non-64-bit integers, throwing a modulo operation here and there to keep up the appearances. Similarly, float64 and float32 are actually also the same number under the hood. That said, JavaScript does give us a tool to pretend there are signed and unsigned integers: >> and >>> respectively. In JS, the following are true: (4294967295 >> 0) === -1 and (4294967295 >>> 0) === 4294967295 (yeah, the infamous === has a little >>> brother, don't ask what happened to <<<).

Going back to GopherJS, it uses this trick to maintain signed and unsigned integers. We almost have our smoking gun. The last piece of the puzzle has to do with the unit32 → uint64 conversion step. When GopherJS tries to interpret a negative number as uint64, it assumes the number is signed, and sets the high 32 bits to FFFFFFFF as it should. Let's look at fastrand again:

func fastrand() uint32 {
	return uint32(js.Global.Get("Math").Call("random").Float() * (1<<32 - 1))
}

The js.Global.Get("Math").Call("random").Float() * (1<<32 - 1) is not very interesting, it just gives us a float64 in the range of [0, 2^32). But then it gets converted to unit32. The correct way of doing it is using the >>> 0 trick, but the compiler was emitting >> 0. Which meant any value in the [2^31, 2^32) range would be converted to its two's complement negative number. Boom! The gun fires. When our negative supposedly uint32 value is converted to uint64 the high bits get set to FFFFFFFF; when it gets |'ed with another number, it overrides whatever higher bits the other side had. Bad things happen then.

#golang #gopherjs

https://hachyderm.io/@Mara/111357549294998960 gave me a good chuckle.

It also reminded me of a thing that I never really considered until I started contributing to #GopherJS: a += b and a = a + b are not semantically equivalent statements. In the former a is evaluated only once, and in the latter twice. That doesn't matter when a is just a variable, but if it includes a function call with side effects the double evaluation may cause all sorts of havoc.

Also, it's written in Go and #Ebitengine. One of those days I should try and compile it with #GopherJS. I know that around Go 1.11 days Ebitengine actually used to support GopherJS, but when GopherJS development fell behind, the support was dropped. Might be cool to bring it back.

A funny thing about GopherJS that took me a while to figure out: struct data is actually stored in what is technically a runtime representation of a pointer-to-struct type, and the struct object itself almost never exists

You'd say, but wait, no way it works without structs, surely they do exist and are represented by a plain old JavaScript object. Well, sort of. In reality, there is a pointer to a struct type that is represented by a JavaScript object. So when you write something like this:

type S struct { f int }
s := S{} // ← non-pointer type!
ptr := &s
reflect.TypeOf(s)

...it translates into something similar to:

var S = $newType(...., function() { this.f = 0 })
var s = new S.ptr()  // ← actually uses pointer constructor!
var ptr = s; // ← noop, it's already kind of a pointer.
reflect.TypeOf(new S(s))  // ← adds a wrapper with the correct type information when interfaces get involved.

This is counter-intuitive, but also makes sense because JavaScript's pass-by-reference semantics for objects is a lot like a pointer in Go, and GopherJS actually has to take extra steps to emulate value type semantics on top of it.

#golang #gopherjs

Before I became a GopherJS maintainer I had no idea how incredible of an achievement that would be: https://mastodon.social/@andrewrk/109621905821062620

Growing an active contributor community is hard. Even harder when it comes to folks willing to do the unfancy, grindy work such as triaging old issues — very few people consider it an enjoyable leisure activity, or are willing to do it regardless of not being paid. So hats off to the Zig authors for getting there, I'm sure it required a lot of persistence.

As for GopherJS, I am truly delighted that virtually all changes that went into 1.18.0-beta2 release came from contributors outside of the core maintainer group. Many of them are non-trivial too. We are still struggling to find volunteers for recurring tasks such as new Go version support, but hey, hopefully we'll get there.

#golang #gopherjs #opensource