go1.20升级风波。。。

用户3904122

发布于 2023-09-02 10:19:39

5690

发布于 2023-09-02 10:19:39

文章被收录于专栏：光华路程序猿

unexpected fault address 0x0
fatal error: fault
unexpected fault address 0x0
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0x478dbf]

goroutine 49 [running]:
runtime.throw({0x1f08eaf?, 0x194665d?})
 /usr/local/go/src/runtime/panic.go:1047 +0x5d fp=0xc0015ad388 sp=0xc0015ad358 pc=0x445afd
runtime.sigpanic()
 /usr/local/go/src/runtime/signal_unix.go:851 +0x28a fp=0xc0015ad3e8 sp=0xc0015ad388 pc=0x45c82a
aeshashbody()
 /usr/local/go/src/runtime/asm_amd64.s:1370 +0x39f fp=0xc0015ad3f0 sp=0xc0015ad3e8 pc=0x478dbf
runtime.mapiternext(0xc000cb3200)
 /usr/local/go/src/runtime/map.go:936 +0x2eb fp=0xc0015ad460 sp=0xc0015ad3f0 pc=0x41eb6b
runtime.mapiterinit(0xc0015ad4f8?, 0xf51c96?, 0x1b57420?)
 /usr/local/go/src/runtime/map.go:863 +0x236 fp=0xc0015ad480 sp=0xc0015ad460 pc=0x41e836
reflect.mapiterinit(0xc0015ad4f8?, 0xf53727?, 0xc000cb3200?)
 /usr/local/go/src/runtime/map.go:1375 +0x19 fp=0xc0015ad4a8 sp=0xc0015ad480 pc=0x474e79
github.com/modern-go/reflect2.(*UnsafeMapType).UnsafeIterate(...)
 /root/go/pkg/mod/github.com/modern-go/reflect2@v1.0.1/unsafe_map.go:112
github.com/json-iterator/go.(*sortKeysMapEncoder).IsEmpty(0xc0015ad570?, 0xc000fe8cd0?)
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_map.go:333 +0x28 fp=0xc0015ad4e8 sp=0xc0015ad4a8 pc=0xf451c8
github.com/json-iterator/go.(*structFieldEncoder).IsEmpty(0xc001647260, 0x18ab3c2?)
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:118 +0x42 fp=0xc0015ad508 sp=0xc0015ad4e8 pc=0xf51e42
github.com/json-iterator/go.(*structEncoder).Encode(0xc001647350, 0xc00082a1d8?, 0xc0011b0f60)
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_struct_encoder.go:148 +0x29b fp=0xc0015ad5f0 sp=0xc0015ad508 pc=0xf521db
github.com/json-iterator/go.(*OptionalEncoder).Encode(0xc000190320?, 0x0?, 0x0?)
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect_optional.go:70 +0x9d fp=0xc0015ad640 sp=0xc0015ad5f0 pc=0xf4989d
github.com/json-iterator/go.(*onePtrEncoder).Encode(0xc000722ba0, 0xc000fe8cd0, 0xc000fd35f0?)
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:219 +0x82 fp=0xc0015ad678 sp=0xc0015ad640 pc=0xf3cae2
github.com/json-iterator/go.(*Stream).WriteVal(0xc0011b0f60, {0x1d20640, 0xc000fe8cd0})
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/reflect.go:98 +0x158 fp=0xc0015ad6e8 sp=0xc0015ad678 pc=0xf3bdf8
github.com/json-iterator/go.(*frozenConfig).Marshal(0xc000190320, {0x1d20640, 0xc000fe8cd0})
 /root/go/pkg/mod/github.com/json-iterator/go@v1.1.10/config.go:299 +0xc9 fp=0xc0015ad780 sp=0xc0015ad6e8 pc=0xf332a9

今天尝试把工程从古老的1.13版本升级到最新的1.20，打算坐下泛型的快车。升级之后运行时立马就panic掉了。打印堆栈日志，发现是内部common库依赖的github.com/json-iterator/go@v1.1.10导致的。翻了翻源码，问题出在github.com/json-iterator/go@v1.1.10/reflect_map.go:333

type sortKeysMapEncoder struct {
 mapType     *reflect2.UnsafeMapType
 keyEncoder  ValEncoder
 elemEncoder ValEncoder
}
func (encoder *sortKeysMapEncoder) IsEmpty(ptr unsafe.Pointer) bool {
 iter := encoder.mapType.UnsafeIterate(ptr) //line 333
 return !iter.HasNext()
}

这里的mapType 使用了reflect2（github.com/modern-go/reflect2@v1.0.1）,追根溯源

func (type2 *UnsafeMapType) UnsafeIterate(obj unsafe.Pointer) MapIterator {
 return &UnsafeMapIterator{
  hiter:      mapiterinit(type2.rtype, *(*unsafe.Pointer)(obj)),//问题根因
  pKeyRType:  type2.pKeyRType,
  pElemRType: type2.pElemRType,
 }
}

这里reflect2弄了个骚操作

// m escapes into the return value, but the caller of mapiterinit
// doesn't let the return value escape.
//go:noescape
//go:linkname mapiterinit reflect.mapiterinit
func mapiterinit(rtype unsafe.Pointer, m unsafe.Pointer) *hiter

利用go:linkname这个编译器指令，骗过编译器检查直接使用了reflect.mapiterinit。其具体含义：

mapiterinit 函数是一个与 reflect 包中的 mapiterinit 函数相关联的低级函数，用于初始化 map 的迭代器。它利用了 Go 编译器的 go:linkname 指令，以便在 reflect2 包中直接调用 Go 运行时（runtime）中实现的 reflect.mapiterinit 函数。

而go1.18版本将reflect.map.iterinit做了调整

//1.18之前
// m escapes into the return value, but the caller of mapiterinit
// doesn't let the return value escape.
//go:noescape
func mapiterinit(t *rtype, m unsafe.Pointer) unsafe.Pointer

//1.18
//go:noescape
func mapiterinit(t *rtype, m unsafe.Pointer, it *hiter)

显然go1.18将mapiterinit新增了一个参数，但是因为reflect2使用了骚操作，骗过了编译器，但运行时panic。

解决办法:

目前json-iterator和reflect2均已兼容go1.18升级到最新版本即可