Type Mapping Rules

Because MessagePack uses a schema-less, polymorphic type system, and Go is a strongly-typed language, any Go implementation of MessagePack serialization will have to make choices about how Go types map onto MessagePack types, and vice-versa. This document aims to explain the rules that msgp uses, and the justifications behind them.

Numerical Precision and Overflow

Rule 1: No Overflow

msgp always attempts to encode Go values in the smallest wire representation possible without any loss in numerical precision. For example, even though a Go int is 64 bits on 64-bit hardware, the encoding of int(5) is one byte on the wire, and it will still be 5 when decoded.

As a consequence of this rule, msgp will never let you decode a value that would overflow the object you are decoding into. For instance, if you use msgp.ReadInt16Bytes() or (*Reader).ReadInt16() to read out an integer value, the method will only succeed if the value of the integer is between math.MinInt16 and math.MaxInt16. For clarity's sake, here is the actual code for (*Reader).ReadInt16():

// ReadInt16 reads an int16 from the reader
func (m *Reader) ReadInt16() (i int16, err error) {
	var in int64
	in, err = m.ReadInt64()
	if in > math.MaxInt16 || in < math.MinInt16 {
		err = IntOverflow{Value: in, FailedBitsize: 16}
		return
	}
	i = int16(in)
	return
}

Note that all the methods that read int64 values will never return overflow errors, since MessagePack does not support integers wider than 64 bits.

Tip: Use int/uint or int64/uint64 when you cannot be sure of the magnitude of the encoded value.

Rule 2: No Loss of Precision

msgp will always encode a Go float32 as a 32-bit IEEE-754 float, and a float64 as a 64-bit IEEE-754 float.

When decoding, it is legal to decode a 32-bit float on the wire as a Go float64, but the opposite is illegal. This is to avoid the possibility of losing numerical precision.

Tip: Don't mix-and-match; pick either float32 or float64 and use it everywhere.

Rule 3: Sign Matters

msgp will not allow a value that is a uint on the wire to be decoded into an int, and vice-versa.

The justification behind this is to prevent applications from failing sporadically because one implementation encodes int values and the other decodes uint values, for example. Those types are not strictly compatible, and thus it is treated as a type error.

(This is unlike the floating-point conversion rules, as neither uint nor int is a strict sub- or super-set of the other.)

Tip: Use mostly signed integers.

Structs

Like JSON, MessagePack has no notion of strongly-typed data structures. msgp encodes Go struct objects as MessagePack maps.

TODO

Null

TODO