[go-jsonnet] stack overflow when trying to format string longer than ~48kb

[discordianfish]

When trying to format strings longer than 48409 bytes, jsonnet fails with the following error:

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc02bd004a8 stack=[0xc02bd00000, 0xc04bd00000]
fatal error: stack overflow

runtime stack:
runtime.throw(0x11a1ee1, 0xe)
	/usr/local/Cellar/go/1.14.5/libexec/src/runtime/panic.go:1116 +0x72
runtime.newstack()
	/usr/local/Cellar/go/1.14.5/libexec/src/runtime/stack.go:1034 +0x6ce
runtime.morestack()
	/usr/local/Cellar/go/1.14.5/libexec/src/runtime/asm_amd64.s:449 +0x8f

goroutine 1 [running]:
github.com/google/go-jsonnet.(*interpreter).evaluate(0xc000054140, 0x11ff320, 0x1390ac0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/johannes_ziemke/dev/go/src/github.com/google/go-jsonnet/interpreter.go:268 +0x4471 fp=0xc02bd004b8 sp=0xc02bd004b0 pc=0x111c1b1
github.com/google/go-jsonnet.(*interpreter).evaluate(0xc000054140, 0x11fee40, 0x1435620, 0x1, 0x1, 0xc00d38a780, 0xc00d38a700, 0x0)
	/Users/johannes_ziemke/dev/go/src/github.com/google/go-jsonnet/interpreter.go:423 +0x6a8 fp=0xc02bd00eb8 sp=0xc02bd004b8 pc=0x11183e8
...

This can be reproduced by using this jsonnet:

{
  foo: (importstr 'bar.txt') % {},
}

And running:

$ dd if=/dev/zero of=bar.txt bs=1 count=48409
48409+0 records in
48409+0 records out
48409 bytes transferred in 0.156383 secs (309554 bytes/sec)
$ jsonnet test.jsonnet > /dev/null
$ echo $?

...still works but one byte more and it crashes:

$ dd if=/dev/zero of=bar.txt bs=1 count=48410
48410+0 records in
48410+0 records out
48410 bytes transferred in 0.157608 secs (307154 bytes/sec)
$ jsonnet test.jsonnet > /dev/null
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc020120748 stack=[0xc020120000, 0xc040120000]
fatal error: stack overflow
...

Not sure whether to call this bug report or feature request, but it limits jsonnet use for templating larger config files.

[sbarzowski]

Thanks for reporting. We're aware that the performance of std.format is not very good and I would like to do something about it.

In the meantime, maybe you can try a different way to handle templating. The operator % implements a pretty fancy Python-like formatting which is an overkill in many cases. In many cases you could get away even with something as simple as std.substr.

I rolled out simpler custom templating once, which should be way faster. I'll try to find it later.