Parsing JSON in Golang

What happens when Go’s type safety gets in the way of working with arbitrary JSON data?

Introduction

Recently, I wanted to write a simple service (in Golang). This service is an API that accepts some JSON data and performs a few simple actions.

Parsing arbitrary JSON in Golang

Say we would like to parse a simple JSON object with the following format:

{"string": "value", "bool": true}

Before I started writing Go, my main programming language for dealing with JSON data was Python, for which a simple solution would be to do the following:

import json

j = json.loads('{"string":"value", "bool":true}')
print(j['string'], j['bool'])    # value True

How can we do the same thing in Go? Naturally, someone that has been accustomed to the pattern above and not too familiar with Go, would gravitate towards a solution using map[string]interface{} (at least I did when I was getting started):

// not-a-good-idea.go
package main

import (
    "encoding/json"
    "fmt"
    "log"
)

func main() {
    s := []byte(`{"string":"value","bool":true}`)
    var j map[string]interface{}

    if err := json.Unmarshal(s, j); err != nil {
        log.Fatalf("Failed to parse JSON: %q", err)
    }
    fmt.Println(j["string"], j["bool"])
}

This works, and we do get our JSON data regardless of the format, but using it afterwards (and dealing with lots of interface{} types) is incredibly hard, prone to errors, and throws type-safety out of the window.

Parsing JSON in Golang

Instead, the proper way to deal with JSON data in Go is to define a struct and use tags:

// a-better-idea.go
package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// MyStruct is used to parse the JSON object.
type MyStruct struct {
    String string `json:"string"`
    Bool bool `json:"bool"`
}

func main() {
    s := []byte(`{"string":"value","bool":true}`)
    var j MyStruct

    if err := json.Unmarshal(s, &j); err != nil {
        log.Fatalf("Failed to parse JSON: %q", err)
    }
    fmt.Println(j.String, j.Bool)
}

This is already much better than before. It reads much easier, your favourite IDE will help you with autocompletion. More complex types are also supported, e.g. []string. If we try to parse an invalid JSON format (e.g. {"string":4123}), then json.Unmarshal will return an error like this:

2021/12/31 12:32:24 Failed to parse JSON: "json: cannot unmarshal number into Go struct field MyStruct.string of type string"

This is neat! We can define (and document) our structs, using standard Go types, and we get parsing, validation and type-checking for free.

So what can Go wrong?

Back to our problem

What was written above works great, with one caveat: We are very strict with the kind of data that we are allowed to use. In my case, the Go service was aimed to replace a Flask server. One of the JSON input fields, (we will call this input-field from now on) was (documented as) a boolean value, but in truth the client would send one of the following:

{"input-field": false}

{"input-field": "yes"}

The problem becomes apparent here: How do we handle input-field in our Go struct? We cannot use either bool or string, since one of them will always fail.

How do we get out of this situation?

The ugly solution: Back to interface

Again, the first instinct is to backtrack on using types, and fall back to interface{} and adding some ugly code to handle string, bool and nil.

// not-a-good-idea.go

package main

import (
    "encoding/json"
    "fmt"
    "log"
)

// MyStruct is used to parse the JSON object.
type MyStruct struct {
    InputField interface{} `json:"input-field"`
}

func main() {
    s := []byte(`{"input-field":"yes"}`)
    var j MyStruct

    if err := json.Unmarshal(s, &j); err != nil {
        log.Fatalf("Failed to parse JSON: %q", err)
    }

    // Handle nil
    if j.InputField == nil {
        j.InputField = false
    }
    if s, ok := j.InputField.(string); ok {
        if s == "yes" {
            // handle true
        }
    } else if v, ok := j.InputField.(bool); ok {
        if v {
            // handle true
        }
    }
}

This obviously comes with many caveats. interface{} will parse anything, so we lose type-safety. The code to handle each case becomes quite ugly, and has been left incomplete on purpose (consider what happens if {"input-field": [2]} is passed).

So what can we do?

Using the json.Unmarshaler interface

The encoding/json package offers the json.Unmarshaler interface. This allows us to unmarshal (parse) any custom type that we want, by implementing the method UnmarshalJSON([]byte) error.

This is what we will do in this case. We define a new custom type (aptly named InputFieldType), which is just an alias for a boolean value. We will then implement the json.Unmarshaler interface for our type. Here is the complete code:

// solution.go
package main

import (
    "bytes"
    "encoding/json"
    "fmt"
    "log"
)

// InputFieldType is a type alias for bool types, but implements json.Unmarshaler.
type InputFieldType bool

// UnmarshalJSON implements the json.Unmarshaler interface. It accepts the raw JSON value
// that was parsed from the JSON data. It returns nil on success, an error otherwise.
//
// Note that "yes" is the quoted string, whereas true and false are not.
func (v *InputFieldType) UnmarshalJSON(b []byte) error {
    switch {
    case bytes.Equal(b, []byte(`"yes"`)) || bytes.Equal(b, []byte(`true`)):
        *v = true
    case bytes.Equal(b, []byte(`false`)):
        // Not really needed for bool values (the default value is false).
        *v = false
    default:
        return fmt.Errorf("could not parse %q for input field type", b)
    }
}

// MyStruct is used to parse the JSON object.
type MyStruct struct {
    InputField InputFieldType `json:"input-field"`
}

func main() {
    s := []byte(`{"input-field":"yes"}`)
    var j MyStruct

    if err := json.Unmarshal(s, &j); err != nil {
        log.Fatalf("Failed to parse JSON: %q", err)
    }

    if j.InputField {
        // handle true
    }
}

Overall, this approach is much cleaner for the following reasons:

  • All the logic of parsing our custom type is encapsulated within InputFieldType. Users only need to know about json.Unmarshal, they do not have to handle arbitrary types every time they parse a MyStruct object.
  • Our type is just an alias for a bool value.
  • We retain error checking and proper handling for invalid values.
  • We can write proper unit tests for parsing our custom type.