Introduction

By default, json.Unmarshal silently ignores any JSON fields that do not have a matching struct field. This is convenient for forward compatibility but dangerous for API validation - a typo in a field name or an unexpected API response change goes undetected, leading to zero values being used instead of the expected data.

Symptoms

  • Struct fields contain zero values even though JSON has data
  • API contract violations not caught (e.g., "usernmae" instead of "username")
  • Downstream failures due to missing data that was silently dropped
  • Schema changes in upstream API not detected
  • Tests pass with incorrect JSON because fields are silently ignored

go type User struct { ID int json:"id" Name string json:"name" Email string json:"email"` }

// API sends: {"id": 1, "usernmae": "John", "email": "john@example.com"} // Typo: "usernmae" instead of "name" var user User json.Unmarshal(data, &user) // user.Name = "" (empty, no error reported!) ```

Common Causes

  • Typos in JSON field names from upstream APIs
  • API version changes adding or renaming fields
  • Missing struct tags causing field name mismatch (camelCase vs snake_case)
  • Nested structs with different JSON conventions
  • Testing with mock data that does not match production schema

Step-by-Step Fix

  1. 1.Use json.Decoder with DisallowUnknownFields:
  2. 2.```go
  3. 3.import (
  4. 4."encoding/json"
  5. 5."strings"
  6. 6.)

func strictUnmarshal(data []byte, v interface{}) error { dec := json.NewDecoder(strings.NewReader(string(data))) dec.DisallowUnknownFields() // Key: rejects unknown fields return dec.Decode(v) }

// Usage var user User err := strictUnmarshal(data, &user) if err != nil { // json: unknown field "usernmae" return err } ```

  1. 1.Validate with custom UnmarshalJSON:
  2. 2.```go
  3. 3.type User struct {
  4. 4.ID int json:"id"
  5. 5.Name string json:"name"
  6. 6.Email string json:"email"
  7. 7.}

func (u *User) UnmarshalJSON(data []byte) error { // First, use strict decoding dec := json.NewDecoder(bytes.NewReader(data)) dec.DisallowUnknownFields()

// Alias to avoid infinite recursion type Alias User aux := &struct{ *Alias }{Alias: (*Alias)(u)}

if err := dec.Decode(aux); err != nil { return fmt.Errorf("invalid user JSON: %w", err) }

// Additional validation if u.Name == "" { return errors.New("user name is required") } if u.Email == "" { return errors.New("user email is required") } return nil } ```

  1. 1.Use a validation library for schema enforcement:
  2. 2.```go
  3. 3.import "github.com/go-playground/validator/v10"

var validate = validator.New()

func ParseAndValidate(data []byte) (*User, error) { dec := json.NewDecoder(bytes.NewReader(data)) dec.DisallowUnknownFields()

var user User if err := dec.Decode(&user); err != nil { return nil, err }

if err := validate.Struct(user); err != nil { return nil, fmt.Errorf("validation failed: %w", err) }

return &user, nil } ```

  1. 1.Detect field changes in tests:
  2. 2.```go
  3. 3.func TestStrictJSONUnmarshal(t *testing.T) {
  4. 4.tests := []struct {
  5. 5.name string
  6. 6.data string
  7. 7.wantErr bool
  8. 8.}{
  9. 9.{"valid", {"id":1,"name":"John","email":"j@e.com"}, false},
  10. 10.{"unknown field", {"id":1,"name":"John","age":30}, true},
  11. 11.{"typo field", {"id":1,"usernmae":"John"}, true},
  12. 12.{"missing required", {"id":1}, false}, // Decodes, validation catches
  13. 13.}

for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { dec := json.NewDecoder(strings.NewReader(tt.data)) dec.DisallowUnknownFields() var user User err := dec.Decode(&user) if (err != nil) != tt.wantErr { t.Errorf("got err = %v, wantErr %v", err, tt.wantErr) } }) } } ```

Prevention

  • Always use DisallowUnknownFields() for API request parsing
  • Run contract tests against production API responses
  • Use OpenAPI/JSON Schema validation in addition to struct unmarshaling
  • Log unknown fields at WARN level even if not rejecting them
  • Set up monitoring for JSON decode errors in production
  • Use github.com/tidwall/gjson for flexible JSON parsing when strictness is not needed