Introduction

By default, Go's json.Unmarshal silently ignores JSON fields that do not map to struct fields. This behavior can cause subtle bugs where API clients send typos in field names or send deprecated fields that the server no longer processes, and the server silently accepts the request without the intended data. For example, a client sending {"usernmae": "john"} instead of {"username": "john"} results in an empty username being stored with no error. Using json.Decoder with DisallowUnknownFields() enforces strict schema validation and catches these errors early.

Symptoms

API accepts requests with typoed or extra fields silently:

go type User struct { Username string json:"username" Email string json:"email"` }

var u User json.Unmarshal([]byte({"usernmae": "john", "email": "john@example.com"}), &u) // u.Username is "" - typo silently ignored! ```

In production, this manifests as: - Users created with empty required fields - Configuration values not applied because of key typos - Feature flags not enabled because flag names were misspelled - Data silently dropped during ETL processes

Common Causes

  • Default json.Unmarshal behavior: Unknown fields are silently discarded
  • Client sends deprecated fields: API evolution where old clients send fields the server no longer reads
  • Typos in JSON keys: usernmae instead of username, emial instead of email
  • Extra fields from client SDK: Client SDK adds metadata fields the server does not expect
  • No input validation layer: Relying solely on Go's type system without field validation

Step-by-Step Fix

Step 1: Use json.Decoder with DisallowUnknownFields

```go // For HTTP request bodies func (h *Handler) CreateUser(w http.ResponseWriter, r *http.Request) { var user User decoder := json.NewDecoder(r.Body) decoder.DisallowUnknownFields()

if err := decoder.Decode(&user); err != nil { http.Error(w, fmt.Sprintf("invalid request: %v", err), http.StatusBadRequest) return }

// user is now validated - no unknown fields accepted } ```

The error message is specific about which field is unknown:

bash
invalid request: json: unknown field "usernmae"

Step 2: Create a reusable strict unmarshal function

```go // StrictUnmarshal strictly unmarshals JSON, rejecting unknown fields. func StrictUnmarshal(data []byte, v interface{}) error { dec := json.NewDecoder(bytes.NewReader(data)) dec.DisallowUnknownFields() return dec.Decode(v) }

// Usage var config AppConfig if err := StrictUnmarshal(rawJSON, &config); err != nil { return fmt.Errorf("invalid config: %w", err) } ```

Step 3: Handle API versioning with unknown fields

When you need to accept extra fields for forward compatibility, use a two-pass approach:

go type APIRequest struct { Version int json:"version" Username string json:"username" Email string json:"email"` }

func (h *Handler) HandleRequest(w http.ResponseWriter, r *http.Request) { // First, read raw body body, err := io.ReadAll(r.Body) if err != nil { http.Error(w, "read error", http.StatusBadRequest) return } defer r.Body.Close()

// Parse with a generic map to check for unknown fields var raw map[string]interface{} if err := json.Unmarshal(body, &raw); err != nil { http.Error(w, "invalid JSON", http.StatusBadRequest) return }

// Known fields for v1 knownFields := map[string]bool{ "version": true, "username": true, "email": true, }

// Log unknown fields as warnings (do not reject) for key := range raw { if !knownFields[key] { log.Printf("WARNING: unknown field %q in request (client may be using newer API version)", key) } }

// Parse into struct var req APIRequest if err := json.Unmarshal(body, &req); err != nil { http.Error(w, fmt.Sprintf("invalid request: %v", err), http.StatusBadRequest) return } } ```

Step 4: Test for strict unmarshaling

go func TestStrictUnmarshal(t *testing.T) { tests := []struct { name string input string wantErr bool }{ { name: "valid request", input: {"username": "john", "email": "john@example.com"}, wantErr: false, }, { name: "typo in field name", input: {"usernmae": "john", "email": "john@example.com"}, wantErr: true, }, { name: "extra unknown field", input: {"username": "john", "email": "john@example.com", "admin": true}`, wantErr: true, }, }

for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { var user User err := StrictUnmarshal([]byte(tt.input), &user) if (err != nil) != tt.wantErr { t.Errorf("StrictUnmarshal() error = %v, wantErr %v", err, tt.wantErr) } }) } } ```

Prevention

  • Always use json.Decoder.DisallowUnknownFields() for HTTP request parsing
  • Add a StrictUnmarshal helper function to your project's validation utilities
  • Use go-json (github.com/goccy/go-json) as a drop-in replacement that supports strict mode with struct tags
  • Add integration tests that send requests with extra fields to verify rejection
  • Document the strict behavior in your API specification so clients know to expect 400 on unknown fields
  • Use OpenAPI validation middleware (like kin-openapi) as an additional validation layer