Understanding the terraform HTTP backend

Terraform is one of the de-facto infrastructure-as-code tools out there. It keeps track of the described infrastructure state and tries to help you not shoot yourself in the foot. It offers a wide variety of backends to store the state, the most popular ones being S3 and keeping state locally (as a beginner). Among the options provided is for example a Postgres backend. However lately my friend Evangelos Balaskas reminded me of the more generic HTTP backend, with an excellent blog post that combines GitLab and the HTTP backend.

I wanted to know more about the HTTP backend for some time now and I thought I should try to write a toy implementation to figure it out. I’ve never written any serious code with Golang and it seemed like a good oportunity to do something more than Go by example.

This is the simplest terraform HTTP backend configuration that assumes a web server running locally:

terraform {
  backend "http" {
    address = ""
    lock_address = ""
    unlock_address = ""

The simplest web server that we could write in Golang has to be something like

package main

import (

func hello_terraform(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello\n")

func main() {
    http.HandleFunc("/", hello_terraform)
    http.ListenAndServe(":8090", nil)

Running terraform init results in the following error message:

 Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
Error refreshing state: 2 problems:

- Unsupported state file format: The state file could not be parsed as JSON: syntax error at byte offset 1.
- Unsupported state file format: The state file does not have a "version" attribute, which is required to identify the format version.

And ever if we modified the terraform_hello() function to return an empty JSON string we would still get an error about the version attribute. Note that we’d get the same errors with a simple nginx instead of our golang HTTP server too (and this is because we send back a 200 response which terraform cannot parse instead of a 404 or other that would make it think this is a new state). Changing however the function just a bit

func hello_terraform(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "{"version": 0\n")


Error refreshing state: Unsupported state file format: The state file uses JSON syntax but has a version number of zero. There was never a JSON-based state format zero, so this state file is invalid and cannot be processed.

So we change the version number to 1 and:

PS C:\Users\...\terraform-http-backend> .\terraform.exe init
Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

But we can even have the function serve a pre-existing file on disk:

switch req.Method {
	case "GET":
	   http.ServeFile(w, req, "tf.state")

This also has the added value that we are not served with an error about the JSON file; if the file to be served does not exist terraform will get a HTTP 404 and it knows how to handle it.

Terraform is now convinced that somwhere there’s an HTTP server storing its state. In fact you could say that we have just implemented a /dev/null terraform backend.

But we sure want more than that. We want our backend to be able to create, store, serve and update the state file whenever we access it. A function that does this would start looking like:

func hello_terraform(w http.ResponseWriter, req *http.Request) {

	fmt.Printf("%v %v\n", req.Method, req.URL.Path)

	switch req.Method {
	case "GET":
	   http.ServeFile(w, req, "tf.state")
	case "POST":
	   body, _ := ioutil.ReadAll(req.Body)
           b1 :=[]byte(string(body))
           os.WriteFile("tf.state", b1, 0644)
        case "LOCK":
        case "UNLOCK":
	    /* code */

There: 20 lines of code and stuff I learned from Go by example and you have the beginning of a working terraform HTTP backend. You can remove the fmt.Printf() line as it is there just to show you the HTTP dialogue that takes place. A proper application, using a web framework will deal with this better.

You can find a bigger example, based on the above and far from complete here:


You can expand from here and add whatever functionality you like, like database support, some locking mechanism of your liking, versioning via git or other system, RBAC, anything.

I am in my fourth attempt of learning Go

I am not particularly fond of Go, but I work with Kubernetes, and Go is to Kubernetes what C is to Unix. So after a point, you have to know some Go in order to understand more of Kubernetes’s implementation, design and other quirks (and why not, implement something too).

My first try was with The Go Programming Language. Up in the mountains, no Internet, just me, the book and my laptop. It felt like when reading K&R, only it didn’t. Times have changed. This is not the way. The Book is OK to be by your side, but I need something else.

The second time was when I was asked to write a review of Go Systems Programming by Mihalis Tsoukalos. I read the book through and through and submitted corrections on errors and such and my opinion of it to the author and the publisher. But life happened and I paused from immediate need for Go and Kubernetes. I forgot almost all of Go that I learned through the process.

I then tried the Exercism track for Go. I’ve tried many languages on Exercism and I consider it a valuable tool for everyone. It is just not for me. That effort faded quickly.

I am now in my fourth attempt. Mind you, I am not trying to become super proficient in Go, or even idiomatic Go. I want to have the relative ease to understand code that I read and to be able to write 100 lines of Go that work. This time I’ve chosen Go by Example. I’m following this tutorial one example per day. I had a small hiatus during the holidays, but today I came back. This looks like it may work.

That’s why I am writing the post. It is a sort of public commitment. Like Stickk without Stickk.


Last year I promised myself that I would revisit Haskell. Well I did not, so I did not escape the new year’s resolutions cliche. It was an interesting year though, considering that I left my country, worked for Intel, resigned and returned back to Greece and to my previous work.

So for this year I will promise myself something simpler, as a continuation of things I still do in 2017: simply improve my Go-fu. And yes, I also tried to learn Go and miserably failed. Let’s see about that too.