golang

One of the things that Go community is the proudest of is the toolchain that is shipped alongside the language. Apart from dependency vendoring (this is in progress), you can run the whole show of taking your source code, testing it, cross-compiling, all the way to building a single binary output file. One of the lesser known tools that you should get familiar with is the one that just lists stuff...

go list

If you run the tool without going into its full capabilities, you may be a bit disappointed:

$ cd $GOPATH/src/github.com/spf13/cobra
$ go list
github.com/spf13/cobra

Wow, that's... little. Maybe we can get more information if we list the packages recursively:

$ go list ./...
github.com/spf13/cobra
github.com/spf13/cobra/cobra
github.com/spf13/cobra/cobra/cmd
github.com/spf13/cobra/doc

Now, this is actually more helpful, especially if you are running Go version prior to 1.9. Before the latest (at the time of writing this post) release whenever you ran go test or some linter tools on your project, you executed it both on your and vendored files (stored in vendor/ subdirectory). The problem was, that you may not want to test those or see if any of them do not follow go vet rules. This is where go list helped because you could list all the packages, grep-out all paths that contained vendor/ and pass that filtered list as input to eg. go test:

$ go test -race -v `go list ./... | grep -v /vendor/`

Now you don't have to do that, but that doesn't mean we're not going to appreciate go list anymore.

More than packages list

If you check out help for the tool, you might get a bit overwhelmed, since the message is quite large and includes snippets in Go with lengthy structs. Most of the message explains how you can use go templates as a parameter to get more details about your code. For example, if you want to list all imports used in a package you should use {{.Imports}} template and type:

$ go list -f {{.Imports}} .
[bytes fmt github.com/spf13/pflag io os path/filepath reflect sort strconv strings text/template unicode]

You can see those imports actually being used in the files, eg.:

$ cat cobra.go
package cobra

import (
        "fmt"
        "io"
        "reflect"
        "strconv"
        "strings"
        "text/template"
        "unicode"
)
...

You must remember, that go list does not check anything here, if we add some non-existing, unused import to one of the files, it will be returned in the list as well.

You can go further and ask what are all the dependencies that are being used in your app? You want to know not only about the packages you explicitly list, but also all the things that your dependencies depend on. For that, we need to use {{.Deps}}:

$ go list -f {{.Deps}} .
[bufio bytes context encoding/csv errors flag fmt github.com/spf13/pflag internal/cpu internal/nettrace internal/poll internal/race internal/singleflight io io/ioutil math math/rand net net/url os path/filepath reflect runtime runtime/cgo runtime/internal/atomic runtime/internal/sys sort strconv strings sync sync/atomic syscall text/template text/template/parse time unicode unicode/utf8 unsafe vendor/golang_org/x/net/route]

You can see that the list is much longer now. The output consists of one slice of strings, which might be a bit difficult to process, so let's join them with new-line character (note that we need to put our template in single-quotes because it contains spaces now):

$ go list -f '{{join .Deps "\n"}}' .
bufio
bytes
context
...
vendor/golang_org/x/net/route

Another useful thing is checking if the package comes from the standard library, which means that it's not actually vendored (since it comes with the language) - it can be done by checking {{.Standard}} flag on each package:

$ go list -f '{{.ImportPath}} - {{.Standard}}' `go list -f '{{join .Deps "\n"}}' .`
bufio - true
bytes - true
context - true
...
github.com/spf13/pflag - false
...
vendor/golang_org/x/net/route - true

Finally, let's list all recursive dependencies that do not come from standard library (using {{if not X}} helper):

$ go list -f '{{if not .Standard}}{{.ImportPath}}{{end}}' `go list -f '{{join .Deps "\n"}}' .`
github.com/spf13/pflag

Use case: gvt-unused

In the project I am currently working we are using gvt as a vendoring tool. Since we are making lots of refactorings all the time (who doesn't?) it's hard to track which dependencies are still being used, and which are just dangling entries in vendor/manifest file. In order to solve this problem, I created a small script called gvt-unused that parses that file, compares it with a recursive list of dependencies and suggests which ones could be safely removed. If you are using gvt as well, go ahead and play around with it.

Full source code of this tiny application is available on Github: github.com/slomek/gvt-unused.