Using Go and fsnotify to watch a directory for new files

April 8, 2017

I’ve been very busy at work, mastering infrastructure as code tools and work-flows, but somehow I’ve found the time to start learning Go!

I had already learned some python and even used it for a couple of small projects at work. Python is a great language with beautiful syntax that makes coding a joy, but it’s slow as molasses and managing dependencies is no fun. Docker can help here (that’s how I solved my dependency and environment parity issues), but you just can’t beat static binaries for portability and ease of deployment, not to mention speed.

Go is a newer language compared to Python or Java, but it has quickly gained popularity. Most of my favorite DevOps tools are written in Go: Docker, Kubernetes, Consul, Nomad, Packer, Terraform … the list goes on. When new open source DevOps tools appear, it’s more surprising at this point when they’re not written in Go!

For a previous project, I wrote a little tool in python that watches for newly uploaded files and feeds them into an ETL pipeline. It wasn’t pretty but it got the job done. Since that’s a problem I can wrap my head around, I decided to build the same thing in Go, or at least the file system watching part. Moving files around the filesystem is fairly trivial in any language.

The code:

I’ve since created a more powerful version of this which I hope to finish up soon and post here, but I figured this little snippet would be useful for anyone just getting started with Go or fsnotify

Tags