Download the internet with the Google Blobstore downloader

As many engineers know, a (Google Cloud Platform) project that isn’t in use, is bound to be thrown away one day… probably in the near future… and all data stored in the Blobstore will be lost… Forever! So what to do with important data?
Let’s start from the beginning.

It was early in the year 2013 when we at Q42 started using our self built declaration tool. We were all very enthusiastic as this was the end of our pile of receipts and frustration about the mess. With this nice little tool we just took a photograph of our receipts and threw them away afterwards. It was built in Java and hosted on Google AppEngine, thus using Google Blobstore.

Now it was not so early this year when, during our recent streak of make-or-buy decisions, we decided to start doing our declarations with Xpenditure. But before we could move from our old carefully crafted tool, we needed a backup of all our data, for possible future tax audits.

Where there is Azure CLI for the Azure Blobstore and GSUtils for the Google Cloud Storage to manage your files, there is no such thing for the Google Blobstore. I realized we needed a backup, offline and in our own safe-deposit. Because as soon as we didn’t use this Google Cloud Platform project anymore, the information would be lost someday and will most likely not survive the term set by the tax authorities.

So I was left with two choices: download all blobs by hand or write a little piece of code to do it for you… Of course I chose the latter.
While writing the code I wanted other people to be able to use it themselves and decided to create a little tool for every developer to use.

So without further ado I present, the Blobstore downloader.

Use it to download all files from every Google Blobstore you have access to, alter the code to download your files to Google Cloud Storage or fork the repository and help to make it better!

Thanks to Christiaan and Bart for their help and code reviews.

Download your Blobstore
Link: https://github.com/Q42/download-blobstore
Readme: https://github.com/Q42/download-blobstore/blob/master/README.md