{"id":528,"date":"2017-12-01T08:36:03","date_gmt":"2017-12-01T08:36:03","guid":{"rendered":"http:\/\/buklijas.info\/blog\/?p=528"},"modified":"2017-11-23T12:57:46","modified_gmt":"2017-11-23T12:57:46","slug":"automatic-backup-of-git-repositories-to-dropbox-with-python","status":"publish","type":"post","link":"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/","title":{"rendered":"Automatic backup of git repositories to Dropbox with Python"},"content":{"rendered":"

Published on:<\/strong> 01.12.2017<\/p>\n

Intro<\/h4>\n

I will show how to upload files to Dropbox from Python<\/strong> code.<\/p>\n

Why do I need this?<\/h4>\n

Currently, I am only using WebFaction<\/a> for all my web services and also as my private git server.<\/p>\n

I wanted to make an automatic backup of my git repositories to Dropbox<\/strong>.<\/p>\n

Dropbox App<\/h4>\n

I order to upload files to Dropbox you need to have an access token<\/strong>.<\/p>\n

And for the access token, you need to register your app on DBX platform<\/strong>.<\/p>\n

All of this must be done on Dropbox website.<\/p>\n

The first step is to go to https:\/\/www.dropbox.com\/developers\/apps\/<\/a> and press “Create App” button.<\/p>\n

Step 1<\/h6>\n

\"Just<\/p>\n

Just click “Create app” button<\/p>\n

Step 2<\/h6>\n

\"New<\/p>\n

We will use Dropbox API.<\/p>\n

We will choose “App folder” because we will just upload one backup to Dropbox, we do not need full access to all our files.<\/p>\n

Name your app and click “Create App” button.<\/p>\n

Step 3<\/h6>\n

\"Use<\/p>\n

We will use defaults settings, here we will get the access token so, click “Generate access token” button.<\/p>\n

Step 4<\/h6>\n

\"Access<\/p>\n

Now you have your access token, you will need it in your code, so copy it.<\/p>\n

Step 5<\/h6>\n

\"Dropbox<\/p>\n

Now we have our “my_git_backup” Dropbox app.<\/p>\n

pip install<\/h4>\n
\npip install dropbox\npip install fabric\n<\/pre>\n
It is always recommended to use virtual environments<\/strong> inside python.<\/p>\n
At least I use them always.<\/p>\n
Code<\/h3>\n
\nfrom fabric.api import local, lcd\nimport datetime\nimport dropbox\n\nremote_directory = 'full path to root git repository'\nNUMBER_OF_BACKUP_TO_KEEP = 10\n<\/pre>\n
I am using fabric to make my life(code) easier.<\/p>\n
I use fabric<\/strong> every time when I am calling CLI command from Python<\/strong>.<\/p>\n
I will explain NUMBER_OF_BACKUP_TO_KEEP<\/code> later, I use it at the end of the program.<\/p>\n
\ndef git_backup_to_dropbox():\n    with lcd(remote_directory):\n<\/pre>\n
All code that follows is inside with lcd(remote_directory):<\/code> Python context manager<\/strong>.<\/p>\n
The context manager is used so all code that follows is executed inside remote_directory<\/code> directory.<\/p>\n
\nname_of_backup = datetime.datetime.today().strftime('%Y-%m-%d_%H%M%S')\nname_of_backup += '_git_backup.zip'\n\nlocal('zip --password LAME_PASSWORD -r ' + name_of_backup + ' *.git')\n<\/pre>\n
Name of backup file will be YYYYMMDD_HHMM_git_backup.zip<\/code> where upper case letters are date and time when a program was executed.<\/p>\n
Eg. 20171121_1856_git_backup.zip<\/code> so that we know from when is this backup file.<\/p>\n
For making an actual backup, zip CLI command<\/strong> is used, we are only doing the backup of files that end on *.git<\/code> (in my case only git repositories).<\/p>\n
I also have LAME_PASSWORD<\/strong> for basic protection.<\/p>\n
This is why I used fabric, just by calling local()<\/code> function you can execute CLI commands.<\/p>\n
\ndbx = dropbox.Dropbox('YOUR ACCESS TOKEN FROM DROPBOX!!!')\n\nwith open(remote_directory + '\/' + name_of_backup, 'rb') as f:\n    dbx.files_upload(f.read(), '\/' + name_of_backup )\n\nlocal('rm ' + name_of_backup)\n<\/pre>\n
The first line is the opening connection to your Dropbox application<\/strong>, you need to add your own access token as an argument.<\/p>\n
Next two lines are for upload, you are: opening file, reading it and uploading bytes to Dropbox.<\/p>\n
In Dropbox documentation<\/a> is mention that this is only working for files till 150MB<\/strong> in size.<\/p>\n
With last line program is deleting the local backup.<\/p>\n
\nall_backup_files = []\nfor entry in dbx.files_list_folder('').entries:\n    all_backup_files.append(entry.name)\n\nfor file in sorted(all_backup_files[:-NUMBER_OF_BACKUP_TO_KEEP]):\n    dbx.files_delete('\/' + file) \n<\/pre>\n
First for loop is getting all files from your backup folder in a list.<\/p>\n
Second for loop is deleting all files except, last few files.<\/p>\n
How many files to keep<\/strong> (otherwise we need manually to delete old backup files) is define in NUMBER_OF_BACKUP_TO_KEEP from the beginning of the code.<\/p>\n
I keep it at 10, more than that I do not need.<\/p>\n
Because we have date and time in our filename we can use Python sort function to sort files by when the backup was done.<\/p>\n
The program can be run with
\nfab -f fabfile git_backup_to_dropbox<\/code><\/p>\n
First is fab<\/code> because we used fabric, fabfile<\/code> because fabfile.py<\/code> is file of our source code and git_backup_to_dropbox<\/code>is name of the function that we are executing from fabfile.py<\/code> file.<\/p>\n
How I run this automaticaly<\/h4>\n
I personally run this command from crontab once per day<\/strong>.
\n35 02 * * * \/home\/user_name\/code\/venv\/bin\/fab -f \/home\/user_name\/code\/fabfile git_backup_to_dropbox<\/code><\/p>\n
Conclusion<\/h4>\n
This can be used for backup of any folder as zip file automaticaly to Dropbox<\/strong>.<\/p>\n
For any questions, please write them in comments.<\/p>\n","protected":false},"excerpt":{"rendered":"
Published on: 01.12.2017 Intro I will show how to upload files to Dropbox from Python code. Why do I need this? Currently, I am only using WebFaction for all my web services and also as my private git server. I wanted to make an automatic backup of my git repositories to Dropbox. Dropbox App I […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[27],"tags":[31,4],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"yoast_head":"\nAutomatic backup of git repositories to Dropbox with Python - Sasa Buklijas<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Automatic backup of git repositories to Dropbox with Python - Sasa Buklijas\" \/>\n<meta property=\"og:description\" content=\"Published on: 01.12.2017 Intro I will show how to upload files to Dropbox from Python code. Why do I need this? Currently, I am only using WebFaction for all my web services and also as my private git server. I wanted to make an automatic backup of my git repositories to Dropbox. Dropbox App I […]\" \/>\n<meta property=\"og:url\" content=\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/\" \/>\n<meta property=\"og:site_name\" content=\"Sasa Buklijas\" \/>\n<meta property=\"article:published_time\" content=\"2017-12-01T08:36:03+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-23T12:57:46+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/buklijas.info\/blog\/wp-content\/uploads\/2017\/11\/1_create_app.png\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\">\n\t<meta name=\"twitter:data1\" content=\"Sasa Buklijas\">\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\">\n\t<meta name=\"twitter:data2\" content=\"3 minutes\">\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"http:\/\/buklijas.info\/blog\/#website\",\"url\":\"http:\/\/buklijas.info\/blog\/\",\"name\":\"Sasa Buklijas\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":\"http:\/\/buklijas.info\/blog\/?s={search_term_string}\",\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/i0.wp.com\/buklijas.info\/blog\/wp-content\/uploads\/2017\/11\/1_create_app.png?fit=1441%2C273\",\"width\":1441,\"height\":273},{\"@type\":\"WebPage\",\"@id\":\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/#webpage\",\"url\":\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/\",\"name\":\"Automatic backup of git repositories to Dropbox with Python - Sasa Buklijas\",\"isPartOf\":{\"@id\":\"http:\/\/buklijas.info\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/#primaryimage\"},\"datePublished\":\"2017-12-01T08:36:03+00:00\",\"dateModified\":\"2017-11-23T12:57:46+00:00\",\"author\":{\"@id\":\"http:\/\/buklijas.info\/blog\/#\/schema\/person\/780025d597f1c5df3cc156eaffc8c561\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"http:\/\/buklijas.info\/blog\/2017\/12\/01\/automatic-backup-of-git-repositories-to-dropbox-with-python\/\"]}]},{\"@type\":\"Person\",\"@id\":\"http:\/\/buklijas.info\/blog\/#\/schema\/person\/780025d597f1c5df3cc156eaffc8c561\",\"name\":\"Sasa Buklijas\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"http:\/\/buklijas.info\/blog\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"http:\/\/0.gravatar.com\/avatar\/9f6f7de5a4882517ca0e4a8ebd607925?s=96&d=mm&r=g\",\"caption\":\"Sasa Buklijas\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p5YHGV-8w","_links":{"self":[{"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/posts\/528"}],"collection":[{"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/comments?post=528"}],"version-history":[{"count":18,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/posts\/528\/revisions"}],"predecessor-version":[{"id":567,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/posts\/528\/revisions\/567"}],"wp:attachment":[{"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/media?parent=528"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/categories?post=528"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/buklijas.info\/blog\/wp-json\/wp\/v2\/tags?post=528"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}