Session Backup Strategies & Workflows

00:00:09
Good morning, children! Today, we're gonna talk about backup strategies and since no one has the same backup strategy, because there are many backup strategies and no exact guidelines on how to backup, I thought we'd do it with somebody who probably has a very good backup strategy: Andrew Scheps! What's your strategy? - Well, it changes all the time.

00:00:30
'cause the tools keep changing. But..

00:00:32
my philosophy is that if it's not in 3 places and one of those places isn't somewhere else then it's not backed up, and..

00:00:42
that's only because I've actually been bitten several times in different ways by not adhering to both sides of that rule.

00:00:50
- Ok..

00:00:51
- The 'having one somewhere else' is so that somebody else can get to it, or, if the place where your other two are burns down, that place is still there..

00:01:00
and the being in 3 places is because it's very difficult to have one on-site backup as your only backup.

00:01:06
So you need one local backup and one elsewhere.

00:01:09
So that's the general theory.

00:01:11
How I do it changes constantly.

00:01:13
- How is that reliable if it changes constantly? You have to keep track of where your stuff is.

00:01:17
- Yeah, my local backup is always a second hard drive.

00:01:20
That's it. And so..

00:01:22
Which software I use to back it up changes..

00:01:25
for a long time I used Synchronize Pro from Qdea.. like 'idea' but with a 'Q'.

00:01:32
And I would run that backup at the end of every day.

00:01:35
And this is back before all the discussion 'Pro Tools got good' and like you definitely didn't want anything else going on in the background ever..

00:01:41
all the way back to the days when you needed to defragment every morning before you hit Record.

00:01:45
Now it seems like you could record an infinite number of tracks at any sample rate to any hard drive while backing it up and looking at a YouTube video.. so..

00:01:54
that part of it has gone away, so I started looking for some more real-time backup options..

00:01:58
'cause I got sick of waiting at the end of the day..

00:02:00
you finally finish and then: 'oh no, I have to backup..' So, I started using Synk Pro ..can't remember the name of the company that makes it..

00:02:10
very clever, integrates itself into the finder in a lot of ways and gives you a way to setup scripts that will auto run as soon as anything changes. So, if the folder gets touched it just starts backing up.. but it's also very clever about not backing up files that were in use.. because Pro Tools will pre-allocate files..

00:02:30
A lot of the backup programs will very happily back up the pre-allocated version, which is the wrong size and then it will go back and have to back up, again..

00:02:37
and at least temporary files and thing likes that, where Synk Pro was really good about having those be identical but every once in a while I would get a file that wouldn't be properly encapsulated or something, where a folder would look like a file, with the creation date of 1969 and there was no way to repair it..

00:02:57
and when you went to run a backup it would say 'no..

00:02:59
it's all fine, all the files are there.' But you couldn't actually open the backed-up file.

00:03:04
And the last thing I want in any backup is to have to check it.

00:03:07
- Yes. So now I've gone back to Syncronize Pro which has all sorts of scheduling features, which are accessible only by clicking on the little backup window down where it says the last time it was run and opens up this myriad of options..

00:03:23
So I've got that set to auto-backup every 10 minutes.

00:03:25
So that's my local backup thing.

00:03:28
It's every 10 minutes, my active drive backs up to an exact copy of the active drive.

00:03:34
- Well, that's very commendable! - Thank you! - That's very commendable! Somebody said: 'if you have one copy, you are stealing it.' 'If you have 2 copies, you are borrowing it.' 'If you have 3 copies, you can think of owning it later.' - That sounds good! - Yes, it sounds good.

00:03:55
I used to work the way you did.

00:03:58
But not as thoroughly.

00:04:00
Because I am too lazy for that.

00:04:02
And I don't want to script anything..

00:04:04
so what we did is.. we had an internal drive that was filling up rapidly, but it was an internal drive. And then we had incremental drives like we had Audio A, we're probably at Audio K right now..

00:04:18
and then we have Audio A Backup.

00:04:20
So we have the internal working copy then we had the A backup copy, and then the backup of the backup.

00:04:27
And we made sure that at the - in theory - at the end of every day, we would copy the internal drive to the backup drive. At the end of the day.

00:04:39
We still haven't lost a file, so far.

00:04:42
But what we found, is that..

00:04:45
at the end of the 12-hour day nobody stays to do the backup and make sure that it doesn't crash. - Exactly, because if you are tracking a band, that's 45 minutes of backup time, even with fast drives. - Yes.

00:04:55
Nobody stayed. Not even the assistant who was supposed to stay, I found several times that the stuff was not backed up.

00:05:00
And so, if anything had happened to the work drive we would have been.. and..

00:05:05
I'm gonna used a super-technical term..

00:05:07
F*CKED. We used..

00:05:10
Chronosync.

00:05:11
Because Chronosync is very fast on incrementals, meaning it doesn't copy all the files every time.

00:05:16
Either..

00:05:18
that was wonderful and all but..

00:05:21
you know, we still haven't had a problem but there were risks.

00:05:25
The opposite of backup is 'risk', the idea of it is not having a risk.

00:05:29
And also, we would backup so the backup, to the backup of the backup, once a week or once a month.

00:05:36
Which also didn't always happen, depending on who got fired, who got hired who was travelling and everything.

00:05:41
So, that sucked.

00:05:43
So we had to change it. I came up with a different system. So, what's your new system? Well, my new system is: I'll still do my local backups the same way.

00:05:52
And they're running at 10-minute increments using, the Qdea Synchronize Pro X.

00:05:59
Which is also really good for making bootable backups of your startup drive. Which is another thing I try to remember to do, every week or so..

00:06:06
Because you update things and you forget.

00:06:08
And if you have to go back a month on your startup drive you've got sessions that won't open. So, I try and be good about that, there's tons of other software which works great: Carbon Copy Cloner works really well for backups that will only happen every time both drives appear.

00:06:25
And then it will run in the background and you can unmount the drive.

00:06:28
But I like to leave my local drives mounted, so the off-site backup is what's changed a lot.

00:06:33
So I used to have drives that I would bring in to the house as opposed to in the studio. Which was at my house, not in my house.

00:06:41
I guess if the fire would be big enough..

00:06:43
that wouldn't be enough. But I am not driving to a bank vault.

00:06:47
Every day.

00:06:48
Since cloud storage..

00:06:51
I've tried a million different ways to run backups. SugarSync is a open-source Linux-based backup strategy thing, which now there are a lot of cloud backup software that's based on it..

00:07:06
and I tried running that for a little while, and that would run much the way Synk Pro did work. It would see the folder that changed..

00:07:13
but because you're going through Linux and onto servers your modification dates could change and all of a sudden, the incremental backups don't work, anymore.

00:07:22
I'm big fan of Dropbox, the functionality of it, so at the moment what I do is I have a second backup script that is copying.. so, the..

00:07:33
'main drive to backup drive' is a backup script.

00:07:36
Then I have a synchronize script, which is a 2-way backup, that goes from my main drive on my active project into a folder that's in my Dropbox folder and I've got the Terabyte plan for Dropbox.

00:07:49
And that would be uploading in the background, so that's backing up to an offsite thing. Of course we've just learnt that 68 million passwords for Dropbox were compromised in 2012.. so change your password right now, if you haven't already.

00:08:04
There are obviously issues with putting stuff on other people's drive, so good part is the way most of these, and I believe all of these companies work is that your data can only be accessed by you. There's an encrypted database to the files, which points to where they are in the drive that is encrypted with your password.

00:08:23
So it's not someone from Dropbox can go through your stuff..

00:08:26
So unless they're credential files get compromised, which they did..

00:08:32
your stuff is safe.

00:08:34
Anyway, to get back to what I'm doing at the moment, which I'm not sure it's gonna last too much longer but I have these backups into Dropbox, and what it allows me to do is I can share that folder with somebody else, if I need to collaborate on a session. So if I have somebody else working on a song musically, or I've got..

00:08:51
let's say a friend of mine in LA, which works with me quite a bit, will sometimes print versions, or do final tweaks, 'cause I don't have time..

00:08:59
he'll always have the most current version of the song working out of the Dropbox folder..

00:09:03
As soon as he's done, I'm downloading all of his updates, so so if I have to work on the song..

00:09:08
give it 20 minutes and now I am right where he left off, and that will synchronize itself back onto my main work drive which then backs itself up onto my backup drives.

00:09:16
- So what you do when your 1Tb is full? - As an Amazon Prime subscriber for $5 a month or $60 a year you get unlimited space on Amazon servers.

00:09:28
And I would say of all of the servers in the world Amazon server have got to be the most robust.

00:09:35
They have the most people stuff on them as well as their own stuff.

00:09:39
A company like Dropbox which isn't creating their own data..

00:09:42
they could go under and just have the servers go..

00:09:45
Gobbler had a problem, where they were renting server space from someone and all of a sudden those servers are no longer available, for a minute.

00:09:52
Everybody's data is - not lost - but it is not accessible.

00:09:55
Amazon is staying online.

00:09:57
So now what I do is I then copy from Dropbox to Amazon and they're a lot of different servers which allow you to copy from cloud drive to cloud drive and the key to this, especially because I live in the country and my internet is terribly, terribly slow..

00:10:15
is I don't wanna be copying through my computer. When I was in LA, screaming fast internet, it didn't really bother me to have to download and re-upload it. Now, if I've got to download and re-upload 1Gb that's a full day.

00:10:28
So, at the moment I am using a company called MultCloud which allows me to have multiple cloud stuff mounted, including FTP shares, so I've got my web server as well as my Box account, my Dropbox account and my Amazon account all mounted, and they are in a web-browser sort of hard drives so you can actually set up backup scripts that run incrementally.

00:10:50
So I can archive to Amazon from Dropbox, continue to work on the album and then run that archive script again and only the new stuff gets copied.

00:11:02
So far that's working, we'll see what happens in the future.

00:11:04
- You are a brave soul! - Thank you. - I am way, way lazier than you! - Well I'm lazy 'cause it's all automated.

00:11:13
- ..ish! So for those of you whose heads are spinning right now, trying to figure out what's going on..

00:11:18
Andrew has a local backup and he has - for the current project - a sync to another folder in his computer, that one syncs to Dropbox, online, the files are gonna be online.. then he has another system where the Dropbox files get copied online to online, to the Amazon S3 servers.

00:11:35
- Correct. - It's impressive! - What I am getting to is where on Amazon will actually be everything I've worked on in the last 12 years.

00:11:43
- So that's the idea.

00:11:44
- That is the idea, and I actually had to use it, when we were shooting the mixing videos here, I had an archive hard drive with me, of older material and one of the songs we're mixing was just one drive before the individual drives I brought with me that 4Tb backup drive, took it down.. about 2 weeks ago.

00:12:03
And I forgot. And the day before we were supposed to shoot I said: 'mh! Where is that song?' And fortunately, it was on Amazon. So I downloaded the 10Gb overnight and was able to show up with the song. And otherwise I would have needed someone to FedEx the drive to me.

00:12:18
And we would not have been able to shoot the video.

00:12:21
- Saved the day! So, here's what I do: I work off Dropbox.

00:12:27
End of story. - Directly? - Directly.

00:12:29
It used to be horrible.

00:12:31
But because it was simple and low-impact, I still did it.

00:12:36
What used to happen is: sometimes Dropbox would try to upload the file that you had open.. and corrupted it.

00:12:43
Which was a tremendous amount of fun.

00:12:45
But since I am paranoid and I have my auto save on Pro Tools on 1 minute and I save the last 100 backups..

00:12:51
that has saved my ass many, many times! I always got away with it.

00:12:56
But, they fixed that problem.

00:12:59
And I have expanded my Dropbox strategy to something that's maybe a little overkill but it's working great.

00:13:04
I have unlimited storage on Dropbox. I pay dearly for it.

00:13:07
They have a minimum of 5 users per year..

00:13:10
5 users per account, sorry.

00:13:12
$12 a month, per user $15.. it's costing me $700 a year.

00:13:17
That's my backup strategy. - Which is.. in..

00:13:20
for a business, that's not terrible.

00:13:22
For writing the checks, it sucks. - Yeah. I hate it.

00:13:26
But I no longer by hard drives.

00:13:28
I don't have a local backup. I don't want a local backup.

00:13:31
I don't want that responsibility.

00:13:32
I'd rather be able to scream at Dropbox.

00:13:35
And they have amazing redundancy on their servers.

00:13:39
It's unbelievable. - Yeah, it's all around the world encrypted, but available to you. - Exactly! So, what happens is this: I create I have assistants. They all have an account.

00:13:51
I have an account for my main machine and one for my laptop.

00:13:55
And everybody is considered a user even if it's a machine.

00:13:59
And so, my main machine, I sit at it, whatever I do, is being uploaded to Dropbox in real time, because I am in NY and I have a decent connection.

00:14:07
- Yeah. - What that does, it means that about 5 minutes after I'm done, it's already on my assistant's computer and on my laptop.

00:14:15
Already. I don't have to decide to do it, I don't have to think about doing it. It just does it.

00:14:21
So, if I do, whatever, I open a backup, I open a session and I am doing a quick recall..

00:14:26
I do my quick recall. I print it. I close it.

00:14:29
That is the end of my brain share to that recall.

00:14:33
It's now on my laptop, it's on my assistant's laptop and it's also available online and it's available, here.

00:14:40
So that means I can launch the Dropbox app here and right now I can access anything I've done with no process on my behalf.

00:14:48
Anything I've done. Any mix. Send it to people, play it on my phone, do whatever I want. Since I've done that the communication with my clients have come down, drastically..

00:14:59
tracking of what's what has gone down, drastically..

00:15:01
like..

00:15:02
'which version is which', 'you didn't send me, you didn't communicate..' You have a Dropbox link and you click on it, and you have every single mix we've done properly incremented with a very, very, very OCD naming system.

00:15:15
And that, for me, although it's costly has totally changed the way I work.

00:15:22
Because, basically, if there's not no backing up there's no not backing up.. if..

00:15:28
- Yeah. 'If it exists, it's backed up.' - Period. So you can't fail! - Yeah! - This is perfect for my team.

00:15:35
And when the apocalypse comes and the internet shuts down we won't really care anymore. - No, I just really want an apple, - An actual apple, to it. - Yeah, not that one..yeah.

00:15:42
- Yes. So it will be fine! - So that's another strategy! That's not as thorough and as immediate, but it's..

00:15:49
it's only possible if you have a fierce connection. - And the only thing I'd say, is: if you are looking for cheaper ways to do the same thing, you've got to be 100% sure about where things are encrypted, where they are not encrypted, because you're putting other people's stuff, up there. I know for a fact that nobody else, unless they get my password, which I do change, and I try and keep it not the same as any other service etc. etc.

00:16:14
that..

00:16:16
I'm the only who can get to the stuff, no one else can get to that stuff and that it's actually secure.

00:16:20
Because that is a big deal. Intellectual property value, even if no one wants to pay for the records themselves it's still a very valuable commodity. So security is as important as the backup.

00:16:32
To that purpose, there's another option which is kind of the same as Dropbox, which is a local cloud system.

00:16:39
If you have a network, if you have a network..

00:16:43
there's a couple companies that make local clouds.

00:16:45
- OnCloud is a good PHP-based one.

00:16:49
That's more advanced. There's one called Transporter.

00:16:52
So, Transporter is a little box, it's a little box, you plug it ethernet in your network you put a hard drive in it and basically you designate a folder, you can work off that folder on your internal drive and it will backup automatically to that drive.

00:17:07
In real time. And it's non-destructive. It works very well.

00:17:11
But, you don't have dual redundancy.

00:17:13
You can buy 2 Transporters and they talk to each other and they back up each other.

00:17:16
And then once your 2 drives are full then you have to remove them and put other drives in them.

00:17:20
- And you need to put that other one somewhere - Somewhere else.. the thing that's amazing about Transporter is actually, you could put, your 2nd Transporter, which will be your redundancy, at your girlfriend, if you still live with your mom and it will back up, secretly, encrypted, in a tunnel, from Transporter to Transporter, over the internet.

00:17:39
- And I suppose, if you're not really sure about your girlfriend it gives you an excuse to still live with your mom.

00:17:44
- Correct! That works! You need to have your own apartment for security reasons.

00:17:48
- Yes. - That works, that's great.

00:17:50
And the cost is low, because it's a one-time cost.

00:17:53
But you still have to buy a hard drive..

00:17:55
But there's no intellectual brain-share all I have to back up.

00:18:01
What I see is: most of the clients I work with 99% of the clients I work with show up with a drive at the studio or FedEx me the drive and I say: 'thank you!' You have a backup, right?' - No! - 'No, that's the master.' *claps* - Yeah.

00:18:22
Don't ever let that happen.

00:18:24
Kids.

00:18:24
- Friends don't let friends do that. - No.

00:18:27
- All right! Well, that was very informative! - Yeah, I think so. I think that works.

00:18:31
Talk to us in a week and it will be totally different.

00:18:33
I think the key to all of this is, the tools change, get better, get worse, get different..

00:18:41
but you need to have a backup philosophy and make sure the tools you're using adhere to that philosophy.

00:18:47
So, for me the philosophy is: one local backup, one remote backup, that are up-to-date at least daily on all of my work and I hope every week on my startup drive, though I am not nearly as good about that.

00:18:59
And I don't have an off-site backup for my startup drive, but no matter what tools I use that is exactly where I back up my stuff all the time, and then there's the file security: whatever your concept is about that make sure that whatever tools you're using adhere to that even when you change tools.

00:19:16
- And..

00:19:17
my strategy is to make sure there are no human beings involved in the backup process, because - no matter what - human beings will fail! More often than machines will fail! And that's why I removed the human being from my backup strategy.

00:19:29
The main human being failing at backup being me, of course, but you gotta know thyself! The end word here, is: no matter what, if you don't backup, you don't own it.

00:19:41
Et voila'!