You’ve likely never heard the term “Disk Fragmentation” before, unless of course, you’ve been using computers for so long that you can remember when they came with a 64 MB hard drive, or smaller. That was an awful long time ago and for the better part of the past decade, disk fragmentation is an issue which plagued only audio, video, photography, and other professionals who use their computers to move very large files around.
With the advent of inexpensive, high-capacity storage drives, and as computer processors have pushed into new territory, fragmentation is beginning to rear its ugly head at home, and not just in the professional studio environments.
Wikipedia has this to say about the issue:
In computing, file system fragmentation, sometimes called file system aging, is the inability of a file system to lay out related data sequentially (contiguously), an inherent phenomenon in storage-backed file systems that allow in-place modification of their contents. It is a special case of data fragmentation. File system fragmentation increases disk head movement or seeks, which are known to hinder throughput. The correction to existing fragmentation is to reorganize files and free space back into contiguous areas, a process called defragmentation.
[Read the entire Wikipedia article HERE. You may find this article overly technical. If so, feel free to skip it, as I've outlined the important bits below.]
An easy way to think about fragmentation is this:
If your garage is a hard drive, and you’ve got hundreds of boxes of all different sizes that you need to store there, you’d be much better off planning ahead, cataloging everything, bringing in the larger boxes first, the medium boxes next, the smallest boxes last and stacking them all neatly. You’ll keep an index of all of the boxes, their contents, and where you’ve stacked them. You’ll also remember to create a path between the rows of boxes so that you can easily travel around in search of what you need. Tidy.
Now imagine that same garage filled up with the same boxes, but without any prior planning or system. You now have an enormous, disorganized mess. You can’t find what you’re looking for when you need it, and if you ever do need something, it will take you hours, perhaps even days to locate it and remove it from the piles, assuming the item doesn’t get damaged in the process.
Admittedly, this example is a gross simplification of the issue, but it’s really not too far off. And in truth, given that your hard drive is storing and managing not hundreds or thousands of files, but millions of files, you can replace your garage in this example, with the government warehouse scene at the end of Raiders of the Lost Ark.
The most worrisome results of disk fragmentation are:
- Overall computer slow-down
- Files and documents can get lost, or become corrupted
- Certain programs will no longer operate properly, or even launch
- Computer stops working altogether
Here’s what a normal, healthy hard drive looks like, as a chart:
(click on the image to view larger)
This is what my own hard drive looks like as of about 12 seconds ago. The black area represents free space on the drive, while the purple represents data. Notice that there are some little spots of black (free space) spread out amongst the purple (used space) areas. The smallest of the black spots are technically free and open to be used, but like will never be used, because the audio, video, and photo files that I work with regularly are too large to fit in those small free spaces.
What this means, is that while the computer sees this as “free” space, it’s really not. I’m already familiar with how quickly it takes for me to fragment a drive, given the nature of my own work. I perform a drive defragment procedure every 90 days or so on my MacBook Pro. My main video workstation, on the other hand, undergoes a defrag every 30 days or so, depending on how much video editing I’ve done that month.
Okay, now let’s have a look at a computer that I’ve been working on for the past couple of days.
This is not good:
This is the second-worse drive fragmentation I’ve seen. Ever. This iMac is currently sitting on my workbench as I type, and is the inspiration for this article.
The iMac’s owner came in complaining of overall slowness and erratic behavior. The first thing I noticed was that it took 5 full minutes for the machine to start up. There are a few procedures one can run to remedy this.
I use Prosoft’s Drive Genius software to defrag hard drives. It’s excellent and very fast (as fast as these things can be anyhow.)
It’s important to note, however, that a drive that’s as fragmented as this one is, is better off reformatted than defragmented. The likelihood of damaged files becomes very high when the problem is this advanced. However, there’s no harm in trying, as long as you have very current backups of your entire hard drive. [Refer to articles on the importance of including a regular and reliable backup routine in your workflow, also under the Mac Maintenance category on this site.]
Some more scary information for you, or “how I dealt this this disastrous hard drive”:
We had agreed that it would be better and faster (and therefore cheaper for the client) for me to erase the hard drive, reinstall Mac OSX, all of the applications, and then bring her user data over from her backup. Sadly, this client was unsure of when her last full backup was, so no problem, I’ll just back it up here in my office. Well, as fate would have it, 4 attempts to back the computer up failed, and the poor computer would freeze during the process. Following a handful of restarts, the computer simply refused to start up again.
Attempts to access the hard drive from another computer (a secret Mac technician trick) failed, and we had to resort to using ProSoft’s Data Rescue to recover the information on the drive. We got almost everything, and there was some data lost. Other data was recoverable, but many file names and locations were lost.
After wiping the drive, and loading all of her apps, I’m now in the process of restoring personal and project data, and I’m noticing that some of the data is simply damaged. Not much of it, but some. I’m sure some of it is important. Emails, photo projects, etc.
Needless to say, the client and I have conferred regarding setting up a workflow situation which will prevent this same thing from happening again in the future. An ounce of prevention, and all that.
As I mentioned earlier in this article, drive fragmentation is no longer an issue for the professionals. As we deal more and more with editing our home movies in iMovie, or putting all of our digital photos into iPhoto, recording music in GarageBand, or storing all of our music and DVDs in iTunes, we are going to have to start working like the professionals do.
—————————————————————-
Tips and Preventive To-Do’s:
Here are some tips to minimize drive fragmentation, and some ways to combat it once it’s happened. If you are using your Mac to do any of the following on a regular basis, this section is for you. If not, skip ahead to the next dotted line.
High-risk users:
- Photographers. Are you managing a large volume of digital photos? Are you using Adobe Photoshop to edit them?
- Musicians. Are you doing multi-track recording, or movie scores?
- Videographers. You are the most at-risk category as video files are very large. Are you shooting in High Definition?
1. Whenever possible, do not work to the same hard drive that your system lives on. Professionals know this. If you work on a drive other than your system drive, your work files will not cause your system drive to fragment. Sure, the work drive will eventually fragment, but you can easily run a defrag on your work drive without worrying about damaging your system installation or your applications. Also, if you want, you can move all of your work files to a backup drive, erase the work drive, and then move the files back from the backup drive. In this way, you can keep your fragmentation under control, and greatly minimize the risk of lost or damaged files, and downtime resulting in having to reinstall your entire computer.
For example:
If you have a MacPro tower, the standard configuration is to load at least 2 hard drives into it. The system and all of your programs go onto one drive, and the second drive is used to house all of your projects. A 3rd, external drive is used to backup the 2 internal drives. There are, of course many possible configurations, and individual needs will vary.
If you are using a Mac Mini, an iMac, or a MacBook/MacBook Pro, you only have one hard drive inside. The rule still applies however, if possible, do not work to your system drive. I use a LaCie Rugged drive when I edit video, or when I record audio with my MacBook Pro. [Read more about this in this article HERE.]
2. In this day and age, I should not have to state the obvious, but here it is anyway, make sure that you have a system in place to backup your system drive AND your work drives. Redundancy is good. If your data does not exist in 2 places, you’re asking for trouble.
3. If you are a Photoshop user, create a partition on your system drive to use exclusively as the Photoshop scratch disk. Photoshop creates multiple temporary files every time you open an image, and the room these files require adds up very quickly. Luckily, Photoshop allows you to decide where those temporary files are going to go. Don’t allow them to occupy the same space as your system files, and you’ll be good. [You can use Drive Genius to create additional partitions, or Apple's own built-in Disk Utility program.]
4. iMovie and Final Cut Pro both allow you to choose where new projects get created, and where new footage imports to. Choose your work drive, NOT the Movies folder in your Home folder. Again, do your work to an external work drive, whenever possible, especially if you’re shooting with an HD camera. HD files are enormous, and will fragment a drive faster than anything else I can think of.
I generally erase my Video work drives following the completion of each video production project.
5. Get the fastest drives you can. If your computer has a FireWire 800 port, make sure you get a drive with a FW800 port. I don’t recommend using USB2.0 for video editing. You’ll end up with dropped frames, and other weird issues.
6. (this tip for Videographers only) There are many reasons you may be unable to work to an external in certain situations: You’re on the road, forgot your external work drive, it died or doesn’t have enough free space left on it. If that’s the case, you’ll simply have to work to your system drive, out of sheer necessity. Well, no problem, but as soon as you can, move the project to an external drive, and erase it from your system drive, especially if your project exceeds 15 or 20GB. And remember that you’ve done this, and that you’ll need to run a defrag on your drive a few weeks sooner for having done it.
—————————————————————-
Routine Prevention:
1. Perform Regular Maintenance on your Mac! If you don’t want to do that, I am available to do it for you. I visit my regular customers once a month to make sure this happens. It makes a huge difference, and keeps everything happy. [Refer to this article HERE, to learn how to keep your Mac healthy and happy.]
2. Defragment your drive every 6 months. I use Prosoft’s Drive Genius tool to perform this task. The frag-map screenshots above are from Drive Genius, and defrag is simply the push of a button. BE SURE that you’ve run a backup before you defrag, because sometimes defragging your drive can cause files that were sick, but still hanging on, to stop working completely.
3. Backup backup backup Backup backup backup Backup backup backup Backup backup backup Backup backup backup Backup backup backup. Clear?
—————————————————————-
In my experience, most folks don’t ever need to deal with this issue. If you’ve got 2,000 photos in your iPhoto Library, you’re fine. I’m speaking to those of you who have 12,000 photos, and who sometimes shoot weddings; the semi-professional amateur.
I’m not talking to the woman recording herself singing with her acoustic guitar. I’m talking to the person recording their band’s next demo.
I am talking to the person who’s using their Mac, and iMovie to put together wedding videos or music videos, or to edit more than an average helping of home movies.
Again, most people end up traveling through their Digital-Life without every needing to address this issue, though you’ve undoubtedly noticed that after a couple of years, your Mac is just not performing like it used to when it was new. That’s normal, and does not suggest that it’s time to buy a new computer. It’s likely your drive just needs some TLC.
I can tell you from experience, that when people do have to deal with this, they’re almost never prepared, they almost never have a current backup, and the people who do end up losing data, lose big. Don’t let this happen to you. It’s all easily preventable, and now you know how.
Thanks for reading!
~ Chris
