DIY Fusion Drive: an attempt to retrofit a pre-fall 2012 Mac with an SSD and a traditional hard disk

Just a couple days ago Apple released news that new Macs would have an option called Fusion Drive, a technology that would take a fast SSD and a traditional hard disk to bond the two together to create a “self optimizing” single volume. Apple announced that the new technology was included in Mountain Lion, which made me curious as to whether existing Macs may be able to use the technique and have a Fusion Drive “retrofitted”.

I did a quick search on the web to find this excellent article on Anandtech and a FAQ page by Apple, evidently showing that Fusion Drive is simply a volume created using LVM (Logical Volume Management) that aggregates two physical disks. This is a fairly common practice in the Unix world. It also turns out that Apple had introduced LVM support back in 10.7.4 and some previous analysis was already available on how to use diskutil to manage LVMs, but people had failed to use the addDisk command to add a second physical drive to a LVM group. It is possible that Apple made LVM smarter in 10.8 to distribute data over a group in a more efficient manner, maybe determined by the speed of the underlying physical storage technology. If this theory is correct, that would mean it should all be just a matter of creating a LVM and persuade the OS to run from it.

In the following guide I managed to create a LVM using an SSD and a traditional hard disk, to combine the two to one volume. The machine I used is a MacBook Pro, 17 inch, early 2011. I removed the optical drive and replaced it with a big, traditional hard disk. The original HDD was replaced with a fast, 240GB SATA-3 SSD.

It is difficult to prove that this is exactly the same as to having Apple’s new Fusion Drive enabled, as it is very difficult to determine where individual pieces of the file system are physically stored and if these pieces are actually managed between the two disks actively to achieve better performance. There is a fair chance that there really isn’t more to this than setting up the LVM. Unfortunately I don’t have access to a Mac with a real Fusion Drive to compare the outputs of commands such as”diskutil cs list”. It’s also possible that additional settings are required to enable the active management of the LVM, but according to Apple, every Mac running Mac OS X 10.8.2 should be able to read Fusion Drives in target mode, which at least seems to indicate that current Macs using the latest OS should technically be equipped to deal with a Fusion Drive without requiring further software.

So far the technique has proven itself to run at the speeds I was used to have when I was hosting all apps and the OS on the SSD exclusively. I will continue to verify the performance and see whether it behaves in the same manner as described by Apple and Anandtech. Suggestions on how to most efficiently verify this would be very welcome.

Be warned, this guide is for the tinkerer only and those that don’t mind a total loss of their data. Messing around with diskutil will wipe your disks and you have to rely on a functioning backup to get your data back.

  1. Make a full backup of your data, all of it! If you don’t know how to do this, don’t attempt to follow this guide any further. You have been warned! I used TimeMachine for the backup.
    1. In order to speed things up, you may want to connect your Mac to a Time Capsule using a wired Ethernet connection and turn WiFi off.
    2. If you have a bootcamp partition with Windows, make sure to also backup all of that data.
  2. Make sure you are on Mac OS X 10.8.2
  3. Create a USB restore key with the latest OS X version. You can find instructions on this article on CNET. Apple also provides a tool, but I have not verified whether it will create a key with the latest version of OS X. I downloaded Mac OS X 10.8.2 from the AppStore on a Mac OS X 10.7 machine. That ensured that the restore image (ESD) would be coming with the latest version of OS X available.
  4. Reboot to Recovery Mode with the USB key, holding Alt/Option and choosing the USB key to boot from. This takes you into the recovery mode from where you can access both physical drives properly online task manager.
  5. Delete all partitions on both physical drives. You can use Disk Utility for that.
  6. Now open the terminal by going to Utilities -> Terminal
  7. Create a new Logical Volume group with both disks
    1. Use “diskutil list” to identify the names of your physical disks. In my case it was disk0 (SSD) and disk1 (HDD)
    2. Run “diskutil coreStorage create FusionDrive disk0 disk1”. This is the step where others seem to have failed. I figured you could create the LVM by supplying both disks at the same time, instead to creating a LVM with one disk first and then trying to use “addDisk” for the second disk, which doesn’t seem to function. You can also replace the “FusionDrive” string with your own choice of LVM group name. It does not matter what the name is.
    3. List the new LVG using “diskutil coreStorage list”. You should see something like this:



      Get the overall group size and copy the UUID of your LVM, in my case “F628F010-3CFF-4BC7-90CE-CD61EC7C44E1”, then run “diskutil coreStorage createVolume F628F010-3CFF-4BC7-90CE-CD61EC7C44E1 jhfs+ MacintoshFD 975g”. Replace the UUID with your copied LVG UUID and replace 975 with the actual size of your LVG. You can also rename MacintoshFD to whatever you want your main volume to be named to. If the name contains spaces, make sure to contain the string in quotes (“”). Also, make sure to subtract some 10G from the size to leave enough room for a recovery partition.

  8. I did proceed to close the Terminal window and restore a fresh Mac OS X install. You should also be able to restore all your data from a Time Capsule at this point. I did install the OS first to see whether the LVM was bootable and then returned into Recovery Mode to restore from my Time Capsule. After a couple hours my machine booted back into my previous Mac OS X environment.
  9. Congratulations, now you have a Fusion Drive-esque volume. Not only will have higher performance from this setup, you also get the aggregate sizes of your mechanical disk and your SSD, together in one volume.

Like I pointed out earlier, this may just be all the magic behind the new Fusion Drive and I’m currently trying to verify whether this setup behaves the same, or I just created a dumb LVM that lacks the “secret sauce”, like a particular flag being set in the LVM header I don’t know yet about. It would be interesting to compare this setup with a real Fusion Drive and if you have access to a Mac with an actual Fusion Drive shipped by Apple, I encourage you to run the “diskutil coreStorage list/info” commands and post your results in the comments. I’d love to take a look at the real setup. Please let me know if you have an idea at a good strategy to test the setup and verify that it does indeed move data around between the SSD and the HDD in a smart manner.

UPDATE 10/31/2012:

While I wrote this on the weekend and didn’t have much time to dig into verifying whether this setup works like a real Fusion Drive this week ( I do have day job 🙂 ), jollyjinx posted a guide on tumbler on how he went ahead to verify that the above setup does indeed work like you would expect a Fusion Drive to behave. I do have quite a bit of background IO “noise” on my machine, because the LVM is actually hosting the actual OS, but after following JollyJinx’s steps and verifying the results on my own machine and I am pretty much convinced at this point, that Fusion Drive just works “automagically” if you use it with an SSD and a plain old hard drive. After all, the OS has several ways to know that it is actually attached to an SSD by either checking the storage type (I’m sure there is some vendor PID that can be queried from the hardware) or by just keeping performance logs on how the hardware behaves, which would let the OS infer which drive is faster. Despite not leaving much control over whether such behavior is really desired, this is what I would expect from Apple; to employ such heuristics in order to just do what they think is the right thing to do on such hardware.

As to my own results, using iostat in a shell, I was able to see how the IO load was distributed to either one of the disk when writing/reading data, but rarely to both at the same time (which would indicate proper RAID 0 style striping). I could also see that data was moved over from the mechanical disk to the SSD, after accessing some data that was previously read from the HDD. The moment you stop reading such data you can observe IO activity transferring over to the SSD for a while, roughly to the amount of data you were accessing. Accessing the same data again then returns IO activity almost exclusively from the SSD instead of the HDD, which seems to indicate that the OS had decided to move the data over to the faster drive. This sure looks like it’s exactly what Apple was promising with the Fusion Drive.

Sweet, I think I’ll keep this setup!

The perfect SSD and HDD hybrid MacBook setup

The SSD and HDD happily together, with plenty of space for more data. Note the separation of different categories of data, most suitable for either drive.

The performance of a Macbook pro by itself is quite amazing, but enthusiast know that the quest for higher specs never ends. It used to be that adding more memory (RAM) would be the best investment in increasing a machine’s performance and it sure still is one of the first things one should consider upgrading. With the introduction of solid state disks (SSD) however, replacing the magnetic hard-disk has become at least equally rewarding, if not even more. A typical 2.5 inch hard-disk will get about 60-90 MB/s in sequential read/write measurements, while a modern SandForce SSD on a SATA3 connection can easily get up to 400-500 MB/s. More importantly, seek times are dramatically lower on an SSD, given that the drive does not have to position a head physically on several disk platters. Since typical usage patterns on modern OS include loading many small files from completely disjoint regions of a mechanical hard-disk, an SSD can cut load times for a typical application to fractions of a second, whereas a mechanical drive would load data for seconds. However, SSDs are still much more expensive than mechanical drives, eclipsing $1 per GB, which makes them too expensive to keep your vast collection of music, pictures and movies on them. Fortunately, storing such files is more like keeping an archive of rarely accessed data and therefore there is no real benefit in storing hundreds of gigabytes of pictures of your cats, dogs or family on an SSD.

The following is a step by step guide on upgrading a typical Macbook with an SSD and preserving the old mechanical drive for rarely loaded data, such as your iPhoto pictures, iTunes music and movie folders. Some guides will recommend moving the whole /Users/<yourname> folder to the mechanical drive. Personally, I think that having parts of your user folder on the SSD will benefit performance, as many applications cache data in the ~/Library folder, and store settings and metadata in this location. The load time of applications can be improved if such data also resides on the SSD.

First, a couple of preparatory steps need to be performed.

Preparation

a. Make sure you have a current backup

Needless to say that if you are dealing with moving data and manipulating your long-term storage system of your computer, you should make sure that your backups are up-to-date. The following guide has the potential to destroy all of your data, so beware before performing any of these steps. Safeguarding your data is YOUR responsibility!

b. Find out where your data lives

In order to make the right decision on how big of an SSD drive you should get, it’s probably best to know what kind of data takes up the most space on your drive. Data such as your iTunes library, movies or iPhoto library do not need to be placed on an SSD, as such data usually doesn’t load very often and when requested, the performance of a mechanical drive will just do fine. Applications like WhatSize can make it easier identifying your storage requirements.

WhatSize helps identifying where you’re storing most of your data

c. Get an optical bay adapter

These days, most people don’t use built-in optical drives anymore. There are some vendors out there manufacturing bays that will allow replacing the optical drive with a second hard-disk. If you want to keep your mechanical drive to store rarely accessed data, this is the way to go. MCE Tech sells the Optibay online. Make sure to get the right adapter for your Macbook, as some MacBooks have SATA interfaces, while older ones have a PATA interface for the optical drive and that would require you to get a bay that converts PATA to SATA, so you can keep using the SATA drive that came with your notebook.

d. Choose an SSD

New Macbook Pros come with a SATA3 interface, which will allow you to take advantage of new SandForce SSD drives. SATA3 has a theoretical transfer speed of 6 Gb/s. SandForce powered SATA3 SSD drives can deliver up to 500 MB/s in sequential data transfer rates, which is easily a 5-8 times improvement over a regular mechanical drive. When choosing the size, make sure you also have at least 20 GB or more left on top of your storage needs, as the OS performance starts degrading dramatically, if you run into low hard-disk space.

e. Have an external USB enclosure ready

Transferring the data from the original hard-disk to the new SSD is much easier if you have an external drive available, as it will reduce the amount of times you’ll need to disassemble your notebook. Now proceed with attach the USB enclosure to your Macbook.

That’s it. We’re done with the preparations. Let’s move on to the real migration work.

1. Partition the new SSD drive

Use Disk Tool from the Utilities folder to partition the new SSD. Make sure to choose “GUID Partition Table” under Options of the Partition tab before creating the new partition. Name the new partition to “Macintosh HD” in order to preserve the naming of the drives and select “Mac OS Extended (Journaled)” as the file system type. Hit “Apply” to finalize your changes.

<a href="http://www.petralli project planning tools.net/wp-content/uploads/2011/12/Screen-Shot-2011-12-25-at-12.34.20-PM.png”>

Choose GPT when partitioning to ensure that your drive is bootable

2. Transfer all data using Carbon Copy Cloner

While there are other tools available for this, Carbon Copy Cloner is a great and free (donation ware) tool that allows you to efficiently clone your original drive to the SSD, without requiring any re-installation of Mac OS X. I’ve used this tool for years now and it has never failed me so far.

When running CCC make sure to exclude your media data folders. Exclude the following folders:

  • /Users/<yourname>/Movies
  • /Users/<yourname>/Music
  • /Users/<yourname>/Pictures

Deselect your media folders when cloning your data from the HDD drive to the SSD

3. Make room for Bootcamp and Windows (optional)

If you are interested using some of the prospective available space of your mechanical drive, you should consider preparing your bootcamp partition now. Windows requires the optical drive to be attached to the SATA bus of your machine and won’t allow you to install from an externally attached USB drive. Therefore it’s most advisable to perform the Windows installation at this time, so that you won’t have to disassemble your machine multiple times. This guide won’t go into details on how to install Windows on your Macbook, as there are plenty of good guides elsewhere. Just make sure to detach your SSD drive from the USB port before proceeding. That will ensure you’re not accidentally removing the newly created parathion on the SSD drive or re-format it inadvertently.

4. Don’t reboot into Mac OS X

Once the main hard-disk is cloned, you should avoid continuing to use or rebooting into Mac OS X as changes won’t be present on your SSD. If you’ve installed Windows in the previous step, you can make sure to avoid booting to Mac OS X by pressing the “alt” key during startup. That will allow you to choose the Windows partition instead.

5. Remove primary drive and remove built-in superdrive

It’s time to replace your primary drive with the SSD and install the optical bay replacement to hold your mechanical drive. Follow the guide that came with your optical bay replacement or check out the many guides you can find by searching Google.

6. Your first boot from the new SSD

It’s time to boot Mac OS X from the SSD for the first time. When turning on your notebook, make sure to keep the “alt” key depressed. That will allow you to choose from which drive to boot. If you’ve installed the SSD as the primary drive and put the mechanical drive into an optical bay adapter, you will see two “Macintosh HD” volumes available. The left volume is the first from the SSD. Choose the left “Macintosh HD” volume and hit return. You should already notice by now that you’re booting from the SSD, as your usual boot time should be cut down dramatically.

7. Rename the mechanical drive to “Data”

Beware that any removals should only be performed if you are confident that your data is available on a backup. The following steps have some severe data loss potential! Proceed with caution and at your own risk!

Once Mac OS X is fully booted go to the Finder and hit control-shift-g. Enter “/Volumes” and hit enter.

Go to the /Volumes location using the Finder’s goto function

You should see a “Macintosh HD 1” volume, the old mechanical drive, which you can rename to anything you like, or just “Data” for the purpose of this guide. From here you can also proceed in removing all System directories such as “Applications”, “Libraries”, “System” and so on, as these directories are fully cloned and available on your SSD. Just ensure not to remove the “Users” directory, as it contains all your pictures, movies and music that you have not transferred to the SSD.

8. Move your media folders to the new Data volume

Go to the finder again and navigate to your old Users directory. Hit control-shift-g and enter “/Volumes/Data/Users/<yourname>” . Grab the Movies, Pictures and Music folders and drag them to the “/Volumes/Data” location. Then you can proceed in removing the rest of the “Users” folder from the old drive, as anything else should have been moved when cloning the drive initially.

9. Redirect your Music, Movies and Pictures folders to the Data volume

Open a Terminal window and type the following commands:

 

Create the symlinks to your Data volume

These commands will create symlinks from your user folder to the mounted Data volume. Symlinks are like shortcuts redirecting data to another location on the drive. iPhoto, iTunes and iMovie will continue to think that your data is in its usual location and the applications won’t need any reconfiguration to access the data from the mechanical drive. Additionally, if you’ve set up Time Machine to perform backups, TM will follow these symlinks and continue to backup your data as usual, despite being located on a different drive than the operating system.

10. Restart your backups

Now would also be a good time to ensure your backups are up-to-date again. If you’re running Time Machine or any other backup, make sure to check that the backups are being performed as expected.

11. Done!

You’re now a lucky owner of a hybrid SSD and HDD setup. Access to your applications should be dramatically faster, while not wasting precious SSD storage on rarely accessed data.

Pictures, Music and Movies all mounted on the mechanical drive

I hope this guide was helpful to you. If you have questions or contributions, feel free to leave a comment.