Notes on Migrating from Aperture to Photos for OS X

I have enjoyed photography for many years, particularly since the transition to digital. I've been shooting exclusively digitally since I got my first digital camera in 2002 and the result is rather a large body of digital pictures.

This is the story of migrating from a system that involved Aperture and a bunch of jury-rigged hacks to Apple's new Photos for OS X.

Background

Since Aperture first shipped, I used it to manage all my digital photographs - until the iPhone came along and wrecked my workflow. I can't properly explain what went wrong but I think we all recognise now that there is a sense in which photos "live" on an iPhone in a way that they don't live on a digital camera. At least they didn't for us "serious" photographers but, if you recall the days before smartphones, many regular users did store photos on their digital camera as a way to take the photos with them and show friends.

Since the iPhone, I rather relied on occasionally dumping photos out into Aperture and then, later, relying on the never-very-good iCloud PhotoStream to get photos into Aperture. The implementation of Photo Stream in both Aperture and iPhoto was a mess, as evidenced by the dozens of projects I have named "PhotoSteam (month year)", each roughly correlating to the dates I opened Aperture.

So, at the end of this era, I had:

  • About 31,000 photographs in Aperture, totalling over 300GB.
  • The Aperture library residing on my Mac's internal SSD storage taking up 38GB
  • All master images referenced on a 3TB external hard drive
  • About 6,000 photographs in iCloud Photo Library, totalling around 6-7GB of iCloud storage space.
  • A folder of unknown number of other photographs of around 40GB in size from the time between early iOS 7 betas and iOS 8 shipping.

So, how to get all of this into iCloud Photo Library and back down to my iPhone and iPad?

Migrating the Library

The first step after installing the 10.10.3 beta was to migrate the Aperture library. Photos did this more or less automatically and quite well. The software is obviously still in flux, so exact details of UI are not worth discussing right now and I want to focus on the data migration.

Photos correctly maintained the connection of photos in the library to their referenced masters on the external drive. Everything worked well as long as the drive was connected. At this point I discovered that images with referenced masters cannot be uploaded to iCloud Photo Library. Only images with managed masters can.

The result at this point was that I could browse around 36,000 photos on my Mac but my iOS devices only showed about 6,000 photos. These were the 'native' iOS photos taken on my various devices in the iCloud Photo Library era, but none of my 'legacy' DSLR photos were coming across.

It is also possible to consolidate the masters into the Photos library (using File > Consolidate). Now, recall that my Aperture library was over 300GB in size. My MacBook Air only has 512GB of internal storage and a fair amount of that is being used for other things. This seemed like a bit of a stalemate since I couldn't consolidate the library for lack of disk space.

At this point I left things for a while as other parts of life interfered....and a few new 10.10.3 betas arrived.

Migrating the Data to the Cloud

The story resumes this week as I sat down to try and solve the problem once and for all. The impetus only somewhat increased by an unexpected burst of techno-lust for the new MacBook Apple announced this week.

My first try was to consolidate the entire library. This quickly failed due to lack of disk space, as expected. An indicated 200GB was required over and above what was available.

The next step was to consolidate some images. I started off consolidating a year at a time, which allowed me to make some progress. The default setup for Photos is to keep the full-resolution masters on the Mac's storage but there is an option to optimise instead. I turned this on.

At this point, it's worth noting that I'm still struggling a bit with a mental model of where all this data is going and what's happening to it. As a precaution, I have made backups of my entire Aperture library and masters on the external hard drive.

The result of this was that I was able to, in batches, consolidate around 25,000 of the legacy images into the Photos library. The library on disk grew to around 214GB. At the same time, I was monitoring outbound network traffic from my Mac in Activity Monitor. This indicated that uploads were ongoing, which I assumed to be Photos sending these images to the cloud now that they were no longer referenced.

I opened Photos on my iPhone and iPad and could see photos streaming down to these devices. It was helpful to turn off Summarize Photos (Settings > Photos & Camera > Summarize Photos) on iOS to see the full extent of progress when viewing the Years view on the devices.

The Impact of 30,000 Photos

As the number of photos in my iOS device libraries grew, I started to notice some impacts on performance and correctness in some apps on my devices. Twitter correspondents were telling me that they had seen poor performance on iPhone 5s devices.

Once I crossed 20,000 images, I started to notice the following on my iPhone 6 Plus running iOS 8.2:

  • The first effect was occasional instability in the Photos app on iOS. It was by no means unusable but the crash rate went from zero to not-zero (note: this is much better - but still not perfect - on iOS 8.3).
  • The next effect was that iOS apps that implement their own photo-picking UI started to really struggle to either remain performant or even show a correct view of the photos on the device. Particular offenders include Instagram and Explain Everything.
  • Apps that use the system photo picker UI continued to work well.
  • The Camera app started to be slightly slower to launch.

Time Taken

Having started the migration on Friday evening, by the time I woke up on Tuesday morning my devices were showing a more-or-less consistent view of my photo library. My iPhone had downloaded tiny thumbnails for all images although my iPad was still catching up.

The image counts were a little inconsistent at this point:

  • Mac: 30,931 Photos, 211 Videos
  • iPhone and iPad: 30,190 Photos, 211 Videos
  • iCloud Settings Panel: 31,030 Photos, 213 Videos (Settings > iCloud > Storage > Manage Storage > iCloud Photo Library)

The only reliable way I found to determine whether my Mac was completely finished migrating all the data to the cloud was to observe the Networking tab in Activity Monitor. When Photos was migrating, there was a very obvious pattern to the upstream bandwidth usage.

During this time, the system did not instantly sync anything. If I deleted an image on one device, it persisted for days on the others. This is hardly surprising given the background workload that was going on and I wasn’t particularly concerned about it.

Mike Bradshaw let me know via Twitter that he had observed image counts being incorrect for around 48 hours after migration as the various devices reached a quiescent state between them.

Data Usage and Optimisations

As I said earlier, I started with over 300GB of Aperture master images. In the final tally, here is how much data the system used after all optimisations:

  • Mac Library: 95GB (as reported by du(1))
  • iCloud Storage: 269.3GB (as reported by Settings > iCloud > Storage)
  • iPhone on-board storage: 10GB (as reported by Settings > General > Usage > Manage Storage)
  • iPad on-board storage: 8.6GB

I have turned on “optimise storage” on every device, including the Mac.

I still don’t have a perfect mental model of the data placement in Photos but my current understanding of what has happened is this:

  • I consolidated (copied) all my master files from my external drive to my Mac’s internal storage
  • Photos proceeded to upload that entire collection to iCloud
  • Once the photos were in iCloud, the consolidated masters were replaced by mac-optimised versions
  • On the iPhone and iPad, Photos was aware of the existence of these images but no data was downloaded to any device until required.
  • When I accessed the Years view, and tiny thumbnails were required, these were loaded from iCloud
  • When I accessed any individual image, a high-resolution version was downloaded from iCloud. A circular pie-chart progress meter in the corner tells you this.

Steady State Operation

7 days into the migration, I woke up to find my iPad and iPhone in total agreement about the number of photos I finally had:

  • Mac Library: 109GB (as reported by du(1))
  • iCloud Storage: 269.3GB (as reported by Settings > iCloud > Storage)
  • iPhone on-board storage: 10.3GB (as reported by Settings > General > Usage > Manage Storage)
  • iPad on-board storage: 8.6GB

At this point, my Mac’s photo count was still ahead by 21 photos. I’m now assuming that there are 21 photos that are either corrupt or missing their master files in some way somewhere in my photo library. I doubt I’ll ever find them.

Sync performance is good at this scale:

  • Photos deleted from one device disappear from the other in under 5 seconds.
  • Photo edits on one device appear on the other in about 15 seconds.
  • New photos from the phone appear on the Mac in under a minute (I have image optimisation turned on at both ends so this is likely to take some extra time).

It’s rare that I’ll be wanting or needing faster syncing than this. What most people really want, I’d guess, is “my photos are on the other device when I get back to it” and Photos certainly seems to be offering that. There is one caveat, however: Photos/iOS will not upload newly-taken photos unless on WiFi and there seems to be no way around this. I have Settings > Cellular > Use Cellular Data For: turned ON for Photos but it won’t upload unless the phone is on WiFi. I understand why that is the way it is but I have an eat-all-you-can data plan for my phone and it’d be nice to have this.

Overall, I can say that I'm really very pleased with Photos for OS X and with iCloud Photo Library.