New Year Outlook

No, I’m not going to talk about the way I think 2012 will go. Nothing that exciting. Instead, I’m going to talk about how I create a new Outlook Data file every year to contain e-items for that year. I did this originally because the PST file format had a 2GB limit. This limit has since been lifted, but I still practice this annual ritual of creating a new pst file for the year and cleaning out my inbox. I went and compressed each pst file and checked the sizes. Here they are:

Outlook Archive 2000.pst – 11,921 KB
Outlook Archive 2001.pst – 532,721 KB
Outlook Archive 2002.pst – 1,218,193 KB
Outlook Archive 2003.pst – 631,921 KB
Outlook Archive 2004.pst – 799,817 KB
Outlook Archive 2005.pst – 889,345 KB
Outlook Archive 2006.pst – 1,017,065 KB
Outlook Archive 2007.pst – 1,412,129 KB
Outlook Archive 2008.pst – 1,904,657 KB
Outlook Archive 2009.pst – 3,220,297 KB
Outlook Archive 2010.pst – 5,073,353 KB
Outlook Archive 2011.pst – 3,666,449 KB

I also have a 2010 Alerts folder… mostly automated e-mails I had on for a while, weighing in at 2,694,289.

How’s that for New Year Minutiae!

iPhoto and iMovie optimization on Mac OS X

If your iPhoto and iMovie libraries on your Mac grow to anything beyond a few gigs, these programs can begin to slow down. In this post, I discuss how I resolved many of these performance issues. If your video archive is small, you won’t notice any of these problems and you probably don’t need any of these solutions. Most of what you’ll see below are ways to make iPhoto and iMovie see only the important stuff; divide and conquer.

I hope you find this post useful for helping to manage your growing iPhoto and iMovie libraries. If you have your own techniques, I’d love to hear them.

iPhoto and iMovie take a long time to load

For some reason, iPhoto and iMovie are very sensitive to how big your library is. Very sensitive.

iMovie in particular has a nasty habit of looking through all of your events and making sure they are all “optimized” when it starts up. Having a big library makes this very slow, particularly for certain kinds of videos like 3g phone videos iMovie doesn’t like to deal with. The solution here is to remove the content from the UI so it’s not scanned.

One way to accomplish this is to simply delete the event. Create a project, make a movie, export it to iTunes, and delete the event. If you delete the clips, though, you may as well delete the project that used the footage because it’s useless. If you’re like me, you like to keep around the original clips so you can make multiple movies from a single event. I wish I could purge like this, my data would be a lot easier to manage.

What I did to solve this problem without losing the original clips was to move all of my events to a “special” folder. By “special” I mean a folder not managed by iMovie. Then, I created symlinks into the standard ~/Movies/iMovie Events folder and iMovie Events folders at the root of each hard drive. These folders is where iMovie looks for events. This allows me to instantly filter what iMovie can see without moving gigs and gigs around my hard drive.

[code lang=”bash”]
lrwxr-xr-x 1 nic staff 58 Jun 5 18:06 Antonio Math Bee Grade 5 -> ../Movies/iMovie Events.localized/Antonio Math Bee Grade 5
drwxr-xr-x 157 nic staff 5338 Sep 11 23:03 Broncos 2011 Week 1
drwxr-xr-x 44 nic staff 1496 Jul 23 09:57 July 2011
lrwxr-xr-x 1 nic staff 44 Jun 21 16:22 June 2011 -> ../Movies/iMovie Events.localized/June 2011a
lrwxr-xr-x 1 nic staff 44 Jun 21 08:55 March 2011 -> ../Movies/iMovie Events.localized/March 2011
lrwxr-xr-x 1 nic staff 41 Jun 5 18:06 Ohio TC -> ../Movies/iMovie Events.localized/Ohio TC
lrwxr-xr-x 1 nic staff 45 Jun 21 08:54 Shananigans -> ../Movies/iMovie Events.localized/Shananigans
drwxr-xr-x 24 nic staff 816 Sep 11 23:03 The Fight
lrwxr-xr-x 1 nic staff 44 Jun 21 16:25 Track 2011 -> ../Movies/iMovie Events.localized/Track 2011

The entries with -> after them are symlinks and the path to the right is the actual location of the clip directories. iMovie doesn’t know the difference so it just works. Notice how some events are not symlinked. This is because iMovie by default writes real directories. It would be great if iMovie used symlinks, more on this soon.

To create symlinks, move to the directory where you want the symlink and type this at the Terminal:

[code lang=”bash”]
ln -s /path/to/original/file

Time Machine backups are a bit of a problem, though. By default, the movies are imported into ~/Movies/iMovie Events and iMovie Events folders at the root of each hard drive. These obviously aren’t symlinked as I mentioned. So, when I have accumulated enough Events that I want to start hiding some, I’ll need to physically move these to my special folder. Not such a big deal, but this kind of move prompts Time Machine to backup the files in the new location again. This was a big problem since my backup drive is not much larger than my regular storage. I had to wipe the backup and backup clean to avoid out of backup space warnings.

As I mentioned earlier, this whole mess of managing movies would be best solved by a future version of iMovie and not by hacking the file system. For instance, it would be nice for iMovie to facilitate hiding events and projects from the main UI so the main UI doesn’t spend so much time accessing these older events.

Clicking around iMovie cases a lot of access, so the UI is largely unresponsive in many cases

This is a ridiculous feature, IMHO. The point of the feature is you can click on a project name in the browser and play the project without actually opening the project. The problem is that to move a project into a folder, you need to first click on it. This is an EPIC FAIL in UI as clicking causes enormous delays while the preview feature is launched.

The “iPhoto Videos” event just gets bigger and bigger and takes longer and longer to load

Most cameras can take pictures and video. iPhoto allows you to import videos taken on your camera and places them in events. This is not such a bad thing. What’s bad is that the videos all funnel into a single event in iMovie called “iPhoto Videos”. Clicking on “iPhoto Videos” causes iMovie to load previews of every video in your entire iPhoto library and as your library grows, so does your wait time.

It would be better, IMHO, to have the iPhoto Videos folder expand and show you the events where your movies lay. This would make things much faster. Since this isn’t the way it works, I have banned myself from importing videos into iPhoto. This is kind of a pain as I need to introduce a second step when grabbing content off my camera. I use the Image Capture application to grab the videos as a first step and drop them onto my hard drive. Then, I drag the videos into iMovie. Finally, I import the rest of the pictures into iPhoto.


Moving clips from event to event takes forever, a lot longer than manually moving the underlying files

It’s possible that this problem is specific to my system. At least the crashing part. Yes, in many cases moves never finish and eventually result in a crash. Your milage may vary. Moving clips is generally a complex operation since I believe all projects referencing the clips are scanned and updated with the new location if necessary. The net effect is that moving clips from event to event is massively time consuming. Again, this is a basic problem that can probably be solved behind the scenes with symlinks or pointers to clips somehow. Moving clips on the same hard drive should be more or less instantaneous.

Still, I try to move clips often to make sure Events aren’t too large and clicking on a event doesn’t need to scan too many clips.

Projects can easily lose access to the clips they use if you go mucking around with moving events

In the original iMovie HD, clips were stored internally in the movie project file (a package) and things were very reliable. The advantage to the new UI, of course, is more flexibility. You can use event footage in more than one project. If you use the symlink technique I spoke about earlier, you may start getting these kinds of errors when referencing clips that have no symlinks from projects. All you need to do is create new symlinks as described above.

iPhoto slows to a crawl

iPhoto performance began to get intolerable when my library started to exceed 50GB or so. At my library’s peak, it was almost 90GB, over 30k photos, and 10 years of events. I am sure this is not large by professional standards, but it’s plenty too large for iPhoto to deal with.

The easiest way to bring sanity to performance is to simply create a second iPhoto library: divide and conquer. This is easy as all you need to do is to a) close iPhoto, b) rename the default library, and c) restart iPhoto. When you do this, a new library will be created automatically for you. You can get the old library back by clicking on it in the finder. Once you do that, the library you clicked on will be the default o ne. So, to get the new library back, you need to exit iPhoto and double-clic on the new library.

One caveat to this is that you lose your Faces database. But I do not use that feature.

There are some good iPhoto library managers out there but IMHO, this is the kind of thing that should be built into iPhoto. Or, better yet, Apple could spend time optimizing iPhoto and make this whole technique unnecessary. After all, Google does fine with Picasa.


I’ll conclude with a bunch of tips and links you may find useful:

  • Other than creating symlinks, don’t muck with event or project folders in the finder or the terminal. The names of the event and project folders is essential to keeping projects properly linked to the underlying clips.
  • You can rebuild your iPhoto library, thumbnail caches, etc., by holding down the Command and Option keys on the keyboard after launching iPhoto before the main window shows up. IMHO, this is stupid. These features should be available via preferences. Perhaps they can fuck things up for the user so Apple would rather they be hidden. Meh.

Primordial Ooze WordPress Theme is up on GitHub

Although the ¬†theme is still not ready for prime-time, here is the link to GitHub where it’s ready for forking.¬† Upcoming changes will be around:

  • An article summary main index page (possibly)
  • Links to archives, rss
  • Links to the static files that make up the legacy for the original Movable Type Ooze (from 2003 to 2008)
  • Sleeker comment threads
  • More contact information in the footer
I’ll also try and add +1, tweet this, reddit this, etc. to my posts though I’m not so bold as to think anyone will ever actually click on these.