Green Screen

My son Antonio makes a lot of movies with his friends and puts them up on youtube. And by lots, I mean every few days he uploads a new video. Lots.

iMovie has a green screen feature which wasn’t working out so well against our semi-green walls. So, Antonio ordered a green screen and paid for it with his own money. It took about a week and finally arrived the other day. I didn’t expect such great results. I mean, the lighting was crude and the sheet came terribly folded. It seemed like the video would look simply terrible. Below is a video of his first test using the wonderful sheet of green. And by wonderful, I mean holy shit it works so well!

NO BE CHICKEN

Love this.

NO BE CHICKEN

CHICKEN LIVE IN CAGE. NO CAN HAVE PERSONALITY INSIDE CAGE.

LAST STEP IS SMASH CAGE, LIGHT BARN ON FIRE.

DO THAT, YOU WIN.

Via http://www.avc.com/a_vc/2011/09/minimum-viable-personality.html

Football

So, football season has begun on many levels. I’m happy the Jets have won their second straight game. They looked good today, as certainly handled the Jaguars better than they did Dallas. I was at the game today and the mood was pretty damn good! The Patriots are also 2-0 as are the Buffalo Bills. I have a soft spot for the Bills since I was a pretty big fan in the early 90′s when they made four straight runs for the Super Bowl, yet were denied each time. Brad Smith went to Buffalo from the Jets this year, I hope the Bills make good use of him and they have a great season.

Marco and Antonio are doing the Long Island Broncos again this year and each are having a ball. Last year they were rookies but this year they are both stronger and more confident. Each has been having pretty good game time and both are playing running back this year. They’re both on the small side, but both very quick so that position is a good fit for them.

And my oldest is starting to work his way into athletics which I’m thrilled about. Giovanni was never much into sports. But this year, things have changed. This past Spring he did track and field and this fall he’s doing flag football. He’s on a team with all his friends from school and he’s having a great time. Lisa and I are so happy for him.

And on the tech side of things…

I’m video taping every game and am starting to run out of disk and backup space again. Tonight, I’m spending some time going through some old footage and compressing it from 1080i/25Mbps to a more modest 720p/123Kbps. That’s a 200x space savings and the quality difference is minimal for every day viewing. If I want to use the footage in a second video, quality will degrade further but this is a sacrifice I need to make right now.

I maintain 2TB of drive space and 2TB of backup space. At some point, I’ll need to double this to 4TB/4TB but to do this, I need to start thinking about RAID devices like a Drobo or Cisco array. My budget can’t handle that right now so I’ll just need to compress things for now.

Robot

Robot

iPhoto and iMovie optimization on Mac OS X

If your iPhoto and iMovie libraries on your Mac grow to anything beyond a few gigs, these programs can begin to slow down. In this post, I discuss how I resolved many of these performance issues. If your video archive is small, you won’t notice any of these problems and you probably don’t need any of these solutions. Most of what you’ll see below are ways to make iPhoto and iMovie see only the important stuff; divide and conquer.

I hope you find this post useful for helping to manage your growing iPhoto and iMovie libraries. If you have your own techniques, I’d love to hear them.

iPhoto and iMovie take a long time to load

For some reason, iPhoto and iMovie are very sensitive to how big your library is. Very sensitive.

iMovie in particular has a nasty habit of looking through all of your events and making sure they are all “optimized” when it starts up. Having a big library makes this very slow, particularly for certain kinds of videos like 3g phone videos iMovie doesn’t like to deal with. The solution here is to remove the content from the UI so it’s not scanned.

One way to accomplish this is to simply delete the event. Create a project, make a movie, export it to iTunes, and delete the event. If you delete the clips, though, you may as well delete the project that used the footage because it’s useless. If you’re like me, you like to keep around the original clips so you can make multiple movies from a single event. I wish I could purge like this, my data would be a lot easier to manage.

What I did to solve this problem without losing the original clips was to move all of my events to a “special” folder. By “special” I mean a folder not managed by iMovie. Then, I created symlinks into the standard ~/Movies/iMovie Events folder and iMovie Events folders at the root of each hard drive. These folders is where iMovie looks for events. This allows me to instantly filter what iMovie can see without moving gigs and gigs around my hard drive.

lrwxr-xr-x    1 nic  staff    58 Jun  5 18:06 Antonio Math Bee Grade 5 -> ../Movies/iMovie Events.localized/Antonio Math Bee Grade 5
drwxr-xr-x  157 nic  staff  5338 Sep 11 23:03 Broncos 2011 Week 1
drwxr-xr-x   44 nic  staff  1496 Jul 23 09:57 July 2011
lrwxr-xr-x    1 nic  staff    44 Jun 21 16:22 June 2011 -> ../Movies/iMovie Events.localized/June 2011a
lrwxr-xr-x    1 nic  staff    44 Jun 21 08:55 March 2011 -> ../Movies/iMovie Events.localized/March 2011
lrwxr-xr-x    1 nic  staff    41 Jun  5 18:06 Ohio TC -> ../Movies/iMovie Events.localized/Ohio TC
lrwxr-xr-x    1 nic  staff    45 Jun 21 08:54 Shananigans -> ../Movies/iMovie Events.localized/Shananigans
drwxr-xr-x   24 nic  staff   816 Sep 11 23:03 The Fight
lrwxr-xr-x    1 nic  staff    44 Jun 21 16:25 Track 2011 -> ../Movies/iMovie Events.localized/Track 2011

The entries with -> after them are symlinks and the path to the right is the actual location of the clip directories. iMovie doesn’t know the difference so it just works. Notice how some events are not symlinked. This is because iMovie by default writes real directories. It would be great if iMovie used symlinks, more on this soon.

To create symlinks, move to the directory where you want the symlink and type this at the Terminal:

ln -s /path/to/original/file

Time Machine backups are a bit of a problem, though. By default, the movies are imported into ~/Movies/iMovie Events and iMovie Events folders at the root of each hard drive. These obviously aren’t symlinked as I mentioned. So, when I have accumulated enough Events that I want to start hiding some, I’ll need to physically move these to my special folder. Not such a big deal, but this kind of move prompts Time Machine to backup the files in the new location again. This was a big problem since my backup drive is not much larger than my regular storage. I had to wipe the backup and backup clean to avoid out of backup space warnings.

As I mentioned earlier, this whole mess of managing movies would be best solved by a future version of iMovie and not by hacking the file system. For instance, it would be nice for iMovie to facilitate hiding events and projects from the main UI so the main UI doesn’t spend so much time accessing these older events.

Clicking around iMovie cases a lot of access, so the UI is largely unresponsive in many cases

This is a ridiculous feature, IMHO. The point of the feature is you can click on a project name in the browser and play the project without actually opening the project. The problem is that to move a project into a folder, you need to first click on it. This is an EPIC FAIL in UI as clicking causes enormous delays while the preview feature is launched.

The “iPhoto Videos” event just gets bigger and bigger and takes longer and longer to load

Most cameras can take pictures and video. iPhoto allows you to import videos taken on your camera and places them in events. This is not such a bad thing. What’s bad is that the videos all funnel into a single event in iMovie called “iPhoto Videos”. Clicking on “iPhoto Videos” causes iMovie to load previews of every video in your entire iPhoto library and as your library grows, so does your wait time.

It would be better, IMHO, to have the iPhoto Videos folder expand and show you the events where your movies lay. This would make things much faster. Since this isn’t the way it works, I have banned myself from importing videos into iPhoto. This is kind of a pain as I need to introduce a second step when grabbing content off my camera. I use the Image Capture application to grab the videos as a first step and drop them onto my hard drive. Then, I drag the videos into iMovie. Finally, I import the rest of the pictures into iPhoto.

Meh.

Moving clips from event to event takes forever, a lot longer than manually moving the underlying files

It’s possible that this problem is specific to my system. At least the crashing part. Yes, in many cases moves never finish and eventually result in a crash. Your milage may vary. Moving clips is generally a complex operation since I believe all projects referencing the clips are scanned and updated with the new location if necessary. The net effect is that moving clips from event to event is massively time consuming. Again, this is a basic problem that can probably be solved behind the scenes with symlinks or pointers to clips somehow. Moving clips on the same hard drive should be more or less instantaneous.

Still, I try to move clips often to make sure Events aren’t too large and clicking on a event doesn’t need to scan too many clips.

Projects can easily lose access to the clips they use if you go mucking around with moving events

In the original iMovie HD, clips were stored internally in the movie project file (a package) and things were very reliable. The advantage to the new UI, of course, is more flexibility. You can use event footage in more than one project. If you use the symlink technique I spoke about earlier, you may start getting these kinds of errors when referencing clips that have no symlinks from projects. All you need to do is create new symlinks as described above.

iPhoto slows to a crawl

iPhoto performance began to get intolerable when my library started to exceed 50GB or so. At my library’s peak, it was almost 90GB, over 30k photos, and 10 years of events. I am sure this is not large by professional standards, but it’s plenty too large for iPhoto to deal with.

The easiest way to bring sanity to performance is to simply create a second iPhoto library: divide and conquer. This is easy as all you need to do is to a) close iPhoto, b) rename the default library, and c) restart iPhoto. When you do this, a new library will be created automatically for you. You can get the old library back by clicking on it in the finder. Once you do that, the library you clicked on will be the default o ne. So, to get the new library back, you need to exit iPhoto and double-clic on the new library.

One caveat to this is that you lose your Faces database. But I do not use that feature.

There are some good iPhoto library managers out there but IMHO, this is the kind of thing that should be built into iPhoto. Or, better yet, Apple could spend time optimizing iPhoto and make this whole technique unnecessary. After all, Google does fine with Picasa.

Conclusion

I’ll conclude with a bunch of tips and links you may find useful:

  • Other than creating symlinks, don’t muck with event or project folders in the finder or the terminal. The names of the event and project folders is essential to keeping projects properly linked to the underlying clips.
  • You can rebuild your iPhoto library, thumbnail caches, etc., by holding down the Command and Option keys on the keyboard after launching iPhoto before the main window shows up. IMHO, this is stupid. These features should be available via preferences. Perhaps they can fuck things up for the user so Apple would rather they be hidden. Meh.

Clancy Brown

I was watching an episode of Phineas and Ferb, an excellent kids show on Disney. The episode was Phineas and Ferb Get Busted and a prominent character in that episode was the drill sergeant. I recognized his voice, thinking he was maybe Lex Luthor from the DC Animated Universe cartoons and I even thought he sounded like Mr. Kraabs from Spongebob Squarepants.

Well, I was right as the actor playing the drill seargent was Clancy Brown and he played all of those voice roles and a ton more… in addition to countless TV and movie roles. Recall he was the Kurgan in the original Highlander movie?

If you haven’t seen Phineas and Pherb, I highly recommend you introduce your kids to it.

Clancy Brown

Logo Contest Source Code

The source code for my previous post on the TechCon 2011 logo is up at github: https://github.com/NickCody/TechCon2011.

Data URI’s

Inlining images in base64 encoding is a pretty neat trick. Not all that useful in most scenarios, since http connections are most likely all persistent these days and browsers tend to cache image resources. Still, I find this approach interesting when you want to limit the number of files you deliver or you can’t really reference external resources at all.

The basic technique involves two things: encoding your image in base64 format and inlining the resulting string into your img tag. Here is an example:


<img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAGAAAABICAMAAAA+uQBRAAAAGXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAAABJQTFRF////zMzMmZmZZmZmMzMzAAAA8496aQAABHBJREFUeNrsWNGW6jAIzAD9/1++DCRNWrVtovt2czyuusokMAyEUu4XRIDyV0t0y2Wm6kA/RYIYLbtZEVXbOtQvzoSwqDJ6SkCghmTqn61uPRyjdZfgEYQeav9HvtVNl7du3bq4R7j78xe5ickToJq3wZo7myd4Z96/OglgNYAPyOW7QJkGGCh5SRNnl0lsaBKA+5KRJm9hSF6tJ54kK+ovGdgRZnQarNNrGqC4ucEUYbYjDM0b+tdnAez1Jzgk8ZbO3wFmc0A/7Wn3mR49Oi1tl6njBrfje5vO4u0q+X8AwFS75MDXAB6Ego+pfAKQBa1jlDFq3RXJVgAiykFHRvss9yeAFbWOKHuCQmhLTpw6Asi0FLUohwKIv8AVABNjEUDDPW+yqAP4EU2XAOgaYS3mCYp+OoGYA8xLUQZBUIXHeWkhEzgqiYu4+v4XxLRF2QXTC7o7wLasQrXeJ4BXfO+RTNYA/Fcadt1a7JFHImvRAJyc6hGQsgigEWVkOVEGRHcRjDRUYdnLirnUKzp7MkWN1iVzwQYA52fWnAUpyiCkWCSSJJh2gI09V/kKQGpX6n+9/IaZAUBLZRXWACLK4SsSJoWj6ZokwBdq3aLc3yjtoOraEcCWOtO0cv6gFuMgEx7VppsoH68gThzpAN0+3l+HuFQuC6McAbU+jQBnIUL0+O5LQ6wrhLNvI7HiM+wAetyFJzYvPVBow1VcRRkvzaiVEeDQ3oC1od8t9pdXUT7XZKCMANU+2uptNuJiFGEwverO39d91ErNJBRPuB2+FgqECIe7/H6lt/fLwx1wB9C4Y/LSs3sy+xAuXu6CDv6QJ1e1I0YCYP+CDC29ZTTSe2aXUf6IkeWuM1VrDOqVRKzRlwLwLM2PGIiLn45wEWPJNopcShb52eSxjnQMlKzVwRJ3ODcL1gZkxaPNxHdQnek4oPv9I6Fy52xs2F3ECRBh7eG2SSUcblPpZETBwUZ/BwBv/zFpoNNwS6KLc7Dwu6GoB/D2hW8zcyhGbBUU9oREL7mBOtxBiE8WHH7qKBp9lG9fsnnWonO2JZKo3tVJG43QGpdGVWJofOfsQ/xJ7lmaAxbaLW3O4m6VPEHUfamcAiWaRGU3o7zAuH/uSaR1uiXDaWugCbzf071oS3T7sXWJAzHY96V5V76uOY2tnGU0AJekIhF/tkz3g5R2NsibqRDSQYhE6gDuIsvJxr31GspSpNf21Edk7jpXJB47QAkC3M/w0GZmtZEehNIRNEcVfM0S6pANQJ/4JQwJxlqX+d7GFJR3iwkUAYzZq/v88WmajuEF2zkSNV0UucuUCgCwtWSVnMtVHSpm3ge4vwrg2Ym0GsUy1qwUtB80VQsArdR0005MniCVdKljRNJIgOp2L4lSG0cCuWm1L+a9Imm9ecVFOJRSUicbj74YJbPCShQkrdqbNDIOlOUHQ3HuVtkOZIlVq4XrZ7P20AjZ3vTXvxrmZ3tbq8lfrKDIYcr+4+U+wkICTRDVVKX85RKU/+tl/RNgAMFwGP4bvg4lAAAAAElFTkSuQmCC">

The tag’s src=”data:…” bit is called the Data URI scheme and has been around for a while and is now implemented in most browsers.

The encoding bit is easy and can be done in most frameworks. I spend a lot of time in grails so to do this in groovy you just do this:

f = new File("tcsmall.png")
b = f.readBytes()
b.encodeBase64().toString()

This approach has some overhead. Base64 encoding only uses 62 or 64 characters of a possible 256 in a byte so overall bytes going over the wire will be more. You would normally only use this for very small images.

For the life of me, I don’t know why browsers don’t use this technique when you save web pages locally. My gut says loading the base64 encoded versions from disk is negligible. If browsers used this, they could save an entire image-rich web page into a single html page. Instead, what browsers do is they write a single html file and all image /css resources in a directory alongside the html. References to images are all then re-pointed to the files in this directory. Yuck. Maybe browsers do this specifically so you can have the images as files for easy access. Who knows.

Anyway, I see some browsers (like Chrome) use this feature for things like “file/folder” icons when viewing ftp sites. Unless the browser is using OS native calls to render these directly, it needs to do something if it renders html to display this kind of interface. In the case of Chrome, it uses data uri’s and plain html to show a ftp user interface. I suppose a browser could reference files on your local hard drive, but seeing links in “view source” like: file:///C:/Users/ncodignotto/AppData/Local/Google/Chrome/Application/icons/folder.png, well, that shit just seems wrong.

Logo Contest

My company is holding a logo contest for our upcoming third annual in-house technology convention called Tech Con. Lots of people submitted logos and they were all awesome. My first idea was inspired by circuit boards and how they work words onto the actual circuit. The theme of TechCon this year is “The Big Picture” so this is what I came up with first, in about 10 minutes:
Still, I wanted to try something else. I am not sure why I had the idea for children-style alphabet blocks but I figure I could create something with povray, a ray tracing program that’s been around for a while. I stopped using it when I migrated to the Mac since it’s Mac support is firmly based in PPC land and emulated in Intel-land. Anyway, povray happened to come with a sample scene file which rendered blocks spelling “povray”. I was able to quickly modify this sample and change this to “techcon”. Techcon had the same number of different letters, 6, one for each side of the cube.
The first logo I created was this one:
First of all, if you click on any of these images they expand into a larger version that has a lot more detail. Notice how smooth the shadows are. Earlier versions looked terrible. The shadows are all rough and posterized. I dig into the documentation and found that since the scene used radiosity (where ray tracer takes lit diffuse objects lighting each other up into account), I had to increase a few settings in order to get enough samples to make everything smooth which was more than merely supersampling more.
The next logo compressed things a bit more by stacking the blocks, this was an easy change to some basic translations.
The next version, below, introduces some logic that I had previously disabled in the povray demo scene file I used as a basis for my work. You can see below that there are many more cubes randomly dropped via a trace function collision algorithm written by Greg M. Johnson in 2001. The main enhancement I added to this is a focal blur effect. Notice the blocks in front are in-focus and the ones in back are fuzzy. This is another expensive effect that I had to learn to use effectively. There was yet another setting I needed to bump up to get the cubes to be blurred in a smooth manner.
I even experimented in black, wondering what it would look like. To me, this version mainly looks like puke.
Knowing that the logo may need to be printed on the cover of a faux-moleskine notebook we give out every year, I brought a few of the logos into Photoshop and used the posterize and threshold filters to create a hip black and white version of  the logo.
So, the powerful ray tracers is thus reduced to a fancy way to create a low-fidelity graphic. But it did look cool.  You would not believe how obsessed I was at the creation of this. Rendering hundreds of variations, making tiny adjustments to lighting, camera, and block positions. After a while, I had most of the settings I wanted from a look and feed perspective, so I started experimenting with putting the year in, since it was an annual event:
Next, I incorporated the “Big Picture” into the logo and adjusted the color scheme. Here, the TechCon blocks are a mere annotation to the “Big Picture” cube.
Finally, I submitted my masterpiece. The result of like 30 hours of work. A hip black and white version of the logo that had all of the elements I was playing with and would look good on the front of a notebook.
Sweet, yes?
Guess which logo people liked? Yes, you guessed it. The circuit-board logo I started off with:

Batten down the hatches

Tomorrow, I leave for California with my family. Today, I battened down the hatches in preparation for hurricane Irene. The brilliant sunshine of the day was a bit ominous as the storm heads up the East coast. While the storm has just been downgraded, it still represents a bit of risk to my home since I won’t be here to address any issues should they come up. I took the day off to pack and it turns out that I was quite a bit busier than I expected.

Moving all possible projectiles into my garage was just the start. I emptied my pool a few inches and left the evacuation hose out. If the pool floods, I may be able to have someone turn on the pump to empty it a bit more. I secured my BBQ to our jungle gym. I might even secure the jungle gym to the trees in my backyard. I have a special pump that activates every 60 seconds and turns off after a few seconds if it detects no water. It wasn’t set up right so I set it up properly and ran the hose away from my house. If a tree falls on my house or we lose power then there is not much I can do. Flooding from the excessive rain is my main concern and with the pump set up, there is little more I can do.

The goods news is my mom is going to ride out the storm in my house while I’m away. Her house is more or less in the flood zone so it’s smart to be out of there so you don’t have to evacuate during the storm.

I wish all my NY friends luck this weekend.