There are many possible techniques to use for cleaning up a shot, ranging in difficulty from extremely easy, to mind-numbingly complex. How hard it is depends on how complex the background behind the object being removed is, and what might be occluding the object being removed in various parts of the shot.

For example, if there is a large unfortunate piece of rig that happens to be in front of a complex and defined tree-branch blowing wildly in the wind, occluded in the foreground by a healthy wisp of smoke, cue the nightmare scenario. Basically the aim of cleanup work is to re-create the background behind the object needing to be removed, such that a person can’t tell there was ever anything there.

Here are some of the techniques used to do this.
2D or 3D tracking of still “cleanup” images into shots: this works well for background objects that are not deforming, for example, the sides of buildings, trucks, rocks, and other hard things. This technique does not work as well for soft moving things like people, clothes, energetic trees, and water. Another thing that confounds this technique is interactive lighting changes. If there is a flickering light on the side of a building, using a still image to clean up something on the wall of said building will look out of place, unless a keyframed color correction is applied to match the lighting changes.

Cloning one area of an image to another area, in order to cover something up: This works well for shots where the background of the object needing to be removed has a moving texture. For example, for something like ripples in water, still image “patching” will not work because the ripples in the water have to move. Since the texture of the water is ideally relatively consistent in its pattern of ripples, cloning from one area of the frame to the other might not be noticeable. However, if the background’s pattern is non-repeating or complex, this technique might easily be foiled.

Clone Painting: This technique is varied and quite effective with the right tool in skilled hands. It is similar to wielding the “rubber stamp” tool in Photoshop, except that it must be kept in mind at all times that one is working with a moving shot. One can clone areas from adjacent frames to replace the background over a moving wire. In order to do this effectively, the plate has to be motion tracked and stabilized to the object being manipulated, so that the object being removed doesn’t change position from frame to frame. Clone painting from the same frame using an offset to remove something on a moving object also can work well. When cloning with an offset on consecutive frames, one has to be very careful in order to avoid motion artifacts that result from the cloning happening slightly differently on each frame. A first inclination might be to just clone out an object on each frame and be done with, but when you watch it back in motion, horrible boiling artifacts will occur over the object that is removed so perfectly when looking at each frame individually. Generally, offset cloning is easier to get away with on edges, and objects in motion, and harder to get away with on static objects that have subtle gradients.

Here are a couple of video tutorials on how to accomplish some of the things discussed, using The Foundry’s Nuke 6.0.

Watch in HD: 2D Tracking and Cloning from Jed Smith on Vimeo.

A simple tutorial in Nuke on how to clone from one area of a moving image to another, using 2D tracking and stabilization, and basic compositing. Uses a shot from The Hotdog Cycle, produced by The Last Quest in Seattle.
The Hotdog Cycle Trailer

Watch in HD: Temporal Clone Painting, Patching, and Grain Manipulation from Jed Smith on Vimeo.

A demonstration of a method of cleanup using clone-painting from adjacent frames on a stabilized plate, in NukeX 6.0.
The shot used is from the animation “High Strung”, produced by Tommy Thompson at The Evergreen State College.
Tommy Thompson’s Production Blog
A Short Documentary About The Project.

A collection of timelapses shot over the last year by myself, using my modest photographic equipment: a Canon 350d (Rebel XT), a Canon EF 35-105mm f/3.5-5.6 zoom lens, and a Sigma EF-S 10-20mm f/4.5-5.6 zoom lens. Most were shot in Raw, and especially the cloud sequences have extensive post color correction.

Download 720p Version
Watch on Vimeo
The music is Buralta by Fedaden, off of his new LP Broader (Beatport.com is the only place that has it in lossless, and it costs a ridiculous $25).

I shot my first timelapse a little more than a year ago. Above is a compilation of the best ones that I’ve created. I have learned a few things about timelapse:

1). Shutter angle in timelapses is very important. In stop motion animation, the strobing look of objects moving without motion blur is part of its visual aesthetic (except when counteracted by techniques such as Go Motion). In timelapse, since the subjects move by themselves, very filmic results can be achieved. The trick is to think about shutter angle, and to adjust your camera’s settings accordingly. Tyler Ginter wrote a more in-depth post about the technical and aesthetic considerations of Shutter Angle, but my description of it in application to timelapse follows.

Most films are shot with what is called a 180º shutter angle of a rotary shutter. This means for every frame, the exposure time is about half of the frame interval. Traditional film runs at 24 frames per second, so the time interval between each frame for both capture and display is 1/24th of a second. At a 180º shutter angle, each frame of film is exposed for 1/2 that time, or 1/48th of a second. This produces a level of motion blur that we as an audience are familiar with and are quite aware of as an innate aspect of our viewing experience, even if we don’t understand what we are seeing technically. The blurring of moving objects mimics the way our eyes see (just try flailing your hand around in front of your face), and helps the space between frames disappear, producing a pleasant illusion of movement. If you are shooting a timelapse of some clouds, and your frame interval is 2 seconds, your exposure time should then be about 1/2 of that, or 1 second.

Of course increasing or reducing your shutter angle has dramatic perceptual effect, which should be fully understood, and used carefully. Two dramatic examples of this are Saving Private Ryan and Apocalypto. Saving Private Ryan used a “narrow” shutter angle of 90º or even 45º for some combat shots, for a shorter exposure time and less motion blur on each frame, to produce a dramatic strobing effect, which accentuates the violence and physical proximity of the images. ( This effect is quite apparent in the final battle scene from the film ). Using a very short exposure duration interrupts a natural viewing experience.

If you take exposure too far on the other side of a “normal” shutter angle, strange results can occur. Many people who saw Apocalypto in the theater were confused about how some scenes looked like they were “shot on video”. They had a very fluid and “lifelike” motion aesthetic, which evoked “soap opera” or “news footage”. In addition to media being shot at higher framerates like 30i (NTSC DV), or 60p (capable by some HD cameras), this perceptual effect can occur at 24 frames per second if the shutter angle is set to be 360 degrees. Normally this setting would be impossible in a conventional film camera, but the shutter mechanisms of digital cinema cameras such as the Panavision Genesis easily allow the shutter duration to equal the frame interval. Some other movies that are afflicted with this problem are 2012 (shot by Dean Semler, the same DoP as Apocalypto), as well as Public Enemies (360º shutter is clearly visible at 00:58 in Trailer 1).

2). Make your shot look good as a compelling still photograph should before turning it into a timelapse. Composition counts doubly as much in Timelapse than in still photography. This should probably be pretty obvious, but it took me a while to realize, and even longer to implement properly. If you shoot a still photograph with bad or uninteresting composition, you can always stop looking at it, or move on to the next photo you shot. Viewing timelapse, you will be looking at a static composition for the duration of the shot, unless you have a motorized tripod head, or a motion-controlled dolly system. Make a conscious effort to take test shots and make sure your composition is spot-on, before you start shooting frames. Once you start shooting, the best plan is to walk away from your camera until the quantity of time you have calculated will be sufficient to get as many frames as you need. 15 seconds, or 360 frames is probably the minimum duration desirable for a single shot, depending on what you are shooting for.

3). Preplanning is just as important in timelapse as in animation. It is distinctly valuable to analyze the motion of the subject you are shooting, and determine what the best choices for achieving the result you want. Adjust your frame capture interval based on what you are shooting, and adjust your shutter speed accordingly in order to achieve the amount of motion blur you want. Try to imagine what the final timelapse will look like before you shoot anything, and think about how you could make it better.

4). Have fun, because timelapse is awesome, and can visualize movement in a completely novel way compared to our normative perceptions.

Most of these little projects are primarily technical experiments, which serve to demonstrate a skill, and which provided me with some much needed experience. Therefore they are significantly lacking in any sort of conceptual and creative capacities, which are so essential to good work. However, competency applies much more than creativity in a junior level rotoscoping job. Don’t worry, I feel a boiling of creative output gearing up to burst forth sometime in the next 1-5 years.

This is a piece of music written by family friend David Lamb in memorial of the death of my mother’s father. It is played by my grandmother (violin) and her brother Greg (piano), who is now also passed away.

Hearing it again reminds me that life is short. I have not been making the best use of my time. I moved to the Bay Area nearly 7 months ago. Since I’ve been living here, I have done a little timid exploring, very little extremely timid socializing, and a whole lot of keeping my cognitive tendrils embedded in the extended reality of my computer, with its vast, tempting, and marginalizing wealth of information and connectivity. While enabling great feats of externalized memory storage and access, and augmenting capabilities of information processing, storage, and organization, it seems at times that living life so absorbed in this abstracted processing tool results in an overwhelming reduction of critical thinking ability and other aspects of intelligent behavior.

I recently went to a ‘lecture’ by Gerry Fialka, as recommended to me by my friend Will Erokan. This consisted of a whimsical introduction of sorts to some of the theories of Marshall McLuhan, centered around his ideas about the tools humans use and how they affect us. “We shape our tools and they in turn shape us.” This approach of Technological Determinism is an interesting one. It posits that humans create inventions and spur on the growth of technology, and then this technology in turn affects humans in ways we cannot understand or predict. McLuhan urged the importance of studying the effects of our technological inventions. This is not an easy challenge, but an important one, if we are to maintain enough self-awareness and self-knowledge to remain respectably intelligent creatures.

I just finished reading Glasshouse by Charles Stross. This book and the prequel of sorts, Accelerando, took me by surprise and got me all excited. Strossian speculative fiction forms a remarkably imaginative post-cyberpunk vision of future technology, building on the work of Stephenson and Vinge, and also more obviously extrapolating from contemporary technological trends. He grapples with the idea of the technological singularity, and imagines a potential and compelling post-singularity environment. Extremely interesting concepts are explored such as the augmentation of human sensorium and cognitive ability by means of integrated ‘wetware’, the abstraction of self and consciousness from physical identity by means of matter compilers and scanners, as well as the abstracting of reality by means of artificial reality subsystems so sophisticated as to be indistinguishable from ‘real.’

In his theory of anamnesis, Plato posits that writing is a device of artificial memory, both in its storage of knowledge through written language, and its ability to cement information in the memory through the act of writing itself. If writing is a device of artificial memory, the computer could also be considered a device of external information storage. However a computer possesses much more generalized and powerful capabilities of information storage than simply written language. It can capture and store audio and video, pictures, documents, books, magazines. In addition to this, when connected to the internet, it can easily function as a universal device to quickly find information.

I spend a lot of time on my computer, searching for specific things, finding things accidentally by association with other things I happen to be looking at, and reading about new things. Increasingly, I find myself depending on my computer as an external information storage device instead of committing things to memory in my own brain. For example, I will try to remember the name of a particular movie I recently watched, and can’t remember what the name is, but I can remember exactly what folder it is in on my computer, and what letter the name begins with — I have a generalized sense of the data’s location in my own mind, but I don’t have the data itself.

If you can access information in external memory storage, then why commit it to memory? I think this will only become an increasingly relevant question as technology progresses. Or maybe it will become increasingly irrelevant. It seems relevant now because there is such a large distinction between external and internal memory storage, but 50 years from now, this distinction will likely be irrelevant.

Then again, maybe it’s just how I’m using the computer that is causing me to become cognitively impaired. In Anathem, Neal Stephenson lays out a clear distinction between syntactics and semantics. In the world he creates, there is a separation between the Mathic order, and the Saecular powers. The Mathic world has willingly abstained from technology, yet is devoted to “theorics” (academic study of math, sciences, philosophy, and other disciplines). The outside Saecular world is abundant with technology, yet the users of the technology are primarily unintelligent. The allegory is pretty clear. The interesting idea is that there is a clear separation between academic study and the effects of technological advancement on people. The ideas — the concept and the meaning are always more important than the technology. The technology provides tools to implement the ideas. A syntactic device running by itself will do nothing interesting. Only through input structured by ideas will it output meaningful information. The ideas are what is important.

How technology affects people, the way that they think and act, their ideologies, and their culture, is a very interesting and increasingly relevant question. I find that often while working at my computer I become overwhelmed, and begin to multitask and to split off into iterative threads of distractment, until my original focus on a task at hand is nearly forgotten about. I will literally stop the furious clicking and typing suddenly, realizing that I have forgotten what I was supposed to be doing. Maintaining self-awareness and direction and focus is a difficult thing to sustain in the virtual realm of endless information, minimized barriers, and infinite distraction. There is much to learn, but the brain can only absorb so much at one time; that is, until computing systems and cognitive systems are more closely integrated.

The act just completed of finishing a book was a refreshing one, and I think gave me new perspective on this problem. Reading on printed paper engages one’s brain in a different way than reading webpages on the internet, and I think it might just be possible to understand this difference in thought process, and control it to one’s own advantage.

I had the interesting experience of going to a club and seeing Tipper perform two nights before. For quite a while now, I have been excited by electronic music, and the potentialities for interesting new sonic and musical territories to be explored. All too often (as in many disciplines), I find the majority of electronic music to be uninteresting or even repulsive, because it strongly adheres to established patterns of style and form, is often rhythmically unsophisticated, does not experiment nor innovate, but instead self-congratulates and regurgitates itself endlessly.

As a relatively Anti-Sphexish human, I am predisposed to be repulsed by things that self-regurgitate endlessly. I tend to be interested and excited by things that push accepted boundaries and experiment, and that offer compelling, internally consistent, emotionally powerful, tantalizingly complex, and genuine (in the sense of sincere, profound, and non-cynical) “art”.

The experience of seeing Tipper, Beats Antique, and Anten-nae at the Ten15 Club in San Francisco was an ambivalent one for me. I was excited by the environment of the experience. The 1015 club is truly quite impressive in regard to its light and sound technology. Entering the club was an experience akin to what stepping into the future might feel like. You know when you’re watching a movie, and they have a scene in a nightclub that is intended to evoke “future”? Like that. The ceiling is made of illuminated and animated color, there are projections of abstract patterns sweeping the floor and flaring in your eyes. In the middle of the main dance floor, the sound system is so precisely tuned and so powerful that it sounds amazing, does not hurt, and has close to the most insanely intensely powerful bass I have ever experienced.

With things that impress at first however, often the initial awe breaks down as the experience continues. The show that I saw here was primarily “DJ Entertainment”, meaning that there was a corner of the room with a guy behind a large stack of complicated electronic sound devices and a laptop computer, who (depending on the ‘performer’), would occassionaly wave an arm around or rhythmically adjust headphones on his or her head.

Electronic music is a strange phonomenon. Usually when you go to a concert (historically speaking), you are expecting a performance. You stand facing a stage, on which there are musical performers that you idolize performing songs that are ingrained into your musical memory. This breeds an excitement and an experience of anticipation and release, which is … enjoyable.

With a DJ Performer, there is a disconnect that happens, because while there is a person there making the sounds happen, said person is not necessarily performing said sounds in the concretely recognizable way that a person playing a guitar and singing performs his songs. Additionally, there aren’t individual songs, but rather a long continuous evolving musical structure. The music itself becomes as much of an attraction as the performers being physically present. What then is the difference between sitting in your room alone listening to the music on a home stereo system, and going out to a club and listening to a DJ set? Most notably, there is a sense of camaraderie in being with a large group of other people enjoying the same music as yourself, and dancing. Also the sound system is a lot better than your home stereo. Still, it seems like there is an important difference between a ‘traditional’ concert, and a ‘DJ set’.

In my limited experience of such things, it seems that in the culture of those who go to electronic music shows a lot, the primary attraction is electronic music, dancing, and drugs. Often the experience of the dancing and the “party” atmosphere seems to be considered more important than the music itself, and the quality of the music suffers. Some people might not care, but this environment is not an attractive one to me.

In the last year or so I became relatively enamored of the underground Bay Area ‘crunky’ ‘glitch-hop’ style electro dubstep characterized by the music of edIT, Ooah, Boreta, Bill Bless (Squarnch, Heyoka), Skeetaz, EPROM, and others. edIT’s amazingly nuanced and beautifully emotional album Crying Over Pros With No Reason was one of the records that got me interested in electronic music in the the early days (Summer 2005). His newer music (Certified Air Raid Material) forms an interesting hybrid of the DJ set style of performance and the more traditional song-based structure. He and a group of like-minded musicians have been touring together under the name The Glitch Mob, using an interesting performance structure where they play each others songs in a linear structure, but there is preserved a nearly improvisational performance structure, where the core rhythmic and textural components of the songs are in place, but the structure and the nuances of the songs can be varied each time. There is something compelling about the fact that music is being created on the spot in a performance, and The Glitch Mob come closer to this notion of “performance” than more traditional ‘rave’ or ‘discotech’ style events.

There still seems to be an aura of the “dance party” mindset to even this marginally avant-garde collection of underground electronic musical stylings which I find to be distasteful however.

The real subject that this post is about is the Danish band Efterklang. Efterklang embody just what I love to see in electronic music. Their music is fundamentally constructed around the idea of compositions — songs that are structured in such a way as to have emotional dynamics, crescendos, harmonies, and real depth of feeling. The electronically generated or manipulated sounds are treated as just another instrument with new expressive capabilities, and exists with a larger structure of many other instruments; guitar bass and drums, piano, violin, trumpet, flute, homemade whistle and rattles, and vocals. This plethora of instruments are utilized each in their own uniquely expressive way to create a whole that is beautiful, complex, powerful, and affecting. Much of the purely electronic music I just described is lacking in this “whole”, and relies too much on rhythmic repetition of simple musical ideas, which may be good for dancing too, but to what end does one dance?

The point of this post is actually not to pontificate at length about the intricacies of musical preference and politics, but rather to post some of the Efterklang concert that I recorded on 2009-03-10 at the Bottom of the Hill pub in San Francisco, California.

Here are a couple of songs from the show. If you want to watch it all, there is a youtube playlist. Be warned there are some audio synch issues with youtube and the mpeg4-avc files I uploaded there. I have been too lazy to fix it so far. You can also download the show in 720p files, split by song.

Here is an older song called Chapter 6.
[flv:http://www.youtube.com/watch?v=gJ9GaYvEcx4 640 360]

This is a new song roughly titled “Piano Song”.
[flv:http://www.youtube.com/watch?v=B81xVD-JZOk 640 360]

Here is a professionally shot video of Jojo.

This is what Efterklang’s music really sounds like.

The show was recorded with a Canon HV20 HDV camcorder. I had never been to this venue before, but managed to find a good spot for recording perched on the drink counter at the edge of the room. It was a good spot for video recording, but unfortunately near a subwoofer, and in a bad spot for capturing the midrange PA speakers. I recorded audio with an iRiver H120 + binaural mics also, but the audio from them turned out overdriven and unusable. The HV20 mics actually did a really good job (with ATT turned on). The audio you here is just the straight camera audio, with a bit of multi-band compression to bring out the highs. The video was shot in 24F HDV captured with Final Cut Pro, edited, and brought into After Effects, and exported at 1280×720 23.976p as Avid DNxHD, and then encoded using Mpeg Streamclip and x264 as dual-pass 3000kbps video and AAC audio at 192kbps.

After 4 years living in Olympia, I am moving away. I graduated with a Bachelor of Arts in Media studies and film/video animation photography and music technology from the Evergreen State College in Spring 2008. Now in about 10 days I am driving my car, containing all of my possessions south to Berkley California, where I will store my things and board my dad’s sailboat, and whenupon we shall depart southward, with the destination of Mexico.

As the picture above indicates, I have not been sailing in a while. So long ago it was that I do not remember it. If the suffering of motion sickness and the terror of waves and of being on a boat does not consume me first, perhaps I will adopt the safety measure of a tether, as I had when a toddler.

After my quarterlife crisis comes to a state of calm, I will embark upon a mission to find residence in the bay area, and employment.

I had the good fortune to see them live in Seattle on September 13th. They were opening for Delerium, and sadly played without video projections, and for only the short period of 28 minutes, however their show was amazing. Here is the 3rd song they played, “Mend”.

Elsiane – Seattle, 2008 – 03 Mend from Jed Smith on Vimeo.

You can download the full show in 720p MP4-AVC, or see the other songs from this show on vimeo.

Due to the inevitable migration of my sleep schedule during times without large quantities of obligation, I happened to be awake at about 7am one morning. I felt like going for a walk, for it had just stopped raining. The rain was the first in a couple of months. The sun was just rising behind some clouds, and the light looked quite pleasant. Armed with my Chinese macro extension tube, I took pictures of leaves and water droplets, and other inane subject matter. Brilliant.

The hardest part of the job was the setup, which took two days. We were working on a barge just near Glacier Northwest Concrete in Tacoma. Pyrotechnics work is not as glamorous as it might seem, it consisting mostly of shoveling sand, stringing squib wire, and loading mortars in tubes in the right order. I had altogether too much fun doing even this, however.
And, some photographic evidence of the fireworks show itself. Officially, I got to wear a fireman’s suite and be on “fire duty,” which was supposed to consist of running around with a fire extinguisher and putting out flames that might threaten fuses that might launch the wrong shells at the wrong time, however, because 3 other people were already doing that, I ran around with cameras instead.

Also, if you are into pictures of me, and captions, check out Dave Cramton’s pictures.

Doing work with fellow film geeks, of course we ended up making a video glorifying the 10″ fireworks shell, creating a somewhat silly loading tube ceremony. This ended up being an intro to the other footage of the fireworks show, and the aftermath, that I shot.

2008 Tacoma Fireworks from badur on Vimeo.