Condividi tramite

Collection Preview vs. Timeline Preview

  • Articolo

If you drag one clip to the timeline within Windows Movie Maker and preview it you may think that the process that goes on is very similar to when you preview the clip within the collection view. Well, although there are a few similarities there are a lot of differences. When we preview the clip from the collection we render the file directly using DirectShow. When we preview the timeline we use the DirectShow Editing Services (this was codenamed Dexter as you can see if look at the name of its type library, and this is still the way we refer to it internally).

Above is the graph that gets generated when we preview a file from the collection (the Windows Movie Maker sample file in this case). As you can see it’s fairly straightforward. The source filter on the left reads the bits from disc and outputs the raw video and audio streams. You can then see two DMO’s (DirectX Media Objects) that are responsible for decoding those bits into the actual video and audio data. The audio data is then passed to the DirectSound Device filter which will send the bits to your audio card, and the video data is sent to the VMR (Video Mixing Renderer) which will send the video frames to your graphics card.

OK. Now let's see what the graph looks like if you play a clip using DirectShow Editing Services.

This time you’ll see two source filters for the file as Dexter doesn’t share source filters across the video and audio streams. Note that if the same source file is used multiple times within the same timeline Dexter will share the same source filter within the same stream as long as it doesn’t overlap with itself e.g. if you have a transition.

On the left side you’ll also notice another source filter which is used to generate a solid colour, black in this case. Dexter uses this filter whenever there is a gap in video on the timeline, although this shouldn’t actually happen with the way Movie Maker uses it.

Dexter expect all audio within a single stream to be at the same format so you’ll see that after the audio is decoded there are a couple of other filters that convert and repackage the audio to Dexter’s liking. Similarly Dexter expects all the video within one stream to be at a single frame rate so it supplies a Frame Rate Converter filter to do the conversion if necessary. Movie Maker 2 tells Dexter to use a frame rate of 15 frames per second for NTSC and 12.5 for PAL. We use half the normal frame rate so that less data has to be processed through the timeline so we can get better performance. If the supplied video isn’t 320x240 then an extra filter will be inserted before the Frame Rate Converter that will resize the video to 320x240 as this is the resolution we use for preview. (See my blog about preview for further details on this).

After these, come two filters that are essentially the heart of Dexter and they are fundamentally big switches. They have multiple pins coming through and one main pin coming out. In this case we only have one clip so there aren't that many inputs connected. However if you imagine a timeline with a lot of non-contiguous clips then these switches will have lots of connected input pins. As the timeline is played Dexter will output the data from the pin connected to the relevant source at that time on the timeline and it will switch which pin is supplying the data as the timeline progresses as it moves from clip to clip.

After the switches, the audio data is sent to Dexter’s audio renderer and the video data is again sent to the VMR (after going through a color space converter which convert the video to match your graphics card if necessary).

So as you can see, even in this simple case there are quite a few differences between collection playback and timeline playback and you can probably just begin to imagine what some of these graphs can look like on a fairly complicated timeline.

Comments

  • Anonymous
    November 28, 2004
    Nice pictures...

    Would the picture be the same or similar to Dexter's project preview when you save a movie and render it to WMV.... and to DV-AVI?
  • Anonymous
    November 28, 2004
    Good question. When we render the timeline to a WMV or a DV-AVI file we will use a graph similar to the big one above (the one from Direct Show Editing Serices). The graph for saving the movie file is very similar to the one for timeline preview except that instead of hooking up the video and the audio streams to renderers they will be hooked up to an encoder to encode the video/audio data and a filter that will actually write those bits to a file on your disc.
  • Anonymous
    November 29, 2004
  1. One thing that is not clear is that Where is the place for custom effects and transitions in this graph? or is the big switch Dexfilt00000000A actually the custom filter?

    2. For the final render (not preview) does the resize from source size -> output size, still happens at the same place i.e. prior to processing?
  • Anonymous
    November 29, 2004
  1. Effects and transitions are rather interesting beasts as far as Dexter is concerned. They occur after the switch but their outputs are fed back into the switches as inputs. I'll try to post another entry shortly with more details and a diagram that will hopefully make this clearer.

    2. Yes the resize will happen at the same place (if necessary). Again I'll try to post something which includes some publish graphs in the next few days.

  • Anonymous
    December 03, 2004
    About resizing: Can you elaborate on the algorithm used to resize the inputs? By the way I dont understand why do you have to do the resizing early in the pipeline! I think it should be the last thing after all processing is finished. If the filters haven't produced an output of desired size only then you can enforce resizing for the output size. In any case the filters should be able to work on the the full resolution. This early resizing spoils the quality for many effects. For example pan zoom on still images. This also causes the chroma keying to fail in various subtle ways.

  • Anonymous
    December 03, 2004
    The comment has been removed

  • Anonymous
    December 03, 2004
    The comment has been removed

  • Anonymous
    December 16, 2004
    I asked around and no one seemed to know exactly what algorithm is used for the resize code. I understand your point about the early resize not being very good for transitions or effects that zoom in on the frame. Unfortunately that is the way that Direct Show Editing Services work today and it would probably be non-trivial to change it. I can’t go into any details but we’re certainly going to look to see if there is anything we can do to improve the situation in future versions of MovieMaker.

  • Anonymous
    December 31, 2007
    PingBack from http://movies.247blogging.info/?p=1313

  • Anonymous
    June 08, 2009
    PingBack from http://insomniacuresite.info/story.php?id=5510