COMPRESSION SYSTEMS applicable to Television and Multimedia Production

The difference between component and composite video and how that relates to COMPRESSION standards.

COMPOSITE VIDEO: luminance and chrominance signals are combined along with scanning control information as they leave the camera. ( Whittaker p. 261)

Sometimes this general approach to combining parts of the video into one composite signal is referred to as COLOR UNDER [color info. is put under the luminance signal in the bandwidth],

ADVANTAGES: Convenience -- especially for VCR recording

DISadvantages: Quality is reduced when the full bandwidth of the color signal is reduced and multipelxed into one signal.

• Interactions take place between the various signals, further reducing quality.
• Once they're together, the various signals can't effectively be separated and that makes further processing of the signal difficult.
• Quality is lost on a massive scale with each generation of tape copied. After the 3rd or fourth generation, the product is unusable

COMPONENT VIDEO: newer format (used with beta cam and S-VHS )

Luminance and chrominance signals are SEPARATED and recorded as two separate signals or components.

Y/C Video signals: (made possible by new metal tape formats)

• Luminance (Y) and chrominance (C) are kept separate during the encoding process, (write stage) and the decoding process (read stage). But both signals are combined and laid down on the same track. Still two wires are required to transport the component signal and all the equipment used to process it, must also keep the signals separate, and therefore are not compatible with NTSC.
• Advantage: Less generational loss.

Y/C/C :

• Three signals or components
• 1 luminance
• 2 color signals
• Currently y/c is most popular,
• Simpler, cheaper and can't tell the difference between the two in less than 5 generations
• RESULT: better resolution and detail.

[SEE: Migration to Digital from BSTG 2/28/94]

RGB Component Video: Here the red, green and blue signals are all kept separate and treated as individual components. Three wires are required for transport, and therefore all equipment used to process the signal must be RGB and cannot be NTSC compatible! RGB Component: Because of the detail retained in the signal, this is exceptionally resistant to generational loss, and is excellent for sophisticated special effects.

COMPRESSION is a means of reducing the signal in some way to make the most efficient use of the bandwidth available.

• In a sense, an interlaced system is a 2:1 compression system
• Component video is a compression system as well....combining the three color signals together.
• Digitizing component video produced an excellent result, but took a VERY long time to take place.
• TV needed to be transmitted in REAL TIME, (a 1:1relationship between the time on the clock and the duration of the image on the screen) but using component video could take hours to digitize a single frame.
• Digitizing composite signals was cheaper, faster and easier, but the result wasn't as good.
• Cost for component storage was still very expensive!
• Time was a factor here too.

So....

Composite digital storage is used for large quantities of information, but component systems are better for short piece..

Technology has increased the speed the process requires.

You get an idea of how MUCH data is involved when you consider that 1 second of digital film is equal to approximately 24,000 pages of text! (Cinebase, 1998)

Compression is the key to the information super high way --

Information is lost during all of the compression systems, but not as much as is lost in analog recording.

The terms are :

• LOSSY -- data is lost during the encoding process
• LOSSLESS -- all the data encoded can be recovered during decoding

More compression costs more money, but actually reduces the quality of the product and is more "lossy."

The most important forms of compression are

JPEG -- Joint Photographic Experts Group standard

• used primarily for still images, uses DCT (discrete cosine transformation)
• good for some post production
• uses frame by frame compression so gets good detail
• but is "lossy" and picture decoded won't be as good as a the one encoded

Motion JPEG

• used for video editing.
• Each frame is compressed using JPEG/DCT

MPEG -1 Moving Picture Experts Group standard

• Compression to VHS and lower quality.
• Used for video CDs

MPEG 2 will be a kind of standard for transmitting and /or transporting digital video

MPEG 2+ -- under development; will offer editing of broadcast-quality of DCT compressed video

MPEG 3 -- abandoned as a standard for HDTV; merged into MPEG 2

MPEG 4 -- low bit rate video telephony compression over standard phone lines or ISD. Will be used for video phone or telco delivery of video; may employ fractal or wavelet compression systems.

Jpeg, Wavelet and Fractal will be used for special effects, editing, and post production.

In the very near future, we will probably have a whole job-description (position) just devoted to MPEG encoding. This is a skill you may want to learn --- "Compressionist" much like a "colorist" is used today in digital post production work.

Now that you have some very basic information, it's necessary to approach this in more detail. It is the intricacies of compression that are key to understanding digital television and the subsequent transfer to HDTV. The problems broadcasters and other face are keyed very closely to the nature of compression technology. Camera technology is similarly linked. So, here we go.....

The Key difference between MPEG codecs and others is that MPEG uses a "interpicture" concept to take advantage of two characteristics of video:

• The large amounts of identical areas in sequential frames of video.
• The high degree of predictability of movement between one frame and another.

MPEG creates three types of pictures or frames:

• Intrapictures (I): completely self contained (all the information about the picture is there) and like a jpeg file.
• Predicted (P) Pictures: contain information about the predicted forward movement of the elements of the picture.
• Bi-directional (B) Pictures: which contain information about the forward and backward predicted movement of the elements of the frame.

Both the B and P MPEG pictures reduce the amount of data necessary to represent an image but require lots of computation power. These predictions are made based on comparison between where a part of the frame (macroblock) is in one frame compared to where it is in the previous or next frame. From these comparisons a predicted motion vector is established which predicts the movement in the next frame and the rest of the picture is compressed. This allows a B or P frame to have all the information necessary for a complete I frame...at six times less storage space required. This is the key to MPEG's efficiency.

The I, B, and P frames are organized into GOPs (not Republicans) or Groups of Pictures. GOPs can be large or small, but are efficient. These groups are organized into five "profiles" and four "levels" depending on the features the GOP has or the application for which it is to be used.

MPEG-1

• Originally designed for putting video on CD-Rom.
• Released in 1992
• Used also for satellite television (DSS)
• Max resolution = 352 by 240 (standard input format)
• 30 frames per second
• no interlace capability

MPEG-2

• adopted for DVD and DVD-ROM video
• will be built into new digital TV sets
• can handle high definition tv
• resolution up to 1920 to 1080 at 30 fps
• encoder prices have fallen substantially increasing its popularity (1997)
• convert regular video into MPEG2 files
• good quality from 50:1 to 100:1 compression
• Most experts consider DVD with MPEG better than laser disc
• Laser disk horizontal resolution is 425 lines
• DVD goes to 500 lines
• Higher bit rates make that possible: DVD bit rate=3.5 Mbps
• At 6 Mbps, MPEG 2 delivers digital video as good as the best broadcast digital videotape (D1 format) with 720 x 480 resolution at 30 fps.
• it will also handle HDTV but highest current resolution is 720 by 576. ( Roth, 1997)

Advantages of MPEG-2:

• This is a TRUE STANDARD, not owned by any particular corporate interest; this should preclude competitive standard wars.
• SCALABLE: The standard works for a large range of applications, but is designed so that the more complex profiles or levels are build on the simpler ones; the allows for flexibility of use.
• TRANSPORTABIITY: The standard was designed to move large amounts of video data and it does just that.
• EFFICIENCY: Especially for broadcasters and others with large amounts of video inventory to store and archive, MPEG-2 offers an efficient means. By reducing file sizes, it takes less storage space and that saves money! This is also good for video providers who want to use servers and coax or other means to deliver video....smaller file sizes mean less bandwidth required for delivery.

What are the DISadvantages?

All this forward and backward prediction rerquires lots of computation power and that means BIG, expensive computers. But as processing prices continue to dip this is not expected to be a long-term problem.

Harder to implement and hard to edit; special challenges for equipment manufacturers

Editing is especially hard to do with three separate pictures to deal with.

This is a key factor in some of the problems broadcasters face with implementation of HDTV. (Bennett, 1996)

Codecs (compression/decompression systems) are designed to reduce the size of digital video files by reducing the amount of redundant information the frames contain.

MPEG-4: This newer compression system is designed to compress multimedia signals (such as web pages) and is good for handling half-screen or quarter-screen pictures. It is expected to be of significant advantage to streaming web pages on cable or as data streams broadcast with STDV or HDTV signals over a standard 6 Mhz channel. (Brown, Peter J.)

MPEG-21: "Multimedia Framework" Is under development at the present time and will establish a uniform and flexible code for "declaring digital items" and will make sure that those items are separable after compression.  The goal is to allow one compression system to handle all types of multimedia components.  Ideally it will combine the best of previous compression systems into one super-system which will handle everything.

For an excellent resource on video compression: See Ali Sama Tosun's page " Video Compression: MPEG-4 and beyond."  It's way more detail than is required for this class, but if you have questions, you'll find what you need there at http://www.cis.ohio-state.edu/~jain/cis788-99/compression  

Three types of compression systems exist:

DCT: (discrete cosine transform) includes JPEG, (used for nonlinear editing)

MPEG-1 and MPEG-2 (used for play-back applications) Used in production in DV(digital video) and DVCPro and Digital-S formats.

Wavelets (or sub-band compression) used only in the ImMix VideoCube non-linear editing system. This compression system has developed some advantages that make it more efficient than MPEG2. Wavelet compression breaks a frame into sections for analysis rather than analyzing the entire frame for changes.

Only those sections which need the extra bandwidth get it.

Statistical Coding -- Used by NewTech in the popular VideoToaster.

Other, newer codec under consideration:

• Fractals
• Horizons Technology's True-Motion-S: Software-only algorithm which provides the high degree of frame-accuracy for interactive applications
• Intel's Indeo Video Interactive.
• Model-based Video Coding
• Scalable Video Coding
  
  

Other ways of classifying video codecs:

SPATIAL CLASSIFICATIONS:

Spatial Compression: (intra-frame compression) video files undergo a bit-rate reduction on a frame by-frame compression; that means that each frame remains a complete entity which can be called up, examined and manipulated. Motion J-peg is the prime example. This is why jpeg is the preferred codec for digital editing and special effects.

Temporal Compression: (inter-frame compression) is a motion-estimated compression system which depends on reducing the amount of duplicated information between sequential pictures.

It can reduce files significantly. It can also introduce some artifacts or mistakes. MPEG-1 and MPEG-2 are prime examples here.

LOST INFORMATION CLASSIFICATIONS:

Lossless: In these systems, when the video is regenerated there is very little if any lost information.

Lossy: In this process, some of the material is lost in the encoding/decoding process. The artifacts may or may not be noticeable. (as mentioned earlier in your notes.)

Problems with uncompressed digital video:

• Uncompressed digital video signal is called D-1 4:2:2 serial component video.
• Transmission rate: 270 megabytes per second.
• Sampling rate: 13/5 MHZ

Problems: recorders can only hold about 30 to 60 seconds of digital video

Sony and Panasonic offer D1 and D5 digital tape systems, but they don't allow random access to footage (in other words, the tape is linear - like analog tape; you have to wind and rewind)

This is a real problem when you want to do digital effects or editing.

The Digital tape data can be loaded into non-linear editing systems, but still are limited by the mass quantities of information and the storage limitations of the host computers.

There is not much likelihood that a standard compression format will develop for all purposes, because some companies have interests in their own hardware and corresponding software based on specific codecs.

ACQUISITION:

The collection of video is called acquisition and the type of equipment used is called the acquisition format.

Today, most broadcast and production studios use analog equipment. Be sure you know the characteristics of the classic analog camera types:

Image Orthicon: made practical television broadcasting possible. The tube was very large, very fragile, and very expensive. It was nicknamed "IMMY" and from that came the television award, the EMMY.

Vidcon: (developed during the 60's) Smaller, more rugged, warms up quickly, less expensive, and lasts longer. This is the type of camera we have in our studio.

Plumbicon: Lower illumination, higher video and color quality; noise is high in the "reds" and it "blooms" or puddles in bright light. Recent improvements mean better pictures under difficult shooting conditions.

Saticon tube: More frequently used; more stable and longer lived than the lead-oxide targets of earlier tubes; higher definition and more even color responses; less image lag and comet trails tend to be white rather than red or green. Not as good as lead-oxide tubes under low light conditions

Charged-Coupled device (CCD) may be known as (MOS) Metal Oxide Semi-Conductor chips. With these, the tubes are replaced by semiconductor memory chips which produce the color and brightness parts of the picture. In fact the target is a solid-state image sensor that uses a small silicon chip containing thousands of individual elements called PHOTO DIODES which when combined with transistor correspond to the phosphors on the kinescope.

Advantages of CCD image sensor

• Small (smaller lightweight cameras possible)
• Rugged -- vibrations and shocks don't hurt it
• Consumes low power so battery life extended
• No image "burn-in" ; so no after-images after lights etc.
• No comet tailing behind moving lights
• No geometric distortion (straight lines don't lean)
• Low lag under low-light conditions
• Free from magnetic field interference (no glitches in pictures shot near radar or radio transmitters)
• High sensitivity
• Long life compared with camera tube
• Less blooming (where picture appears to "run" or "blur")
• No drift (performance change during use)

Most all cameras made today are CCD cameras, although many tube cameras are still in use.

Digital acquisition formats include:

DV

Introduced in 1995 is a Panasonic format aimed at high-end consumers.

• It is now available from JVC, Panasonic, Sharp and Sony.
• Uses 1/4 inch metal-evaporated tape with helical scan recording.
• The format uses a 4:1:1 eight-bit sampling rate (that means the luminance signal is sampled
• four times as many times as the two color signals)
• 500 lines of horizontal resolution (this is 20% higher than the best analog laser disc or or S-VHS recordings and better than current NTSC broadcast standard)
• Luminace sampling rate is 13.5 million times per second; color sampling three times higher
• than S-VHS
• 1.5 MHZ chrominance bandwidth
• DCT-based 5:1 compression ratio
• 25 megabit/sec data stream
• intra-frame recording for frame-accurate editing
• Interlaced: two fields per frame
• note: DV does not compress audio, but has three separate sampling rates for audio.

DVCAM: Digital video camera and the first of the Sony products

Targeted for the industrial and business product, not for consumer or broadcast news acquisition format Sony now offers the "Digital HANDYCAM", a DV format. A consumer/professional camcorder with lots of bells and whistles. See the brochure in your packet for complete details.

DVC-Pro: Is the professional digital acquisition format also introduced by Panasonic.

• uses 1/4 inch metal particle tape and a faster tape speed
• supports an audio cue and audio control tracks
• can play back DV tapes
• DVC-Pro tapes cannot be played back on DV machines.
• DVCPRO: primarily a news acquisition format
• 1/4 inch tape
• DCT-based compression at 5:1 ratio

These two formats are not compatible, although they are similar. The difference is in the track width. DVCPRO offers the most because it has the largest track width. Because of that it can playpack DVCPRO, DVCAM and DV tapes without any content loss. DVCAM can play DV as well, but not DVCPRO.

210 Panasonic DVCPROs were used by NBC and the Olympic Committee to cover the 1996 Olympic Games in Atlanta.. They also used 1000 D-3 VTRS, 100 DVCPRO recorders and 40 D-5 recorders)

Panasonic offers a Full-sized camcorder with three CCDs at about $16,000. They also offer a Handheld camocoreder which is component quality using DAT sized cassettes at $4500.

DVCPRO 50 is the upgrade version with more features

See brochure in your packet for more information about how the DVCPRO is being used in the field.

These formats were developed as a result of research and a standard adopted by a group of equipment manufacturers including Hitachi, JVC, Mitsubishi, Philips, Sanyo, Sharp, Sony, Thomson and Toshiba.

Sony also offers the BETACAM SX and DIGITAL BETACAM as news acquisition formats The SX records up to 60 min. of video at a mpeg2 based 18MB/second. These can run the old Betacam SP tapes as well; this an advantage to newsrooms using the SP camera system.

DIGITAL-S by JVC is another DV format

• component digital video
• backward compatible with S-VHS
• aimed at industrial video user
• designed to replace S-VHS and U-Matic equipment
• alternative to Betacam  

24P

• High Definition Television in 24 frames per second format
• Allow for smooth downconversion to NTSC
• Smooth transfer to 35 mm. film (also at 24 fps)
• Provides "filmic" look which is warmer, softer than sharp contrast video
• Can be "undercranked" to slow motion without shutter or increased to 33, 40 and 60 fps for special effects
• Advantage for film-makers because it is quieter, more secure, lighter and easier to use than film cameras, especially on location or "nature" shooting where conditions may not be pristine.
• HD tape stock is much cheaper than film; $1500 vs. $104,000 for film and processing.
• $65,000 in 2001 approximate cost  per camera

AVID's CAMCUTTER/NEWSCUTTER:

• records video on a hard drive as it is being shot
• hard drive is removable
• data is compressed using motion jpeg codec
• drives are inserted directly into the editing system
• eliminates the need to transfer and digitize footage
• very expensive
• low storage capacity

Shooting on video tape has advantages over shooting to hard drive:

• tape can acquire video at a high data rate than hard drives
• tremendous storage capacity at much cheaper prices
• BUT can't provide random-access of hard-drive based systems.

Both tape-based systems and hard-drive systems will exist side by side, each doing what best serves the needs of the producer.

EDITING:

JPEG is the codec of choice for most of the non-linear systems because of...

• mature technology
• low-cost chip sets
• low compression ratios
• high-quality playback

Not actually a standard and variations exist among the various hardware designers.

MPEG is much more standardized on an international basis. BUT MPEG is hard to use for editing because the frames are not isolated and therefore not addressable. I-frames contain all the information necessary to reproduce a picture. P and B frames contain manipulated date which results from mathematical predictions about how the frames are going to change based on motion in portions of the frame. These frames aren't addressable and therefore have to be decoded before they can be edited. Some companies are trying to work out an editing system for MPEG, but not really there yet.

PLAYBACK:

MPEG is best for delivery and playback.

• It is more efficient and moves data faster.
• It does need a very powerful computing system to encode it, but
• DECODING is very fast and simple.
• So simple in fact, that set-top boxes can be used to play back MPEG data streams as real-time video and audio.

• A number of television stations are starting to use MPEG servers for insertion of commercials, station IDs and promos and news stories.

VIDEO STREAMING:

Web Theater 2.2 by VXtreme will stream video to quarter-screen (352-288) over 56 kbps connection and scale down to 28 K connections in streaming fashion.

MICROSOFT has bought both the company and the Web Theater. They have also recently licensed rights to REAL VIDEO technology which is the foundation for Microsofts NETSHOW product.

Other steaming software companies like VDOnet, Vivo, and Progressive

Networks have said they'll support NETSHOW, but nobody knows what that means.

What NETSHOW offers is: Consolidation of lots of different streaming plug-ins

That may make it easier, but it also means that servers and endusers will have to buy into it.

The MacIntosh streaming offering is QUICKTIME streaming, again a consolidation of lots of various efforts. Quicktime has proved a successful video file format but the streaming version hasn't been really seen yet.

Those who don't want to wait, have opted for a lower bandwidth solution.

SORENSON VIDEO does that...

It competes with CLEAR VIDEO which is optimized for low data rates, but it doesn't work for CD-Rom speeds. Sorenson will be available for Mac and PC/windows and can be run on 486 cpus and (Mac) 68040 rather than Pentiums or PowerPC.

Intel Indeo Video can handle higher data rates but requires big computer power.

Eidos Technologies has introduced ESCAPE based on codecs used for video games. Designed for 600 kps and 4x CD-Rom

MPEG-1 is the biggest competitor to escape, but Escape is faster at encoding and requires less CPU power on playback.

"The consolidation of the streaming video market is inevitable. Video content providers need to get their content delivered to the broadcast audience possible and fragmented market with a surplus of competing codes and browser plug-ins simply can't deliver the needed market share for any single solution. Placing those different codecs under a single umbrella, whether it is NETSHOW or QUICKTIME will bring the necessary order to the market." (Currier, 1997 )

• Go back to PART 1
• Go back to PART 2
• Go on to PART 4
  
  

  ---

Resources:

• Bennet, Chris. "Slaying the Myths of MPEG Compression". TV Technology 4/12/96p7
• Brown, Peter. "The Promise of Datacasting" Broadcacing and Cable. 2/7/2000. p. 54
• Careless, James. "Wavelet Compression Emerges." TV Technology. 2/23/98 p. 14
• Cinebase Software. "The Network Story" Supplement to Film and Video. 4/98.
• Cover, Robin. THE XML Cover Pages: MPEG 21 Part 2: Digital Item Declaration Language (DIDL).   6/05.01 Http://www.xml.coveragepages.org/mpeg-21didl.html  Accessed 11/13/01
• Currier, Bob, "Making Sense of Codec Chaos" TV Technology 9/25/95/97 p. 48
• Drabick, Matt. "Power of the Press." AV VIDEO 4/96 p. 84
• Kienzle, Claudia.  "Panasonic's HD Camera Gets Cranking."  TV Technology. 10/31/01 p. 12
• Livingston, Philip and Johann Safar. "Not an all-MPEG World" Advanced Imaging. 4/98 p. 37.
• Prikios, Karen Anderson.  "Cameras Flex Their Resolution." Broadcasting and Cable.  4/6/01 p. 38
• Roth, Cliff. "MPEG 2 to Die FOr" NewMedia 11/3/97 p67
• Tsoun, Ali Saman.  "Video Compression: Mepeg-4 and Beyond."  ftp://ftp.netlab.ohio-state.edu/pub/jain/courses/cis788-99/compresssion/indiex.html