00001 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
00002 <html>
00003 <head>
00004
00005 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15"/>
00006 <title>Ogg Vorbis Documentation</title>
00007
00008 <style type="text/css">
00009 body {
00010 margin: 0 18px 0 18px;
00011 padding-bottom: 30px;
00012 font-family: Verdana, Arial, Helvetica, sans-serif;
00013 color: #333333;
00014 font-size: .8em;
00015 }
00016
00017 a {
00018 color: #3366cc;
00019 }
00020
00021 img {
00022 border: 0;
00023 }
00024
00025 #xiphlogo {
00026 margin: 30px 0 16px 0;
00027 }
00028
00029 #content p {
00030 line-height: 1.4;
00031 }
00032
00033 h1, h1 a, h2, h2 a, h3, h3 a {
00034 font-weight: bold;
00035 color: #ff9900;
00036 margin: 1.3em 0 8px 0;
00037 }
00038
00039 h1 {
00040 font-size: 1.3em;
00041 }
00042
00043 h2 {
00044 font-size: 1.2em;
00045 }
00046
00047 h3 {
00048 font-size: 1.1em;
00049 }
00050
00051 li {
00052 line-height: 1.4;
00053 }
00054
00055 #copyright {
00056 margin-top: 30px;
00057 line-height: 1.5em;
00058 text-align: center;
00059 font-size: .8em;
00060 color: #888888;
00061 clear: both;
00062 }
00063 </style>
00064
00065 </head>
00066
00067 <body>
00068
00069 <div id="xiphlogo">
00070 <a href="http://www.xiph.org/"><img src="fish_xiph_org.png" alt="Fish Logo and Xiph.org"/></a>
00071 </div>
00072
00073 <h1>Ogg logical bitstream framing</h1>
00074
00075 <h2>Ogg bitstreams</h2>
00076
00077 <p>The Ogg transport bitstream is designed to provide framing, error
00078 protection and seeking structure for higher-level codec streams that
00079 consist of raw, unencapsulated data packets, such as the Vorbis audio
00080 codec or Tarkin video codec.</p>
00081
00082 <h2>Application example: Vorbis</h2>
00083
00084 <p>Vorbis encodes short-time blocks of PCM data into raw packets of
00085 bit-packed data. These raw packets may be used directly by transport
00086 mechanisms that provide their own framing and packet-separation
00087 mechanisms (such as UDP datagrams). For stream based storage (such as
00088 files) and transport (such as TCP streams or pipes), Vorbis uses the
00089 Ogg bitstream format to provide framing/sync, sync recapture
00090 after error, landmarks during seeking, and enough information to
00091 properly separate data back into packets at the original packet
00092 boundaries without relying on decoding to find packet boundaries.</p>
00093
00094 <h2>Design constraints for Ogg bitstreams</h2>
00095
00096 <ol>
00097 <li>True streaming; we must not need to seek to build a 100%
00098 complete bitstream.</li>
00099 <li>Use no more than approximately 1-2% of bitstream bandwidth for
00100 packet boundary marking, high-level framing, sync and seeking.</li>
00101 <li>Specification of absolute position within the original sample
00102 stream.</li>
00103 <li>Simple mechanism to ease limited editing, such as a simplified
00104 concatenation mechanism.</li>
00105 <li>Detection of corruption, recapture after error and direct, random
00106 access to data at arbitrary positions in the bitstream.</li>
00107 </ol>
00108
00109 <h2>Logical and Physical Bitstreams</h2>
00110
00111 <p>A <em>logical</em> Ogg bitstream is a contiguous stream of
00112 sequential pages belonging only to the logical bitstream. A
00113 <em>physical</em> Ogg bitstream is constructed from one or more
00114 than one logical Ogg bitstream (the simplest physical bitstream
00115 is simply a single logical bitstream). We describe below the exact
00116 formatting of an Ogg logical bitstream. Combining logical
00117 bitstreams into more complex physical bitstreams is described in the
00118 <a href="oggstream.html">Ogg bitstream overview</a>. The exact
00119 mapping of raw Vorbis packets into a valid Ogg Vorbis physical
00120 bitstream is described in <a href="vorbis-stream.html">Vorbis
00121 bitstream mapping</a>.</p>
00122
00123 <h2>Bitstream structure</h2>
00124
00125 <p>An Ogg stream is structured by dividing incoming packets into
00126 segments of up to 255 bytes and then wrapping a group of contiguous
00127 packet segments into a variable length page preceded by a page
00128 header. Both the header size and page size are variable; the page
00129 header contains sizing information and checksum data to determine
00130 header/page size and data integrity.</p>
00131
00132 <p>The bitstream is captured (or recaptured) by looking for the beginning
00133 of a page, specifically the capture pattern. Once the capture pattern
00134 is found, the decoder verifies page sync and integrity by computing
00135 and comparing the checksum. At that point, the decoder can extract the
00136 packets themselves.</p>
00137
00138 <h3>Packet segmentation</h3>
00139
00140 <p>Packets are logically divided into multiple segments before encoding
00141 into a page. Note that the segmentation and fragmentation process is a
00142 logical one; it's used to compute page header values and the original
00143 page data need not be disturbed, even when a packet spans page
00144 boundaries.</p>
00145
00146 <p>The raw packet is logically divided into [n] 255 byte segments and a
00147 last fractional segment of < 255 bytes. A packet size may well
00148 consist only of the trailing fractional segment, and a fractional
00149 segment may be zero length. These values, called "lacing values" are
00150 then saved and placed into the header segment table.</p>
00151
00152 <p>An example should make the basic concept clear:</p>
00153
00154 <pre>
00155 <tt>
00156 raw packet:
00157 ___________________________________________
00158 |______________packet data__________________| 753 bytes
00159
00160 lacing values for page header segment table: 255,255,243
00161 </tt>
00162 </pre>
00163
00164 <p>We simply add the lacing values for the total size; the last lacing
00165 value for a packet is always the value that is less than 255. Note
00166 that this encoding both avoids imposing a maximum packet size as well
00167 as imposing minimum overhead on small packets (as opposed to, eg,
00168 simply using two bytes at the head of every packet and having a max
00169 packet size of 32k. Small packets (<255, the typical case) are
00170 penalized with twice the segmentation overhead). Using the lacing
00171 values as suggested, small packets see the minimum possible
00172 byte-aligned overheade (1 byte) and large packets, over 512 bytes or
00173 so, see a fairly constant ~.5% overhead on encoding space.</p>
00174
00175 <p>Note that a lacing value of 255 implies that a second lacing value
00176 follows in the packet, and a value of < 255 marks the end of the
00177 packet after that many additional bytes. A packet of 255 bytes (or a
00178 multiple of 255 bytes) is terminated by a lacing value of 0:</p>
00179
00180 <pre><tt>
00181 raw packet:
00182 _______________________________
00183 |________packet data____________| 255 bytes
00184
00185 lacing values: 255, 0
00186 </tt></pre>
00187
00188 <p>Note also that a 'nil' (zero length) packet is not an error; it
00189 consists of nothing more than a lacing value of zero in the header.</p>
00190
00191 <h3>Packets spanning pages</h3>
00192
00193 <p>Packets are not restricted to beginning and ending within a page,
00194 although individual segments are, by definition, required to do so.
00195 Packets are not restricted to a maximum size, although excessively
00196 large packets in the data stream are discouraged; the Ogg
00197 bitstream specification strongly recommends nominal page size of
00198 approximately 4-8kB (large packets are foreseen as being useful for
00199 initialization data at the beginning of a logical bitstream).</p>
00200
00201 <p>After segmenting a packet, the encoder may decide not to place all the
00202 resulting segments into the current page; to do so, the encoder places
00203 the lacing values of the segments it wishes to belong to the current
00204 page into the current segment table, then finishes the page. The next
00205 page is begun with the first value in the segment table belonging to
00206 the next packet segment, thus continuing the packet (data in the
00207 packet body must also correspond properly to the lacing values in the
00208 spanned pages. The segment data in the first packet corresponding to
00209 the lacing values of the first page belong in that page; packet
00210 segments listed in the segment table of the following page must begin
00211 the page body of the subsequent page).</p>
00212
00213 <p>The last mechanic to spanning a page boundary is to set the header
00214 flag in the new page to indicate that the first lacing value in the
00215 segment table continues rather than begins a packet; a header flag of
00216 0x01 is set to indicate a continued packet. Although mandatory, it
00217 is not actually algorithmically necessary; one could inspect the
00218 preceding segment table to determine if the packet is new or
00219 continued. Adding the information to the packet_header flag allows a
00220 simpler design (with no overhead) that needs only inspect the current
00221 page header after frame capture. This also allows faster error
00222 recovery in the event that the packet originates in a corrupt
00223 preceding page, implying that the previous page's segment table
00224 cannot be trusted.</p>
00225
00226 <p>Note that a packet can span an arbitrary number of pages; the above
00227 spanning process is repeated for each spanned page boundary. Also a
00228 'zero termination' on a packet size that is an even multiple of 255
00229 must appear even if the lacing value appears in the next page as a
00230 zero-length continuation of the current packet. The header flag
00231 should be set to 0x01 to indicate that the packet spanned, even though
00232 the span is a nil case as far as data is concerned.</p>
00233
00234 <p>The encoding looks odd, but is properly optimized for speed and the
00235 expected case of the majority of packets being between 50 and 200
00236 bytes (note that it is designed such that packets of wildly different
00237 sizes can be handled within the model; placing packet size
00238 restrictions on the encoder would have only slightly simplified design
00239 in page generation and increased overall encoder complexity).</p>
00240
00241 <p>The main point behind tracking individual packets (and packet
00242 segments) is to allow more flexible encoding tricks that requiring
00243 explicit knowledge of packet size. An example is simple bandwidth
00244 limiting, implemented by simply truncating packets in the nominal case
00245 if the packet is arranged so that the least sensitive portion of the
00246 data comes last.</p>
00247
00248 <h3>Page header</h3>
00249
00250 <p>The headering mechanism is designed to avoid copying and re-assembly
00251 of the packet data (ie, making the packet segmentation process a
00252 logical one); the header can be generated directly from incoming
00253 packet data. The encoder buffers packet data until it finishes a
00254 complete page at which point it writes the header followed by the
00255 buffered packet segments.</p>
00256
00257 <h4>capture_pattern</h4>
00258
00259 <p>A header begins with a capture pattern that simplifies identifying
00260 pages; once the decoder has found the capture pattern it can do a more
00261 intensive job of verifying that it has in fact found a page boundary
00262 (as opposed to an inadvertent coincidence in the byte stream).</p>
00263
00264 <pre><tt>
00265 byte value
00266
00267 0 0x4f 'O'
00268 1 0x67 'g'
00269 2 0x67 'g'
00270 3 0x53 'S'
00271 </tt></pre>
00272
00273 <h4>stream_structure_version</h4>
00274
00275 <p>The capture pattern is followed by the stream structure revision:</p>
00276
00277 <pre><tt>
00278 byte value
00279
00280 4 0x00
00281 </tt></pre>
00282
00283 <h4>header_type_flag</h4>
00284
00285 <p>The header type flag identifies this page's context in the bitstream:</p>
00286
00287 <pre><tt>
00288 byte value
00289
00290 5 bitflags: 0x01: unset = fresh packet
00291 set = continued packet
00292 0x02: unset = not first page of logical bitstream
00293 set = first page of logical bitstream (bos)
00294 0x04: unset = not last page of logical bitstream
00295 set = last page of logical bitstream (eos)
00296 </tt></pre>
00297
00298 <h4>absolute granule position</h4>
00299
00300 <p>(This is packed in the same way the rest of Ogg data is packed; LSb
00301 of LSB first. Note that the 'position' data specifies a 'sample'
00302 number (eg, in a CD quality sample is four octets, 16 bits for left
00303 and 16 bits for right; in video it would likely be the frame number.
00304 It is up to the specific codec in use to define the semantic meaning
00305 of the granule position value). The position specified is the total
00306 samples encoded after including all packets finished on this page
00307 (packets begun on this page but continuing on to the next page do not
00308 count). The rationale here is that the position specified in the
00309 frame header of the last page tells how long the data coded by the
00310 bitstream is. A truncated stream will still return the proper number
00311 of samples that can be decoded fully.</p>
00312
00313 <p>A special value of '-1' (in two's complement) indicates that no packets
00314 finish on this page.</p>
00315
00316 <pre><tt>
00317 byte value
00318
00319 6 0xXX LSB
00320 7 0xXX
00321 8 0xXX
00322 9 0xXX
00323 10 0xXX
00324 11 0xXX
00325 12 0xXX
00326 13 0xXX MSB
00327 </tt></pre>
00328
00329 <h4>stream serial number</h4>
00330
00331 <p>Ogg allows for separate logical bitstreams to be mixed at page
00332 granularity in a physical bitstream. The most common case would be
00333 sequential arrangement, but it is possible to interleave pages for
00334 two separate bitstreams to be decoded concurrently. The serial
00335 number is the means by which pages physical pages are associated with
00336 a particular logical stream. Each logical stream must have a unique
00337 serial number within a physical stream:</p>
00338
00339 <pre><tt>
00340 byte value
00341
00342 14 0xXX LSB
00343 15 0xXX
00344 16 0xXX
00345 17 0xXX MSB
00346 </tt></pre>
00347
00348 <h4>page sequence no</h4>
00349
00350 <p>Page counter; lets us know if a page is lost (useful where packets
00351 span page boundaries).</p>
00352
00353 <pre><tt>
00354 byte value
00355
00356 18 0xXX LSB
00357 19 0xXX
00358 20 0xXX
00359 21 0xXX MSB
00360 </tt></pre>
00361
00362 <h4>page checksum</h4>
00363
00364 <p>32 bit CRC value (direct algorithm, initial val and final XOR = 0,
00365 generator polynomial=0x04c11db7). The value is computed over the
00366 entire header (with the CRC field in the header set to zero) and then
00367 continued over the page. The CRC field is then filled with the
00368 computed value.</p>
00369
00370 <p>(A thorough discussion of CRC algorithms can be found in <a
00371 href="ftp://ftp.rocksoft.com/papers/crc_v3.txt">"A
00372 Painless Guide to CRC Error Detection Algorithms"</a> by Ross
00373 Williams <a
00374 href="mailto:ross@guest.adelaide.edu.au">ross@guest.adelaide.edu.au</a>.)</p>
00375
00376 <pre><tt>
00377 byte value
00378
00379 22 0xXX LSB
00380 23 0xXX
00381 24 0xXX
00382 25 0xXX MSB
00383 </tt></pre>
00384
00385 <h4>page_segments</h4>
00386
00387 <p>The number of segment entries to appear in the segment table. The
00388 maximum number of 255 segments (255 bytes each) sets the maximum
00389 possible physical page size at 65307 bytes or just under 64kB (thus
00390 we know that a header corrupted so as destroy sizing/alignment
00391 information will not cause a runaway bitstream. We'll read in the
00392 page according to the corrupted size information that's guaranteed to
00393 be a reasonable size regardless, notice the checksum mismatch, drop
00394 sync and then look for recapture).</p>
00395
00396 <pre><tt>
00397 byte value
00398
00399 26 0x00-0xff (0-255)
00400 </tt></pre>
00401
00402 <h4>segment_table (containing packet lacing values)</h4>
00403
00404 <p>The lacing values for each packet segment physically appearing in
00405 this page are listed in contiguous order.</p>
00406
00407 <pre><tt>
00408 byte value
00409
00410 27 0x00-0xff (0-255)
00411 [...]
00412 n 0x00-0xff (0-255, n=page_segments+26)
00413 </tt></pre>
00414
00415 <p>Total page size is calculated directly from the known header size and
00416 lacing values in the segment table. Packet data segments follow
00417 immediately after the header.</p>
00418
00419 <p>Page headers typically impose a flat .25-.5% space overhead assuming
00420 nominal ~8k page sizes. The segmentation table needed for exact
00421 packet recovery in the streaming layer adds approximately .5-1%
00422 nominal assuming expected encoder behavior in the 44.1kHz, 128kbps
00423 stereo encodings.</p>
00424
00425 <div id="copyright">
00426 The Xiph Fish Logo is a
00427 trademark (™) of Xiph.Org.<br/>
00428
00429 These pages © 1994 - 2005 Xiph.Org. All rights reserved.
00430 </div>
00431
00432 </body>
00433 </html>
