Internet DRAFT - draft-rey-avt-3gpp-timed-text
draft-rey-avt-3gpp-timed-text
Internet Draft
draft-rey-avt-3gpp-timed-text-02.txt J. Rey
Y. Matsui
Matsushita
Expires: August 2004 February 2004
RTP Payload Format for 3GPP Timed Text
Status of this document
This document is an Internet-Draft and is in full conformance
with all provisions of Section 10 of RFC 2026.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract
This document specifies an RTP payload format for the transmission of
3GPP (3rd Generation Partnership Project) timed text. 3GPP timed
text is time-lined decorated text media format whose defined storage
format is the ISO (International Standardisation Organisation) Base
Media File Format. As of today, 3GP files containing timed text
contents can be downloaded via HTTP and be synchronised with
audio/video contents. There is however no available mechanism for
streaming 3GPP timed text contents. In the following sections the
problems of streaming timed text are addressed and a payload format
for streaming 3GPP timed text over RTP is specified.
IETF draft - Expires August 2004 [Page 1]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
Table of Contents
1. Change Log......................................................2
2. Introduction....................................................3
3. Terminology.....................................................5
4. RTP Payload Format for 3GPP Timed Text..........................5
5. Resilient Transport............................................13
6. Congestion control.............................................13
7. SMIL usage.....................................................14
8. MIME Type usage Registration...................................14
9. SDP usage......................................................16
10. Examples of RTP packet structure..............................17
11. IANA Considerations...........................................17
12. Security considerations.......................................17
13. References....................................................18
14. Annex A Basics of 3GP File Structure..........................19
15. Author's Addresses............................................20
16. IPR Notices...................................................21
17. Full Copyright Statement......................................21
1. Change Log
1.1 Changes from draft-rey-rtp-avt-3gpp-tt-00.txt
Major changes:
- completed empty sections from -00 draft.
- abstract and introduction re-arranged. Moved section "Basics of the
3GP File Structure" to end of the document as Annex B.
- SLEN, SIDX and SDUR lengths fixed to 16, 16 and 24 bits,
respectively.
- New OPTIONAL header, SPLDESC, added to transport sample description
in-band.
- Section 4 on payload format expanded: text header, fragment header
and sample description header are fully specified.
- SMIL usage section added.
1.2 Changes from draft-rey-rtp-avt-3gpp-tt-01.txt
Major changes:
- Terminology, some terms introduced to clarify text.
- Section 4
- rules and recommendations on fragmentation are given.
- payload headers were calssified into five types, with a common
field section and specific fields for each type.
Rey & Matsui [Page 2]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
- header structure similar to RFC 3640 for easy transformation.
2. Introduction
The purpose of this draft is to provide a means to stream the 3GPP
timed text using RTP.
3GPP timed text is a 3GP file format for time-lined decorated text
specified in [1]. The 3GP file format itself follows the ISO Base
Media File Format recommendation [2]. Besides plain text, the 3GPP
timed text format allows the display of decorated text (e.g. blinking
text, scrolling, hyperlinks) synchronised or not with other media.
The 3GPP timed text format was developed for 3GPP Transparent End-to-
end Packet-switched Streaming Services (PSS) [1]. The scope of the
3GPP PSS includes downloading and streaming of multimedia content
over 3G packet-switched networks. The PSS adopts multimedia codecs
(such as MPEG-4 Visual, AMR, MPEG-4 AAC, and JPEG) and protocols like
SMIL [9] for presentation layouts or RTP for streaming. The current
usage of the 3GPP timed text file format is limited to downloading
via HTTP (with or without audio contents) due to the lack of an
appropriate RTP payload format.
In general, a multimedia presentation might consist of several
audio/video/text streams or tracks (in 3GP file format jargon).
Different tracks may have different contents and tracks of different
media may be spatially synchronised using the information within the
tracks or a scene description language like SMIL. An example of this
would be a media session with three different media tracks: 1 audio,
1 video and 1 timed text that reproduces a music video with karaoke
subtitles. The information contained in each track defines the
regions where each media is displayed, how the media looks like and
how it is synchronised, e.g., the song lyrics is displayed below the
video and the words are highlighted and synchronised with the
soundtrack.
Basically the 3GPP timed text format can be summarised as consisting
of four differentiated functional components:
- initial setup information for text tracks: these are the height
and width of the text region where the text track (contents) are
displayed, the translation offsets tx and ty relative to the video
track region and the layer or proximity of the text to the user. In
the 3GPP timed text format, these pieces of information are extracted
from Track Header Box, "tkhd".
- general formatting information about the text track: default font,
default background colour, default horizontal and vertical
justification, default line width, default scrolling, etcetera. In
the 3GPP timed text format, these pieces of information are extracted
from the Sample Description Box, "stsd".
- the actual text, conveyed as plain text using either UTF-8 or UTF-
16 encoding and,
Rey & Matsui [Page 3]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
- the "decoration": whether it is highlighted text, blinking text,
karaoke, hypertext, scroll delay, other text styles/formatting than
the defaults, etcetera. In the 3GPP timed text format, these pieces
of information are extracted from the various Modifier Boxes: "hlit",
"hclr", "blnk", "krok", "href", "dlay", "styl" or "tbox".
For details refer to Annex A that summarises the basics of the 3GP
file format and to [1], where a more detailed description of the
setup information, format parameters and modifiers is contained.
2.1 Requirements
In this section a set of requirements is listed. A justification for
each of them is also given. An RTP Payload Format for 3GPP timed
text SHALL:
1. Keep the 3GP text sample structure. A text sample consists
of the text length, the text string (either UTF-8 or UTF-16 encoded),
and one or several Modifier Boxes containing the text "decoration",
as defined in [1]. This is important to foster interoperability of
3GP file and RTP payload formats.
2. Transmit the text sample size, sample duration and sample
description index in-band. In RTP it is important to transmit it in-
band because this information might change from sample to sample.
This is also important for buffering purposes as described in Section
4.1.
3. Enable the transmission of the formatting information
(contained in the Sample Description Box, "stsd") by out-of-band and
in-band means. In general, a single sample description may be used
by different text samples. Therefore, to save overhead it is
sensible to transmit a default formatting once at the initialisation
phase and update this on demand. On the other hand, these pieces of
information may become large so that out-of-band transmission might
not be the most appropriate method. Also, out-of-band channels might
not be always available. For these reasons, the payload format SHALL
enable also the in-band transmission of sample description
information. This is especially useful for live streaming (where the
contents are not known a priori) or to protect this information
through other mechanisms like FEC [4] or retransmission [13]. RFC
2354 [8] discusses available mechanisms for packet loss resiliency.
4. Enable the aggregation of text samples in one single RTP
packet. In a mobile communication environment a typical text sample
size is around 100-200 bytes. Thus, transporting several text
samples in one RTP packet makes the transport over RTP more
efficient.
5. Enable the fragmentation and reassembly of a text sample
into several RTP packets in order to cover a wide range of
applications and network environments. In general, fragmentation is
Rey & Matsui [Page 4]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
a rare event given the low bit rates and text sample sizes. However,
the 3GPP Timed Text media format allows for larger text samples and
so SHALL the payload format cover this possibility.
6. Enable the use of resilient transport mechanisms, such as
repetition, retransmissions and FEC.
3. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [5].
Furthermore, the following definitions are used in this document:
- text sample or whole text sample: this refers to a unit of text
data as contained in the source 3GP file. Its equivalent in
audio/video would be an audio/video frame (see Section 14 for
details). A text sample contains a text length indication, a text
string, and zero or several modifier boxes (the decoration of the
text).
- fragment or text sample fragment: a fraction of a text sample as
defined above. A fragment may either text strings and modifiers or
just one of these.
- sample contents: general term to identify timed text data
transported when using this payload format. In addition to text
samples and fragments, it may also be used to refer to the sample
descriptions associated to the text samples.
4. RTP Payload Format for 3GPP Timed Text
The format of an RTP packet containing 3GPP timed text is shown
below:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ RTP payload |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Marker bit (M): the marker bit must be set to 1 if the RTP packet
includes the last fragment of a text sample; otherwise set to 0. For
Rey & Matsui [Page 5]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
RTP packets containing several text samples, being at least one of
them non-fragmented, the marker bit MUST be set to 1.
Timestamp: the timestamp indicates the sampling instant of the timed
text sample contained in the RTP packet. The initial value is
randomly determined. If the RTP packet includes more than one text
sample (i.e. text sample aggregation), the timestamp indicates the
sampling instant of the oldest text sample in the RTP packet. The
samples MUST be placed in playout order, whereas the oldest sample is
placed first in the payload. The timestamp of the subsequent samples
is obtained by adding the timed text sample duration (if present) to
the timestamp value. For example, let sdur(0), sdur(1) and sdur(2)
be the durations of three subsequent timed text samples included in
an RTP packet. Let rtpts be the timestamp as present in the RTP
header. The timestamp ts(i) for each sample (i=0,1,2) would be:
ts(i)=rtpts + sum[sdur (i-1)];
ts(0)=rtpts,
ts(1)=rtpts + sdur(0)
ts(2)=rtpts + (sdur(0)+ sdur(1))
Some text samples may become large and have to be fragmented and so
spread over several RTP packets. In this case, the receiver needs to
associate fragments of the same text sample. This is done using the
timestamp. The order of the fragments is resolved using the fields
available in the following headers.
The value timestamp clockrate is copied directly from the 3GP file:
the value of "timescale" in the Media Header Box is to be used.
Payload Type (PT): the payload type is set dynamically and sent by
out-of-band means.
The usage of the remaining RTP header fields follows the rules of RTP
[3] and the profile in use.
4.1 Fragmentation of Timed Text Samples
This section justifies why text samples MAY be fragmented and
discusses some of the possible approaches to do it. A solution is
proposed together with rules and recommendations for fragmenting and
transporting text samples using this payload format.
3GPP Timed Text applications are expected to operate at low bit rates.
This fact added to the small size of timed text samples (typically
one or two hundred bytes) makes fragmentation of text samples a rare
event. Samples should usually fit into the MTU size of the used
network path.
Nevertheless, the text string (e.g. ending roll in a movie) and some
modifier boxes, i.e. for hyperlinks ("href"), for karaoke ("krok") or
Rey & Matsui [Page 6]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
for fonts ("styl") might become large and need fragmentation. This
may also apply for future modifier boxes. While the text string is
recommended as per [1] to take a maximum of 2048 bytes for maximum
client interoperability, there is no recommendation on the amount of
space occupied by modifier boxes.
In order to transport these larger text samples using RTP, it could
be argued that a careful encoding be used to transform the original
large sample into smaller self-contained text samples that fit into
the transport MTU. This would comply with the ALF principle, as per
RFC 2367 [14]. It would also need additional pre-processing previous
to RTP encapsulation. Given the low probability of fragmentation, it
is believed that the overhead of this pre-processing (careful
encoding) is not worth. It appears more appropriate to encode text
samples without taking the path MTU into account. In this manner,
this payload format meets a trade-off by intentionally leaving out
this pre-processing and making some text samples more sensitive to
packet losses.
However, a minimum set of fragmentation rules and recommendations
SHALL be observed to guarantee a minimum resiliency and guide in the
task of fragmentation. Text samples and fragments thereof are
aggregated in the RTP payload according to the rules and
recommendations specified as follows:
- it is RECOMMENDED that text samples are fragmented as seldom as
possible. E.g. if a previous packet has some free space and a new
text sample fits in one MTU, a new RTP packet SHOULD be sent, instead
of sending two or more fragments out of it. In order to fill up the
remaining bits, piggybacking of sample descriptions MAY be performed.
- text strings MUST split at character boundaries. Otherwise, it is
not possible to display the text of a fragment if the previous is
lost.
- it is RECOMMENDED to include as fewer text sample fragments as
possible in an RTP packet. This reduces the effects of packet loss.
RTP packets using this payload format MAY include zero or more whole
text samples, zero or more text sample fragments and zero or more
sample descriptions.
- sample descriptions SHALL NOT be fragmented, since they contain
important information that may affect several text samples.
- for enhanced resiliency against packet loss it is RECOMMENDED that
fragments containing decoration are especially protected using FEC
[4], retransmission [13], packet repetition or a similar technique.
- when fragmenting text samples, the start of the decoration
(modifiers) MUST be indicated. Otherwise, if packets are lost, a
client may be unable to identify where the modifiers start and the
text ends.
Rey & Matsui [Page 7]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
Usually, RTP applications use the information on packet size from UDP
or lower layers to find out the length of the RTP payload. This
means that if several text samples (or fragments) are contained in
the payload a length indication MUST be present for all fragments,
but the last one. Similarly, those transported as unique payload do
not need of a length indication.
However, some transport schemes for RTP, e.g. RFC 3640 [15], require
that the length of each fragment is indicated. This payload format
does not mandate to comply with such requirement, but OPTIONALLY
allows to do so. Implementations subject to such requirement MUST
include an explicit length indication,i.e. the LEN field, by setting
the L bit in all cases.
Note also that the order of the text samples and fragments in the RTP
payload is important. As described above in the definition of the
RTP timestamp usage, these MUST be placed chronologically in the RTP
payload, so that the SDUR field allows calculating the timestamp of
the samples following. At the same time, other samples MAY follow if
the current and following samples contain each a length indication.
Otherwise, the sample is either placed at the end or as unique RTP
payload. Fragments carrying modifier box contents and sample
descriptions MAY be placed in any order (no timing requirements) and
MAY be present as often as needed. Modifier box fragments SHOULD be
placed as close as possible to the text strings, which they belong
to.
4.2 Payload Header Definitions
An RTP packet using the payload headers defined in this document has
the following format:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Res.|U|L|TYPE | |
+-+-+-+-+-+-+-+-+ |
: (variable payload header depending on TYPE value) :
: :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
: SAMPLE CONTENTS = Text Sample(s), Fragment(s), Sample :
: Description(s) :
: :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Rey & Matsui [Page 8]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
The payload headers specified in this document consist of a set of
common fields followed by specific fields for each header type.
The structure of the payload headers resembles that of the 'access
units' in RFC 3640. This similarity is intentional, in order to ease
the transport using MPEG4 elementary streams. In this manner, the
'AU header' of that document finds an equivalent in the common header
fields for all TYPE values: R, U, L, TYPE and LEN. The specific
fields plus the sample contents would be similar to the 'AU data
section'.
The payload header the following format:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN (if L=1) | specific |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| fields (variable)...|
+-+-+-+-+-+-+-+-+-+-+-+-+
where,
- R (3 bits) "Reserved bits": this field MUST be set to zero.
- U (1 bit) "UTF Transformation": indicates whether the text
characters are encoded using UTF-8 (U=0) or UTF-16 (U=1). It MUST
also be present in text sample fragments if they contain text
strings. If the payload does not carry text strings, this bit has
no meaning and SHALL be disregarded.
- L (1 bit) "Length Field" flag: if set indicates the presence of the
OPTIONAL length field, LEN. By default this flag is reset. Where
several 'access units' are included in an RTP packet, the length MUST
be used to parse the payload. In this case, it SHALL be present in
all 'access units' but the last one. In some cases, as in RFC 3640
[15], the length field is always needed, and so MUST L flag be always
set.
- LEN (16 bits) "Length Field": indicates the size (in bytes) of this
field and all the bits following, i.e. specific payload header fields
for each TYPE (excluding initial byte) plus the sample contents. If
present, LEN has the following values:
- TYPE = 1, LEN >= 6,
- TYPE = 2, LEN > 9,
- TYPE = 3, LEN > 3,
- TYPE = 4, LEN > 3,
- TYPE = 5, LEN > 3.
For whole text samples (TYPE=1), the sample contents length
corresponds to the entry value in the Sample Size Box, "stsz" (see
also SLEN below). Otherwise the sample contents length MUST be
Rey & Matsui [Page 9]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
calculated when fragmenting the sample, in order to obtain the
correct LEN value.
Empty text samples do not have sample contents. For this case, TYPE 1
with a LEN of 6 (0x0006) MUST be used.
The following payload header compositions (including the specific
fields) result for the different values of TYPE:
- TYPE = 0,6 and 7 are reserved.
- TYPE = 1,
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN (if L=1, always >=6)| SIDX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SDUR |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This header type is used to transport whole text samples. If several
text samples are sent in an RTP packet, every sample has its own
header.
The LEN field MUST be always equal (for empty text samples) or
greater than 6 (0x0006).
The fields above have the following meaning:
- SIDX (8 bits) "Text Sample Entry Index": indicates the reference
index for the text sample, which corresponds to the index field in
the Sample to Chunk Box, "stsc", for the sample. This field is used
to map the corresponding sample description information. The SIDX is
unequivocally linked to one particular sample description.
Therefore, sample descriptions SHALL not be modified during a
streaming session. A maximum of 126 SIDX values is allowed per text
stream. To allow SIDX wrap-up, clients SHALL keep as valid only
those values of SIDX outside of the interval (X+128) modulo 255,
where X is the last SIDX value received. The SIDX values 0 (0x00)
and 255 (0xff) are reserved for possible future extensions.
- SDUR (24 bits) "Text Sample Duration": indicates the sample
duration in timestamp units of the text sample, which corresponds to
the entry value in the Decoding Time to Sample Box, "stts", for that
sample. This field allows by a clockrate of 1000 Hz a maximum
duration of approximately 279 hours (16 bits is would allow for just
65 seconds, which might be too short for some streams).
It is assumed that all text samples have a known duration at the time
of transmission. In some cases however, e.g. live streaming, the
SDUR value might not be known. To cover this exception, the value
zero (0x000000) is reserved to signal unknown duration. For all
other cases SDUR MUST be different from 0x000000.
Rey & Matsui [Page 10]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
The ordering of 'access units' in the RTP payload is important.
Logically, samples of unknown duration SHALL NOT precede any text
samples or text sample fragments in the RTP payload. Otherwise it
would not be possible to find out the timestamp of these. However,
samples of unknown duration MAY precede sample descriptions, as they
have no duration.
In general, text sample contents expire when the next sample becomes
valid. As an exception, samples of unknown duration (SDUR=0x000000)
are valid until new packets arrive.
Note also, that samples of unknown duration SHALL NOT use features,
such as scrolling or karaoke, which would need to know the duration
of the sample up-front.
- TYPE = 2,
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN(if L=1, always >9) | SIDX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SLEN | SDUR |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SDUR (cont.) |TOTAL | THIS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This header type is used to transport text sample fragments
containing text strings.
The LEN field (16 bits) has the same meaning as above. If present,
the length of LEN MUST be greater than a value of nine (0x0009).
The SLEN field (16 bits) indicates the size (in bytes) of the (whole)
text sample to which this fragment belongs. As seen above, the text
sample length corresponds to the entry value in the Sample Size Box,
"stsz". Clients MAY use SLEN to buffer space for the remaining
fragments of the text sample.
The fields TOTAL (4 bits) and THIS (4 bits) indicate the total number
of fragments in which the original text sample has been fragmented
and which order occupies the current fragment in that sequence,
respectively. The usual 'byte offset' field is not used here for two
reasons: a) it would take one more byte and b) it does not provide
any useful information on the character offset. UTF-8/16 text
strings have, in general, a variable character length ranging from 1
to 6 bytes. Therefore, the TOTAL/THIS solution is preferred.
The R, U, L, TYPE, SIDX, and SDUR fields have identical
interpretation as above. The U, SIDX and SDUR fields are useful
since partial text strings MAY also be displayed with the
corresponding decoration.
Rey & Matsui [Page 11]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
- TYPE = 3,
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN(if L=1, always >3) |TOTAL | THIS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This header type is used to transport either whole modifier boxes or
just the first fragment of these. This depends on whether the
modifier boxes fit into one RTP packet.
In case fragmentation is needed, this header type identifies first
fragment. As explained above, the rules for fragmentation require
that the start of the modifier boxes be signaled.
The R, U, L, TOTAL/THIS and LEN fields are used as above. If present,
the LEN field MUST be greater than three (0x0003).
Note that the SLEN, SIDX and SDUR fields are not present. This is
because: a) these fragments do not contain text strings and b) these
types of fragments are applied over text string fragments, which
already contain this information.
- TYPE = 4,
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN(if L=1, always >3) |TOTAL | THIS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
This header type is used to transport modifier fragments, other than
the first one.
The R, U, L, TOTAL/THIS and LEN fields are used as above. If present,
the LEN field MUST be greater than three (0x0003).
Note that the SLEN, SIDX and SDUR fields are not present. This is
because: a) these fragments do not contain text strings and b) these
types of fragments are applied over text string fragments, which
already contain this information.
- TYPE = 5
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| R |U|L|TYPE | LEN(if L=1, always >3) | SIDX |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Rey & Matsui [Page 12]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
This header type is used to transport sample descriptions. The L
flag MUST be always set. Consequently, the LEN field MUST be greater
than three (0x0003). Every sample description MUST have its own TYPE
5 header.
5. Resilient Transport
Apart from the basic fragmentation measures described in the section
above, the simplest option for packet loss resilient transport is to
send the same text samples (or fragments) again, i.e. repetition. A
server MAY decide to send the same contents again as a measure for
error resilience.
Repetition of text samples (or fragments) is only allowed if exactly
the same RTP payloads are sent, so that a receiver can use original
and repeated fragments together to reconstruct the text samples. If
a text sample was originally sent as non-fragmented text sample, a
repetition of that sample MUST be sent also as a single non-
fragmented text sample. Likewise, if the original text sample was
fragmented and spread over several RTP packets, repeated fragments
SHALL also be observe the same byte boundaries and use the same
headers and bytes per fragment. Finally, if several text samples
resolve to the same timestamp, the receiver SHOULD use the one
received in the RTP packet with the highest sequence number and
discard the rest.
In repeated text samples (or fragments), all RTP header fields MUST
keep their original values except the sequence number that MUST be
increased to comply with RTP.
6. Congestion control
The RTP profile under which this payload format is used defines an
appropriate congestion control mechanism in different environments.
Following the rules under the profile, an RTP application can
determine its acceptable bitrate and packet rate in order to be fair
to other TCP or RTP flows.
If an RTP application using this payload format uses retransmission,
the acceptable packet rate and bitrate includes both the original and
retransmitted data. This guarantees that an application using
retransmission achieves the same fairness as one that does not. Such
a rule may translate in practice into the following actions:
If enhanced service is used, it should be made sure that the total
bitrate and packet rate do not exceed that of the requested service.
It should be further monitored that the requested services are
actually delivered. In a best-effort environment, the sender SHOULD
NOT send retransmission packets without ensuring first that enough
bandwidth for retransmission is available. Other solutions like
Rey & Matsui [Page 13]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
reducing the packet rate and bitrate of the original stream (for
example by encoding the data at a lower rate) MAY be used.
Similar considerations apply, if an RTP application using this
payload format implements forward error correction, FEC [4]. Hereby,
the sender should take care that the amount of FEC does not actually
worsen the problem.
Therefore, it is RECOMMENDED that applications implementing this
payload format also implement congestion control. The actual
mechanism for congestion control is out of the scope of this document
but should be suitable for real-time flows. As an example, RFC 3448
[11] specifies an equation-based congestion control that fulfils this
requirement.
7. SMIL usage
The SMIL recommendation [9] specifies a means for synchronising
different media streams.
This payload format defines the spatial layout parameters for a timed
text stream. These specify the location of the text display area
relative to the top left corner of the video display area, when a
text stream is played with a single video stream without SMIL. In
cases where several media streams shall be synchronized, SMIL MAY be
used to specify the spatial layout parameters.
It shall be noted that even if SMIL scene description is used the
track header information pieces SHOULD be sent anyway as they
represent the intrinsic media properties.
8. MIME Type usage Registration
8.1 3GPP Timed Text MIME Registration
MIME type: video
MIME subtype: 3gpp-tt
Required parameters:
rate: the RTP timestamp clockrate is equal to the clockrate of
the media. The value timestamp clockrate is copied directly
from the 3GP file: the value of "timescale" in the Media Header
Box is to be used.
brand=<brand-name>, where <brand-name> identifies a Release
specification of 3GPP Timed Text being transmitted over RTP. A
brand indicates the "best use" of the contents: the brand value
"3gp5" indicates Release 5 of 3GPP Technical Specification (TS)
26.245 [1].
Rey & Matsui [Page 14]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
spldesc=<flag>, where <flag> may take three different values:
- spldesc=in, for in-band transmission of sample descriptions
- spldesc=out, for out-of-band and,
- spldesc=both, when both methods are allowed.
tx3g=<base64-value-1>, <base64-value-2>,... where <base64-value-
i> represents a list of sample description entries using base64
encoding. This parameter MAY be used to convey sample
descriptions out-of-band. The list of sample entries is not
required to follow any particular order. Each value <base64-
value-i> represents the concatenation of the SIDX and sample
descriptions contents for that SIDX. The LEN field is not
needed.
width=<integer-value> indicates the width in pixels of the text
track or area where the text is actually displayed.
height=<integer-value> indicates the height in pixels of the
text track.
tx=<integer-value>, indicates the horizontal translation offset
in pixels of the text track with respect to the origin of the
video track.
ty=<integer-value>, indicates the vertical translation offset in
pixels of the text track.
layer=<integer-value>, indicates the proximity of the text track
to the viewer. Higher values means closer to the viewer. This
parameter has no units.
Optional parameters:
mver=<version-value>, "Minor version" where <version-value> is a
positive integer. It identifies the oldest compatible version.
How the version is defined can be found in TS 26.245 "3GPP
Transparent end-to-end packet switched streaming service (PSS);
Timed Text Format (Release 6)".
cbrand=<value1,value2,...>, "List of Compatible Brands" where
value1 is a brand. This list MUST at least contain the <brand-
name> as in "brand".
Encoding considerations: this type is only defined for transfer via
RTP.
Security considerations: please refer to Section 12 of RFCXXXX.
Interoperability considerations: the 3GPP Timed Text media format is
specified in 3GPP Release 5 version of TS 26.245 "Transparent end-to-
Rey & Matsui [Page 15]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
end packet switched streaming service (PSS); Timed Text Format
(Release 6)". In later releases, 3GPP may specify extensions or
updates to the media format in a backwards-compatible way, e.g. new
modifier boxes or extensions to the sample descriptions. The payload
format RFCXXXX allows for such extensions. For future 3GPP Releases
of the Timed Text Format, the therein parameters "brand", "mver" and
"cbrand" are used to identify the current version of the media
format, the oldest compatible version and a list of compatible
versions.
Published specification: RFC XXXX
Applications which use this media type: multimedia streaming
applications.
Additional information: the 3GPP Timed Text media format is specified
in 3GPP TS 26.245 "Transparent end-to-end packet switched streaming
service (PSS); Timed Text Format (Release 6)". This document and
future extensions to the 3GPP Timed Text format are publicly
available at http://www.3gpp.org.
Person & email address to contact for further information:
rey@panasonic.de
matsui.yoshinori@jp.panasonic.com
Intended usage: COMMON
Author/Change controller:
Jose Rey
Yoshinori Matsui
IETF AVT WG
9. SDP usage
This document defines the MIME subtype name "3gpp-tt" and introduces
several REQUIRED payload-format-specific parameters: "brand",
"width", "height", "tx", "ty", "layer", "spldesc" and "tx3g" and two
OPTIONAL parameters "mver" and "cbrand".
9.1 Mapping to SDP
The information carried in the MIME media type specification has a
specific mapping to fields in SDP [4], which is commonly used to
describe RTP sessions. When SDP is used to specify transmission
using this payload format, the mapping is done as follows:
- The MIME type ("video") goes in the SDP "m=" as the media name.
The "video" MIME Type is used as timed text is considered visual
media.
- The MIME subtype ("3gpp-tt") goes in SDP "a=rtpmap" as the
encoding name. The value timestamp clockrate is copied directly from
Rey & Matsui [Page 16]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
the 3GP file, the value of "timescale" in the Media Header Box is to
be used. Other values MAY be specified by out-of-band means.
- The REQUIRED payload-format-specific parameters "brand", "width",
"height", "tx", "ty", "layer", "tx3g" and "spldesc" go in the SDP
"a=fmtp" as a semicolon separated list of parameter=<value> (or
parameter=<value1,value2,value3> for "tx3g") pairs.
- The OPTIONAL payload-format-specific parameters "mver", "cbrand"
go in the SDP "a=fmtp" as a semicolon-separated list of
parameter=<value> pairs.
- Any remaining parameters go in the SDP "a=fmtp" attribute by
copying them directly from the MIME media type string as a semicolon
separated list of parameter=value pairs.
In the following sections some example SDP descriptions are
presented.
10. Examples of RTP packet structure
In this section, some examples of RTP packet structure are explained
for better understanding of this payload format. The wrap-around of
the long lines is indicated by the backslash character "\". The
examples assume aggregate control of stream container files. The
session descriptions are not complete but limited to the example
purposes.
10.1 An RTP packet containing multiple text samples
<TODO>
11. IANA Considerations
This document introduces the MIME subtype name "3gpp-tt" in Section
8.
12. Security considerations
RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP
specification [3]. This implies that confidentiality of the media
streams is achieved by encryption.
Furthermore, the main security issues are confidentiality and
authentication of the text itself. The payload format itself does
not have any support for security. These issues have to be solved by
a payload external mechanism, e.g. SRTP [10].
Rey & Matsui [Page 17]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
13. References
13.1 Normative References
1 3GPP, "Transparent end-to-end packet switched streaming service
(PSS); Timed Text Format (Release 6)", TS 26.245 v 0.1.6, Working
Draft, July 2003.
2 ISO/IEC 14496-1:2001/AMD5, "Information technology û Coding of
audio-visual objects û Part 1: Systems, ISO Base Media File
Format", 2003.
3 H. Schulzrinne, S. Casner, R. Frederick and V. Jacobson, "RTP: A
Transport Protocol for Real-Time Applications", RFC 3550, July
2003.
4 M. Handley, V. Jacobson, "SDP: Session Description Protocol", RFC
2327, April 1998.
5 S. Bradner, "Key words for use in RFCs to indicate requirement
levels," BCP 14, RFC 2119, IETF, March 1997.
13.2 Informative References
6 C. Perkins, I. Kouvelas, O. Hodson, V. Hardman, M. Handley, J.C.
Bolot, A. Vega-Garcia, S. Fosse-Parisis, "RTP Payload for
Redundant Audio Data", September 1997.
7 J. Rosenberg, H. Schulzrinne, "An RTP Payload Format for Generic
Forward Error Correction", RFC 2733, December 1999.
8 C. Perkins, O. Hodson, "Options for Repair of Streaming Media",
RFC 2354, June 1998.
9 W3C, "Synchronised Multimedia Integration Language (SMIL 2.0)",
August, 2001.
10 M. Baugher, D. A. McGrew, D. Oran, R. Blom, E. Carrara, M.
Naslund, K. Norrman, "The Secure Real-Time Transport Protocol",
draft-ietf-avt-srtp-05.txt, June 2002.
11 Handley, et al., "TCP Friendly Rate Control (TFRC): Protocol
Specification ", RFC 3448, January 2003.
12 R. Hovey, S. Bradner, "The Organizations involved in the IETF
Standards Process", BCP 11, RFC 2028, October 1996.
13 J. Rey et al., "RTP Retransmission Payload Format", draft-ietf-
avt-rtp-retransmission-10.txt, work in progress, January 2004.
14 M. Handley, C. Perkins, "Guidelines for Writers of RTP Payload
Format Specifications", RFC 2367, December 1999.
Rey & Matsui [Page 18]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
15 Van der Meer et al., "RTP Payload Format for Transport of MPEG-4
Elementary Streams ", RFC3640, November 2003.
14. Annex A Basics of 3GP File Structure
Each 3GP file consists of "Boxes". Boxes start with a header which
indicates both size and type contained. The 3GP file contains the
File Type Box (ftyp), the Movie Box (moov), and the Media Data Box
(mdat). The Movie Box and the Media Data Box, serving as containers,
include own boxes for each media. Similarly, each box type may
include a number of boxes, see ISO Base Media file Format [2] for a
complete list of possibilities.
In the following, only those boxes are mentioned, which are useful
for the purposes of this payload format.
The File Type Box identifies the type and properties of a 3GP file.
The File Type Box contents comprise the major brand, the minor
version and the compatible brands. These are communicated via out-
of-band means, such as SDP, when streamed with RTP. For the 3GPP
timed text file format, the set of compatible-brands MUST include
"3gp5".
The Movie Box contains one or more Track Boxes (trak) which include
information about each track. A Track Box contains, among others,
the Track Header Box (tkhd), the Media Header Box (mdhd) and the
Media Information Box (minf). The Media Header Box contains the
timescale or number of time units that pass in one second. The Media
Information Box includes the Sample Table Box (stbl) which itself
contains the Sample Description Box (stsd), the Decoding Time to
Sample Box (stts), the Sample Size Box (stsz) and the Sample to Chunk
Box (stsc). Sample descriptions for each text sample are encoded as
"tx3g" sample entries in the Sample Description Box (stsd).
The Track Header Box specifies the characteristics of a single track,
where a track is, in this case, the streamed text during a session.
Exactly one Track Header Box is needed for a track. It contains
information about the track, such as the spatial layout (width and
height), the video transformation matrix and the layer number. Since
these pieces of information are essential and static, i.e. constant
for the duration of the session, they MUST be sent prior to the
transmission of any text samples. See the ISO base media file format
[2] for details about the definition of the conveyed information.
When using scene description in SMIL [9], it is possible to specify
the layer and the position of the text track. However, in this case,
the transmission of the Track Header Box (tkhd) is still RECOMMENDED,
as the intrinsic track information is specified there. Otherwise,
the Track Header Box information MUST be sent prior to the start of
the text streaming.
Rey & Matsui [Page 19]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
The Sample Table Box (stbl) contains all the time and data indexing
of the media samples in a track. Using the tables here, it is
possible to locate samples in time, determine their type, and
determine their size, container, and offset into that container.
From the Sample Table Box (stbl) the following information is carried
in each RTP packet using this payload format: the Sample Description
Box (stsd), the Decoding Time to Sample Box (stts), the Sample Size
Box (stsz) and the Sample to Chunk Box (stsc). The Decoding Time to
Sample Box (stts) is mapped to the field SDUR (Text Sample Duration);
the Sample Size Box (stsz) is mapped the field SLEN (Text Sample
Length) and the Sample to Chunk Box is mapped to the field SIDX (Text
Sample Entry Index). The Sample to Chunk Box (stsc) associates the
text sample and its corresponding sample description entry in the
Sample Description Box (stsd, see below). The Sample to Chunk Box
can be used to associate a text sample with a sample description
entry. Since the sample description may vary during the session, the
association SDIX must be sent together with the text samples using
this payload format.
The Sample Description Box (stsd) provides information on the basic
characteristics of text samples. Each entry is a sample entry box of
type "tx3g". An example of the information contained in a sample
entry could be the font size or the background colour. Since these
pieces of information are commonly used by many text samples during
the session, it is sent by out-of-bands means. A complete list of
text characteristics can be found in [1].
Finally, the Media Data Box contains the media data itself. In 3GPP
timed text tracks this box contains text samples. Its equivalent to
audio and video is audio and video frames, respectively. The text
sample consists of the text length, the text string, and one or
several Modifier Boxes. The text length is the size of the text in
bytes. The text string is plain text to render. The Modifier Box is
information to render in addition to the text such as colour, font,
etc.
15. Author's Addresses
Jose Rey rey@panasonic.de
Panasonic European Laboratories GmbH
Monzastr. 4c
D-63225 Langen, Germany
Phone: +49-6103-766-134
Fax: +49-6103-766-166
Yoshinori Matsui matsui.yoshinori@jp.panasonic.com
Matsushita Electric Industrial Co., LTD.
1006 Kadoma
Kadoma-shi, Osaka, Japan
Phone: +81 6 6900 9689
Fax: +81 6 6900 9699
Rey & Matsui [Page 20]
Internet Draft RTP Payload Format for 3GPP Timed Text February 2004
16. IPR Notices
The IETF takes no position regarding the validity or scope of any
intellectual property or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; neither does it represent that it
has made any effort to identify any such rights. Information on the
IETF's procedures with respect to rights in standards-track and
standards-related documentation can be found in BCP 11 [12]. Copies
of claims of rights made available for publication and any assurances
of licenses to be made available, or the result of an attempt made to
obtain a general license or permission for the use of such
proprietary rights by implementers or users of this specification can
be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights which may cover technology that may be required to practice
this standard. Please address the information to the IETF Executive
Director.
17. Full Copyright Statement
"Copyright (C) The Internet Society (2003). All Rights Reserved.
This document and translations of it may be copied and furnished to
others, and derivative works that comment on or otherwise explain it
or assist in its implementation may be prepared, copied, published
and distributed, in whole or in part, without restriction of any
kind, provided that the above copyright notice and this paragraph are
included on all such copies and derivative works. However, this
document itself may not be modified in any way, such as by removing
the copyright notice or references to the Internet Society or other
Internet organizations, except as needed for the purpose of
developing Internet standards in which case the procedures for
copyrights defined in the Internet Standards process must be
followed, or as required to translate it into languages other than
English.
The limited permissions granted above are perpetual and will
not be revoked by the Internet Society or its successors or
assigns.
This document and the information contained herein is provided on an
"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE."
Rey & Matsui [Page 21]
