Specification
DESCRIPTION Title of Invention: INFORMATION PROCESSING DEVICE AND METHOD, AND PROGRAM Technical Field
[0001]
The present invention relates to an information processing device and method, and a program, and more particularly, it relates to an information processing device and method, and a program that make it possible to perform streaming playback so as to prevent image or sound disturbance from occurring even when the playback is performed for a long time. Background Art
[0002]
Hitherto, live streaming playback using networks has been performed. When the live streaming playback is performed, there is a case in which control of, using time stamps included in streams, timing at which data is transferred to a decoder is not performed. Summary of Invention Technical Problem
[0003]
However, in such a case, there is no effective means for synchronizing an encoder clock of a server and a decoder clock of a client. For this reason, there is a risk of occurrence of image or sound disturbance because of clock drift when playback is performed for a long time.
[0004]
The present invention has been made in order to solve such a problem, and makes it possible to reduce clock drift that occurs in streaming playback and to perform playback with more stability. Solution to Problem
[0005]
An information processing device according to an aspect of the present invention includes: reception means for receiving data for streaming playback; holding means for holding the data that has been received by the reception means; decoding means for reading and decoding the data that is held in the holding means and that has been encoded in a predetermined manner; determination means for determining, from a temporal data amount of the data that is held in the holding means, whether or not a speed of a decoding process performed by the decoding means is appropriate; and control means for controlling, on the basis of a result of determination performed by the determination means, the speed of the decoding process performed by the decoding means.
[0006]
The determination means may include: information acquisition means for acquiring, as data-amount calculation information, out of the data that is held in the holding means, information that is necessary for calculating a temporal data amount of all data which is held in the holding means; calculation means for calculating a temporal data amount using the data-amount calculation information that has been acquired by the information acquisition means; and detection means for detecting, by comparing the temporal data amount that has been calculated by the calculation means with a predetermined threshold, drift of a clock of the decoding process with respect to a clock of an encoding process to which the data has been subjected.
[0007]
The information acquisition means may acquire, out of the data that is held in the holding means, as the data-amount calculation information, a time stamp of data that has been first held and a time stamp of data that has been last held.
[0008]
The information acquisition means may acquire, as the data-amount calculation information, the number of packets that are held in the holding means and that have a PI D of zero.
[0009]
The information acquisition means may acquire, as the data-amount calculation information, the number of PCR packets that are held in the holding means.
[0010]
On the basis of a result of detection performed by the detection means, when the clock of the decoding process is too fast, the control means may make the clock of the decoding process slower, and, when the clock of the decoding process is too slow, the control means may make the clock of the decoding process faster.
[0011]
The detection means may compare the temporal data amount that has been calculated by the calculation means with the threshold that has been set using a temporal data amount of the data which is held in the holding means at a time of starting decoding of the data with the decoding means.
[0012]
The detection means may compare the temporal data amount that has been calculated by the calculation means with the threshold that has been set using a temporal data amount of the data which is held in the holding means at a time of starting decoding of the data with the decoding means and using jitter that occurs at a time of transmission of the data.
[0013]
The determination means may further include addition means for adding, to the data that has been received by the reception means, on a predetermined-data-amount-by-predetermined-data-amount basis, arrival-time information indicating a reception time of the data. The information acquisition means may acquire earliest arrival-time information and latest arrival-time information that are added to the data which is held in the holding means.
[0014]
An information processing method or a program according to an aspect of the present invention includes the steps of: receiving data for streaming playback; holding the data that has been received; reading and decoding the data that is held and that has been encoded in a predetermined manner; determining, from a temporal data amount of the data that is held, whether or not a speed of a decoding process is appropriate; and controlling the speed of the decoding process on the basis of a result of determination.
[0015]
According to an aspect of the present invention, data for streaming playback is received. The data that has been received is held. The data that is held and that has been encoded in a predetermined manner is read and decoded. Whether or not a speed of a decoding process is appropriate determined from a temporal data amount of the data that is held. The speed of the decoding process is controlled on the basis of a result of determination.
Advantageous Effects of Invention
[0016]
According to the present invention, information can be processed. More particularly, clock drift that occurs in streaming playback can be reduced, and playback can be performed with more stability. Brief Description of Drawings
[0017]
[Fig. 1] Fig. 1 is a diagram illustrating an example of a main configuration of a communication system to which the present invention is applied.
[Fig. 2] Fig. 2 is a block diagram illustrating an example of a detailed configuration of a receiving device.
[Fig. 3] Fig. 3 is a flowchart for explaining a flow of a synchronization control process.
[Fig. 4] Fig. 4 is a block diagram illustrating another example of the configuration of the receiving device.
[Fig. 5] Fig. 5 is a flowchart for explaining another flow of the synchronization control process.
[Fig. 6] Fig. 6 is a diagram illustrating an example of a configuration of a personal computer to which the present invention is applied. Description of Embodiments
[0018]
Fig. 1 is a block diagram illustrating an example of a configuration of a communication system to which the present invention is applied.
[0019]
A communication system 100 illustrated in Fig. 1 is a communication system that plays back content data such as images or sounds between a plurality of devices via streaming. The communication system 100 has a sending device 101 and a receiving device 103 that is connected to the sending device 101 via a network 102.
[0020]
The sending device 101 is a device that sends content data, which is supplied from the outside, as streaming data to the receiving device 103 via the network 102. The content data is data serving as content including, for example, images or sounds. The sending device 101 has an encoder 111 and a sending section 112.
[0021]
The encoder 111 encodes and compresses the content data, which has been supplied, using a predetermined encoding scheme such as MPEG (Moving Picture Experts Group) or JPEG (Joint Photographic Experts Group) 2000, and supplies the encoded data to the sending section 112. The sending section 112 packetizes the encoded data, which has been supplied from the encoder 111, as data (streams) for streaming playback, and sends the packetized streams to the receiving device 103 via the network 102.
[0022]
Note that the sending device 101 may have other functions. For example, the sending device 101 may have a content-data generating function such as a camera or a microphone. Furthermore, for example, the sending device 101 may have a storage section that stores content data. The content data may be read from the storage section, and may be supplied to the encoder.
[0023]
The streams, which have been sent by the sending section 112, are supplied to the receiving device 103 via the network 102. The network 102 is any network for connecting a plurality of devices to each other and for allowing the devices to perform communication therebetween, such as the Internet or a LAN (Local Area Network). The network 102 may be configured using a plurality of networks. Furthermore, the network 102 may be configured using a wired technology, may be configured using a wireless technology, or may be configured using both wired and wireless technologies.
[0024]
While the receiving device 103 is receiving the streams. which have been supplied via the network 102, the receiving device 103 decodes the streams, decompresses the content data, plays back the content data (via streaming), and outputs the content data to a monitor, a speaker, or the like. As a matter of course, the decompressed content data may be supplied to another device that is provided outside the receiving device 103.
[0025]
Fig. 2 is a block diagram illustrating an example of a detailed configuration of the receiving device 103 illustrated in Fig. 1.
[0026]
As illustrated in Fig. 2, the receiving device 103 has a receiving section 201, a buffer 202, a decoder 203, an output section 204, and a control section 211.
[0027]
The receiving section 201 is a communication interface that is connected to the network 102. The receiving section 201 acquires streams, which have been supplied via the network 102, and supplies the streams to the buffer 202. The buffer 202 is configured using a storage medium such as a RAM (Random Access Memory), and temporarily holds data that has been supplied from the receiving section 201. The buffer 202 supplies, to the decoder 203, encoded data that has been requested by the decoder 203, out of the encoded data which is held, or supplies the encoded data, which is held, to the decoder 203 at timing that is predetermined in accordance with the processing speed of the decoder 203.
[0028]
The decoder 203 decompresses and decodes the encoded data, which has been acquired from the buffer 202, and supplies content data that has been decompressed, to the output section 204. The output section 204 has, for example, a monitor or a speaker, and outputs the content data that has been supplied from the decoder 2 03. For example, the output section 204 displays images on the monitor, or outputs sounds from the speaker.
[0029]
The control section 211 controls an operation (for example, a clock) of the decoder 203 on the basis of an information item that has been extracted from the data which is held in the buffer 202. The control section 211 has a synchronization determination unit 221 and a clock control unit 222.
[0030]
The synchronization determination unit 221 performs, on the basis of the data which is held in the buffer 202, a process associated with determination of whether or not the speed of a decoding process that the decoder 203 performs is appropriate. More specifically, the synchronization determination unit 221 performs a process associated with determination of whether or not a clock of the decoder 203 and a clock of the encoder 111 of the sending device 101 are synchronized with each other.
[0031]
The synchronization determination unit 221 has an information acquisition part 231, a calculation part 232, and a detection part 233. The information acquisition part 231 extracts necessary information items from the data that is held in the buffer 202. The calculation part 232 calculates, using the information items that have been acquired by the information acquisition part 231, a temporal data amount of the data that is held in the buffer 202.
[0032]
Note that the term "temporal data amount" does not refer to a simple information amount (a data size) , but rather to an amount corresponding to a playback time over which data is played back as content. In other words, the calculation part 232 calculates a "temporal data amount" as a value indicating the amount of data that is held in the buffer 202 for a playback time.
[0033]
The detection part 233 compares the temporal data amount, which has been calculated by the calculation part 232, with a predetermined threshold, thereby determining whether or not the speed of the clock of the decoder 203 is appropriate.
[0034]
The clock control unit 222 controls the clock of the decoder 203 on the basis of a result of determination performed by the synchronization determination unit 221 (the detection part 233) so that the clock of the decoder 203 has an appropriate speed.
[0035]
An example of a flow of a specific process will be described with reference to a flowchart illustrated in Fig. 3.
[0036]
When the receiving device 103 starts reception of data for streaming playback, the control section 211 starts a synchronization control process.
[0037]
When the synchronization control process starts, in step SlOl, the information acquisition part 231 of the synchronization determination unit 221 analyzes, using a predetermined method, streams (data) that are held in the buffer 202, and acquires, as data-amount calculation information items, information items necessary for calculating a data amount.
[0038]
For example, when the streams are TTSs (TimedTransportStreams), a time stamp of the TTS that has been last held in the buffer 2 02 is compared with a time stamp of the TTS that has been first held in the buffer 202 (a difference is calculated), whereby a temporal data amount of the streams.that are held in the buffer 202 can be calculated.
[0039]
Accordingly, in this case, the information acquisition part 231 acquires the time stamp of the TTS that has been last held in the buffer 2 02 and the time stamp of the TTS that has been first held in the buffer 202 as data-amount calculation information items.
[0040]
Furthermore, for example, a case in which it is guaranteed that PATs (Program Association Tables) serving as PSI (Program Specific Information) are inserted between the streams at intervals each of which corresponds to a content playback time of 100 msec, the number of packets that have a PID of zero and that exist in the buffer 202 is counted, whereby a temporal data amount of the streams that are held in the buffer 202 can be calculated.
[0041]
PSI is an information item describing which program streams belong to. A PMT indicates an information item in which each of PIDs of images or sounds included in programs is stored. A PAT is a table in which a list of programs included in streams is stored using a list of PIDs stored in the PMT. It is determined that a PID of each PAT is necessarily zero. In other words, counting of the number of packets having a PID of zero is equivalent to counting of the number of PATs that are inserted every 100 msec.
[0042]
Accordingly, in this case, the information acquisition part 231 counts the number of packets having a PID of zero, and acquires a result of the counting as a data-amount calculation information item. Note that, in this case, because it is necessary to refer to TS headers of all of the packets that are held in the buffer 2 02, a load imposed on the information acquisition part 231 is higher than that in the above-described case in which TTS time stamps are used.
[0043]
Furthermore, for example, a case in which it is guaranteed that PCRs (Program Clock References) are inserted between the streams at intervals each of which corresponds to a content playback time of 100 msec, the number of PCR packets is counted, whereby a temporal data amount of the streams that are held in the buffer 202 can be calculated.
[0044]
A PCR indicates a time information item that is used as a clock reference. Between the streams, a PCR is inserted every 100 msec. Accordingly, a temporal data amount of the streams that are held in the buffer 202 can be calculated from a count value that is obtained by counting the number of PCRs. The information acquisition part 231 analyzes all PATs and PMTs for the streams that are held in the buffer 202. The information acquisition part 231 counts the number of PCR packets, and acquires a result of the counting as a data-amount calculation information item.
[0045]
Note that, in this case, because it is necessary to analyze all PATs and PMTs held in the buffer 202, a load imposed on the information acquisition part 231 is higher than that in the above-described case in which TTS time stamps are used or the above-described case in the number of PATs is counted.
[0046]
When data-amount calculation information items are acquired, in step S102, the calculation part 232 calculates, using the data-amount calculation information items, a temporal data amount of the streams that are held in the buffer 202.
[0047]
In step S103, the detection part 233 detects, from a result of the calculation, "drift" (hereinafter, referred to as "decoder clock drift") of the clock (decoder clock) of the decoder 203 with respect to the clock (encoder clock) of the encoder 111.
[0048]
A transmission bit rate at which streams are transmitted via the network 102 corresponds to a speed at which content is played back. It is supposed that, while the reception device 103 is receiving streams that have been sent, the reception device 103 performs live streaming playback in which the streams are played back substantially immediately (substantially in real time).
[0049]
Supposing that the receiving device 103 is in an ideal state in which network jitter is zero and there is also no decoder clock drift, the input/output rates of the buffer 202 are substantially the same as each other. In other words, during such live streaming playback, the temporal data amount of streams that are held in the buffer 2 02 is substantially constant. For example, when streaming playback starts, supposing that the temporal data amount of streams that are held in the buffer 202 is an amount of data corresponding to m seconds, data corresponding to m seconds is always held in the buffer 202.
[0050]
Accordingly, in this case, the detection part 233 monitors the temporal data amount of streams that are held in the buffer 202, and detects drift of the temporal data amount from m, thereby detecting network jitter or decoder clock drift.
[0051]
Meanwhile, m can be approximated using a time from the start of reception of data from a network to the start of playback. The reason for this is that, under an ideal environment in which there is no network jitter as described above, if data is received for m seconds, data corresponding to m seconds for which content is played back should be acquired. In other words, in order to hold, in the buffer 202, data corresponding to m seconds for which content is played back, reception for m seconds is necessary. Accordingly, as described above, m can be approximated using a time from the start of reception of data from a network to the start of playback.
[0052]
Supposing that network jitter is not zero, the reception rate changes with time. Accordingly, the temporal data amount of streams that are held in the buffer 202 is not constant because the temporal data amount is influenced by the network jitter. When the temporal data amount is denoted by m' in this case and the network jitter is at most n seconds, an appropriate value of the m' can be represented by Expression (1) given below.
[0053]
m-n
Documents