DESCRIPTION
SENDING DEVICE, SENDING METHOD, AND PROGRAM
TECHNICAL FIELD
5 [000l]
The present disclosure relates to a sending device,
a sending method, and a program. More particularly, the
present disclosure relates to a sending device, a sending
method, and a program that can restrain a disturbance or
10 the like of an image, the disturbance or the like being
generated when the image, transmitted from a sending
device through a data transmission channel such as a
network, for example, is displayed with low delay.
15 BACKGROUND ART
[0002]
In recent years, remote surgeries have been
performed where, for example, a surgery is performed on a
patient in a remote location by operating robot arms.
20 During such a remote surgery, a doctor performing the
surgery operates robot arms while looking at moving
images obtained by taking images from the ongoing
surgery; thus it is desirable that the moving images be
transmitted with low delay of several frames (or several
25 fields) or less (substantially in real time).
[0003]
Given such a situation, encoding techniques have
been proposed to take several lines of each picture
constituting a moving image as a unit of block data and
30 to encode (compress) each unit of block data by wavelet
transform (see Patent Document 1, for example).
[0004]
According to this type of encoding technique, a
sending device starts encoding the block data without
waiting for the input of the complete units of the block
5 data within a picture and sends the resultant encoded
data. A receiving device starts decoding (expanding) the
encoded data before receiving the complete encoded data
from the sending device.
[OOOS]
10 The receiving device thus can decode the encoded
data and make the picture displayable in time for a
predetermined display timing when a picture should be
displayed. Hence, the receiving device can allow a
monitor to display the picture in synchronization with
15 the display timing.
CITATION LIST
PATENT DOCUMENT
[0006]
20 Patent Document 1: Japanese Patent Application Laid-Open
NO. 2007-311924
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
25 [0007]
Congestion of a data transmission channel such as a
network, however, may prevent the sending device from
sending a moving image with low delay of several frames
or less.
30 [0008]
In this case, the receiving device cannot make a
picture displayable in time for the predetermined display
timing. The picture that has not been made displayable
in time for the display timing may not be, for example,
displayed on the monitor and be skipped, possibly causing
5 a disturbance or the like to an image to be displayed on
the monitor.
[0009]
The present disclosure has been provided in light
of such a situation, and is to restrain a disturbance or
10 the like of an image, the disturbance or the like being
generated when the image, transmitted from a data
transmission channel, is displayed with low delay.
SOLUTIONS TO PROBLEMS
15 [OOlO]
A sending device according to an aspect of the
present disclosure includes an obtaining part that
obtains an adjustment quantity for adjusting a capture
timing to start capturing an image; a capture adjusting
20 part that adjusts the capture timing based on the
adjustment quantity; a capture part that captures the
image in synchronization with the adjusted capture
timing; and a sending part that sends the captured image
to a receiving device.
25 [OOll]
The receiving device includes a holding part that
temporarily holds the image, writes into the holding part
the image received from the sending part, and allows the
image written in the holding part to be displayed in
30 synchronization with a predetermined display timing, and
the obtaining part may obtain the adjustment quantity
calculated based on a result of comparison between an
allowable delay time representing a delay time allowed
when the writing of the image into the holding part is
not completed within a predetermined needed time from a
5 write start time when the writing has started, and a
necessary time at least taken from the write start time
for the image that has been written with a delay time of
the allowable delay time or less to the display timing.
[0012]
10 A first calculating part that calculates the
allowable delay time, a second calculating part that
calculates the necessary time, a comparing part that
compares the allowable delay time and the necessary time,
and an adjustment quantity calculating part that
15 calculates the adjustment quantity based on a comparison
result of the comparing part may be further included, and
the obtaining part may obtain the adjustment quantity
calculated by the adjustment quantity calculating part.
[0013]
20 If a comparison result is obtained to show that the
necessary time is longer than the allowable delay time,
the adjustment quantity calculating part may calculate
the adjustment quantity for adjusting the capture timing
such that the necessary time is reduced to an extent that
25 a condition that the necessary time is longer than the
allowable delay time is met.
[0014]
If a comparison result is obtained to show that the
necessary time is not longer than the allowable delay
30 time, the adjustment quantity calculating part may
calculate the adjustment quantity for adjusting the
capture timing such that the necessary time becomes
longer than the allowable delay time.
[0015] . -
The obtaining part may obtain the adjustment
5 quantity by receiving the same from the receiving device.
[0016]
A first measuring part that measures a write time
taken to write the image into the holding part of the
receiving device may be further included, and the first
10 calculating part may calculate the allowable delay time
based on a distribution of the write time.
[0017]
A second measuring part that measures a write
preparation time taken to start writing the image into
15 the holding part of the receiving device may be further
included, and the second calculating part may calculate
the necessary time based on a distribution of the write
preparation time.
[0018]
20 A sending method according to an aspect of the
present disclosure is for a sending device that captures
and sends an image, the sending method including the
steps of, by the sending device: obtaining an adjustment
quantity for adjusting a capture timing to start
25 capturing the image; adjusting the capture timing based
on the adjustment quantity; capturing the image in
synchronization with the adjusted capture timing; and
sending the captured image to a receiving device.
[0019]
30 A program according to an aspect of the present
disclosure is for allowing a computer to function as: an
obtaining part that obtains an adjustment quantity for
adjusting a capture timing to start capturing an image; a
capture adjusting part that adjusts the capture timing
based on the adjustment quantity; a capture part that
5 captures the image in synchronization with the adjusted
capture timing; and a sending control part that allows
the captured image to be sent to a receiving device.
[0020]
According to an aspect of the present disclosure,
10 an adjustment quantity for adjusting a capture timing to
start capturing an image is obtained, the capture timing
is adjusted based on the adjustment quantity, the image
is captured in synchronization with the adjusted capture
timing, and the captured image is sent to a receiving
15 device.
EFFECTS OF THE INVENTION
[0021]
According to the present disclosure, a disturbance
20 or the like of an image can be restrained, the
disturbance or the like being generated when the image,
transmitted from a data transmission channel, is
displayed with low delay.
25 BRIEF DESCRIPTION OF DRAWINGS
[0022]
Fig. 1 is a block diagram of an exemplary
configuration of a sending/receiving system according to
the present disclosure.
3 0 Fig. 2 is a graph of an example of displaying of an
image in synchronization with a display timing.
Fig. 3 is a graph of an exemplary method of
measuring write information.
Fig. 4 is a graph of an exemplary distribution of a
write preparation time.
5 Fig. 5 is a graph of an exemplary distribution of a
buffer write time.
Fig. 6 is a graph of an exemplary method of
calculating a necessary time and an allowable delay time.
Figs. 7A and 7B are first graphs of an exemplary
10 case where a write start time to start writing an image
changes depending on a capture timing.
Figs. 8A and 8B are first graphs of an example of
adjusting a capture timing.
Figs. 9A and 9B are first graphs of an exemplary
15 case where the write start time to start writing the
image has changed due to the adjusted capture timing.
Figs. 10A and 10B are second graphs of an exemplary
case where the write start time to start writing the
image changes depending on the capture timing.
20 Figs. 11A and 11B are second graphs of an example
of adjusting the capture timing.
Figs. 12A and 12B are second graphs of an exemplary
case where the write start time to start writing the
image has changed due to the adjusted capture timing.
25 Fig. 13 is a first graph of an example of varying
of a transmitting time.
Fig. 14 is a second graph of an example of varying
of the transmitting time.
Fig. 15 is a flow chart for describing a sending
30 process performed by the sending device.
Fig. 16 is a flow chart for describing a capture
adjusting process in step S6 in Fig. 15.
Fig. 17 is a flow chart for describing an
adjustment quantity calculating process performed by the
receiving device.
5 Fig. 18 is a graph of an exemplary method of
measuring write information of an odd-numbered field
image.
Fig. 19 is a graph of an exemplary method of
measuring the write information of a progressive image.
10 Fig. 20 is a block diagram of an exemplary
configuration of a computer.
MODE FOR CARRYING OUT THE INVENTION -.
[0023]
15 An embodiment of the present disclosure
(hereinafter referred to as the present embodiment) will
now be described below. Note that the description will
be in the following order.
1. The present embodiment (an example of restraining of a
20 disturbance or like of an image by adjusting a capture
timing Vblank)
2. Exemplary variations
[0024]
<1. The present embodiment>
25 [Exemplary configuration of sending/receiving system]
Fig. 1 illustrates a sending/receiving system
according to the present disclosure.
[0025]
This sending/receiving system includes a sending
30 device 21, a receiving device 22, and a network 23 such
as the Internet.
100261
In this sending/receiving system, the sending
device 21, for example, adjusts a capture timing Vblank
to start capturing (obtaining) an image and sends the
5 image captured at the adjusted capture timing to the
receiving device 22.
[0027]
This allows the receiving device 22 to display the
image received from the sending device 21 with low delay
10 without a disturbance or the like.
[0028]
[Exemplary configuration of sending device 211
The sending device 21 includes a capture part 41,
an encoding part 42, a packetizing part 43, an RTP (real-
15 time transport protocol) communication part 44, a capture
adjusting part 45, an operation part 46, and a control
part 47.
[0029]
The capture part 41, in synchronization with the
20 capture timing Vblank adjusted by the capture adjusting
part 45, obtains (captures) image data externally input
(corresponding to Video IN) and supplies the image data
to the encoding part 42. As the image data, a frame
image including multiple lines, for example, may be used.
25 As another example, a field image including either oddnumbered
lines or even-numbered lines among multiple
lines may be used.
[0030]
The present embodiment will be described with
30 reference to Figs. 1 to 18 with an assumption that a
field image is used as the image data. In this case, the
capture part 41, in synchronization with the capture
timing Vblank adjusted by the capture adjusting part 45,
obtains an odd-numbered field image and an even-numbered
field image alternately as the image data and supplies
5 the image data to the encoding part 42.
[0031]
Here, the odd-numbered field image refers - to a
field image 2 odd-numbered lines among the
multiple lines. The even-numbered field image refers to
10 a field image consisting of even-numbered lines among the
multiple lines.
[0032]
Note that the use of a frame image as the image
data will be described with reference to Fig. 19.
15 [0033]
The encoding part 42 performs an encoding process
to encode the image data from the capture part 41 and
supplies the encoded data obtained through the encoding
process to the packetizing part 43.
20 [0034]
As the encoding process, a wavelet encoding process
to encode image data, for example, by using wavelet
transform to compress the image data may be used.
[0035]
25 The packetizing part 43 packetizes (converts) the
encoded data from the encoding part 42 into multiple RTP
packets and outputs the-RTP packets to the RTP
communication part 44. An RTP packet refers to a packet
in accordance with RTP described in IETF RFC3550.
30 [0036]
The RTP communication part 44, in accordance with
RTP, adds to the RTP packets from the packetizing part 43
the time or the like for sending the RTP packets, for
example, to the receiving device 22 as a timestamp. The
RTP communication part 44 then sends the RTP packets,
5 with the timestamp added, through the network 23 to the
receiving device 22.
[0037]
In addition, the RTP communication part 44, in
accordance with RTP, receives an RTCP message supplied
10 through the network 23 from the receiving device 22 and
supplies the message to the capture adjusting part 45.
[0038]
Here, the RTCP message includes adjustment
quantities Zforward and Zbackward for adjusting the
15 capture timing Vblank. The adjustment quantities
Zforward and Zbackward are, for example, calculated and
sent by the receiving device 22. One of the adjustment
quantities Zforward and Zbackward is set to, for example,
zero.
20 [0039]
The capture adjusting part 45 performs a capture
adjusting process to adjust the capture timing Vblank of
the capture part 41 based on the adjustment quantities
Zforward and Zbackward from the RTP communication part 44.
25 [0040]
In other words, for example, if the adjustment
quantity Zforward = 0, the capture adjusting part 45
makes an adjustment such that the capture timing Vblank
is advanced by the adjustment quantity Zbackward. If,
30 for example, the adjustment quantity Zbackward = 0, the
capture adjusting part 45 makes an adjustment such that
the capture timing Vblank is delayed by the adjustment
quantity Zforward. The capture adjusting process
performed by the capture adjusting part 45 will be
described in detail below with reference to Fig. 16.
5 [0041]
The operation part 46 includes operating buttons
and the like to be operated by a user and supplies an
operating signal according to the user operation to the
control part 47.
10 [0042]
The control part 47 controls the capture part 41,
the encoding part 42, the packetizing part 43, and the
RTP communication part 44 based on, for example, an
operating signal from the operation part 46.
15 [0043]
[Exemplary configuration of receiving device 221
The receiving device 22 includes an RTP
communication part 61, a packet assembling part 62, a
decoding part 63, a write control part 64, a buffer 65, a
20 display control part 66, a display part 67, a write
information measuring part 68, a write information
storage part 69, an adjustment quantity. calculating part
70, an operation part 71, and a control part 72.
[0044]
25 The RTP communication part 61 receives the RTP
packets sent from the RTP communication part 44 through
the network 23 and supplies them to the packet assembling
part 62. The RTP communication part 61 generates, based
on adjustment quantities from the adjustment quantity
30 calculating part 70, an RTCP message including the
adjustment quantities, and sends the message through the
network 23 t o the RTP communication p a r t 44.
[0045]
The packet assembling part 62 assembles the RTP
packets from the RTP communication p a r t 61 t o generate
5 encoded data t o be decoded and supplies the data t o the
decoding p a r t 63.
[0046]
The decoding part 63 performs a decoding process
for the encoded data from the packet assembling part 62
10 and supplies t h e r e s u l t a n t image data t o the write
control part 64. The decoding process corresponds t o the
encoding process performed by the encoding part 42. As
the decoding process, a decoding process of decoding and
expanding by using inverse transform of wavelet transform,
15 for example, is used.
[0047]
The write control part 64 supplies the image data
from the decoding part 63 to t h e b u f f e r 65 f o r storing
(writing). In addition, the w r i t e c o n t r o l part 64
20 supplies a write s t a r t time Ts when writing i n t o the
buffer 65 has s t a r t e d and a write end time Te when the
writing has ended t o the write information measuring part
68 f o r each u n i t of image data.
[0048]
25 The buffer 65 temporarily s t o r e s t h e image data
from the write control part 64.
[0049]
The display c o n t r o l part 66 reads the image data
from the buffer 65 and supplies the data t o the display
30 part 67. The display c o n t r o l part 66 then allows the
display p a r t 67 t o display the image data read from the
buffer 65 in synchronization with a predetermined display
timing Vblank.
[0050]
Fig. 2 illustrates an example where the display
5 control part 66 allows each unit of image data to be
displayed at the predetermined display timing Vblank.
[0051]
The image data is, for example, a field image,
which is one of an odd-numbered field image nodd
10 consisting of odd-numbered lines among the multiple lines
or an even-numbered field image neven consisting of evennumbered
lines among the multiple lines.
[0052]
In Fig. 2, the horizontal axis represents time; the
15 vertical axis represents a line number for a line
displayed at a corresponding time. An assumption here is
that the odd-numbered field image nodd and the evennumbered
field image n,,,, each consist of 540 lines.
[0053] -.
20 A display timing Vblank (nodd) represents a time to
start displaying the odd-numbered field image nodd. A
display timing Vblank (n,,,,) represents a time to start
displaying the even-numbered field image n,,,,.
[ 0 0 5 4 ]
25 The display control part 66 reads the odd-numbered
field image nodd from the buffer 65 and allows the
display part 67 to display the image in synchronization
with the predetermined display timing Vblank (nodd). The
display control part 66 also reads the even-numbered
30 field image n,,,, from the buffer 65 and allows the
display part 67 to display the image in synchronization
with the predetermined display timing Vblank (n,,,,).
[0055]
The display part 67 displays each unit of image
data with, for example, an interlacing scheme. In other
5 words, for example, the display part 67 displays the oddnumbered
field image nodd and the even-numbered field
image n,,,, alternately as a unit of image data under the
control of the display control part 66. The display part
67 may be a component of the receiving device 22 as shown
10 in Fig. 1, or may be separate from the receiving device
22 and provided independently. In this case, the display
control part 66 within the receiving device 22 is
connected to the display part 67 provided outside the
receiving device 22 through, for example, a cable or the
15 like.
[0056]
The write information measuring part 68 measures
write information of the even-numbered field image n,,,,
written in the buffer 65. The write information
20 measuring part 68 may measure write information of the
odd-numbered field image nodd written in the buffer 65 in
lieu of, or together with, the write information of the
even-numbered field image n,,,,. This will be described
below with reference to Fig. 18.
25 [0057]
If, for example, the frame image is used as the
image data and the display part 67 is designed to display
.each unit of image data through a progressive scheme, the
write information measuring part 68 will take the frame
30 image as each unit of image data and measure write
information of the frame image. This will be described
below with reference to Fig. 19.
[0058]
The description below assumes that the write
information measuring part 68 measures write information
5 of the even-numbered field image neven written in the
buffer 65.
[0059]
The write information measuring part 68 measures,
as the write information of the even-numbered field image
10 n,,,, written in the buffer 65, a write preparation time
Xn that is a time necessary until the writing of the
even-numbered field image neve, starts and a buffer write
time Pn that is a time taken for the writing of the evennumbered
field image n,,,,, for example.
15 [0060]
Fig. 3 illustrates an example where the write
information measuring part 68 measures the write
preparation time Xn and the buffer write time Pn for the
even-numbered field image n,,,,.
20 [0061]
Here, in Fig. 3, the write start time Ts represents
the time when the writing of the even-numbered field
image neven into the buffer 65 has started. The write end
time Te represents the time when the writing of the even-
25 numbered field image neven into the buffer 65 has ended.
[0062]
Note that the write information measuring part 68
is supplied by the write control part 64 with the write
start time Ts and the write end time Te for the even-
30 numbered field image neven.
[0063]
As shown in Fig. 3, the write information measuring
part 68 measures the write preparation time Xn (= Vblank
(n - leven) - Ts) based on the write start time Ts from
the write control part 64 and a predetermined display
5 timing Vblank (n-leve)n .
[0064]
In addition, for example, the write information
measuring part 68 measures, as shown in Fig. 3, the
buffer write time Pn (= Te - Ts) based on the write start
10 time Ts and the write end time Te from the write control
part 64.
[0065]
The write information measuring part 68 supplies
the write preparation time Xn and the buffer write time
15 Pn measured for each even-numbered field image neve, to
the write information storage part 69 for storage.
[0066]
With reference to Fig. 1 again, the write
information storage part 69 stores the write preparation
20 time Xn and the buffer write time Pn from the write
information measuring part 68 as the write information.
[0067]
The adjustment quantity calculating part 70
performs, based on the write information stored in the
25 write information storage part 69, an adjustment quantity
calculating process to calculate the adjustment
quantities Zforward and Zbackward for adjusting the
capture timing Vblank at the capture part 41 of the
sending device 21. Note that the adjustment quantity
30 calculating process will be described in detail with
reference, for example, to Fig. 17.
The adjustment quantity calculating part 70
supplies adjustment quantities Zforward and Zbackward
calculated through the adjustment quantity calculating
5 process to the RTP communication part 61.
[0069]
The operation part 71 includes operating buttons
and the like to be operated by a user and supplies an
operating signal according to the user operation to the
10 control part 72.
[0070]
The control part 72 controls the RTP communication
part 61, the packet assembling part 62, the decoding part
63, the write control part 64, the display control part
15 66, the write information measuring part 68, and the
ad~ustment quantity calculating part 70 based on, for
example, an operating signal from the operation part 71.
[0071]
[Details of adjustment quantity calculating process and
20 capture adjusting process]
Now, with reference to Figs. 4 to 12, the
adjustment quantity calculating process performed by the
adjustment quantity calculating part 70 of the receiving
device 22 and the capture adjusting process performed by
25 the capture adjusting part 45 of the sending device 21
will be described.
[0072]
Fig. 4 illustrates an exemplary distribution of the
write preparation time Xn stored in the write information
30 storage part 69.
[0073]
In Fig. 4, the horizontal axis represents the write
preparation time Xn; the vertical axis represents the
frequency of the write preparation time Xn.
[0074]
5 The adjustment quantity calculating part 70
generates, based on the write preparation time Xn stored
in the write information storage part 69, X distribution
information representing a distribution of the write
preparation time Xn as shown in Fig. 4.
10 [0075]
The adjustment quantity calculating part 70 then
calculates, based on the X distribution information of
the write preparation time Xn, a threshold Xth for
distinguishing the write preparation time Xn occupying an
15 upper a (where a = 95, for example) percent from the
write preparation time Xn occupying the remaining lower
(100 - a) percent, with the write preparation time Xn
sorted in an ascending order.
[0076]
20 Here, the threshold Xth is the maximum value or
more of the write preparation time Xn occupying the upper
a (where a = 95, for example) percent and is less than
the minimum value of the write preparation time Xn
occupying the lower (100 - a) percent, with the write
25 preparation time Xn sorted in an ascending order. The
value a is, for example, predetermined by user operation.
[ 0 0 7 7 1
Fig. 5 illustrates an exemplary distribution of the
buffer write time Pn stored in the write information
30 storage part 69.
[0078]
In Fig. 5, the horizontal axis represents the
buffer write time Pn; the vertical axis represents the
frequency of the buffer write time Pn.
5 The adjustment quantity calculating part 70
generates, based on the multiple buffer write times Pn
stored in the write information storage part 69, P
distribution information representing a distribution of
the buffer write time Pn as shown in Fig. 5.
10 [0080]
The adjustment quantity calculating part 70 then
calculates, based on the P distribution information of
the buffer write time Pn, a threshold Pth for
distinguishing the buffer write time Pn occupying an
15 upper a percent from the buffer write time Pn occupying
the lower (100 - a) percent, with the buffer write time
Pn sorted in an ascending order.
[0081]
Here, the threshold Pth is the maximum value or
20 more of the buffer write time Pn occupying the upper a
and is less than the minimum value of the buffer write
time Pn occupying the lower (100 - a) percent, with the
buffer write time Pn sorted in the ascending order.
[0082]
25 Fig. 6 illustrates an example where the adjustment
quantity calculating part 70 calculates an allowable
delay time Yth based on the threshold Pth, and a
necessary time Y based on the threshold Xth.
[0083]
3 0 The adjustment quantity calculating part 70
calculates, based on the threshold Pth calculated from
the P distribution information of the buffer write time
Pn as shown in Fig. 6, the allowable delay time Yth with
expression (1) .
Yth = Pth - (1/60) (1
[0084]
Here, the allowable delay time Yth refers to a
delay time used when a writing of the even-numbered field
image n,,,, cannot be completed within a predetermined
expected write time 1/60 from the write start time Ts
when the writing has started. The allowable delay time
Yth is a delay time allowed to display the even-numbered
field image n,,,, without a disturbance or the like with a
probability of at least a percent.
[0085]
Note that the expected write time 1/60 refers to a
write time of the even-numbered field image n,,,, (or the
odd-numbered field image nodd) expected when there is no
delay or the like in data writing at the write control
part 64.
[0086]
The expected write time 1/60 is determined
depending on, for example, the time taken to display the
even-numbered field image n,,,, (or the odd-numbered field
image nodd). In the current case, the display part 67 is
configured to display the odd-numbered field image and
the even-numbered field image each for a display time
1/60. Hence, for example, the expected write time is
determined similarly to be 1/60. When the display part
67 displays the odd-numbered field image and the evennumbered
field image each for a display time l/m, the
expected write time is, for example, l/m.
[0087]
The adjustment quantity calculating part 70, for
example, calculates, based on the threshold Xth
calculated from the X distribution information of the
5 write preparation time Xn, a necessary time Y for the
even-numbered field image n,,,, to be received with a
probability of a percent, or in other words, the evennumbered
field image neven to be written with a delay time
of the allowable delay time Yth or less.
10 [0088]
Here, the necessary time Y refers to a time at
least taken from the write start time Ts to the display
timing (neve,) for the even-numbered field image neven to
be written into the buffer 65 with a delay time of the
15 allowable delay time Yth or less.
[0089]
In other words, for example, the adjustment
quantity calculating part 70 calculates, based on a
display interval Vblank Interval between the display
20 timing Vblank (n-lev,,) and the display timing (neven),
and the threshold Xth as shown in Fig. 6, the necessary
time Y with expression (2) below.
Y = Vblank Interval - Xth (2
[0090]
25 The adjustment quantity calculating part 70
determines whether the even-numbered field image neven can
be displayed without a disturbance or the like with a
probability of at least a percent based on whether the
calculated necessary time Y meets expression (3) below.
30 Y > Yth (3
[0091]
The adjustment quantity calculating part 70 then,
based on the result of the determination, calculates the
adjustment quantities Zforward and Zbackward. In other
words, for example, the adjustment quantity calculating
5 part 70 calculates the adjustment quantity Zforward (= Y
- Yth) as shown in Fig. 6, if expression (3) is met. The
adjustment quantity calculating part 70 then supplies the
calculated adjustment quantity Zforward and the
adjustment quantity Zbackward set to 0 to the RTP
10 communication part 61.
[a0921
Now, with reference to Figs. 7 to 9, the adjustment
quantity calculating process and the capture adjusting
process, both of which are performed if expression (3) is
15 met, will be described. The adjustment quantity
calculating process is performed by the adjustment
quantity calculating part 70 and the capture adjusting
process is performed by the capture adjusting part 45.
[0093]
20 Note that the following description assumes that
the display timing at the receiving device 22 side is in
synchronization with the capture timing at the sending
device 21 side; however, the display timing may not be in
synchronization with the capture timing.
25 [0094]
As shown in Fig. 7A, the sending device 21 side
starts capturing the even-numbered field image n,,,, in
synchronization with a capture timing Vblank (n,,,,).
[0095]
30 If expression (3) is met, as shown in Fig. 7B, the
receiving device 22 side starts writing the even-numbered
field image n,,,, into the buffer 65 at the write start
time Ts. A line (i) with the line number i then will be
written into the buffer 65 before the display timing of
the line (i) .
5 [0096]
In this case, the adjustment quantity calculating
part 70 calculates the adjustment quantity Zforward (= Y
- Yth) as shown in Fig. 7B and sets the adjustment
quantity Zbackward to zero, and supplies the adjustment
10 quantities Zforward and Zbackward to the RTP
communication part 61. The RTP communication part 61
generates an RTCP message including the adjustment
quantities Zforward and Zbackward from the adjustment
quantity calculating part 70, and sends the message
15 through the network 23 to the RTP communication part 44
of the sending device 21.
[0097]
The RTP communication part 44 of the sending device
21 supplies the RTCP message received through the network
20 23 from the RTP communication part 61 to the capture
adjusting part 45.
[0098]
The capture adjusting part 45 adjusts the capture
timing Vblank of the capture part 41 based on the
25 adjustment quantities Zforward and Zbackward included in
the RTCP message from the RTP communication part 44.
[0099]
In the current case, the adjustment quantity
Zbackward included in the RTCP message is zero.
30 Accordingly, the capture adjusting part 45 adjusts the
current capture timing Vblank as shown in Fig. 8A so as
to obtain the capture timing Vblank as shown in Fig. 8B
by delaying the current capture timing Vblank by the
adjustment quantity Zforward.
[0100]
5 Hence, the capture part 41 will capture an external
even-numbered field image n,,,, with the adjusted capture
timing Vblank that is delayed by the adjustment quantity
Zforward as shown in Fig. 9A, for example.
[0101]
10 The write control part 64 of the receiving device
22 thus will start writing the even-numbered field image
neven at the write start time Tsr (= Ts + Zforward) that
is delayed by the adjustment quantity Zforward as shown
in Fig. 9B. In this case, the necessary time Y is
15 reduced limitlessly to an extent that expression (3) is
met.
[0102]
Now, with reference to Figs. 10 to 12, the
adjustment quantity calculating process and the capture
20 adjusting process, both of which are performed if
expression (3) is not met, will be described. The
adjustment quantity calculating process is performed by
the adjustment quantity calculating part 70 and the
capture adjusting process is performed by the capture
25 adjusting part 45.
[0103]
Note that, as shown in Fig. 10A, the sending device
21 side starts capturing the even-numbered field image
n,,,, in synchronization with the capture timing Vblank
30 (neven) -
[0104]
If expression (3) is not met, the receiving device
22 side starts writing the even-numbered field image neven
into the buffer 65 at the write start time Ts as shown in
. :' ;:,y~.$pQ?''F''%w vO-T -~g" "^" ; :, :-.".t >" . ::' ." -" T~',;.: . .. . 5 9 :.: I -9 Fig . 1 0B . ki -l~x~&GL(&~j.rri~ ~thkii~r.li j=;.'G, a eer;:-i
5 line with the line number L or beyond (i 2 L), for
example, will be written into the buffer 65 after the
display timing of the line (i 2 L) .
[0105]
Here, the line with the line number L refers to, in
10 Fig. 10B, a line when the thick broken line representing
the write timing of each line of the even-numbered field
image neVen and the solid line representing the display
timing of each line of the even-numbered field image neven
cross.
15 [0106]
In this case, the adjustment quantity calculating
part 70 calculates, as shown in Fig. 10B, the adjustment
quantity Zbackward (= Yth - Y) and sets the adjustment
quantity Zforward to zero, and supplies the adjustment
20 quantities Zforward and Zbackward to the RTP
communication part 61. The RTP communication part 61
generates an RTCP message including the adjustment
quantities Zforward and Zbackward from the adjustment
quantity calculating part 70, and sends the message
25 through the network 23 to the RTP communication part 44
of the sending device 21.
[0107]
The RTP communication part 44 of the sending device
21 supplies the RTCP message, received through network 23
30 from the RTP communication part 61, to the capture
adjusting part 45.
[OlOB]
The capture adjusting part 45 adjusts the capture
timing Vblank of the capture part 41 based on the
adjustment quantities Zforward and Zbackward included in
5 the RTCP message from the RTP communication part 44.
[0109]
In the current case, the adjustment quantity
Zforward included in the RTCP message is zero.
Accordingly, the capture adjusting part 45 adjusts the
10 current capture timing Vblank as shown in Fig. 11A so as
to obtain the capture timing Vblank as shown in Fig. 11B
by advancing the current capture timing Vblank by the
adjustment quantity Zbackward.
[OllO]
15 The capture part 41 thus, for example, will capture
an external even-numbered field image n,,,, at the
adjusted capture timing Vblank that is advanced by the
adjustment quantity Zforward as shown in Fig. 12A.
[Olll]
20 The write control part 64 of the receiving device
22 thus will start writing the even-numbered field image
n,,,, at the write start time Ts' (Ts - Zbackward) that is
advanced from the write start time Ts by the adjustment
quantity Zbackward as shown in Fig. 12B. In this case,
25 as shown in Fig. 12B, expression (3) will be met.
[0112]
Note that the adjustment quantity calculating part
70 performs the adjustment quantity calculating process
as appropriate. This is because the transmitting time of
30 the image data varies depending on congestion and other
situations of the network 23, which brings about a state
as shown in Figs. 7A and 7B or in Figs. 10A and 10B even
with the adjusted capture timing Vblank.
[0113]
Now, Figs. 13 and 14 each illustrate an example
5 where the transmitting time varies depending on
congestion and other situations of the network 23.
[0114]
In Figs. 13 and 14, the vertical axis represents a
packet number that stands for an RTP packet of the image
10 data. The horizontal axis represents the time that is
each taken for a "capture" process performed by the
capture part 41, an "encoding" process performed by the
encoding part 42, a "packetizing" process performed by
the packetizing part 43, an RTP packet "transfer" process
15 performed through the network 23 between the RTP
communication part 44 and the RTP communication part 61,
a "packet assembling" process performed by the packet
assembling part 62, and a "decoding" process performed by
the decoding part 63. These processes are performed for
20 the data constituting the image data.
[0115]
As shown in Figs. 13 and 14, the time taken for the
"transfer" process will significantly vary depending on
the situation of the network 23. That is, if congestion
25 of the network 23 is relatively minor, the time taken for
the "transfer" process will be short as shown in Fig. 13.
If the congestion of the network 23 is relatively major,
the time taken for the "transfer" process will be long as
shown in Fig. 14.
30 [0116]
The time taken for the "transfer" process and
consequently the transmitting time vary depending on the
congestion and other situations of the network 23, which
in turn brings about a state as shown in Figs. 7A and 7B
or Figs. 10A and 10B. Hence, the adjustment quantity
5 calculating part 70 needs to perform the adjustment
quantity calculating process as appropriate.
[0117]
The capture adjusting part 45 then needs to adjust
the capture timing Vblank of the capture part 41 based on
10 the adjustment quantities Zforward and Zbackward supplied
from the adjustment quantity calculating part 70 through
the RTP communication part 61, the network 23, and the
RTP communication part 44.
[0118]
15 [Description of operation of sending device 211
Now, with reference to a flow chart in Fig. 15, a
sending process performed by the sending device 21 will
be described.
[0119]
20 This sending process starts, for example, when the
operation part 46 is operated so as to capture and send
image data. The control part 47 then performs the
sending process by controlling the capture part 41
through the capture adjusting part 45 according to an
25 operating signal from the operation part 46.
[0120]
In other words, at step S1, the capture part 41
obtains image data input externally and supplies the data
to the encoding part 42 under the control of the control
30 part 47 and in synchronization with the capture timing
Vblank adjusted by the capture adjusting process of step
S6 to be described below.
[0121]
Here, if the capture adjusting process of step S6
is not performed yet, the capture part 41, in
5 synchronization with the predetermined capture timing
Vblank, for example, obtains image data input externally
and supplies the data to the encoding part 42.
[0122]
In step S2, the encoding part 42 performs the
10 encoding process to encode the image data from the
capture part 41 and supplies the encoded data obtained
through the encoding process to the packetizing part 43.
[0123]
In step S3, the packetizing part 43 packetizes
15 (converts) the encoded data from the encoding part 42
into multiple RTP packets and outputs the packets to the
RTP communication part 44.
[0124]
In step S4, the RTP communication part 44, in
20 accordance with RTP, adds to the RTP packets from the
packetizing part 43 the time or the like for sending the
RTP packets, for example, to the receiving device 22 as a
timestamp. The RTP communication part 44 then sends the
RTP packets, with the timestamp added, through the
25 network 23 to the receiving device 22.
[0125]
In step S5, the RTP communication part 44
determines whether the RTCP message including the
adjustment quantities Zforward and Zbackward has been
30 received through the network 23 from the receiving device
22. If it is determined that the message has not been
received, the process reverts to step S1 and, from then
on, the process is performed in a similar manner.
[0126]
If the RTP communication part 44 determines in step
5 S5 that the RTCP message including the adjustment
quantities Zforward and Zbackward has been received
through the network 23 from the receiving device 22, the
RTCP message is supplied to the capture adjusting part 45.
[0127]
10 In step S6, the capture adjusting part 45 performs
the capture adjusting process to adjust the capture
timing Vblank based on the RTCP message from the RTP
communication part 44. Also in step S6, the capture
adjusting part 45, after the completion of the capture
15 adjusting process, reverts to step S1 of the process, and
from then on, the process is performed in a similar
manner. Note that the sending process is finished when,
for example, the operation part 46 is operated by a user
so as to stop sending image data.
20 [0128]
Now, with reference to the flow chart of Fig. 16,
the capture adjusting process in step S6 of Fig. 15 will
be described.
[0129]
25 As described above, the capture adjusting part 45
is supplied with the RTCP message by the RTP
communication part 44.
[0130]
In step S21, the capture adjusting part 45 obtains
30 (extracts), from the RTCP message supplied by the RTP
communication part 44, the adjustment quantities Zforward
and Zbackward t h a t are included i n the RTCP message.
[0131]
In step S22, the capture adjusting p a r t 45
determines whether the obtained adjustment quantity
5 Zbackward is zero. I f the adjustment quantity Zbackward
is determined t o be zero, the process moves on t o s t e p
S23.
[0132]
In s t e p S23, the capture adjusting p a r t 45 controls
10 the capture p a r t 41 so as t o delay the capture timing
Vblank of the capture p a r t 4 1 by the obtained other
adjustment quantity Zforward.
[0133]
In other words, f o r example, the capture adjusting
15 p a r t 45 a d j u s t s t h e current capture timing Vblank a s
shown i n Fig. 8A so as t o obtain the capture timing
Vblank as shown i n Fig. 8B.
[0134]
The capture p a r t 4 1 thus captures each unit of the
20 image data i n synchronization with the adjusted capture
timing Vblank as shown i n Fig. 9A. The display c o n t r o l
p a r t 66 of the receiving device 22 hence displays the
image data, as shown i n Fig. 9B, a t the display timing
Vblank with the necessary time Y t h a t is reduced as much
25 as possible t o the extent t h a t expression (3) is met.
[0135]
In step S22, the capture adjusting p a r t 45
determines whether the obtained adjustment quantity
Zbackward is zero. I f the adjustment quantity Zbackward
30 is determined t o be not zero, or i n other words, i f the
adjustment quantity Zforward is determined t o be zero,
the process moves on t o step S24.
[0136]
In step S24, the capture adjusting p a r t 45 controls
the capture part 41 so as t o advance the capture timing
5 Vblank of the capture part 41 by the obtained other
adjustment quantity Zbackward.
[0137]
In other words, f o r example, the capture adjusting
part 45 adjusts the current capture timing Vblank as
10 shown i n Fig. 11A so as t o obtain the capture timing
Vblank as shown i n Fig. 11B. The capture p a r t 41 thus
captures each unit of the image data i n synchronization
with the adjusted capture timing Vblank as shown i n Fig.
12A. The display c o n t r o l part 66 of the receiving device
15 22 hence displays the image data, as shown i n Fig. 12B,
a t the display timing Vblank t h a t meets expression ( 3 ) .
[0138]
The capture adjusting process is thus completed and
returned to step S6 i n Fig. 15. The process reverts t o
20 step S1. In step S1, the capture part 4 1 captures each
unit of the image data i n synchronization with the
capture timing Vblank adjusted through the capture
adjusting process.
[0139]
25 As described above, according t o the capture
adjusting process, i f expression (3) is not met, the
capture adjusting p a r t 45 a d j u s t s t h e capture timing
Vblank i n step S24 so as t o meet expression ( 3 ) .
[0140]
30 The receiving device 22, therefore, can r e s t r a i n a
s i t u a t i o n where a disturbance or the l i k e is generated i n
the image because a part of the image is skipped.
[0141]
In addition, for example, even if expression (3) is
met, the capture adjusting part 45 adjusts the capture
5 timing Vblank in step S23 so as to reduce the necessary
time Y as much as possible to the extent that expression
(3) is met.
[0142]
Thus, the overall time taken from the write start
10 time to the display timing is reduced for the oddnumbered
field image nodd and the even-numbered field
image n,,,,, enabling the image to be displayed with
minimized low delay.
[0143]
15 [Description of operation of receiving device 221
Now, with reference to the flow chart of Fig. 17,
the adjustment quantity calculating process performed by
the receiving device 22 will be described.
101441
20 The adjustment quantity calculating process starts,
for example, when the RTP packets are sent through the
network 23 from the sending device 21. The RTP
communication part 61 receives the RTP packets sent
through the network 23 from the RTP communication part 44
25 and supplies the packets to the packet assembling part 62.
The packet assembling part 62 assembles the RTP packets
from the RTP communication part 61 to generate the
encoded data to be decoded and supplies the data to the
decoding part 63. The decoding part 63 performs the
30 decoding process for the encoded data from the packet
assembling part 62 and supplies the resultant image data
to the write control part 64. The decoding process
corresponds to the encoding process performed by the
encoding part 42.
[0145]
5 The write control part 64 supplies the image data
(the odd-numbered field image and the even-numbered field
image) from the decoding part 63 to the buffer 65 for
storing (writing). In addition, the write control part
64 supplies the write start time Ts when the writing into
10 the buffer 65 has started and the write end time Te when
the writing has ended to the write information measuring
part 68 for each even-numbered field image n,,,,.
[0146]
The display control part 66 reads the odd-numbered
15 field image and the even-numbered field image as the
image data from the buffer 65 and allows the display part
67 to display the image data in synchronization with the
predetermined display timing Vblank.
[0147]
20 In step S41, based on the write start time Ts from
the write control part 64 and the predetermined display
timing Vblank, the write information measuring part 68
measures the write preparation time Xn for the evennumbered
field image n,,,, and supplies the write
25 preparation time Xn to the write information storage part
69 for storage.
[0148]
In step S42, based on the disclosure time Ts and
the end time Te from the write control part 64, the write
30 information measuring part 68 measures the buffer write
time Pn for the even-numbered field image n,,,, and
SP333120W000
supplies the buffer write time Pn to the write
information storage part 69 for storage.
[0149]
In step S43, the adjustment quantity calculating
5 part 70 determines whether a time measured by a built-in
timer part (not shown) has passed a predetermined time.
If the time measured is determined to have not passed the
predetermined time, the process reverts to step S41 and,
from then on, the process is performed in a similar
10 manner.
[0150]
If the adjustment quantity calculating part 70
determines in step S43 that the time measured has passed
the predetermined time, the process moves on to step S44.
15 [0151]
In step S44, the adjustment quantity calculating
part 70 reads the write preparation time Xn from the
write information storage part 69 and calculates the X
distribution information representing the distribution of
20 the write preparation time Xn that has been read.
[0152]
In step S45, the adjustment quantity calculating
part 70 calculates the threshold Xth based on the X
distribution information calculated in step S44.
25 [0153]
In step S46, the adjustment quantity calculating
part 70 calculates the necessary time Y based on the
display interval Vblank Interval and the threshold Xth,
using expression (2) .
30 [0154]
In step S47, the adjustment quantity calculating
part 70 reads the buffer write time Pn from the write
information storage part 69 and calculates the P
distribution information representing the distribution of
the buffer write time Pn that has been read.
5 [01551
In step S48, the adjustment quantity calculating
part 70 calculates the threshold Pth according to the P
distribution information calculated in step S47.
[0156]
10 In step S49, the adjustment quantity calculating
part 70 calculates the allowable delay time Yth based on
the threshold Pth calculated in step S48 and the expected
write time 1/60, using expression (1).
101571
15 In step S50, the adjustment quantity calculating
part 70 determines whether expression (3) is met based on
the necessary time Y calculated in step S46 and the
allowable delay time Yth calculated in step S49. If
expression (3) is determined to be met, the process moves
20 on to step S51.
[0158]
In step S51, the adjustment quantity calculating
part 70 calculates the adjustment quantity Zforward (= Y
- Yth) based on the necessary time Y and the allowable
25 delay time Yth as shown in Fig. 7B. The adjustment
quantity Zforward will be an adjustment quantity for
adjusting the capture timing Vblank such that the
necessary time Y is reduced to the extent that expression
(3) is met.
30 [01591
In step S52, the adjustment quantity calculating
part 70 sets the adjustment quantity Zbackward to zero
and supplies the adjustment quantities Zforward and
Zbackward to the RTP communication part 61. The process
moves on to step S53.
5 [0160]
Then, in step S53, the RTP communication part 61
generates the RTCP message including the adjustment
quantities Zforward and Zbackward from the adjustment
quantity calculating part 70 and sends the message
10 through the network 23 to the RTP communication part 44
of the sending device 21.
[0161]
In step S50, if the adjustment quantity calculating
part 70 determines, based on the necessary time Y
15 calculated in step S46 and the allowable delay time Yth
calculated in step S49, that expression (3) is not met,
the process moves on to step S54.
[0162]
In step S54, the adjustment quantity calculating
20 part 70 calculates the adjustment quantity Zbackward (=
Yth - Y) based on the necessary time Y and the allowable
delay time Yth as shown in Fig. 10B. The adjustment
quantity Zbackward will be an adjustment quantity for
adjusting the capture timing Vblank such that expression
25 (3) is met.
[0163]
In step S55, the adjustment quantity calculating
part 70 sets the adjustment quantity Zforward to zero and
supplies the adjustment quantities Zforward and Zbackward
30 to the RTP communication part 61. The process moves on
to step S53, where the process in step S53 described
above is performed. The adjustment quantity calculating
process is thus completed.
[0164]
As described above, according to the adjustment
5 quantity calculating process, the adjustment quantities
Zforward and Zbackward are calculated, according to, for
example, the write information of the even-numbered field
image n,,,,, so as to obtain the capture timing Vblank
that allows expression (3) to be met.
10 [0165]
Then, in the capture adjusting process, the capture
timing Vblank is adjusted based on the adjustment
quantities Zforward and Zbackward calculated through the
adjustment quantity calculating process.
15 [0166]
The receiving device 22, therefore, writes the
image with expression (3) met, and hence can restrain a
situation where a disturbance or the like is generated in
the image because a part of the image is skipped.
20 [0167]
In addition, for example, according to the
adjustment quantity calculating process, even if
expression (3) is met, the adjustment quantities Zforward
and Zbackward for adjusting the capture timing Vblank are
25 calculated so as to reduce the necessary time Y as much
as possible to the extent that expression (3) is met.
[0168]
Thus, the overall time taken from the write start
time to the display timing is reduced for the odd-
30 numbered field image nodd and the even-numbered field
image n,,,,, enabling the image to be displayed with
minimized low delay.
[0169]
<2. Exemplary variations>
Note that, in step S22, if the adjustment quantity
5 Zbackward = 0, the capture adjusting process allows the
process to move on to step S23 to advance the capture
timing Vblank by the adjustment quantity Zforward, but
this is not limiting.
[0170]
10 That is, for example, if the adjustment quantity
Zbackward = 0, the capture timing Vblank may be left
unadjusted because expression (3) is met. In this case,
as shown in Fig. 7B, the even-numbered field image n,,,,
will be displayed in synchronization with the display
15 timing Vblank (n,,,,).
[0171]
In addition, the adjustment quantity calculating
part 70 of the receiving device 22 calculates the
adjustment quantities Zforward and Zbackward, but the
20 adjustment quantity calculating part 70 may calculate
just an adjustment quantity to be used for adjusting the
capture timing. In this case, the RTP communication part
61 will not send an adjustment quantity that is set to
zero but send just an adjustment quantity to be used for
25 adjusting the capture timing.
[0172]
The capture adjusting part 45 of the sending device
21 will then use the adjustment quantity obtained through
the RTP communication part 44 and the network 23 from the
30 RTP communication part 61 to adjust the capture timing.
[0173]
Note that, in the present embodiment, the
adjustment quantity calculating part 70 of the receiving
device 22 calculates the adjustment quantities Zforward
and Zbackward. Nonetheless, the sending device 21, for
5 example, may be provided with the write information
measuring part 68 through the adjustment quantity
calculating part 70, so that the adjustment quantities
Zforward and Zbackward are calculated at the sending
device 21 side.
10 [0174]
In this case, the capture adjusting part 45 adjusts
the capture timing based on the adjustment quantities
Zforward and Zbackward calculated by the adjustment
quantity calculating part 70 at the sending device 21
15 side.
[0175]
If the sending device 21 is provided with the write
information measuring part 68 through the adjustment
quantity calculating part 70, the write control part 64
20 of the receiving device 22 supplies the write start time
Ts, the write end time Te, and the display timing Vblank
for each unit of image data to the RTP communication part
61. The RTP communication part 61 will then supply the
write start time Ts, the write end time Te, and the
25 display timing Vblank from the write control part 64
through the network 23 to the write information measuring
part 68 at the sending device 21 side.
[0176]
Note that if the sending device 21 is .provided with
30 the write information measuring part 68 through the
adjustment quantity calculating part 70, the receiving
device 22 may not be provided with the write information
measuring part 68 through the adjustment quantity
calculating part 70.
[0177]
5 In the present embodiment, the necessary time Y and
the allowable delay time Yth are calculated based on the
write information of the even-numbered field image n,,,,.
The necessary time Y and the,allowable delay time Yth may
also be calculated based on, for example, the write
10 information of the odd-numbered field image n,dd as shown
in Fig. 18. Alternatively, the necessary time Y and the
allowable delay time Yth may be calculated based on, for
example, the write information of the odd-numbered field
image nodd and the write information of the even-numbered
15 field image n,,,,.
[0178]
In addition, for example, if the image is displayed
with the progressive scheme, the display part 67 may use
a progressive image (frame image) as each image and
20 calculates the necessary time Y and the allowable delay
time Yth based on the write information of the
progressive image as shown in Fig. 19. The adjustment
quantity Zforward (= Y - Yth) or the adjustment quantity
Zbackward (= Yth - Y) then is calculated based on the
25 necessary time Y and the allowable delay time Yth that
have been calculated.
[0179]
Furthermore, when generating the X distribution
information representing the distribution of the write
30 preparation time Xn as shown in Fig. 4, the adjustment
quantity calculating part 70 generates the X distribution
information with the frequency of the write preparation
time Xn at 1 regardless of the timing of the measurement
of the write preparation time Xn.
[0180]
5 Nonetheless, the adjustment quantity calculating
part 70 may generate the X distribution information with
the frequency of the write preparation time Xn added more
as the measurement timing is more recent. This is
because, the more recently the write preparation time Xn
10 is measured, the more accurately the write preparation
time for the image to be received is likely represented.
[0181]
This also applies to the generation of the P
distribution information representing the distribution of
15 the buffer write time Pn as shown in Fig. 5.
[0182]
This allows the adjustment quantity calculating
part 70 to generate the X distribution information more
accurately representing the write preparation time for
20 the image to be received and to generate the P
distribution information more accurately representing the
buffer write time for the image to be received.
[0183]
The adjustment quantity calculating part 70 thus
25 will be able to calculate the necessary time Y and the
allowable delay time Yth for the image to be received
more accurately based on such X distribution information
and such P distribution information. Hence, the
adjustment quantity calculating part 70 will be able to
30 calculate a more accurate adjustment quantity based on
the result of a comparison between the necessary time Y
and the allowable delay time Yth (the result of the
determination whether expression (3) is met). The
capture adjusting part 45 of the sending device 21,
therefore, will be able to adjust the capture timing
5 Vblank more appropriately based on the adjustment
quantity calculated by the adjustment quantity
calculating part 70.
[0184]
Note that this technique may take the following
10 configurations.
(1) A sending device including an obtaining part that
obtains an adjustment quantity for adjusting a capture
timing to start capturing an image, a capture adjusting
part that adjusts the capture timing according to the
15 adjustment quantity, a capture part that captures the
image in synchronization with the adjusted capture timing,
and a sending part that sends the captured image to a
receiving device.
(2) The sending device according to (1) above, wherein
20 the receiving device includes a holding part that
temporarily holds the image, writes into the holding part
the image received from the sending part, and allows the
image written in the holding part to be displayed in
synchronization with a predetermined display timing, and
25 the obtaining part obtains the adjustment quantity
calculated based on a result of comparison between an
allowable delay time representing a delay time allowed
when the writing of the image into the holding part is
not completed within a predetermined needed time from a
30 write start time when the writing has started, and a
necessary time at least taken from the write start time
for the image that has been written with a delay time of
the allowable delay time or less to the display timing.
(3) The sending device according to (2) above, further
including a first calculating part that calculates the
5 allowable delay time, a second calculating part that
calculates the necessary time, a comparing part that
compares the allowable delay time and the necessary time,
and an adjustment quantity calculating part that
calculates the adjustment quantity based on a comparison
10 result of the comparing part, wherein the obtaining part
obtains the adjustment quantity calculated by the
adjustment quantity calculating part.
(4) The sending device according to (3) above, wherein,
if a comparison result is obtained to show that the
15 necessary time is longer than the allowable delay time,
the adjustment quantity calculating part calculates the
adjustment quantity for adjusting the capture timing such
that the necessary time is reduced to an extent that a
condition that the necessary time is longer than the
20 allowable delay time is met.
(5) The sending device according to (3) or (4) above,
wherein, if a comparison result is obtained to show that
the necessary time is not longer than the allowable delay
time, the adjustment quantity calculating part calculates
25 the adjustment quantity for adjusting the capture timing
such that the necessary time becomes longer than the
allowable delay time.
(6) The sending device according to (2) above, wherein
the obtaining part obtains the adjustment quantity by
30 receiving the same from the receiving device.
(7) The sending device according to (3) above, further
including a first measuring part that measures a write
time taken to write the image into the holding part of
the receiving device, wherein the first calculating part
calculates the allowable delay time based on a
5 distribution of the write time.
(8) The sending device according to (3) or (7) above,
further including a second measuring part that measures a
write preparation time taken to start writing the image
into the holding part of the receiving device, wherein
10 the second calculating part calculates the necessary time
based on a distribution of the write preparation time.
[0185]
Note that the series of processes described above
may be executed by hardware or software. When the series
15 of processes is executed by software, a program
constituting the software is installed through a program
recording medium into a computer integrated in special
hardware, or, for example, into a general-purpose
computer that can execute different functions with
20 different programs installed.
[0186]
[Exemplary configuration of computer]
Fig. 20 is a block diagram of an exemplary
configuration of hardware for a computer that executes,
25 by a program, the series of processes described above.
[0187]
A CPU (Central Processing Unit) 201 executes
different processes according to the program stored in a
ROM (Read Only Memory) 202 or a storage part 208. A RAM
30 (Random Access Memory) 203 stores the program to be
executed by the CPU 201 and data as appropriate. The CPU
201, the ROM 202, and the RAM 203 are mutually connected
through a bus 204.
[0188]
The CPU 201 is also connected to an input/output
5 interface 205 through the bus 204. The input/output
interface 205 is connected to an input part 206 including
a keyboard, a mouse, a microphone, and the like and to an
output part 207 including a display, a speaker, and the
like. The CPU 201 executes different processes
10 corresponding to an instruction input from the input part
206. The CPU 201 then outputs a result of the processes
to the output part 207.
[0189]
The storage part 208 connected to the input/output
15 interface 205 includes, for example, hardware, and stores
the program to be executed by the CPU 201 and different
types of data. The communication part 209 communicates
with an external device through a network such as the
Internet and a local area network.
20 [0190]
Furthermore, the program may be obtained through
the communication part 209 and stored in the storage part
208.
[0191]
25 A drive 210 connected to the input/output interface
205 drives a removable medium 211 such as a magnetic disk,
an optical disk, a magneto-optical disk, or a
semiconductor memory when mounted and obtains the program,
data, and the like stored therein. The obtained program
30 and data are transferred to the storage part 208 as
needed for storage.
[01921
A recording medium to record (store) the program
that is to be installed in the computer and made
executable by the computer includes, as shown in Fig. 20,
5 the removable medium 211 that is a packaged medium
including a magnetic disk (including a flexible disk), an
optical disk (including a CD-ROM (Compact Disc-Read Only
Memory) and a DVD (Digital Versatile Disc)), a magnetooptical
disk (including an MD (Mini-Disc)), a
10 semiconductor memory or the like, a ROM 202 storing the
program temporarily or permanently, a hard disk
constituting the storage part 208, or the like. The
program is recorded into the recording medium through the
communication part 209 that is an interface such as a
15 router and a modem as needed by using a wired or wireless
communication medium such as a local area network, the
Internet, and digital satellite broadcasting.
[0193]
Note that, in this specification, the steps
20 describing the series of processes described above of
course include processes to be performed in time sequence
in the described order but may also include processes
executed in parallel or individually even if not
processed in time sequence.
25 [0194]
In addition, in this specification, a system refers
to equipment including multiple devices.
[0195]
Note that the present embodiment described above
30 should not be construed to limit embodiments in the
present disclosure, and various modifications are
possible without departing from the s p i r i t of the present
disclosure.
REFERENCE SIGNS LIST
5 [0196]
21 Sending device
22 Receiving device
23 Network
4 1 Capture p a r t
10 42 Encoding p a r t
43 Packetizing p a r t
44 RTP communicationpart
Capture adjusting p a r t
Operation p a r t
Control p a r t
RTP communication p a r t
Packet assembling part
Decoding p a r t
Write control p a r t
Buffer
Display control p a r t
Display p a r t
Write information measuring p a r t
Write information storage p a r t
Adjustment quantity calculating p a r t
Operation p a r t
Control p a r t

CLAIMS
1. A sending device comprising:
an obtaining part that obtains an adjustment
5 quantity for adjusting a capture timing to start
capturing an image;
a capture adjusting part that adjusts the capture
timing according to the adjustment quantity;
a capture part that captures the image in
10 synchronization with the adjusted capture timing; and
a sending part that sends the captured image to a
receiving device.
2. The sending device according to claim 1, wherein
15 the receiving device comprises a holding part that
temporarily holds the image,
writes into the holding part the image
received from the sending part, and
allows the image written in the holding part
20 to be displayed in synchronization with a predetermined
display timing, and
the obtaining part obtains the adjustment quantity
calculated based on a result of comparison between
an allowable delay time representing a delay
25 time allowed when the writing of the image into the
holding part is not completed within a predetermined
needed time from a write start time when the writing has
started, and
a necessary time at least taken from the
30 write start time for the image that has been written with
a delay time of the allowable delay time or less to the
display timing.
3. The sending device according to claim 2, further
comprising:
5 a first calculating part that calculates the
allowable delay time;
a second calculating part that calculates the
necessary time;
a comparing part that compares the allowable delay
10 time and the necessary time; and
an adjustment quantity calculating part that
calculates the adjustment quantity based on a comparison
result of the comparing part,
wherein the obtaining part obtains the adjustment
15 quantity calculated by the adjustment quantity
calculating part.
4. The sending device according to claim 3, wherein,
in a case where a comparison result is obtained to
20 show that the necessary time is longer than the allowable
delay time, the adjustment quantity calculating part
calculates the adjustment quantity for adjusting the
capture timing such that the necessary time is reduced to
an extent that a condition that the necessary time is
25 longer than the allowable delay time is met.
5. The sending device according to claim 4, wherein,
in a case where a comparison result is obtained to
show that the necessary time is not longer than the
30 allowable delay time, the adjustment quantity calculating
part calculates the adjustment quantity for adjusting the
capture timing such that the necessary time becomes
longer than the allowable delay time.
6. The sending device according to claim 2, wherein
5 the obtaining part obtains the adjustment quantity
by receiving the same from the receiving device.
7. The sending device according to claim 3, further
comprising
10 a first measuring part that measures a write time
taken to write the image into the holding part of the
receiving device,
wherein the first calculating part calculates the
allowable delay time based on a distribution of the write
15 time.
8. The sending device according to claim 3, further
comprising
a second measuring part that measures a write
20 preparation time taken to start writing the image into
the holding part of the receiving device,
wherein the second calculating part calculates the
necessary time based on a distribution of the write
preparation time.
25
9. A sending method for a sending device that captures
and sends an image, the sending method comprising the
steps of, by the sending device:
obtaining an adjustment quantity for adjusting a
30 capture timing to start capturing the image;
adjusting the capture timing based on the
adj ustment quantity;
capturing the image i n synchronization with the
adjusted capture timing; and
sending the captured image to a receiving device.
5
10. A program for allowing a computer to function as:
an obtaining part that obtains an adjustment
quantity for adjusting a capture timing to s t a r t
capturing an image;
10 a capture adjusting p a r t t h a t adjusts the capture
timing based on the adjustment quantity;
a capture part that captures the image in
synchronization with the adjusted capture timing; and
a sending control part that allows the captured
15 image to be sent to a receiving device.