r
i
1
SP345665WO00
DESCRIPTION
DECODING DEVICE AND DECODING METHOD, AtJU ENCODING DEVICE
AND ENCODING METHOD
5 TECHNICAL L-'lELD
[0001]
The present technique relates to a decoding device
and a decoding method, and an encoding device and an
encoding method, and more particularly to a decoding
10 device and a decoding method, and an encoding device and
an encoding method capable of performing enoodi ng and
decoding independently in the time direction for each
tile.
15 BACKGROUND ART
[0002]
Currently, st.andardination of encoding system
oat 1 eri High Rf iiici enoy Video Cod.i ng (HKVC) i s promoted by
Joint Coli aborati on Team-Video Codi.nq (JCTVC), a joint
20 standardisation onjan i *«Li on of TTU-T and ISG/IEC, aiming
at I ijrl.her improvement ot H. 2 64/AVC i n onoodi.ng
eMieieney. Conoi?rn i rig HRVC standards, a committee draft
as the initial draft version is issued in February, 2012
(cii* , see Non PeUont T.i.Lerature 1} .
25 [0003]
According to IIEVC standards, a picture can be split
into units of tiles or slices for encoding. In decoding
Etn encoded stream split into these units and encoded, no
correlation exists between the split units in the process
30 of creating information about Context-based Adaptive
Binary Arithmetic Coding (CABAC), intra prediction modes,
2
SP345665WO00
quantization values, and the like.
[0004]
However, according to inter prediction, no
restriction is set to motion vectors - In this case, an
5 encoded image ol a di I fcrent tile at a different time can
be used as a re Terence image. Accordingly, independent
encoding and decoding in the time direction for each Life
1 a not a flowed,
10 More specifically, as shown in Fig. 1, for exaciiplo,
each of a frame #t having a Picture Order Count [POCJ of
L" and a frame #t-l having a POC of t-1 is split into four
tiles and inter-predicted, all the encoded Images within
the four tiles of the frame Jft-1 can be determined as
15 possible reference images for a CU [Codi nq Unit) of the
frame #t.
[0006]
Accordingly, there is a case when a decoded image
12 within a tile fJ-2 having a speoi I ic TD {hereinafter
20 referred to as a tile ID) til 2 and contained in the frame
#t-l is determined as a reference i mage for a CU 11 of a
tile fll having a tile ID oi. 1 and contained in the frame
Ut, for example. Tn other words, there is a case when a
vector which has an i.ni t. i a I pel n t of a CU11 and a
25 terminal point of an area 12A of the frame (Jt
corresponding to I. he decoded image 12 is detected as a
motion vector 13. In this case, reference to the decoded
image ~\7 of" the tile #2 different from the tile tfl
containing the CIJ 11 is needed; therefore, independent.
30 encoding and decoding in the time direction I or each tile
is not allowed.
3
SF345665WO00
[0007]
Accordingly, a decoding device needs to have a
common decoding Decoded Picture Buffer (DPB) which
retains decoded imaqes i'or all tiles.
5 [0008]
Fig. ? \ s a bE ot:k di agrain showing a constitution
example oi: a decoding device of this type,
[0009]
A deeod i no, deviee 30 in Fig. 2 is constituted by
10 decoding uniIs 31-1 through 31-N, DFB 32-1 through 32-N,
and a common DFB 33.
10010]
An encoded stream containing split N tiles (N is an
arbitrary positive number) and encoded for each nnl L of
15 tiles is inputted to the decoding device 30. Encoded
data of each tile is supplied to the correspond i no,
decoding units 31-1 through 31-N.
[0011]
Each of the decoding units 31-1 through 3I~N
20 decodes the encoded data o t: the cor responding tile by
using the corresponding image of decoded i mao.es stored in
the common DPB 33 for all ti les contained i n Lhe
corresponding frame as a reference imago.
[0012]
2b More speelf icaI Iy, lhe decoding unit 31-1 decodes
encoded dala of a I i .1 e #1 having a tile ID of 1 using a
reference i mage, and supplies a decoded image of the tile
ftl obtained as a result of the decoding to the DPB 32-1.
Similarly; the decoding units 31-2 through 31-N decode
30 data of a tile #2 having a tile ID of 2, a tile If3 having
a tile ID of 3, and up to a tile UN having a tite TD of" N
4
SP345665WO00
using reference images, respect! voJ.y. Then, the decoding
units 31-2 through 31-N suppIy the decoded images of the
tile fJ2, tile fJ3, and up i.o Li l.e ttN obtained by decoding
to the DPB 32-2, DPB 32-3, and up to DPB 32-tJ,
5 respectively.
[0013]
The DPB 32-1 Lhrouqh 32-N store the decoded images
supplied by the corresponding decoding units 31-1 through
31-N. The DPB 32-1 through 32-N supply the stored
10 decoded images Lo tho common DPB 33 and allow these
decoded images to bo stored in the common DPB 33.
[00141
Tho common DPB 33 stores the decoded images of." the
tiJo #1 thi.-oi.igh tho tile #N at the same time supplied by
15 the DPB 32-1 through 32-N as decoded images of. one f rattie.
The common DPB 33 outputs the stored decoded images tor
each unit of frames as decoded results.
[0015]
in addition, though not shown in the l:\ gures, a
20 common DPD needs to be provided on an encoding dovice for
inter prediction similarly to the DPB on (.he decoding
device 30.
CITATION LIST
25 NOK PATENT DOCDMRTCT
[001ft]
Non P a t e n t Document 1 : Berrjamin B r o s 3 / Woo-Jin Han, J e n s -
R a t n e r Ohm, Gary J. S u l l i v a n , Thomas W i e g a n t , "'High
e f f i c i e n c y v i d e o c o d i n g (HJiVC) t e x t s p e c i f i c a t i o n d r a f t:
30 6" JCTVC-H1Q0U3 vor 2 1 , 2 0 1 2 , 2 . 17
5
SE'345665WO00
SUMMARY OF THE INVENTION
PRORT.EMS TO lit; SOLVED BY THE INVENTION
loci n
As described above, according Lo IJEVC standards, no
b restriction is set to motion vector^ m inter prediction.
In this case, a decoded image or a different tile at a
different time can be used as a reference image.
Accordingly, independent. encoding and decoding in the
time direction for each tile i s not allowed.
10 [0013]
The present 1-eohni.que has been developed in
consideration of these si Luationa, and is provided as a
technique capable QV performing encoding and deood Lily in
the time direction i ndeperidontly for each tile.
15
SOLUTIONS TO PROBLEMS
[0019]
According Lo a first aspect of the present
technique, there i 3 provided a decoding device,
20 inetudirnj; a motion compensation unit generating a
predicl.i on i nictqe by performing, for each of L i I os, motion
oompensaLi on of a reference image within a co-I ocated
tile based on tile splittable informal, i on indicating that
deood i EKJ I 3 allowed for each of the tiles and motion
2b vector information represent! riq a mot Ion vector used for
generating encoded data oi a decod Lny target current
image when a picture of the current image is split into
the tiles and decoded; and a decoding unit decoding the
encoded data using the prod I ction image generated by I.be
30 motion compensation uni I..
[0020J
6
SP345665WG00
The decoding method according to the first aspect
of the present technique corresponds to the decoding
device according to the first aspect of the present
technique.
5 [0021]
According to the first aspect of the present
technique, a prediction image is generated by perform?ng,
for each of tiles, rnoti on compensat i on ol a reference
image within a co-located tile based on tile spl ittable
10 information indicating that decoding is at lowed lor each
of. the tites and motion vector information representing a
motion vector used for genera t E nq encoded data ol" a
decoding target current image when a picture of the
current image is split into the tiles and decoded, 'i'he
lb encoded data is decoded using the prediction image,
Accordi.mi to a second aspect of the present
technique; l here is provided an encoding device,
including: a motion compensation unit generating a
20 prediction image by performing motion compensation of a
reference image at a time different from the time of an
encoding target current image based on a motion vector
detected within a tile when a picture of the current
image is split into the tiles and encoded; an encoding
25 unit encoding the current image and generating encoded
data using the prediction image generated by the rnoti on
compensation unit; a setting uni t setting tile spli ttabIe
in lormation indicating that deoodi aq is all owed I or eaoh
ntii I: of the t i "les; and a trans mi ssi on tjn i t t ran sill i tt. i ng
30 the encoded daUi generated by the encodinq unit, mul the
trie sp t i I.tab I e i n I oritw I.i on nc.\ by the set. tinq unit,
7
SP345665BO00
[0023f
The enoodinq method according to the second aspect
of the present technique corresponds to the encoding
device according to the second aspect of the present
5 technique,
£00341
According to tho second aspect of the present
technique, a prediction image is generated by performing
motion compensation of a reference image at a time
3 0 different from the time of an encoding target current
image based on a motion vector detected within a tile
when a picture of the current image is spli t into tho
tiles and encoded. Encoded data is qenera Led by encoding
the current image using the predict'on fmage. Tile
15 splittable information indicating that decoding is
allowed for each unit of the tiles in set. The encoded
data and the tile split table i nl ormat-' on are transmitted.
[0025]
Further, the decodj nq dev i ce oi t-hc- first aspect
20 and the encodi ttq dev i co of" the second aspect can be
realized by making a computer execute a program.
[0026]
Moreover, (.he program executed by the computer lor
reali ?.\ nq the decod i riq device of the first aspect and the
7b encoding device of the second aspect can be provided by
Lransm.i Lting the program via a transmission medium, or by
recording the program on a recording medium.
[0027]
in addition, the decoding device of the I i rsL
30 aspect and the encoding device of the second aspect may
be separate devices, or may be inside hi in:ks constituting
a
SPJ456S5WO00
one device,
KFFECTS OF THE INVENTION
[0028]
5 According to the first ^specL of the present
technique, independent deoodi nq i n the time direction for
each tile is allowed.
[0029]
Moreover, accord3nq to the second aspect of the
10 present technique, i ndepejldent encoding in the time
direction lor each lile is allowed.
BRIF.F DESCRIPTION OF DRAWINGS
(0030f
15 Flq, 1. is a diagram describing a reference image
I or conventional inter prediction.
Fig. 2 is a block diagram showing a constitution
example of a conventional decoding device.
Fig. 3 is a block diagram showing a constitution
20 example of an encoding device to which the present
technique is applied according to a first embodi ment.
Fig . A is a block diagram showing a const i. tut ion
example of an encoding unit in Fig. 3 -
L'ig. 5 is a diagram describing ti I es.
25 Fig. 6 is a diagram deser i b i ng a restriction set
when a motion vector i s detected.
Fig. 7 is A d'\ aqrrtiri describing a reference image
for inter predi cti on ,
Fig. 8 is
[Constitution example of Encoding Llevice in First
25 Embodiment)
Fig. 3 is a block diagram showing a coiiaLi Lul-i Oft
example of an encoding device to whi oh (.he pre^enl.
technique is applled according Lo a lirsL embod imenL.
[00321
30 An enoodi nq dev f a-. hO in Fiq. ~3 i s const! tubed by
an A/D con vers i on uri 1 I. bl , a scroen rear rangement buffer
12
SP34b665WG0O
52, a split unit 53, encoding units 54-1 111 rough 54-fJ, a
setting unit 55, and a transmission unit b6. The
encoding device 50 performs, for each Li I e, eomprossionencoding
of an image in each unit of Iraracs inputted as
5 input signals by a system corresponding to HEVC system,
[0033]
More specifically, the A./D conversion unit 51 of
the encoding device bO perforins A/D conversion of images
in each unit of frames inputted as input signals, and
10 outputs the converted imager to the screen rearrangement
buffer 52 and allows the screen rearrangement buffer 52
to store the images. The screen rearrangement buffer 52
rearranges the stored images in the respective units of
frames 1 oca Led i n the order of display in such positions
15 that: the images are located in the order of encoding in
accordance with Group of Picture (GOP) structure, and
supp I i cs Lhe rearranged images to the split unit 53.
f0034|
The sp.l i.L unit 53 splits each of the images
20 supplied by the screen rearrangement buffer 52 i nLo N
ti.l es based on information indicating split posi t i on a ot"
tiles and a split number b\ specified for each uni t ot
sequences in correspondence with operati on O l a not-shown
input unit operated by a user (hereinaf ter referred to as
25 tile split information) . The spl i I. unit b'3 supplies the
images of the N tiles to the encoding units 54-1 through
54-N, respective! y, as encori f ng target images .
{0035]
The encoding units b4 I through 54-N perform
30 compression-encoding ol the images of the corresponding
Li 1 es rfiipp I i nd by the .spJ it unit 53 independently in the
13
SP3456G5WO00
time direction by a system corresponding to IIKVC system.
The encoding units 54-1 ihrouqh 54-N supply encoded data
of the respective tiles obtained da a result of the
compression encoding to Lhe seLLlng unit 55, I'iirther, in
5 the following description, lho encoding units 54-1
through 54-N are cottecUvely referred to as the encoding
units 54 when distinction between the encoding units 54-1
Lhrouqh 54-N is not particularly needed,
[0036]
10 The seLLinq un.it 55 synthesizes the encoded data of
the respective tiles supplied by the encoding units 54-1
through 54-N based on the tile split information. In
addition, the setting unit 55 sets Sequence Parameter Set
fSPS), Picture Parameter Set {PPS), Video Usability
15 Information (VU1), Adaption Parameter Set (APS), and Lho
like based en the tile split information. The sel.li.nq
unit 55 generates an encoded stream by adding SPE, PPS,
VUi, APS and the like to the synthesized enooded dal.a,
and supplies the generated encoded stream to the
20 transmission unit 56.
E00371
The transmission unit 56 transmi Ls Lhe encoded
stream supplied by the seU.lnq unit 55 to a decoding
device described later.
25 [0038]
[Constitution example of Encoding Unit}
Fiq. '4 i a a block diagram showing a constitution
exarrl e of Lhe encoding unit 54 in L'ig. 3.
{0039 J
30 The encoding unit 54 m i"ig. 4 is constituted by a
calculation unit 71, an orthogonal transformation uni L >7,
14
SP345665WO00
a quantization unit 73, a lossless encoding unit 74, a
store buffer 75, an inverse quanti nation unit 76, an
inverse orthogonal trans forma I. i on unit 77, an addition
unit 78, a deblock filter '( a
prediction block adj acent to the co-1 oca ted bIock in tho
15 spatial direction, and the like are determi ned &3
prediction vectors.
EGCM8]
Farther, according to this ftpet;! I ication, the
condition "co-located" refers to a tionti i I. i on so disposed
?0 as to have tho saino pos i t i.ofial f.eJ ationship [positioned
at the same place) in different pictures (frames, fields).
Accordingly, the co-locatod block rciuja to a block
having tho same positional relationship (positioned at
tho same p I ace) i n dl/.Terei] t pictures (frame, fields} .
25 Also, co-located pixels are pixels having the same
positional relationship (positioned at the same place) in
different pictures (frame, fields).
[00491
In addition, according to this specification, the
30 adjacent (neighboring) condition refers to such a
condition as to have a positj onal rei a t ionsh i p a I I ow i noj
17
SPl4566bWO0U
reference from a current picture fframe, field}. It is
preferable that this positional relationship cyrresponds
to the position immediately before or immediatef y aftor
in view of time. However, this relationship i ^ not
i> required its long as the effects of the present technique
can be offered. Further, the adjacent eond i LLon iit the
time direction and the adjacent oondf 11 on i.ll the spatial
direction are collectivet y referred be as an adjacent
condition when distinction therebetween is not
10 particularly needed. The adjacent i:ondi L.i on in the time
direction represents a pos i Li ona.t relationship allowing
reference in the time direct!on. The adjacent condition
in the spatial direct i on represents to a positional
relationship allowinq re Terence within the same picture.
15 [0050]
The lossless eneod i.nq unit 7 4 restricts the
prediction b]oek oI the motion vector defined as the
predict!on vector Ie Lhe prediction blocks within the
same tile as Lhe Li le of the prediction block of the
20 motion vector information. In this case, the decoding
dftvkift is not required to refer to motion vectors of
other tiles- Accordingly, the encoded data after inter
prediction encoding can be decoded independent I y f n Lhe
Li.me direction for each tile.
25 [0051]
In addition, merging in Tor ma I. i on may be used as the
motion vector information. The mere, i ruj i.nformation is
information indicating whether a [jredi etion block, of a
motion vector is to be merged wi l.h another prediction
30 block, and which predi c|. i on bl ock the; motion vector is to
be mei ged wi th at the L ime of" merging . Possible
18
prediction blocks for merging involve a prediction btock
adjacent to the prediction block of the motion vector in
the spatial direction, a co-located block, and a
prediction block adjacent to the co-located block in the
5 spatial directiony for example.
[0052]
lis this case, however, the possible prediction
blocks for merging are restricted t:o the prediction
blocks within the same tile as the tile of the prediction
10 block of the motion vector information. Aeeordingly, the
decoding device is not: required l.o refer to motion
vectors of other tiles; therefore, the encoded data after
inter prediction encoding can be i ndepcridently decoded in
the time direction Cor each Li le.
15 [0053]
The lossless encod i nq uil it 7 4 determines whether
the motion vector supplied by the inter prediction unit
83 is j denti oal to any of" the motion vectors of the
possible prediction blocks Tor merging when the merging
?0 information is used as the motion vector information.
When determi n i rig as 1 dent real, the lossless encoding unit
IA qeneral.es, as moUon vector information, merging
information indicating that merging is performed with the
possi.b le prediction block for merging determined as a
25 vector identical block. On the other hand, when
determining as not identical, the lossless encodi n§ unit
7 4 generates, as motion vector inf orinal. 1 on, morq i ng
information indicating that mergi ng is not performed,
[0051]
30 The lossless encod i nq unit M per fof.uis lossless
encoding of the intra pred i c|. i on mod'.' information after
19
SP345665CTO00
differential encoding, or the mter prediction mode
information, motion vector i nf.otmation, information l:oi"
specifying a reference i maqe, and the like, and
determines the informs I: i on a J" Lor lossless encoding as
5 encoded information about encoding* The lossless
encoding unit 14 supp t ies the coefficients and enooded
information after loss I oas encoding to the store buf"f er
75 as encoded da La and allows the store buffet: 7 b to
store the encoded da La- [•'urther, the encoded i ni oi illation
10 may be determined as header information of the
ooeli tii oi.enl.s after lossless encoding.
[0055]
The store buffer 75 temporarily stores the encoded
data suppl i ed by the lossless encoding tm i L 74 > In
15 addiL.ion, Lhe store buffer 75 supplies (.he stored encoded
data Lo Lhe setting unit 55 in Fig. 3.
[OObdl
Furthermore, the quantized ooer\ ioients outputted
i roul Lhe quantization unit 73 are at^o inputted to the
^0 inverse quantization unit 76, $\\.)
[0060]
D is the difference (distort.! on) between I.he
25 original image and the decoded imago, Header__Bit is
header bits for a prediction mode. QPtoQuant Js a
f unci.i on given as a Tunction of a quantizal. i on parameter
QT?.
[0069J
30 in Low Complexity mode, generation of" decoded
i mages for all the prediction modes is only needed, and
?3
SP34b665WO0Q
the necessity of performing Lossless encoding i ft
slim i natod. Accordingly, |:ho amount of" calculit". i.on
decreases.
[0070]
5 The inter prediction unit S3 is constituted by a
motion detection unit 83A and a motion compensation unit
83R, and performs (notion prediction and compensation
processing of:" all the possible inter prediction modes.
More specifically, the motion detection unit G3A performs
10 motion prediction within the tile or the encoding target
image by usi ny the encoding target image supplied by tho
split unit 53, and a reference image read from the DPB 80
via the switch 81, positioned at a time different from
the time of" the corresponding encoding target image, and
lb filtered by the deblock filter 753.
[0071]
More specific 1 I.yi the motion detection unit 83A
detects motion vectors for all the possibE o inter
predict"! on modes by using the encoding target image, and
20 a reference image J i.ltered, ooritciined wi thin the same
tile as the tile ol tho encodi ny target i mage, and
posil'.i oned in a frame different from the frame of the
encoding target image. The motion compensation unit 83B
per f i.>rrri3 inter prediction by performing motion
25 compensation or the reference image filtered by the
deblock, filter 79 based on the motion vectors detected by
the motion detection unit 83A, and generates a prediction
i mage.
[0072]
30 At this time, the inter prediction unit S3
calculates the cost f miction val ijos for all the possi bi e
24
SP.345665WO00
inter predf ction modes leased on the encoding target imago
and the prediction imago, and determines the inter
prediction [node where the cost function va J ije becomes the
minimum as the optimum inter measurement mode. Then, the
b inter prediction unil. 83 supplies the cost function value
for the optimum inter prediction mode, and the
corresponding prediction image to the prediction image
selecUon unit 84. In addition, the inter prediction
unit 83 outputs the inter preriiction mode i.nformation,
10 the corresponding motion vector, the information for
specifying a reference image, and the like to the
lossless encoding unit 74 when notified I rum the
prediction image selection unit 84 about selection of the
prediction image generated in the optimum inter
15 prediction mode,
[0073]
The prediction image selection unit 34 selects the
prediction mode where the correspond.! ng cost function
value is smaller from the optimum intra predicti on mode
20 and the opti mum inter prediction mode based on the cost
I."unction values supplied by the intra prediction unit 02
and the inter prediction unit 03, and determiner the
selected prediction mode as the opiimum prediction mode,
Then, the prediction imago selecti on unit Q4 supplies the
2.5 predicts on image in Uie optimum prediction mode to the
calculation unit 7"l and the addition unit /H. Moreover!
the prediction image selection unit 84 notifies the intra
prediction unit 8? or the inter prediction unit 03 about
setection of the prediction image in the optimum
30 prediction mode.
E00MJ
25
SP345665WO00
The i:ate c o n t r o l u n i t 85 c o n t r o l s the f a t e of t he
q u a n t i z i n g o p e r a t i o n of the q u a n t i z a t i o n u n i t 73 based on
t h e encoded d a t a s t o r e d in t h e s t o c o b u f f e r 7b such t h at
n e i t h e r o v e r f l ow nor underflow o c c u r s.
5 [0075]
[ D e s c r i p t i o n of Til e}
F i q . 5 is a diagram showinq t i l e s .
fflOVGJ
As shown in F i g . 5, one p i c t u r e (frame) can be
10 s p i i I i n t o a p l u r a l i t y of t i l e s and encoded- According
Lo the example i n l^'ig. 5, otto p i c t u r e i y s p l i t i n t o four
ti l e s - ftach t i l e is g i v e n a I- Lie ID s t a r I. i.ng f rovn 0 in
Ule o r d e r of r a s t e r scan. F u r t h e r , L a r g e s t Codinq U n i ts
£LCUsJ w i t h i n a t i l e a r e encoded in t h e o r d e r of r a s t er
15 scan.
[0077]
Moreover, one p i c t u r e can a l s o be s p l i t i n t o a
p l u r a l i t y of M f . i e e s . The b o u n d a r i e s between t he
r e s p e c t i v e t:i ley may be e f t h e r i d e n t i c a l to or d i f f e r e nt
20 from t h e b o u n d a r i e s between t h e r e s p e c t i v e si i c e s ,
According to the examp] e i.n li'ig. 5, each of a t i l e #0
h a v i n g a t.i k1 1EJ of 0 and a t i l e fll having a ti le ID of 1
i s coiifll.i l u t e d by two s l i c e s . Also, each oi a t i l e U3
h a v i n g a ( . l i e ID of 3 and a t i l e fj4 having a Li l.e ID of 4
25 c o n s t i t u t e s one s l i c e . According to t h i s embodiment,
however, a p l u r a l i t y of t i l e s does not c o n s t i t u t e one
s l i c e . In o t h e r words, a t i l e c o n t a i n s at l e a s t one
s i i c e . Accordingly, encoded d a t a of each t i l e always
c o n t a i n s a s l i c e h e a d e r ; t h e r e Tore, encod i rig can be
30 performed for each u n i t of t i I e s . Furthermore, when one
t i l e c o n t a i n s a p l u r a l i t y of s l i c e s , the s l i c e s wi t h in
?6
SP34S665WOG0
the corresponding Li lo are encoded in (.he order ol." raster
scan.
[007R]
(Description of R e s t r i c t i o n on Motion Vector Detection)
5 Fig, 6 is a diagram showing a r e s t r i c t i o n s e t when
t h e motion d e t e c t i o n unit 83A in Fig. 4 d e t e c t s motion
v e c t o r s .
[0079]
As shown E n t,big. 6, the motion d e t e c t i o n u n i t 03A
10 perl onus motion p r e d i c t i o n w i t h i n a 1. i le by s e t t i n g such
a r e s t r i c t i o n that p o s s i b l e reference images of a GU
with In a t i l e if i having a t i l e ID of" i. ttre on I y images
within the t i l e # i . Accordingly, a motion vector MV (KIVX,
mvy} {unit: pixel) s a t J s t i c s the following equation {3} .
15 [0030]
[Equation 3]
x + mux >. minX in TilelD i
y -f mvy £ minY if, T i l e lO i
x + w f mvy. < maxX_in_TileTn_i
90 y + h I mvy < maxi_in TileID_i
[OOtfl]
F u r t h e r , in the equation (3), (x, y) are
coordinates of a pixel unit corresponding to a pixel
25 located at the upper l e f t ol the CCJ, and each ol w and h
is -a I ength of a p i x e l uni t In the hor \ c e n t a l i-/ i d th and
the v e r t i c a l width, respect I v e l y . Moreover,
mtiiX_iri_i'ileID i corresponds to t h e x coordinate value of
the p i x e l at the upper 1 e I t. of t h e ti l.e #i, and
30 mi nY_in_TileTD_i corresponds to the y coordiiirtte value ot
the pixel at the upper left of the t i l e Jfi. Furthermore,
27
SP345665WO00
maxX in_TMclU_i a or responds to Lhe x coordinate value of
the pixel at the lower right of Lhe tile Ifi, and
maxY in_Tilu±D_i corresponds to the y coordinate value of
the pixel at the lower right of the tile fji.
5 [0082]
The restriction established at the time of
detect i on of the mo Lien vector as discussed above
el frninates the necessity of using a decoded image of
ano Liter tile, as shown in Fig. 1, as a re Jeronce imaqe in
10 the Lime direction at the time ef inter prediction*
[0083]
More spec! I i.eally, as shown in Fig. i, inter
prediction of Lhe CU within the tile JM of the frame #t
is performed by using an image within Lhe tile #1 of the
15 Trame #t-l as a reference imago when each of the (",-:ame #t
having a POC of" t and the frame #t-l having a POC of t-1
is split into four tiles. Similarly (e the tile fit,
inter predicl.i.on is performed fur the respective CUs of
tiles fl2 through #4 by using images within their own tile
20 #2, tile (13, and tile ft 4 /i.^ that decoding is not allowed for
30
KP31566bWO00
oach tile based on the eonsideral: i on that the tile
splittable information is false (0).
[0095]
[Description of Process performed by F-neoding Dev i co)
5 Fig. 3 2 is a flowchart describing an encoded stream
generating process performed by the encoding device 50 in
li'ig. 3.
[0096]
In stop Sll in Fiq. 12, the A/D conversion unit 51
10 performs A/D conversion of an image of each unib of
frames inputted as input signals, and output..1? the
converted Image to the screen rearrangement buffor 52 and
allows the screen rearrangement buffer 52 to store the
image,
15 [009/J
In step S12, the screen rearrangement buffer 52
rearranges the stored images of the respective frames
located in the <\\ splay ord^r i u such pos i Lions that Lho
intakes are located i E* the order of encodi nq in accordance
20 w i Lh the GOP structure, and supplJ es the rearranged
.images to the split unit 53,
[0098]
In step S13, the split unit 53 splits eaoh of the
images supplE ed by the soreon rearrangement bufI or 52
2b into N ti.les based on Lho tile spl i t informal- i on. The
split uni I. YS supplies each of the images conl.a ining the
N tiles f.o the corresponding encoding units 54-1 through
54-N as an image of an encoding unit.
[00991
JO Tti stop S14, Lho encoding units 54 perform an
enood i ruj process whieh compress i ou-oncodes Lho images of
31
SP345665WO00
the corresponding tiles supplied by the split un I L 53
independently in the time direct. I on by a system
corresponding to HEVC system. The details of the
enoodi nq process will be described wi Ut reference t.o Figs.
5 13 and 14 discussed befow,
[01001
Tn step SI5, Lhe setting unit. 55 synthesizes
encoded data of Lho respective tiles supplied by the
encoding units 54-1 through 54-N based on the 1:1 Le split
10 in!omation.
10101]
In step S16, the setting unit 55 sets the tile
splittable information of VJJI to 1. In slop IS17, the
setting unit 5b sets the deblock filter i n r c-t.mation oi
lb SPS and PPS to 0. In addition, the setting unit 55 sets
information on SPS, P£il, VUI, APS and the Like other than
the tile splil.Lablo information based on Uie trie split.
information and the like.
[0102]
20 At this time, the setting unit 55 sets
sao repeat rowf lctg and sao_meryo_up flag contained in
APS and indicating whether SAO processing is performed
using parameters of SAO processing of Uie adjacent: image
to false (0) when the adjacent image is an image of a
25 di f f eren t tile. Morfiovt-t, the set Li no, unxt 55 sets
al Irepeatrow flag and alf_merge jj[j_lVlag contained in
APS and indicating whether ALF processing is per i or.mcd
using parameters oi ALLL processing of the adjaoonU image
f.o false {0) when (.he adjacent Image xs an image of a
30 different tile. Tn this case, parameters for SAO
processing and parameters for AT,F processing are not
3?
SP345665WO00
shared botween different tiles, According^ y, filterE rig
is performed for each unit of tiles for encoding.
[010-ij
As discussed above, sao repeat row ("lag,
5 sa to the
decoding device described later, and terminates the
20 process.
[0106]
(Descr i pti.on of Process Performed by Encoding Device)
figs. 13 and 14 are a l\'\ owe hart describing step SI 4
of the encoding process in Fig. 12. This encoding
25 process is performed for each unit of CDs, for exarrip J e.
[0107]
in step SJ0, the intra prediction unit 32 performs
an intra prediction process for performing intra
prediction (of all possible intra prediction modes by
30 using an imaqe stored .in the DPB B0f located in the same
tile as the t tie of the encoding ta rgct image, and not
33
SPr^!}665WO00
filtered as a reference Linage. At Ul i.s time, I:ho intra
prediction unit 32 calcu I ates cost ("unction vat uos for
all the possible intra prediction modes based on the
encoding target image supplied by Uie split imlt 53 and a
b prediction i mage generated as a result of the 3 ntra
prediction. Then, the intra prediction unit £2
determines the intra prediction mode where the? cost
function value becomes the minimum as the optimum intra
prediction mode, and supplies a prediction .i mage
10 generated in the op I. i mum intra prediction mode and the
corresponding cost function value to the prediction Image
selection unit Bt.
[O10GJ
In addition, the inter prediction unit 83 performs
15 motion prediction and Giotion compensation within a tile
for all the possible inter prediction modes by using a
f i I torcd image stored in the DPii 80, and Located in the
same tile as the? tile of the encoding target image rts a
reference image. At this time, the inter prediction unit
20 83 calculates cost fund: i Oil values for all the possible
inter predicti on modes based on the encoding tarqet image
supplied by the split uni t b'S, and a prediction imago
generated as a result of" the motion cempensat E on. Then,
tho inter prediction uni t 83 determi nes the inter
2t* predicti on mode where the cost f tine tion valuer bo comes
the mini rmjili as the optimum inter prediction niedo, and
supplies a prediction image generated in tho optimum
inter prediction mode and the corecspondinq cost function
value to the predi ct.i on image selection un ft 84 .
30 f 0 "I 09J
In s t e p S31 , the predi i: I. i on image siock filter 79 supplies the
decoded image obtained as a result of the I i Itering for
each unit ot tiles to the DPR 30.
30 [0124]
In step S46, the DPR 80 stores the decoded images
SP345665WOG0
Cor each unit of tiles before and after the filtering.
More specif iea IE y, the DPB 80 stores the decoded images
for each unii: <<(: tiles suppii.ed by the addition unit /tf
and the decoded images for each unit or l.ilos supplied by
5 the deblock f"i Iter 79. The decoded images for each unit
of tiles stored in the DPR 80 are output.ted via the
switch Gl to Lhe intra prediction unil. 82 or the Enter
prediction unit 83 as reference images, Then, the
process returns to step S1.4 in Fig. 12, and proceeds to
1.0 step SIB.
[0126]
Further, according to the encoding process in l|Lig.
"13 and Fig. 14, both the intra pred Lotion, and motion
predi o t i.on and moti on compensation are always performed
15 for ai mplification of the descriptfon. However, in
practical cases, onty either of these processes may be
perf onned depending on picture types or other conditions.
[.01?7J
As discussed herein, the encoding devi ee 50
20 pecforms motion prediction within a tile, and generates a
motion vector by us i IKJ an encod I ng target, image and a
re Terence image at a time dii {erent from tin? time of the
encoding target image. Accordingly, independent encod i.ng
i i] the time di rection for each tile is all owed,
25 [0128]
Further, while the encoding device 50 is provided
with the N encoding units 54 for encoding images of.'
respective tiles, the encoding device 50 may be provided
with only one encoding unit. In this case, the encoding
30 unit has a DP!3 storing a decoded imatjc for each tile, and
encodes Linages per Lile in the order of the I. i I e ID
39
^345665^000
number in t:he d i r e c t i o n from smaller number to l a r g er
number, that _is, the order oC r a s t e r scan.
[0129]
(Constitution example of Decoding Device i n First
5 Embodiment:)
Fi q, 15 is a bock d iagram showing a constitution
example of a decoding dev i ce to which the present
teohnique is applied according to the first embodiment.
This decoding device decodes an encoded stream
10 t ran sin i tted from the encoding device 50 in Ij'ig. 3.
(01301
A decoding device 90 in Fig. 15 Is constituted by a
reception unit 91, an extraction unit 92, a split unit 93,
decoding units 94-1 through 94-N, a screen rearrangement
15 buffer 95, and a D/A conversion unit 9G.
[0131]
The reception unit 91 of the decoding device 90
receives an enooded stream transmuted from the encoding
device 50, and supplies the encoded stream to the
?.0 extraction un it 92 .
[0132]
The e x t r a c t i o n unit 92 e x t r a c t s ttl?S, PPS, VUl, APS,
encoded data and the l i k e I rem the encoded stream, and
s u p p l i e s the e x t r a c t i o n to the s p l i t unit 9.3, in
25 addition, the e x t r a c t i o n imit 92 supplies t i l e s p l it
information contained in SPS and PFS to the screen
rearrangement buffer 9b-
[013.31
The split unit 93 splits the encoded data into
30 units of tiles based on tile splitLabie information
contained in VUT supplied by the extraction unit 9^, and
40
SP345665WO00
the tile split information contained in SPS and PPS. The
split unit 93 supplies the encoded data of N tiles
obtained as a result, of: Lhe spl.iL Lo the decoding units
94-1 through 94-N for each (.. i I o, Jn addition, the split
5 unit 93 supplies SPS, PPS, APS and the liko supplied by
the extraction imi I. 92 Lo Lhe decoding unit 94-N.
[0134]
fiach of the decoding units 94-1 through 94-N
decodes encoded data of the corresponding tile supplied
10 by the ftp I i L un.il. 93 by a system corresponding to EEVC
system while referring to SPS, PPS, APS and the I ike
supplied by the split unit 93. in other words, the
decoding units 94-1 through 94-N decode the encoded daLa
independently in the time direction for each l:i le wh i I e
15 referring to 3PS, PPS, APS and the like. The decoding
units 94-1 through 94-N supply the decoded images
obtained as a result of the decoding to the screen
rearrangement buffer 95. In the follow! ng descrl pLi on,
the decoding units 94-1 through 94-N are collecLi.veEy
20 referred to as decoding uni l.s 94 when di sLinetion
therebetween is not particularly reqtji r^;t.i.
[0135]
The screen rearrangement buffer 95 synthesizes the
decoded images of the respecl-ivo Li les supplied by the
25 decoding nni.l.s 94-1 Lhrough 94-N by arranging the
re spec Li ve decoded i maries and storing the respective
decoded images for each unit of frames based on the tile
sp[ i L in i'n riiia Lion supplied by the extraction unit 92 .
The scj.een cearrangemont buffer 95 rearranges the s t.ort?d
30 i mages for each unit of frames located in the order oI
encoding in such positions that the respeol.i ve images arc
41
SP345665KO00
located in the order of the original di.sp lay, and
supplies the rearranged imago s to the D/A con vers i on unit.
[0136]
b The D/R conversion unit 96 performs D/A convorsi on
of- the images for each unit of frames supplied by the
screen rearrangement buffer 95, and supplies the
converted image as output signals.
[013/]
10 (Corls t i t.uLiovt example of Decoding Unit)
Fig. 16 is a block diagram showing a constitution
exauip I o of the decoding units 94 in fig. 15 according to
the first embodiment.
[0133]
15 The decod.i rig unil. 94 in Fig. 16 is constituted by a
store buffer 101, a lossless decoding unit 102, an
inverse quantize I i.orl un it 103, an inverse orthogonal
transformation uni.L 104, an addition unit 105, a deblock
filter 106, a DPB 1.0V, a switch 10B, an intra prediction
20 unit 109, a motion compensate on un i t 110, and a swi. [".chill.
[0139]
The store buffer 101 of the decoding un it 94
receives encoded data of the cor respond i ruj Li le siippl i ed
?b by the split unit 93 in Fig. 15, and stores the received
data. The store buffer 101 supplies the stored encoded
data t.o Lhe lossless decoding unit 102.
[0140|
Tho Iossless decoding unit 102 performs lossless
30 decodinq, nvc.lt as variable codeword length decoding and
arithmet i.e decod i nq, J or the encoded data received from
•12
SF345665WO0O
the store buffer 101 to obtain quantized coefficients and
encoded information. The lossless decoding unit 102
sopplies the quantized coefficients to the inverse
quantization unit 103.
b [0141]
Tn addition, the lossless decoding unit 102 obtains
i Fitra prediction mode information of the current
prediction block by adding intra prediction mode
inf ormati on a t. Ler di l; lie rent ial encoding as encoded
10 inf oniiatl on Lo i n tra predi ction mode information of a
prediction block adjacent to Lho current prediction block
within Lho sarne Li I e* The lossless decoding unit 102
supplies tho current intra prediction mode in formation
and the like to the intra prod Loti on un i L 109.
15 [0142]
Moreover, the lossless deeodi itg uili t 102 functions
as a motion vector generation unit, and calculates a
motion vector of the current prediction block by adding
motion vector information as encoded information to a
20 motion vector of another prediction block within the same
tile. The lossless decoding unit 102 supplies tho
obtained motion vector, information for specifying a
reference image as encoded information, inter prediction
mode information and the like to the motion compensation
2b un it 110, Furthermore, the lossless decoding unit 102
supfj f i os intra predi oLion mode i ii format ion or inter
prodicti on mod*-? i n I ormaLi on Lo the switch 111.
[0143]
Tho inverse quanLi^aLi on unlL 103, Lhe inverse
30 orthogonal trans formal: ' on tin it 1 04, tho add i Li on unit 1 0b,
the deblock filter 106, lho l)fc>R 10/, Lho switch 10W, (.ho
43
SPU4566bWO00
i ntra prediction unit 109, and the motion compensation
i.tn i |. 110 perform operations similar to the corresponding
operations of the inverse quantization unit 7 6, the
i Averse orthogonal transformation unit 77, the addition
5 unit. 78, the deblock filter IB, the Upn SO, the switch 91,
the intra prediction unit 02, and the motion compensation
unit 83 in Fig. A. The images are decoded by these
operations.
[0144]
10 More spec! F" i oal.l y, the inverse quantization unit
103 performs inverse quanLi *a ti on of the quantized
coefficients supplied by the lossless decoding unit 102,
and supplies the coef Ei ei ents obtained as a result of the
inverse quantization to the i nverse orthoqonal
15 transformation unit 104.
E0145]
The inverse orthogonal trans f."o rma t ion unit 104
performs inverse orthogonal trans format ist:r i pL i on ot Out t ine of Process Performed by Decoding
30 Dev.ice)
F\ i|, 17 is a (ti aqram d e s c r i b i n g the o u t l i n e of a
46
SP345665WO0G
process performed by the deeod i nq devi oe 90 in Fig. 1T>.
[0153]
As shown in L'ig, 17, an encoded stream divided into
N tiles and encoded is input tod Lo Lhe decoding device 90
5 from the encoding devico 50 - Further, Lhe l.ile
splittable information is Hot to truo (1.) J or (.hi s
encoded stream.
10154]
The decoding device 90 receiver the encoded 3Lream,
10 exl.raots SPS, PP£, VUI, AFS, encoded data and the like
from Lhe encoded stream, and splits the encoded data into
units o£ tiles based on tile split information contained
in SPS and PPS. The encoded data for each tile obtained
by spl.iL is supplied to the corresponding decoding units
15 94-1 through 94-N for each tile. Moire specifically, each
of the encoded da La of tile Jfi, tile #2, and up to tile
#N is supplied Lo Lhe corresponding decoding unit 94-1,
decoding un i I. 94-?, and up to decoding unit 94-N.
[0155]
20 The decoding uniL 94-1 is constituted by a decoding
processing unit 121-1 and a DPB 122-1. The decoding
processing unit 121-1 is consLiLuted by the store buffer
101, the lossless decoding unit 102, |.he i n^erse
quantization unit 103, Lhe I nverse orl.hogonal
2b transformation unit 104, the addition un1L 10b, Lhe
deblock filter 106, the DPU 107, the swi.Lch 108, \.ha
i nLra prediction unit 109, the motion compensaL.i on un i t
110, and the switch ill (Fig. 16) of the decoding un i L
94-1 . The decoding processing unit 121-1 decodes the
30 encoded data of Lhe tile fj-l.
[01561
47
SP345665WO00
Moreover, the DPR 12P-1 is oonsU t.ul:ed by the DPB
107 of the decoding unit 94-1, and s|:ores (:he decoded
image of the tile fll oblai.ned as a result", of. the decoding
by the decoding processing uniL 121-1, The decoded image
5 of the tile #1 stored in the DPR 172-1 is used for
decoding by the decoding process i ng un i I- 121-1-
[0157]
Each of the decoding units 94-2 through 94-N has a
constitution similar to the structure o£ the decoding
10 unit 94-1. Accordingly, the decoded images of the t.i f e
ft2 l.hrough tile «N are stored in the DPiJ 122-2 through
I22-N, respectively.
101581
In addition, the decoded images of the tile #1
l1? Lhrough l:ile JfH obtained by the decoding processing unit
121 -1 L-hrough 121-N are also supplied to the screen
rearrangement buffer 95, synthesized by arrangement based
on Lhe Lile split, in formation, and stored for each unit
of: frames-
20 [0159]
As di scijMSed above, l.he encoded data for each tile
is decoded i ndependenLIy by using the decoding image of
the corresponding tile- Aeeord i rig I y, thy decoding device
90 is not required Lo Include a common decoding DPB
2!j retaining decoded images for all t.i ley -
[0160]
{Description of Process Performed by decoding Device)
Fig. 18 is a flowchart describing an encoded stream
decoding process performed by the decoding device 90 in
30 Fig. "LS.
10161 1
A8
SP34S66bWQ00
III step S61 in Fjq. 18, the reoepti on Urt i. L 91 of
the deoodt ng devi ce 90 reoei ves an encoded stream
Lransrni tted from the enoodi nq dt^ui ce bO, and aupp[ ies the
encoded stream to the extraction un it 92.
b [016?[
In step Sb2, the extraction uni I 92 extracts SPS,
PPS, VUT, APS, encoded data and the like from the encoded
stream, and supplies the extraction to the split unit 93.
lit addition, the extraction unit 92 supplies tile split
10 information contained in SPS and PPS to the screen
rearrangement buffer 95,
[01G3]
In step IJ63 , the split unit 93 determines whether
tile splittable information contained in VUI supplied by
15 the extraction unit 92 is true [1) or not. When the tile
splittable information is not true {1}, that is, when the
tile splittable information is false {0), the split unit
93 terminates the process.
[0164]
20 On the other hand, when i t i a determi ned that, t he
tile split table information is true (1} in step SEiil, the
split unit 93 splits the encoded data 5 nto units ol I. i I es
based on the tile split information contained in the
split: unit 93, EPS and PPS En step Sf>4.
?S [01bb)
In step 3Gb, the split unit 93 supplies the encoded
data of the respective split N tiles to the corresponding
decoding units 94-1 through 94-N. Moreover, the split
unit 93 supplies SPI?y PPS and the like supplied by the
30 extraction unit 92 to the decoding unit 94-tJ.
[0166]
49
Sf345665WO00
In stop -366, the decoding units 94 perform decoding
of the encoded data of the corresponding tiles supplied
by the split unit 93 by a system corresponding to ilKVC
system while referring to JjFS, PPS and the like supplied
5 by the split unit 93, The details of this decoding
process will be described with reference to Fig, 19
discussed later.
[0167]
In step SG7, the screen rearrangement buffer 95
10 synthesizes the decoded images of the respective tiles
auppl3 ed by the decoding units 94-1 through 94-N by
arranging the respective decoding images and storing the
respective decoding images for each unit ot iiramea based
on Die tile spl i t informs t E on soppt led by the extract, i on
lb uni I. 92.
[0l68j
In s t e p S6B, Lhe tf<;reen rearrangement, bu I fer 9b
r-oarranqes the at.ci-od images for each u n i t o!" I ramos
l o c a t o d in l.ho order lor- oncodinq in su<:h p o s i t i o n s that
2 0 the r e s p e c t i v e images arc located in the order of tho
o r i q i n a 1 di sp Lay, and supp i i es tho rearranqed iEitaqos to
the D/A conversion u n i t 90[0169]
in stop S69, the D/A conversion unit 96 performs
25 D/A conversion of the images for each unit of frames
supplied by the screen rearrangement buffer 95, and
outputs the converted images as output signals.
[0170]
Fig. 19 is a flowchart describing the decoding
30 process performed i n step S6E> in Fig. 18.
[01 /-ll
SP34b6GbWO00
Tn step S100 in Fiq, J9, the store buffer 101 of
the decoding un i L 94 ruticivcs encoded da La of the
correspond i nij bile from the split, unit 9J in Fig. 15, and
stores the data- The store buffer 101 supplied the
b encoded data stored therein to the lossless decoding unit
102, Further, the following processes from 3101 to S110
aie per formed for each unit of CUs, for example,
[0172]
in step 5101, the lossless decoding unit 102
10 performs lossless decoding of the encoded data received
from the store buffer 101, and obtains quantized
coefficients and encoded information. The lossless
decoding unit 102 supplies the quantized coefficients to
the inverse quantization unit 103.
15 [0173]
In a d d i t i o n , the l o s s l e s s decoding unit 102 obtains
i n t r a p r e d i c t i o n mode information of the current
p r e d i c t i o n block, by adding i n t r a p r e d i c t i o n mode
information a f t e r d i f f e r e n t i a l enoodi ng as encoded
20 information to i n t r a p r e d i c t i o n mode information of a
p r e d i c t i o n block adjacent to the current, hi ook w i (,h i n the
same t i J e . The l o s s l e s s deoodi fit; unit, 1 OP suppl i es i.n t r a
predi ct I on mode in format Loit of Lhe current p r e d i c t i on
block to the i n t r a p r e d i c t i o n unit 109 and the switch 111.
?.b (0174)
In step SI02, the lossless decoding unit 102
generates a motion vector of the current prediction block
by adding motion vector information DLS encoded
information to a motion vector of another prediction
30 block within the same tile. The lossless decoding unit
102 supplies the generated motion vector, information I or
51
SF345665WO00
specifying a reference image as encoded information,
inter prediction mode information and the like to the
motion compensation unit 110. In addition, the lossless
decoding unit 102 supplies the inter prediction mode
5 information to the switch 111.
[0175]
In step 3103, the inverse quantization unit 103
performs inverse quanti zation of the quantized
coefficients received from the lossless decoding unit 102,
10 and supplies the coefficients obtained as a result of the
Inverse quantisation Lo the inverse orLhoqonat
transformation unit 104.
[01/61
In step ST04, the motion compensat i on unit HO
15 determines whether Lhe inter predioLlon mode i nforrtiation
is auppiied f.'rom Lhe .lossless decoding un i L 102. When it
is dote mill ned thaL (.".he inter predictJ en mode information
is supplied in step 5104, the process proceeds to atop
S105,
20 [0177]
In step 3105, the motion compensation unit 110
performs motion compensation by using a reference image
filtered by the deblock filter 10 6 and contained in the
same tile as the tile of the decoding target image based
25 on the motion vector, the inter prediction mode
information, and the information for specifying a
reference image supplied by the lossless deeodi nq uni I.
10?. The motion compensation uni I: "110 suppl ies a
predi cti on image qeneiviLed as d resuIt of the mo 11en
30 compensat i on to Lhe add i I. i on unit. 1 0b via the aw i ten 111,
and a I 1 ows the oroeosa to proceed to step SI U /.
52
5i>345665WO00
[0178]
On the other hand, when it is determined that the
inter prediction mode information is not supplied in step
Si04, that is, when the intra prediction mode information
b is supplied to the intra prediction unit 109, the process
proceeds to step fllOfi.
[01791
Tn slop Si Ob, the i nl.ra predict! on uni t: 109
performs i nl.ra predi tit ton process which performs intra
10 predi ct i on ol the i nLra prediction mode informal.ion by
using a rcfeconce linage read i'roiit the F)PB 1Q> via lite
sw.i toh 10H, not fit to red by the debt ock [iilLer 106, and
located within the same Lile as the Liie ol" the decoding
target image- The intra prediction unit 109 suppl i.es a
15 prediction image generated ad a result of." the intra
prediction to tho addition unit 105 via the switch 111,
and allows the process to proceed to step 5107.
[0130]
In step Iil07, the inverse orthogonal transformation
20 unit 104 performs inverse orthogonal transformation of
the coefficients received from, the inverse quantization
unit 103, and supplies residual information obtained as a
result of the inverse orthogonal transformation to the
addition unit 105.
2b [01811
Tn s t e p Si 08, i.he addj ti on uni t 105 performs
dei:od i nq by add i nij I.he r e s i dual i nl orrnati on snppl i ed by
I. he f n v o r s e orthotjona t t r a n s I ormal. i on uni L 1 04 as a
decoding t a r g e t i mage to the prod i <:l.l on i rnaqe s u p p l i e d by
30 the switch 111. The a d d . i t i o n unit. 1 0^ s u p p l i e s a decoded
image o b t a i n e d ctd a r e s u l t of l. he deeod Lng Lo the debl oek
53
SP34SG65WO00
f i l t e r 106, and a l s o s u p p l i e s Lhe decoded image to Lhe
DFli 107.
[0132]
In step 5109, tho deblock filLor 100 performs
5 filtering of tho decoded image supp.l i ed by Lho add i L i on
unit 105 for each unit of tiled bayed on deb Look f i I Lor
information contained in SE^S and PPS supplied by tho
split unit 93. The deblock, filter 106 supplies a decoded
image after filtering to the DFIi 107 and the screen
10 rearrangement buffer 95 (fig, 15),
[0133]
In step 3110, the DPB 107 stores the decoded image
before filtering supplied by the addition unit 105, and
Lhe decoded image after filtering supplied by the deblock.
lb f"i 1 Ler lflfi. The decoded image stored in the DPB 107 is
supplied l.o the motion compensation unit 110 or to the
i [lira predi otion unit 109 via the switch 108. Then, the
process returns to step S66 in Fig. 10, and proceeds to
step 3 67.
20 [0184]
As d i seussed shove, the decoding device 90 performs
motion cortipen sa Li on ior each Lite by usirig a reference
image positioned at a time diff etenl. I r-pin Lite L i me of Lho
decoding target .image and coEiLainod within (.he same l.Ue
25 as the tile of the decoding target image based on the
tile splittable information and the motion vector
information. Accordingly, independent decoding rn the
time direction for each tile is allowed. As a result,
the decoding device 90 can reproduce only a predetermined
no t'.ile of the W tiles at a high speed, for example.
[0-IBbl
54
3P345665WO00
Furtherj while the decoding device 90 is provided
with the H decoding units 94 for decoding images of the
respective tiles, the decoding device 90 may be provided
with the one decoding unit 94. In this c&ti&, the
b decoding unit includes a DPS3 storing decoded images for
each l.i le, and performs decoding of images for each ti le
i n Lhe order of ti Le TT) number i n the direction from
smaller number; to larger number, that in, in the order of
raster scan,
10 [0186J

(Example of" Kncoding Target Ima.:\.- image is t i l e - s p ] i I: i nl.o d i £ I e r e n t I. i I e±s i or I. lie T.
i matje and l o r Lhe R. iniriqe, Ay a r e s u l t . , l-he L i l e ior T,
f iliaqe becomes a Li Le ftO, and Lhe Li le Tor Lhe R imago
becomes a LL.le tfl -
'30 [0139]
L'urther, tlio L imago and the K imago of the 3D
55
SP345665WO00
image may be disposed on the upper half and the I owe r
half of the screen, respectively.
[0190]
(Constitution example of Encoding Device in Second
5 Embodime n t)
The encoding device to which the present technique
is applied according to the second embod i men t i s the
encoding device bO which sobs N to 2. This encoding
device independently encodes the T. imago and the R image,
10 and transmits an encoded stream obtained as a result of
the encodi nq*
[0191]
fConst.i tut i on example of Decoding Device for 2D image in
an Kmbod i client j
1 b Fig. 21 is a block diagram showing a constitution
example of a decoding device for a 2D image according to
an embodiment. This device decodes an encoded stream of
a 3D image encoded by the encoding device according to
the second embodiment.
20 [0192]
In the constitution shown in Fig. 21, constitutions
similar to the constitutions in Fig. 15 are given sinti tar
reference numbers. The same explanation i s oinf tted when
appropriate.
25 [0193]
The constitution of: a dei:od i ntj dev i ee 140 in l''ig.
21 is different trom the cons t f tut f on In Fig. 15 in that
a tile extraction unit 141 is provided instead of the
split unit 93, and that a screen rearrangement buffer 142
30 is prov i (fed i na Lead ol. the screen rearrangement buffer 95 .
56
SP345665WO00
The tile oxtiaction unit 141 splits encoded data
into units of tiles based on tile splittable information
contained in VUI supplied by the extraction unit 92, and
tile split information contained in SPIi and FPU. The
5 tile extraction unit 141 supplies encoded data of the
tile #1 included in encoded data of two tiles to the
decoding unit 94-1. It is assumed herein that 2D display
is performed by using the L image. However, the R image
may be used for performing 2D display. In this case, not.
10 the encoded data of the tile #1, but the encoded data of
the tile 12 is supplied to the decoding unit 94-1.
[0195]
The screen rearrangement buffer 143 stores the
decoded image of the tile fll Auppli ed by the deeodi nq
15 unit 94-1 for each unit of frames - The screen
rearrangement buffer 142 rearranges the stored i rnaqes Tor
each unit of frames located i n (.he order I or cncodi ng .> n
such posi Li on a that the irnaqes are I oca Led i n Use order.
of the original display, and supplies the rearranged
20 images Lo the D/A conversion unit 96.
101961
{Description of" Process Performed by Decoding Device for
2D image)
F.i q - 22 is a flowchart" describing an encoded stream
25 decoding process performed by the decoding device 140 in
i'ig. 21.
[0197]
The processing performed from steps S131 through
Si34 in Fig. 22 is similar to the corresponding
30 processing from step S61 through S£i4 in Fig. IS;
therefore, the explanation oi these steps i s omi I.Led,
57
SP345GG5WO00
[0198]
I n s t e p S135, the t i l e e x t r a c t i o n u n i t 141 s u p p l i es
t h e encoded d a t a of the t i l e fl i n c l u d e d in the encoded
d a t a of t h e s p l i t two t i l e s to the decoding u n i t 9 4 - 1,
5 In s t e p S136, the dece-diny u n i t 94-1 performs decoding Ln
li'ig. 19.
[0199]
In step 13137, the decoded image of the tile #1
supplied by the decoding unit 94-1 is stored for each
10 unit of frames.
[0200]
The processing in steps S138 and SI39 is similar to
the processing in step E6S and 369. in Fig. 13; therefore,
the explanation of these steps is omitted.
15 [0201]
As discussed above, when the encoded stream is an
encoded stream tile-split to provide different L image
and TC image and encoded, independent encoding of the L
i mage and the R image is allowed. Accordingly, the
20 decoding device 140 can decode only Lhe encoded {lata of
the T, i mage of the l.i le ft"L i neluded in the decoding
target encoded data« As a reaut I., h.i qh-apeed
reproduction of a 2D image J s res E i./.ed. Trt add i. t ton, the
decoding devi.ee 140 a I I ows reduction of L.ht: capacity ol
25 the DP13, and reduction of power consumption at the time
of decoding.
[0202]
liiinilarly, when the encoded stream is an encoded
stream tile-split into a central area within the screen
30 and into the other area, high-speed reproduction of only
the central area to which attention is given is allowed.
SP345665WO00
[0203]
{Constitution example of. r>ei:oding Device for 3D Image)
A decoding devi cu lor 3D image shown in Fig. 20 for
decoding a 3D image encoded a I; ream i a a decoding device
5 which sets N to 2 in Fig. lb. Th E.a 3D image decoding
device obtains a 3D image by independent: 1 y decoding
encoded data for an L image and for ^n R i mage, and
synthesizing the decoded data, further, the doeod.i ng
device for 3D image may bo constituted to output the T.
1 0 image and the R image obtained as a (caul L of' the
deoodi.ng without synthesizing these images,
[0204]
In addition, while each of the L image and the R
.image la split into one tile according to the second
15 embodiment, each of these images may be split into a
plurality of tiles. In other words, tiles may be split
in any ways as long as the tiles are so split as not to
conta i n both the L image and R image.
[0205]
20
(Constitution example of Television Conference System)
Fig. 23 Is a block diagram showing a constitution
example of a television con Terence system to which the
present technique i a apfj I i ed according to an embodiment.
25 [02061
A television conference system 1 GO in Fig- 23 1 a
constituted by imaging devices 161L-1 throuyh 1 61 -M,
encoding devices 162-1 through 162-M, a synthes.i v. i ng
devt ce 163, decoding devices 164-1 through 1G4--M, and
30 d i splay devices 165-1 through 165-M. The television
r.on I erence system 160 captures images of tA participants
59
SP345665WO00
of a conference positioned at different locations,
encodes and synthesizes the images, and decodes and
displays the images.
[0207]
5 More specifically, the imaging devices 161-1
through 161-M of the television conference system 160 are
positioned at the respective locations of the M
participants of the conference. The imaging devices Ifil-
1 through 161-M capture images of the corresponding
10 participants of the conference, and supply the images to
the encoding devices 162-1 through 162-M.
[0208]
Each of the encoding dev.i oes 162-1 through 1 62-M
has a constitution simit ar to l.he struebure of Lhe
IS encoding device 50 in Ft g* 3 , The encodi nq dev i CO a 162-1
through I62-M compression-encode the images supp i. i.ed by
the imaging devices 1 61 i n depend en 1.1 y for each bi lo by a
sysbem correspond i tig bo HF.VC syMberiu F.aeh of" the
encoding devices 163-1 bhromjh 162-M supplies an oneedod
20 sbream obbainod as a result of the compression encoding
Lo bho synl.hes i •/. i nq dev i co 163 .
[0209J
Tho synthesizing device 163 receives the encoded
stroams transitutted from tho encoding devices 162-1
25 through 162-M, The synthesizing device 163 synthesizes
each of encoded data contained in the synthesized streams
as encoded data of different tiles. The synthesizing
device 163 generates tile split information indical.E ng
the positions of the encoded data of the respecti ve tiles
30 and showing K as the number of splits from the encoded
data obtained as a result ot" the synbhesi s. Tin;
60
SPJ45665WO00
synthes i y. i nq dev ice 1.63 sots SPS containing tile split
i nformaU on and dcbJook filter in formation sot to false
(0}. Moreover, Ule synthesizing device 163 sots VUI
con ta i.n i nq mold on restriction information set to false
5 (0}, PPS containing deblock filter information set to
false (0) , and APS. The synthesizing device 163
generates a synthesized stream by adding SPS, PPS, VUI,
APS and the like to the encoded data obtained as a result.
of the synthesis. The synthesizing device 163 transmits
10 the synthesized stream to the decoding device 164-1
through 164-K,
[0210]
Each of the decoding devices 164-1 through 164-M
has a constitution similar to the structure of the
15 decoding device 90 in Fig. 15. Each of the decoding
devices 164-1 through 164-M receives the syni.hesi ^ed
stream transmitted from the synthesizing devi oe 163.
Each of the decoding devices 16*1-1 throuqh 1 64-M
independently decodes the synthesized stream f or each
20 tile, and supplies l.he decoded i maqe obtained as a result
of the decoding to the corresponding one oJ Lhe dispJ ay
devices 165-1 through "I 6.^-M.
(0211j
The respective display devices 165--1 through 165-M
25 are disposed at the corresponding locations of the M
participants of the conference. The display devices 165-
1 through 165-M display the decoded images supplied by
the decoding devices 164-1 through 164-M.
[0212}
30 further^ according to the television conference
system 160, the display devices 165-1 through "L6h-M are
61
SP345665WO00
disposed al. the cor respond tnq locations of the M
participants o\~ l.he con I erence. However, the display
devices may be d i sposed at L coations of a part of the M
participants ol" Lho con Terence.. Tn addition, the decoded
5 images may bo dispEayed on the display devices of the
persons not parLi o I pal i rig in Lhe conference.
[0213]
As discussed above, according Lo the to I evis i on
conference system 100, the encoding devioe t62-1 through
10 162-M perform encoding independently Cor each Li I.e.
Accordingly, the motion vector in inter prediction always
becomes a vector designating an image wi.Lhi.ri a Li le
containing a prediction block as a reference imago.
[0211]
15 In this case, even when encoded data contained in
encoded bit-streams supplied by the encoding devices 162-
t through 162-M is synthesized by the synthesizing device
163 as it is as a part of encoded data of one screen, a
decoded image corresponding to the encoded data of a
20 dii.l erenL encoding device after synthesis of the encoded
data is not referred to at the time of decoding. In this
case, normal decoding of the encoded data after synthesis
allowed. Accord1nqIy, Lhe synthesizing device 163 can
eas.l ly synthesize encoded bit-streams supplied by the
25 encoding devices 1.62-1 through 162-M without changing
layers of Video Coding Layer (VCI.) and lower Eayers.
[0215]
This point is particularly advantageous Tor a
television conference system where the number of encoded
30 bit-streams to be synthesized is dynamically changeable
by addition of a new participant of the conference, or
62
yP345665WO00
removal of a participant of the conference in the middle
of the conference.
[0216]
In addition, a decoded image of different encoded
5 data is not referred to for each of the M encoded data
contained in the synthesized stream. Accordingly, the
synthesized stream can be again split into encoded
streams conta i ni ng the respectj ve encoded data . As a
resut I'.f processing associated with the synthesized stream
10 f tf easi l.y performed.
[0217]
Further, according to the foregoing d e s c r i p t i o n,
encoding and decodi EIreeLi orO is sot lor each
63
UF345665WO00
line in VUI as shown in lines 5 through 7 in Fig. 24.
[0221]
In this case, encoding and decoding can be
performed only for a predetermined tile for each unit of
b t".i J es contained in the tiles constituting a picture. For
example, assuming that the number of tiles is A, with the
Lile spit I. table informal".! on of the tile IJ1 set to true
[1} and wi.Lh Lhe Lilt? ypiiLLabl.e i nE orinaL i on of" Lhe Li fes
#2 through if 4 sel: l.o la \ se (0) , \ ndependenL decoding oJ:
10 only Lhe Lile ft! is allowed*
[0222]
Further, when b i LsLruam_resL.r i cl. i on_£ I ag lor It ne 1
is 0, the decoding aide recognizes that all the tiles arc
not decodablo for each tilo based on the dototmination
15 that the tilo splittablo information for all the tiles is
set to false (0).
[0223]

[Applicability to Multi-view Image Encoding and Multi-
20 view Image Decoding)
A series of processes described above are
applicable to multi-view image encoding and multi-view
image decoding. Fig. 25 shows an example of multi-view
image encoding system.
AH shown in Fig. 2h, a rnuE I: i -vi ew i niaqe con La i ns
images (>l a p I ij ra I i Ly Ol v i GWK { v i ews) . The pi uraI v i t:ws
of. I ho [iivi I I. i -v iow imafjo ^ r{' COEISI. i Lul.ed by base v I ews Lor.
encoding and deeod * nq us i nq on I y i maqey of Lhei r own
30 views without usi.nq ullages ot" oLher v.i ewsf and rionj-base
views for encoding and decoding using imayes of other
&4
3P345665WO0G
v i ews , The non-base views may use images of base views,
or may use images of other non-base views.
I0?2b]
For enoodi nq and decoding a mul I: i -view image as
5 shown .i n Fl g , 2b, i mages ol the re spec l:i ve vi ews are
encoded <>r decoded* In this case, Lhe methods in the
first through third enibodi meats described above may bo
applied to encoding and decoding of tho respective views,
When these methods are ajjp.I Led, iiidopendent encoding and
10 decoding in the time direction for each tile is allowed.
[0226]
Moreover, the flags and parameters used in tho
methods according to the first through third embodiments
described above may be shared in encoding and decoding
15 the respective views. More specifically! syntax elements
or the like of UPS, PPI3, VUI, and AP^ may be shared in
encoding and decoding of the respective views, for
example. Needless to say, necessary information other
than these may be shared in encoding and decoding of the
20 respective views.
[0227]
When these are shared, suppression of redundant
i n I ormati on i.ransmi ssi on, and reduction ol the amount ol
i nformation (amount of codes) to be transmitted are
25 achieved (thai, is, lowering Of encoding ylfiuicncy is
suppressed)•
[0228]
(Multi-view Imago E'lneoding Device)
l^'ig, 2 6 is a diagram showing a multi--viow image
30 encoding device which performs the foregoing multi-view
image encoding. As shown in L'ig. 26, a multi-view image
65
SP345665WOQ0
encoding device 600 includes an enoodi ng un i I: 601 , an
encoding unit 602, and a multipl exinq unit 603 ,
(0229|
The encoding tin it: 601 encodes base v I ew i mages, and
5 generates a base vi ew imaqe encoded ftL.ream* The encoding
unit 602 encodes non-base vi ew imaqeft, and generates a
non-base view imaqe encoded stream. The multiplexing
uni t 603 niu tLiplexes tho base view imago encoded stream
qeneraLed by the encoding unLL 601 and tho non-base view
10 image encoded a I. ream qeneraLed by the encoding unit 602,
and generates ei multi-view image encoded stream.
[Q23G]
The oneoding device 50 (li'ig. 3) , and the encoding
devices 162-1 through 162-M {trig. 23) are applicable to
15 the encoding unit 601 and the encoding unit 602 of this
multi-view image encoding device 600. In other words, in
encoding the respective views, independent encoding in
the time direction for each tile is allowed. Moreover,
the encoding unit 601 and the encoding unit 602 can
20 perform encoding using the same flags and parameters
(e.g., syntax elements associated with processing between
images) {that is, flags and parameters can be shaded
between the encoding units 601 and 60? ) . Aeoordi ruj I v,
lowering of the encoding efficiency can be suppressed.
2S [0231]
(Mul Li - v i e w Trrirtqe Decod i t\<\ Tii;v i CO)
Fig* 7'i i ft a di aqraiii showing a multi-view decoding
dev i oc per J ormi iiq Uie foregoing multi-view image decoding.
As shown in F'nj, 2 1, a multi-view imago decoding device
SO 610 has an I nverse multiplexing unit 611, a decoding unit
61^i and a decoding unit 613.
66
UF345665WO00
[0232]
The inverse multiplexing unit 611 performs inverse
multiplex!nq of the muU.i-view image encoded stream
generated by mulliplexsnq of the base view image encoded
5 stream and the non-base view image encoded stream, and
extracts the base view image encoded stream and the nonbase
view image encoded streari;. The decoding unit 612
decodes the base view image encoded stream extracted by
the inverse multiplexing unit 611, and obtains base view
10 images. The decoding unit 613 decodes (.he non-base view
linage encoded stream extracted by the inverse
multiplexing unit 611, and obtains Don-base view images.
[0233]
The decoding device 90 (fig* 15}, and the decoding
1.5 devioe 140 {Fig. 21) or the decoding devices 164-1
through 164-M [Fig. 23) are applicable to the decoding
unit 612 and the decoding unit 613 of this multi-view
Imaqe decoding device 610. In other words, in decoding
the respective views, independent decoding in the time
20 direction for each tile is allowed. Moreover, the
decoding unit 612 and the decoding unit 613 can perform
decoding using the same flags and parameters {e.g.,
syntax elements associated with processing between
images) (that is, flags and parameters can be shared
25 between the decoding units 612 and 613). Accordingly,
lowering of the encod i nnq to the first through third
69
SP345665WO00
embodimenta are applicable to encoding and decoding of
the respective layers. When these methods are applied,
independent encoding and decoding in the time direction
for each tile is allowed.
5 [G240J
Moreover, the flags and parameters used in the
methods according to the first through third embodiments
described above may be shared in encoding and decoding oi
the respective layers. More specifically, syntax
10 elements or the like of SPS, PPS, VUI, and APS may be
shared in encoding and deoodi nq of the respect i ve Iayor^,
for example. Needless to say, necessary informal".! on
other than these may be shared i n encodi nq end decoding
of. the respecti ve 1 ayers .
lb [0241 1
When these are shared, suppression of redundant
in formal, ion transmi ss i on, and reduction of." the acnount of
informal.lort (amount of codes) lo be transmitted can be
achieved (that is, lowering of encoding efficiency can bo
20 suppressed).
(Scalable Parameter)
According to these hierarchical image encoding and
hierarchical image decoding {scalable encoding and
25 scalable decoding), the parameter having scalability
{scalability) function is an arbitrary parameter. For
example, the spatial resolution shown in Fig. 2 9 may he
determined as the parameter {spatial sea lab it il:y) . In
case of the spatial scalability {spatial sealabi I il.y},
30 the resolution of L.he image i s var iahl e ( or each I ayer.
More specifica11y, 3 n th i s case, each pi chu re is d i v i ded
70
3P345665WO0Q
into two types of hierarchies of base layers having lower
spatial resolution than the resolution of the original
image, and enhancement layers obtaining the original
spati al resolution when synthesized with the base layers
b as shown in Fig. 23. Needless to say, this number of
hierarchies i s an example, and the number ol: hi erarchi es
may be an arbitrary number.
E0243]
Alternati ve \ y, the parameter hav.inq this
10 scalability may bo temporal resolution (temporal
scalability} as shown iit fc'ig. 30, far example. fit case
of this temporal sua lab.l L i by (tempora I scalability), the
frame rate is variable for each layer. More specifically,
in this case, each picture is divided into two types of
15 hierarchies of base layers having a lower frame rate than
the frame rate of the original image, and enhancement
layers obtaining the original frame rate when synthesized
with the base layers as shown in Ij'ig, 30. Heedless to
say, this number of hierarchies is an example, and the
2 0 number of hierarchies may be an arbitrary number.
[0244]
Moreover, the parameter having this scalability may
be a signal to noise ratio (Signal to Noise rat.io (SNRJ )
{SNR scalability) , for example. In case of this SNR
2 b seal abi It ty, the SNR ratio is var i abl e for each layer .
More speci licially, in thi s <:rtse, each pi eture is d i v I ded
into two types of hierrtreliie^ of X><%^<\ layers having a
lower SNR than tile SNR of the original image, and
enhancement layers obta i n i nq the or.lqinal SNH when
30 synthesized with the base layers as shown in fig, 31 .
Needless to say, this number of hierarchies is an example,
71
and the number <>f" hierarchies may be an. arbitrary number.
[0245]
Obviously, the parameter having scalability may be
a parameter other than the foregoing parameters. For
5 example, the parameter having scalability may be a bit
depth {bit-depth scarabs 1 i ty) . In case of this bit-depth
scalability, the bit depth is variable for each layer.
In this case, each of base layers is constituted by an 8-
bit image, for example. An enhancement layer is added to
10 this image so as to obtain a 10-bil. (bit) image.
In addition, the parameter having scalability may
be a chroma format {chroma scalability) . Trl case o(" |:hi s
chroma scalability, the chroma format is variable I or
lb each layer. In this case, each of base layers (base
layers) is constituted by a component imaye hav i ng 4:2:0
lormat., for example. An enhancement layer is added to
this layer so as to obtain a component image having 4:2:2
Tormat.
20 [0247]
(Hi erarchical image Encoding Device)
Fig. 32 shows a hierarchical image encoding device
which perlorms the i oregoing hierarchical image encoding.
As shown in FE q. 32, a hierarchical image encoding device
25 620 includes an encoding mtU: 621, an encoding unit 622,
and a multiplexing unit 1>23.
[024B]
The encoding unit 621 encodes b^se layer .images,
and generates a base layer image encoded stream. The
30 encoding unit 622 encodes non,jbase .1 ayer imaqes, and
generates a non-base layer image encoded stream. Tin;
72
SP345665WO00
multiplexing unit 623 multiplexes the base layer image
encoded stream generated by the encoding unit 621 and the
non-base layer image encoded stream generated by the
encoding unit 622, and generates a hierarchical image
5 encoded stream,
[0249]
The encoding device 50 (Fig. 3), and the encoding
devices 162-1 through 162-M {Fig. 23) are applicable to
the encoding unit 621 and the encoding unit 622 of this
10 hierarchical image encoding device 620. In other words,
in encoding the respective layers, independent enoodi no;
in the time direction for each tile is allowed. Moreover,
the encoding unit 621 and the encoding un i I: 622 can
perform control of f il tering for i.nl.ra predi cl i on and the
15 like using the same flags and parameters [e, q w syrttaK
elements aasooi a ted wi th process! nq between iETiaqes) ( that
is, flags rind parameters can be shared between Lite
encoding units 6?1 arid 6?2) . Accord! itq I y, towering of
the enoodi nq ef'l'ici ccicy eari be suppressed.
20 [0250]
[Hi erarchi ca i image Decodi nq Dev ice)
Fi q. 33 i s a d.iag rani showing a hierarchical image
decoding device which performs the foregoing hierarchical
image decoding. As shown in i-'ig, 33, a hierarchical
25 image decoding device 630 includes an inverse
multiplexing unit 631, a decoding unit 632, and a
decoding unit 633.
[0251]
The i n v e r s e multiplexing u n i t 631 performs i nvorse
30 multiplexing of the hierarohi cat image encoded stream
generated by mul ti plexi nq oi: the base layer i rttaqie encoded
73
SP345665WO00
stream and the non-base layer image encoded stream, and
extracts the base layer image encoded stream and the nonbase
layer image encoded stream. The decoding unit 632
decodes the base layer image encoded stream extracted by
5 the inverse multiplexing unit 631, and obtains base layer
images. The decoding unit 633 decodes the non-base layer
.image encoded stream extracted by the inverse
multiplexing uiiiL 631, and obtains non-base layer images.
10 The decodi ng devi oe 90 {Fi g. 15), the decoding
device 140 (Fig. 21}, or the decoding devices 164-J
through 164-M (Fig- 23) are applicable to the decoding
unit 632 and the decoding unit 633 ol" th i.s hi.orarchi.oal
image decoding device 630. In ether words, in decoding
15 the respective layers, independent decoding in the time
direction for each tile is allowed- Moreover, the
decoding unit 612 and the decoding unit 613 can perform
decoding using the same flags and parameters (o,g.,
syntax elements associated with processing between
20 images) (that is, flags and parameters can be shared
between the decoding units 612 and 613) . Accordingly,
lowering of the encoding efficiency can be suppressed,
[0253[

2 b {Descriptton of Computer to Which This Technique Ts
Appt i ed)
A seri es ot the Ioregoi ng processes may be executed
by hardware, ot" m^y bo executed by sot I.ware. When the
series ol Lhe processes &n.-*. executed by software, a
30 proguacii coils L.i tuL i ng Lhe sof Lware is i lis La I [ ed i n a
computer. Examples ot" thi s computer, i no I ude a computer
74
SPn4566SWO00
incorporated in dedicatod hardware, and a yonoral-purpose
computer or the liko capable of executiny various types
of functions under various typos of prog raids installed
therein,
5 [0254]
Fig. 34 is a block diagram showiny a constitution
example of hardware of a computer executing tho series of
the foregoing processes under a program.
[02551
10 In the computer! a Central Processing Unit [CPU)
601, a Read Only Memory (ROM) 802, and a Random Access
Memory (RAM) 803 are connected with each other via a bus
£04.
[02^61
15 An input/output interface 005 is further connected
with the bus BOA. An i nput unit G06, an output unit 8 07,
a storaye uni E. 008, a communication unit 809, and a drive
810 are connected wi l:h the input/output interface 805.
[0257]
20 '1'he input unit 300 is constituted by a keyboard, a
mouse, a itrl cropsone and the f i ke* The output uni 1: 807 is
constituted by a display, a speaker and the like. The
storage unit 808 is constituted by a hard di.sk, a nonvolatile
memory and tho J i ke . The coroniuri i cat i on un it. 809
2 5 is constituted by a network, interface and tho like, Tho
drive 810 drives a removable medium 811 in the form of a
magnetic disk, an optical disk; a magneto-optical disk, a
semiconductor memory or the like.
(o^sat
30 According to the computer thus constituted, the
sori es of the foregoing processes are performed by the
75
CPU 301 which loads the program stored in the storage
unit 000 to the RAM 303 via the input/output interface
005 and the bus 304, and executes the program, for
example.
5 [0259]
The program executed by the computer (CPU 801) can
be recorded on the removable medium Sll as a package
medium or the like, and provided in the form of the
removable medium 811, for example. In addition, the
10 program can be provided via a wired or wireless
transmi salon medium such as a local area network, the
Internet, and digital satellite broadcasting.
[0260J
According to the computer, the program can be
15 inatal led into the al.orage unit 808 from the removable
medium 811 attached l.o the drive 810 via the input/output
interf.ace SO 5- AI tenia Li ve \ y, the program can be
received by the communication unit 809 via a wired or
wireless transmits i on med iurn, and inatalt ed i nto the
20 storage unit 303. Instead, the program can be ins taJ I ed
beforehand in the ROM 802 of the storage unit fi08.
[0261]
Further, the program to bo executed by the computer
may be a program under which processes are perfonmed in
25 time series in the order described in this specification,
or performed in parallel, or at the time of necessity
suoh as at the time of accesses.
[0262]

30 (Constitution example of Cellular Phono)
Fig. 36 shows an example of the general structure
79
yP345665WO00
of a cellular phone to which the present technique is
applied. A cellular phone 920 includes a communication
unit 922, an audio codec 923, a camera unit 92 6, ^.n image
processing unit 927, a multiplexing split unit 928, a
5 recording and reproducing unit 929, a display unit 930,
and a control unit 931. These are connected with each
other via a bus 933.
(027b)
An antenna 921 .is. connected vn |:h the commim i cat i.on
1 0 uni t 922, whi .lea speaker 924 and a mi erophone 92 b arc
connected wi th the audio codec 923. Moreover, an
operation uni L 932 is connected with Lhc control. unit 931.
[0276]
The cellular phone 92 0 performs various types of
15 operations such as transmission and reception of audio
signals, transmission and reception of e-mails and image
data, imaging, data recording and the like in various
types of modes including audio communication mode and
data communication mode.
20 [0277]
In the audio communication mode, audio s i g n a ls
generated by the microphone 925 are converted i n t o audio
data and are subjected to data compression by the audi o
codec 923, and supplied to the communication u n i t 922.
25 The communication unit. 922 performs modul.at 5 on, I r^quoncy
transformation and other prooessi nq ol the audi o data,
and generates L.ransmi a s i on si qua I s. In add i Li on, L!ie
communi c a t i o n uni.L. 922 suppE i es l.hc Lrarisrru ss i.on s i g n a ls
L
(Constitubion example of Recording and Reproducing
Device)
fig- 3/ shows an example of the general structure
of a record Lug and reproduoi ng device to which the
15 present technique is applied. A recording and
reproducing device 940 records audio data and video data
of a received btoadcasb proqram on a recording medium,
and provides bhe recorded data to a user at a time
corresponding to instruct! oua of the user, for example.
20 In addition, the recording and reproducing device 940 can
obtain audio data and video da La from another device, and
record these on a recordiEtg medium, lor example.
Furthermore, the recording and reproducing device 940 can
achieve image display and voice output. I rom a mon i Lor i ng
2b device or the like by decoding audio data and video daba
recorded on a recording medium and outputting the result,
[02841
The recording and reproducing device 940 includes a
tuner 941 , an external interface unit 942, an encoder 943,
30 a Hard ni.sk Drive (HDD) unit 944, a disk drive 945, a
selector 94 6, a decoder 04 7, an On-Screen Display (OSD)
33
SP345665WO00
unit 94B, a control unit 949, and a user interface unit
950.
[0205]
The tuner 941 selects a desired channel from
5 broadcast signals received by a not-shown antenna. The
tuner 941 outputs an encoded bit-stream obtained by
demodulating reception signals of the desired channel to
the sc Lector 946 .
[0286E
10 The externa! i nterface uni t 9 A? is constituted by
at least any of an IEEE 1394 interface, a network
Interface unit, a USB interface, a Mash memory interface
and the like. The external interlace unit 942 is an
interface for connection with an external device, a
15 network, a memory card or the like, and receives data to
be recorded such as video data and audio data.
[0237]
The encoder 943 performs encoding by a
predetermined system when the video data and audio data
20 supplied by the external interface unit 942 are not
encoded, and outputs an encoded bit-stream to the
selector 94 6.
[0238]
The KDD unit 944 records contents data such as
2h pi ctures and voice, various types of programs, other data
and the like on a built-in hard disk, and reads these
Iroifi the corresponding hard disk at the time of
reproducLi on, Tor exampte ,
102^9!
3D The disk drive 94b records siifriais on an attached
optical di.sk, and reproduces si'jna I s f roni the optical
$4
SP345665WO00
disk. The optical, disk is a DVD disk [DVD-Video, DVD-RAM,
DVD-KW, DVDiR, DVDiRW, for example) , Til vi-ray (registered
trademark} disk, or the J ike,
[02901
5 The selector 94S selects any of the encoded bitstreams
front the tuner 941 or the encoder 943 at the time
of recording of pictures or voice, and suppl i es the
selected bit-stream to either the !!DD unit 944 or the
disk drive 945. in addition, the selector 94 6 supplies
10 the encoded bit-stream outputtod from the flDD unit 944 or
the disk drive 945 to the decoder 947.
[0291]
The decoder 947 performs decoding of the encoded
bit-stream. The decoder 947 supplies video data
15 generated by decoding to the OliD unit 948. Ln addition,
the decoder 947 outputs audio data generated by decoding.
[0292]
The OSD unit 94G generates video data for
displaying a menu screen associated with selection of
20 iteitts or the 1i ke, superimposes the video data on video
data outputted from the decoder 947, and outputs the
resuIt,
[0293]
The user inter! ace unit 950 i ?, connected with the
25 control unit 949. The user .interface unit 9b0 is
constituted by an operation switch, a remote control
signal receiving unit and the like, and supplies
operation signals corresponding to user operation to the
control unit 949.
30 [0294]
The control unit 949 is constituted by a CPU, a
85
SP."J4b665WO00
memory and the like. The memory stores a program
executed by the CPU and various data necessary for
processing performed by the CPU. The program stored in
the memory is read and executed by the CPU at a
5 predetermined time such as the start of the recording and
reproducing device 940. The CPU controls the respective
parts by executing the program such that the recording
and reproducing device 940 operates in accordance with
user operation.
10 [0295]
Accordinq Lo the recording and reproducing device
thus constituted, the decoder 947 is provided with the
J.uiict Lon of Lhe decodi ng devi ce (decoding method J
•according Lo the presenL appl i.oaLion . According 1 y,
15 indcpendenL decoding in l.he Lime direction for each tile
is a.l lowed.
[0296|

{ConstiLution examp.l o of Jmaq i tig novice)
20 Ii'ig. 38 shows an example of the general structure
of an imaging device Lo which Lhe present LcehnicHie is
applied. An imaging device 960 images an ob ioeL, and
displays an image of the object on a display unit, and
records the image on a recording medium as image data.
25 [0297]
The imaging device 960 includes an optical block.
961, an imaging unit 962, a camera signal processing unit
9 6!?, an image data processing unit 964, a display unit
9 6h, an external interface unit 966, a memory unit 967, a
."JO media drive 968, cin OSD unit 969, and a control unit 970.
Moreover, a u^er i nl.er I 34b66bWO0a
interface unit 966 and the media drivo 968. The imago
data processing unit 964 supplies image data generated by
decoding to the display unit 965, lA.tr thermoro, the image
data processing unit 964 supplies imago data supplied by
5 the camera signal processing unit 963 to the display unit
965. In addition, the image data processing unit 964
superimposes display data received from the 0I3D unit 969
on image data and supplies the result to the display unit
965.
10 [0301]
The GRD unit 969 generates display data such as a
menu screen and icons in the form of symbols, characters,
or Mqures, and outputs the display data to the image
data processing uni L 963.
lb 103031
The exl.ernal i liter Jiaoe un.i I: 966 is consLi Luted by a
USR i rtpuL/ou LpuL terminal and the like, and oonneoLed
with a prifiler al. Lhe Lime oS pr inLi nq o £ images. Tn
addition, a drive is oonnecled wtLh Lhe eKLernal
20 interface unit 966 as necessary. A computer program is
read from a removable medium, .such as a magnetic disk and
an optica.l disk, appropr: Lately aLLaeJied Lo I ho drive, and
the computer program read from tho medium is installed as
necessary. Moreover, tho externa1 interfaee unit 966 has
25 a network interface connected with a predetermined
network such as a LAN and the Internet. The control unit
970 can read encoded data from the memory unit 967 in
accordance with instructions from the user interface unit
9/1, for example, and allow the external interface unit
rs(] 966 to supply the data to another device connected via
Lhe network. Further more, Lhe control uni 1. 970 al lows
SF345665WO00
I.he external interface unit 966 to obtain encoded data
and image data supplied by another device via the network,
and supplies the data to the image data processing unit
964.
b [03031
The recording medium driven by the media drive 968
ia a magnet i o di.sk, a magneto-optical disk, an optical
disk, a serai conductor memory, or other arbitrary readable
and writable removable medium, for example. In addition,
10 tho recording medium may be an arbitrary type of
removable medium, such as a tape devj oe, a d i.sk, and a
memory card, Needless to say, the recording medium may
be a non-contact 1C card or the like.
[0304]
15 Moreover, the media drive 968 and the recording
medium may be unified, and constituted by a non-port"able
recording medium such
[first System)
A specific application example of sea lab l.o encoded
data after scalable encoding {hierarchical encoding) is
10 now described, for example, scalab l.o onood i nq is used
for selection of data to be transmitted as an example
shown in fig. 39,
[0300]
In a data transmission system 1000 shown in fig, 39,
lEi a distribution server 1002 reads scalable encoded data
stored in a scalable encoded data memory unit 1001, and
distributes the data to terminal devices such as a
personal computer 1004, an AV device 1005, a tablet
device 1006, and a cellular phone 1007 via a network. 1003.
20 103091
At this time, the distribution server 1002 selects
and transmits encoded data having an appropriate quality
in accordance w t l.h the capacities, communication
env i ronments and the like of the terminal devices. When
25 the quality of Lho data transmi l.l.ed f rein the distribution
server 1002 is excess i ve I y n i qh, li i qh-quat i l".y imaqes are
not necessarily produced by the Loriiii na I devices. Tn
this condition, there is a poss ibf I i Ly ol do lay or
overflow, and further a possibil i.ty of." unnecessary
30 occupation of the communication bands or unnecessary
increase in loads on the terminal devices . I n contrast",
90
SP345665WO00
when the quality of the data transmitted from the
distribution server 1002 is excessively low, images
having sufficient quality may be difficult to he produced
by the terminal devices. Accordingly, the distribute on
5 server 1002 reads scalable encoded data stored in l.he
scalable encoded data memory unit 1001 as encoded da La
having quality appropriate for the oapaoJ ti es,
environments and the li ke of Lite Lentil na I devices, and
transmits the data appropriately.
10 [0310]
For example, it is assumed that the scalable
encoded data memory unit 1001 stores sea I abl e encoded
data (RL+RTiJ 1011 obLaincd by scalable encoding. The
scalable encoded daLa (RLi-EL] 1011 is encoded data
15 oorit.ain.inq both base fayeivs and enhancement layers, and
produces both base .1 ay or i triages and enhancement layer
irnaqes when decoded.
[0311]
The distrLbution server 1002 selects appropriate
20 layers in accordance with capacities, communication
erlv I i/etiiitcEi ts and the like of the terminal devices to
which data is transmitted, and reads data of the selected
layers. Ij'or example, the distribution server 1003 reads
high-quality scalable encoded data [BL+ELt 1011 from the
25 scalable encoded data memory unit 1001, and l.ransmi Ls the
data as it is to the personal computer 1004 and tile
tablet device 1006 having high processinq abi I ity- On
the other hand, for example, the di st.r i blil-i oil ser.vcr 1002
extracts data of base }ayers Irom Lhe sea lab!e encoded
30 data (BL+ET.} 1011, and transmits I he extracted data as
scalable data (BT,) 101? having the same contents as the
91
SP345665WO00
contents of: the-, scalable encoded dat& (BL-HEL) 1011 but
havjng lower quality than the quality of the scalable
encoded data {RL+RL) 1011 to the AV device 1005 and the
cellu.l ar phone 1007 having lower processing ability.
b [0312]
As can be understood, the amount of data can be
easily controlled bv the use of the scalable encoded data.
Accordingly, the possibility oJ" de I ay and overflow, and
further the possibility oE unnecessary increase in loads
10 on the torciiiEial devices and communication media can be
suppressed. Moreover, in case of the ^ca I abl e encoded
data (BL-iEL) 1011, the redundancy between layers is
reduced; therefore, the amount cf data becomes smaI fer in
comparison with the case when the encoded data of" the
1 5 respective layers is handled as discrete data.
Accordingly, the memory area of the scalable encoded data
memory unit 1001 can be more efficiently ut.ilized.
{0313]
Further, the terminal devices may be various types
20 of" dei red and wire I ess types such
as the Internet and a LocaI Area Network (LAN). Thus,
30 the data transmission capac i ty is vari able. In addi ti on,
variations may be produced by other c>rmiuui i ea t i OHM, lor
92
3F345665WO00
example.
[0314|
Accordingly, the distribution server 1002 may
communicate with the terminal devices corresponding to
5 Uie data transmission destinations before starting data
t ran SIR i ssion so as to obtain information about the
capac i Lies of the terminal devices such as the hardware
performance of the terminal devices, the capacity of the
applieati on (sol Lware) executed by the terminal devices,
10 and informa Li on about the communication environments such
as the usable band range of the network 1003, Then, the
distributf on server 1002 may select the appropriate
layers based on the information thus obtained*
[0315]
15 further, extraction of layers may be performed by
the terminal devices. fc'or example, the persona? eociipu ter
1004 may decode the transmitted scalable encoded data
(BL-f-EL) 1011, and display images of base layers, or
display images of enhancement layers. In addition, the
?0 personal computer 1004 may extract the scalable encoded
data (BL) 1012 of base layers from the transmitted
scalable encoded data {13L+KL) 1011, store the data 1012,
trainsiier the data 1012 to another device, and decode the
data 1012 to display images of base layers, for example.
25 [031 Eil
Needless to say, each number of the s c a l a b le
encoded data memory vini I. 1001, the d i s t r i b u t E on server
1002, (.he network 1003, and the Lermi nal s 5 s an arbi t r a ry
number- In a d d i t i o n , while the example in which the
30 d i s t r i b u t i o n setver 1002 transmits data to (he Lermirirtl
devices has been discussed here f n, appI i c a t i on exampIes
93
SP345665WO00
are not limited to this example. The data transmission
system 1000 is applicable to any system as long as the
system selects appropriate layers in accordance with the
Crtpaci ties of the terminal devices, communication
b envi ronment and the like and transmits the selected
layers when transrnittinq encoded data after scalable
encodJ nq |:o the terminal devices.
[0317]
{Second System)
10 Moreover, scalable encoding is applicable Lo
transmission via a plurality of communication rnedia as in
an example shown in Fig- 40, for example,
[0318]
In a data transmission systom 1100 shown in ti'ig. 40,
15 a broadcasting station 1101 transmits scalable encoded
data (till) 1121 of base layers 1121 through ground wave
broadcast 1111, Moreover, the broadcasting station 1101
transmits (e.g,, transmits in the form of packets)
scalable encoded data [KL) 1122 of enhancement layers via
?0 an arbitrary network. 1112 constituted by a wired,
wireless, or both wired and wireless communication
network.
A terminal device 1102 is provided with the
25 rune I- i on of" receiving the ground wave broadcast 1111
bjoadcrtst.ed by the broadcasting station 1101 , and
receives the scalable encoded data {RT,J 1121 of base
layers tranymi tied via the rj round wave broadcast 1111.
In addition, the Lermi.ua I. dev I ce 1 1 02 i ikrttier has the
30 communication function of" prov i di.mj eorii'uuni.eat ion v.i a the
network 1112, and receives the scalabl c encoded data (t^L)
94
SP345665WO00
1122 of enhancement layers transmitted via the network.
1112.
[0320]
The terminal device 1102 decodes the scalable
5 encoded data (13L) 1121 of base layers obtained via the
ground wave broadcast 1111 in accordance with user
instructions or the like, for example, and obtains images
of base layers, stores the images, and tran^m i ts the
images to another device.
10 [0321]
Moreover, the terminal dev ice 1102 synthes i HCS the
s c a l a b l e encoded data {BL) 1121 of base layers obLainod
via the ground wave broadcast 1111 and the s c a l a b le
encoded data (ET»J 112? obtained via the network 1112 in
1 ^ accordance with user i.nstruoLl oris oc the l i k e , for
example* to obtain s c a l a b l e encoded data (BLiHL), obtains
images of enhancement Layers by decoding of the data,
s t o r e s the images, and transm.its the images to another
dev f co«
20 [03223
As described above, the scalable encoded data can
be transmitted via transmission media different for each
layer, for example. Accordingly, loads can be dispersed,
and the possibility of delay and overflow can be
25 suppressed.
[0323]
Furthermore( the communication medium t.o he used
for transmission may be selected for each layer depend Ing
on situations. For example, t.he seal able encoded data
30 {BT,J 1121 of base layers havi nq a rel a t. i vo I y la rqo amount
ol data may be l.ransrni [".Led via a oommvin i {:rfL Ion medium
9b
SP345G65WO0Q
having a wide band range, while the scalable encoded data
[KL) 1122 of enhancement layers having a relatively small
amount of data may bo transitu l.Led through a oovfimunication
medium having a narrow band range. In addition, for
5 example, the communication rafidium transmitting the
scalable encoded data (RLJ 1122 of enhancement layers may
be switched between the network 111? and the ground wave
broadcast 1111 in accordance with the usab l.e band range
of the network 1112, Needless to say, this applies to
10 data of arbitrary layers,
[0324]
This control can further suppress .i n^rcase i n I oads
.imposed on data transmission,
[0325]
1 b Obviously, the number of layers is an arbitrary
number, and the number of the communication media to bo
used tor transmission is also an arbitrary number,
Moreover, the number of the terminal device 1102 as data
d i 31ri but.ion target is also an arbitrary number.
20 furthermore, whi te the example of broadcasting from the
broadcasting station 1101 has been discussed, application
exaiilpl es are not limited to this example. The data
transmission system 1100 is applicable to an arbitrary
system as Iong as the system splits encoded data after
25 scalable encoding into a pJurality of parts of: layer
units and transmits the data via a plurality of lines.
[0326]
(Third System)
Moreover, scalable encoding is appI I cab Ic to
30 storage of encoded data J,S an example shown in fi IJ - 41 ,
[0327]
96
In an imaging system 1200 shown in Fig. 41, an
imaging device 1201 performs scalable-encoding of image
data obtained by imaging an object 1211, and supplies the
data to a scalable encoded data storage device 1202 as
5 scalable encoded data (I3L+EL) 1221.
[0323]
The scalable encoded data storage device 1202
stores the scalable encoded data (BL+EL) 1221 supplied
from the imaging device 1201 as data having quality in
10 accordance with situations. For example, in the normal
eondi tion, the scalable encoded data storage device 1202
extracts data of base layers from the scalable encoded
data (BTH-F.I,) 1221, and stores the data as scalable
enouded data (BTJ) 1222 of base layers having low qua! 1 l:y
1 b aiid a small amount of data. On the other hand, in the
a t ton t i on condf ti on, for exampl e, the scalabl e encoded
data storage device 1?0? stores the scalable encoded data
(BLiRTr) 12?1 as it is as data havinq hiqh quality and a
Large amount of data,
20 [0329]
By this method, the scalable encoded data storage
device .1^0^ can store i mages tmv i Ely h i.gh quality only as
necessary. Accordingly, this method suppresses increase
in the amount of data while suppressing lowering of
25 values of images caused by deterioration of image quality.
As a result, the utilization efficiency of the memory
area can improve.
10330]
For example, it is assumed herein that the imaging
"*0 devi ce 1201 is a monitoring camera. When a monitoring
Lis r\je |. (such as invader J is not present i n a captured
97
SP345665WO00
image [i.e., in the normal condition) , the possibility
that the contents of the captured image are not important
is high. In this case, reduction of the amount of data
has priority, and the image data (scalable encoded data)
5 is stored as low quality data. On the other hand, when
the monitoring target is present in a captured image as
the object "1211 fi.e. , in the attention condition) , the
poss f bi1iLy that the contents of the captured image are
important is high. Accordingly, the quality of the image
10 hds priority, and the image data (scalable encoded data}
is stored as h i.gh quality data,
[03311
Whether (..he condition is the normat condition or
the attention condition may be determined baaed on
15 analysis oJ the image by the scalable encoded data
storage device 1202- AUernat I ve Ly, the imaging device
1201 may determine the condi.ti.on and transmit the
determination resuLL to the scalable encoded data storage
device 1202.
20 [0332]
Further, the basis foe determination whothei. the
condition is the normal condition or the attention
condition is arbitrarily set, and the contents of the
image corresponding to the basis for determination are
2h arbitrarily established. Heedless to say, conditions
other than the contents of an image may be established as
the basi s for determination. For example, switching may
be made i n accordance with the level of recorded voice,
wave I orrtis or the t i ke, may be made at predetermined Lime
30 in terva I ^i or may be made i n correspondence wj l.h
ins Lnuet i oris I rom the outs 5 de such as nser i nstruct. i OHM .
98
SP345665WO00
[0333]
Koroovor, while the example in which the two
conditions of the normal condition and the attention
condition are switched has been discussed, the number of
5 the conditions is an arbitrary number. For example,
three or more conditions, such as normal condition,
slight attention condition, attention condition, and
extreme attention condition, may be switched. However,
the maximum number of the conditions to be switched
10 depends on the number of layers of scalable encoded data *
[0334]
Furthermore, the imaging device 1201 may deterrni ne
the number of layers of scalable encoding i.n accordance
with conditions . For example, i n the norma I cond i Lion,
15 the imaging device 1201 may generate Lhe sea IabIo encoded
data fBL) 1222 oi: base layers having low qua I i. by and a
small amount of data, and supply Lhe generaLed data to
Lhe scalable encoded dal.a sLorage device VAQ'A. In
addiLion, in Lhe aLLenLion condition, for example, the
?Ai i.mag.i IKJ dev Leo 1201 may generate tho scalable encoded
dal.a (RT.-hF.T.) 1?21 of: base layers having high quality and
a I arge artfouni. of da La, and supply Lhe generated data to
tho scalable encoded data storage device 1202,
[0335]
25 According to the foregoing example, the monitoring
camera has been discussed. However, the purpose of use
of the imaging system 1200 is an arbitrary purpose, and
is not limited to the monitoring camera.
[0336]
30 Further, in this specification, Lhe ays I.em ref ers
t:o a group of plural constj l.uent e t emenLs (dev i cos,
99
SP345665WO00
modules (partst and the like), including both the
structure which contains all the constituent elements E n
the same housing and the structure which contains not al1
the constituent elements in the same housing.
5 Accordingly, a plurality of devices contained in separate
housings and connected with one another via a network,
and a device containing a plurality of rnodu t es wi Lh i n a
housing are both del: \ ned as a ays tern.
(0337]
10 In addition, embodiments accordinq Lo the present
technique are noL limited to the aforementioned
embodiments. Various modifications may bo made without
departing from the scope of the subject matters of the
present lechn iquc.
\b |03JSj
For example, the present technique may have a form
of c I oud eompu tiny which shares and jointly uses one
rune I; Lon between a plurality of devices via a network to
peiform processing.
20 [0339]
Moreover, the respective steps described in
conjunction with the foregoing flowcharts can be executed
by one device, or can be executed jointly with a
plurality of devices.
25 [0310]
Furthermore, when a plurality of processes i s
contained in one step, the p] tirai processes eon t.a i ned in
the one step can be executed by one dev I ce, or- can be
executed jointly with a pIura1 It.y ol devi ces.
:10 [0.1411
Tn add i I. i on, Lhe present Lechn Ujuc can have the
100
SP345G&5WO00
following constitute on a,
[0342]
{1)
A decoding device, including:
5 a motion compensation unit generating a prediction
image by performing, for each of tiles, motion
compensation of. a reference image within a co-located
Lite baaed on tile splittable information indicating that
decoding is allowed for each of the tiles and motion
10 vector information representing a motion vector used for
generating encoded data of a decoding target current
image when a picture of the current image is split into
the tiles and decoded; and
a decoding unit decoding the encoded data usJ ng the
15 prediction image generated by the motion oonipensati.on
unit.
The decoding device according to (t ) abow, further
including:
20 a vector generation unit generating the motion
vector of the encoded data from the mot ton vector
information using a motion vector of ^rl image located
adjacent to the current i fiiao,e arid contained within the
same tile as the tile of the current image,
25 wherein Lite iiiut.i on compensation unit performs
mot \ on oompenaati ost ol: the reference image for each of
the tiles ba.^ed on the tile spiittable information and
the iriotfori vector generated by the motion vector
general. i on uti i.L" ,
JO [S]
The decoding device according to [1) or (2) above,
101
SP345665WO0O
f u r t h e r i n c l u d i n g:
a f i l t e r u n i t performing l i t t e r i n g of the reierentift
image for each u n i t of the t i l e s,
w h e r e in
5 the filtor unit performs the f:\Utcriug of" the
reference image for each of the tiles based on f:.i 1 Lor
information representing that filtering of the reference
image is not performed across the tiles, and
the motion compensation unit performs, for each of
10 the tiles, the motion compensation of the reference image
obtained after the filtering by the filter unit based on
the tile splittable information and the motion vector
i nf orination.
M)
1 b The decoding device according to (3) above, wherein
L.he I i Iter unit performs, for each of the tiles, the
I i I taring of the reference image using a parameter for
the Ii1Lering associated with an image contained within
Uie corresponding tile based on the filter information
20 and parameter- shari nq i n( ormation representing that the
parameter is not shared between the tiles.
n
25 the picture is split into two of the tiles and
encoded,
the image of one of the tiles is an image for !efb
eye constituting a 3D image, and
the image of the other tile is an image for rf qht
30 eye constituting a 3D image.
(1&)
105
SP^45&65WO00
An encodinq method, including:
a rtiol.ion compensation atop performed by an encoding
device which qenerates a prediction image by performing
motE on ^ompensation ef a roforeneo image at a time
5 di f IcrerU. firom the time of an encoding target current
iiriaqe based on a motion vector detected within a tile
when a picture of tho current image is split into the
tiles and encoded;
an encoding step performed by the encoding device
10 which encodes tho current image and generating encoded
data using the prediction image generated by the
processing of the motion compensation step;
a setting step performed by the encoding device:
which sets tile split table information indicating t.hat
15 decoding is allowed for each unit of the tiles; and
a transmission step performed by the encodi ng
device which transmits the encoded data generated by the
processing of the encoding step, and the l:il e. sp I f L tabl e
information set by the processing of the sel.Li ng stop-
REFERENCE SIGNS LIST
[0343]
50
5h
56
71
74
79
S3A
83B
90
Fin cod i uq Coy i ce
Setl.i nq uni I.
Transmission unit
Calculation unit
Lossless encoding unit
Deblock filter
Motion detection unit
Motion compensation unit
Decoding device
10b
SP34bt;G5WOaO
91 Recepl i on urii t
102 T,osslcs& decoding u n it
10b ftddi Lion uni t
JOfc Deblock f.UU;r
110 Mo Li on compensation u n it
140 Deeodiny d e v i ce
162-1 to 162-M Encoding d e v i ce
164-1 to 164-M Decoding d e v i ce
107
SP345665WO00
CLAIMS
1 , ft decoding device, comprising:
a motion compensate on uni L generating a peed to Li on
b i niaqe by peri.ormi nq, tor each ol t i .1 es, motion
comp^nsa Li on ol a re Terence image wi thin a oo-located
tile based on Lite spliLLable E.n f'ormati on indicating that
decoding is allowed for each of the tiles and motion
voctot inrorfiiation representing a motion vector used for
10 generating encoded data of a decoding target current
image when a picture of the current image is split into
the tiles and decoded; and
a decoding unit decoding the encoded data using the
prediction image generated by the motion compensation
15 unit,
2, The decoding device according to claim 1, further
comprising:
a vector generation unit generating the motion
?G vector of the encoded data from the motion vector
information using a motion vector of an image located
adjacent to the current image and contained within the
tfariii? tile as the Li I e of the current image,
wherein the motion compensation unit performs
25 mo IJ on noiiiperisa Li on oJ the ref erenoe image I or each <>E
the tiles bayed on the tile spliLLable information and
the motion vector generated by the motion vecLor
generation unit.
30 3. The decoding device according to claim 1, further
comprising:
100
SP345665WO0O
a filter unit performing filtering of the reference
image for each unit of the tiles,
wherein
the filter unit performs the filtering of the
5 reference image for each of the tiles based on filter
information representing that filtering of the reference
image is not performed across the tiles, and
the motion compensation unit performs, for each of
the tiles, the motion compensation of the reference image
10 obtained after the filtering by the filter unit based on
the tile spl -i LLabl e in Forma Li on and the mot: .ion vector
i n Formation.
4. The decod i nq device according to claim 3, wherein
lb the filler unit, performs, lor each of." the tiles, the
f j l.tori IKJ oi the reference i raaqe us i nq a parameter lor
the filtering associated with an Imaqe contained within
the coj.respundi.iiy t.i to based on the filter information
and paraciteter sharing i nfennat i.on representing that the
2 0 parameter is not shared between the tiles.
5. The decoding device according to claim 1, wherein
tile split of a picture contained within the same
sequence is the same split.
6 . The decoding device according to claim 1, wherein
each of the tiles includes one or more slices.
/ . The decodi ng device according to claim 1, wherein
"30 Lin- fj i el.ii re is spl i t: i nLo two of the Li les and
decoded,
109
SP345665WO00
the image of one of the til es is an image lof 1 et t
eye constituting a 3D image, and
the image of the other tile is an image for right
eye constituting a 3D image.
8 . A deoodi ng method, oomprisi rig:
a motion compensation step performed by a decoding
device which generates a prediction image by performing,
lor each of tiles, motion compensation of: a reference
]0 image within a co-located tito based on tile splittable
information indicating that decoding is allowed for each
of the tiles and motion vector information representing a
motion vector used for generating encoded data of a
decoding target current image when a picture of the
15 current image is split into the tiles and decoded/ and
a decoding stop performed by the decoding device
which decodes the encoded data using the prediction image
generated by the processing of the motion compensation
step.
20
9. An encoding device, comprising:
a motion compensation unit generating a prediction
image by performing motion compensation of a reference
image at a time different from the time of an encoding
2 b target current image based on a moti on vector detected
w i th i ii a tile when a pi cture of the current i mage i y
split 1nto the ti Ies and encoded;
an eticodl ng unit oncod i no. the current Image and
gener a t i ng encoded data using the predict J on f mage
30 generated by l. he mot 1 on compensation unit;
a setting unit setting tile splittable information
110
SP34h66bWOOO
indicating that decoding is allowed (;or each unit of tho
tiles; and
a transmission unit transmitti ng the encoded data
generated by the encoding unit, and the tile splittable
5 information set by the setting uni {..
10 . The encoding device accordi ng to claim 9, further
eompri si ng :
a vector generation unit generating the motion
10 vector information based Of) a motion vector of an image
located adjacent to the current image and contained
within the same tile ax the Lite of tho current image,
and a moH on vector of tho current image.
lb 11. The encoding device according to claim 9, further
compri sing;
a F.i Itet unit performing filtering of the reference
i [fiagc toJ." each unit of the tiles,
whenein
20 the motion compensation unit performs motion
compensation of the reference image obtained after the
filtering by the filter unit using the current image and
the reference image obtained after filter E ng by the
filter unit based on the motion vector detected within
25 the tile,
the setting uni L sets f il t«r i n formation
representing that lettering ol the reference image is not
performed across the ti lea, and
the trans mi ss i on un f t LCcuisinits the filter
30 information an|. by the aetling unit,
11.1
HP345665WO00
12, The encoding devf ce according to claim 11, where i o
the filter utii I". performs, for each of the tile^,
the filtering of the re rereneo image using a parameter of:
an image contained wiLb f rt the corresponding tile,
5 the setting un it HC ta parameter sharing inf ormaLi on
representing that |.he parameter is not shared between the
tilesj and
the transmission unit transmits the parameter
sharing inforinati oi) set by the setting unit.
10
13. The encoding device according to claim 9, wherein
tile spliL ol a picture contained within the sarfte
sequence i.a the same split.
15 14. The encoding device according to c La Ltd 9, wherein
eaoh of the tiles includes one or more si iocs,
1-b. The encoding device according to claim 9, wherein
the picture is split into two of the tiles and
^0 encoded,
the image of one of the Li.Lea i s an image for left
eye constituting a 3D image, and
the image oi: the other tile is an image for ri qht
eye constituting a 3D imaqe.
25
16. An enood f nq method, comprising:
a mot i on compensation step performed by an encoding
device which qenevatcs a prediction image by performing
moti on compensation of a reference image at a I-line
30 di £ iercnl. I rem the time of an encoding target current
im,-iq.e baaed on a motion vector detected wlL.hi.n a tile
112
SP34566bCTO00
when a picture of the current image is split into the
tiles and encoded;
an encoding step performed by the encoding device
which encodes the current image and generating encoded
data using the prediction image generated by the
process i nq of the motion compendial i on & top;
a suLLing slop porlormaci by the encoding dovico
which sets tile splittable information indicating that
decoding is allowed for each unit of the tiles; and
a transmission, stop performed by the encoding
device which transmits the encoded data generated by the
processing of the encoding step, and the tile splittable
i EIJ omiri L i on set by the prooessi ng '.>\~ the a-ei-.L i nq sLep*