Sign In to Follow Application
View All Documents & Correspondence
View All Documents & Correspondence
Encoding Device And Encoding Method, And Decoding Device And Decoding Method
Abstract: This technology pertains to an encoding device and encoding method which make it possible to improve the encoding efficiency of images having multiple viewpoints , and to a decoding device and decoding method. A motion- parallax- prediction/compensation unit sets the following information in a list (L0) in a first order ,and in a list (L1) in a second order which differs from the first order: first- image- identifying information for identifying a first image having a viewpoint extending in a first direction from the viewpoint of the image to be encoded; and second- image- identifying information for identifying a second image having a viewpoint extending in a second direction which is opposite from the first direction. The motion- parallax- prediction/compensation unit and a calculation unit encode the image to be encoded and generate encoded data on the basis of the list (L0) and the list (L1). It is possible to use this technology in , for example , an encoding device or the like for encoding a 3D image.
Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence
Notices, Deadlines & Correspondence
Patent Information
Application #
Filing Date
03 June 2015
Publication Number
48/2015
Publication Type
INA
Invention Field
COMMUNICATION
Status
Email
remfry-sagar@remfry.com
Parent Application
Applicants
SONY CORPORATION
1- 7- 1, Konan, Minato- ku, Tokyo
1080075
Inventors
1. TAKAHASHI Yoshitomo
c/o SONY CORPORATION, 1- 7- 1, Konan ,Minato- ku ,Tokyo
1080075
2. NAKAGAMI Ohji
c/o SONY CORPORATION, 1- 7 -1, Konan, Minato- ku ,Tokyo
1080075
Specification
TECHNICAL FTELD
EOOO]|
The p r e s e n t : t e c h n o l o g y r o l a f . e s t o a n e n c o d i n g d e v i c e ,
a n e n c o d i n g m e t h o d , a d e c o d i n g d e v i c e , and a d e e o d i v i q m e t h o d ,
itnd p a r t i c u l a r l y L.o an e n c o d i n g d e v i c e , f an .image corresponding Lo the VPS i M set. In addition,
as ill ustrated in Lhe "J 1 th row to 1 4 th row, in the Vpy, i.lio
number (num. direct i.ef layers) of r^f orence images of an image
of eaidi viewpoint. and re I orence image spe<: Ifyinq inl ocmati on
(re I I iiyer id) specify i ng the re ter. once Images 3,ri' set.
10 [OOObj
Ecich view ID, f o r example, is used l.or r e p r e s e n t i n g a
2
UP35I082WOQG
re l."ererlce re t at ion between viewpoints and corresponds to the
rererencc image specifying information. Here, the view IDs
may be ass i gned regardless of the arrangement of cameras , More
spei: i I i caI Iy, I or example, as il] ustratod in iig. 2, a 3D image
b is configured by images of fi.ve viewpoints, and, in a case
whore cameras 11 lo lb photographing the 3D imago are arranged
such I.ha I. ea<:h camera i s arranged in an oblique direction with
the camera 1 3 being located at the center, view IDs, for example,
may be assigned as i 1 l.u.strated in A of tig, 2 or Ii of tig.
10 ?..
[0Q0bIn other words, as i l l u s t r a t e d in A of i ' i g . 2, a view
TD "0" may be ass i gned bo the camera 1 3 l o c a t e d at the c e n t e r,
and view IDs " 1 " , *?", ".!", and n4" may be assigned in order
1 h oJ the I;1082WO0G
t r a n s m i t l e d t h r o u g h a trrtit3itiisR.ionrfiedii.mor by b e i n g r e c o r d ed
on a r e c o r d i n g medium.
[0022]
Here, t h e encoding d e v i c e a c c o r d i n g t o the f i r a t a s p e c t
3 and t h e decoding d e v i c e a c c o r d i n g t:o t h e second a s p e c t may
be #n i n d e p e n d e n t d e v i c e or an i n t e r n a l b l o c k t h a t c o n f i g u r es
eric d e v i c e,
EFFECTS OF THE INVENTION
10 [0023]
According, to a I i r s t a s p e c t of t h e p r e s e n t t e c h n o l o g y!
t h e cod t ng e f f i c i e n c y of an image of a p l u r a I i t y ot viewpo i r t ts
c a n be i reproved•
[0024]
15 In a d d i t i o n , a c c o r d i n g t o a s e c o n d a s p e c t of t h e p r e s e n t
t e c h n o l o g y , coded d a t a of which t h e coding e f f i c i e n c y of an
icnage o/ a p l u r a L i t y o[ viewpc">i t i t s is Improved can be decoded-
RfUl^F DESCRIPTION OF DRAWINGS
20 J0025I
F i g , 1 is fi d i a g r am thai" i l l n s t . r a t e s an example of t he
s y n t a x oi a VPS of a 3ft coding system.
F i g . 2 is a diagram t h a i i l l u s l . a t e s an example of a
r e L a t i o n between cameras and view ir>s.
25 Fig. 3 is a d i a g r am t h a i , i i l u s l i aires an example of a
r e l a t i o n between cameras and view TfJs.
F i g . 4 is a b l o c k d i a g r am t h a t i I . l u s t . r a t e s an example
ef t h e oonf.igural i o n o i an encod i ngdev i ce of a f i r s t embod imont
t o which the p r e s e n t Lechnot oqy i^ a p p l i e d.
30 Fig. 5 is a d i a g r am t h a i , t l l u s t r a t e s an example of" t he
s y n t a x oJ a VPS s e t hy a s e t I. i rig uni t i l l u s t r a t e d in Fi g . 4.
8
SP35"J082WO00
Fig, 6 is a diagram that illustrates an example of a
posi i. iona I rel afcioEi between a vi cw ID and a camera .
Fig, 7 is a block diagram that i frustrates an example
of the configuration of art encoding unit i.l lustrated in Pig.
5 4.
Fig, 3 is a diagram that illustrates the calculation
of a time scaling process and a para J lax seal i.ng process.
Fig, 9 is a diagram Uiat iliustrat.es a conventional
method of regi ster Lng a lisL..
10 Flq, 10 is a diagram ihat illustrat.es a conventional
method of reg i.stering a list.
Fi t|. 11 is a diagram that, illustrates a method off
reg i ster, fig a list that is performed by a mot. i. on parallax
predtctio'i/compensation uni L il.l iistrated in Fig . 7.
15 Fig H 12 is a diagram that i llustrai.es a method of"
registering a list that is performed by the motion parallax
prediction/compensation unit illustrated in Fig. 1.
Fig.13 i s a flowohart that illustrates a generati on
process performed by an encoding dev i ce.
20 Fig- 14 .isafl owchart that il lust rate.s a coding procoss
illustrated in Fig. 13 E ri detail.
Fig . 15 is a f loivcha J- t that il I List rates a setting process
illustrated in Fig. 14 in detail*
Fig. 16 3 s a rlowehart that ilJ us Crates .-3 list
25 regi strat i on process ilJ ustrated in Fig. 15 in detail.
Fig, 17 is a block diagram that i (lu:i I. rates an example
of the con f iguraf:ion of a decod f ng device of a (first embodiment
to which the present technology is applied.
Fig. 13 i s a b Lock diagram that i I.lusf rates an example
30 of the con figm ation of a decoding uni t il I ustrated in Fig.
17.
J /
- ? •
9
SP351082WOG0
F i g . 19 is a i Jowchart t h a t E M u s i . r a t e s tin .image
g e n e r a t i o n p r o c e s s performed by t h e d e c o d i n g d e v i ce
i l l u s t r a t e d i n f i g . 17.
F i g . 2 0 \ a a f I owchart t h a t i I l u s t r a t e s a decoding
5 prorM^s il f . u s t r a t e d Lri F.ig. 19 in d o t a i . l .
l|Lig. 21 is a b l o c k diagram t h a i : . i l l u s t r a t e s an e^amptr?
of the c o n f i g u r a t i o n of an encoding dev.i co of # second
ernbodimen t to which the p r e s e n t techno"! ogy J a app i i e d .
Fig* 22 is a d i a g r am (.hat i M u s l . r a t e s an example of t he
10 syrUaK oi. a VPS s e t by a s e t t i n g u n i t i l l u s t r a t e d in F i g . 2 1.
F i g . 23 "fa a b i o c k diagram t h a i , i l l u a t r a l . e s an example
of the c o n f i g u r a t i o n of ^.'^ encoding u n i t .i Must: r a t e d in F i g.
2 1 .
Fig* 24 .is a f I o wen-art t h a t i.t l.us t r a c e s a g e n e r a t i on
15 process per;formed by an encoding d e v i c e il f u s t r a t e d i.n F i g,
2 1 .
F i g , 25 i s a f l o w c h a r t t h a t i l l u s U r a t e s a s e t t i n g p r o c e ss
in a coding p r o c e s s i I l u s l . r / a t e d in Fig. ?4 in dota.i I.,
j : ' i g . 26 is a b l o c k diagram t h a t i l l u s t r a t e ^ an example
20 of I.ho c o n f i g u r a t i o n of a decoding d e v i c e of Lhe second
embodiment. %o wh i ch ( he p r e s e n t , t e c h n o l o g y is appl l e d .
i ' i g . 7,1 is a b l o c k diagram t h a t i l l u s t r a t e s an example
of t h e c o n f i g u r a t i o n of: a d e c o d i n g u n i t i l l u s t r a t e d in F i g.
26.
25 L'ig. 2% i s a f 1 owcha J- t t h a t il I its t r a c e s an image
g e n e r a t i o n p r o c e s s performed by the d e c o d i n g d e v i ce
i l l u s t r a t e d i n F i g . 26.
l ' i g* 29 i s a bJ EJCJC d i a g r am t h a t i l l u s t r a t e s an example
of t h e con! 1 g u r a L i o n of" an e n c o d i n g d e v i c e o ! a t h i rdembodiment
.30 to which the p r e s e n t t e c h n o l o g y i s a p p l i e d.
P i g . "30 is a d i a g r am I h a t i M u s t r a t e s an e*ump I e of Lhe
10
SP3hlO&?VlOOO
syntax of a VPS sot by a set. Ling unit i I i ust rated i n Fig - 29.
Fig. 31 is a bloek diagram I.hat i ilustrates an example
of L.hc coni i.cnirabion of-" an encoding urn b illustrated in Fig.
29.
Fig- 32 is a flowchart thai illustrates a generabion
process performed by an encoding device illustrated in Fig.
29.
Fig. 33 isa flowchart that illustrates a cod i rig process
illustrated in Fig. 32 in dctai I.
Fig, 34 .is a flowchart thai, illustrates a registration
process illustrated in Fig. 33 in detail-
Fig. 35 is a block diagram that > l.lustrates an example
o£ Lho con figuration of a decoding device of a third erribod.i mont
to which the present technology is applied,
Fig. 36 is a block diagram that i I l.ustrates an example
of the conf i.gorabion of a denuding device illustrated in Fig.
35.
Fig. 37 i s a flowchart thai, illustrates an irnage
generation process per formed by the decoding dev i <-e
illustrated in Ftg- JS.
Fig- 38 is a f I owcharI thai, illustrates a decodi ng
process il i ustra bod J u Fig- 37 in deta i 1 .
Fig. 39 is a block diaq ram that i 1 I ustrates an example
of the hardware eonfigurat.ion or a computer.
Fig. 40111 ustra bos an example of" the schematic
cotirigural. i.on of u te J evisi on apparatus to wh i.ch the present
technology is applied.
Fig. 41 .i M.ustrates an example of the schematic
conTigura t ion of a rrtohi \ o phone to which the present techno I ogy
is applied,
Fig. 42 i I l.ustrates an example at the schematic
• J l
SP3410B2WO00
iion f i g u r e I LonoJ: a r e c o r d i n g a n d r e p r o d u c i n g a p p a r ' a I . u s t o w h i ch
I.he p r e s e n t t e c h n o l o g y is appl i e d .
F i g . 43 i l l u s t r a t e s an exampl o of t h e s c h e m a t ic
c o n f i g u r a t i o n o f an i m a g i n g a p p a r a t u s t o w h i c h t h e p r e s e n t
5 t e c h n o l o g y i s a p p l i e d -
F i g , 44 i s a b l o c k d i a g r a m t h a i i l l u s t r a t e s a n e x a m p le
of." t h e sr;huraatiti c o n f i g u r a t i o n of a v i d e o s e t to which t he
p r e s e n t techno\c>gy i s a p p l i e d ,
F i g . 45 i s a b l o c k d i a g r a m t h a t i l l u s t r a t e s a n e x a m p le
1 0 of t h e s c h e m a t i c c o n f i g u r a t i o n o f a v i d e o p r o c e s s o r t o w h i ch
t h e p r e s e n t t e c h n o l o g y i s a p p l i e d.
F t q . 4 6 i s a bl i>ck d i a g r a m t h a t il I u s t r a t e s a n o t h er
e x a m p l e of" t h e s c h e m a t i c c o n f i g u r a t i o n of t h e v i d e o p r o c e s s o r
I o which the p r e s e n t t e c h n o l o g y i s a p p l i e d.
15
MODE FOR CARRYING OUT TFIlil INVENTION
10026]
< F i r s t Embodiments
( C o n f i g u r e I ; i on E x a m p l e oC lincod i i tg Dev i.ee A c c o r d i n g to
20 Fi j . s t Embodiment.)
F i g . 3 is a b l o c k d i a g r a m thai", i l l u s t r a t e d an e x a m p le
of" t h e conf i g u r a t i on o f a n e n c o d i n g d e v i c e of a f I r s t e m b o d i m e n t
t o w h i c h the p r e s e n t t e c h n o l o g y i s a p p l i e d,
[ 0 0 2 7J
25 An e n c o d i n g d e v i c e 40 i l l u s t r a t e d i.n F i g - A i s c o n f i g u r e d
by a s e t . t i r i g un i I . 51 arid an e n c o d i n g u n i t "r>2. The e n c o d i ng
d e v i c e 50 i]one r a t e s a Coded S t r e a m by codi rig an i raage
( h o r e i n a J l o r , r e f e r r e d to as a mull, i ' v i e w p o i n t image) of a
p Lur£jli t y of v i e w p o i n t s p h o t o g r a p h e d by c a m e r a s as a plur-y t i t y
3 0 of i m a g i n g u n i t s I h a t a r e a l i g n e d i n p a r a i l e l i n a o n t ? - d i m e n s i on
arrangemi-!riL' by u s i n g a 3D cod i n g s y s t e m tint} t r a n s m i t s t he
12
SP3M082WO00
genera Led coded stream.
[0O2OJ
More s p e c i f i o a l l y , the s e t t i n g unit 511 of t h e encoding"
device bO s e ty an SPi> [Sequence Parameter Set) , a PPS ( P i c t u re
5 faramel.er Set) , and the 1 i fce. In addi I-ion, the p e l t i n g u n it
51 s e q u e n t i a l l y a s s i g n s view IDs (viewpoint, i d e n t i f i c a t i on
informstion} from l.he viewpoint, of a camera disposed at the
end among a p l u r a l i t y of" cameras aJ i gned in p a r a l l e J in a
one-dimensional arrangement. Tn add-i Lion, Lhe s e t t i n g unit
10 51 qonoi'ateii an order r e l i a b i l . i ty I fag (order r e l i a b i l i t y
i n f or [nation} t h a t is used for 3 dent i Tying Lhat the visw IDs
a r e .•sequent i a l l y assigned from the vrevrpoint of the camera
disposed at Lhe end amonq the p l u r a l i t y oJ cameras a l i g n ed
i n t h e one-dimensional arrangement. Then, lhe s e l t i n g u n it
15 51 s e t s ti VPS including t h e view IDs and the order r e l i a b i I i ty
f l a g . The s e t t i n g unit b\ supp Lies par ante tor s^ts such as
the SPS, the fPS, the VPS, and lhe l i k e to the encoding unit
52,
[0029 J
20 The encoding u n i t 52 codes a mult 5 -'viewpoint image i nput
from (.he out.Hi.de by usinq the 3ft coding system based on the
order i c l i a b i l.ity I lag and the ^ i.ow TJ~)s i n c I uded in the VPS
suppi fed from the s i t t i n g unit ^1., thereby g e n e r a t i n g coded
d a t a . The encoding u n i t h'A adds the parameter s e t s suppl ted
25 from the s e t t i ng uni I 51 to lhe coded da t a , t h e r e b y qenerrfl ing
a coded slirenm. The encoti ing on i t 52 s e r v e s #s a t.ransm.i ysion
u n i t and t r a n s m i t s the generated coded stream.
[0030|
[CJonf i qurat: i on Example of Syntax ol VPS}
30 fig"- 4 is a diagram t h a t i l l u s l . r a t e s ^n example of t he
syntax of a VPS s e t by t h e s e t t i rnj uni t 51 i 1 I u s t r a t e d i n f i g .
13
SP3!}1002WQ00
4.
[00.31 |
As il lustra Led in the (ilh row of Fit) - 5, an order
reli ability Flag fvfow order idc) is J ncluded in the VPS. The
5 order reliability flag ia v0" in a case where it represents
that Lhe view IDs are assigned Lo sequential ty increase front
the v i owpoint of the camera disposedat Lhe end among a plura I ity
of cameras aligned in par all el in Lhe ont^ '-dimensions I
arrangement add is ,ll" in a case where it rep'.esenl.s thai, the
10 vi ew IDS xr& assigned Lo sequential f.y decrease from Lhe
viewpoint of the camera disposed al. the end. in add.i Lion,
the order reliabj \ ity f."lay is "0" in a case whore iL represents
thaL the vi ew IDs are assigned not sequentially from the
viewpoint of the oarnora d i sposed at the end among the plura I ity
1^> of cameras aligned in parallel in the one-dimeiis ionaJ
arrangement.,
{0032]
for example, in a oase where the came* as are aliqned
in I.ho hori/.ontat direcLion, AH'3. the view TLis are assigned
?0 sequentialt y from the camera disposed at Lhe lei I. end such
thai. Lhe view ID of the viewpoint of Lhe camera disposed at
the r Lght end is Lhe largest, i.he order re J i abili t y f laq is
"0". In adi"i i tion, in a <;?ise where the earner as are alt qited
in I.he horizontal direct.! on, A.rn:l the view IDs are assigned
?b sequentially from i. he camera disposed at the lei I. end such
thai, the view ID of the viewpoint of the camera disposed at
the r iqht end rs the snia.l I t;st, I.he order rel i abil i I y f la2 p e r l orms an A/D c o n v e r s i o n of rfn i n p u t image of a
p r e d e t e r m i n e d v i e w p o i n t and s u p p l i e s om image l.hat ia d i g i t a l
d a t a a f t e r t h e c o n v e r s i o u t o t h e s c r e e n rearrangement. b u f f er
102 s o a £ to b e s t o r e d t h e r e i n . The sei een rearranqemerU" buf I or
102 r e a r r a n g e s s t o r e d images, which a r e in d i s p l a y o r d e r of
25 frames, in o r d e r of frames for coding in a c c o r d a n c e w i t h a
COP (Group Of P i c t u r e ) . The s c r e e n r e a r r a n g e m e n t b u f f e r 1 02
s-uppli e£ the image of which I-he o r d e r of" frames have boon
r e a r r a n g e d t o t h e c a l c u l a t i o n unit. 103 t o g e t h e r w i t h the view
ID anrf t h e POC (P i c t u r e Order Count) of t h e image,
30 [0041 J
In addition, the screen rearranqement bufIer 102
16
SPJ51082WO00
sitppf j.os the imatjo of which L-ho order ol frames have"? been
rearranged also to the intra predict.i on uni I. 1 14 and the motion
para 11 as predict.! on/componsat. i on un it 115 together with the
view 1L? and the POC of the image.
b [0042J
The oalcula Lion unit 103 serves as an encod '< rig un i t and
peri orms coding by subtracting a fired E cted image supplied from
the intra prediction unit 114 or lite motion parallax
predi.ction/eompenscition unit 1 1 5 through the predicted Image
1 0 selecting unit 116 from an J mage road from the screen
rearrangement buffer 102. The calculation unit. 103 outputs
di I" ferent.i al information acqn i ted ay a result thereof to the
orthogonal t. rans form nil i.t 104 .
[0043]
1^ fi'or example, in a ease where intra codi.ng is performed,
the calculat i.on un i t 103 subtracts a predicted imaqo supplied
frtim the intra prediction unit 114 Trom an image read from
the screen rearrangement, Jjuffer 102. On the othei hand, in
a ca:-;e where inter codinq is performed, the calenfation unit
2 0 103 subtracts a predicted imaqo supplied I rom Lho mo J-ion
para I ] ax prediction/compensati en uni I- 115 I rom an Image road
from J..he screen rearranqement tn.iffer 102.
[00441
The orl.hogona i. trans form unit 104 performs an orthogonal
25 transform such as a discrete cosine L rans fonn or a
Karhunen-Leove transform for the d i f"ferential i nfomiction
suppl i od f rotfi the ca f.cula t i.on un i t 103 . Here, the method used
for I.his orthogonal transform is arbitrary. The orthogonal
transform nni.t 104 supp I ios a transform coefficient thereof
30 to (.he quantization unit 105.
[004b]
T7
SP3510S^WO00
The quantj nation unit 1 05 perl eirms quantization of the
transform coefficient that is supp I i.ed front the orthogonal
transform unit 1 04 * The guantizal. i on un f t IflS sets a
q^antizat 1 on parameter based on in format i.on rt [.atimj to a
5 target value of a oodinq amount. and performs the quantization,
Here, the method used for. this quanti z#tion i.s arb i.trary. Tim
quantizat ion un i t 105 suppl i es the quant 3 zed transf orm
coefficient to the lossless encoding unit 106.
[0046]
10 The lossl ess encoding unit 106 performs lossless coding
of" the transform coefficient quantised by the quantization
uriit 105 vising an arbitrary oodinq system, thereby generating
coded data. In addition, the lossless encoding unit 106
acquires- i.ntra prediction information including information
15 represent, i.ng thy mode of the intra prediction and the like
from the intra prediction unit 114 and acquirer inter
prediction information including information ropresentinq
thf? mode of the inter prediction, motion paral lax vector
information/ anti the tike from the motion parallax
'A0 prediction/compensation unit 115 . Tri add i iron, the lossless
encoding unit "j.06 acquires a f i.lter coefficient and the like
used in the filter ill.
[0047]
Th£ J os si ess encoding un E t 10f> porf orms lossless crjdinq
25 of such v#r i ous k i nds ol i nformatron using ^n arbi (. vary eodinq
system and sets the coded information as a part of hoader
information of tho coded data {mull, i olexi rig) . The lossless
encoding unit 11 00 supp i ies tho coded data i.n wh i oh the hoade.
information is multiplexed to the accumulation buffer 107 so
30 as to be accum^ i a ted (.here t n .
10048]
IB
SP3.L3l092WO00
As JJte c o d i n g s y s t em of the l o s s l e s s e n c o d i n g unit. 106,
lui; example, t h e r e is v a r i a h I e - l e n g tit coding, a r i t h m e t ic
c o d i n g , or t h e l i k e . As t h e v a r i a b I o - l e n g t h coding- for
example, t h e r e is a CAV?,C {Contost-Adaptive VarJrtble Length
5 Coding) d e f i n e d in tht* H.264/AVC system or t h e l i k e . As the
a r i t h m e t i c c o d i n g , for example, t h e r e is CAUAC
{ C o n t e x t - A d a p t i v e B i n a r y Ar-i Uimeti<: Coding) or the l i k e
[0049J
The accumu l a t i o n b u f f e r 1 07 t e m p o r a r i I y s t o r p ^ t h e coded
10 data s u p p l i e d from t h e l o s # I e s s ^neoding u n i t 10 6 . The
a c c u m u l a t i o n b u f f e r 107 o u t p u t s t h e s t o r e d coded d a t a <\t
p t o d e t e r n i i n e d t i m i n g , for example, to a r e c o r d i n g a p p a r a t us
[ r e c o r d i ny medi um) , a t r a n s m i s s i o n 1 i n o , or t h e l i k e of a I a t e r
s t a g e , which i s not i l J u s t r a t e : d i n the f i g u r e , t o g e t h e r w i th
15 the p a r a m e t e r s e t s u p p l i e d from t h e s e t t i n g u n i t 51 "i I l u s t r a t o d
in f i g . A as a coded s t r e a m . In o t h e r words, the oodod s t r e am
f a suppl \cd to [.he decoding s i d e.
[0050]
I n rtddition, t h e t r a n s f o rm c o e l iTicient q u a n t E zed b y t he
20 q u a n t i z a t i o n un i L 105 is s u p p l i e d rtlso to t h e i n v e r se
q v i a n t i z aL i on uni t 108 . The i a v e r s e q u a n t 1 n a t i o n u n i t 103
per forms i nverse? quant i n a t i o n of t h e q u a n t i z e d I.j.ansf urm
ooef f i c i en J. by us i n g a m ^ t hod c o r r e s p o n d i n g to I . he q u a n t i z a l i on
p r o c e s s performed by t h e q u a n t i z a t i o n u n i t 105. The method
25 used f o r t h i s i n v e r s e q u a n t i z a t i o n p r o c e s s f i i a y b e any method
ay l o n g as the me ( h o d cor r e s p o n d s to the quan t i z a t i on p r o c e s s
pei- formed by t h e q u a n t i n a t i o n u n i t 105. Tho i n v e r se
q u a n t i z a |. ion un it lOfi suppl i 03 thi^ acqu.i rc-.fl t r a n s f o rm
c o e f f i c i e n t t o the i n v e r s e o r t h o g o n a l t r a n s f o rm u n i t 109.
30 10051]
The i n v e r t o r t h o g o n a l t r a n s I orm uii i t 100 p e r f o r i n s an
19
yP3h1O82WOO0
inverse orthogonal trans J orm of the tr.ansfor.m coeffioi cut
suppli ed froitf bhe ! nverse quanti zation unit 10B by using a
method corresponding "to the orthogonal transform process
performed by l.he orthogonal transform unit 104, The method
5 used for the E nverse orthogonal transform process may be rtny
method as long as the method corresponds to the orthogonal
trains I oon process performed by the orthogonal trans form unit
104. An output (the differential information that Is locally
restored} for which the inverse orthogonal transform has boon
10 performed is suppl i.od to the calculation unit 110,
[005?The ca I culaf. i on un i t 110 adds the predicted image
supplied from the intra prediction unit 114 or the mot.ion
paral I aM pred lotion/compensation unit 115 through the
15 predicted image selecting unit 116 to a result of the inverse
orthogonal transform that is supplied from the inverse
orthogonal transfo ml unit 109, in other words, thedi J rorenl.i.al
i information that is local I y restored, thereby acquiring a
local 1 y-decoded image [hereinafter, referred to as a
20 reconf i gured Image) . The roconl i.gured ima£fe is supplied to
the filter 111 or the decoded picture buffer 112.
[0053]
The filter 111 i ncludes a deb Locking filter, an adapt 1 ve
[ oop f i I tor, or" the 1 f ke and appropriately performs a filter
25 process for the reconfigured image supplied from Lhe
calcti I at ion unit 110 . iTor exampl <:, the filter 111 e I iminaics
a block distortion oT the reconl i gured image by performing
a deblocking filter process for the reconfigured image. In
addit i on, the filter 111 Improves the imago qua I ity by
30 performing a I oop f i I ter process for a result (a reconfigured
image from which the blo<;[i distortion has been el iminated)
• J /
20
yP3blOB2WOOG
oT t h e d e b l o c k i n g f i I t e r p r o c e s s by u s i n g a wiener f i l L o r .
10054]
The f i l L o r 111 maybe conf i g u r e d l.o p e r l orra an a r b i t r a r y
I r I t e r p r o c e s s for t h e r e c o n f i g u r e d image • Tn add i t i o n , t he
5 f i l t e r 111, as is n e c e s s a r y , may lie c o n f i g u r e d l.o s u p p ly
i TI format, i on of a f i l t e r COP f f i c i e n t u s e d f o r t h e f i l l , or p r o c e s s
and t h e like l.o t h e l o s s l e s s encoding u n i t 106 so as Lo be
J osslessly coded.
[0055]
10 The f i l (.or 111 suppl i o s a r e s u l t there* n a f t f i r , r e f e r r ed
l o as a decoded im^qo) oi the f i l t e r p r o c e s s to l.ho decoded
p i c t u r e buffer; 112,
1U056]
The decoded p i e t u r e b u f f e r 1 12 s i . o r e s l.ho r e c o n f i g u r ed
i.5 image s u p p l i e d from Lhe c a l c u l a t i o n u n i t 110 and t h e decoded
image s u p p l i e d from t h e f i l I.or 111 . In a d d i t i on, t h e decoded
pi e t u r e b u f f e r 112 s t o r e s v i ew IDs a n d POCs of l.he r e c o n f i g u r e d
image and the decoded image,
[0057]
20 The. decoded p i c Lure bu f f e r I 1 2 s u p p l i e s l.he r e c o n f i g u r e d
image and the view ID and t h e I?OC of t h e recon f i g u r e d imaqo,
which a r e s t o r e , to t h e i n t r a p r e d i c t i o n u n i t 114 t h r o u g h the
se \ e c t i o n u n i t 1 1 3 at pr e d e t e i mined t i m i n q or b a s e d on a r e q u e s t
from t h e o u t s i d o such as the i n t r a p r e d i c t i o n u n i t 114. in
?^ a d d i t i o n , Lhe decoded p i c t u r e b u t I er 11? suppf i es t h o decoded
ifiitige and the view ID and t h e POC of t h e decoded image, winch
arc- s t o r e d , to Lhe mo Lion p a r a l l a x p r e d i c t i o n / c o m p e n s a l ion
tin it l l i i Llirouoh t h e s e l e c t i o n u n i i 113 a I. p r e d e torrid nod t i ruing
ort>asedt>ri a r e q u e s t i rorcith^ o u t s i d e s u c h as t h e rriotiotipa.ra 1 l ax
30 prcdici. i on/componsaL i on u n i t 11 b.
10058]
21
SP351082WQ00
The se lection uni I. 113 represents # supply desi.i nation
of the reconfigured imaqe or the decoded image that is output
from l.ho decoded picture buffer 11?, More spec: i f ica I ly, i n
a case vjhere intr^ coding is performed, Lhe selection unit
5 113 reads the reconfigured imaqo for which Lhe f i l.ter process
has not been performed From the decoded picture but ("or 11 2
and supplies the road roconJ. i gured image to the inl.ru
pred fction unit 114 as an image (peripheral linage} of a
peripheral area 1 ocated on the periphery of l.ho current
10 prediction area.
[00!J9]
in addition, in a case whore .i rtter coding i s per formed,
the selection uni L 113 reads l.ho decoded image lor which the
filter process has been performed fv.om the decoded picture
l£j buffer 112 and supplies Lhe read decoded image l.o the motion
para I lax predict! on/compensate on unit l"Lb us a reference
image.
[0060]
When l.ho peripheral J image is acquired from the decoded
20 picture buflt-r If.?, the intra prediction unit 114 perforin-;
an int.r.vii prediction process i n whi oh a predicted image i.1-;
generaLod basically usinq a prediction unit (PU) a3 a
processing un i t by using a pixel value of the peripheral i mage.
'['he inl.tn prediction unit. 114 performs an intrrf prediction
25 i.n each of al 1 the intra predict i on modes that are eandi dates .
[0061|
The intra prediction uni I- 114 evaluates cost function
values of predicted imacjos by usino the predi cted i rnages
genera tod by the intra predict i ens of all the intra predict! on
30 modes that artj candidates and (he input image supplied from
the ^eroen rearrangement builer 102 and .selects an optimal.
22
SP351GH2WO00
intra predicti on mode . Then, the intra predial. i.on unit 11 4
supplies the predicted image generated in the optimal intra
prediction mode and the coat function value to the predicted
image selecting unit 116,
[0062]
In addition, in a case where the selection is notified
from the predicted Image selecting unit 116, Lhe intra
ptediol-lon unit 124 appropriately supplies intra, prediction
in format ion ineludi rig in I ormal. i.on relating te the i ntra
predict i ons of" the optima 1. intra prediction mode and the li ko
to the [ossless encoding unit 10 6 so as to be coded<
[0063]
The mot i on parallax prediction/compensation unit Hi)
registers the reference iiviaye specifyi rig inTormal. ton included
in the VPS i.na .1 i st based on Lhe order re liabi I ity J. Fag included
in the VPS supplied from Lhe setting unit bl ilti.istral.od in
Fig, A, thereby generate ng the list..
[0064]
More specifically, the motion paral fax
predict:<>n/componsa t ton unit 115 r£g i aters the reference
imago specIfyinif in 1 or: mat: i on e^eh hav i ng an image of the saiue
viewpoint of which the FOC is before that of the current
predicting image as a reference image in a List T,Q in order
of sotting in Lhe VPS, In addition- the motion parallax
prediction/compens^ t i on mi it 11 5 reg i titers the reference
i m#ge specify] ng inf ormal. i on each hav Ing an image of Lhe same
viewpoint, of which the POC is after that ot the current
predicting linage as a reference image in rf list LI in ordt:r
of the setting in the VPS. lj-urLhermor e, the mot. i on pa ruli ri#
predict! ou/conipensatIon unit 11 5 series as a set.L ing unit and
registers (sets) the reference i mage spect Tying Informati on
2'3
spjsioaawooo
each havinq an image of a viewpoint different from that ol
the current pred.i <: ting .i ruage as a refor.once linage 3 n the Lists
1*0 and ill based on the order reliability flagb
The motion parallax prediction/compeEisnt.ion unit 11F>
reads decoded images specified by the relcrenot; imago
specifying information registered in the list in. order of
registration in the list from the decoded picture frufJor 112
as reference images. The motion pacallax
1 0 prediction/compensate onunil: 115 performs a motionpred i ction
or a para f l.ax prediction as an inter prediction bas i ecilly
having the ^u as a prooessinq unit by using the input iniage
supplied from the screen rearrangement buffer 102 and the
rei orence i mages *
1U [0060]
The mot ion predict! on is a prediction using a cyrr els tion
of the time direction, and the parallax prediction is a
pred i ction using a correlation of-" the parallax direction. The
motion parallax predict i on/oompensa tion unit lib detects a
20 mot i on vector as a t &sul t of the (notion pred lotion and detects
a parallax vector as a rosul I. of the para I lax predicti on .
Here i naf ter, in a case where the motion vector and the para I lax
vector do not need to £>e parties I arly d l.scr i ruinated from each
other, the vectors will be collectivet y refer rod to a£ a mot. ion
2b parallax vector.
[OOiWj
The moLion parallax prediction/compensation unit 115
perlorms a compensation process accordinq to the mot ion
pars: J lax vector detected as a result of the i nter predi ction,
.ii) thereby generat-i ncj a predicted iinaqe. The motion parallax
pred Letion/compensation unit 115 pci fornix the inter
24
SP351082WOG0
p r e d i c t i o n and t h e compensation profioss in each of aJL t he
i n t e r p r e d i c t i o n modes t h a t a r e c a n d i d a t e s.
[00681
Tho motion parallax prediction/compensation unit llh
5 evaluates cos I function values of predicted images by usinq
the predicted images of all tho inter prediction modes that.
are candidates and the input linage supplied from the screen
rearrangement buf for 102 and selecta an optimal int.or
prediction mode. Then, the motion parallax
10 ptedi etion/eompennation unit 115 supplies L he predicted imago
generated in the optimal inter- prediction mode and tho cost
funct. i on va I uo to the predicted image se I ecting uni i. 116 .
[0069[
Tn addition, in a case where a selection Is notified
15 f.'rom l.ho pred Lcted image selecting unit 116, tho mol. i on
parall ax pred Iction/compensat i on un it ll.S generates a
prediction vector of a motion parallax vector cor.responding
l.o the predid ed image generated in the optimal inter
predict Ion mn parallax predict ion/compensation
unit ii.r> is supplied bo the calculation unit 103 ^nd the
calculation unit. 110, arid in* er coding is performed. The
mo I. ion parallax pi edict i.on/compensabion unit 11S rioti I ics the
selected supply source of the selection.
30 J0076]
Whi Jo the decoded pictu re buf ier It? stores the decoded
21
SP3f>1032WO00
image and t h e rcconfiqujccd imago ot the c u r r e n t p r o c e s s i ng
viowpoinL. and Lire vievj TL>s and POCs of t h e decoded i mage and
t h e r e f i o n f i g u r e d image, t h e m i ] I t i - v i e w p o i n t decoded p i c t u re
b u f f e r 1.1 s t o r e s a decoded image oi each viowpoinL and t he
f t view ID rfnd t h e fOC oi the decoded image. Tri o t h e r words,
t h e mult i - v i e w p o i n t decoded pi c t u r e b u f f e r 1 21 acqu Lues t he
decoded linages s u p p l i e d t o t h e decoded p i c t u r e b u f f e r 112 and
t h e view IDs and t h e POCs oJ the decoded images and s t o r es
them t o g e t h e r w i t h t h e decoded p i c t u r e b u i f e r 11?.
10 [0077!
When t h e c u r r e n t p r o c e s s i n g v i e w p o i n t i s changed, I.lie
decoded p i c t u r e b u f f e r 112 eKminal.es the decoded image of
t h e p r e v i o u s p r o c e s s 1 ny v i e w p o i n t , the mtjJ | . i - v i e w p o i n t
decoded p i c t u r e b u f f e r 121 m a i n t a i n s the decoded image oC
15 previous p r o c e s s i n g viowpoinL a s i t i s . Then, a c c o r d i n g L:o
a r e q u e s t from I.ho decoded p i c t u r e b u f f e r IT 2 or Lhe l i k e ,
t h e decoded imrtgo and t h e view ID and t h e POC of t h e decoded
image, which a r e s t o r e d , a r e s u p p l i e d Lo t h e decoded p i c t u re
b u f f e r T12 as w# decoded image of a n o n - c u r r e n t p r o c e s s i ng
20 viewpoint and Lhe view Tf.> and Lhc l5OC of t h e decoded image",
The decoded p i c t u r e b u f f e r 11? suppl i o s " t h e decoded image
of a n o n - c u r r e n t p r o c e s s i n g v i e w p o i n t and t h e view JTi and t he
POC of t h e decoded image" r e a d from t h e mu i t i - v i ewpoinl. d e c o d ed
p i c t u r e buffer 121 to Lhe motion parn I.lax
25 p r e d i c t f o n / c o m p e n s a t i o n u n i t 1 1 a t h r o u g h t h e s e l e c t i on uni L
113.
[0078J
( D e s c r i p t i o n oi Time S c a l i n g P r o c e s s and P a r a l l ax
S c a l i n g Process)
30 Fig. 3 is a d i a g r am t h a i i l l u s t r a t e s the CH f c u l a l . i on
of t h e I. i jito sea I i ng p r o c e s s and t h e pai a l l a x s c a l i n g p r o c e s s .
yP3hl Q32WO00
[00791
Tn the (.Lino soa J ing process, by multiplying the
predict i.on vector of the time peripheral area image by a
coef lii ci ont d.i a t^caleFactor, the prediction vector of the 1. i me
b peripheral area image is corr.eeted* This coefficient
distSca lei'acV.or. i as illustrated in I he 1st and 2nd rows of
Fig, B, .i.y acqn i.rod basod on a coef f i<": Lent tb arid a coeffieJ ftnt
td.
[OOSO!
10 The coefficient tb, ay illustrated in the fth row, is
acqu i redbased onadi [ f or encode t weft rt the PGC (PicOrdorCntVal)
of (.he current predicting linage and the POC PioOrderCnt
[refpi c List A j ref IdxA] ) ) of Lhe refeconce .image of the imaqe ,
In add Ition, the coeIficient td, a^ illustrated in the 8th
lb row, is acqui red based on a difference between the POC
(PicOrderCntVrt L) of the time peripheral area imaqe and the
POC (PicOrderCnt[Re IpicListX[refldxLX]}) of the reference
image of the time per i phera.l area i mage .
[00G1 i
2fl On the other hand/ in (.lie para I lax s(:a Ling process, by
multAp f ying the predi oJ ion vector ol Lhe parallaH peripheral
area linage by a coefficient d i stUcal el-'actoi", the prediction
vector of the paralJax peripFieral area image is corrected.
This coefficient disl. Scale Factor, s i milar to the case of the
2S time scaling process, as illustrated in the 1st and 2nd rows
of Fiq, 3, is acquired based en tho coef I icient ib and the
coefti ci ent td,
[0082]
However, the coefficient tb at Lhis t i me, as 3 I lust rated
.iO in the 10th row, is acquired based on a difference between
the view ID (Vi ewOrderTdx Va I ) of the"; current pred i cting image
?erformed by Encoding r)evice>
J'"ig. 13 is a floy/charl. that i l l u s t r a t e s a generation
33
SP351US2WO00
process performed by the encoding device-
[0094]
in Step Sll represenbod in Fig. 13, the setting unit
51 of the encoding device bO sets an SPS. In Step S12, the
5 sotting unit hi sets a PPR, in Step Si 3, the setting unit
hi sets a VPU that iiif:l udes a view TD and an order re"! i abili ty
i lag. The setting unit 51 supplies parameter sets such as
the SPS, the PPS, ^nd the VPS to the encod i rig unit 52.
[0095]
10 Tn Step S14, Lhe encoding unit 52 performs a coding
process f n whi<:hamu I bi-viewpoint imacje input from the outside
i s coded using a 3D cod ing system based oil bhe order re? I.iabi.l i. ty
J I ag and the view ID i ncluded in the VPS that i s suppf ied from
tlio setting unit 51. This coding process will, be described
1.5 in detai i Later with re frerence to Fi g. 14 bo be described latfir.
[0096]
In Step Si 5, theencodi nguni b 52 generates a coded stream
by adding bhe parameter set suppl i.od from the setting unit
hi to the: coded data accumulated in the accumulation bui for
20 107 and transmits the generated coded stream. Then, the
process ends.
[0097]
F±q. Id is a flowchart that illustrates the r:odirt(]
process o\ Htep Sid represented in Fig, 13 >n detail,
?b [0098]
In Step Si 01 represented i" Fig. 1 d, the A / D converter
101 of the encoding unit 52 performs ^ A/D conversion Qi an
i nput image of d predetermined viewpoint and supplies an image
al bor the conversion, whii;h is digital dat^a, to the screen
30 rearrangement buffer 102 ao as Io be stored.
10099]
3d
SP3.1)1082WO00
Tn Step S102, the screen rearrangement buMct 102
rearranges the slorod imago of frames, which are oonfigured
in the display order, in the order of frantos for eoding in
accordance with Lho COP. The screen rearrangement buffer 3 02
t> supplies (.he image of which the order oi frames has been
rearranged also Le the oalculation unit 103, the intra
prediction unit 114, and the motion parallax
predi c I ion/compensation unit lib together with Lho view ID
and the POC of Lllc image.
10 [0100!
Tn Step S1.03, the intra prediction unit 114 performs
an intra predict i on for generating a predi otod image basioa I E y
using l.he PU as a procsssi ay unit by using l.he peripheral imago
supplied from l.he decoded picture buffer 112 through the
lb selection unit "J 1 ,"j. The intra prediction unit 1"J4 perforin
this intra predict, i™ in each of al I the intra predi o Lien mode s
that aro candidates.
[0101]
Til addition, the .intra prediction unit 114 evaluates
20 cost funct ion vat ucs of predicted i mages by using the predicted
images generated by the intra predictions of all the intra
prediction modes that are candidates and the input image
supplied from the screen r ear rang ement bui ftc 102 and selects
an optimal intra prediction mode . Then, the intra predicts on
2 5 unit XIA supplies Lho predi etcd image generated in l.he optima 1
intra prediction [node and the cost funot ton value to the
predicted image se Looting unit 1 "I 6 .
[0102]
Tn Step SI 1)4, the rnotion parallax
."^0 predict i on/compensation unit 1I!J performs a selt.ing prot.-oss
for set I. i ng presence/no-presence of the para3 Mx sea.} i ny
35
SP3510S2WO00
process by r.egisteri ng t-ho lists LO and LI b^sed on the order
Leliability flay included in bhe Vpy that is supplied from
thesettinq anil. 51 ±1 I usf. rated LnFJu . 4 . This setting process
will bo described wi th reference to Fiq. 15 to he described
5 Later.
[0103j
In Slop fil05, the motion parallax
pred i ction/cofTipenaa tion unit. 115 performs an inter predict i on
basic-ally using th£ PU as the processing unit by us.i rig the
10 reference iniag.es based an the lisis LO and Ll and the input
imago supplied J'rom the screen rearrangement buffer 10? . Then,
the motion para J la^ prediction/compensation unit. 115 pcrforms
a compensation process accord.i ng to the motion parallax vector
detected as a resu.l t of the i nter predict.i on, thereby
15 qenerating a predicted .image* The motion pa.ra.1 fax
predj etion/compensation unit 115 performs such tin i (iter
prediction and the compensation process in each of all (.lie
inter pred5ction modes that a re candidates.
[0104|
2 0 In additi on, t.he motion parallax
predi ction/compensation unit 115 evaluates cost function
values of predicted images by using the pred i.eted imaqes of
all the inter prediction modes that are candidates *ud the
i fiput i rnage supp I ied I rom I he screen rearrangement, buf I et 102
?5 (3 rid se I ects an opt. r.ma.l i.nter pred leti onmodo. Then, the motion
para 1 I ax predi ction/compensation unit. 115 supp I ias the
predicted image generated i n the optimal i nter pred i c tion mode
and the cost function va.l no to the predicted imatje se I ect f ng
unit 116.
30 I010h|
in Step ST 0 6, the p r e d i c t e d Imaqe s e l e c t i n g unit 116
36
SP3510fl^WO00
determines whether to select the intra prediction unit 114
as a supply source of (.he predicted image to be stippl i od to
the calculation unit I 0 3 and the calculation unit 110 based
an the cos L fund: i on val uos supplied from the intra predi ction
5 uii i t 11^ and the motion par si 3 ax prediction/compensation unit
115.
[0106]
Iii n cast', where the intra prediction unit 114 is
determined to he selected in Step S106, Ule predicted image
10 selecting unit 116 supplies the predicted image suppli ed from
l.ho intra prediction unit 114 to tho calculation unit T 03 and
thti calculation unit 110 and notifies the intra prediction
unit 114 of the selection. Then, in Step £?107, the Intra
prediction unit 1 14 supplies Intra prediction in Format, i.on to
15 tho lossless encoding unit IDG, and the process proceeds to
Step S1Q9,
[0107]
On the other hand, in a case where the intra prediction
tin i t is determined not to be sel ected i n ytep.SlOG, the predicted
?.Q imaqc selecting unit lib suppl I os the predicted image supplied
from the mot ion parallax pred i ction/compensation unit 1 13 to
the calculation unit 103 arid the oalcula) len unit 110 and
not i fies the mot i on para I Lax predict i cn/compensal ion un i t 115
of l.ho selection. Then, in Stop S1U8, the motion parallax
2b prediction/compensation unit 115 generates a predict i.on
vector oi a mot i on pa rallax vector correspond i rig to the
predicted image generated in the optima 1. inter pred i ctioiiinode .
(0103]
Kore speci I leal 1 y, in a case whore the parallax soalint;
30 process is valid, L he mo I ion pa rallax predi ft ion/compensation
UII i t 115 generates a predict! on vector by performing tho time
'A'I
SP3M082WOOO
scaling process or the parallax scaling process . On L-ho other
hand, in a c^ao whore the para 1 fax scaling process is Invalid,
the motion parallax predict i on /compensation uni t. 115
generates a prediction vector by performing the time scaling
5 process or sf?ts a parallax vfd.gr of the parallax peripheral
area image as a prediction vector ay it is.
[0109]
Then, the motion para 3 I ax prediction/compensat ion unit
115 acquires a difference between the prediction vector and
10 themot.i on parallax vector corresponding to thepred i cLed image
generated in the optimal inter prediction mode as motion
parallax vector information. Then, the motion parallax
predict] on/compensation unit 115 supplies the i nl.er
prediction i n [."or mat ion inclurti nq the motion paral 1 ax vector
15 information, Lhe optimal inter pred i ction mode, and Ule like
to the lossless encoding unit 106, and the process proceeds
to Step SI 09.
(0110}
In Step S109, the calculation unit 103 calculates a
y.i) difference be L.ween the image read I roni Lhe screen rearrangement
buffer 102 and Lhe predicted imaije supplied from the intra
prediction uni I. i.14 or the motion parallax
prediction/compensation unit l"l h through the predicted image
selecting unit 116, The calculation unit 103 outputs
25 differentia! 1 n I ormation acquired ^HJ a result thereof to the
orthogonal transform unit "J 04 .
[0111]
Tn Step 13110, the orl.hoo.ona 1. transform \m i I. 1 04 performs
j,n orthogonal transform oi lhe differentirtl information
30 suppli ed ( rom the calculal.l on mi i t 103 and supp I i es a transform
coefficient acquired as a result thereof t.o l.lie quantization
W7
W/ ~j'
••
SP35I082WO00
i
u n i t 10b-
[0112]
Tn S t e p S i l l , the truanti n a t i o n u n i t 105 q u a n t i s e s t ht
Lransl OUR c o e f f i c i e n t s u p p l i e d from t h e o r t h o g o n a t t r a n s f o rm
5 u n i t 104 and iHuppl.i cs t h e quant i zed [.ransform coef fio.i e n t ti>
f: tie l o s s l e s s encoding u n i t 10(i and I.ho i n v e r s e rfuantj n a t i on
u n i t IDS.
Tn Step SI 12, Lhe it-worse quantf nation unit 1 08 perforin?-;
10 inverse quantization oJ L-he quantised transform coefficient
suppl 3 ed from the quant i.zati on uni b 105 using a method
corresponding to Lhe quantization process performed £>y the
quant i nation unit 105 and supplies the acquired transform
eoef t i cient Le tht; inverse orthogonal transform uni U 109,
15 [0111}
T(i S t e p S113, t h e i n v e r s e o r t h o g o n a l l . t a n s l orm un i t 109
performs ^n I n v e r s o orl.hogona L t r a n s f o rm of (.lie t r a n s f o rm
o e o f f i o i e n t s u p p l i e d from t h e i n v e r s e q u a n t i z a t i o n u n i t lOfi
by u s i n g a method c o r r e s p o n d i n g to Lire o r t h o g o n a l t r a n s f o rm
20 process performed by t h e orthogonai tmns(:orni u n i t 104 and
s u p p l i oy d i f I ftrent i l-ion par alia*
predict, i on/coinpensal. i on un i t 11!J registers t.lie r&l oren^f!
image specifying in! <;rmat i on in i-fre lisl.s L0 and LI in order
25 set in I-he- VPS. Then, the process is returned to SLep S'104
represented in L'"ig. 1 4, and the process proceeds to Stop Sll 05 .
10127]
Fi.ij . 16 is a f f.owohart that ill iistrat.es the list.
registration process of Sl.cp Si 33 illustrated in Fig. 15 Ln
30 detail,
[0128]
- ^ /
"7/
42
I5P351O82W0OO
In Stop SI 51 represented in Fig. 1 h, the [notion para J lax
prediction/compensal. i.on unit 115 sets the reference i mage
specifying informati on eaoh having an image oi a view ID sma i ler
than the v i ow ID of the current predict} itg image as a reference
5 image among the reference specifying in Forma kion included in
the VPS in Ref Pi eljetTvCJurrRefore in order of highest degree
of closeness to the view in of the current predicting image
( i.n order of largest to smallest view ID) •
[0129]
10 Til Step 3152, the motion para I tax
predioti on/compensation unit 115 sets the reference linage
specify i ng in I ormal-i en eaoh hav.i rig m\ i mage of" a view ID larger
than the view ID of the current predict E ng image as a reference
image among the reference specifying J nforma Lion .i deluded in
15 the VPS in ReiPicSeLTvCurrAfter i.n order oi highest degree
of closeness to the vl.cw In of the current predict i.ng i mage
(in order or smallerb to largest view ID),
[0130]
in Step S153, Lhe motion parallax
20 prediction/compensation un i.t 115 performs registration for
the? list LO in order of ReFpicfietlvCtirrliefoce and
Ref PicSel.TvCurrAfter. iri Step S154, the motion parallax
predict ion/compi:nsaf. i on unit US pertoims registration for
the list. 1.1 in order of Refb>icSeklvCui rAftor. and
25 Re I PicSetTvCurrRofore . Then, the process is returned to Step
SI 33 represented in Fi.g. 15, the prooess is returned to Step
S1D4 represented in Fig, 1A , and the process proceeds to Seep
SI 05.
[0131]
30 As above, since a mu I t i-viewpoint rmaqe is ceded based
on the order reliab i Ilty I lag, the encoding dev ice SO can
43
SP35l0fl2RO00
p e r t o n n t h e 1 i s t r e g i s t r a t i e n p r o c e s s and Lhc p a r a I l a j ; s i d l i n g
p r o c e s s only i n a fiase whnro t h e view TDs a r e a s s i g n ed
s e q u e n t i a l l y from t h e v i e w p o i n t ol t h e curlier a di s p o s e d at t he
end. As a r e s u l t , the: r e g j j - i t u a t i o " o r d e r of r e f e r e n c e images
b [red P r e n c e pf c t u r e s of t h e v i.ewpoi ' i t d i r e c t i o n J oC v i e w p o i n ts
d i f i-etont from t h e v i owpo.i nt of the c u r r e n t p r e d i c t i n g i.mage
c a n bo c o n f i g u r e d t o b e di ("Cerent. between t h e l i s t LO and t he
l i t f t L i . Tn a d d i t i o n ; the para ( l a x s c a l i n g p r o c e s s can tie
c o r r e c t l y p e r f o r m e d . According] y , t h e c o d i n g e f"ficienoy c an
1 [1 be improved*
[0132]
in add i tion, i n d caav. where the view IDs are not assigned
sequentially from l.ho viewpoint of." the oatnora arranged at the
end, by per forming the 1i«t reg i stration process and the
lb para I lax 30a I ing process, i t can be prevented to degrade the
coding efficiency.
[013"J]
{Configuration t^xampE c of Decoding novice accord i ug to
First Embodiment)
2 0 Lb'ig* 1 I is a bf ock di arjram that iJ I ustral-^s an example
of the configuration of a decoding do vice oi a first embod iment,
to which the present technology is applied, decoding a coded
stream transmitted f r.om the encoding device 50 illnsl rated
in Fig- 4.
2b [0111]
A decoding device ?00 ill ustrated in FUj. 17 is
conf i gured by i^n extraction unit 201 and d decod i rtg uni I 2 02 .
The decoding device 200 decodes the coded stream transmitted
from the encoding device bO by using rt decoding mef.hod
30 corresponding to the encoding method used by the encoding
dev i ce 50, thereby (jonera Ling a uutlti-viewpoi nt inviqc .
44
SP351Q82WOG0
[0135]
Mote specifically, I.ho extraction unit 201 oi the
decoding device 200 serves as a exception unit and ret'oives
a coded stream. transmitted from the encoding device 50* The
5 extraction un i t 201 ^tracl.s parameter sots such as tho SPS,
the PPS, andVPS#nd coded data from thocod^d a tream esnd supplies
the parameter ^ets and the coded data to the decoding unit
202.
[01361
] 0 The decoding unit 202 decodes the coded data supplied
from the extraction unit 201 based on the ord^r. reliability
flag and the view TD included in the VFS supplied irom the
extraction unit 201 , thereljy generating a mu I ti-viewpoinf.
iniage* At th i s time, the decoding unit 202 ref Ufa to the liPS,
15 the PPS, and l.ho like as is necessary. Tho decoding tnil.t 202
outputs the multi-v iewpoinL image,
[013/J
fConfiguration fxamp I o of Decoding Unit]
Tig. 1R is a block diagram t h a t i] lugtraU;.1.; an example
20 of t h e c o n f i g u r a t i o n of t h e decodE rig uni t 202 i 1 Lustra Led in
f i g , 1 /.
[013*1]
The decoding un i t 202 illustrated i n lj'ig. 1 3 inc i udes :
an accumulation bufter 301; & lossless dt^codiny unit 302; aft
25 inverse quanti vat ion unit 303; an indorse or thogdal transform
unit 304; a oa l.culat I on un it 305; a loop filter 306; a screen
rearrangement buf fer 307; and a D/fl converter 308 , In addition,
the decoding unit 202 includes : a decoded pictu re but I or 309;
a sel eclion unit 310; an j ni ret predict! y using a
system corresponding i.o the orthogonal transform system of
the orthogonal transform unit 101 illustrated in Fig. 1,
thereby acquiri rig di I forenI ial in f'orma!. ion-
[01431
10 The differential, information that is acquired by the
inverse orthogonal transform process is supplied to the
calculation uniL 305. In addition, a predicted image is
supplied to the calculation un i t 305 J.'rom the intra predi ction
un i t 311 or the motion parallax compensate ng unit 312 through
1$ thff selection unit 313.
[0143]
The? calculation unit 305 serves as a decoding unit and
per forms decoding by add! rig the differentia t information and
the predicted image togel her, I.hereby acqul ri nga reconfigured
;>U image. The calculation unit 305 supplies the reconfigured
imago to the loop filter 30ft or the decoded picture buffer
309.
[0145]
The loop filter 306, similar to the filter 11]
?h illustrated in Fig . 7, appropriately performs a fi I Lor process
for the reconfigured image supplied I rom the calculation uni I.
305, thereby generating a decoded image. Jn addition, in a
case where a filter coefficient is acquired by the lossless
decoding process performed by t he lossless dccodl rig un i t 302,
30 the 1 oop f i \ ter 306 performs a I liter process us.i n and the FOC of the decoded
4H
yP3S1082WOO0
ifrifUjGi which a r e s t o r e d , to the: motion p a r a l l a x compensating
im i t 312 t h r o u g h l.lio s e l e c t i o n u n i t 310 at p r e d e t e r m i n e d t i m i rig
lirliaseclofiri r e q u e s t from t h e o u t . s i d e s u c h a s t h e motion p a r a l I ax
p r e d i c t i o n / c o m p e n s a t i o n , u n i t 1 15.
5 [0150]
The I n t r a p r e d i c t i o n u n i t 311 b a s i c a l l y performs Lhe
same p r o c e s s a? t h a t of the i n t r a p r e d i c t i o n u n i t li1*
i l l u s t r a t e d i n Fig. 7. However, the i n t r a p r e d i c t i o n u n it
311 performs an i n t r a p r e d i o l . i . o n oil the opLimal I n t ra
10 p r e d i c t J on mode of t h e i n t r a p r e d i c t ion i n l o m i c t i o n suppJ lod.
("torn t h e [ o s s l e ^ s d e c o d i n g un i t 302 o n l y f o r an a r e a in wh \.ch
Lhe p r e d i c t e d image is g e n e r a t e d by Lhe i n t r a p r e d i c t i o n at
t h e time of t h e c o d i n g p r o c e s s . The i n t r a p r e d i c t i o n u n i t
31 1 suppl i es t h e p r e d i c t e d image a c q u i r e d as a r e s u l t of Lhe
1 5 i n t r a p r e d i c t i o n t o t h e s e l e c t i o n u n i t 3 1 3.
[01511
The motion p a r a l J ax compens^Ll rig u n i t . 312 s e r v e s as a
s e t t i n g u n i t and, s i m i l a r to Lhe mot.Eon p a r a l l ax
p r e d i c t i o n / c o m p e n s a t i o n unit. 115 ii t u s t r a l . t a i in Fig. 7,
20 r e g i s t e r s Lhe r e f e r e n c e 1 mage s p e c i f y i ng i n f o r m a t i o n i n c l u d ed
i n the VPS i.n t h e l i s t ( r e f e r e n c e l i s t ) based on (he o r d er
r e l i a b i l i t y f l a g included in Lhe VPS s u p p l i e d from t he
e x t r a c t i o n u n i t 201 i l l u s t r a t e d LrtJTicf, 17. Then, (.he m o t i on
p a r a l l a x ooiupensal i.ng un i t 312 reads images s p e c i f i ed by t he
25 r e f e r e n c e i mage s p e c i f yi nq i n f o r m a t i o n r e g i s t e r e d i n t h e l i s t
f. rem t h e decoded p i c t u r e b u f f e r 309 in o r d e r of r e q i strr at. i on
i n Lhe l i s t as re I e r e i i o e images -
£0152]
I n a d d i t i o n , t h e m o t i o n p a r a l l a x c o m p e n s a t i n g u n i t . 31 2,
30 s i m i l a r to Lhe mo I ion p a r a l l a x p r e d i etion/coKipensa t i o n u n i t
115, g e n e r a t o s a p r e d i c t i on veeJ or of a m o t i o n p a r a I 1 ax v e c t o r
49
SP3S1082WO00
corresponding to the pred i oted image gene r a ted ' n Lht? opL ima J
inter predict! on mode oi. the inter predict i on i n I or ma I. i on
supplied from the lossless decoding unit 302 based on the order
reliability flag. In addition, the motion paraMax
5 compensating unit 312 generates a motion parallax vector by
adding the generated prediction vector and the moti on parallax
vector information of the inter prediction information
supplied from the 1ossless decoding um t 30? Lo^ethcr.
[0153]
10 The motion parallax compensating unit 31? performs a
compensation process of the optimal inter pred i oti.cn mode
basically using the PU as a processing uni 1. based on the
generated motion parallax vector and the reference image,
thereby generating a predicted image . In addition, the motion
15 parallax compensating unit 312 performs a compensation process
only for an area for which the inter prediction is performed
at the time of the coding process based on the inter predi ot i on
information supplied from the lossless decoding unit 302 . The
motion parallax compensating unit 312 supplies the generated
2 0 predicted image to the calculation unit 305 through the
selection unit 313.
[0154]
The selection unit 313 supplies the predicted image
supplied from the intra prediction unit 311 or the predicted
25 image supplied from the motion parallax compensating smi 1. 31 ?
to the calculation unit 305.
[0155]
While the decoded picture bul f er 309 stores the decoded
image and the reconfigured image of the current jirocowsinq
30 viewpoint and the view IDs and the POCs of (.lie decoded uiiagc
and the reconfigured imaqe, the mu I I. i -v 1 ev/po ml decoded
bO
3P351I0S2WO00
picture buf f er 321 stores the decoded imago of (?ach vi cwpoi nt
and the view TDs ?nit} the POCs of the decoded images. Tn other
words, theme l.ti-v i.owpo i nt decodedpf cturebuf f er/321 acquires
the decoded images supplied i r.om the decoded picture buffer
5 309 and the view TDs and the POCs of the decoded images arid
storey the decoded intaqes and the view TDs and the E^OCs of
the decoded images together with the decoded, picture? buffer
309,
[0156J
10 When the current processing viewpoint is changed, the
decoded picture buffer 309 eliminates the decoded image of
the previous processing viewpoint, the mult i -viewpoint.
decoded picture buffer 321 ma i ntainy the decoded image of the
previ yus processi rig viewpoint as i t is - '1'hen, according to
15 a request from the decoded picture buffer 309 or the like,
the decoded image and the view ID and the POC oT the decoded
image, which are stored, are supp] i od to the decoded picture
buffer 309 as "a decoded image of a non-eurrent processing
viewpoint and the view TP and the POC of the decoded image".
20 The decoded picture buffer 309 supplies "'the decoded imaqe
of a non-curn;nt process i rig viewpoint and the v iew ID and the
POC of the deceded j mage" read i rom the mult i -viewpoint decoded
picture buffer 3211 to the motion parallax compensating unit
312 through the seIecti en unit 310,
25 [01571
(Description of Process Perlormed by Decoding Device-}
Fig, 19 is a flowchart that i l l u s l r a t e s an image
genera t i o n process perf ormed by the decoding devj co 200
i l l u s t . s a t e d in Fig . 17 .
30 [0i5fif
Tn Step 5170 represented in Fig. 19, the ex traction unit
bl
SP351082WO0G
201 of tho decoding device 200 n>ci>iu^^ a coded stream that
is transmitted frora the encoding device h(i. In Step SI71,
the extraction unit 201 extracts dn SPS l rom the coded stream
and supplies the extracted SPS to the decoding unit 202. In
5 Step 13172, the extraction unit 201 extracts a PPS from the
coded stream and suppli OH the extracted PPS to the decoding
unit 202. In Step SI 73, the extraction unit 201 extracts a
VPS including the view ID and the order re I i abi I i.ty iil ag from
the coded stream and supp I i es the extracted VPS to the decoding
10 unit 2 02.
[0159]
in Step 5174, tho extraction unit 201 extracts coded
data from the coded stream and supplies the extracted coded
data to the decoding unit 202. In Step S175, the decoding
15 unit 202 pot forma a decoding process for decoding the coded
data supp I i ed £ roiu the extraction unit 201 based on the order
reliability flay and tht; w i «w ID included in the VPS that is
supplied from the extraction unit 201, and the process ends.
[0160]
20 L-'iq. yQ in a flowchart that illustrates the decoding
process or Step S"l7b i 1 i net rated in Fig. 19 in detail.
[0161J
In Step S i00 represented i n Fig .20, the motion parallax
compensal-inq unit 31? performs a setting process similar to
25 the yetflrii] process illustrated in Fig. IS. In Step S301,
the accumulation buMor .iQl accumulates the coded data ol" a
predeterm i ned v i ewpoinf that i s supplied from the extraction
unit 201 illustrated in Fiq, 17 and supplies the coded data
to the lossless dei:od i rnj unit 302 at fj rede term i ued Lira i mj.
30 [0162]
i n t j t e p S30?, the E o s s f e s y deeocjirn; u n i t 302 [ l e r l o i r ns
SP351082WO00
Lossless deoodinq of I.IK; coded data supplied irom the
accumulation bul i.er 301 us i nq a .-jysLom corresponding to the
codinq system of the Loss I ess encod i rtq uii i L J 06 i. Llustrated
in Fig . 7 . The 1 ossl ess deood i nq uri i I. 302 ^uppl I es a quantized
5 transform coef tioient aoqui red by l-ht: I oss I IMS& decoding
process to the inverse quanl.i /;a I. i on vm I L 303 together with
the view ID and the POC of. I.he i maqe correspond i.nqj to the
trans form ooe I l"i ci en I.,
[0163[
10 Tnaddt (ion, i na case where i ntra production information
is acquired by decodinq I.he cod^d data, the lossless decoding
unit 302 supplies L.he intra pred i ot.i on i n I orrtia t i on to the intra
prediction unit 311. On (.he oilier hand, i n a esse whore inter
prediction information is acqui red, l.he I oss I essdecod i ngun.it
15 302 supplies the inter prediction 5 n( onnal. i on to Uie motion
parallax compensating unit 312.
[0164]
In Step 3303, the inverse quantizat i on unf L 303 performs
inverse quantization of the quanl. i zed transform coeiTi cient
20 supplied from the lossless deoodi nq un i I. 30^ UH i IKJ a yys torn
corresponding to the quantizahi on system of 1.1 if: quanl. i nation
unit 105 illustrated in Fig. 7 and snppl i es an aequ i.red
transform coefficient to the inverseorl.hoqona 1 trans form unit
30^ . In Step S304, the inverse orthoqona I traiiy form unit J 04
25 performs an inverse orthogonal l.rans I orm of Liie transform
coefficient supplied from Hit: inverse; quantization unit 303
by using a system correspond! nq l.o the ori.hogona.l transform
system of the orthogonal transform unit 104 illustrated in
Fig. 1, thereby acquirinq d i I ( erent i al information. The
30 di (:ferential informal. i on ^cqu i \<-t\ by the inverse orthogonal
transform is suppliod |.o the calculation unit 305.
b3
SP3S1 082WO00
S0165J
In Step S3 05, the intra predict i on nni t 311 determines
whether or: not the intra prediction informs Lion is supplied
Trotfi the lossless decod' rig un.i I- 302 . Tna case where the intra
5 prediction information is detetmined to be supplied in Stop
3305, in Step S306, the intra prediction unit 311 performs
an intra prediction that is basically the same as that of the
intra prediction unit 114 il l.ustratod in Fig. 1. The intra
prediction unit 311 supplies a predicted image acquired as
10 a result of the intra prediction to the se Lection unit 313,
and the process proceeds to Step 5i309.
[0166]
On the other hand, in a case where the intra prediction
information is determined not to be suppl.ied in Step S305,
15 i n other words, i n a case where thomotj on para f J ax compensatE ng
unit 31? acquires the in terpred i ction i n forms t ion, 1. ho process
proceeds to Stop S307. in Step 3301, the motion parallax
oompensa!. ing uni t 312 generates a motion par. all ax vector
corresponding to the predicted image generated in 1 he optimal
20 i nter prediction mode ol the inter pred > ction mode inf ormat i on
that is supplied from t he lossless decoding unit 302.
[0167]
In Stop S30ri, the motion paral] ax compensating unit Til 2
performs a compensation process of the optimal i nter predict i on
25 iiindo basically using the ^Ll as a processing "nn.it based on the
qonerated motion para I tax vector and the reference ieiiaqo,
thereby general. E ng a predicted image. The motion parallax
conipensrt L ing un i t 312 supplies the generated predicted j mage
to the calculation unit 305 through the selection unit. 313.
30 [016G)
Ti\ Step S309, the calci? 1 ation unit 3U5 adds the
S3
SP351Qa2tf>O00
dif f erential i nformat ion suppl led from the inverse orthoqonal
transform uni I. 304 and the pred feted .image suppl icd from the
selection uni !. 313 together, thereby acquiring a rroconl i qured
image. The o$ l.culati on unit 305 supplies the iroconf i qured
5 image to the loop filter 306 or. the decoded picture buffer
309.
f03 69J
in Step S310, Lhc loop r i Iter 306, simil av to the filter
111 i llustral.ed in Fiq, 7, appropriately per f ornts a filter
10 process for the reconfigured image supplied from the
cd I epilation UEiit 305, thereby qenerat i tig a decoded irnaqe, In
addition, in # czis& where a I i.Iter coefficient is acquired
by the lossless decod i.ng process performed by the lossless
decoding unit. 302, the I oop fill, err 306 performs a J" titer process
15 by us i rig the f i I tor coefficient.. The loop filter 306 supplies
the decoded image that, is a result of the filler process to
the screen rearrangement buflor 307 and the decoded picture
buffer 309.
[01V0J
20 in Step S311, l.hc screen rearrangement buffer 307
rearranges the decoded image supplied from the loop ( i Iter
306. In Step SJ12, the U/A converter J06 performs a D/A
conversion of the decoded imjge ef a itiul ti-viewpoint that is
suppl i ed from the screen rearrangement buffer 30 / and outputs
^5 a im.il I. i ,jviewpo i nt imaqe acqui red as a result thereof to a
display not i F lustrai.ed in the figure so as to be displayed
thereon-
[0-|Vl[
in Step S313, the decoded picture buffer 309 stores the
30 reconfigured 3 maye supplied from the ca I eulati on unit 305 and
the v 1 <:\J ID and the I'OC of" the reconfigured image, in add i.tion,
S!?3510G2WO00
tho docoded picture bi>[ I er .}GQ and the multi-viewpoint decoded
picturo bni.ler ?>?~\ store the decoded image and the view ID
and the POC ol I.he decoded image that are supplied from the
loop i"i ll.er 306* Furthermore, the decoded picture buffer 309
5 toads and stores the decoded image and the view ID and the
POC g(' the decoded image, which are stored in the
Klul l.i-viewpoint decoded picture buffer 321, as "the decoded
imago oi: a non—current processing viewpoint and the view ID
and the POC of the decoded image".
10 [Q172J
The rftconli qured image and the view ID and the PCC of
the rooonf1 i qured image, which are stored in the decoded picture
but ("or 309, are supplied to the intra prediction unit 311
throuqh the selection unit 310 . In addition, the decoded image
lb and the view ID and the POC of the decoded image, which are
stored in the decoded picture buffer 309, are supplied to the
motion parallax compensating unit 312 through the selection
unit 310. Then, the process is returned to litep liL7ri
represented in Fig. 19, and the process ends,
?{) LOmi
As above, since the decoding device 200 receives the
order reliability flag and decodes a multi-viewpoint image
liased on the order reliability flag, a coded stream transmitted
i rom the encoding device 50 can be decoded.
7^ fOVM]
i s inct uded
in the VPS. In addition, as illustrated in the 11th row, a
20 l i s I. flag {inter_ vifv/_def atj I t_rel «r;cnce_l lag) i s included in
the VPS. The l i s t i lag is "J." in H, case whore .i L represents
thai, the list., registration process is performed and is "0"
in a case where it represents that the List recjistrati on process
is not performed.
25 [0181]
In addition, as illustrated in the 12th row, a scaling
flag (inter view scaling 1 lag) is included in l.ho VPS. The
sea I ing flat} is "1 " in a ease v'bero it represents that the
parallax scaling process is performed and is "0" in a case
30 where it represents that the para J l.ax sea I ing process is riot
pe r formed.
s a
I5P351032WO00
[0132]
F u r t h e r m o r e , as j f l u s t r a ( . o d in t h e 13Ui row l.o t h e i l> th
row, in tlx; VPS, s i m i l a r to th.
[0139]
25 In addition, in a case where a selection i.y notified
from the predicted imago selecting un i t 116, the mo I. toil
parallax pr.odicl. i on/compensation unit A1 1 generates a
prediction vector of a motion parallax vector corresponding
to the predicted image generated in the optj mal inter
30 predict i i^n mode based on the serf I trig f 1 sq . Then, the motion
parallax predict Ion/compensation unit 411 ca J cul^Les a
60
SP3510G2WOD0
difference between the generated prod i eL i on vector and the
mgLion parallax vector correspond, i ttq fc.o t.ho predicted image
generated in the optimal inter prediction mode and sets the
calculated difference as motion para. I l^x vector information.
b Tn addition, the motion parallax prod i cl. i on /compensation unit
411 supplies the inter prediction 1 n I orinati on including the
motion parallax vector informal.i on, Uie opl:imal inter
prediction mode, and the like to the I oas 1 t^s encoding unit
106 so as to be losslessly coded*
0 101903
(Description of Process Performed by Encoding Device)
Fig. 24 is a flowchart that i I Susl.ral.es a generation
process performed by tho encodyno. device 400 illustrated in
Fig. 21.
•5 [0191]
The process of Stops S.J31 and S332 illustrated in Fig.
7A is similar to that of Steps 31 1 and SI 2 LI lust rated in Fig.
13, and the description thereof will not be presented.
[0192]
2 After the process or SlepSiU?, instep 5333, the setting
unit -101 sets a VPS f.hal includes a view TD, a scaling flag,
and a list flag, Tho sel-L i ruj nn 1 I. 40"! supplies parameter sets
such as tho SPS, the PPS, and i.he VPS to the encoding unit
102.
i [0193]
I n litep S334, the encoding u n i t 40? performs a coding
p r o c e s s in which amu I V i -v i t:wpo i nl. i iitaq^ i n p u t from t h e o u t s i d e
i s coded u s i n g a 3D coding system based on the vi.eu TD, the
s c a l i n g f l a g , and t h e l i s t V\nq included in I.he VPS t h a t is
J s u p p l i e d from t h e s e t t i n g uni.t 4U1 - Do |.a i Is o f (.hi s cod 1 no,
p r o c e s s a r e t h e same as t h e s e of I:he codi TH] proof?:-! S i I I iisl.ra t.t:d
61
Sfc>35I082WO00
in Fig . 14 e x c e p t for a s o t t i n g p r o c e s s of S t e p S104 i l l u s t r a t e d
i n Fi g . 14. Thus, only t h e d e t a i l s of t h e s o t t i n g p r o c e ss
wi I.I bo d e s c r i b e d w i t h r e f e r e n c e to L'Mq. 25 to bo d e s c r i b ed
l a t e r .
5 [0194]
I n S t e p S33b, the encoding u n i t 402 g e n e r a t e s a coded
s t r e am by addi rig the p a r a m e t e r s o t s suppl. i ed from t h e s e t t i n g
u n i t 401 to the coded d a t a accumulated in t h e a c c u m u l a t i on
b u f f e r 10/ and l.raiisnii t s the g e n e r a t e d coded s t r e a m . Then,
10 the p r o e e s s e n d s.
[0195 J
Fig, 2 b i s a I I ov/ehart t h a t i 1 l u s t j . a l e s a s e t t i n g p r o c e s s
in a coding p r o c e s s of Stop S334 i l l u s t r a t e d in h'ig, 24 in
d o t a l i.
15 £0196)
Tn Step S3 51 represented in F"i q . 9 b, the mot. ion, pa fa 1 lax
prediction/compensation unit 411 determines whether or not
the scaling flag included in the VPS suppl \ ed I rom l.ho set-Li ng
nn i I. 401 is *'l" . Tn a case where l.ho sea \ i ng f \ ag is do term J ned
20 l.o be "1" in Step S351, in Step S.15?, the motion parallax
predi otion/oompensation unf I 4"l 1 val i dates \M%> para I lax
scaling process, and the process proceeds l.o Step S354.
[0197]
On the other hand, i n a case where I.lie sea I i rig I \ ag is
25 determined not to be ul" in Step S^51 , in Stop 5353, the motion
parallax predi ci.i on/eomperisa t i on unit 411 Invalidates the
parallax scaling process, and l.ho process proceeds to Step
S."554.
lOTJHl
30 Tn Step S354, I he metJon parallax
predi ct i on/comperisa11 on unit 411 determines whether or not
62
SP35I0BPWO00
the list. J lag included in the VPS thai: is supplied from the
sotting urt.it 40"J is vl" - In a case where the list J I ag is
determined to be r'l" in Step S3M, in Step S3bb, the motion
parallax predict ion/conipensatiod unit 411 performs the list
5 rogistral: i on process il I ustrated in Fiq, 16 . Then, the
setting process ends.
[0159]
On the other hand, in a case where the list Hag is
determined not to be "I" in Step S354, in Step S3b£*, the motion
10 par all ax predict! on/oorkipensal.3 on un.i L 411 registers the
reference image specifying information in the lists L0 and
LI in order of setting Tor the VPS. Then, the setting process
ends .
[0200]
15 As above, since the encoding device 400 codes a
multi-viewpoint image baaed on the list Hag and the scaling
flag^ on I y in a case when? I he view IDs are assigned sequentially
I rom the vi ewpoinl. of the camera arranged at the odd, the list
registration process or the parallax scaling process can be
20 performed. As a result thereof, the registration order of
reference images (reference pi clures ol the v lt?wpoi tit
direction] of viewpoints different from that of the ovurent
predicting image can be configured to be different lintueen
the lists 1.0 and Li - In addition, the para i.lax scaling process
2£j can lie correctly performed. Accordingly, the coding
officiency can bo improved.
10201]
In addition, in a ccivt; where the vi cw IDs a re not assigned
sequent ia 1 ly from the vi ewpoint of the camera ai ranged at the
30 end, by perform E rig the J ist req 1st ration process Ai\t;{ the
parallax scaling process, it can be prevented l.o degrade the
6."j
SP351082WOQ0
cod J ng ef f E c:ienoy.
[0202f
[Conf Lgural. ion Example of Decoding Device accord Lng to
Second liSnfoodiiaen t.J
5 fig. 26 is a block diagram that, illustraf.es an example
of the configurati on of a decodi rigdev i ceof a second embodiment,
to which the present technology is applied, decoding a coded
stream transmitted from the encoding devi.ee -100 illustrated
in Fig. 21.
10 [0203]
Among the configurations illustrated in fig. 26, the
same reference numeral is assigned (o the same configuration
as that illustrated in fig. If. Duplicate description wilt.
not bo presented as is appropriate.
15 [0204j
The configuration of a decoding device 500 illustrated
in Ffg , 2 6 is dill orent from that of the decod.i rig dev i ce 200
illustrated i u fig . 17 in that A decod i rig un i t 501 i.s arranged
instead of the decoding unit 202. The decoding device 500
20 decodes the coded stream transmitted from the enoodinq devioo
400 by using a decoding method corresponding to the encoding
method used by the encoding device 400, thereby generating
a mult, i -viewpoint Image.
[020M
2C:> More specifically, the decoding unit. ^01 of the decoding
device 500 general.os a multi-viewpoint image by decoding the
coded data supplied from the ext raction uni t 201 based on the
l i s t ("lag, tlio scaling I Lag, and the view Tn included in I.lie
MPS supplied Crom the extraction unj 1.201. At this t i me, the
iQ decotf I rig unit. 501 refers to the UPS, the PPIi, emd the like
as i M necessary. rLLhenf the decoding uni |. 5011 outputs tho
64
5P351Q82MOG0
mu J t i - v i e w p o i nt iviiaqt:,
(0206]
{Confi j u r a t i o n ftxamp.l e of Decoding On i t}
P i g . 2. / is a hi ocfc d i a q r ym t h a i , i l l u s f . r a t c s an example
5 ol the c o n f i g u r a t i o n of t h e d e c o d i n g u n i t 501 i 1 l u s t r a bed in
Fi q . 2 6.
[0207]
Amonq the configurations illustrated in Fig. 27, the
same reference numeral is assigned to the same configuration
10 as that il] titrated in Fig. IS. Duplicate description will
not tie presented as is appropriate,
[020£]
The configuration of the decoding unit bUl illustrated
in Fig. 27 is different from that ol the decoding unit 20?
15 il I ustrated i n l-"ig. 1 8 in that a motion parallax compensating
unit 511 is arranged instead of the motion parallax
compensating unit .31 2 .
[0209]
The motion parallax compensating unit M l , similar to
20 the motion parallax prediction/compensation unit 411
illustrated Mi fig. 23, registers the reference imaqc
spe<: i iiying in formation included in the VPS in the list based
on the list Mag included in (.he VPS that is supplier! from
the extraction unit ?Ql illustrated i n Fig. 26. Than, the
25 motion parallax compensating unit 511 reads images specified
by the reference image speciIying information regi atered in
the I i st in order of req i strati on for the list I rom the decoded
pit-Lure buffet 309 as reference imager.
E0210]
30 In add f tion, the motion parallax compensating unit 5"11 ,
similar to the motion jiaralla* predi ction/oonipensai-ion un i t
65
SP351032WO00
•111 , genera Los a prediction vector ofamoi.i on para I lax veclor
fiorrcspondi Jig to the predicted image generated in I.tic opti mal
inl.ef prediction mode of Lho inter prediction information
supplied from the lossless decoding unil. 302 bribed on the
5 scaling flag, Then, the motion parallax compensating unit
511 adds the generated prediction vector i3]
The process of Steps S370 to $312 i I lustra Led in Kig.
30 2R is similar to thai of Steps SI 70 to SI 12 illustrated in
Fig, 19, and the description thereof will not be presented.
6(>
SP"J510G2ffOO0
In Step S373, the extraction unit 201 uKtracU a VPS (.hat
includes the vi ev/ ID, the list f t ag, and the scd I i ?tg flag from
the coded stream and supplies the oxtracLcd VPS to the decoding
unit bOl,
5 [0214]
In Step S374, the extraction unit 201 extracts eroded
data from the coded stream aitd supplies the extracted coded
data to the decoding unit 501 , in Stop S375, the decoding
unit !?0l performs a decoding process i o*. decoding the coded
10 data supplied from the extraction un.i I. 201 based on the view
ID, the list flag, and the scaling flay included in the VPS
that is supplied from the extraction unit 201, Details of
this decoding process are the same as those ot the decoding
process illustrated in Fig. 20 except that a setting process
f b of Step S3 00 represented i.n 1'ig. 20 replaces the setting process
similar to the setting process represented in Fiq . 25 . After
the process ol Step AJ75, the process ends.
[0215]
As abovtf, sinc£ the deeod.i rig device 500 j.oceives the
20 list M ag and the seal 3 ng flag and decodes a mu I ti-viewpoint
image based on the list ["lag and the sea Iing f I ay, the coded
stream transmitted from the encod i ng devi ce 400 caFt be decoded.
{0216 J
.rst image, the second image,
and the current decoding image such thai: values ai:o
seguenl. Lally increased from the viewpo i nL present most in the
'50 first direction .
120
SP3510fl^WO00
I I . Tin? d e c o d i n g d e v i c e a c c o r d i n g to c l a im 9, wheroln t he
i s e t t i n t j u n i t s e t s t h e l i r n t image s p e c i f y i n g i n f oriiiri I. i.on in
t h e f irtf(. r e f e r e n c e l i s t and t h e second r e f e r e n c e l i s t in o r d er
of l a r g e s t to smr-Qlest Vd 1 ue of t h e 'i d o n t i f i c r f l i o n i ni o r m a t i on
and set.-; t h e second image s p e c i f y i i\q i n f o r m a t i o n i n t h e f irsL"
r e f e r e n c e l i s l . tind the second r e f e r e n c e List i n o r d e r n[
s.mal I {'.at to lrt r t f o s t va 1 no of t h e i d e n t i f i c a t i o n i n formal, i on.
12 . A decodi rig method using' a d e c o d i n g d e v i c e , the d e c o d i ng
ine l.hod compr i s i n g :
a s e t t i riq s t e p of s e t l.i nq f i r s t i rnage s p e c i f y i rig
i n f o r m a t i o n s p e c i f y i n g a f i r s t i.mage of a v i e w p o i n t p r o s en I
in a \ ^.£3t d i r e c t i o n fresa a. v i ^.wpoiviL of a cui rant decoding
image and second imsqe s p e c i f y i n g i n f o r m a t i o n s p e c i T y i n g a
second image of o\ v i ewpoint p r e s e n t i ri a second di r e c t i or t
oppos i to to t h e f i r s t d i r e c t i on in a j i <_-yt xe I c r e n » s I i s t in
f i r s t o r d e r and s e t t i rig t h e ti r s t image: s p e c i I yi.ng J n format: i on
&nd t h e second image s p e c i f y ' EKJ i n t o n a t i o n in a second
r e f e r e n c e l i s I Ln second o r d e r t h a t is d i f f e r e n t from t h e I i r,st
o i d e r ; and
a decod i rig s t o p of d e c o d i n g coded d a t a oi the c u r r e nt
decod i rig image based
Documents
Application Documents
| # | Name | Date |
|---|---|---|
| 1 | 4798-delnp-2015-Form-1-(15-06-2015).pdf | 2015-06-15 |
| 2 | 4798-delnp-2015-Correspondence Others-(15-06-2015).pdf | 2015-06-15 |
| 3 | 4798-DELNP-2015.pdf | 2015-06-16 |
| 4 | Other relevant documents.pdf | 2015-06-24 |
| 5 | GPA.pdf | 2015-06-24 |
| 6 | Form PCT-IB-304.pdf | 2015-06-24 |
| 7 | Form 5.pdf | 2015-06-24 |
| 8 | Form 3.pdf | 2015-06-24 |
| 9 | Form 2 + Specification.pdf | 2015-06-24 |
| 10 | Drawings.pdf | 2015-06-24 |
| 11 | 4798-delnp-2015-Form-1-(09-07-2015).pdf | 2015-07-09 |
| 12 | 4798-delnp-2015-Correspondence Others-(09-07-2015).pdf | 2015-07-09 |
| 13 | 4798-delnp-2015-Form-3-(09-10-2015).pdf | 2015-10-09 |
| 14 | 4798-delnp-2015-Correspondence Others-(09-10-2015).pdf | 2015-10-09 |
| 15 | 4798-delnp-2015-Form-3-(17-02-2016).pdf | 2016-02-17 |
| 16 | 4798-delnp-2015-Correspondence Others-(17-02-2016).pdf | 2016-02-17 |
| 17 | Form 18 [20-10-2016(online)].pdf | 2016-10-20 |
| 18 | 4798-DELNP-2015-FER.pdf | 2019-10-24 |
Search Strategy
| 1 | 2019-03-1316-22-53_13-03-2019.pdf |