Entitled: encoding device and encoding method, decoding apparatus and decoding method
Technical field
[0001]
　This technique, coding apparatus and coding method, a decoding apparatus and decoding method, in particular, the encoding device and encoding method to be able to improve the coding efficiency of parallax images using information on the parallax image , on the decoding apparatus and decoding method.
Background technique
[0002]
　Recently, a 3D image is focused, the coding method of parallax images used to generate the 3D image of multi-view has been proposed (e.g., see Non-Patent Document 1). Incidentally, the parallax and the parallax image, representing the each pixel of the color image of the view corresponding to the parallax images, corresponding to the pixel, the horizontal distance position on the screen of the pixel of the color image of the view to be a base point is an image of values.
[0003]
　Further, now, as AVC (Advanced Video Coding) in order to improve the further coding efficiency than scheme, HEVC (High Efficiency Video Coding) and has been advanced standardization of coding method called, August 2011, Draft as a non-patent document 2 have been issued.
CITATION
Non-Patent Document
[0004]
非特許文献1 : "Call for Proposals on 3D Video Coding Technology",ISO/IEC JTC1/SC29/WG11,MPEG2011/N12036,Geneva,Switzerland,March 2011
非特許文献2 : Thomas Wiegand,Woo-jin Han,Benjamin Bross,Jens-Rainer Ohm,GaryJ.Sullivian,"WD3:Working Draft3 of High-Efficiency Video Coding",JCTVC-E603_d5(version5),2011年5月20日
Summary of the invention
Problems that the Invention is to Solve
[0005]
　However, the coding method for improving the coding efficiency of parallax images using information on the parallax image has not been devised.
[0006]
　This technology has been made in view of such circumstances, it is to be able to improve the coding efficiency of parallax images using information on the parallax image.
Means for Solving the Problems
[0007]
　Encoding apparatus of the first aspect of the present technique, by using the information about the parallax image of the reference viewpoint, and a correction unit that corrects the predicted image of parallax images of the reference viewpoint, the predicted image corrected by said correction unit using an encoding unit for encoding the parallax image of the reference viewpoint, and a transmission unit for transmitting the information about the parallax image of the reference viewpoint parallax image of the reference viewpoint encoded by the encoding unit is an encoding device comprising a.
[0008]
　Coding method of the first aspect of the present technology corresponds to the coding apparatus of the first aspect of the present technology.
[0009]
　In the first aspect of the present technique, by using the information about the parallax image of the reference viewpoint is corrected predicted image of parallax images of the reference viewpoint, using the corrected said predicted image, the parallax image of the reference viewpoint is encoded, the information and are transmitted about parallax image coded the reference viewpoint parallax image and the reference viewpoint of.
[0010]
　Decoding apparatus according to the second aspect of the present technique, the parallax image coded the reference viewpoint using a predicted image of parallax images of the reference viewpoint corrected using information on parallax image of the reference viewpoint, the a receiving unit that receives information about the parallax image of the reference viewpoint, using information on parallax image of the reference viewpoint received by the receiving unit, a correcting unit that corrects the predicted image of parallax images of the reference viewpoint, the using the predictive image corrected by the correction unit, received by the receiving unit, a decoding device and a decoding unit for decoding the parallax images coded the reference viewpoint.
[0011]
　Decoding method of the second aspect of the present technology corresponds to the decoding device of the second aspect of the present technology.
[0012]
　In a second aspect of the present technique, the parallax image coded the reference viewpoint using a predicted image of parallax images corrected the reference viewpoint using information on parallax image of the reference viewpoint, the reference viewpoint is received and information on the parallax images, using information about the parallax image of the reference viewpoint received, the predicted image is corrected parallax image of the reference viewpoint, using the corrected said prediction image, coding parallax images has been the reference viewpoint is decoded.
[0013]
　Incidentally, the decoding device of the encoding device and the second side of the first aspect can be realized by executing the program in the computer.
[0014]
　Further, in order to realize a decoding apparatus of an encoding apparatus and a second side surface of the first side, a program to be executed by a computer, by transmitting via a transmission medium or by being recorded on a recording medium, it is possible to provide.
Effect of the invention
[0015]
　According to a first aspect of the present technology, it is possible to improve the coding efficiency of parallax images using information on the parallax image.
[0016]
　According to the second aspect of the present technology, it is possible to decode the encoded data of the parallax image coding efficiency is improved by coding using information on parallax images.
Brief description of the drawings
[0017]
Is a block diagram showing a configuration example of the first embodiment of FIG. 1 coding apparatus according to the present technique.
It is a diagram illustrating a disparity maximum value and the parallax minimum value of [2] viewpoint generating information.
It is a diagram illustrating a parallax accuracy parameter [Figure 3] viewpoint generating information.
It is a diagram illustrating a camera distance of [4] viewpoint generating information.
Is [5] a block diagram illustrating a configuration example of the multi-view image coding unit of FIG.
6 is a block diagram showing a configuration example of the encoding unit.
Is a diagram illustrating a configuration example of FIG. 7 encoded bit stream.
[8] is a diagram showing an example of the syntax of the PPS in Figure 7.
9 is a diagram illustrating an example of a syntax of a slice header.
It is a diagram illustrating an example of the syntax of FIG. 10 slice header.
FIG. 11 is a flowchart for explaining the encoding process of the encoding device of FIG. 1.
12 is a flowchart illustrating the details of the multi-view coding process of FIG. 11.
13 is a flowchart illustrating the details of the parallax image encoding process in FIG. 12.
14 is a flowchart illustrating the details of the parallax image encoding process in FIG. 12.
It is a block diagram showing a configuration example of the first embodiment of FIG. 15 decoding device according to the present technique.
It is a block diagram showing a configuration example of the multi-view image decoding unit of FIG 16 FIG 15.
17 is a block diagram showing a configuration example of the decoding unit.
18 is a flowchart for explaining a decoding process of the decoding apparatus 150 of FIG. 15.
19 is a flowchart illustrating the details of the multi-view decoding process of FIG. 18.
FIG. 20 is a flowchart illustrating the details of the parallax image decoding process in FIG. 16.
21 is a diagram for explaining a method for transmitting information used for correction of the prediction image.
22 is a diagram illustrating a configuration example of a coded bit stream in the second transmission method.
23 is a diagram illustrating a configuration example of a coded bit stream in the third transmission method.
Is a diagram illustrating a configuration example of an embodiment of FIG. 24 computer.
Is a diagram showing a schematic configuration example of FIG. 25 the television apparatus embodying the present technology.
FIG. 26 is a diagram showing a schematic configuration example of a mobile phone according to the present technique.
FIG. 27 is a diagram showing a schematic configuration example of a recording and reproducing apparatus according to the present technique.
[FIG. 28] is a diagram showing a schematic configuration example of an imaging device according to the present technique.
DESCRIPTION OF THE INVENTION
[0018]
　
　[Configuration example of a first embodiment of the coding device]
　FIG. 1 is a block diagram showing a configuration example of a first embodiment of the applied coding apparatus to which the present technology.
[0019]
　Encoding device 50 of FIG. 1, a multi-view color image the imaging unit 51, the multi-view color image correction section 52, a multi-view parallax image correction section 53, point of view generation information generation section 54, and the multi-view image coding unit 55 constructed.
[0020]
　Encoder 50, a parallax image of a predetermined view point, is encoded with information about the parallax image.
[0021]
　Specifically, multi-view color image pickup unit 51 of the encoding device 50 captures a color image of multi-view, and supplies the multi-view color image correcting unit 52 as a multi-view color image. Also, multi-view color image pickup unit 51 generates an external parameter, the disparity maximum value, and disparity minimum value (the details will be described later). Multi-view color image pickup unit 51, the external parameters, the disparity maximum value, and supplies the parallax minimum value to the viewpoint generation information generating unit 54, supplying the disparity maximum value and the parallax minimum value multiview parallax image generator 53 to.
[0022]
　The external parameter is a parameter that defines the horizontal position of the multi-view color image pickup unit 51. Also, the disparity maximum value and the parallax minimum value, respectively, the maximum value of the disparity values ​​on the world coordinate can take the multi-viewpoint parallax image, the minimum value.
[0023]
　Multi-view color image correcting unit 52, with respect to multi-view color image supplied from the multi-view color image pickup unit 51 performs color correction, brightness correction, distortion correction, and the like. Thus, the focal length in the horizontal direction (X direction) of the multi-view color image pickup unit 51 of the multi-view color image after correction becomes common to all viewpoints. Multi-view color image correcting unit 52 supplies the multi-view color image after correcting the multi-view parallax image generator 53 as a multi-view correcting color image to the multi-view image coding unit 55.
[0024]
　Multiview parallax image generator 53, based on the disparity maximum value and the parallax minimum value supplied from the multi-view color image pickup unit 51, the multi-viewpoint corrected color image supplied from the multi-view color image correcting unit 52, the multi- to generate a parallax image point of view. Specifically, multi-view parallax image generator 53, for each viewpoint of the multi-view (reference viewpoint), the multi-view correcting obtains a disparity value for each pixel from the color image, the disparity value disparity maximum value and the parallax minimum value normalized on the basis of. The multi-view parallax image generator 53, for each viewpoint of a multi-view, to generate a parallax image for the disparity values ​​of the pixels normalized pixel values ​​of the pixels of the parallax image.
[0025]
　Also, multi-view parallax image generator 53 supplies the multi-view image coding unit 55 parallax images of the created multi-view as a multi-view parallax images. Furthermore, multi-view parallax image generator 53 generates a parallax accuracy parameter representative of the accuracy of the pixel values ​​of the multi-view parallax images, and supplies to the viewpoint generation information generating unit 54.
[0026]
　Viewpoint generation information generating unit 54 uses the corrected color image and the parallax image multi-view, generates viewpoint generating information (viewpoint generation information) used when generating a color image of the view outside the multi-view . Specifically, viewpoint generation information generating unit 54, based on external parameters supplied from the multi-view color image pickup unit 51 obtains the inter-camera distance. The inter-camera distance, for each viewpoint of the multi-view parallax images, the position of the horizontal direction of the multi-view color image pickup unit 51 at the time of capturing a color image of the view point, the parallax corresponding to the color image and the parallax image a distance in the horizontal direction position of the multi-view color image pickup unit 51 at the time of capturing a color image having.
[0027]
　Viewpoint generation information generating unit 54, a parallax maximum value and the parallax minimum value from the multi-view color image pickup unit 51, a parallax accuracy parameter viewpoint generating information from the inter-camera distance, and multi-view parallax image generator 53 . Viewpoint generation information generating unit 54 supplies the generated viewpoint generating information to the multi-view image coding unit 55.
[0028]
　Multi-view image encoding unit 55, a multi-view correction color image supplied from the multi-view color image correcting unit 52 for encoding in HEVC manner. Further, the multi-view image encoding unit 55, a parallax maximum value of viewpoint generating information supplied from the viewpoint generation information generating unit 54, by using parallax minimum, and the inter-camera distance as information on parallax, multi the multi-view parallax image supplied from the viewpoint parallax image generator 53 encodes in a manner analogous HEVC scheme.
[0029]
　Further, the multi-view image encoding unit 55, a parallax maximum value of viewpoint generating information supplied from the viewpoint generation information generating unit 54, a parallax minimum, and the inter-camera distance and difference coding, multi-view parallax images including information (coding parameters) about the coding used when encoding. Then, the multi-view image encoding unit 55, the information encoded multi-view correction color image and the multi-view parallax images, differentially encoded disparity maximum value, disparity minimum value, and a coding comprising the inter-camera distance, the bit stream of disparity accuracy parameter or the like from the viewpoint generation information generating unit 54, and transmitted as encoded bit stream.
[0030]
　As described above, the multi-view image encoding unit 55, a parallax maximum value, disparity minimum, and therefore transmits the inter-camera distance and differential encoding, it is possible to reduce the code amount of the viewpoint generating information. In order to provide a comfortable 3D image, the disparity maximum value, disparity minimum, and because inter-camera distance is likely not to change greatly between pictures, by performing the differential encoding is effective in reducing the code quantity .
[0031]
　Incidentally, the encoding apparatus 50, the multi-view parallax images, but is generated from multi-view correcting color image, when imaging of the multi-view color image may be generated by a sensor for detecting a disparity value.
[0032]
　[Viewpoint Description of generating information]
　FIG. 2 is a diagram illustrating a parallax maximum value and the parallax minimum value of the viewpoint generating information.
[0033]
　In FIG. 2, the horizontal axis is the disparity value before normalization, the vertical axis represents the pixel value of the parallax images.
[0034]
　As shown in FIG. 2, the multi-view parallax image generator 53, a disparity value for each pixel, by using the parallax minimum Dmin and the parallax maximum value Dmax, it is normalized to a value of for example 0 to 255. The multi-view parallax image generator 53, a disparity value for each pixel of the normalized is any value from 0 to 255 as the pixel value, to generate a parallax image.
[0035]
　That is, the pixel value I of each pixel of the parallax images before normalization disparity value d of the pixel, the disparity minimum Dmin, and disparity maximum value Dmax is expressed by the following equation (1).
[0036]
[Formula 1]

[0037]
　Accordingly, the decoding device to be described later, by the following equation (2), from the pixel value I of each pixel of the parallax image by using a parallax minimum Dmin and the parallax maximum value Dmax, to restore the pre-normalization disparity value d There is a need.
[0038]
[Number 2]

[0039]
　Thus, the disparity minimum value Dmin and the parallax maximum value Dmax is transmitted to the decoding device.
[0040]
　3 is a diagram illustrating a parallax accuracy parameter of the viewpoint generating information.
[0041]
　As shown in the upper part of FIG. 3, when the parallax value before normalization disparity values ​​per the normalized is 0.5, parallax accuracy parameter is a representative of the accuracy 0.5 disparity values. Further, as shown in the lower part of FIG. 3, when the parallax value before normalization disparity values ​​per the normalized is one, parallax accuracy parameter is a representative of the accuracy 1.0 disparity values.
[0042]
　In the example of FIG. 3, the disparity values ​​of the previous viewpoint # normalization 1 is the first viewpoint is 1.0, the disparity values ​​of the previous second normalization viewpoint # 2 is a perspective of 0.5. Therefore, viewpoint # disparity value after normalization 1, even 1.0 even accuracy of the parallax values ​​of 0.5, 1.0. On the other hand, the disparity values ​​of the viewpoint # 2 is 0.5 when the accuracy of the parallax value is 0.5, which is 0 if the accuracy of the parallax value is 1.0.
[0043]
　Figure 4 is a diagram for explaining the inter-camera distance of the viewpoint generating information.
[0044]
　As shown in FIG. 4, the viewpoint # 1, the inter-camera distance parallax images that originates viewpoint # 2 is set to the position indicated by the external parameters of the viewpoint # 1, at a distance of a position indicated by the external parameters of the viewpoint # 2 is there.
[0045]
　[Configuration example of the multi-view image encoding unit]
　FIG. 5 is a block diagram showing a configuration example of the multi-view image encoding unit 55 of FIG. 1.
[0046]
　Multi-view image encoding unit 55 of FIG. 5 is constituted by the slice coding unit 61, the slice header encoding unit 62, PPS encoder 63 and SPS coding section 64,.
[0047]
　The slice coding unit 61 of the multi-view image encoding unit 55, with respect to multi-view correcting color image supplied from the multi-view color image correcting unit 52 performs encoding in units of slices in HEVC manner. Furthermore, the slice coding unit 61, by using the disparity maximum value of the viewpoint generating information supplied from the point of view generation information generating unit 54 1, the parallax minimum, and the inter-camera distance as information on parallax, for multi-view parallax images from multiview parallax image generator 53 performs encoding in units of slices in a manner analogous HEVC scheme. The slice coding unit 61 supplies the encoded data or the like in units of slices obtained as a result of encoding in the slice header encoding unit 62.
[0048]
　Slice header encoding unit 62, a parallax maximum value of viewpoint generating information supplied from the viewpoint generation information generating unit 54, a parallax minimum, and the inter-camera distance, the parallax maximum value of the slice of the current processing target , parallax minimum value, and the inter-camera distance, to hold.
[0049]
　Further, the slice header encoding unit 62, a parallax maximum value of the slice of the current processing target, the parallax minimum, and the inter-camera distance, respectively, the previous slice disparity maximum in coding order than the slice, parallax minimum value, and whether or not to coincide with the inter-camera distance, the unit in which the same PPS is added (hereinafter, referred to as the same PPS units) determined in.
[0050]
　Then, the disparity maximum value of all the slices that make up the same PPS unit, parallax minimum value, and the inter-camera distance, the parallax maximum value of the previous slice in coding order, the disparity minimum value, and the inter-camera distance match Then when it is determined, slice header encoding unit 62, a slice header of the coded data for each slice constituting the same PPS unit, disparity maximum value of the slice, the disparity minimum, and other than the inter-camera distance code adding information relating to reduction, and supplies the PPS encoder 63. Further, the slice header encoding unit 62, a parallax maximum value, disparity minimum value, and supplies the transmission flag indicating the absence of transmission of the differential coding result of the inter-camera distance PPS encoder 63.
[0051]
　On the other hand, at least one slice disparity maximum of constituting the same PPS unit, parallax minimum, and the inter-camera distance, disparity maximum value of the previous slice in coding order, the parallax minimum, and the inter-camera distance and If they do not match is determined, the slice header encoding unit 62, the encoded data of the intra type slice coding comprising a slice header, the disparity maximum value of the slice, the disparity minimum, and the inter-camera distance information about the added, and supplies it to the PPS coding unit 63.
[0052]
　Further, the slice header encoding unit 62, for inter-type slices, the disparity maximum value of the slice, the differentially encoded parallax minimum, and the inter-camera distance. Specifically, the slice header encoding unit 62, a parallax maximum value of inter-type slices, the disparity minimum value, and the inter-camera distance, the previous slice disparity maximum in coding order than the slice, parallax minimum value, and the inter-camera distance, respectively, subtraction, and differential encoding result. Then, the slice header encoding unit 62, the encoded data of inter-type slices, the addition disparity maximum value as a slice header, the disparity minimum, and information about coding, including differential encoding result of the inter-camera distance, supplied to the PPS coding unit 63.
[0053]
　In this case, the slice header encoding unit 62, a parallax maximum value, disparity minimum value, and supplies the transmission flag indicating presence of a transmission of the inter-camera distance difference coding results of the PPS encoder 63.
[0054]
　PPS encoder 63 is provided with a transmission flag is supplied from the slice header encoding unit 62, a PPS containing parallax accuracy parameter of the viewpoint generating information supplied from the point of view generation information generating unit 54 1 generated. PPS encoder 63 is the same PPS units, adds a PPS in the encoded data of the supplied slice headers added slices from the slice header encoding unit 62, and supplies the SPS coding unit 64.
[0055]
　SPS coding unit 64 generates the SPS. Then, SPS encoding unit 64 is a sequence unit, adds the SPS to the encoded data PPS supplied is added from PPS encoder 63. SPS coding unit 64 functions as a transmission unit and transmits the resulting bit stream as the coded bit stream.
[0056]
　[Configuration example of a slice coding unit]
　FIG. 6 is a block diagram showing a configuration example of an encoding unit for encoding the parallax images for any one viewpoint of the slice coding unit 61 of FIG. In other words, encoding unit for encoding a multi-view parallax images of the slice coding unit 61 is composed of coding section 120 of the viewpoint fraction Figure 6.
[0057]
　Encoding unit 120 of FIG. 6, the A / D conversion unit 121, a screen rearrangement buffer 122, arithmetic unit 123, orthogonal transform unit 124, quantization unit 125, a reversible encoding unit 126, storage buffer 127, dequantizer 128, inverse orthogonal transform unit 129, addition unit 130, deblocking filter 131, a frame memory 132, intra prediction unit 133, the motion prediction and compensation unit 134, the correction unit 135 is constituted by the selection unit 136 and rate control unit 137, that.
[0058]
　A / D conversion unit of the coding unit 120 121, the multiplex image in units of frames of a predetermined viewpoint supplied from the multi-view parallax image generator 53 of FIG. 1 converts A / D, the screen rearrangement buffer 122 output to be stored. Screen rearrangement buffer 122, a parallax image in units of frames of the stored display order, according to the GOP (Group of Picture) structure, rearranged in the order for coding, arithmetic unit 123, intra prediction unit 133 , and outputs it to the motion prediction and compensation unit 134.
[0059]
　Operation unit 123 functions as an encoding unit, and a prediction image supplied from the selection unit 136, by calculating the difference between the output coded target parallax images from the screen rearrangement buffer 122, to be encoded encoding the parallax image. Specifically, operation unit 123, the output coded target parallax images from the screen rearrangement buffer 122, subtracts the predicted image supplied from the selection unit 136. Operation unit 123, an image obtained as a result of the subtraction, and outputs to the orthogonal transformation unit 124 as the residual information. Incidentally, if the predicted image from the selection unit 136 is not supplied, the arithmetic unit 123 outputs the parallax image read from the screen rearrangement buffer 122 as it is to the orthogonal transformation unit 124 as the residual information.
[0060]
　Orthogonal transform unit 124, a discrete cosine transform on the residual information from the operation unit 123 performs orthogonal transformation such as Karhunen-Loeve transform, and supplies the coefficients obtained as a result to the quantization unit 125.
[0061]
　Quantization unit 125 quantizes the coefficient supplied from the orthogonal transformation unit 124. The quantized coefficients are inputted to the reversible encoding unit 126.
[0062]
　Lossless encoding unit 126, the quantized coefficients supplied from the quantization unit 125, variable length coding (e.g., CAVLC (Context-Adaptive Variable Length Coding), etc.), arithmetic coding (e.g., CABAC (Context-Adaptive Binary Arithmetic coding), etc.) make a reversible encoding, such as. Lossless encoding unit 126 supplies the encoded data obtained as a result of lossless coding in the storage buffer 127, to accumulate.
[0063]
　Storage buffer 127, the coded data supplied from the lossless coding unit 126, temporarily stored, and supplies the slice header encoding unit 62 per slice.
[0064]
　Moreover, output from the quantization unit 125, the quantized coefficients is also input to the inverse quantization unit 128, after being inverse quantized and supplied to the inverse orthogonal transformation unit 129.
[0065]
　Inverse orthogonal transform unit 129, the coefficient supplied from the inverse quantization unit 128, inverse discrete cosine transform, applies inverse orthogonal transformation, such as transformation inverse Karhunen-Loeve, adding unit residual information resulting 130 supplied to.
[0066]
　Adding section 130 obtains the residual information as a parallax image to be decoded which is supplied from the inverse orthogonal transform unit 129 adds the predicted image supplied from the selection unit 136, a locally decoded parallax image . Incidentally, if the predicted image from the selection unit 136 is not supplied, the addition unit 130, a locally decoded parallax image residual information supplied from the inverse orthogonal transformation unit 129. Adding section 130 supplies the locally decoded parallax image is supplied to a deblocking filter 131, the intra prediction unit 133 as a reference image.
[0067]
　Deblocking filter 131 filters the locally decoded parallax image supplied from the adder 130, to remove the block distortion. Deblocking filter 131 supplies the parallax images obtained as a result to the frame memory 132, to accumulate. Stored parallax image in the frame memory 132 is output to the motion prediction and compensation portion 134 as a reference image.
[0068]
　Intra prediction unit 133 uses the reference image supplied from the adding unit 130 performs intra prediction of all intra-prediction modes as candidates, and generates a predicted image.
[0069]
　Further, the intra prediction unit 133 calculates the cost function values ​​for all intra prediction modes which are candidates (the details will be described later). Then, the intra prediction unit 133 determines an intra-prediction mode cost function value is minimized in the optimal intra prediction mode. Intra prediction unit 133, the optimal intra prediction mode prediction image generated by, and the corresponding cost function value, and supplies to the selecting unit 136. Intra prediction unit 133, when the selection of the prediction image generated in the optimal intra prediction mode from the selection unit 136 has been notified, the slice header encoding unit 62 of FIG. 5 the intra prediction information indicating the optimal intra prediction mode, and the like supplied to. The intra-frame prediction information is included in the slice header as information regarding the encoding.
[0070]
　It should be noted that the cost function value is also called RD (Rate Distortion) cost, for example, H. 264 / AVC system, such as those defined by the JM (Joint Model) is a reference software in, or High Complexity mode, is calculated on the basis of any of the methods of the Low Complexity mode.
[0071]
　Specifically, when the High Complexity mode is adopted as a method of calculating the cost function value, for all prediction modes as candidates, are performed to temporarily reversible encoding, represented by the following formula (3) cost function value is calculated for each prediction mode.
[0072]
　Cost(Mode)=D＋λ・R　　　　　　　　　　　　　　　　・・・（３）
[0073]
　D is the original image and the decoded image difference (distortion), R is inclusive generated code amount by a factor of orthogonal transform, lambda is the Lagrangian multiplier given as a function of the quantization parameter QP.
[0074]
　On the other hand, if the Low Complexity mode is adopted as a method of calculating the cost function value to all prediction modes that are candidates, generating a decoded image, and the calculation of the header bits, such as information indicating the prediction mode is performed , the cost function expressed by the following equation (4) is calculated for each prediction mode.
[0075]
　Cost(Mode)=D＋QPtoQuant(QP)・Header_Bit　　　　　　・・・（４）
[0076]
　D is the original image and the decoded image difference (distortion), Header_Bit the header bits, QPtoQuant for the prediction mode is a function given as a function of the quantization parameter QP.
[0077]
　In Low Complexity mode, for all prediction modes, it is only necessary to generate a decoded image, there is no need to perform the lossless encoding, requires the calculation amount is small. Here, it is assumed that the High Complexity mode is adopted as a method of calculating the cost function value.
[0078]
　The motion prediction and compensation unit 134 performs a parallax image supplied from the screen rearrangement buffer 122, based on the reference image supplied from the frame memory 132, a motion prediction process for all the inter prediction modes as candidates, It generates a motion vector. Specifically, the motion prediction and compensation unit 134, for each inter prediction mode, perform the reference image, the matching of the parallax image supplied from the screen rearrangement buffer 122, to generate the motion vector.
[0079]
　Note that the inter prediction mode, the size of the block of interest in the inter prediction, which is information representing a prediction direction, and reference indices. The prediction direction, the prediction before the direction of using the target early reference image display time than the parallax images of inter-prediction (L0 prediction), the slow reference image display time than the parallax image to be the subject of inter-prediction direction of prediction after using (L1 prediction), and a bidirectional prediction (Bi-prediction) using target display time than parallax images is earlier reference picture and slow reference picture inter prediction. Moreover, the reference index is a number for identifying a reference picture, for example, as the reference index of the image close to the parallax images subjected to the inter prediction number is small.
[0080]
　Further, the motion prediction and compensation portion 134 functions as a predictive picture generating unit, for each inter prediction mode, based on the generated motion vector, by reading the reference image from the frame memory 132, performs a motion compensation process. Motion predicting and compensating section 134 supplies the predicted image to be the result generated in the correction unit 135.
[0081]
　Correcting unit 135, disparity maximum value of viewpoint generating information supplied from the point of view generation information generating unit 54 1, the parallax minimum value, and using the inter-camera distance as information on a parallax image, a prediction image generating a correction coefficient used when correcting (setting). Correcting unit 135, a predicted image for each inter prediction mode supplied from the motion prediction and compensation portion 134 is corrected using the correction coefficient.
[0082]
　Here, the depth direction position Z of the object of the parallax image to be encoded c position Z in the depth direction of the object and the prediction image p is represented by the following formula (5).
[0083]
[Number 3]

[0084]
　In Expression (5 of 5), L c , L p- respectively, the inter-camera distance parallax images to be coded, a camera distance of the predicted image. f is a common focal length to the prediction image and the parallax image to be coded. Further, d- c , d- p- respectively, the absolute value of the disparity values ​​of the previous normalization parallax image to be coded, the absolute value of the disparity values ​​of the previous normalization prediction image.
[0085]
　Also, the disparity value I of the parallax image to be coded c parallax value I of the predicted image p , the absolute value d of the normalized previous disparity value c , d p using, represented by the following formula (6) that.
[0086]
[Number 4]

[0087]
　In Expression (. 6), D c min , D p- min , respectively, parallax minimum value of the parallax image to be coded, a disparity minimum value of the predicted image. D c max , D p- max , respectively, parallax maximum value of the parallax image to be coded, a disparity maximum value of the predicted image.
[0088]
　Therefore, the position Z in the depth direction of the subject of the parallax image to be encoded c position Z in the depth direction of the subject of the predicted image p even in the same, inter-camera distance L c and L p , disparity minimum value D c min and D P min , and the disparity maximum value D C Max , D P Max and at least one of the different, disparity value I C and the parallax value I P would be different.
[0089]
　Therefore, the correction unit 135, the position Z- c and the position Z- p- disparity value I when the identical c parallax value I p- generates a correction coefficient for correcting the predicted image to have the same.
[0090]
　Specifically, the position Z- c and the position Z- p- If are the same, the equation (5) described above, the following equation (7) holds.
[0091]
[Number 5]

[0092]
　In addition, when modifying the formula (7), consisting of the following equation (8).
[0093]
[Number 6]

[0094]
　Then, using equation (6) described above, the absolute value d of the normalized previous disparity value of the formula (. 8) c , d p- disparity value the I c and disparity values ​​the I p- Substitution in the following formula (9 )become.
[0095]
[Number 7]

[0096]
　Thus, the disparity value the I c is the disparity value the I p- expressed by the following equation using the (10).
[0097]
[Number 8]

[0098]
　Accordingly, the correction unit 135 generates a and b of formula (10) as a correction coefficient. Then, the correction unit 135, the correction coefficients a, b and disparity value I p- using parallax value I in the formula (10) c determined as the parallax value of the prediction image after correcting.
[0099]
　Further, the correction unit 135, using the predicted image after the correction to calculate the cost function value for each inter prediction mode, determines the inter prediction mode cost function value is minimized in the optimal inter measurement mode. Then, the correction unit 135, and supplies the selecting unit 136 and the predictive image and the cost function values ​​generated in the optimal inter prediction mode.
[0100]
　Further, the correction unit 135, when the selection of the prediction image generated in the optimal inter prediction mode from the selection unit 136 is notified, and outputs the motion information in the slice header encoding unit 62. This motion information, the optimal inter prediction mode, the prediction vector index, constituted by the motion vector residual such a difference obtained by subtracting the motion vector from the current motion vector indicated by the prediction vector index. Incidentally, the predicted vector index is information identifying one of the motion vectors of the motion vectors, which are candidates used for the generation of the predicted image of the decoded parallax images. Motion information is included in the slice header as information regarding the encoding.
[0101]
　Selecting unit 136, based on the cost function value supplied from the intra prediction unit 133 and the correction unit 135, one of the best intra prediction mode and the optimal inter prediction mode, to determine the optimal prediction mode. Then, the selection unit 136, a predicted image of the optimum prediction mode, and supplies to the arithmetic unit 123 and the addition unit 130. In addition, the selection unit 136 notifies the selection of the predicted image of the optimal prediction mode in the intra-frame prediction unit 133 or the correction unit 135.
[0102]
　The rate control unit 137, based on the encoded data accumulated in the accumulation buffer 127, as an overflow or underflow does not occur, and controls the rate of the quantization operation of the quantization unit 125.
[0103]
　[Configuration example of a coded bit stream]
　FIG. 7 is a diagram illustrating a configuration example of a coded bit stream.
[0104]
　In FIG. 7, for convenience of explanation, has been described only slice coded data of the multi-view parallax images, in practice, the coded bit stream, a slice of the encoded data of the multi-view color image is also It is located. This also applies to the 22 and 23 will be described later.
[0105]
　In the example of FIG. 7, the 0th PPS # 1 single intra types of slices and disparity maximum of two inter-type slices constituting the same PPS units of 0 is PPS, parallax minimum, and the inter-camera distance, each parallax maximum value of the previous slice in coding order, the parallax minimum, and does not coincide with the inter-camera distance. Therefore, the PPS # 0 includes a transmitted flag "1" indicating the presence of a transmission. In the example of FIG. 7, the parallax accuracy of 0.5 slices constituting the same PPS units of PPS # 0, the PPS # 0 includes a "1" indicating the parallax accuracy 0.5 as the parallax accuracy parameter.
[0106]
　Further, in the example of FIG. 7, a disparity minimum value of the intra-type slice 10 constituting the same PPS units of PPS # 0, a parallax maximum value is 50, the inter-camera distance is 100. Therefore, the slice header of the slice, the disparity minimum value "10", the disparity maximum value "50", and a camera distance "100".
[0107]
　In the example of FIG. 7, a disparity minimum value of the first inter-type slices constituting the same PPS units of PPS # 0 is 9, the parallax maximum value is 48, the inter-camera distance is 105. Therefore, the slice header of the slice, from the disparity minimum value "9" of the slice, the disparity minimum value of the slice of the previous intra type coding order "10" difference "-1" obtained by subtracting the found It is included as a differential encoding a result of the disparity minimum value. Similarly, the difference "-2" parallax maximum value is included as differential encoding result of the disparity maximum value, the difference of the camera distance "5" is included as differentially encoded result of the inter-camera distance.
[0108]
　Further, in the example of FIG. 7, a disparity minimum value of the second inter-type slices constituting the same PPS units of PPS # 0 7, the parallax maximum value is 47, the inter-camera distance is 110. Therefore, the slice header of the slice, from the disparity minimum value "7" of the slice, the difference "-2 obtained by subtracting the first parallax minimum of inter-type slices immediately preceding in coding order" 9 " "it is included as a differentially encoded result of the disparity minimum. Similarly, the difference "-1" parallax maximum value is included as differential encoding result of the disparity maximum value, the difference of the camera distance "5" is included as differentially encoded result of the inter-camera distance.
[0109]
　In the example of FIG. 7, the first PPS # 1 single intra types of slices and disparity maximum of two inter-type slices constituting one of the same PPS unit is PPS, parallax minimum, and the inter-camera distance but, respectively, consistent disparity maximum value of the previous slice in coding order, the disparity minimum value, and the inter-camera distance. That, PPS # 1 single parallax minimum of the intra-type slices and two inter-type slices constituting the same PPS units of 1, parallax maximum value, the inter-camera distance, respectively, constitute the same PPS units of PPS # 0 "7" the same and the second inter-type of slice you, "47", which is "110". Therefore, the PPS # 1 includes a transmitted flag "0" indicating the absence of transmission. In the example of FIG. 7, the parallax accuracy of 0.5 slices constituting the same PPS units of PPS # 1, the PPS # 1 includes a "1" indicating the parallax accuracy 0.5 as the parallax accuracy parameter.
[0110]
　[Example of PPS of Syntax]
　FIG. 8 is a diagram illustrating an example of a syntax of PPS in FIG.
[0111]
　As shown in FIG. 8, the PPS, it includes parallax accuracy parameter (disparity_precision) and the transmission flag (dsiparity_pic_same_flag). Parallax accuracy parameters, for example, is a case that represents the disparity accuracy 1 "0", is a case that represents the parallax accuracy 0.25 "2". As described above, the parallax accuracy parameter is "1" when referring to parallax accuracy 0.5. The transmission flag is, as described above, is "1" when referring to presence of transmission is when representing the absence of transmission "0".
[0112]
　Slice syntax example of a header]
　FIGS. 9 and 10 are diagrams showing an example of a syntax of a slice header.
[0113]
　As shown in FIG. 10, a transmission flag is 1, if the type of the slice is an intra type, the slice header, parallax minimum value (minimum_disparity), disparity maximum value (maximum_disparity), and inter-camera distance (translation_x) It is included.
[0114]
　On the other hand, a transmission flag is 1, and the type of slice is an inter type, in the slice header, differential encoding a result of the disparity minimum value (delta_minimum_disparity), differential encoding a result of the disparity maximum value (delta_maximum_disparity), and the camera between distance differential encoding result of (delta_translation_x) are included.
[0115]
　[Explanation of the processing of the encoding apparatus]
　FIG 11 is a flowchart for explaining the encoding process of the encoding device 50 of FIG. 1.
[0116]
　In step S111 in FIG. 11, the multi-view color image pickup unit 51 of the encoding device 50 captures a color image of multi-view, and supplies the multi-view color image correcting unit 52 as a multi-view color image.
[0117]
　In step S112, multi-view color image pickup unit 51, a parallax maximum value, disparity minimum value, and generating an external parameter. Multi-view color image pickup unit 51, a parallax maximum value, disparity minimum value, and supplies the external parameter to the viewpoint generation information generating unit 54, supplying the disparity maximum value and the parallax minimum value multiview parallax image generator 53 to.
[0118]
　In step S113, the multi-view color image correcting unit 52, with respect to multi-view color image supplied from the multi-view color image pickup unit 51 performs color correction, brightness correction, distortion correction, and the like. Thus, the focal length in the horizontal direction (X direction) of the multi-view color image pickup unit 51 of the multi-view color image after correction becomes common to all viewpoints. Multi-view color image correcting unit 52 supplies the multi-view color image after correcting the multi-view parallax image generator 53 as a multi-view correcting color image to the multi-view image coding unit 55.
[0119]
　In step S114, the multi-view parallax image generator 53, based on the disparity maximum value and the parallax minimum value supplied from the multi-view color image pickup unit 51, the multi-view correcting color supplied from the multi-view color image correcting unit 52 from image to generate a parallax image for multi-view. The multi-view parallax image generator 53 supplies the multi-view image coding unit 55 parallax images of the created multi-view as a multi-view parallax images.
[0120]
　In step S115, the multi-view parallax image generator 53 generates a parallax accuracy parameters, and supplies it to the point of view generation information generation section 54.
[0121]
　In step S116, the viewpoint generation information generating unit 54, based on external parameters supplied from the multi-view color image pickup unit 51 obtains the inter-camera distance.
[0122]
　In step S117, the viewpoint generation information generating unit 54, the disparity maximum value and the disparity minimum value from the multi-view color image capturing section 51, inter-camera distance, as well as the disparity accuracy parameter point of view generated from the multi-view parallax image generation unit 53 to generate as use information. Viewpoint generation information generating unit 54 supplies the generated viewpoint generating information to the multi-view image coding unit 55.
[0123]
　In step S118, the multi-view image coding unit 55, a multi-view coding process of encoding the multi-view parallax images from the multi-viewpoint correction color image and the multi-view parallax image generating unit 53 from the multi-view color image correction section 52 I do. Details of this multi-view coding process will be described with reference to FIG. 12 described later.
[0124]
　In step S 119, the multi-view image encoding unit 55 transmits the encoded bit stream obtained as a result of the multi-view coding process, the process is terminated.
[0125]
　Figure 12 is a flowchart illustrating a multi-view coding process in step S118 in FIG. 11.
[0126]
　In step S131 in FIG. 12, the slice coding unit 61 of the multi-view image encoding unit 55 (FIG. 5) includes a multi-view correcting color image from the multi-view color image correcting unit 52, the multi-view parallax image generator 53 the multi-view parallax images, encoded in units of slices. Specifically, the slice coding unit 61 performs a color image encoding process of encoding the multi-view correcting color image HEVC manner per slice. Furthermore, the slice coding unit 61, by using the disparity maximum value of the viewpoint generating information supplied from the point of view generation information generating unit 54 1, the parallax minimum, and the inter-camera distance, multi-view parallax images the conduct of the parallax image encoding process of encoding in the slice unit in a manner that conforms to HEVC method. Details of the parallax image encoding process will be described with reference to FIGS. 13 and 14 described later. The slice coding unit 61 supplies the encoded data in units of slices obtained as a result of encoding in the slice header encoding unit 62.
[0127]
　In step S 132, the slice header encoding unit 62, inter-camera distance of viewpoint generating information supplied from the viewpoint generation information generating unit 54, a parallax maximum value, and the parallaxes minimum value, the slice of the current process target inter-camera distance, the parallax maximum value, and the disparity minimum value, to hold.
[0128]
　In step S 133, the slice header encoding unit 62, all of the inter-camera distance of slices constituting the same PPS unit, disparity maximum value, and disparity minimum value, respectively, the previous slice in coding order than the slice inter-camera distance, it is determined whether or not to match the disparity maximum value, and the disparity minimum value.
[0129]
　Inter-camera distance in step S 133, if it is determined that the parallax maximum value, and disparity minimum values ​​match, at step S 134, the slice header encoding unit 62, inter-camera distance, the parallax maximum value, and the difference the sign of the disparity minimum generating a transmission flag indicating the absence of transmission of the quantization result, and supplies the PPS encoder 63.
[0130]
　In step S 135, the slice header encoding unit 62, for each slice of the encoded data constituting the same PPS units to be processed in step S 133, as the slice header, the inter-camera distance for that slice, disparity maximum value, and disparity adding information about the encoding of non-minimum. Note that the information about the coding includes intra prediction information or motion information supplied from the slice coding unit 61. Then, the slice header encoding unit 62 supplies the encoded data of each slice constituting the same PPS units resulting in PPS encoder 63, the process proceeds to step S 140.
[0131]
　On the other hand, inter-camera distance, the parallax maximum value in step S 133, and if the disparity minimum value is determined not to match, in step S 136, the slice header encoding unit 62, inter-camera distance, the parallax maximum value, and disparity minimum value and it supplies the transmission flag that represents the presence of the transmission of the differential encoding result in PPS coding unit 63. The processing of steps S137 to S139 described below is performed for each slice constituting the same PPS units to be processed in step S 133.
[0132]
　In step S 137, the slice header encoding unit 62, the type of slices constituting the same PPS units to be processed in step S133 determines whether an intra type. If the slice type at step S137 is judged to be intra type, in step S 138, the slice header encoding unit 62, the encoded data of that slice as a slice header, the inter-camera distance for that slice, parallax maximum adding information about the encoding, including the value, and the parallaxes minimum. Note that the information on the coding, the intra prediction information or motion information supplied from the slice coding unit 61 is also included. Then, the slice header encoding unit 62 supplies the encoded data of the resulting slices into PPS encoder 63, the process proceeds to step S 140.
[0133]
　On the other hand, if the slice type in step S137 is determined not to be an intra type, that is, when the slice type is an inter type, the process proceeds to step S 139. Information In step S 139, the slice header encoding unit 62, inter-camera distance for that slice, disparity maximum value, and the parallaxes minimum value differential encoding, the encoded data of the slice, about encoding including the differential coding result It is added as a slice header. Note that the information on the coding, the intra prediction information or motion information supplied from the slice coding unit 61 is also included. Then, the slice header encoding unit 62 supplies the encoded data of the resulting slices into PPS encoder 63, the process proceeds to step S 140.
[0134]
　In step S 140, PPS encoder 63 is provided with a transmission flag is supplied from the slice header encoding unit 62, a parallax accuracy parameter of the viewpoint generating information supplied from the point of view generation information generating unit 54 1 It generates a PPS, including.
[0135]
　In step S 141, PPS encoder 63 is the same PPS units, adds a PPS in the encoded data of the supplied slice headers added slices from the slice header encoding unit 62, to the SPS coding section 64 supplies.
[0136]
　In step S142, SPS coding unit 64, to generate the SPS.
[0137]
　In step S 143, SPS encoding unit 64 is a sequence unit, adds the SPS to the encoded data PPS is added supplied from the PPS encoder 63, generates an encoded bit stream. Then, the processing returns to step S118 in FIG. 11, the process proceeds to step S 119.
[0138]
　13 and 14 are flowcharts illustrating the details of the parallax image encoding process of the slice coding unit 61 of FIG. This parallax image encoding process is performed for each point of view.
[0139]
　In step S160 in FIG. 13, A / D converter 121 of the encoding unit 120, a parallax image for each frame of a predetermined viewpoint inputted from the multiview parallax image generator 53 converts the A / D, the screen rearranging and stores and outputs it to the buffer 122.
[0140]
　In step S 161, the screen rearranging buffer 122, a parallax image of the stored display order of the frame, depending on the GOP structure, rearranged in order for encoding. Screen sorting buffer 122, the parallax image of the frame unit after sorting, and supplies it to the arithmetic unit 123, the intra prediction unit 133 and the motion prediction and compensation unit 134,.
[0141]
　In step S 162, the intra prediction unit 133 uses the reference image supplied from the adder 130, the intra prediction process for all the intra prediction modes as candidates performed. At this time, the intra prediction unit 133 calculates the cost function values ​​for all intra prediction modes as candidates. Then, the intra prediction unit 133 determines an intra-prediction mode cost function value is minimized in the optimal intra prediction mode. Intra prediction unit 133, the optimal intra prediction mode prediction image generated by, and the corresponding cost function value, and supplies to the selecting unit 136.
[0142]
　In step S 163, the motion prediction and compensation unit 134, a parallax image supplied from the screen rearrangement buffer 122, based on the reference image supplied from the frame memory 132, performs the motion prediction and compensation processing.
[0143]
　Specifically, the motion prediction and compensation unit 134, the screen rearranging the parallax image supplied from the buffer 122, based on the reference image supplied from the frame memory 132, all motion of the inter prediction modes as candidates It performs prediction processing, and generates a motion vector. Further, the motion prediction and compensation unit 134, for each inter prediction mode, based on the generated motion vector, by reading the reference image from the frame memory 132, performs a motion compensation process. Motion predicting and compensating section 134 supplies the predicted image to be the result generated in the correction unit 135.
[0144]
　In step S 164, the correction unit 135, disparity maximum value of viewpoint generating information supplied from the point of view generation information generating unit 54 1, the parallax minimum value, and based on the inter-camera distance, calculates the correction factor to.
[0145]
　In step S 165, the correction unit 135, a predicted image for each inter prediction mode supplied from the motion prediction and compensation portion 134 is corrected using the correction coefficient.
[0146]
　In step S 166, the correction unit 135, using the predicted image after the correction to calculate the cost function value for each inter prediction mode, determines the inter prediction mode cost function value is minimized in the optimal inter measurement mode . Then, the correction unit 135, and supplies the selecting unit 136 and the predictive image and the cost function values ​​generated in the optimal inter prediction mode.
[0147]
　In step S 167, selecting unit 136, based on the cost function value supplied from the intra prediction unit 133 and the correction unit 135, towards the cost function value of the optimal intra prediction mode and the optimal inter prediction mode is minimum It determines the optimal prediction mode. Then, the selection unit 136, a predicted image of the optimum prediction mode, and supplies to the arithmetic unit 123 and the addition unit 130.
[0148]
　In step S 168, the selection unit 136 determines whether the optimum prediction mode is the optimal inter prediction mode. If the optimum prediction mode is determined to be the optimal inter prediction mode in step S 168, selecting unit 136 notifies the correction section 135 to select the prediction image generated in the optimal inter prediction mode.
[0149]
　Then, in step S 169, the correction unit 135 outputs the motion information in the slice header encoding unit 62 (FIG. 5), the process proceeds to step S 171.
[0150]
　On the other hand, if the optimum prediction mode is determined not to be optimal inter prediction mode in step S 168, that is, if the optimum prediction mode is the optimal intra prediction mode, the selection unit 136, the prediction image generated in the optimal intra prediction mode to notify the selection on the screen in the prediction unit 133.
[0151]
　Then, in step S 170, the intra prediction unit 133 outputs the intra prediction information in the slice header encoding unit 62 advances the process to step S 171.
[0152]
　In step S 171, operation unit 123, from the parallax image supplied from the screen rearrangement buffer 122, subtracts the predicted image supplied from the selection unit 136. Operation unit 123, an image obtained as a result of the subtraction, and outputs to the orthogonal transformation unit 124 as the residual information.
[0153]
　In step S 172, orthogonal transform unit 124, an orthogonal transform performed with respect to residual information from the operation unit 123 supplies the coefficient obtained as a result to the quantization unit 125.
[0154]
　In step S 173, the quantization unit 125 quantizes the coefficient supplied from the orthogonal transformation unit 124. The quantized coefficients are inputted to the inverse quantization unit 128 and the lossless coding unit 126.
[0155]
　In step S 174, the reversible encoding unit 126 lossless codes the quantized coefficients supplied from the quantization unit 125.
[0156]
　In step S175 in FIG. 14, the reversible encoding unit 126 supplies the encoded data obtained as a result of lossless encoding processing to the storage buffer 127, to accumulate.
[0157]
　In step S 176, the storage buffer 127 outputs the encoded data accumulated in the slice header encoding unit 62.
[0158]
　In step S 177, the inverse quantization unit 128 inversely quantizes the quantized coefficients supplied from the quantization unit 125.
[0159]
　In step S 178, the inverse orthogonal transformation unit 129 performs inverse orthogonal transform to the coefficient supplied from the inverse quantization unit 128, supplies the residual information obtained as a result to the adder 130.
[0160]
　In step S 179, the addition unit 130 adds the residual information supplied from the inverse orthogonal transform unit 129, a prediction image supplied from the selection unit 136, to obtain a locally decoded parallax images. Adding section 130 supplies the parallax images obtained in the deblocking filter 131, and supplies the intra prediction unit 133 as a reference image.
[0161]
　In step S 180, deblock filter 131, by performing filtering on the locally decoded parallax image supplied from the adder 130, to remove the block distortion.
[0162]
　In step S 181, the deblocking filter 131 supplies the parallax image after the filtering in the frame memory 132, to accumulate. Stored parallax image in the frame memory 132 is output to the motion prediction and compensation portion 134 as a reference image. Then, the process is terminated.
[0163]
　The processing of steps S162 through S181 of FIG. 13 and FIG. 14 is performed, for example, in increments of coding unit having a hierarchical structure. Further, in the parallax image encoding process of FIG. 13 and FIG. 14, in order to simplify the description, always it has been so intra prediction process and the motion compensation process is performed, in practice, either by picture type or the like in some cases only one is carried out.
[0164]
　As described above, the encoding apparatus 50 corrects the predicted image by using the information about the parallax image, it encodes the parallax image using the prediction image after correction. More particularly, the encoding apparatus 50, the inter-camera distance as information on a parallax image, the disparity maximum value, and using the disparity minimum value, between the predictive image and the parallax image, the position in the depth direction of the object is the same disparity value is corrected predicted image to be identical in some cases, encoding the parallax image using the prediction image after correction. Thus, the reduced difference between the predicted image and the parallax images resulting information about the parallax image is improved coding efficiency. In particular, when the information about the parallax image changes on a picture-by-picture basis, to improve the coding efficiency.
[0165]
　The encoding apparatus 50, as information used for correction of the predicted image, not the correction coefficient itself, and transmits the inter-camera distance used to calculate the correction coefficient, the parallax maximum value, and the parallaxes minimum. Here, the inter-camera distance, the parallax maximum value, and disparity minimum value is part of the view point generating information. Therefore, inter-camera distance, the parallax maximum value, and the parallaxes minimum value can be shared as part of the information and the viewpoint generating information used for correction of the predicted image. As a result, it is possible to reduce the information amount of the coded bit stream.
[0166]
　[First configuration example of an embodiment of the decoding device]
　FIG. 15 decodes the coded bit stream transmitted from the encoding device 50 of FIG. 1, the first embodiment of the applied decoding device to which the present technology it is a block diagram showing a configuration example.
[0167]
　Decoding device 150 of Figure 15 is composed of a multi-view image decoding unit 151, the viewpoint synthesizing unit 152, and the multi-view image display unit 153. Decoding device 150 decodes the encoded bit stream transmitted from the encoder 50, the multi-view color image obtained as a result, multi-view parallax images, and a color image of display viewpoints with viewpoint generating information It generates and displays.
[0168]
　Specifically, multi-view image decoding unit 151 of the decoding device 150 receives a coded bit stream transmitted from the encoding device 50 of FIG. 1. Multi-view image decoding unit 151 extracts the transmission flag parallax accuracy parameter from PPS included in the received coded bit stream. Further, the multi-view image decoding unit 151, in accordance with the transmission flag, and extracts inter-camera distance from the slice header of the coded bit stream, the disparity maximum value, and the parallaxes minimum. Multi-view image decoding unit 151 supplies the parallax accuracy parameter, the inter-camera distance, the parallax maximum value, and generates viewpoint generating information consisting of parallax minimum value, the viewpoint combining section 152.
[0169]
　Further, the multi-view image decoding unit 151, the coded data of the multi-view correcting color image of the slice units included in the coded bit stream, in a corresponding manner to the encoding method of the multi-view image encoding unit 55 of FIG. 1 decoded, to generate a multi-view correction color image. In addition, the multi-view image decoding unit 151, to function as a decoder. Multi-view image decoding unit 151, the inter-camera distance, the parallax maximum value, and using the disparity minimum value, the encoded data of the multi-view parallax images included in the coded bit stream, of the multi-view image coding unit 55 codes and decoding in a manner corresponding to the scheme, to produce a multi-view parallax images. Multi-view image decoding unit 151 supplies the multi-view correction color image and the multi-view parallax images generated in the viewpoint combining section 152.
[0170]
　Perspective synthesis unit 152, a multi-view using a perspective for generating information from the image decoding unit 151, a multi-view to multi-view parallax image from the image decoding unit 151, the number of viewpoints that corresponds to the multi-view image display unit 153 perform the warping process to the display point of view. Specifically, the viewpoint synthesizing unit 152, the inter-camera distance included in the viewpoint generating information, the disparity maximum value, and based on the parallax minimum value or the like, with a precision corresponding to the parallax accuracy parameter, with respect to multi-view parallax images perform the warping process to the display point of view on. Here, the warping process is a process of geometric transformation from the image of a viewpoint to the image of another viewpoint. Further, the display point of view, include viewpoints other than the viewpoint corresponding to multi-view color image.
[0171]
　Further, the viewpoint synthesizing unit 152, by using the parallax image display viewpoint obtained as a result of warping process, for multi-view correcting color image supplied from the multi-view image decoding unit 151, performs the warping process to display viewpoint . Viewpoint synthesizing unit 152 supplies the color image display viewpoint obtained as a result of the multi-view image display unit 153 as a multi-view composite color image.
[0172]
　Multi-view image display unit 153, a multi-view composite color image supplied from the viewpoint synthesizing unit 152, possible angles viewing is differently displayed for each viewpoint. The viewer by looking at the respective images of any two viewpoints at left and right eyes can see a 3D image from a plurality of viewpoints without wearing glasses.
[0173]
　As described above, viewpoint synthesizing unit 152, based on the parallax accuracy parameter, with a precision corresponding to the viewpoint accuracy parameter, since the warping process to display perspective on the multi-view parallax images, viewpoint synthesizing unit 152, vain there is no need to perform warping process with high accuracy.
[0174]
　Further, the viewpoint synthesizing unit 152 based on the inter-camera distance, since the warping process to display perspective on the multi-view parallax images, the extent parallax appropriate for the disparity values ​​of the multi-view parallax image after warping process If not, on the basis of the inter-camera distance, it can be modified to a value corresponding to the parallax of the appropriate range of disparity values.
[0175]
　[Configuration example of the multi-view image decoding unit]
　FIG. 16 is a block diagram showing a configuration example of the multi-view image decoding unit 151 in FIG. 15.
[0176]
　Multi-view image decoding unit 151 in FIG. 16 is composed of SPS decoding unit 171, PPS decoding unit 172, a slice header decoding unit 173 and the slice decoder 174,.
[0177]
　SPS decoding unit 171 of the multi-view image decoding unit 151 functions as a receiving unit receives the coded bit stream transmitted from the encoding device 50 of FIG. 1, extracts the SPS of the coded bit stream . SPS decoding unit 171 supplies the coded bit stream other than the extracted SPS and SPS to PPS decoding unit 172.
[0178]
　PPS decoding section 172, to extract the PPS from SPS other than the encoded bit stream, which is supplied from the SPS decoding section 171. PPS decoding unit 172, the extracted PPS, SPS, and supplies the coded bit stream other than the SPS and PPS in the slice header decoding unit 173.
[0179]
　Slice header decoding unit 173, to extract a slice header from SPS and non-PPS encoded bit stream supplied from the PPS decoding section 172. Slice header decoding unit 173, if the transmission flag contained in the PPS from PPS decoding unit 172 is "1" indicating the presence of transmission, inter-camera distance included in the slice header, the disparity maximum value, and the parallaxes minimum value or held, or, inter-camera distance, and updates the disparity maximum value, and the differential encoding result to inter-camera distance which is held on the basis of the parallax minimum disparity maximum value, and the parallaxes minimum. Slice header decoding unit 173, the inter-camera distance is held, the disparity maximum value, and disparity minimum value, and generates the viewpoint generating information from the parallax accuracy parameter contained in the PPS, and supplies the viewpoint synthesizing unit 152.
[0180]
　In addition, the slice header decoding unit 173, SPS, PPS, and the inter-camera distance of the slice header, other than the information disparity maximum value, and to a disparity minimum value, as well as some in the SPS, PPS, and other than the slice header coded bit stream the encoded data in units of slices, and supplies it to the slice decoding section 174. Further, the slice header decoding unit 173, and supplies the inter-camera distance, the parallax maximum value, and the parallaxes minimum value to the slice decoder 174.
[0181]
　Slice decoding unit 174, the slice SPS supplied from the header decoding unit 173, PPS, and the slice header of the inter-camera distance, on the basis of other than the information disparity maximum value, and a disparity minimum value, the slice coding unit 61 (FIG. 5 in a manner corresponding to the encoding scheme in), decodes the encoded data multiplexed color image in units of slices. Further, the slice decoder 174, SPS, PPS, inter-camera distance slice header, parallax maximum value, and the non-disparity minimum value relates information, as well as inter-camera distance, on the basis of the disparity maximum value, and the parallaxes minimum value, the slice in a manner corresponding to the coding method in the encoding unit 61, decodes the encoded data of the multiplexing parallax images in units of slices. Slice header decoding unit 173, a multi-view correction color image and the multi-view parallax images obtained as the result of decoding to the viewpoint synthesizing unit 152 of FIG. 15.
[0182]
　[Configuration example of the slice decoder]
　FIG. 17 is a block diagram showing a configuration example of a decoding unit for decoding the parallax images for any one viewpoint of the slice decoder 174 in FIG. 16. That is, the decoding unit for decoding a multi-view parallax images of the slice decoder 174 is composed of a decryption unit 250 of the visual point fraction Figure 17.
[0183]
　Decoding unit 250 in FIG. 17, storage buffer 251, reversible decoding unit 252, an inverse quantization unit 253, inverse orthogonal transform unit 254, addition unit 255, deblocking filter 256, a screen rearrangement buffer 257, D / A conversion unit 258 , the frame memory 259, intra prediction unit 260, motion vector generation unit 261, a motion compensation unit 262, correction unit 263, and a by the switch 264.
[0184]
　Storage buffer 251 of the decoding unit 250 receives the coded data of the parallax image of a predetermined viewpoint in units of slices from the slice header decoding unit 173 in FIG. 16, accumulates. Accumulation buffer 251 supplies the encoded data accumulated in the lossless decoding unit 252.
[0185]
　Lossless decoding unit 252 obtains the coded data from the storage buffer 251, a variable length decoding and, by performing the lossless decoding of the arithmetic decoding and the like, the quantized coefficients. Lossless decoding unit 252 supplies the quantized coefficients to the inverse quantization unit 253.
[0186]
　Inverse quantization unit 253, inverse orthogonal transform unit 254, addition unit 255, deblocking filter 256, a frame memory 259, intra prediction unit 260, motion compensation unit 262, and the correction unit 263, the inverse quantization unit 128 in FIG. 6 , inverse orthogonal transform unit 129, addition unit 130, deblocking filter 131, a frame memory 132, intra prediction unit 133 performs the respective same processing as the motion prediction compensation unit 134 and the correction unit 135, thereby, a predetermined parallax image of the view is decoded.
[0187]
　Specifically, the inverse quantization unit 253 inversely quantizes the quantized coefficients from the lossless decoding unit 252 supplies the coefficient obtained as a result to the inverse orthogonal transformation unit 254.
[0188]
　Inverse orthogonal transform unit 254, the coefficient from the inverse quantization unit 253, inverse discrete cosine transform, applies inverse orthogonal transformation, such as transformation inverse Karhunen-Loeve, supplying residual information obtained as a result to the adding unit 255 to.
[0189]
　Adding section 255 functions as a decoder, and the residual information as a parallax image to be decoded which is supplied from the inverse orthogonal transform unit 254, by adding the predicted image supplied from the switch 264, the decoded parallax to decode the image. Adding section 255 supplies the parallax images obtained as a result is supplied to a deblocking filter 256, the intra prediction unit 260 as a reference image. Incidentally, if the predicted image from the switch 264 is not supplied, the adding unit 255 supplies the parallax image is a residual information supplied from the inverse orthogonal transform unit 254 to the de-block filter 256, intra prediction unit as a reference image supplied to the 260.
[0190]
　Deblock filter 256 filters the parallax image supplied from the adder 255, to remove the block distortion. Deblocking filter 256 supplies the parallax images obtained as a result to the frame memory 259, together with the to accumulate, and supplies to the screen rearrangement buffer 257. Stored parallax image in the frame memory 259 is supplied to the motion compensation unit 262 as a reference image.
[0191]
　Screen rearrangement buffer 257 stores the parallax image supplied from the deblocking filter 256 in units of frames. Screen sorting buffer 257, the parallax images on a frame-by-frame basis of the order for the stored coded, sorted in the order of the original display, supplied to the D / A conversion unit 258.
[0192]
　D / A conversion unit 258, a parallax image in units of frames supplied from the screen rearrangement buffer 257 converts D / A, and supplies the viewpoint synthesizing unit 152 (FIG. 15) as the parallax images of a predetermined viewpoint.
[0193]
　Intra prediction unit 260 performs by using the reference image supplied from the addition unit 255, a slice header decoding unit 173 intra prediction of the optimal intra prediction mode indicated by the intra prediction information supplied from (FIG. 16), and it generates a prediction image. Then, the intra prediction unit 260 supplies the predicted image to the switch 264.
[0194]
　Motion vector generating unit 261, from among the motion vectors that are held, by adding a motion vector representing the prediction vector index included in the motion information supplied from the slice header decoding unit 173, and a motion vector residual motion to restore the vector. Motion vector generation unit 261 holds the restored motion vector. The motion vector generation unit 261, a motion vector is reconstructed, and supplies the optimal inter prediction mode or the like included in the motion information to the motion compensation unit 262.
[0195]
　The motion compensation unit 262 functions as a predictive picture generating unit, based on the motion vector and the optimal inter prediction mode supplied from the motion vector generating unit 261, by reading the reference image from the frame memory 259, motion compensation processing . The motion compensation unit 262 supplies the predicted image to be the result generated in the correction unit 263.
[0196]
　Correction unit 263, similarly to the correcting unit 135 in FIG. 6, the parallax maximum value supplied from the slice header decoding unit 173 in FIG. 16, the parallax minimum value, and based on the inter-camera distance, when correcting the predicted image to generate a correction coefficient used. Further, the correction unit 263, similarly to the correcting unit 135, a prediction image of the optimal inter prediction mode supplied from the motion compensation unit 262 is corrected using the correction coefficient. Correcting unit 263 supplies the predicted image after correction to the switch 264.
[0197]
　Switch 264, when a predicted image from the intra prediction unit 260 is supplied, it supplies the predicted image to the adding unit 255, when a predicted image from the motion compensation unit 262 is supplied to the adding unit 255 and the predictive picture supplies.
[0198]
　[Explanation of the processing of the decoding apparatus]
　FIG 18 is a flowchart illustrating a decoding process of the decoding apparatus 150 of FIG. 15. The decoding process is, for example, when the encoded bit stream has been transmitted from the encoding device 50 of FIG. 1, is started.
[0199]
　In step S201 in FIG. 18, the multi-view image decoding unit 151 of the decoding device 150 receives a coded bit stream transmitted from the encoding device 50 of FIG. 1.
[0200]
　In step S202, the multi-view image decoding unit 151, performs a multi-view decoding process of decoding the received coded bit stream. Details of the multi-view decoding process will be described with reference to FIG. 19 described later.
[0201]
　In step S203, the viewpoint synthesizing unit 152, by using function as a color image generation unit, viewpoint generating information supplied from the multi-view image decoding unit 151, the multi-viewpoint corrected color image, and a multi-view parallax images, multi-view to generate a composite color image.
[0202]
　In step S204, the multi-viewpoint image display unit 153, a multi-view composite color image supplied from the viewpoint synthesizing unit 152, possible angles viewing is displayed differently for each viewpoint, the process is terminated.
[0203]
　Figure 19 is a flowchart illustrating the details of the multi-view decoding process in step S202 in FIG. 18.
[0204]
　In step S221 in FIG. 19, SPS decoding unit 171 (FIG. 16) of the multi-view image decoding unit 151 extracts the SPS of the received coded bit stream. SPS decoding unit 171 supplies the coded bit stream other than the extracted SPS and SPS to PPS decoding unit 172.
[0205]
　In step S222, PPS decoding section 172, to extract the PPS from SPS other than the encoded bit stream, which is supplied from the SPS decoding section 171. PPS decoding unit 172, the extracted PPS, SPS, and supplies the coded bit stream other than the SPS and PPS in the slice header decoding unit 173.
[0206]
　In step S 223, the slice header decoding unit 173, and supplies the viewpoint synthesizing unit 152 parallax accuracy parameters contained in the PPS supplied from PPS decoding unit 172 as a part of the viewpoint generating information.
[0207]
　In step S 224, the slice header decoding unit 173 determines the transmission flag contained in PPS from PPS decoding unit 172 whether the "1" representing the presence of the transmission. The processing of steps S225 to S234 of the subsequent is carried out in slice units.
[0208]
　If the transmission flag in step S224 is determined to be "1" representing the presence of the transmission, the process proceeds to step S 225. In step S 225, the slice header decoding unit 173, the SPS and PPS non-encoded bit stream supplied from the PPS decoding unit 172, the disparity maximum value, disparity minimum value, and the inter-camera distance or, parallax maximum value, disparity the minimum value, and extracts a slice header including a differential encoding result of the inter-camera distance.
[0209]
　In step S 226, the slice header decoding unit 173, determines whether the type of a slice is an intra type. If the slice type is determined to be the intra type in the step S 226, the process proceeds to step S 227.
[0210]
　In step S 227, the slice header decoding unit 173, holds the parallax minimum value included in the slice header extracted in step S 225, and supplies the viewpoint synthesizing unit 152 as a part of the viewpoint generating information.
[0211]
　In step S 228, the slice header decoding unit 173, holds the parallax maximum value included in the slice header extracted in step S 225, and supplies the viewpoint synthesizing unit 152 as a part of the viewpoint generating information.
[0212]
　In step S 229, the slice header decoding unit 173 holds the inter-camera distance included in the extracted slice header in step S 225, and supplies the viewpoint synthesizing unit 152 as a part of the viewpoint generating information. Then, the process proceeds to step S 235.
[0213]
　On the other hand, if the slice type in step S226 is determined not to be an intra type, that is, when the slice type is an inter type, the process proceeds to step S 230.
[0214]
　In step S 230, the slice header decoding unit 173 adds the differential encoding result of the disparity minimum value included in the slice header extracted in step S 225, the disparity minimum value held. Slice header decoding unit 173, and supplies the viewpoint synthesizing unit 152 parallax minimum value restored by the addition as a part of the viewpoint generating information.
[0215]
　In step S 231, the slice header decoding unit 173 adds the differential encoding result of the disparity maximum values ​​contained in the slice header extracted in step S 225, the disparity maximum value held. Slice header decoding unit 173, and supplies the viewpoint synthesizing section 152 the restored disparity maximum value by the addition as a part of the viewpoint generating information.
[0216]
　In step S 232, the slice header decoding unit 173, adds the inter-camera distance differential coding results of which are contained in the slice header extracted in step S 225, the inter-camera distance being held. Slice header decoding unit 173, and supplies the viewpoint synthesizing section 152 to inter-camera distance restored by the addition as a part of the viewpoint generating information. Then, the process proceeds to step S 235.
[0217]
　On the other hand, if the transmission flag is determined not to be "1" represents the presence of a transmission in step S 224, that is, if the transmission flag is "0" representing the absence of the transmission, the process proceeds to step S 233.
[0218]
　In step S 233, the slice header decoding unit 173, the SPS and non PPS coded bit stream supplied from the PPS decoding unit 172, the disparity maximum value, disparity minimum value, and the inter-camera distance, and the disparity maximum value, disparity the minimum value, and extracts a slice header including no differential encoding result of the inter-camera distance.
[0219]
　In step S 234, the slice header decoding unit 173, disparity maximum value held, the disparity minimum, and inter-camera distance, i.e. the disparity maximum value of the previous slice in coding order, the parallax minimum, and between the cameras distance, disparity maximum value of the slice to be processed, the parallax minimum, and by the inter-camera distance, the parallax maximum value of the slice to be processed, to recover parallax minimum, and the inter-camera distance. Then, the slice header decoding unit 173, the restored disparity maximum value, disparity minimum value, and the inter-camera distance, and supplies to the viewpoint synthesizing unit 152 as a part of the viewpoint generating information, the process proceeds to step S 235.
[0220]
　In step S 235, the slice decoder 174, in a manner corresponding to the coding method in the slice coding unit 61 (FIG. 5), decodes the encoded data in units of slices. Specifically, the slice decoder 174, based on SPS from slice header decoding unit 173, PPS, and the inter-camera distance, the parallax maximum value, and the slice header other than the information relating to the disparity minimum, the slice coding unit 61 in a manner corresponding to the encoding scheme in, it decodes the encoded data of the multi-view color image in units of slices. Further, the slice decoder 174, a slice SPS from the header decoding unit 173, PPS, inter-camera distance, the parallax maximum value, and disparity minimum value other than information about the slice header, and, the inter-camera distance, the parallax maximum value, and disparity based on the minimum value, in a manner corresponding to the coding method in the slice coding unit 61, performs a parallax image decoding processing for decoding encoded data of multi-viewpoint corrected image in units of slices. Details of the parallax image decoding process will be described with reference to FIG. 20 described later. Slice header decoding unit 173, a multi-view correction color image and the multi-view parallax images obtained as the result of decoding to the viewpoint synthesizing unit 152 of FIG. 15.
[0221]
　Figure 20 is a flowchart illustrating the details of the parallax image decoding processing of the slice decoder 174 in FIG. 16. The parallax image decoding processing is performed for each viewpoint.
[0222]
　In step S261 in FIG. 20, the storage buffer 251 of the decoding unit 250 receives the encoded data in units of slices of the parallax image of a predetermined viewpoint from the slice header decoding unit 173 in FIG. 16, it accumulates. Accumulation buffer 251 supplies the encoded data accumulated in the lossless decoding unit 252.
[0223]
　In step S 262, the reversible decoding unit 252, the coded data supplied from the storage buffer 251 and the lossless decoding and supplies the resulting quantized coefficients to the inverse quantization unit 253.
[0224]
　In step S 263, the inverse quantization unit 253 inversely quantizes the quantized coefficients from the lossless decoding unit 252 supplies the coefficient obtained as a result to the inverse orthogonal transformation unit 254.
[0225]
　In step S 264, the inverse orthogonal transformation unit 254 performs inverse orthogonal transform to the coefficient from the inverse quantization unit 253, supplies the residual information obtained as a result to the adder 255.
[0226]
　In step S 265, the motion vector generation unit 261 determines whether the motion information is supplied from the slice header decoding unit 173 in FIG. 16. If it is determined that the motion information is supplied in the step S 265, the process proceeds to step S 266.
[0227]
　In step S 266, the motion vector generation unit 261 restores the motion vector based on the motion vector holding the motion information held. Motion vector generation unit 261, a motion vector is reconstructed, and supplies the optimal inter prediction mode or the like included in the motion information to the motion compensation unit 262.
[0228]
　In step S 267, the motion compensation unit 262, based on the motion vector and the optimal inter prediction mode supplied from the motion vector generating unit 261 performs motion compensation processing by reading the reference image from the frame memory 259. The motion compensation unit 262 supplies the predicted image generated as a result of the motion compensation processing in the correction unit 263.
[0229]
　In step S 268, the correction unit 263, similarly to the correcting unit 135 in FIG. 6, the parallax maximum value supplied from the slice header decoding unit 173 in FIG. 16, the parallax minimum value, and based on the inter-camera distance, the correction coefficient calculate.
[0230]
　In step S 269, the correction unit 263, similarly to the correcting unit 135, a prediction image of the optimal inter prediction mode supplied from the motion compensation unit 262 is corrected using the correction coefficient. Correcting unit 263, a prediction image after the correction, via a switch 264 supplies to the adder 255, the process proceeds to step S 271.
[0231]
　On the other hand, if the motion information is determined not to be supplied in the step S 265, that is, when the intra prediction information is supplied to the intra prediction unit 260 from the slice header decoding unit 173, the process proceeds to step S 270.
[0232]
　In step S 270, the intra prediction unit 260 is performed, the intra prediction processing of the optimum intra prediction mode indicated by the intra prediction information supplied from the slice header decoding unit 173 using the reference image supplied from the adder 255 . Intra prediction unit 260, a prediction image that is a result generated, via the switch 264 supplies to the adder 255, the process proceeds to step S 271.
[0233]
　In step S 271, the adding unit 255 adds the residual information supplied from the inverse orthogonal transform unit 254, a prediction image supplied from the switch 264. Adding section 255 supplies the parallax images obtained as a result is supplied to a deblocking filter 256, the intra prediction unit 260 as a reference image.
[0234]
　In step S 272, the deblocking filter 256 performs filtering on the parallax image supplied from the adder 255, to remove the block distortion.
[0235]
　In step S273, the deblocking filter 256 supplies the parallax image after filtering in the frame memory 259, along with the to accumulate, and supplies it to the screen sorting buffer 257. Stored parallax image in the frame memory 259 is supplied to the motion compensation unit 262 as a reference image.
[0236]
　In step S 274, the screen rearranging buffer 257 stores the parallax image supplied from the deblocking filter 256 in units of frames, the parallax images in units of frames of the sequence for storing coded, the order of the original display Sort, supplied to the D / A conversion unit 258.
[0237]
　In step S275, D / A conversion section 258 supplies the parallax images in units of frames supplied from the screen rearrangement buffer 257 converts D / the A, the viewpoint combining unit 152 in FIG. 15 as the parallax image of a predetermined viewpoint .
[0238]
　As described above, the decoding device 150, the coded data of the parallax image coding efficiency is improved by coding with reference to predictive image corrected using information on the parallax image, information about the parallax image receive an encoded bit stream, including. Then, the decoding device 150, corrects the predicted image by using the information about the parallax image, decodes encoded data of the parallax image by using the prediction image after correction.
[0239]
　More particularly, the decoding device 150, inter-camera distance as information on a parallax image, the disparity maximum value, and the encoded data by using the prediction image corrected by using the parallax minimum value, the inter-camera distance receives disparity maximum value, and the parallax minimum value. Then, the decoding device 150, the inter-camera distance, the parallax maximum value, and corrects the predicted image by using the parallax minimum value, decodes the encoded data of the parallax image by using the prediction image after correction. Thus, the decoding apparatus 150 can decode the encoded data of the parallax image coding efficiency is improved by coding with reference to predictive image corrected by using the information on the parallax image.
[0240]
　Incidentally, the encoding apparatus 50, as information used for correction of the prediction image, the disparity maximum value, disparity minimum value, and transmitting the inter-camera distance is included in the slice header, the transmission method is not limited thereto.
[0241]
　[Description of method of transmitting information used for correction of the predicted image]
　FIG. 21 is a diagram for explaining a method for transmitting information used for correction of the prediction image.
[0242]
　The first transmission method of FIG. 21, as described above, as the information used for correction of the prediction image, the disparity maximum value, a method of transmitting including disparity minimum, and the inter-camera distance in the slice header. In this case, to share information and viewpoint generating information used for correction of the prediction image, it is possible to reduce the information amount of the coded bit stream. However, the decoding apparatus 150, the disparity maximum value, disparity minimum, and it is necessary to calculate a correction coefficient using inter-camera distance, the processing load of the decoding device 150 is larger than the second transmission method described below.
[0243]
　On the other hand, a second transmission method of FIG. 21 is a method for transmitting included in the slice header correction coefficient itself as information used for correction of the prediction image. In this case, the disparity maximum value, disparity minimum value, and the inter-camera distance, because not used for the correction of the predicted image, as a part of the viewpoint generating information, for example, need not be referred to when coding SEI (Supplemental Enhancement is transmitted is included in the Information). In the second transmission method, the correction coefficient is transmitted, there is no need to calculate correction coefficients in the decoding apparatus 150, the processing load of the decoding device 150 is smaller than that of the first transmission method. However, since the correction coefficient is newly transmitted, the information amount of the coded bit stream increases.
[0244]
　Incidentally, in the above description, the prediction image, the disparity maximum value, disparity minimum, and has been corrected by using the inter-camera distance, the other parallax related information (e.g., multi-view color image pickup unit 51 in the depth direction it is also possible to be corrected by also using such image pickup position information) indicating the image pickup position.
[0245]
　In this case, the third method of the transmission of Figure 21, as the information used for correction of the prediction image, the disparity maximum value, disparity minimum value, between the camera distance, and other correction coefficients generated using information on parallax there additional correction factors are transmitted is included in the slice header. Thus, the disparity maximum value, disparity minimum value, and if the information about the parallax other than inter-camera distance be employed predictive image is corrected to reduce further the difference between the predicted image and the parallax image by the information relating to the disparity, the encoded thereby improving the efficiency. However, since the additional correction factor is newly transmitted, the information amount of the coded bit stream is increased in comparison with the first transmission method. Also, the disparity maximum value, disparity minimum, and since by using the inter-camera distance is necessary to calculate the correction coefficient, the processing load of the decoding apparatus 150 in comparison with the second transmission method is large.
[0246]
　Figure 22 is a diagram illustrating a configuration example of a coded bit stream in the case of transmitting information used for correction of the predicted image by the second transmission method.
[0247]
　In the example of FIG. 22, the correction coefficient of the slice of one intra types of slices and two inter-type constituting the same PPS units of PPS # 0, respectively, consistent with the correction factor of the previous slice in coding order do not do. Therefore, the PPS # 0 includes a transmitted flag "1" indicating the presence of a transmission. Here, the transmission flag is a flag indicating whether the transmission of the correction coefficient.
[0248]
　In the example of FIG. 22, the correction coefficient a of the intra type of slices that make up the same PPS units of PPS # 0 is 1, the correction coefficient b is zero. Therefore, the slice header of the slice comprises the correction coefficient b "0" and the correction coefficient a "1".
[0249]
　Further, in the example of FIG. 22, the first correction coefficient a of the inter type of slices constituting the same PPS units of PPS # 0 is 3, the correction coefficient b is 2. Therefore, the slice header of the slice, from the correction coefficient slice a "3", the difference "+2" obtained by subtracting "1" correction coefficient a of the previous intra-type slice coding order, It is included as a differential encoding result of the correction coefficient. Similarly, the difference "+2" of the correction coefficient b is included as differential encoding result of the correction coefficient b.
[0250]
　In the example of FIG. 22, the second correction coefficient a of the inter type of slices constituting the same PPS units of PPS # 0 is 0, the correction coefficient b is -1. Therefore, the slice header of the slice, from the correction coefficients a "0" on the slice, a difference "-3 obtained by subtracting the correction coefficient a" 3 "of the slice of the first inter-type one before in the encoding order "it is included as a differentially encoded result of the correction coefficient. Similarly, the difference "-3" of the correction coefficient b is included as differential encoding result of the correction coefficient b.
[0251]
　In the example of FIG. 22, the correction coefficient of one intra types of slices and two inter-type slices constituting the same PPS units of PPS # 1, respectively, the correction coefficient of the previous slice in coding order consistent with. Therefore, the PPS # 1 includes a transmitted flag "0" indicating the absence of transmission.
[0252]
　Figure 23 is a diagram illustrating a configuration example of a coded bit stream in the case of transmitting information used for correction of the prediction image in the third transmission method.
[0253]
　In the example of FIG. 23, PPS # 1 single parallax minimum of the intra-type slices and two inter-type slices constituting the same PPS units of 0, the disparity maximum value, the inter-camera distance, and additional correction factor, respectively, parallax minimum of the previous slice in coding order, the parallax maximum value, the inter-camera distance, and do not match the additional correction factor. Therefore, the PPS # 0 includes a transmitted flag "1" indicating the presence of a transmission. Here, the transmission flag, disparity minimum value, a flag indicating the presence or absence of transmission of the disparity maximum value, the inter-camera distance, and additional correction factor.
[0254]
　In the example of FIG. 23, the parallax minimum value of slices constituting the same PPS units of PPS # 0, the disparity maximum value, and the inter-camera distance is the same as in the case of FIG. 7, included in the slice header of each slice parallax minimum value, disparity maximum value, and since the information about the inter-camera distance is the same as in FIG. 7, description thereof is omitted.
[0255]
　In the example of FIG. 23, additional correction coefficient of the intra type of slices that make up the same PPS units of PPS # 0 is 5. Therefore, the slice header of the slice, includes additional correction coefficient "5".
[0256]
　Further, in the example of FIG. 23, additional correction factor of the first inter-type slices constituting the same PPS units of PPS # 0 is seven. Therefore, the slice header of the slice, from the additional correction coefficient "7" of the slice, the difference "+2" obtained by subtracting "5" Additional correction factors of the previous intra-type slice coding order, It is included as a differential encoding a result of additional correction coefficient.
[0257]
　In the example of FIG. 23, additional correction factor for the second inter-type slices constituting the same PPS units of PPS # 0 is 8. Therefore, the slice header of the slice, from the additional correction factor "8" of the slice, the difference "+1 obtained by subtracting the added correction coefficient" 7 "of the previous first inter-type slice in coding order "it is included as a differential encoding result of additional correction coefficient.
[0258]
　In the example of FIG. 23, PPS # 1 single parallax minimum of the intra-type slices and two inter-type slices constituting the same PPS units of 1, parallax maximum value, the inter-camera distance, and additional correction factor, respectively, parallax minimum value of the previous slice in coding order, the disparity maximum value, inter-camera distance, and consistent with the additional correction factor. Therefore, the PPS # 1 includes a transmitted flag "0" indicating the absence of transmission.
[0259]
　Encoder 50, by any method of the first to third methods of the transmission of Figure 21, may be transmitted information used for correction of the predicted image. The encoding apparatus 50 has adopted as a transmission method, identification information for identifying one transmission method of the first to third methods of transmission (e.g., a flag, IDs, etc.), included in the coded bit stream it may be transmitted on. Further, the first to third methods of the transmission of Figure 21, depending on the application using the coded bit stream, be appropriately selected in consideration of the balance between the amount of data and the decoding of the processing load of the coded bit stream possible it is.
[0260]
　Further, in this embodiment, information used for correction of the prediction image has been placed in the slice header as information regarding encoding, placement area information used for correction of the predicted image is referred to when coding if area is not limited to the slice header. For example, information used for correction of the prediction image, and existing NAL (Network Abstraction Layer) unit, such as a PPS NAL unit, a new NAL such NAL units of the APS proposed in HEVC standard (Adaptation Parameter Set) it can be made to be placed in the unit.
[0261]
　For example, when the correction coefficient and the additional correction factor is common among a plurality of pictures, the plurality of adaptive NAL unit in the picture (for example, NAL units, etc. PPS) to, by placing the common value, thereby improving the transmission efficiency. That is, in this case, since a common correction coefficient and adding the correction coefficient among a plurality of pictures may be transmitted, as in the case of arranged in the slice header, there is no need to transmit correction coefficient and adding the correction factor for each slice .
[0262]
　Thus, for example, a color image, when a color image having a flash and fading effects, parallax minimum value, since the disparity maximum value, parameters such as inter-camera distance and the like tend not to change the correction coefficient and the additional correction factor It is arranged like NAL unit of PPS, improving the transmission efficiency.
[0263]
　Correction factor and adding the correction factor arrangement, for example, be different for each picture, are arranged in the slice header, if it is shared among a plurality of pictures, the layer above the slice header (e.g., PPS NAL unit, etc.) can do.
[0264]
　Further, the parallax image may be an image made up of depth value representative of the position of the depth direction of the object for each pixel of the viewpoint color image corresponding to the parallax image. In this case, the disparity maximum value and the parallax minimum value, Zorezore, the maximum value of the world coordinate in the depth direction of the position that can be taken in the multi-viewpoint parallax image, the minimum value.
[0265]
　Further, the techniques can be applied to coding systems such as AVC than HEVC scheme, MVC (Multiview Video Coding).
[0266]
　
　[Description of Computer to which the present technology]
　Next, the series of processes described above can be executed by hardware, it may otherwise be executed by software. When the series of processes by software, a program constituting the software is installed in a general-purpose computer or the like.
[0267]
　Therefore, FIG. 24 shows a configuration example of an embodiment of a computer program for executing the series of processes described above is installed.
[0268]
　The program can be recorded in advance in the storage unit 808 or the ROM (Read Only Memory) 802 as a recording medium built in the computer.
[0269]
　Alternatively, the program may be stored in the removable medium 811 (recording). The removable medium 811 may be provided as so-called package software. Examples of the removable medium 811, for example, a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto Optical) disk, DVD (Digital Versatile Disc), magnetic disk, and semiconductor memory.
[0270]
　The program may be installed from the removable medium 811 as described above into the computer via the drive 810, via a communication network or broadcasting network and can be installed in the storage unit 808 to download the computer, built. That is, the program, for example, from a download site via an artificial satellite for digital satellite broadcasting, or transferred wirelessly to the computer, LAN (Local Area Network), via a network such as the Internet, is transferred by wire to the computer be able to.
[0271]
　Computer, CPU incorporates a (Central Processing Unit) 801, the CPU 801, via a bus 804, output interface 805 is connected.
[0272]
　CPU801 through the input-output interface 805, the user, when the command is input by the input unit 806 is operated or the like, the CPU 102 executes the program stored in the ROM 802. Alternatively, CPU 801 may stored the program in a storage unit 808, and executes the loaded to RAM (Random Access Memory) 803.
[0273]
　Thus, CPU 801 performs the processes according to the flowcharts described above or processing performed by configurations in the above-described block diagrams. Then, CPU 801 is the processing result, as required, for example, via the input-output interface 805, an output from the output unit 807, or transmits from the communication unit 809, or further records such as the storage unit 808.
[0274]
　The input unit 806 includes a keyboard, a mouse, a microphone or the like. Further, the output unit 807, an LCD (Liquid Crystal Display), a speaker or the like.
[0275]
　In this specification, the processing computer performs in accordance with the program need not be performed chronologically according to the order described as a flow chart. That is, the process in which the computer performs in accordance with the program also includes processing executed in parallel or individually (e.g., parallel processing or object processing).
[0276]
　Further, the program may be one that is processed by a single computer (processor), it may be subjected to distributed processing by a plurality of computers. Furthermore, the program, or may be executed by being transferred to a remote computer.
[0277]
　This technology, satellite broadcasting, cable TV (television), when to communicate through the network media such as the Internet, and mobile phone, or, light, to process on a storage medium such as a magnetic disk and flash memory, it can be applied to the encoding apparatus and the decoding apparatus used in.
[0278]
　Further, the above-described encoding apparatus and decoding apparatus can be applied to any electronic device. It explained the example below.
[0279]
　
　[television device configuration example of]
　FIG 25 illustrates a schematic structure of a television device according to the present technique. Television apparatus 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, display unit 906, audio signal processor 907, a speaker 908, and has an external interface unit 909. Further, the television apparatus 900, the control unit 910, and a user interface unit 911 or the like.
[0280]
　The tuner 902 performs demodulation selects a desired channel from a broadcast wave signal received by the antenna 901, and outputs the resulting coded bit stream to the demultiplexer 903.
[0281]
　The demultiplexer 903 extracts the program video and audio packets to be encoded bit stream from the viewing object, and outputs the data of the extracted packet to the decoder 904. Further, the demultiplexer 903 supplies the packet data such as EPG (Electronic Program Guide) to the control unit 910. In the case where scrambling is performed, to release the scramble by the demultiplexer and the like.
[0282]
　The decoder 904 performs decoding processing of the packet, and outputs the image data generated by the decoding process of the video signal processor 905, the audio data to the audio signal processing unit 907.
[0283]
　The video signal processing unit 905, the video data, performs video processing and the like in accordance with the noise elimination and user settings. The video signal processing unit 905, and video data of a program to be displayed on the display unit 906, and generates an image data by the processing based on the application supplied via a network. The video signal processing unit 905 generates image data for displaying a menu screen or the like as selection item, is superimposed it to the program of the video data. The video signal processing unit 905 generates a driving signal for driving the display unit 906 based on the image data generated in this way.
[0284]
　Display unit 906 drives the display device based on a drive signal from the video signal processing unit 905 (e.g. a liquid crystal display element or the like), and displays and the video of the program.
[0285]
　The audio signal processing unit 907 performs predetermined processing such as noise removal on the audio data, performs audio output by supplying to the speaker 908 performs D / A conversion processing and amplification processing of the audio data after processing.
[0286]
　The external interface unit 909 is an interface for connecting to an external device or network, performs data transmission and reception of video data and audio data.
[0287]
　The user interface unit 911 is connected to the control unit 910. The user interface unit 911, the operation consists of a switch or the remote control signal receiving unit or the like, and supplies an operation signal corresponding to the user operation to the control unit 910.
[0288]
　The control unit 910 is configured with a CPU (Central Processing Unit), memory and the like. The memory stores various necessary data in terms of program or CPU that runs performs processing by CPU, EPG data, the data acquired via a network or the like. Program stored in the memory is read and executed by the CPU at a predetermined timing such as during start-up of the television apparatus 900. The CPU executes the program to control each unit so that the operation of the television apparatus 900 according to the user operation.
[0289]
　Incidentally, the television apparatus 900, a tuner 902, a demultiplexer 903, a video signal processing unit 905, audio signal processor 907, a bus 912 for connecting the control unit 910 and the external interface unit 909 or the like are provided.
[0290]
　In the thus constructed television apparatus, the function of the decoding apparatus of the present application (decoding method) is provided to the decoder 904. Therefore, it is possible to decode the encoded data of the parallax image coding efficiency is improved by coding using information on parallax images.
[0291]
　
　[Configuration example of a cellular phone]
　FIG. 26 illustrates a schematic configuration of a cellular phone according to the present technique. The mobile phone 920, a communication unit 922, voice codec 923, a camera unit 926, image processing unit 927, the demultiplexing unit 928, a recording reproduction unit 929, the display unit 930 has a control unit 931. These are connected to each other via a bus 933.
[0292]
　Further, the communication unit 922 has an antenna 921 is connected to the voice codec 923, speaker 924 and a microphone 925 are connected. More control unit 931, operation unit 932 is connected.
[0293]
　The mobile phone 920, in a variety of modes such as voice communication mode and data communication mode, perform transmission and reception of the audio signal, send and receive e-mail and image data, imaging, or the various operations of the data recording and the like.
[0294]
　In voice communication mode, audio signals generated by the microphone 925, the voice codec 923 converts and data compression to the audio data supplied to the communication unit 922 is performed. The communication unit 922 performs modulation processing and frequency conversion processing and the like of the audio data to generate a transmission signal. Further, the communication unit 922 transmits the transmission signal to the base station (not shown) is supplied to the antenna 921. The communication unit 922 performs amplification and frequency conversion processing and demodulation processing of the received signals received by the antenna 921, and supplies the resulting audio data to the audio codec 923. Audio codec 923, and outputs it to a speaker 924 performs conversion into data decompression, analog audio signals of the audio data.
[0295]
　Further, in the data communication mode, when performing the mail transmission, the control unit 931 accepts a character data inputted by operating the operation unit 932, the display unit 930 to input characters. Further, the control unit 931 supplies the communication unit 922 to generate the mail data based on the user instruction or the like in the operation unit 932. The communication unit 922 performs modulation processing and frequency conversion processing and the like of the mail data, and transmits a transmission signal obtained from the antenna 921. The communication unit 922 performs amplification and frequency conversion processing and demodulation processing of the received signals received by the antenna 921, restores the mail data. The mail data is supplied to the display unit 930 performs display of the mail content.
[0296]
　Incidentally, the mobile phone 920, a mail data received, it is also possible to store in the storage medium by the recording and reproducing unit 929. A storage media may be any storage medium rewritable. For example, the storage medium, RAM and built-in flash memory or the like of a semiconductor memory, hard disk, magnetic disk, optical magnetic disk, an optical disk, a USB memory or a removable medium such as a memory card.
[0297]
　When transmitting image data in the data communication mode, and supplies the image data generated by the camera unit 926, the image processing unit 927. The image processing unit 927 performs encoding processing of the image data to generate encoded data.
[0298]
　Demultiplexing unit 928, the encoded data generated by the image processing unit 927, and supplies to the communication unit 922 multiplexes the voice data supplied in a predetermined manner from the audio codec 923. The communication unit 922 performs modulation processing and frequency conversion processing of the multiplexed data, and transmits a transmission signal obtained from the antenna 921. The communication unit 922 performs amplification and frequency conversion processing and demodulation processing of the received signals received by the antenna 921, restores the multiplexed data. And it supplies the multiplexed data to the demultiplexing unit 928. Demultiplexing unit 928 performs the separation of the multiplexed data, and supplies the encoded data image processing unit 927, the audio data to the audio codec 923. The image processing unit 927 performs decoding processing of the encoded data, generates image data. And supplies this image data to the display unit 930, and displays the received image. Audio codec 923 is supplied to a speaker 924 converts the audio data into analog audio signals, outputting an audio received.
[0299]
　In the thus constructed portable telephone device, the function of the image processing unit 927 present in the coding apparatus and decoding apparatus (coding method and decoding method) is provided. Therefore, it is possible to improve the coding efficiency of parallax images using information on the parallax image. Further, it is possible to decode the encoded data of the parallax image coding efficiency is improved by coding using information on parallax images.
[0300]
　
　[Configuration example of a recording and reproducing apparatus]
　Fig. 27 illustrates the schematic configuration of a recording and reproducing apparatus according to the present technique. Recording reproducing apparatus 940, the audio data and the video data of the broadcast program instance received, recorded on the recording medium, provided to the user at a timing corresponding to the recorded data to a user instruction. Further, the recording and reproducing apparatus 940, for example, acquires the audio data and video data from another apparatus can also be recorded on them recording medium. Further, the recording and reproducing apparatus 940, by outputting the decoded audio data and video data recorded on the recording medium, to be able to display an image and sound output on the monitor device or the like.
[0301]
　Recording and reproducing apparatus 940 includes a tuner 941, the external interface unit 942, encoder 943, HDD (Hard Disk Drive) 944, disk drive 945, the selector 946, decoder 947, OSD (On-Screen Display) unit 948, the control unit 949, and a user interface unit 950.
[0302]
　The tuner 941 selects a desired channel from the broadcast signal received by an antenna (not shown). The tuner 941 outputs a coded bit stream obtained by demodulating the reception signal of a desired channel to the selector 946.
[0303]
　The external interface unit 942, IEEE 1394 interface, a network interface unit cable, USB interface is composed of at least one of such as a flash memory interface. The external interface unit 942 is an interface for connecting an external device or a network, a memory card, etc. and performs data reception such as video data and audio data to be recorded.
[0304]
　The encoder 943 performs encoding in a predetermined manner when the video data and audio data supplied from the external interface unit 942 has not been encoded, and outputs a coded bit stream to the selector 946.
[0305]
　The HDD unit 944, recorded content data such as video and audio, various programs and other data, and the like in a built-in hard disk, and reads them from the hard disk to the reproduction or the like.
[0306]
　Disk drive 945 performs recording and reproducing of a signal on the optical disk that is mounted. Optical disc, for example DVD disc (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.) or a Blu-ray disc or the like.
[0307]
　The selector 946, when the video and audio recording, and select one of the encoded bit stream from the tuner 941 or the encoder 943, and supplies to any one of the HDD unit 944 and disk drive 945. The selector 946, when reproduction of video and audio, supplies the output encoded bit stream from the HDD unit 944 or disk drive 945 to the decoder 947.
[0308]
　The decoder 947 performs a decoding process of the coded bit stream. The decoder 947 supplies the image data generated by performing decoding processing of the OSD unit 948. Further, the decoder 947 outputs audio data generated by performing a decoding process of.
[0309]
　OSD unit 948 generates video data for displaying a menu screen or the like as selection items, and outputs it superimposed on the video data output from the decoder 947.
[0310]
　The control unit 949, a user interface unit 950 is connected. The user interface unit 950, the operation consists of a switch or the remote control signal receiving unit or the like, and supplies an operation signal corresponding to the user operation to the control unit 949.
[0311]
　The control unit 949 is configured with a CPU, a memory, and the like. The memory stores various data required in terms of program or CPU that is executed by the CPU performs the processing. Program stored in the memory is read and executed by the CPU at a predetermined timing, such as during start of the recording and reproducing apparatus 940. The CPU executes the program, the recording and reproducing apparatus 940 controls each unit so that the operation corresponding to the user operation.
[0312]
　In the thus constructed recording apparatus, the function of the decoding apparatus of the present application (decoding method) is provided to the decoder 947. Therefore, it is possible to decode the encoded data of the parallax image coding efficiency is improved by coding using information on parallax images.
[0313]
　
　[Configuration Example of Imaging Apparatus]
　FIG. 28 illustrates the schematic configuration of the applied imaging apparatus to which the present technology. Imaging device 960, imaging an object, or to display the image of the object on the display unit, it as the image data, is recorded on the recording medium.
[0314]
　Imaging device 960 includes an optical block 961, an imaging unit 962, a camera signal processing unit 963, the image data processing unit 964, display unit 965, external interface unit 966, a memory unit 967, media drive 968, OSD unit 969, a control unit 970 It has. The control unit 970, a user interface unit 971 is connected. Further, the image data processing unit 964 and the external interface unit 966, a memory unit 967, media drive 968, OSD unit 969, the control unit 970 or the like, are connected through a bus 972.
[0315]
　The optical block 961 is formed by using the focus lens, an aperture mechanism, and the like. The optical block 961, for forming an optical image of a subject on the imaging surface of the imaging unit 962. Imaging unit 962 is configured with a CCD or CMOS image sensors, and supplies to the camera signal processing unit 963 generates an electric signal corresponding to an optical image by the photoelectric conversion.
[0316]
　The camera signal processing unit 963, knee correction, gamma correction, various types of camera signal processing such as color correction performed on the electric signal supplied from the imaging unit 962. The camera signal processing unit 963 supplies the image data after the camera signal processing to the image data processing unit 964.
[0317]
　The image data processing unit 964 performs encoding processing of the image data supplied from the camera signal processing unit 963. The image data processing unit 964 supplies the encoded data generated by performing coding processing to the external interface unit 966 and media drive 968. Further, the image data processing unit 964 performs a decoding process of encoded data supplied from the external interface unit 966 and media drive 968. The image data processing unit 964 supplies to the display unit 965 the image data generated by performing the decoding process. Further, the image data processing unit 964, the processing and supplies it to the display unit 965 the image data supplied from the camera signal processing unit 963, the display data acquired from the OSD unit 969, the display unit 965 is superimposed on the image data supplied to.
[0318]
　OSD unit 969, symbols, and generates display data such as a menu screen or an icon consisting of characters or graphics, and outputs the image data processing unit 964.
[0319]
　The external interface unit 966, for example, be constituted by a USB input and output terminals, when printing an image, is connected to the printer. Further, the external interface unit 966 is connected to the drive as required, a magnetic disk, removable medium such as an optical disk is mounted as appropriate, a computer program read out therefrom, if required, be installed. Furthermore, external interface unit 966 has a network interface connected to a predetermined network such as a LAN or the Internet. Control unit 970, for example, in accordance with an instruction from the user interface unit 971, reads the encoded data from the memory unit 967, it from the external interface unit 966, it is fed to another device connected via a network it can. Further, the control unit 970, encoded data or image data supplied from another apparatus via a network, be obtained via the external interface unit 966, or supplies it to the image data processing unit 964 it can.
[0320]
　The recording medium driven by the media drive 968, for example, a magnetic disk, a magneto-optical disk, an optical disk or a semiconductor memory, any removable media that can be read and written, are used. Further, the recording media, type of the removable medium also optional, may be a tape device, may be a disc, may be a memory card. Of course, it may be a non-contact IC card or the like.
[0321]
　Further, a medium drive 968 and the recording medium are integrated, for example, as such as built-in hard disk drive or SSD (Solid State Drive), may be constituted by a non-portable storage medium.
[0322]
　The control unit 970 is configured with a CPU, a memory, and the like. The memory stores various kinds of data required in terms of program or CPU to be executed perform processing by the CPU. Program stored in the memory is read and executed by the CPU at a predetermined timing such as during start-up of the image pickup device 960. The CPU executes the program, the imaging apparatus 960 controls the respective units so that the operation corresponding to the user operation.
[0323]
　In such configured image pickup apparatus, the function of the image data processing unit 964 encoding device and the decoding device of the present application (the encoding method and decoding method) is provided. Therefore, it is possible to improve the coding efficiency of parallax images using information on the parallax image. Further, it is possible to decode the encoded data of the parallax image coding efficiency is improved by coding using information on parallax images.
[0324]
　Embodiments of the present technique is not intended to be limited to the embodiments described above, but various modifications are possible without departing from the scope of the present technique.
[0325]
　The present technique can also take the following configuration.
[0326]
　(1)
　using the information about the parallax image of the reference viewpoint, and a correction unit that corrects the predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by said correcting unit, a parallax image of the reference viewpoint an encoding unit for encoding, the
　a transmission unit for transmitting the information about the parallax image of the parallax image and the reference viewpoint of the reference viewpoint encoded by the encoding unit
　encoding apparatus comprising a.
　(2)
　said information on a parallax image of the reference viewpoint, the viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint hints,
　the transmission unit, the viewpoint generation information, and transmits the disparity image of the reference viewpoint as an encoding parameter used for encoding
　the encoding apparatus according to the above (1).
　(3)
　The correction unit uses a correction coefficient used when correcting the prediction image based on the viewpoint generation information, the predicted image is corrected,
　the transmission unit, the correction coefficient, the coded transmitted as a parameter
　coding apparatus according to (2).
　(4)
　Information about the parallax image of the reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the correcting unit, based on the image pickup position information the prediction image by correcting the predictive image using the correction coefficient and said viewpoint generating information used for correcting,
　the transmission unit, the correction factor is transmitted as the coding parameters
　described in (2) encoding device.
　(5)
　The correcting unit sets the correction coefficient
　encoding apparatus according to the above (3) or (4).
　(6)
　the coding unit, a parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　encoding apparatus according to any one of (1) to (5).
　(7)
　the reference viewpoint predicted image generation unit for generating a predicted image of parallax images
　further comprises a
　coding device according to any one of (1) to (6).
　(8)
　encoder is,
　by using the information about the parallax image of the reference viewpoint, and a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by the processing of the correction step an encoding step of encoding the parallax image of the reference viewpoint,
　and a transmission step of transmitting the information about the parallax image of the parallax image and the reference viewpoint of the reference viewpoint encoded by the processing of the encoding step
　the coding method comprising.
　(9)
　and the parallax image coded the reference viewpoint using a predicted image of parallax images of reference viewpoint the reference viewpoint corrected using information on parallax images, and information related to the parallax image of the reference viewpoint a receiving unit that receives,
　using information on parallax image of the reference viewpoint received by the receiving unit, a correcting unit that corrects the predicted image of parallax images of the reference viewpoint,
　the predicted image corrected by said correction unit using, received by the receiving unit, a decoding unit for decoding the parallax images coded the reference viewpoint
　decoding apparatus comprising a.
　(10)
　and the parallax image of the reference viewpoint decoded by the decoding unit, by using the color image of the reference viewpoint, the color image generation unit that generates a color image of a different viewpoint and the reference viewpoint
　further comprising a
　said information about the parallax image of the reference viewpoint, comprising the viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint,
　the receiving unit receives the viewpoint generating information to be transmitted as an encoding parameter for use in decoding the parallax images coded the reference viewpoint,
　the color image generation unit is decoded the reference by said decoder using perspective and parallax images, and a color image of the reference viewpoint, and the viewpoint generating information received by said receiving unit, to generate a color image of the view different from the reference viewpoint
　decoding according to the above (9) apparatus.
　(11)
　The receiving unit receives the correction coefficient used for correction of the prediction image when the coding is transmitted as the coding parameter,
　the correction coefficient is generated on the basis of the viewpoint generation information,
　the correction parts are by using the correction coefficient received by said receiving unit, corrects the predicted image
　decoding apparatus according to the above (10).
　(12)
　information on the parallax image of the reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the receiving unit is transmitted as the coding parameter that the time to encode receive a correction coefficient used for correction of the prediction image,
　the correction unit uses the viewpoint generation information and the correction coefficient received by said receiving unit, corrects the predicted image
　above ( decoding apparatus according to 10).
　(13)
　said correction coefficient is set based on the viewpoint generation information
　decoding apparatus according to (12).
　(14)
　a parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　decoding apparatus according to any one of (9) to (13).
　(15)
　the reference viewpoint predicted image generation unit for generating a predicted image of parallax images
　further comprises a
　decoding device according to any one of (9) to (14).
　(16)
　decoding apparatus,
　the parallax images coded the reference viewpoint using a predicted image of parallax images of the reference viewpoint corrected using information on parallax image of the reference viewpoint, the parallax image of the reference viewpoint a receiving step of receiving the information about,
　using information on parallax image of the reference viewpoint received by the processing of the receiving step, a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　the correction step using the predictive image corrected by the processing, the received by the processing of the receiving step, a decoding step of decoding parallax images coded the reference viewpoint
　decoding method comprising.
Description of the code
[0327]
　50 encoding device, 64 SPS coding section, 123 computing unit, 134 motion prediction and compensation unit, 135 correction unit, 150 decoding apparatus, 152 point of view synthesis unit, 171 SPS decoding section, 255 adding section, 262 motion compensation unit, 263 correction unit
The scope of the claims
[Claim 1]
　Using information on parallax image of the reference viewpoint, the correction unit that corrects the predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by said correcting unit, encodes the parallax image of the reference viewpoint an encoding unit, for
　a transmission unit for transmitting the information about the parallax image of the parallax image and the reference viewpoint of the reference viewpoint encoded by the encoding unit
　encoding apparatus comprising a.
[Claim 2]
　Information about the parallax image of the reference viewpoint includes viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint,
　the transmission unit, the viewpoint generation information, and transmits the disparity image of the reference viewpoint as an encoding parameter used for encoding
　the encoding apparatus according to claim 1.
[Claim 3]
　The correction unit uses a correction coefficient used when correcting the prediction image based on the viewpoint generation information, the predicted image is corrected,
　transmitting the transmission unit, the correction coefficient, as the coding parameter and
　coding apparatus according to claim 2.
[Claim 4]
　Information about the parallax image of the reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the correction unit, the predicted image on the basis of the image pickup position information correcting the predicted image using the correction coefficient and said viewpoint generating information used for correcting,
　the transmission unit, the correction factor is transmitted as the coding parameter
　encoding apparatus according to claim 2 .
[Claim 5]
　The correction unit sets the correction coefficient
　encoding apparatus according to claim 4.
[6.]
　The encoding unit, a parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　encoding apparatus according to claim 5.
[7.]
　It predicted image generation unit for generating a predicted image of parallax images of the reference viewpoint
　further comprising an
　encoding device according to claim 1.
[8.]
　Coding apparatus,
　using information on parallax image of the reference viewpoint, and a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by the processing of the correction step, the reference an encoding step of encoding the parallax image of the view,
　a transmission step of transmitting the information about the parallax image of the parallax image and the reference viewpoint encoded the reference viewpoint by the process of the encoding step
　encodes containing Method.
[9.]
　Receiving unit for receiving a parallax image coded the reference viewpoint using information on parallax image of the reference viewpoint by using a predicted image of parallax images corrected the reference viewpoint, and information about the parallax image of the reference viewpoint When,
　by using the information about the parallax image of the reference viewpoint received by the receiving unit, a correcting unit that corrects the predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by said correction unit , received by the receiving unit, a decoding unit for decoding the parallax images coded the reference viewpoint
　decoding apparatus comprising a.
[10.]
　And a parallax image of the reference viewpoint decoded by the decoding unit, by using the color image of the reference viewpoint, the color image generation unit that generates a color image of a different viewpoint and the reference viewpoint
　further comprising a,
　of the reference viewpoint information about the parallax image includes viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint,
　the receiving unit receiving said viewpoint generating information to be transmitted as an encoding parameter for use in decoding the parallax images coded the reference viewpoint,
　the color image generation unit, a parallax of the reference viewpoint decoded by the decoding unit using an image, a color image of the reference viewpoint, and the viewpoint generating information received by said receiving unit, to generate a color image of the view different from the reference viewpoint
　decoding apparatus according to claim 9.
[11.]
　The receiving unit receives the correction coefficient used for correction of the prediction image when the coding is transmitted as the coding parameter,
　the correction coefficient is generated on the basis of the viewpoint generation information,
　the correction unit, by using the correction coefficient received by said receiving unit, corrects the predicted image
　decoding apparatus according to claim 10.
[12.]
　Information about the parallax image of the reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the receiving unit is coded to be transmitted as the coding parameter sometimes receives a correction coefficient used for correction of the prediction image,
　the correction unit uses the viewpoint generation information and the correction coefficient received by said receiving unit, corrects the predicted image
　according to claim 10 decoding apparatus.
[13.]
　The correction coefficient is set based on the viewpoint generation information
　decoding apparatus according to claim 12.
[14.]
　Parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　decoding apparatus according to claim 13.
[15.]
　Predicted image generation unit for generating a predicted image of parallax images of the reference viewpoint
　further comprising a
　decoding device as claimed in claim 9.
[16.]
　Decoding apparatus,
　the parallax images coded the reference viewpoint using a predicted image of parallax images of the reference viewpoint corrected using information on parallax image of the reference viewpoint, and information about the parallax image of the reference viewpoint a receiving step of receiving and
　using information on a parallax image of the reference viewpoint received by the processing of the receiving step, a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　corrected by the processing of the correction step by using said predicted image, said received by the processing of the receiving step, a decoding step of decoding parallax images coded the reference viewpoint
　decoding method comprising.
It corrected the scope of the claims (Convention Article 19)
[January 16, 2013 (16.01.2013) The International Bureau acceptance]
[1]
　Using information on parallax image of the reference viewpoint, the correction unit that corrects the predicted image of parallax images of the reference viewpoint,
　using the predicted image corrected by said correcting unit, encodes the parallax image of the reference viewpoint an encoding unit, for
　a transmission unit for transmitting the information about the parallax image of the parallax image and the reference viewpoint of the reference viewpoint encoded by the encoding unit
　encoding apparatus comprising a.
[2]
　Information about the parallax image of the reference viewpoint includes viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint,
　the transmission unit, the viewpoint generation information, and transmits the disparity image of the reference viewpoint as an encoding parameter used for encoding
　the encoding apparatus according to claim 1.
[3]
　The correction unit uses a correction coefficient used when correcting the prediction image based on the viewpoint generation information, the predicted image is corrected,
　transmitting the transmission unit, the correction coefficient, as the coding parameter and
　coding apparatus according to claim 2.
[4]
[Corrected] the transmission section includes a parallax maximum value, and the parallax minimum value, and the inter-camera distance, and transmits the information not referred to when or decoding when encoding
　the encoding apparatus according to claim 3.
[5]
[Corrected] the transmission unit includes: the disparity maximum value, said parallax minimum value, and transmits the distance between the camera, as SEI (Supplemental Enhancement Information)
　encoding apparatus according to claim 4.
[6]
[Corrected] information about the parallax image of the reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the correction unit based on the image pickup position information wherein said prediction image is corrected using the correction coefficient used when correcting the predicted image and the viewpoint generation information on,
　the transmission unit, the correction factor is transmitted as the coding parameters
　according to claim 2 encoding device.
[7]
[Corrected] The correcting unit sets the correction coefficient
　encoding apparatus according to claim 6.
[8]
[Corrected] the encoding unit, a parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　encoding apparatus according to claim 7.
[9]
[Corrected] predicted image generation unit for generating a predicted image of parallax images of the reference viewpoint
　further comprising an
　encoding device according to claim 1.
[10]
[Corrected] coding apparatus,
　using information on parallax image of the reference viewpoint, and a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　the predicted image corrected by the processing of the correction step used on an encoding step of encoding the parallax image of the reference viewpoint,
　and a transmission step of transmitting the information about the parallax image by the processing of the coding step and parallax image coded the reference viewpoint the reference viewpoint
　encoding method, including.
[11]
And parallax images [corrected] using information on parallax image of the reference viewpoint by using a predicted image of parallax images corrected the reference viewpoint coded the reference viewpoint, and information about the parallax image of the reference viewpoint a receiving unit that receives,
　using information on parallax image of the reference viewpoint received by the receiving unit, a correcting unit that corrects the predicted image of parallax images of the reference viewpoint,
　corrected said predicted by the correction unit using the image, received by the receiving unit, a decoding unit for decoding the parallax images coded the reference viewpoint
　decoding apparatus comprising a.
[12]
And parallax images [corrected] the reference viewpoint decoded by the decoding unit, by using the color image of the reference viewpoint, the color image generation unit that generates a color image of a different viewpoint and the reference viewpoint
　further comprising a
　information about the parallax image of the reference viewpoint includes viewpoint generation information is information used to generate the color images of different viewpoints and the reference viewpoint using a parallax image of the color image and the reference viewpoint of the reference viewpoint,
　said receiving portion receives the viewpoint generating information to be transmitted as an encoding parameter for use in decoding the parallax images coded the reference viewpoint,
　wherein said color image generation unit, which is decoded by the decoding unit and a parallax image of the reference viewpoint, and a color image of the reference viewpoint, by using the above viewpoint generating information received by said receiving unit, and generates a color image of the reference viewpoint different from the viewpoint
　decoding according to claim 11 apparatus.
[13]
[Corrected] The receiving unit receives the correction coefficient used for correction of the prediction image when the coding is transmitted as the coding parameter,
　the correction coefficient is generated on the basis of the viewpoint generation information,
　the correction unit, by using the correction coefficient received by said receiving unit, corrects the predicted image
　decoding apparatus according to claim 12.
[14]
[Corrected] The receiving section includes a parallax maximum value, and the parallax minimum value, and the inter-camera distance, receives as information not referred to when or decoding during encoding
　decoding apparatus according to claim 13.
[15]
[Corrected] The receiving unit includes: the disparity maximum value, said parallax minimum value, and a distance between said camera receives as SEI (Supplemental Enhancement Information)
　decoding apparatus according to claim 14.
[16]
Information about the parallax image in the corrected] The reference viewpoint includes image pickup position information indicating the image pickup position in the depth direction of the imaging unit for capturing a color image of the reference viewpoint,
　the receiving unit is transmitted as the coding parameter receive a correction coefficient used for correction of the prediction image when the coding is,
　the correction unit uses the viewpoint generation information and the correction coefficient received by said receiving unit, corrects the predicted image
　according decoding apparatus according to item 12.
[17]
Add the correction coefficient is set based on the viewpoint generation information
　decoding apparatus according to claim 16.
[18]
Add parallax image of the reference viewpoint is encoded in unit having a hierarchical structure
　decoding apparatus according to claim 17.
[19]
Add further comprising a prediction image generation unit that generates a predicted image of parallax images of the reference viewpoint
　decoding apparatus according to claim 11.
[20]
Add decoding apparatus,
　the parallax images coded the reference viewpoint using a predicted image of parallax images of the reference viewpoint corrected using information on parallax image of the reference viewpoint, the parallax image of the reference viewpoint a receiving step of receiving the information about,
　using information on parallax image of the reference viewpoint received by the processing of the receiving step, a correction step of correcting a predicted image of parallax images of the reference viewpoint,
　the correction step using the predictive image corrected by the processing, the received by the processing of the receiving step, a decoding step of decoding parallax images coded the reference viewpoint
　decoding method comprising.