Technical field
[0001]The present disclosure is an image coding apparatus and method, and relates to an image decoding apparatus and method, in particular, the image encoding device and method capable of suppressing the increase of the load of encoding or decoding, as well, the image decoding apparatus and a method for.
BACKGROUND
[0002]Recently, handling image information as digital, time, transmission of a high efficiency information, accumulated for the purpose of, by using the image information inherent redundancy, coding compressed by orthogonal transform and motion compensation such as a discrete cosine transform employs a method apparatus for compressing code images have become popular. The encoding method include, for example, MPEG (Moving Picture Experts Group).
[0003]
　In particular, MPEG2 (ISO / IEC 13818-2) is defined as a general-purpose image encoding scheme, both the interlaced scanning images and progressive scan images as well as standard to cover standard resolution images and high-definition images. For example, MPEG2 is currently widely used in a wide range of applications for professional applications and consumer applications. By using the MPEG2 compression method, 4 to a code amount of 8 Mbps (bit rate) is allocated if the standard resolution interlaced image having 720 × 480 pixels, for example. Further, by using the MPEG2 compression method, the code amount of 18 to 22 Mbps if high-resolution interlaced scan image (bit rate) is allocated with e.g. 1920 × 1088 pixels. Thus, it is possible to realize a high compression ratio excellent image quality.
[0004]
　MPEG2 had mainly target conforming quality coding for broadcasting, but less code amount than MPEG1 (bit rate), was not compatible with the encoding scheme above, that the compression ratio. With the spread of portable terminals, probably it is enhanced needs for such coding method future standardization of the MPEG4 encoding system has been performed correspondingly. With respect to the image encoding method, the standard has been approved by the international standard in December 1998 as ISO / IEC 14496-2.
[0005]
　Furthermore, in recent years, for the purpose of initial image encoding for television conference, H.26L (ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Q6 / 16 VCEG (Video Coding Expert Group)) standard normalization that is advanced It was. H.26L compared to conventional encoding methods such as MPEG2 and MPEG4, the coding, although many calculation amount required by the decoding, it is known that higher encoding efficiency is realized. Also, currently, as part of MPEG4 activities, lines the H.26L-based, also incorporating functions that are not supported by H.26L, standardized to realize higher coding efficiency as Joint Model of Enhanced-Compression Video Coding We were.
[0006]
　The standardization of schedule, in March 2003, became the international standard in the name of H.264 and MPEG-4 Part10 (Advanced Video Coding, hereinafter referred to as AVC).
[0007]
　In addition, the H. 264 / AVC as an extension of, RGB or 4: 2: 2 and 4: 4: 4, such as, and the coding tools necessary for business, including also the 8x8DCT and quantization matrix is ​​specified by the MPEG-2 FRExt (Fidelity Range Extension) standardization was completed in February 2005. As a result, H. With 264 / AVC, so that the possible encoding schemes that better represent also the film noise included in movies, became carried for use in a wide range of applications of such Blu-Ray Disc (trademark).
[0008]
　However, these days, four times the high-definition image, want to compress an image of about 4000 × 2000 pixels, or such as the Internet, in a limited transmission capacity of the environment, but would like to deliver high-definition images, even higher compression ratio there is a growing need for coding. Therefore, the foregoing, in VCEG affiliated ITU-T, Study on improvement of the coding efficiency has been performed is continued.
[0009]
　Therefore, currently, for the purpose of further improvement of the encoding efficiency than the AVC, and ITU-T, is a standards body of the joint ISO / IEC JCTVC - by (Joint Collaboration Team Video Coding), HEVC (High Efficiency Video Coding) standardization of coding method called is in progress. For HEVC standard, Committee draft has been issued a draft specification in January 2013 (for example, see Non-Patent Document 1).
[0010]
　In HEVC, in addition to the slice (Slice) that has been defined also in AVC, it is possible to perform parallel processing by the tile (Tile) or wavefront parallel processing (Wavefront Parallel Processing).
[0011]
　Incidentally, so far, the image encoding method such as MPEG-2 and AVC, hierarchized image into a plurality of layers had scalability (scalability) function for encoding.
[0012]
　That is, for example, such as a cellular phone, for the low throughput terminal transmits the compressed image information only the base layer (base layer), low spatial temporal resolution, or play the poor moving image quality and, such as a television or a personal computer, for high throughput terminal in addition to the base layer (base layer), and transmits the compressed image information of the enhancement layer (enhancement layer), high space-time resolution, Alternatively, as such play a high moving picture image quality without performing a transcoding process, the image compression information according to the terminal and network capabilities, it is possible to transmit from the server.
CITATION
Non-patent literature
[0013]
非特許文献1 : Benjamin Bross, Woo-Jin Han, Gary J. Sullivan, Jens-Rainer Ohm, Gary J. Sullivan, Ye-Kui Wang, Thomas Wiegand, " High Efficiency Video Coding (HEVC) text specification draft 10 (for FDIS & Consent)", JCTVC-L1003_v4, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 12th Meeting: Geneva, CH, 14-23 Jan. 2013
Summary of the Invention
Problems that the Invention is to Solve
[0014]
　However, in the conventional method, the encoding and decoding of enhancement layer, for example, when referring to information related to the coding of the base layer such decoded image information and movement information, etc., the entire picture of the base layer is the subject of reference.
[0015]
　Therefore, in the encoding and decoding of enhancement layer, etc. of the memory access number for referring to the information related to the coding of the base layer is increased, there is a possibility that the load is increased.
[0016]
　The present disclosure has been made in view of such circumstances, it is desirable to make it possible to suppress an increase in the load of the encoding or decoding.
Means for Solving the Problems
[0017]
　According to an embodiment of the present technology, for the current layer of the image data comprising a plurality of layers, to control the area referenced other layer encoded for each predetermined area which divides the picture into a plurality of information related to the coding a generation unit for generating control information according to the control of the control information generated by the generation unit, by referring to information about the encoding of a partial area of ​​the other layers, code the current layer of the image data an encoding unit for reduction, the encoded data of the image data generated by the encoding unit, an image coding apparatus and a transmission unit for transmitting said control information generated by the generation unit.
[0018]
　Information the control information, the other layer, to specify a region to allow a reference to information on the coding, to specify a region for prohibiting reference information about the encoding, or relates to the encoded by specifying the area that reference may be information for limiting a region capable of referring to the information about the encoding.
[0019]
　Wherein the control information indicates the area, raster scan order assigned identification number, information indicating the longitudinal and lateral position of the region in the picture, or the location of the data of said region in the encoded data it can be specified by the information.
[0020]
　The transmission unit, the information indicating whether the control region refers to the information about the encoding can be further transmitted.
[0021]
　Information about the coding may be information to be used in generating a predicted image to be used in encoding of the image data.
[0022]
　Information that is used to generate the prediction image, and information used for texture prediction of the image data, and a information used for syntax prediction of the image data, the control information, the information used for the texture prediction a reference region and a region that refers to information used for the syntax prediction may be information to be controlled independently of each other.
[0023]
　The generation unit of the current layer of the image data, generates the control information for each predetermined area which divides the picture into a plurality of the encoding unit, the current layer of the image data for each of the areas under the control of the control information of each region in which the generator has generated, by referring to the information about the encoding of a partial region of the other layer, it can be encoded.
[0024]
　The transmission unit, the can Segmentation current layer transmits further information indicating whether the same as the region division of the other layers.
[0025]
　The region can be designed such that a slice or tile of the image data.
[0026]
　According to an embodiment of the present technology, also, the current layer of the image data composed of a plurality layers, the other layers that are coded for each predetermined area which divides the picture into a plurality of the regions to view information about coding generates control information for controlling, in accordance with the control of the generated control information refers to information relating to the encoding of a partial area of ​​the other layers, it encodes the current layer of the image data, the image data There is an image coding method for transmitting a coded data generated by coding the generated and the control information.
[0027]
　Another aspect of the present technology, the current layer of the encoded data of the image data composed of a plurality layers, the image data, the other layers which are encoded for each predetermined region to divide the picture into a plurality of coding references and receiving unit to receive the control information for controlling the area that refers to information under the control of said control information received by the receiving unit, information related to the encoding of a partial region of the other layer about and an image decoding apparatus and a decoding unit for decoding the encoded data.
[0028]
　Information the control information, the other layer, to specify a region to allow a reference to information on the coding, to specify a region for prohibiting reference information about the encoding, or relates to the encoded by specifying the area that reference may be information for limiting a region capable of referring to the information about the encoding.
[0029]
　Wherein the control information indicates the area, raster scan order assigned identification number, information indicating the longitudinal and lateral position of the region in the picture, or the location of the data of said region in the encoded data it can be specified by the information.
[0030]
　The receiving unit may further receive information indicating whether the control region refers to the information on the coding.
[0031]
　Information about the coding may be information to be used in generating a predicted image to be used upon decoding the encoded data.
[0032]
　Information that is used to generate the prediction image, and information used for texture prediction of the image data, and a information used for syntax prediction of the image data, the control information, the information used for the texture prediction a reference region and a region that refers to information used for the syntax prediction may be information to be controlled independently of each other.
[0033]
　The receiving unit receives the encoded data for each predetermined region dividing the current layer of the image data, the picture into a plurality, and the control information for each of the areas, the decoder, the the encoded data received by the receiving portion for each of the areas under the control of the control information of each area refers to information relating to the encoding of a partial region of the other layer, it can be decoded .
[0034]
　The receiving unit may be area division of the current layer further receives information indicating whether the same as the region division of the other layers.
[0035]
　The region can be designed such that a slice or tile of the image data.
[0036]
　Another aspect of the present technology, also, the current layer of the encoded data of the image data composed of a plurality layers, the image data, the other layers which are encoded for each predetermined region to divide the picture into a plurality, It receives the control information for controlling the area that refers to information related to the coding, under the control of the received control information refers to information relating to the encoding of a partial area of ​​the other layers, the coding an image decoding method for decoding data.
[0037]
　In one aspect of the present technology, for the current layer of the image data comprising a plurality of layers, to control the area referenced other layer encoded for each predetermined area which divides the picture into a plurality of information related to the coding control information is generated according to the control of the generated control information, information related to the coding of the partial area of ​​the other layers is referred to, the image data current layer is encoded, the image data is encoded product and encoded data, and the generated control information is transmitted.
[0038]
　In another aspect of the present technology, the current layer of the encoded data of the image data composed of a plurality layers, the image data, the other layers which are encoded for each predetermined region to divide the picture into a plurality of coding about received control information for controlling the area that refers to information according to the control of the received control information, information related to the coding of the partial area of ​​the other layers is referred to, the encoded data is decoded.
The invention's effect
[0039]
　According to the present disclosure, images may be encoded and decoded. In particular, it is possible to suppress an increase in the load of the encoding or decoding.
BRIEF DESCRIPTION OF THE DRAWINGS
[0040]
Is a diagram illustrating a configuration example of FIG. 1-coding unit.
Is a diagram illustrating an example of FIG. 2 layer image coding method.
It is a diagram illustrating an example of FIG. 3 Spatial scalable coding.
Is a diagram illustrating an example of FIG. 4 temporal scalable coding.
Is a diagram illustrating an example of a scalable encoding of FIG. 5 signal to noise ratio.
6 is a diagram illustrating an example of a slice.
7 is a diagram illustrating an example of a tile.
8 is a diagram illustrating an example of control of the base layer reference.
Is a diagram illustrating an example of FIG. 9 tile set.
[10] the control of the base layer reference is a diagram for explaining another example.
11 is a diagram illustrating an example of parallel processing.
Is a diagram illustrating an example of a way of assigning [12] tile identification number.
13 is a diagram illustrating an example of syntax of the picture parameter set.
[Figure 14] illustrates an example of a syntax of the picture parameter set, a view subsequent to Fig. 13.
Is a diagram illustrating an example of FIG. 15 slice header syntax.
[Figure 16] illustrates an example of a slice header syntax diagrams subsequent to Fig.
[Figure 17] illustrates an example of a slice header syntax diagrams subsequent to Fig.
18 is a block diagram showing a main configuration example of an image encoding apparatus.
Is a block diagram showing a main configuration example of FIG. 19 the base layer image encoding unit.
It is a block diagram showing a main configuration example of FIG. 20 enhancement layer image encoding unit.
It is a block diagram showing a main configuration example of FIG. 21 region synchro unit.
22 is a flowchart illustrating an example of the image encoding processing flow.
Is a flowchart illustrating an example of FIG. 23 base layer coding processing flow.
Is a flowchart illustrating an example of the flow of FIG. 24 enhancement layer coding processing.
[Figure 25] describing an example of the flow of the enhancement layer coding process is a flowchart subsequent to FIG. 24.
FIG. 26 is a block diagram showing a main configuration example of an image decoding apparatus.
It is a block diagram showing a main configuration example of FIG. 27 base layer image decoding unit.
[FIG. 28] is a block diagram showing a main configuration example of the enhancement layer image decoding unit.
It is a block diagram showing a main configuration example of FIG. 29 region synchro unit.
It is a flowchart illustrating an example of the flow of FIG. 30 the image decoding processing.
[FIG. 31] is a flowchart for explaining an example of the flow of the base layer decoding process.
FIG 32 is a flowchart illustrating an example of a flow of the enhancement layer decoding processing.
[Figure 33] describing an example of the flow of the enhancement layer decoding processing, a flowchart subsequent to FIG. 32.
FIG. 34 is a diagram showing a multi-view image encoding method.
Is a diagram illustrating a main configuration example of FIG. 35 multi-view image encoding apparatus to which the present technology is applied.
[FIG. 36] is a diagram showing a main configuration example of the multi-view image decoding apparatus according to the present technology.
It is a block diagram showing a main configuration example of FIG. 37 computer.
[FIG. 38] is a block diagram showing an example of a schematic configuration of a television device.
[39] is a block diagram showing an example of a schematic configuration of a mobile phone.
Is a block diagram showing an example of a schematic configuration of FIG. 40 recording and reproducing apparatus.
Is a block diagram showing an example of a schematic configuration of a [41] an imaging device.
[FIG. 42] is a block diagram showing an example of a scalable coding utilized.
[43] is a block diagram showing another example of a scalable coding utilized.
[FIG. 44] is a block diagram showing still another example of a scalable coding utilized.
[FIG. 45] is a block diagram showing an example of a schematic configuration of a video set.
[FIG. 46] is a block diagram showing an example of a schematic configuration of the video processor.
[FIG. 47] is a block diagram showing another example of a schematic configuration of the video processor.
Is an explanatory view showing the FIG. 48 of the content reproduction system configuration.
[FIG. 49] is an explanatory view showing the flow of data in the content reproduction system.
Is an explanatory view showing a specific example of FIG. 50] MPD.
[FIG. 51] is a functional block diagram showing a configuration of the content server of the content playback system.
[FIG. 52] is a functional block diagram showing a configuration of the content reproducing apparatus of a content reproduction system.
[FIG. 53] is a functional block diagram showing a configuration of the content server of the content playback system.
[FIG. 54] is a sequence chart showing a communication process example according to the device of a wireless communication system.
[FIG. 55] is a sequence chart showing a communication process example according to the device of a wireless communication system.
[Figure 56] a configuration example of a frame format (frame format) to be transmitted and received in the communication process by each unit of the wireless communication system is a diagram schematically illustrating.
[FIG. 57] is a sequence chart showing a communication process example according to the device of a wireless communication system.
DESCRIPTION OF THE INVENTION
[0041]
　The following describes embodiments of the present disclosure (hereinafter referred to as embodiments). The description will be made in the following order.
　1. The main explanation of this technology
　2. First Embodiment (image coding apparatus)
　3. Second Embodiment (image decoding apparatus)
　4. Third Embodiment (the multi-view image encoding and multi-view image decoding apparatus)
　5. Fourth Embodiment
　(Computer) 6. Application Example
　7. Applications of scalable coding
　8. Fifth Embodiment (Set Unit module processor)
　9. Applications of MPEG-DASH content reproduction system
　10. Application of the wireless communication system of the Wi-Fi standard
[0042]
　<1. The main description of the techniques>
　　
　below is an example when applied to an image encoding and decoding of HEVC (High Efficiency Video Coding) scheme, to illustrate the present technology.
[0043]
　　
　In AVC (Advanced Video Coding) scheme, the hierarchical structure of a macroblock and sub-macroblock is defined. However, in the macro block of 16 × 16 pixels, such that the target of the next generation coding method, UHD; not optimal for large image frame such (Ultra High Definition 4000 pixels × 2000 pixels).
[0044]
　In contrast, in HEVC method, as shown in FIG. 1, the coding unit (CU (Coding Unit)) is defined.
[0045]
　CU is also called a Coding Tree Block (CTB), plays the same role as a macroblock in AVC method, a partial area of ​​the image in units of pictures. The latter, while being fixed to a size of 16 × 16 pixels, the former size is not fixed, in each sequence, will be designated in the image compression information.
[0046]
　For example, the sequence parameter set included in the output encoded data (SPS (Sequence Parameter Set)), maximum size of CU (LCU (Largest Coding Unit)) and minimum size (SCU (Smallest Coding Unit)) is defined that.
[0047]
　Within each LCU, does not fall below the size of the SCU, by a split-flag = 1, it can be divided into smaller size CU. In the example of FIG. 1, the magnitude of the LCU 128, a maximum hierarchical depth is 5. CU size of 2N × 2N, when the value of split_flag is "1", and one lower level, is divided into CU of size of N × N.
[0048]
　Furthermore, CU is divided into an intra or inter a processing unit and a region of the predicted (partial area of ​​the image in units of pictures) is prediction unit (Prediction Unit (PU)), also serving as a processing unit of orthogonal transform domain is a (partial area of ​​the image in units of pictures), is divided into the transform unit (transform unit (TU)). Currently, in HEVC scheme, in addition to the 4 × 4 and 8 × 8, it is possible to use a 16 × 16 and 32 × 32 orthogonal transform.
[0049]
　Thus the HEVC scheme defines a CU, when the coding scheme as perform various processes the CU as a unit, a macroblock in AVC method corresponds to LCU, the block (sub-block) is equivalent to the CU Then it can be considered. The motion compensation block in the AVC scheme may be considered to correspond to PU. However, CU, since having a hierarchical structure, the size of the LCU of the highest hierarchical level, for example, as a 128 × 128 pixels, it is generally set larger than the macro block of the AVC method.
[0050]
　Thus, the following, LCU is intended to include a macroblock in AVC method, CU is also intended to include a block (sub-block) in the AVC scheme. That is, "blocks" used in the following description refers to any partial area in the picture, its size, shape, and characteristics, etc. are not limited. That is, the "block", for example, TU, PU, ​​SCU, CU, LCU, subblock includes macroblocks or slices like any area, (processing units). Of course, other than these partial regions (processing units) are also included. If you need to limit the size and process unit or the like, appropriately described.
[0051]
　　
　Incidentally, in the AVC and HEVC coding scheme, to achieve higher coding efficiency, selection of an appropriate prediction mode is important.
[0052]
　Examples of such selection method, JM (Joint Model) and H.264 / MPEG-4 AVC reference software called (published in Http://Iphome.Hhi.De/suehring/tml/index.Htm) it can be a method that is implemented in.
[0053]
　In JM, described below, it is possible to select the High Complexity Mode, a mode determination two ways Low Complexity Mode. Both calculates a cost function value for each of the prediction modes Mode, which selects a prediction mode that minimizes the optimal mode for the block to macro block.
[0054]
　The cost function in the High Complexity Mode is shown by the following equation (1).
[0055]
[Number 1]

[0056]
　Here, Omega, the entire set of candidate modes for encoding the block to macro block, D is, when coding in the prediction mode, which is the difference energy of the decoded image and the input image. λ is a Lagrange undetermined multiplier given as a function of a quantization parameter. R contained the orthogonal transform coefficients, the total code amount when encoding in the mode.
[0057]
　That is, to do the encoding in High Complexity Mode, for calculating the parameters D and R, by all candidate modes, once it is necessary to perform a temporary encoding process requires a higher amount of calculation.
[0058]
　The cost function in the Low Complexity Mode is shown by the following equation (2).
[0059]
[Number 2]

[0060]
　Here, D is, unlike in the case of High Complexity Mode, the differential energy input image and the prediction image. QP2Quant (QP) is given as a function of the quantization parameter QP, HeaderBit does not include the orthogonal transformation coefficients, and motion vectors, such as mode, a code amount relating to information belonging to Header.
[0061]
　That is, in the Low Complexity Mode, for each candidate mode, it is necessary to perform a prediction process, since it is not necessary to decode the image, need not be performed until the coding process. Therefore, it is possible to realize a low amount of calculation than the High Complexity Mode.
[0062]
　　
　Incidentally, so far, the image encoding method such as MPEG2, AVC had scalability (scalability) function. The scalable coding (layered coding), multiple layers of images and (hierarchical), a method of encoding for each layer. Figure 2 is a diagram showing an example of a hierarchical image coding method.
[0063]
　As shown in FIG. 2, in the hierarchy of the image, first image with reference to a predetermined parameter having a scalability function is divided into a plurality of layers (layers). That, layered image (hierarchical image) includes an image of the predetermined parameter value is a plurality of different hierarchical (layer). Multiple layers of this hierarchy image coding and by using a base layer of coding and decoding using only the image of its own layer without using the image of the other layer, the image of the other layers performing decoding becomes by a non-base layer (also called enhancement layer). Non-base layer, may be using the image of the base layer, may be utilized images of other non-base layer.
[0064]
　Generally, non-base layer, as the redundancy is reduced, and its own image, constituted by the data of the difference image between the image of the other layers (difference data). For example, when two layers into one of the image-based layer and a non-base layer (also called enhancement layer), the low-quality images can be obtained than the original image only data of the base layer, the base layer data and non by combining the data of the base layer, the original image (i.e., high-quality image) is obtained.
[0065]
　By layering these images, it is possible to easily obtain various quality images depending on the situation. Such as a mobile phone, for the low throughput terminal transmits the compressed image information only the base layer (base layer), low spatial temporal resolution, or play the poor moving image quality, such as a television or a personal computer, for high throughput terminal in addition to the base layer (base layer), and transmits the compressed image information of the enhancement layer (enhancement layer), high space-time resolution, or, as such reproducing a high moving picture image quality without performing a transcoding process, the image compression information according to the terminal and network capabilities, it is possible to transmit from the server.
[0066]
　　
　In such a hierarchical image coding, hierarchical image decoding (scalable coding and scalable decoding), the parameter having scalability (scalability) function is optional. For example, the spatial resolution may be as a parameter as shown in FIG. 3 (spatial scalability). For this spatial scalability (spatial scalability), the resolution of the image is different for each layer. That is, as shown in FIG. 3, the enhancement of each picture, a base layer of a spatially lower resolution than the original image, the original image by combining the image of the base layer (original spatial resolution) is obtained It is stratified into two layers of the layer. Of course, the number of layers is an example, it can be layered into any number of hierarchies.
[0067]
　Further, as a parameter to provide such scalability properties, other, for example, as shown in FIG. 4, it may be applied to temporal resolution (temporal scalability). For this temporal scalability (temporal scalability), frame rates are different for each layer. That is, in this case, as shown in FIG. 4, are layered in different frame rates of the layers to each other, the low frame rate of the layer, by adding a high frame rate of the layer, a moving image of a higher frame rate can be obtained, by adding all the layers, it is possible to obtain the original moving image (original frame rate). The number of layers is an example, it can be layered on any number of hierarchies.
[0068]
　Further, as a parameter to provide such scalability properties, for example, signal-to-noise ratio (SNR (Signal to Noise ratio)) may be applied (SNR scalability). For this SNR scalability (SNR scalability), SN ratio is different for each layer. That is, as shown in FIG. 5, each picture, and low base layers of SNR than the original image, two-level enhancement layer original image (original SNR) is obtained by synthesizing the image of the base layer It is layered on. That is, in the base layer (base layer) image compression information, and information about the image of the low PSNR is transmitted, in this, the addition of the enhancement layer (enhancement layer) image compression information, reconstructs the high PSNR image It is possible. Of course, the number of layers is an example, it can be layered into any number of hierarchies.
[0069]
　Parameter to have the scalability properties, be other than the above-described example, of course good. For example, the base layer (base layer) is made of eight bits (bit) image, by adding the enhancement layer (enhancement layer) to 10-bit (bit) bit depth scalability image is obtained (bit-depth scalability) is is there.
[0070]
　Also, the base layer (base layer) is 4: 2: 0 consists format component image, by which the addition of the enhancement layer (enhancement layer), 4: 2: 2 format chroma scalability component image is obtained of (chroma scalability) there is.
[0071]
　　
　Incidentally, in the HEVC, in addition to the slice (Slice) that has been defined also in AVC, it is possible to perform parallel processing by the tile (Tile) or wavefront parallel processing (Wavefront Parallel Processing).
[0072]
　Figure 6 is a diagram showing an example of a slice (Slice) which is defined in HEVC. As with AVC, slices, as a unit for performing encoding in the raster scan order, within the picture, as shown in FIG. 6, an area divided into a plurality. However, in HEVC, slice division is only possible in LCU unit. 6, the entire square indicates picture, small squares therein indicates the LCU. In addition, a group of LCU, which is divided into pattern indicates a slice. For example, a slice consisting of LCU over from the first row and the second row shown by hatching pattern is the first slice of the picture (Slice # 1). Further, a slice consisting of LCU in the third row and the fourth row from the top, shown in white is the second slice of the picture (Slice # 2). Furthermore, a slice consisting of LCU of lines 5 and 6 from the top, shown in gray areas is the third slice of the picture (Slice # 3). Further, a slice consisting of LCU from the top the seventh and eighth rows represented by hatched is the fourth slice of the picture (Slice # 4). Of course, the number of slices and LCU formed picture, as well as how to slice division is arbitrary and is not limited to the example of FIG.
[0073]
　Figure 7 is not an example of a tile (Tile) which is defined in HEVC. Tiles, like the slices above an area for dividing the inside of a picture as a unit LCU. However, slices, whereas each LCU is a region to divide the picture to be processed in raster scan order, tiles, as shown in FIG. 7, is an area which divides the picture into arbitrary rectangular .
[0074]
　7, the whole of the square indicates a picture, a small square in it indicates the LCU. In addition, a group of LCU, which is divided into pattern indicates a tile. For example, a slice consisting of the upper left of the vertical four x lateral four LCU indicated by cross-hatched is the first tile of the picture (Tile # 1). Also, tiles made from the upper right of the vertical four x lateral four LCU represented by white is the second tile of the picture (Tile # 2). Furthermore, tiles made from the bottom left of the vertical four x lateral four LCU indicated by gray areas is the third tile of the picture (Tile # 3). Also, tiles made of vertical four x lateral four LCU lower right indicated by hatched is the fourth tile of the picture (Tile # 4). Of course, the number of tiles and LCU formed picture, as well as how the tile division is arbitrary and is not limited to the example of FIG.
[0075]
　Within each tile is formed as described above, each LCU is processed in raster scan order. Such tiles, because it requires a short length of the boundary than the slice, has a feature that requires less deterioration in the coding efficiency caused by the split screen.
[0076]
　The above respective slices or tiles divided as is, because the dependencies such as prediction and CABAC in the encoding and decoding is not present, it is possible to perform the process independently of each other. That is, for example, the data for each slice (or tiles), using different CPU (Central Processing Unit) (or different cores) from each other, can be processed in parallel.
[0077]
　　
　Incidentally, in the hierarchical coding mentioned above, in the encoding of the enhancement layer, it is possible to utilize the information related to the coding of the base layer. This content of the information related to the coding is arbitrary, for example, texture information such as decoded image, there is a syntax information such as such as motion information, intra prediction mode information.
[0078]
　In hierarchical coding, after the coded picture in the base layer is made, the enhancement layer reference information related to the coding of the base layer, coding the picture corresponding to the picture is performed. That is, after the encoding of the base layer, information related to the coding of the resulting base layer is supplied to the encoding of the enhancement layer is appropriately utilized. Decoding is also performed in the same procedure.
[0079]
　However, in the conventional method, the encoding and decoding of the thus enhancement layer, there was no way to control the area to be reference destination information related to the coding. That is, for example, information about the encoding, even different things in each area, always it has been the entire picture of the base layer and the reference object. Therefore, the picture of the base layer, since the unwanted areas as clearly referenced also are referenced, increased number of memory accesses and the like unnecessary, possibly load of the encoding and decoding of enhancement layer increases unnecessarily there were.
[0080]
　Also in hierarchical encoding, by eliminating the processing of dependencies between the regions of the slice and tiles or the like as described above, that the independent the process for each area, parallel processing of the processing of each region e.g. can. That is, in this case, for each area, and the encoding and decoding of the base layer can be sequentially performed and encoding and decoding the enhancement layer.
[0081]
　However, the encoding and decoding of enhancement layer, when referring to information related to the coding of the base layer, in the conventional method, becomes the entire picture and the reference objects, cause a dependency between the other region put away. Therefore, there is a risk that parallel processing of each region becomes difficult.
[0082]
　　
　Therefore, in the encoding and decoding of enhancement layer, other layers (e.g. the base layer or other enhancement layer), so as to control the area to be referenced for encoding information. For example, an area for additional information about coding, the picture of the other layers, so as to limit the portion of the region.
[0083]
　Figure 8 is a diagram showing an example of the way of such a reference target limit. In the example shown in FIG. 8, only the tiles represented by mesh pattern of the base layer has been designated as the reference target information related to the coding. In this case, information related to the coding of the other regions (white regions) are not included in the reference target, the encoding and decoding of enhancement layer, it is not to be read out from the memory. Therefore, correspondingly, increase in the load of the encoding and decoding of enhancement layer is suppressed.
[0084]
　Although the method of this limitation is arbitrary, for example, the other layers, may be designated a region to allow a reference to information related to the coding. Further, for example, the other layers, may be designated a region for prohibiting reference information about the encoding. Furthermore, for example, the other layers, may be designated a region refers to the information related to the coding.
[0085]
　Incidentally, the control unit of the reference target information about the encoding, for example, by a region serving as a processing unit of encoding and decoding such tiles and slice, it is possible to reduce the dependency between the regions, the process it can be easier to separate reduction and parallelization.
[0086]
　　
　be described a more specific example of such control.
[0087]
　For example, as in the example of FIG. 8, when the encoding of the base layer, divides the picture into tiles, a part of the tile only, controlled so as to be able to view information about the encoding. In this case, for example, the part of the tile, to allow a reference to information related to the coding. For example, in the coding of the base layer generates control information specifying the tile to allow a reference to information related to the coding and provides it to the encoding of the enhancement layer.
[0088]
　Encoding the enhancement layer is performed according to the control information. That is, in the coding of the enhancement layer may be reference only information related to the coding of allowed tile by the control information.
[0089]
　Note that the encoding of the base layer, which region, or setting to allow a reference to information related to the coding is arbitrary. For example, such as a user or an application may specify a region to allow a reference to information related to the coding regions to allow a reference to information related to the coding may be predetermined.
[0090]
　For example, to each picture common location of a moving image, if there is clearly referring unwanted areas such as letterbox excludes the area from "area to allow a reference to information related to the coding", i.e., the other regions as "area to allow a reference to information related to the coding", before encoding each picture of the moving image data may be designated in advance.
[0091]
　Further, for example, the user, may be designated a "region that allows the reference information related to the coding" of each picture, the user specifies the characteristics of the image, the application or the like of its features in each picture an area including, may be designated as "area to allow a reference to information related to the coding." Furthermore, application and the like, in each picture, may perform area division so as to form a region comprising (features specified in or user) predetermined characteristic (e.g., tiled or slice division, etc.).
[0092]
　For example, in the coding of the base layer, the input image is assumed to be an image including a person (A in Figure 9). Applications for the image performs face recognition processing and detects a partial area including a face of a person (B in Figure 9). Then, the application, the partial region to one of the tiles, tiles divides the picture (C in FIG. 9). Then, the application, the tile (i.e. detected partial region) including the face of a person is designated as "area to allow a reference to information related to the coding" (the tracery in D in FIG. 9 tiles).
[0093]
　Thus, it is possible to perform an area division (the formation of tiles or slices) aware that the information about the encoding is referenced by the encoding of the enhancement layer. In this way, it is possible to reduce the "area to allow a reference to information related to the coding." That is, in the coding of the enhancement layer, it is possible to further narrow the range of referable base layer, it is possible to suppress the increase of the load.
[0094]
　As described above, control of the area refers to the information about the coding is performed in larger units than at least regions (tiles and slices, etc.). For example, it may be performed for each picture. Further, for example, it may be executed for each sequence. Further, it may be performed for each video image data. It is also possible to prepare such control information in advance.
[0095]
　In the above has been described as specifying the "area to allow a reference to information related to the coding", the control method is not limited thereto, for example, on the contrary, inhibited the reference information about the "coding it is also possible to specify the area "to be. In this case, the tiles other than the tile part that is prohibited references are referenced.
[0096]
　Again, for example, in the coding of the base layer generates control information specifying the tile for prohibiting reference information related to the coding, it is sufficient it to provide the encoding of the enhancement layer.
[0097]
　Encoding the enhancement layer, as in the case to allow a reference, it is performed according to the control information. That is, in the coding of the enhancement layer may be reference only information related to the coding of the tiles than the tiles that are prohibited by the control information.
[0098]
　Of course, the setting method in this case is the same as any in the case to allow reference. Incidentally, the base layer, to allow a reference to information related to the coding (or prohibits) regions, may be one or more.
[0099]
　Even if prohibiting may allow a reference to information on such encoded, the encoding of the enhancement layer, whether to divide the picture into tiles (or slice) is optional. It is also arbitrary how to divide as to divide. If the enhancement layer, even when the encoded for each region, such as tiles and slice, coding each region, respectively, is performed based on the above-described control information. That is, even in coding of any area, it is possible to see only the information related to the coding of the permitted by the control information (or forbidden tiles (or slice) other) tile (or slice).
[0100]
　Then, when coding thus the enhancement layer for each region, such as tiles and slices, to allow a reference to information related to the coding (or prohibits) region, it is set for each region of the enhancement layer good. That is, the base layer, to allow a reference to information related to the coding (or prohibits) regions may not be unified with each region of the enhancement layer.
[0101]
　For example, control information, linking the respective areas of each region and the base layer of the enhancement layer may be information for performing (synchro) (correspondence table, etc.). In that case, in the encoding of each region of the enhancement layer, only the information related to the coding region of the linking base layer is referable by the correspondence table.
[0102]
　Thus, for each region of the enhancement layer, by controlling the reference destination information related to the coding enables more appropriate control. Therefore, it is possible to suppress an increase in the load of the encoding or decoding. Further, it is possible to reduce the dependency between the regions.
[0103]
　For example, as shown in FIG. 10, each region of the enhancement layer, may be allowed to refer to information related to the coding of the different regions of the base layer. For example in FIG. 10, the tiles E the enhancement layer 0 in the coding of the reference destination information related to the coding of the base layer, a tile B in the base layer 0 it is limited. Also, tiles E of enhancement layer 1 in the coding of the reference destination information related to the coding of the base layer, a tile B in the base layer 1 is limited to. Furthermore, tile E of enhancement layer 2 in the coding of the reference destination information related to the coding of the base layer, a tile B in the base layer 2 is limited to. The tile E of enhancement layer 3 in the encoding of the reference destination information related to the coding of the base layer, a tile B in the base layer 3 is limited to.
[0104]
　As in the example of FIG. 10, such that each region of the enhancement layer, by such additional information related to the coding of the different regions of the base layer, which reduces the dependency between the regions, as shown in FIG. 11 , it is possible to make it easier to parallel processing.
[0105]
　In the example of FIG. 11, the first CPU # 0, the tile # 0 (B of the base layer frame # 0 0 _0), the frame # tiles # 0 of enhancement layer 0 (E 0 _0), Frame # 1 tile base layer # 0 (B 0 - 1), the tile # 0 (E enhancement layer frame # 1 0 - 1), the frame # tiles # 0 of the second base layer (B 0 _2), enhancement of the frame # 2 layer tiles # 0 (E 0 in the order such as _2), performs coding for the tiles # 0 of each frame.
[0106]
　In parallel with this, the second CPU # 1 is the tile # 1 (B of the base layer frame # 0 1 _0), Tile # 1 (E enhancement layer frames # 0 1 _0), the frame # 1 base tile # 1 (B layer 1 _1), tile # 1 (E enhancement layer frame # 1 1 - 1), the frame # tiles # 1 of 2 of the base layer (B 1 _2), the frame # 2 enhancement layer tile # 1 (E 1 in the order, such as _2), it is possible to perform encoding of the tiles # 1 of each frame.
[0107]
　In parallel with these processes, the third CPU # 2, the tile # 2 (B base layer frame # 0 2 _0), tiles enhancement layer frames # 0 # 2 (E 2 _0) frame # 1 of the base layer of the tile # 2 (B 2 _1), the frame # 1 of the enhancement layer of the tile # 2 (E 2 _1), the frame # 2 of base layer tile # 2 (B 2 _2), the frame # the second enhancement layer tile # 2 (E 2 forward as _2) by, it is possible to perform encoding of the tiles # 2 of each frame.
[0108]
　Furthermore, in parallel with these processes, the 4 th CPU # 3, tile # 2 (B base layer frame # 0 3 _0), enhancement layer of the tile # 3 of frame # 0 (E 3 _0) frame # 1 of the base layer of the tile # 3 (B 3 _1), the frame # 1 of the enhancement layer of the tile # 3 (E 3 _1), tile # 3 of frame # 2 of the base layer (B 3 _2), the frame # tile # 3 of the second enhancement layer (E 3 in the order, such as _2), it is possible to perform encoding of the tiles # 3 of each frame.
[0109]
　Incidentally, the designation of regions of the base layer in such control information (tile or sliced, etc.) is to be performed by the position of the data in each region (e.g., an offset value from the head) included in the coded data (bit stream) it may be, but assigned an identification number to each region of the base layer, may be designated by the identification number.
[0110]
　For example, as shown in FIG. 12, assigned an identification number in the raster scan order in each area, with this identification number may be performed the specified region to allow or prohibit the reference information about the encoding. Of course, how to assign the identification number is optional, the above-mentioned raster scan order is an example.
[0111]
　In the above, a case has been described of coding as an example, the same applies to the case of decoding.
[0112]
　　
　The control information for controlling the reference information related to the coding, such as described above, may be transmitted from the encoding side to the decoding side. By transmitting control information to the decoding side, it is possible to utilize the control information in the decoding. That is, as in the case of coding, it is possible to reduce the load of the decoding. Incidentally, in which case, the control information, for example, may be defined in a picture parameter set (PPS (Picture Parameter Set)) or slice header (SliceHeader). Of course, the control information can be transmitted in any manner. For example, it may be defined in a sequence parameter set or a video parameter set or the like. Also, it may be transmitted as separate data from the encoded data of the image data control information.
[0113]
　In the case of transmitting control information by a picture parameter set, an example of syntax of the picture parameter set of enhancement layer shown in FIGS. 13 and 14.
[0114]
　In this example, as shown in FIG. 13, Tile_setting_from_ref_layer_flag as information indicating area division of a processed current layer (i.e. enhancement layer) is, or is similar to segmentation of other layers (i.e., base layer) There is transmitted. If this value is 1, it indicates that the way of area division (e.g. tile division) in the enhancement layer is the same as in the base layer.
[0115]
　For example, if the segmentation of the enhancement layer is the same as the region division of the base layer, the decoding of the enhancement layer, it is possible to grasp the segmentation of the enhancement layer by referring to the area division information of the base layer, enhancement layer information about the region segmentation (e.g., Num_tile_columns_minus1 in FIG 13, num_tile_rows_minus1, uniform_spacing_flag etc.) transmission is not required. Therefore, it is possible to suppress the reduction of the coding efficiency.
[0116]
　Further, as shown in FIG. 14, Inter_layer_tile_prediction_restriction_flag is transmitted as information indicating whether the control region refers to the information about the coding. When the value is 1, the control information for controlling the reference information related to the coding as described above is transmitted (second line to the ninth line from the top in FIG. 14). In the example shown in FIG. 14, the enhancement layer is encoded for each region, for each region of the enhancement layer, the control information for controlling whether to refer to the information related to the coding of any region of the base layer is transmitted .
[0117]
　Information Thus, by transmitting the information indicating whether the control region refers to the information related to the coding, to (coding omitted transmission of control information when no control region refers to the information related to the coding only when controlling the area that reference, so as to transmit the control information) can be. Therefore, it is possible to suppress the reduction of the coding efficiency.
[0118]
　In the example shown in FIG. 14, the current area to be processed of the enhancement layer is specified by the position in the horizontal direction and the vertical direction (i, j) in the region array. Also, the each region, the number of the reference destination becomes base layer region (num_ref_tiles_minus1) and its area is designated. Furthermore, the region of the reference destination becomes base layer is specified by the identification number (ref_tile [k]). The identification number for each region of the base layer are those assigned to the raster scan order as in the example of FIG. 12.
[0119]
　Above, such as a designation of the current region of the enhancement layer, the designation of regions of the base layer that is referenced can be specified in any manner other than the above-described example. For example, the current area of ​​the enhancement layer, may be designated by the identification number. Also shown for example, an area that is referenced base layer, the position of the horizontal and vertical direction in the region array (i, j) may be specified by the position of the region data in the encoded data it may be specified by information (e.g., an offset value from the beginning).
[0120]
　In the case of transmitting control information by a slice header, an example of a syntax of a slice header of the enhancement layer shown in FIGS. 15 to 17. As shown in FIGS. 15 to 17, even if the slice header, by methods similar to that of the picture parameter set described with reference to FIGS. 13 and 14, the control information is transmitted.
[0121]
　In the example of FIGS. 13 to 17 has been described as an example tile as an area, the same applies if the region is a slice.
[0122]
　As described above, the information related to the coding, for example, texture information such as decoded image includes syntax information such as motion information, intra prediction mode information. That is, for example, the inter-layer prediction to perform prediction with reference to the information of the other layers (also referred to as inter-layer prediction (Inter-layer Prediction)), is used to predict the texture information such as the decoded image information of the base layer inter-layer texture prediction (the interlayer texture prediction (inter-layer texture prediction) and also referred to), the inter-layer syntax prediction using the syntax information such as the motion information and the intra prediction mode information of the base layer to the prediction (inter-layer syntactic spray prediction (Inter-layer syntax prediction) and also referred to) and there is. This technology, the control of the reference destination information related to the coding in each prediction processing may be carried out independently of each other. Thus, for example, a reference destination region of texture information, and a reference destination region syntax region may be specified independently of each other.
[0123]
　<2. First Embodiment>
　　
　Next, a description will be given apparatus and method for implementing the present technology as described above. Figure 18 shows an embodiment of an image processing apparatus according to the present technology is a diagram showing an image coding apparatus. The image coding apparatus 100 shown in FIG. 18 is a device for performing the hierarchical image coding. As shown in FIG. 18, the image coding apparatus 100, the base layer image encoding section 101, enhancement layer video encoder 102, and a multiplexing unit 103.
[0124]
　Base layer image coding unit 101, a base layer picture is encoded to produce a base layer image encoding stream. Enhancement layer picture coding unit 102, an enhancement layer picture is encoded, to generate an enhancement layer image encoding stream. Multiplexing unit 103 includes a base layer image coded stream generated in the base layer image coding unit 101, and an enhancement layer image coded stream generated in the enhancement layer video encoder 102 multiplexes hierarchical image coding generating a stream. Multiplexing unit 103 transmits the generated layer image encoded stream to the decoding side.
[0125]
　Base layer image coding unit 101, the encoding of the base layer image performs the area division such as tiles or slices for the current picture, encoding is performed for respective areas (tiles and slices, etc.). The base layer image encoding unit 101 supplies information about the encoding of the resulting base layer in the encoding, the enhancement layer video encoder 102.
[0126]
　Enhancement layer picture coding unit 102, the encoding of the enhancement layer image performs the area division such as tiles or slices for the current picture, encoding is performed for respective areas (tiles and slices, etc.). At that time, an enhancement layer video encoder 102 controls the area to be reference destination information related to the coding of the base layer. More specifically, the enhancement layer video encoder 102, and the region of the enhancement layer performs the linking of the base layer in the regions to be the reference destination information related to the coding, the control information indicating the corresponding relationship generated.
[0127]
　Enhancement layer picture coding unit 102, under the control of the control information, can refer to information related to the coding of the base layer appropriately encodes the enhancement layer image. Further, the enhancement layer video encoder 102, the control information, (as hierarchical image coding stream) through the multiplexing unit 103, and transmits the decoding side.
[0128]
　　
　FIG. 19 is a block diagram showing a main configuration example of a base layer image encoding unit 101 of FIG. 18. As shown in FIG. 19, the base layer image encoding unit 101, A / D conversion unit 111, a screen rearrangement buffer 112, arithmetic unit 113, orthogonal transform unit 114, a quantization unit 115, a lossless encoding section 116, having a storage buffer 117, dequantizer 118 and inverse orthogonal transformer 119,. The base layer image encoding unit 101 includes a calculation unit 120, a loop filter 121, a frame memory 122, selection unit 123, an intra prediction unit 124, an inter prediction unit 125, the predicted image selection unit 126 and rate control unit 127, . Furthermore, the base layer image encoding unit 101 has a base layer region division setting unit 128.
[0129]
　A / D conversion unit 111, input image data (base layer image information) into A / D, the converted image data (digital data) is supplied to the screen rearrangement buffer 112 for storage. Screen rearrangement buffer 112, an image of the stored display order of the frame, depending on the GOP (Group Of Picture), rearranged in the frame order for coding, an image rearranged the order of the frame, and supplies the calculation unit 113. Also, the screen rearrangement buffer 112, an image rearranged the order of the frame, and supplies to the intra prediction unit 124 and the inter prediction section 125.
[0130]
　Operation section 113, from the image read from the screen rearrangement buffer 112, subtracts a prediction image supplied from the intra prediction unit 124 or the inter prediction unit 125 via the predicted image selection unit 126, orthogonal to the difference information and outputs to the converter 114. For example, in the case of an image on which intra encoding is performed, the arithmetic unit 113, the image read from the screen rearrangement buffer 112, subtracts a prediction image supplied from the intra prediction unit 124. For example, when an image inter-coding is performed, the arithmetic unit 113, the image read from the screen rearrangement buffer 112, subtracts a prediction image supplied from the inter prediction unit 125.
[0131]
　Orthogonal transform unit 114, to the difference information supplied from the calculation unit 113 performs orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform. Orthogonal transform unit 114 supplies the transform coefficients to the quantization unit 115.
[0132]
　Quantization unit 115 quantizes the transform coefficient supplied from the orthogonal transform unit 114. Quantization unit 115 sets a quantization parameter based on information on a target value of the code amount supplied from the rate control unit 127 performs the quantization. Quantization unit 115 supplies the transform coefficient quantized in the lossless encoding section 116.
[0133]
　Lossless encoding unit 116 encodes the transform coefficient quantized in the quantization unit 115 in any encoding method. Coefficient data because it is quantized under the control of the rate control unit 127, the code amount (approximate to or target value) becomes a target value the rate control unit 127 is set.
[0134]
　Also, the lossless encoding section 116 acquires such information indicating the mode of the intra prediction from the intra prediction unit 124 acquires such information and difference motion vector information indicating the mode of inter prediction from the inter prediction unit 125. Furthermore, the lossless encoding unit 116, appropriately generates NAL units of the base layer comprising a sequence parameter set (SPS), and a picture parameter set (PPS) and the like.
[0135]
　Also, the lossless encoding unit 116, the information about the set base layer region (e.g., a tile or a slice, etc.) divided by the base layer area division setting unit (also referred to as base layer area division information) coding.
[0136]
　Lossless encoding unit 116, these various types of information encoded in any encoding method, a part of the encoded data (also referred to as encoded stream) (multiplexing). Lossless encoding unit 116, to accumulate and supplies the encoded data obtained by encoding in the storage buffer 117.
[0137]
　The encoding method of the lossless coding unit 116, for example, variable length coding or arithmetic coding or the like. The variable length coding, for example, H. Such as 264 / AVC scheme in are defined CAVLC (Context-Adaptive Variable Length Coding) and the like. The arithmetic coding, for example, like CABAC (Context-Adaptive Binary Arithmetic Coding).
[0138]
　The storage buffer 117, the coded data supplied from the lossless encoding section 116 (base layer coded data), temporarily holds. The storage buffer 117 at a predetermined timing, the base layer encoded data held, for example, is output to the subsequent stage of the not shown recording device (recording medium) or a transmission line. That is, the storage buffer 117 is also a transmission unit for transmitting the encoded data.
[0139]
　The conversion coefficient quantized in the quantization unit 115 is also supplied to the inverse quantization unit 118. Inverse quantization unit 118, the quantized transform coefficients to inverse quantization in a manner corresponding to the quantization by the quantization unit 115. Inverse quantization unit 118, the resulting transform coefficients, and supplies the inverse orthogonal transform unit 119.
[0140]
　Inverse orthogonal transform unit 119, the transform coefficient supplied from the inverse quantization unit 118, inverse orthogonal transformation in a method corresponding to the orthogonal transform processing by the orthogonal transform unit 114. Inverse orthogonal transformed output (restored difference information) is supplied to the arithmetic unit 120.
[0141]
　Calculation unit 120, an inverse orthogonal transform result supplied from the inverse orthogonal transform unit 119, the restored difference information, a prediction image from the intra prediction unit 124 or the inter prediction unit 125 via the predicted image selection unit 126 summed to obtain a locally decoded image (decoded image). The decoded image is supplied to the loop filter 121 or the frame memory 122.
[0142]
　Loop filter 121 includes a deblocking filter and adaptive loop filter or the like, and appropriate filtering on the reconstructed image supplied from the calculating unit 120. For example, the loop filter 121 removes block distortion of the reconstructed image by performing deblocking filtering on the reconstructed image. Further, for example, a loop filter 121, for the deblocking filtering result (reconstructed image removal of block distortion is performed), image quality improvement by performing the loop filter processing by using the Wiener filter (Wiener Filter) I do. Loop filter 121 includes a filter processing result (hereinafter, referred to as decoded image) to the frame memory 122.
[0143]
　Incidentally, the loop filter 121 further on the reconstructed image may be performed to any other filtering. Further, the loop filter 121 as needed, information such as filter coefficients used in the filtering process is supplied to the lossless encoding unit 116, it may be so as to be encoded.
[0144]
　The frame memory 122 stores the decoded image supplied, at a predetermined timing, as a reference image the decoding image stored therein and supplies the selector 123.
[0145]
　More specifically, the frame memory 122 stores the reconstructed image supplied from the calculating unit 120, and a decoded image supplied from the loop filter 121, respectively. The frame memory 122 supplies the predetermined timing, or based on a request from the outside such as the intra prediction unit 124, the reconstructed image stored therein to the intra prediction unit 124 via the selector 123. The frame memory 122 supplies the predetermined timing, or based on a request from the outside such as the inter prediction unit 125, the decoded image stored therein via the selection unit 123, the inter prediction section 125 .
[0146]
　Selecting unit 123 selects the destination of the reference image supplied from the frame memory 122. For example, in the case of intra prediction, the selection unit 123 supplies the reference image supplied from the frame memory 122 (pixel values ​​in the current picture) to the intra prediction unit 124. Further, for example, in the case of inter prediction, the selection unit 123 supplies the reference image supplied from the frame memory 122 in the inter prediction section 125.
[0147]
　The intra prediction unit 124, for the current picture is an image of the processing target frame, performs prediction processing, and generates a prediction image. The intra prediction unit 124, the prediction process, (as a processing unit blocks) every predetermined block performed. In other words, the intra prediction unit 124 generates a prediction image of the current block is the current picture, the processing target. At that time, the intra prediction unit 124 performs prediction processing (intraframe prediction (referred to as intra prediction)) using the reconstructed image supplied as the reference image from the frame memory 122 via the selector 123. In other words, the intra prediction unit 124 is included in the reconstructed image to generate a predictive image using the pixel values ​​around the current block. Neighboring pixel value to be used for the intra prediction, the current picture, the pixel value of pixels that are processed in the past. This is the intra prediction (i.e., the way of generating a predicted image), a plurality of methods (both intra-prediction mode referred to), are prepared in advance as candidates. The intra prediction unit 124 performs the intra prediction in the previously prepared plurality of intra prediction modes.
[0148]
　The intra prediction unit 124 generates prediction images in all the candidate intra prediction modes, evaluates the cost function value of each prediction image using the input image supplied from the screen rearrangement buffer 112, an optimal mode select. The intra prediction unit 124 has selected the best intra prediction mode, a prediction image generated in the optimum mode, and supplies the predicted image selection unit 126.
[0149]
　In addition, as described above, the intra prediction unit 124, an intra prediction mode information indicating the adopted intra prediction mode, and supplies an appropriate lossless coding unit 116, is encoded.
[0150]
　Inter prediction unit 125, for the current picture, performs prediction processing, and generates a prediction image. Inter prediction unit 125, the prediction process, (as a processing unit blocks) every predetermined block performed. That is, the inter prediction unit 125 generates a prediction image of the current block is the current picture, the processing target. At that time, the inter prediction unit 125 uses the image data of the input image supplied from the screen rearrangement buffer 112, and the image data of the decoded image supplied as the reference image from the frame memory 122, performs a prediction process. The decoded image is an image of a frame processed prior to the current picture (other pictures not current picture). That is, the inter prediction unit 125 performs a prediction process for generating a prediction image using an image of another picture (inter prediction (also referred to as inter prediction)).
[0151]
　The inter prediction is made from the motion prediction and motion compensation. More specifically, the inter prediction unit 125 uses the reference image and the input image, performs motion prediction for the current block, for detecting a motion vector. The inter prediction unit 125 uses a reference image, performs motion compensation processing in accordance with the detected motion vector to generate a prediction image of the current block (inter prediction image information). This is the inter-prediction (that is, the method of generating a predicted image), a plurality of methods (both inter-prediction mode referred to), are prepared in advance as candidates. Inter prediction unit 125 performs such inter prediction in this previously prepared plural inter prediction modes.
[0152]
　Inter prediction unit 125 generates a prediction image in all inter prediction modes which are candidates. Inter prediction unit 125, an input image supplied from the screen rearrangement buffer 112, by using a generated difference motion vector information, it evaluates the cost function value of each prediction image, and selects an optimal mode. Inter prediction unit 125, selecting the optimal inter prediction mode, the prediction image generated in the optimum mode, and supplies the predicted image selection unit 126.
[0153]
　Inter prediction unit 125, and information indicating the inter prediction mode adopted, when decoding the encoded data, and supplies the information necessary for processing carried out on the the inter prediction mode to the reversible encoding unit 116, to be encoded. The required information, e.g., information of the generated difference motion vector, and the like flag indicating the index of the prediction motion vector as a prediction motion vector information.
[0154]
　Predicted image selecting unit 126 selects the source of the predicted image supplied to the arithmetic unit 113 and the execution unit 120. For example, in the case of intra-prediction image selecting unit 126 selects the intra prediction unit 124 as the supply source of the predicted image supplied to the predicted image supplied from the intra prediction unit 124 to the arithmetic unit 113 and the execution unit 120 to. Further, for example, in the case of inter encoding, the predicted image selecting unit 126 selects the inter prediction section 125 as a supply source of the predicted image, the inter prediction image supplied from the prediction unit 125 and the execution unit 113 operation unit 120 supplied to.
[0155]
　Rate control unit 127, based on the code amount of the encoded data accumulated in the accumulation buffer 117, as an overflow or underflow does not occur, controlling the rate of the quantization operation of the quantization unit 115.
[0156]
　Base layer area division setting unit 128 sets the area division for a picture of the base layer (e.g., a tile or sliced ​​etc.). Base layer area division setting unit 128, the setting, as a base layer region segmentation information, are supplied to each section of the base layer image coding unit 101. Each section of the base layer image encoding unit 101 executes processing for each region shown in the base layer area division information. Encoding of each region is processed independently of each other. Thus, for example, by using a plurality of CPU, it is also possible to parallel processing of the encoding of each region.
[0157]
　The base layer image coding unit 101 performs coding without reference to other layers. In other words, the intra prediction unit 124 and the inter prediction section 125 does not refer to information related to the coding of the other layers.
[0158]
　The frame memory 122 supplies the image data of the decoded pictures stored are the base layer, as the information related to the coding of the base layer, the enhancement layer video encoder 102.
[0159]
　Likewise, the intra prediction unit 124, an intra prediction mode information such as information related to the coding of the base layer, is supplied to the enhancement layer video encoder 102.
[0160]
　Similarly, the inter prediction unit 125 supplies the motion information such as information related to the coding of the base layer, the enhancement layer video encoder 102.
[0161]
　Furthermore, the base layer area division setting unit 128, a base layer region segmentation information is supplied to the enhancement layer video encoder 102.
[0162]
　　
　FIG. 20 is a block diagram showing a main configuration example of an enhancement layer video encoder 102 of FIG. 18. Figure As shown in 20, the enhancement layer video encoder 102 includes a base and a base layer image coding unit 101 to the same structure of Figure 19.
[0163]
　That is, the enhancement layer video encoder 102, as shown in FIG. 20, A / D conversion unit 131, a screen rearrangement buffer 132, an arithmetic unit 133, orthogonal transform unit 134, a quantization unit 135, a lossless encoding section 136, has a storage buffer 137, an inverse quantization unit 138, and the inverse orthogonal transform unit 139. Further, the enhancement layer video encoder 102, an arithmetic unit 140, a loop filter 141, a frame memory 142, selection unit 143, an intra prediction unit 144, an inter prediction unit 145, predicted image selecting unit 146 and rate control unit 147, .
[0164]
　These A / D converter 131 to the rate control unit 147 corresponds to the A / D converter 111 to the rate control unit 127 of FIG. 19, respectively, performs the same processing as the corresponding processing unit. However, each part of the enhancement layer video encoder 102, not the base layer, the processing associated with coding of enhancement layer picture information. Therefore, explanation of the processing in the A / D converter 131 to the rate control unit 147, can be applied to the description of the A / D converter 111 to the rate control unit 127 of FIG. 19 described above, in which case the process data, not the data of the base layer, it is necessary assumed to be data of an enhancement layer that. The input source and output destination of the processing unit of the data, as appropriate, it is necessary to read by replacing the corresponding processing unit of the A / D converter 131 to the rate control unit 147.
[0165]
　Further, the enhancement layer video encoder 102 does not have a base layer region division setting unit 128 has a region synchro unit 148 and the up-sampling section 149.
[0166]
　Region synchro unit 148 sets the area division for picture enhancement layer (e.g., a tile or sliced ​​etc.). Region synchro unit 148 supplies the setting, as enhancement layer area division information, to each part of the enhancement layer video encoder 102.
[0167]
　The region synchro unit 148 in the coding of the enhancement layer, to control the area that refers to encoding information of the base layer. For example, the region synchro unit 148 generates control information for controlling the area that refers to encoding information of the base layer, in accordance with the control information, controls the intra prediction unit 144 and the inter prediction section 145. In other words, the region synchro unit 148, intra prediction unit 144 and inter prediction unit 145 when performing inter-layer prediction, controls the area of ​​the base layer that refers to encoding information.
[0168]
　Furthermore, the region synchro unit 148 supplies the control information to the reversible encoding unit 136, is coded, is transmitted to the decoding side.
[0169]
　Incidentally, the enhancement layer video encoder 102, encoding is performed by referring to the information related to the coding of the other layers (e.g., base layer).
[0170]
　Region synchro unit 148 obtains a base layer region division information supplied from the base layer image coding unit 101. Region synchro unit 148, by using the base layer area division information, and generates the control information described above.
[0171]
　Upsampling section 149 obtains information related to the coding of the base layer is supplied from the base layer image coding unit 101. For example, up-sampling section 149, a texture information such as the decoded image of the base layer (also the base layer decoded image referred to), it is obtained as information about the encoding. For example, when the syntax prediction process between layers (interlayer prediction) is performed, the up-sampling unit 149, syntax information such as the motion information and the intra prediction mode information of the base layer is also obtained as the information about the coding.
[0172]
　Upsampling section 149, the information related to the coding of the thus obtained base layer upsampling process. Between layers in the hierarchical coding, the value of a predetermined parameter having a scalability function (e.g. resolution, etc.) are different from each other. Therefore, up-sampling section 149, (performs conversion processing of the scalable parameters) the value of the parameter as converted to the enhancement layer reference, the information up sample processing related to the coding of the base layer. By thus be up sample processing, information related to the coding of the base layer, it is possible to use in the coding of the enhancement layer.
[0173]
　Upsampling section 149, the information related to the coding of the up sample processing base layer is supplied to a frame memory 142 for storage. Information related to the coding of the base layer, for example, as a reference image, is supplied to the intra prediction unit 144 and the inter prediction section 145. Incidentally, similarly syntax information, is supplied to the intra prediction unit 144 and the inter prediction section 145.
[0174]
　　
　FIG. 21 is a block diagram showing a main configuration example of a region sync portion 148 in FIG. 20.
[0175]
　As shown in FIG. 21, the region synchro unit 148 has a base layer region segmentation information buffer 171, an enhancement layer area division setting unit 172 and the region synchro setting unit 173,.
[0176]
　The base layer region segmentation information buffer 171 acquires a base layer region division information supplied from the base layer image coding unit 101, holds. The base layer region segmentation information buffer 171 supplies at a predetermined timing, or according to a request from the outside such as the region synchro setting unit 173, a base layer region division information held in the region synchro setting unit 173.
[0177]
　Enhancement layer area division setting unit 172 sets area division of the picture of the enhancement layer (for example, tiles and slices, etc.). Setting this area division is arbitrary. For example, such as a user or an application may be set, or may be predetermined. The area division of the enhancement layer may be the same as the region division of the base layer, it may be different.
[0178]
　Enhancement layer area division setting unit 172 supplies the setting, as enhancement layer area division information, to each part of the enhancement layer video encoder 102. Each part of the enhancement layer video encoder 102 performs the processing for each region shown in the enhancement layer area division information. Encoding of each region is processed independently of each other. Thus, for example, by using a plurality of CPU, it is also possible to parallel processing of the encoding of each region.
[0179]
　Further, the enhancement layer area division setting unit 172, the generated enhancement layer area division information, and supplies to the region synchro setting unit 173.
[0180]
　Furthermore, the enhancement layer area division setting unit 172, the generated enhancement layer area division information is supplied to the lossless encoding unit 136, is coded, it is transmitted to the decoding side. Accordingly, the decoding side since it is possible to perform decoding by referring to this information, it is possible to reduce the load of the decoding.
[0181]
　Region synchro setting unit 173 uses the base layer area division information and the enhancement layer division information supplied, to associate the region between layers. In other words, the region synchro setting unit 173, for each region of the enhancement layer sets an additional information related to the coding of the base layer during the encoding.
[0182]
　Region synchro setting unit 173 generates the sync area information indicating this setting. The sync area information, as long as it controls the area of ​​the base layer to be reference destination information related to the coding, it may be information of any specification. For example, for each region of the enhancement layer may be information associating a region of the base layer to be reference destination information related to the coding. For example, <1. It may be information such as syntax described above in the main description of the techniques>.
[0183]
　Note that this setting process is arbitrary. That is, whether to refer to any region in the intra prediction unit 144 and inter prediction unit 145, is determined by any method. For example, such as a user or an application may be set, or may be predetermined.
[0184]
　Region synchro setting unit 173, using the generated sync area information, to be processed current region to identify the region of the base layer to be reference destination information related to the coding, upsampling processing is stored in the frame memory 142 coded infos (see, for example, images and the like texture information, or syntax information, such as motion information, intra prediction mode information) to generate a sync address information indicating the position of data of the area in the data (address), It supplies the sync address information to the intra prediction unit 144 and the inter prediction section 145.
[0185]
　An intra prediction unit 144 and inter prediction unit 145, since the inter-layer prediction according to the sync address information, only a partial region of a picture of the base layer to be a reference destination, the number of accesses to the frame memory 142 it is possible to suppress the increase. In other words, the region synchro setting unit 173, by performing such processing, it is possible to suppress the increase of the load of encoding.
[0186]
　The region synchro setting unit 173, the generated sync area information, and supplies the lossless encoding unit 136, is coded, is transmitted to the decoding side. Accordingly, since it is possible to perform decoding by referring to the sync area information at the decoding side, it can also be suppressed similarly increased number of accesses to the memory in the decoding, thereby reducing the load of the decoding be able to.
[0187]
　　
　Next, a flow of each process executed by the image encoding apparatus 100 will be described. First, with reference to the flowchart of FIG. 22, an example of a flow of the image encoding process.
[0188]
　When the image coding process is started, in step S101, the base layer image encoding unit 101 of the image encoding apparatus 100 encodes the image data of the base layer.
[0189]
　In step S102, an enhancement layer video encoder 102 encodes the image data of the enhancement layer.
[0190]
　In step S103, the multiplexing unit 103 includes a base layer image coded stream generated by the processing in step S101, and an enhancement layer image coding stream generated by the processing in step S102 (i.e., each layer bit stream the) multiplexed to generate a layer image encoded stream of one system.
[0191]
　When the process of step S103 is completed, the image coding apparatus 100 ends the image encoding process. One picture is processed by such image coding processing. Accordingly, the image encoding apparatus 100 repeatedly executes for each picture of layered video data such image coding processing.
[0192]
　　
　Next, in step S101 of FIG. 22, an example of base layer coding the flow of the processing executed by the base-layer image coding unit 101, with reference to the flowchart of FIG. 23 described to.
[0193]
　When the base layer encoding process is started, the base layer area division setting unit 128 of the base layer image coding unit 101, at step S121, determines the segmentation of the base layer in a predetermined manner, the base layer area division information to generate. The base layer area division setting unit 128, the base layer region segmentation information, are supplied to each section of the base layer image coding unit 101.
[0194]
　In step S122, the base layer area division setting unit 128 supplies the base layer area division information generated in step S121 to the lossless encoding unit 116, is transmitted.
[0195]
　Each processing after is executed for each area set in step S121. Thus, each treatment, the region, or, is performed a small predetermined unit than the area as a processing unit.
[0196]
　In step S123, A / D conversion unit 111, the image to the A / D conversion of each frame of the moving image input (picture).
[0197]
　In step S124, the screen rearrangement buffer 112 stores the A / D converted image in step S123, performs the rearrangement of the order of encoding the display order of each picture.
[0198]
　In step S125, the intra prediction unit 124 performs intra prediction processing of the intra prediction modes.
[0199]
　In step S126, the inter prediction unit 125 performs inter prediction process for performing motion prediction and motion compensation or the like in the inter prediction mode.
[0200]
　In step S127, the predicted image selecting unit 126, based on the cost function values, etc., selects a prediction image. That is, the predicted image selecting unit 126 selects the predicted image generated by the intra prediction in step S125, the one of the predicted image generated by the inter prediction step S126.
[0201]
　In step S128, the arithmetic unit 113 calculates an input image rearranged frame order by the processing of step S124, the difference between the prediction image selected by the processing in step S127. That is, the arithmetic unit 113 generates an image data of a difference image between the input image and the prediction image. Image data of the difference image obtained in this way, the data amount is reduced as compared to the original image data. Therefore, as compared with the case of directly coding the image, it is possible to compress the data amount.
[0202]
　In step S129, the orthogonal transform unit 114, orthogonal transform image data of the generated differential image by the processing in step S128.
[0203]
　In step S130, the quantization unit 115, using the quantization parameters calculated by the rate control unit 127 quantizes the orthogonal transform coefficient obtained by the processing in step S129.
[0204]
　In step S131, the inverse quantization unit 118, the quantized coefficients generated by the processing in step S130 the (also referred to as quantized coefficients), inverse quantization with characteristics corresponding to the characteristics of the quantization unit 115.
[0205]
　In step S132, the inverse orthogonal transform unit 119 performs inverse orthogonal transformation on the orthogonal transformation coefficient obtained by the processing in step S131.
[0206]
　In step S133, the arithmetic unit 120, the restored difference image by the process of step S132, by adding the predictive image selected by the processing in step S127, and generates the image data of the reconstructed image.
[0207]
　Step loop filter 121 in S134 performs the loop filter processing on the image data of the reconstructed image generated by the processing in step S133. Thus, the block distortion of the reconstructed image are removed.
[0208]
　In step S135, the frame memory 122 stores data, such as reconstructed image obtained by the processing of the decoded image and the step S133 obtained by the processing in step S134.
[0209]
　In step S136, the lossless encoding section 116, obtained by the processing in step S130, it encodes the quantized coefficients. That is, for the data corresponding to the difference image, the variable length coding or arithmetic coding lossless encoding or the like is performed.
[0210]
　At this time, the reversible encoding unit 116, the information about the prediction mode of the prediction image selected by the processing in step S127 is encoded and added to the encoded data obtained by the difference image is coded. That is, lossless encoding unit 116, the optimum intra prediction mode information supplied from the intra prediction unit 124, or even coded such information according to the optimum inter prediction mode supplied from the inter prediction unit 125, the encoded data added to.
[0211]
　Furthermore, the lossless encoding unit 116, syntax elements such as various NAL units also set, coded, adding it to the encoded data.
[0212]
　Storage buffer 117 at step S137 stores the encoded data obtained by the processing in step S136. Encoded data accumulated in the storage buffer 117 is appropriately read out, and transmitted to the decoding side via a transmission path or a recording medium.
[0213]
　Rate control unit 127 in step S138, on the basis of the code amount of encoded data accumulated in the accumulation buffer 117 by the processing in step S137 (generation code amount), so that overflow or underflow does not occur, the quantization unit 115 controlling the rate of the quantization operation. Moreover, the rate control unit 127, the information about the quantization parameter, and supplies the quantization unit 115.
[0214]
　In step S139, the frame memory 122, an intra prediction unit 124, an inter prediction unit 125, and a base layer region division setting unit 128, the information related to the coding of the resulting base layer in the base layer coding processing as described above, supplied to the encoding process of an enhancement layer.
[0215]
　When the process of step S139 is completed, the base layer coding process is completed, the process returns to FIG. 22.
[0216]
　　
　Next, in step S102 of FIG. 22, an example of a flow of the enhancement layer encoding process performed by the enhancement layer video encoder 102, reference to the flowcharts of FIGS. 24 and 25 and it will be described.
[0217]
　When the enhancement layer encoding process is started, the base layer area division information buffer 171 of the enhancement layer video encoder 102, in step S151, the generated in the base layer coding processing, the base layer area division information supplied get.
[0218]
　In step S152, the up-sampling unit 149 is generated in the base layer coding process, to obtain a base layer decoded image supplied (that texture information) as information about the encoding. Incidentally, if the inter-layer syntactic spray prediction is performed, the up-sampling unit 149 is generated in the base layer coding process also acquires the information related to the coding syntax information supplied.
[0219]
　In step S153, the upsampling section 149, information related to the coding of the obtained base layer in step S152 (e.g., base layer decoding image) upsampling processes.
[0220]
　In step S154, the frame memory 142, information related to the coding of the up sample processing base layer by the process of step S153 (e.g., base layer decoding image) stores.
[0221]
　In step S155, the enhancement layer area division setting unit 172 determines the segmentation of the enhancement layer in a predetermined manner, generating an enhancement layer area division information. Further, the enhancement layer area division setting unit 172 supplies the enhancement layer area division information, to each part of the enhancement layer video encoder 102.
[0222]
　In step S156, the area synchro setting unit 173, by using the base layer area division information acquired in step S151, the an enhancement layer area division information generated in step S155, a predetermined method, generating a sync region information to. In other words, the region synchro setting unit 173, for each region of the enhancement layer, and sets an area of ​​the base layer to be reference destination information related to the coding.
[0223]
　In step S157, the area synchro setting unit 173, using the sync area information generated by the processing in step S156, generates a sync address information indicating a data region of the base layer to be reference destination information related to the coding.
[0224]
　In step S158, the area synchro setting unit 173 supplies the sync region information generated by the processing in step S156 to the lossless encoding unit 136, is transmitted. Further, the enhancement layer area division setting unit 172, the enhancement layer area division information generated by the processing in step S155 is supplied to the lossless encoding unit 136, it is transmitted.
[0225]
　When the process of step S158 ends, the process proceeds to step S161 in FIG. 25.
[0226]
　Each processing after is executed for each area set in step S155. Thus, each treatment, the region, or, is performed a small predetermined unit than the area as a processing unit.
[0227]
　The processes of steps S161 to step S176 of FIG. 25 corresponds to the processing in steps S123 to step S138 of FIG. 23, is executed as well as their processing.
[0228]
　When the process of step S176 is completed, the enhancement layer coding process is completed, the process returns to FIG. 22.
[0229]
　By executing the processes as described above, the image coding apparatus 100, the inter-layer prediction to reduce memory access for referring to the information related to the coding of the other layers, increase in the load of the encoding and decoding it is possible to suppress.
[0230]
　<3. Second Embodiment>
　　
　Next, the decoding of the coded data is described as above. Figure 26 shows an embodiment of an image processing apparatus according to the present technique, a block diagram illustrating a main configuration example of an image decoding apparatus corresponding to the image encoding apparatus 100 of FIG. 18.
[0231]
　The image decoding apparatus shown in FIG. 26 200, the encoded data by the image encoding apparatus 100 is generated and decoded by the decoding method corresponding to the coding method (i.e., a hierarchical decoding the hierarchically encoded data to).
[0232]
　As shown in FIG. 26, the image decoding apparatus 200 includes a demultiplexer 201, a base layer image decoding section 202 and enhancement layer image decoding unit 203,.
[0233]
　Demultiplexing section 201, transmitted from the encoding side, receives the hierarchical image coding stream the base layer image encoded stream and an enhancement layer image encoding stream are multiplexed, demultiplexes it, the base layer an image encoded stream and an enhancement layer image encoding stream extracted.
[0234]
　Base layer image decoding section 202 decodes the base layer image coded stream extracted by the demultiplexer 201 to obtain the base layer image. At that time, the base layer image decoding unit 202, based on the base layer area division information supplied from the encoding side, the region set the encoding side (tiles and slices, etc.) decodes each.
[0235]
　Enhancement layer image decoding section 203 decodes the enhancement layer image encoding stream extracted by the demultiplexer 201 to obtain an enhancement layer image. At that time, the enhancement layer image decoding unit 203, based on the enhancement layer area division information supplied from the encoding side, the region set the encoding side (tiles and slices, etc.) decodes each.
[0236]
　Further, the enhancement layer image decoding unit 203, use is supplied from the encoding side, the respective regions of the enhancement layer, the sync area information is control information for controlling the region of the base layer to be reference destination information related to the coding to perform the inter-layer prediction. That is, the enhancement layer image decoding unit 203, the decoding of the enhancement layer, when performing inter-layer prediction refers to the information related to the coding region of the base layer is designated by the sync area information.
[0237]
　　
　FIG. 27 is a block diagram showing a main configuration example of a base layer image decoding unit 202 of FIG. 26. Base layer image decoding unit 202 as shown in FIG. 27, a storage buffer 211, a lossless decoding unit 212, an inverse quantization unit 213, inverse orthogonal transform unit 214, calculation unit 215, a loop filter 216, a screen rearrangement buffer 217, and a D / a converter unit 218. The base layer image decoding unit 202 includes a frame memory 219, selection unit 220, an intra prediction unit 221, an inter prediction unit 222 and the predicted image selecting section 223,.
[0238]
　The storage buffer 211 is also a receiving unit receiving the encoded data transmitted. Accumulation buffer 211 receives the encoded data transmitted, accumulating and supplies the encoded data to the lossless decoding unit 212 at a predetermined timing. The encoded data, information necessary for decoding, such as prediction mode information is added. Lossless decoding unit 212, supplied from the accumulation buffer 211, the information encoded by the lossless coding unit 116 is decoded by the decoding method corresponding to the coding method. Lossless decoding unit 212, the coefficient data quantized in the difference image obtained by decoding to the inverse quantization unit 213.
[0239]
　Also, the lossless decoding unit 212 determines whether the inter prediction mode intra prediction mode is selected is selected optimum prediction mode, information about the optimum prediction mode, the intra prediction unit 221 and the inter prediction section 222 among supplied towards the mode is determined to have been selected. That is, for example, when the intra prediction mode is selected as the optimum prediction mode in the encoding side, information about the optimum prediction mode is supplied to the intra prediction unit 221. For example, when the inter prediction mode is selected as the optimum prediction mode in the encoding side, information about the optimum prediction mode is supplied to the inter prediction unit 222.
[0240]
　Furthermore, the lossless decoding unit 212, for example, such as quantization matrices and quantization parameter and supplies information necessary for inverse quantization in the inverse quantization unit 213.
[0241]
　Also, the lossless decoding unit 212, a base layer region segmentation information supplied from the encoding side, and supplies to each processing unit of the base layer image decoding unit 202. Each section of the base layer image decoding unit 202 performs the processing for each region shown in the base layer area division information. Decoding each region is processed independently of each other. Thus, for example, by using a plurality of CPU, it is also possible to parallel processing of the decoding of each region.
[0242]
　Inverse quantization unit 213, the coefficient data quantized obtained by decoding by the lossless decoding unit 212, inverse quantization in a manner corresponding to the quantization method of the quantization unit 115. Incidentally, the inverse quantization unit 213 is a processing unit similar to the inverse quantization unit 118. In other words, the description of the inverse quantization unit 213 may be applied mutatis mutandis to the inverse quantization unit 118. However, the input source or output destination of data, it is necessary to read by replacing the processing of the base layer image decoding unit 202.
[0243]
　Inverse quantization unit 213 supplies the obtained coefficient data to the inverse orthogonal transform unit 214.
[0244]
　Inverse orthogonal transform unit 214, an orthogonal transformation coefficient supplied from the inverse quantization unit 213, as necessary, to the inverse orthogonal transform by the corresponding method to the orthogonal transform scheme of the orthogonal transform unit 114. Incidentally, the inverse orthogonal transform unit 214, a processing unit similar to the inverse orthogonal transform unit 119. In other words, the description of the inverse orthogonal transform unit 214, it can be applied mutatis mutandis to the inverse orthogonal transform unit 119. However, the input source or output destination of data, it is necessary to read by replacing the processing of the base layer image decoding unit 202.
[0245]
　Image data of the difference image is restored by the inverse orthogonal transform processing. Image data of the restored difference image corresponding to the image data of the previous difference image orthogonal transform in the image encoding apparatus. Hereinafter, this was obtained by the inverse orthogonal transformation processing of the inverse orthogonal transform unit 214, the image data of the restored difference image, also referred to as decoded residual data. Inverse orthogonal transform unit 214 supplies the decoded residual data, to the arithmetic unit 215. Further, the arithmetic unit 215 via the predicted image selection unit 223, the image data of the predictive image from the intra prediction unit 221 or the inter prediction unit 222 is supplied.
[0246]
　Calculation unit 215, by using the image data of the decoded residual data and the predicted image to obtain image data of the reconstructed image by adding the predicted image and the differential image. The reconstructed image corresponding to the previous input image predictive image by calculating unit 113 is subtracted. Calculation unit 215 supplies the reconstructed image to the loop filter 216.
[0247]
　Loop filter 216 subjects the supplied reconstructed image to generate a suitably subjected to decoded image loop filter processing including deblocking filtering and adaptive loop filter processing. For example, the loop filter 216, by performing deblock filter processing on the reconstructed image, to remove the block distortion. Further, for example, a loop filter 216, for the deblocking filtering result (reconstructed image removal of block distortion is performed), image quality improvement by performing the loop filter processing by using the Wiener filter (Wiener Filter) I do.
[0248]
　The type of filtering loop filter 216 performs is arbitrary, it may be performed filtering other than those described above. Further, the loop filter 216, the filter coefficient supplied from the image encoding apparatus may perform a filtering process using. Furthermore, the loop filter 216, skip this filtering process, it is also possible to output the input data without filtering.
[0249]
　Loop filter 216 supplies the decoded image is a filter processing result (or reconstructed image) to rearrangement buffer 217 and the frame memory 219 screen.
[0250]
　Screen rearrangement buffer 217 rearranges the order of the frame for the decoded image. That is, the screen rearrangement buffer 217, an image of each frame are sorted into coding order by the screen rearrangement buffer 112 rearranges the original display order. That is, the screen rearrangement buffer 217, an image data of a decoded image of each frame supplied to the coding order, and stores in this order, the image data of a decoded image of each frame stored in the coding order, reads the display order D / A converter unit 218. D / A conversion unit 218, the decoded image of each frame supplied from the screen rearrangement buffer 217 (digital data) to convert D / A, as analog data, and outputs the display not illustrated in the drawing, is displayed.
[0251]
　The frame memory 219 stores the decoded image supplied, at a predetermined timing, or based on an external request, such as the intra prediction unit 221 and inter prediction unit 222, as a reference image the decoded image stored, supplied to the intra prediction unit 221 and the inter prediction section 222 via the selector 220.
[0252]
　The intra prediction unit 221, intra prediction mode information and the like are properly supplied from the lossless decoding unit 212. The intra prediction unit 221 performs intra prediction in the intra prediction mode used in the intra prediction unit 124 (the optimal intra prediction mode), to generate a prediction image. At that time, the intra prediction unit 221 performs intra prediction using the image data of the reconstructed image supplied from the frame memory 219 via the selector 220. That is, the intra prediction unit 221 utilizes the reconstructed image as a reference image (peripheral pixels). The intra prediction unit 221 supplies the generated predicted image to the predicted image selection unit 223.
[0253]
　The inter prediction unit 222, optimum prediction mode information and the motion information and the like are properly supplied from the lossless decoding unit 212. Inter prediction unit 222 performs inter prediction using at inter prediction mode indicated by the optimum prediction mode information obtained from the lossless decoding unit 212 (the optimal inter prediction mode), the decoded image obtained from the frame memory 219 (reference image) , to generate a prediction image.
[0254]
　Predicted image selecting unit 223, a prediction image supplied from the predicted image or inter prediction unit 222 is supplied from the intra prediction unit 221, and supplies the arithmetic unit 215. Then, the arithmetic unit 215, the reconstructed image is obtained by adding the decoded residual data from the predicted image and the inverse orthogonal transform unit 214 (differential image information).
[0255]
　The base layer image decoding unit 202 performs decoding without referring to the other layers. In other words, the intra prediction unit 221 and the inter prediction section 222 does not refer to information related to the coding of the other layers.
[0256]
　The frame memory 219 stores the image data of the decoded pictures stored are the base layer, as the information related to the coding of the base layer, is supplied to the enhancement layer image decoding unit 203.
[0257]
　Likewise, the intra prediction unit 221, an intra prediction mode information such as information related to the coding of the base layer, is supplied to the enhancement layer image decoding unit 203.
[0258]
　Similarly, the inter prediction unit 222, motion information, etc., as information related to the coding of the base layer, is supplied to the enhancement layer image decoding unit 203.
[0259]
　Furthermore, the intra prediction unit 221 or the inter prediction unit 222 (or, like the lossless decoding unit 212, any processing of the base layer image decoding unit 202) supplies a base layer region segmentation information, the enhancement layer image decoding unit 203 .
[0260]
　　
　FIG. 28 is a block diagram showing a main configuration example of the enhancement layer image decoding unit 203 of FIG. 26. As shown in FIG. 28, an enhancement layer image decoding unit 203 has basically the same configuration as the base layer image decoding unit 202 of FIG. 27.
[0261]
　That is, the enhancement layer image decoding unit 203, as shown in FIG. 28, a storage buffer 231, a lossless decoding unit 232, an inverse quantization unit 233, inverse orthogonal transform unit 234, calculation unit 235, a loop filter 236, the screen rearrangement a buffer 237 and the D / a converter 238,. Further, the enhancement layer image decoding unit 203 includes a frame memory 239, selection unit 240, an intra prediction unit 241, an inter prediction unit 242 and the predicted image selecting section 243,.
[0262]
　These storage buffer 231 to the predicted image selection unit 243 corresponds to the storage buffer 211 to the predicted image selection unit 223 of FIG. 27, respectively, performs the same processing as the corresponding processing unit. However, each part of the enhancement layer image decoding unit 203, not the base layer, the processing associated with coding of enhancement layer picture information. Therefore, explanation of the processing in the storage buffer 231 to the predicted image selection unit 243, can be applied to the description of the storage buffer 211 to the predicted image selection unit 223 of FIG. 27 described above, in which case, the data to be processed, not the data of the base layer, it is necessary assumed to be data of an enhancement layer. The input source and output destination of the processing unit of the data, as appropriate, of the enhancement layer image decoding section 203, it is necessary to read by replacing the processor a corresponding.
[0263]
　Further, the enhancement layer image decoding unit 203 includes a region sync unit 244 and the up-sampling section 245.
[0264]
　Region synchro unit 244 obtains the enhancement layer area division information supplied from the lossless decoding unit 232 and the sync area information. This information is generated at the decoding side, those that have been transmitted from the decoding side. Furthermore, the region synchro unit 244 also obtains a base layer region division information supplied from the base layer image decoding unit 202.
[0265]
　Region synchro unit 244 uses these information, the decoding of the enhancement layer, to control the area that refers to encoding information of the base layer. For example, the region synchro unit 244 uses these information, the intra prediction unit 241 and inter prediction unit 242 when performing inter-layer prediction, controls the area of ​​the base layer that refers to encoding information. By doing so, the area synchro unit 244, similarly to the time of encoding, it is possible to control the area referenced in the decoding of the enhancement layer, the information related to the coding of the base layer. Therefore, the region synchro unit 244 reduces the memory access, it is possible to suppress an increase in the decoding load.
[0266]
　Enhancement layer image decoding unit 203 performs encoding by referring to the information related to the coding of the other layers (e.g., base layer).
[0267]
　Upsampling section 245 obtains information related to the coding of the base layer is supplied from the base layer image decoding unit 202. For example, the upsampling unit 245, a texture information such as the decoded image of the base layer (also the base layer decoded image referred to), is obtained as information about the encoding. For example, when the syntax prediction process between layers (interlayer prediction) is performed, the up-sampling unit 245, syntax information such as the motion information and the intra prediction mode information of the base layer is also obtained as the information about the coding.
[0268]
　Upsampling unit 245, the information related to the coding of the thus obtained base layer upsampling process. Between layers in the hierarchical coding, the value of a predetermined parameter having a scalability function (e.g. resolution, etc.) are different from each other. Therefore, up-sampling section 245 (performs conversion processing of scalable parameters) the value of the parameter as converted to the enhancement layer reference, the information up sample processing related to the coding of the base layer. By thus be up sample processing, information related to the coding of the base layer, it is possible to use in the decoding of the enhancement layer.
[0269]
　Upsampling section 149, the information related to the coding of the up sample processing base layer is supplied to a frame memory 239 for storage. Information related to the coding of the base layer, for example, as a reference image, is supplied to the intra prediction unit 241 and the inter prediction section 242. Incidentally, similarly syntax information, is supplied to the intra prediction unit 241 and the inter prediction section 242.
[0270]
　　
　FIG. 29 is a block diagram showing a main configuration example of a region sync portion 244 in FIG. 28.
[0271]
　As shown in FIG. 29, the region synchro unit 244 has a base layer region segmentation information buffer 271, an enhancement layer area division information buffer 272 and the sync area information decoding section 273,.
[0272]
　The base layer region segmentation information buffer 271, a base layer region division information supplied from the base layer image decoding unit 202, i.e., to get the base layer area division information supplied from the encoding side, to hold. The base layer region segmentation information buffer 271, at a predetermined timing, or according to a request from the outside such as synchronous area information decoding section 273 supplies the base layer area division information held in the sync area information decoding section 273.
[0273]
　Enhancement layer area division information buffer 272, an enhancement layer area division information supplied from the lossless decoding unit 232, i.e., obtains the enhancement layer area division information supplied from the encoding side, to hold. Enhancement layer area division information buffer 272 at a predetermined timing, or according to a request from the outside such as synchronous area information decoding section 273 supplies the enhancement layer area division information held in the sync area information decoding section 273.
[0274]
　Sync area information decoding unit 273 obtains a base layer region segmentation information from the base layer area division information buffer 271, obtains the enhancement layer region segmentation information from the enhancement layer area division information buffer 272. Further, sync area information decoding section 273, sync area information supplied from the lossless decoding unit 232, i.e., acquires the sync area information supplied from the encoding side, to hold.
[0275]
　The sync area information, for each region of the enhancement layer, which is information to control the region of the base layer to be reference destination information related to the coding. Sync area information decoding section 273, by using the base layer area division information and the enhancement layer area division information, decodes the sync area information. In other words, sync area information decoding section 273 grasps the positional relationship between the regions of the layer between using a base layer area division information and the enhancement layer region segmentation information, in accordance with the positional relationship, the region between the layers indicated synchrotron area information to analyze the correspondence.
[0276]
　More specifically, sync area information decoding section 273, the information of the data related to the coding, such as the reference image supplied from the frame memory 239, reference destination information about the current region coding to be processed in the enhancement layer identifying the location of the data area of ​​the base layer to. Sync area information decoding section 273 generates a sync address information indicating the position of the data, and supplies it to the intra prediction unit 241 or the inter prediction unit 242.
[0277]
　Since sync area information decoding section 273 is information information supplied from all the encoding side to use, by doing so, sync area information decoding section 273, as well as to generate a region sync setting section 173 it can generate sync address information. In other words, sync area information decoding unit 273 can perform the same control as the region synchro setting unit 173.
[0278]
　An intra prediction unit 241 and inter prediction unit 242, since the inter-layer prediction according to the sync address information, can be only the referenced portion of the region of the picture of the base layer, the number of accesses to the frame memory 239 it is possible to suppress the increase. In other words, sync area information decoding section 273, by performing such processing, to reduce memory access, it is possible to suppress an increase in the decoding load.
[0279]
　　
　Next, a flow of each process executed by the image decoding apparatus 200 will be described. First, with reference to the flowchart of FIG. 30, an example of a flow of the image decoding processing.
[0280]
　When the image decoding process is started, in step S201, the demultiplexer 201 of the image decoding apparatus 200 demultiplexes for each layer of the hierarchical image coding stream transmitted from the encoding side.
[0281]
　In step S202, the base layer image decoding unit 202 decodes the base layer image coded stream extracted by the processing in step S201. Base layer image decoding unit 202 outputs the data of the base layer image generated by the decoding.
[0282]
　In step S203, an enhancement layer image decoding section 203 decodes the enhancement layer image encoding stream extracted by the processing in step S201. Enhancement layer image decoding unit 203 outputs the data of the enhancement layer image generated by the decoding.
[0283]
　When the process of step S203 is completed, the image decoding apparatus 200 terminates the image decoding processing. One picture is processed by such an image decoding processing. Thus, the image decoding apparatus 200 repeatedly executes for each picture of layered video data such image decoding processing.
[0284]
　　
　Next, in step S202 of FIG. 30, an example of a flow of the base layer decoding process performed by the base layer image decoding unit 202 will be described with reference to the flowchart of FIG. 31.
[0285]
　When the base layer decoding process is started, the base lossless decoding unit 212 of the base layer image decoding unit 202, in step S221, which decodes the encoded data acquired via the storage buffer 211, which is supplied from the encoding side to get the layer region segmentation information. Also, the lossless decoding unit 212, the base layer region segmentation information, are supplied to each section of the base layer image decoding unit 202.
[0286]
　Each processing after is executed for each area set in step S221. Thus, each treatment, the region, or, is performed a small predetermined unit than the area as a processing unit.
[0287]
　In step S222, the accumulation buffer 211 accumulates the bit stream (coded data) that has been transmitted. In step S223, the lossless decoding unit 212 decodes the bit stream (encoded data) supplied from the accumulation buffer 211. That is, the encoded I-picture by the lossless coding unit 116, P picture, and picture data, such as B-picture is decoded. At this time, various information other than the image data that is included in the bit stream, such as header information are also decoded.
[0288]
　In step S224, the inverse quantization unit 213, obtained by the processing in step S223, inverse-quantizes the quantized coefficients.
[0289]
　In step S225, the inverse orthogonal transform unit 214 performs inverse orthogonal transformation to the inverse quantized coefficients in step S224.
[0290]
　In step S226, the intra prediction unit 221 or the inter prediction unit 222 performs prediction processing, and generates a prediction image. That is, it is determined at the reversible decoding unit 212, the prediction processing with the applied prediction mode in the encoding is performed. More specifically, for example, if intra prediction has been applied in encoding, the intra prediction unit 221 generates a prediction image in the intra prediction mode optimal during encoding. For example, when the inter prediction is applied during encoding, inter-prediction unit 222 generates a prediction image in the inter prediction mode optimal during encoding.
[0291]
　In step S227, the arithmetic unit 215, the inverse orthogonal transformation has been obtained difference image in step S225, adds the predicted image generated in step S226. Thus, the image data of the reconstructed image is obtained.
[0292]
　In step S228, the loop filter 216, the image data of the obtained reconstructed image by the processing in step S227, performs appropriate loop filter processing including deblocking filtering and adaptive loop filter processing.
[0293]
　In step S229, the screen rearrangement buffer 217 rearranges the frames of the filtered reconstructed image at step S228. That is, the order of frames rearranged during coding, is rearranged to the original display order.
[0294]
　In step S230, D / A conversion unit 218, the image sequence frames sorted D / A conversion in step S229. This image is output to a display (not shown), an image is displayed.
[0295]
　In step S231, the frame memory 219 stores data such as the reconstructed image obtained by the processing of the decoded image and the step S227 obtained by the processing in step S228.

CLAIM

The scope of the claims

[Requested item 1]For the current layer of the image data comprising a plurality of layers, generating unit for generating control information for controlling the area referenced other layer encoded for each predetermined area which divides the picture into a plurality of information related to the coding When,
　under the control of said control information generated by the generation unit, and a coding unit for the reference information about the coding, coding the current layer of the image data in the partial region of the other layers,
　a transmission unit for transmitting the encoded data of the image data generated by the encoding unit, and said control information generated by the generating unit
　image coding apparatus comprising a.
[Requested item 2]
　Information the control information, the other layer, to specify a region to allow a reference to information on the coding, to specify a region for prohibiting reference information about the encoding, or relates to the encoded by specifying the area that reference is information to limit the area that can be referred to the information about the coded
　image coding apparatus according to claim 1.
[Requested item 3]
　Wherein the control information indicates the area, raster scan order assigned identification number, information indicating the longitudinal and lateral position of the region in the picture, or the location of the data of said region in the encoded data specifying the information
　image coding apparatus according to claim 2.
[Requested item 4]
　The transmission unit may further transmit the information indicating whether the control region refers to the information about the coded
　image coding apparatus according to claim 1.
[Requested item 5]
　Information about the coding is information to be used in generating a predicted image to be used in encoding of the image data
　the image coding apparatus according to claim 1.
[Requested item 6]
　Information that is used to generate the prediction image, and information used for texture prediction of the image data, and a information used for syntax prediction of the image data,
　the control information, the information used for the texture prediction a reference region and a region that refers to information used for the syntax prediction is information for controlling independently of one another
　image coding apparatus according to claim 5.
[Requested item 7]
　The generation unit of the current layer of the image data, generates the control information for each predetermined area which divides the picture into a plurality of
　the encoding unit, the current layer of the image data for each of the areas under the control of the control information of each region in which the generator has generated, by referring to the information about the encoding of a partial area of the other layers, to encode
　the image coding apparatus according to claim 1.
[Requested item 8]
　The transmission unit, area division of the current layer is further for transmitting information indicating whether the same as the region division of the other layer
　picture coding apparatus according to claim 7.
[Requested item 9]
　The region is a slice or tile of the image data
　the image coding apparatus according to claim 1.
[Requested item 10]
　For the current layer of the image data comprising a plurality of layers, it generates control information for controlling the area referenced other layer encoded for each predetermined area which divides the picture into a plurality of information related to the coding,
　generating under the control of the control information refers to information relating to the encoding of a partial area of the other layers, the current layer of the image data is encoded,
　code the image data is generated by coding transmitting the data, the generated and the control information
　image coding method.
[Requested item 11]
　And the encoded data of the current layer of the image data composed of a plurality layers, the image data, the other layers which are encoded for each predetermined region to divide the picture into a plurality of the regions to view information about coding a receiving unit for receiving a control for controlling the information,
　under control of said control information received by the receiving unit, by referring to information about the encoding of a partial area of the other layers, decoding the encoded data a decoding unit for
　image decoding apparatus comprising a.
[Requested item 12]
　Information the control information, the other layer, to specify a region to allow a reference to information on the coding, to specify a region for prohibiting reference information about the encoding, or relates to the encoded by specifying the area that reference is information to limit the area that can see information about the encoded
　image decoding apparatus according to claim 11.
[Requested item 13]
　Wherein the control information indicates the area, raster scan order assigned identification number, information indicating the longitudinal and lateral position of the region in the picture, or the location of the data of said region in the encoded data specifying the information
　image decoding apparatus according to claim 12.
[Requested item 14]
　The receiving unit further receives information indicating whether the control region refers to the information about the encoded
　image decoding apparatus according to claim 11.
[Requested item 15]
　Information about the coding is information to be used in generating a predicted image to be used upon decoding the encoded data
　image decoding apparatus according to claim 11.
[Requested item 16]
　Information that is used to generate the prediction image, and information used for texture prediction of the image data, and a information used for syntax prediction of the image data,
　the control information, the information used for the texture prediction a reference region and a region that refers to information used for the syntax prediction is information for controlling independently of one another
　image decoding apparatus according to claim 15.
[Requested item 17]
　The receiving unit receives the encoded data for each predetermined region dividing the current layer of the image data, the picture into a plurality, and the control information for each of the areas,
　the decoder, the the encoded data received by the receiving portion for each of the areas under the control of the control information of each area refers to information relating to the encoding of a partial area of the other layers, to decode
　claim 11 the image decoding apparatus according to.
[Requested item 18]
　The receiving unit, area division of the current layer further receives information indicating whether the same as the region division of the other layer
　picture decoding apparatus according to claim 17.
[Requested item 19]
　The region is a slice or tile of the image data
　image decoding apparatus according to claim 11.
[Requested item 20]
　And the encoded data of the current layer of the image data composed of a plurality layers, the image data, the other layers which are encoded for each predetermined region to divide the picture into a plurality of the regions to view information about coding receives the control information for controlling,
　in accordance with the control of the received control information refers to information relating to the encoding of a partial area of the other layers, to decode the encoded data
　image decoding method.