Title of invention: Image processing apparatus and method
Technical field
[0001]
　The present disclosure relates to an image processing device and method, and more particularly, to an image processing device and method capable of suppressing quality reduction due to two-dimensional projection of 3D data.
Background technology
[0002]
　Conventionally, as a method for encoding 3D data representing a three-dimensional structure such as a point cloud, the position and color information of the point cloud are projected on a two-dimensional plane for each small area and used for a two-dimensional image. There is proposed an approach (hereinafter, also referred to as a video-based approach) in which encoding is performed by the above encoding method (see, for example, Non-Patent Documents 1 to 3).
Prior art documents
Non-patent literature
[0003]
Non-Patent Document 1: Tim Golla and Reinhard Klein, "Real-time Point Cloud Compression", IEEE, 2015
Non-Patent Document 2: K. Mammou, "Video-based and Hierarchical Approaches Point Cloud Compression", MPEG m41649, Oct. 2017
Non-Patent Document 3: "PCC Test Model Category 2 v0", N17248 MPEG output document, October 2017
Summary of the invention
Problems to be Solved by the Invention
[0004]
　However, the point cloud to be encoded has points other than the object surface due to noise and the characteristics of the imaging system. Therefore, it may be difficult to project onto the two-dimensional plane, and there is a risk that the quality is reduced by the encoding accompanied by the projection onto the two-dimensional plane.
[0005]
　The present disclosure has been made in view of such a situation, and makes it possible to suppress reduction in quality due to two-dimensional projection of 3D data.
Means for solving the problems
[0006]
　An image processing apparatus according to one aspect of the present technology is an image processing apparatus including a two-dimensional projection unit that projects data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers.
[0007]
　An image processing method according to one aspect of the present technology is an image processing method that projects data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers.
[0008]
　An image processing apparatus according to another aspect of the present technology includes a three-dimensional projection unit that projects data for every position of 3D data projected on a two-dimensional plane having the number of layers indicated by the number-of-layers information into a three-dimensional space. It is an image processing device.
[0009]
　An image processing method according to another aspect of the present technology is an image processing method that projects data for every position of 3D data projected on a two-dimensional plane having the number of layers indicated by the number-of-layers information, into a three-dimensional space.
[0010]
　In the image processing device and method according to one aspect of the present technology, data for every position included in 3D data representing a three-dimensional structure is projected onto a two-dimensional plane of a plurality of layers.
[0011]
　In the image processing device and method according to another aspect of the present technology, data for every position of the 3D data projected on the two-dimensional plane having the number of layers indicated by the number-of-layers information is projected on the three-dimensional space.
Effect of the invention
[0012]
　According to the present disclosure, information can be processed. In particular, it is possible to suppress the quality reduction due to the two-dimensional projection of 3D data.
Brief description of the drawings
[0013]
FIG. 1 is a diagram illustrating an example of a point cloud.
FIG. 2 is a diagram illustrating an example of an outline of a video-based approach.
FIG. 3 is a diagram summarizing the present technology described in each embodiment.
FIG. 4 is a block diagram showing a main configuration example of an encoding device.
FIG. 5 is a diagram illustrating a main configuration example of a patch decomposition unit.
FIG. 6 is a block diagram showing a main configuration example of a decoding device.
FIG. 7 is a block diagram showing a main configuration example of a 3D reconstruction unit.
FIG. 8 is a diagram showing an example of a conventional two-dimensional projection.
FIG. 9 is a diagram showing an example of a state of two-dimensional projection to which the present technology is applied.
FIG. 10 is a flowchart illustrating an example of the flow of encoding processing.
FIG. 11 is a flowchart illustrating an example of the flow of patch decomposition processing.
FIG. 12 is a flowchart illustrating an example of the flow of two-dimensional projection processing.
FIG. 13 is a flowchart illustrating an example of the flow of decoding processing.
FIG. 14 is a flowchart illustrating an example of the flow of point cloud reconstruction processing.
FIG. 15 is a flowchart illustrating an example of the flow of three-dimensional projection processing.
FIG. 16 is a diagram showing an example of a conventional two-dimensional projection.
FIG. 17 is a diagram showing an example of a state of two-dimensional projection to which the present technology is applied.
FIG. 18 is a flowchart illustrating an example of the flow of two-dimensional projection processing.
FIG. 19 is a flowchart illustrating an example of the flow of three-dimensional projection processing.
FIG. 20 is a diagram showing an example of a conventional two-dimensional projection.
FIG. 21 is a diagram showing an example of a state of two-dimensional projection to which the present technology is applied.
FIG. 22 is a flowchart illustrating an example of the flow of two-dimensional projection processing.
FIG. 23 is a flowchart illustrating an example of the flow of three-dimensional projection processing.
FIG. 24 is a block diagram showing a main configuration example of a computer.
MODE FOR CARRYING OUT THE INVENTION
[0014]
　Hereinafter, modes for carrying out the present disclosure (hereinafter, referred to as embodiments) will be described. The description will be given in the following order.
　1. Video-based approach
　2. First Embodiment (Variable number of layers)
　3. Second Embodiment (defined without points)
　4. Third Embodiment (Variable depth parameter)
　5. Note
[0015]
　<1. Video-based approach>
　　
　The scope disclosed by the present technology is not limited to the contents described in the examples, but also the following non-patent documents known at the time of application. The contents described in are also included.
[0016]
　Non-Patent Document 1: (above)
　Non-Patent Document 2: (above)
　Non-Patent Document 3: (above)
　Non-Patent Document 4: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04/2017
　Non-Patent Document 5: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12/2016
　Non-Patent Document 6: Jianle Chen, Elena Alshina, Gary J Sullivan, Jens-Rainer, Jill Boyce, "Algorithm Description of Joint Exploration Test Model 4", JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/ WG 11 7th Meeting: Torino, IT, 13-21 July 2017
[0017]
　That is, the contents described in the above non-patent documents also serve as the basis for determining the support requirement. For example, even if the Quad-Tree Block Structure described in Non-Patent Document 5 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Non-Patent Document 6 are not directly described in the examples, the present invention It is within the technical disclosure range and satisfies the support requirements of the claims. Further, for example, technical terms such as Parsing, Syntax, and Semantics are also within the scope of the disclosure of the present technology even when there is no direct description in the examples. Shall meet the support requirements in the range of.
[0018]
　　
　Conventionally, such as a point cloud that represents a three-dimensional structure based on the position information and attribute information of a point cloud, a mesh that is composed of vertices, edges, and faces and that defines a three-dimensional shape using polygonal representation. There was data.
[0019]
　For example, in the case of a point cloud, a three-dimensional structure as shown in A of FIG. 1 is expressed as a set of many points (point group) as shown in B of FIG. That is, the point cloud data is composed of position information and attribute information (for example, color) of each point of this point cloud. Therefore, the data structure is relatively simple, and an arbitrary three-dimensional structure can be represented with sufficient accuracy by using a sufficient number of points.
[0020]
　　 A video-based approach
　that projects such point cloud position and color information on a two-dimensional plane for each small area and encodes them using a two-dimensional image encoding method (Video-based approach) approach) is proposed.
[0021]
　In this video-based approach, for example, as shown in FIG. 2, an input point cloud is divided into a plurality of segmentations (also referred to as regions or patches), and each region is projected on a two-dimensional plane. The data for each position of the point cloud (that is, the data of each point) is composed of the position information (Geometry (also referred to as Depth)) and the attribute information (Texture) as described above, and is 2 for each region. It is projected on a dimensional plane.
[0022]
　The 3D data (point cloud) projected on the two-dimensional plane is encoded by a two-dimensional plane image encoding method such as AVC (Advanced Video Coding) or HEVC (High Efficiency Video Coding). ..
[0023]
　　 The present technology relating to the video-based approach
　as described above will be described. FIG. 3 is a list of the present technology described in each embodiment.
[0024]
　The first column from the top of this table (excluding the column of item name) describes the video-based approach in the conventional (TMC2). That is, in the conventional video-based approach, two-dimensional projection of 3D data is performed on a two-layer (two-layer) two-dimensional plane. This specification was common to the whole screen (the same projection was performed for every segment). Therefore, the projection control information used to control such projection is signaled from the encoding side to the decoding side as information on a frame-by-frame basis.
[0025]
　However, the point cloud to be encoded has points other than the object surface due to noise and the characteristics of the imaging system. Therefore, it may be difficult to project two layers onto a two-dimensional plane as in the conventional method described above. Therefore, there is a possibility that a point that cannot be projected on the two-dimensional plane is generated and the quality of the data is reduced by the encoding accompanied by the projection on the two-dimensional plane.
[0026]
　In addition, for example, when the compression target has a Sparse (sparse and sparse) property, it is necessary to project in units of minute areas (Point), and many small areas are encoded, resulting in poor processing amount and encoding efficiency. There was a risk of becoming.
[0027]
　In the second row from the top of the table in FIG. 3 (excluding the row of item names), the present technology described in the first embodiment (Example 1) is described. In this method, the number of layers of a two-dimensional plane onto which 3D data is projected is variable, and the number of layers is set so that data of all points that overlap in the depth direction (data for each position) can be projected. How to set.
[0028]
　In this case, for example, information indicating the number of layers may be signaled from the encoding side to the decoding side for each area.
[0029]
　By doing so, it is possible to more accurately reproduce a thick point cloud on the object surface. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0030]
　In the third row from the top of the table in FIG. 3 (excluding the row of item names), the present technology described in the second embodiment (Example 2) is described. This method is a method of adding the definition of "absent point" when projecting 3D data onto a two-dimensional plane.
[0031]
　In this case, for example, the definition of the pixel value of the point to be deleted on the decoding side may be signaled from the encoding side to the decoding side.
[0032]
　By doing so, the Sparse point cloud can be reproduced more accurately. That is, it is not necessary to project in units of minute areas (Point), and it is possible to suppress an increase in processing amount and a reduction in encoding efficiency.
[0033]
　In the fourth row from the top of the table of FIG. 3 (excluding the row of item names), the present technology described in the third embodiment (Example 3) is described. This method is a method that enables depth parameters for controlling the depth range of 3D data projected on a two-dimensional plane to be set for each region.
[0034]
　In this case, for example, the depth parameter may be signaled from the encoding side to the decoding side.
[0035]
　By doing so, it is possible to control the image quality for each area, and it is possible to suppress a reduction in the efficiency of position information (Geometry).
[0036]
　　
　Next, a configuration for realizing each of the above methods will be described. FIG. 4 is a block diagram showing an example of the configuration of an encoding device that is one aspect of an image processing device to which the present technology is applied. The coding apparatus 100 shown in FIG. 4 is an apparatus that projects 3D data such as a point cloud onto a two-dimensional plane and performs coding by a coding method for a two-dimensional image.
[0037]
　For example, the encoding device 100 implements the techniques described in Non-Patent Documents 1 to 6 and encodes 3D data by a method that conforms to the standards described in any of those documents. ..
[0038]
　Note that FIG. 4 illustrates main components such as a processing unit and a data flow, and the components illustrated in FIG. 4 are not limited to all. That is, in the encoding device 100, there may be a processing unit not shown as a block in FIG. 4 or a process or data flow not shown as an arrow or the like in FIG. This also applies to other drawings that describe the processing unit and the like in the encoding device 100.
[0039]
　As shown in FIG. 4, the encoding device 100 includes a patch decomposition unit 111, a packing unit 112, an auxiliary patch information compression unit 113, a video encoding unit 114, a video encoding unit 115, an OMap encoding unit 116, and a multiplexer 117. Have.
[0040]
　The patch decomposition unit 111 performs processing related to decomposition of 3D data. For example, the patch decomposition unit 111 acquires 3D data (for example, point cloud) representing a three-dimensional structure input to the encoding device 100 (arrow 121). Moreover, the patch decomposition unit 111 decomposes the acquired 3D data into a plurality of patches, and projects the 3D data for each patch on a two-dimensional plane.
[0041]
　The patch decomposition unit 111 supplies the 3D data projected on the two-dimensional plane for each patch to the packing unit 112 (arrow 122). Further, the patch disassembling unit 111 supplies the auxiliary patch information, which is information related to the disassembly, to the auxiliary patch information compressing unit 113 (arrow 123).
[0042]
　The packing unit 112 performs processing related to data packing. For example, the packing unit 112 acquires data of a two-dimensional plane onto which 3D data is projected for each patch supplied from the patch decomposition unit 111 (arrow 122). Also, the packing unit 112 packs the acquired layers of the two-dimensional plane as different video frames. For example, the packing unit 112 includes position information (Gepmetry) indicating the position of the point, attribute information (Texture) such as color information added to the position information, and an occupancy map (Occupancy Map) indicating the presence or absence of the point. ) Are each packed as a video frame.
[0043]
　The packing unit 112 supplies the generated video frame to the subsequent processing unit (arrow 124). For example, the packing unit 112 supplies the generated video frame of the position information (Geometry) to the video encoding unit 114. Further, for example, the packing unit 112 supplies the generated video frame of the attribute information (Texture) to the video encoding unit 115. Furthermore, for example, the packing unit 112 supplies the generated video frame of the occupancy map to the OMap encoding unit 116.
[0044]
　Further, the packing unit 112 supplies control information regarding the packing to the multiplexer 117 (arrow 125).
[0045]
　The auxiliary patch information compression unit 113 performs processing relating to compression of auxiliary patch information. For example, the auxiliary patch information compression unit 113 acquires the data supplied from the patch decomposition unit 111 (arrow 123). The auxiliary patch information compression unit 113 encodes (compresses) the auxiliary patch information included in the acquired data. The auxiliary patch information compression unit 113 supplies the obtained encoded data of the auxiliary patch information to the multiplexer 117 (arrow 126).
[0046]
　The video encoding unit 114 performs processing regarding encoding of a video frame of position information (Geometry). For example, the video encoding unit 114 acquires the video frame of the position information (Geometry) supplied from the packing unit 112 (arrow 124). In addition, the video encoding unit 114 encodes the acquired video frame of the position information (Geometry) by an arbitrary encoding method for a two-dimensional image such as AVC or HEVC. The video encoding unit 114 supplies the encoded data (encoded data of the video frame of the position information (Geometry)) obtained by the encoding to the multiplexer 117 (arrow 127).
[0047]
　The video encoding unit 115 performs a process related to encoding a video frame of attribute information (Texture). For example, the video encoding unit 115 acquires the video frame of the attribute information (Texture) supplied from the packing unit 112 (arrow 124). Further, the video encoding unit 115 encodes the acquired video frame of the attribute information (Texture) by an arbitrary encoding method for a two-dimensional image such as AVC or HEVC. The video encoding unit 115 supplies the encoded data (encoded data of the video frame of the attribute information (Texture)) obtained by the encoding to the multiplexer 117 (arrow 128).
[0048]
　The OMap encoding unit 116 performs a process related to encoding a video frame of an occupancy map. For example, the OMap encoding unit 116 acquires the occupancy map video frame supplied from the packing unit 112 (arrow 124). Also, the OMap encoding unit 116 encodes the acquired video frame of the occupancy map by an arbitrary encoding method for a two-dimensional image such as AVC or HEVC. The OMap encoding unit 116 supplies the encoded data (encoded data of the video frame of the occupancy map) obtained by the encoding to the multiplexer 117 (arrow 129).
[0049]
　The multiplexer 117 performs processing regarding multiplexing. For example, the multiplexer 117 acquires the encoded data of the auxiliary patch information supplied from the auxiliary patch information compression unit 113 (arrow 126). Further, for example, the multiplexer 117 acquires control information regarding packing supplied from the packing unit 112 (arrow 125). Further, for example, the multiplexer 117 acquires the encoded data of the video frame of the position information (Geometry) supplied from the video encoding unit 114 (arrow 127). Further, for example, the multiplexer 117 acquires the encoded data of the video frame of the attribute information (Texture) supplied from the video encoding unit 115 (arrow 128). Further, for example, the multiplexer 117 acquires the encoded data of the video frame of the occupancy map supplied from the OMap encoding unit 116 (arrow 129).
[0050]
　The multiplexer 117 multiplexes the acquired information to generate a bitstream. The multiplexer 117 outputs the generated bit stream to the outside of the encoding device 100 (arrow 130).
[0051]
　　
　FIG. 5 is a block diagram showing a main configuration example of the patch decomposition unit 111. As shown in FIG. 5, the patch decomposition unit 111 in this case includes a normal direction estimation unit 151, a segmentation initial setting unit 152, a segmentation updating unit 153, a two-dimensional projection unit 154, and a pixel distribution analysis unit 155.
[0052]
　The normal direction estimation unit 151 performs processing related to estimation of the normal direction of the surface of 3D data. For example, the normal direction estimation unit 151 acquires the input 3D data. Further, the normal direction estimating unit 151 estimates the normal direction of the surface of the object represented by the acquired 3D data. For example, the normal direction estimating unit 151 estimates the normal direction by constructing a kd-tree, searching for a neighborhood, and calculating an optimum approximate tangent plane. The normal direction estimation unit 151 supplies the estimation result of the normal direction to the segmentation initial setting unit 152 together with other data.
[0053]
　The segmentation initial setting unit 152 performs processing relating to initial setting of segmentation. For example, the segmentation initial setting unit 152 acquires the data supplied from the normal direction estimation unit 151. In addition, for example, the segmentation initial setting unit 152 determines the surface of the 3D data corresponding to the normal direction based on the components of the six directions of the normal direction estimated by the normal direction estimation unit 151. Classify. The segmentation initial setting unit 152 supplies the classification result to the segmentation updating unit 153 together with other data.
[0054]
　The segmentation updating unit 153 performs processing related to segmentation updating. For example, the segmentation updating unit 153 acquires the data supplied from the segmentation initial setting unit 152. Then, the segmentation updating unit 153 collects the too small areas in the segmentation of the initial setting set by the segmentation initial setting unit 152 so as to be a sufficiently large area. The segmentation updating unit 153 supplies the updated information about the segmentation to the two-dimensional projection unit 154 together with other information.
[0055]
　The two-dimensional projection unit 154 performs processing relating to two-dimensional projection of 3D data. For example, the two-dimensional projection unit 154 acquires the data supplied from the segmentation updating unit 153. In addition, the two-dimensional projection unit 154 uses the pixel distribution analysis unit 155 to generate an occupancy map of 3D data included in the acquired data, and to generate the occupancy data of the 3D data and the occupancy data on a two-dimensional plane for each region. Or project it on. The two-dimensional projection unit 154 supplies the 3D data projected on the two-dimensional plane to the packing unit 112 together with other data.
[0056]
　The pixel distribution analysis unit 155 performs processing related to analysis of the pixel distribution of the 3D data that the two-dimensional projection unit 154 processes.
[0057]
　　
　FIG. 6 is a block diagram showing an example of the configuration of a decoding device which is one mode of the image processing device to which the present technology is applied. The decoding device 200 shown in FIG. 6 decodes coded data obtained by projecting and coding 3D data such as a point cloud on a two-dimensional plane by a decoding method for a two-dimensional image and projecting the data onto a three-dimensional space. It is a device that does.
[0058]
　For example, the decoding device 200 implements the techniques described in Non-Patent Documents 1 to 6 and decodes encoded data of 3D data by a method that conforms to the standard described in any of those documents. I do.
[0059]
　Note that FIG. 6 illustrates main components such as a processing unit and a data flow, and the components illustrated in FIG. 6 are not limited to all. That is, in the decoding device 200, a processing unit not shown as a block in FIG. 6 may exist, or a process or data flow not shown as an arrow or the like in FIG. 6 may exist. This also applies to other drawings that explain the processing unit and the like in the decoding device 200.
[0060]
　As shown in FIG. 6, the decoding device 200 includes a demultiplexer 211, an auxiliary patch information decoding unit 212, a video decoding unit 213, a video decoding unit 214, an OMap decoding unit 215, an unpacking unit 216, and a 3D reconstruction unit 217. Have.
[0061]
　The demultiplexer 211 performs processing relating to data demultiplexing. For example, the demultiplexer 211 acquires the bitstream input to the decoding device 200. This bit stream is supplied from the encoding device 100, for example. The demultiplexer 211 demultiplexes this bit stream, extracts encoded data of auxiliary patch information, and supplies it to the auxiliary patch information decoding unit 212. Further, the demultiplexer 211 extracts the encoded data of the video frame of the position information (Geometory) from the bitstream by demultiplexing and supplies it to the video decoding unit 213. Further, the demultiplexer 211 extracts the encoded data of the video frame of the attribute information (Texture) from the bitstream by demultiplexing and supplies it to the video decoding unit 214. Further, the demultiplexer 211 extracts the encoded data of the occupancy map video frame from the bitstream by demultiplexing and supplies it to the OMap decoding unit 215.
[0062]
　The auxiliary patch information decoding unit 212 performs processing regarding decoding of encoded data of auxiliary patch information. For example, the auxiliary patch information decoding unit 212 acquires the encoded data of the auxiliary patch information supplied from the demultiplexer 211. Further, the auxiliary patch information decoding unit 212 decodes the encoded data of the auxiliary patch information included in the acquired data. The auxiliary patch information decoding unit 212 supplies the auxiliary patch information obtained by the decoding to the 3D reconstruction unit 217.
[0063]
　The video decoding unit 213 performs a process regarding decoding of encoded data of a video frame of position information (Geometory). For example, the video decoding unit 213 acquires encoded data of a video frame of position information (Geometory) supplied from the demultiplexer 211. In addition, for example, the video decoding unit 213 decodes the acquired encoded data to obtain a video frame of position information (Geometory). The video decoding unit 213 supplies the video frame of the position information (Geometory) to the unpacking unit 216.
[0064]
　The video decoding unit 214 performs a process regarding decoding of encoded data of a video frame of attribute information (Texture). For example, the video decoding unit 214 acquires the encoded data of the video frame of the attribute information (Texture) supplied from the demultiplexer 211. In addition, for example, the video decoding unit 214 decodes the acquired encoded data to obtain a video frame of attribute information (Texture). The video decoding unit 214 supplies the video frame of the attribute information (Texture) to the unpacking unit 216.
[0065]
　The OMap decoding unit 215 performs processing regarding decoding of encoded data of video frames of the occupancy map. For example, the OMap decoding unit 215 acquires the encoded data of the video frame of the occupancy map supplied from the demultiplexer 211. Also, for example, the OMap decoding unit 215 decodes the acquired encoded data to obtain a video frame of the occupancy map. The OMap decoding unit 215 supplies the video frame of the occupancy map to the unpacking unit 216.
[0066]
　The unpacking unit 216 performs processing related to unpacking. For example, the unpacking unit 216 acquires a video frame of position information (Geometory) from the video decoding unit 213, acquires a video frame of attribute information (Texture) from the video decoding unit 214, and acquires an occupancy map from the OMap decoding unit 215. Get the video frame of. The unpacking unit 216 unpacks these video frames. The unpacking unit 216 supplies the position information (Geometory) data, the attribute information (Texture) data, and the occupancy map data obtained by the unpacking to the 3D reconstruction unit 217.
[0067]
　The 3D reconstruction unit 217 performs processing relating to reconstruction of 3D data. For example, the 3D reconstruction unit 217 includes auxiliary patch information supplied from the auxiliary patch information decoding unit 212, position information (Geometory) data, attribute information (Texture) data, and occluded data supplied from the unpacking unit 216. The 3D data is reconstructed based on the Pancy map data and the like. The 3D reconstruction unit 217 outputs the 3D data obtained by such processing to the outside of the decoding device 200.
[0068]
　The 3D data is supplied to the display unit to display its image, recorded on a recording medium, or supplied to another device via communication, for example.
[0069]
　　<3D Reconstruction Unit>
　FIG. 7 is a block diagram showing a main configuration example of the 3D reconstruction unit 217 of FIG. As shown in FIG. 7, the 3D reconstruction unit 217 includes a three-dimensional projection unit 251, a pixel distribution analysis unit 252, an inverse segmentation updating unit 253, an inverse segmentation initial setting unit 254, and an inverse normal direction estimation unit 255. ..
[0070]
　The three-dimensional projection unit 251 projects the 3D data projected on the two-dimensional plane for each area onto the three-dimensional space. The pixel distribution analysis unit 252 performs processing such as analysis of pixel distribution when the three-dimensional projection unit 251 projects a three-dimensional space.
[0071]
　The inverse segmentation updating unit 253 performs the inverse process of the segmentation updating unit 153. The inverse segmentation initial setting unit 254 performs the reverse process of the segmentation initial setting unit 152. The inverse normal direction estimation unit 255 performs the inverse process of the normal direction estimation unit 151.
[0072]
　<2. First embodiment>
　　 In the
　conventional method, 3D data is composed of two layers (layer 0 and layer 1) as in the example shown in FIG. Was projected on the two-dimensional plane.
[0073]
　On the layer 0 (Layer 0), the data of the points on the surface is projected as viewed from the projection surface of the 3D data. On the layer 1 (Layer 1), the data of the point farthest from the layer 0 within a predetermined threshold value (Default Th = 4Voxel) is projected. Those separated by a predetermined threshold value Th or more are dropped. In Layer 1, the difference value of the distance from Layer 0 is the pixel value.
[0074]
　In this way, the information of other points is lost (cannot be reproduced) because the algorithm expresses the object surface with only two fixed layers. Therefore, there is a possibility that a point that cannot be projected on the two-dimensional plane is generated and the quality of the data is reduced by the encoding accompanied by the projection on the two-dimensional plane.
[0075]
　Therefore, the number of layers on the two-dimensional plane onto which the 3D data is projected may be variable. For example, the data for every position included in the 3D data representing the three-dimensional structure is projected on the two-dimensional plane of a plurality of layers. For example, the image processing apparatus is provided with a two-dimensional projection unit that projects data for all positions included in the 3D data representing the three-dimensional structure onto a two-dimensional plane of a plurality of layers.
[0076]
　For example, in the case of FIG. 9, two-dimensional planes of layers 0 (Layer 0) to 3 (Layer 3) (two-dimensional planes of more than two layers) are set, and 3D data is projected on these layers. By doing so, it is possible to more accurately reproduce a thick point cloud on the object surface. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0077]
　For example, the two-dimensional projection unit 154 may project the data of the 3D data for each position where the positions overlap in the depth direction when viewed from the projection surface, on different layers of the two-dimensional plane of a plurality of layers. ..
[0078]
　In the case of the example in FIG. 9, the data overlapping in the depth direction is projected on different layers of layers 0 to 3. By doing so, it is possible to project all data of the 3D data for each position where the positions overlap in the depth direction when viewed from the projection surface, onto the two-dimensional plane. That is, the loss of information can be suppressed. Therefore, it is possible to suppress the quality reduction due to the two-dimensional projection of the 3D data.
[0079]
　Further, for example, the two-dimensional projection unit 154 may generate the same number of layers as the maximum number of data for each position where the positions overlap with each other in the depth direction when viewed from the projection surface of the 3D data on the two-dimensional plane. ..
[0080]
　In the case of the example in FIG. 9, in the 3D data of the area (Local Bounding Box), the maximum number of pieces of data that overlap in the depth direction is 4. Therefore, this 3D data is projected onto the two-dimensional plane of four layers (layer 0 to layer 3).
[0081]
　By doing so, the data for every position of the 3D data can be projected on the two-dimensional plane. Therefore, since it is possible to suppress the loss of information, it is possible to suppress the quality reduction due to the two-dimensional projection of the 3D data.
[0082]
　In this case, the information indicating the number of layers of the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit 154 may be signaled to the bitstream. That is, the multiplexer 117 that functions as a bitstream generation unit encodes the two-dimensional plane by the video encoding unit 114 and the information indicating the number of layers of the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit 154. A bitstream including the obtained encoded data is generated.
[0083]
　By doing so, by referring to the information indicating the number of layers of the two-dimensional plane on the decoding side, it is possible to easily project the 3D data projected on all the layers of the two-dimensional plane into the three-dimensional space. be able to.
[0084]
　　
　An example of the encoding process flow executed by the encoding device 100 will be described with reference to the flowchart in FIG. 10.
[0085]
　When the encoding process is started, the patch decomposition unit 111 of the encoding device 100 decomposes 3D data into patches and projects the data of each patch onto a two-dimensional plane in step S101. In step S102, the auxiliary patch information compression unit 113 compresses the auxiliary patch information obtained by the process of step S101.
[0086]
　In step S103, the packing unit 112 packs the 3D data projected on the two-dimensional plane for each patch by the patch decomposition unit 111 as a video frame. In step S104, the video encoding unit 114 encodes the geometry video frame, which is the video frame of the position information obtained by the process of step S103, by the encoding method for a two-dimensional image.
[0087]
　In step S105, the video encoding unit 114 encodes the color video frame, which is the video frame of the attribute information obtained by the process of step S103, by the encoding method for a two-dimensional image. In step S106, the video encoding unit 114 encodes the occupancy map obtained by the process of step S103 by the encoding method for a two-dimensional image.
[0088]
　In step S107, the multiplexer 117 multiplexes the various types of information generated as described above, and generates a bitstream including these types of information.
[0089]
　In step S108, the multiplexer 117 outputs the bitstream generated by the process of step S107 to the outside of the encoding device 100.
[0090]
　When the process of step S108 ends, the encoding process ends.
[0091]
　　
　Next, an example of the flow of the patch disassembly processing executed in step S101 of FIG. 10 will be described with reference to the flowchart of FIG.
[0092]
　When the patch decomposition process is started, the normal direction estimation unit 151 estimates the normal direction in step S121. In step S122, the segmentation initial setting unit 152 performs initial setting of segmentation. In step S123, the segmentation updating unit 153 updates the segmentation in the initial state set in step S122 as needed. In step S124, the two-dimensional projection unit 154 projects the 3D data on the two-dimensional plane.
[0093]
　When the process of step S124 ends, the patch decomposition process ends, and the process returns to FIG.
[0094]
　　
　Next, an example of the flow of the two-dimensional projection processing executed in step S124 of FIG. 11 will be described with reference to the flowchart of FIG.
[0095]
　When the two-dimensional projection process is started, the two-dimensional projection unit 154 performs area extraction by segmentation in step S141. In step S142, the two-dimensional projection unit 154 initializes the layer number i=0.
[0096]
　In step S143, the pixel distribution analysis unit 155 determines whether or not there is an unprojected pixel (data for each position of 3D data that is not projected on the two-dimensional plane). If it is determined that there are unprojected pixels, the process proceeds to step S144.
[0097]
　In step S144, the two-dimensional projection unit 154 projects the area on the layer i (Layer i) to be processed. In step S145, the two-dimensional projection unit 154 increments the variable i (i++). When the process of step S145 ends, the process returns to step S143, and the subsequent processes are repeated.
[0098]
　If it is determined in step S143 that there are no unprojected pixels (all pixels in the area have been projected), the process proceeds to step S146.
[0099]
　In step S146, the two-dimensional projection unit 154 supplies information indicating the number i of layers and encodes the information. Further, in step S147, the two-dimensional projection unit 154 encodes the corresponding i-frame geometry image. That is, the two-dimensional projection unit 154 supplies the two-dimensional plane onto which the 3D data is projected to the packing unit 112, and packs the respective layers as different frames.
[0100]
　When the process of step S147 ends, the two-dimensional projection process ends, and the process returns to FIG. 11.
[0101]
　By executing each process as described above, it is possible to more accurately reproduce a point cloud having a thick surface on the object. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0102]
　　 On the
　decoding side, by using the information indicating the number of layers of the two-dimensional plane provided from the encoding side, the number of layers as described above is used. It is possible to realize the reconstruction of 3D data projected on a two-dimensional plane with variable V.
[0103]
　That is, the data for every position of the 3D data projected onto the two-dimensional plane having the number of layers indicated by the number-of-layers information is projected onto the three-dimensional space. For example, the image processing apparatus is provided with a three-dimensional projection unit that projects data at all positions of the 3D data projected onto the two-dimensional plane having the number of layers indicated by the number-of-layers information, into a three-dimensional space.
[0104]
　By doing so, reconstruction of 3D data projected on a two-dimensional plane with a variable number of layers can be realized. That is, it is possible to more accurately reproduce a thick point cloud on the object surface. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0105]
　　
　An example of the flow of the decoding process executed by the decoding device 200 will be described with reference to the flowchart of FIG.
[0106]
　When the decoding process is started, the demultiplexer 211 of the decoding device 200 demultiplexes the bitstream in step S201.
[0107]
　In step S202, the auxiliary patch information decoding unit 212 decodes the auxiliary patch information extracted from the bitstream by the process of step S201. In step S203, the video decoding unit 213 decodes the encoded data of the geometry video frame (video frame of position information) extracted from the bitstream by the process of step S201.
[0108]
　In step S204, the video decoding unit 214 decodes the encoded data of the color video frame (video frame of attribute information) extracted from the bitstream by the process of step S201. In step S205, the OMap decoding unit 215 decodes the encoded data of the occupancy map extracted from the bitstream by the process of step S201.
[0109]
　In step S206, the unpacking unit 216 unpacks the geometry video frame, the color video frame, and the occupancy map decoded in steps S203 to S205.
[0110]
　In step S207, the 3D reconstruction unit 217 reconstructs 3D data such as a point cloud based on the auxiliary patch information obtained in step S202 and the various information obtained in step S206.
[0111]
　When the process of step S207 ends, the decoding process ends.
[0112]
　　
　Next, an example of the flow of the point cloud reconstruction processing executed in step S207 of FIG. 13 will be described with reference to the flowchart of FIG.
[0113]
　When the point cloud reconstruction processing is started, the three-dimensional projection unit 251 projects the two-dimensional image in the three-dimensional space in step S221.
[0114]
　In step S222, the inverse segmentation updating unit 253 updates the segmentation in the opposite direction and divides the combined segmentation.
[0115]
　In step S223, the inverse segmentation initial setting unit 254 performs an inverse process of the initial setting of segmentation, and collects the classified points.
[0116]
　In step S224, the inverse normal direction estimation unit 255 performs the inverse process of the normal direction estimation to reconstruct the point cloud.
[0117]
　When the process of step S224 ends, the point cloud reconstruction process ends, and the process returns to FIG. 13.
[0118]
　　
　Next, an example of the flow of 3D projection processing executed in step S221 of FIG. 14 will be described with reference to the flowchart of FIG.
[0119]
　When the three-dimensional projection process is started, the three-dimensional projection unit 251 decodes the information indicating the number of layers i in step S241.
[0120]
　In step S242, the three-dimensional projection unit 251 decodes the corresponding i-frame geometry image.
[0121]
　In step S243, the pixel distribution analysis unit 252 initializes the variable k=0.
[0122]
　In step S244, the pixel distribution analysis unit 252 determines whether or not the variable k
　　 When sparse (3D) data is projected onto a two-dimensional plane, pixels on which the 3D data is not projected, that is, a pixel value There may be a pixel in which is not set (also referred to as a non-existing pixel).
[0129]
　In the conventional method, when the 3D data is sparse, as shown in FIG. 16, the pixel value is complemented to such “absent pixel”. For example, in the case of FIG. 16, the pixel value is copied (pixel complementation is performed) from the pixel on the left of the “non-pixel” of layer 0 (Layer0). The reason for performing such complementing processing is that there is no concept that data does not exist (blank) in a two-dimensional image encoding method (for example, AVC or HEVC).
[0130]
　However, if such complementary processing is performed, points that do not originally exist will be added to the 3D data. Therefore, the 3D data may be deteriorated. That is, there is a fear that the quality of the 3D data due to the two-dimensional projection is reduced.
[0131]
　If such a complementary process is not performed, it is necessary to set the area so that "absent pixels" do not occur on the two-dimensional plane onto which the 3D data is projected. Therefore, projection in units of minute areas (Point) is required, which may increase the amount of processing or reduce the coding efficiency.
[0132]
　Therefore, the data for each position included in the 3D data representing the three-dimensional structure is projected onto the two-dimensional plane, and it is shown that the data for each position does not exist at the position on the two-dimensional plane where the data for each position does not exist. Make sure to set a predetermined value. For example, in the image processing apparatus, the data for each position included in the 3D data representing the three-dimensional structure is projected onto a two-dimensional plane, and the data for each position is stored at a position where the data for each position on the two-dimensional plane does not exist. A two-dimensional projection unit that sets a predetermined value indicating that it does not exist is provided.
[0133]
　For example, as shown in FIG. 17, a prescribed value X is used as a pixel value indicating “absent pixel” on a two-dimensional plane onto which 3D data is projected. The value of X may be, for example, a predetermined fixed value. For example, X=255 (upper limit of 8 bits) may be set.
[0134]
　In this case, it is necessary to signal the value of X in the bitstream (it is necessary to notify the decoding side).
[0135]
　Further, for example, as the value of X, an arbitrary value that satisfies X>D may be used. Here, D represents the maximum value of the depth of the bounding box. A pixel value on the two-dimensional plane cannot be set to a value equal to or larger than the maximum value of the depth of the bounding box. Therefore, the unused value may be used as X.
[0136]
　In this case, on the decoding side, it is possible to determine whether or not the pixel value set on the two-dimensional plane is an unused value from the information on the size of the bounding box. Therefore, it is not necessary to signal this value of X to the bitstream (ie notify the decoder of this value of X). However, of course, the value of X may be notified to the decoding side.
[0137]
　By doing so, the encoding apparatus 100 can represent “non-existing pixels” without requiring data complementation, and thus can suppress deterioration of 3D data. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0138]
　Further, by doing so, the coding apparatus 100 can reduce the number of “pixels that cannot be coded” in the two-dimensional plane, and thus the area is set to be larger than that in the case where no complementary processing is simply performed. be able to. Therefore, it is not necessary to project in units of minute areas (Point), and it is possible to suppress an increase in processing amount and a reduction in encoding efficiency.
[0139]
　　 In
　this case as well, the encoding processing and the patch decomposition processing are performed in the same manner as the case described in the first embodiment. Therefore, their description will be omitted.
[0140]
　An example of the flow of the two-dimensional projection process in this case executed in step S124 of FIG. 11 will be described with reference to the flowchart of FIG.
[0141]
　When the two-dimensional projection process is started, the two-dimensional projection unit 154 performs area extraction by segmentation in step S301. In step S302, the pixel distribution analysis unit 155 determines whether or not the area includes a sparse point cloud. When it is determined that the area includes the sparse point cloud, the process proceeds to step S303.
[0142]
　In step S303, the two-dimensional projection unit 154 signals information that defines “absent pixel” for the region. That is, the above-mentioned X is set and the X is used to represent the “non-existing point” projected on the two-dimensional plane. When the process of step S303 ends, the process proceeds to step S305.
[0143]
　If it is determined in step S302 that the area does not include a sparse point cloud, the process proceeds to step S304.
[0144]
　In step S304, the two-dimensional projection unit 154 projects the area on each layer of the two-dimensional plane. When the process of step S304 ends, the process proceeds to step S305.
[0145]
　In step S305, the two-dimensional projection unit 154 encodes the geometry image of the area.
[0146]
　When the process of step S305 ends, the two-dimensional projection process ends, and the process returns to FIG. 11.
[0147]
　As described above, by performing the two-dimensional projection process, the encoding device 100 can express the “absent pixel” without the need for data complement. Therefore, deterioration of 3D data can be suppressed. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0148]
　Further, by doing so, the coding apparatus 100 can reduce the number of “pixels that cannot be coded” in the two-dimensional plane, and thus the area is set to be larger than that in the case where no complementary processing is simply performed. be able to. Therefore, it is not necessary to project in units of minute areas (Point), and it is possible to suppress an increase in processing amount and a reduction in encoding efficiency.
[0149]
　　 On the
　decoding side, when projecting 3D data projected on a two-dimensional plane onto a three-dimensional space, on the two-dimensional plane signaled as described above The pixel value (X described above) indicating the “absent pixel” of is detected and deleted (not projected).
[0150]
　That is, out of the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane, data other than the data of the predetermined value indicating that the data for each position does not exist in the three-dimensional space. To be projected on. For example, in the image processing apparatus, data other than data having a predetermined value indicating that there is no data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane Is provided in a three-dimensional space.
[0151]
　By doing so, “non-existing pixels” can be expressed without the need for data complementation, and therefore deterioration of 3D data can be suppressed. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0152]
　Further, by doing so, the number of “pixels that cannot be coded” in the two-dimensional plane can be reduced, and thus the area can be set larger than in the case where the complementary processing is not simply performed. Therefore, it is not necessary to project in units of minute areas (Point), and it is possible to suppress an increase in processing amount and a reduction in encoding efficiency.
[0153]
　　 Also in
　this case, the decoding processing and the point cloud reconstruction processing are performed in the same manner as the case described in the first embodiment. Therefore, their description will be omitted.
[0154]
　An example of the flow of the three-dimensional projection process in this case, which is executed in step S221 of FIG. 14, will be described with reference to the flowchart of FIG.
[0155]
　When the three-dimensional projection process is started, the three-dimensional projection unit 251 decodes the geometry image of the processing target area in step S321.
[0156]
　In step S322, the pixel distribution analysis unit 252 determines whether or not the geometry image includes information (a pixel value indicating “absent pixel”) that defines “absent pixel”. If it is determined that there is one, the process proceeds to step S323.
[0157]
　In step S323, the three-dimensional projection unit 251 deletes the pixel having the pixel value indicating the detected "absent pixel" (set not to project in the three-dimensional space). When the process of step S323 ends, the process proceeds to step S324.
[0158]
　If it is determined in step S322 that the pixel value indicating “non-existing pixel” does not exist, the process of step S323 is omitted and the process proceeds to step S324.
[0159]
　That is, the pixel value indicating “absent pixel” is detected in the geometry image of the processing target region, and if detected, the pixel is deleted.
[0160]
　In step S324, the three-dimensional projection unit 251 projects the geometry image of the processing target area onto the three-dimensional space.
[0161]
　When the process of step S324 ends, the three-dimensional projection process ends, and the process returns to FIG.
[0162]
　By performing the three-dimensional projection process as described above, the decoding device 200 can represent “non-existing pixels” without requiring data complementation, and thus can suppress deterioration of 3D data. .. That is, it is possible to suppress the reduction in quality due to the two-dimensional projection of 3D data.
[0163]
　Further, by doing so, the decoding device 200 can reduce the number of “pixels that cannot be coded” in the two-dimensional plane, and therefore, the region should be set to be larger than in the case where no complementing process is performed. You can Therefore, it is not necessary to project in units of minute areas (Point), and it is possible to suppress an increase in processing amount and a reduction in encoding efficiency.
[0164]
　<4. Third Embodiment>
　　
　When projecting 3D data onto a two-dimensional plane, a depth parameter th that controls a range in the depth direction of the 3D data projected onto the two-dimensional plane is used. Since the points within the range specified by the depth parameter th are projection targets on the two-dimensional plane, the value of the depth parameter th is related to the length of the region (Local bounding box) in the depth direction. For example, when the value of the depth parameter th is larger than the length of the area in the depth direction, points in other areas can also be projection targets. That is, the length of the area in the depth direction needs to be longer than the depth parameter th.
[0165]
　In the conventional method, as shown in FIG. 20, this depth parameter th is controlled in frame units. Therefore, all depth parameters th in the frame have a common value.
[0166]
　However, there may be cases where the surface density of the point cloud is not constant in the frame, for example, the face and feet, the central portion and the peripheral portion. Therefore, the value of the depth parameter th is not always optimum, and the coding efficiency may be reduced. For example, a case may occur in which the area is divided in the depth direction and the coding efficiency is better, but the value of the depth parameter th is large and the area cannot be divided.
[0167]
　Therefore, when the data for each position of the 3D data representing the three-dimensional structure is projected on the two-dimensional plane for each predetermined area of ​​the three-dimensional space, a three-dimensional structure that can be projected on one layer set for each area is set. The data for each position within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction for each position of the represented 3D data is projected on the two-dimensional plane. For example, in the image processing apparatus, when the data for each position of the 3D data representing the three-dimensional structure is projected on the two-dimensional plane for each predetermined area of ​​the three-dimensional space, it is possible to project on one layer set for each area. A two-dimensional projection unit that projects the data for each position within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction for the data for each position of 3D data that represents a different three-dimensional structure To do so.
[0168]
　For example, as shown in FIG. 21, the depth parameter is expanded so that it can be transmitted for each patch, and the position (TH) of the pixel to be projected on the layer is transmitted for each area. By doing so, for example, the depth parameter th can be set for the area 1 and the depth parameter th′ can be set for the area 2. Therefore, the efficiency of layer 1 (Geometry coding efficiency) is improved.
[0169]
　　 In
　this case as well, the encoding processing and the patch decomposition processing are performed in the same manner as the case described in the first embodiment. Therefore, their description will be omitted.
[0170]
　An example of the flow of the two-dimensional projection process in this case executed in step S124 of FIG. 11 will be described with reference to the flowchart of FIG.
[0171]
　When the two-dimensional projection process is started, the pixel distribution analysis unit 155 initializes the region number i=0 in step S401.
[0172]
　In step S402, the pixel distribution analysis unit 155 determines whether or not there are unprocessed pixels. If it is determined that there are unprocessed pixels, the process proceeds to step S403.
[0173]
　In step S403, the two-dimensional projection unit 154 extracts the area i by segmentation.
[0174]
　In step S404, the two-dimensional projection unit 154 performs depth parameter adjustment and encoding (RD determination). That is, the optimum depth parameter th and the range of the area are set by the RD determination.
[0175]
　In step S405, the two-dimensional projection unit 154 projects the geometry image of the processing target region i on the two-dimensional surface based on the settings made in step S404.
[0176]
　In step S406, the two-dimensional projection unit 154 encodes the geometry image projected on the two-dimensional plane.
[0177]
　In step S407, the pixel distribution analysis unit 155 increments the variable i (i++). When the process of step S407 ends, the process returns to step S402, and the subsequent processes are repeated.
[0178]
　If it is determined in step S402 that there are no unprocessed pixels, the process proceeds to step S408.
[0179]
　In step S408, the two-dimensional projection unit 154 encodes an occlusion map. When the process of step S408 ends, the two-dimensional projection process ends, and the process returns to FIG. 11.
[0180]
　By performing the two-dimensional projection process as described above, the encoding device 100 can obtain the depth parameter th more suitable for each region, and can suppress the reduction in encoding efficiency.
[0181]
　　 On the
　decoding side, when the 3D data projected on the two-dimensional plane is projected on the three-dimensional space, the 3D data is within the range indicated by the depth parameter th controlled as described above. To project.
[0182]
　That is, when the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane is projected into the three-dimensional space for each predetermined region of the three-dimensional space, the data for each position is , So as to project within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction of the data for each position of 3D data that can be projected on one layer set for each region in the three-dimensional space. To do. For example, in the image processing apparatus, when the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane is projected into the three-dimensional space for each predetermined area of ​​the three-dimensional space, Within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction of the data for each position of the 3D data that can project the data for each position in one layer set for each region in the three-dimensional space A three-dimensional projection unit for projecting onto
[0183]
　By doing so, it is possible to project the 3D data onto the three-dimensional space by using the depth parameter th set for each area, and thus it is possible to suppress the reduction in encoding efficiency.
[0184]
　　 Also in
　this case, the decoding processing and the point cloud reconstruction processing are performed in the same manner as the case described in the first embodiment. Therefore, their description will be omitted.
[0185]
　An example of the flow of the three-dimensional projection process in this case executed in step S221 of FIG. 14 will be described with reference to the flowchart of FIG.
[0186]
　When the three-dimensional projection process is started, the three-dimensional projection unit 251 decodes the Occlusion map in step S421.
[0187]
　In step S422, the pixel distribution analysis unit 252 initializes the area number k=0.
[0188]
　In step S423, the pixel distribution analysis unit 252 determines whether or not the variable k
　　 The control information
　regarding the present technology described in each of the above embodiments may be transmitted from the encoding side to the decoding side. For example, control information (for example, enabled_flag) that controls whether to permit (or prohibit) application of the above-described present technology may be transmitted. Further, for example, control for designating a range (for example, an upper limit or a lower limit of a block size, or both, a slice, a picture, a sequence, a component, a view, a layer, etc.) that permits (or prohibits) the application of the present technology described above. Information may be transmitted.
[0194]
　　 The
　series of processes described above can be executed by hardware or software. When the series of processes is executed by software, a program forming the software is installed in the computer. Here, the computer includes a computer incorporated in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.
[0195]
　FIG. 24 is a block diagram showing a configuration example of hardware of a computer that executes the series of processes described above by a program.
[0196]
　In a computer 900 shown in FIG. 24, a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 are connected to each other via a bus 904.
[0197]
　An input/output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input/output interface 910.
[0198]
　The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal and the like. The output unit 912 includes, for example, a display, a speaker, an output terminal and the like. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, or the like. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0199]
　In the computer configured as above, for example, the CPU 901 loads the program stored in the storage unit 913 into the RAM 903 via the input/output interface 910 and the bus 904 and executes the program to execute the above-described series of operations. Is processed. The RAM 903 also appropriately stores data necessary for the CPU 901 to execute various processes.
[0200]
　The program executed by the computer (CPU 901) can be applied by being recorded in the removable medium 921 such as a package medium, for example. In this case, the program can be installed in the storage unit 913 via the input/output interface 910 by mounting the removable medium 921 in the drive 915.
[0201]
　The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting. In that case, the program can be received by the communication unit 914 and installed in the storage unit 913.
[0202]
　In addition, this program can be installed in the ROM 902 or the storage unit 913 in advance.
[0203]
　　 In the
　above, the case where this technology is applied to Voxel conversion of point cloud data has been described, but this technology is not limited to these examples, and can be applied to Voxel conversion of 3D data of any standard. Can be applied to That is, the specifications of various processes such as encoding/decoding methods and various data such as 3D data and metadata are arbitrary as long as they do not conflict with the present technology described above. Further, as long as it does not conflict with the present technology, some of the processes and specifications described above may be omitted.
[0204]
　In the above, the encoding device 100 and the decoding device 200 have been described as application examples of the present technology, but the present technology can be applied to any configuration.
[0205]
　For example, the present technology is applied to a transmitter or a receiver (for example, a television receiver or a mobile phone) in satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication, or It can be applied to various electronic devices such as a device (for example, a hard disk recorder or a camera) that records an image on a medium such as an optical disc, a magnetic disc, and a flash memory, or reproduces an image from these storage media.
[0206]
　In addition, for example, the present technology includes a processor (for example, a video processor) as a system LSI (Large Scale Integration) or the like, a module (for example, a video module) that uses a plurality of processors or the like, or a unit (for example, a video unit) that uses a plurality of modules or the like. Alternatively, it can be implemented as a part of the configuration of a device such as a set (for example, a video set) in which a unit is added with other functions.
[0207]
　Further, for example, the present technology can also be applied to a network system including a plurality of devices. For example, the present technology may be implemented as cloud computing in which a plurality of devices share and jointly process via a network. For example, this technology is implemented in a cloud service that provides services related to images (moving images) to arbitrary terminals such as computers, AV (Audio Visual) devices, portable information processing terminals, and IoT (Internet of Things) devices. You may do it.
[0208]
　In the present specification, the system means a set of a plurality of constituent elements (devices, modules (components), etc.), and it does not matter whether or not all the constituent elements are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and one device housing a plurality of modules in one housing are all systems. ..
[0209]
　　 Systems, devices, processing units, etc. to which this technology is applied include, for example, transportation, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, home appliances, weather, and nature monitoring. Etc. can be used in any field. Further, its use is also arbitrary.
[0210]
　For example, the present technology can be applied to a system or device provided for providing ornamental content and the like. Further, for example, the present technology can be applied to a system or device used for traffic such as traffic condition supervision and automatic driving control. Furthermore, for example, the present technology can be applied to a system or device used for security. Further, for example, the present technology can be applied to a system or device used for automatic control of a machine or the like. Furthermore, for example, the present technology can be applied to systems and devices used for agriculture and livestock. Further, the present technology can also be applied to a system or device for monitoring natural conditions such as volcanoes, forests, and oceans, and wildlife. Furthermore, for example, the present technology can be applied to a system or device used for sports.
[0211]
　　 In
　this specification, the “flag” is information for identifying a plurality of states, and not only information used to identify two states of true (1) or false (0), Information that can identify three or more states is also included. Therefore, the value that the “flag” can take may be, for example, a binary value of 1/0, or a ternary value or more. That is, the number of bits forming this "flag" is arbitrary and may be 1 bit or multiple bits. Further, since the identification information (including the flag) is not limited to the form in which the identification information is included in the bitstream, the form in which the difference information of the identification information with respect to certain reference information is included in the bitstream is also assumed. In the above, "flag" and "identification information" include not only that information but also difference information with respect to reference information.
[0212]
　Further, various types of information (metadata, etc.) regarding the encoded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the encoded data. Here, the term "associate" means that, for example, when processing one data, the other data can be used (linked). That is, the data associated with each other may be collected as one data or may be individual data. For example, the information associated with the encoded data (image) may be transmitted on a transmission path different from that of the encoded data (image). Further, for example, the information associated with the encoded data (image) may be recorded in a recording medium (or another recording area of ​​the same recording medium) different from that of the encoded data (image). Good. Note that this "association" may be a part of the data instead of the entire data. For example, the image and the information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part of the frame.
[0213]
　In this specification, “composite”, “multiplex”, “add”, “integrate”, “include”, “store”, “insert”, “insert”, “insert”. A term such as “” means to combine a plurality of objects into one, for example, to combine encoded data and metadata into one data, and means one method of “associating” described above.
[0214]
　The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present technology.
[0215]
　For example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). On the contrary, the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Further, it is of course possible to add a configuration other than the above to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or another processing unit). ..
[0216]
　Further, for example, the program described above may be executed in any device. In that case, the device may have a necessary function (function block or the like) so that necessary information can be obtained.
[0217]
　Further, for example, each device may execute each step of one flowchart, or a plurality of devices may share and execute the steps. Further, when one step includes a plurality of processes, one device may execute the plurality of processes, or a plurality of devices may share the processes. In other words, a plurality of processes included in one step can be executed as a process of a plurality of steps. On the contrary, the processes described as a plurality of steps can be collectively executed as one step.
[0218]
　Further, for example, in the program executed by the computer, the processing of the steps for writing the program may be executed in a time series in the order described in this specification, or in parallel or by calling. It may be executed individually at a necessary timing such as when it is released. That is, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the process of the step of writing this program may be executed in parallel with the process of another program, or may be executed in combination with the process of another program.
[0219]
　Further, for example, a plurality of techniques related to the present technique can be independently implemented as a single unit unless a contradiction occurs. Of course, a plurality of arbitrary present techniques can be used in combination. For example, part or all of the present technology described in any of the embodiments can be combined with part or all of the present technology described in other embodiments. Further, a part or all of the present technology described above may be implemented in combination with other technology not described above.
[0220]
　Note that the present technology may also be configured as below.
　(1) An
　image processing apparatus including a two-dimensional projection unit that projects data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers .
　(2) The two-dimensional projection unit projects data of the 3D data at each position where the positions overlap in the depth direction when viewed from the projection surface, on different layers of the two-dimensional planes of the plurality of layers.
　The image processing device according to (1).
　(3) the two-dimensional projection unit, for the two-dimensional plane, the position viewed from the projection plane of the 3D data to generate the maximum number as many layers of data for each of the position overlapping in the depth direction
　(2) The image processing device described.
　(4)
　The image processing device according to any one of (1) to (3), further including a coding unit that codes the 3D data projected on the two-dimensional plane by the two-dimensional projection unit .
　(5)
　The image processing device according to (4), wherein the encoding unit encodes the position information, the attribute information, and the occupancy map of the 3D data projected on each layer of the two-dimensional plane .
　(6) Information indicating the number of layers of the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit, and encoded data obtained by encoding the two-dimensional plane by the encoding unit
　The image processing device according to (4) or (5), further including a bitstream generation unit that generates a bitstream including the bitstream .
　(7) A packing unit that packs the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit as a video frame is further provided, and the
　encoding unit packs the two-dimensional plane by the packing unit.
　The image processing device according to any one of (4) to (6), which is configured to encode the video frame .
　(8)
　The image processing device according to any one of (1) to (7), wherein the two-dimensional projection unit projects the 3D data on the two-dimensional plane for each predetermined area .
　(9)
　The image processing device according to any one of (1) to (8) , wherein the 3D data is a point cloud .
　(10) An
　image processing method of projecting data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers .
[0221]
　(11) An
　image processing apparatus including a three-dimensional projection unit that projects data at all positions of 3D data projected onto a two-dimensional plane having the number of layers indicated by the number-of-layers information, into a three-dimensional space .
　(12) An extraction unit that extracts the number-of-layers information included in the bitstream is further provided, and the
　three-dimensional projection unit projects onto the two-dimensional plane having the number of layers indicated by the number-of-layers information extracted by the extraction unit.
　The image processing device according to (11), configured to project all the data for each of the positions of the generated 3D data into the three-dimensional space .
　(13) A decoding unit that decodes encoded data of the 3D data projected onto the two-dimensional plane included in the bitstream is further included, and the
　three-dimensional projection unit decodes the encoded data by the decoding unit.
　The image processing device according to (12), which is configured to project all the data for each position of the 3D data projected on the two-dimensional plane obtained in this way into the three-dimensional space .
　(14) the decryption unit, of the 3D data projected on each layer of the two-dimensional plane, the position information, attribute information, and decodes the respective encoded data occupancy map
　image according to (13) Processing equipment.
　(15) An unpacking unit that unpacks a video frame packed with the 3D data projected on the two-dimensional plane, which is obtained by decoding the encoded data by the decoding unit,
　The three-dimensional projection unit projects, on the three-dimensional space, all the data for each position of the 3D data projected on the two-dimensional plane, which is obtained by unpacking the video frame by the unpacking unit.
　The image processing device according to (13) or (14) configured to :
　(16)
　The image processing according to any one of (11) to (15), wherein the three-dimensional projection unit projects the 3D data projected on the two-dimensional plane for each predetermined region into the three-dimensional space. apparatus.
　(17)
　The image processing device according to any of (11) to (16) , wherein the 3D data is a point cloud .
　(18) An
　image processing method of projecting data for every position of 3D data projected onto a two-dimensional plane having the number of layers indicated by the number-of-layers information onto a three-dimensional space .
[0222]
　(21) The data for each position included in the 3D data representing the three-dimensional structure is projected onto a two-dimensional plane, and the data for each position does not exist at a position on the two-dimensional plane where the data for each position does not exist. An
　image processing apparatus including a two-dimensional projection unit that sets a predetermined value indicating
　(22)
　The image processing device according to (21), wherein the predetermined value is a predetermined fixed value .
　(23)
　The image processing device according to (21), wherein the predetermined value is a value larger than a maximum depth value of the 3D data .
　(24)
　The image processing device according to any one of (21) to (23), further including an encoding unit that encodes the 3D data projected on the two-dimensional plane by the two-dimensional projection unit .
　(25)
　The image processing device according to (24), wherein the encoding unit encodes the position information, the attribute information, and the occupancy map of the 3D data projected on the two-dimensional plane .
　(26) A bitstream generation unit that further generates a bitstream including information indicating the predetermined value and encoded data obtained by encoding the two-dimensional plane by the encoding unit
　(24) Alternatively, the image processing device according to (25).
　(27) A packing unit that packs the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit as a video frame,

　The image processing device according to any one of (24) to (26), 　wherein the encoding unit is configured to encode the video frame in which the two-dimensional plane is packed by the packing unit.
　(28)
　The image processing device according to any one of (21) to (27), wherein the two-dimensional projection unit projects the 3D data on the two-dimensional plane for each predetermined region .
　(29)
　The image processing device according to any of (21) to (28) , wherein the 3D data is a point cloud .
　(30) The data for each position included in the 3D data representing the three-dimensional structure is projected onto a two-dimensional plane, and the data for each position does not exist at a position on the two-dimensional plane where the data for each position does not exist. An
　image processing method for setting a predetermined value indicating .
[0223]
　(31) Among the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane, the data other than the data of the predetermined value indicating that the data for each position does not exist is three-dimensional. An
　image processing apparatus including a three-dimensional projection unit that projects in space .
　(32)
　The image processing apparatus according to (31), wherein the predetermined value is a predetermined fixed value .
　(33)
　The image processing device according to (31), wherein the predetermined value is larger than a maximum depth value of the 3D data .
　(34) An extraction unit that extracts information indicating the predetermined value included in the bitstream is further provided, and the
　three-dimensional projection unit extracts the data for each position included in the 3D data by the extraction unit.
　The image processing device according to any one of (31) to (33) , configured to project data other than the data of the predetermined value shown in the generated information in a three-dimensional space .
　(35) A decoding unit that decodes encoded data of the 3D data projected onto the two-dimensional plane included in the bitstream is further included, and the
　three-dimensional projection unit decodes the encoded data by the decoding unit. Among the data for each position included in the 3D data projected on the two-dimensional plane obtained as described above, data other than the data of the predetermined value shown in the information extracted by the extraction unit is
　The image processing device according to (34), which is configured to project in a three-dimensional space .
　(36)
　The image described in (35), in which the decoding unit decodes the respective encoded data of the position information, the attribute information, and the occupancy map of the 3D data projected on each layer of the two-dimensional plane. Processing equipment.
　(37) An unpacking unit for unpacking a video frame packed with the 3D data projected on the two-dimensional plane, which is obtained by decoding the encoded data by the decoding unit, and further includes the
　three-dimensional The projection unit extracts the data for each position included in the 3D data projected on the two-dimensional plane, which is obtained by unpacking the video frame by the unpacking unit, and is extracted by the extraction unit.
　The image processing device according to (35) or (36), which is configured to project data other than the data having the predetermined value indicated in the information into a three-dimensional space .
　(38)
　The image processing according to any one of (31) to (37), wherein the three-dimensional projection unit projects the 3D data projected on the two-dimensional plane for each predetermined region into the three-dimensional space. apparatus.
　(39)
　The image processing device according to any one of (31) to (38) , wherein the 3D data is a point cloud .
　(40) Among the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane, the data other than the data of the predetermined value indicating that the data for each position does not exist is three-dimensional.
　Image processing method for projecting into space .
[0224]
　(41) A three-dimensional structure capable of projecting onto one layer set for each area when the data for each position of the 3D data representing the three-dimensional structure is projected on the two-dimensional plane for each predetermined area of ​​the three-dimensional space
　Image processing including a two-dimensional projection unit that projects the data at each position within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction of the data at each position of the 3D data representing apparatus.
　(42)
　The image processing device according to (41), further including an encoding unit that encodes the 3D data projected on the two-dimensional plane by the two-dimensional projection unit .
　(43)
　The image processing device according to (42), wherein the encoding unit encodes the position information, the attribute information, and the occupancy map of the 3D data projected on the two-dimensional plane .
　(44) A bitstream generation unit that further generates a bitstream including the depth parameter set for each area and the encoded data obtained by encoding the two-dimensional plane by the encoding unit.
　The image processing device according to (42) or (43).
　(45) A packing unit that packs the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit as a video frame is provided, and the
　encoding unit packs the two-dimensional plane by the packing unit.
　The image processing device according to any one of (42) to (44), which is configured to encode the video frame .
　(46)
　The image processing device according to any of (41) to (45) , wherein the 3D data is a point cloud .
　(47) A three-dimensional structure capable of projecting onto one layer set for each area when the data for each position of the 3D data representing the three-dimensional structure is projected on the two-dimensional plane for each predetermined area of ​​the three-dimensional space The
　image processing method of projecting the data for each position within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction for the data for each position of the 3D data indicating the two-dimensional plane .
[0225]
　(51) The data for each position when the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane for each predetermined region in the three-dimensional space is projected to the three-dimensional space. In the depth direction range indicated by the depth parameter that limits the range in the depth direction of the data for each position of the 3D data that can be projected onto one layer set for each region in the three-dimensional space. An
　image processing apparatus including a three-dimensional projection unit .
　(52) The depth parameter extracted by the extraction unit may further include an extraction unit that extracts the depth parameter included in the bitstream, and the
　three-dimensional projection unit may extract the data for each position included in the 3D data by the extraction unit.
　The image processing device according to (51), which is configured to project within the range in the depth direction indicated by .
　(53) A decoding unit that decodes encoded data of the 3D data projected onto the two-dimensional plane included in the bitstream may be further included, and the
　three-dimensional projection unit may decode the encoded data by the decoding unit. The data for each position of the 3D data projected on the two-dimensional plane obtained as described above is projected within the range in the depth direction indicated by the depth parameter extracted by the extraction unit. that
　the image processing apparatus according to (52).
　(54)
　The image according to (53), wherein the decoding unit decodes the respective encoded data of the position information, the attribute information, and the occupancy map of the 3D data projected on each layer of the two-dimensional plane. Processing equipment.
　(55) An unpacking unit for unpacking a video frame packed with the 3D data projected on the two-dimensional plane, which is obtained by decoding the encoded data by the decoding unit, and further includes the
　three-dimensional The projection unit extracts the data for each position of the 3D data projected on the two-dimensional plane, which is obtained by unpacking the video frame by the unpacking unit, by using the depth parameter extracted by the extraction unit.
　The image processing device according to (53) or (54) configured to project within the range in the depth direction shown .
　(56)
　The image processing device according to any of (51) to (55) , wherein the 3D data is a point cloud .
　(57) The data for each position when the data for each position included in the 3D data representing the three-dimensional structure projected on the two-dimensional plane is projected in the three-dimensional space for each predetermined region of the three-dimensional space In the depth direction range indicated by the depth parameter that limits the range in the depth direction of the data for each position of the 3D data that can be projected onto one layer set for each region in the three-dimensional space. the
　image processing method.
Explanation of symbols
[0226]
　100 coding device, 111 patch decomposition unit, 112 packing unit, 113 auxiliary patch information compression unit, 114 video coding unit, 115 video coding unit, 116 OMap coding unit, 117 multiplexer, 151 normal direction estimation unit, 152 Segmentation initial setting unit, 153 segmentation updating unit, 154 two-dimensional projection unit, 155 pixel distribution analysis unit, 200 decoding device, 211 demultiplexer, 212 auxiliary patch information decoding unit, 213 video decoding unit, 214 video decoding unit, 215 OMap decoding Section, 216 unpacking section, 217 3D reconstruction section, 251 three-dimensional projection section, 252 pixel distribution analysis section, 253 inverse segmentation updating section, 254 inverse segmentation initial setting section, 255 inverse normal direction estimation section
The scope of the claims
[Claim 1]
　An
　image processing apparatus comprising a two-dimensional projection unit that projects data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers .
[Claim 2]
　The two-dimensional projection unit, the 3D data, the data for each of the positions located when viewed from the projection plane overlaps the depth direction, of the two-dimensional plane of the plurality of layers, projecting the different layers
　according to claim 1 The image processing device according to.
[Claim 3]
　The two-dimensional projection unit, the two-dimensional plane, the position viewed from the projection plane of the 3D data to generate a maximum as many layers of data for each of the position overlapping in the depth direction
　image according to claim 2 Processing equipment.
[Claim 4]

　The image processing apparatus according 　to claim 1, wherein the two-dimensional projection unit sets a predetermined value indicating that data for each position does not exist at a position on the two-dimensional plane where data for each position does not exist .
[Claim 5]

　The image processing device according to claim 4, 　wherein the predetermined value is a predetermined fixed value .
[Claim 6]

　The image processing apparatus according to claim 4, 　wherein the predetermined value is a value larger than a maximum depth value of the 3D data .
[Claim 7]
　The two-dimensional projection unit, when projecting the data for each position of the 3D data onto a two-dimensional plane for each predetermined region of a three-dimensional space, can project onto one layer set for each region.
　The image processing apparatus according to claim 1 , wherein the data for each position within the range in the depth direction indicated by the depth parameter that limits the range in the depth direction for the data for each position of the 3D data is projected on the two-dimensional plane. ..
[Claim 8]

　The image processing apparatus according to claim 1, 　further comprising an encoding unit that encodes the 3D data projected by the two-dimensional projection unit onto the two-dimensional plane .
[Claim 9]
　A bitstream including information indicating the number of layers of the two-dimensional plane onto which the 3D data is projected by the two-dimensional projection unit, and encoded data obtained by encoding the two-dimensional plane by the encoding unit.
　The image processing apparatus according to claim 8, further comprising a bitstream generation unit that generates
[Claim 10]

　The image processing apparatus according to claim 1 　, wherein the 3D data is a point cloud .
[Claim 11]
　An
　image processing method for projecting data for all positions included in 3D data representing a three-dimensional structure onto a two-dimensional plane of a plurality of layers .
[Claim 12]
　An
　image processing apparatus comprising a three-dimensional projection unit that projects data for all positions of 3D data projected onto a two-dimensional plane having the number of layers indicated by the number-of-layers information, into a three-dimensional space .
[Claim 13]
　The three-dimensional projection unit, among the data for each of the positions included in the 3D data, the data for each of the position data other than the data of a predetermined value indicating that there is no, projecting the three-dimensional space
　according to Item 12. The image processing device according to item 12.
[Claim 14]

　The image processing device according to claim 13, 　wherein the predetermined value is a predetermined fixed value .
[Claim 15]

　The image processing apparatus according to claim 13, 　wherein the predetermined value is a value larger than a maximum depth value of the 3D data .
[Claim 16]
　The three-dimensional projection unit, when projecting the data for each position included in the 3D data into the three-dimensional space for each predetermined region of the three-dimensional space, sets the data for each position as the three-dimensional space. space, projecting in the range of the depth direction indicated by the depth parameter that limits the depth direction of the range of data for each position of the 3D data representing the three-dimensional structures capable projected on first layer which is set for each of the areas
　according Item 12. The image processing device according to item 12.
[Claim 17]
　The
　three-dimensional projection unit may further include an extraction unit that extracts the layer number information included in the bitstream, and the three-dimensional projection unit may project the two-dimensional plane having the number of layers indicated by the layer number information extracted by the extraction unit.
　The image processing device according to claim 12 , wherein the data for all the positions of 3D data is configured to be projected onto the three-dimensional space .
[Claim 18]
　The decoding unit may further include a decoding unit that decodes the encoded data of the 3D data projected on the two-dimensional plane included in the bitstream, and the
　three-dimensional projection unit may obtain the encoded data by the decoding unit.
　18. The image processing apparatus according to claim 17 , wherein all the data for each of the positions of the 3D data projected on the two-dimensional plane are projected on the three-dimensional space .
[Claim 19]

　The image processing apparatus according to claim 12 　, wherein the 3D data is a point cloud .
[Claim 20]
　An
　image processing method for projecting data for every position of 3D data projected onto a two-dimensional plane having the number of layers indicated by the number-of-layers information onto a three-dimensional space .