Title of Invention: Image Processing Device and Method
Technical field
[0001]
　The present disclosure relates to an image processing device and a method, and more particularly to an image processing device and a method capable of suppressing an increase in a load when generating a point cloud from a mesh.
Background technology
[0002]
　Conventionally, as a method for coding 3D data representing a three-dimensional structure such as a point cloud, for example, there has been coding using Octree (see, for example, Non-Patent Document 1).
[0003]
　In recent years, it has been proposed that a target 3D object is voxel-coded and then coded by combining Octree coding and Mesh coding (Triangle soup) (see, for example, Non-Patent Document 2).
Prior art literature
Non-patent literature
[0004]
Non-Patent Document 1: R. Mekuria, Student Member IEEE, K. Blom, P. Cesar., Member, IEEE, "Design, Implementation and Evaluation of a Point Cloud Codec for Tele-Immersive Video", tcsvt_paper_submitted_february.pdf
Non-Patent Document 2: Ohji Nakagami, Phil Chou, Maja Krivokuca, Khaled Mammou, Robert Cohen, Vladyslav Zakharchenko, Gaelle Martin-Cocher, "Second Working Draft for PCC Categories 1, 3", ISO / IEC JTC1 / SC29 / WG11, MPEG 2018 / N17533 , April 2018, San Diego, US
Outline of the invention
Problems to be solved by the invention
[0005]
　However, in the conventional method, when a point cloud is generated from a mesh, points are densely sampled on the surface of the mesh to generate a high-density point cloud, and then a voxel with the same resolution as the input is generated. It was resampling to the data. Therefore, the amount of processing and the amount of data to be processed are large, and there is a risk that the load when generating the point cloud from the mesh will increase.
[0006]
　The present disclosure has been made in view of such a situation, and makes it possible to suppress an increase in the load when generating a point cloud from a mesh.
Means to solve problems
[0007]
　The image processing device on one side of the present technology includes a point cloud generator that generates point cloud data by arranging points at the intersection of the mesh surface and the vector whose starting origin is the position coordinates corresponding to the specified resolution. It is an image processing device.
[0008]
　The image processing method of one aspect of the present technology is an image processing method that generates point cloud data by arranging points at the intersection of a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution.
[0009]
　In the image processing apparatus and method of one aspect of the present technology, point cloud data is generated by arranging points at the intersection of a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution.
A brief description of the drawing
[0010]
[Fig. 1] Fig. 1 is a diagram illustrating a process for generating a point cloud from a mesh.
[Fig. 2] Fig. 2 is a diagram illustrating a process for generating a point cloud from a mesh.
[Fig. 3] Fig. 3 is a diagram illustrating an example of how an intersection is calculated.
[Fig. 4] Fig. 4 is a block diagram showing a main configuration example of a point cloud generator.
[Fig. 5] Fig. 5 is a flowchart illustrating an example of a flow of point cloud generation processing.
[Fig. 6] Fig. 6 is a diagram illustrating an example of how an intersection is derived.
[Fig. 7] Fig. 7 is a diagram illustrating an example of how an intersection is derived.
[Fig. 8] Fig. 8 is a diagram illustrating an example of how an intersection is derived.
FIG. 9 is a block diagram showing a main configuration example of a decoding device.
FIG. 10 is a flowchart illustrating an example of a flow of decoding processing.
FIG. 11 is a block diagram showing a main configuration example of a coding device.
FIG. 12 is a flowchart illustrating an example of a flow of coding processing.
[Fig. 13] Fig. 13 is a diagram illustrating an example of how the Triangle soup is made scalable.
[Fig. 14] Fig. 14 is a diagram illustrating an example of a state of point generation.
[Fig. 15] Fig. 15 is a diagram illustrating an example of a state of point generation.
[Fig. 16] Fig. 16 is a diagram illustrating an example of a state of point generation.
[Fig. 17] Fig. 17 is a diagram illustrating an example of a state of point generation.
[Fig. 18] Fig. 18 is a diagram illustrating an example of a state of point generation.
[Fig. 19] Fig. 19 is a flowchart illustrating an example of a flow of point cloud generation processing.
[Fig. 20] Fig. 20 is a block diagram showing a main configuration example of a computer.
Mode for carrying out the invention
[0011]
　Hereinafter, embodiments for carrying out the present disclosure (hereinafter referred to as embodiments) will be described. The explanation will be given in the following order.
　1. 1. Generation of point cloud
　2. First embodiment (point cloud generator)
　3. Second embodiment (decoding device)
　4. Third Embodiment (encoding device)
　5. Fourth Embodiment (scalable Triangle soup)
　6. Addendum
[0012]
　<1. Generation of point cloud>
　　
　The scope disclosed in this technology is not limited to the contents described in the embodiments, but the following non-public information at the time of filing The contents described in the patent document are also included.
[0013]
　Non-Patent Document 1: (above)
　Non-Patent Document 2: (above)
　Non-Patent Document 3: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "Advanced video coding for generic audiovisual services", H.264, 04/2017
　Non Patent Document 4: TELECOMMUNICATION STANDARDIZATION SECTOR OF ITU (International Telecommunication Union), "High efficiency video coding", H.265, 12/2016
　Non-Patent Document 5: Jianle Chen, Elena Alshina, Gary J. Sullivan, Jens-Rainer, Jill Boyce, "Algorithm Description of Joint Exploration Test Model 4", JVET-G1001_v1, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11 7th Meeting: Torino, IT , 13-21 July 2017
[0014]
　In other words, the contents described in the above-mentioned non-patent documents are also the basis for determining the support requirements. For example, even if the Quad-Tree Block Structure described in Non-Patent Document 4 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Non-Patent Document 5 are not directly described in the embodiment. It is within the scope of disclosure of this technology and shall meet the support requirements of the scope of claims. Similarly, technical terms such as Parsing, Syntax, and Semantics are also within the scope of disclosure of the present technology even if there is no direct description in the embodiment. It shall meet the support requirements of the claims.
[0015]
　　
　Conventionally, a point cloud that represents a three-dimensional structure based on the position information and attribute information of a point cloud, and a three-dimensional shape that is composed of vertices, edges, and faces and is defined using a polygonal representation. There was 3D data such as a mesh to be used.
[0016]
　For example, in the case of a point cloud, a three-dimensional structure (three-dimensional object) is expressed as a set (point cloud) of a large number of points. That is, the point cloud data (also referred to as point cloud data) is composed of position information and attribute information (for example, color, etc.) of each point in this point cloud. Therefore, the data structure is relatively simple, and an arbitrary three-dimensional structure can be expressed with sufficient accuracy by using a sufficiently large number of points.
[0017]
　　
　Since the amount of such point cloud data is relatively large, a coding method using voxels is used to compress the amount of data due to coding or the like. it was thought. A voxel is a three-dimensional region for quantizing the position information to be encoded.
[0018]
　That is, the three-dimensional area containing the point cloud is divided into small three-dimensional areas called voxels, and each voxel indicates whether or not the points are included. By doing so, the position of each point is quantized in voxel units. Therefore, by converting the point cloud data into such voxel data (also referred to as voxel data), the increase in the amount of information is suppressed (typically, the amount of information is reduced). Can be done.
[0019]
　　
　Furthermore, it was considered to construct an Octree using such voxel data. Octree is a tree-structured version of voxel data. The value of each bit of the lowest node of this Octree indicates the presence or absence of a point for each voxel. For example, a value "1" indicates a voxel containing points, and a value "0" indicates a voxel containing no points. In Octree, one node corresponds to eight voxels. That is, each node of Octtree is composed of 8 bits of data, and the 8 bits indicate the presence or absence of points of 8 voxels.
[0020]
　Then, the upper node of the Octtree indicates the presence or absence of a point in the area in which the eight voxels corresponding to the lower node belonging to the node are combined into one. That is, the upper node is generated by collecting the voxel information of the lower node. If a node having a value of "0", that is, all eight corresponding voxels do not contain points, that node is deleted.
[0021]
　By doing so, a tree structure (Octree) consisting of nodes whose value is not "0" is constructed. That is, Octree can indicate the presence or absence of voxel points at each resolution. Therefore, by converting the voxel data into an Octree and encoding it, it is possible to more easily restore the voxel data having various resolutions at the time of decoding. In other words, voxel scalability can be realized more easily.
[0022]
　Further, by omitting the node having the value "0" as described above, the voxel in the area where the point does not exist can be reduced in resolution, so that further increase in the amount of information can be suppressed (typically, the amount of information). Can be reduced).
[0023]
　　 In
　recent years, for example, as described in Non-Patent Document 2, after voxelizing a target 3D object, a combination of Octree coding and Mesh coding (Triangle soup) is performed. It was proposed to be.
[0024]
　For example, as shown in A of FIG. 1, Octree data is decoded to generate voxel data. In the example of A in FIG. 1, voxels 11-1, voxels 11-2, and voxels 11-3 are generated.
[0025]
　Next, as shown in B of FIG. 1, for example, the mesh shape (that is, the surface of the mesh) is restored from the voxel data. In the example B of FIG. 1, the surface 12 of the mesh is restored based on voxels 11-1, voxels 11-2, and voxels 11-3.
[0026]
　Next, as shown in C of FIG. 1, for example, the point 13 is arranged on the surface 12 of the mesh with a resolution of 1 / (2 * block width). The blockwidth indicates the longest side of the bounding box including the mesh.
[0027]
　Then, for example, as shown in D of FIG. 1, the point 13 is voxelized again at a specified resolution d. At that time, the mesh data (surface 12 and the like) is removed. That is, when generating point cloud data having a desired resolution from the mesh data, resampling is performed so as to reduce the resolution (number of points) of the points 13 once sampled to a high resolution.
[0028]
　However, in such a method, sampling must be performed twice, which is redundant in processing. In addition, the amount of data increases because a high-density point cloud is sampled. Therefore, there is a risk that the load when generating the point cloud from the mesh will increase. As a result, there is a risk that the processing time will increase and the amount of resources such as memory will increase.
[0029]
　　
　Therefore, by utilizing the fact that the resolution of the output point cloud is the same as the resolution of the input point cloud converted into voxels, the point cloud is generated at high speed by limiting the number of voxel judgments. To do so.
[0030]
　More specifically, the point cloud data is generated by arranging the points at the intersection of the surface of the mesh and the vector whose starting origin is the position coordinates corresponding to the specified resolution.
[0031]
　For example, the image processing device is provided with a point cloud generation unit that generates point cloud data by arranging points at the intersection of a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution.
[0032]
　By doing so, voxel data corresponding to the input resolution can be generated from the mesh in one process. Therefore, it is possible to suppress an increase in the load when generating a point cloud from the mesh. As a result, it is possible to suppress an increase in processing time and an increase in resource usage such as memory. Typically, the processing time can be reduced and the amount of resources such as memory used can be reduced. In addition, the point cloud can be generated at higher speed.
[0033]
　　
　Next, the method of deriving the point cloud will be described more specifically. First, as in the example of FIG. 2, the sides of the bounding box containing the data to be encoded and the vector Vi having the same direction and the same length are generated at intervals k * d. In FIG. 2, a vector Vi as shown by an arrow 23 is set for the surface 22 of the mesh existing in the bounding box 21. d is the quantization size when voxels the bounding box. k is any natural number. That is, the vector Vi whose starting origin is the position coordinates corresponding to the specified voxel resolution is set.
[0034]
　Next, the intersection of the set vector Vi (arrow 23) and the surface 22 (that is, triangular mesh) of the decoded mesh is determined. When the vector Vi and the surface 22 of the triangle intersect, the coordinate values ​​of the intersection 24 are calculated.
[0035]
　As the direction of this vector Vi, two positive and negative directions can be set for each of the x, y, and z directions perpendicular to each other (directions parallel to each side of the bounding box). That is, the intersection determination may be performed for each of the six types of vector Vis. In this way, by performing the intersection determination in more directions, the intersection can be detected more reliably.
[0036]
　The starting point of the vector Vi may be limited to the range of the three vertices of the triangular mesh. By doing so, the number of vector Vis to be processed can be reduced, so that an increase in load can be suppressed (for example, the processing speed can be further increased).
[0037]
　Further, as an auxiliary process, when the coordinate values ​​of the intersections overlap due to different vectors or meshes, one point may be left and deleted. By removing the overlapping points in this way, it is possible to suppress an increase in unnecessary processing and an increase in load (for example, the processing can be made faster).
[0038]
　Further, as an auxiliary process, when the coordinate value of the intersection is outside the bounding box, the position of the intersection may be clipped (moved) into the bounding box by the clip process. .. Alternatively, the intersection may be deleted.
[0039]
　As described above, the points having the obtained coordinate values ​​are output as the decoding result. That is, the points are arranged at the obtained coordinate values. By doing so, voxel data corresponding to the input resolution can be generated from the mesh in one process. Therefore, it is possible to suppress an increase in the load when generating a point cloud from the mesh.
[0040]
　　
　The method of intersection determination and coordinate value calculation is arbitrary. For example, it may be calculated using Cramer's rule as shown in FIG. For example, P is the coordinates of the intersection, origin is the coordinates of the ray, ray is the direction vector, and t is the scalar value, and the intersection passing through the ray is expressed in a straight line as follows.
[0041]
　P = origin + ray * t
[0042]
　Also, vo is the vertex coordinates of the triangle, edge1 is the vector obtained by subtracting v0 from the coordinates v1, and edge2 is also the vector obtained by subtracting v0 from the coordinates v2 in the same way. The point P is u (scalar value) from v0 in the vector edge1 direction, and the intersection on the triangle is represented by the edge vector as follows.
[0043]
　P = v0 + edge1 * u + edge2 * v
[0044]
　Combining these two equations gives a system of equations.
[0045]
　origin + ray * t = v0 + edge1 * u + edge2 * v
[0046]
　It can be expressed as follows.
[0047]
　edge1 * u + edge2 * v ・ ray * t = origin ・ v0
[0048]
　Since it is in the form of three-dimensional simultaneous one-dimensional equations in this way, it can be calculated mechanically by Cramer's rule as a determinant.
[0049]
　<2. First Embodiment>
　　
　Next, a configuration for realizing the above processing will be described. FIG. 4 is a block diagram showing an example of a configuration of a point cloud generation device, which is an aspect of an image processing device to which the present technology is applied. The point cloud generator 100 shown in FIG. 4 has <1. As explained in Generating a point cloud>, it is a device that generates a point cloud from a mesh.
[0050]
　Note that FIG. 4 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the point cloud generator 100, there may be a processing unit that is not shown as a block in FIG. 4, or there may be a processing or data flow that is not shown as an arrow or the like in FIG.
[0051]
　As shown in FIG. 4, the point cloud generation device 100 includes a vector setting unit 111, an intersection determination unit 112, an auxiliary processing unit 113, and an output unit 114.
[0052]
　The vector setting unit 111 sets (generates) the vector Vi for intersection determination, for example, as described above in . As described above, this vector Vi is a vector having the same direction and the same length as the side of the bounding box containing the data to be encoded. The vector setting unit 111 supplies vector information indicating the set vector Vi to the intersection determination unit 112.
[0053]
　The intersection determination unit 112 acquires the mesh data input to the point cloud generation device 100, and further acquires the vector information supplied from the vector setting unit 111. The intersection determination unit 112 determines the intersection between the mesh surface indicated by the acquired mesh data and the vector Vi indicated by the vector information, as described above in, for example, or . I do. When an intersection is detected, the intersection determination unit 112 calculates the coordinate value thereof. The intersection determination unit 112 supplies the calculated coordinate values ​​of the intersection (intersection coordinates) to the auxiliary processing unit 113.
[0054]
　The auxiliary processing unit 113 acquires the intersection coordinates supplied from the intersection determination unit 112, and performs auxiliary processing for the intersection as described above in, for example, . The auxiliary processing unit 113 supplies the output unit 114 with the coordinates of the intersection where the auxiliary processing has been performed as needed.
[0055]
　The output unit 114 outputs the intersection coordinates supplied from the auxiliary processing unit 113 to the outside of the point cloud generator 100 as (position information) of the point cloud data. That is, point cloud data in which points are arranged at the derived intersection coordinates is generated and output.
[0056]
　Note that these processing units (vector setting unit 111 to output unit 114) have an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Further, each processing unit has, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and the above-mentioned processing is realized by executing a program using them. You may do so. Of course, each processing unit may have both configurations, and a part of the above-mentioned processing may be realized by a logic circuit, and the other may be realized by executing a program. The configurations of the respective processing units may be independent of each other. For example, some processing units realize a part of the above-mentioned processing by a logic circuit, and some other processing units execute a program. The above-mentioned processing may be realized by the other processing unit by both the logic circuit and the execution of the program.
[0057]
　With such a configuration, the point cloud generator 100 can be <1. The effect as explained in Point cloud generation> can be obtained. For example, voxel data corresponding to the input resolution can be generated from the mesh in one process. Therefore, it is possible to suppress an increase in the load of point cloud data generation. Therefore, for example, point cloud data can be generated at a higher speed. Further, for example, the manufacturing cost of the point cloud generator 100 can be reduced.
[0058]
　　
　Next, an example of the flow of the point cloud generation process executed by the point cloud generation device 100 will be described with reference to the flowchart of FIG.
[0059]
　When the point cloud generation process is started, the intersection determination unit 112 acquires mesh data in step S101.
[0060]
　In step S102, the vector setting unit 111 sets the same direction and the same length as the side of the vector Vi (bounding box containing the data to be encoded) whose starting origin is the position coordinate corresponding to the specified voxel resolution. Vector to have) is set.
[0061]
　In step S103, the intersection determination unit 112 determines the intersection of the vector Vi set in step S102 and the surface (triangle) of the mesh indicated by the mesh data acquired in step S101.
[0062]
　In step S104, the intersection determination unit 112 calculates the coordinates of the intersection detected in step S103.
[0063]
　In step S105, the auxiliary processing unit 113 deletes the overlapping intersections, leaving one point.
[0064]
　In step S106, the auxiliary processing unit 113 processes (for example, clips or deletes) an intersection outside the bounding box.
[0065]
　In step S107, the output unit 114 outputs the coordinates of the intersection obtained as described above as point cloud data (position information).
[0066]
　When the process of step S107 is completed, the point cloud generation process is completed.
[0067]
　Note that each of the above processes is described in <1. It is performed in the same manner as the above-mentioned example in Point cloud generation>. Therefore, by executing each of the above processes, the point cloud generator 100 can be set to <1. The effect as explained in Point cloud generation> can be obtained. For example, voxel data corresponding to the input resolution can be generated from the mesh in one process. Therefore, it is possible to suppress an increase in the load of point cloud data generation. Therefore, for example, point cloud data can be generated at a higher speed. Further, for example, the manufacturing cost of the point cloud generator 100 can be reduced.
[0068]
　　 In
　the above intersection determination, the intersection determination with respect to the inside of the surface may be performed using a vector Vi that is sparser than the case of the intersection determination with respect to the end of the surface. .. For example, as in the example of FIG. 6, the intersection determination may be performed on the surface 201 by using the vectors Vi202-1 to Vi202-8. In this example, the intervals between the vectors Vi202-1 to the vectors Vi202-8 are narrow between the vectors Vi202-1 to the vector Vi202-3 and the vectors Vi202-6 to Vi202-8. In other words, the intervals between the vectors Vi202-3 and the vectors Vi202-6 are set wider than the intervals between the other vectors Vi202-6. That is, the intervals between the vectors Vi202-1 to Vi202-3 and the vectors Vi202-6 to Vi202-8 used for determining the intersection with respect to the end of the surface 201 are set narrow (dense), and the inside of the surface 201 is set. The intervals between the vectors Vi202-3 and the vectors Vi202-6 used for determining the intersection with the vector Vi202-3 are widely set (sparse).
[0069]
　In this way, the number of points generated inside the triangle can be reduced by intentionally performing collision determination of the vector Vi with a sparse width (roughening the width of the start origin). Therefore, it is possible to suppress an increase in the coding bit rate of the attribute information (color information, etc.) of the point cloud.
[0070]
　　 Further
　, the coordinates for which the intersection determination has been performed once may not be calculated twice. For example, as in the example of FIG. 7, when there are a plurality of mesh faces (face 212 and face 213) for one vector Vi211, the intersection is determined for one vector Vi211 at the same time. The processing can be made faster.
[0071]
　　 Further , as shown in FIG. 8, when one vector Vi221 intersects a plurality of triangles (face 222 and face 223) and there is a space between the triangles, a point (point (face 222)) is provided in the space. In the figure, black dots) may be generated to fill in the holes (denoise). By doing so, a more accurate point cloud can be generated. That is, it is possible to suppress the reduction in the image quality of the displayed image (typically, the image quality can be improved).
[0072]
　　 In the
　intersection determination as described above, a plurality of processes may be performed in parallel. For example, the intersection determination of a plurality of vectors for one surface of the mesh may be processed in parallel with each other (processed in parallel). That is, the processing may be performed independently for each vector. By doing so, the intersection determination can be performed at a higher speed.
[0073]
　Further, for example, the intersection determination of each of the plurality of surfaces for one vector may be processed in parallel with each other (processed in parallel). That is, the processing may be performed independently for each surface of the mesh. By doing so, the intersection determination can be performed at a higher speed.
[0074]
　<3. Second Embodiment>
　　
　FIG. 9 is a block diagram showing an example of a configuration of a decoding device which is an aspect of an image processing device to which the present technology is applied. The decoding device 300 shown in FIG. 9 is a decoding device corresponding to the coding device 500 of FIG. 11 described later, and for example, decodes the bit stream generated by the coding device 500 and restores the point cloud data. It is a device.
[0075]
　Note that FIG. 9 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the decoding device 300, there may be a processing unit that is not shown as a block in FIG. 9, or there may be a processing or data flow that is not shown as an arrow or the like in FIG.
[0076]
　As shown in FIG. 9, the decoding device 300 includes a reversible decoding unit 311, an Octree decoding unit 312, a mesh shape restoration unit 313, a Point cloud generation unit 314, and an Attribute decoding unit 315.
[0077]
　The reversible decoding unit 311 acquires a bit stream input to the decoding device 300, decodes the bit stream, and generates Octree data. The reversible decoding unit 311 supplies the Octree data to the Octree decoding unit 312.
[0078]
　The Octree decoding unit 312 acquires the Octree data supplied from the reversible decoding unit 311, constructs an Octree from the Octree data, and generates voxel data from the Octree. The Octree decoding unit 312 supplies the generated voxel data to the mesh shape restoration unit 313.
[0079]
　The Mesh shape restoration unit 313 restores the mesh shape using the voxel data supplied from the Octree decoding unit 312. The Mesh shape restoration unit 313 supplies the generated mesh data to the Point cloud generation unit 314.
[0080]
　The Point cloud generation unit 314 generates point cloud data from the mesh data supplied from the Mesh shape restoration unit 313, and supplies the generated point cloud data to the Attribute decoding unit 315. The point cloud generation unit 314 has the same configuration as the point cloud generation device 100 (FIG. 4), and performs the same processing. That is, the Point cloud generator 314 has <1. Generation of point cloud> and <2. The point cloud data is generated from the mesh data by the method as described above in the first embodiment>.
[0081]
　Therefore, the point cloud generation unit 314 can obtain the same effect as the point cloud generation device 100. For example, the Point cloud generation unit 314 can generate voxel data corresponding to the input resolution from the mesh in one process. Therefore, the point cloud generation unit 314 can suppress an increase in the load of point cloud data generation. Therefore, the point cloud generation unit 314 can generate point cloud data at a higher speed, for example. Further, for example, the manufacturing cost of the Point cloud generation unit 314 can be reduced.
[0082]
　The Attribute decoding unit 315 performs processing related to decoding of the attribute information. For example, the Attribute decoding unit 315 decodes the attribute information corresponding to the point cloud data supplied from the Point cloud generation unit 314. Then, the Attribute decoding unit 315 includes the decoded attribute information in the point cloud data supplied from the Point cloud generation unit 314 and outputs the decoded attribute information to the outside of the decoding device 300.
[0083]
　These processing units (reversible decoding unit 311 to Attribute decoding unit 315) have an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Further, each processing unit may have, for example, a CPU, ROM, RAM, etc., and execute a program using them to realize the above-mentioned processing. Of course, each processing unit may have both configurations, and a part of the above-mentioned processing may be realized by a logic circuit, and the other may be realized by executing a program. The configurations of the respective processing units may be independent of each other. For example, some processing units realize a part of the above-mentioned processing by a logic circuit, and some other processing units execute a program. The above-mentioned processing may be realized by the other processing unit by both the logic circuit and the execution of the program.
[0084]
　With such a configuration, the decoding device 300 can be <1. Generation of point cloud> and <2. The effects as described in the first embodiment> can be obtained. For example, since voxel data corresponding to the input resolution can be generated from the mesh in one process, the decoding device 300 can suppress an increase in the load of point cloud data generation. Therefore, for example, the decoding device 300 can generate point cloud data at a higher speed. Further, for example, the manufacturing cost of the decoding device 300 can be reduced.
[0085]
　　
　Next, an example of the flow of the decoding process executed by the decoding device 300 will be described with reference to the flowchart of FIG.
[0086]
　When the decoding process is started, the reversible decoding unit 311 acquires a bit stream in step S301.
[0087]
　In step S302, the reversible decoding unit 311 reversibly decodes the bit stream acquired in step S301.
[0088]
　In step S303, the Octree decoding unit 312 builds the Octree and restores the voxel data.
[0089]
　In step S304, the Mesh shape restoration unit 313 restores the mesh shape from the voxel data restored in step S303.
[0090]
　In step S305, the point cloud generation unit 314 executes the point cloud generation process (FIG. 5), and <1. Generation of point cloud> and <2. A point cloud is generated from the mesh shape restored in step S304 by the method described above in the first embodiment>.
[0091]
　In step S306, the Attribute decoding unit 315 decodes the attribute information (Attribute).
[0092]
　In step S307, the attribute decoding unit 315 includes the attribute information decoded in step S306 in the point cloud data and outputs the data.
[0093]
　When the process of step S307 is completed, the decoding process is completed.
[0094]
　By executing each process as described above, the decoding device 300 can perform <1. Generation of point cloud> and <2. The effects as described in the first embodiment> can be obtained.
[0095]
　<4. Third Embodiment>
　　
　FIG. 11 is a block diagram showing an example of a configuration of a coding device which is an aspect of an image processing device to which the present technology is applied. The coding device 500 shown in FIG. 11 is a device that encodes 3D data such as a point cloud using voxels and Octree.
[0096]
　Note that FIG. 11 shows the main things such as the processing unit and the data flow, and not all of them are shown in FIG. That is, in the coding apparatus 500, there may be a processing unit that is not shown as a block in FIG. 11, or there may be a processing or data flow that is not shown as an arrow or the like in FIG. This also applies to other figures for explaining the processing unit and the like in the coding apparatus 500.
[0097]
　As shown in FIG. 11, the coding device 500 includes a Voxel generation unit 511, a Geometry coding unit 512, a Geometry decoding unit 513, an Attribute coding unit 514, and a bitstream generation unit 515.
[0098]
　The Voxel generation unit 511 acquires the point cloud data input to the encoding device 500, sets a bounding box for the area including the acquired point cloud data, and further divides the bounding box. , By setting voxels, the position information of the point cloud data is quantized. The Voxel generation unit 511 supplies the voxel (Voxel) data thus generated to the Geometry coding unit 512.
[0099]
　The Geometry coding unit 512 encodes the voxel data supplied from the Voxel generation unit 511 and encodes the position information of the point cloud. The Geometry coding unit 512 supplies the generated point cloud position information coded data to the bitstream generation unit 515. Further, the Geometry coding unit 512 supplies the Octree data generated when encoding the position information of the point cloud to the Geometry decoding unit 513.
[0100]
　The Geometry decoding unit 513 decodes the Octree data to generate the location information of the point cloud. The Geometry decoding unit 513 supplies the generated point cloud data (position information) to the Attribute coding unit 514.
[0101]
　The Attribute encoding unit 514 encodes the attribute information corresponding to the point cloud data (position information) based on the input encode parameter. The Attribute coding unit 514 supplies the coded data of the generated attribute information to the bitstream generation unit 515.
[0102]
　The bitstream generation unit 515 generates and encodes a bitstream including the coded data of the position information supplied from the Geometry coding unit 512 and the coded data of the attribute information supplied from the Attribute coding unit 514. Output to the outside of the device 500.
[0103]
　　　 The
　Geometry coding unit 512 includes an Octree generation unit 521, a Mesh generation unit 522, and a lossless coding unit 523.
[0104]
　The Octree generation unit 521 constructs an Octree using the voxel data supplied from the Voxel generation unit 511, and generates Octree data. The Octree generation unit 521 supplies the generated Octree data to the Mesh generation unit 522.
[0105]
　The Mesh generation unit 522 generates mesh data using the Octree data supplied from the Octree generation unit 521, and supplies the mesh data to the lossless coding unit 523. Further, the Mesh generation unit 522 supplies Octree data to the Geometry decoding unit 513.
[0106]
　The lossless coding unit 523 acquires the Mesh data supplied from the Mesh generation unit 522. Further, the lossless coding unit 523 acquires an encoding parameter input from the outside of the coding device 500. This encoding parameter is information that specifies the type of encoding to be applied, and is input by a user operation or supplied from an external device or the like, for example. The lossless coding unit 523 encodes the mesh data with the type specified by this encoding parameter and generates the coded data of the position information. The lossless coding unit 523 supplies the position information to the bitstream generation unit 515.
[0107]
　　　 The
　Geometry decoding unit 513 includes an Octree decoding unit 531, a Mesh shape restoration unit 532, and a Point cloud generation unit 533.
[0108]
　The Octree decoding unit 531 decodes the Octree data supplied from the Geometry coding unit 512 and generates voxel data. The Octree decoding unit 531 supplies the generated voxel data to the mesh shape restoration unit 532.
[0109]
　The Mesh shape restoration unit 532 restores the mesh shape using the voxel data supplied from the Octree decoding unit 531 and supplies the mesh data to the Point cloud generation unit 533.
[0110]
　The Point cloud generation unit 533 generates point cloud data from the mesh data supplied from the Mesh shape restoration unit 532, and supplies the generated point cloud data to the Attribute coding unit 514. The point cloud generation unit 533 has the same configuration as the point cloud generation device 100 (FIG. 4), and performs the same processing. That is, the Point cloud generator 533 has <1. Generation of point cloud> and <2. The point cloud data is generated from the mesh data by the method as described above in the first embodiment>.
[0111]
　Therefore, the point cloud generation unit 533 can obtain the same effect as the point cloud generation device 100. For example, the Point cloud generation unit 533 can generate voxel data corresponding to the input resolution from the mesh in one process. Therefore, the point cloud generation unit 533 can suppress an increase in the load of point cloud data generation. Therefore, the point cloud generation unit 533 can generate point cloud data at a higher speed, for example. Further, for example, the manufacturing cost of the Point cloud generation unit 533 can be reduced.
[0112]
　These processing units (Voxel generation unit 511 to Attribute coding unit 514, Octree generation unit 521 to lossless coding unit 523, and Octree decoding unit 531 to Point cloud generation unit 533) have an arbitrary configuration. For example, each processing unit may be configured by a logic circuit that realizes the above-mentioned processing. Further, each processing unit may have, for example, a CPU, ROM, RAM, etc., and execute a program using them to realize the above-mentioned processing. Of course, each processing unit may have both configurations, and a part of the above-mentioned processing may be realized by a logic circuit, and the other may be realized by executing a program. The configurations of the respective processing units may be independent of each other. For example, some processing units realize a part of the above-mentioned processing by a logic circuit, and some other processing units execute a program. The above-mentioned processing may be realized by the other processing unit by both the logic circuit and the execution of the program.
[0113]
　With such a configuration, the coding device 500 has <1. Generation of point cloud> and <2. The effects as described in the first embodiment> can be obtained. For example, since voxel data corresponding to the input resolution can be generated from the mesh in one process, the coding device 500 can suppress an increase in the load of point cloud data generation. Therefore, for example, the coding device 500 can generate a bit stream at a higher speed. Further, for example, the manufacturing cost of the coding apparatus 500 can be reduced.
[0114]
　　
　Next, an example of the flow of coding processing executed by the coding apparatus 500 will be described with reference to the flowchart of FIG.
[0115]
　When the coding process is started, the Voxel generation unit 511 acquires the point cloud data in step S501.
[0116]
　In step S502, the Voxel generation unit 511 generates voxel data using the point cloud data.
[0117]
　In step S503, the Octree generation unit 521 constructs an Octree using the voxel data and generates Octree data.
[0118]
　In step S504, the Mesh generation unit 522 generates mesh data based on the Octree data.
[0119]
　In step S505, the lossless coding unit 523 losslessly encodes the mesh data and generates coded data of the position information of the point cloud.
[0120]
　In step S506, the Octree decoding unit 531 restores the voxel data using the Octree data generated in step S503.
[0121]
　In step S507, the mesh shape restoration unit 532 restores the mesh shape from the voxel data.
[0122]
　In step S508, the point cloud generation unit 533 executes the point cloud generation process (FIG. 5), and <1. Generation of point cloud> and <2. Point cloud data is generated from the mesh shape by the method described above in the first embodiment>.
[0123]
　In step S509, the Attribute coding unit 514 encodes the attribute information using the point cloud data.
[0124]
　In step S510, the bitstream generation unit 515 generates a bitstream including the coded data of the position information generated in step S505 and the coded data of the attribute information generated in step S509.
[0125]
　In step S511, the bitstream generation unit 515 outputs the bitstream to the outside of the coding device 500.
[0126]
　When the process of step S511 is completed, the coding process is completed.
[0127]
　By executing each process as described above, the coding apparatus 500 can be set to <1. Generation of point cloud> and <2. The effects as described in the first embodiment> can be obtained.
[0128]
　<5. Fourth Embodiment>
　　 In the
　above, in Triangle soup, a point is generated at the intersection of the vector and the surface of the mesh whose starting origin is the position coordinate corresponding to the specified voxel resolution. Explained that point cloud data is generated. Not limited to this, point cloud data may be generated from the mesh at an arbitrary resolution.
[0129]
　For example, as shown in FIG. 13, it is assumed that Octree is applied to a lower resolution layer (LoD = 0 to 2), and Triangle soup is applied to a higher resolution layer. For the tier to which Octree is applied, it is possible to realize resolution scalability in decoding (select which tier resolution to generate point cloud data).
[0130]
　For the lower layers, by setting the vector Vi interval d = 1, Triangle soup can obtain point cloud data with a resolution equivalent to LoD = 4. For example, in the case of FIG. 13, the voxel 601 corresponding to LoD = 2 (the rightmost voxel 601 in the figure) includes one triangular surface 602 of the mesh.
[0131]
　Then, each surface of the voxel 601 is set as a start origin, and a vector Vi603 perpendicular to the surface is set at an interval (d = 1) that divides each side of the voxel 601 into four equal parts. In FIG. 13, only one arrow is signed, but all the arrows in voxel 601 (including the end of voxel 601) are the vector Vi603.
[0132]
　Then, the point 604 located at the intersection of the vector Vi603 and the surface 602 of the mesh is derived. In FIG. 13, only one point is designated, but all the points shown in the voxel 601 (including the end of the voxel 601) are the points 604.
[0133]
　By doing so, point cloud data with a resolution equivalent to LoD = 4 can be obtained.
[0134]
　That is, by setting the vector Vi whose starting origin is the position coordinates corresponding to the specified voxel resolution, the point cloud data of the final resolution can be obtained. The final resolution indicates a predetermined maximum resolution. For example, in the case of coding / decoding, the maximum resolution indicates the resolution of the point cloud data before encoding using Octree, a mesh, or the like.
[0135]
　Here, instead of doing this, if the interval of the vector Vi603 is set to d = 2 (that is, the interval of the vector Vi603 is doubled), the point 604 is as shown in the second voxel 601 from the right in FIG. (Surface 602 and vector Vi603) are derived. In other words, point cloud data with a resolution equivalent to LoD = 3 can be obtained.
[0136]
　FIG. 14 shows the state of the voxel 601 in a plane for the sake of simplification of the description. All solid and dotted lines within voxel 601 (including the ends of voxel 601) parallel to any of the four sides of voxel 601 have a vector Vi603 with a final resolution (eg Lod = 4) spacing (d = 1). It shows. Point 604 located at the intersection of these vectors Vi603 and the surface 602 of the mesh is derived.
[0137]
　In FIG. 14, the vector Vi603 shown by the solid line and the vector Vi603 shown by the dotted line are arranged alternately. That is, the distance between the vectors Vi603 shown by the solid line is d = 2. That is, the vector Vi603 shown by the solid line is the vector Vi603 of a layer one level higher than the final resolution (for example, Lod = 3). By expanding the interval d in this way, the number of vectors Vi603 is reduced, so that the number of points 604 located at the intersection is also reduced. That is, the resolution of the point cloud data is reduced.
[0138]
　As described above, the interval d of the vector Vi makes it possible to derive point cloud data having an arbitrary resolution. Therefore, it is possible to realize the scalability of the resolution of Triangle soup.
[0139]
　The interval d of this vector Vi can be set to any value. For example, the interval d of the vector Vi may be set to a power of 2. By doing so, the scalability of the resolution for each layer of Octree is realized. In other words, it is possible to derive point cloud data with a resolution corresponding to each layer of Octree. For example, if the difference between the desired layer (derivated layer) of Octree and the lowest layer (layer of final resolution) is L (L is a non-negative integer) , the desired level is set to d = 2 L. It is possible to derive point cloud data with a resolution corresponding to the hierarchy.
[0140]
　Note that L may be a negative value. By setting L to a negative value, it is possible to derive point cloud data with a resolution higher than the final resolution.
[0141]
　Further, the value of the interval d of the vector Vi may be other than the power of 2. The interval d of the vector Vi may be an integer or a decimal as long as it has a positive value. For example, by setting the value of the interval d of the vector Vi to a power other than the power of 2, it becomes possible to derive point cloud data having a resolution other than the resolution corresponding to the Octtree hierarchy. For example, by setting the value of the interval d of the vector Vi to 3, point cloud data having a resolution between LoD = 2 and LoD = 3 can be obtained.
[0142]
　　
　For example, in the case of FIG. 14, both the vertical vector Vi603 in the figure and the horizontal vector Vi603 in the figure are identification numbers 0, 2, 4, 6 shown in the figure. , 8 vector Vi603 is adopted as the vector Vi603 one layer higher. In other words, the vectors Vi603 (vector Vi603 shown by the dotted line) of the identification numbers 1, 3, 5, and 7 shown in the figure are thinned out in the upper layer.
[0143]
　In this way, the vector Vi603 (in other words, the vector Vi603 that is thinned out) adopted in the upper layer is in each direction of the vector Vi603 (that is, in each of the three axial directions (x, y, z directions) perpendicular to each other). May be set independently. In other words, the positions of the starting origin of each vector Vi603 in the three axial directions (x, y, z directions) perpendicular to each other may be independent of each other in each direction.
[0144]
　For example, in the case of FIG. 15, as for the vector Vi603 in the vertical direction in the figure, the vector Vi603 of the identification numbers 1, 3, 5, and 7 shown in the figure is adopted as the vector Vi603 one layer higher. On the other hand, as for the vector Vi603 in the horizontal direction in the figure, the vector Vi603 of the identification numbers 0, 2, 4, 6, and 8 shown in the figure is adopted as the vector Vi603 one layer higher.
[0145]
　In other words, in the upper layer, the vertical vector Vi603 (vector Vi603 shown by the dotted line) of the identification numbers 0, 2, 4, 6, and 8 shown in the figure is thinned out. On the other hand, in the upper layer, the horizontal vector Vi603 (vector Vi603 shown by the dotted line) of the identification numbers 1, 3, 5, and 7 shown in the figure is thinned out.
[0146]
　By doing so, the point 604 can be generated at a position different from that in the case of FIG. 14 without changing the resolution of the derived point cloud data.
[0147]
　　
　For example, in the case of FIGS. 14 and 15, half of the vectors in the vertical direction in the figure and the vector Vi603 in the horizontal direction in the figure are thinned out in the upper layer. It is being drawn. That is, the interval d of the vectors Vi is the same in the vertical direction and the horizontal direction in the figure.
[0148]
　In this way, the number of vectors Vi603 (in other words, the vectors Vi603 that are thinned out) adopted in the upper layer is the direction of the vector Vi603 (that is, the three-axis directions (x, y, z directions) perpendicular to each other). It may be set independently for each). In other words, the intervals between the starting origins of each vector Vi603 in the three axial directions (x, y, z directions) perpendicular to each other may be independent of each other in each direction.
[0149]
　For example, in the case of FIG. 16, assuming that only the vector Vi603 shown by the solid line is adopted (the vector Vi603 shown by the dotted line is thinned out), the vector Vi603 in the vertical direction in the figure has identification numbers 0 to 0 to While all the vectors Vi603 of 8 are adopted, only the vectors Vi603 of the identification numbers 0, 2, 4, 6, and 8 shown in the figure are adopted as the vector Vi603 in the horizontal direction in the figure. .. That is, the intervals between the vector Vi603 in the vertical direction in the figure and the vector Vi603 in the horizontal direction in the figure are different from each other. Therefore, the vertical spacing in the figure and the horizontal spacing in the figure of the generated points are different from each other. That is, the resolution of the point cloud data differs between the vertical direction in the figure and the horizontal direction in the figure.
[0150]
　That is, by doing so, it is possible to set the resolution of the point cloud data independently of each other in each direction of the vector Vi603 (that is, in each of the three axial directions (x, y, z directions) perpendicular to each other).
[0151]
　　
　Note that points may be generated at a part of the intersection between the vector Vi and the surface of the mesh. In other words, it does not have to generate points even at intersections. That is, by reducing the number of intersections that generate points, the resolution of the point cloud may be reduced (that is, the scalability of the resolution may be realized).
[0152]
　The method of selecting intersections that generate points (or do not generate points) is arbitrary. For example, as shown in FIG. 17, points may be generated in a staggered pattern (at every other intersection in each of the three axial directions).
[0153]
　By doing so, it is possible to realize resolution scalability regardless of the interval (or the number of vector Vis) of the vector Vis.
[0154]
　　
　A point that is not located at the intersection of the vector Vi and the surface of the mesh may be generated and included in the point cloud data. For example, as shown in FIG. 18, points 611 are generated at positions on the vector Vi that are close to each side of the mesh surface 602 (triangle) even if they are not intersections, and are included in the point cloud data. May be good. In FIG. 18, only one point is coded, but all the points indicated by the white circles are the points 611 generated as described above.
[0155]
　The method of determining the position where the point is generated (in the case of the example of FIG. 18, the method of determining the approximate point from each side) is arbitrary.
[0156]
　By doing so, points can be added independently of the position of the intersection, so that the resolution of the desired portion can be improved more easily. For example, in the case of FIG. 18, by including the approximate points of each side of the surface 602 in the point cloud data, the resolution around each side of the surface 602 can be improved as compared with the other parts. By doing so, the configuration of each side of the surface 602 can be expressed more accurately in the point cloud data. Therefore, the three-dimensional structure represented by the mesh can be represented more accurately in the point cloud data.
[0157]
　　
　Each of the above-described methods in the present embodiment can be applied by combining any plurality of methods. In addition, each of the above-mentioned methods in the present embodiment can be applied in combination with any of the above-mentioned methods in .
[0158]
　　 Further
　, a desired method (or a combination thereof) may be selected and applied from a part or all of the methods described above in the present specification. In that case, the selection method is arbitrary. For example, all applicable patterns may be evaluated and the best may be selected. By doing so, the point cloud data can be generated by the method most suitable for the three-dimensional structure or the like.
[0159]
　　
　Each method described above in the present embodiment also applies to <1. Similar to each method described in> Point cloud generation>, it can be applied to the point cloud generation device 100 described above in the first embodiment. The configuration of the point cloud generator 100 in that case is the same as that described with reference to FIG.
[0160]
　An example of the flow of the point cloud generation process executed by the point cloud generation device 100 in this case will be described with reference to the flowchart of FIG.
[0161]
　When the point cloud generation process is started, the intersection determination unit 112 acquires mesh data in step S601.
[0162]
　In step S602, the vector setting unit 111 is perpendicular to each surface of the voxel (parallel to each side of the voxel), starting from the position coordinates corresponding to the resolution specified by the user, for example. Set the vector Vi.
[0163]
　In step S603, the intersection determination unit 112 determines the intersection between the vector Vi set in step S602 and the mesh surface (triangle) indicated by the mesh data acquired in step S601.
[0164]
　Each process of steps S604 to S607 is executed in the same manner as each process of steps S104 to S107.
[0165]
　When the process of step S607 is completed, the point cloud generation process is completed.
[0166]
　It should be noted that each of the above processes is performed in the same manner as the above-described example in the present embodiment, for example. Therefore, by executing each of the above processes, the point cloud generator 100 can obtain the effects as described in the present embodiment, for example. For example, voxel data of any resolution can be generated from a mesh in a single process. In other words, the scalability of the resolution of point cloud data can be realized.
[0167]
　In addition, it is possible to suppress an increase in the load of point cloud data generation. Therefore, for example, point cloud data can be generated at a higher speed. Further, for example, the manufacturing cost of the point cloud generator 100 can be reduced.
[0168]
　　 Further
　, each method described above in the present embodiment is described in <1. Similar to each method described in Point cloud generation>, it can be applied to the decoding device 300 described above in the second embodiment. The configuration of the decoding device 300 in that case is the same as that described with reference to FIG.
[0169]
　The Point cloud generation unit 314 has the same configuration as the point cloud generation device 100 described above in the present embodiment, and generates point cloud data from the mesh data as described above in the present embodiment.
[0170]
　Therefore, the point cloud generation unit 314 can obtain the same effect as the point cloud generation device 100 of the present embodiment. For example, the Point cloud generation unit 314 can generate voxel data of any resolution from the mesh in one process. In other words, the scalability of the resolution of point cloud data can be realized.
[0171]
　In addition, the Point cloud generation unit 314 can suppress an increase in the load of point cloud data generation. Therefore, the point cloud generation unit 314 can generate point cloud data at a higher speed, for example. Further, for example, the manufacturing cost of the Point cloud generation unit 314 can be reduced.
[0172]
　In this case, the Attribute decoding unit 315 may decode the attribute information in a scalable manner. That is, the resolution scalability of the attribute information may be realized.
[0173]
　Further, the decoding process executed by the decoding device 300 in this case is executed in the same flow as in the case of the second embodiment (FIG. 10). Therefore, the decoding device 300 can obtain the above-mentioned effect (for example, the same effect as the point cloud generation device 100) in the present embodiment.
[0174]
　<6. Note>
　　 The
　above-mentioned series of processes can be executed by hardware or software. When a series of processes are executed by software, the programs constituting the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.
[0175]
　FIG. 20 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.
[0176]
　In the computer 900 shown in FIG. 20, the CPU (Central Processing Unit) 901, the ROM (Read Only Memory) 902, and the RAM (Random Access Memory) 903 are connected to each other via the bus 904.
[0177]
　The input / output interface 910 is also connected to the bus 904. An input unit 911, an output unit 912, a storage unit 913, a communication unit 914, and a drive 915 are connected to the input / output interface 910.
[0178]
　The input unit 911 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like. The output unit 912 includes, for example, a display, a speaker, an output terminal, and the like. The storage unit 913 includes, for example, a hard disk, a RAM disk, a non-volatile memory, or the like. The communication unit 914 includes, for example, a network interface. The drive 915 drives a removable medium 921 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0179]
　In the computer configured as described above, the CPU 901 loads the program stored in the storage unit 913 into the RAM 903 via the input / output interface 910 and the bus 904 and executes the above-described series. Is processed. The RAM 903 also appropriately stores data and the like necessary for the CPU 901 to execute various processes.
[0180]
　The program executed by the computer (CPU901) can be recorded and applied to the removable media 921 as a package media or the like, for example. In that case, the program can be installed in the storage unit 913 via the input / output interface 910 by mounting the removable media 921 in the drive 915.
[0181]
　The program can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting. In that case, the program can be received by the communication unit 914 and installed in the storage unit 913.
[0182]
　In addition, this program can be pre-installed in the ROM 902 or the storage unit 913.
[0183]
　　 The
　case where this technology is applied to the coding / decoding of point cloud data has been described above, but this technology is not limited to these examples, and the code of 3D data of any standard is used. It can be applied to conversion / decryption. That is, as long as it does not contradict the present technology described above, various processes such as coding / decoding methods and specifications of various data such as 3D data and metadata are arbitrary. In addition, some of the above-mentioned processes and specifications may be omitted as long as they do not conflict with the present technology.
[0184]
　The present technology can be applied to any configuration. For example, this technology is a transmitter or receiver (for example, a television receiver or mobile phone) for satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and distribution to terminals by cellular communication, or It can be applied to various electronic devices such as devices (for example, hard disk recorders and cameras) that record images on media such as optical disks, magnetic disks, and flash memories, and reproduce images from these storage media.
[0185]
　Further, for example, in the present technology, a processor as a system LSI (Large Scale Integration) or the like (for example, a video processor), a module using a plurality of processors (for example, a video module), a unit using a plurality of modules (for example, a video unit) Alternatively, it can be implemented as a configuration of a part of the device, such as a set (for example, a video set) in which other functions are added to the unit.
[0186]
　Further, for example, the present technology can also be applied to a network system composed of a plurality of devices. For example, the present technology may be implemented as cloud computing that is shared and jointly processed by a plurality of devices via a network. For example, this technology is implemented in a cloud service that provides services related to images (moving images) to arbitrary terminals such as computers, AV (Audio Visual) devices, portable information processing terminals, and IoT (Internet of Things) devices. You may try to do it.
[0187]
　In the present specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..
[0188]
　　 Systems, equipment, processing departments, etc. to which this technology is applied include, for example, transportation, medical care, crime prevention, agriculture, livestock industry, mining, beauty, factories, home appliances, weather, nature monitoring. It can be used in any field. Moreover, the use is arbitrary.
[0189]
　　 In
　the present specification, the "flag" is information for identifying a plurality of states, and is not limited to information used for identifying two states of true (1) or false (0). Information that can identify three or more states is also included. Therefore, the value that this "flag" can take may be, for example, 2 values ​​of 1/0 or 3 or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be 1 bit or a plurality of bits. Further, the identification information (including the flag) is assumed to include not only the identification information in the bit stream but also the difference information of the identification information with respect to a certain reference information in the bit stream. In, the "flag" and "identification information" include not only the information but also the difference information with respect to the reference information.
[0190]
　Further, various information (metadata, etc.) regarding the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, to make the other data available (linkable) when processing one data. That is, the data associated with each other may be combined as one data or may be individual data. For example, the information associated with the coded data (image) may be transmitted on a transmission path different from the coded data (image). Further, for example, the information associated with the coded data (image) may be recorded on a recording medium (or another recording area of ​​the same recording medium) different from the coded data (image). Good. Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part within the frame.
[0191]
　In addition, in this specification, "synthesize", "multiplex", "add", "integrate", "include", "store", "insert", "insert", "insert". A term such as "" means combining a plurality of objects into one, for example, combining encoded data and metadata into one data, and means one method of "associating" described above.
[0192]
　Further, the embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.
[0193]
　For example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). On the contrary, the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Further, of course, a configuration other than the above may be added to the configuration of each device (or each processing unit). Further, if the configuration and operation of the entire system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). ..
[0194]
　Further, for example, the above-mentioned program may be executed in any device. In that case, the device may have necessary functions (functional blocks, etc.) so that necessary information can be obtained.
[0195]
　Further, for example, each step of one flowchart may be executed by one device, or may be shared and executed by a plurality of devices. Further, when a plurality of processes are included in one step, the plurality of processes may be executed by one device, or may be shared and executed by a plurality of devices. In other words, a plurality of processes included in one step can be executed as processes of a plurality of steps. On the contrary, the processes described as a plurality of steps can be collectively executed as one step.
[0196]
　Further, for example, in a program executed by a computer, the processing of the steps for writing the program may be executed in chronological order in the order described in the present specification, or may be executed in parallel or in calls. It may be executed individually at the required timing such as when it is broken. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-mentioned order. Further, the processing of the step for writing this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.
[0197]
　Further, for example, a plurality of techniques related to this technique can be independently implemented independently as long as there is no contradiction. Of course, any plurality of the present technologies can be used in combination. For example, some or all of the techniques described in any of the embodiments may be combined with some or all of the techniques described in other embodiments. It is also possible to carry out a part or all of any of the above-mentioned techniques in combination with other techniques not described above.
[0198]
　The present technology can also have the following configurations.
　(1) An
　image processing device including a point cloud generator that generates point cloud data by arranging points at the intersection of a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution .
　(2) 　The image processing apparatus according 　　to (1), wherein the point cloud generation unit
　　determines an intersection between the surface and the vector, and when it is determined that the surface intersects the vector
, calculates the coordinates of the intersection
.
　(3)
　The image processing apparatus according to (2), wherein the point cloud generation unit determines the intersection of the vector in each of the positive and negative directions in the three axial directions perpendicular to each other and the surface .
　(4)
　The image processing according to (3), wherein when the coordinate values ​​of a plurality of intersections overlap, the point cloud generation unit leaves one of the overlapping intersections and deletes the other intersections. apparatus.
　(5) The point cloud generation unit according
　to any one of (2) to (4), which determines the intersection of the vector whose start origin is located within the range of each vertex of the surface and the surface . Image processing device.
　(6) When the calculated coordinates of the intersection are outside the bounding box, the point cloud generator clips the coordinates of the intersection into the bounding box.
　The image processing apparatus according to any one of (2) to (5).
　(7)
　The image processing device according to any one of (2) to (6), wherein the point cloud generation unit deletes the intersection when the calculated coordinates of the intersection are outside the bounding box .
　(8) The point cloud generation unit uses the vector, which is sparser than the case of the intersection determination for the end portion of the surface, to perform the intersection determination for the inside of the surface according
　to any one of (2) to (7). The image processing apparatus described.
　(9) The point cloud generation unit adds points to the space when the vector intersects the plurality of surfaces and a space exists between the plurality of surfaces
　(2) to (8). ). The image processing apparatus according to any one of.
　(10)
　The image processing apparatus according to any one of (2) to (9), wherein the point cloud generation unit determines the intersection of a plurality of the vectors with respect to one surface in parallel with each other .
　(11)
　The image processing apparatus according to any one of (2) to (10), wherein the point cloud generation unit determines the intersection of a plurality of the surfaces with respect to one vector in parallel with each other .
　(12)
　The image processing apparatus according to any one of (2) to (11), wherein the vector has position coordinates corresponding to a designated voxel resolution as a starting origin .
　(13)
　The image processing apparatus according to any one of (2) to (12), wherein the vector has a position coordinate corresponding to a power of 2 of a specified voxel resolution as a starting origin .
　(14) 　The image processing apparatus according to any one of (2) to (13),
　wherein the positions of the start origins of the vectors in the three axial directions perpendicular to each other are independent of each other
.
　(15)
　The image processing apparatus according to any one of (2) to (14), wherein the distance between the start origins of the vectors in the three axial directions perpendicular to each other is independent of each other .
　(16)
　The image processing device according to any one of (2) to (15), wherein the point cloud generation unit includes points not located at the intersection in the point cloud data .
　(17) A mesh shape restoration unit that restores the shape of the mesh from voxel data is further provided, and the
　point cloud generation unit uses the intersection of the surface and the vector restored by the mesh shape restoration unit as a point.
　The image processing apparatus according to any one of (2) to (16) for generating cloud data .
　(18) A lossless decoding unit that reversibly decodes a bit stream to
　generate Octree data and an Octtree decoding unit that generates the voxel data using the Occtree data generated by the reversible decoding unit
　are further provided.

　The image processing apparatus according 　to (17), wherein the mesh shape restoration unit restores the shape of the mesh from the voxel data generated by the Octree decoding unit .
　(19)
　An Octree that generates the voxel data using a position information coding unit that encodes the position information of the point cloud data and the Octree data that the position information coding unit generates when encoding the position information. 　The image processing apparatus according to (17),
　further including a decoding unit
.
　(20) An
　image processing method for generating point cloud data by arranging points at intersections between a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution .
Code description
[0199]
　100 point cloud generator, 111 vector setting unit, 112 intersection judgment unit, 113 auxiliary processing unit, 114 output unit, 300 decoding unit, 311 reversible decoding unit, 312 Octree decoding unit, 313 Mesh shape restoration unit, 314 Point cloud generator , 315 Attribute decoding unit, 500 encoding device, 511 Voxel generator, 512 Geometry encoding unit, 513 Geometry decoding unit, 514 Attribute encoding unit, 515 bitstream generator, 521 Octree generator, 522 Mesh generator, 523 Reversible coding part, 531 Octree decoding part, 532 Mesh shape restoration part, 533 Point cloud generation part
The scope of the claims
[Claim 1]
　An
　image processing device including a point cloud generator that generates point cloud data by arranging points at the intersection of a mesh surface and a vector whose starting origin is a position coordinate corresponding to a specified resolution .
[Claim 2]
　The image processing device according 　　to claim 1, 　wherein the point cloud generation unit
　　determines an intersection between the surface and the vector, and when it is determined that the plane
intersects, calculates the coordinates of the intersection
.
[Claim 3]

　The image processing device according to claim 2, 　wherein the point cloud generation unit determines the intersection of the vector in each of the positive and negative directions in the three axial directions perpendicular to each other and the surface .
[Claim 4]

　The image processing device according 　to claim 3, wherein when the coordinate values ​​of a plurality of intersections overlap, the point cloud generation unit leaves one of the overlapping intersections and deletes the other intersections .
[Claim 5]

　The image processing device according to claim 2, 　wherein the point cloud generation unit determines the intersection of the vector whose start origin is located within the range of each vertex of the surface and the surface .
[Claim 6]

　The image processing device according 　to claim 2, wherein the point cloud generation unit clips the coordinates of the intersection into the bounding box when the calculated coordinates of the intersection are outside the bounding box .
[Claim 7]

　The image processing device according 　to claim 2, wherein the point cloud generation unit deletes the intersection when the calculated coordinates of the intersection are outside the bounding box .
[Claim 8]

　The image processing apparatus according to claim 2, 　wherein the point cloud generation unit uses the vector, which is sparser than the case of the intersection determination with respect to the end portion of the surface, to perform the intersection determination with respect to the inside of the surface .
[Claim 9]

　The image processing apparatus according 　to claim 2, wherein the point cloud generation unit adds points to the space when the vector intersects the plurality of surfaces and a space exists between the plurality of surfaces. ..
[Claim 10]

　The image processing apparatus according to claim 2, 　wherein the point cloud generation unit determines the intersection of a plurality of the vectors with respect to one surface in parallel with each other .
[Claim 11]

　The image processing apparatus according to claim 2, 　wherein the point cloud generation unit determines the intersection of a plurality of the surfaces with respect to one vector in parallel with each other .
[Claim 12]

　The image processing apparatus according 　to claim 2, wherein the vector has a position coordinate corresponding to a designated voxel resolution as a starting origin .
[Claim 13]

　The image processing apparatus according 　to claim 2, wherein the vector has a position coordinate corresponding to a power of 2 of a specified voxel resolution as a starting origin .
[Claim 14]

　The image processing apparatus according to claim 2, 　wherein the positions of the start origins of the vectors in the three axial directions perpendicular to each other are independent of each other .
[Claim 15]

　The image processing apparatus according to claim 2, 　wherein the distance between the start origins of the vectors in the three axes perpendicular to each other is independent of each other .
[Claim 16]

　The image processing device according to claim 2, 　wherein the point cloud generation unit includes points not located at the intersection in the point cloud data .
[Claim 17]
　A mesh shape restoration unit that restores the shape of the mesh from voxel data is further provided, and the
　point cloud generation unit uses the point cloud data at the intersection of the surface and the vector restored by the mesh shape restoration unit as a point.
　The image processing apparatus according to claim 2 to be generated .
[Claim 18]
　The 　mesh shape restoration unit 　further includes a reversible decoding unit that reversibly decodes a bit stream to
　generate Octree data, and an Octree decoding unit that generates voxel data using the Octree data generated by the reversible decoding unit. 　17 is the image processing apparatus according to claim 17, wherein the mesh shape is restored from the voxel data generated by the Octree decoding unit .

[Claim 19]
　A position information coding unit that encodes the position information of the point cloud data, and
　an Octree decoding unit that generates the voxel data using the Octree data generated when the position information coding unit encodes the position information. 　The image processing apparatus according to claim 17,
　further comprising
.
[Claim 20]
　An
　image processing method that generates point cloud data by arranging points at the intersection of a mesh surface and a vector whose starting origin is the position coordinates corresponding to the specified resolution .