JP2018116645A - Image processing system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of curtailing generation of noise in a fine shape of an edge part in a shape expressed by three-dimensional shape data.SOLUTION: The image processing system includes: selection means for selecting a correction object vertex to be corrected in each vertex in a shape expressed by three-dimensional shape data including normal information and a peripheral vertex near the correction object vertex; rotation means for rotating a normal vector in the peripheral vertex; calculation means for calculating an inner product between a vector from the peripheral vertex towards a candidate position of the movement destination of the correction object vertex and the rotated vector; determination means for determining the position of the movement destination of the correction object vertex on the basis of the calculated inner product; and correction means for moving the correction object vertex to the determined position of the movement destination.SELECTED DRAWING: Figure 10

Description

  The present invention relates to a technique for optimizing shape data representing a three-dimensional shape.

  A technique such as a multi-view stereo method for reproducing a three-dimensional surface shape (hereinafter also simply referred to as “shape”) of a subject from a plurality of images (multi-view images) obtained by viewing the same subject from different viewpoints is known. In addition, normal information on the surface shape of the subject is acquired from a plurality of images obtained by viewing the same subject irradiated from different light source directions from the same viewpoint, and the shape of the subject is reproduced based on the acquired normal information. Techniques such as the photometric stereo method are also known. In recent years, it has become possible to acquire normal information on the surface shape of an object with high accuracy, and to reproduce a fine shape with a finer scale, such as a fine relief pattern.

  Patent Document 1 discloses a technique for optimizing shape data representing the shape of a subject. Specifically, for the shape represented by the input shape data, the consistency between the normal information and the position information at each vertex is evaluated, and the movement destination of each vertex is sequentially searched based on the evaluation result. Correction is performed.

US Patent Application Publication No. 2012/0268571

  In the technique of Patent Document 1, an inner product of a vector indicating the positional relationship between a target vertex and a peripheral vertex adjacent to the target vertex and a normal vector at the peripheral vertex is calculated, and the larger the calculated inner product is, the larger the calculated vertex product is. It is evaluated that the position is away from the ideal position. That is, it is estimated that the larger the calculated inner product is, the higher the possibility of noise occurring in the shape near the target vertex, and the position of the target vertex is moved in a direction closer to the ideal position.

  However, in the technique of Patent Document 1, it is sometimes difficult to correct the fine shape at the edge portion that is the boundary of the angle change among the shapes represented by the input shape data. This is because the calculated inner product may be small even though the position of the target vertex is far from the ideal position. For this reason, the fine shape is not properly corrected, and noise may occur in the shape represented by the output shape data.

  In view of the above problems, the shape correction method of the present invention aims to suppress the occurrence of noise in the fine shape of the edge portion.

  The image processing apparatus of the present invention includes a selection unit that selects a correction target vertex to be corrected at each vertex in the shape represented by the three-dimensional shape data including normal line information and a peripheral vertex adjacent to the correction target vertex; Rotating means for rotating normal vectors at peripheral vertices, calculating means for calculating an inner product of the vector directed from the peripheral vertices to a candidate position to which the correction target vertex is moved, and the rotated vector, the calculated And determining means for determining the movement destination position of the correction target vertex based on the inner product, and correction means for moving the correction target vertex to the determined movement destination position.

  The shape correction method of the present invention has an effect that generation of noise in the fine shape of the edge portion can be suppressed.

It is a figure which shows the hardware constitutions of the image processing apparatus in embodiment. It is a figure which shows the software function structure of the image processing apparatus in embodiment. It is a flowchart which shows the procedure of the shape correction process in embodiment. It is a schematic diagram which shows an example of the positional relationship between the correction target vertex s and the peripheral vertex t in the embodiment. It is a figure explaining the principle of the conventional evaluation value calculation method. It is a figure explaining the principle of the evaluation value calculation method of this embodiment. FIG. 7A is a distribution diagram of evaluation values calculated by a conventional evaluation value calculation method. FIG. 7B is a gradient diagram showing the slope of the distribution of FIG. 7A on the XY plane. FIG. 8A is a distribution diagram of evaluation values calculated by the evaluation value calculation method of the present embodiment. FIG. 8B is a gradient diagram showing the slope of the distribution of FIG. 8A on the XY plane. In this embodiment, it is an example of sectional drawing of the shape which input shape data represents. It is a figure which shows the effect at the time of applying the shape correction method of this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and all the combinations of features described in the present embodiment are not necessarily essential to the solution means of the present invention. In addition, about the same structure, the same code | symbol is attached | subjected and demonstrated.

[Embodiment 1]
In the present embodiment, the shape (particularly the fine shape) represented by the input shape data is corrected using normal line information included in the three-dimensional shape data (hereinafter also simply referred to as “shape data”) input to the image processing apparatus 100. An embodiment to be described will be described.

(Hardware configuration)
FIG. 1 is a block diagram illustrating a hardware configuration of the image processing apparatus 100 according to the present embodiment. The image processing apparatus 100 includes a CPU 101, a RAM 102, a ROM 103, a storage unit 104, an input interface 105, an output interface 106, and a system bus 107. The CPU 101 is a processor, and reads a program stored in the ROM 103 into the RAM 102 and executes it. When the CPU 101 executes the program, various processes in the image processing apparatus 100 described later are executed. The storage unit 104 is a storage device, and stores image data used for processing in the image processing apparatus 100, parameters used for processing, and the like. As the storage unit 104, for example, a secondary storage device such as an HDD (Hard Disk Drive), an optical disk drive, or a flash memory is applied.

  The input interface 105 is a serial bus interface such as USB or IEEE1394. The image processing apparatus 100 acquires shape data, normal map (normal information), and the like to be processed from an external memory 108 (for example, a memory card, CF card, SD card, USB memory) via the input interface 105. can do. Alternatively, the input interface 105 may have a network interface function such as Ethernet (registered trademark). In this case, the image processing apparatus 100 can receive shape data, a normal map, and the like to be processed from an external apparatus (such as a PC) that is communicably connected via a network. The output interface 106 is a serial bus interface such as USB or IEEE1394. The image processing apparatus 100 can output shape data subjected to predetermined processing to the external memory 109 via the output interface 106. Alternatively, the output interface 106 may have a network interface function such as Ethernet. In this case, the image processing apparatus 100 can transmit shape data that has been subjected to predetermined processing to an external apparatus via a network. Note that the components of the image processing apparatus 100 exist in addition to those described above.

(Software function configuration)
FIG. 2 is a block diagram illustrating a software functional configuration of the image processing apparatus 100 according to the present embodiment. The functions of the blocks in FIG. 2 are realized by the CPU 101 reading out the program code stored in the ROM 103 to the RAM 102 and executing it. Alternatively, some or all of the blocks in FIG. 2 may be realized by hardware such as an ASIC or an electronic circuit.

  In the image processing apparatus 100 according to the present embodiment, the data acquisition unit 201 reads shape information and normal information from the storage unit 104 or the like. The correction target vertex selection unit 202 selects a target vertex s (hereinafter also referred to as “correction target vertex s”) to be corrected from the read shape information and normal line information. The peripheral vertex selection unit 203 selects a peripheral vertex t adjacent to the correction target vertex s on the side based on information indicating the adjacent relationship of the vertex group. The normal rotation unit 204 rotates a normal vector extending in the normal direction from the peripheral vertex t by a predetermined angle around the position of the peripheral vertex. The evaluation value calculation unit 205 calculates an evaluation value corresponding to the candidate position based on the vector from the corrected vertex t toward the candidate position to which the correction target vertex s is moved and the rotated rotation vector. The evaluation value comparison unit 206 compares the evaluation values corresponding to the candidate positions, and extracts the smallest value among the calculated evaluation values. The movement destination determination unit 207 determines the position of the movement destination of the correction target vertex s based on the extracted evaluation value having the smallest value. The shape correcting unit 208 moves the correction target vertex s to the destination position. The data output unit 209 outputs the shape data in which all the vertices are moved (corrected) as output shape data.

(Flowchart showing the procedure of shape correction processing)
FIG. 3 is a flowchart showing the procedure of the shape correction process in this embodiment. 3 is performed by the CPU 101 reading out the program code stored in the ROM 103 to the RAM 102 and executing it. Each symbol S in the description of FIG. 3 means a step in the flowchart.

  In S301, the data acquisition unit 201 acquires shape information and normal information. Specifically, the data acquisition unit 201 stores three-dimensional shape data representing a predetermined shape in a three-dimensional space, and normal data indicating normals at each vertex, each side, and each surface of the shape. Read from 104. Alternatively, the data acquisition unit 201 may read the three-dimensional shape data and normal data from the external memory 108 via the input interface 105. In the present embodiment, mesh data that is shape data and normal line data and that can be processed by the image processing apparatus 100 is read from the storage unit 104. The mesh data is data representing a set of vertices, sides, and faces, and each vertex, each side, and each face has a normal vector that is a vector of length one. Further, the CPU 101 can acquire data indicating the adjacency relationship of the vertex group indicating which vertex a certain vertex is adjacent to by analyzing the mesh data. For example, the mesh data according to the present embodiment can be data in which a normal vector is assigned to each vertex, each side, and each surface by texture mapping, which is generated from point cloud data by a surface reconstruction technique. In the present embodiment, data obtained by adding a normal vector to lattice polygon data representing the surface shape of a subject is used as three-dimensional shape data.

  In S302, the correction target vertex selection unit 202 selects the target vertex s to be corrected from the read mesh data. In S302, the correction target vertex selection unit 202 also acquires the position information of the target vertex s and the normal vector extending in the normal direction from the target vertex s. In the initial state in which S302 is executed for the first time, the position information of the top left vertex and the normal vector at the top left vertex of the shape represented by the read mesh data are selected. The correction target vertex selection unit 202 of this embodiment sequentially selects each correction target vertex s included in the mesh data in the main scanning direction.

  In S303, the peripheral vertex selection unit 203 selects the vertex t where the correction target vertex s is adjacent on the side based on the target vertex s acquired in S302 and the data indicating the adjacent relationship of the vertex group. In this specification, the vertex t adjacent to the correction target vertex s in this way is referred to as a “peripheral vertex”. Then, the peripheral vertex selection unit 203 analyzes the mesh data to acquire position information of the peripheral vertex t and a normal vector extending in the normal direction from the peripheral vertex t. In the present embodiment, in the shape data (mesh data), the number of sides extending from all vertices is unified, and the quadrilateral formed by the sides extending from the vertices is configured according to a predetermined rule. Therefore, the peripheral vertex selection unit 203 selects, as peripheral vertices, four vertices that are adjacent vertically and horizontally around the target vertex s. As described above, in this embodiment, four vertices that are adjacent in the vertical and horizontal directions around the target vertex s are selected as peripheral vertices, but the number of peripheral vertices t varies depending on the definition of the polygon in the shape data.

  In S304, the normal rotation unit 204 rotates the normal vector at the peripheral vertex t by θ based on the position information of the correction target vertex s and the position information and normal vector of the peripheral vertex t. Rt is calculated. At this time, the rotation axis vt used when calculating the rotation matrix Rt of the normal vector at the peripheral vertex t is determined by the following equation.

  Here, ps represents the position of the correction target vertex s in the three-dimensional space, pt represents the position of the peripheral vertex t, and dt represents a normalized vector from the peripheral vertex t toward the correction target vertex s. Further, ns represents a normalized normal vector extending from the correction target vertex s in the normal direction, and nt represents a normalized normal vector extending from the peripheral vertex t in the normal direction.

  FIG. 4 is a diagram schematically illustrating an example of a positional relationship between the correction target vertex s and two peripheral vertices t1 and t2 adjacent to the correction target vertex s in the present embodiment. In FIG. 4, a shape 401 is an example of a shape represented by three-dimensional shape data (mesh data). The shape 401 includes a correction target vertex s and two peripheral vertices t1 and t2 adjacent to the correction target vertex s. A normal vector ns extends from the position ps of the correction target vertex s. A normal vector nt1 and a vector dt1 extending to the position ps of the correction target vertex s extend from the position pt1 of the peripheral vertex t1. Similarly, a normal vector nt2 and a vector dt2 toward the position Ps of the correction target vertex s extend from the position pt2 of the peripheral vertex t2. The shape correction method of the present embodiment is based on the shape information (vectors indicating the positions of the vertices and the positional relationship between the vertices) and the normal information (normal vectors at the vertices). The shape to represent can be modified. In the example of FIG. 4, a shape 401 when the XZ plane direction is viewed around the Y axis is shown.

  When the normal vector nt is rotated θ around the rotation axis vt calculated by Equation 1, the rotation matrix Rt (θ) is determined by the following equation (Rodriguez formula).

  In the present embodiment, when the normal vector nt is rotated 90 degrees in the positive direction, the rotation matrix Rt (θ) is as follows.

  A vector n′t obtained by rotating the normal vector nt by 90 degrees can be calculated by the calculation using Expression 5.

  A method of calculating a vector n′t obtained by rotating the normal vector nt by 90 degrees by calculation using Expression 5 will be described with reference to FIG. In FIG. 6, the positional relationship between the correction target vertex s and two peripheral vertices t1 and t2 adjacent to the correction target vertex s is the same as in FIG. In the example of FIG. 6, the normal vector nt2 at the position pt2 of the peripheral vertex t2 is rotated 90 degrees by the calculation using Expression 5, and becomes the rotation vector n′t2.

  In the present embodiment, the rotation direction in which the normal vector nt at the peripheral vertex t is rotated is determined according to the positional relationship between the position ps of the correction target vertex s and the position pt of the peripheral vertex t. Specifically, the normal vector nt2 at the peripheral vertex t2 is rotated around the position pt2 of the peripheral vertex t2 on the plane defined by the normal vector nt2 and the vector dt2 from the peripheral vertex t2 toward the correction target vertex s. The The normal rotation unit 204 of the present embodiment rotates the normal vector nt2 by 90 degrees in the main scanning direction, which is the direction in which the correction target vertex s is selected. However, the rotation direction of the normal vector nt is not limited to the above embodiment. As shown in FIG. 6, for example, when the XZ plane is centered on the Y axis, the normal vector nt is rotated in the positive direction for the peripheral vertex t positioned first in the main scanning direction, and is positioned last. For the peripheral vertex t, the normal vector nt may be rotated in the negative direction.

  In S305, the evaluation value calculation unit 205 determines the candidate position ps + σ, ps, corrected for the correction target vertex s based on the rotation vector n′t, the position pt of the peripheral vertex t, and the position ps of the correction target vertex s. Evaluation values χ1, χ2, and χ3 are calculated for each of ps−σ. The evaluation value χ corresponding to the corrected candidate position p of the correction target vertex s is expressed by the following equation.

  Here, in Expression 7, A and B each represent a constant. In practice, A and B may each be “1”. If “1” is not applied to A and B, the values of A and B can be adjusted with reference to the output of Expression 7.

  In S306, the evaluation value comparison unit 206 compares the evaluation values χ1, χ2, and χ3 corresponding to the corrected candidate positions ps + σ, ps, and ps−σ calculated in S305, and the evaluation value χ1. , Χ2, χ3, the minimum value is extracted.

  In S307, the movement destination determination unit 207 determines the movement destination of the correction target vertex s based on the evaluation value extracted in S306. The movement destination position ps ′ of the correction target point s can be calculated by the following equation.

  In S308, the shape correcting unit 208 moves the correction target vertex s to the movement destination position ps'.

  In S309, it is determined whether or not all the vertices included in the shape data have been corrected. If all the vertices have been corrected (S309: YES), the data output unit 209 outputs corrected three-dimensional shape data with the position coordinates of the movement destination position ps' as vertices. On the other hand, when all the vertices included in the shape data have not been corrected (S309: NO), the process returns to S302 and the shape correction process for the next target vertex s is repeated.

(Principle of evaluation value calculation method)
FIG. 5 is a diagram for explaining the principle of the evaluation value calculation method in the conventional shape correction processing. The conventional evaluation value calculation unit 205 calculates the inner product of the normal vector nt at the peripheral vertex t and the vector dt from the position pt of the peripheral vertex t toward the position ps of the correction target vertex s as the evaluation value. In FIG. 5, the positional relationship between the correction target vertex s and the two peripheral vertices t1 and t2 adjacent to the correction target vertex s is the same as in FIG. As shown in FIG. 5, the inner product of the normal vector nt2 at the peripheral vertex t2 and the vector dt2 from the position pt2 of the peripheral vertex t2 to the position ps of the correction target vertex s is “0.25”, and this value is Used for evaluation value. Then, the conventional destination determination unit 207 corresponds to the smallest evaluation value among the evaluation values χ1, χ2, and χ3 corresponding to the candidate positions ps + σ, ps, and ps−σ after the correction of the correction target vertex s. The candidate position is determined as the movement destination. For example, in the positional relationship of FIG. 5, when the smallest evaluation value is “0.2”, the conventional movement destination determination unit 207 moves the position of the correction target vertex s in the ideal position (FIG. 5) direction on the search axis. The position where the inner product of nt2 and dt2 is “0.2” is determined as the destination vertex. 5 and 6, the “search axis” is an axis determined by the ideal position and the position ps of the correction target vertex s, and the evaluation value calculation unit 205 of the present embodiment uses the candidate position ps + σ. , Ps, ps-σ are acquired from the position on the search axis. 5 and FIG. 6, the “ideal position” is determined according to the positional relationship between the position ps of the correction target vertex s and the position pt of the peripheral vertex t. Specifically, in the plane defined by the normal vector nt2 and the vector dt2 from the peripheral vertex t2 toward the correction target vertex s, the inner product of nt2 and dt2 is “0”.

  However, in the conventional evaluation value calculation method, there is a case where it is difficult to correct the fine shape at the edge portion that is the boundary of the angle change. An example in the case where the edge portion cannot be corrected is shown in FIG. When the fine shape is corrected in the edge portion, the selected correction target vertex s 'and the peripheral vertices t1 and t2 adjacent to the correction target vertex s' may be positioned as shown in FIG. More specifically, the correction target vertex s ′ may be located beyond the normal line of the peripheral vertex t2 (or the peripheral vertex t1) with the ideal position as a reference. The example of FIG. 5 shows that the correction target vertex s ′ is located beyond the normal vector nt2 direction of the peripheral vertex t2 with the ideal position as a reference. In this case, the inner product of the normal vector nt2 and the vector d′ t2 from the position pt2 of the peripheral vertex t2 toward the position ps ′ of the correction target vertex s ′ is “0.75”, and this value is used as the evaluation value. It is done. The evaluation value becomes lower as the position ps ′ of the correction target vertex s ′ is farther from the ideal position. As a result, the position ps 'of the correction target vertex s' is moved in a direction away from the ideal position (for example, a position where the inner product of nt2 and d't2 is "0.5"). If the correction target vertex s ′ moves in a direction away from the ideal position, the noise included in the shape represented by the input shape data is not removed, and a problem such as occurrence in the shape represented by the output shape data occurs. May end up.

  FIG. 6 is a diagram for explaining the principle of the evaluation value calculation method in the present embodiment. In FIG. 6, the positional relationship between the correction target vertex s and two peripheral vertices t1 and t2 adjacent to the correction target vertex s is the same as in FIG. Similarly, the positional relationship between the correction target vertex s 'and the two peripheral vertices t1 and t2 adjacent to the correction target vertex s' is the same as in FIG. The evaluation value calculation unit 205 of the present embodiment includes a rotation vector n′t2 obtained by rotating the normal vector nt2 by 90 degrees in the positive direction, and a vector dt2 from the position pt2 of the peripheral vertex t2 toward the position ps of the correction target vertex s. The inner product of is calculated. As shown in FIG. 6, the inner product of n′t2 and dt2 is “0.14”, and this value is used as the evaluation value. Although the embodiment using the inner product of n′t2 and dt2 as the evaluation value has been described, the correction target vertex, such as the sum of the inner product of n′t1 and dt1 and the inner product of n′t2 and dt2, is described. The sum of inner products calculated for each peripheral vertex t of s may be used as the evaluation value.

  On the other hand, when the correction target vertex s ′ is located beyond the normal direction of the peripheral vertex t2 with respect to the ideal position, the calculated evaluation value is as follows. The evaluation value calculation unit 205 of the present embodiment calculates the inner product of the rotation vector n′t2 and the vector d′ t2 from the position pt2 of the peripheral vertex t2 toward the position ps ′ of the correction target vertex s ′. As shown in FIG. 6, the inner product of the rotation vector n′t2 and the vector d′ t2 is “1.5”, and this value is used as the evaluation value. Thus, in this embodiment, since the evaluation value is calculated using the rotation vector n′t2, the evaluation value decreases as the position of the correction target vertex s approaches the ideal position, and the position of the correction target vertex s is ideal. The evaluation value increases as the distance from the position increases. As a result, the movement destination determination unit 207 of this embodiment can determine a candidate position close to the ideal position direction as the position of the movement destination vertex of the correction target vertex s.

  FIG. 7A is a distribution diagram of evaluation values calculated by a conventional evaluation value calculation method. In the XYZ coordinate system of FIG. 7A, the X axis, the Y axis, and the Z axis are coordinate axes respectively indicating the X coordinate, the Y coordinate, and the Z coordinate of the position ps of the correction target vertex s. The distribution 701 is shown in the XYZ coordinate system. In FIG. 7A, the position pt1 of the peripheral vertex t1 is fixed at (-1, 0, 0), the position pt2 of the peripheral vertex t2 is fixed at (1, 0, 0), and the ideal position is (0, 0, 0). An example in the case of 0) is shown. When the position ps is arranged at an arbitrary position with Z coordinates ≧ 0, a point group of evaluation values (inner product of the vector nt and the vector dt) corresponding to each of them constitutes the distribution 701.

  The slope of the distribution 701 indicates that the evaluation value increases or decreases. Of the slopes of the distribution 701, an arrow 702 indicates a downward slope toward the origin in the distribution 701. This indicates that the evaluation value constituting the downward slope indicated by the arrow 702 decreases as the origin approaches the XYZ coordinate system. That is, the arrow 702 indicates that the position ps of the correction target vertex corresponding to the evaluation value constituting the downward slope (for example, the position ps where the evaluation value is “0.25” in FIG. 5) is obtained by the conventional shape correction processing. It means moving in the direction approaching the ideal position.

  However, in the distribution of evaluation values calculated by the conventional evaluation value calculation method, a downward slope in a direction different from the origin direction may be formed. An arrow 703 indicates a downward slope in the distribution 701 in a direction different from the origin direction. The evaluation value constituting the downward inclination in the direction different from the origin direction decreases as the distance from the origin of the XYZ coordinate system increases. That is, the arrow 703 indicates that the position ps of the correction target vertex corresponding to the evaluation value (for example, the position ps ′ where the evaluation value is “0.75” in FIG. 5) is changed from the ideal position by the conventional shape correction processing. It means moving in the direction of leaving.

  FIG. 7B is a gradient diagram showing the slope of the distribution 701 in FIG. 7A on the XY plane with the Z axis as the center. Each thin line arrow included in the gradient diagram of FIG. 7B represents the direction of inclination in the distribution 701 and the inclination of the inclination. A coordinate point 704 represents the position ps of the correction target vertex whose position coordinates are (0, 1, 1). In the conventional shape correction process, the coordinate point 704 is moved so as to approach the origin direction according to the inclination represented by the thin line arrow. A coordinate point 705 represents the position ps of the correction target vertex whose position coordinates are (2, 2, 2). In the conventional shape correction process, it can be seen that the coordinate point 705 is moved in a direction away from the origin direction according to the inclination represented by the thin line arrow. A region where the downward slope of the distribution 701 is formed in a direction away from the origin is represented by a hatched portion in FIG.

  FIG. 8A is a distribution diagram of evaluation values calculated by the evaluation value calculation method of the present embodiment. In the XYZ coordinate system of FIG. 8A, the meanings of the coordinate axes of the X axis, the Y axis, and the Z axis, and the point group of evaluation values corresponding to the position ps constitute a distribution. ). Unlike the conventional method, in the evaluation value distribution 801 calculated by the evaluation value calculation method of the present embodiment, all the downward inclinations are directed toward the origin. As shown in FIG. 8A, any arrow 802 indicates a downward slope toward the origin in the distribution 801. That is, the position ps of the correction target vertex is the position before correction (for example, the position ps where the evaluation value is “0.14” in FIG. 6 or the position ps ′ where the evaluation value is “1.5”). Regardless, this means that the shape is moved in a direction approaching the ideal position by the shape correction processing of the present embodiment.

  FIG. 8B is a gradient diagram showing the slope of the distribution 801 in FIG. 8A on the XY plane with the Z axis as the center. The thin line arrows included in the gradient diagram of FIG. 8B represent the direction of inclination in the distribution 801 and the steepness of the inclination. A coordinate point 803 represents the position ps of the correction target vertex having the same position coordinate (0, 1, 1) as the coordinate point 704. Similar to the conventional shape correction processing, the coordinate point 803 is moved so as to approach the origin direction according to the inclination represented by the thin line arrow. A coordinate point 804 represents the position ps of the correction target vertex having the same position coordinate (2, 2, 2) as the coordinate point 705. In the shape correction process of the present embodiment, the coordinate point 804 is also moved so as to approach the origin direction according to the inclination represented by the thin line arrow. As described above, according to the shape correction method of the present embodiment, the position ps of the correction target vertex is moved in the direction approaching the ideal position regardless of the position before the correction. As a result, the image processing apparatus 100 of the present embodiment can appropriately execute the fine shape correction in the edge portion, and can suppress the occurrence of noise in the shape represented by the output shape data.

  The effect when the shape correction method of this embodiment is applied will be described with reference to FIGS. FIG. 9 is an example of a sectional view of the shape represented by the input shape data in the present embodiment. The shape shown in FIG. 9 is a shape in which a plurality of images (multi-view images) obtained by viewing the same subject from different viewpoints are reproduced by a multi-view stereo method or the like. The example of FIG. 9 shows a cross-sectional view of the shape of a “human face” reproduced from a subject. Further, normal line information corresponding to vertices, sides, and faces constituting the shape of the “human face” is represented by arrows in the graph of FIG. 9.

  FIG. 10 is a diagram illustrating a result of correcting a fine shape using normal line information acquired from input shape data. FIG. 10A illustrates a correction result when a conventional shape correcting method is applied. 10 (b) shows the correction results when the shape correction method of this embodiment is applied. In FIG. 10A, it can be seen that a steep angle change occurs in the solid line showing the cross section of the corrected shape. Thus, a region including a steep angle change in a fine shape is visually recognized as noise. On the other hand, in FIG. 10B, a steep angle change does not occur in the solid line indicating the cross section of the corrected shape. That is, it can be seen that the generation of noise is suppressed in the fine shape.

[Embodiment 2]
In the shape correction method according to the first embodiment, the inner product of the rotation vector n′t and the vector dt from the correction vertex t toward the candidate position of the correction target vertex s is calculated, and the calculated inner product is used as the evaluation value. It was done. However, when the CPU 101 executes shape correction processing based on a predetermined program, evaluation values other than the inner product are often used. In the present embodiment, a method for correcting a shape using an inner product calculated from position information and normal information and an evaluation value other than the inner product will be described. In the description of the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and description of overlapping contents is omitted.

  In S305, the evaluation value calculation unit 205 determines the candidate position ps + σ, ps, corrected for the correction target vertex s based on the rotation vector n′t, the position pt of the peripheral vertex t, and the position ps of the correction target vertex s. Evaluation values χ1, χ2, and χ3 are calculated for each of ps−σ. The evaluation value χ corresponding to the corrected candidate position p of the correction target vertex s is expressed by χ (p) (Equation 7).

  Next, the evaluation value calculation unit 205 acquires an evaluation value φ (p) for the smoothness of the shape represented by the shape data, which can be calculated based on the positional relationship between the correction target vertex and the vertex group near the correction target vertex. In the technical field of image processing, a technique for calculating the smoothness based on the positional relationship between the target vertex and a group of vertices near the target vertex is well known, and thus the description thereof is omitted.

  The comprehensive evaluation value ψ (p) corresponding to the candidate position p after the correction of the correction target vertex s can be calculated according to the following equation.

  Here, ω1 and ω2 represent weighting factors, respectively. As described above, the evaluation value calculation unit 205 according to this embodiment is obtained by a combination of values obtained by multiplying the inner product of the rotation vector n′t and the vector dt and the evaluation value for the smoothness of the shape by the weight coefficient. A comprehensive evaluation value is calculated.

  As described above, according to the present embodiment, in addition to the effects of the first embodiment, the shape represented by the input shape data can be corrected in consideration of the smoothness of the shape.

[Modification]
In the above-described embodiment, the evaluation value calculation method for calculating the inner product (evaluation value) using the rotation vector n′t obtained by rotating the normal vector nt at the peripheral vertex t has been described. In the modification, the normal vector nt may be rotated only for a necessary region in the shape represented by the three-dimensional shape data. For example, the normal line rotation unit 204 executes the process of rotating the normal vector nt only when the position ps of the correction target vertex s and the position pt of the peripheral vertex t are in the positional relationship constituting the edge part. May be. Alternatively, the normal rotation unit 204 rotates the normal vector nt only when the correction target vertex s is located beyond the normal vector nt direction of the peripheral vertex t with reference to the ideal position. May be executed. As described above, the normal value vector nt is rotated only for a necessary region, and the evaluation value calculation process is performed for a region such as a non-edge portion without rotating the normal vector nt. 100 processing speed can be improved and the processing load can be reduced.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 ... Image processing apparatus 202 ... Correction target vertex selection part 203 ... Peripheral vertex selection part 204 ... Normal rotation part 205 ... Evaluation value calculation part 207 ... Destination determination part 208 ... Shape correction part

Claims (11)

Selection means for selecting a correction target vertex to be corrected at each vertex in the shape represented by the three-dimensional shape data including normal line information and a peripheral vertex adjacent to the correction target vertex;
A rotating means for rotating a normal vector at the peripheral vertex;
A calculating means for calculating an inner product of a vector from the peripheral vertex toward the candidate position of the correction target vertex and the rotated vector;
Determining means for determining a position of a movement destination of the correction target vertex based on the calculated inner product;
An image processing apparatus comprising: correction means for moving the correction target vertex to the determined movement destination position. The image processing apparatus according to claim 1, wherein the rotation unit rotates the normal vector in a plane defined by the normal vector and a vector from the peripheral vertex toward the correction target vertex. The image processing apparatus according to claim 2, wherein the rotation unit rotates the normal vector in a main scanning direction that is a direction in which the selection unit selects the correction target vertex. The image processing apparatus according to claim 2, wherein the rotation unit rotates the normal vector by 90 degrees around the position of the peripheral vertex. A determination means for determining whether or not the correction target vertex and the peripheral vertex are in a predetermined positional relationship;
5. The rotation unit according to claim 1, wherein the rotation unit rotates a normal vector at the peripheral vertex when it is determined that the correction target vertex and the peripheral vertex are in the predetermined positional relationship. The image processing apparatus according to any one of the above. The calculation means performs an operation of normalizing a vector from the peripheral vertex toward a candidate position of a movement destination of the correction target vertex, and calculates the inner product using the normalized vector. Item 6. The image processing apparatus according to any one of Items 1 to 5. The calculation means calculates a plurality of inner products corresponding to a plurality of candidate positions,
7. The determination unit according to claim 1, wherein the determination unit determines the candidate position where the smallest value among the plurality of calculated inner products is calculated as a movement destination of the correction target vertex. The image processing apparatus according to item. The calculation means calculates at least one evaluation value other than the inner product, calculates a comprehensive evaluation value obtained by a combination of values obtained by multiplying each of the inner product and the evaluation value by a weighting factor,
The image processing apparatus according to claim 1, wherein the determination unit determines a movement destination of the correction target vertex based on the comprehensive evaluation value. The evaluation value other than the inner product is a smoothness of a shape represented by the three-dimensional shape data that can be calculated from a positional relationship between the correction target vertex and a vertex group near the correction target vertex. An image processing apparatus according to 1. A selection step of selecting a correction target vertex to be corrected at each vertex in the shape represented by the three-dimensional shape data including normal line information and a peripheral vertex adjacent to the correction target vertex;
A rotation step of rotating a normal vector at the peripheral vertex;
A calculation step of calculating an inner product of a vector from the peripheral vertex toward a candidate position of the movement destination of the correction target vertex and the rotated normal vector;
A determination step of determining a position of a movement destination of the correction target vertex based on the calculated inner product;
And a correction step of moving the correction target vertex to the determined position of the destination.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1-9.