JP5956875B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing technique for expressing an image as a vector, and more particularly to a vector expression technique for expressing a raster image by a mesh group.

  Conventionally, a technique for expressing an image as a vector is widely known. The vector-represented image has advantages such as little image quality degradation at the time of enlargement / reduction, easy image editing, and high compression ratio.

  In order to represent an illustration or a character in a vector, a method of approximating the contour of an object with a Bezier or spline function is used. The object area can be painted with a single color, linear gradation, radial gradation, etc., but it is difficult to express complex gradation.

  In order to represent an object including a complex gradation as a vector, an Adobe Illustrator (registered trademark) gradient mesh tool is generally used. In a gradient mesh, an object including a complex gradation can be drawn by generating a cubic function by giving a color and a gradient to the mesh (Patent Document 1).

  A gradient mesh is a curved mesh composed of four vertices, but a technique for expressing a more complex gradation vector by using a curved mesh composed of three vertices is known (Non-Patent Document 1). Edges in the image are extracted as line segments (hereinafter referred to as edge lines), and a vector expression is performed by constructing a 3-vertex curve mesh group that reproduces the edge lines. Here, the three-vertex curve mesh is assumed that the number of vertices of the mesh is 3, and the sides of the mesh connecting the vertices are straight lines or curves.

Special registration 04220010

Tian Xia, Binbin Liao, and Yizhou Yu, “Patch-Based Image Collection with Automatic Curvilinear Feature Alignment”, ACM SIGGRAPH Asia Via.9. 28, no. 5.

  Consider a vector representation of an object such as a natural image whose color changes in a complex manner using a three-vertex curve mesh group. Since a curved mesh approximates the shape and characteristics of an object with a curved line, the object can be approximated with a smaller number of meshes. In order to approximate an image with a three-vertex curve mesh group with high accuracy, it is necessary to generate a mesh group that reproduces edge lines and captures the characteristics of global color change of the image.

  For example, in Non-Patent Document 1, an edge line in an image is extracted, and a mesh group along the edge line is generated. After that, for each of the three RGB components for each mesh, a thin plate spline function that paints the colors in the mesh is optimized based on the color of the input image. At this time, only when the adjacent meshes do not cross the edge line, the thin plate spline function is continued to express a steep color change in the edge portion and a gentle color change in the gradation portion. However, this method requires an optimization operation for color determination for each mesh, so that the image approximation accuracy is good, but the calculation amount increases.

  Otherwise, the mesh is further subdivided into three fine vertices, the colors are obtained from the input image using the vertices of the subdivided mesh as sampling points, and the colors in these small triangles are compensated by bilinear interpolation. A method of expressing the color of the mesh can be considered. However, according to this method, since the same vertex is acquired as the sampling point in the edge portion, it becomes impossible to express a sharp color change of the edge. In particular, at the intersections where the edge lines meet, color oozes out, leading to degradation of image quality.

  The present invention has been made in view of such problems, and an object thereof is to provide a technique for generating a three-vertex curve mesh group having good color reproducibility even at an edge line intersection.

  In order to achieve the object of the present invention, for example, the image processing apparatus of the present invention has a mesh having a detection line detected from an input image consisting of three vertices and a side connecting the three vertices being a straight line or a curve. An image processing apparatus that divides the input image into units of the mesh so as to be one side, divides the mesh into a plurality of sub-mesh, and assigns colors to the color points using the vertices of the sub-mesh as color points. The mesh side corresponding to the detection line among the mesh sides in the input image is the corresponding mesh side, and the color point of interest among the color points on the corresponding mesh side is an intersection of the corresponding mesh sides of three or more. The mesh group sharing the intersection is divided into a plurality of groups with the three or more corresponding mesh sides as a boundary, and the color at a position close to the intersection in each group is defined as the target color point. Allocation And a color point that is not the intersection point on the corresponding mesh side, the color at a position close to the color point in each mesh that shares the color point is the color point. Second assigning means for assigning to each other, third assigning means for assigning a color at the position of the other color point in the input image to a color point other than the color point on the corresponding mesh side, And means for outputting the color assigned by one to three assigning means.

  With the configuration of the present invention, it is possible to generate a three-vertex curve mesh group with good color reproducibility even at the edge line intersection.

FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus. The figure explaining the case where an undesirable result is obtained as an edge line. The figure explaining operation | movement of the feature point extraction part 103. FIG. The figure explaining operation | movement of the feature point extraction part 103. FIG. The flowchart of the process which the curved mesh production | generation part 104 performs. The figure which shows the result of having performed the 3 vertex curve mesh group production | generation process with respect to the characteristic line of Fig.3 (a). The figure explaining shape information and color information. The figure explaining the division | segmentation of a Bezier patch. The figure explaining the color setting with respect to a color point. The figure explaining the color setting with respect to a color point. The flowchart of the process which the intersection color complementation part 107 performs. FIG. 2 is a block diagram illustrating a hardware configuration example of a computer applicable to an image processing apparatus. The figure explaining the color setting with respect to a color point. The flowchart of the process which the intersection color complementation part 107 performs. The figure which shows the structural example of code | symbol data. The figure which shows the structural example of the apparatus which produces | generates code data, and the apparatus which decodes this code data.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
First, the three-vertex curve mesh used in this embodiment will be described. As described above, each side of the three-vertex curve mesh is a curve or a line segment, but this curve may be constituted by a Bezier curve or a B-spline. That is, the type of curve is not limited to a specific type. In the following description, a three-vertex Bezier patch that is one of parametric curved surfaces is used for the three-vertex curved mesh. Among the Bezier patches, in particular, the cubic Bezier patch has an advantage that each side of the mesh becomes a cubic Bezier curve, so that the user can easily edit after generating the three-vertex curve mesh. The points in the cubic Bezier patch having the vertices at P1, P2, and P3 are expressed by the following equations.

  Here, s, t, and u are patch parameters (center-of-gravity coordinates in the patch). Further, p1, p2, and p3 are coordinate values of the vertices P1, P2, and P3, respectively. c1, c2, c3, c4, c5, c6 and c7 are parameters for determining the shape of the curved patch. Here, c1 to c6 are parameters corresponding to control points of three Bezier curves that determine the contour of the Bezier patch. c7 is a parameter that does not relate to the contour of the Bezier patch. In this embodiment, the parameter is determined from the coordinates of the vertex of the Bezier patch and the parameter corresponding to the control point of the Bezier curve forming the contour by the following equation.

  Next, a functional configuration example of the image processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. The configuration illustrated in FIG. 1 is merely an example of a configuration for executing processing described below.

  A raster representation image (input image) is input to the image input unit 101. The input image may be a color image, a gray image, or a black and white image. Further, the input image may be a rectangular image, and when it is desired to vectorize a part of the image, a clipped region such as an image in which the background portion is set to a transparent color may be given. Then, the image input unit 101 transfers this input image to the subsequent feature line detection unit 102.

  The feature line detection unit 102 configures a curve-shaped feature line representing a feature in the input image. In the present embodiment, edge lines are used as feature lines. The process for configuring the feature line is roughly divided into an edge detection process and an edge line configuration process.

  First, the edge line detection process will be described. In the edge line detection process, first, pixels (edge pixels) constituting an edge are detected from the input image. In the present embodiment, the edge pixel is detected using a known method called the Canny method, but the edge pixel may be detected using another method. In addition, preprocessing such as brightness adjustment and edge enhancement may be performed on the input image so that the Canny method functions effectively.

  Then, an edge line configuration process is performed. In the edge line configuration process, first, a line segment that reproduces an edge line is configured by configuring a line segment that connects adjacent edge pixels. When a line segment group is formed by connecting adjacent edge pixels, there is a four-connection connection in which four edge pixels located above, below, left, and right of the target edge pixel are considered to be adjacent. Further, in addition to these four directions, there are eight connected connections in which four edge pixels on the upper right, lower right, upper left, and lower left are considered to be adjacent. In the present embodiment, eight linked connections are used, but this is only an example.

  Here, when edge pixels that can be considered to be adjacent to each other by 8-connection are connected to edge pixels detected using the Canny method, an undesirable result may be obtained as an edge line. This case will be described with reference to FIG.

  FIGS. 2A to 2D show a pixel group constituting a certain area in the image, and one square represents one pixel. In addition, a rectangle indicated by hatching represents an edge pixel, and a rectangle without hatching represents a pixel other than the edge pixel (non-edge pixel).

  As shown in FIG. 2B, when the edge pixel group shown in FIG. 2A is connected to adjacent edge pixels by 8-connection, a loop that connects the edge pixels 201, 202, and 203 is generated. End up. When an edge pixel as shown in FIG. 2A is detected, it is necessary to suppress the occurrence of such a loop. Therefore, in the present embodiment, this problem is solved by using a known technique called the Hiditch thinning method before performing 8-connection. When the thinning is performed on the edge pixel group shown in FIG. 2A, the edge pixel 201 shown in FIG. 2B becomes a non-edge pixel. As a result, as shown in FIG. 2C, the edge pixels (204 to 207) are connected by one edge line having no loop.

  At this time, as shown by the edge pixel 208 in FIG. 2D, a pattern in which a plurality of edge lines meet at one place may occur. In such a case, the meeting edge lines are divided and treated as independent edge lines. In the example of FIG. 2D, there are three edge lines.

  In addition to the edge line, for example, the contour of an object region in the input image can be considered as the feature line. Here, the object refers to a character, an object image, or the like. As already mentioned, for the purpose of vectorizing a part of the input image, if a part of the input image is set to transparent color, the pixel at the boundary between the transparent color and the non-transparent color is the outline of the object. It becomes a pixel. The process of combining the contour pixels of the object into one line segment group can be realized by connecting adjacent pixels according to the same process as the edge line configuration process. The same applies to pixels on the boundary line between regions having different attributes in the input image.

  The feature line detection unit 102 may divide the input image into regions and use the boundary lines as feature lines. Many methods are known for area division, such as those using the nearest neighbor method and those using an EM algorithm. In the present embodiment, a threshold value is set as a reference for density change, an edge pixel is detected by applying the Canny method, and a feature line is configured using the edge pixel.

  Returning to FIG. 1, next, the feature point extraction unit 103 determines a sample point to be finally used as a feature point among the sample points on the feature line. The operation of the feature point extraction unit 103 will be described with reference to FIGS.

  FIG. 3A shows a characteristic line configured by a certain area in the input image. The feature point extraction unit 103 sets a plurality of sample points on the feature line. This sample point may be the edge pixel described above.

  A sample point group on a characteristic line in a certain section is shown in FIG. In FIG. 4A, reference numerals 401 to 405 indicate sample points. Starting from the end point of the feature line, adjacent sample points are sequentially deleted to generate an approximation function.

  First, as shown in FIG. 4B, starting from the sample point 401 that is the end point of the feature line, the sample point 402 is first deleted, and the sample point 401 and the sample point 403 are used as end points of the approximate curve. An approximate curve 406 is generated so as to approximate. Here, the approximation function is a cubic Bezier function generated by least square approximation, but other functions such as a B-spline function may be used. An error is calculated from the distance between the generated approximate curve 406 and the deleted sample point 402. If the error is within the threshold, the next sample point is deleted. In the present embodiment, the error threshold is set to one pixel. In FIG. 4B, the next process will be described on the assumption that the error is within the threshold.

  As shown in FIG. 4C, the next adjacent sample point 403 is deleted and the approximate curve 407 is approximated so that the sample point 402 and the sample point 403 are approximated by using the sample point 401 and the sample point 404 as end points of the approximate curve. Is generated. The error between the approximate curve 407 and the sample points 402 and 403 is calculated. In FIG. 4C, the next process will be described on the assumption that the error is within the threshold.

  As shown in FIG. 4D, next, the adjacent sample point 404 is deleted, and the approximate curve 408 is approximated so that the sample points 402, 403, 404 are approximated with the sample point 401 and the sample point 405 as the end points of the approximate curve. Generate. The error between the approximate curve 408 and the sample points 402, 403, 404 is calculated. Here, it is assumed that the error between the approximate curve 408 and the sample points 402 and 404 exceeds the threshold value. At this time, the approximate curve 408 is not employed, and the previously generated approximate curve 407 is employed as an effective approximate function.

  In this case, as shown in FIG. 4E, the sample point 401 that is the end point of the feature line is used as the starting point, the sample points 402 and 403 are deleted, and the sample point 404 that is the end point of the approximated curve 407 is used as the feature point. Sampling. That is, the sample point 401 and the sample point 405 which are the end points of the feature line, and the sample point 404 sampled as the feature point are employed as the first feature point. Here, when there are more sample points, the same process is repeated starting from the sample point 404 to determine the feature points. With the above processing, feature points are extracted in addition to the intersections of feature lines such as edge pixels 204 and 207 in FIG. 2C and feature lines such as edge pixel 208 in FIG. The feature points finally sampled with respect to the feature line in FIG. 3A are as indicated by white circles 301 and asterisks 302 in FIG. Of course, as long as the features of the image can be reproduced, other methods may be used such as sampling evenly between the end points of the feature lines to obtain the feature points.

  Returning to FIG. 1, the curved mesh generation unit 104 divides the input image into a three-vertex curved mesh group so that the feature line can be reproduced. The process performed by the curved mesh generation unit 104 will be described with reference to FIG. 5 showing a flowchart of the process. Of course, the main subject of the processing in each step shown in the flowchart of FIG. 5 is the curved mesh generation unit 104.

  First, in step S501, a side to be curved is set. Here, in the feature points determined by the feature point extraction unit 103, a line segment connecting the feature points adjacent on the feature line is set as a side to be curved.

  Next, in step S502, by using the Delaunay triangulation with constraints using the side set in step S501 as a constraint condition, a three-vertex line mesh group in which the set side becomes a side of the mesh is generated. The constrained Delaunay triangulation is an algorithm that, when given a point group and a constraint condition, generates a straight line mesh with the given point group as a vertex and satisfying the constraint condition. Here, the constraint condition is a line segment connecting a part of the point group. Of course, other methods may be used as long as a three-vertex straight line mesh that reproduces the set side can be generated.

  In step S503, it is determined whether or not all sides to be curved have been curved. As a result of the determination, if all the sides to be curved have been curved, the processing according to the flowchart of FIG. 5 is completed, and the sides that have not yet been curved remain among the sides to be curved. If yes, the process proceeds to step S504.

  In step S504, one side that has not yet been curved is selected from the sides to be curved. In step S505, the side selected in step S504 is curved. In this curving, the sides of the mesh are curved so as to approximate the feature line. In the present embodiment, the cubic Bezier function generated by the feature point extraction unit 103 is used as it is as a mesh side, so there is no need to recalculate the curve. FIG. 6 shows the result of such a three-vertex curve mesh group generation process performed on the feature line of FIG.

  Returning to FIG. 1, the color point extraction unit 105 then determines color information necessary for applying each mesh set on the input image by the curved mesh generation unit 104. Various methods are conceivable for generating color information. One possible method is to paint each mesh with a single color. As a method for determining one color, there is a method in which an average value of pixel values of pixels included in the mesh is used as color information of the mesh. As a method for expressing the colors in the mesh in a more complicated manner, a method for generating a function for expressing the color for each mesh may be considered. For example, in Non-Patent Document 1, a thin plate spline function that paints colors in a mesh is generated for three RGB components based on the color of the input image. In this embodiment, each mesh is further divided into fine meshes (sub-mesh) composed of three vertices, and the color of the pixel at the vertex of the divided sub-mesh is acquired from the input image and assigned to the vertex (color assignment). When rendering, the colors in each small triangle can be supplemented by bilinear interpolation to paint each mesh color.

  Here, shape information and color information necessary for configuring each mesh will be described. FIG. 7A shows shape information relating to a mesh (three-vertex curve mesh) composed of three vertices and connected between three vertices by a curve (all sides are expressed by a curve). Reference numerals 701, 702, and 703 denote vertices of the curved mesh, and reference numerals 704 to 709 denote control points of the curved mesh. This shape information is generated by the curved mesh generation unit 104.

  FIG. 7B shows color information relating to the curved mesh shown in FIG. Reference numerals 710 to 724 denote vertexes (color points) of the sub-mesh in the curved mesh, and each color point has color information. Here, the color point is described as having RGB values as color information. However, the color point may have color information in another color space such as CMYK. In addition, when the input image is a gray image, the color point holds the luminance information of the corresponding pixel as color information. The color point is derived by regularly dividing the Bezier patch. Here, 15 points are used as color points. However, the number of Bezier patches may be changed to change the number of color points. For example, in FIG. 8A, the number of divisions is 1, and the sampling number of color points is 3. In FIGS. 8B, 8C, and 8D, the number of divisions is 2, 3, and 4, and the number of color point samplings is 6, 10, and 15, respectively.

  Returning to FIG. 1, the on-edge color point complementing unit 106 complements the color information of the color point in order to express a steep color change that occurs when the edge line is crossed. In the present embodiment, the color change described above is expressed by holding two types of colors in one color point for each color point of a side curved along the feature line (referred to as a curved side). ing. Since the color points 901 to 905 in FIG. 9A are color points that are not on the curved side, one color is held at such a color point. On the other hand, since the color points 906 to 910 in FIG. 9B are color points on the curved side, two colors are held at such color points. That is, in the present embodiment, the curved side has two kinds of colors. As a color to be actually given to the color point, the color of the input image at a position where the curved side is shifted by several pixels in the direction perpendicular to the curved side can be considered. In the example of FIG. 9B, the color points 906 to 910 have white and black colors, and when the right mesh is rendered, the right color of the side is used. When rendering the left mesh, the color on the left side of the side is used. In addition, intersection points as indicated by the asterisk 302 in FIG. 3 where three or more curved edges meet are complemented in the next step and are not subject to complementation in this step.

  Returning to FIG. 1, the intersection color complementing unit 107 complements the color of the color point on the intersection point in order to express a steep color change at the intersection point where three or more curvilinear sides meet. In the present embodiment, the above-described color change is expressed by holding the same number of colors as the number of curved edges associated with the color point on the intersection. The broken lines (1002, 1003, 1004) in FIG. 10A represent the curved edges, and the solid lines represent the other mesh edges. Further, the color point 1001 in FIG. 10A is an intersection of the curved sides 1002, 1003, and 1004. Reference numerals 1005 to 1010 denote meshes that share the color point 1001. At this time, since the number of curvilinear sides meeting at the color point 1001 is 3, the color point 1001 has three colors. A process performed by the intersection color complementation unit 107 to determine the color at such an intersection part will be described with reference to FIG. 11 showing a flowchart of the process. FIG. 10B is an enlarged view of the vicinity of the intersection in the example of FIG.

  First, in step S1101, the number Enum of curved edges sharing the intersection is counted. In the example of FIG. 10, since there are three curved edges 1002, 1003, and 1004 that share the intersection (color point 1001), Enum = 3. Next, in step S1102, a variable i used in the following processing is initialized to 1.

  In step S1103, when Enum curved edges are set to E1,. The number Minum of j = 1 to Minum) is counted. In the example of FIG. 10, when E1 is a curved edge 1002, E2 is a curved edge 1003, and E3 is a curved edge 1004, the mesh sandwiched between E1 and E2 is mesh 1005 (M11 ), 1006 (M12), so that Minum = 2. Further, since the mesh sandwiched between E2 and E3 is one of the meshes 1007 (M21), Minum = 1. Further, since there are three meshes 1008 (M31), 1009 (M32), and 1010 (M33) that are sandwiched between E3 and E1, Minum = 3.

  In step S1104, it is determined whether Minum = 1. If Minum = 1 as a result of this determination, the process proceeds to step S1105. If Minum ≠ 1, the process proceeds to step S1106.

  In step S1105, the color near the intersection in the mesh Mi1 is acquired, and the acquired color is set as the color of the intersection. Specifically, the color point closest to the intersection is specified in each of Ei and Ei + 1, and the midpoint position of each specified color point is obtained. This midpoint position is a position in the mesh Mi1. The color at the midpoint position is acquired from the input image, and the acquired color is set for the intersection.

  In the example of FIG. 10, when i = 2, the process proceeds from step S1004 to step S1005. In this case, the color points closest to the color point 1001 in the curved sides 1003 and 1004 are the color points 1012 and 1013. However, the midpoint position (position in the mesh 1007) between the color point 1012 and the color point 1013 is obtained, the color at this midpoint position is acquired from the input image, and the acquired color is set for the color point 1001. . In this way, the color of the intersection can be complemented with a stable color inside the patch without taking an intermediate color near the edge.

  On the other hand, in step S1006, in the mesh sandwiched between Ei and Ei + 1, the color of the color point near the intersection and not on the curvilinear side is acquired, and the calculated color calculated from the acquired color is calculated at the intersection. Set as color. Specifically, in a mesh sandwiched between Ei and Ei + 1, a plurality of color points that are not on the curved side in the vicinity of the intersection are specified, and a calculated color calculated from the colors at the specified color points (For example, an average color) is obtained, and the obtained calculated color is set as the color of the intersection.

  In the example of FIG. 10, when i = 1 or i = 3, the process proceeds from step S1004 to step S1005. When i = 3, meshes sandwiched between the curved side 1004 and the curved side 1002 are meshes 1008 to 1010. Among the color points in the meshes 1008 to 1010, the color points that are not on the curvilinear side and are close to the color point 1001 (here, closest) are the color points 1014 and 1015. In this case, a calculated color is obtained from the colors of the color points 1014 and 1015, and the obtained calculated color is set as the color point 1001. By doing so, it is possible to complement the color of the intersection point without taking an intermediate color near the edge and maintaining the continuity of the color with a plurality of meshes.

  When i = 1, the color points in the meshes 1005 and 1006 sandwiched between the curved side 1002 and the curved side 1003 are not on the curved side and in the vicinity of the color point 1001 ( The closest color point here is only the color point 1011. In such a case, the color of the color point 1011 is set to the color point 1001.

  In step S1107, it is determined whether i = Enum. If i = Enum as a result of this determination, the processing according to the flowchart of FIG. 11 ends. On the other hand, if i <Enum, the process proceeds to step S1108. In step S1108, the value of variable i is incremented by one, and the processing from step S1103 is repeated.

  By executing the processes described above, it is possible to set shape information and color information necessary for image representation using a mesh.

  Returning to FIG. 1, the image output unit 108 generates an image by performing rendering processing based on the shape information and color information necessary for image representation by the mesh obtained in the above processing, The generated image is output. The output destination of the generated image is not limited to a specific output destination, and may be displayed on a CRT or a liquid crystal screen, output to an appropriate memory, or output to an external device. Also good. The shape information and color information obtained by the above processing may be processed as a known mesh encoding target.

  By performing the above processing on the input image, there is no jaggy that occurs when the image represented by the bitmap is enlarged as it is, and there is a steep color change around the edge or at the intersection. Can be obtained.

<Modification of First Embodiment>
Each unit shown in FIG. 1 may be configured by hardware, but may be implemented as software (computer program). In this case, the software is installed in a memory of a general computer such as a PC (personal computer). When the CPU of the computer executes the installed software, the computer realizes the functions of the above-described image processing apparatus (functions of each unit illustrated in FIG. 1). That is, this computer can be applied to the above-described image processing apparatus. A hardware configuration example of a computer applicable as the image processing apparatus according to the first embodiment will be described with reference to FIG.

  The CPU 1201 controls the entire computer using computer programs and data stored in the RAM 1202 and the ROM 1203, and executes the above-described processes described as being performed by the image processing apparatus. In other words, the processes described above are performed as performed by each unit illustrated in FIG.

  The RAM 1202 is an example of a computer-readable storage medium. The RAM 1202 has an area for temporarily storing computer programs and data loaded from the external storage device 1207 and the storage medium drive 1208, data received from the external device via the I / F (interface) 1209, and the like. Further, the RAM 1202 has a work area used when the CPU 1201 executes various processes. That is, the RAM 1202 can provide various areas as appropriate. The ROM 1203 is an example of a computer-readable storage medium, and stores computer setting data, a boot program, and the like.

  A keyboard 1204 and a mouse 1205 can be operated by a computer operator to input various instructions to the CPU 1201. The display device 1206 is configured by a CRT, a liquid crystal screen, or the like, and can display the processing result by the CPU 1201 with images, characters, and the like. For example, the input image can be displayed, a three-vertex curve mesh group, the rendering result, or the like can be displayed.

  The external storage device 1207 is an example of a computer-readable storage medium, and is a large-capacity information storage device represented by a hard disk drive device. The external storage device 1207 stores an OS (operating system), computer programs and data for causing the CPU 1201 to realize the functions of the respective units shown in FIG. 1, data of the input image, information described as known information, and the like. Has been. Computer programs and data stored in the external storage device 1207 are appropriately loaded into the RAM 1202 under the control of the CPU 1201 and are processed by the CPU 1201.

  The storage medium drive 1208 reads a computer program and data recorded on a storage medium such as a CD-ROM or DVD-ROM, and outputs the read computer program or data to the external storage device 1207 or the RAM 1202. Note that part or all of the information described as being stored in the external storage device 1207 may be recorded on this storage medium and read by this storage medium drive 1208.

  The I / F 1209 is for connecting an external device to a computer. For example, a device for acquiring the input image such as a digital camera is connected to the I / F 1209, and the input image is acquired from the device to the RAM 1202 or the external storage device 1207 via the I / F 1209. good. A bus 1210 connects the above-described units.

  In the above configuration, when the computer is turned on, the CPU 1201 loads the OS from the external storage device 1207 to the RAM 1202 in accordance with the boot program stored in the ROM 1203. As a result, an information input operation can be performed via the keyboard 1204 and the mouse 1205, and a GUI can be displayed on the display device 1206. When the user operates the keyboard 1204 or the mouse 1205 and inputs an activation instruction for an image processing application program stored in the external storage device 1207, the CPU 1201 loads the program into the RAM 1202 and executes it. As a result, the computer functions as the image processing apparatus.

  The image processing application program executed by the CPU 1201 basically includes the processing units shown in FIGS. 1 and 16 and the functions corresponding to the components shown in the steps of FIGS.

[Second Embodiment]
In the first embodiment, in the intersection color complement processing shown in FIG. 11, the color of the color point near the intersection or the color itself at the midpoint position of the straight line connecting the color points is applied as the color of the intersection. However, this does not reflect the color transition in the mesh. In the present embodiment, this is taken into consideration, and linear extrapolation is performed using colors in the mesh. This embodiment is the same as the first embodiment except for the points described below. That is, only the difference from the first embodiment will be described below.

  In FIG. 13A, broken lines (1302, 1303, 1304) represent curved edges, and solid lines represent other mesh edges. Further, the color point 1301 in FIG. 13A is an intersection of the curved edges 1302, 1303, and 1304. Reference numerals 1305 to 1310 denote meshes that share the color point 1301. At this time, since the number of feature lines meeting at the color point 1301 is three, the color point 1301 has three colors. The process performed by the intersection color complementation unit 107 to determine the color at such an intersection part will be described with reference to FIG. 14 showing a flowchart of the process. FIG. 13B is an enlarged view of the vicinity of the intersection in the example of FIG. Steps S1401 to S1403 are the same as steps S1101 to S1103 described above, and a description thereof will be omitted.

  In step S1404, it is determined whether Minum is an even number or an odd number. As a result of the determination, if Minum is an even number, the process proceeds to step S1406. If Minum is an odd number, the process proceeds to step S1405.

  In step S1405, the center mesh is specified from among the meshes sandwiched between Ei and Ei + 1, and among the color points on the sides passing through the intersection in the identified mesh, the color points near the intersection are identified and identified. The color of the intersection is determined using the color point.

  In the example of FIG. 13, when E1 is a curved edge 1302, E2 is a curved edge 1303, and E3 is a curved edge 1304, three meshes (mesh 1308, 1309, and 1310) exist between E1 and E3. Is sandwiched. Among these, since the mesh at the center is the mesh 1309, the side (sides 1312 and 1312) passing through the color point 1301 among the sides of the mesh 1309 is specified, and the color point 1301 among the color points on the specified sides 1312 and 1313 is specified. The closest color point to the second color point is specified. In the case of FIG. 13, the color points closest to the color point 1301 are color points 1316 and 1318, and the second closest color point is the color points 1317 and 1319.

  Then, the color at the midpoint position between the closest color points and the color at the intermediate position between the second closest color points are acquired from the input image. In the case of FIG. 13, the color at the intermediate position 1320 between the color point 1316 and the color point 1318 and the color at the intermediate position 1321 between the color point 1317 and the color point 1319 are acquired from the input image. Then, using the coordinate positions and colors of the intermediate position 1320 and the intermediate position 1321, the color of the intersection is determined by linear extrapolation.

  On the other hand, in step S1406, two meshes at the center of the meshes sandwiched between Ei and Ei + 1 are specified, and among the color points on the side shared by the two specified meshes, the color points near the intersection are selected. Identify and determine the color of the intersection using the identified color points.

  In the example of FIG. 13, two meshes (mesh 1305, 1306) are sandwiched between E1 and E2. Among these, the two meshes at the center are meshes 1305 and 1306, and the side shared by these meshes is the side 1311. Then, the color point closest to the color point 1301 and the second closest color point among the color points on the side 1311 are obtained. In the example of FIG. 13, the color point closest to the color point 1301 is the color point 1314, and the second closest color point is the color point 1315. Then, the color at the closest color point and the color at the second closest color point are acquired from the input image. In the case of FIG. 13, the color at the color point 1314 and the color at the color point 1315 are acquired from the input image. Then, using the coordinate positions and colors of the color point 1314 and the color point 1315, the color of the intersection is determined by linear extrapolation.

  Steps S1407 and S1408 are the same as steps S1107 and S1108 in FIG. By performing the above processing, it is possible to complement the intersection color reflecting the color transition in the mesh.

  The configuration described in the first and second embodiments is merely a configuration as an embodiment, and is merely an example of the configuration described below. In other words, any of the embodiments is merely an example of the following image processing apparatus. That is, the input image is divided into units of the mesh so that the detection line detected from the input image becomes one side of the three-vertex curve mesh, the mesh is divided into a plurality of sub-mesh, and each vertex of the sub-mesh Is an image processing apparatus that assigns a color to the color point.

  In this image processing apparatus, when a mesh side corresponding to a detection line among mesh sides in an input image is a corresponding mesh side, an intersection of corresponding mesh sides having three or more target color points among color points on the corresponding mesh side. In this case, a color is assigned to the target color point as follows. That is, the mesh group sharing the intersection is divided into a plurality of groups with the three or more corresponding mesh sides as boundaries, and the color at the position close to the intersection in each group is assigned to the target color point (first 1 allocation). In the example of FIG. 10, mesh groups (mesh 1005 to 1010) sharing the color point 1001 as the intersection are bounded by E1 to E3, and (mesh 1005, 1006), (mesh 1007), (mesh 1008 to 1010) It is divided into three groups. Then, the color of the position close to the intersection in the group (mesh 1005, 1006), the color of the position close to the intersection in the group (mesh 1007), and the color of the position close to the intersection in the group (mesh 1008 to 1010) , Are assigned to the color point 1001.

  For a color point that is not an intersection on the corresponding mesh side, a color at a position close to the color point in each mesh sharing the color point is assigned to the color point (second assignment). . Further, a color at a position of the other color point in the input image is assigned to a color point other than the color point on the corresponding mesh side (third assignment). Then, the colors assigned by the first to third assignments are output.

  In such a configuration, in the first embodiment, when a group composed of one mesh is included in a plurality of groups, the following processing is performed as the first assignment. . That is, the color point closest to the intersection among the color points on one corresponding mesh side sharing the intersection in the one mesh and the color point closest to the intersection among the color points on the other corresponding mesh side are specified. . Then, the color on the input image at the intermediate position between the specified color points is assigned to the target color point.

  In addition, when a group composed of two or more meshes is included in the plurality of groups, the other color point close to the intersection among the other color points in the two or more meshes is assigned as the first assignment. Is assigned to the target color point.

  In the second embodiment, when a group composed of an odd number of meshes is included in a plurality of groups, the following processing is performed as the first assignment. That is, the color point closest to the target color point and the second closest color point are specified from the mesh sides passing through the target color point of the center mesh among the odd number of meshes. A color at an intermediate position of a color point closest to the target color point specified from each mesh side; and a color at an intermediate position of the color point closest to the target color point specified from the respective mesh side; The color obtained using is assigned to the color point of interest.

  Further, when a group composed of an even number of meshes is included in the plurality of groups, the following processing is performed as the first assignment. That is, among the even number of meshes, the color point closest to the target color point is identified from the color points on the mesh side shared by the two central meshes, and the closest color point is specified. And the color obtained using the color at the second closest color point are assigned to the target color point.

[Third Embodiment]
In the first embodiment, the feature point extraction unit 103 determines whether or not to make a feature point only on the feature line due to the error of Bezier approximation. However, the feature point extraction unit 103 always has a feature in a place where the distance from the intersection is constant. The feature points may be corrected so as to take points. By doing so, the mesh near the intersection can be kept at a certain size, so that it is not necessary to perform the intersection color correction with a minute mesh, and more stable color interpolation of the intersection can be performed. .

[Fourth Embodiment]
In the first embodiment, output data is image data, and mesh shape information and color information are used only as intermediate data in the process. However, as described in the first embodiment, the shape of the mesh is used. Information and color information may be output as code data. The code data is, for example, as shown in FIG. 15, vertex information 1501, which is information related to each vertex constituting the mesh, mesh edge information 1502, which is information related to each edge constituting the mesh, and information related to the color of the mesh. The color information 1503 may be output in a form including it.

  The vertex information 1501 includes a mesh vertex number 1504, a mesh vertex ID 1505, and mesh vertex coordinates 1506. The mesh side information 1502 includes a mesh side number 1507, a mesh side ID 1508, a mesh vertex ID 1509 forming the mesh side, and control point coordinates 1510 for curving the mesh side. The color information 1503 includes a mesh number 1511, an ID 1512 of a mesh side forming the mesh, and a color 1513 of a color point in the mesh.

  In decoding, a desired image can be obtained by reading the above information and rendering the mesh as bitmap data. Such generation of code data and decoding for obtaining image data from the code data may be performed by the image processing apparatus of FIG. 1 or may be performed by another apparatus.

[Fifth Embodiment]
In the fourth embodiment, when the code data is generated, the intersection color is complemented and output. However, since the color of the intersection can be complemented from other set color points, the intersection color complement is performed. Is not necessarily performed at the time of encoding. For example, the configuration of the device on the code data generation side may be configured as shown in FIG. 16A, and the configuration of the device on the code data decoding side may be configured as shown in FIG. .

  In FIG. 16A, each part of 1601-1606 is the same as 101-106 of FIG. The intersection extraction unit 1607 extracts and outputs information for identifying a mesh vertex using the intersection, for example, an ID. The code data output unit 1608 outputs information specifying the vertex using the intersection in addition to the mesh shape information and color information.

  In FIG. 16B, the code data input unit 1609 receives the data output from the code data output unit 1608 and transfers it to the subsequent intersection color complement unit 1610. The intersection color complementation unit 1610 extracts a mesh using the intersection based on the information for specifying the vertex using the intersection in the code data. Then, in the extracted mesh, a process equivalent to the intersection color complement described in the first embodiment is performed. The image output unit 1611 renders the mesh data and outputs the rendering result as an image.

  With such a configuration, since it is not necessary to have a plurality of colors at the intersections at the time of encoding, the size of the code data can be made smaller than the size of the code data described in the fourth embodiment.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

The input image is divided into units of the mesh so that the detection line detected from the input image is composed of three vertices and the side connecting the three vertices is one side of a mesh that is a straight line or a curve. Is divided into a plurality of sub-mesh, and each vertex of the sub-mesh is used as a color point to assign a color to the color point,
The mesh side corresponding to the detection line among the mesh sides in the input image is the corresponding mesh side, and the color point of interest among the color points on the corresponding mesh side is the intersection of three or more corresponding mesh sides First, a mesh group sharing the intersection is divided into a plurality of groups with the three or more corresponding mesh sides as a boundary, and a color at a position close to the intersection in each group is assigned to the target color point. Assigning means,
For a color point that is not the intersection on the corresponding mesh side, a second assigning unit that assigns a color at a position close to the color point in each mesh sharing the color point to the color point; ,
Third assigning means for assigning a color at a position of the other color point in the input image to a color point other than the color point on the corresponding mesh side;
An image processing apparatus comprising: means for outputting colors assigned by the first to third assigning means. The first assigning means includes
When the group consisting of one mesh is included in the plurality of groups, the color point closest to the intersection among the color points on one corresponding mesh side sharing the intersection in the mesh, Specifying the color point closest to the intersection among the color points on the other corresponding mesh side, and assigning the color on the input image at an intermediate position of the specified color points to the target color point The image processing apparatus according to claim 1, wherein: The first assigning means includes
In the case where a group composed of two or more meshes is included in the plurality of groups, the color of the other color point close to the intersection among the other color points in the two or more meshes or an average thereof The image processing apparatus according to claim 1, wherein a color is assigned to the target color point. The first assigning means includes
When a group consisting of an odd number of meshes is included in the plurality of groups, the target color is determined from each mesh side passing through the target color point of the center mesh among the odd number of meshes. The color point closest to the point is specified, the color point closest to the second is specified, the color at the middle position of the color point closest to the target color point specified from the respective mesh sides, and the above specified from the respective mesh sides The image processing apparatus according to claim 1, wherein a color obtained by using a color at an intermediate position of a color point that is second closest to the target color point is assigned to the target color point. The first assigning means includes
In the case where a group consisting of an even number of meshes is included in the plurality of groups, the attention point is determined from a color point on a mesh side shared by two meshes at the center among the even number of meshes. A color point closest to the color point is specified, a color point closest to the second color point is specified, and a color obtained using the color at the closest color point and the color at the color point closest to the second is determined as the color point of interest. The image processing apparatus according to claim 1, wherein the image processing apparatus is assigned to the image processing apparatus. The input image is divided into units of the mesh so that the detection line detected from the input image is composed of three vertices and the side connecting the three vertices is one side of a mesh that is a straight line or a curve. Is divided into a plurality of sub-mesh, and an image processing method performed by an image processing apparatus that assigns a color to each color point using each vertex of the sub-mesh as a color point,
The first assigning unit of the image processing apparatus sets a mesh side corresponding to the detection line among mesh sides in the input image as a corresponding mesh side, and among the color points on the corresponding mesh side, a target color point is 3 When the intersection of the corresponding mesh sides is the above, the mesh group sharing the intersection is divided into a plurality of groups with the three or more corresponding mesh sides as a boundary, and the position close to the intersection in each group A first assigning step of assigning the color of the color to the target color point;
For a color point that is not the intersection point on the corresponding mesh side, the second assigning unit of the image processing apparatus selects a color at a position close to the color point in each mesh that shares the color point. A second assigning step assigning to the color point;
A third allocating step in which a third allocating unit of the image processing apparatus allocates a color at a position of the other color point in the input image to a color point other than the color point on the corresponding mesh side; ,
An image processing method, comprising: an output unit of the image processing apparatus outputting a color assigned in the first to third assignment steps.   A computer program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 5.

Images