TWI525585B - An image processing system and method - Google Patents

Description

Image processing system and method

The present invention relates to an image processing system and method, and more particularly to an image processing system and method for applying a non-realistic artistic (eg, cartoon) image generation environment, inputting an original image via Feature detection, feature measurement, and feature comparison, to obtain a first image, and the original image is detected by edge detection to obtain a second image, and based on the first image, or based on the first image The image and the second image result in a third image of the cartoon image of the object.

The general digital image processing technology pursues how to produce a digital image that is closest to the original real object/person, for example, photo-realistic rendering (PR). However, in recent years, in addition to the original In addition to image processing techniques (for example, photo-realistic display PR), in order to satisfy entertainment effects, such as cartoon shapes and artistic effects, for example, photo images are like sketches/oil paintings, and new image processing techniques have emerged, such as Non-photorealistic rendering (NPR) image processing technology is based on the pursuit of entertainment and artistic effects.

Any digital image from a photo can basically be divided into two types according to the condition of real display/reality or not: one is photorealistic rendering technology equivalent to the quality of photo digital image. The other is a non-photorealistic rendering (NPR) technique with a specific rendering effect (for example, cartoon shapes, sketches, oil paintings, or other artistic effects).

In the invention publication No. I267798 "Three-dimensional object contour drawing method" of the Republic of China Patent Publication, a method of how to describe a stylized outline of a three-dimensional object composed of a plurality of polygons is disclosed, and the contour extraction method is The silhouette, surface boundary and wrinkles of the three-dimensional object are taken out, and the wire is drawn on the extracted side line and various brush materials are applied. In other words, the "three-dimensional object contour drawing method" belongs to the conventional non-realistic display NPR technology, which draws a network line and applies various brush stroke materials on the side line of a three-dimensional object composed of a plurality of polygons.

In the publication of the Chinese Patent Publication No. 200923836, "Method for Producing a Sketch Image by Image Separation", a method for generating a sketch image by image color separation is disclosed, which is a black layer image, a black layer. Layer maps, gray layer maps, and outline maps are combined into a sketch map. In other words, the "method of generating sketch images by image color separation" belongs to the non-realistic display NPR technology which is known to produce artistic effects of images.

At present, most of the image editing software available on the market provides a combination of non-realistic effects filters, which are usually converted into cartoon-like styles by means of blur filters, color blocks, and edge enhancement. Based on the processing of the pixels, rather than the structure of the brushstroke, the cartoonist's hand-painted style cannot be presented.

SmithMicro's Manga Studio line of products is used as an aid to professional artists to draw their own comics. Among them, the automatic conversion photo is a comic-style function, and the color is replaced by various hatch patterns, and the basic details of the original color photo cannot be retained due to the addition of a large number of lines.

The Richness-Preserving Manga Screening color photo developed by the Chinese University of Hong Kong is transformed into a comic background technology. For the lack of shadow pattern replacement, the converted comic image can show more basic details by preserving the degree of visual difference between color and pattern. Still can't avoid the noise caused by a lot of lines.

Automatic conversion of real-life photos into cartoon-like non-realistic images is a clear market demand, but there is still no suitable solution in the industry. In the market demand for real-life photos to be automatically converted into cartoon styles, it is most eager to convert face photos into cartoon characters, but there is no solution to the demand in the market.

Microsoft Microsoft's Microsoft cartoon player (MCM) is used in the digital image of the photo to analyze the facial and facial features of the facial features, and draws a cartoon-like style according to the specific cartoon sample specified by the user. Portrait of a person. When the system is unable to accurately analyze the outline and lines of the image, the rendered result presents a normal face that is not generally recognized.

Also, Yahoo Yahoo launched Yahoo Avatar, which uses a "combination/synthesis" method to organize a cartoon figure character. Users can use the [tab] button to select appearance/apparel/matching accessories (extras ), and can select the background of the cartoon figure character (background), etc., in other words, to form a picture with a cartoon figure character by selecting the pattern component, and does not perform image processing action on any photo digital image. . The system uses a manual collage method to allow the user to select various parts of the overall shape from the component library, and combine the corresponding face shape and body shape. The real person similarity of the cartoon portrait is determined by the operator's portrait analysis and drawing skills.

In addition, on the FYM (FaceYourManga) website, you can create a mangatar with the online fan-like drawing machine FYM, and use the Japanese manga style to select "face components (for example, face, ear, eye, nose, mouth). In the way of ", etc.", a combination of facial comics styles similar to Japanese manga styles, in other words, the use of the method of selecting and combining facial components, to produce a facial pattern with a Japanese comic cartoon character style, and No image processing is performed on any photo digital image. With Yahoo! Avatar is also a manual collage system.

Reallusion launched the CrazyTalk product to help users locate the facial features of people in photos or pictures in a semi-automatic way. The computer generated (CG, Computer Generated) method allows the characters to speak and perform, without the role of cartoonization. In the same series of FaceFilter products, only the shape, texture or color correction of a single static photo is made, and the artificial or environmental photography is improved to beautify the portrait in the photo.

Another product of Reallusion, iClone, also assists the user in locating the facial features of the characters in the photos or pictures in a semi-automatic manner. Based on the positioning results, the 3D character model is constructed and the animation is synthesized based on the generated model. Its 3D rendering method is a well-known technology in the computer synthesis animation industry, and is drawn using the graphics card hardware function.

So, how can we solve it, whether it is Microsoft's cartoon icon and / or Yahoo Yahoo combination and / or comics FYM (FaceYourManga) website selection, combined facial components and / or Reallusion dynamic simulation / manipulation of photo images The expression of the character can not produce a cartoon image with image features, which is a problem to be solved.

The main object of the present invention is to provide an image processing system and method for applying a non-realistic art (for example, cartoon) image generation environment, and using the image processing system of the present invention to perform an image method flow, inputting a The original image is obtained by feature detection, feature measurement, and feature comparison, and the first image is obtained, and based on the first image, a third image of the cartoon image of the object is obtained.

It is still another object of the present invention to provide an image processing system and method for use in a non-realistic art (e.g., cartoon) image generation environment, using the image processing system of the present invention to perform an image method flow, input An original image is obtained by feature detection, feature measurement, and feature comparison, and the original image is obtained by edge detection to obtain a second image based on the first image and the first image. The second image, and a third image of the cartoon image of the object.

A further object of the present invention is to provide an image processing system and method for use in a non-realistic art (eg, cartoon) image generation environment, based on the first image and the second image, the first image and the first image The two images are compared to find the difference in the details. After calculating the difference between the two images, and fine-tuning the position of the control points of the vector elements at the difference according to the difference, the iterative optimization method is used to approximate the most similar ones. The shape, in order to obtain a third image of the cartoon image of the object.

According to the above, the present invention provides an image processing system including an image processing module, an image generation module, and a data library.

An image processing module, wherein the image processing module generates a first image of a vector image of a cartoon style with original image features by means of feature detection, feature measurement, and feature comparison; Here, the image processing module first performs feature detection and feature measurement on the input original image, and obtains feature data of the original image. When performing feature comparison, the image processing module is directed to the original image. The feature data of each structural component is compared in the database by means of an optimized feature comparison method, and each drawing component closest to the feature data of each structural component is separately compared and combined with the respective drawing components. a first image of a new vector graphic, that is, the image processing module can compare the feature data of each structural component of the original image with the feature data of each drawing component in the database. And obtaining each drawing element of each structural element closest to the original image, and combining the obtained drawing elements into a vector graphic, Words, the vector graphics system by the library of the respective drawing elements consisting, while the first image is the original image having a particular feature of the artistic style (e.g., a cartoon style) the vector graphics.

In addition, and/or, the image processing module performs edge detection processing on the original image, in order to highlight the shape of each structure, filtering out image information other than the original image shape, and filtering out color and detail texture, in color and detail After the texture is filtered out, the edge features in the image are enhanced and smoothed, resulting in a second image of the line art.

An image generation module, the image generation module may obtain a third image that is a cartoon image of the object according to the first image; and/or, based on the first image and the second image, the first image and the second image Perform an alignment to find the difference in the details. After calculating the difference between the two images, and fine-tuning the position of the control points of each vector element at the difference according to the difference, the iterative optimization method is used to approximate the most similar shape. In order to obtain a third image of the cartoon image of the object.

a database in which the drawing elements are stored, and each drawing element in the database is drawn in a specific artistic style (for example, a cartoon style), and the measurement of the characteristic data is simultaneously performed at the time of drawing, so that each The drawing component has its characteristic data; the image processing module can compare the feature data of each structural component of the original image with the feature data of each drawing component, and obtain the structural components closest to the original image. Individual drawing elements.

When the image processing system of the present invention is used to perform the image method flow, first, an original image is input to the image processing module; wherein the original image may be an original image having structural features, for example, a human face image, In other words, because of the characteristics of the relative position of the facial features (the relative positions of the eyes, nose, ears, mouth, and eyebrows), it can be performed in the following manner, and the original image can be detected without pre-preparation. The original image of the structural features, for example, the house, because of the various room types, sizes, structures, and no specific patterns, there is no preset structural feature, so, in the follow-up, there are no preset structural features. The detection method is carried out.

Then, the image processing module processes the input original image to generate at least a first image of a vector graphic having a cartoon image of the original image feature, and/or the image processing module inputs the original image through The method of edge detection is derived as the second image of the line draft. Here, the image processing module can generate the first image of the vector graphic of the cartoon style with the original image feature by using the original image input, feature detection, feature measurement, and feature comparison; The image processing module first performs feature detection and feature measurement on the input original image, and obtains feature data of the original image. When performing feature comparison, the image processing module targets each structural component of the original image and / or the feature data of the non-predetermined structural component, in the optimized feature comparison manner, respectively, in the database to find the feature data closest to each structural component and / or no preset structural component Drawing a component, and combining the first image of the new vector graphic with the respective drawing components, that is, the image processing module can be configured for each structural component of the original image and/or without the predetermined structural component The feature data is compared with the feature data of each drawing component in the database, and the structural elements closest to the original image and/or the non-predetermined structural elements are obtained. Each drawing element of the piece, and the resulting drawing elements are combined into a vector graphic, in other words, the vector graphic is composed of each drawing element in the database, and the first image is specific to the original image feature. This vector graphic of artistic style (for example, cartoon style). Moreover, and/or, the image processing module performs edge detection processing on the original image, in order to highlight the shape of each structure, filtering out image information other than the original image shape, and filtering out color and detail texture, in color and detail After the texture is filtered out, the edge features in the image are enhanced and smoothed, resulting in a second image of the line draft.

Then, the image generation module obtains a third image of the cartoon image of the object according to the at least the first image, and the image generation module may further select the second image according to the at least the first image. And obtaining a third image of the cartoon image of the object; in other words, the image generation module generates a third image of the cartoon image of the object based on the at least the first image, and/or the image generation module can be based on the first image The image and the second image are compared with the second image to find the difference in the details. After calculating the difference between the two images, the position of the control points of each vector component at the difference is fine-tuned according to the difference, The iterative optimization approach approximates the most similar shape, resulting in a third image of the cartoon image of the object.

In order to make the person skilled in the art understand the purpose, features and effects of the present invention, the present invention will be described in detail by the following specific embodiments and the accompanying drawings.

1 is a schematic diagram of a system for illustrating the system architecture and operation of the image processing system of the present invention. As shown in FIG. 1 , the image processing system 101 includes an image processing module 102 , an image generation module 103 , and a database 104 . The image processing system 101 can be located in a computer device. For example, the server and/or the notebook computer and/or the desktop computer, and/or the device can be located in a handheld device, such as an Android phone and/or an iPhone and/or an Android tablet and/or an iPad and/or Or iPad2, depending on the actual implementation.

The image processing module 102 is configured to generate an original image (not shown) by feature detection, feature measurement, and feature comparison to generate a cartoon style with original image features. The first image of the vector graphic (not shown); the image processing module 102 first performs feature detection and feature measurement on the input original image to obtain characteristic data of the original image (not The feature processing data of the image processing module 102 for each structural element (not shown) of the original image and/or without a predetermined structural element (not shown) is shown. And optimizing, in the optimized comparison manner, the plurality of drawing elements 410 that are closest to each of the structural elements and/or the feature data of the non-predetermined structural elements are respectively compared in the database 104, and the one is Each of the above drawing elements 410 combines a first image of a new vector graphic, that is, the image processing module 102 can be used for each structural element of the original image and/or characteristic data of the non-predetermined structural element. And database The feature data of each of the drawing elements 410 in 104 is compared, and each drawing element 410 closest to the original image and/or the non-predetermined structural element is obtained, and the respective drawing is obtained. The elements 410 are combined into a vector graphic (first image), in other words, the vector graphic is composed of various drawing elements 410 in the database 104, and the first image is a specific artistic style having original image features (for example, cartoon drawing) The vector graphics of the wind).

In addition, and/or, the image processing module 102 performs edge detection processing on the original image, in order to highlight the shape of each structure, filtering image information other than the original image shape, and filtering out the color and the detailed texture in the color and After the detail texture is filtered, the edge features in the image are enhanced and smoothed, resulting in a second image of the line draft (not shown).

The image generation module 103 is configured to generate a third image (not shown) of the cartoon image of the object based on at least the first image generated by the image processing module 102; and/or Based on the first image and the second image generated by the image processing module 102, the first image and the second image are compared to find a difference in details, and after calculating the difference between the two images, and according to the difference The position of the control points of each vector element at the difference is fine-tuned to approximate the most similar shape in an iteratively optimized manner, thereby obtaining a third image (not shown) of the cartoon image of the object.

The database 104 stores more than one drawing component 410, and each drawing component 410 in the database 104 is drawn in a specific artistic style (for example, cartoon style), and is simultaneously completed at the time of drawing. The measurement of the characteristic data is such that each of the drawing elements has its characteristic data (not shown); the image processing module 102 can be used for the characteristic data of the structural elements of the original image and/or the non-predetermined structural elements, and The feature data of each of the drawing elements 410 is compared to obtain the respective drawing elements 410 that are closest to the structural elements of the original image and/or without the predetermined structural elements.

Figure 2 is a flow chart showing the flow of steps for performing an image processing method using the image processing system of the present invention as shown in Figure 1. As shown in FIG. 2, first, in step 111, first, an original image is input to the image processing module 102, and the process proceeds to step 112. Here, the original image may be an original image with structural features, for example, a person. Face image, in terms of face, because of the relative position of the facial features (the relative positions of the eyes, nose, ears, mouth, and eyebrows), it is, in the following, the detection method of the preset structural features can be used, and The original image can also be an original image without preset structural features. For example, in the case of a house, due to various room types, sizes, structures, and no specific patterns, there is no preset structural feature. Therefore, in the following, It is based on the detection method without preset structure features.

In step 112, the image processing module 102 processes the input original image to generate at least a first image of a vector graphic having a cartoon image of the original image feature, and/or the image processing module 102 inputs the image. The original image is obtained as a second image of the line draft by means of edge detection, and proceeds to step 113.

In step 113, a third image is generated for the cartoon image of the object. The image generation module 103 obtains a third image of the cartoon image of the object according to at least the first image generated by the image processing module 102. The image generating module 103 can obtain a third image of the cartoon image of the object according to the second image generated by the image processing module 102, in other words, according to at least the first image; in other words, The image generation module 103 is based on the first image generated by the image processing module 102 to obtain a third image of the cartoon image of the object, and/or the image generation module 103 can be based on the image processing module. 102, the first image and the second image are generated, and the first image is compared with the second image to find the difference in the details, after calculating the difference between the two images, and fine-tuning the difference according to the difference The position of the control point of the vector element approximates the most similar shape in an iteratively optimized manner, thereby obtaining a third image of the cartoon image of the object.

Figure 3 is a flow chart showing a more detailed procedure for the steps of processing the original image in Figure 2. As shown in FIG. 3, first, in step 211, the image processing module 102 first performs feature detection and feature measurement on the input original image, and obtains feature data of the original image, and proceeds to step 212. .

In step 212, the feature comparison is performed, and the image processing module 102 selects the feature data of the original image and/or the feature data of the non-predetermined structural component in an optimized feature comparison manner in the database 104. Aligning each drawing element 410 that is closest to the feature data of each structural element and/or no preset structural element, and combining the first image of a brand new vector graphic with the respective drawing elements 410, and Go to step 113; that is, the image processing module 102 can perform the feature data of each structural component of the original image and/or the non-predetermined structural component, and the feature data of each of the mapping components 410 in the database 104. Comparing, the respective drawing elements 410 closest to the original image and/or the non-predetermined structural elements are obtained, and the resulting respective drawing elements 410 are combined into a vector graphic, in other words, the vector graphic system It is composed of each drawing element 410 in the database 104, and the first image is the vector graphic having the specific artistic style (cartoon style) of the original image feature.

Figure 4 is a flow chart showing still another more detailed procedure for the steps of processing the original image in Figure 2. As shown in FIG. 4, first, in step 221, the image processing module 102 first performs feature detection and feature measurement on the input original image, and obtains feature data of the original image, and proceeds to step 222. .

In step 222, the feature comparison is performed, and the image processing module 102 selects the feature data of the original image and/or the feature data of the non-predetermined structural component in the optimized database. Aligning each drawing element 410 that is closest to the feature data of each structural element and/or no preset structural element, and combining the first image of a brand new vector graphic with the respective drawing elements 410, and Go to step 113; that is, the image processing module 102 can perform the feature data of each structural component of the original image and/or the non-predetermined structural component, and the feature data of each of the mapping components 410 in the database 104. Comparing, the respective drawing elements 410 closest to the original image and/or the non-predetermined structural elements are obtained, and the resulting respective drawing elements 410 are combined into a vector graphic, in other words, the vector graphic system It is composed of each drawing element 410 in the database 104, and the first image is the vector graphic having the specific artistic style (cartoon style) of the original image feature.

In step 231, the image processing module 102 performs edge detection processing on the original image, so as to highlight the shape of each structure, the image information other than the original image shape is filtered out, and the color and the detailed texture are filtered out, and the color and the detailed texture are filtered. After being filtered out, the edge features in the image are enhanced and subjected to smoothing processing to obtain a second image of the line draft, and the process proceeds to step 113.

Figure 5 is a schematic diagram showing an embodiment of the image processing system of the present invention and its operation. As shown in FIG. 5, the image processing system 101 includes an image processing module 102, an image generation module 103, and a database 104. Here, the image processing system 101 is located in a computer device ( For example, a server (not shown), but the image processing system 101 can also be located in a handheld device (not shown) and its implementation is the same, similar to that described in this embodiment. I will not repeat them here.

The image processing module 102 inputs an original image 200 to the image processing module 102, as shown in FIG. 7 as the original image 200; the image processing module 102 first performs the input original image 200. Feature detection, when the original image 200 is a human face image, the image processing module 102 detects the features of the original image 200 by detecting the preset structural features. The image 200 having the feature points 1, 2, 3, ..., 71, 72, 73 as indicated in Fig. 8 is produced.

Here, since the original image 200 is a human face image with structural features, in terms of a human face, due to the relative position of the facial features (eye, nose, ear, mouth, and relative position of the eyebrow), the preset is available. The feature detection method is performed to obtain a feature point that is consistently defined to facilitate subsequent processing; the image processing module 102 detects and analyzes the color, shape, or texture of the original image 200, and marks the same as 8 feature points 1, 2, 3, ..., 71, 72, 73 having characteristic properties, and only the feature points 1, 2, 3, ..., 71, 72, 73 conforming to the preset feature structure are taken out; The image processing module 102 can predefine each feature point to be pointed out in each part of the facial features, and find corresponding feature points on the input image; in the field of graphic recognition research, the machine learning property can be The algorithm completes this action.

After the image processing module 102 detects the feature points 1, 2, 3, ..., 71, 72, and 73 of the original image 200, the feature measurement as shown in FIG. 9 is performed, and the original image 200 is obtained. Feature data (not shown); before the original image 200 (face image) has preset structural features, the structure (five features) of each feature point 1, 2, 3...71, 72, 73 belongs to Geometric measurements such as position, width, height, and angle can be obtained by using a mathematical calculation formula for the coordinate sets of the relevant feature points 1, 2, 3, ..., 71, 72, and 73 based on the preset structure, and the measurement content includes the positions of the structures. Width, angle, radians, and various quantifiable shape parameters. As shown in FIG. 9, it is used to display a schematic diagram illustrating local measurements, where features can be separately made for the eyes and/or the eyebrows and/or the nose and/or the mouth in the face image (original image) 200. measuring.

After the feature detection and feature measurement are performed on the original image 200, the image processing module 102 performs feature comparison. When performing feature comparison, the image processing module 102 is as shown in FIG. 9 for the original image 200. Structural data of structural elements 201 (eyebrows), structural elements 202 (eyes), structural elements 203 (nose), and structural elements 204 (mouth), as well as hairstyles and face shapes in the original image 200, to optimize features The alignment mode, as shown in FIG. 10, compares each drawing element 420, 430, 440, which is closest to the feature data of each structural element, and the drawing element 450 related to the hairstyle, respectively, in the database 104. Corresponding to the face-shaped drawing component 460, and combining the respective image elements 420, 430, 440, 450, 460 to form a new image of the first image 300 (as shown in FIG. 11), That is, the image processing module 102 can perform the feature data of each of the structural elements 201, 202, 203, and 204 of the original image 200 and the feature data of each of the drawing elements 420, 430, and 440 in the database 104. Compare and get the closest Each of the drawing elements 420, 430, 440 of each structural element of the initial image, and the resulting respective drawing elements 420, 430, 440 and drawing elements 450 and 460 are combined into a vector graphic (first image 300), in other words The vector image is composed of each of the drawing elements 420, 430, 440 and the drawing elements 450 and 460 in the database 104, and the first image 300 is a specific artistic style (cartoon style) having the characteristics of the original image 200. The vector graphic.

The image generation module 103 generates a third image 500 that is a cartoon image of the object. The image generation module 103 is obtained based on at least the first image 300 generated by the image processing module 102. A third image 500 of the cartoon image of the object, as shown in FIG.

a database 104 in which the drawing elements 420, 430, 440, and 450 and 460 are stored, and the respective drawing elements 420, 430, 440, and 450 and 460 in the library 104 are in a specific artistic style ( The cartoon style is drawn, and the measurement of the feature data is completed at the same time, so that each drawing component has its characteristic data (not shown); the image processing module 102 can be used for each structural component 201 of the original image 200. The feature data of 202, 203, and 204 are compared with the feature data of each of the drawing elements 420, 430, and 440, and the respective drawing elements 420, 430, and 440 of the structural elements closest to the original image are obtained. .

Figure 6 is a flow chart showing a flow of steps for performing an image processing method using an embodiment of the image processing system of the present invention as shown in Figure 5. As shown in FIG. 6, first, in step 510, an original image 200 is input to the image processing module 102, and the process proceeds to step 511.

In step 511, feature detection is performed on the original image 200. When the image processing module 102 performs feature detection on the input original image 200, since the original image 200 is a human face image, the image processing module 102 The feature of the original image 200 is detected by the detection of the preset structural feature, and the image 200 having the indicated feature points 1, 2, 3, ..., 71, 72, 73 is generated, and the process proceeds to step 512.

After the image processing module 102 detects the feature points 1, 2, 3, ..., 71, 72, and 73 of the original image 200, the feature measurement is performed, and the feature data of the original image 200 is obtained. Go to step 513.

In step 513, the feature comparison is performed, and the image processing module 102 optimizes the feature data of each structural element 201, 202, 203, 204 of the original image 200 and the hairstyle and face in the original image 200. In the feature comparison manner, the respective drawing elements 420, 430, 440 which are closest to the feature data of the respective structural elements 201, 202, 203, 204, and the drawing element 450 related to the hairstyle are respectively compared in the database 104. The face type drawing component 460 combines the first image 300 of a brand new vector graphic with the respective drawing elements 420, 430, 440, 450, 460, and proceeds to step 514.

In step 514, a third image 500 is generated for the cartoon image of the object. The image generating module 103 obtains a third image of the cartoon image of the object based on at least the first image 300 generated by the image processing module 2. 500.

Figure 7 is a schematic view showing the original image.

Figure 8 is a schematic view showing an image with feature points.

Fig. 9 is a schematic view showing a case where feature measurement is performed on a feature point.

Figure 10 is a schematic diagram showing the drawing elements in the library.

Figure 11 is a schematic view showing the first image.

Figure 12 is a schematic view showing the third image.

Figure 13 is a schematic view showing still another embodiment of the image processing system of the present invention, and the operation. As shown in FIG. 13 , the image processing system 1 includes an image processing module 2 , an image generation module 3 , and a database 4 . The image processing system 101 is located in a handheld device. (for example, an Android mobile phone and/or an iPhone mobile phone) (not shown), but the image processing system 101 can also be located in a computer device (not shown) and its implementation is the same, similar to that described in this embodiment. Therefore, it is not mentioned here.

The image processing module 102 inputs an original image 200 to the image processing module 102, as shown in FIG. 7 as the original image 200; the image processing module 102 first performs the input original image 200. Feature detection, when the original image 200 is a human face image, the image processing module 102 detects the features of the original image 200 by detecting the preset structural features. The image 200 having the feature points 1, 2, 3, ..., 71, 72, 73 as indicated in Fig. 8 is produced.

Here, since the original image 200 is a human face image with structural features, in terms of a human face, due to the relative position of the facial features (eye, nose, ear, mouth, and relative position of the eyebrow), the preset is available. The feature detection method is performed to obtain a feature point that is consistently defined to facilitate subsequent processing; the image processing module 102 detects and analyzes the color, shape, or texture of the original image 200, and marks the same as 8 feature points 1, 2, 3, ..., 71, 72, 73 having characteristic properties, and only the feature points 1, 2, 3, ..., 71, 72, 73 conforming to the preset feature structure are taken out; The image processing module 102 can predefine each feature point to be pointed out in each part of the facial features, and find corresponding feature points on the input image; in the field of graphic recognition research, the machine learning property can be The algorithm completes this action.

After the image processing module 102 detects the feature points 1, 2, 3, ..., 71, 72, and 73 of the original image 200, the feature measurement as shown in FIG. 9 is performed, and the original image 200 is obtained. Feature data (not shown); before the original image 200 (face image) has preset structural features, the structure (five features) of each feature point 1, 2, 3...71, 72, 73 belongs to Geometric measurements such as position, width, height, and angle can be obtained by using a mathematical calculation formula for the coordinate sets of the relevant feature points 1, 2, 3, ..., 71, 72, and 73 based on the preset structure, and the measurement content includes the positions of the structures. Width, angle, radians, and various quantifiable shape parameters. As shown in FIG. 9, it is used to display a schematic diagram illustrating local measurements, where features can be separately made for the eyes and/or the eyebrows and/or the nose and/or the mouth in the face image (original image) 200. measuring.

After the feature detection and feature measurement are performed on the original image 200, the image processing module 102 performs feature comparison. When performing feature comparison, the image processing module 102 is as shown in FIG. 9 for the original image 200. Structural data of structural elements 201 (eyebrows), structural elements 202 (eyes), structural elements 203 (nose), and structural elements 204 (mouth), as well as hairstyles and face shapes in the original image 200, to optimize features The alignment mode, as shown in FIG. 10, compares each drawing element 420, 430, 440, which is closest to the feature data of each structural element, and the drawing element 450 related to the hairstyle, respectively, in the database 104. Corresponding to the face-shaped drawing component 460, and combining the respective image elements 420, 430, 440, 450, 460 to form a new image of the first image 300 (as shown in FIG. 11), That is, the image processing module 102 can perform the feature data of each of the structural elements 201, 202, 203, and 204 of the original image 200 and the feature data of each of the drawing elements 420, 430, and 440 in the database 104. Compare and get the closest Each of the drawing elements 420, 430, 440 of each structural element of the initial image, and the resulting respective drawing elements 420, 430, 440 and drawing elements 450 and 460 are combined into a vector graphic (first image 300), in other words The vector image is composed of each of the drawing elements 420, 430, 440 and the drawing elements 450 and 460 in the database 104, and the first image 300 is a specific artistic style (cartoon style) having the characteristics of the original image 200. The vector graphic.

Moreover, the image processing module 102 performs edge detection processing on the original image 200, and converts the original image 200 into a second image 400 as a line draft as shown in FIG. 15; here, in order to highlight each structure The shape will filter out the image information other than the original image shape, and filter out the color and the detailed texture. After the color and the detailed texture are filtered out, the edge features in the image are strengthened and smoothed, and the line draft is obtained. The second image 400, the second image 400 obtained here, or slightly non-realistic drawing style.

The image generation module 103 generates a third image 600 that is a cartoon image of the object. The image generation module 103 is based on the first image 300 and the second image 400 generated by the image processing module 102. Aligning the first image 300 with the second image 400, as shown in FIG. 16, finding the difference in detail, after calculating the difference between the two images, and fine-tuning the vector elements at the difference according to the difference Controlling the position of the point, in an iteratively optimized manner, as shown in Figure 17, approximating the most similar shape, thereby yielding a third image 600 of the cartoon image of the object (as shown in Figure 18) Wherein, when calculating the difference between the two graphs, the first image 300 of the vector graphics is pre-transformed into a bitmap format, and then the difference value is compared with the second image 400 of the line draft to determine the displacement of the control point. After obtaining the deformed vector graph, if it is still necessary to continue the iterative approximation, continue to draw the bitmap, the difference comparison, the movement control point, and the like.

a database 104 in which the drawing elements 420, 430, 440, and 450 and 460 are stored, and the respective drawing elements 420, 430, 440, and 450 and 460 in the library 104 are in a specific artistic style ( The cartoon drawing style is drawn, and the measurement of the characteristic data is completed at the same time, so that each drawing element has its characteristic data (not shown); the image processing module 102 can be used for each structural element 201 of the original image, The feature data of 202, 203, and 204 is compared with the feature data of each of the drawing elements 420, 430, and 440, and the respective drawing elements 420, 430, and 440 of the structural elements closest to the original image are obtained.

Figure 14 is a flow chart showing still another flow of steps for performing an image processing method using yet another embodiment of the image processing system of the present invention as in Figure 13. As shown in FIG. 14, first, in step 181, an original image 200 is input to the image processing module 102, and proceeds to step 182 and step 192.

In step 182, feature detection is performed on the original image 200. When the image processing module 102 performs feature detection on the input original image 200, since the original image 200 is a face image, the image processing module 102 The feature of the original image 200 is detected by the detection of the preset structural feature, and the image 200 having the indicated feature points 1, 2, 3, ..., 71, 72, 73 is generated, and the process proceeds to step 183.

In step 183, after the image processing module 102 detects the feature points 1, 2, 3, ..., 71, 72, and 73 of the original image 200, the feature measurement is performed, and the feature data of the original image 200 is obtained. Go to step 184.

In step 184, the feature comparison is performed, and the image processing module 102 optimizes the feature data of each structural element 201, 202, 203, 204 of the original image 200 and the hairstyle and face in the original image 200. In the feature comparison manner, the respective drawing elements 420, 430, 440 which are closest to the feature data of the respective structural elements 201, 202, 203, 204, and the drawing element 450 related to the hairstyle are respectively compared in the database 104. The face-shaped drawing component 460 combines the first image 300 of a brand new vector graphic with the respective drawing elements 420, 430, 440, 450, 460, and proceeds to step 185.

In step 192, the image processing module 102 performs edge detection processing on the original image 200, and converts the original image 200 into the second image 400 of the line draft, and proceeds to step 185; here, in order to highlight the shape of each structure, Image information other than the original image shape will be filtered out, and the color and detail texture will be filtered out. After the color and the detailed texture are filtered out, the edge features in the image are strengthened and smoothed, and the line draft is obtained. Two images 400.

In step 185, a third image 600 is generated for the object cartoon image. The image generation module 103 is based on the first image 300 and the second image 400 generated by the image processing module 102. The two images 400 are compared to find the difference in the details. After calculating the difference between the two images, and fine-tuning the position of the control points of the vector elements at the difference according to the difference, the iterative optimization method is used to approximate the most similar The shape is used to derive a third image 600 of the cartoon image of the object.

Figure 15 is a schematic view showing a second image illustrated as a line draft.

Figure 16 is a schematic diagram showing the difference in detail of the comparison between the first image and the second image.

Figure 17 is a schematic diagram for illustrating the calculation of the difference between the first image and the second image, and approximating the most similar shape in an iteratively optimized manner.

Figure 18 is a schematic view showing a third image showing a cartoon image of an object.

Figure 19 is a schematic view showing still another embodiment and operation of the image processing system of the present invention. As shown in FIG. 19, the image processing system 101 includes an image processing module 102, an image generation module 103, and a database 104. Here, the image processing system 101 can be located in a computer device. For example, the server and/or the notebook computer and/or the desktop computer, and/or the device can be located in a handheld device, such as an Android phone and/or an iPhone and/or an Android tablet and/or an iPad and/or Or iPad2, depending on the actual implementation.

The image processing module 102 inputs an original image 700 to the image processing module 102. The original image 700 is shown in FIG. 21; the image processing module 102 first performs the input original image 700. Feature detection, when the feature image is detected, since the original image 700 is a house image, the image processing module 2 detects the feature of the original image 700 by detecting the feature without a preset structure. The image 800 having the feature points indicated in FIG. 22 is generated; here, the original image 700 is an original image without a preset structural feature. In the case of a house, there are no specific modes due to various room types, sizes, structures. So there are no default structural features.

The image processing module 102 detects and analyzes the color, shape, or texture of the original image 700, and marks each feature point having the characteristic property as shown in FIG. 22, and extracts only the features corresponding to the preset feature structure. The feature points, the obtained feature points without the preset structure, can be combined into groups to obtain the basic geometric shape, combined one by one, or merged into a straight line, a triangle, a rectangle, an arc, a circle and Oval shape, etc., after the completion of the geometric component merge action, feature measurement can be performed.

After the image processing module 102 detects the feature points of the original image 700, the feature measurement as shown in FIG. 23 is performed, and the components 801, 802, 803, 804, and 805 as shown in FIG. 24 are calculated. The feature measurement parameters of 806, 807, and 808 are used to obtain feature data (not shown) of the original image 700.

After the feature detection and feature measurement is performed on the original image 700, the image processing module 102 performs feature comparison. When performing feature comparison, the image processing module 102 is configured as shown in FIG. 24 for the original image 700. The feature data of each component 801, 802, 803, 804, 805, 806, 807, 808, as shown in FIG. 25, is respectively compared in the database 4 to find the feature data closest to each structural component. Each of the drawing elements 401, 402, 403, 404, 405, 406, 407, 408 combines a first image 900 of a brand new vector graphic (as shown in FIG. 26), that is, the image processing The module 102 can be used for the feature data of the components 801, 802, 803, 804, 805, 806, 807, 808 of the original image 700, and the respective drawing components 401, 402, 403, 404, 405 in the database 104. The feature data of 406, 407, 408 are compared, and each drawing element 401, 402, 403, 404, 405, 406, 407, 408 of each structural element closest to the original image is obtained, and the resulting Each of the drawing elements 401, 402, 403, 404, 405, 406, 407, 4 08 is combined into a vector graphic (first image 900), in other words, the vector graphic is composed of each of the drawing elements 401, 402, 403, 404, 405, 406, 407, 408 in the database 104, and the first image 900 is the vector graphic of the specific artistic style (cartoon style) with the original image 700 feature.

Moreover, when the feature comparison is performed, the image processing module 2 has the feature data of each component 801, 802, 803, 804, 805, 806, 807, 808 of the original image 700 as shown in FIG. As shown in FIG. 27, the respective drawing elements 501, 502, 503, 504, 505, 506, 507, 508 which are closest to the feature data of the respective structural elements are respectively compared in the database 104. A first image 910 of a new vector graphic (as shown in FIG. 28), where the first image 900 and the first image 910 have different image styles, because the components in the component library 104 are When the style of painting is different, the image styles obtained by the combination are also different.

The image generation module 103 generates a third image 610 which is a cartoon image of the object, at least according to the first image 900 generated by the image processing module 102, as shown in FIG. And/or, depending on the style of the painting, the image generating module 103 obtains a third image 620 that is a cartoon image of the object, at least according to the first image 910 generated by the image processing module 102. Figure 30 shows.

a database 104 in which the drawing elements 401, 402, 403, 404, 405, 406, 407, 408 and 501, 502, 503, 504, 505, 506, 507, 508 are stored in the database 4 Each of the drawing elements 401, 402, 403, 404, 405, 406, 407, 408 and 501, 502, 503, 504, 505, 506, 507, 508 is drawn in a specific artistic style, at the time of drawing At the same time, the measurement of its characteristic data is completed, so that each drawing element has its characteristic data (not shown).

For the case where the image processing module 102 performs edge detection processing on the original image 700 and converts the original image 700 into a second image (not shown) of the line draft, the same is true, similar to FIG. 13 and The contents in the description of Fig. 19 are the same and will not be described again.

Figure 20 is a flow chart showing still another flow of steps for performing an image processing method using still another embodiment of the image processing system of the present invention as in Figure 19. As shown in FIG. 20, first, in step 281, an original image 700 is input to the image processing module 102, and the process proceeds to step 282.

In step 282, feature detection is performed on the original image 700. When the image processing module 102 performs feature detection on the input original image 700, since the original image 700 is a house image, the image processing module 102 The method for detecting the feature of the original image 700 is detected by the feature of the original image 700, and the image 800 having the marked feature point is generated, and the process proceeds to step 283.

In step 283, after the image processing module 102 detects the feature points of the original image 700, the feature measurement is performed to obtain the feature data of the original image 700, and the process proceeds to step 284.

In step 284, feature matching is performed, and the image processing module 2 compares the feature data of each component 801, 802, 803, 804, 805, 806, 807, 808 of the original image 700 with optimized features. In the manner, the respective drawing elements 401, 402, 403, 404, 405, 406, 407, 408 which are closest to the feature data of each structural element are respectively found in the database 104, and a combination of the elements is combined. The first image 900 of the brand new vector graphic proceeds to step 285.

In step 285, a third image 610 is generated for the cartoon image of the object. The image generation module 103 obtains a third image of the cartoon image of the object based on at least the first image 900 generated by the image processing module 102. 610.

Here, for the first image 910 composed of the drawing elements 501, 502, 503, 504, 505, 506, 507, 508, the flow processing steps of the image processing method are the same, similar to the above, I will not repeat them here.

Figure 21 is a schematic view showing another original image.

Figure 22 is a schematic diagram showing an image showing the feature points indicated.

Figure 23 is a schematic diagram showing feature measurements illustrating images with feature points.

Figure 24 is a schematic diagram showing the characteristic parameters of each component.

Figure 25 is a schematic diagram showing the characteristic parameters of each component.

Figure 26 is a schematic view showing a first image.

Figure 27 is a schematic diagram showing the various drawing elements.

Figure 28 is a schematic view showing another first image.

Figure 29 is a schematic view showing a third image.

Figure 30 is a schematic view showing another third image.

In summary of the above embodiments, we can obtain an image processing system and method of the present invention, which is applied to a non-realistic artistic (eg, cartoon) image generation environment, using the image processing system of the present invention to perform an image method flow. Inputting an original image, and performing feature detection, feature measurement, and feature comparison to obtain a first image of a vector graphic having a cartoon image of the original image feature, and/or the input original image And obtaining, by means of edge detection, a second image of the line draft; and then, based on the first image, obtaining a third image of the cartoon image of the object, and/or based on the first image and the second image Comparing the first image with the second image to find the difference in the details, after calculating the difference between the two images, and fine-tuning the position of the control points of the vector elements at the difference according to the difference, to iteratively optimize the In this way, the most similar shape is approximated, thereby obtaining a third image of the cartoon image of the object. The image processing system and method of the present invention includes the following advantages:

1. Input an original image, and obtain a first image through feature detection, feature measurement, and feature comparison, and based on the first image, obtain a third image of the cartoon image of the object.

2. inputting an original image, and obtaining a first image by means of feature detection, feature measurement, and feature comparison, and the original image is obtained by edge detection to obtain a second image, based on the first image The image and the second image result in a third image of the cartoon image of the object.

3. Based on the first image and the second image, compare the first image with the second image to find the difference in the details, after calculating the difference between the two images, and fine-tuning the vector elements of the difference according to the difference The position of the control point is approximated to the most similar shape in an iteratively optimized manner, thereby obtaining a third image of the cartoon image of the object.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the present invention; all other equivalent changes or modifications which are not departing from the spirit of the present invention should be included in the following patents. Within the scope.

1, 2, 3...71, 72, 73. . . Feature points

101. . . Image processing system

102. . . Image processing module

103. . . Image generation module

104. . . database

111 112 113. . . step

181 182 183 184 185 192. . . step

200. . . Original image

201. . . Structural component

202. . . Structural element

203. . . Structural element

204. . . Structural element

211 212. . . step

221 222 231. . . step

281 282 283 284 285. . . step

300. . . First image

400. . . Second image

401 402 403 404 405 406 407 408. . . Drawing component

410. . . Drawing component

420. . . Drawing component

430. . . Drawing component

440. . . Drawing component

450. . . Drawing component

460. . . Drawing component

500. . . Third image

501 502 503 504 505 506 507 508. . . Drawing component

511 512 513 514. . . step

600. . . Third image

610. . . Third image

620. . . Third image

700. . . Original image

800. . . image

801 802 803 804 805 806 807 808. . . element

900. . . First image

910. . . First image

1 is a schematic diagram of a system for illustrating the system architecture and operation of the image processing system of the present invention;

Figure 2 is a flow chart showing the flow of steps for performing an image processing method using the image processing system of the present invention as shown in Figure 1;

Figure 3 is a flow chart showing a more detailed procedure for the steps of processing the original image in Figure 2;

Figure 4 is a flow chart showing still another more detailed procedure for the steps of processing the original image in Figure 2;

Figure 5 is a schematic view showing an embodiment of the image processing system of the present invention and its operation;

Figure 6 is a flow chart showing a flow of steps for performing an image processing method using an embodiment of the image processing system of the present invention as in Figure 5;

Figure 7 is a schematic view showing the original image;

Figure 8 is a schematic view showing an image showing a feature point;

Figure 9 is a schematic view showing a situation in which feature measurement is performed on a feature point;

Figure 10 is a schematic diagram showing the drawing elements in the description database;

Figure 11 is a schematic view showing the first image;

Figure 12 is a schematic view showing the third image;

Figure 13 is a schematic view showing another embodiment and operation of the image processing system of the present invention;

Figure 14 is a flow chart showing another flow of steps for performing an image processing method using still another embodiment of the image processing system of the present invention as in Figure 13;

Figure 15 is a schematic view showing a second image illustrated as a line draft;

Figure 16 is a schematic view showing the difference in details of the comparison between the first image and the second image;

Figure 17 is a schematic diagram for illustrating the calculation of the difference between the first image and the second image, and approximating the most similar shape in an iteratively optimized manner;

Figure 18 is a schematic view showing a third image showing a cartoon image of the object;

Figure 19 is a schematic view showing still another embodiment and operation of the image processing system of the present invention;

Figure 20 is a flow chart for showing another flow of steps for performing an image processing method using still another embodiment of the image processing system of the present invention as in Figure 19;

Figure 21 is a schematic view showing another original image;

Figure 22 is a schematic view showing an image showing the feature points marked;

Figure 23 is a schematic view showing the feature measurement of the image with the feature points;

Figure 24 is a schematic view showing the characteristic parameters of each component;

Figure 25 is a schematic view showing the characteristic parameters of each component;

Figure 26 is a schematic view showing a first image;

Figure 27 is a schematic view showing the various drawing elements;

Figure 28 is a schematic view showing another first image;

Figure 29 is a schematic view showing a third image; and

Figure 30 is a schematic view showing another third image.

111. . . step

112. . . step

113. . . step

Claims (8)

An image processing method is applied to a non-realistic artistic image generation environment, which comprises the following steps: inputting an original image to an image processing module; the image processing module adopts a preset structural feature to the original Performing a feature detection on the image to define a feature point of the original image; the image processing module performs a feature measurement on the feature point to obtain a shape parameter that can be quantized by each feature point; the image processing module is In a database, a drawing component closest to each of the shape parameters is separately compared, and each drawing component is combined to form a first image of a vector graphic having the original image feature; The original image performs an edge detection process, which filters image information and detailed textures other than the original image shape to enhance edge features of the original image to generate a second image of the line draft; and an image generation module a group, the first image is compared with the second image to find a difference in detail between the first image and the second image, and Each element of the vector control of the position fine adjustment decapitated, in an iterative optimization manner, approximation of the most similar shape, so as to obtain a third image as a cartoon image of the object. The image processing method of claim 1, wherein the original image is an image having no structural features, and the feature detecting mode is a detecting mode without a preset structural feature. The image processing method according to claim 1, wherein, in the feature comparison mode, the feature data of the component of the original image is compared by an optimized feature comparison method. Finding each of the drawing elements that are closest to each of the shape parameters, and combining the first image with each of the drawing elements. The image processing method according to claim 2, wherein, in the feature comparison mode, the feature data of the component of the original image is compared by an optimized feature comparison method. Each of the drawing elements closest to each of the shape parameters is found, and the first image is combined with the respective drawing elements. The image processing method of claim 3, wherein the first image is a vector graphic having a cartoon style of the original image feature. The image processing method of claim 4, wherein the first image is a vector graphic having a cartoon style of the original image feature. An image processing system is applied to a non-realistic art image generation environment, comprising: a database storing more than one drawing component; an image processing module, an input original image, by the image The processing module obtains, by a feature detection, a feature measurement, and a feature comparison, a first image having the original image feature composed of one or more drawing elements in the database; and the input The original image, the edge detection process is a second image of the line draft; and an image generation module, the first image is compared with the second image to find the first image The difference between the image and the second image is finely adjusted, and the position of the control point of each vector element at each difference is fine-tuned, and the most similar shape is approximated by an iterative optimization method, thereby obtaining A third image of the cartoon image of the object. The image processing system of claim 7, wherein the original image is an image with a structural feature, and the feature detection mode is a method for detecting a predetermined structural feature.