KR20220057391A - Image Processing Method and Apparatus Using Thereof - Google Patents

Abstract

An image processing method according to the present invention includes the steps of: receiving at least one image data; assigning a plurality of feature points to the image data according to a predetermined criterion; and changing a part of the image data based on a feature line created by connecting at least two of the feature points or a feature area created by connecting at least three of the feature points.

Description

Image Processing Method and Apparatus Using Thereof

The present invention relates to an image processing method and an apparatus using the same.

The smile line refers to the tooth line, the shape of the teeth, the degree of exposure to the gingiva when smiling, the gingival line, the length and thickness of the lips, the length of the pharynx, the muscles around the lips and the activity level and direction of the muscles related thereto. is made With orthodontic treatment, the patient can have a desired smile line. In particular, patients who are considering treatment with veneer or laminate are curious about how their face (more specifically, a smile line) changes after treatment. A person's impression changes depending on the smile line, and making such a smile line through orthodontic treatment or surgery is sometimes called a smile design.

On the other hand, the smile design selects an optimal orthodontic template for the patient while applying templates having information on the corrected teeth on the image data showing the patient's face. In this process, there is a possibility of sharing templated image data between therapists or patients, and sharing of image data exposes the patient's face and may infringe personal privacy.

Therefore, there is a need for a method that can guarantee the anonymity of the patient even by the image data to which the template is applied.

Republic of Korea Patent Publication KR10-2007-0093841 A (published on Sep. 14, 2007)

In order to solve the above problems, an image processing method according to the present invention receives image data having human face data, assigns a plurality of feature points to the image data, and based on a feature line or a feature area generated from the feature points An image processing method for changing a part of image data is provided.

In addition, there is provided an image processing apparatus for changing a part of image data by performing the image processing method.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

The image processing method according to the present invention includes the steps of receiving image data, assigning a plurality of feature points to the image data, and a feature line generated by connecting at least two of the feature points, or at least three of the feature points and changing a portion of the image data based on the feature region generated by concatenation.

In addition, the image processing apparatus according to the present invention provides a database unit in which input image data is stored, a plurality of feature points are assigned to the input image data, and a portion of image data is generated based on a feature line or a feature area generated from the feature points. It includes a control unit to change.

According to the present invention, the user or patient can change a part of the image data when sharing the patient's image data with others to ensure the patient's anonymity, and the user or patient can discuss the template applied to the image data through discussion. This has the advantage of being able to establish an optimal remediation plan.

1 and 2 are flowcharts of an image processing method according to the present invention.
3 is a diagram for explaining image data.
4 is a diagram for explaining that feature points are provided.
5 is a diagram for explaining generation of a feature line and a feature region based on some of feature points assigned to image data.
6 and 7 are diagrams for explaining a process of fitting image data.
8 is a diagram for explaining application of a template to image data.
9 is a diagram for explaining a second feature region in the image processing method according to the present invention.
FIG. 10 is another embodiment of FIG. 9 .
11 and 12 are diagrams for explaining a change in the second characteristic area of image data.
13 is a diagram for explaining a blur processing process.
14 is a diagram for explaining a Gaussian blur processing process.
15 is a block diagram of an image processing apparatus according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail with reference to exemplary drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function interferes with the understanding of the embodiment of the present invention, the detailed description thereof will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the elements from other elements, and the essence, order, or order of the elements are not limited by the terms. In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present application. does not

1 and 2 are flowcharts of an image processing method according to the present invention, and FIG. 3 is a diagram for explaining image data.

1 to 3 , the image processing method according to the present invention includes a step of receiving image data (S110), a step of assigning a plurality of feature points to the image data according to a predetermined criterion (S120), and among the feature points The method may include changing a part of the image data based on a feature line generated by connecting at least two or a feature region generated by connecting at least three of the feature points ( S140 ).

Hereinafter, each step of the image processing method according to the present invention will be described in detail.

In step S110 of receiving image data, at least one image data of a pre-imaged patient may be input. Here, the image data is stored in a data storage element called a database unit (DB unit), and may be input by a user's manipulation. The database unit (DB unit) where image data is stored is a physical hard disk drive (Hard Disk Drive), a floppy disk drive (Floppy Disk Drive), a solid state drive (Solid State Drive), a flash memory drive (Flash Memory Drive), etc. It may be a storage device, or it may be a cloud-type virtual database.

Preferably, the image data may be human face data. More specifically, the image data may be face data 400 in which a person's eyes, nose, mouth, etc. are visually displayed, and in particular, may be face data in which a person's mouth is opened and teeth are exposed. The face data 400 may be general image data having a natural smile, and portions of teeth and gingiva may be exposed as both corners of the mouth rise upward. A portion of the exposed teeth and gingiva may be included in a feature region to be described later, and a template for orthodontic treatment may be virtually applied within the region. In this case, the template refers to a sample model of orthodontic teeth, and the template may have standard information on the position, size, and shape of at least one tooth. The user and the patient can predict the shape of the teeth after orthodontic treatment by overlaying the template on the patient's teeth among the image data. In addition, a plurality of templates may be provided in order to provide an optimal orthodontic treatment to the patient, and the user and the patient may determine a template that best matches the patient's teeth by selectively applying the plurality of templates on the patient's teeth among the image data. can

Meanwhile, the face data may be general face data having a smile line, or face data with a larger opening than general face data by forcibly opening the mouth with a mouthpiece or the like. The general face data is used by the user and the patient to establish a correction plan by applying the template, and the opening face data can be used by the laboratory to produce the correction. In this case, the orthodontic product may mean an object worn by the patient. Illustratively, the orthodontic appliance may refer to a veneer, a prosthesis including a laminate, and an orthodontic appliance.

The input image data may be displayed on a user interface (UI), and the image data may be displayed to a user and a patient through an output device. Various types of devices may be used as the output device, and among them, a display device capable of visually displaying image data may be used.

Hereinafter, a process in which feature points are assigned to image data will be described.

4 is a diagram for explaining that feature points are provided.

Referring to FIG. 4 , in the step of assigning the feature points ( S120 ), a plurality of feature points P may be assigned to the feature parts on the image data according to a predetermined criterion based on the face data. More specifically, the characteristic parts may be an eyebrow, an eye contour between the upper and lower eyelids, a nose, an outer lip and an inner lip, and a face contour. The number of the feature points P provided in the step of assigning the feature points ( S120 ) may be a number that can effectively represent the characteristic parts of the above-described image data.

Exemplarily, the step of giving the feature points ( S120 ) may use an AI Facial Landmarks Detection method to express characteristic parts of image data. Hereinafter, the present invention has been described based on the Face Landmark 68 algorithm that assigns 68 feature points (P) to the image data among the AI Facial Landmarks Detection methods, but the present invention is not limited thereto. Any algorithm can be used.

Meanwhile, when the image data is input, if the input image data is excessively rotated in one direction, the feature points P may not be smoothly provided to the image data. Accordingly, in the step of assigning the feature points ( S120 ), a plurality of feature points P may be assigned according to a predetermined criterion based on the face data 400 while rotating the image data at a predetermined angular interval.

Illustratively, in the step of assigning the feature points ( S120 ), the feature points P may be assigned while rotating the image data in a clockwise or counterclockwise direction at 90° intervals. According to another example, in the step of assigning the feature points ( S120 ), the feature point P may be assigned while rotating the image data clockwise or counterclockwise at 15° intervals. In this way, by rotating the image data and giving the feature points P, accurate recognition of the image data and the assignment of the feature points P are possible, and there is an advantage in that it is possible to change a part of the image data in a normal position and apply a template.

According to another embodiment, the step of assigning the feature points ( S120 ) uses artificial intelligence to select image data that best matches the face data 400 from among a plurality of two-dimensional image data and continuous video data, and select the selected image. Feature points P may be assigned to data. When the step of assigning feature points using artificial intelligence ( S120 ) is performed, even if the user does not directly input image data that best matches the face data 400 , appropriate image data is automatically selected and the feature points P are given. There is an advantage.

Hereinafter, as an additional step in the image processing method according to the present invention, the step of fitting image data based on feature points assigned to the image data ( S130 ) will be described.

5 is a diagram for explaining generation of a feature line and a feature region based on some of the feature points assigned to image data, and FIGS. 6 and 7 are diagrams for explaining a process of fitting image data.

5 to 7 , when feature points are provided on the image data, a step of fitting ( S130 ) of fitting the image data to have at least one of a predetermined size or a predetermined angle based on the assigned feature points is additionally performed can be That is, the position of the feature points P, the distance between the feature points P, and the angle formed by the feature points P can be obtained, and such information can be used as an element for performing the image processing method according to the present invention.

In the fitting step ( S130 ), a predetermined feature line may be generated from the feature points (P) provided in the step ( S120 ) of assigning the aforementioned feature points. Exemplarily, the feature line may be generated by connecting at least two feature points among the assigned feature points, and the generated feature line is feature points P1' and P2' corresponding to the lateral canthus of each eye in the face data. It may be a line connecting This characteristic line is called a first characteristic line L1' or an eye line. Meanwhile, the key points P1' and P2' may be the No. 37 key point and No. 46 key point, respectively, but these are merely exemplary and are not limited thereto.

In the fitting ( S130 ), the size of the image data may be enlarged or reduced based on the generated feature line. More specifically, in the fitting ( S130 ), the size of the image data may be enlarged or reduced so that the length of the first feature line L1 ′ matches a predetermined length. Exemplarily, when the length of the first feature line L1' is 300 pixels and the preset target length is 400 pixels, the image data is multiplied by a constant value so that the fitted first feature line L1 has a length of 400 pixels. size can be enlarged. On the other hand, when the length of the first feature line L1' is greater than the preset target length, the size of the image data is reduced by multiplying the length of the fitted first feature line L1 by a predetermined value so that the length becomes the preset target length. can

In addition, the fitting ( S130 ) may rotate the image data based on the generated feature line. In order to rotate the image data, the fitting ( S130 ) may rotate the image data based on a second characteristic line L2 ′ that is different from the first characteristic line L1 ′. More specifically, in the fitting step ( S130 ), a second feature line (or midpoint) generated by connecting a point of the first feature line L1 ′ and a feature point spaced apart from the first feature line L1 ′. Line, L2') can be used to rotate the image data. The second characteristic line L2' may be generated by connecting the central point P3' and the characteristic point P4' of the first characteristic line L1'. Exemplarily, the feature point P4 ′ may correspond to the feature point number 34.

Based on the generated second characteristic line L2', in the fitting step S130, the image data is fitted so that a smile line can be designed at a normal angle. For example, an angle of the fitted second feature line L2 may be perpendicular to a horizontal direction among a user interface (UI) plane on which image data is input and displayed.

As described above, by fitting the face data 400 ′ to match the image data to at least one of a predetermined size and a predetermined angle, the user can accurately apply a template to the fitted face data 400 and establish a highly reliable correction plan. can do.

Optionally, when the angle of the image data is different from the user's intention, the user may manually rotate the image data by a predetermined angle. Illustratively, the image data may be rotated clockwise or counterclockwise around the center point P3 of the fitted first feature line L1, and the predetermined angle at which the image data may be rotated is clockwise. Alternatively, it may be within about 5° in a counterclockwise direction. In this way, by manually finely rotating the image data by a predetermined angle according to the user's judgment, the user can acquire image data that is more suitable for image data analysis and smile line design.

Hereinafter, a process of applying a template to image data will be described.

8 is a diagram for explaining application of a template to image data.

5 and 8 , after the fitting of the image data is completed, a step (not shown) of designing a smile line by applying at least one template to a predetermined area of the image data based on the feature point may be performed. The template may be orthodontic data pre-stored in the database, and a plurality of templates may be provided so that an optimal template may be applied to the patient's image data. The template can be customized by the user according to the oral condition of the patient. For example, the user may change the shape of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template. At this time, the shape of the teeth may be changed by adjusting the tooth contours forming the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template. In addition, the user may change the position, size, color, or shade of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template. By changing the positions or sizes of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template, the positions or sizes of adjacent teeth may be changed in conjunction with each other. In addition, by changing the color or shade of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template, a comprehensive dental treatment plan in consideration of whitening procedures other than prosthetics may be considered. By customizing the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template in this way, the user can provide an optimal treatment plan to the patient.

Meanwhile, in the designing step, a template may be applied to a feature area generated by connecting at least three feature points among the plurality of feature points given in the step of assigning the feature points ( S120 ). More specifically, a lipline generated by connecting the feature points forming the inside of the lips among the feature points given on the image data may be formed as the first feature region C1 among the feature regions, and the first feature region ( C1) A template may be applied on the tooth images Ta', Tb', Tc', Td', Te', and Tf' existing therein. More specifically, in the designing step (S160), the feature area C1 generated by connecting at least three feature points is recognized as a lip line (lip line recognition step), and a tooth image is displayed inside the feature area C1. The region is recognized (tooth region recognition step), and a template matching algorithm can be used for the tooth region to match the tooth image with the template inside the feature region C1 constituting the lip line (template matching step). At this time, the template matching algorithm is a method of matching the template to the position of the tooth image expressed inside the feature area (C1), taking into consideration factors such as shape, size, or color, automatically to the tooth area having elements similar to the template. to match the template. By applying the template to the feature region as described above, it is possible to prevent the template from being erroneously applied to a region other than the first feature region C1 of the image data.

On the other hand, when the feature region C1 is generated, the positions of the feature points P are unclear, and thus an erroneous feature region may be generated. In this case, the contours of the feature points P and the first feature region C1 may be corrected using the feature point correction function formed on the user interface, and a more accurate feature region acquisition and template application process may be performed.

Hereinafter, the step ( S140 ) of changing a part of image data based on the feature line or the feature region will be described in detail.

9 is a diagram for explaining a second characteristic region in the image processing method according to the present invention, FIG. 10 is another embodiment of FIG. 9, and FIGS. 11 and 12 are diagrams for explaining a change in the second characteristic region of image data It is a road for Also, FIG. 13 is a diagram for explaining a blur processing process, and FIG. 14 is a diagram for explaining a Gaussian blur processing process.

5 and 9 , in the changing (S140), when a plurality of feature points (P) are given to image data, a feature line or feature area generated by connecting at least two of the feature points (P) is selected. It is possible to change a part of the image data based on it. In this case, 'changing a part of the image data' may mean performing a predetermined process on the part to make it difficult to identify whose face data the image data has.

In the following, the predetermined processing has been described as an example of changing a part of image data using a predetermined color or a predetermined pattern. However, the predetermined processing is not limited thereto, and may be interpreted as meaning various processing such as cutting a part of the image data or inserting a figure into a part of the image data.

The feature region may include the first feature region C1 to which the template is applied as described above. Also, the feature region may further include a second feature region C2 generated based on at least two feature points in the face data 400 . In this case, the second characteristic region C2 may be formed to be spaced apart from the first characteristic region C1 to which the template is applied. Accordingly, even if the inside of the second feature region C2, which will be described later, is changed, the first feature region C1 may not be affected, and the template applied to the first feature region C1 may not be damaged. A configuration in which the inside of the second characteristic region C2 is changed will be described later.

Meanwhile, the second feature area C2 may include a first feature point P1 and a second feature point P2 forming an eye outline among the face data 400 . More specifically, the first feature point P1 and the second feature point P2 may be feature points corresponding to the lateral canthal angle of each eye among the face data in the above-described fitting step S130 . Exemplarily, the feature points P1 and P2 may be feature points 37 and 46, respectively.

Since the second feature region C2 includes the first feature point P1 and the second feature point P2 , the second feature region C2 including the patient's eye part in the face data 400 may be changed. .

A range covered by the second characteristic region C2 will be described in more detail. A first feature line L1 generated by connecting the first feature point P1 and the second feature point P2 may be included. At this time, the second characteristic region C2 is a first point Pa spaced apart from the first characteristic point P1 by a first distance da in the longitudinal direction of the first characteristic line L1, and the first characteristic line A second point Px spaced apart by a second distance dx from the second feature point P2 in the longitudinal direction of L1 may be included. The first point Pa and the second point Px may be formed side by side in the longitudinal direction of the first characteristic line L1 , and may be formed in a direction opposite from the first characteristic line L1 . In addition, the first point Pa and the second point Px may form a horizontal length of the second characteristic region C2 .

In addition, the second characteristic region C2 includes a third point Pb spaced apart from the first characteristic point P1 by a third distance db in a direction perpendicular to the longitudinal direction of the first characteristic line L1, a third The fourth point Pc may include a fourth point Pc spaced apart from the first feature point P1 by a fourth distance dc in a direction opposite to the direction in which the point Pb is formed. In this case, the third point Pb and the fourth point Pc may constitute horizontal lines Lby and Lcz of the second characteristic region C2, respectively.

The second characteristic region C2 is a fifth point Py spaced apart from the second characteristic point P2 by a fifth distance dy in a direction perpendicular to the longitudinal direction of the first characteristic line L1, and a fifth point A sixth point Pz spaced apart from the second feature point P2 by a sixth distance dz in a direction opposite to the direction in which Py is formed may be included. In this case, the fifth point Py and the sixth point Pz may constitute horizontal lines Lby and Lcz of the second characteristic region C2, respectively.

Referring to FIG. 9 , first to sixth points Pa, Px, Pb generated from the first feature line L1 and the first feature point P1 and the second feature point P2 constituting the first feature line , Pc, Py, and Pz) may form a boundary of the second characteristic region C2. For example, the first horizontal line Lby may be formed to pass through the third point Pb and the fifth point Py, and the second line Lby may be formed to pass through the fourth point Pc and the sixth point Pz. A horizontal line Lcz may be formed. In addition, a first vertical line La extending in a direction perpendicular to the longitudinal direction of the first feature line L1 may be formed by the first point Pa, and the first vertical line La is a first horizontal line ( Lby) and the second horizontal line Lcz, respectively. In addition, a second vertical line Lx extending in a direction perpendicular to the longitudinal direction of the first feature line L1 may be formed by the second point Px, and the second vertical line Lx is a first horizontal line ( Lby) and the second horizontal line Lcz, respectively. Accordingly, the first horizontal line Lby, the second horizontal line Lcz, the first vertical line La, and the second vertical line Lx may be connected to form a second characteristic region C2 having a rectangular shape, 2 Enables predetermined processing inside the feature area.

Meanwhile, the first to sixth distances (da, dx, db, dc, dy, dz) may all be the same, and no matter what face data is used, the eyes may be included in the second feature region C2. .

Referring to FIG. 10 , the second characteristic region C2 may have a bar shape in which both sides are rounded. Exemplarily, a first point Pa, a third point Pb, and a fourth point Pc may be formed to be spaced apart from the first feature point P1 by a first distance d1, and the second feature point A second point Px, a fifth point Py, and a sixth point Pz may be formed to be spaced apart from (P2) by a second distance d2. In this case, the first point Pa, the third point Pb, and the fourth point Pc may form a first arc lbac, and the second point Px and the fifth point Py. , and the sixth point Pz may form a second arc lyxz. Accordingly, the first arc lbac, the second arc lyxz, the first horizontal line Lby, and the second horizontal line Lcz may be connected to each other to form the second characteristic region C2 . In this way, when the second feature region C2 is formed using the first arc lbac and the second arc lyxz, the user minimizes the processed second feature region C2 portion while exposing the patient’s face. It can be effectively prevented and the anonymity of the patient can be guaranteed.

Hereinafter, a process in which the second characteristic region C2 is changed will be described.

Referring to FIG. 11 , in the changing ( S140 ), at least one of a predetermined color and a pattern may be applied to the second characteristic area C2 . For example, the pixels in the second characteristic region C2 may have image data information, and the information may include a color, a curvature, and the like. In this case, a specific color may be assigned to pixels inside the second characteristic area C2 . For example, black may be assigned to pixels in the second characteristic region C2 . Accordingly, the pixels inside the second feature region C2 display the assigned black color when the image data is displayed, and the second feature region C2 including the eyes among the face data of the image data is processed into a silhouette. can be As described above, through the silhouette processing of the second characteristic region C2, the user can provide an optimal correction plan while minimizing exposure of the patient's personal information.

In addition, it is also possible to change the second characteristic area C2 so that a predetermined pattern is overlaid on the inside of the second characteristic area C2, and any processing for minimizing exposure of the patient's personal information is possible.

Referring to FIG. 12 , the inner portion of the second characteristic region C2 may be changed to be dimly processed. This processing is called blurring, and the blurring may be an average of pixel values obtained by averaging colors of pixels constituting the second feature region C2. Referring to FIG. 13 , for example, a 3×3 averaging filter kernel for a part of the second feature region C2 is illustrated. When one pixel constituting the second characteristic region C2 is a central pixel, pixel values of nine adjacent pixels including the central pixel are added to the central pixel. After that, the added value can be divided by 9 and the corresponding value can be assigned as the central pixel value. This process may be applied to all pixels constituting the second characteristic region C2, and accordingly, the pixels constituting the second characteristic region C2 may be blurred. In addition to the above-described averaging method to blur the second feature region C2 in the image data, as shown in FIG. 14 , Gaussian filtering in which differentially changed pixel values are assigned to each pixel, the center Various schemes may be used, including median filtering, and bilateral filtering. By changing the second feature area using the above blurring, the user can provide an optimal correction plan while minimizing exposure of the patient's personal information.

Meanwhile, image data to which the change of the second characteristic region C2 is applied may be extracted as the anonymized image data 500 . In one embodiment, the anonymized image data 500 is facial data in which a part of the image data is changed by performing predetermined processing on the second feature region C2 , and the patient's face is exposed by sharing the anonymized image data 500 . This can be minimized and an optimal correction plan can be provided to the patient through discussion among therapists.

At least a part of the above-described process may be displayed through an output device such as a display device, and a user and a patient can easily check a process in which the image processing method according to the present invention is performed. The user can customize the feature points, feature lines, feature regions, and templates on the user interface through the input device, and there is an advantage in that the user can provide a correction product desired by the patient.

In addition, changes in the second feature area C2 that are changed and applied to the image data may be applied or released. By applying or disabling changes in the second feature area C2, when the user and the patient discuss the correction plan, the patient can easily see how the template will be applied to the entire face, and the image data is shared with other therapists, etc. This has the advantage of minimizing exposure of the patient's face and guaranteeing anonymity.

Hereinafter, an image processing apparatus according to the present invention will be described in detail. In describing the image processing apparatus, content overlapping with the image processing method will be briefly mentioned or omitted.

15 is a block diagram of an image processing apparatus according to the present invention.

Referring to FIG. 15 , the image processing apparatus 10 according to the present invention receives at least one image data from the database unit 100 and the database unit 100 in which image data is stored, and applies a plurality of feature points to the image data. and a control unit 200 that changes a part of image data based on at least one characteristic line or a characteristic region generated by connecting at least two characteristic points among the characteristic points, and a display that visually displays the result of the operation of the control unit 200 It may include a part 300 .

Hereinafter, the configuration of each part will be described.

The database unit 100 may store image data. In this case, the image data may be face data in which the oral cavity is opened and teeth are exposed. When the oral cavity is opened, it may mean that the lips are opened and a part of the teeth or gingiva is exposed, and the face data may be general face data having a natural smile line or face data with a forced open mouth. That is, any type of face data can be used as long as the teeth are exposed and the template can be applied. The database unit 100 may include not only image data but also an algorithm for generating feature points, and at least one tooth template data to be applied to the image data.

As described above in the image processing method according to the present invention, the database unit 100 may be a physical storage device or a cloud.

Meanwhile, the controller 200 may receive at least one image data from the database unit 100 , assign feature points to the image data, fit the image to meet a predetermined standard, and design a smile line. Also, the controller 200 may change a part of the image data. Hereinafter, a detailed configuration of the control unit 200 will be described.

The controller 200 may include a feature point generating unit 210 that assigns a plurality of feature points to the image data according to a predetermined criterion.

The feature point generator 210 assigns feature points to a characteristic part of the input image data. In this case, the characteristic part may include the eyebrows, the lower eyelid, the upper eyelid, the nose, the outer side of the lips, the inner side of the lips, the contour of the face, and the like, as described above. In order to represent the characteristic parts, characteristic points may be given to the characteristic parts, and a plurality of the characteristic points may be provided to the image data. The feature point may be used as a reference point for generating a feature line and a feature region to be described later. In addition, the feature point generator 210 may assign feature points to the image data at an optimal angle while rotating the image data at predetermined angular intervals in order to provide accurate feature points to the image data. Meanwhile, the feature point generating unit 210 may use a predetermined criterion based on the face data to assign the feature points to the image data, and the criterion is the same as described above.

Also, the control unit 200 may further include an image data fitting unit 220 . The image data fitting unit 220 may fit the image data to have at least one of a predetermined size and a predetermined angle based on at least one characteristic line generated by connecting the characteristic points provided by the characteristic point generator 210 . For example, the image data fitting unit 220 may enlarge or reduce the size of the image data so that the length of the first feature line generated by connecting the feature points generated on the lateral canthus of both eyes corresponds to a predetermined length. Also, the image data fitting unit 220 may rotate the image data so that a second feature line generated by connecting a point of the first feature line and a feature point spaced apart from the first feature line has a predetermined angle. In this case, the image data fitting unit 220 may rotate the image data so that the angle of the second feature line is perpendicular to the horizontal direction of the user interface plane.

Hereinafter, a method of fitting the image data received by the image data fitting unit 220 has been described above in the image processing method.

When fitting is completed by the image data fitting unit 220 , the smile line design unit 250 may design a smile line of the fitted image data. In this case, designing the smile line may mean applying at least one template to a feature area generated by the feature points among image data generated based on the feature point. The feature region of the image data may be generated by the region generating unit 230 of the controller. The feature region generated by the region generator 230 may be a first feature region generated by connecting at least three feature points among the plurality of feature points provided by the feature point generator 210 . For example, the first feature region may be an internal region of a lip line generated by connecting feature points on the inner side of the lips among face data. The first characteristic area includes data representing teeth, and the smile line design unit 260 may indicate the state of the patient's teeth after orthodontic treatment by applying a template related to orthodontic teeth to the first characteristic area.

Meanwhile, the feature region generated by the region generator 230 may further include a second feature region generated based on at least two feature points in the face data. Exemplarily, the second feature region may include eyes in the face data, and the second feature region may include first feature points and second feature points forming an outline of the eyes in the face data. Also, the second characteristic region may be formed to be spaced apart from the first characteristic region. Since the first feature region and the second feature region are formed to be spaced apart, it is possible to prevent the template applied to the first feature region from being damaged when a change to the second feature region is performed.

In addition, the second feature region may include a first feature line generated by connecting the first feature point and the second feature point, and the second feature region is a first distance from the first feature point in a longitudinal direction of the first feature line. It may include a first point spaced apart by a distance and a second point spaced apart by a second distance from the second feature point in the longitudinal direction of the first feature line. In addition, the second feature region may include a third point spaced apart from the first feature point by a third distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fourth point from the first feature point in a direction opposite to the direction in which the third point is formed. A fourth point spaced apart by a distance, a fifth point spaced apart by a fifth distance from the second feature point in a direction perpendicular to the longitudinal direction of the first feature line, and a second feature point in a direction opposite to the direction in which the fifth point is formed It may include a sixth point spaced apart by 6 distances. In this case, the first to sixth distances may have the same length.

The first to sixth points are formed from the first and second feature points, and the process of forming the second feature region from the first to sixth points is the same as described above in the image processing method according to the present invention. .

The control unit 200 may further include a region change unit 240 for changing the second characteristic region. The region changer 240 may apply at least one of a predetermined color and a predetermined pattern to the second characteristic region generated by the region generator 230 . In this case, the predetermined color may be a single color, or in some pixels constituting the second characteristic region, an average of pixel values of the corresponding pixels. The average of the pixel values may be allocated to a central pixel among the partial pixels, and accordingly, a blurring process of the second feature region may be performed. In order to perform the blurring process, various filtering methods as described above may be used.

The control unit 200 may further include an image data extraction unit 260 . The image data extraction unit 260 may extract image data to which the change of the second characteristic region is applied as anonymized image data. The extracted anonymized image data may be stored in the database unit 100, and through the anonymized image data, the user can discuss a correction plan with other therapists, and in this process, exposure of the patient's personal information can be minimized. and, thereby, has the advantage of guaranteeing anonymity of the patient.

On the other hand, at least a part of the operation performing process of the control unit 200 as described above may be visually displayed through the display unit 300 , and the user and the patient can select an optimal correction plan for the patient through the display unit 300 . can be established In addition, predetermined processing is performed so that the second feature region is changed, thereby minimizing exposure of the patient's face and guaranteeing anonymity.

The above description is merely illustrative of the technical spirit of the present invention, and various modifications and variations will be possible without departing from the essential characteristics of the present invention by those skilled in the art to which the present invention pertains.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

S110: step of receiving image data
S120: Step of assigning feature points
S130: Fitting step
S140: step to change
400: face data
500: Anonymized image data
10: image processing unit
100: database unit
200: control unit
300: display unit

Claims (36)

receiving at least one or more image data;
assigning a plurality of feature points to the image data; and
changing a portion of the image data based on a feature line generated by connecting at least two of the feature points or a feature region generated by connecting at least three of the feature points Image processing method. The method according to claim 1,
The image data is an image processing method, characterized in that the mouth is opened and teeth are exposed face data. The method according to claim 1,
The step of giving the characteristic points is,
An image processing method characterized in that a plurality of feature points are assigned to the image data according to a predetermined criterion. The method according to claim 1,
The step of giving the characteristic points is,
An image processing method, characterized in that while rotating the image data at a predetermined angular interval, a plurality of feature points are assigned according to a predetermined criterion based on the image data. The method according to claim 1,
The method further comprising: fitting the image data to have at least one of a predetermined size and a predetermined angle based on at least one characteristic line generated by connecting at least two of the characteristic points; Image processing method. 6. The method of claim 5,
The fitting step is
An image processing method, characterized in that the size of the image data is enlarged or reduced so that a length of a first feature line generated by connecting at least two feature points corresponds to a predetermined length. 7. The method of claim 6,
The fitting step is
and rotating the image data so that an angle of a second characteristic line generated by connecting a point of the first characteristic line and a characteristic point spaced apart from the first characteristic line corresponds to a predetermined angle. 8. The method of claim 7,
The angle of the second characteristic line is an image processing method, characterized in that perpendicular to the horizontal direction of the user interface (UI) plane. 3. The method according to claim 2,
The feature area is
and a first feature region generated by feature points forming the inner side of the lips among the face data. 10. The method of claim 9,
The feature area is
It further comprises a second feature region generated based on at least two feature points in the face data,
The image processing method, characterized in that the first characteristic region and the second characteristic region are formed to be spaced apart from each other. 11. The method of claim 10,
The image processing method according to claim 1, wherein the second feature area includes a first feature point and a second feature point forming an eye outline among the face data. 12. The method of claim 11,
The second feature region includes a first feature line generated by connecting the first feature point and the second feature point,
The second feature region may include a first point spaced apart from the first feature point by a first distance in the longitudinal direction of the first feature line, and a second feature spaced apart from the second feature point by a second distance in the longitudinal direction of the first feature line. An image processing method comprising two points. 13. The method of claim 12,
The second characteristic region is
A third point spaced apart by a third distance from the first feature point in a direction perpendicular to the longitudinal direction of the first feature line, and a fourth distance apart from the first feature point in a direction opposite to the direction in which the third point is formed A fourth point, a fifth point spaced apart from the second feature point by a fifth distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fifth point from the second feature point in a direction opposite to the direction in which the fifth point is formed An image processing method comprising a sixth point spaced apart by 6 distances. 14. The method of claim 13,
The image processing method, characterized in that the first distance to the sixth distance are the same. 11. The method of claim 10,
The changing of the part includes applying at least one of a predetermined color and a predetermined pattern to the second feature area. 16. The method of claim 15,
The color is an image processing method, characterized in that the average of the pixel values of the pixels constituting the second characteristic area. The method according to claim 1,
The image data is an image processing method, characterized in that extracted as anonymized image data to which the change is applied. 10. The method of claim 9,
and designing a smile line by applying at least one template to the first feature area. a database unit storing at least one or more image data;
control unit; and
a display unit for visually displaying the result of the control unit; including,
The control unit is
a feature point generating unit that receives the image data from the database unit and assigns a plurality of feature points to the image data; and
a region changing unit for changing a part of the image data based on at least one feature line or feature region generated by connecting at least two of the feature points; Image processing apparatus comprising a. 20. The method of claim 19,
The image data is an image processing apparatus, characterized in that the mouth is opened and teeth are exposed face data. 20. The method of claim 19,
The image processing apparatus according to claim 1, wherein the feature point generator assigns a plurality of feature points according to a predetermined criterion based on the image data. 22. The method of claim 21,
The feature point generating unit,
The image processing apparatus according to claim 1, wherein the plurality of feature points are provided according to the predetermined criterion while rotating the image data at predetermined angular intervals. 20. The method of claim 19,
The control unit is
An image data fitting unit for fitting the image data to have at least one of a predetermined size and a predetermined angle based on at least one characteristic line generated by connecting at least two of the characteristic points; image processing unit. 24. The method of claim 23,
The image data fitting unit,
The image processing apparatus according to claim 1, wherein the size of the image data is enlarged or reduced so that a length of a first feature line generated by connecting at least two feature points corresponds to a predetermined length. 25. The method of claim 24,
The image data fitting unit,
and rotating the image data so that an angle of a second characteristic line generated by connecting a point of the first characteristic line and a characteristic point spaced apart from the first characteristic line corresponds to a predetermined angle. 26. The method of claim 25,
The angle of the second characteristic line is an image processing apparatus, characterized in that perpendicular to the horizontal direction of the user interface (UI) plane. 20. The method of claim 19,
The control unit further includes a region generator for generating the feature region based on the feature points provided to the image data from the feature point generator;
The feature area is
and a first feature region generated by feature points forming the inner side of the lips among the image data. 28. The method of claim 27,
The feature area is
Further comprising a second feature region generated based on at least two feature points in the image data,
The image processing apparatus according to claim 1, wherein the first characteristic area and the second characteristic area are spaced apart from each other. 29. The method of claim 28,
The image processing apparatus according to claim 1, wherein the second characteristic area includes first and second characteristic points forming an outline of an eye among the image data. 30. The method of claim 29,
The second feature region includes a first feature line generated by connecting the first feature point and the second feature point,
The second feature region may include a first point spaced apart from the first feature point by a first distance in the longitudinal direction of the first feature line, and a second feature spaced apart from the second feature point by a second distance in the longitudinal direction of the first feature line. An image processing apparatus comprising two points. 31. The method of claim 30,
The second characteristic region is
A third point spaced apart by a third distance from the first feature point in a direction perpendicular to the longitudinal direction of the first feature line, and a fourth distance apart from the first feature point in a direction opposite to the direction in which the third point is formed A fourth point, a fifth point spaced apart from the second feature point by a fifth distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fifth point from the second feature point in a direction opposite to the direction in which the fifth point is formed An image processing apparatus comprising a sixth point spaced apart by 6 distances. 32. The method of claim 31,
The image processing apparatus, characterized in that the first distance to the sixth distance are the same. 29. The method of claim 28,
and the region change unit applies at least one of a predetermined color and a predetermined pattern to the second characteristic region. 34. The method of claim 33,
and the color is an average of pixel values of pixels constituting the second feature area. 26. The method of claim 25,
The control unit is
a smile line design unit for designing a smile line by applying at least one template to the feature area among the fitted image data; Image processing apparatus further comprising a. 20. The method of claim 19,
The control unit is
an image extraction unit for extracting the image data to which the change is applied as anonymized image data; Image processing apparatus further comprising a.

Images