KR102536646B1 - Image Processing Method and Apparatus Using Thereof - Google Patents

Abstract

An image processing method according to the present invention includes the steps of receiving at least one image data, assigning a plurality of feature points to the image data according to a predetermined criterion, and a feature line generated by connecting at least two of the feature points. , or changing a part of the image data based on a feature region generated by connecting at least three of the feature points.

Description

Image processing method and apparatus using the same {Image Processing Method and Apparatus Using Thereof}

The present invention relates to an image processing method and an apparatus using the same.

Smile line refers to the tooth line, tooth shape, degree of gingiva exposed when smiling, gingival line, length and thickness of the lips, length of the philtrum, and the degree and direction of activity of the muscles around the lips and related muscles. is made Through corrective treatment, patients can have desired smile lines. In particular, patients who are concerned about treatment with veneers, laminates, etc., wonder how their face (more specifically, a smile line) changes after treatment. A person's impression is changed according to the smile line, and making such a smile line through corrective treatment or surgery is also called a smile design.

Meanwhile, in the smile design, an optimal correction template for the patient is selected while applying templates having information about corrected teeth on image data in which the face of the patient is displayed. In this process, there is a possibility of sharing image data to which a template is applied between therapists or patients, and sharing of image data exposes the patient's face and may infringe personal privacy.

Therefore, there is a need for a method capable of guaranteeing anonymity of a patient even by using template-applied image data.

Republic of Korea Patent Publication KR10-2007-0093841 A (2007.09.14. Publication)

In order to solve the above problem, the image processing method according to the present invention receives image data having human face data, assigns a plurality of feature points to the image data, and based on a feature line or feature area generated from the feature points, An image processing method for changing a part of image data is provided.

In addition, an image processing device that changes part of image data by performing the above image processing method is provided.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

An image processing method according to the present invention includes the steps of receiving image data, assigning a plurality of feature points to the image data, and a feature line generated by connecting at least two of the feature points or at least three of the feature points. and changing a part of the image data based on the feature region created by the concatenation.

In addition, the image processing apparatus according to the present invention, a database unit for storing input image data, assigns a plurality of feature points to the input image data, and displays a portion of the image data based on feature lines or feature regions generated from the feature points. It includes a control unit that changes.

According to the present invention, when the user or patient shares the patient's image data with others, the patient's anonymity can be guaranteed by changing a part of the image data, and the user or patient can discuss the template applied to the image data. It has the advantage of being able to establish an optimal calibration plan.

1 and 2 are flow charts of an image processing method according to the present invention.
3 is a diagram for explaining image data.
4 is a diagram for explaining that feature points are assigned.
5 is a diagram for explaining that feature lines and feature regions are generated based on some of feature points assigned to image data.
6 and 7 are diagrams for explaining a process of fitting image data.
8 is a diagram for explaining application of a template to image data.
9 is a diagram for explaining a second feature region in the image processing method according to the present invention.
10 is another embodiment of FIG. 9 .
11 and 12 are diagrams for explaining that a second feature region of image data is changed.
13 is a diagram for explaining a blur processing process.
14 is a diagram for explaining a Gaussian blur processing process.
15 is a configuration diagram of an image processing device according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through exemplary drawings. In adding reference numerals to components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing an embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function hinders understanding of the embodiment of the present invention, the detailed description will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the corresponding component is not limited by the term. In addition, unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

1 and 2 are flowcharts of an image processing method according to the present invention, and FIG. 3 is a diagram for explaining image data.

1 to 3, the image processing method according to the present invention includes receiving image data (S110), assigning a plurality of feature points to the image data according to a predetermined criterion (S120), and among the feature points A step of changing part of the image data based on a feature line created by connecting at least two features or a feature region created by connecting at least three feature points ( S140 ) may be included.

Hereinafter, each step of the image processing method according to the present invention will be described in detail.

In the step of receiving image data ( S110 ), at least one image data of a previously photographed patient may be input. Here, the image data is stored in a data storage element called a database unit (DB unit) and can be input by a user's manipulation. The database unit (DB unit) in which image data is stored is a physical disk such as a hard disk drive, a floppy disk drive, a solid state drive, or a flash memory drive. It may be a storage device or a virtual database in the form of a cloud.

Preferably, the image data may be human face data. More specifically, the image data is face data 400 in which a person's eyes, nose, mouth, etc. are visually displayed, and in particular, it may be face data in which a person's oral cavity is opened and teeth are exposed. The face data 400 may be general image data having a natural smile, and portions of teeth and gingiva may be exposed as both mouth corners rise upward. A part of the exposed tooth and gingiva is included in a feature region to be described later, and a template for orthodontic treatment may be virtually applied to the corresponding region. In this case, a template refers to a sample model of orthodontic teeth, and the template may have standard information about the position, size, and shape of at least one tooth. The user and the patient can predict the shape of the tooth after correction in advance by overlaying the template on the patient's tooth among the image data. In addition, a plurality of templates may be provided to provide the patient with an optimal orthodontic appliance, and the user and the patient may determine a template that best matches the patient's teeth by selectively applying the plurality of templates to the patient's teeth among image data. can

Meanwhile, the face data may be general face data having a smile line, or may be open face data having a greater degree of opening than normal face data by forcibly opening the mouth with an opening device or the like. Normal face data is used by users and patients to apply templates to establish correction plans, and aperture face data can be used by laboratories to create corrections. In this case, the corrector may refer to an object worn by the patient. Illustratively, the orthodontic product may refer to a veneer, a prosthesis including a laminate, and an orthodontic device.

The input image data may be displayed on a user interface (UI), and the image data may be displayed to a user and a patient through an output device. Various types of devices may be used as an output device, and among them, a display device capable of visually displaying image data may be used.

Hereinafter, a process of assigning feature points to image data will be described.

4 is a diagram for explaining that feature points are assigned.

Referring to FIG. 4 , in the step of assigning feature points ( S120 ), a plurality of feature points P may be assigned to characteristic parts of image data according to a predetermined criterion based on face data. More specifically, the characteristic parts may be eyebrows, eye contours between the upper and lower eyelids, nose, outer and inner lips, and face contours. The number of feature points P assigned in the step of assigning feature points ( S120 ) may be a number sufficient to effectively represent the aforementioned characteristic parts of the image data.

Illustratively, in the step of assigning feature points (S120), an AI Facial Landmarks Detection method may be used to express characteristic parts of image data. Hereinafter, the Face Landmark 68 algorithm, which assigns 68 feature points (P) to the image data among the AI Facial Landmarks Detection methods in the present invention, has been described as a standard, but is not limited thereto, and can easily represent the characteristics of image data Any algorithm may be used.

Meanwhile, when image data is input, if the input image data is excessively rotated in one direction, feature points P may not be smoothly assigned to the image data. Accordingly, in the step of assigning feature points ( S120 ), a plurality of feature points P may be assigned according to a predetermined criterion based on the face data 400 while rotating the image data at predetermined angular intervals.

Illustratively, in the step of assigning feature points ( S120 ), feature points P may be assigned while rotating the image data clockwise or counterclockwise at 90° intervals. According to another example, in the step of assigning feature points ( S120 ), feature points P may be assigned while rotating the image data clockwise or counterclockwise at 15° intervals. In this way, by rotating image data and assigning feature points P, it is possible to accurately recognize image data and assign feature points P, and there are advantages in that a part of image data can be changed at a normal position and a template can be applied.

According to another embodiment, the step of assigning feature points (S120) selects image data that most closely matches the face data 400 from among a plurality of two-dimensional image data and continuous video data using artificial intelligence, and selects the selected image. Feature points (P) may be assigned to data. When the step of assigning feature points using artificial intelligence (S120) is performed, even if the user does not directly input image data that most closely matches the face data 400, appropriate image data is automatically selected and feature points P are assigned. There is an advantage.

Hereinafter, as an additional step of the image processing method according to the present invention, fitting image data based on feature points assigned to the image data (S130) will be described.

5 is a diagram for explaining that feature lines and feature regions are generated based on some of feature points assigned to image data, and FIGS. 6 and 7 are diagrams for explaining a process of fitting image data.

5 to 7, when feature points are assigned to image data, a step of fitting the image data to have at least one of a predetermined size or a predetermined angle based on the assigned feature points (S130) is additionally performed. It can be. That is, the location of the feature points P, the distance between the feature points P, and the angle formed by the feature points P can be obtained, and these information can be used as elements for performing the image processing method according to the present invention.

In the fitting step (S130), predetermined feature lines may be generated from the feature points P assigned in the step of assigning the feature points described above (S120). Exemplarily, the feature line may be generated by connecting at least two feature points among the assigned feature points, and the generated feature line is the feature points P1' and P2' corresponding to the lateral canthus of each eye in the face data. may be a line connecting This feature line is called a first feature line L1' or an eye line. Meanwhile, the feature points P1' and P2' may be the 37th feature point and the 46th feature point, respectively, but this is merely an example and is not limited thereto.

In the fitting step ( S130 ), the size of image data may be enlarged or reduced based on the generated feature line. More specifically, in the fitting step ( S130 ), the size of the image data may be enlarged or reduced so that the length of the first feature line L1' matches a predetermined length. Exemplarily, when the length of the first feature line L1' is 300 pixels and the preset target length is 400 pixels, the length of the fitted first feature line L1 is multiplied by a constant value so that the length of the first feature line L1 is 400 pixels. size can be enlarged. Meanwhile, when the length of the first feature line L1' is greater than the predetermined target length, the size of the image data may be reduced by multiplying the length of the fitted first feature line L1 by a predetermined value so that the length of the first feature line L1 reaches the predetermined target length. can

Also, in the fitting step ( S130 ), image data may be rotated based on the generated feature line. To rotate the image data, in the fitting step S130, the image data may be rotated based on a second feature line L2' different from the first feature line L1'. More specifically, the fitting step (S130) may include a second feature line (or mid) generated by connecting a point of the first feature line L1' and a feature point spaced apart from the first feature line L1'. Image data can be rotated using the line, L2'). The second feature line L2' may be generated by connecting the central point P3' of the first feature line L1' to the feature point P4'. Illustratively, the feature point P4' may correspond to the 34th feature point.

In the fitting step (S130) based on the generated second feature line L2', image data is fitted so that a smile line can be designed at a normal angle. Illustratively, the angle of the fitted second feature line L2 may be perpendicular to the horizontal direction among the planes of the user interface (UI) on which image data is input and displayed.

As described above, by fitting the face data 400' so that the image data conforms to at least one of a predetermined size and a predetermined angle, the user can accurately apply the template to the fitted face data 400 and establish a highly reliable correction plan. can do.

Optionally, when the angle of the image data is fitted differently from the user's intention, the user may manually rotate the image data by a predetermined angle. Exemplarily, the image data may be rotated clockwise or counterclockwise around the central point P3 of the fitted first feature line L1, and the predetermined angle at which the image data can be rotated is clockwise. Alternatively, it may be within about 5° counterclockwise. In this way, by manually finely rotating the image data at a predetermined angle according to the user's judgment, the user can acquire image data that is more suitable for image data analysis and smile line design.

Hereinafter, a process of applying a template to image data will be described.

8 is a diagram for explaining application of a template to image data.

Referring to FIGS. 5 and 8 , after fitting of image data is completed, a step (not shown) of designing a smile line by applying at least one template to a predetermined region of the image data based on feature points may be performed. The template may be pre-stored orthodontic tooth data in a database, and a plurality of templates may be provided so that an optimal template may be applied to the patient's image data. The template can be customized by the user according to the patient's oral condition. For example, the user can change the shape of the teeth (Ta, Tb, Tc, Td, Te and Tf) of the template. At this time, the shape of the tooth may be changed by adjusting the tooth contour constituting the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template. In addition, the user may change the position, size, color, or shade of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template. By changing the positions or sizes of the teeth (Ta, Tb, Tc, Td, Te and Tf) of the template, the positions or sizes of adjacent teeth can be linked together and changed. In addition, by changing the color or shade of the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template, a comprehensive tooth treatment plan considering whitening procedures other than prosthesis can be considered. By customizing the teeth (Ta, Tb, Tc, Td, Te, and Tf) of the template in this way, the user can provide an optimal treatment plan to the patient.

Meanwhile, in the designing step, a template may be applied to a feature region created by connecting at least three feature points among the plurality of feature points assigned in the step of assigning feature points ( S120 ). More specifically, a lipline generated by connecting feature points constituting the inside of the lips among feature points assigned to image data may be formed as a first feature region C1 among feature regions, and the first feature region ( C1) A template may be applied to the tooth images Ta', Tb', Tc', Td', Te', and Tf' existing inside. More specifically, in the designing step (S160), the feature region C1 generated by connecting at least three feature points is recognized as a lip line (lip line recognition step), and the tooth image appears inside the feature region C1. A region is recognized (tooth region recognizing step), and a tooth image inside the feature region C1 constituting the lip line may be matched with a template using a template matching algorithm for the tooth region (template matching step). At this time, the template matching algorithm is a method of matching the template to the position of the tooth image expressed inside the feature region (C1), and considers factors such as shape, size, or color to automatically match the tooth region having elements similar to the template. You can match the template with . In this way, by applying the template to the feature region, it is possible to prevent misapplication of the template to regions other than the first feature region C1 among image data.

Meanwhile, when the feature region C1 is created, positions of the feature points P are unclear, so that an incorrect feature region may be created. In this case, the feature points P and the outline of the first feature region C1 may be modified using the feature point correction function formed on the user interface, and a more accurate feature region acquisition and template application process may be performed.

Hereinafter, the step of changing a part of image data based on the feature line or feature region (S140) will be described in detail.

9 is a diagram for explaining a second feature region in the image processing method according to the present invention, FIG. 10 is another embodiment of FIG. 9, and FIGS. is for 13 is a diagram for explaining a blur processing process, and FIG. 14 is a diagram for explaining a Gaussian blur processing process.

5 and 9 , in the step of changing (S140), when a plurality of feature points P are assigned to image data, a feature line or feature area generated by connecting at least two of the feature points P is changed. Based on this, a part of the image data can be changed. At this time, 'changing a part of the image data' may mean that it is difficult to identify whose face data the image data has by performing a predetermined process on the part.

In the following description, a predetermined process is described as an example of changing a part of image data using a predetermined color or a predetermined pattern. However, the predetermined process is not limited thereto, and may be interpreted as meaning various processes such as cutting a part of the image data or inserting a figure into a part of the image data.

As described above, the feature region may include the first feature region C1 to which the template is applied. Also, the feature region may further include a second feature region C2 generated based on at least two feature points of the face data 400 . In this case, the second feature region C2 may be formed to be spaced apart from the first feature region C1 to which the template is applied. Therefore, even if the inside of the second feature region C2, which will be described later, is changed, the first feature region C1 may not be affected, and the template applied to the first feature region C1 may not be damaged. A configuration in which the inside of the second feature region C2 is changed will be described later.

Meanwhile, the second feature region C2 may include a first feature point P1 and a second feature point P2 that form an eye contour among the face data 400 . More specifically, the first feature point P1 and the second feature point P2 may be feature points corresponding to the lateral canthus of each eye among face data in the above-described fitting step ( S130 ). For example, the feature points P1 and P2 may be the 37th feature point and the 46th feature point, respectively.

By including the first feature point P1 and the second feature point P2 in the second feature region C2, the second feature region C2 may be changed including the patient's eyes in the face data 400. .

The range covered by the second feature region C2 will be described in more detail. A first feature line L1 generated by connecting the first feature point P1 and the second feature point P2 may be included. At this time, the second feature region C2 includes a first point Pa spaced apart from the first feature point P1 by a first distance da in the longitudinal direction of the first feature line L1, and the first feature line A second point Px spaced apart from the second feature point P2 by a second distance dx in the longitudinal direction of (L1) may be included. The first point Pa and the second point Px may be formed side by side in the longitudinal direction of the first feature line L1, but may be formed in opposite directions from the first feature line L1. Also, the first point Pa and the second point Px may form the horizontal length of the second feature region C2.

In addition, the second feature region C2 includes a third point Pb spaced apart from the first feature point P1 by a third distance db in a direction perpendicular to the longitudinal direction of the first feature line L1, and a third point Pb. A fourth point Pc spaced apart from the first feature point P1 by a fourth distance dc in a direction opposite to the direction in which the point Pb is formed may be included. In this case, the third point Pb and the fourth point Pc may constitute horizontal lines Lby and Lcz of the second feature region C2, respectively.

The second feature region C2 includes a fifth point Py spaced apart from the second feature point P2 by a fifth distance dy in a direction perpendicular to the longitudinal direction of the first feature line L1, and a fifth point Py. A sixth point Pz spaced apart from the second feature point P2 by a sixth distance dz may be included in a direction opposite to the direction in which Py is formed. In this case, the fifth point Py and the sixth point Pz may constitute horizontal lines Lby and Lcz of the second feature region C2, respectively.

Referring to FIG. 9 , first to sixth points (Pa, Px, Pb) generated from the first feature line L1 and the first feature points P1 and second feature points P2 constituting the first feature line , Pc, Py, Pz) may form a boundary of the second feature region C2. Illustratively, the first horizontal line Lby may be formed to pass through the third point Pb and the fifth point Py, and the second horizontal line Lby may pass through the fourth point Pc and the sixth point Pz. A horizontal line Lcz may be formed. In addition, a first vertical line La extending in a direction perpendicular to the longitudinal direction of the first feature line L1 may be formed by the first point Pa, and the first vertical line La may include a first horizontal line ( Lby) and the second horizontal line Lcz. In addition, a second vertical line Lx extending in a direction perpendicular to the longitudinal direction of the first feature line L1 may be formed by the second point Px, and the second vertical line Lx may be a first horizontal line ( Lby) and the second horizontal line Lcz. Accordingly, the first horizontal line Lby, the second horizontal line Lcz, the first vertical line La, and the second vertical line Lx are connected to form a rectangular second feature region C2. 2 Enables predetermined processing inside the feature area.

Meanwhile, the first to sixth distances da, dx, db, dc, dy, and dz may all be the same, and any face data may be used so that the eyes are included in the second feature region C2. .

Referring to FIG. 10 , the second feature region C2 may have a rod shape with both sides round. Illustratively, the first point Pa, the third point Pb, and the fourth point Pc may be formed to be spaced apart from the first feature point P1 by the first distance d1, and the second feature point A second point Px, a fifth point Py, and a sixth point Pz may be formed to be spaced apart from (P2) by the second distance d2. In this case, the first point Pa, the third point Pb, and the fourth point Pc may form a first arc lbac, and the second point Px and the fifth point Py , and the sixth point Pz may form a second arc lyxz. Accordingly, the first arc lbac, the second arc lyxz, the first horizontal line Lby, and the second horizontal line Lcz may be connected to each other to form the second feature region C2. In this way, when the second feature region C2 is formed using the first arc (lbac) and the second arc (lyxz), the user can minimize exposure of the patient's face while minimizing the processed portion of the second feature region (C2). This can be effectively prevented and the patient's anonymity can be guaranteed.

Hereinafter, a process of changing the second feature region C2 will be described.

Referring to FIG. 11 , in the changing operation S140, at least one of a predetermined color and pattern may be applied to the second feature region C2. Illustratively, pixels within the second feature region C2 may have image data information, and the information may include color, curvature, and the like. In this case, a specific color may be assigned to pixels within the second feature region C2. For example, black may be assigned to pixels within the second feature region C2. Accordingly, the pixels inside the second feature region C2 cause the assigned black color to be displayed when the image data is displayed, and the second feature region C2 including eyes among the face data of the image data is processed as a silhouette. It can be. In this way, through the silhouette processing of the second feature region C2, the user can provide an optimal correction plan while minimizing exposure of personal information of the patient.

In addition, it is also possible to change the second characteristic region C2 so that a predetermined pattern is overlaid on the inside of the second characteristic region C2, and any processing to minimize the exposure of personal information of the patient is possible.

Referring to FIG. 12 , the inner portion of the second feature region C2 may be changed to be dimly processed. This processing is called blurring, and the blurring may be an average of pixel values obtained by averaging the colors of the pixels constituting the second feature region C2. Referring to FIG. 13 , a 3×3 averaging filter kernel for a part of the second feature region C2 is illustrated as an example. If one pixel constituting the second feature region C2 is the center pixel, all pixel values of nine adjacent pixels including the center pixel are added based on the center pixel. Then, the added value can be divided by 9 and the value can be assigned as the center pixel value. This process may be applied to all pixels constituting the second feature region C2, and accordingly, the pixels constituting the second feature region C2 may be subjected to a blurring process. In addition to the above-described averaging method for blurring the second feature region C2 of image data, Gaussian filtering in which differentially changed pixel values are assigned to each pixel as shown in FIG. 14, center Various schemes may be used, including median filtering, and bilateral filtering. By changing the second feature region using the blurring as described above, the user can provide an optimal correction plan while minimizing exposure of personal information of the patient.

Meanwhile, image data to which a change of the second feature region C2 is applied may be extracted as anonymized image data 500 . In one embodiment, the anonymized image data 500 is face data in which a portion of the image data is changed by performing a predetermined process on the second feature region C2, and the patient's face is exposed by sharing the anonymized image data 500. This is minimized and the optimal correction plan can be provided to the patient through discussion among therapists.

At least some of the above processes may be displayed through an output device such as a display device, and the user and the patient can easily check the process in which the image processing method according to the present invention is performed. The user can customize feature points, feature lines, feature regions, and templates on the user interface through the input device, and the user has the advantage of being able to provide corrections desired by the patient.

In addition, changes in the second feature region C2 applied to the image data may be applied or canceled. By applying or canceling the changes in the second feature area C2, when the user and the patient discuss the correction plan, the patient can easily check how the template will be applied to the entire face, and image data can be shared with other therapists. It has the advantage of minimizing the exposure of the patient's face and guaranteeing anonymity.

Hereinafter, an image processing apparatus according to the present invention will be described in detail. In describing the image processing apparatus, contents overlapping with the image processing method are briefly mentioned or omitted.

15 is a configuration diagram of an image processing device according to the present invention.

Referring to FIG. 15 , the image processing apparatus 10 according to the present invention receives at least one image data from the database unit 100 in which image data is stored, and assigns a plurality of feature points to the image data. and a controller 200 that changes part of the image data based on at least one feature line or feature area generated by connecting at least two of the feature points, and a display that visually displays the performance result of the controller 200. A portion 300 may be included.

Hereinafter, the configuration of each part will be described.

The database unit 100 may store image data. At this time, the image data may be face data in which the mouth is opened and the teeth are exposed. Opening of the oral cavity may mean that the lips are open and part of the teeth or gingiva is exposed, and the face data may be normal face data having a natural smile line or open face data that is forcibly opened. That is, as long as teeth are exposed and a template can be applied, any type of face data can be used. The database unit 100 may include not only image data, but also algorithms for generating feature points, at least one tooth template data to be applied to image data, and the like.

As described above in the image processing method according to the present invention, the database unit 100 may be a physical storage device or a cloud.

Meanwhile, the control unit 200 may receive at least one image data from the database unit 100, assign feature points to the image data, fit the image to meet a predetermined standard, and design a smile line. Also, the controller 200 may change part of image data. Hereinafter, a detailed configuration of the control unit 200 will be described.

The controller 200 may include a feature point generator 210 that assigns a plurality of feature points to image data according to a predetermined criterion.

The feature point generation unit 210 assigns feature points to characteristic parts of the input image data. In this case, the characteristic parts may include the eyebrows, the lower eyelid, the upper eyelid, the nose, the outside of the lips, the inside of the lips, and the outline of the face, as described above. In order to represent the characteristic parts, feature points are assigned to the characteristic parts, and a plurality of feature points may be assigned to image data. Feature points can be used as reference points for generating feature lines and feature regions, which will be described later. In addition, the feature point generation unit 210 may assign feature points to the image data at an optimal angle while rotating the image data at predetermined angular intervals in order to assign accurate feature points to the image data. Meanwhile, the feature point generation unit 210 may use a predetermined criterion based on face data to assign feature points to the image data, and the criterion is as described above.

Also, the controller 200 may further include an image data fitting unit 220 . The image data fitting unit 220 may fit the image data to have at least one of a predetermined size and a predetermined angle based on at least one feature line generated by connecting the feature points assigned from the feature point generation unit 210 . Illustratively, the image data fitting unit 220 may enlarge or reduce the size of the image data so that a length of a first feature line generated by connecting feature points generated at the lateral canthus of both eyes matches a predetermined length. Also, the image data fitting unit 220 may rotate the image data so that a second feature line generated by connecting a point of the first feature line and a feature point spaced apart from the first feature line has a predetermined angle. At this time, the image data fitting unit 220 may rotate the image data so that the angle of the second feature line is perpendicular to the horizontal direction of the user interface plane.

Hereinafter, the method of fitting the image data input by the image data fitting unit 220 has been described in the above image processing method.

When fitting is completed in the image data fitting unit 220, the smile line design unit 250 may design a smile line of the fitted image data. In this case, designing the smile line may mean applying at least one template to a feature region created by feature points among image data generated based on feature points. The feature region of the image data may be generated by the region generator 230 among the controllers. The feature region generated by the region generator 230 may be a first feature region generated by connecting at least three feature points among a plurality of feature points assigned by the feature point generator 210 . Exemplarily, the first feature region may be an inner region of a lip line generated by connecting feature points of the inside of the lips among facial data. The first feature region includes data representing teeth, and the smile line design unit 260 may indicate a state of the patient's teeth after orthodontics by applying a template related to corrected teeth to the first feature region.

Meanwhile, the feature region generated by the region generator 230 may further include a second feature region generated based on at least two feature points of face data. Exemplarily, the second feature region may include an eye among face data, and the second feature region may include a first feature point and a second feature point forming an eye contour among face data. Also, the second feature region may be formed to be spaced apart from the first feature region. Since the first feature region and the second feature region are spaced apart from each other, a template applied to the first feature region may be prevented from being damaged when a change to the second feature region is performed.

Also, the second feature region may include a first feature line generated by connecting the first feature point and the second feature point, and the second feature region is a first distance from the first feature point in a longitudinal direction of the first feature line. and a second point spaced apart from the second feature point by a second distance in the longitudinal direction of the first feature line. In addition, the second feature region may include a third point spaced apart from the first feature point by a third distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fourth point from the first feature point in a direction opposite to the direction in which the third point is formed. A fourth point spaced apart by a distance, a fifth point spaced apart by a fifth distance from the second feature point in a direction perpendicular to the longitudinal direction of the first feature line, and a fifth point from the second feature point in a direction opposite to the direction in which the fifth point was formed. Sixth points spaced apart by 6 distances may be included. In this case, the first to sixth distances may have the same length.

The process of forming the first to sixth points from the first feature point and the second feature point and forming the second feature region from the formed first to sixth points is the same as described above in the image processing method according to the present invention. .

The controller 200 may further include a region changing unit 240 that changes the second characteristic region. The region changer 240 may apply at least one of a predetermined color and a predetermined pattern to the second feature region generated by the region generator 230 . In this case, the predetermined color may be a single color or may be an average of pixel values of corresponding pixels in some pixels constituting the second characteristic region. An average of the pixel values may be assigned to a center pixel among the partial pixels, and accordingly, a blurring process of the second feature region may be performed. Various filtering schemes as described above may be used to perform the blurring process.

The controller 200 may further include an image data extractor 260 . The image data extractor 260 may extract image data to which the changes in the second feature region are applied as anonymized image data. The extracted anonymized image data can be stored in the database unit 100, and the user can discuss a correction plan with other therapists through the anonymized image data, and exposure of the patient's personal information can be minimized in this process. This has the advantage of ensuring patient anonymity.

Meanwhile, at least a part of the process of performing the operation of the control unit 200 as described above may be visually displayed through the display unit 300, and the user and the patient may find an optimal correction plan for the patient through the display unit 300. can be established In addition, by performing a predetermined process to change the second characteristic region, exposure of the patient's face can be minimized and anonymity can be guaranteed.

The above description is merely an example of the technical idea of the present invention, and various modifications and variations can be made to those skilled in the art without departing from the essential characteristics of the present invention.

Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed according to the claims below, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

S110: step of receiving image data
S120: Step of assigning feature points
S130: Step of fitting
S140: Step to change
400: face data
500: anonymized image data
10: image processing unit
100: database unit
200: control unit
300: display unit

Claims (36)

Receiving at least one image data that is face data in which teeth are exposed when the mouth is opened;
assigning a plurality of feature points to the image data, including feature points corresponding to lateral canthus angles, according to a predetermined criterion;
designing a smile line by applying at least one template to a first feature region among at least two feature regions generated by connecting at least three of the feature points; and
Changing a part of the image data based on a second feature region among at least two feature regions generated by connecting at least three of the feature points;
The first feature region represents a lip line generated by connecting feature points constituting the inside of the lips among the face data and is a region to which at least one template for orthodontic treatment is virtually applied,
The image processing method of claim 1 , wherein the second feature region is spaced apart from the first feature region and is generated based on a first feature line connecting first feature points and second feature points of the face data. delete delete The method of claim 1,
The step of assigning the feature points,
The image processing method characterized in that, while rotating the image data at predetermined angular intervals, a plurality of feature points are assigned according to a predetermined criterion based on the image data. The method of claim 1,
Further comprising the step of fitting the image data to have at least one of a predetermined size and a predetermined angle based on at least one feature line generated by connecting at least two of the feature points. Image processing method. The method of claim 5,
The fitting step is
The image processing method characterized by enlarging or reducing the size of the image data so that a length of a first feature line generated by connecting at least two feature points matches a predetermined length. The method of claim 6,
The fitting step is
and rotating the image data so that an angle of a second feature line generated by connecting a point of the first feature line and a feature point spaced apart from the first feature line matches a predetermined angle. The method of claim 7,
The image processing method, characterized in that the angle of the second feature line is perpendicular to the horizontal direction of the user interface (UI) plane. delete delete delete The method of claim 1,
The second feature region includes a first point spaced apart from the first feature point by a first distance in the longitudinal direction of the first feature line, and a second feature spaced apart by a second distance from the second feature point in the longitudinal direction of the first feature line. An image processing method comprising two points. The method of claim 12,
The second feature region,
A third point spaced apart from the first feature point by a third distance in a direction perpendicular to the longitudinal direction of the first feature line, and spaced apart from the first feature point by a fourth distance in a direction opposite to the direction in which the third point was formed. A fourth point, a fifth point spaced apart from the second feature point by a fifth distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fifth point from the second feature point in a direction opposite to the direction in which the fifth point was formed. An image processing method comprising a sixth point spaced apart by 6 distances. The method of claim 13,
The image processing method, characterized in that the first distance to the sixth distance is the same. The method of claim 1,
In the changing of the portion, at least one of a predetermined color and a predetermined pattern is applied to the second feature region. The method of claim 15
The image processing method according to claim 1 , wherein the color is an average of pixel values of pixels constituting the second feature region. The method of claim 1,
Wherein the image data is extracted as anonymized image data to which the change is applied. delete a database unit storing at least one image data representing facial data in which the mouth is opened and teeth are exposed;
control unit; and
a display unit that visually displays a result of the execution of the control unit; including,
The control unit,
a feature point generation unit that receives the image data from the database unit and assigns a plurality of feature points to the image data, including feature points corresponding to lateral canthus angles according to a predetermined criterion;
a region generator for generating at least two feature regions based on the feature points assigned to the image data from the feature point generator;
a smile line design unit designing a smile line by applying at least one template to a first feature region among the at least two feature regions generated by connecting at least three of the feature points; and
a region changer configured to change part of the image data based on a second feature region among the at least two feature regions generated by connecting at least three of the feature points; including,
The first feature region represents a lip line generated by connecting feature points constituting the inside of the lips among the face data and is a region to which at least one template for orthodontic treatment is virtually applied,
The image processing apparatus according to claim 1 , wherein the second feature region is spaced apart from the first feature region and is generated based on a first feature line connecting the first feature point and the second feature point of the face data. delete delete The method of claim 19
The feature point generation unit,
The image processing apparatus characterized in that the plurality of feature points are assigned according to the predetermined criterion while rotating the image data at predetermined angular intervals. The method of claim 19
The control unit,
and an image data fitting unit for fitting the image data to have at least one of a predetermined size and a predetermined angle based on at least one feature line generated by connecting at least two of the feature points. image processing unit. The method of claim 23
The image data fitting unit,
The image processing device characterized in that the size of the image data is enlarged or reduced so that a length of a first feature line generated by connecting at least two feature points matches a predetermined length. The method of claim 24
The image data fitting unit,
and rotating the image data such that an angle of a second feature line generated by connecting a point of the first feature line and a feature point spaced apart from the first feature line matches a predetermined angle. The method of claim 25
The image processing device, characterized in that the angle of the second feature line is perpendicular to the horizontal direction of the user interface (UI) plane. delete delete delete The method of claim 19
The second feature region includes a first point spaced apart from the first feature point by a first distance in the longitudinal direction of the first feature line, and a second feature spaced apart by a second distance from the second feature point in the longitudinal direction of the first feature line. An image processing device comprising two points. The method of claim 30
The second feature region,
A third point spaced apart from the first feature point by a third distance in a direction perpendicular to the longitudinal direction of the first feature line, and spaced apart from the first feature point by a fourth distance in a direction opposite to the direction in which the third point was formed. A fourth point, a fifth point spaced apart from the second feature point by a fifth distance in a direction perpendicular to the longitudinal direction of the first feature line, and a fifth point from the second feature point in a direction opposite to the direction in which the fifth point was formed. An image processing device comprising a sixth point spaced apart by 6 distances. The method of claim 31
The image processing device according to claim 1 , wherein the first distance to the sixth distance are the same. The method of claim 19
The image processing apparatus according to claim 1 , wherein the area changing unit applies at least one of a predetermined color and a predetermined pattern to the second feature area. The method of claim 33,
The image processing device according to claim 1 , wherein the color is an average of pixel values of pixels constituting the second feature region. delete The method of claim 19
The control unit,
an image extraction unit extracting the image data to which the change is applied as anonymized image data; Image processing device characterized in that it further comprises.

Images