KR102553141B1 - Method and device for providing alopecia information - Google Patents

Abstract

A method for providing hair loss state information according to an embodiment of the present application includes obtaining a head image of a user; extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point; extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image; selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points; obtaining an upper face part calculation value determined based on a distance between a feature point of the first group and a matching boundary point corresponding thereto with respect to a first axis of the head image; obtaining a mid-lower face part calculation value determined based on a distance between the feature points of the first group and the feature points of the second group based on the first axis; and providing hair loss state information of the user based on the ratio of the calculated value of the upper face part and the calculated value of the lower middle face part.

Description

Method and apparatus for providing hair loss condition information {METHOD AND DEVICE FOR PROVIDING ALOPECIA INFORMATION}

The present application relates to a method for providing hair loss condition information of a user and an apparatus for providing hair loss condition information.

Hair loss refers to a state in which there is no hair in an area where hair should normally exist, and generally refers to the loss of terminal hair on the scalp. Hair loss is a disease with a high prevalence, and when it occurs, it can cause problems such as psychologically negative effects on individuals, such as stress, lack of confidence, social avoidance, and depression, and can even lead to mental illness in severe cases. Due to the negative effects of such hair loss, hair loss is recognized as a serious social problem, and accordingly, the market related to hair loss is steadily growing.

In order to diagnose and manage hair loss conditions and, if necessary, treat hair loss, it is essential to accurately determine how much hair loss has progressed from an individual's head image.

However, the conventional method for determining the hair loss state has been made by a method in which an individual directly visits a hospital or clinic and observes with the naked eye. In this case, there was the inconvenience of having to visit a hospital or clinic directly for consultation or diagnosis for hair loss treatment, and there was a limitation that it was difficult to obtain objective judgment results compared to normal people regarding the current hair loss progress.

Accordingly, there is a demand for development of a method for easily identifying and diagnosing a hair loss condition without an individual visiting the site.

The prior art document below discloses a hair management integrated diagnosis system and its operation method for observing and diagnosing hair loss and condition.

* Republic of Korea Intellectual Property Office (KR) Patent Publication 10-2004-0048668 (2004. 06. 10.)

One object of the present invention is to provide a method for providing hair loss condition information and an apparatus for providing hair loss condition information.

The problem to be solved by the present invention is not limited to the above-mentioned problems, and problems not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings. .

The method for providing hair loss state information disclosed in the present application includes obtaining a head image of a user; extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point; extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image; selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points; obtaining an upper face part calculation value determined based on a distance between a feature point of the first group and a matching boundary point corresponding thereto with respect to a first axis of the head image; obtaining a mid-lower face part calculation value determined based on a distance between the feature points of the first group and the feature points of the second group based on the first axis; and providing hair loss state information of the user based on the ratio of the calculated value of the upper face part and the calculated value of the middle and lower face part.

The solutions to the problems of the present invention are not limited to the above-described solutions, and solutions not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings. You will be able to.

According to an embodiment of the present application, the user can easily obtain information about his/her hair loss condition without visiting a site such as a hospital.

Effects of the present invention are not limited to the above-mentioned effects, and effects not mentioned will be clearly understood by those skilled in the art from this specification and the accompanying drawings.

1 is a diagram for explaining a method of providing hair loss state information according to an embodiment.
2 is a diagram for explaining the configuration of a user terminal.
3 is a diagram for explaining a hair loss condition information providing system according to an embodiment.
4 is a diagram for illustratively describing an operation of capturing a user's head through an image capturing unit.
5 is a diagram illustrating an image of a user's head acquired through an image capturing unit.
6 is a diagram for explaining modules included in the control unit of the user terminal described in FIGS. 1 to 3 and their operations.
7 is a diagram for explaining feature points extracted from a head image.
8 and 9 are views for explaining boundary points extracted from a head image.
10 illustrates an embodiment of a structure of a neural network for obtaining boundary points from a head image.
11 is a diagram for explaining a method of selecting a matching boundary point from a head image.
12 is a diagram illustrating feature points of a first group and matching boundary points corresponding thereto according to an exemplary embodiment.
13 is a diagram for explaining a method of selecting a matching boundary point according to an exemplary embodiment.
14 is a diagram for explaining a method of selecting a matching boundary point according to another embodiment.
15 is a diagram for explaining a process of providing hair loss state information according to an embodiment.
16 is a diagram for illustratively explaining a method of obtaining a first calculation value and a second calculation value based on a head image.
17 is a diagram for illustratively explaining a reference ratio used when determining a user's hair loss state.
18 is a flowchart illustrating a method for determining a user's hair loss state according to another embodiment.
19 is a diagram for explaining a method of determining a reference point in a head image.
20 is a diagram for explaining a method of calculating an area of an upper facial part according to an embodiment.
21 and 22 are diagrams illustrating an area of an upper facial portion according to an exemplary embodiment.
23 is a diagram for explaining a method for determining a user's hair loss state according to another embodiment.
24 is a diagram for explaining a method of recalibrating an image based on an inclined angle of a user's head in a head image.
25 is a diagram for explaining a detailed ratio related to a main part of a user's face.
26 and 27 are views for explaining that an output unit outputs information related to a user's hair loss state based on user input information.
28 and 29 are diagrams for exemplarily explaining that an output unit outputs a head image of a user after an operation.

The foregoing objects, features and advantages of the present application will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. However, the present application can apply various changes and can have various embodiments. Hereinafter, specific embodiments will be illustrated in the drawings and described in detail.

Like reference numerals designate essentially like elements throughout the specification. In addition, components having the same function within the scope of the same idea appearing in the drawings of each embodiment will be described using the same reference numerals, and overlapping descriptions thereof will be omitted.

If it is determined that a detailed description of a known function or configuration related to the present application may unnecessarily obscure the subject matter of the present application, the detailed description thereof will be omitted. In addition, numbers (eg, first, second, etc.) used in the description process of this specification are only identifiers for distinguishing one component from another component.

In addition, the suffixes "module" and "unit" for components used in the following embodiments are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinguished from each other by themselves.

In the following examples, expressions in the singular number include plural expressions unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have mean that features or components described in the specification exist, and do not preclude the possibility that one or more other features or components may be added.

In the drawings, the size of components may be exaggerated or reduced for convenience of explanation. For example, the size and thickness of each component shown in the drawings are arbitrarily shown for convenience of explanation, and the present invention is not necessarily limited to those shown.

If an embodiment is otherwise implementable, the order of specific processes may be performed differently from the order described. For example, two processes that are described in succession may be performed substantially concurrently, or may proceed in an order reverse to that described.

In the following embodiments, when components are connected, a case in which the components are directly connected as well as a case in which components are interposed between the components and connected indirectly is included.

For example, when it is said that components are electrically connected in this specification, not only the case where the components are directly electrically connected, but also the case where the components are interposed and electrically connected indirectly is included.

In the hair loss state information providing method according to an embodiment of the present application, obtaining a head image of a user; extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point; extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image; selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points; obtaining an upper face part calculation value determined based on a distance between a feature point of the first group and a matching boundary point corresponding thereto with respect to a first axis of the head image; obtaining a mid-lower face part calculation value determined based on a distance between the feature points of the first group and the feature points of the second group based on the first axis; and providing hair loss state information of the user based on the ratio of the calculated value of the upper face part and the calculated value of the middle and lower face part.

The plurality of boundary points are extracted based on probability values corresponding to the boundary lines of the user's hair and forehead, and the selecting of the matching boundary points includes at first points and second points corresponding to the feature points of the first group. setting a matching area between a first straight line and a second straight line respectively extending in the first axis direction; and selecting, as the matching boundary point, at least one of matching candidate boundary points having the probability value equal to or greater than a predetermined threshold among a plurality of boundary points included in the matching area, wherein the first point and the second point are the matching boundary points. The points may be points separated from the first group feature points by a first distance and a second distance, respectively, in a second axis direction orthogonal to the first axis.

The matching boundary point may be a point located farthest from the feature points of the first group among the matching candidate boundary points in the first direction of the first axis.

The upper face part calculation value includes the upper face part area value of the head image set based on the feature points of the first group and the matching boundary points, and the middle and lower face part calculation values include the feature points of the first group and the second group feature points. It may include a mid-lower face area value of the head image, which is set based on feature points of .

The selecting of the matching boundary points may include selecting a plurality of feature points including at least a first feature point and a second feature point from the feature points of the first group; and selecting a first matching boundary point and a second matching boundary point corresponding to the first feature point and the second feature point, respectively, wherein the providing hair loss state information comprises: Extracting a first reference point and a second reference point corresponding to the second feature point - The first reference point and the second reference point calculate the mid-lower face portion in a first direction of the first axis from the first feature point and the second feature point, respectively. It is a point separated by a distance of a preset percentage of the value -; obtaining a first upper face area value determined based on the first feature point, the second feature point, the first matching boundary point, and the second matching boundary point; calculating a second upper face area value based on the first feature point, the second feature point, the first reference point, and the second reference point; and providing hair loss state information of the user based on the ratio of the first upper facial area value and the second upper facial area value.

The providing of the user's hair loss state information includes providing information on a treatment region in the image in which the user needs treatment, wherein the treatment region is the first upper facial area value and the second image. It may be determined based on the difference in facial part area values.

The first upper face area value and the second upper face area value are calculated based on a reference horizontal length and a vertical length of the user's face, and the reference horizontal length is based on the aspect ratio of the user's face and a preset standard face length. can be determined based on

The providing of the user's hair loss state information includes obtaining information on the amount of hair treatment input from the user or a third party and outputting the forehead region after the treatment of the user according to the amount of hair treatment. can do.

The plurality of boundary points are obtained using a neural network model, and the neural network model obtains a plurality of reference boundary points corresponding to the boundary line between the hair and the forehead based on a learning head image including at least a part of the boundary line between the hair and the forehead. can be learned to do.

The neural network model may include a first part that acquires a region of interest including the hair and the forehead based on the training head image, and a second part that is learned to acquire the plurality of reference boundary points based on the region of interest. can

According to another embodiment, obtaining a head image of the user; extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point; extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image;

selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points; obtaining an upper face part calculation value determined based on a distance between the feature points of the first group and the matching boundary point with respect to a first axis of the head image; obtaining a mid-lower face part calculation value determined based on a distance between the feature points of the first group and the feature points of the second group based on the first axis; And providing a program stored in a computer readable recording medium to execute a hair loss condition information providing method comprising the step of providing hair loss condition information of the user based on the ratio of the calculated value of the upper face part and the calculated value of the middle and lower face part. can

Hereinafter, a hair loss condition information providing device, a hair loss condition information providing system, and a hair loss condition information providing method of the present application will be described with reference to the drawings.

1 is a diagram for explaining a method of providing hair loss state information according to an embodiment, and FIG. 2 is a diagram for explaining a configuration of a user terminal.

Referring to FIG. 1 , a method of providing hair loss state information according to an embodiment may be performed through a user terminal 1000 . Referring to FIG. 2, the user terminal 1000 includes a control unit 100, an image capture unit 200, a storage unit 300, a user input unit 400, an output unit 500, a power supply unit 600, and a communication unit ( 700) may be included. At this time, the user terminal 1000 may include a portable information communication device, for example, a smart phone, a tablet, and the like.

The image capturing unit 200 is a digital camera and may include an image sensor and an image processing unit. An image sensor is a device that converts an optical image into an electrical signal, and may be composed of a chip in which a plurality of photo diodes are directly incorporated. Illustratively, the image sensor may include a Charge Coupled Device (CCD), Complementary Metal Oxide Semiconductor (CMOS), and the like. Meanwhile, the image processing unit may generate image information by processing a photographed result.

The storage unit 300 is a storage means for storing data that can be read by a microprocessor, and includes a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), ROM, RAM, It may include CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. Data received by the user terminal 1000 may be stored in the storage unit 300 .

The user input unit 400 receives a user's input for the user terminal 1000 . The received input may be transmitted to the control unit 100 . According to an embodiment, the user input unit 400 may receive a user's input through a touch display. Also, the user input unit 400 may refer to a user interface screen through which a command is input from a user.

The output unit 500 outputs various types of information according to a control command of the control unit 100 . According to one embodiment, the output unit 500 may output information through a display panel. More specifically, the output unit 500 may output information related to the user's hair loss state through the display panel. However, the output unit 500 is not limited to a display panel, and may include various means capable of outputting information such as a speaker.

The power supply unit 600 includes a battery, and the battery may be built into the user terminal 1000 or detachably provided from the outside. The power supply unit 600 may supply power required by each component of the user terminal 1000 .

The communication unit 700 may include a wireless communication module and/or a wired communication module. Here, the wireless communication module may include a Wi-Fi communication module, a cellular communication module, and the like.

The controller 100 may include at least one processor. At this time, each processor may execute a predetermined operation by executing at least one instruction stored in the memory. Specifically, the controller 100 may control overall operations of elements included in the user terminal 1000 . In other words, the user terminal 1000 may be controlled or operated by the control unit 100 .

Meanwhile, although not shown in FIG. 2 , the user terminal 1000 may include various sensors. For example, the user terminal 1000 may include a gyro sensor, a temperature sensor, a motion sensor, a touch sensor, a proximity sensor, and the like.

According to an embodiment of the present application, a user may be provided with information about his or her hair loss condition, eg, whether or not hair loss has progressed, the degree of hair loss progressed, the type of hair loss, hair transplant simulation, and the like, through the user terminal 1000 . For example, a user may capture an image of his or her head through the image capture unit 200 provided in the user terminal 1000, and then receive information about the hair loss state through the output unit 500. there is.

As a more specific example, the user terminal 1000 obtains an image of the user's head photographed through the image capture unit 200, and then obtains information related to the user's hair loss state using a pre-learned neural network model, and obtains information related to the user's hair loss condition through the output unit. It can be provided to the user through 500. A more detailed method for the user terminal 1000 to obtain information related to the user's hair loss condition will be described later.

3 is a diagram for explaining a hair loss condition information providing system according to an embodiment. Referring to FIG. 3 , the hair loss condition information providing system may include a user terminal 1000 and a server (S).

According to an embodiment, the server S may perform learning of a neural network model for outputting hair loss state information of a corresponding user based on the user's head image. The user terminal 1000 may receive information on the learned neural network model from the server S. In this case, the user terminal 1000 may perform a series of operations for providing hair loss state information of the user by using the learned neural network model received from the server S. More specifically, the user terminal 1000 may obtain an image of the user's head and generate information about the user's hair loss state based on this. In addition, the user terminal 1000 may provide information about the user's hair loss state to the user.

According to another embodiment, a series of operations for determining the hair loss state information of the user described above may be performed by the server (S). That is, the server S may acquire the user's head image from the user terminal 1000 . Thereafter, the server S may determine the hair loss state of the user from the head image using the learned neural network model. In addition, the server (S) may transmit the user's hair loss state determination result to the user terminal (1000). In this case, the user terminal 1000 may provide the user with a result of determining the user's hair loss condition.

Meanwhile, the user terminal 1000 described above with reference to FIG. 3 may be a diagnostic device. In this case, a series of operations performed by the user terminal 1000 may be identical or correspondingly performed in the diagnostic device.

For example, the diagnosis device may receive information about the learned neural network model from the server S. In this case, the diagnosis device may perform a series of operations for providing hair loss condition information of the user by using the learned neural network model received from the server S.

As another example, the diagnosis device may transmit the user's head image to the server (S). Thereafter, the server S may determine the hair loss state of the user from the head image using the learned neural network model, and transmit the result to the diagnosis device.

Meanwhile, the diagnosis device may be a scalp condition measurement device used in hospitals, clinics, and the like. Alternatively, the diagnosis device may be a device used to diagnose a patient's hair loss condition in a hospital, clinic, etc., but is not limited thereto, and may include various types of known medical devices or devices.

Hereinafter, operations performed in the user terminal 1000 are described for convenience of description, but the same or corresponding operations may be performed in the diagnostic device according to implementation examples.

The user terminal 1000 may utilize the user's head image to determine the user's hair loss state. Hereinafter, a method of acquiring a head image by the user terminal 1000 will be described, and the obtained head image will be described in detail with reference to drawings.

The user terminal 1000 may obtain an image of the user's head through the image capturing unit 200 of FIG. 2 .

The user terminal 1000 may display a photographing guide through the output unit 500 when the user photographs his/her own head or another person's head through the image capturing unit 200 . The photographing guide may relate to the outline of a person's head or face, but is not limited thereto, and may include various types of known photographing guides.

When the user photographs the head based on the photographing guide and the head is positioned within the photographing guide by satisfying a predetermined criterion, the control unit 100 of the terminal activates the photographing button to allow the user's head to be photographed and stored. can

In this case, the predetermined criterion may be determined based on whether the user's head is fully positioned within the photographing guide. Preferably, the predetermined criterion may be determined based on whether the entire face including the user's hair is located in the photographing guide. As another example, the predetermined criterion may relate to whether a specific part of the user's head, eg, a boundary between hair and forehead, is located in the photographing guide.

Meanwhile, as described above, the user terminal 1000 may be equipped with a gyro sensor. In this case, the predetermined criterion may be related to an inclination degree of the user terminal 1000 obtained through a gyro sensor.

More specifically, the control unit 100 of the user terminal 1000 may obtain tilt information of the user terminal 1000 through a gyro sensor. Thereafter, the controller 100 may control the image capture unit 200 to capture a head image based on the tilt information. For example, the control unit 100 may allow the image capture unit 200 to capture a head image only when the degree of inclination of the user terminal 1000 is equal to or less than a certain level.

4 is a diagram for illustratively describing an operation of capturing a user's head through an image capturing unit. Referring to FIG. 4 , the user terminal 1000 may output a photographing guide message through the output unit 500 .

FIG. 4(a) is a diagram illustrating a case where the degree of inclination of the user terminal 1000 from side to side or back and forth satisfies a predetermined criterion. On the other hand, (b) of FIG. 4 is a diagram illustrating a case where the user terminal 1000 is tilted left and right, and (c) of FIG. 4 shows a case where the user terminal 1000 is tilted forward and backward. It is a drawing that represents

As shown in (a) of FIG. 4 , when the degree of inclination of the user terminal 1000 satisfies a predetermined criterion, the control unit 100 of the user terminal 1000 may activate the capture button of the image capture unit 200. there is. However, as shown in (b) or (c) of FIG. 4 , when the user terminal 1000 tilts left and right or back and forth, the user terminal 1000 disables the capture button of the image capture unit 200 and then tilts the terminal A photographing button may be activated when the level is below a certain level.

5 is a diagram illustrating an image of a user's head acquired through an image capturing unit. The head image obtained by the image capturing unit 400 may be transmitted to the controller 100 . Hereinafter, a user's head image will be described with reference to FIG. 5 .

Referring to FIG. 5 , the head image may be an image in which all or part of the user's head is photographed.

For example, if the head image is an image of the user's entire head, the head image may be an image including the user's entire face including hair. Alternatively, the head image may be an image including hair, forehead, and facial contours. Alternatively, the head image may be an image including all major parts of the face.

As another example, when the head image is an image obtained by capturing a part of the user's head, the head image may be an image obtained by capturing at least the user's hair and forehead. Alternatively, the head image may be an image obtained by capturing at least a portion of the user's hair and at least a portion of the forehead. Alternatively, the head image may be an image including at least one of main parts constituting the user's face. In this case, the main part may be a body organ such as the eyes, nose, mouth, and ears, or a specific part of the face such as the forehead, forehead, cheeks, and chin.

As another example, the head image may be an image of the front, left side or right side of the user's head. In this case, the image of the front of the user's head may be a face image of the user including at least one of the user's eyes, nose, and mouth. The images of the left and right sides of the user's head may be side images of the user's head in which at least a part of the user's forehead is included.

The head image may be an image in which the subject does not move or may be a video in which the subject moves. In this case, if the head image is a video, the video may include a plurality of images in which the front, left side and/or right side of the user's head are photographed. Alternatively, the video may refer to a video in which all of the boundary regions of the user's hair and forehead are photographed.

Meanwhile, a first axis AX1 and a second axis AX2 may be defined to indicate specific coordinates on the head image.

According to an embodiment, the first axis AX1 and the second axis AX2 may mean the y-axis and the x-axis of the image, respectively. For example, the head image may be an image captured in a state in which the user's head is not tilted, as shown in (a) of FIG. 5 . In this case, the first axis AX1 may correspond to the y-axis of the image. Also, the second axis AX2 may correspond to the x-axis on the image. In this case, the second axis AX2 may be an axis orthogonal to the first axis AX1.

According to another embodiment, the first axis AX1 and the second axis AX2 may be defined corresponding to the degree of inclination of the user's head on the head image. For example, the head image may be an image captured with the user's head tilted left and right, as shown in (b) of FIG. 5 . As shown in (b) of FIG. 5 , when it is determined that the user's head is tilted by an angle a with respect to the y-axis in the head image, the first axis AX1 may be defined as an axis tilted by a with respect to the y-axis. , and the second axis AX2 may be defined as an axis tilted by a with respect to the x-axis.

The head image may be divided into a plurality of regions. According to one embodiment, the head image may be divided into two regions. For example, the head image may be divided into an upper region and a lower region of the face. Alternatively, the head image may be divided into a forehead area and a lower face area. Alternatively, the head image may be divided into an upper face part and a middle lower face part. However, the embodiments of the present application are not limited thereto, and a head image may be classified by various terms capable of classifying one region into two regions. Hereinafter, for convenience of explanation, the two regions on the head image are defined as an upper face part and a middle lower face part, respectively, and then described.

In this case, a criterion for dividing the head image into two regions may be determined based on the user's eyebrows. For example, the upper facial part may refer to an upper portion of the eyebrows, and the lower middle facial portion may refer to a lower portion of the eyebrows.

According to another embodiment, the head image may be divided into three regions. For example, the head image may be divided into an upper region, a central region, and a lower region. Alternatively, the head image may be divided into a forehead region, a nose region, and a lip region. Alternatively, the head image may be divided into an upper face part, a middle face part, and a lower face part. However, the embodiments of the present application are not limited thereto, and a head image can be classified by various terms that can classify one region into three regions. Hereinafter, for convenience of description, three regions on the head image are defined as an upper face, a middle face, and a lower face, respectively, and then described.

In this case, criteria for dividing the head image into three regions may be determined based on the user's eyebrows and lips. For example, the upper face part may mean the upper part of the eyebrows, the middle face part may mean the lower part of the eyebrows and the upper part of the lips, and the lower face part may mean the lower part of the lips, but is not limited thereto. .

6 is a diagram for explaining modules included in the control unit of the user terminal described in FIGS. 1 to 3 and their operations.

The control unit 100 may determine the user's hair loss state based on the user's head image obtained by the image capturing unit 200 of the user terminal 1000 . More specifically, the controller 100 may receive the user's head image described in FIGS. 4 and 5 and determine the user's hair loss state based on this. Hereinafter, when the control unit 100 determines the user's hair loss condition, functions performed by each module included in the control unit 100 will be described.

Referring to FIG. 6 , the control unit 100 includes a boundary point extraction module 110, a feature point extraction module 120, a matching boundary point selection module 130, a first calculated value acquisition module 140, and a second calculated value acquisition module ( 150) and a user hair loss condition determination module 160.

The boundary point extraction module 110 may extract boundary points BP from the head image. Also, the feature point extraction module 120 may extract feature points (FP) from the head image. The matching boundary point selection module 130 may select a matching boundary point MBP from boundary points BP based on the feature point FP.

The first calculation value acquisition module 140 may obtain a first calculation value based on the feature point FP and the matching boundary point MBP. Also, the second calculation value obtaining module 150 may obtain a second calculation value based on the feature point FP. The user's hair loss state determination module 160 may determine the user's hair loss state, eg, whether hair loss progresses, the degree of hair loss, the type of hair loss, etc., based on the first calculated value and the second calculated value.

Meanwhile, the modules shown in FIG. 6 are sub-elements of the control unit 100 and may be integrated and performed by one or a plurality of processors according to embodiments. Hereinafter, the operation of each module constituting the control unit 100 will be described in detail with reference to the drawings.

7 is a diagram for explaining feature points extracted from a head image. Hereinafter, referring to FIG. 7, a method of extracting feature points (FP) from a head image will be described.

The feature point (FP) may represent a landmark for a major part of the face. Here, the main parts of the face are body organs constituting the face (eg, eyes, eyebrows, nose, mouth, ears, etc.) or specific parts of the face (eg, forehead, forehead, cheeks, chin, face) outlines, etc.).

Referring to FIG. 6 , feature points FP may be classified into a first group feature point G1FP and a second group feature point G2FP. In this case, each of the feature points G1FP of the first group and the feature points G2FP of the second group may include one or more feature points.

For example, the feature points G1FP of the first group may express landmarks related to a main part of the face located at an upper part based on a predetermined point, location, or part in the longitudinal direction of the face. In addition, the second group of feature points (G2FP) may represent landmarks related to a main part of the face located at a lower part based on a predetermined point, location, or part in the longitudinal direction of the face. Meanwhile, the feature points G2FP of the second group may be located lower than the feature points G1FP of the first group in the longitudinal direction of the face on the head image.

As another example, the feature points G1FP of the first group may express landmarks corresponding to the first main part of the face, and the feature points G2FP of the second group may express landmarks corresponding to the second main part of the face. can In this case, the first main part and the second main part of the face may be different from each other. For example, the feature points G1FP of the first group may represent landmarks corresponding to eyebrows, and the feature points G2FP of the second group may represent landmarks corresponding to the facial contour (eg, jaw line). can Alternatively, the first main part and the second main part of the face may be identical to each other. For example, the feature point G1FP of the first group may represent a landmark corresponding to the left eyebrow of the face, and the feature point G2FP of the second group may represent a landmark corresponding to the right eyebrow of the face.

The feature point extraction module 120 may extract a plurality of feature points FP from the head image captured by the image capturing unit 200 . More specifically, the feature point extraction module 120 may extract feature points (FP) from the head image using a preset algorithm. Here, the algorithm may refer to a computer algorithm that automatically detects a facial landmark corresponding to a major part of the face from a face image, but is not limited thereto, and various known face detection algorithms can include

According to an embodiment, the user terminal 1000 may store software related to the above algorithm. In this case, the user terminal 1000 may extract feature points (FP) from the head image according to the stored algorithm.

According to another embodiment, software related to the above algorithm may be stored in the server (S). In this case, the user terminal 1000 may extract feature points FP from the head image by transmitting and receiving data in real time through the server S.

8 and 9 are views for explaining boundary points extracted from a head image. Hereinafter, a method of extracting a plurality of boundary points BP from a head image will be described with reference to FIGS. 8 and 9 .

Referring to FIG. 8 , the boundary point BP may be a point included in the boundary region between the hair and the forehead. Alternatively, the boundary point BP may be a plurality of points corresponding to the boundary line between the hair and the forehead. More specifically, the boundary point BP may refer to a pixel having a probability of being a boundary point between the hair and the forehead, among a plurality of pixels constituting the head image of the user, that is equal to or greater than a predetermined standard. According to an embodiment, the probability corresponding to the boundary point between the hair and the forehead may be determined by a pre-learned neural network model.

According to an embodiment, the boundary point BP may be extracted based on a pixel corresponding to the boundary between the hair and the forehead, as shown in FIG. 8 . In this case, the boundary point BP may be extracted based on the segmentation image of the region including the hair and the forehead.

According to another embodiment, the boundary point BP may be extracted based on a region corresponding to the boundary between the hair and the forehead, as shown in FIG. 9 . In this case, one or more bounding boxes corresponding to the boundary area may be obtained in the area including the hair and the forehead. Thereafter, the boundary point extraction module 110 may extract a point at a preset position within the obtained bounding box as a boundary point. In this case, the preset position may be any one of a middle point or a corner of the bounding box.

10 illustrates an embodiment of a structure of a neural network for obtaining boundary points from a head image. Hereinafter, referring to FIG. 10, a method of extracting a boundary point BP from a head image through a pre-learned artificial neural network will be described.

Referring to FIG. 10 , the boundary point extraction module 110 may extract a boundary point BP from a head image through a pre-learned neural network model.

Referring to (a) of FIG. 10 , according to an embodiment, the boundary point extraction module 110 may extract a plurality of boundary points BP from a head image using one neural network model (NN). The neural network model (NN) may be trained to obtain a region of interest (ROI) and a boundary point (BP') by receiving a head image. In this case, the region of interest (ROI) may refer to a region including at least a part of the hair and forehead boundary and the surrounding hair and forehead.

A neural network model (NN) may be trained using training data including a head image and labeling data. In this case, the labeling data may include a first labeling value corresponding to the region of interest (ROI). Also, the labeling data may include a second labeling value corresponding to the boundary point.

Specifically, the neural network model (NN) may obtain an output value after receiving a head image. Thereafter, the neural network model NN may be learned by updating the neural network model NN based on an error value calculated by considering the difference between the output value and the labeling data. In this case, the output value may include a first output value corresponding to the first labeling value and a second output value corresponding to the second labeling value.

The boundary point extraction module 110 may obtain the region of interest (ROI) and the boundary point (BP') using a neural network model (NN). Thereafter, the boundary point extraction module 110 may finally extract only points located within the ROI from among the obtained boundary points BP′ as boundary points BP.

In detail, the boundary points BP' obtained through the neural network model NN may include both boundary points located within the region of interest (ROI) and boundary points located outside the region of interest (ROI). At this time, the boundary points BP finally extracted by the boundary point extraction module 110 may mean only points located within the region of interest ROI among the boundary points BP′ acquired through the neural network model NN.

Referring to (b) of FIG. 10 , according to another embodiment, the boundary point extraction module 110 uses a plurality of neural network models, for example, two neural network models NN1 and NN2 to extract a plurality of boundary points (from a head image). BP) can be extracted. The first neural network model NN1 may be trained to obtain a region of interest (ROI) by receiving a head image. In addition, the second neural network model NN2 may be trained to acquire the boundary point BP by receiving the region of interest ROI. At this time, the region of interest (ROI) is as described above through (a) of FIG. 10 .

The first neural network model NN1 and the second neural network model NN2 may refer to independent and separate neural network models, but are not limited thereto, and may be physically or logically separated from one neural network model. That is, the first part of the neural network model may acquire the region of interest (ROI) from the head image, and the second part may be trained to acquire the boundary point (BP) from the region of interest (ROI).

The first neural network model NN1 may be trained using training data including a head image and labeling data. In this case, the labeling data may include a labeling value corresponding to a boundary between the hair and the forehead and a region including at least a part of the hair and the forehead around the boundary. Specifically, the first neural network model NN1 may obtain an output value after receiving the head image. Thereafter, the first neural network model NN1 may be learned by updating the first neural network model NN1 based on the error value calculated by considering the difference between the output value and the labeling data.

The second neural network model NN2 may be trained using training data including an image of the region of interest (ROI) and labeling data. In this case, the labeling data may include a labeling value corresponding to the boundary point between the hair and the forehead. Specifically, the second neural network model NN2 may obtain an output value after receiving an image of the region of interest (ROI). Thereafter, the second neural network model NN2 may be learned by updating the second neural network model NN2 based on the error value calculated by considering the difference between the output value and the labeling data.

In other words, the boundary point extraction module 110 may obtain a region of interest (ROI) from the head image by using the first neural network model NN1, and obtain a region of interest (ROI) from the region of interest (ROI) by using the second neural network model NN2. A boundary point (BP) may be acquired. For example, the boundary point extraction module 110 uses the second neural network model NN2 to obtain the boundary point BP by using the ROI acquired through the first neural network model NN1 as input data. can As another example, the boundary point extraction module 110 uses the region of interest (ROI) including the boundary between the hair and the forehead and at least a part of the hair and the forehead around the boundary through the second neural network model NN2 as input data, and obtains the boundary point (BP) can be obtained.

Meanwhile, the neural network model is at least one of a Convolutional Neural Network (CNN), a Recursive Neural Network (RNN), a Deep Belief Network (DBN), a Long Short-term Memory (LSTM), a Gated Recurrent Neural Network (GRU), and variations and combinations thereof. can be one

11 is a diagram for explaining a method of selecting a matching boundary point from a head image. Hereinafter, referring to FIG. 11, a method of selecting a matching boundary point (MBP) from among a plurality of boundary points (BP) based on a plurality of feature points (FP) extracted from a head image will be described.

Referring to FIG. 11 , the matching boundary point selection module 130 may select a matching boundary point (MBP) through a matching region setting step (S131), a matching candidate boundary point extraction step (S133), and a matching boundary point selection step (S135). Hereinafter, each step performed by the matching boundary point selection module 130 will be described in more detail with reference to the drawings.

The matching boundary point MBP may refer to at least one of a plurality of boundary points BP extracted from the boundary point extraction module 110 . For example, the matching boundary point MBP may mean a boundary point corresponding to the feature point FP among the plurality of boundary points BP extracted from the boundary point extraction module 110 . Alternatively, the matching boundary point MBP may mean a boundary point corresponding to the feature point G1FP of the first group among the plurality of boundary points BP.

12 is a diagram illustrating feature points of a first group and matching boundary points corresponding thereto according to an exemplary embodiment. The feature points (G1FP) and matching boundary points (MBP) of the first group shown in FIG. 12 are according to an embodiment, and the number and location of the feature points (G1FP) and matching boundary points (MBP) of the first group are shown in FIG. 12. may be different from

12, the first feature point FP1 to the tenth feature point FP10 are the feature points G1FP of the first group, and the first matching boundary point MBP1 to the eleventh matching boundary point MBP11 are matching boundary points ( MBPs). In this case, the matching boundary point MBP may be selected as a boundary point corresponding to at least one feature point among the plurality of feature points FP. Alternatively, the matching boundary point MBP may be selected as a boundary point corresponding to one or more feature points among the feature points G1FP of the first group.

According to an embodiment, the matching boundary point MBP may be selected as a boundary point corresponding to any one feature point among the feature points G1FP of the first group. For example, referring to FIG. 12 , the matching boundary point MBP may be selected based on one of the first feature points FP1 to tenth feature points FP10 corresponding to the feature points G1FP of the first group. . For example, the first matching boundary point MBP1 may be a boundary point corresponding to the first feature point FP1 among the plurality of boundary points BP, and the second matching boundary point MBP2 may be a second boundary point among the plurality of boundary points BP. It may be a boundary point corresponding to the feature point FP2.

According to another embodiment, the matching boundary point MBP may be selected as a boundary point corresponding to two or more feature points among the feature points G1FP of the first group. More specifically, as the matching boundary point MBP, a boundary point corresponding to two neighboring points among the feature points G1FP of the first group may be selected as the matching boundary point. For example, referring to FIG. 12 , the matching boundary point MBP may be selected based on a plurality of feature points among the first feature points FP1 to tenth feature points FP10 corresponding to the feature points G1FP of the first group. . For example, the sixth matching boundary point MBP6 may be selected based on the fifth feature point FP5 and the sixth feature point FP6. More specifically, the sixth matching boundary point MBP6 may be a boundary point corresponding to the x-coordinate of an intermediate point between the fifth and sixth feature points FP5 and FP6.

The matching boundary point MBP may be a boundary point corresponding to a characteristic part of the face among the plurality of boundary points BP. In this case, the characteristic part of the face may mean a part corresponding to a feature point among the boundaries between the hair and the forehead. For example, the characteristic part of the face may mean a part corresponding to the central part of the forehead or a part corresponding to a specific part of the eyebrows among the boundaries between the hair and the forehead. Referring to FIG. 12, the above-mentioned matching boundary point corresponding to the central part of the forehead may be the sixth matching boundary point MBP6, and among the forehead boundaries, the matching boundary point corresponding to the specific part of the eyebrow may be the fourth matching boundary point MBP4. ) or the seventh matching boundary point MBP7.

According to an embodiment, the user hair loss state determination module 160 of FIG. 6 may determine the user's hair loss state based on the matching boundary point obtained by the matching boundary point selection module 130 . More specifically, the matching boundary point selection module 130 may select a matching boundary point (MBP) corresponding to one or more feature points satisfying a predetermined criterion among a plurality of feature points (FP), and the user hair loss state determination module 160 may The hair loss state of the user may be determined based on the selected matching boundary point (MBP).

For example, referring to FIG. 12 , the matching boundary point selection module 130 includes one or more feature points (eg, a third feature point FP3) and a fifth feature point (eg, a third feature point FP3) that satisfy a predetermined criterion among feature points G1FP of the first group. FP5), a matching boundary point (MBP) corresponding to the sixth feature point (FP6) or the eighth feature point (FP8) (eg, the fourth matching boundary point (MBP4), the sixth matching boundary point (MBP6), or the eighth matching boundary point (MBP8) )), and the user's hair loss state determination module 160 may determine the user's hair loss state based on the selected matching boundary point.

According to another embodiment, the user's hair loss state determination module 160 may determine the user's hair loss state based on one or more boundary points satisfying a predetermined criterion among the matching boundary points obtained by the matching boundary point selection module 130. . More specifically, the matching boundary point selection module 130 may select matching boundary points (MBPs) corresponding to the feature point (FP), and the user hair loss state determination module 160 may select a predetermined criterion from among the selected matching boundary points (MBPs). The hair loss state of the user may be determined based on one or more matching boundary points (MBP) that are satisfied.

For example, referring to FIG. 12 , the matching boundary point selection module 130 selects a plurality of matching boundary points corresponding to the feature points G1FP of the first group, for example, the first matching boundary point MBP1 to the eleventh matching boundary point MBP11. may be selected, and the user hair loss state determination module 160 selects a matching boundary point (MBP) that satisfies a predetermined criterion among the selected matching boundary points (MBP), for example, a fourth matching boundary point (MBP4), a sixth matching boundary point (MBP6), or The user's hair loss state may be determined based on the eighth matching boundary point MBP8.

Referring to FIG. 11 , in a matching area setting step (S131), the matching boundary point selection module 130 may set a matching area. As will be described later, the matching area may be used as a reference area for selecting a matching boundary point MBP among a plurality of boundary points BP.

The matching area may be set based on a single feature point or a plurality of feature points. 13 is a diagram for explaining a method of selecting a matching boundary point according to an exemplary embodiment. Referring to FIG. 13 , a matching area may be set based on a single feature point.

According to an embodiment, the matching area is formed by first straight lines extending in the direction of the first axis AX1 on the head image from two points P1 and P2 determined based on one first group feature point G1FP. L1) and the second straight line L2. In this case, the first point P1 is a point separated by a first distance from the first group feature point G1FP in the direction of the second axis AX2 on the head image, and the second point P2 is the first group feature point G1FP may be a point separated from by a second distance in the direction of the second axis AX2.

According to an embodiment, the first point P1 is a point separated by a first distance from the first group feature point G1FP in the first direction of the second axis AX2, and the second point P2 is the first group feature point G1FP. It may be a point separated from the feature point G1FP by a second distance in the second direction of the second axis AX2. In this case, the first direction may be a negative direction of the corresponding axis, and the second direction may be a positive direction of the corresponding axis. Also, the first distance and the second distance may be the same or different.

According to another embodiment, the first point P1 and the second point P2 may be points separated from the first group feature point G1FP by a first distance and a second distance in the same direction of the second axis AX2, respectively. can At this time, the first distance and the second distance are set to different values.

The second axis AX2 may be set as an axis orthogonal to the first axis AX1. According to an embodiment, the first axis AX1 may indicate the y-axis of the head image, and the second axis AX2 may indicate the x-axis of the head image, but the present invention is not limited thereto. For example, when it is determined in the obtained head image that the user's head is tilted by an angle a with respect to the y-axis of the image, the first axis AX1 and the second axis AX2 are the y-axis and the y-axis of the head image, respectively. It can be set as an axis tilted by a based on the x-axis.

14 is a diagram for explaining a method of selecting a matching boundary point according to another embodiment. Referring to FIG. 14 , a matching area may be set based on a plurality of feature points.

According to an embodiment, the matching area is an area between a first straight line L1 and a second straight line L2 extending from the plurality of first group feature points G1FP in the direction of the first axis AX1 on the head image, respectively. can be set. Specifically, the matching area includes first straight lines L1 and second straight lines extending in the direction of the first axis AX1 on the head image from the first and second feature points constituting the first group of feature points G1FP, respectively. It can be set to the area between (L2). In this case, the plurality of first group feature points G1FP may mean two feature points FP that are adjacent to each other among the feature points G1FP of the first group.

Meanwhile, definitions of the first axis AX1 and the second axis AX2 shown in FIG. 14 are the same as those of the first axis AX2 and the second axis AX2 described above with reference to FIG. 13 .

Referring back to FIG. 11 , in step of extracting matching candidate boundary points ( S133 ), the matching boundary point selection module 130 may extract matching candidate boundary points. More specifically, the matching boundary point selection module 130 may extract at least one of the boundary points located in the matching area among the plurality of boundary points BP as the matching candidate boundary point MCBP.

Here, the matching candidate boundary point MCBP may mean at least one boundary point located in the matching area among the plurality of boundary points BP. According to an embodiment, the matching candidate boundary point MCBP may mean at least one boundary point that satisfies a predetermined criterion among a plurality of boundary points BP located in the matching area. For example, the matching candidate boundary point MCBP may refer to a boundary point having a probability value corresponding to the boundary line between the hair and the forehead, among the plurality of boundary points BP located in the matching area, greater than or equal to a predetermined threshold value.

Referring to (a) of FIG. 13 , a matching candidate boundary point MCBP may be a boundary point corresponding to a single first group feature point G1FP. According to an embodiment, the matching candidate boundary point MCBP is formed extending in the direction of the first axis AX1 on the head image from two points P1 and P2 determined based on one first group feature point G1FP. It may be a boundary point located in a region between the first straight line L1 and the second straight line L2. Here, the first point P1 and the second point P2 are as described above in the matching area setting step S131. According to another embodiment, the matching candidate boundary point MCBP is formed extending in the direction of the first axis AX1 on the head image from two points P1 and P2 determined based on one first group feature point G1FP. Among the boundary points located in the region between the first straight line L1 and the second straight line L2, a boundary point having a probability value corresponding to the boundary line between the hair and the forehead may be equal to or greater than a predetermined threshold value.

Referring to (a) of FIG. 14 , the matching candidate boundary points MCBP may be boundary points corresponding to the plurality of first group feature points G1FP. According to an embodiment, the matching candidate boundary points MCBP include first straight lines L1 and second straight lines L2 extending from the plurality of first group feature points G1FP in the direction of the first axis AX1 on the head image, respectively. It may be a boundary point located in an area between. In this case, the plurality of first group feature points G1FP may mean two feature points FP that are adjacent to each other among the feature points G1FP of the first group. According to another embodiment, the matching candidate boundary point MCBP is a first straight line L1 and a second straight line L2 extending from the plurality of first group feature points G1FP in the direction of the first axis AX1 on the head image, respectively. Among the boundary points located in the region between, the boundary point may be a boundary point having a probability value corresponding to the boundary line between the hair and the forehead equal to or greater than a predetermined threshold value.

Referring back to FIG. 11 , in the matching boundary point selection step ( S135 ), the matching boundary point selection module 130 may select a specific matching boundary point (MBP) from among matching candidate boundary points. More specifically, the matching boundary point selection module 130 may select a matching boundary point MBP corresponding to the feature point G1FP of the first group. Alternatively, the matching boundary point selection module 130 may select at least one matching boundary point (MBP) that satisfies a certain criterion from among the matching candidate boundary points (MCBP) included in the above-described matching area.

The matching boundary point selection module 130 may select, as a matching boundary point (MBP), a boundary point whose representative value satisfies a predetermined criterion among the matching candidate boundary points (MCBP). In this case, the representative value may include a probability value that the corresponding boundary point corresponds to the boundary line between the hair and the forehead, the coordinate value of the first axis AX1 and/or the second axis AX2 of the corresponding boundary point on the head image, and the like. .

According to an embodiment, the matching boundary point selection module 130 may select, as the matching boundary point (MBP), a boundary point having the highest probability value corresponding to the boundary between the hair and the forehead among the matching candidate boundary points (MCBP) in the matching area. Referring to FIG. 13(b) and FIG. 14(b) , the matching boundary point selection module 130 selects the probability value corresponding to the boundary between the hair and the forehead among the matching candidate boundary points (MCBP) shown in each drawing. A high boundary point can be selected as a matching boundary point (MBP).

According to another embodiment, the matching boundary point selection module 130 may select a boundary point having the highest coordinate value of the first axis AX1 on the head image among the matching candidate boundary points MCBP in the matching area as the matching boundary point MBP. Referring to FIG. 13(c) and FIG. 14(c), the matching boundary point selection module 130 selects the head of the matching candidate boundary points (MCBP) shown in FIGS. 13(a) and 14(a), respectively. A boundary point having the highest coordinate value of the first axis AX1 on the image may be selected as the matching boundary point MBP.

According to another embodiment, the matching boundary point selection module 130 selects the first axis AX1 on the head image from among matching candidate boundary points (MCBP) having a probability value corresponding to the boundary between the hair and the forehead within the matching area equal to or greater than a predetermined standard. A boundary point having the highest coordinate value may be selected as a matching boundary point (MBP). Referring to FIGS. 13(c) and 14(c) , the matching boundary point selection module 130 selects a matching candidate whose probability value corresponding to the boundary between the hair and the forehead shown in each drawing is equal to or greater than a predetermined standard. Among the boundary points MCBP, a boundary point having the highest coordinate value of the first axis AX1 on the head image may be selected as the matching boundary point MBP.

According to another embodiment, although not shown in the drawing, the matching boundary point selection module 130 selects the second axis AX2 of the feature point FP, which is the criterion for selecting the matching region, among the matching candidate boundary points MCBP in the matching region. A boundary point corresponding to the coordinate value may be selected as a matching boundary point (MBP). Alternatively, the matching boundary point selection module 130 may select the second axis AX2 of the feature point FP that is the criterion for the selection of the matching region among the matching candidate boundary points MCBP having a probability value corresponding to the boundary between the hair and the forehead equal to or greater than a predetermined criterion. ) may select a boundary point corresponding to the coordinate value as a matching boundary point (MBP).

More specifically, the matching boundary point selection module 130 is the most suitable for the coordinate value of the second axis AX2 of the single first group feature point G1FP serving as the criterion for selecting the matching region among the matching candidate boundary points MCBP in the matching region. A boundary point having an adjacent coordinate value of the second axis AX2 may be selected as the matching boundary point MBP. Alternatively, the matching boundary point selection module 130 determines the median value of the coordinate values of the second axis AX2 of two neighboring feature points FP serving as criteria for selecting the matching region among the matching candidate boundary points MCBP in the matching region. A boundary point having the closest coordinate value of the second axis AX2 may be selected as the matching boundary point MBP.

In this way, the user terminal may select a matching boundary point (MBP) according to various criteria and determine the user's hair loss state based on the selected matching boundary point (MBP). In the above-described embodiments, when the boundary point having the highest coordinate value of the first axis AX1 on the head image is selected as the matching boundary point (MBP), the matching boundary point ( MBP), it is possible to provide an effect of being able to more conservatively determine the hair loss state of the user.

As described above, the matching boundary point (MBP) may be selected based on the feature point (FP), and the controller 100 may determine the user's hair loss state using at least one of the selected matching boundary points (MBP).

Meanwhile, in some cases, matching boundary points (MBP) required to determine the user's hair loss state may be omitted or insufficient. In this case, coordinates of missing matching boundary points may be determined based on at least one of the pre-selected matching boundary points (MBP). Hereinafter, a method of defining a missing matching boundary point based on at least one of the pre-selected matching boundary points (MBP) will be described.

According to a further embodiment, at least some of the matching boundary points (MBP) may be defined based on the pre-selected matching boundary points (MBP). For example, when there is a missing matching boundary point in the right region of the forehead based on the center of the head image, the missing matching boundary point (MBP) corresponding to the left region of the forehead among the previously selected matching boundary points (MBP) is selected. Matching boundary points can be defined.

Referring to FIG. 12, assuming that the eighth matching boundary point MBP8 is missing in the right region of the forehead based on the center of the head image, the fourth matching boundary point corresponding to the left region of the forehead among the previously selected matching boundary points MBP The missing eighth matching boundary point MBP8 may be defined based on (MBP4). In other words, the eighth matching boundary point MBP8 may be defined by the coordinates of the fourth matching boundary point MBP4. More specifically, the coordinates of the first axis AX1 of the eighth matching boundary point MBP8 may be defined as the coordinates of the first axis AX1 of the fourth matching boundary point MBP4. Also, the coordinates of the second axis AX2 of the eighth matching boundary point MBP8 may correspond to the coordinates of the second axis AX2 of the seventh feature point FP7. In this case, the seventh feature point FP7 may be a feature point corresponding to the fourth feature point FP4 that is the basis for selecting the fourth matching boundary point MBP4.

Meanwhile, although the case where the missing matching boundary point corresponds to the right region of the forehead based on the center of the head image has been described above, even when there is a missing matching boundary point in the left region of the forehead, the same method as the above method is used. The missing matching boundary point may be defined based on the matching boundary point MBP corresponding to the right region of the forehead among the selected matching boundary points MBP.

As another example, if the central matching boundary point corresponding to the center of the second axis AX2 of the head image is missing, the missing central matching boundary point corresponds to a matching region of the left or right eyebrow among the pre-selected matching boundary points MBP. It can be defined by the boundary point (MBP). Referring to FIG. 12 , assuming that among the boundary points between the hair and the forehead, the sixth matching boundary point MBP6 corresponding to the center of the second axis AX2 of the head image is missing, the first axis of the sixth matching boundary point MBP6 The (AX1) coordinate may be determined as the first axis AX1 coordinate of the fourth matching boundary point MBP4 corresponding to a region of the left eyebrow among the pre-selected matching boundary points. In this case, the matching boundary point MBP corresponding to one region of the left or right eyebrow may be the same as or correspond to the matching boundary point MBP described above with reference to FIG. 13 .

Meanwhile, the coordinates of the second axis AX2 of the missing sixth matching boundary point MBP6 are the second axis of the first feature point corresponding to one region of the left eyebrow and the second feature point corresponding to one region of the right eyebrow AX2) may be determined based on coordinates. For example, the second axis AX2 coordinate of the missing sixth matching boundary point MBP6 is the second axis AX2 coordinate of the first feature point and the second axis AX2 coordinate of the second feature point. It can be set as an intermediate coordinate. In this case, one region of the left eyebrow and one region of the right eyebrow may be symmetrical with respect to the first axis AX1 on the head image.

As another example, the coordinates of the first axis AX1 of the matching boundary point MBP are the first axes of another matching boundary point MBP that are symmetrical within the error range with respect to the first axis AX1 at the center of the head image. (AX1) can be determined by the coordinates. More specifically, assuming that the first matching boundary point and the second matching boundary point are symmetrical to each other within the error range with respect to the first axis AX1 at the center of the head image, the first axis AX1 of the first matching boundary point ) coordinates are greater than the coordinates of the first axis (AX1) of the second matching boundary point, the coordinates of the first axis (AX1) of the second matching boundary point are determined as the coordinates of the first axis (AX1) of the first matching boundary point. can Referring to FIG. 12 , the fourth matching boundary point MBP4 and the eighth matching boundary point MBP8 are symmetrical to each other within an error range with respect to the first axis AX1 on the head image. In this case, when the coordinates of the first axis AX1 of the fourth matching boundary point MBP4 are greater than the coordinates of the first axis AX1 of the eighth matching boundary point MBP8, the coordinates of the first axis AX1 of the eighth matching boundary point MBP8 The coordinates of the axis AX1 may be determined as the coordinates of the first axis AX1 of the fourth matching boundary point MBP4.

So far, various embodiments of extracting boundary points, feature points, and matching boundary points from a user's head image have been described. According to an embodiment of the present application, the user terminal may provide hair loss state information of the user based on the extracted boundary points, feature points, and/or matching boundary points.

15 is a diagram for explaining a process of providing hair loss state information according to an embodiment.

Referring to FIG. 15 , the process of providing hair loss state information according to an embodiment includes obtaining a head image of a user (S100), extracting boundary points and feature points from the obtained head image (S110, S120), and Selecting a matching boundary point (S130), obtaining a first calculation value and a second calculation value of the head image (S140, S150), determining the user's hair loss state based on the first calculation value and the second calculation value It may include step (S160) and providing hair loss state information to the user based on the result of determining the user's hair loss state (S170). Meanwhile, the flow chart of FIG. 15 is exemplary, and a partial order of each step shown in FIG. 15 may be changed.

Referring to FIG. 15 , the controller 100 of FIG. 2 may extract a boundary point BP from the head image through a boundary point extraction step ( S110 ). Specifically, the controller 100 may obtain an image of the user's head from the image capture unit 200 and extract a boundary point BP based on the image. In addition, the controller 100 may extract feature points (FP) from the head image through the feature point extraction step ( S120 ). Specifically, the controller 100 may obtain an image of the user's head from the image capture unit 200 and extract feature points (FP) based on the obtained image.

In addition, the controller 100 may select a matching boundary point MBP based on the boundary point BP and feature point FP of the head image through the matching boundary point selection step S130. Specifically, the controller 100 may select one or more matching boundary points MBP corresponding to the feature point FP from among the plurality of boundary points BP. Since the method for extracting the plurality of boundary points (BP) and the plurality of feature points (FP) from the head image and the method for selecting the matching boundary point (MBP) have been described above, a detailed description thereof will be omitted.

According to an embodiment, the controller 100 of FIG. 2 may calculate calculated values according to a preset criterion from a head image, and diagnose the user's hair loss condition based on the calculated values. In obtaining the first calculated value of FIG. 15 ( S140 ), the controller 100 may obtain the first calculated value for the head image. According to an embodiment, obtaining the first calculated value may be performed by the first calculated value obtaining module 140 of the controller 100 . The first calculation value acquisition module 140 may obtain a first calculation value based on the feature point FP and the matching boundary point MBP. Here, the first calculated value may mean a calculated value related to the upper face part described above.

For example, the first calculated value may be determined based on the distance between the feature point FP and the matching boundary point MBP. For example, the first calculated value is the distance between the feature point FP and the matching boundary point MBP determined based on the first axis AX1 or the feature point FP and the matching boundary point determined based on the second axis AX2. It can be determined based on the distance between (MBP). In other words, the first calculated value may mean a distance value between the first axis AX1 coordinate of the feature point FP and the first axis AX1 coordinate of the matching boundary point MBP corresponding to the feature point.

As another example, the first calculation value may be determined based on an area calculated by at least one feature point FP and at least one matching boundary point MBP. For example, the first calculated value may mean an area value calculated by a plurality of feature points FP and a plurality of matching boundary points MBP corresponding to each of the plurality of feature points FP.

16 is a diagram for illustratively explaining a method of obtaining a first calculation value and a second calculation value based on a head image. Hereinafter, referring to FIG. 16, a method of obtaining a first calculated value will be described in detail.

Referring to FIG. 16 , the first calculation value is determined based on the distance between the feature point G1FP of the first group and the matching boundary point MBP corresponding thereto with respect to the first axis AX1 on the head image. Can include output values. In this case, the calculated value of the upper facial portion may include the length d1 of the upper facial portion or may include an area value of the upper facial portion. In other words, the first calculated value may mean the length d1 of the upper facial part determined based on the preset region of the face. At this time, FIG. 16(a) is a diagram showing the length d1 of the upper facial part obtained based on the center region of the face by way of example.

Referring to (a) of FIG. 16 , the first calculated value may mean the length d1 of the upper facial part obtained based on the center region of the face. Here, the length d1 of the upper facial part, which is obtained based on the center region of the face, may mean the length in the direction of the first axis AX1 from the center of the upper facial part. In this case, the length d1 of the upper face part may be determined based on the central part of the face based on the second axis AX2 of the head image. According to an embodiment, the first calculation value may be calculated based on a single feature point G1FP of the first group and a single matching boundary point MBP selected based thereon. According to another embodiment, the first calculation value may be calculated based on a plurality of feature points G1FP of the first group and a single matching boundary point MBP selected based thereon.

For example, a single matching boundary point (MBP) may be selected based on a first feature point corresponding to one region of the left eyebrow and a second feature point corresponding to one region of the right eyebrow. In this case, one region of the left eyebrow and one region of the right eyebrow may be symmetrical with respect to the first axis AX1 on the head image. As a more specific example, the matching boundary point MBP may be selected to correspond to an intermediate point between the rightmost feature point of the left eyebrow and the leftmost feature point of the right eyebrow among the feature points G1FP of the first group. In this case, the first calculated value d1 may be determined by a difference between a coordinate value of the intermediate point and a coordinate value of the matching boundary point MBP based on the first axis AX1 on the head image. That is, the first calculation value d1 may be determined by a difference between the coordinate value of the first axis AX1 of the middle point and the coordinate value of the first axis AX1 of the matching boundary point MBP.

According to another embodiment, the first calculation value d1 is based on a plurality of first group feature points G1FP and a plurality of matching boundary points MBP selected based on each of the plurality of first group feature points G1FP. can be derived.

16(b) is a diagram showing the length d1 of the upper facial part obtained based on the area above the eyebrows by way of example. Referring to (b) of FIG. 16 , the first calculated value d1 may mean the length d1 of the upper face part obtained based on the area above the eyebrows. In this case, the first calculated value may be calculated by using at least one or more feature points G1FP of the first group and at least one or more matching boundary points MBP selected based on the feature point G1FP, as described in FIG. 16(a). Since it is the same as or corresponds to the bar, redundant description will be omitted.

For example, the first calculation value d1 may be calculated based on the feature point G1FP of the first group located in one region of the left or right eyebrow and the matching boundary point MBP selected correspondingly thereto. As a more specific example, the first calculation value d1 is determined by the difference between the coordinate value of the feature point G1FP of the first group and the coordinate value of the matching boundary point MBP based on the first axis AX1 on the head image. can be determined That is, the first calculation value d1 may be determined by a difference between the coordinate value of the first axis AX1 of the feature point G1FP of the first group and the coordinate value of the first axis AX1 of the matching boundary point MBP. there is.

Meanwhile, various embodiments have been described for the case where the first calculated value d1 represents the length through (a) and (b) of FIG. 16, but is not limited thereto, and the first calculated value d1 is at least It may be a value related to the area or width calculated by one or more feature points (FP) and one or more matching boundary points (MBP). For example, although not shown in the drawing, the first calculated value may be an area value calculated by at least one feature point FP and at least one matching boundary point MBP. Specifically, the first calculated value may mean an area value of the upper face part calculated using the coordinates of at least one feature point FP and the coordinates of at least one matching boundary point MBP. As another example, although not shown in the drawing, the first calculated value may be a width value calculated by at least one feature point FP and at least one matching boundary point MBP. Specifically, the first calculated value is a width determined by the coordinates of the second axis AX2 of at least one feature point FP and the coordinates of the second axis AX2 of at least one matching boundary point MBP, for example, a head image. This may mean a distance value in the direction of the second axis AX2 of the image.

In obtaining the second calculated value of FIG. 15 ( S150 ), the controller 100 may obtain the second calculated value for the head image. According to an embodiment, obtaining the second calculated value may be performed by the second calculated value obtaining module 150 of the controller 100 . The second calculated value acquisition module 140 may obtain a second calculated value based on the feature point FP. Here, the second calculated value may mean a calculated value related to the above-described lower middle face.

For example, the second calculated value may be determined based on the distance between the plurality of feature points FP. Alternatively, the second calculated value may be determined based on the distance between the feature points G1FP of the first group and the feature points G2FP of the second group. For example, the second calculated value is a distance between the feature points G1FP of the first group and the feature points G2FP of the second group determined based on the first axis AX1 or determined based on the second axis AX2. It may be determined based on the distance between the feature points G1FP of the first group and the feature points G2FP of the second group. As a more specific example, the second calculated value is the coordinates of the first axis AX1 of the feature points G1FP of the first group and the first axis of the feature points G2FP of the second group corresponding to the feature points G1FP of the first group. (AX1) may mean a distance value between coordinates.

As another example, the second calculation value may be determined based on an area calculated by at least a plurality of feature points FP. For example, the second calculated value is an area value calculated by a plurality of first group feature points G1FP and a plurality of second group feature points G2FP corresponding to each of the plurality of first group feature points G1FP. can mean

Hereinafter, referring to FIG. 16, a method of obtaining the second calculated value will be described in detail. Referring to FIG. 16 , the second calculated value is determined based on the distance between the first group feature point G1FP and the second group feature point G2FP based on the first axis AX1 on the head image. It may include the calculated value of the mid-lower facial part. In this case, the calculated value of the lower middle facial portion may include the length d2 of the lower middle facial portion, or may include an area value of the lower middle facial portion.

Referring to (a) of FIG. 16 , the second calculated value may mean the length d2 of the lower middle face part obtained based on the center region of the face. Here, the length d2 of the lower middle facial part, which is obtained based on the center region of the face, may mean a length in the first axial direction from the center of the lower middle facial part. In this case, the length d2 of the lower middle facial part may be determined based on the central part of the face based on the second axis AX2 on the head image.

According to an embodiment, the second calculated value may be calculated based on a single feature point G1FP of the first group and a single feature point G2FP of the second group selected based thereon. Alternatively, the second calculated value may be calculated based on the single feature point G2FP of the second group and the single feature point G1FP of the first group selected based thereon.

According to another embodiment, the second calculation value may be calculated based on a plurality of first group feature points G1FP and a single second group feature point G2FP selected based thereon. Alternatively, the second calculation value may be calculated based on a plurality of second group feature points G2FP and a single first group feature point G1FP selected based thereon.

Exemplarily, the second group of feature points (G2FP) is selected based on the first group of feature points (G1FP) corresponding to one region of the left eyebrow and the first group of feature points (G1FP) corresponding to one region of the right eyebrow. It can be. In this case, one region of the left eyebrow and one region of the right eyebrow may be symmetrical with respect to the first axis AX1 on the head image. As a more specific example, the feature point G2FP of the second group may be selected to correspond to the midpoint between the feature point located on the rightmost side of the left eyebrow and the feature point located on the leftmost side of the right eyebrow among the feature points G1FP of the first group. there is. In this case, the second calculated value d2 may be determined by a difference between the coordinate value of the intermediate point and the coordinate value of the feature point G2FP of the second group based on the first axis AX1 on the head image. That is, the second calculation value d2 may be determined by a difference between the coordinate value of the first axis AX1 of the intermediate point and the coordinate value of the first axis AX1 of the feature point G2FP of the second group.

According to another embodiment, the second calculated value d2 is a plurality of first group feature points G1FP and a plurality of second group feature points G2FP selected based on each of the plurality of first group feature points G1FP. can be calculated based on

FIG. 16(b) is a diagram showing the length d2 of the upper facial part obtained based on the area under the eyebrows by way of example. Referring to (b) of FIG. 16 , the second calculated value d2 may mean the length d2 of the lower middle face part obtained based on the area under the eyebrows. In this case, the second calculated value may be calculated by at least one feature point G1FP of the first group and at least one feature point G2FP of the second group selected based on the at least one feature point G2FP of the first group. ), so that it is the same as or equivalent to that described in, duplicate descriptions are omitted.

For example, the second calculation value d2 may be calculated based on the first group of feature points G1FP located in one region of the left or right eyebrow and the second group of feature points G2FP selected correspondingly thereto. . As a more specific example, the second calculated value d2 is the coordinate value of the first group feature point G1FP and the second group feature point G2FP based on the first axis AX1 on the head image. difference can be determined. That is, the second calculated value d2 is based on the difference between the coordinate values of the first axis AX1 of the feature points G1FP of the first group and the coordinate values of the coordinates of the first axis AX1 of the feature points G2FP of the second group. can be determined by

Meanwhile, various embodiments have been described for the case where the second calculated value d2 represents the length through (a) and (b) of FIG. 16, but is not limited thereto, and the second calculated value d2 is at least It may be a value related to the area or width calculated by one or more feature points (FP), for example, the feature points (G1FP) of the first group and the feature points (G2FP) of the second group.

For example, although not shown in the drawing, the second calculation value may be an area value calculated by a plurality of feature points FP. Specifically, the second calculated value may mean an area value of the mid-lower face part calculated using coordinates of the feature points G1FP of the first group and the feature points G2FP of the second group.

As another example, although not shown in the drawings, the second calculated value may be a width value calculated by a plurality of feature points FP, for example, the first group of feature points G1FP and the second group of feature points G2FP. Specifically, the second calculated value is a width determined by the second axis AX2 coordinates of the feature points G1FP of the first group and the coordinates of the second axis AX2 of the feature points G2FP of the second group, for example, a head image. This may mean a distance value in the direction of the second axis AX2 of the image.

Meanwhile, the first calculation value and the second calculation value may be calculated in consideration of the tilt angle of the user's head in the head image. Specifically, when the user's head is tilted in the head image as shown in (b) of FIG. 5, the first calculation value or the second calculation value may be calculated after correcting the reference axis of the image or the image itself. For example, in the head image, when the user's head is tilted by an angle a with respect to the y-axis, the first axis AX1 and the second axis AX2 on the head image are defined as a relative to the y-axis and x-axis of the image, respectively. The first calculation value and the second calculation value may be calculated by defining an axis tilted by . As another example, after correcting the head image itself based on the degree of inclination of the head image, the first calculation value and the second calculation value may be calculated based on the corrected head image.

Meanwhile, as described above, the first calculation value and the second calculation value may be obtained based on a preset area of the face. At this time, in relation to the preset regions of the face, in (a) and (b) of FIG. 16 , the preset regions of the face are exemplified as the center region of the face and the upper/lower eyebrow regions of the face, respectively, but are limited thereto. it is not going to be Although not shown in the drawing, the preset region of the face may mean various regions of the main part of the face.

In addition, in (a) of FIG. 16, it is shown that the first calculated value d1 and the second calculated value d2 are obtained based on the center region of the face, and in (b) of FIG. 16, the first calculated value d2 is obtained. Although the value d1 and the second calculated value d2 are calculated based on the area above/below the eyebrows of the face, the present invention is not limited thereto. For example, the first calculated value d1 may be obtained based on the center region of the face, and the second calculated value d2 may be obtained based on the area under the eyebrows of the face. As another example, the first calculation value d1 may be obtained based on the area above the eyebrows of the face, and the second calculation value d2 may be obtained based on the center area of the face. In addition, although not shown in the figure, the first calculated value d1 and the second calculated value d2 may be obtained based on different preset regions of the face.

Referring back to FIG. 15 , the controller 100 may determine the user's hair loss state through the user's hair loss state determination step (S160). More specifically, the user's hair loss state determination module 160 of the controller 100 may determine the user's hair loss state based on the first calculated value and the second calculated value. More specifically, the user's hair loss state determination module 160 determines the user's hair loss state based on the ratio of the first calculated value (eg, the upper facial part calculated value) and the second calculated value (eg, the middle and lower facial part calculated value). can

According to one embodiment, the user's hair loss state determination module 160 may determine the ratio of the first calculated value and the second calculated value by comparing it with a predetermined ratio. For example, the user's hair loss state determination module 160 may determine that the user's hair loss state is normal when the ratio of the first calculated value and the second calculated value is compared with a predetermined ratio and satisfies the first criterion. As another example, the user's hair loss state determination module 160 may determine that the user's hair loss state is in progress when the ratio of the first calculated value and the second calculated value does not satisfy the first criterion compared to a predetermined ratio.

As a more specific example, the user's hair loss state determination module 160 may determine that the user's hair loss state is in progress when the ratio of the first calculated value to the second calculated value is equal to or greater than a predetermined ratio. Alternatively, the user's hair loss state determination module 160 may determine that the user's hair loss state is normal when the ratio of the first calculated value to the second calculated value is less than a predetermined ratio. In this case, the predetermined ratio may mean a ratio between parts or regions constituting the face. Alternatively, the predetermined ratio may refer to a predetermined ratio so that each part or region constituting the face feels stable and balanced.

17 is a diagram for illustratively explaining a reference ratio used when determining a user's hair loss state. Referring to FIG. 17 , the predetermined ratio may include a ratio between the length (a) of the upper facial part, the length (b) of the middle facial part, and the length (c) of the lower facial part. Alternatively, the predetermined ratio may include a ratio between the length (a) of the upper facial part and the length (b+c) of the lower middle facial part. More specifically, the predetermined ratio may include a ratio of the length (a) from the boundary line of the hair and the forehead to the eyebrows and the length (b+c) from the lower part of the eyebrows to the tip of the chin. In this case, the length (a) of the upper facial part, the length (b) of the middle facial part, and the length (c) of the lower facial part may mean lengths determined based on the first axis AX1 on the head image.

Meanwhile, the predetermined ratio may vary according to a user's selection. That is, the predetermined ratio may be determined based on a ratio considered ideal by the user. For example, the predetermined ratio may be determined based on a sample head image that is preferred by the user or considered ideal. Accordingly, the user's hair loss state determination module 160 may determine the user's hair loss state based on the ratio in which the user's taste is reflected.

18 is a flowchart illustrating a method for determining a user's hair loss state according to another embodiment.

Referring to FIG. 18, a process for determining a user's hair loss state according to another embodiment includes obtaining a head image of the user (S200), extracting boundary points and feature points from the obtained head image (S210, S220), and head image. Selecting a matching boundary point (S230), extracting a reference point from the head image (S240), obtaining a first upper face calculated value and a second upper face calculated value of the head image (S250, S260), Determining the user's hair loss state based on the first upper face part calculation value and the second upper face part calculation value (S270), and providing hair loss state information to the user based on the user's hair loss state determination result (S280). can Meanwhile, the flow chart of FIG. 18 is exemplary, and a partial order of each step shown in FIG. 18 may be changed.

Meanwhile, the user's head image acquisition step (S200) to the matching boundary point selection step (S230) of FIG. 18 are performed the same as or corresponding to the user's head image acquisition step (S100) to the matching boundary point selection step (S130) of FIG. 15, respectively. It can be.

According to another embodiment, the control unit 100 may determine the user's hair loss state by comparing the calculated value of the upper face part with the reference value. The reference value may refer to a reference value for determining the user's hair loss condition. More specifically, the upper face part calculation value may be a value corresponding to the user's current forehead region obtained from the head image. In this case, the reference value determines whether the upper face part calculation value is greater than when the user's hair loss state is normal Alternatively, it may be a reference value for determining whether it is small. In this case, the reference value may be determined based on a predetermined ratio.

That is, the controller 100 may determine whether the hair loss state of the user is normal or ongoing by comparing the calculated value of the upper face part obtained from the head image with the reference value. Illustratively, the controller 100 may determine the user's hair loss state based on whether the calculated value of the upper face part is greater than or less than the reference value. As another example, the control unit 100 may determine the user's hair loss state based on whether the upper face part calculation value falls within a predetermined ratio of the reference value. Hereinafter, for convenience of description, the upper face part calculation value is defined as a first upper face part calculation value, and the reference value is defined as a second upper face part calculation value.

The controller 100 may obtain a first upper face calculated value based on the feature point FP and the matching boundary point MBP corresponding thereto through the first upper face calculated value obtaining step ( S250 ). A specific embodiment for this corresponds to the method for obtaining the first calculated value described above with reference to FIGS. 15 and 16, and duplicate descriptions are omitted.

The controller 100 may obtain the second upper face part calculation value based on the feature point FP and the reference point RP corresponding thereto through the second upper face part calculation value obtaining step ( S260 ). Alternatively, the control unit 100 obtains the second upper face calculated value based on the calculated value of the middle and lower face based on the plurality of feature points (FP) and a predetermined ratio through the second calculated value of the upper face (S260). can do.

Meanwhile, the second upper face part calculation value may be determined based on a distance between a feature point FP determined based on the first axis AX1 on the head image and a reference point RP corresponding thereto. Alternatively, the second upper face part calculation value may be determined based on the distance between the coordinates of the feature point FP and the coordinates of the reference point RP corresponding thereto. For example, the second upper face part calculation value may mean a distance value between the first axis AX1 coordinate of the feature point FP and the first axis AX1 coordinate of the reference point RP corresponding thereto.

The control unit 100 may extract a reference point from the head image through step S240 of extracting the reference point. Specifically, the reference point RP may be determined as a point separated from the feature point FP on the first axis AX1 by a distance of a preset ratio of the calculated value of the mid-lower facial part. For example, the reference point RP is the feature point G1FP of the first group by a distance of a preset ratio of the calculated value of the mid-lower face part determined based on the feature point G1FP of the first group and the feature point G2FP of the second group. It may be determined as a point away from in the first direction of the first axis AX1. Hereinafter, a method of determining the reference point RP will be described in detail with reference to the drawings.

19 is a diagram for explaining a method of determining a reference point in a head image. Referring to FIG. 19 , the reference point RP may be a point separated from the feature point G1FP of the first group in the first direction of the first axis AX1 on the head image by a distance of a preset ratio of the calculated value of the mid-lower facial part. . In other words, the reference point RP may be a point away from the feature points G1FP of the first group in the first direction of the first axis AX1 on the head image by a value obtained by multiplying or dividing the mid-lower facial part calculation value by a preset ratio. . Meanwhile, the mid-lower face part calculation value may be determined based on a plurality of feature points (FP), which may correspond to the second calculation value described above with reference to FIGS. 15 and 16 . In addition, the preset ratio may correspond to the ratio described above with reference to FIG. 17, and redundant descriptions in this regard will be omitted.

The controller 100 may determine the user's hair loss state based on the calculated value of the first upper face part and the second calculated value of the upper face part in the user's hair loss state determination step ( S270 ). For example, the controller 100 may determine that the hair loss state of the user is normal when the first upper face part calculation value is equal to or less than the second upper face part calculation value. Alternatively, the controller 100 may determine that the user's hair loss condition is in progress when the first upper face part calculation value exceeds the second upper face part calculation value. Thereafter, the controller 100 may provide information on the user's hair loss state based on the user's hair loss state determination result in the user's hair loss state information providing step (S280).

According to another embodiment, the user's hair loss state determination module 160 of FIG. 6 may determine the user's hair loss state based on the upper facial area value. More specifically, the user's hair loss state determination module 160 may determine the user's hair loss state based on the area value of the first upper face part and the area value of the second upper face part.

Here, the area value of the first upper face part and the area value of the second upper face part may be determined based on the user's head image. However, the area value of the first upper facial part means the area of the user's current forehead area (for example, the area from above the eyebrows to the area below the boundary line between the hair and the forehead in the head image), and the area value of the second upper facial part is defined in advance. It may refer to the area of the user's ideal forehead region determined based on the predetermined ratio. That is, the user's hair loss state determination module 160 may determine whether the user's hair loss is progressing or normal by comparing the area of the user's current forehead region determined through the head image with the area of the user's ideal forehead region.

Meanwhile, the area value of the first upper face part may be calculated based on a plurality of feature points FP and a plurality of matching boundary points MBP corresponding thereto. Also, the area value of the second upper face part may be calculated based on a plurality of feature points FP and a plurality of reference points RP corresponding to the plurality of feature points FP. Hereinafter, a method of calculating the area value of the first upper facial part and the area value of the second upper facial part will be described with reference to the drawings.

The feature point extraction module 120 of FIG. 6 may extract a plurality of feature points (FP) from the head image. In this case, the plurality of feature points FP extracted by the feature point extraction module 120 may include a first feature point FP1 and a second feature point FP2.

20 is a diagram for explaining a method of calculating an area of an upper facial part according to an exemplary embodiment. Referring to FIG. 20 , each of the first feature point FP1 and the second feature point FP2 may be any one of the feature points G1FP of the first group. For example, the first feature point FP1 may be any one of a plurality of feature points located in one region of the left eyebrow among the feature points G1FP of the first group. Also, the second feature point FP2 may be any one of feature points located in one region of the right eyebrow among the feature points G1FP of the first group. The first feature point FP1 and the second feature point FP2 may be any one of the feature points located symmetrically with each other within the error range with respect to the first axis AX1 at the center of the head image among the feature points G1FP of the first group. can For example, the first feature point FP1 may be a feature point corresponding to the center of the left eyebrow among the feature points G1FP of the first group, and the second feature point FP2 may be the feature point of the right eyebrow among the feature points G1FP of the first group. It may be a feature point corresponding to the center of .

The matching boundary point selection module 130 of FIG. 6 may select a plurality of matching boundary points (MBP) from among boundary points extracted from the head image. In this case, the plurality of selected matching boundary points MBP may include a first matching boundary point MBP1 and a second matching boundary point MBP2. Referring to FIG. 20 , the matching boundary point selection module 130 may select a first matching boundary point MBP1 corresponding to the first feature point FP1. Also, the matching boundary point selection module 130 may select a second matching boundary point MBP2 corresponding to the second feature point FP2. Meanwhile, since the method of selecting the matching boundary point (MBP) corresponding to the feature point (FP) has been described above with reference to FIGS. 11 to 14, a duplicate description is omitted.

According to an additional embodiment, the controller 100 may extract a plurality of reference points from the head image. In this case, the plurality of extracted reference points may include a first reference point BP1 and a second reference point BP2. Referring to FIG. 20 , the controller 100 may extract a first reference point RP1 corresponding to the first feature point FP1. Also, the controller 100 may extract a second reference point RP2 corresponding to the second feature point FP2. Meanwhile, the plurality of reference points RP may be determined based on the feature point FP and a predetermined ratio, and since this has been described above with reference to FIG. 19, duplicate descriptions will be omitted.

21 and 22 are diagrams illustrating an area of an upper facial portion according to an exemplary embodiment. Referring to FIG. 21 , as described above, the upper face part may refer to an area corresponding to an area between the boundary line of the hair and the forehead in the head image and the upper part of the eyebrows.

Referring to (a) of FIG. 21 , the first upper face part AR1 is based on the first feature point FP1, the second feature point FP2, the first matching boundary point MBP1, and the second matching boundary point MBP2. It can be a defined area. That is, the first upper face part AR1 may correspond to the user's current forehead region, for example, an area from above the eyebrows to below the boundary line between the hair and the forehead. Also, referring to (b) of FIG. 21 , the second upper face part AR2 is based on the first feature point FP1, the second feature point FP2, the first reference point RP1, and the second reference point RP2. It can be a defined area. That is, the second upper face part AR2 may correspond to an ideal forehead area of the user determined based on a preset ratio, for example, an area between the first reference point RP1 and the second reference point RP2 from above the eyebrows.

According to an embodiment, the controller 100 determines the first upper face part AR1 based on the first feature point FP1, the second feature point FP2, the first matching boundary point MBP1, and the second matching boundary point MBP2. The area value of can be calculated. The controller 100 determines the area value of the first upper face part AR1 based on the coordinates of the first feature point FP1, the second feature point FP2, the first matching boundary point MBP1 and the second matching boundary point MBP2, respectively. can be calculated. For example, the controller 100 performs a trigonometric function calculation method based on the coordinates of the first feature point FP1, the second feature point FP2, the first matching boundary point MBP1, and the second matching boundary point MBP2. 1 The area value of the upper face part AR1 may be calculated, but is not limited thereto, and various known methods for obtaining the area using coordinates may be used.

Meanwhile, the controller 100 calculates the area value of the second upper face part AR2 based on the first feature point FP1, the second feature point FP2, the first reference point RP1, and the second reference point RP2. And since the specific method is the same as the method of calculating the area value of the first upper face part AR1, overlapping descriptions will be omitted.

According to another embodiment, the controller 100 may perform the first phase based on the reference horizontal length, the first feature point FP1, the second feature point FP2, the first matching boundary point MBP1, and the second matching boundary point MBP2. An area value of the face part AR1 may be calculated. Specifically, the controller 100 obtains the vertical length of the first upper face part AR1 based on the feature point FP and the corresponding matching boundary point MBP, and uses the obtained vertical length and the reference horizontal length. Thus, the area value of the first upper face part AR1 may be calculated.

Referring to FIG. 21 , the control unit 100 determines the vertical length of the first upper face part AR1 (eg, the length determined based on the first feature point FP1 and the corresponding first matching boundary point MBP1 or An area value of the first upper face part AR1 may be calculated by multiplying a length determined based on the second feature point FP2 and the corresponding second matching boundary point MBP2 by the reference horizontal length. In this case, the reference horizontal length may be determined based on an aspect ratio of the user's face and a preset standard face length. In this case, the aspect ratio of the user's face may be the ratio of the horizontal length to the vertical length of the user's face determined based on at least one of a feature point (FP) or a boundary point (BP) existing in the head image. Specifically, the vertical length of the user's face may be determined based on any one of the feature points FP and any one of the boundary points BP. Also, the horizontal length of the user's face may be determined based on a plurality of feature points (FP) or a plurality of boundary points (BP).

Meanwhile, the preset standard face length may include a preset standard face horizontal length or vertical length. For example, the preset standard face length may be an average vertical (or horizontal) length of male (or female) faces. For example, the reference horizontal length may be determined based on a preset vertical length of a standard male face and an aspect ratio of a user's face.

FIG. 22 is a diagram for explaining a method of determining a user's hair loss state based on a first upper facial part and a second upper facial part by a user's hair loss state determination module. Referring to FIG. 22 , the user's hair loss state determination module 160 may determine the user's hair loss state based on the area value of the first upper face part AR1 and the area value of the second upper face part AR2.

More specifically, the user's hair loss state determination module 160 may determine the user's hair loss state based on the difference between the area value of the first upper face part AR1 and the area value of the second upper face part AR2. Alternatively, the user's hair loss state determination module 160 may determine the user's hair loss state based on the ratio between the area value of the first upper face part AR1 and the area value of the second upper face part AR2.

For example, the user's hair loss state determination module 160 may determine whether the area value of the first upper facial part AR1 is larger or smaller than the area value of the second upper facial part AR2 (ie, the reference area value). of hair loss can be judged. For example, the user's hair loss state determination module 160 may determine that the user's hair loss state is normal when the area value of the first upper face part AR1 is smaller than the area value of the second upper face part AR2. In addition, the user's hair loss state determination module 160 may determine that the user's hair loss state is in progress when the area value of the first upper facial portion AR1 is greater than the area value of the second upper facial portion AR2. As another example, the user's hair loss state determination module 160 may determine the user's hair loss state in stages based on the ratio of the area value of the first upper face part AR1 to the area value of the second upper face part AR2.

23 is a diagram for explaining a method for determining a user's hair loss state according to another embodiment. Hereinafter, a method for determining a user's hair loss state according to another embodiment will be described with reference to FIG. 23 .

Referring to FIG. 23 , the user's hair loss state determination module 160 may determine the user's hair loss state based on the feature point G1FP and the matching boundary point MBP of the first group. The user's hair loss state determination module 160 may determine the user's hair loss state based on the first matching boundary point MBP1 and the second matching boundary point MBP2 selected in correspondence with at least one feature point G1FP of the first group. there is.

The first matching boundary point MBP1 may be a boundary point having the highest coordinate value of the first axis AX1 on the head image among matching candidate boundary points MCBP corresponding to feature points located in one region of the left or right eyebrow. Since this has been described above in (c) of FIG. 13, overlapping descriptions are omitted. The second matching boundary point MBP2 may be a boundary point corresponding to the central part of the forehead in the head image. According to an embodiment, the second matching boundary point MBP2 may be selected based on the plurality of feature points G1FP of the first group. For example, the second matching boundary point MBP2 corresponds to an intermediate point between a first feature point located in one region of the left eyebrow and a second feature point located in one region of the right eyebrow among the feature points G1FP of the first group. can be chosen Meanwhile, the second matching boundary point MBP2 in FIG. 23 may correspond to the sixth matching boundary point MBP6 in FIG. 12 .

The user state determination module 160 determines the user's hair loss state based on the angle a formed by the second axis AX2 and a straight line connecting the first matching boundary point MBP1 and the second matching boundary point MBP2. can do. For example, the user state determination module 160 determines an angle (M-shaped angle) between a straight line connecting the first matching boundary point MBP1 and the second matching boundary point MBP2 and the second axis AX2. If it is equal to or greater than a standard, it may be determined that the user's hair loss state is in progress, and if it is less than a predetermined standard, it may be determined that the user's hair loss state is normal.

As such, according to determining whether or not the user has hair loss based on the boundary point between the user's hair and the forehead, the user's hair loss state determination module 160 can more accurately determine the user's M-shaped hair loss.

24 is a diagram for explaining a method of recalibrating an image based on an inclined angle of a user's head in a head image.

On the head image, the user's head may be photographed in a three-dimensionally tilted state. In this case, the measured lengths of main parts of the user's face included in the head image, such as the eyes, nose, leaves, and ears, may differ from the actual lengths according to the tilt angle. Accordingly, the measured length of the main part included in the head image needs to be corrected based on the tilt angle of the user's head in the head image. In this case, the measurement length may mean a length obtained by measuring the length of a main part of the user's face in the X-axis or Y-axis direction on the head image.

Referring to FIG. 24 , in a photographed head image, the user's head may be tilted at a predetermined angle with respect to the X-axis or Y-axis of the image. Specifically, the user's head may be inclined by an angle (theta_a) formed by a straight line extending the upper part of the user's left eyebrow along the X-axis and a straight line connecting the upper part of the left eyebrow and the upper part of the right eyebrow. Alternatively, the user's head may be tilted by an angle theta_b formed by a straight line connecting the tip of the nose and the tip of the chin of the user and a straight line extending the tip of the nose on the Y-axis.

In this case, the actual length of the main part of the user's face may be determined according to the ratio of the measured length and the inclination angle (theta_a or theta_b). Exemplarily, when the user's face is tilted so that the right eyebrow is higher than the left eyebrow, the actual length of the left eyebrow may be the length obtained by dividing the measured length of the left eyebrow by the cos(theta_a) value, and the actual length of the right eyebrow may be a length obtained by multiplying the measured length of the right eyebrow by the cos(theta_b) value.

As another example, when the user's face is tilted such that the left eyebrow is higher than the right eyebrow, the actual length of the left eyebrow may be the length obtained by multiplying the measured length of the left eyebrow by the cos(theta_a) value, and the actual length of the right eyebrow is It may be the length obtained by dividing the measured length of the right eyebrow by the cos(theta_a) value. As another example, the actual length from the end of the center of the user's nose to the bottom of the center of the lower lip may be the length obtained by dividing the measured length by the cos(theta_b) value. In this case, the measured length may be a length in the Y-axis direction from the tip of the center of the nose to the bottom of the center of the lower lip. As another example, the actual length from the bottom of the center of the user's lower lip to the tip of the chin may be the length obtained by dividing the measured length by the cos(theta_b) value. In this case, the measurement length may be a length in the Y-axis direction from the bottom end of the center of the lower lip to the end of the chin.

The actual length related to the main part of the user's face measured by the above method may be used to determine the user's hair loss state, or may be used to compare and determine the ratio of the main part of the user's face with a predetermined ratio.

25 is a diagram for explaining a detailed ratio related to a main part of a user's face. Referring to FIG. 25 , it is possible to measure various detailed ratios related to the main parts of the face from the user's head image, and determine the user's hair loss state based on the measured detailed ratios. Alternatively, the measured detailed ratio may be used when comparing and determining a ratio of a main part of a user's face with a predetermined ratio.

The output unit 500 may output information related to the user's hair loss state determined by the control unit 100 through the user's hair loss state determination module 160 . For example, the output unit 500 may output user hair loss state information determined based on the first calculation value and/or the second calculation value in the control unit 100 . Alternatively, the output unit 500 may output user hair loss state information determined based on at least one of a plurality of feature points (FP) and a plurality of boundary points (BP) extracted or selected by the controller 100 . Alternatively, the output unit 500 may output user hair loss state information determined by the control unit 100 based on the first upper facial area value and/or the second upper facial area value. The above-described information related to the user's hair loss state may include, for example, various information such as whether the user is experiencing hair loss, the degree of hair loss, the type of hair loss, and information about a region requiring treatment.

The output unit 500 may output information about the treatment area. The output unit 500 may output information about a treatment area determined based on information obtained through the user hair loss state determination module 160 . The treatment area may refer to an area requiring hair transplantation on the head due to hair loss occurring in the user. Alternatively, the treatment area may refer to an area where the user desires to have hair transplanted on the head. In other words, the treatment area may refer to an area determined by the user to require hair transplantation on the head.

Illustratively, the treatment area may be determined based on the area value of the first upper face part AR1 and the area value of the second upper face part AR2 described with reference to FIGS. 21 and 22 . More specifically, the treatment area may be determined based on the area AR3 corresponding to the difference between the first upper facial portion AR1 and the second upper facial portion AR2.

The output unit 500 may output the treatment area in text or image format. For example, when the output unit 500 outputs the treatment area in text format, the output unit 500 may also output information about the area value of the area to be treated, the number of hair follicles to be treated in the treatment area, and the like. As another example, when the output unit 500 outputs the treatment area in the form of an image, the output unit 500 may output a head image after hair is transplanted to the treatment area, that is, an image of the user's head after treatment.

26 and 27 are diagrams for explaining that the output unit outputs information related to the user's hair loss state based on user input information, and FIGS. 28 and 29 illustrate that the output unit outputs an image of the user's head after treatment It is a drawing for explanatory purposes. Hereinafter, with reference to FIGS. 26 to 29 , a method of outputting information related to a user's hair loss condition based on user input information by an output unit will be described.

Referring to FIGS. 26 and 27 , the output unit 500 may output information related to a hair loss condition based on user input information acquired through the user input unit 400 . The output unit 500 may output a postoperative head image based on user input information obtained through the user input unit 400 .

The hair treatment amount information acquisition module 510 of the output unit 500 may acquire hair treatment amount information through the step of acquiring hair treatment amount information from a user or a third party (S510). Here, the hair treatment amount information is information input by a user or a third party, and may include information on the number of hair follicles, the type of hair to be treated, the treatment area, treatment area, treatment method, and the like. The post-surgery image generation module 530 may generate an image of the user's post-surgery head through a post-surgery image generation step (S530).

Referring to FIG. 28 , the post-surgery image output module 550 may output an image of the user's post-surgery head in a post-surgery image output step (S550). The post-operative image output module 550 may output an image of the treatment region determined based on user input information. The treatment area may include a first treatment area and a second treatment area different from the first treatment area, and the first treatment area is determined when the hair treatment amount input by the user is the first treatment amount, The second treatment area may be an area determined when the amount of hair treatment input by the user is the second treatment amount.

Referring to (a) of FIG. 28 , when the hair treatment amount obtained by the hair treatment amount information acquisition module 510 is the first treatment amount, the treatment area may be determined as the first treatment area AR1. Referring to (b) of FIG. 28 , when the hair treatment amount obtained by the hair treatment amount information acquisition module 510 is the second treatment amount, the treatment area may be determined as the second treatment area AR2. 2 The value of the treatment amount may be greater than the value of the first treatment amount.

Referring to FIG. 29 , for example, the post-treatment head image output by the post-treatment image output module 550 is a first axis AX1 on the head image from the boundary line between the user's hair and the forehead based on the input hair treatment amount. It may be an image of an area VAR simulated so that hair is gradually transplanted in a second direction (a direction from the hair to the chin). As another example, in the post-surgery head image output by the post-surgery image output module 550, hair is gradually transplanted in the direction of the second axis AX2 on the head image from the boundary line between the user's hair and the forehead based on the input amount of hair treatment. It may be an image related to a simulated area (HAR). As the output unit 500 provides a head image after hair transplantation based on the direction of the first axis and/or the second axis, more accurate simulation information can be provided to the user.

The features, structures, effects, etc. described in the embodiments above are included in at least one embodiment of the present invention, and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. illustrated in each embodiment can be combined or modified with respect to other embodiments by those skilled in the art in the field to which the embodiments belong. Therefore, contents related to these combinations and variations should be construed as being included in the scope of the present invention.

In addition, although the embodiment has been described above, this is only an example and does not limit the present invention, and those skilled in the art to the present invention pertain to the above to the extent that does not deviate from the essential characteristics of the present embodiment. It will be appreciated that various modifications and applications not exemplified are possible. That is, each component specifically shown in the embodiment can be implemented by modifying it. And differences related to these modifications and applications should be construed as being included in the scope of the present invention as defined in the appended claims.

Claims (13)

In the hair loss condition information providing method performed by a hair loss condition information providing device or a processor of a server,
obtaining an image of the user's head;
extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point;
extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image;
selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points;
obtaining an upper face part calculation value determined based on the coordinates of one or more feature points of the first group and the coordinates of a matching boundary point corresponding thereto;
obtaining a mid-lower facial part calculation value determined based on the coordinates of one or more feature points of the first group and the coordinates of one or more feature points of the second group;
obtaining a reference value based on the calculated value of the mid-lower facial part and a predetermined ratio; and
Comprising the step of providing hair loss state information of the user by comparing the upper facial part calculated value and the reference value,
How to provide hair loss status information.
According to claim 1,
The plurality of boundary points are extracted based on probability values corresponding to the boundary lines of the user's hair and forehead,
The step of selecting the matching boundary point,
setting a matching area between a first straight line and a second straight line respectively extending in a first axial direction on the head image at a first point and a second point corresponding to the feature points of the first group; and
selecting at least one of matching candidate boundary points having a probability value equal to or greater than a predetermined threshold among a plurality of boundary points included in the matching area as the matching boundary point;
The first point and the second point are points separated from the first group feature points by a first distance and a second distance in a second axis direction orthogonal to the first axis, respectively.
How to provide hair loss status information.
According to claim 2,
The matching boundary point is a point located farthest from the feature points of the first group among the matching candidate boundary points in the first direction of the first axis,
How to provide hair loss status information.
According to claim 1,
The upper face part calculation value includes an upper face part length value of the head image set based on the feature points of the first group and the matching boundary points,
The lower middle face part calculation value includes a middle lower face part length value of the head image set based on the feature points of the first group and the feature points of the second group.
How to provide hair loss status information.
According to claim 1,
The calculated upper face value includes an upper face area value of the head image set based on the feature points of the first group and the matching boundary points;
The middle lower face part calculation value includes a middle lower face area value of the head image set based on the feature points of the first group and the feature points of the second group,
How to provide hair loss status information.
According to claim 1,
The step of selecting the matching boundary point,
selecting a plurality of feature points including at least a first feature point and a second feature point from the feature points of the first group; and
selecting a first matching boundary point and a second matching boundary point corresponding to each of the first feature point and the second feature point;
The hair loss state information providing step,
Extracting a first reference point corresponding to the first feature point and a second reference point corresponding to the second feature point from the head image - the first reference point and the second reference point are respectively selected from the first feature point and the second feature point. A point separated by a distance of a preset ratio of the calculated value of the mid-lower facial part in the first direction of the first axis on the head image -;
The upper face part calculation value is determined based on the coordinates of the first feature point, the coordinates of the second feature point, the coordinates of the first matching boundary point, and the coordinates of the second matching boundary point;
The reference value is determined based on the coordinates of the first feature point, the coordinates of the second feature point, the coordinates of the first reference point, and the coordinates of the second reference point.
How to provide hair loss status information.
According to claim 6,
The upper face part calculation value and the reference value are calculated based on the reference horizontal length and vertical length of the user's face,
The reference horizontal length is determined based on the aspect ratio of the user's face and a preset standard face length.
How to provide hair loss status information.
According to claim 1,
The providing of the user's hair loss state information includes providing information about a treatment region in which the user's treatment is required in the image,
The treatment area is determined by comparing the calculated value of the upper face with the reference value,
How to provide hair loss status information.
According to claim 8,
Providing the user's hair loss condition information,
receiving a user's input for determining a treatment area in which the user's treatment is required;
Including the step of outputting a head image after the user's treatment according to the received user's input,
How to provide hair loss status information.
According to claim 9,
The user's input includes information on the amount of hair treatment,
How to provide hair loss status information.
According to claim 1,
The plurality of boundary points are obtained using a neural network model,
The neural network model is trained to acquire a plurality of reference boundary points corresponding to the boundary line between the hair and the forehead based on a learning head image including at least some of the boundary lines between the hair and the forehead.
How to provide hair loss status information.
According to claim 11,
The neural network model includes a first part that acquires a region of interest including the hair and the forehead based on the training head image, and a second part that is learned to acquire the plurality of reference boundary points based on the region of interest. ,
How to provide hair loss status information.
obtaining an image of the user's head;
extracting a plurality of feature points from the head image, the plurality of feature points including a first group feature point and a second group feature point;
extracting a plurality of boundary points corresponding to the boundary lines of the user's hair and forehead from the head image;
selecting matching boundary points corresponding to feature points of the first group from among the plurality of boundary points;
obtaining an upper face part calculation value determined based on the coordinates of one or more feature points of the first group and the coordinates of a matching boundary point corresponding thereto;
obtaining a mid-lower facial part calculation value determined based on the coordinates of one or more feature points of the first group and the coordinates of one or more feature points of the second group;
obtaining a reference value based on the calculated value of the mid-lower facial part and a predetermined ratio; and
A program stored in a computer readable recording medium to execute a hair loss condition information providing method comprising the step of comparing the upper face part calculation value and the reference value to provide hair loss condition information of the user.

Images