KR20190043776A - Apparatus and method for photographing tongue image using depth information - Google Patents

Abstract

The present invention relates to an image providing apparatus and, more specifically, to a tongue image photographing apparatus comprising: at least one depth camera for photographing a tongue image of a user; a display unit for displaying the image; and a processor for calculating depth information of the image, determining whether a tongue shape of the user is included in a first threshold range designated in advance, and transmitting a retake signal of the image when the first threshold is not included in the first threshold range.

Description

[0001] APPARATUS AND METHOD FOR PHOTOGRAPHING TONGUE IMAGE USING DEPTH INFORMATION [0002]

The present invention relates to an apparatus and method for providing an image, and more particularly to an apparatus and method for capturing and analyzing a tongue image using a depth camera and providing the same to a user.

It is a unique diagnostic method of Oriental medicine that examines the tongue to understand the health condition and disease characteristics of the human body. Specifically, it is based on the shape of the tongue, the skin, the sex, and the movement.

If there is an abnormal thickness of white tongue on the tongue, it may be related to aphasia, throat, and fever, so it may be suspicious of the state of the tongue. In recent years, research is underway to diagnose the possibility of the disease by imaging the tongue.

Korean Patent No. 10-1265849 (published on May 20, 2013) proposes a tongue imaging device. And a re-shooting notification signal when the tongue image is not included in the critical range.

According to one embodiment, there is provided an apparatus comprising: at least one depth camera for capturing a tongue image of a user; A display unit for outputting the image; And a processor for calculating depth information of the image, determining whether the tongue shape of the user is included in a first threshold range that is designated in advance, and sending a retake signal of the image if the tongue shape is not included in the first threshold range A tongue image photographing apparatus is disclosed.

According to another embodiment, the processor calculates tongue position information of the image and determines whether the tongue position of the user is included in a second threshold range that is designated in advance. If the tongue position is not included in the second threshold range A reticle signal of the image is sent and a storage signal of the image is sent when the tongue shape is included in a first threshold range and the tongue position is included in the second threshold range.

According to another embodiment, the processor can determine the tongue shape using the depth distribution of the image in the X-axis direction and the Y-axis direction.

In addition, the image may include a tongue front image and a tongue side image, and the processor may extract a color value of the tongue image from a plurality of tongue images to provide an average value.

According to another embodiment, there is also provided a tongue imaging apparatus including a storage unit for receiving a storage signal from a processor and storing a tongue image.

In addition, the display unit may combine and display the grid image for dividing the tongue image into a plurality of regions.

According to one aspect, at least one depth camera photographs a tongue image of a user; The display unit outputting the image; Calculating a depth information of the image by the processor and determining whether the tongue shape of the user is included in a predetermined first threshold range; And transmitting a retake signal of the image if the first threshold range is not included in the first threshold range.

According to another aspect, the processor may calculate tongue position information of the image and determine whether the tongue position of the user is included in a second threshold range that is previously designated; Capturing a resampling signal of the image if it is not included in the second threshold range; And sending a storage signal of the image if the tongue shape is included in a first threshold range and the tongue position is included in the second threshold range.

The processor may further include determining the tongue shape using the depth distribution of the image in the X-axis direction and the Y-axis direction. The image may include a tongue front image and a tongue side image, and the processor may further include extracting a color value of the tongue image from a plurality of tongue images to provide an average value.

According to another aspect of the present invention, there is also provided a tongue image capturing method, further comprising the step of storing the image by receiving a storage signal of the image.

The step of outputting the image may be a tongue image capturing method of combining and displaying a grid image that divides the tongue image into a plurality of regions.

According to one embodiment, a computer-readable recording medium storing a program for performing a tongue-capturing method is also disclosed.

1 is a block diagram of a tongue imaging apparatus according to an embodiment.
2 is a flowchart of a tongue imaging method according to an embodiment.
3 is a detailed flowchart of a tongue imaging method according to an embodiment.
4A shows a tongue image actually taken according to an embodiment.
FIG. 4B shows a simplified version of a tongue image according to an embodiment.
FIG. 5A shows a state in which another tongue image is photographed according to an embodiment.
FIG. 5B illustrates a simplification of another tongue image according to an embodiment.
FIG. 6A illustrates the tongue being located at an incorrect location according to one embodiment.
FIG. 6B illustrates the tongue being located at another point that is not correct according to one embodiment.
Figure 6c shows the tongue according to one embodiment located at the correct point.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the rights is not limited or limited by these embodiments. Like reference symbols in the drawings denote like elements.

The terms used in the following description are chosen to be generic and universal in the art to which they are related, but other terms may exist depending on the development and / or change in technology, customs, preferences of the technician, and the like. Accordingly, the terminology used in the following description should not be construed as limiting the technical thought, but should be understood in the exemplary language used to describe the embodiments.

Also, in certain cases, there may be a term chosen arbitrarily by the applicant, in which case the meaning of the detailed description in the corresponding description section. Therefore, the term used in the following description should be understood based on the meaning of the term, not the name of a simple term, and the contents throughout the specification.

1 is a block diagram of a tongue imaging apparatus according to an embodiment. The tongue imaging apparatus may be composed of a depth camera, a processor, a display unit, and a storage unit.

The depth camera may be composed of at least one depth camera and may be, but not limited to, a time-of-flight (TOF) depth camera using a difference in reflection time of infrared rays.

The depth camera may provide depth information for analyzing the shape of the tongue by photographing the tongue of the user. In addition, the depth camera can photograph a frontal image and a lateral image of the tongue. The processor determines the shape of the tongue using the depth information from the image taken by the depth camera. The processor can determine or determine the shape of the tongue using the depth distribution in the X-axis direction and the Y-axis direction of the tongue image.

The processor determines the shape of the tongue and determines whether it is included in a first threshold range that is specified in advance. A resume signal is sent when the determined tongue shape is not included in the first threshold range, and a tongue image storage signal if the tongue shape is included in the first threshold range.

The display unit can display tongue images taken by the depth camera in real time. More specifically, the display unit can directly output tongue images taken by the depth camera without separately storing them. The display unit may combine and output a grid image to the tongue image. The grid image may be a grid image that divides the tongue into a plurality of portions in the tongue image. An example of the image output by the display unit can be seen in detail in FIGS.

The processor determines the position of the tongue using the grid image and determines whether or not the tongue is included in a second threshold range designated in advance. Capturing signal when the determined tongue position is not included in the second threshold range, and sends a tongue image storage signal if the tongue position is included in the second threshold range.

The storage unit may store the tongue image when the tongue shape of the user is included in the first threshold range and the tongue position is included in the second threshold range to receive the storage signal of the tongue image. Since the proper tongue shape is maintained when the tongue shape of the user is included in the first threshold range and the tongue position is located at a proper position when the tongue position is included in the second threshold range, .

In addition, the processor may extract the color values of the tongue image from a plurality of tongue images and provide an average value. So as to provide more accurate color information by extracting the color value of the tongue portion from a plurality of stored tongue images.

2 is a flowchart of a tongue imaging method according to an embodiment. The tongue image capturing method according to one embodiment includes a tongue image capturing step 210, a front side, a side, a depth image and a grid pattern output step 220, a posture correction feedback step 230, (Step < RTI ID = 0.0 > 240). ≪ / RTI > Hereinafter, each step will be described in detail.

The tongue image capturing step 210 is a step of capturing a tongue image using at least one depth camera. Illustratively, but not exclusively, two front-view depth-of-field depth cameras can be used to capture the frontal and lateral images of the user's tongue.

Next, the front, side, depth image, and grid pattern output step 220 is a step in which the display unit outputs an image captured by the depth camera. More specifically, when two depth cameras photograph the front and side images, each image can be displayed so that the user can view the images. Also, a grid image having a grid pattern may be added to the front and side tongue images and output. The plaid may be used to determine the position of the tongue. The user can directly determine the shape and position of the tongue by looking at the outputted tongue image and the grid image.

The posture correction feedback step 230 is a step of receiving correction feedback on the shape and posture of the tongue by the processor. The processor can determine the shape and posture of the tongue from the grid image including the depth information of the tongue image and the grid pattern, and determines whether the shape and the posture of the tongue are included in a predetermined threshold range.

The processor sends a resume signal when any one of the tongue shape or the posture is out of a predetermined threshold range. In addition, we can show how to correct the posture. For example, if the shape of the tongue is not straightened properly and the tip of the tongue is curled, it is possible to provide posture correction feedback such as "Please spread the tongue" when there is irregularity and "Please lower the end of the tongue" .

Or even if the position of the tongue is not correct, posture correction feedback is possible. For example, if the tongue is located behind the standard, please push the jaw forward. If it is in front of the reference, please pull the jaw backward. If the tongue is in the proper position, I can not. If it is located at the proper position, it is also possible to provide feedback such as " Please keep the current posture for 3 seconds "

Finally, when the tongue shape and position are satisfied, the storage step 240 stores the tongue image at that point by making the tongue suitable for the photographing through the posture correction feedback 230 step.

Specifically, when the tongue shape does not deviate from the first threshold range and the tongue position does not deviate from the second threshold range, the storage signal can be received from the processor to store the tongue image at that time point. A tongue image having a consistent position and shape can be obtained through the steps 210 to 240. [

3 is a detailed flowchart of a tongue imaging method according to an embodiment. The tongue imaging method according to an exemplary embodiment may include a tongue imaging step 310, a depth information calculation step 320, a tongue area separation step 330, a tongue depth normalization and segmentation step 340, Step 350 and step 360 of determining if the tongue location and shape meet the criteria. A step 380 is performed or a re-photographing 370 is performed according to the satisfaction of the criterion.

The tongue image capturing step 310 is a step of capturing a tongue image of a user using at least one depth camera. More specifically, at least one depth camera can photograph the front and side images of the tongue of the user, and the shooting position is not limited.

Next, the depth information calculation step 320 calculates depth information from the tongue image photographed by the depth camera. Illustratively, the depth camera may be a time-of-flight (TOF) depth camera, and depth information may be obtained by calculating a TOF using an image captured by the depth camera.

The tongue region separation step 330 is a step of extracting a tongue region from an image to be photographed. More specifically, the tongue image to be photographed may include a part of a face such as a tongue, a chin, and a nose. Therefore, except for these parts, only the tongue area required for seulching is extracted. Unlike the segmentation step described later, this means extracting the entire tongue region from the face.

The tongue depth normalization and segmentation step 340 is a step of normalizing the tongue depth calculated in the depth information calculation step 320 and dividing the tongue area extracted in the tongue area separation step 330. In the depth information calculation step, normalization is performed to determine the position and shape of the tongue because only tongue depth information is calculated. Subsequently, the zone is divided. And divides the area into a grid image in the separated tongue area. The lattice image is merged into the tongue image to divide the area so that the user can easily judge the position and shape of the tongue.

The tongue position and shape determination step 350 is a step of determining the position and shape of the tongue during photographing using the depth information and the grid image. The shape of the tongue is determined using depth information. Specifically, it is possible to determine which portion of the tongue is deep and which portion is shallow by using the depth information, and the shape of the tongue can be determined accordingly. The position of the tongue can be determined based on the grid image. The position of the current tongue centered on the reference position is determined by overlapping the grid image at the center of the captured image.

Finally, it is determined whether the position and shape of the tongue meet the criteria. When the position and shape of the tongue satisfy a predetermined criterion (suitable for photographing), the storage unit receives the storage signal and stores the tongue image. On the other hand, if any one of the position or the shape of the tongue does not satisfy the predetermined criterion (unsuitable for photographing), it is possible to photograph again and output a feedback message.

In the case of satisfying the predetermined criteria, the shape of the tongue is flattened and the shape of the end portion or both sides is not curled. The position of the tongue is a position where the entire tongue region is included in the grid image have.

A retake is performed and a feedback message can be output when the predetermined criteria are not satisfied. Illustratively, the feedback message may include, for example, "stretch your tongue", "lower the tip of the tongue down", "push the jaw forward", "pull the jaw backward", " But it is not limited thereto.

After the re-photographing, the tongue image photographed again is used again from step 320. The position and shape of the tongue are re-performed until a predetermined criterion is satisfied, and when the tongue is satisfied, the image is stored and terminated.

4A shows a tongue image actually taken according to an embodiment. Figure 4A illustrates an image of a tongue curved inward illustratively. The front, side, and depth images are shown in order from the left. The grid images overlap each other. The position of the tongue photographed did not deviate greatly from the lattice range, but when the depth image was viewed, both sides of the tongue protruded and the inside depressed. Therefore, the shape of the tongue should be flat.

FIG. 4B shows a simplified version of a tongue image according to an embodiment. FIG. 6 shows an X-axis direction distribution and a Y-axis direction distribution in the tongue image bent inward. When the distribution in the X-axis direction is viewed, it can be seen that the irregularities are not constant. The Y-axis distribution shows a downward slope. Therefore, considering the proposed distribution of the X-axis direction and the Y-axis direction, it can be judged as the shape of the tongue curved inward. The shape of the tongue bent in the above does not satisfy a predetermined criterion and can be re-photographed. When re-shooting, a message such as "Please stretch your tongue" can be output.

FIG. 5A shows a state in which another tongue image is photographed according to an embodiment. The end portion shows the tongue image protruding forward. As in FIG. 4A, the front image, the side image, and the depth image are sequentially arranged from the left, and the grid images overlap each other.

The position of the tongue to be photographed may be included in a range previously specified within the lattice range. However, if we look at the depth image, we can see that the tongue area is not constant and the tip is red.

FIG. 5B illustrates a simplification of another tongue image according to an embodiment. Specifically, the distribution in the X-axis direction and the distribution in the Y-axis direction are relatively flat in the X-axis direction, but the tip of the tongue is excessively protruded in the Y-axis direction distribution. Therefore, the shape of the tongue can be judged to be a shape in which the tip portion protrudes forward by using the X-axis and Y-axis direction distributions. The shape of the tongue protruding from the end portion does not satisfy the predetermined criteria, so that the tongue shape can be re-photographed. At the time of re-photographing, a message such as 'Please lower the tip of the tongue down' can be outputted.

FIG. 6A illustrates the tongue being located at an incorrect location according to one embodiment. When viewed from the front, it is appropriately located at the center of the grid image, but when viewed from the side image, the tongue touches the right end of the grid image. In addition, even when looking at the front depth image, the whole area of the tongue is displayed in blue and is located too far away. In this case, the tongue is flat and well-formed, but the position of the tongue is not appropriate. Therefore, it is possible to perform re-photographing without satisfying the position reference of the tongue specified in advance, and to output a message such as 'Please push forward the jaw' as a feedback message at the time of re-photographing.

FIG. 6B illustrates the tongue being located at another point that is not correct according to one embodiment. And the tongue is positioned in front of the reference in accordance with another embodiment. In the side image, it can be seen that it is out of the left line of the grid image, and in the depth image, the entire tongue area is displayed in red and is located too close. Therefore, in this case, the tongue is flat and spread out, but does not satisfy the predetermined positional reference of the tongue, and retakes. A message such as 'Please pull the jaw backward' may be outputted as a feedback message at the time of re-shooting.

Finally, FIG. 6C shows the tongue according to one embodiment being located at the correct point. It can be seen that the position of the tongue is located inside the grid image in both the front and side images, and that the entire tongue area is displayed in green in the depth image, and is located at an appropriate distance. A storage signal is transmitted to the tongue image at this time, and the storage unit can receive and store the storage signal.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the drawings, various modifications and variations may be made by those skilled in the art. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (15)

At least one depth camera for capturing a tongue image of a user;
A display unit for outputting the image; And
A processor for calculating depth information of the image and determining whether the tongue shape of the user is included in a first threshold range that is designated in advance and sending a retake signal of the image if the tongue shape is not included in the first threshold range,
And a tongue image capturing device.
The method according to claim 1,
The processor comprising:
Determining whether the tongue position of the user is included in a second threshold range that is designated in advance, sending a retake signal of the image if the tongue position is not included in the second threshold range,
And transmits a storage signal of the image when the tongue shape is included in a first threshold range and the tongue position is included in the second threshold range.
3. The method of claim 2,
The processor comprising:
And determines the shape of the tongue using the depth distribution in the X-axis direction and the Y-axis direction of the image.
The method of claim 3,
Wherein the image includes:
A tongue front image, and a tongue side image.
5. The method of claim 4,
The processor comprising:
And extracting a color value of the tongue image from the plurality of tongue images to provide an average value.
6. The method of claim 5,
A storage unit for receiving a storage signal from the processor and storing a tongue image,
Wherein the tongue image capturing device further comprises:
The method according to claim 6,
And a display unit for combining and displaying the grid image for dividing the tongue image into a plurality of regions
Wherein the tongue image capturing device further comprises:
At least one depth camera capturing a tongue image of a user;
The display unit outputting the image;
Calculating a depth information of the image by the processor and determining whether the tongue shape of the user is included in a predetermined first threshold range; And
Capturing a resampling signal of the image if it is not included in the first threshold range
/ RTI >
9. The method of claim 8,
The processor comprising:
Calculating tongue position information of the image and determining whether the tongue position of the user is included in a second threshold range designated in advance;
Capturing a resampling signal of the image if it is not included in the second threshold range; And
Transmitting a storage signal of the image when the tongue shape is included in a first threshold range and the tongue position is included in the second threshold range;
Further comprising the steps of:
10. The method of claim 9,
Wherein the processor determines the shape of the tongue using the depth distribution of the image in the X-axis direction and the Y-axis direction
Further comprising the steps of:
11. The method of claim 10,
Wherein the image includes:
A tongue frontal image and a tongue lateral image.
12. The method of claim 11,
Wherein the processor extracts a color value of the tongue image from a plurality of tongue images and provides an average value
Further comprising the steps of:
13. The method of claim 12,
Wherein the outputting of the image comprises combining grid images for dividing the tongue image into a plurality of regions and displaying the grid image.
14. The method of claim 13,
Wherein the storage unit receives the storage signal of the image and stores the image
Further comprising the steps of:
15. A method according to any one of claims 8 to 14
A computer-readable recording medium storing a program for performing a tongue image capturing method.

Images