WO2015156039A1

WO2015156039A1 - Organ imaging apparatus

Info

Publication number: WO2015156039A1
Application number: PCT/JP2015/054706
Authority: WO
Inventors: 松田　伸也; 楠田　将之
Original assignee: コニカミノルタ株式会社
Priority date: 2014-04-08
Filing date: 2015-02-20
Publication date: 2015-10-15

Abstract

An organ imaging apparatus (1) provided with an illumination unit (2), an imaging unit (3), and a calculating unit (16). The illumination unit (2) illuminates an organ of a living body using illumination light comprising near infrared light. The imaging unit (3) acquires images of the organ by receiving illumination light reflected at the surface of the organ. The calculating unit (16) classifies the degrees of organ diagnosis categories on the basis of the image data of at least the near infrared images obtained using received near infrared light contained in the illumination light from among the images acquired by the imaging unit (3).

Description

Organ imaging device

The present invention relates to an organ image photographing apparatus for photographing an organ of a living body and detecting a diagnosis item of the organ from the photographed image.

In oriental medicine, a diagnostic method (tongue diagnosis) is known for diagnosing a health condition or medical condition by observing the state of the tongue. In the tongue examination, the physical condition and health level are diagnosed based on the color and shape of the tongue and moss, and the diagnosis items include the color of the tongue, the thickness of the tongue, and the crack on the surface of the tongue.

In a healthy state, the tongue is light red, but if there is a problem with the respiratory or circulatory system, the tongue will turn blue, and if there is fever or dehydration, the tongue will turn red. Further, when the metabolism of water is poor, the so-called swelled state is formed, and the tongue swells and becomes thick. Conversely, when the blood flow is insufficient or the water is insufficient, the tongue becomes thin and thin. Furthermore, when the immunity decreases due to lack of nutrition, poor blood flow, stress, etc., the regenerative power of the tongue cells decreases and the surface of the tongue cracks.

At present, these diagnoses are carried out by specialist doctors, but because they rely on experience and intuition, there are individual differences and poor objectivity.

Therefore, a system has been proposed in which a subject is photographed using a digital camera, and the features of the photographed image are digitized and recorded and diagnosed. For example, in Patent Document 1, a tongue is photographed with a camera to extract a region of interest such as a tongue apex, a tongue, a tongue base, a tongue base, and the tongue quality (tongue color) and tongue tongue (moss color) of the region of interest. This makes it easy to diagnose the health status of individuals. In Patent Document 2, an index for diagnosing the state of blood or blood vessels is obtained by photographing the tongue with a camera and detecting the color and gloss of the tongue separately.

In Patent Document 3, the ratio of the red (R) image data and the green (G) image data in each pixel of the captured image of the tongue is calculated, and the ratio is compared with a threshold value, whereby the tongue The color and the color of moss are judged. Furthermore, the number of connections in the vertical and horizontal directions of pixels on the base of the tongue (parts other than the nipple) is counted, and the number of connections in one direction (for example, the vertical direction) is equal to or greater than a predetermined value and the connection in the other direction (for example, the horizontal direction). A portion where the number is equal to or less than a predetermined value is determined as a crack in the tongue.

Japanese Patent No. 3854966 (see claim 1, paragraphs [0004], [0009], [0022] to [0024], etc.) JP 2011-239926 A (refer to claim 1, paragraph [0028] etc.) JP 2005-137756 A (refer to claim 3, paragraphs [0059] to [0063], [0079] to [0081], FIG. 4 to FIG. 6, FIG. 8, etc.)

However, in the systems of Patent Documents 1 to 3, depending on the state of the tongue surface, there are cases in which the diagnostic items of the tongue cannot be accurately detected.

That is, in the systems disclosed in Patent Documents 1 to 3, a captured image (visible image) of the tongue is acquired by irradiating the tongue with visible light. In the visible image, the part where the moss exists on the surface of the tongue is white or yellow, and the color of the underlying tongue cannot be detected, so the image data of the part where the moss does not exist on the surface is selected from the visible image Thus, the color of the tongue is detected. For this reason, when the entire surface of the tongue is covered with moss, it is impossible to select image data of a portion where the moss is not present on the surface, and it is impossible to detect the color of the tongue.

In the visible image, if the distribution of moss on the surface of the tongue is uneven, it is impossible to distinguish between the moss-free portion and the crack, so that the tongue crack cannot be accurately detected.

Also, since the cross-sectional shape of the tongue is different between the case where the tongue is thick and the case where the tongue is thin, it is considered that the thickness of the tongue can be detected by using a method such as a light cutting method. The light cutting method is a method of detecting the shape of the subject by irradiating the subject with linear light (eg, visible light) and detecting the shape of the reflected light. However, even if this method is used, if the moss distribution on the surface of the tongue is uneven, the reflected light of the irradiated light will be disturbed in the portion where the moss is present. It becomes impossible to accurately detect the thickness of the.

The present invention has been made to solve the above problems, and an object of the present invention is to provide an organ imaging apparatus capable of accurately detecting a diagnosis item of an organ regardless of the surface state of the organ of a living body. It is to provide.

An organ imaging apparatus according to one aspect of the present invention receives an illumination unit that illuminates a living organ with illumination light including near-infrared light, and the illumination light reflected from the surface of the organ. An imaging unit that acquires an image of the organ, and at least image data of a near-infrared image obtained by receiving the near-infrared light included in the illumination light among the images acquired by the imaging unit. And an arithmetic unit for classifying the degree of the diagnosis item of the organ.

According to the above configuration, the calculation unit classifies the degree of the diagnosis item of the organ based on the image data of the near-infrared image, thereby accurately determining the diagnosis item of the organ regardless of the state of the organ surface that causes noise. Can be detected.

It is a perspective view which shows the external appearance of the organ imaging device which concerns on one Embodiment of this invention. It is a block diagram which shows the schematic structure of the said organ image imaging device. It is a graph which shows the light emission characteristic of the xenon lamp used for the illumination part of the said organ imaging device. It is explanatory drawing which shows the positional relationship of the said illumination part with respect to imaging | photography object, and an imaging part. It is explanatory drawing which shows typically the structure of the color filter which the said imaging part has. It is explanatory drawing which shows the other structure of the said imaging part. It is explanatory drawing which shows typically the structure of the color filter which the imaging part of FIG. 6 has. It is explanatory drawing which shows the visible image and near-infrared image of a tongue obtained by image | photographing a tongue under illumination by the said illumination part. It is a graph which shows the spectral distribution of the color of a tongue, and the color of moss. It is a graph which shows the absorption characteristic of hemoglobin. It is explanatory drawing which shows together the visible image of a tongue, the cross-sectional shape of a tongue, and the distribution of RGB image data in one direction when a tongue is thin and moss is thin. It is explanatory drawing which shows together the visible image of a tongue, the cross-sectional shape of a tongue, and the distribution of RGB image data in one direction when a tongue is thick and moss is thick. 9 is a graph showing a distribution of RGB image data in one direction of the visible image in FIG. 8. It is a graph which shows distribution of the image data in one direction of the near-infrared image of FIG. It is explanatory drawing which shows the cross-sectional shape of a tongue, and the distribution of the image data in one direction of the near-infrared image of a tongue. FIG. 16 is an explanatory diagram illustrating the distribution of the image data in FIG. 15 together with a region set when detecting the thickness of the tongue. It is a graph which shows the frequency distribution of the image data of the one part area | region of the visible image shown in FIG. It is a graph which shows the frequency distribution of the image data of the one part area | region of the near-infrared image shown in FIG. It is a flowchart which shows the flow of operation | movement in the said organ imaging device.

An embodiment of the present invention will be described below with reference to the drawings. In this specification, when the numerical range is expressed as A to B, the numerical value range includes the values of the lower limit A and the upper limit B.

[Overall configuration of organ imaging system]
FIG. 1 is a perspective view showing an external appearance of an organ image photographing apparatus 1 of the present embodiment, and FIG. 2 is a block diagram showing a schematic configuration of the organ image photographing apparatus 1. The organ image capturing apparatus 1 captures an organ of a living body and detects information (diagnostic items) necessary for health diagnosis. Here, as an example, a case where a living organ is a tongue will be described. Therefore, as the diagnostic items, for example, the color of the tongue, the thickness of the tongue, and the crack on the surface of the tongue can be considered.

The organ image photographing apparatus 1 includes an illumination unit 2, an imaging unit 3, a display unit 4, an operation unit 5, a communication unit 6, and an audio output unit 7. The illumination unit 2 is provided in the housing 21, and the configuration other than the illumination unit 2 (for example, the imaging unit 3, the display unit 4, the operation unit 5, the communication unit 6, and the audio output unit 7) is provided in the housing 22. ing. Although the housing | casing 21 and the housing | casing 22 are connected so that relative rotation is possible, rotation is not necessarily required and one side may be completely fixed to the other. In addition, said illumination part 2 grade | etc., May be provided in the single housing | casing. Moreover, the organ image photographing device 1 may be composed of a multifunctional portable information terminal.

The illuminator 2 illuminates a living organ (herein, the tongue) that is an object to be imaged with illumination light including near-infrared light, and includes an illuminator that illuminates the object to be imaged from above. As the light source of the illuminating unit 2, in order to improve color reproducibility and to shoot by irradiating near infrared light, a light source that emits a daylight color such as a xenon lamp is used.

FIG. 3 shows the light emission characteristics of the xenon lamp used as the light source of the illumination unit 2. Xenon lamps are often used in camera flashes, and are almost flat from the visible wavelength range (400 nm to 700 nm) to the near infrared wavelength range (700 nm to 1000 nm). Emission characteristics close to the distribution.

Although the brightness of the light source varies depending on the sensitivity of the imaging unit 3 and the distance to the shooting target, as an example, it is possible to consider the brightness at which the illuminance of the shooting target is 1000 to 10000 lx. The illumination unit 2 includes a lighting circuit and a dimming circuit in addition to the above light source, and lighting / extinguishing and dimming are controlled by a command from the illumination control unit 11.

The imaging unit 3 acquires an image of the organ by receiving illumination light from the illumination unit 2 reflected from the surface of the organ. FIG. 4 is an explanatory diagram showing a positional relationship between the illumination unit 2 and the imaging unit 3 with respect to the imaging target. The imaging unit 3 is disposed so as to face the tongue that is the subject of imaging.

The imaging unit 3 includes an imaging lens 31, an infrared reflecting mirror 32, a visible sensor 33, and an infrared sensor 34. Illumination light (reflected light) emitted from the illumination unit 2 and reflected by the tongue passes through the imaging lens 31 and is separated into visible light and infrared light (including near infrared light) by the infrared reflection mirror 32. Is done. The visible light separated by the infrared reflecting mirror 32 is guided to the visible sensor 33 and forms an image there. The visible sensor 33 has a color filter 33a shown in FIG. 5 on the light incident side of the light receiving surface, and thereby, for each pixel of the sensor, red (R), green ( G) and blue (B) light can be received. As a result, the visible sensor 33 obtains a visible image composed of each color of RGB. On the other hand, the infrared light separated by the infrared reflecting mirror 32 is guided to the infrared sensor 34 and forms an image there. The infrared sensor 34 acquires a near-infrared image by receiving near-infrared (IR) light.

The aperture (lens brightness), shutter speed, and focal length of the imaging lens 31 are set so that the entire range to be photographed is in focus. As an example, F number: 16, shutter speed: 1/120 seconds, focal length: 20 mm.

The area sensor (the visible sensor 33 and the infrared sensor 34) is composed of an image sensor such as a CCD (Charge-Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), for example, and has sufficient color and shape to be photographed. Sensitivity, resolution, etc. are set so that they can be detected easily. As an example, sensitivity: 60 db, resolution: 10 million pixels.

Imaging by the imaging unit 3 is controlled by the imaging control unit 12. In addition to the imaging lens 31 and the area sensor, the imaging unit 3 includes a focus mechanism (not shown), a diaphragm mechanism, a drive circuit, an A / D conversion circuit, and the like, and in response to a command from the imaging control unit 12. Focus, aperture control, A / D conversion, and the like are controlled. In the imaging unit 3, for example, data of 0 to 255 in 8 bits is acquired for each of R, G, B, and IR as captured image data.

The illumination unit 2 described above is arranged so as to illuminate the imaging target at an angle A of, for example, 0 ° to 45 ° with respect to the imaging optical axis X of the imaging unit 3 passing through the imaging target. The imaging optical axis X refers to the optical axis of the imaging lens 31. When the angle A at the time of illumination is large, the measurement accuracy of the unevenness is improved, but the range in which the tongue can be photographed is reduced by the shadow of the upper lip. On the other hand, when the angle A is small, the photographing range is enlarged, but the measurement accuracy is lowered, and the color jump due to regular reflection is increased. Considering the above, the preferable range of the angle A at the time of illumination is 15 ° to 30 °.

FIG. 6 is an explanatory diagram showing another configuration of the imaging unit 3. The imaging unit 3 may include an imaging lens 31, a color filter 35, and an imaging element 36. The image sensor 36 is composed of an image sensor such as a CCD or CMOS, and has a Si photodiode having detection sensitivity from visible light to near infrared light. The color filter 35 is disposed on the light incident side of the image sensor 36, and transmits visible light and near infrared light to the image sensor 36.

FIG. 7 schematically shows the configuration of the color filter 35. The color filter 33a shown in FIG. 5 includes a G filter that transmits green light at a ratio of 2 pixels to 4 pixels, and a B filter that transmits blue light and a red light transmitted to the remaining pixels. The R filter is arranged at a ratio of 1: 1. However, the color filter 35 in FIG. 7 uses one of the G filters arranged at a ratio of 2 pixels per 4 pixels as near-infrared light. Is replaced with an IR filter that transmits light. By using such a color filter 35, the detection of visible light and the detection of near-infrared light can be shared by one image sensor 36. That is, by decimating the G filter of the color filter 33a, both visible light and near infrared light can be detected by one sensor. Further, in this configuration, since the infrared reflection mirror 32 and the infrared sensor 34 in FIG. 4 can be omitted, it is possible to realize downsizing and cost reduction.

Referring back to FIGS. 1 and 2, the description will be continued. The display unit 4 includes a liquid crystal panel (not shown), a backlight, a lighting circuit, and a control circuit. The display unit 4 calculates and outputs an image acquired by photographing with the imaging unit 3 and a calculation unit 16 described later. Displayed information. Display of various types of information on the display unit 4 is controlled by the display control unit 13.

The operation unit 5 is an input unit for instructing imaging by the imaging unit 3, and includes an OK button (imaging execution button) 5a and a CANCEL button 5b. In the present embodiment, the display unit 4 and the operation unit 5 are configured by a common touch panel display device 41, and the display area of the display unit 4 and the display area of the operation unit 5 in the touch panel display device 41 are separated. The display of the operation unit 5 on the touch panel display device 41 is controlled by the operation control unit 14. The operation unit 5 may be configured by an input unit other than the touch panel display device 41 (the operation unit 5 may be provided at a position outside the display area of the touch panel display device 41).

The communication unit 6 transmits the image data acquired by the imaging unit 3 and the information calculated and output by the calculation unit 16 described below to the outside via a communication line (including wired and wireless). It is an interface for receiving information from the outside. Transmission / reception of information in the communication unit 6 is controlled by the communication control unit 18.

The audio output unit 7 outputs various types of information as audio, and is composed of, for example, a speaker. The information output by voice includes the result calculated by the calculation unit 16. The sound output in the sound output unit 7 is controlled by the sound output control unit 19.

In addition, the organ imaging apparatus 1 further includes an illumination control unit 11, an imaging control unit 12, a display control unit 13, an operation control unit 14, an image processing unit 15, a calculation unit 16, a storage unit 17, a communication control unit 18, and a voice. An output control unit 19 and an overall control unit 20 that controls these units are provided. As described above, the illumination control unit 11, the imaging control unit 12, the display control unit 13, the operation control unit 14, the communication control unit 18, and the audio output control unit 19 are the illumination unit 2, the imaging unit 3, the display unit 4, and the operation. The unit 5, the communication unit 6, and the audio output unit 7 are controlled. The overall control unit 20 is composed of, for example, a CPU (Central Processing Unit). The illumination control unit 11, the imaging control unit 12, the display control unit 13, the operation control unit 14, the image processing unit 15, the calculation unit 16, the communication control unit 18, the audio output control unit 19, and the overall control unit 20 are: It may be configured integrally (for example, with one CPU).

The image processing unit 15 has a function of performing various types of image processing, such as extracting an outline of an organ from an image acquired by the imaging unit 3. The outline of the organ can be extracted by extracting the luminance edge of the captured image (the portion where the brightness has changed abruptly in the image) using a filter. The edge extraction filter is a filter that weights pixels in the vicinity of the target pixel when performing first-order differentiation (when obtaining a difference in image data between adjacent pixels).

The storage unit 17 stores image data acquired by the imaging unit 3, data acquired by the image processing unit 15, data calculated by the calculation unit 16, information received from the outside, and the like. A memory for storing a program for operating the unit.

The calculation unit 16 determines the degree of the diagnostic item of the organ based on at least the image data of the near-infrared image obtained by receiving the near-infrared light included in the illumination light among the images acquired by the imaging unit 3. Classify. The above classification may be performed by quantifying the degree of the diagnosis item, or may be performed by an output other than the numerical value. Hereinafter, a specific example for classifying the degree of diagnosis items will be described.

[Detection of tongue color]
FIG. 8 shows a visible image and a near-infrared image of the tongue acquired by the imaging unit 3 when the imaging unit 3 images the tongue under illumination by the illumination unit 2. In FIG. 8, the moss region on the tongue surface is indicated by hatching for convenience. In the visible image, it is confirmed that there is moss on almost the entire surface of the tongue, and the moss distribution is uneven. On the other hand, in the near-infrared image, the difference between the color of the tongue and the color of the moss disappears, and the moss cannot be seen.

FIG. 9 shows the spectral distribution of tongue color and moss color. In the B wavelength range (400 to 500 nm) and the G wavelength range (500 to 600 nm), there is a large difference in reflectance between the tongue color and moss color, but in the wavelength range exceeding 650 nm, the tongue color and moss There is almost no difference in reflectance between the colors of and. Accordingly, when the near-infrared light exceeding the wavelength of 650 nm is used to illuminate the tongue, and the tongue is photographed to obtain a near-infrared image, the tongue and the moss have the same output (same color) in the near-infrared image. The difference between the color of moss and the color of moss disappears. By using this feature in the near-infrared image, regardless of the surface condition of the tongue (whether the whole surface of the tongue has moss or uneven moss distribution), and even if the moss is thick The tongue color can be accurately detected. In addition, since the moss unevenness does not appear in the near-infrared image, the thickness and crack of the tongue, which will be described later, can be accurately detected by using light having any wavelength of 650 to 1000 nm.

Here, the color of the blood reflects the color of the blood. The color of blood changes mainly depending on the degree of oxidation of hemoglobin. FIG. 10 shows the absorption characteristics of oxygenated hemoglobin (HbO ₂ ) and reduced hemoglobin (Hb). From the figure, it can be seen that in the wavelength region of 600 to 800 nm, reduced hemoglobin has a larger absorption coefficient than oxygenated hemoglobin. That is, reduced hemoglobin has a higher degree of light absorption in the wavelength region of 600 to 800 nm than oxyhemoglobin. For this reason, blood with a high amount of deoxyhemoglobin looks relatively blue because of its large red absorption. Therefore, it is possible to detect the color of the tongue reflecting the color of blood by determining whether the reduced hemoglobin is high or low.

Note that FIG. 10 shows that the difference between the absorption coefficient of reduced hemoglobin and the absorption coefficient of oxyhemoglobin is maximum when the wavelength of light is 660 nm and is equal to 805 nm.

In detecting the color of the tongue, conventionally, the color of the tongue was detected in the region of the left or right end of the tongue without the moss or the lower part (lingual apex), but in this embodiment, it is not affected by the moss, The color may be detected in any region of the tongue. Here, the central region C of the near-infrared image of the tongue shown in FIG. 8 (the region in the tongue between the tip of the tongue and the base of the tongue) is set as a diagnosis target region, and the tongue C based on the image data of this region C. The color of is detected as follows.

The calculation unit 16 obtains an average value Bm of the B image data and an average value IRm of the IR image data in the region C of the near-infrared image acquired by the imaging unit 3, and uses these ratios IRm / Bm as a tongue. It is used as an index when detecting the color. When the ratio IRm / Bm is large, the color of the tongue is red (the degree of oxidation of blood is large) because blue is relatively small. On the other hand, when the ratio IRm / Bm is small, the color of the tongue is blue (the degree of oxidation of blood is small) because blue is relatively large.

The calculation unit 16 classifies the tongue color into a plurality of ranks according to the value of the ratio IRm / Bm. For example, the value of IRm / Bm is the rank "1" less than L _1, the value of IRm / Bm is the rank "2" less than L ₁ or L _2, the value of IRm / Bm is less than L ₂ or L ₃ The rank is set to “3”, and the calculation unit 16 classifies the tongue color into any one of “1” to “3” according to the value of IRm / Bm. In addition to the numerical values “1” to “3” shown above, this classification indicates the degree of tongue color such as “dark red”, “normal red”, “light red”, etc. (other than numerical values) ). Thereby, even if the whole tongue surface is covered with moss or the moss distribution is uneven, the color of the tongue (including the degree of redness) can be accurately detected.

Since the difference in absorption coefficient between oxyhemoglobin and reduced hemoglobin is large for light with a wavelength of 600 to 800 nm, illumination including light with a wavelength of 600 to 800 nm is used when detecting the color of the tongue (degree of oxidation of hemoglobin). It is desirable to use light.

The index used when detecting the color of the tongue is not limited to the ratio IRm / Bm, but may be, for example, the ratio IRm / (IRm + B).

[Detection of tongue thickness]
FIGS. 11 and 12 show the visible image of the tongue, the cross-sectional shape of the tongue along the line AA ′, and the distribution of RGB image data of each pixel aligned in the AA ′ direction in the visible image. FIG. 11 shows a case where the tongue is thin and the moss is thin, and FIG. 12 shows a case where the tongue is thick and the moss is thick. The AA ′ direction refers to a direction that passes through the tongue between the tongue tip and the tongue base of the tongue in the visible image and is perpendicular to the direction connecting the tongue tip and the tongue base, and corresponds to the horizontal direction.

When the tongue is thin, the surface of the tongue has a valley shape with the central part recessed downward. For this reason, when illumination light reflected from the surface of the tongue is received by the imaging unit and a distribution of image data in the direction AA ′ is created, the above distribution has a shape recessed downward in both RGB (FIG. 11). reference).

On the contrary, when the tongue is thick, the surface of the tongue bulges in the center and becomes a mountain (convex upward). Therefore, when illumination light reflected from the surface of the tongue is received by the imaging unit and a distribution of image data in the direction AA ′ is created, the distribution has a convex shape with a high central portion in both RGB (FIG. 12).

Further, as shown in FIG. 11, when the moss is thin, the distribution of RGB image data is smooth, and therefore the shape (thickness) of the tongue can be accurately detected based on this distribution. However, as shown in FIG. 12, when the moss is thick, the RGB image data distribution includes noise (due to the reflected light from the moss). Become.

Therefore, in this embodiment, the thickness of the tongue is detected as follows. FIGS. 13 and 14 show the distribution of image data of each pixel arranged in the A-A ′ direction of the visible image and the near-infrared image shown in FIG. 8, respectively. The AA ′ direction of the near-infrared image refers to a direction perpendicular to the direction connecting the tongue apex and the tongue base, passing through the tongue between the tongue apex and the tongue base of the near-infrared image, and in the horizontal direction. It corresponds.

In the distribution of the image data of the near-infrared image, it can be seen that noise due to moss is reduced and the distribution is smoother than the distribution of the image data of the visible image. The calculation unit 16 detects the thickness of the tongue as follows using the distribution of the image data of the near-infrared image.

FIG. 15 shows a sectional shape of the tongue and a distribution of image data of each pixel arranged in the A-A ′ direction of the near-infrared image of the tongue. FIG. 16 shows the distribution of the image data in FIG. 15 together with the region S set when the thickness is detected. This region S is a region closer to the end than the central portion in the AA ′ direction. For example, when the width of the tongue determined from the contour line of the tongue is W (mm), the dimensional relationship shown in FIG. (Region including a region of width W / 4 at a distance of W / 8 from the end of the tongue) can be considered. The contour line of the tongue is obtained by the image processing unit 15 described above.

Depending on the movement of the tongue, the cross-sectional shape of the tongue changes variously. However, when the tongue is thin, in the region S, the distribution shape is horizontal, or the horizontal to concave shape, no matter how the tongue moves. To change. For this reason, when the shape of the distribution in the region S is approximated by a quadratic polynomial, the quadratic coefficient of the approximate polynomial is 0 or a positive value. On the other hand, if the tongue is thick, no matter how the tongue moves, the distribution shape in the region S becomes a convex shape, so the quadratic coefficient of the approximate polynomial of the distribution in the region S is a negative value. . Note that the thicker the tongue, the greater the negative value of the coefficient. Therefore, the thickness of the tongue can be detected by looking at the quadratic coefficient of the approximate polynomial in the region S of the distribution of the image data of the near-infrared image.

For example, if the quadratic coefficient of the approximate polynomial is less than M ₁ , the rank is “1”, M ₁ or more and less than M ₂ is rank “2”, and M ₂ or more and less than M ₃ is rank “3”. The calculation unit 16 classifies the thickness of the tongue into any one of “1” to “3” according to the second-order coefficient of the approximate polynomial in the image data distribution region S. In addition to the numerical values “1” to “3” shown above, this classification indicates the level of tongue thickness (other than numerical values) such as “thin”, “normal”, and “thick”. May be. Thereby, the thickness of the tongue (including the thickness) can be accurately detected regardless of the presence or absence of moss on the tongue surface and the unevenness of the moss.

[Detection of tongue surface cracks]
17 and 18 show the frequency distribution of the image data in the region C of the visible image and the near-infrared image shown in FIG. Since many cracks occur near the center of the surface of the tongue, in the present embodiment, a region having a length of ¼ of the length in the vertical direction is defined as a region C in the tongue near the vertical center of the tongue. This is set as a crack detection area (diagnosis area). Note that the frequency distribution in the region C of the visible image in FIG. 17 indicates the frequency distribution for the B image data of RGB.

If there is a crack on the surface of the tongue, the base of the tongue appears more than when there is no crack, so the range of values that can be taken by the image data of the pixels that make up the base is expanded for both RGB. For this reason, when the frequency distribution of the image data of the captured image is created, the width of the frequency distribution is widened. In addition, as described above, the crack on the tongue surface is caused by a decrease in immunity due to lack of nutrition, poor blood flow, stress, or the like.

On the other hand, moss is a keratinized papillary tissue of the lingual mucosa, with exfoliated cells, mucus, food, and bacteria attached to the tongue surface. Since the base of the tongue appears in the portion without moss, the range of values that can be taken by the image data of the pixels constituting the base is also expanded for both RGB. For this reason, in the frequency distribution of the visible image in FIG. 17, the standard deviation (or variance) increases as a result of including both the background image data of the portion without moss and the background image data of the crack portion. Incidentally, the standard deviation of the frequency distribution of FIG. 17 is 26.78 (the variance is the square of the standard deviation).

On the other hand, in the near-infrared image of FIG. 8, the influence of moss is eliminated, so that the standard deviation (or variance) is smaller than the frequency distribution of the visible image as in the frequency distribution of FIG. Incidentally, the standard deviation of the frequency distribution in FIG. 18 is 13.18. Therefore, by creating a frequency distribution of the image data of the near-infrared image and looking at its standard deviation (or variance), it is possible to accurately crack the tongue regardless of the surface condition of the tongue (uneven distribution of moss). Can be detected.

More specifically, when the standard deviation σ (or variance σ ² ) of the frequency distribution of the image data of the near-infrared image is less than N ₁ , the rank is “1”, and N ₁ or more and less than N ₂ is the rank “2”. A rank “3” is set between N _{2 and} N ₃ . The calculation unit 16 classifies the cracks of the tongue into any one of “1” to “3” according to the value of the standard deviation σ (or variance σ ² ) of the frequency distribution. In addition to the numerical values “1” to “3” shown above, this classification indicates the degree of cracking of the tongue (other than numerical values), such as “small”, “medium”, and “large”. Also good. This makes it possible to accurately detect cracks in the tongue (including the degree of cracking) regardless of the presence or absence of moss on the tongue surface and unevenness of the moss.

[Control flow]
FIG. 19 is a flowchart showing an operation flow in the organ image photographing apparatus 1 of the present embodiment. When the organ image photographing apparatus 1 receives a photographing instruction from the operation unit 5 or an unillustrated input unit, the illumination control unit 11 turns on the illumination unit 2 (S1) and sets photographing conditions such as illuminance (S1). S2). When the setting of the photographing conditions is completed, the imaging control unit 12 controls the imaging unit 3 to photograph the tongue that is the photographing target (S3). Thereby, the visible image and near-infrared image of a tongue are obtained.

When the photographing is completed, the image processing unit 15 extracts the contour line of the tongue from the photographed image of the tongue (S4). Then, the computing unit 16 detects the upper and lower ends and left and right ends of the tongue from the extracted outline, and sets areas for detecting diagnostic items (tongue color, thickness, crack) (S5). Specifically, the calculation unit 16 sets the region C in the tongue (see FIG. 8) when detecting the color of the tongue, and in the horizontal direction (A− When a region (each pixel) in the A ′ direction) is set and a crack on the tongue surface is detected, a region C in the tongue is set.

Next, the calculation unit 16 uses a feature amount (ratio IRm / Bm, region S) for detecting the color, thickness, and crack of the tongue based on the photographed image (visible image, near infrared image) of the tongue. The quadratic coefficient of the approximate polynomial and the standard deviation σ of the frequency distribution are extracted (S6), and these are classified (for example, digitized) by the above-described method (S7). The calculated numerical value is output from the calculation unit 16 and displayed on the display unit 4 and is also stored in the storage unit 17. However, the calculated numerical value is output from the audio output unit 7 by voice as necessary, or an output device (not shown). Or transferred to the outside via the communication unit 6 (S8).

As described above, in the present embodiment, the calculation unit 16 determines the tongue diagnosis item (tongue) based on at least the image data of the near-infrared image among the images obtained by photographing the tongue with the imaging unit 3. Color, thickness, cracks). In the near-infrared image, noise (eg, moss) that was visible in the visible image can no longer be seen, so the diagnostic items for the tongue can be accurately detected regardless of the state of the tongue surface (whether moss is present or moss is uneven). Can do. Therefore, the accuracy of diagnosis based on the detected diagnosis item is improved.

Also, the calculation unit 16 classifies the degree of tongue thickness as a diagnostic item based on the distribution of image data in one direction in the near-infrared image. In the near-infrared image, the influence of the moss on the tongue surface that becomes noise is eliminated, so that the distribution of the image data in one direction is smoother than that of the visible image. Accordingly, it becomes possible to accurately and easily grasp the cross-sectional shape of the tongue surface based on the distribution of the image data in one direction of the near-infrared image, and to detect the thickness of the tongue accurately. .

In particular, as the distribution of the image data in the near-infrared image, the horizontal surface of the tongue is obtained by using the distribution of the image data of each pixel that passes through the tongue and is aligned in a direction perpendicular to the direction connecting the tongue tip and the tongue base. By grasping the shape, the thickness of the tongue can be accurately detected.

In addition, the calculation unit 16 obtains a second order polynomial that approximates the above distribution in a predetermined region (region S) of the distribution of the image data, and classifies the degree of tongue thickness based on the second order coefficient. Therefore, it becomes easy to classify the thickness of the tongue into a plurality of ranks based on the magnitude and sign (positive / negative) of the second-order coefficient of the approximate polynomial.

Further, the calculation unit 16 classifies the degree of cracking of the tongue as a diagnostic item based on the frequency distribution of the image data of a partial region of the tongue in the near-infrared image. In the near-infrared image, the influence of the moss on the tongue surface, which becomes noise, is eliminated. Therefore, the frequency distribution of the near-infrared image becomes smaller than the frequency distribution of the visible image. Therefore, it is possible to accurately detect the presence or absence of a crack on the tongue surface based on the frequency distribution of the near-infrared image.

Especially, by using the frequency distribution of the region in the tongue in the near-infrared image, it is possible to reliably detect the cracks in the tongue that are likely to occur in the vicinity of the tongue.

Further, since the standard deviation or variance of the frequency distribution of the near-infrared image indicates the extent of the frequency distribution, the calculation unit 16 determines the degree of cracking of the tongue based on the standard deviation or variance of the frequency distribution. This makes it possible to accurately and reliably detect cracks on the tongue surface.

Also, the calculation unit 16 classifies the degree of tongue color as a diagnostic item based on the near-infrared image data and the B image data in the visible image. B image data is less likely to fluctuate due to the presence of moss than other G and R image data, so the near-infrared image data and B image data are used for tongue color detection. The color of the tongue can be accurately detected regardless of the presence or absence of moss.

In particular, the calculation unit 16 sets the ratio (IRm / Bm) of the average value IRm of the image data in a partial region of the near-infrared image and the average value Bm of the B image data in a partial region of the visible image. Based on the degree of tongue color. In this case, it is possible to determine whether the tongue is reddish or bluish by a relative comparison between IRm and Bm, and the color of the tongue can be easily detected.

[Others]
In the above, the case where the subject to be photographed is a human tongue has been described. However, as long as it is a living body (living object), it may not be a human but may be an animal other than a human. For example, even for the tongue of an animal such as a pet or livestock, the diagnostic item can be accurately detected by applying the method of the present embodiment. In this case, it is possible to quickly and accurately determine the poor physical condition of an animal that cannot communicate its intention.

In the present embodiment, the case where the organ of the living body is the tongue has been described as an example. However, for example, the eyelid or the like can be used as the organ to be diagnosed.

The organ image capturing apparatus described above can be expressed as follows, and has the following effects.

The organ imaging apparatus described above receives an illumination unit that illuminates a living organ with illumination light including near-infrared light, and receives the illumination light that is reflected from the surface of the organ. Based on image data of a near-infrared image obtained by receiving at least the near-infrared light included in the illumination light, among the image acquired by the imaging unit and the image acquired by the imaging unit, And an arithmetic unit for classifying the degree of diagnosis items of the organ.

The image acquired by the imaging unit includes a near-infrared image of the organ. In this near-infrared image, the noise (for example, moss if the organ is a tongue) cannot be seen in the visible image acquired by receiving visible light. Therefore, the operation unit classifies the degree of the diagnosis item of the organ based on the image data of the near-infrared image, thereby accurately detecting the diagnosis item of the organ regardless of the state of the organ surface that causes noise. Is possible.

The organ may be a tongue. In this case, in the near-infrared image, there is almost no color difference between the tongue and the moss on the surface, and the moss cannot be seen. Therefore, the arithmetic unit is affected by the moss on the tongue surface by classifying the degree of the diagnostic items of the tongue (for example, the color, thickness of the tongue, cracks on the surface) based on the image data of the near-infrared image. Therefore, the diagnostic item can be detected accurately. That is, even if the entire surface of the tongue is covered with moss or the moss distribution is uneven, it is possible to accurately detect the diagnostic items of the tongue.

The calculation unit may classify the degree of the thickness of the tongue as the diagnostic item based on the distribution of image data in one direction in the near-infrared image.

In the near-infrared image, the influence of the moss on the tongue surface that becomes noise is eliminated, so that the distribution of the image data in one direction in the near-infrared image is smoother than the distribution of the image data in one direction in the visible image. Therefore, it is possible to accurately and easily grasp the cross-sectional shape of the tongue surface based on the former distribution, and it is possible to accurately detect the thickness of the tongue.

The calculation unit may obtain a second order polynomial that approximates the distribution in a predetermined region of the distribution of the image data, and classify the degree of the thickness of the tongue based on the second order coefficient of the polynomial. .

In this case, the thickness of the tongue can be detected based on the magnitude and sign (positive / negative) of the second-order coefficient of the polynomial, and the level of the thickness of the tongue can be easily classified.

The distribution of the image data is a distribution of image data of each pixel that passes through the tongue between the tongue tip and the tongue base of the tongue in the near-infrared image and is aligned in a direction perpendicular to the direction connecting the tongue tip and the tongue base. Also good.

In this way, the surface shape in the horizontal direction of the tongue is grasped by classifying the degree of the thickness of the tongue based on the distribution of horizontal image data passing through the center of the tongue (in the tongue) in the near-infrared image. Thus, the thickness of the tongue can be accurately detected.

The calculation unit may classify the degree of cracking of the tongue as the diagnostic item based on a frequency distribution of image data of a partial region of the tongue in the near-infrared image.

In the near-infrared image, the influence of the moss on the tongue surface that causes noise is eliminated, so the spread of the frequency distribution of the image data of a partial region of the tongue in the near-infrared image is the same region as the above region in the visible image Smaller than the spread of the frequency distribution of data. Therefore, it is possible to accurately detect the presence or absence of a crack on the tongue surface based on the former frequency distribution.

The calculation unit may classify the degree of tongue cracking based on the standard deviation or variance of the frequency distribution. Since the standard deviation or variance indicates the extent of the frequency distribution, it is possible to accurately and reliably detect cracks on the tongue surface.

The partial region may be a region in the tongue between the tongue tip and the base of the tongue in the near-infrared image. In this case, it is possible to reliably detect a crack in the tongue that is likely to occur in the vicinity of the tongue surface on the tongue surface based on the frequency distribution.

The illumination light includes visible light, and the calculation unit receives the image data of the near-infrared image and the visible light included in the illumination light through the surface of the organ. The degree of tongue color as the diagnostic item may be classified based on the blue image data in the visible image.

The blue (B) image data of the visible image is less likely to fluctuate due to the presence of moss (less susceptible to moss) than the other green (G) and red (R) image data. Therefore, the color of the tongue can be accurately detected regardless of the presence of moss by classifying the degree of the color of the tongue based on the image data of the near-infrared image and the B image data of the visible image. It becomes possible.

When a partial region on the surface of the organ is a target region for diagnosis, the calculation unit calculates an average value of image data in the target region of the near-infrared image and a blue color in the target region of the visible image. The degree of tongue color may be classified based on the ratio to the average value of the image data.

Redness or blueness of the tongue is detected by relative comparison between the average value (IRm) of the image data in the target region of the near-infrared image and the average value (Bm) of the B image data in the target region of the visible image. Therefore, it is easy to detect the color of the tongue.

The present invention can be used in an apparatus for photographing a living organ and detecting a diagnosis item of the organ from the photographed image.

DESCRIPTION OF SYMBOLS 1 Organ imaging device 2 Illumination part 3 Imaging part 16 Calculation part

Claims

An illumination unit that illuminates a living organ with illumination light including near infrared light; and
An imaging unit that acquires an image of the organ by receiving the illumination light reflected from the surface of the organ;
Based on the image data of the near-infrared image obtained by receiving the near-infrared light included in the illumination light among the images acquired by the imaging unit, the degree of diagnosis items of the organ is determined. An organ image capturing apparatus comprising: an arithmetic unit for classifying.
2. The organ imaging apparatus according to claim 1, wherein the organ is a tongue.
The organ imaging according to claim 2, wherein the calculation unit classifies the degree of the thickness of the tongue as the diagnosis item based on a distribution of image data in one direction in the near-infrared image. apparatus.
The calculation unit obtains a second order polynomial that approximates the distribution in a predetermined region of the distribution of the image data, and classifies the degree of the thickness of the tongue based on the second order coefficient of the polynomial. The organ imaging apparatus according to claim 3.
The distribution of the image data is a distribution of image data of each pixel that passes through the tongue between the tongue tip and the tongue base of the tongue in the near-infrared image and is aligned in a direction perpendicular to the direction connecting the tongue tip and the tongue base. The organ image photographing device according to claim 3 or 4, characterized in that:
The said calculating part classify | categorizes the grade of the crack of the tongue as the said diagnostic item based on the frequency distribution of the image data of the partial region of the tongue in the said near-infrared image. An organ imaging apparatus according to any one of the above.
The organ imaging apparatus according to claim 6, wherein the calculation unit classifies the degree of cracking of the tongue based on a standard deviation or variance of the frequency distribution.
The organ imaging apparatus according to claim 6 or 7, wherein the partial region is a region in the tongue between a tongue tip and a tongue base in the near-infrared image.
The illumination light includes visible light,
The arithmetic unit is based on image data of the near-infrared image and blue image data in a visible image obtained by receiving the visible light included in the illumination light through the surface of the organ. The organ imaging apparatus according to claim 2, wherein the degree of color of the tongue as the diagnostic item is classified.
When a region on the surface of the organ is a target region for diagnosis,
The calculation unit calculates the degree of tongue color based on a ratio between an average value of image data in the target region of the near-infrared image and an average value of blue image data in the target region of the visible image. The organ imaging apparatus according to claim 9, wherein the organ imaging apparatus is classified.