WO2023163051A1

WO2023163051A1 - Oral-cavity cleaning device, control device, oral-cavity cleaning method, and program

Info

Publication number: WO2023163051A1
Application number: PCT/JP2023/006498
Authority: WO
Inventors: 泰雄大塚; 岳史浜崎; 祥光浅井; 俊幸中嶋
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2022-02-22
Filing date: 2023-02-22
Publication date: 2023-08-31

Abstract

An oral-cavity cleaning device (100) cleans teeth by causing a sprayed cleaning solution to collide with the teeth, the oral-cavity cleaning device (100) comprising: a cleaning solution spraying unit (103) that sprays the cleaning solution in a prescribed direction from a spray port (103a); a spray controller (153) that controls a flow rate and/or a spray pressure of the cleaning solution; a light projection unit (LED illumination unit (102)) that projects projection light in a projection region that encompasses a prescribed point and that extends across a line of extension; and a light detection unit (camera unit (101)) that encompasses a prescribed point and that performs detection in a two-dimensional detection region (101a) contained in the projection region. The spray controller (153) estimates a dental plaque portion in the detection region (101a) on the basis of a detection result of the camera unit (101), determines a control amount on the basis of the dental plaque portion estimation result, and sprays the cleaning solution from the spray port (103a) so as to achieve the determined control amount.

Description

MOUTH CLEANING DEVICE, CONTROL DEVICE, MOUTH CLEANING METHOD, AND PROGRAM

The present disclosure relates to an oral cleaning device, a control device, an oral cleaning method, and a program.

As an oral cavity cleaning device using a liquid such as a cleaning liquid, Patent Document 1 discloses that a liquid having fine bubbles is supplied into the oral cavity, and the steps of brushing and rinsing the teeth are integrated, thereby cleaning the oral cavity. A cleaning device is disclosed that can simultaneously clean and clean the tooth surface.

JP 2017-118940 A JP 2019-141582 A

By the way, it is important to control the detergency in an oral cleaning device that uses a liquid such as a cleaning liquid. The present disclosure provides an oral cavity cleaning device and the like that can appropriately control cleaning power.

An oral cavity cleaning device according to an aspect of the present disclosure is an oral cavity cleaning device for cleaning teeth, comprising: a cleaning control unit that controls a cleaning force acting on teeth via a tooth cleaning unit; and a cleaning effect by the tooth cleaning unit. a light irradiation unit that irradiates irradiation light onto an irradiation region that includes a cleaning effective region; and a light detection unit that detects within the detection area, wherein the cleaning control unit estimates the plaque portion within the detection area based on the detection result of the light detection unit, and estimates the plaque portion. Based on, the control amount of the cleaning power is determined, and the tooth cleaning unit is controlled so that the cleaning power of the tooth cleaning unit becomes the determined control amount.

A control device according to an aspect of the present disclosure is a control device for an oral cavity cleaning device that cleans teeth with a tooth cleaning unit, comprising: a cleaning control unit that controls a cleaning force that the tooth cleaning unit acts on the teeth; A two-dimensional detection area that includes the effective cleaning area and is included in the irradiation area includes the detection light emitted in response to the irradiation light applied to the irradiation area that includes the effective cleaning area that has the cleaning effect of the tooth cleaning unit. and an acquisition unit configured to acquire detection results detected within the detection area, wherein the cleaning control unit estimates a plaque portion within the detection region based on the acquired detection results, and estimates the plaque portion. Based on, the control amount of the cleaning power acting on the teeth via the tooth cleaning unit is determined, and the cleaning power is controlled so as to be the determined control amount.

A program according to an aspect of the present disclosure is a program for causing a computer to execute a control method for an oral cavity cleaning device that cleans teeth with a tooth cleaning unit, and in the control method, the cleaning effect of the tooth cleaning unit is A detection result obtained by detecting, in a two-dimensional detection area including the effective cleaning area and included in the irradiation area, the detection light emitted in response to the irradiation light irradiated to the irradiation area including the effective cleaning area. and estimating the plaque portion within the detection area based on the acquired detection result, and controlling the cleaning force acting on the tooth via the tooth cleaning unit based on the estimation result of the plaque portion. The amount is determined, and the detergency is controlled so as to be the determined control amount.

An oral cavity cleaning method according to an aspect of the present disclosure is an oral cavity cleaning method using an oral cavity cleaning apparatus that cleans teeth with a tooth cleaning unit, wherein an irradiation area includes a cleaning effective area having a cleaning effect by the tooth cleaning unit. a detection result obtained by detecting the detection light emitted in accordance with the irradiation light applied to the area within a two-dimensional detection area that includes the effective cleaning area and is included in the irradiation area, and the obtained detection result is Based on the result of estimating the dental plaque portion within the detection area, determining a control amount of the cleaning power acting on the tooth via the tooth cleaning unit based on the estimation result of the dental plaque portion, and determining the cleaning power , is controlled to be the determined control amount.

According to the oral cavity cleaning device and the like of the present disclosure, the oral cavity can be cleaned with appropriately controlled cleaning power.

FIG. 1 is a diagram showing a usage example of an oral cavity cleaning device, etc., according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of the oral cavity cleaning device etc. according to the embodiment. FIG. 3 is a flow chart showing an operation example of the mouthwash device according to the embodiment. FIG. 4 is a flowchart showing an operation example in the plaque detection injection mode of the oral cavity cleaning apparatus according to the embodiment. FIG. 5 is a graph showing the relationship between the detection result and the control amount in the mouthwash device according to the embodiment. FIG. 6 is another graph showing the relationship between the detection result and the control amount in the mouthwash device according to the embodiment. FIG. 7 is a block diagram showing the functional configuration of an oral cavity cleaning device and the like according to Modification 1 of the embodiment. FIG. 8 is a flowchart showing an operation example of the mouthwash apparatus according to Modification 1 of the embodiment. FIG. 9 is a flowchart showing an operation example in the plaque detection injection mode and the malfunction detection injection mode of the oral cavity cleaning apparatus according to Modification 1 of the embodiment. FIG. 10 is a block diagram showing the functional configuration of an oral cavity cleaning device etc. according to Modification 2 of the embodiment. 11A is a perspective view of an intraoral camera in the intraoral camera system according to the embodiment; FIG. FIG. 11B is a cross-sectional view schematically showing an imaging optical system incorporated in the intraoral camera in the intraoral camera system according to the embodiment; FIG. 12 is a schematic configuration diagram of an intraoral camera system according to an embodiment. FIG. 13 is a diagram showing the flow of intraoral imaging operation in the intraoral camera system according to the embodiment. FIG. 14 is a functional block diagram of the mobile terminal according to the embodiment; FIG. 15 is a diagram showing a state in which a user is photographing using the intraoral camera system according to the embodiment. FIG. 16A is a diagram showing an example of a first RGB image of the front tooth photographed in the state shown in FIG. 15; FIG. 16B is a diagram showing an example of color difference data between a plaque region and a tooth region when the tooth is irradiated with light including a blue light wavelength region without a blue light cut filter. FIG. 16C is a diagram showing an example of color difference data between a plaque region and a tooth region when the tooth is irradiated with light including a blue light wavelength region using a blue light cut filter. FIG. 16D is a diagram showing an example of color difference data between a plaque region and a tooth region when the tooth is irradiated with light including a wavelength range of blue light and white light without a blue light cut filter. FIG. 17A is a diagram showing an example of a second RGB image of front teeth captured in the state shown in FIG. FIG. 17B shows the difference between the plaque region and the tooth region after performing the exposure control processing and the white balance adjustment processing when the teeth are irradiated with light including the wavelength range of blue light without a blue light cut filter. It is a figure which shows an example of color difference data. FIG. 17C shows the color difference between the dental plaque region and the tooth region after performing the exposure control processing and the white balance adjustment processing when the tooth is irradiated with light including the blue light wavelength region using the blue light cut filter. It is a figure which shows an example of data. FIG. 17D shows the plaque region and the tooth after exposure control processing and white balance adjustment processing when the tooth is irradiated with light including the wavelength range of blue light and white light without a blue light cut filter. 2 is a diagram showing an example of color difference data of an area of . FIG. 18 is a diagram showing an example of the fourth RGB image of the front teeth photographed in the state shown in FIG. FIG. 19 is a diagram showing another example of the fourth RGB image of the front teeth photographed in the state shown in FIG. FIG. 20 is a flowchart of image processing in the mobile terminal 1070. FIG.

An oral cavity cleaning device according to a first aspect of the present disclosure is an oral cavity cleaning device for cleaning teeth, comprising: a cleaning control unit for controlling cleaning force acting on the teeth via the tooth cleaning unit; and a cleaning effect by the tooth cleaning unit. A light irradiator that irradiates an irradiation area that includes an effective cleaning area with irradiation light, and a detection light emitted in response to the irradiation light is emitted in a two-dimensional detection area that includes the effective cleaning area and is included in the irradiation area. and a light detection unit that detects with the light detection unit, the cleaning control unit estimates the plaque portion within the detection area based on the detection result of the light detection unit, and determines the cleaning power based on the estimation result of the plaque portion. A control amount is determined, and the tooth cleaning unit is controlled so that the detergency of the tooth cleaning unit becomes the determined control amount.

In such an oral cavity cleaning device, the cleaning power can be a control amount determined based on the estimated result of the plaque portion. The estimation result of the dental plaque portion is estimated by detecting the detection light emitted in accordance with the irradiation light in a two-dimensional detection area when the irradiation light is irradiated. For example, if the irradiation light is light capable of selectively coloring dental plaque, detecting the light generated by the coloring of dental plaque as the detection light increases the detection accuracy of the dental plaque portion. be able to. In this way, it is possible to appropriately estimate the plaque portion within the detection area and determine the control amount of at least one of the contact amount and the contact pressure for the detection area according to the estimated plaque portion. . Therefore, the oral cavity cleaning apparatus can clean the oral cavity with the appropriately controlled contact amount and contact pressure of the cleaning material.

Further, for example, an oral cavity cleaning device according to a second aspect of the present disclosure is the oral cavity cleaning device according to the first aspect, wherein the tooth cleaning unit is a brush, and the cleaning control unit is a brush as a cleaning power of the brush. and a drive control unit that controls the drive mode of.

In such an oral cavity cleaning device, the drive amount of the brush can be a control amount determined based on the estimated result of the dental plaque portion. The estimation result of the dental plaque portion is estimated by detecting the detection light emitted in accordance with the irradiation light in a two-dimensional detection area when the irradiation light is irradiated. For example, if the irradiation light is light capable of selectively coloring dental plaque, detecting the light generated by the coloring of dental plaque as the detection light increases the detection accuracy of the dental plaque portion. be able to. In this way, it is possible to appropriately estimate the plaque portion within the detection region and determine the control amount of the drive amount for the detection region according to the estimated plaque portion. Therefore, in the oral cavity cleaning apparatus, the oral cavity can be cleaned by appropriately controlling the driving amount of the brush.

Further, for example, the oral cavity cleaning device according to the third aspect of the present disclosure is the oral cavity cleaning device according to the first aspect, wherein the tooth cleaning unit causes the cleaning liquid ejected from the ejection port toward the teeth to collide with the teeth. The washing controller functions as an injection controller that controls at least one of the flow rate of the washing liquid as the washing power of the washing liquid injection section, the washing liquid, and the injection pressure of the injection frequency.

In such an oral cavity cleaning device, at least one of the flow rate of the cleaning liquid ejected from the ejection port, the cleaning liquid, and the ejection frequency can be set as a control amount determined based on the estimation result of the dental plaque portion. The estimation result of the dental plaque portion is estimated by detecting the detection light emitted in accordance with the irradiation light in a two-dimensional detection area when the irradiation light is irradiated. For example, if the irradiation light is light capable of selectively coloring dental plaque, detecting the light generated by the coloring of dental plaque as the detection light increases the detection accuracy of the dental plaque portion. be able to. Thus, appropriately estimating the dental plaque portion within the detection area and determining the control amount of at least one of the flow rate, the washing liquid, and the injection frequency corresponding to the estimated dental plaque portion for the detection area. can be done. Therefore, in the oral cavity cleaning apparatus, the oral cavity can be cleaned by appropriately controlling the flow rate of the cleaning liquid, the cleaning liquid, and the ejection frequency. In addition, as for the injection frequency, by setting an appropriate injection frequency according to the state of the gingiva, in other words, by performing an appropriate intermittent injection according to the state of the gingiva, it is possible to reduce the gingival It has the effect of suppressing damage that would further aggravate the condition.

Further, for example, the oral cavity cleaning device according to the fourth aspect of the present disclosure is the oral cavity cleaning device according to the third aspect, in which the injection control unit determines, based on the detection result, the area of the plaque portion within the detection region is calculated from the area of the portion where the detection light is detected in the detection area, and the control amount of at least one of the flow rate and injection pressure of the cleaning liquid injected from the injection port is determined so that the larger the calculated area, the larger the control amount. do.

According to this, the plaque portion within the detection region is appropriately estimated as the calculated area, and the control amount of at least one of the flow rate and the injection pressure corresponding to the estimated area of the plaque portion for the detection region. can be determined. Therefore, the oral cavity cleaning apparatus can clean the oral cavity with the appropriately controlled flow rate and injection pressure of the cleaning liquid.

Further, for example, the oral cavity cleaning device according to the fifth aspect of the present disclosure is the oral cavity cleaning device according to the third aspect, in which the injection control unit determines, based on the detection result, The amount is calculated from the luminance value of the detected light within the detection area, and the control amount of at least one of the flow rate and the injection pressure of the cleaning liquid injected from the injection port is determined so that the larger the calculated adhesion amount, the larger the amount.

According to this, the plaque portion in the detection area is appropriately estimated as the amount of adhesion calculated from the luminance value of the detection light, and the flow rate and the amount corresponding to the estimated amount of plaque adhesion to the detection area Control variables for at least one of the injection pressures can be determined. Therefore, the oral cavity cleaning apparatus can clean the oral cavity with the appropriately controlled flow rate and injection pressure of the cleaning liquid.

Further, for example, an oral cavity cleaning device according to a sixth aspect of the present disclosure is the oral cavity cleaning device according to any one of the third to fifth aspects, wherein the irradiation light kills bacteria contained in dental plaque. The wavelength of light that excites the metabolite is included, and the detected light is the fluorescence emitted by the excited metabolite.

According to this, the irradiated light excites the metabolites of bacteria contained in the dental plaque, and the fluorescence emitted by the excited metabolites can be detected as the detection light, so that the light selectively emitted by the dental plaque can be detected. can improve the detection accuracy of the plaque portion.

Further, for example, an oral cavity cleaning device according to a seventh aspect of the present disclosure is the oral cavity cleaning device according to any one of the third to sixth aspects, wherein the injection control unit is configured such that the light detection unit emits the detection light. Injection of the cleaning liquid from the injection port is prohibited during a prohibition period that at least partially overlaps with a detection period from the start to the end of detection.

According to this, the sprayed cleaning liquid is likely to scatter as droplets, which may affect detection. can be created to detect the detected light.

Further, for example, an oral cavity cleaning device according to an eighth aspect of the present disclosure is the oral cavity cleaning device according to any one of the first to seventh aspects, wherein the brightness of the detected light is a predetermined value or less. If there is a shield that blocks at least one of (i) the detection light between the detection region and the light detection unit, or (ii) the irradiation light between the light irradiation unit and the irradiation region, when it continues for a period of time or longer. It further includes a determination unit that makes determinations and an output unit that outputs determination results of the determination unit.

According to this, (i) the detection light between the detection region and the light detection unit, or (ii) the presence of a shield that blocks at least one of the irradiation light between the light irradiation unit and the irradiation region is detected. It is possible to determine whether or not the state in which the brightness of the light is equal to or less than a predetermined value has continued for a predetermined time or longer, and output the determination result.

Further, for example, an oral cavity cleaning device according to a ninth aspect of the present disclosure is the oral cavity cleaning device according to any one of the first to eighth aspects, wherein the cleaning control unit detects Based on this, the dental plaque portion is estimated in a cleaning effective region which is a partial region within the detection region and has a cleaning effect by the tooth cleaning section, and based on the estimation result of the dental plaque portion, the control amount of cleaning power is determined. decide.

Further, a control device according to a tenth aspect of the present disclosure is a control device for an oral cavity cleaning device that cleans teeth by a tooth cleaning unit, and includes a cleaning control unit that controls the cleaning force that the tooth cleaning unit acts on the teeth. , the detection light emitted in response to the irradiation light irradiated to the irradiation area including the effective cleaning area having the cleaning effect by the tooth cleaning unit is detected within the two-dimensional detection area including the effective cleaning area and included in the irradiation area. and an acquisition unit for acquiring the detection result detected by the cleaning control unit, based on the acquired detection result, estimates the plaque portion within the detection region, and based on the estimation result of the plaque portion, A control amount of the cleaning power acting on the tooth through the cleaning unit is determined, and the cleaning power is controlled to be the determined control amount.

Such a control device can make the mouthwash device using the control device have the same effect as the mouthwash device described above.

Further, a program according to an eleventh aspect of the present disclosure is a program for causing a computer to execute a control method for an oral cavity cleaning apparatus that cleans teeth by a tooth cleaning unit, wherein the control method includes the cleaning effect of the tooth cleaning unit Acquisition of the detection result obtained by detecting the detection light emitted in response to the irradiation light irradiated to the irradiation area that includes the effective cleaning area and detecting it in the two-dimensional detection area that includes the effective cleaning area and is included in the irradiation area. estimating the plaque portion within the detection area based on the acquired detection result, determining a control amount of the cleaning force acting on the tooth via the tooth cleaning unit based on the estimation result of the plaque portion, The detergency is controlled so as to be the determined control amount.

Such a program can use a computer to achieve the same effects as the control device described above.

In addition, an oral cavity cleaning method according to a twelfth aspect of the present disclosure is an oral cavity cleaning method using an oral cavity cleaning device that cleans teeth with a tooth cleaning unit, and includes a cleaning effective area having a cleaning effect by the tooth cleaning unit. A detection result obtained by detecting the detection light emitted in accordance with the irradiation light applied to the irradiation region in a two-dimensional detection region that includes the effective cleaning region and is included in the irradiation region, and based on the obtained detection result Based on the result of estimating the plaque portion, the control amount of the cleaning force acting on the tooth via the tooth cleaning unit is determined, and the cleaning force is determined by the determined control amount. Control so that

With such an oral cavity cleaning method, it is possible to achieve the same effect as the oral cavity cleaning device described above.

Further, an oral cavity cleaning apparatus according to another aspect of the present disclosure is an oral cavity cleaning apparatus that cleans teeth by causing the ejected cleaning liquid to collide with the teeth, wherein the cleaning liquid is ejected from the ejection port in a predetermined direction. an injection control unit that controls at least one of the flow rate and injection pressure of the cleaning liquid injected from the injection port; and a detection area that includes a predetermined point on an extension line extending in a predetermined direction from the injection port, a light detection unit that detects an image in a detection area that extends two-dimensionally across the line, and the ejection control unit detects the image based on the detected image when the detected image includes the gingiva portion. , estimating the state of the gingiva, determining the control amount of at least one of the flow rate and the injection pressure based on the estimation result of the state of the gingiva, and making at least one of the flow rate and the injection pressure equal to the determined control amount. to spray the cleaning liquid from the injection port.

In such an oral cavity cleaning device, at least one of the flow rate and the ejection pressure of the cleaning liquid ejected from the ejection port can be a control amount determined based on the estimated state of the gingival portion. This estimation result of the state of the gingiva is estimated based on the detected image when the gingiva is included in the image detected in the two-dimensional detection area. In this way, it is possible to appropriately estimate the state of the gingival portion within the detection region and determine the control amount of at least one of the flow rate and the injection pressure for the detection region according to the estimated state of the gingival portion. can. Therefore, the oral cavity cleaning apparatus can clean the oral cavity with the appropriately controlled flow rate and injection pressure of the cleaning liquid.

Also, for example, the estimation result indicates the presence or absence of at least one of swelling and bleeding in the gingival portion, and the ejection control unit, if the estimation result indicates that there is at least one of swelling and bleeding in the gingival portion, The amount of control of at least one of the flow rate and injection pressure of the cleaning liquid injected from the injection port may be determined so as to be smaller than when it is indicated that there is no swelling or bleeding in the part.

According to this, the presence or absence of at least one of swelling and bleeding in the gingival portion is estimated as the estimated state of the gingival portion, and when the estimation result indicates that there is at least one of swelling and bleeding in the gingival portion. , the amount of control of at least one of the flow rate and injection pressure of the cleaning liquid injected from the injection port can be determined so as to be smaller than when it is shown that there is no swelling or bleeding in the gingival region. Therefore, the oral cavity cleaning apparatus can clean the oral cavity with the appropriately controlled flow rate and injection pressure of the cleaning liquid.

Further, for example, the injection control unit inhibits injection of the cleaning liquid from the injection port during a prohibition period that at least partially overlaps with the detection period from when the light detection unit starts detecting the detection light until it ends. good too.

In addition, a control device according to another aspect of the present disclosure is a control device for an oral cavity cleaning device for cleaning teeth by causing jetted cleaning liquid to collide with teeth, wherein the cleaning liquid is directed in a predetermined direction and jetted. and a detection area including a predetermined point on an extension line extending in a predetermined direction from the injection port, the extension line and an acquisition unit that acquires an image detected in a detection area that extends two-dimensionally by intersecting the , estimating the state of the gingiva, determining the control amount of at least one of the flow rate and the injection pressure based on the estimation result of the state of the gingiva, and making at least one of the flow rate and the injection pressure equal to the determined control amount. to spray the cleaning liquid from the injection port.

Such a control device can cause the mouthwash device in which the control device is used to have the same effect as the other mouthwash device described above.

Further, a program according to an aspect of the present disclosure is a program for causing a computer to execute a control method for an oral cavity cleaning device for washing teeth by colliding the jetted cleaning liquid against the teeth, the control method comprising: A detection area that includes a predetermined point on an extension line extending in a predetermined direction from an injection port for injecting cleaning liquid in a predetermined direction, and detected in a detection area that extends two-dimensionally across the extension line. An image is acquired, and if the acquired image includes a gingival portion, the state of the gingival portion is estimated based on the acquired image, and at least one of the flow rate and the injection pressure is calculated based on the estimation result of the state of the gingival portion. is determined, and the cleaning liquid is jetted from the injection port so that at least one of the flow rate and the injection pressure of the cleaning liquid becomes the determined control amount.

Such a program can use a computer to achieve the same effect as the other control device described above.

Further, an oral cavity cleaning method according to another aspect of the present disclosure is an oral cavity cleaning method for cleaning teeth by causing a jetted cleaning liquid to collide with teeth, wherein the cleaning liquid is jetted in a predetermined direction. An irradiation area that includes a predetermined point on an extension line extending from the mouth in a predetermined direction, and that is an irradiation area that spreads across the extension line is irradiated with irradiation light, and according to the irradiated irradiation light, the plaque is removed. The emitted detection light is detected in a two-dimensional detection area including at least a part of the irradiation area, and based on the detection result of the detection light, at least one of the flow rate and injection pressure of the cleaning liquid injected from the injection port is controlled. The amount of the cleaning liquid is determined, and the cleaning liquid is jetted from the injection port so that at least one of the flow rate and the jet pressure of the cleaning liquid becomes the determined control amount.

With such an oral cavity cleaning method, it is possible to achieve the same effect as the other oral cavity cleaning device described above.

In addition, these general or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

It should be noted that the embodiments described below are all comprehensive or specific examples. Numerical values, shapes, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are examples, and are not intended to limit the present disclosure. Further, among the constituent elements in the following embodiments, constituent elements not described in independent claims will be described as optional constituent elements.

In addition, each figure is a schematic diagram and is not necessarily strictly illustrated. Therefore, for example, scales and the like do not necessarily match in each drawing. Moreover, in each figure, the same code|symbol is attached|subjected about the substantially same structure, and the overlapping description is abbreviate|omitted or simplified.

Also, in this specification, terms that indicate the relationship between elements such as parallel, coincident, and orthogonal, terms that indicate the shape of elements such as ring-shaped, and numerical values and numerical ranges are strictly meaning only is not an expression that represents a substantially equivalent range, for example, a difference of about several percent (for example, about 10%).

(Embodiment)
An oral cavity cleaning device according to the present embodiment will be described below with reference to FIGS. 1 to 6. FIG.

[1. Configuration of oral cavity cleaning device]
First, the configuration of an oral cavity cleaning apparatus according to this embodiment will be described with reference to FIGS. 1 and 2. FIG. FIG. 1 is a diagram showing an example of use of an oral cavity cleaning device, etc., according to the present embodiment. In addition, in FIG. 1, (a) shows the oral cavity cleaning apparatus 100 in use, and (b) shows the state of the terminal device 200 connected at that time. Moreover, (c) of FIG. 1 shows an oral cavity cleaning device 100c according to another example of the present embodiment. In FIG. 1C, for the purpose of making it easier to see the configuration of the portion that overlaps the brush 103b, illustration of some tufts that constitute the brush 103b is omitted. Moreover, FIG. 2 is a block diagram showing the functional configuration of the oral cavity cleaning device etc. according to the embodiment.

As shown in (a) of FIG. 1 , the oral cavity cleaning apparatus 100 ejects cleaning liquid toward the user's oral cavity from a cleaning liquid ejection section 103 as an example of a tooth cleaning section, thereby cleaning the teeth 99 of the user. It is a water-jet cleaning device that cleanses the oral cavity by removing deposits attached to the mouth. Alternatively, as shown in (c) of FIG. 1, the oral cavity cleaning device 100c can be realized as an electrically driven toothbrush device in which a brush 103b as an example of a tooth cleaning section is arranged. The user holds the holding part (not shown in FIGS. 1(a) and 1(c)) and moves the holding hand to spray the cleaning liquid. While changing the position and attitude of the mouth 103a or the brush 103b driven on the tooth surface, the jetted cleaning liquid is made to collide with or contact the brush 103b at an arbitrary position in the oral cavity (for example, the tooth 99). The colliding cleaning liquid can remove deposits adhering to the colliding portion by the impact. In addition, the contacting brush 103b can be driven to reciprocate in a fixed direction along the tooth surface, thereby removing deposits adhering to the track of the brush. Such adhering substances include plaque 99a and calculus, which consist of food debris, bacteria grown in the food debris, and the like. In the following, unless otherwise specified, an example of a water jet type oral cavity cleaning device 100 that injects cleaning liquid will be described as the oral cavity cleaning device.

In the present embodiment, the oral cavity cleaning device 100 is configured so that the position where the cleaning liquid collides can be visually recognized. Specifically, the oral cavity cleaning device 100 is provided with a camera unit 101, and an image captured by detecting intraoral light in this camera unit is captured by, for example, the terminal device 200 owned by the user. It is displayed on the display unit 203 . As a result, the user can clean the oral cavity while visually confirming to which position the injection port 103a currently faces and to which cleaning liquid is being injected.

By the way, since the color of the plaque 99a among the deposits is similar to the color of the tooth 99 (white to light white such as beige), it is difficult to visually recognize it only by using a normal camera. . Therefore, in the oral cavity cleaning apparatus 100 of the present embodiment, by selectively coloring the plaque 99a, the color of the plaque 99a can be distinguished from the color of the tooth 99 as shown in FIG. 1(b). Generate the image as displayed. Furthermore, by analyzing this image, the oral cavity cleaning apparatus 100 can determine the strength of the cleaning liquid jetting (the flow rate, jetting pressure, and jetting frequency of the cleaning liquid) when the plaque 99a is located in the region facing the jetting port 103a. at least one of them) can be changed to appropriately clean the oral cavity by jetting the cleansing liquid with appropriate strength (in other words, cleansing power).

In the example of the oral cavity cleaning device 100c, setting the strength of the detergency as described above corresponds to selecting the driving mode of the brush 103b. Drive modes of the brush 103b include a horizontal brushing mode in which the brush 103b is driven along the direction in which the tooth and another adjacent tooth are arranged, and a brushing mode between the tooth and another adjacent tooth (that is, between the teeth). ) in which the brush 103b is driven along the gap, there are several drive modes corresponding to set values such as drive period and drive torque. In other words, the driving modes of the respective brushes 103b have different strengths in the horizontal polishing mode and the strength in the horizontal polishing mode. It is possible to perform appropriate cleaning of the oral cavity by Among such drive modes, there is also a drive mode for only horizontal polishing in which the strength of the horizontal polishing mode is 0, and a drive mode for only horizontal polishing in which the strength of the horizontal polishing mode is 0.

Hereinafter, detailed configurations of the oral cavity cleaning device 100 and the terminal device 200 used in combination with the oral cavity cleaning device 100 will be described with continued reference to FIGS.

As shown in FIG. 2 , the oral cavity cleaning device 100 includes a camera section 101 , an LED lighting section 102 , a cleaning liquid injection section 103 , an input section 104 and a control device 150 . The camera unit 101, the LED illumination unit 102, the cleaning fluid injection unit 103, and the input unit 104 are hardware portions in the oral cavity cleaning device 100, and are sometimes called hardware units. On the other hand, the control device 150 is a functional part that generates and outputs various control signals for controlling the operation of the hardware section and acquires information obtained as a result of the operation of the hardware section. Control device 150 is realized by executing a predetermined program using a computer or the like, more specifically, a processor and memory. Therefore, it can be said that the main function of the control device 150 is the program executed using the processor and the memory.

In the following description, as shown in FIG. 2, an example in which the control device 150 is provided inside the oral cavity cleaning device 100 will be described. can be placed in For example, the control device 150 can be built in the terminal device 200, or can be built in a server device (not shown) such as an edge server or a cloud server. When the control device 150 is incorporated in such a server device, the hardware unit and the control device 150 are connected via a network such as a wide area communication network or a local communication network, and the hardware unit connected via the network It is implemented as an oral cavity cleaning system in combination with the control device 150 .

The control device 150 includes a camera control unit 151 connected to the camera unit 101, an LED lighting control unit 152 connected to the LED lighting unit 102, an injection control unit 153 connected to the cleaning liquid injection unit 103, and an input unit 104. It has a memory unit 154 connected thereto. In addition, the control device 150 has an image processing section 155 and a plaque adhesion amount detection section 156 operating therein.

The camera unit 101 is an example of a light detection unit. As shown in FIG. 1, a plurality of pixels for detecting the brightness of each point in the detection area 101a are arranged in a matrix in correspondence with the detection area 101a extending two-dimensionally. It is a camera realized by combining an image pickup device lined up in a camera with various optical elements such as a lens, an aperture, and a filter. The camera unit 101 receives a shooting start signal from the camera control unit 151 to start shooting, and receives a shooting end signal from the camera control unit 151 to finish shooting. The camera unit 101 then transmits the captured image to the camera control unit 151 . The image here shows luminance values of red, green, and blue detected at each point of the detection area 101a.

Further, the detection area 101a includes a predetermined point on an extension line extending from the injection port 103a in the injection direction (or the predetermined direction), which is the direction in which the cleaning liquid is injected. The predetermined point corresponds to the distance between the injection port 103a and the tooth 99 when an appropriate distance to the tooth 99 is maintained when using the oral cavity cleaning device 100 . As a result, the image captured in the detection area 101a shows the surface of the tooth 99 or the like that the cleaning liquid collides with. For example, in the drawing, a thick dashed arrow extending in a predetermined direction from the injection port 103a penetrates the detection region 101a. Furthermore, the detection area 101a has a thickness in a direction intersecting the surface on which the area extends. This thickness corresponds to the direction in which the camera unit 101 receives light (thin dashed line arrow in the figure), and covers unevenness such as a recess between two teeth 99 within the detection area 101a. there is

The camera control unit 151 controls image capturing by the camera unit 101 and acquires the resulting image by receiving it from the camera unit 101 . That is, the camera control unit 151 is also an example of an acquisition unit. The camera control section 151 then outputs the acquired image to the image processing section 155 .

The LED illumination unit 102 is an example of a light irradiation unit. As shown in FIG. 1, an irradiation area ( A fan-shaped region extending between two dashed lines in the figure) is irradiated with irradiation light. The LED illumination unit 102 is implemented by an LED light source, a lens, a power conversion unit for driving the LED light source, and the like. The LED lighting unit 102 receives an irradiation start signal from the LED lighting control unit 152 to start light irradiation, and receives an irradiation end signal from the LED lighting control unit 152 to end light irradiation. The light (irradiation light) emitted from the LED illumination unit 102 includes light that enables the terminal device 200 to visually recognize the contours of the teeth 99 and gums 98, and excitation light that selectively colors the dental plaque 99a. It is a synthetic light containing

Here, the dental plaque 99a is composed of a mass of bacteria adhering to the teeth 99 and their metabolites, and is also called plaque. These metabolites include a class of substances that fluoresce when excited by light of a particular wavelength. Therefore, in the present embodiment, the irradiation light contains light of the excitation wavelength for exciting such metabolites, and does not contain part of the light of the wavelength-shifted fluorescence wavelength. . Therefore, the LED illumination unit 102 includes an optical element such as a bandpass filter that can select wavelengths.

An example of the above metabolites is porphyrin. Porphyrin, which is a substance produced by bacteria contained in dental plaque 99a, emits reddish purple to orange light around 660 nm when irradiated with blue light around 405 nm. Since such fluorescent emission cannot be obtained from the enamel on the surface of the tooth 99, it is possible to selectively color the dental plaque 99a. Note that this example of porphyrin is only an example, and any substance that is a metabolite of bacteria contained in dental plaque 99a and whose excitation wavelength and fluorescence wavelength are sufficiently separated may be used. Since the wavelengths of the excitation light and the fluorescence are different in this way, it is possible to distinguish the light from the reflected light simply reflected on the surface.

In addition, since porphyrin accumulates according to the breeding period of bacteria, it can also be used as an index for measuring how long the adhering dental plaque 99a has been present. The camera unit 101 is preferably configured to detect fluorescence wavelengths of fluorescence emitted from such metabolites.

On the other hand, the LED lighting unit 102 is configured to prohibit light irradiation under specific conditions. This prevents unnecessary power consumption such as light being emitted outside the oral cavity due to unnecessary light irradiation, suppresses the adverse effects of the irradiated light on external objects, and prevents images from being captured. This is a function to ensure that the image is stabilized. The above specific conditions may be as follows.

When it is determined that the image captured by the camera unit 101 includes a person's face and eyes. In a mode in which the camera unit 101 captures a still image as an image, such as a still image capturing mode, a certain period of time (this In this case, it may also have a function of notifying that shooting has not started). In a state connected to the terminal device 200, within a certain period of time from when the shutter button of the mouthwash device 100 is turned on from the application on the terminal device 200 until the posture of the mouthwash device 100 stabilizes (similarly, it also has a notification function) may be present). In a mode in which the camera unit 101 takes a still image as an image, such as a moving image shooting mode, the light irradiation is linked with the shooting start button and the shooting end button. (b) When the shooting end button ON is detected, the light irradiation ends immediately; (c) shooting within a certain period of time from the shooting start button ON. , automatically terminate the light irradiation. Only when it is connected to the terminal device 200, the application on the terminal device 200 is activated, and the shooting start button is turned on, light is emitted after a certain period of time has passed (similarly, it may also have a notification function). Alternatively, light irradiation ends automatically after a certain period of time has passed since light irradiation.

The cleaning liquid injection unit 103 is composed of an actuator for injecting the cleaning liquid from the injection port 103a, a tank for storing the cleaning liquid, and the like. The cleaning liquid injection unit 103 injects the cleaning liquid from the injection port 103 a at the flow rate and injection pressure of the cleaning liquid determined by the injection control unit 153 . Therefore, the cleaning liquid injection unit 103 receives a control signal designating the flow rate and the injection pressure from the injection control unit 153, and injects the cleaning liquid according to the received control signal. However, since the flow rate and injection pressure of the cleaning liquid jetted from the cleaning liquid jetting section 103 change according to the image of the detection area 101a captured by the camera section 101, the cleaning liquid jetting section 103 is controlled by the injection control section 153 each time. A control signal is received that specifies flow rate and injection pressure.

The cleaning liquid injected by the cleaning liquid injection unit 103 may be simply tap water or the like, or may be a chemical liquid for improving the cleaning effect, or a slurry liquid containing fine granules for polishing off plaque 99a. may

The input unit 104 is a functional unit provided in a grip unit (not shown in FIG. 1) for performing input from the user, and includes various sensors such as a physical switch and a touch panel that can detect user input. realized by The input unit 104 is used, for example, by the user himself/herself to input the set values of the flow rate and the injection pressure of the cleaning liquid that are suitable for the user. These input setting values are stored via the memory unit 154 .

The memory unit 154 is a controller that stores information in a storage device such as a semiconductor memory (not shown) and reads the information as needed. Therefore, the memory unit 154 has a function of accessing the storage device to store information and referring to and reading out the stored information.

Note that the terminal device 200 can also be provided with the same function as the input unit 104. Specifically, similarly to the input unit 104, the terminal input unit 202 is used by the user himself/herself to input the set values of the flow rate and the injection pressure of the cleaning liquid that are suitable for the user, for example. These input setting values are stored via the communication unit 201 and the memory unit 154 .

The image processing unit 155 converts the image output from the camera control unit 151, thereby improving the accuracy of estimating the amount of plaque 99a in the plaque amount detection unit 156, which will be described later. Specifically, the image processing unit 155 performs processing for emphasizing the brightness near the fluorescence wavelength of the target metabolite in the image output from the camera control unit 151 . As a result, it is possible to easily estimate the dental plaque portion in the image to which the dental plaque 99a adheres based on the difference in brightness value. The image processing unit 155 outputs the converted image to the plaque amount detection unit 156 . For details, refer to [5. Image Processing Method, Image Processing Apparatus, and Program Related to Image Processing Method], the image processing unit 155 may output an image after conversion using a technique such as this image processing method. Further, the camera unit 101 may also be realized with a configuration adapted to the technology such as this image processing method.

The plaque adhesion amount detection unit 156 estimates the adhesion amount of plaque 99a in the detection area 101a shown in the image. The amount of plaque 99a estimated by the plaque amount detection unit 156 is output to the injection control unit 153 and used to determine the flow rate and injection pressure of the cleaning liquid. In this way, the amount of plaque 99a adhering to the tooth 99 is estimated from the detected fluorescence, that is, the photographed image. A cleaning liquid can be injected. Note that the image processing unit 155 and the plaque adhesion amount detection unit 156 may be included in the injection control unit 153 as part of their functions. That is, the injection control unit 153 may directly determine the flow rate and injection pressure of the cleaning liquid from the image.

When detecting the amount of plaque 99a, the plaque adhesion amount detection unit 156 calculates the area ratio of the portion where plaque adheres to the entire detection region 101a, and the area ratio is determined in advance. If it is equal to or higher than the threshold value, the cleaning liquid may be jetted at the flow rate and jetting pressure when plaque is attached. Describe detection. Since the injection-type mouthwash device 100 injects the cleaning liquid in a predetermined direction, the cleaning liquid collides with a predetermined point on the extension line.

Then, it is possible to set a cleaning effective area 101aa narrower than the detection area 101a, in which a certain or more cleaning effect can be expected (or simply having a cleaning effect) due to the collision of the cleaning liquid. In the case of a toothbrush device, the effective cleaning area 101aa corresponds to a portion of the toothbrush device that is held at a fixed position and is in contact with the brush in the trajectory of the driven brush 103b when the brush 103b is driven. do. However, the detection itself is performed before the brush comes into contact with the teeth, and the toothbrush device is moved from the position where the detection is made in the direction in which the brush extends (that is, when the toothbrush is moved toward the teeth). It is assumed that the device is held at a fixed position.

In the present embodiment, plaque adhesion amount detection unit 156 calculates the area ratio of the portion where plaque adheres to the entire cleaning effective area 101aa, and the area ratio is determined in advance. If it is equal to or higher than the threshold value, control is performed so that the cleaning liquid is jetted at the flow rate and jetting pressure for when plaque is attached. Therefore, the plaque adhesion amount detection unit 156 or the image processing unit 155 has a function of trimming the image within the cleaning effective area 101aa from the detection area 101a.

As described above, the terminal device 200 is an information processing device such as a smartphone, a tablet terminal, or a PC owned by the user of the oral cavity cleaning device 100, and can communicate with the oral cavity cleaning device 100 via the communication unit 201. . The terminal device 200 includes a communication section 201 , a terminal input section 202 and a terminal display section 203 . Since the functions of these configurations are as described above, descriptions thereof are omitted here.

[2. Operation of Oral Cleaning Device]
Next, the operation (oral cleaning method) of the oral cavity cleaning apparatus 100 configured as described above will be described with reference to FIGS. 3 to 6. FIG. FIG. 3 is a flow chart showing an operation example of the mouthwash device according to the embodiment. FIG. 4 is a flow chart showing an operation example in the plaque detection injection mode of the oral cavity cleaning apparatus according to the embodiment.

First, when the operation of the oral cavity cleaning device 100 is started, the injection control unit 153 reads the normal injection intensity of the cleaning liquid preset by the user (S101). For example, the injection control unit 153 makes an inquiry to the memory unit 154 to read the normal injection intensity stored as the set value in the storage device. The injection control unit 153 then reads the normal injection intensity via the memory unit 154 . The normal injection intensity includes the flow rate and injection pressure of the cleaning liquid, and is a value used as a default value when plaque 99a does not adhere. If the set value of the normal injection intensity is not stored in the storage device, the initial value of the normal injection intensity is read, or the user is notified to input the set value of the normal injection intensity. Output.

Next, the input unit 104 accepts ON/OFF switching input for turning on the plaque detection injection mode from the user (S102). When there is no input, that is, when it is determined that the plaque detection injection mode is not ON (No in S103), the cleaning liquid is always injected at the normal injection strength (S104, also called normal injection mode). ). The process is then terminated.

On the other hand, when the input unit 104 receives an input from the user for turning on the plaque detection injection mode, that is, when it is determined that the plaque detection injection mode is ON (Yes in S103), The washing liquid is injected in the plaque detection injection mode (S105), and the process ends.

The above step S105 will be explained in detail in FIG. As shown in FIG. 4, when the result of step S103 is Yes and injection in the plaque detection injection mode is started, the injection control unit 153 reads the plaque detection injection strength of the cleaning liquid preset by the user ( S201). For example, the injection control unit 153 makes an inquiry to the memory unit 154 to read the plaque detection injection intensity stored as the set value in the storage device. The injection control unit 153 then reads the plaque detection injection intensity via the memory unit 154 . The plaque detection injection strength includes the flow rate and injection pressure of the cleaning liquid, and is a value used when plaque 99a adheres. If the set value of the plaque detection jet strength is not stored in the storage device, the initial value of the plaque detection jet strength is read, or the user inputs the set value of the plaque detection jet strength. Output a notification prompting you to do so.

Here, the normal injection intensity and the plaque detection injection intensity will be described with reference to FIG. FIG. 5 is a graph showing the relationship between the detection result and the control amount in the mouthwash device according to the embodiment. As shown in FIG. 5, the control amount (in this case, including both the flow rate and the injection pressure of the cleaning liquid) is larger in the plaque detection injection intensity than in the normal injection intensity. In such a switching of the injection intensity, the larger the area and the amount of adhesion of the plaque 99a, the greater the control amount, which is set with respect to the area and the amount of adhesion of the plaque 99a. The washing liquid is jetted at a plaque detection jetting intensity of .

In other words, the greater the amount of plaque 99a adhered, the greater the jetting intensity, so that cleaning can be performed intensively. Here, the area and amount of adhered dental plaque 99a will be described. In the present embodiment, the area of the plaque portion to which the plaque 99a adheres within the detection region 101a is used as the amount of plaque 99a. If this area is large, it can be estimated that a large amount of dental plaque 99a is adhered within the detection area.

Then, in order to remove the plaque 99a that has adhered to a relatively large amount, if the cleaning liquid is jetted with a stronger jetting strength, it becomes possible to make the jetting strength appropriate from the viewpoint of the removal efficiency of the dental plaque 99a. Whether plaque 99a adheres or not is determined by whether or not the brightness of the color corresponding to the fluorescence wavelength exceeds the brightness threshold in each pixel of the captured image. Then, if the number of pixels exceeding the luminance threshold exceeds the numerical value corresponding to the threshold in FIG. 5, injection is performed at the flow rate and injection pressure of the cleaning liquid at the plaque removal injection altitude. It should be noted that the brightness threshold is also affected by conditions such as pigmentation, and is therefore preferably set for each user.

As another method for calculating the amount of plaque 99a adhered, the amount of plaque 99a adhered within the detection area 101a is calculated from the total value of the brightness of the color corresponding to the fluorescence wavelength of each pixel within the detection area 101a. can also be estimated. Since the brightness of the color corresponding to the fluorescence wavelength in each pixel corresponds to the accumulated amount of metabolites at the position of the actual tooth 99 surface corresponding to that pixel, the entire detection region 101a of the brightness of the color corresponding to the fluorescence wavelength is considered to be correlated with the amount of plaque 99a adhered, the amount of plaque 99a may be calculated by such a method. In this case, if the total luminance value exceeds the numerical value corresponding to the threshold value in FIG. 5, injection is performed at the flow rate and injection pressure of the cleaning liquid at the plaque removal injection altitude. Further, the injection intensity control amount may be determined by using each of the area and the luminance total value in parallel.

FIG. 6 is another graph showing the relationship between the detection result and the control amount in the mouthwash device according to the embodiment. Although the example of FIG. 5 shows a relationship in which the injection intensity changes abruptly when the total area and luminance values exceed the threshold values, for example, as shown in FIG. may be a function that changes continuously. For example, as shown in (a) of FIG. 6, the injection intensity is rapidly increased in sections where the area and the total brightness value are relatively small, and the injection strength is increased gently in the section where the area and the total brightness value are relatively large. A logarithmic function-like relationship such as Alternatively, for example, as shown in FIG. 6(b), a proportional function relationship may be applied that similarly increases the injection intensity in both sections where the area and total luminance value are relatively small and sections where the total luminance value is relatively large. good. Further, for example, as shown in FIG. 6C, the injection intensity is gradually increased in sections where the area and total luminance value are relatively small, and the injection intensity is rapidly increased in sections where the area and total luminance value are relatively large. An ascending exponential-like relationship may be applied.

Referring to FIG. 4 again, after step S201, the LED lighting unit 102 irradiates the irradiation region with irradiation light containing light having a wavelength that excites metabolites (S202). Then, in the irradiated area, the dental plaque portion to which the dental plaque 99a adheres emits fluorescent light when the excited metabolite returns to the ground state, resulting in a selectively colored state.

The camera unit 101 detects this fluorescence as detection light on an extension line extending in a predetermined direction from the injection port 103a, that is, within a detection area 101a including a position corresponding to the surface of the tooth 99 in the direction in which the cleaning liquid is injected. do. That is, an image containing fluorescence is captured (S203). Since the detection here is performed in the two-dimensional detection area 101a, it is possible to obtain an image that can be comprehensively evaluated within the detection area 101a, such as the area and the total brightness value as described above. can. Note that the operation of the cleaning liquid injection unit 103 is prohibited so that the cleaning liquid is not injected during a prohibited period that at least partially overlaps with the detection period from when the camera unit 101 starts capturing an image until it ends. may When the cleansing liquid is sprayed, droplets of the cleansing liquid scatter in the oral cavity, which may cause optical effects. Therefore, it is effective to prohibit the operation of the cleaning liquid injection unit 103 so that the cleaning liquid is not injected during the prohibition period as described above.

If the area and brightness values are less than the threshold (No in S204), the cleaning liquid is jetted to the detection region 101a at normal jetting strength (S205). If the area and brightness values are equal to or greater than the threshold values (Yes in S204), the cleaning liquid is jetted onto the detection area 101a with the plaque detection jetting strength (that is, strong jetting strength) (S206). After steps S205 and S206, the user moves his or her hand while holding oral cavity cleaning device 100 to perform the same operation on another detection region 101a. Therefore, it is determined whether or not the user has finished cleaning (S207), and if it is determined that the user has finished cleaning, for example, by pressing an end button (not shown) (Yes in S207). , terminate the process. If the user continues washing as it is (No in S207), the process returns to step S202 and repeats the same process.

In this way, the metabolites of the bacteria contained in the dental plaque 99a are selectively colored by fluorescence emission, thereby selectively detecting the attachment sites where the dental plaque 99a is attached. This is done for a two-dimensionally spreading area. As a result, since the cleaning liquid can be jetted at a high jetting strength (flow rate and jetting) to a region where a large amount of dental plaque 99a is adhered, it is possible to appropriately control the jetting strength from the viewpoint of removing the dental plaque 99a. becomes. By repeatedly performing such cleaning and detection while changing the position, it is possible to obtain a high cleaning effect over a wide range of teeth.

[3. Modification 1 of oral cavity cleaning device]
Next, an oral cavity cleaning device according to Modification 1 will be described with reference to FIGS. 7 to 9. FIG. In Modification 1 described below, differences from the above-described embodiment will be described, and descriptions of substantially the same points as the embodiment will be omitted.

FIG. 7 is a block diagram showing the functional configuration of an oral cavity cleaning device etc. according to Modification 1 of the embodiment. As shown in FIG. 7, Modification 1 differs from the embodiment in that oral cavity cleaning device 100a includes malfunction detection section 157 .

The malfunction detection unit 157 estimates the state of the gingiva and controls the jetting of the washing liquid so that the jetting intensity is in accordance with the state of the gingiva. The disorder detection unit 157 is connected to the image processing unit 155, acquires the converted image, and estimates the state of the gingiva based on this image. Note that the malfunction detection unit 157 may acquire the pre-conversion image from the camera control unit 151 .

The disorder detection unit 157 has a pre-learned machine learning model in which the correlation between the gingival part shown in the image and the presence or absence of a disorder of the gingival part is learned. By inputting the acquired image into the machine learning model, the disorder detection unit 157 can obtain the presence or absence of the disorder of the gum part as an output when the gum part is shown in the image. Here, the gingival condition is, for example, at least one of swelling and bleeding in the gingiva. In addition, the state of the gingival portion that is clinically regarded as unsatisfactory may also be included as the unsatisfactory condition. In that case, a machine learning model may be trained using an image showing the affected gingiva and correct data indicating that the problem exists as training data.

Then, if the cleaning liquid is jetted with a high injection intensity to the detection region 101a including the gingiva causing such a malfunction, the malfunction of the gingiva may be exacerbated. Therefore, in Modification 1, when a malfunction is detected in the gingiva, the jetting strength of the cleaning liquid to the detection region 101a is weakened (for example, the jetting strength is weaker than the normal jetting strength). It is possible to suppress the deterioration of the disorder. Specifically, the oral cavity cleaning device 100a operates as follows.

FIG. 8 is a flow chart showing an operation example of the oral cavity cleaning device according to Modification 1 of the embodiment. FIG. 9 is a flowchart showing an operation example in the plaque detection injection mode and the malfunction detection injection mode of the mouthwash apparatus according to Modification 1 of the embodiment.

As shown in FIG. 8, steps S301 and S302 correspond to steps S101 and S102 in FIG. 3, so description thereof will be omitted here. After step S302, the input unit 104 further receives an ON/OFF switching input for turning on the malfunction detection injection mode from the user (S303). Since step S304 corresponds to step S103 in FIG. 3, description thereof is omitted here. If No in step S304, the process proceeds to step S305, and if Yes in step S304, the process proceeds to step S308, in which it is determined whether or not there is an input for turning on the malfunction detection injection mode.

If No in step S305, both the plaque detection injection mode and the malfunction detection injection mode are OFF, so the washing liquid is always injected at the normal injection strength (S306, also referred to as the normal injection mode). The process is then terminated. If Yes in step S305, since the plaque detection injection mode is OFF and the malfunction detection injection mode is ON, cleaning liquid is injected in the malfunction detection injection mode (S307), and the process ends.

If No in step S308, since the plaque detection injection mode is ON and the malfunction detection injection mode is OFF, cleaning liquid is injected in the plaque detection injection mode (S309), and the process ends. If Yes in step S308, both the plaque detection injection mode and the malfunction detection injection mode are ON, so cleaning liquid is injected in both the plaque detection injection mode and the malfunction detection injection mode (S310), and the process ends.

The above step S310 will be explained in detail in FIG. As shown in FIG. 9, when the result of step S308 is YES and the injection in the plaque detection injection mode and malfunction detection mode is started, the injection control unit 153 controls the plaque detection injection of the washing liquid preset by the user. The intensity and malfunction detection injection intensity are read (S201a). For example, the ejection control unit 153 makes an inquiry to the memory unit 154 to read the plaque detection ejection intensity and the malfunction detection ejection intensity stored as set values in the storage device. The injection control unit 153 then reads the plaque detection injection strength and the malfunction detection injection strength via the memory unit 154 . The malfunction detection injection strength includes the injection frequency in addition to the flow rate and injection pressure of the cleaning liquid, and is a value used when there is a malfunction in the gingiva portion within the detection region 101a. As for the frequency of injection, by setting the injection frequency appropriately according to the condition of the gingiva, in other words, by performing an appropriate intermittent injection according to the condition of the gingiva, it is possible to , It has the effect of suppressing the damage that would further aggravate the condition. If the set value of the malfunction detection injection intensity is not stored in the storage device, the initial value of the malfunction detection injection intensity is read, or the user is prompted to input the set value of the malfunction detection injection intensity. Output a prompting notification.

Then, similarly to step S202 in FIG. 4, the LED lighting unit 102 irradiates the irradiation region with irradiation light containing light having a wavelength that excites metabolites (S202a). The irradiation light here includes white light, and it is possible to take not only fluorescence images but also normal images. Then, by photographing the inside of the oral cavity including the detection light (S203a), the plaque portion is selectively colored, and when the gingival portion is captured in the detection area 101a, the gingival portion is detected. An image can be obtained from which the state can be estimated.

Then, the determination in step S204 of FIG. 4 is performed. If No in step S204, the process proceeds to step S401. If Yes in step S204, the process proceeds to step S402. be. When it is determined that a malfunction has been detected in the gingival portion (Yes in S401 or S402), the cleaning liquid is jetted onto the detection area 101a at the malfunction detection jetting intensity (S403).

Others are the same as steps S205 to S207 in FIG. 4, so the description is omitted. Here, an example has been described in which, if a malfunction is detected in the gingival portion, injection is performed at the malfunction detection injection strength even if the amount of plaque 99a adhered, that is, the sum of the area and brightness is greater than or equal to the threshold value. In this case, it can be said that the malfunction detection injection mode has priority over the plaque detection injection mode. For example, if priority is given to the plaque detection injection mode, step S402 may not be executed, and if Yes in step S204, the process may proceed to step S206.

In the above description, the operation when both the plaque detection injection mode and the malfunction detection injection mode are ON has been described. Steps S204, S402, and S206 of are omitted. Therefore, after step S203a, the process proceeds to step S401, where it is determined whether or not a malfunction has been detected in the gingival portion. Then, depending on the determination result, the cleaning liquid is jetted at either the normal jetting strength or the malfunction detection jetting strength.

[4. Modification 2 of oral cavity cleaning device]
Next, an oral cavity cleaning device according to Modification 2 will be described with reference to FIG. 10 . It should be noted that in Modification 2 described below, differences from Modification 1 of the above-described embodiment will be described, and descriptions of substantially the same points as Modification 1 of the embodiment will be omitted.

FIG. 10 is a block diagram showing the functional configuration of an oral cavity cleaning device etc. according to Modification 2 of the embodiment. The mouthwash device 100b according to Modification 2 differs from the mouthwash device 100a according to Modification 1 above in that the mouthwash device 100b includes a determination unit 158 and an output unit 159 .

The determination unit 158 and the output unit 159 determine whether or not the light necessary for generating an image for determining the ejection intensity of the cleaning liquid is properly reaching. It is a functional unit for notifying that. More specifically, the determination unit 158 obtains an image after conversion from the image processing unit, and determines whether the state in which the overall luminance of the image has decreased for a certain period of time or more (brightness of detected light is a predetermined value or less for a predetermined time or longer), (i) detected light between the detection region 101a and the camera unit 101, or (ii) illumination between the LED illumination unit 102 and the irradiation region It is determined that there is a shield that blocks at least one of the lights. Such shields include, for example, oral structures such as lips, teeth, and gums, as well as dust, foreign matter, and the like. In the presence of such a shield, neither plaque detection nor gingival malformation can be accurately detected. In the oral cavity cleaning device 100b according to this modified example, the output unit 159 outputs the determination result indicating the existence of the shield, and for example, the terminal display unit 203 of the terminal device 200 displays it as an image to notify the user. .

Upon seeing this notification, the user can change the posture of the oral cavity cleaning device 100b, remove the shield, etc., so that various detections can be performed normally.

[5. Image processing method, image processing apparatus, and program for image processing method]
[5-1. Background technology and problems]
Japanese Patent Application Laid-Open No. 2002-300002 discloses an intraoral camera system that captures images of teeth in the oral cavity.

With such an intraoral camera system, it is desired to be able to easily identify a dental plaque area, which is an area where dental plaque adheres, in a tooth image.

Therefore, the following provides an image processing method that can easily identify the plaque area in the image of the tooth.

[5-2. means]
The image processing method according to the following aspect acquires a first RGB image obtained by photographing the tooth and dental plaque that undergo fluorescence reaction by irradiating the tooth with light including a wavelength range of blue light. and generating a second RGB image by performing image processing including first image processing on the first RGB image, wherein the first image processing is performed on a plurality of regions of the tooth in the RGB image to be processed. a first red pixel average value of a plurality of red pixel values possessed by the first pixels of, a first green pixel average value of a plurality of green pixel values possessed by the plurality of first pixels, and a first green pixel average value of the plurality of green pixel values possessed by the plurality of first pixels A process of adjusting the gains of at least two of the red, green, and blue components of the RGB image to be processed so that the first blue pixel average value of the plurality of blue pixel values is equal. is.

In addition, these general or specific aspects may be realized by a system, device, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media. Also, the recording medium may be a non-temporary recording medium.

[5-3. effect]
The image processing method and the like described below can easily identify the plaque region in the image of the tooth.

[5-4. Form for implementation]
In the image processing method according to the first aspect described below, the first RGB obtained by photographing the tooth and dental plaque undergoing fluorescence reaction by irradiating the tooth with light including the wavelength region of blue light an image is acquired, and image processing including first image processing is performed on the first RGB image to generate a second RGB image, and the first image processing is performed on the tooth region in the RGB image to be processed. a first red pixel average value of a plurality of red pixel values possessed by the plurality of first pixels constituting the first pixels, a first green pixel average value of a plurality of green pixel values possessed by the plurality of first pixels, and the plurality of first A gain of at least two color components out of the red, green, and blue components of the RGB image to be processed so that the first blue pixel average value of the plurality of blue pixel values of one pixel is equal to the first blue pixel average value. This is the process of adjusting the

According to this, by performing the first image processing, it is possible to adjust the white balance of the first RGB image in which the fluorescently reacting tooth is captured. Therefore, it is possible to generate a second RGB image that makes it easy to distinguish the plaque region, which is the region of the tooth to which dental plaque adheres. Therefore, it is possible to easily identify the plaque region in the image of the tooth. As a result, for example, by taking an image of the tooth after brushing and specifying the plaque area, it is possible to present the user with the area left unbrushed.

An image processing method according to a second aspect described below is the image processing method according to the first aspect, further comprising generating an HSV image by converting the color space of the second RGB image into the HSV space. , one or more of a plurality of fourth pixels of the HSV image that satisfy at least one of a saturation within a first predetermined range, a hue within a second predetermined range, and a lightness within a third predetermined range A fourth RGB image is generated by specifying a specific pixel region in which the fourth pixel of is located, and performing saturation enhancement processing on the specific pixel region in the second RGB image.

According to this, since the specific pixel region as the dental plaque region is specified in the second RGB image and the saturation enhancement processing is performed on the specific pixel region, the third RGB image is further generated in which the plaque region can be easily distinguished. be able to. Therefore, it is possible to easily identify the plaque region in the image of the tooth.

An image processing method according to a third aspect described below is the image processing method according to the first aspect, further comprising generating an HSV image by converting the color space of the second RGB image into the HSV space. , one or more of a plurality of fourth pixels of the HSV image that satisfy at least one of a saturation within a first predetermined range, a hue within a second predetermined range, and a lightness within a third predetermined range A fourth RGB image is generated by specifying a specific pixel region in which the fourth pixel of is located and replacing the specific pixel region in the second RGB image with a predetermined pattern.

According to this, since the specific pixel area as the dental plaque area is specified in the second RGB image and the specific pixel area is replaced with the predetermined pattern, it is possible to further generate the fourth RGB image in which the dental plaque area can be easily distinguished. Therefore, it is possible to easily identify the plaque region in the image of the tooth.

An image processing method according to a fourth aspect described below is an image processing method according to any one aspect of the first aspect to the third aspect, wherein the image processing further includes a second image processing wherein the second image processing includes gain so that an average value of a plurality of index values calculated from a plurality of second pixel values respectively possessed by a plurality of second pixels included in the RGB image to be processed becomes a predetermined value is determined and the determined gain is applied to the RGB image to be processed to generate a third RGB image, and the first image processing is a process to process the third RGB image.

According to this, since the exposure control processing is performed by adjusting the gain of the luminance values of the plurality of second pixels, the conditions of the plurality of luminance values can be kept constant even if the shooting conditions vary. That is, since the condition of the luminance distribution of the third RGB image to be processed for the first image processing can be made constant regardless of the photographing conditions, the first image processing can be performed more effectively.

An image processing method according to a fifth aspect described below is the image processing method according to the fourth aspect, wherein the average value of the plurality of index values is the red component, the green component, and the , is the mean value of the color component with the largest mean value among the blue components.

An image processing method according to a sixth aspect, which will be described below, is the image processing method according to the fourth aspect, wherein the plurality of index values are the second pixel values for each of the plurality of second pixel values. It is obtained by calculation based on the red pixel value, green pixel value, and blue pixel value included in the value.

An image processing method according to a seventh aspect described below is the image processing method according to the fourth aspect, wherein the plurality of second pixels are the plurality of third pixels forming the first RGB image. , the pixel value of the maximum color component is less than the first threshold and the pixel value of the minimum color component is less than or equal to the second threshold.

Therefore, it is possible to perform the second image processing by excluding areas strongly affected by the reflection of the irradiated light in the first RGB image.

An image processing method according to an eighth aspect described below is an image processing method according to any one aspect of the first aspect to the third aspect, and further comprises an oral cavity in which the first RGB image is taken. An intraoral position is detected, and the intraoral position and the second RGB image are associated and stored.

Therefore, it is possible to easily identify the intraoral position of the tooth included in the second RGB image or the fourth RGB image.

An image processing apparatus according to a ninth aspect described below is provided by irradiating a tooth with light including a wavelength region of blue light, and photographing the tooth and dental plaque undergoing a fluorescence reaction to obtain first RGB an acquisition unit that acquires an image; and a generation unit that generates a second RGB image by performing image processing including first image processing on the first RGB image, wherein the first image processing is performed on an object to be processed. A first red pixel average value of a plurality of red pixel values of the plurality of first pixels constituting the tooth region in the RGB image, and a first green of a plurality of green pixel values of the plurality of first pixels. The red, green, and blue components of the RGB image to be processed are adjusted such that the pixel average value is equal to the first blue pixel average value of the plurality of blue pixel values of the plurality of first pixels. This is processing for adjusting the gains of at least two of the color components.

A program according to a tenth aspect described below is a program for causing a computer to execute the image processing method according to any one of the first to eighth aspects.

In addition, these generic or specific aspects may be realized by a system, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. and any combination of recording media.

The form will be described in detail below with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art.

It should be noted that the inventors provide the accompanying drawings and the following description for the full understanding of those skilled in the art and are intended to limit the subject matter of the claims. not a thing

[5-5. Description of form]
FIG. 11A is a perspective view of an intraoral camera in an intraoral camera system according to the present description; As shown in FIG. 11A, the intraoral camera 1010 has a toothbrush-like housing that can be handled with one hand. It includes a gripping handle portion 1010b and a neck portion 1010c connecting the head portion 1010a and the handle portion 1010b.

The imaging optical system 1012 is incorporated in the head portion 1010a and the neck portion 1010c. The imaging optical system 1012 includes an imaging element 1014 and a lens arranged on its optical axis LA.

As shown in FIG. 11B, in this description, the imaging optical system 1012 of the intraoral camera 1010 is incorporated in the head portion 1010a and the neck portion 1010c. The imaging optical system 1012 includes an imaging element 1014 and a lens 1016 arranged on its optical axis LA.

The imaging element 1014 is a photographing device such as a C-MOS sensor or a CCD element, and an image of the tooth D is formed by a lens 1016 . The imaging element 1014 outputs a signal (image data) corresponding to the formed image to the outside.

The lens 1016 is, for example, a condensing lens, and forms an incident image of the tooth D on the imaging device 1014 . Note that the lens 1016 may be one lens, or may be a lens group composed of a plurality of lenses.

Here, the imaging optical system 1012 further includes a mirror 1018 that reflects the image of the tooth D toward the lens 1016, a blue light cut filter (blue blocking element) 1020 arranged between the mirror 1018 and the lens 1016, It includes an aperture 1024 positioned between the lens 1016 and the image sensor 1014 .

A mirror 1018 is arranged on the optical axis LA of the imaging optical system 1012 so as to reflect the image of the tooth D that has passed through the entrance 1012 a of the imaging optical system 1012 toward the lens 1016 .

The blue light cut filter 1020 is a filter that cuts the blue wavelength light component contained in the light incident on the imaging device 1014 . In the case of detecting dental plaque by irradiating a tooth with light including a wavelength range of blue light, if the light including a wavelength range of blue light is increased in order to intensify excitation fluorescence of dental plaque, the entire first RGB image will be tinged with blue. . In this state, the blue pixel value becomes dominant over the red pixel value and the green pixel value, so the plaque area can be easily distinguished by performing image processing (exposure control processing and white balance adjustment processing) to be described later. The effect may become smaller. As a countermeasure, the blue light cut filter 1020 cuts the light including the blue light wavelength region from the light before entering the image sensor 1014 .

The diaphragm 1024 is a plate-like member having a through hole on the optical axis LA of the imaging optical system 1012, and achieves a deep depth of focus. As a result, the depth direction of the oral cavity can be focused, and a row-of-teeth image with a clear outline can be obtained.

In addition, the intraoral camera 1010 is equipped with a plurality of first to fourth LEDs 1026A to 1026D as lighting devices for irradiating light onto the tooth D to be imaged during imaging. The first to fourth LEDs 1026A-1026D are, for example, blue LEDs. Also, as shown in FIG. 11A, here, first to fourth LEDs 1026A-1026D are arranged to surround entrance 1012a. A light-transmitting cover 1028 covering the first to fourth LEDs 1026A to 1026D and the entrance 1012a is provided so that the gums G and the like do not come into contact with the first to fourth LEDs 1026A to 1026D and the illumination light is not insufficient. It is provided in the head portion 1010a. A part of the first to fourth LEDs 1026A to 1026D may be white LEDs. By using white LEDs for some of the first through fourth LEDs 1026A-1026D, the first RGB image can be made brighter and the balance of blue pixel values to red and green pixel values can be improved.

Furthermore, in the case of this description, the intraoral camera 1010 has a composition adjustment mechanism 1030 and a focus adjustment mechanism 1032, as shown in FIG. 11B.

The composition adjustment mechanism 1030 is composed of a housing 1034 that holds the imaging element 1014 and the lens 1016, and an actuator 1036 that moves the housing 1034 in the extending direction of the optical axis LA. By adjusting the position of the housing 1034 with the actuator 1036, the angle of view is adjusted, that is, the size of the row of teeth imaged on the imaging device 1014 is adjusted. Note that the composition adjustment mechanism 1030 automatically adjusts the position of the housing 1034 so that, for example, one entire tooth appears in the photographed image. Also, the composition adjustment mechanism 1030 adjusts the position of the housing 1034 based on the user's operation so that the angle of view desired by the user is obtained.

The focus adjustment mechanism 1032 is held within the housing 1034 of the composition adjustment mechanism 1030 and is composed of a lens holder 1038 that holds the lens 1016 and an actuator 1040 that moves the lens holder 1038 in the extending direction of the optical axis LA. The actuator 1040 adjusts the relative position of the lens holder 1038 with respect to the imaging element 1014 to adjust the focus, ie focus. Note that the focus adjustment mechanism 1032 automatically adjusts the position of the lens holder 1038 so that, for example, the tooth located in the center of the photographed image is in focus. Also, the focus adjustment mechanism 1032 adjusts the position of the lens holder 1038 based on the user's operation.

Note that the components of the imaging optical system 1012 excluding the mirror 1018 may be provided on the handle portion 1010b of the intraoral camera 1010.

The imaging element 1014 is a photographing device such as a C-MOS sensor or a CCD element, and a tooth image is formed by a lens. The imaging element 1014 outputs a signal (image data) corresponding to the formed image to the outside. The image output by the imaging device 1014 is an RGB image in which each of a plurality of pixels forming the image has RGB sub-pixels.

In addition, the intraoral camera 1010 is equipped with a plurality of first to fourth LEDs 1026A to 1026D as illumination devices for irradiating light onto the tooth to be imaged during imaging. The first to fourth LEDs 1026A to 1026D are, for example, blue LEDs that emit blue light having a peak wavelength of 405 nm. The first to fourth LEDs 1026A to 1026D are not limited to blue LEDs as long as they are light sources that emit light including the wavelength range of blue light.

FIG. 12 is a schematic configuration diagram of the intraoral camera system according to this description. As shown in FIG. 12, the intraoral camera system according to the present description is generally configured to photograph a row of teeth using an intraoral camera 1010 and to perform image processing on the photographed image. ing.

As shown in FIG. 12, the intraoral camera system includes an intraoral camera 1010, a mobile terminal 1070, and a cloud server 1080. The mobile terminal 1070 is, for example, a smart phone or a tablet terminal capable of wireless communication. The mobile terminal 1070 includes, as an input device and an output device, a touch screen 1072 capable of displaying a row-of-teeth image, for example. The mobile terminal 1070 functions as a user interface of the intraoral camera system.

The cloud server 1080 is a server that can communicate with the mobile terminal 1070 via the Internet or the like, and provides the mobile terminal 1070 with an application for using the intraoral camera 1010 . For example, a user downloads an application from the cloud server 1080 and installs it on the mobile terminal 1070 . The cloud server 1080 also acquires the row-of-teeth image captured by the intraoral camera 1010 via the mobile terminal 1070 .

The intraoral camera system controls a central control unit 1050 as the main parts that control the system, an LED control unit 1054 that controls a plurality of LEDs 1026A to 1026D, an actuator 1036 for the composition adjustment mechanism, and an actuator 1040 for the focus adjustment mechanism. A lens driver 1056 and a position sensor 1090 are included.

The intraoral camera system also has a wireless communication module 1058 that wirelessly communicates with the mobile terminal 1070, and a power control unit 1060 that supplies power to the central control unit 1050 and the like.

The central control section 1050 of the intraoral camera system is mounted on the handle section 1010b of the intraoral camera 1010, for example. For example, the central control unit 1050 also includes a controller 1062 such as a CPU or MPU that executes various processes described later, and a memory 1064 such as RAM or ROM that stores programs for causing the controller 1062 to execute various processes. contains. In addition to the programs, the memory 1064 stores a row-of-teeth image (image data) captured by the imaging device 1014, various setting data, and the like. The row-of-teeth image captured by the imaging device 1014 is an example of the first RGB image.

The controller 1062 transmits the row of teeth image output from the imaging device 1014 to the mobile terminal 1070 via the wireless communication module 1058 . Portable terminal 1070 displays the transmitted row-of-teeth image on touch screen 1072, thereby presenting the row-of-teeth image to the user.

The LED control unit 1054 is mounted, for example, on the handle portion 1010b of the intraoral camera 1010, and performs lighting and extinguishing of the first to fourth LEDs 1026A to 1026D based on control signals from the controller 1062. The LED control unit 1054 is configured by, for example, a circuit. For example, when the user performs an operation on the touch screen 1072 of the mobile device 1070 to activate the intraoral camera 1010 , a corresponding signal is sent from the mobile device 1070 to the controller 1062 via the wireless communication module 1058 . Controller 1062 transmits a control signal to LED control section 1054 to light first to fourth LEDs 1026A to 1026D based on the received signal.

The lens driver 1056 is mounted, for example, on the handle portion 1010b of the intraoral camera 1010, and controls the actuator 1036 of the composition adjustment mechanism and the actuator 1040 of the focus adjustment mechanism based on control signals from the controller 1062 of the central control unit 1050. . The lens driver 1056 is configured by, for example, a circuit. For example, when the user performs an operation related to composition adjustment or focus adjustment on the touch screen 1072 of the mobile terminal 1070 , a corresponding signal is transmitted from the mobile terminal 1070 to the central control unit 1050 via the wireless communication module 1058 . Controller 1062 of central control unit 1050 transmits a control signal to lens driver 1056 so as to execute composition adjustment and focus adjustment based on the received signal. Further, for example, the controller 1062 calculates control amounts for the

actuators

1036 and 1040 necessary for composition adjustment and focus adjustment based on the row-of-teeth image from the imaging element 1014, and the control signals corresponding to the calculated control amounts are output to the lens driver. 1056.

The wireless communication module 1058 is mounted on the handle portion 1010b of the intraoral camera 1010, for example, and performs wireless communication with the mobile terminal 1070 based on the control signal from the controller 1062. The wireless communication module 1058 performs wireless communication with the mobile terminal 1070 in compliance with existing communication standards such as WiFi (registered trademark) and Bluetooth (registered trademark). A row-of-teeth image showing teeth D is transmitted from the intraoral camera 1010 to the mobile terminal 1070 via the wireless communication module 1058 , and an operation signal is transmitted from the mobile terminal 1070 to the intraoral camera 1010 .

The power control unit 1060 is mounted on the handle portion 1010b of the intraoral camera 1010 in this description, and distributes the power of the battery 1066 to the central control unit 1050, the LED control unit 1054, the lens driver 1056, and the wireless communication module 1058. do. The power control unit 1060 is configured by, for example, a circuit. In the case of this description, the battery 1066 is a rechargeable secondary battery that is wirelessly charged by an external charger 1069 connected to a commercial power source via a coil 1068 mounted on the intraoral camera 1010. be.

The position sensor 1090 is a sensor for detecting the orientation and position of the intraoral camera 1010, and is, for example, a multi-axis (three axes x, y, and z here) acceleration sensor. For example, position sensor 1090 may be a six-axis sensor having a three-axis acceleration sensor and a three-axis gyro sensor. For example, as shown in FIG. 11A, the z-axis coincides with the optical axis LA. The y-axis is parallel to the imaging plane and extends longitudinally of the intraoral camera 1010 . Also, the x-axis is parallel to the imaging plane and orthogonal to the y-axis. The output of each axis of position sensor 1090 may be transmitted to mobile terminal 1070 via central controller 1050 and wireless communication module 1058 .

As the position sensor 1090, a piezoresistive type, capacitance type, or thermal detection type MEMS sensor may be used. Further, although not shown, it is preferable to provide a correction circuit for correcting the sensitivity balance of the sensors on each axis, temperature characteristics of sensitivity, temperature drift, and the like. Also, a band-pass filter (low-pass filter) for removing dynamic acceleration components and noise may be provided. Further, noise may be reduced by smoothing the output waveform of the acceleration sensor.

Next, the intraoral imaging operation of the intraoral camera system will be explained. FIG. 13 is a diagram showing the flow of intraoral imaging operation in the intraoral camera system. Note that the processing shown in FIG. 13 is, for example, processing performed in real time, and is performed each time image data of one frame or a plurality of frames is obtained.

The user uses the intraoral camera 1010 to photograph the teeth and gums in his/her own oral cavity, thereby generating image data (S1101). Next, the intraoral camera 1010 transmits the captured image data to the mobile terminal 1070 (S1102). Note that the image data may be a moving image or one or a plurality of still images. Moreover, when the image data is a moving image or a plurality of still images, the sensor data may be transmitted for each frame of the moving image or for each still image. Note that when the image data is a moving image, the sensor data may be transmitted every multiple frames.

Also, the transmission of image data may be performed in real time, or may be transmitted collectively after a series of images (for example, images of all teeth in the oral cavity) are performed.

The mobile terminal 1070 performs image processing on the received image data (first RGB image) (S1103), and displays the image data after image processing (S1104).

By using such an intraoral camera system, the user can capture an image of the user's own intraoral cavity with the intraoral camera 1010 and check the intraoral condition displayed on the mobile terminal 1070 . This allows the user to easily check the health condition of his/her teeth.

Also, the mobile terminal 1070 may generate, for example, a three-dimensional model of a plurality of intraoral teeth from a plurality of photographed image data. Also, the mobile terminal 1070 may display an image based on the generated three-dimensional model.

Here, an example in which the mobile terminal 1070 performs image processing of the tooth image will be described, but the intraoral camera 1010 may perform part or all of this processing. The mobile terminal 1070 is an example of an image processing device.

FIG. 14 is a functional block diagram of the mobile terminal 1070. FIG. Portable terminal 1070 includes acquisition unit 10101 , generation unit 10102 , and display unit 10103 .

The acquisition unit 10101 acquires image data (first RGB image) transmitted from the intraoral camera 1010 . The acquisition unit 10101 may acquire sensor data in addition to image data from the intraoral camera 1010 . The first RGB image is an image obtained by irradiating the tooth with light including the wavelength region of blue light by the intraoral camera 1010 and photographing the tooth undergoing fluorescence reaction.

The generation unit 10102 performs exposure control processing (second image processing) on the first RGB image to generate a third RGB image, and performs white balance adjustment processing (first image processing) on the third RGB image. may generate a second RGB image.

(Exposure control processing)
In the exposure control process, the generation unit 10102 first extracts a plurality of pixels whose RGB values satisfy the following formulas 1 and 2 from among a plurality of first RGB pixels (third pixels) forming the first RGB image. do.

min (R, G, B) ≤ Ths, and, max (R, G, B) < Thmax (Formula 1)
Gmax−G≦Thb (Formula 2)

min(R, G, B) indicates the minimum value among the pixel values of the three RGB sub-pixels (that is, the red pixel value, the green pixel value, and the blue pixel value) of the first RGB pixel.

Ths is a threshold for excluding areas (for example, glossy areas) that are strongly affected by the reflection of the illuminating light in the first RGB image. Ths is, for example, 900 in 10-bit representation.

"max(R, G, B)" indicates the maximum value among the pixel values of the three RGB sub-pixels (that is, the red pixel value, the green pixel value, and the blue pixel value) of the first RGB pixel.

　Thmax indicates the maximum value that the pixel value can take. Thmax is represented by 1023 in 10-bit representation, for example. Thmax is an example of the first threshold.

　Gmax is the maximum value of a plurality of green pixel values in the first RGB image. That is, it is the pixel value of the green pixel having the maximum pixel value among the green pixels of the plurality of first RGB pixels forming the first RGB image. Thb is a threshold for extracting the green pixel of the second pixel from the first RGB pixels. If the value of Thb is increased, the image becomes too bright, so for example, it is set to a value of 10 or less in 10-bit representation.

The glossy area is excluded by Equation 1, and the tooth area in the first RGB image is extracted by Equation 2. That is, the plurality of pixels extracted by Equations 1 and 2 are the plurality of second pixels forming the tooth region. In this way, the plurality of second pixels are such that the pixel value max (R, G, B) of the maximum color component among the plurality of first RGB pixels (third pixels) forming the first RGB image is the first threshold value ( Thmax) and the pixel value min(R, G, B) of the minimum color component is equal to or smaller than the second threshold (Ths).

The generation unit 10102 calculates the average value of the green pixels of the plurality of second pixels, and determines the gain by which the pixel values of the three sub-pixels of RGB are multiplied according to the calculated average value of the green pixels. The generation unit 10102 determines the gain by which the pixel values of the three sub-pixels of RGB are multiplied, for example, using Equation 3 below. The gain is obtained by dividing the target pixel value by the average value of the green pixels. The generating unit 10102 generates the third RGB image by multiplying each of the plurality of first RGB pixels forming the first RGB image by the determined gain. More specifically, the generation unit 10102 generates the third RGB image by multiplying the pixel values of the three sub-pixels of each of the plurality of first RGB pixels by the determined gain. In other words, the pixel values of the plurality of third RGB pixels forming the third RGB image are pixel values calculated by multiplying the pixel values of the plurality of first RGB pixels forming the first RGB image by the determined gain. If the pixel value exceeds the maximum value (1023 in the case of 10-bit representation) by multiplying by the gain, the generating unit 10102 replaces the pixel value with 1023.

In the above description, the average value of the green pixels of the plurality of second pixels extracted from the first RGB pixels is calculated by Equation 2, and the pixel values of the three sub-pixels of RGB are calculated according to the calculated average value of the green pixels. Although the gain to be multiplied is determined, it is not limited to this. An average value of the red pixels of the plurality of second pixels extracted from the first RGB pixels is calculated, and a gain to be multiplied by the pixel values of the three sub-pixels of RGB is determined according to the calculated average value of the red pixels. good. Similarly, the average value of the blue pixels of the plurality of second pixels extracted from the first RGB pixels is calculated, and the gain to be multiplied by the pixel values of the three sub-pixels of RGB is determined according to the calculated average value of the blue pixels. You may

As described above, the exposure control process is performed based on the plurality of second pixel values of the plurality of second pixels (pixels corresponding to the tooth region) included in the RGB image to be processed (here, the first RGB image). determining gains for the plurality of second pixel values so that the average value of the calculated plurality of index values becomes a predetermined value, and determining the gains for the plurality of first RGB pixel values of the plurality of first RGB pixels included in the first RGB image; is applied to generate a third RGB image. Note that the index value may be a value calculated from pixel values of three sub-pixels of RGB forming one pixel, or may be a pixel value of any one of the three sub-pixels. Here, the average value of the plurality of index values is the average value of the color component having the maximum pixel value among the red, green and blue components of the first RGB image. Further, the color component having the maximum average value is the first red pixel average value of the plurality of red pixel values of the plurality of first pixels forming the first RGB image, and the plurality of green pixel values of the plurality of first pixels. and the first blue pixel average value of the plurality of blue pixel values of the plurality of first pixels. Note that the color component having the maximum average value may be fixed to the green component without calculating and comparing the first red pixel average value, the first green pixel average value, and the first blue pixel average value. may have been determined.

Note that the generation unit 10102 calculates the average value of the green pixels of the plurality of second pixels and determines the gain according to the calculated average value of the green pixels in determining the gain, but the present invention is not limited to this. The generation unit 10102 may calculate the average value of the luminance values of the plurality of second pixels as the average value of the plurality of index values, and determine the gain according to the calculated average value of the luminance values. Thus, the index value may be the pixel value of any one of the three sub-pixels of RGB forming one pixel, or a value calculated from the pixel values of the three sub-pixels. good too. Specifically, the generation unit 10102 calculates the luminance value of each of the plurality of second pixels using the sub-pixel values of the three sub-pixels of the second pixel. For example, the generation unit 10102 calculates the luminance value using Equation 3 below.

　Y=0.21*R+0.72*G+0.07*B (Formula 3)

In Equation 3, Y is the luminance value, R is the red pixel value, G is the green pixel value, and B is the blue pixel value.

In this way, the plurality of luminance values may be obtained by calculating each of the plurality of pixel values based on the red pixel value, green pixel value, and blue pixel value included in the pixel value.

(White balance adjustment processing)
In the white balance adjustment process, the generation unit 10102 extracts a plurality of pixels whose RGB values satisfy Expression 1 and Expression 4 below, among the plurality of third RGB pixels forming the third RGB image to be processed.

　Thl ≤ Y ≤ Thu (Formula 4)

In Expression 4, Thl is a threshold value indicating the lower limit value of the tooth region, and Thu is a threshold value indicating the upper limit value of the tooth region.

The tooth region in the third RGB image is extracted by Equation 4. That is, the plurality of pixels extracted by Equations 1 and 4 are the plurality of second pixels forming the tooth region.

Then, the generation unit 10102 generates a first red pixel average value Rave that is the average value of the plurality of red pixel values in the tooth region that satisfies Equations 1 and 4, and the average value of the plurality of green pixel values in the tooth region. and a first blue pixel average value Bave that is the average value of a plurality of blue pixel values in the tooth region. Then, the generation unit 10102 generates the red component and the green component of the RGB image to be processed so that the first red pixel average value Rave, the first green pixel average value Gave, and the first blue pixel average value Bave are equal to each other. , and adjusting the gain of at least two color components of the blue component.

Specifically, the generation unit 10102 calculates the gain of the plurality of red pixel values (red pixel gain) by dividing the first green pixel average value Gave by the first red pixel average value Rave. The generating unit 10102 also calculates gains of a plurality of blue pixel values (blue pixel gains) by dividing the first green pixel average value Gave by the first blue pixel average value Bave. Then, the generation unit 10102 multiplies each red pixel of the plurality of third RGB pixels forming the third RGB image by the red pixel gain, and multiplies each of the plurality of third RGB pixels of the blue pixel by the blue pixel gain. Thus, a second RGB image is generated. In other words, the pixel values of the plurality of second RGB pixels forming the second RGB image are obtained by multiplying the red pixel values of the plurality of third RGB pixels forming the third RGB image by the red pixel gain and It is a pixel value calculated by multiplying a blue pixel value by a blue pixel gain. Note that the generation unit 10102 performs white balance adjustment by calculating the red pixel gain and the blue pixel gain based on the green pixel average value and multiplying the pixel value of the corresponding color component by each gain. Without being limited to this, the green pixel gain and the blue pixel gain may be calculated using the red pixel average value as a reference, or the red pixel gain and the green pixel gain may be calculated using the blue pixel average value as a reference. You may

Note that the generation unit 10102 replaces the pixel value with 1023 when the pixel value exceeds the maximum value (1023 in the case of 10-bit representation) by multiplying by the gain.

The generation unit 10102 may also perform the following third image processing on the second RGB image to emphasize the plaque area within the tooth area in the second RGB image. Specifically, the generation unit 10102 generates the HSV image by converting the color space of the second RGB image into the HSV space. Then, the generation unit 10102 determines that the saturation is within a first predetermined range (for example, 30 or more and 80 or less in 8-bit expression) and the hue is within a second predetermined range (for example, 8-bit) among the plurality of fourth pixels of the HSV image. 140 or more and 170 or less in expression), and one or more fourth pixels whose brightness satisfies at least one of a third predetermined range (for example, 100 or more and 180 or less in 8-bit expression). Identify it as a dirt area. The first predetermined range, the second predetermined range, and the third predetermined range may be specified by comparing the actual plaque region, the tooth region, and the HSV image. It is not limited. The generation unit 10102 generates a fourth RGB image by performing saturation enhancement processing on the dental plaque region in the second RGB image. Note that the generation unit 10102 may generate the fourth RGB image by replacing the plaque region with a predetermined pattern instead of performing the saturation enhancement process. The predetermined pattern may be, for example, graphics having constant pixel values or graphics including a specific pattern.

A display unit 10103 is a display device included in the mobile terminal 1070 and displays an image after image processing has been performed by the generation unit 10102 . The display unit 10103 may display the second RGB image or may display the fourth RGB image.

(Comparison of color difference data between the plaque region and the tooth region detected from the first RGB image under various irradiation lights)
(1) When the oral cavity is irradiated only with blue light (peak wavelength is 405 nm) FIG. It is a figure which shows the state which is. FIG. 16A is a diagram showing an example of a first RGB image of anterior teeth photographed in the state shown in FIG. 15 by irradiating the oral cavity with blue light (peak wavelength: 405 nm).

FIG. 16B is a diagram showing an example of color difference data between the plaque area 10201 detected in FIG. 16A and the tooth area. In order to compare with the color difference data of the dental plaque region 10201 and the tooth region detected from the first RGB image of the front teeth photographed by cutting the light including the wavelength region of blue light by a blue light cut filter described later, FIG. 11B shows the color difference data of the plaque region 10201 and the tooth region detected from the first RGB image captured without the blue light cut filter 1020 shown in 11B. For reference, in FIG. 16B, the average color difference coordinates of the hue of the plaque region were (0.16, 0.06), and the average color difference coordinates of the hue of the tooth region were (0.25, −0.02). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and the sensitivity of the camera.

As is also known from the quantitative visible light-induced fluorescence method (QLF method), it is known that bacteria in dental plaque fluoresce reddish pink when blue light is irradiated. Further, it is known that when a tooth is irradiated with light including a blue light wavelength region, excitation fluorescence is emitted from the dentin, which passes through the enamel layer and emits green light. On the other hand, since lips and gums are bluish, teeth and plaque can be easily distinguished from lips and gums.

(2) When light including the blue light wavelength region is cut by a blue light cut filter A blue light cut filter is effective as a method for reducing noise. As shown in FIG. 11B , the blue light cut filter 1020 can cut light including the blue light wavelength region from the light before being reflected from the tooth and entering the imaging element 1014 .

FIG. 16C is a diagram showing an example of color difference data between the plaque region 10201 and the tooth region detected from the first RGB image of the front tooth photographed with the blue light cut filter 1020 cutting the light including the wavelength region of blue light. . For reference, in FIG. 16C, the average color difference coordinates of the hue of the plaque area were (0.058, 0.068), and the average color difference coordinates of the hue of the tooth area were (0.050, −0.004). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and the sensitivity of the camera.

Comparing FIG. 16B and FIG. 16C, by cutting light including the wavelength range of blue light with a blue light cut filter, the saturation of blue is lowered (the color is lighter). Since the plaque region fluoresces reddish pink, the brightness of the tooth region can be suppressed while maintaining the brightness of the plaque region. As a result, by reducing the blue pixel value and reducing the difference between the red pixel value and the green pixel value, the white balance gain multiplied by the red pixel value and the green pixel value can be reduced, thereby suppressing noise. can get.

(3) When the oral cavity is irradiated with blue light (peak wavelength is 405 nm) and white light By adding white light, the red pixel value and green pixel value of the tooth region in the first RGB image are increased, and the blue light The exposure correction gain can be kept smaller than in the case of irradiating the oral cavity with only one. However, as the intensity of the white light is increased, the difference between the hue of the plaque region and the hue of the gums and lips becomes smaller, which tends to cause false detection of the gums and lips as plaque. Therefore, the ratio of the intensity of light including the wavelength range of blue light and the intensity of white light must be optimized in advance in order to avoid erroneous detection.

In order to irradiate the oral cavity with blue light and white light, part of the first to fourth LEDs 1026A to 1026D in FIG. 11 may be replaced with white LEDs.

FIG. 16D is a diagram showing an example of color difference data between the plaque region 10201 and the tooth region detected from the first RGB image of the front teeth captured by irradiating the oral cavity with blue light (peak wavelength: 405 nm) and white light. . In order to compare with the plaque region 10201 detected from the first RGB image of the front tooth photographed by cutting the light including the wavelength region of blue light by the blue light cut filter described above and the color difference data of the tooth region (Fig. 16C), Fig. 16D shows the color difference data of the plaque region 10201 and the tooth region detected from the first RGB image captured without the blue light cut filter 1020 shown in FIG. 11B. For reference, in FIG. 16D, the average color difference coordinates of the hue of the plaque area were (0.089, 0.041), and the average color difference coordinates of the hue of the tooth area were (0.15, −0.013). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and white light and the sensitivity of the camera.

Comparing FIG. 16B and FIG. 16D, it can be confirmed that by irradiating the oral cavity with white light in addition to blue light, the brightness of the bluish tooth region is suppressed while maintaining the brightness of the plaque region. . As a result, by reducing the blue pixel value and reducing the difference between the red pixel value and the green pixel value, the white balance gain multiplied by the red pixel value and the green pixel value can be reduced, thereby reducing noise. can get.

Here, the tooth region can be specified by using the color difference data of the tooth region stored according to the lighting conditions for irradiating the tooth. Further, a region having a high luminance and a hue within a predetermined range can be extracted as a tooth region from within a white-hued region surrounded by a red-hued region. As described above, according to the method of detecting a white-hue region surrounded by a red-hue region, a red-hue value close to the color of the lips and gums and a white-hue value of the teeth are detected. It can be roughly detected.

(White balance processing)
By performing image processing (exposure control processing and white balance adjustment processing), the mobile terminal 1070 can generate a second RGB image 10210 in which the plaque region 10211 can be easily distinguished as shown in FIG. 17A. The image processing substantially decolorizes the tooth region, making it easier to distinguish the plaque region 10211 .

FIG. 17B shows the difference between the plaque region and the tooth region after performing the exposure control processing and the white balance adjustment processing when the teeth are irradiated with light including the wavelength range of blue light without a blue light cut filter. It is a figure which shows an example of color difference data. Tooth regions are considered substantially white. For reference, the color difference average coordinates of the hue of the plaque region in FIG. 17B were (0.033, 0.071). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and the sensitivity of the camera.

Similarly, FIG. 17C shows the difference between the plaque region and the tooth after performing exposure control processing and white balance adjustment processing when the tooth is irradiated with light including the wavelength range of blue light using a blue light cut filter. FIG. 4 is a diagram showing an example of color difference data of regions; Tooth regions are considered substantially white. For reference, the color difference average coordinates of the hue of the plaque region in FIG. 17C were (0.033, 0.071). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and the sensitivity of the camera.

Similarly, FIG. 17D shows, without a blue light cut filter, when the teeth are irradiated with light including a wavelength range of blue light and white light, and after performing exposure control processing and white balance adjustment processing, plaque FIG. 10 is a diagram showing an example of color difference data between a region and a tooth region; Tooth regions are considered substantially white. For reference, the color difference average coordinates of the hue of the plaque area in FIG. 17D were (0.014, 0.048). The color difference average coordinates of these hues are reference values, and are values that may change under the influence of the intensity of blue light and white light and the sensitivity of the camera.

From FIGS. 17B to 17D, regardless of the type of light source that irradiates the oral cavity, the hue of the plaque region can be displayed in almost the same reddish pink color.

(highlighting of plaque area)
FIG. 18 is a diagram showing an example of the fourth RGB image of the front teeth photographed in the state shown in FIG. FIG. 19 is a diagram showing another example of the fourth RGB image of the front teeth photographed in the state shown in FIG.

The mobile terminal 1070 acquires the first RGB image 10200 from the intraoral camera 10 as shown in FIG. 16A.

Further, the portable terminal 1070 further performs saturation enhancement processing on the plaque region 10211 of the second RGB image 10210 to generate a fourth RGB image 10220 in which the plaque region 10221 can be easily distinguished as shown in FIG. can do.

On the other hand, portable terminal 1070 further replaces plaque region 10211 of second RGB image 10210 with a predetermined pattern, thereby generating fourth RGB image 10230 in which plaque region 10231 is replaced with a predetermined pattern as shown in FIG. can do.

(Image processing flow)
FIG. 20 is a flowchart of image processing in the mobile terminal 1070. FIG.

The mobile terminal 1070 acquires the first RGB image from the intraoral camera 1010 (S1111).

Next, the mobile terminal 1070 generates a third RGB image by performing exposure control processing on the first RGB image (S1112).

Next, the mobile terminal 1070 generates a second RGB image by performing white balance adjustment processing on the third RGB image (S1113).

Next, the mobile terminal 1070 identifies the plaque area in the second RGB image (S1114).

Next, the mobile terminal 1070 generates a fourth RGB image by performing saturation enhancement processing on the plaque area of the second RGB image or by replacing the plaque area with a predetermined pattern (S1115).

The image processing apparatus (for example, the mobile terminal 1070) according to the above description includes an acquisition unit 10101 and a generation unit 10102. The acquisition unit 10101 acquires a first RGB image obtained by photographing a tooth that reacts with fluorescence by irradiating the tooth with light including a wavelength range of blue light. The generation unit 10102 generates a second RGB image by performing image processing including the first image processing on the first RGB image. In the first image processing, in an RGB image to be processed, a first red pixel average value of a plurality of red pixel values of a plurality of first pixels constituting a tooth region, and a plurality of red pixel values of a plurality of first pixels. The red component, green component, and A process of adjusting the gain of at least two color components of the blue component.

Further, in the image processing apparatus (for example, the mobile terminal 1070) according to the above description, the HSV image is generated by further converting the color space of the second RGB image into the HSV space, and the plurality of fourth pixels of the HSV image are generated. A specific pixel region in which one or more fourth pixels satisfying at least one of the following: saturation within a first predetermined range, hue within a second predetermined range, and brightness within a third predetermined range A fourth RGB image is generated by specifying and performing saturation enhancement processing on a specified pixel region in the second RGB image.

Further, in the image processing apparatus (for example, the mobile terminal 1070) according to the above description, the HSV image is generated by further converting the color space of the second RGB image into the HSV space, and the plurality of fourth pixels of the HSV image are generated. A specific pixel region in which one or more fourth pixels satisfying at least one of the following: saturation within a first predetermined range, hue within a second predetermined range, and brightness within a third predetermined range A fourth RGB image is generated by identifying and replacing the specific pixel area in the second RGB image with a predetermined pattern.

Also, in the image processing apparatus (for example, the mobile terminal 1070) according to the above description, the image processing further includes a second image processing. In the second image processing, a gain is determined so that an average value of a plurality of luminance values calculated from a plurality of second pixel values of a plurality of second pixels included in an RGB image to be processed becomes a predetermined value, This is a process of generating a third RGB image by applying the determined gain to the RGB image to be processed. The first image processing is processing for processing the third RGB image.

Further, in the image processing device (for example, the mobile terminal 1070) according to the above description, the average value of the plurality of luminance values is the maximum average value among the red, green, and blue components of the first RGB image. It is the average value of the color components.

Further, in the image processing apparatus (for example, the mobile terminal 1070) according to the above description, the plurality of luminance values are obtained by determining, for each of the plurality of pixel values, the red pixel value, the green pixel value, and the blue pixel value included in the pixel value. It is obtained by calculation based on the pixel value.

Further, in the image processing apparatus (for example, the mobile terminal 1070) according to the above description, the plurality of second pixels has the pixel value of the largest color component among the plurality of third pixels forming the first RGB image. It is a pixel that satisfies that the pixel value of the minimum color component is smaller than the threshold and is equal to or less than the second threshold.

A modified example of the above description will be described below.

[5-6. Modification 1]
In the above description, the generation unit 10102 performs exposure control processing on the first RGB image and performs white balance adjustment processing on the third RGB image generated by the exposure control processing. , exposure control processing may not be performed. For example, exposure control processing may not be performed as long as the first RGB image is obtained in which variations in luminance distribution are reduced. For example, by performing lighting control so that the shooting conditions are constant, variations in the brightness distribution of the obtained first RGB image may be reduced.

[5-7. Modification 2]
In the above description, when the user takes an intraoral image using the intraoral camera 1010, the display unit 10103 of the mobile terminal 1070 may display guidance for designating the intraoral position to be imaged. Guidance may be output by voice from a speaker (not shown) of mobile terminal 1070 . Accordingly, the user can follow the guidance to move the intraoral camera 1010 to the specified intraoral position so that the image is captured at the specified intraoral position. For example, the intraoral position is the position of the front teeth, the position of the back teeth, and the like.

Then, the portable terminal 1070 generates a second RGB image generated by image processing performed on the intraoral position indicated by the guidance and the first RGB image acquired while the intraoral position is indicated. Alternatively, it may be stored in association with the fourth RGB image.

[5-8. Modification 3]
In the above description, the intraoral camera 1010 transmits the first RGB image to the mobile terminal 1070, and the mobile terminal 1070 performs image processing on the first RGB image. The image may be transmitted to the cloud server 1080 , the cloud server 1080 may perform the above image processing, and the second RGB image or the fourth RGB image of the image processing result may be transmitted to the mobile terminal 1070 . In this case, the first RGB image may be transmitted from intraoral camera 1010 to cloud server 1080 without going through mobile terminal 1070, or may be transmitted from intraoral camera 1010 to cloud server 1080 through mobile terminal 1070. may

Although the intraoral camera system according to the present image processing method and the like has been described above, the present image processing method and the like are not limited to this form.

For example, in the above description, an example using the intraoral camera 1010 whose main purpose is to photograph teeth has been described, but the intraoral camera 1010 may be an intraoral care device equipped with a camera. For example, intraoral camera 1010 may be an intraoral cleaner or the like that includes a camera.

Also, each processing unit included in the intraoral camera system according to the above description is typically realized as an LSI, which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them.

In addition, circuit integration is not limited to LSIs, and may be realized with dedicated circuits or general-purpose processors. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of the circuit cells inside the LSI may be used.

Also, in the above description, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Also, this image processing method and the like may be realized as an image display method and the like executed by an intraoral camera system. Also, the present image processing method and the like may be implemented as an intraoral camera, a mobile terminal, or a cloud server included in an intraoral camera system.

Also, the division of functional blocks in the block diagram is an example, and a plurality of functional blocks can be realized as one functional block, one functional block can be divided into a plurality of functional blocks, and some functions can be moved to other functional blocks. may Moreover, single hardware or software may process the functions of a plurality of functional blocks having similar functions in parallel or in a time-sharing manner.

Also, the order in which each step in the flow chart is executed is for illustrative purposes in order to specifically describe the image processing method, etc., and orders other than the above may be used. Also, some of the above steps may be executed concurrently (in parallel) with other steps.

Although the intraoral camera system and the like according to one or more aspects have been described above, the present image processing method and the like are not limited to this aspect. As long as it does not deviate from the gist of the description regarding this image processing method, etc., various modifications that a person skilled in the art can think of, or a form constructed by combining the constituent elements of different forms, are also part of one or more aspects. may be included within the scope.

[5-9. Modification 4]
In the above description, the case where the oral cavity is irradiated with only blue light (peak wavelength is 405 nm) and the case where the oral cavity is irradiated with blue light (peak wavelength is 405 nm) and white light were explained, but this is not the only case. do not have. White light also contains a wavelength that causes excitation fluorescence of dental plaque, and it is possible to detect the excitation fluorescence of dental plaque, although it is faint, with only white light. In other words, the present invention can be implemented using any irradiation light as long as it is configured to irradiate the oral cavity with light containing at least a blue light component (with a wavelength of about 405 nm).

[5-10. Availability]
This image processing method and the like can be applied to an intraoral camera system.

(Other embodiments)
Although the mouthwash device and the like according to the embodiments of the present disclosure have been described above, the present disclosure is not limited to these embodiments.

For example, each processing unit provided in the oral cavity cleaning apparatus according to the embodiment is typically implemented as an LSI, which is an integrated circuit. These may be made into one chip individually, or may be made into one chip so as to include part or all of them.

In addition, in the above embodiments and the like, each component may be configured with dedicated hardware or realized by executing a software program suitable for each component. Each component may be realized by reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory by a program execution unit such as a CPU or processor.

Further, one aspect of the present disclosure may be implemented as a mouthwash method executed by a mouthwash device, a control method of a mouthwash device, or the like. Moreover, one aspect of the present disclosure may be a computer program that causes a computer to execute each characteristic step included in the oral cavity cleaning method or the control method.

In addition, the oral cavity cleaning device according to the above embodiments and the like may be implemented as a single device, or may be implemented as a plurality of devices. When the oral cavity cleaning device is realized by a plurality of devices, each component of the oral cavity cleaning device may be distributed to the plurality of devices in any way. For example, the control device may be included in the mobile terminal. Also, for example, at least one of the functional configurations of the control device may be implemented by a mobile terminal or a server (for example, a cloud server) capable of communicating with the mobile terminal. When the oral cavity cleaning device is implemented by a plurality of devices, the communication method between the plurality of devices is not particularly limited, and may be wireless communication or wired communication. Also, wireless and wired communications may be combined between devices. Note that the cloud server is a server that can communicate with the mobile terminal via the Internet or the like, and may provide the mobile terminal with an application for using the intraoral camera. For example, a user downloads an application from a cloud server and installs it on a mobile terminal. Also, the cloud server may acquire the image captured by the oral cavity cleaning device via the mobile terminal.

Also, the order in which each step in the flowchart is executed is for illustrative purposes in order to specifically describe the present disclosure, and orders other than the above may be used. Also, some of the above steps may be executed concurrently (in parallel) with other steps.

The present disclosure is not limited to the above one or more aspects, and as long as it does not depart from the spirit of the present disclosure, various modifications that can be made by those skilled in the art can be made to the present embodiment, and different embodiments Forms constructed by combining components may also be included within the scope of one or more aspects.

The present disclosure can be applied to oral cleaning devices.

98 Gum 99

Tooth 99a Plaque

100, 100a, 100b, 100c Mouth cleaning device 101 Camera section 101a Detection area 101aa Cleaning effective area 102 LED illumination section 103 Cleaning fluid injection section 103a Injection port 103b Brush 104 Input section 150 Control device 151 Camera control section 152 LED illumination control unit 153 injection control unit 154 memory unit 155 image processing unit 156 plaque adhesion amount detection unit 157 malfunction detection unit 158 determination unit 159 output unit 200 terminal device 201 communication unit 202 terminal input unit 203 terminal display unit 1010 intraoral Camera 1010a Head 1010b Handle 1010c Neck 1012 Photographic optical system 1014 Imaging element 1026A First LED
1026B Second LED
1026C Third LED
1026D 4th LED
1036, 1040 actuator 1050 central control unit 1052 image processing unit 1054 LED control unit 1056 lens driver 1058 wireless communication module 1060 power control unit 1062 controller 1064 memory 1066 battery 1068 coil 1069 charger 1070 mobile terminal 1072 touch screen 1080 cloud Server 10101 Acquisition unit 10102 generation unit 10103 display unit 10200

first RGB images

10201, 10211, 10221, 10231 plaque region 10210

second RGB images

10220, 10230 fourth RGB images

Claims

An oral cleaning device for cleaning teeth,
a cleaning control unit that controls the cleaning force acting on the teeth via the tooth cleaning unit;
a light irradiation unit that irradiates irradiation light onto an irradiation region that includes a cleaning effective region having a cleaning effect by the tooth cleaning unit;
a light detection unit that detects detection light emitted in response to the irradiation light within a two-dimensional detection region that includes the effective cleaning region and is included in the irradiation region;
The cleaning control unit
estimating a dental plaque portion within the detection area based on the detection result of the light detection unit;
determining a control amount of the detergency based on the estimation result of the plaque portion;
An oral cavity cleaning apparatus that controls the tooth cleaning unit so that the cleansing power of the tooth cleaning unit becomes the determined control amount.
the tooth cleaning unit is a brush,
The oral cavity cleaning apparatus according to claim 1, wherein the cleaning control section includes a drive control section that controls a drive mode of the brush as the cleaning power of the brush.
The tooth washing unit is a water jet type washing liquid spraying unit that causes the washing liquid sprayed from the injection port toward the teeth to collide with the teeth,
The oral cavity cleaning apparatus according to claim 1, wherein the cleaning control unit functions as an injection control unit that controls at least one of the flow rate of the cleaning liquid and the injection pressure of the cleaning liquid as the cleaning power of the cleaning liquid injection section.
The injection control unit is
based on the detection result, calculating the area of the plaque portion in the detection region by the area of the portion where the detection light is detected in the detection region;
The oral cavity cleaning apparatus according to claim 3, wherein the control amount of at least one of the flow rate and the injection pressure of the cleaning liquid injected from the injection port is determined so as to increase as the calculated area increases.
The injection control unit is
calculating, based on the detection result, the amount of plaque adhered within the detection area from the luminance value of the detection light within the detection area;
The oral cavity cleaning apparatus according to claim 3, wherein a control amount of at least one of flow rate and injection pressure of the cleaning liquid injected from the injection port is determined so as to increase as the calculated adhesion amount increases.
the irradiation light includes light having a wavelength that excites the metabolites of bacteria contained in the dental plaque portion;
The mouthwash device according to claim 1, wherein the detection light is fluorescence emitted by the excited metabolite.
The injection control unit inhibits injection of the cleaning liquid from the injection port during a prohibition period that at least partially overlaps with a detection period from when the light detection unit starts to finishes detecting the detection light. A mouthwash device according to claim 3.
(i) the detected light between the detection region and the photodetector, or (ii) the light irradiation unit and the A determination unit that determines that there is a shield that blocks at least one of the irradiation light between the irradiation area and the irradiation area;
The oral cavity cleaning device according to claim 1, further comprising an output section that outputs the determination result of the determination section.
The cleaning control unit
estimating the plaque portion in a cleaning effective area, which is a part of the detection area and has a cleaning effect by the tooth cleaning unit, based on the detection result of the light detection unit;
The oral cavity cleaning device according to claim 1, wherein the control amount of the cleaning power is determined based on the estimation result of the plaque portion.
A control device for an oral cavity cleaning device that cleans teeth with a tooth cleaning unit,
a cleaning control unit for controlling the cleaning force applied to the teeth by the tooth cleaning unit;
The detection light emitted in response to the irradiation light applied to the irradiation area including the cleaning effective area having the cleaning effect by the tooth cleaning unit is detected in a two-dimensional shape including the cleaning effective area and included in the irradiation area. an acquisition unit that acquires detection results detected within the area,
The cleaning control unit
estimating a plaque portion within the detection area based on the acquired detection result;
determining a control amount of the cleaning force acting on the tooth via the tooth cleaning unit based on the estimation result of the plaque portion;
A control device that controls the detergency so that it becomes the determined control amount.
A program for causing a computer to execute a control method for an oral cavity cleaning device that cleans teeth with a tooth cleaning unit,
In the control method,
The detection light emitted in response to the irradiation light applied to the irradiation area including the cleaning effective area having the cleaning effect by the tooth cleaning unit is detected in a two-dimensional shape including the cleaning effective area and included in the irradiation area. Get the detection result detected in the area,
estimating a plaque portion within the detection area based on the acquired detection result;
determining a control amount of the cleaning force acting on the tooth via the tooth cleaning unit based on the estimation result of the plaque portion;
A program for controlling the detergency to be the determined control amount.
An oral cleaning method using an oral cleaning device for cleaning teeth with a tooth cleaning unit,
The detection light emitted in response to the irradiation light applied to the irradiation area including the cleaning effective area having the cleaning effect by the tooth cleaning unit is detected in a two-dimensional shape including the cleaning effective area and included in the irradiation area. Get the detection result detected in the area,
estimating a plaque portion within the detection area based on the acquired detection result;
determining a control amount of the cleaning force acting on the tooth via the tooth cleaning unit based on the estimation result of the plaque portion;
A method for cleaning the oral cavity, wherein the detergency is controlled to be the determined control amount.