JP2014053791A - Image processing device, image processing program and image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device, an image processing program and an image pickup device with which a subject can be recognized by using an image from which an effect of external light components is reduced while displaying an easy-to-see picture.SOLUTION: An image processing device 10 has a recognition image generator 14 which uses images of a subject picked up under first and second illumination states respectively out of images picked up by a camera 20 to generate a recognition image from which an effect of external light components other than illumination is reduced, an image recognizing unit 16 for recognizing the subject by using the recognition image, and a display controller 18 for displaying a display image comprising the image picked up under the first illumination state out of the images picked up by the camera 20.

Description

  The present invention relates to an image processing apparatus, an image processing program, and an image photographing apparatus that display an image including a photographed image and perform image recognition for recognizing a subject in the photographed image.

  In dispensing pharmacies and the like, prior to delivery of a drug to a patient, an audit is performed to confirm the type and dosage of the drug. In recent years, at the time of this audit, the prescribed medicine is photographed, and the picture is output to a predetermined display device to be shown to the patient, and the explanation of the medicine is sometimes performed. In such a case, if the medicine is photographed with a so-called document camera, the pharmacist can explain the medicine with a simple operation while looking at the hand, which is convenient. Note that a document camera is a device that normally includes illumination, irradiates a subject with illumination light, and photographs the subject from above.

  In addition, regarding the drug audit, in order to confirm whether the dispensed medicine matches the medicine specified in the prescription, an image recognition technique is used to check A technique for collating drug images registered in a database in advance has been proposed (see, for example, Patent Document 1). Therefore, it is also assumed that the drug is identified by image recognition while presenting an image of the drug to be delivered to the patient with the above-described document camera.

  However, when image recognition is also performed on a photographed drug image, if a document camera is used, the photographed image is affected by external light other than illumination light of the document camera, and it is difficult to perform image recognition with high accuracy. There is a risk. In such a case, a correction for removing the influence of a shadow or the like due to an external light component is performed using an image of a subject photographed in the presence of illumination light and an image of a subject photographed in the absence of illumination light. (For example, refer patent document 2) By this, the image more suitable for image recognition can be acquired.

JP 2005-122360 A JP 2001-268326 A

  However, if the lighting of the imaging device is repeatedly turned on / off in order to perform such correction, the luminance of the drug image output in real time on the display device is greatly switched, and an image that is difficult to see for the patient is presented. Become.

  The present invention has been made in view of the above-described problems, and an image processing apparatus and image processing capable of recognizing a subject using an image with reduced influence of an external light component while displaying an easy-to-view video. It is an object to provide a program and an image capturing device.

  The image processing apparatus of the present invention uses an image of a subject photographed in a first illumination state and an image of the subject photographed in a second illumination state different from the first illumination state. And recognizing the subject by using a recognition image generation unit that generates a recognition image in which the influence of an external light component other than the illumination is reduced, and the recognition image generated by the recognition image generation unit. An image recognizing unit, and an image captured in the first illumination state among an image captured in the first illumination state and an image captured in the second illumination state. And a display control unit that displays video.

  According to this configuration, among the captured images, a display image composed of an image captured in the first illumination state is displayed, so that an easily viewable image with a stable luminance level is presented. It will be. On the other hand, a recognition image in which the influence of external light components other than illumination is reduced is generated using an image photographed in the first illumination state and an image photographed in the second illumination state. Since the image recognition is performed based on the recognition image, the subject can be recognized with high accuracy. Therefore, according to this configuration, it is possible to accurately identify the subject using an image in which the influence of the external light component is reduced while displaying an easy-to-view video. Note that the illumination state typically means the illumination on / off state, but also includes the intensity of illuminance when the illumination is on.

  The image processing apparatus according to the present invention includes an image of a subject photographed in the first illumination state, and an image of the subject photographed in a second illumination state different from the first illumination state. And a recognition image generation unit that generates a recognition image in which the influence of an external light component other than the illumination is reduced, and the recognition image generated by the recognition image generation unit, and the subject And a display control unit for displaying a display video composed of the recognition image.

  According to this structure, the image for recognition which reduced the influence of external light components other than illumination using the image image | photographed in the state of 1st illumination, and the image image | photographed in the state of 2nd illumination. Is generated and image recognition is performed based on this recognition image, so that the subject can be recognized with high accuracy. On the other hand, since the display video composed of the recognition image is displayed, an easy-to-view video with a stable luminance level is presented. Therefore, according to this configuration, it is possible to accurately identify the subject using an image in which the influence of the external light component is reduced while displaying an easy-to-view video.

  The image processing apparatus may further include a synchronization control unit that synchronizes the control of the illumination state and the photographing of the image of the subject.

  According to this structure, the image image | photographed in the 1st illumination state and the image image | photographed in the 2nd illumination state can be acquired reliably and easily. Furthermore, it is possible to easily identify whether the image is an image taken when the illumination is in the first state or an image taken when the illumination is in the second state.

  The image processing apparatus may further include an illumination state determination unit that determines the state of illumination when the image is captured based on the captured image.

  According to this configuration, it is possible to generate a recognition image and output a display video independently of lighting control.

  The image processing apparatus further includes a subject information storage unit that stores information about the subject, and the image recognition unit extracts information about the recognized subject from the subject information storage unit, and the display control unit May display the extracted information on the subject together with the display video.

  According to this configuration, since information about the subject can be displayed together with the display video, it is possible to present an image that is easy to see and that is easy to understand about the subject.

  Further, the program of the present invention is an image of a subject photographed in a first illumination state and a subject image photographed in a second illumination state different from the first illumination state. And generating a recognition image in which the influence of external light components other than the illumination is reduced, recognizing the subject using the generated recognition image, and the first A step of displaying a display video composed of an image photographed in the first illumination state out of an image photographed in the illumination state and an image photographed in the second illumination state;

  An image photographing apparatus of the present invention includes a photographing stand for placing a subject, a camera for photographing an image of the subject placed on the photographing stand, and an illumination for irradiating light on the subject placed on the photographing stand. Using the image of the subject photographed in the first illumination state and the image of the subject photographed in the second illumination state different from the first illumination state, A recognition image generation unit that generates a recognition image in which the influence of the light component is reduced; an image recognition unit that recognizes the subject using the recognition image generated by the recognition image generation unit; A display control unit that displays a display image including an image photographed in the first illumination state out of an image photographed in the first illumination state and an image photographed in the second illumination state. It has the composition provided with.

  According to this configuration, among the images photographed by the camera, a display image composed of an image photographed in the first illumination state is displayed, so that an easy-to-see image with a stable luminance level is presented. Will be. On the other hand, a recognition image in which the influence of external light components other than illumination is reduced is generated using an image photographed in the first illumination state and an image photographed in the second illumination. Since image recognition is performed based on the recognition image, the subject can be recognized with high accuracy. Therefore, according to this configuration, it is possible to accurately recognize the subject using an image in which the influence of the external light component is reduced while displaying an easy-to-view video.

  In the image photographing device, the camera photographs the subject from above the photographing table, and the illumination is separated from the photographing table in the upward direction and reflected from the photographing table at the time of lighting. The light may be arranged at a position that does not enter the angle of view of the camera.

  According to this configuration, it is possible to prevent the influence of the reflected light from the imaging stand of the illumination light on the captured image, and to perform image recognition with high accuracy using the recognition image generated from the captured image. Can do.

  The image photographing apparatus includes a plurality of the cameras, and the plurality of cameras includes a first camera provided at a position for photographing the subject from above the photographing table, and a downward direction of the photographing table. And a second camera provided at a position where the subject is photographed.

  According to this configuration, the subject can be photographed from the top or bottom of the photographing stand, so that even if the obtained image differs greatly depending on whether the subject is photographed from above or below, image recognition can be performed with high accuracy. Can do.

  In addition, the image photographing apparatus includes a plurality of the illuminations, the plurality of illuminations are spaced apart upward from the photographing table, and reflected light from the photographing table at the time of lighting is the first camera. Illumination arranged at a position that does not fall within the angle of view of the camera, and a position below the photographing stand, and the reflected light from the photographing stand when turned on does not fall within the angle of view of the second camera. And arranged lighting.

  According to this configuration, it is possible to prevent the influence of the reflected light from the imaging stand of the illumination light on the captured image, and to perform image recognition with high accuracy using the recognition image generated from the captured image. Can do.

  In the image processing apparatus, the display control unit may recognize the subject when the image recognition unit can recognize the subject based on the recognition image generated from an image captured by the first camera. The first camera displays a display video composed of an image captured in the first illumination state, and recognizes the subject based on the recognition image generated from the image captured by the first camera. If it is not possible, the second camera may display a display video composed of an image photographed in the first illumination state.

  According to this configuration, as a result of performing image recognition based on an image photographed from above, a display video composed of an image photographed from above is displayed when the image recognition is possible, and an image photographed from below when it is not possible. For example, when shooting a subject that has a different impression depending on whether it is shot from the front side or from the back side, a video that is easier to understand is presented to the viewer. be able to.

  According to the present invention, the recognition image in which the influence of the external light component other than the illumination is reduced by using the image photographed in the first illumination state and the image photographed in the second illumination state. Is generated, and the subject is recognized based on the recognition image, and the display video composed of the image taken in the first illumination state is displayed. Therefore, the subject is accurately displayed while displaying an easy-to-view video. Can be recognized.

1 is a block diagram showing a configuration of an image processing device and an image photographing device according to a first embodiment of the present invention. The figure which shows an example of the chemical | medical agent information in the 1st Embodiment of this invention 1 is a front view of an image capturing apparatus according to a first embodiment of the present invention. The top view which shows arrangement | positioning of the illumination part with respect to the imaging stand of the image imaging device in the 1st Embodiment of this invention The figure explaining arrangement | positioning of the illumination part with respect to the camera of the imaging device in the 1st Embodiment of this invention Operation flow diagram of image processing apparatus according to first embodiment of the present invention The figure explaining the usage example in the pharmacy of the imaging device in the 1st Embodiment of this invention. The figure which shows an example of the display of the taking instruction | indication using the image processing apparatus in the 1st Embodiment of this invention The block diagram which shows the structure of the image processing apparatus in the 2nd Embodiment of this invention, and an image imaging device. The block diagram which shows the structure of the chemical | medical agent check system in the 3rd Embodiment of this invention. (A) A figure for explaining an example of an application using a medicine check system (b) A figure for explaining an example of an application using a medicine check system The figure which shows an example of the information memorize | stored in a taking history memory | storage part Flow chart explaining the dosage confirmation process in the drug check system The figure which shows an example of the image | photographed medicine image in the medicine check system

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The present invention relates to an image processing device and an image photographing device. In the following embodiment, a case where a medicine is photographed and the medicine is identified from the photographed image will be described.

(First embodiment)
FIG. 1 is a diagram showing a configuration of an image photographing apparatus according to the first embodiment of the present invention. The image capturing apparatus 100 includes an image processing apparatus 10, a camera 20, and an illumination unit 30, and the image processing apparatus 10 includes a synchronization control unit 11, a captured image acquisition unit 12, an illumination information addition unit 13, a recognition image generation unit 14, The medicine information storage unit 15, the image recognition unit 16, the display video generation unit 17, and the display control unit 18 are provided. The illumination unit 30 is connected to the synchronization control unit 11, and the camera 20 is connected to the synchronization control unit 11 and the captured image acquisition unit 12. In addition, the image capturing device 100 constitutes an image display system together with the display device 200 connected to the image processing device 10. Note that the image capturing apparatus 100 of the present embodiment includes two cameras 20 as will be described later.

  The synchronization control unit 11 synchronizes lighting / extinguishing of the lighting unit 30 and shooting of the camera 20. The synchronization control unit 11 outputs information on the lighting / light-off state of the lighting unit 30 and the shooting timing of the camera 20 to the lighting information adding unit 13. The captured image acquisition unit 12 acquires a frame image captured by the camera 20 and outputs the frame image to the illumination information addition unit 13. The illumination information adding unit 13 adds illumination information to the acquired frame image based on the information output from the synchronization control unit 11. This illumination information is metadata indicating whether the frame image is taken when the illumination unit 30 is turned on or taken when the illumination unit 30 is turned off. The frame image to which the metadata is added in this way is output to the recognition image generation unit 14 and the display video generation unit 17.

  The display video generation unit 17 refers to the metadata added to the frame image, discards the frame image captured when the illumination unit 30 is turned off, and turns on the illumination unit 30. A display video to be output to the display device is generated using only the frame image taken at the time of shooting. The generated video is output to the display control unit 18. On the other hand, the recognition image generation unit 14 refers to the metadata added to the frame image, the frame image captured when the illumination unit 30 is turned on, and the frame image captured when the illumination unit 30 is turned off. Are extracted, and the difference between them is extracted to generate a recognition image from which the influence of external light is removed. The generated recognition image is output to the image recognition unit 16.

  The drug information storage unit 15 stores information such as a drug name, a medication method, and a dose as drug information. The image recognition unit 16 performs image recognition on the recognition image output from the recognition image generation unit 14 and identifies a medicine that matches the medicine information stored in the medicine information storage unit 15. The identification of the medicine by the image recognition unit 16 is performed based on the name of the medicine, for example, by extracting a character string having the largest font size including the katakana from the recognition image.

  FIG. 2 is a diagram illustrating an example of data stored in the medicine information storage unit 15. As shown in FIG. 2, in the present embodiment, drug information, medication usage, dose, precautions, and an image of a minimum unit (one tablet, one capsule, one bag, etc.) are stored as drug information. Therefore, when a medicine is specified, it is possible to specify which part is a single dose in the photographed medicine image based on the minimum unit image stored in the medicine information storage unit 15.

  The display control unit 18 outputs the video output from the display video generation unit 17 and the information on the medicine specified by the image recognition unit 16 to the display device 200.

  The illumination unit 30 is a light emitter such as an LED, and when illuminated by the synchronization control unit 11, the illumination unit 30 irradiates light on a subject photographed by the camera 20. The camera 20 shoots the drug that is the subject, generates continuous frame images, and outputs them to the captured image acquisition unit 12.

  Each function of the image processing apparatus 10 described above may be realized by the CPU executing arithmetic processing according to a program written in a ROM or the like. Such a program is also included in the scope of the present invention.

  Next, the appearance of the image capturing apparatus 100 according to the present embodiment will be described with reference to the drawings. FIG. 3 is a front view of image photographing apparatus 100 in the present embodiment. As shown in FIG. 3, the image photographing apparatus 100 includes a photographing stand 40 in addition to the camera 20 and the illumination unit 30. In addition, the illumination unit 30 includes a diffusion plate 31. The imaging table 40 is for placing a medicine to be imaged by the camera 20. As will be described later, since the image capturing apparatus 100 captures the medicine placed on the photographing stand 40 from both the top and bottom directions, the photographing stand 40 is preferably made of a transparent plate material such as glass or acrylic. The diffusion plate 31 is for diffusing light from the illumination unit 30 and is made of, for example, a light diffusion resin such as polycarbonate or polypropylene. Note that a display device 200 is connected to the image capturing device 100.

  When the image capturing apparatus 100 is actually used, the drug that is the subject is not always placed on the imaging stand 40 with the surface suitable for image recognition facing up. Depending on the medicine, the medicine name may not be shown depending on which direction the front side / back side is taken. In addition, it may be easier for a user viewing a drug image to present an image taken from the front side (or the back side). Therefore, as shown in FIG. 3, in the present embodiment, the two cameras 20 a and 20 b are arranged substantially vertically symmetrically with respect to the imaging table 40. Accordingly, the illumination unit 30 is also provided with 30a and 30b on the upper side and the lower side of the imaging table 40, respectively.

  FIG. 4 is a plan view of the image capturing apparatus 100 (excluding the camera 20). As shown in FIG. 4, a plurality of LEDs are arranged in an array along the four sides of the imaging table 40 to constitute the illumination unit 30 a. A diffusion plate 31 is disposed in front of the LED array, that is, in the center direction of the imaging table 40. The same applies to the illumination unit 30b.

  In the image capturing apparatus 100 of the present embodiment, the illumination unit 30 is disposed in the vicinity of the imaging table 40 so as to be substantially parallel to the imaging table 40, as shown in FIG. This is because, depending on the position where the illumination unit 30 is arranged, the reflected light from the photographic stand 40 is reflected in the photographed image, which may make it difficult to see the image displayed on the display device or make image recognition difficult. Because there is. FIG. 5 is a diagram for explaining the arrangement position of the illumination unit 30 in consideration of the influence of the reflected light from the imaging table 40. The illumination units 30a and 30b are preferably arranged at positions where the reflected light does not enter the angle of view of the cameras 20a and 20b. However, it is not possible to sufficiently irradiate the medicine with light near the side of the medicine placed on the placement table 40. Therefore, the illumination unit 30a is spaced above the mounting table 40 in consideration of the thickness of the medicine. Specifically, it is preferably arranged at a height position of around 20 mm. After all, it is preferable that the illumination units 30a and 30b be arranged in the range indicated by the oblique lines in FIG.

  Next, the operation of the image processing apparatus according to the present embodiment will be described with reference to the flowchart of FIG.

  First, the photographed image acquisition unit 12 acquires continuous frame images of a medicine that is a subject from the cameras 20a and 20b (step S61). Next, the illumination information adding unit 13 adds information related to the state of the illumination unit 30 to the acquired frame image (step S62). Subsequently, the recognition image generation unit 14 generates a recognition image, that is, an image in which the influence of the external light component is removed from the images photographed by the cameras 20a and 20b (step S63). At this time, the recognition image generation unit 14 refers to the illumination information added to each frame image by the illumination information addition unit 13. Next, the image recognition unit 16 performs image recognition on each of the recognition image derived from the camera 20a and the recognition image derived from the camera 20b generated in step S63 (step S64). The image recognition unit 16 further collates the character string extracted from each of the recognition image derived from the camera 20a and the recognition image derived from 20b with the information stored in the medicine information storage unit 15, The name of the photographed medicine and the number of medicines (number of minimum units) are specified (step S65).

  Then, in step S65, the image recognition unit 16 determines whether the name of the photographed medicine and the number of medicines are specified from the recognition image derived from the camera 20a (step S66). Depending on the medicine, when photographing from the back side, the name may not be specified, or even if the name can be specified, the number may not be identified. Therefore, when the name of the photographed medicine or the number of medicines cannot be specified from the recognition image derived from the camera 20a (NO in step S66), the image recognition unit 16 determines that the camera 20a is the subject. It is determined that the medicine is photographed from the back side, and the recognition result based on the recognition image derived from the camera 20b is output to the display control unit 18 together with information specifying the camera 20b. In this case, the display control unit 18 displays a video photographed from the front side, and among the frame images photographed by the camera 20b, a frame image having metadata indicating that the illumination unit 30 is “lit”. Then, a display video is generated (step S67).

  On the other hand, when the name and number of the medicine are specified from the recognition image derived from camera 20a (YES in step S66), image recognition unit 16 determines that camera 20a has photographed the medicine from the front side. Then, together with information specifying the camera 20a, the recognition result of the recognition image derived from the camera 20a is output to the display control unit 18. The display control unit 18 generates a display image including a frame image having data indicating that the state of the illumination unit 30 is “lit” among the frame images taken by the camera 20a (step S68). In this way, since the medicine image taken from the front side becomes a display image, it is possible to present an image that is easy for the patient to understand. Then, the display control unit 18 outputs the medicine name and number specified in step S65 and the display video generated in step S67 or step S68 to the display device 200 (step S69).

  FIG. 7 shows an actual usage example of the image capturing apparatus 100 in the pharmacy according to the first embodiment of the present invention. As shown in FIG. 7, the pharmacist places the medicine on the placement table 40 of the image capturing apparatus 100, confirms the information of the identified medicine, and explains the medicine to the patient. The medicine image generated from the frame image when the illumination unit 30 of the image capturing apparatus 100 is turned on is output and displayed on the display device 200 in front of the patient.

  Next, display of the medication instruction and creation of the medication instruction using the image processing apparatus 10 of the present embodiment will be described.

  As described above, the image processing apparatus 10 according to the present embodiment can identify a drug name from a captured image and extract information such as usage, dose, and cautions regarding the identified drug. In addition, it is possible to specify how many units of medicine are included from the captured image. Therefore, as shown in FIG. 7, it is possible to display a medication instruction as shown in FIG. 8 together with the medicine image from the state where the medicine image is displayed on the display device 200.

  In FIG. 8, the medication instruction items are displayed in association with each medicine image. The medication instruction includes the drug name, usage, dose, precautions, and number of days prescribed. Among these, the drug name, usage, dose, and precautions are stored in the drug information storage unit 15. The number of prescription days is calculated by dividing the number of taken medicines (number of units) by the dose at the time of image recognition. Also, the dose for one dose is emphasized by displaying a frame superimposed on the drug image.

  As described above, by using the image processing apparatus of the present invention, it is possible to display an actual medicine image and a medication instruction in association with each other, and the medicine can be easily explained to the patient. Furthermore, if a printer is connected to the image processing apparatus 10, a dosing instruction as shown in FIG. 8 can be easily created. In the conventional medication instructions, the medication actually prescribed may not match the image in the medication instructions due to changes in the packaging of the medication. By using the image processing apparatus according to the present embodiment, it is possible to create a dosing instruction using an image of a drug that is actually prescribed. Based on the instructions, it is possible to easily confirm taking instructions.

  As described above, according to the image processing device 10 and the image capturing device 100 including the image processing device 10 according to the present embodiment, the illumination unit 30 is turned on or turned off in the frame image acquired from the camera 20. The data indicating whether or not it is in the state is added. Then, using a frame image to which data indicating that the illumination unit 30 is turned on and a frame image to which data indicating that the illumination unit 30 is turned off are added are used. Then, a recognition image in which the influence of the external light component is reduced or eliminated is generated, and the photographed drug is specified. In addition, a display video is generated using only the frame image to which data indicating that the illumination unit 30 is lit is added. Therefore, even when a drug image is displayed and output while performing image recognition, it is possible to display an easy-to-view image while realizing accurate image recognition.

(Second Embodiment)
FIG. 9 is a block diagram illustrating configurations of the image processing device 10 and the image capturing device 100 according to the second embodiment of the present invention. As shown in FIG. 9, the image capturing device 100 in the second embodiment includes a synchronization control unit 11. Unlike the image processing device in the first embodiment, the image processing device 10 is synchronized. The control unit 11 is not provided. That is, also in the second embodiment, the lighting unit 30 is turned on / off and the camera 20 is synchronized with the photographing, but the synchronization control unit does not output the information to the lighting information adding unit 13. Instead, the image processing apparatus 10 according to the second embodiment includes an illumination state determination unit 19. The other configuration is the same as that of the first embodiment.

  The image acquisition unit 12 outputs the frame image acquired from the camera 20 to the illumination state determination unit 19. The illumination state determination unit 19 determines whether each of the acquired frame images is an image taken when the illumination unit 30 is turned on or an image taken when the illumination unit 30 is turned off. Such illumination state determination can be performed, for example, by analyzing a luminance histogram of each frame image. The illumination state determination unit 19 outputs data indicating the determination result, that is, data indicating whether the illumination unit 30 is turned on or off, to the illumination information adding unit 13.

  The illumination information adding unit 13 adds the determination result by the illumination state determining unit 19 to the acquired image. The operations of the display video generation unit 17 and the recognition image generation unit 14, that is, the processing of each frame image to which data indicating the illumination state is added are the same as those in the first embodiment.

  Thus, according to the image processing apparatus 10 in the second embodiment, each frame image is in a state in which the illumination unit 30 is lit or in a state in which the illuminating unit 30 is turned off based on the frame image acquired from the camera 20. It is determined in which state the image was taken, and data indicating the state of each illumination is added. Then, using a frame image to which data indicating that the illumination unit 30 is turned on and a frame image to which data indicating that the illumination unit 30 is turned off are added are used. Then, a recognition image in which the influence of the external light component is reduced or eliminated is generated, and the photographed drug is specified. In addition, a display video is generated using only the frame image to which data indicating that the illumination unit 30 is lit is added. Therefore, the illumination unit 30 can be controlled independently of the processing in the image processing apparatus 10, and in this case as well, an easy-to-view video can be displayed while realizing accurate image recognition.

(Third embodiment)
Next, a drug check system for a patient who has received a drug will be described. This system is a system that makes it possible to easily check whether a patient is taking medication correctly and in accordance with the instructions.

  FIG. 10 is a block diagram showing the configuration of the present system. As shown in FIG. 10, this system includes a mobile phone 50 including a camera, such as a smartphone, and a medicine information providing server 60. The mobile phone 50 includes a camera 51, an image acquisition unit 52, an image recognition unit 53, a communication unit 54, a drug information acquisition unit 55, a medication history storage unit 56, a medication status determination unit 57, and a display control unit 58. On the other hand, the drug information providing server 60 includes a communication unit 61, a drug information search unit 62, and a drug information storage unit 63.

  The camera 51 is a normal camera module, and takes an image in response to a shooting instruction via the interface of the mobile phone 50. The image acquisition unit 52 acquires an image captured by the camera 51 and outputs it to the image recognition unit 53. The image recognition unit 53 extracts a character string indicating the medicine name from the image and outputs the character string to the communication unit 54. The communication unit 54 transmits the medicine name extracted from the photographed image to the medicine information providing server 60 and receives the medicine information transmitted from the medicine information providing server 60. The drug information acquisition unit 55 acquires the drug information received by the communication unit 54 and outputs the drug information to the image recognition unit 53, the dose history storage unit 56, and the display control unit 58.

  Based on the acquired drug information, the image recognition unit 53 specifies a portion corresponding to the amount of one shot in the photographed image, further specifies how many days the drug has been prescribed, and the display control unit 58. Output to. The dose history storage unit 56 stores the acquired drug information and the drug image acquired from the image acquisition unit 52 in association with each other. The medication status determination unit 57 determines the patient's medication status based on the medicine image stored in the medication history storage unit 56 and the newly photographed drug image. The display control unit 58 displays the acquired drug information in association with the photographed drug image, and displays the determination result of the taking situation.

  The communication unit 61 receives drug name data transmitted from the mobile phone 50 and transmits drug information to the mobile phone 50. The drug information search unit 62 extracts drug information corresponding to the drug name received by the communication unit 61 from the drug information stored in the drug information storage unit 63. The drug information storage unit 63 may include drug information such as a drug name, usage, dose, precautions, and a minimum unit image, similar to the drug information storage unit 15 described above.

  Next, an example of an application using this system will be described with reference to the drawings. There are cases where it is difficult for the patient to understand, for example, that the instruction manual of the medicine delivered together with the medicine does not include an image of the medicine or the image is different from the actual product. In such a case, the patient activates an application for using the medicine check system on the mobile phone 50.

  When the application is activated, for example, as shown in FIG. 11A, a message prompting the patient to take a medicine is displayed on the screen. When a patient takes an image of a medicine for which a user wants to check a medication instruction by operating the mobile phone 50, the medicine name is extracted from the image. The extracted medicine name is transmitted to the medicine information providing server 60. The medicine information providing server 60 transmits medicine information corresponding to the received medicine name to the mobile phone 50. The mobile phone 50 performs image recognition on the photographed medicine image based on the received medicine information, and the received medicine is displayed on the medicine image photographed by the camera 51 as shown in FIG. Display information in association with each other.

  The cellular phone 50 further stores the medicine image taken by the camera 51 in the medication history storage unit 56. FIG. 12 is a diagram illustrating an example of information stored in the medication history storage unit 56. As illustrated in FIG. 12, the medicine image is stored in the medication history storage unit 56 in association with the photographing date and time and the obtained medicine information. Once the medication information is stored in the medication history storage unit 56, the medication instruction can be confirmed on the mobile phone 50 without being connected to the medication information providing server 60.

  Next, the process of checking the taking status in this application will be described with reference to the flowchart of FIG. 13 and FIG.

  First, an image of a medicine taken after being taken by the camera 51 is acquired (step S131). Next, a character string including the name of the medicine is extracted from the image of the medicine photographed in step S131, and the name of the medicine is specified (step S132). If the name of the medicine can be specified, it is determined whether or not medicine information corresponding to the medicine name is stored in the medication history information storage unit 56 (step S133). If the corresponding medication information is not stored in the medication history storage unit 56 (NO in step S133), the corresponding medication information is acquired from the medication information providing server 60 and stored in the medication history information storage unit 56. Store (step S134) and proceed to step S137. On the other hand, if the corresponding drug information is stored (YES in step S133), the taking status is determined (step S135).

  The determination of the taking situation is performed by comparing the image taken in step S131 with the newest taking history image stored in the taking history storage unit 56. For example, when a medication history image as in the example of FIG. 12 is stored and an image as shown in FIG. 14 is taken in step S131, the drug xxx is reduced by 2 capsules. This can be recognized based on one unit image stored in the medication history storage unit 56. In addition, when the image shown in FIG. 14 is taken at 21:10 on August 31, 2012, it will be from 21:10 on August 31, 2012, which is the photographing time of the taking history image, until 21:10. In the meantime, the drug xxx is reduced by 2 capsules. This can be said to be a reasonable reduction in light of the method and dose of drug xxx. In such a case, it is determined that the medicine xxx has been taken correctly.

  On the other hand, when the image shown in FIG. 14 shows a decrease for three capsules, for example, it cannot be said that the decrease corresponds to the dose of the drug xxx. In such a case, it is determined that the medicine xxx is not being taken correctly. And the message regarding the determination result of step S135 is displayed (step S136). When the medicine to be judged is taken correctly, for example, a message such as “Drug xxx is taken correctly. Please take the remaining 0 days” is displayed. If the drug to be judged is not being taken correctly, for example, a message such as “It seems that you forgot to take a capsule. Check the instructions for taking it.” Is displayed. Finally, the medicine image taken in step S131 is stored in the medication history storage unit 56 (step S137).

  Thus, according to the application using this system, even after receiving the delivery of the medicine, the patient himself / herself can confirm the dosing instruction by a simple operation. At that time, since the dosing instruction is displayed in association with the image taken on the spot, an easy-to-understand dosing instruction can be provided to the patient. Also, it can be confirmed by a simple operation by the patient himself whether or not the delivered medicine has been correctly taken.

  The image recognition unit 53, the medication history storage unit 56, and the medication status determination unit 57 may be provided in the medicine information providing server 60 instead of the mobile phone 50. In this case, the mobile phone 50 is taken. The medicine image may be transmitted to the medicine information providing server 60.

(Other variations of the image processing apparatus of the present invention)
In the image processing apparatus 10 of the present invention, in the above embodiment, the display video is generated using the frame image when the illumination unit 30 is lit, but the illumination unit 30 is turned off for the display video. It may be generated using a frame image when the image is being processed, or may be generated using a recognition image generated by the recognition image generation unit 14. Further, the illumination information is not limited to the two cases where the illumination unit 30 is turned on / off, and may be three or more according to the illuminance of the illumination unit 30.

  Further, in the above embodiment, the synchronization control unit 11 synchronizes the lighting / extinction of the illumination unit 30 and the imaging of the camera 20, but the synchronization control unit 11 may not be provided. By synchronizing the lighting state control and shooting, it is possible to reliably acquire an image shot with the illumination turned on and an image shot with the illumination turned off. Due to the relationship between the rate and the control timing of the illumination state, it is not always possible to acquire an image in the same manner when the synchronization control unit 11 is not provided. That is, generally, the lighting / light-off cycle of the illumination unit 30 does not necessarily match the frame rate of the camera 20 and their timing, so that the light-on and light-off states are mixed during the exposure period of each frame image. In other words, it is conceivable that images having a plurality of levels of illumination intensity can be obtained.

  Therefore, the recognition image generation unit 14 extracts, from the plurality of captured frame images, an image captured with the (highest) illumination intensity and an image captured with the (lowest) illumination intensity. Then, a recognition image is generated. Further, the display video generation unit 17 generates an image for display by extracting an image having a predetermined illumination intensity or higher or an image within a predetermined illumination intensity range.

  In the above embodiment, the medicine is identified by extracting the character string including the medicine name from the recognition image, but is stored in the feature amount of the recognition image and the medicine information storage unit 15. You may identify a chemical | medical agent by the comparison with the feature-value of an image. In the above embodiment, the recognition image with reduced influence of the external light component is generated by taking the difference between the image when the illumination unit 30 is turned on and the image when the illumination unit 30 is turned off. However, the recognition image may be generated by using a method for reducing the influence of the external light component other than this.

  In the above embodiment, the case where the image capturing apparatus 100 includes two cameras 20 has been described. However, the number of cameras 20 may be one, or three or more. Further, in the above embodiment, when image recognition is performed and either the drug name or the number cannot be specified using the recognition image generated based on the image captured by the camera 20a ( In step S66, the display video is generated based on the image photographed by the camera 20b. Further, the drug name and the number of medicines are recognized by the image recognition based on the recognition image photographed by the camera 20b. It may be determined whether the image is specified, and if both are specified, a display video may be generated based on an image captured by the camera 20b. In addition, after adding illumination information to the frame image, first, a display video is generated and displayed based on the image taken by the camera 20a (or the camera 20b), and then image recognition is performed using the recognition image. When the medicine name or number is not specified, a display image is generated based on the image captured by the camera 20b (or the camera 20a), the display image is switched to this, and the image is captured by the camera 20b. Image recognition and identification of the name and number of medicines may be performed using a recognition image derived from the obtained image. Further, the display video may be manually selected without using the image recognition result.

  In the above-described embodiment, the case where the two illumination units 30 are provided has been described. However, the illumination unit 30 may be provided only on either the upper side or the lower side of the imaging table 40. Furthermore, the illumination unit 30 may be provided at a position other than the range described in the above embodiment.

  Furthermore, in the above embodiment, the image display apparatus is configured by the image capturing apparatus 100 and the display apparatus 200 connected thereto, but the image capturing apparatus 100 may include a display apparatus.

  In the above embodiment, the image processing apparatus and the image recognition apparatus of the present invention have been described with respect to a case where a medicine is a subject and a photographed medicine is identified. An image may be generated to identify the subject.

  The image processing apparatus according to the present invention has an effect of accurately recognizing a subject using an image in which the influence of an external light component is reduced while displaying an easy-to-view video, and includes a photographed image. It is useful as an image processing apparatus that displays video and recognizes a subject in a captured image.

DESCRIPTION OF SYMBOLS 100 Image photographing device 10 Image processing device 11 Synchronization control part 12 Photographed image acquisition part 13 Illumination information addition part 14 Recognition image generation part 15 Drug information storage part 16 Image recognition part 17 Display image generation part 18 Display control part 19 Illumination state Determination unit 20 Camera 30 Illumination unit 40 Imaging stand 50 Mobile phone 51 Camera 52 Image acquisition unit 53 Image recognition unit 54 Communication unit 55 Drug information acquisition unit 56 Dosing history storage unit 57 Dosing status determination unit 58 Display control unit 60 Drug information providing server 61 Communication Unit 62 Drug Information Search Unit 63 Drug Information Storage Unit 200 Display Device

Claims (11)

Outside light other than the illumination using an image of the subject photographed in the first illumination state and an image of the subject photographed in the second illumination state different from the first illumination state A recognition image generation unit that generates a recognition image with reduced influence of components;
An image recognition unit for recognizing the subject using the recognition image generated by the recognition image generation unit;
Display control for displaying a display video composed of an image photographed in the first illumination state out of an image photographed in the first illumination state and an image photographed in the second illumination state. And
An image processing apparatus comprising: Outside light other than the illumination using an image of the subject photographed in the first illumination state and an image of the subject photographed in the second illumination state different from the first illumination state A recognition image generation unit that generates a recognition image with reduced influence of components;
An image recognition unit for recognizing the subject using the recognition image generated by the recognition image generation unit;
A display control unit for displaying a display image including the recognition image;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, further comprising a synchronization control unit configured to synchronize control of the illumination state and photographing of the subject image.   The image processing apparatus according to claim 1, further comprising an illumination state determination unit that determines a state of illumination when the image is captured based on the captured image. A subject information storage unit for storing information about the subject;
The image recognition unit extracts information on the recognized subject from the subject information storage unit,
The image processing apparatus according to claim 1, wherein the display control unit displays the extracted information about the subject together with the display video. On the computer,
Outside light other than the illumination using an image of the subject photographed in the first illumination state and an image of the subject photographed in the second illumination state different from the first illumination state Generating a recognition image with reduced influence of components;
Recognizing the subject using the generated recognition image;
Displaying an image for display comprising an image photographed in the first illumination state out of an image photographed in the first illumination state and an image photographed in the second illumination state; ,
A program that executes A shooting stand on which the subject is placed;
A camera for shooting an image of a subject placed on the shooting table;
Illumination for irradiating light on a subject placed on the photographing table;
Outside light other than the illumination using an image of the subject photographed in the first illumination state and an image of the subject photographed in the second illumination state different from the first illumination state A recognition image generation unit that generates a recognition image with reduced influence of components;
An image recognition unit for recognizing the subject using the recognition image generated by the recognition image generation unit;
Display control for displaying a display video composed of an image photographed in the first illumination state out of an image photographed in the first illumination state and an image photographed in the second illumination state. And
An image photographing apparatus comprising: The camera photographs the subject from above the photographing table,
8. The light source according to claim 7, wherein the illumination is spaced apart in an upward direction of the imaging table, and is disposed at a position where reflected light from the imaging table at the time of lighting does not enter an angle of view of the camera. The image photographing device described. A plurality of the cameras;
The plurality of cameras include a first camera provided at a position for photographing the subject from above the photographing table, and a second camera provided at a position for photographing the subject from below the photographing table. The image capturing apparatus according to claim 7, further comprising a camera. Comprising a plurality of the lights,
The plurality of illuminations are spaced apart in the upward direction of the imaging table, and the illumination light that is disposed at a position where reflected light from the imaging table at the time of lighting does not enter the angle of view of the first camera; 10. The lighting device according to claim 9, further comprising: an illumination arranged in a downward direction of the imaging table, and light reflected from the imaging table at the time of lighting is located at a position that does not fall within an angle of view of the second camera. The image photographing device described in 1.   When the image recognition unit is able to recognize the subject based on the recognition image generated from an image captured by the first camera, the display control unit is configured to perform the first camera using the first camera. When a display video composed of an image shot in an illumination state is displayed and the subject cannot be recognized based on the recognition image generated from the image shot by the first camera, the second The image photographing apparatus according to claim 9 or 10, wherein a display image including an image photographed in a state of the first illumination is displayed by a camera.

Images