JP2019132771A - Color reaction observation aid system, color reaction observation aid method, and program - Google Patents

Abstract

To provide a color reaction observation aid system capable of accurately observing determination results of the color reaction without using a special and expensive inspection apparatus even for general users who do not have expertise, when a setting detection time set for each inspection item has elapsed.SOLUTION: A color reaction observation aid system of the present invention includes an observation target surface detection unit for assisting an observation of a color reaction by a subject solution that is added dropwise and determining whether the observation target surface including each of the colorimetric reaction region is included in the captured image in the colorimetric reaction region of the analysis kit, an observation position identification unit for identifying a position of each of the colorimetric reaction region, a dropping detection unit for detecting a dropping of the test solution to the colorimetric reaction region, an elapsed time detection unit for detecting whether the reaction time set in the colorimetric reaction region has elapsed from the time of being added dropwise, an observation reaction region instruction unit for encouraging the observation of the colorimetric reaction region in which the reaction time has elapsed, and a reference reaction color pattern presentation unit for presenting a reference reaction color pattern for determining the colorimetric reaction region in which the reaction time has elapsed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a color reaction observation assisting system, a color reaction observation assisting method, and a program for detecting a color reaction of an in-vitro diagnostic agent such as a urine test and a pregnancy test, and a sheet-like simple analysis kit for identifying pathogenic bacteria. About.

  In recent years, sheet-like simple analysis kits for identifying in-vitro diagnostic agents, pathogenic bacteria, and the like have been frequently used for determining diseases, pregnancy, chemical substances, pathogenic bacteria, and the like. For example, a general example of urine test paper will be described below as an example of a simple test kit. A single or a plurality of reaction areas are arranged on the test paper, and when a test solution containing the test object is dropped, the test object causes a color reaction in the reaction area. Then, after dropping the test solution, visually check the color reaction result and the color legend corresponding to the test result when a predetermined time elapses set for each test item to be tested. Analysis is performed.

  There is also an analysis method in which the spectral absorbance of the reaction region where the color reaction has occurred is measured, and the concentration of the analyte as the target substance contained in the test solution is measured.

There is already a dedicated device for analyzing the subject using the urine test paper as described above and a method for detecting the subject using this dedicated device, but the dedicated device used for this analysis is an expert ( It is a device for medical professionals. For this reason, the dedicated device described above is generally installed only at an expensive and extremely limited place, and only an expert can perform an analysis processing operation.
In addition, the detection method using the dedicated device described above is premised on that the reaction region is photographed under a predetermined environment by illuminating using a special optical device incorporated in the dedicated device (for example, patent Reference 1). Further, the position of the urine test paper is mechanically controlled with respect to the photographing optical system, and the illumination conditions and photographing conditions at the time of photographing are ideal for observation and determination of the test paper.

  In recent years, “Point of Care Testing (POCT)” has been proposed, and biochemical test items that could only be processed by dedicated equipment in hospitals can be developed with expertise in biochemical knowledge and operation skills of dedicated equipment. There is an urgent need to create a mechanism that can be easily and inexpensively tested using a simple device at any time, anywhere, and at any time, and there is a tendency to make testing more familiar to patients. In addition, a test result can be obtained immediately at a medical site, and a merit that a doctor can perform a quick treatment is born. By using this simple test kit, it is possible to easily perform a preliminary diagnosis in an ambulance or at home even outside the medical site, and it is possible to efficiently deal with the medical site after being transported to the hospital.

Japanese Patent No. 5351090

However, the above-described analysis method is still based on the premise that analysis is performed using a dedicated device.
Assuming an analysis method that can be used more widely, it is impossible to introduce a dedicated device in a general household from the viewpoint of cost and the like. Moreover, it is not assumed with these inspection papers that the general user who does not have expert knowledge visually determines. For this reason, especially for test papers with many test items with different color reaction times, it is too complicated to understand at which timing and which part should be observed. The result cannot be accurately determined.

For example, in the case of the urine test paper, a plurality of items such as white blood cells, occult blood, and glucose in urine can be inspected. Here, since the time of the color reaction in the reaction region may vary from item to item, it is necessary to observe the coloration at the time of each predetermined time determined for each inspection item.
However, since the reaction proceeds in the colorimetric reaction region even after a predetermined time has elapsed, it is difficult to obtain an accurate determination result of the inspection item if it is less than the predetermined time or longer than the specified time. Therefore, when the subject has been dropped and when the elapsed time since the drop has not been accurately determined, the accuracy of the determination result is affected.

  The present invention has been made in view of such a situation, and setting detection set for each inspection item without using a special and expensive inspection apparatus and even a general user who does not have specialized knowledge. Provided are a color reaction observation assisting system, a color reaction observation assisting method, and a program capable of accurately observing the determination result of the color reaction on the test paper as described above when the time has elapsed.

  In order to solve the above-described problem, the color reaction observation assistance system of the present invention is a test solution dropped in the color reaction region in the color reaction region for each inspection item in the color reaction region of the analysis kit. Is a color reaction observation assisting system that assists in observation of a color reaction by a detection target substance contained in the image, and the captured image supplied from the imaging device includes an observation target surface including each of the colorimetric reaction regions. For each of the colorimetric reaction regions, an observation target surface detection unit for determining whether or not, an observation position specifying unit for specifying the position of each of the colorimetric reaction regions on the observation target surface, and A drip detection unit for detecting that the test solution has been dropped, and a process for detecting whether or not a reaction time set in advance for each of the colorimetric reaction regions has elapsed since the dropping was performed. A time detector and the counter A reference reaction that presents a reference reaction color pattern for determining a color reaction color in the colorimetric reaction region after the reaction time, and an observation reaction region instruction unit that prompts observation of the colorimetric reaction region after the elapse of time And a color pattern presentation unit.

  The color reaction observation assisting system according to the present invention includes, in the data storage unit, a colorimetric reaction region image that is an image of a colorimetric reaction region in which the elapsed time detection unit detects that the reaction time has elapsed in the captured image. A color reaction region image storage control unit for writing and storing, and the observation reaction region instruction unit reads the colorimetric reaction region image from the data storage unit, and the corresponding reaction time on the observation target surface has elapsed. It presents at the position of the colorimetric reaction region.

  In the color reaction observation assistance system according to the present invention, the observation target surface detection unit is configured such that, when the observation target surface is not included in the captured image, the observation target surface is included in the captured image. It is characterized by prompting the user to perform image capturing.

  In the color reaction observation assisting system according to the present invention, the drop detection unit is configured to detect the colorimetric reaction region in each of the colorimetric reaction regions based on a change in coloration anti-dark color of each image of the colorimetric reaction region in the captured image. It is characterized by detecting that the solution has been dropped.

  In the color reaction observation assisting system of the present invention, the elapsed time detection unit measures the elapsed time from the time when the test solution is dropped for each color reaction region, and It is characterized by detecting whether or not the reaction time has elapsed.

  In the color reaction observation assisting system according to the present invention, the observation reaction region instruction unit displays an image indicating the colorimetric reaction region in which the reaction time has elapsed in the color reaction region. It is characterized by prompting observation of the area.

  In the color reaction observation assisting system according to the present invention, when the reference reaction color pattern presentation unit prompts the observation of the colorimetric reaction region by the observation reaction region instruction unit, each of the inspection items includes the colorimetric reaction region. In the color reaction sample in which the reference reaction color pattern is described, the reference reaction color pattern corresponding to the colorimetric reaction region that is prompted to be observed is indicated.

  In the color reaction observation support system according to the present invention, when the reference reaction color pattern presentation unit stores the reference reaction color pattern in the data storage unit in advance, the observation reaction region instruction unit displays the colorimetric color. When the observation of the reaction region is promoted, the reference reaction color pattern that is color calibrated corresponding to the imaging condition of the captured image is presented.

  In the color reaction observation assisting system of the present invention, the analysis kit uses at least a pH (peech) test paper, a urine test paper, as a medium for detecting the detection target substance by a color reaction of the detection target substance, A simple test paper for measuring the degree of oxidation is used.

  In the color reaction observation assisting method of the present invention, in the colorimetric reaction region for each inspection item in the color reaction region of the analysis kit, the display by the detection target substance contained in the test solution dropped in the color reaction region. A color reaction observation assisting method for assisting color reaction observation, wherein the observation target surface detection unit includes an observation target surface including each of the colorimetric reaction regions in a captured image supplied from the imaging device. An observation target surface detection process for determining whether or not, an observation position specifying unit, an observation position specifying process for specifying each position of the colorimetric reaction region on the observation target surface, and a drip detection unit include the ratio A dropping detection process for detecting that the test solution has been dropped on each of the color reaction regions, and an elapsed time detection unit, for each colorimetric reaction region, in advance from the time when the dropping is performed. Whether the set reaction time has elapsed An elapsed time detection process for detecting the observation reaction area indicating unit, an observation reaction area indicating process for urging observation of the colorimetric reaction area after the reaction time has elapsed, and a reference reaction color pattern presentation unit including the reaction time A reference reaction color pattern presenting process for presenting a reference reaction color pattern for determining a color reaction color in the colorimetric reaction region after the elapse of time.

  In the color reaction region for each inspection item in the color reaction region of the analysis kit, the program of the present invention observes the color reaction due to the detection target substance contained in the test solution dropped in the colorimetric reaction region. Is a program for causing a computer to execute the function of an assisting color reaction observation assisting system, and that the computer includes an observation target surface including each of the colorimetric reaction regions in a captured image supplied from an imaging device An observation target surface detecting means for determining whether or not, an observation position specifying means for specifying the position of each of the colorimetric reaction regions on the observation target surface, and the test solution for each of the colorimetric reaction regions Drop detection means for detecting that the dropping has been performed, and an elapsed time for detecting whether or not a reaction time set in advance for each of the colorimetric reaction regions has elapsed since the time when the dropping was performed A detection means, an observation reaction area indicating means for urging observation of the colorimetric reaction area after the reaction time has passed, and a reference reaction color pattern for determining a color reaction color of the colorimetric reaction area after the reaction time has passed. It is a program for functioning as a reference reaction color pattern presenting means to be presented.

  As described above, according to the present invention, even a general user who does not use a special and expensive inspection apparatus and does not have specialized knowledge can easily use a general-purpose digital camera, a smartphone, an eyeglass-type display, or the like. A color reaction observation assist that enables observation of the colorimetric reaction region after the lapse of a predetermined time for each inspection item by using an image picked up by a simple imaging device, and provides accurate and simple inspection results for each of the plurality of inspection items A system, a color reaction observation assisting method, and a program can be provided.

It is a block diagram which shows the structural example of the coloring reaction observation assistance system by the 1st Embodiment of this invention. It is a figure which shows an example of the external appearance of the analysis kit of the analysis object of the color reaction observation assistance system in this embodiment. It is a figure which shows an example of the colorimetric reaction sample for collating the coloration state of each colorimetric reaction area | region in the color reaction area | region of FIG. It is a figure explaining the observation angle of the image pick-up part 101 at the time of imaging a color reaction area | region. It is a figure explaining the example of a display which matched the reference colorimetric part and the reference colorimetric pattern of the inspection item of observation object after reaction time progress. It is a flowchart which shows the operation example of the color reaction detection process with respect to the color reaction detection target using the simple test paper and the simple test | inspection medicine in the color reaction observation auxiliary system of one Embodiment of this invention. It is a figure explaining the other display which matched the reference colorimetric part and the reference colorimetric pattern of the inspection item of the observation object after reaction time progress.

<First Embodiment>
A first embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration example of a color reaction observation assistance system according to the first embodiment of the present invention. In FIG. 1, the color reaction observation assistance system 1 includes an imaging unit 101, an exposure control unit 102, an imaging control unit 103, an object detection unit 104, an observation position specifying unit 105, a drop detection unit 106, an elapsed time detection unit 107, an observation. A reaction region instruction unit 108, a reference reaction color pattern presentation unit 109, a reaction region image storage control unit 110, a display unit 111, an illumination unit 112, and a data storage unit 113 are provided.

  The imaging unit 101 is, for example, a digital camera using an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), or an imaging device provided in a portable terminal. Output captured image data. Moreover, you may image captured image data as a moving image. In the present embodiment, as the captured image data, for example, a color image of a moving image including shades of each color component RGB is targeted. The imaging unit 101 writes and stores the captured image data in the data storage unit 113 with a time stamp added for each frame.

  The exposure control unit 102 controls imaging conditions of the imaging unit 101 such as shutter speed, aperture value, presence / absence of illumination light, and intensity of illumination light as exposure imaging conditions. Further, the exposure control unit 102 corresponds to the brightness around the color reaction region (analysis kit) imaged by the color reaction detection system 1 and turns on illumination light such as a flash light source as necessary at the time of imaging. The instruction is output to an illumination unit (not shown).

FIG. 2 is a diagram illustrating an example of an appearance of an analysis kit to be analyzed by the color reaction observation assistance system according to the present embodiment. The analysis kit to be detected by this embodiment is, for example, an analysis kit 900 having the appearance shown in FIG. The analysis kit 900 includes a color reaction region 902 on the observation target surface 901. In the color reaction region 902, colorimetric reaction regions 903, 904, 905, 906,..., 907, 908 are formed as a plurality of colorimetric reaction regions. In addition, these colorimetric reaction areas are areas colored by the detection target substance contained in the test solution dropped in the color reaction area, and are provided for each detection target substance. Reacts to each target substance.
A color correction patch region 909 is provided on the surface of the analysis kit 900 where the observation target surface 901 exists. In the color correction patch area 909, color correction patches 1001, 1002, and 1003 for correcting the color of the captured image data in the color reaction area 902 are formed (described later).

FIG. 3 is a diagram showing an example of a colorimetric reaction sample for collating the coloration state of each colorimetric reaction region in the color reaction region of FIG. In the reference reaction color pattern sample 900 ′, colorimetric patterns of all colorimetric reactions in which the colorimetric reaction regions 903, 904, 905, 906,..., 907, 908 can be changed are referred to as reference colorimetric patterns 903 ′, 904 ′, 905 ′, 906 ′,..., 907 ′, and 908 ′ are shown corresponding to the respective rows.
For example, in the row of the reference colorimetric pattern 903 ′, the reference colorimetric part 903′_3 located in the center “0” character column is the color of the colorimetric reaction region before the test solution is dropped. The taste is shown. Further, in the row of the reference colorimetric pattern 903 ′, the colorimetric reaction is increased as the color becomes more similar to the colorimetric reference colorimetric portions 903′_2 and 903′_1 from the center reference colorimetric portion 903′_3. An area 903 indicates that the object detected is likely to be negative.

On the contrary, the object detected by the colorimetric reaction region 903 becomes more positive as the color becomes similar to the reference colorimetric parts 903′_4 and 903′_5 in the right coloration state from the central reference colorimetric part 903′_3. It shows that the possibility becomes large. As an example, when the coloration state of the colorimetric reaction region 903 is observed after the reaction time from the time of dropping the test solution defined for the colorimetric reaction region 903, the color of the colorimetric reaction region 903 is the reference colorimetric pattern 903 ′. If the color of the reference colorimetric part 903′_2 that is the second from the left in the row is closest, the target detected by the colorimetric reaction region 903 is determined as “N1 (Negative 1: Negative reaction level 1)”. Further, when the coloration state of the colorimetric reaction region 903 is observed after the reaction time from when the test solution is dropped defined for the colorimetric reaction region 903, the color of the colorimetric reaction region 903 is the row of the reference colorimetric pattern 903 ′. If the colorimetric color is closest to the color of the second reference colorimetric part 903′_5 from the left, the target detected by the colorimetric reaction region 903 is determined to be “P2 (Positive 2: Positive reaction level 2)”.
The reference reaction color pattern sample 900 ′ is provided with a color correction patch pattern 909 ′. Each of the color correction patches 1001 ′, 1002 ′, and 1003 ′ in the color correction patch pattern 909 ′ corresponds to the color correction patches 1001, 1002, and 1003 in FIG.

  Returning to FIG. 2, when the imaging unit 101 captures captured image data that is a captured image of the analysis kit, imaging such as the depth of focus and the sensitivity of the image sensor (ISO (International Organization for Standardization) sensitivity) is performed. The imaging condition of the unit 101 is controlled to a preset numerical value.

  The imaging control unit 103 analyzes the current captured image data when the imaging unit 101 captures captured image data that is a captured image of the analysis kit, and exposes the captured image data in the color reaction region of the analysis kit. Processing to eliminate the influence of the light source environment is performed according to an instruction from the control unit 102. When a general-purpose digital camera has a white balance function, the white balance function may be used to remove the influence of the light source environment.

  The object detection unit 104 determines whether each of the color reaction region 902 and the color correction patch region 909 in FIG. 2 exists in the captured image data. In this determination, the reference image data of the design pattern (three-dimensional shape model) of the analysis kit itself is stored in the data storage unit 113 in advance. Then, the object detection unit 104 determines whether the image feature points of the reference image data stored in the data storage unit 113 and the image data of the design pattern in the captured image data match in the three-dimensional space. No, that is, the similarity between the reference image data and the image data is calculated, and it is determined whether or not they match depending on whether or not the calculated similarity is within a predetermined similarity range. .

For example, the above-described determination method performs window matching in the vicinity of an image feature point, and if the square difference of all luminance values in the window is within a preset threshold value (for example, within a predetermined similarity range), the similarity degree This is a method in which it is determined that there is a match because it is high, and it is determined that they do not match if the square difference of all luminance values in the window is outside the threshold.
If the design feature of the analysis kit itself does not have sufficient image characteristics, a position detection dedicated pattern is arranged outside each of the color reaction region 902 and the color correction patch region 909 in the analysis kit. A configuration may be adopted in which whether or not the image feature points on the position detection dedicated patterns of the reference image data in the storage unit 113 and the image data in the captured image data match is determined by the above-described calculation.

  When the target detection unit 104 determines that each of the color reaction region 902 and the color correction patch region 909 does not exist in the captured image data, the target detection unit 104 displays the color reaction region 902 and the color correction on the display screen of the display unit 111. A notification indicating an instruction prompting the user to image each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ is displayed (presented) so that each of the patch areas 909 is included in the captured image data. This notification is continuously displayed until the captured image data is captured so that each of the color reaction region 902 and the color correction patch region 909 is included in the captured image data.

  If the target detection unit 104 determines that each of the color reaction region 902 and the color correction patch region 909 exists in the captured image data, the target detection unit 104 in the three-dimensional space in which the captured image data is captured, Each of the observation position (coordinate value) that is the position of the imaging unit 101 that has performed imaging and the imaging direction of the imaging unit 101 is determined from a predetermined coordinate conversion formula (a conversion formula obtained from internal parameters of the imaging unit 101 described later). Ask. Here, the observation position indicates a coordinate value captured (or observed) by the imaging unit 101 in a three-dimensional coordinate system indicating a three-dimensional space. That is, the color reaction region 902 of the color reaction region in each captured image data and the color correction for each of the captured image data based on the observation position and the imaging direction of the analysis kit 900 by the imaging unit 101 stored in the data storage unit 113. The relative observation angle or imaging angle of each patch area 909 is estimated.

In the present embodiment, each of the color reaction region 902 and the color correction patch region 909 is imaged by the imaging unit 101 at a predetermined focal length set in advance with respect to the imaging unit 101 to obtain captured image data. . Here, the color reaction region 902 includes the colorimetric reaction regions 903 to 908 as described above.
Then, the object detection unit 104 uses the coordinate conversion formula set in advance from the captured image data of the captured image data as described above, so that the color reaction region 902 and the color correction patch region in the three-dimensional space are used. Each of the observation position and the observation angle of the captured image data obtained by imaging each of 909 is estimated.

  The coordinate conversion formula for obtaining the observation angle described above is used for pre-processing for performing color reaction detection on the color reaction region provided in the analysis kit 900. This preprocessing is performed when a three-dimensional space is reproduced from a plurality of captured image data (captured image data captured from a calibration board described later) in advance, and the pixel positions and three-dimensional coordinates of the plurality of captured image data are reproduced. It is an expression generated when associating coordinate positions in space. The coordinate conversion formula generated in advance is written and stored in advance for each color reaction detection target in the data storage unit 113. Here, the color reaction detection target indicates the type of analysis kit. When the analysis kits have different shapes, it is necessary to prepare coordinate conversion formulas corresponding to the respective container shapes.

  FIG. 4 is a diagram illustrating the observation angle of the imaging unit 101 when imaging the color reaction region. In FIG. 4, when a test solution is dropped on the color reaction region, and there is a target substance to be detected in this test solution, the color of each colorimetric reaction region depends on the concentration or intensity of the target target substance. And the color reaction can be observed.

The normal line 400 is a reference line for indicating the observation angle α in the imaging direction of the captured image data, and is a normal line indicating the surface direction of the surface of the color reaction region 902. The observation angle α is an angle formed by the imaging direction 101A of the imaging unit 101 and the normal 400.
For example, the object detection unit 104 sets the analysis kit in the three-dimensional coordinate system so that the direction parallel to the normal line 400 is the z-axis and each side of the color reaction region is parallel to the x-axis and the y-axis. Deploy. For example, in the three-dimensional coordinate system, the analysis kit is composed of two x-axis and y-axis so that any of the vertices formed by each side of the container of the analysis kit 900 coincides with the origin O of the three-dimensional coordinate system. Place in the dimension plane. For this reason, the thickness direction of the container of the analysis kit 900 is parallel to the z-axis. The three-dimensional shape of the container of the analysis kit 900 is stored in the data storage unit 113 together with the already described coordinate conversion formula as known information.

  Returning to FIG. 1, when the object detection unit 104 obtains the observation angle when the captured image data is captured, the coordinates of the three-dimensional shape of the container of the analysis kit 900 in the three-dimensional coordinate system and the captured image data (2 Each pixel (coordinate) in the dimensional coordinate system is associated with the coordinate conversion formula. Then, the object detection unit 104 obtains the imaging position of the captured image data in the three-dimensional coordinate system of the three-dimensional space and the imaging direction of the captured image data from this imaging position based on the association described above. At this time, as described above, the object detection unit 104 has one vertex of the three-dimensional shape of the color reaction region in the three-dimensional coordinate system as the origin, the normal is parallel to the z-axis, and each side is x A three-dimensional space is defined on the container of the analysis kit 900 so as to be parallel to the axis or the y-axis.

  Then, the target detection unit 104 obtains the imaging position and imaging direction of the captured image data of the imaging unit 101 in the three-dimensional coordinate system with reference to the three-dimensional shape of the container of the analysis kit 900. As a result, the object detection unit 104 obtains the observation angle α formed by the normal 400 and the imaging direction of the imaging unit 101 as described above.

  In the present embodiment, it is necessary on the premise that camera calibration (camera calibration) is performed on the imaging unit 101 in advance. In this camera calibration, a calibration board having a known three-dimensional shape is imaged once or a plurality of times in an imaging region, and the one or a plurality of captured image data is used to capture a three-dimensional space in a three-dimensional space. A coordinate point in the coordinate system is associated with a plurality of coordinate points (two-dimensional pixels) in the two-dimensional coordinate system of the captured image data. As a result, the coordinate conversion formula indicating the relative positional relationship between the imaging unit 101 and the calibration board (hereinafter, external parameters), the optical center of the imaging unit 101 and the light incident direction vector in each pixel (two-dimensional pixel), lens A distortion or the like (hereinafter, an internal parameter of the imaging unit 101) is estimated.

  Here, in the present embodiment, since the object detection unit 104 estimates the observation angle of the captured image data, from the two-dimensional image obtained by imaging the calibration board from a plurality of different viewpoint directions previously captured by the imaging unit 101, that is, A global coordinate system (three-dimensional coordinate system) is reconstructed from multi-viewpoint captured image data. Then, a coordinate conversion formula indicating the correspondence between the coordinate point in the reconstructed three-dimensional coordinate system of the same pixel and the coordinate point in the two-dimensional coordinate system of the captured image data captured by the imaging unit 101 is obtained during camera calibration. Keep it.

  As described above, in this embodiment, the observation angle is estimated by performing camera calibration on the imaging unit 101 in advance, so that the internal parameters of the imaging unit 101 are executed when the color reaction detection process is performed in the detection system. Is known, and the three-dimensional shape of the container and the color reaction region of the analysis kit 900 is known (when the position detection dedicated pattern is used, the three-dimensional shape of the pattern is known). Thus, the imaged image data of the color reaction region is imaged, and a plurality of corresponding point information between the coordinate point in the three-dimensional coordinate system and the pixel in the two-dimensional coordinate system of the imaged image data is obtained by the coordinate conversion formula. The relative positional relationship between the imaging unit 101 and the color reaction region can be estimated from the corresponding point coordinates. That is, it is possible to estimate the observation position and observation angle (imaging direction) of the imaging unit 101 in the three-dimensional coordinate system when imaging the color reaction region.

  The observation position specifying unit 105 analyzes the analysis kit 900 stored in the data storage unit 113 based on the observation position and the observation angle of the color reaction region 902 and the color correction patch region 909 estimated by the target detection unit 104. In order to project the three-dimensional coordinates of the color reaction region 902 in the imaged image data, coordinates of the colorimetric reaction regions 903 to 908 (for example, four corner coordinates in the case of a square region) are calculated. Similarly, the observation position specifying unit 105 is stored in the data storage unit 113 based on the observation position and the observation angle of the color reaction region 902 and the color correction patch region 909 estimated by the target detection unit 104. In order to project the three-dimensional coordinates of the color correction patch region 909 in the analysis kit 900 onto the captured image data, the coordinates of the color correction patches 1001, 1002, and 1003 (for example, four corner coordinates in the case of a rectangular region). Is calculated.

With the configuration described above, the observation position specifying unit 105 corresponds to each position between the coordinate value of each colorimetric reaction region stored in the data storage unit 113 and the coordinate value of each colorimetric reaction region on the captured image data. Ask for. As a result, it is possible to define which position of the captured image data displayed on the display unit 111 corresponds to each colorimetric reaction region of the color reaction region 902 in the analysis kit 900 in the three-dimensional model.
Based on the position correspondence obtained by the observation position specifying unit 105, the observation reaction region instruction unit 108 is the time at which the reaction time of the inspection item to be observed has elapsed from among the colorimetric reaction regions 903 to 908 in the color reaction region 902. In FIG. 5, a mark that shows the colorimetric reaction region of the inspection item corresponding to the reaction time is presented on the image of the display unit 111. Thus, the user can easily recognize the colorimetric reaction region that needs to be observed in the color reaction region 902 after the reaction time has elapsed.

  In addition, the observation reaction region instructing unit 108 displays the mark in a blinking manner from a predetermined time before the time at which the reaction time to be observed elapses, for example, so that the observation reaction region instructing unit 108 is always lit after the reaction time has elapsed. The colorimetric reaction area to be observed may be notified to the user. Thereby, the user can observe the colorimetric reaction region where the reaction time has passed with a margin.

  Further, as will be described later, the observation reaction area instruction unit 108, when the reaction time has elapsed, the frame in which the colorimetric reaction area of the corresponding inspection item exists when the data reaction time has elapsed from the captured image data of the moving image. The captured image data may be extracted, and identification information for identifying the inspection item for which the reaction time has elapsed may be assigned and written and stored in the data storage unit 113. In this case, when the observation reaction region instruction unit 108 sequentially acquires the captured image data for each frame in the moving image, the reaction time elapses from the captured image data that the imaging unit 101 writes to the data storage unit 113 in time series. The captured image data corresponding to the inspection item at the time of the extraction is extracted. At this time, the observation reaction region instruction unit 108 extracts only the colorimetric reaction region of the inspection item at the time when the reaction time has passed as partial image data from the captured image data corresponding to the inspection item at the time when the reaction time has passed. Then, the colorimetric reaction region at the time when the reaction time in the image of the color reaction region 902 on the display unit 111 has elapsed is displayed. That is, the observation reaction region instruction unit 108 displays the image at the colorimetric reaction region coordinates when the reaction time has elapsed in the image of the color reaction region 902 on the display unit 111 and the image at the time when the reaction time has elapsed. to continue.

The observation position specifying unit 105 also includes reference colorimetric units in each row of the reference colorimetric patterns 903 ′, 904 ′, 905 ′, 906 ′,... 907 ′, 908 ′ in the reference reaction color pattern sample 900 ′. Are also calculated by the three-dimensional shape of the reference reaction color pattern sample 900 ′ stored in the data storage unit 113 and the coordinate conversion formula corresponding to the reference reaction color pattern sample 900 ′.
Then, the observation position specifying unit 105 calculates each coordinate value of the reference colorimetric part in each reference colorimetric pattern stored in the data storage unit 113 and each coordinate value of the reference colorimetric part on the captured image data. Each position correspondence is obtained. As a result, it can be defined which position of the reference colorimetric part in each reference colorimetric pattern in the reference reaction color pattern sample 900 ′ in the three-dimensional model corresponds to the captured image data displayed on the display unit 111. .

Based on the position correspondence obtained by the observation position specifying unit 105, the reference reaction color pattern presentation unit 109 responds to the inspection item reaction time to be observed from the reference colorimetric patterns 903 ′ to 908 ′ in the reference reaction color pattern sample 900 ′. At the time when elapses, a mark indicating the row of the reference colorimetric pattern of the inspection item corresponding to the reaction time is presented on the image of the display unit 111. Thus, the user can easily recognize the reference colorimetric pattern corresponding to the colorimetric reaction region that needs to be observed when the reaction time has elapsed in the color reaction region 902.
In addition, the reference reaction color pattern presentation unit 109, for example, blinks and displays the mark from a predetermined time before the time at which the reaction time to be observed elapses, and is always lit after the reaction time has elapsed. The reference colorimetric pattern to be observed in advance may be notified to the user in advance. Thereby, the user can observe the reference colorimetric pattern to be compared with the colorimetric reaction region where the reaction time has passed with a margin.

FIG. 5 is a diagram for explaining a display example in which the reference colorimetric part and the reference colorimetric pattern of the inspection item to be observed after the reaction time has elapsed are associated with each other.
On the display screen 111a of the display unit 111, captured image data including each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ captured by the imaging unit 101 is displayed. This captured image data is displayed, for example, in a gradation expression using a single color that is not colored in the color reaction of all inspection items.
Then, the observation reaction area instruction unit 108 displays the display color of the image of the colorimetric reaction area corresponding to the inspection item by the color of the color actually picked up when the reaction time for the inspection item has elapsed.

The reference reaction color pattern presentation unit 109 actually captures each image of the reference colorimetric part in the reference colorimetric pattern corresponding to the inspection item from the time when the reaction time for the inspection item has elapsed. Display by color.
As described above, after the reaction time has elapsed, the colorimetric reaction region 905 that requires observation in each of the analysis kit 900 and the reference reaction color pattern sample 900 ′, and the reference colorimetric corresponding to the colorimetric reaction region 905 By marking the pattern 905 ′, the user can easily observe the color reaction in the colorimetric reaction region 905 of the inspection item for which the reaction time has passed.

  Returning to FIG. 1, the drop detection unit 106 detects that the subject has been dropped. For example, the drop detection unit 106 selects any color of the colorimetric reaction region of the analysis kit 900 in the captured image data normalized by the target detection unit 104 and any time from the start of imaging of the normalized captured image data. The color of each colorimetric reaction region of the analysis kit 900 in the captured image data after time is compared. Then, the drop detection unit 106 selects the color of the time-series area in each of the colorimetric reaction areas between the start time and the elapse of a predetermined time every arbitrary time from the start of imaging of the captured image data. When the calculated difference value is greater than or equal to a preset dropping threshold value, it is detected that the subject has been dropped.

When the time when the test solution is dropped into the colorimetric reaction region is supplied from the drop detection unit 106, the elapsed time detection unit 107 counts the elapsed time from this time, and the reaction time set for each inspection item Whether or not has passed is detected.
Then, the elapsed time detection unit 107 outputs the inspection item for which the reaction time has elapsed to each of the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109.
Thereby, each of the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109 displays the colorimetric reaction region and the reference colorimetric pattern as described above when the reaction time of each inspection item has elapsed. By marking, the user is prompted to perform observation.

  That is, when the user images each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ with the imaging unit 101, each of the colorimetric reaction region and the reference colorimetric pattern in the color reaction region 902 is detected by the target detection unit 104. Are detected, and each of the observation position and the imaging direction is estimated. Then, the observation position specifying unit 105 calculates coordinates on the captured image of the colorimetric reaction region and the reference colorimetric pattern in the color reaction region 902 from the observation position and the imaging direction. Each of the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109 displays the analysis kit 900 and the reference reaction color pattern sample on the display unit 111 in a state where the luminance value in the coordinate value is highlighted (marked state). By presenting each of 900 ′, the observation timing and observation location of the colorimetric reaction region can be suggested to the user in real time when the reaction time elapses.

  With the configuration described above, according to the present embodiment, the colorimetric reaction region in the analysis kit 900 and the reference colorimetric pattern corresponding to the colorimetric reaction region of the reference reaction color pattern sample 900 ′ printed on the paper surface are imaged. When comparing in an image, a coordinate form in a captured image of a reference colorimetric pattern corresponding to the colorimetric reaction region to be observed is calculated, and a display form in which a luminance value in the coordinate value is highlighted ( As shown in FIG. 5). As a result, the user can simultaneously grasp the observation location of the colorimetric reaction region to be observed and the observation location of the reference colorimetric pattern corresponding to this colorimetric reaction region, so the number of colorimetric reaction regions and reference colorimetric patterns Even when there are a large number of colors, it is possible to easily observe the color reaction in the colorimetric reaction region to be observed.

  In addition, depending on the type of inspection item, there are cases where reaction times are close in a plurality of inspection items. For this reason, when the reaction time has elapsed, the observation reaction region instruction unit 108 receives the partial image data of the colorimetric reaction region of the inspection item for which the reaction time has elapsed, and the reaction region image storage control unit 110 the data storage unit 113. It is good also as a structure which writes and memorize | stores with respect to. When acquiring captured image data as a moving image, the captured image data written in the data storage unit 113 in time series by the image capturing unit 101 is used as an inspection item when the reaction time has elapsed from the captured image data. A configuration may be used in which data of a partial image of the corresponding colorimetric reaction region is extracted and used. The observation reaction region instructing unit 108 reads the partial image data of the colorimetric reaction region from the data storage unit 113, and uses the read partial image data as the colorimetric reaction region in which the reaction time on the display screen 111 a in the display unit 111 has elapsed. Present at the position of.

Then, the user selects the colorimetric reaction region to be observed from the colorimetric reaction regions marked after the reaction time has elapsed on the display screen 111a of the display unit 111, whereby the selected colorimetric reaction region is selected. The colorimetric reaction areas other than are set to the monochrome display already described. Then, the observation reaction region instruction unit 108 outputs the inspection item of the selected colorimetric reaction region to the reference reaction color pattern presentation unit 109.
Thereby, the reference reaction color pattern presentation unit 109 marks the reference colorimetric pattern corresponding to the supplied inspection item.
With this configuration, the user can observe the test items in the desired order without rushing the inspection items that are close to each other in the reaction time. Since the image of the color reaction is written and stored in the data storage unit 113, the color reaction of the inspection item can be observed as necessary.

Next, each color correction of the colorimetric reaction region by the color correction patch will be described. In the above description, an example in which each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ is captured so as to have the same image data has been described. However, as in the case of imaging the reference reaction color pattern specimen 900 ′ first and then observing the color reaction of each colorimetric reaction region of the inspection item in the analysis kit 900, the analysis kit 900, the reference reaction color pattern Each specimen 900 ′ may be captured as different captured image data. In this configuration, the captured image data of the analysis kit 900 and the captured image data of the reference reaction color pattern sample 900 ′ are displayed side by side on the display screen 111a of the display unit 111 of FIG.
In this case, when the imaging environments of the analysis kit 900 and the reference reaction color pattern sample 900 ′ are different, for example, when the illumination light is different, the colors in the captured image data of the analysis kit 900 and the reference reaction color pattern sample 900 ′ are different. Unlikely, the inspection item cannot be accurately determined.

Therefore, in order to accurately determine the inspection item, it is necessary to match the color of the captured image data of the reference reaction color pattern sample 900 ′ with the color of the captured image data of the analysis kit 900 as described below.
For example, the imaging control unit 103 calculates the color tone of the color correction patch pattern 909 ′ in the captured image data captured under a predetermined light source as C _gt (R _gt , G _gt , B _gt ). In addition, the imaging control unit 103 determines that the color of the color correction patch region 909 in the captured image data captured under a light source different from the color correction patch pattern 909 ′ is C _in (R _in , G _in , B _in ). calculate. If, between the color C _gt taste and color C _in, the case of applying the conversion formula of the simplest color correction as an example, the following equation (1) holds.

In the equation (1), as described above, C _gt is the color of the color correction patch pattern 909 ′ under the light source in which the captured image data of the reference reaction color pattern sample 900 ′ is captured, and the color component RGB This is (R _gt , G _gt , B _gt ). Further, C _in is the color of the color correction patch 909 under the light source from which the imaged image data of the analysis kit 900 is imaged, and is described as the color component RGB (R _in , G _in , B _in ).
Thereby, said (1) Formula is represented by the following (2) Formula.

By using three color correction patches with different colors in the reference reaction color pattern sample 900 ′ and the analysis kit 900, the color correction patch color in the captured image data of the reference reaction color pattern sample 900 ′; By obtaining three or more combinations with the color of the color correction patch in the captured image data of the analysis kit 900, the matrix A in the equation (2) is obtained.
Here, the imaging control unit 103 stores the color C _gt (R _gt , G _gt) of each of the three types of color correction patches 1001 ′, 1002 ′, and 1003 ′ of the reference reaction color pattern sample 900 ′ from the data storage unit 113. , B _gt ) and equation (2). Then, the imaging control unit 103 extracts the color C _in (R _in , G _in , B _in ) of the three types of color correction patches from the captured image data of the analysis kit 900, and the reference reaction color pattern sample 900 ′. Using the color correction patch color C _gt (R _gt , G _gt , B _gt ), the expression (2) corresponding to each of the three types of color correction patches 1001, 1002, and 1003, A matrix A for converting the color is calculated. The imaging control unit 103 refers to the color of each pixel of the captured image data of the analysis kit 900 captured under a light source different from that of the reference reaction color pattern sample 900 ′ by using the calculated matrix A, using equation (2). The color of the reaction color pattern sample 900 ′ is converted.

  As described above, by obtaining the matrix A and using the expression (2), the color of the captured image data of the reference reaction color pattern sample 900 ′ captured under an arbitrary light source is captured under a different light source. It is possible to convert the color of the captured image data of the analysis kit 900. As a result, the colorimetric reaction region and the corresponding reference colorimetric pattern in the captured image data of the analysis kit 900 and the reference reaction color pattern sample 900 ′ captured under different light sources are imaged under the same light source. It is possible to compare in the state of the light source unified as the captured image data. That is, in order to compare each of the colorimetric reaction region and the reference colorimetric pattern in the captured image data of each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ captured under different light sources, the analysis kit 900 and It becomes possible to eliminate the influence of illumination (in the light source environment) from the captured image data of the reference reaction color pattern sample 900 ′, and observe the color reaction of the inspection item with high accuracy and perform the color reaction detection process. be able to.

FIG. 6 is a flowchart showing an operation example of the color reaction observation assistance process for the color reaction detection target using the simple test paper or the simple test medicine in the color reaction observation assistance system according to the embodiment of the present invention.
Step S1:
The imaging control unit 103 detects a current imaging condition of a color reaction region that is a color reaction detection target in the imaging unit 101, for example, an observation angle and an exposure condition.

Step S2:
The imaging control unit 103 uses the analysis kit 900 and the reference reaction color as the image quality of an image in which all of the imaging conditions such as the exposure condition can compare the color reaction in the colorimetric reaction region and the reference colorimetric pattern. A determination is made as to whether or not the image capturing conditions allow the captured image data of each of the pattern specimens 900 ′ to be captured.
At this time, when the imaging control unit 103 has imaging conditions capable of imaging each captured image data of the analysis kit 900 and the reference reaction color pattern sample 900 ′ as the image quality of the image that can compare the color reactions. Then, the process proceeds to step S3. On the other hand, the imaging control unit 103 performs processing when the image quality capable of comparing the color reactions is not an imaging condition capable of capturing the captured image data of the analysis kit 900 and the reference reaction color pattern sample 900 ′. Advances to step S4.

Step S3:
The target detection unit 104 extracts the imaging position of each of the color reaction region 902 and the reference reaction color pattern sample 900 ′ in the captured image data. That is, the imaging control unit 103 obtains the three-dimensional shape of each of the color reaction region 902 and the reference reaction color pattern sample 900 ′ within the imaging range of the imaging unit 101. Then, the imaging control unit 103 acquires the three-dimensional shapes of the obtained analysis kit 900 and the reference reaction color pattern sample 900 ′, and the analysis kit 900 and the reference reaction color that are previously written and stored in the data storage unit 113. The three-dimensional shape of each of the pattern specimens 900 ′ is compared, and a color reaction region 902 within the imaging range of the imaging unit 101 is extracted.

Step S4:
The imaging control unit 103 displays a condition that is not satisfied in the imaging condition on the display screen of the display unit 111, and suggests adjustment of the condition that is not satisfied in the imaging condition to the user.

Step S5:
The object detection unit 104 includes the color reaction region 902 and the reference reaction color pattern sample 900 ′ in the captured image data of the image capturing unit 101, and the color reaction region in the three-dimensional shape of the container of the analysis kit 900 stored in advance. 902, each reference reaction color pattern sample 900 ′ is compared. Then, the target detection unit 104 determines whether or not each of the entire color reaction region 902 and the entire reference reaction color pattern sample 900 ′ is included in the captured image data, that is, the coloration of the color reaction region 902 in the observation region. It is determined whether or not the entire color reaction region 902 for observing the reaction and the entire reference reaction color pattern sample 900 ′ are included in the captured image data.
At this time, if the entire color reaction region 902 and the entire reference reaction color pattern sample 900 ′ are included in the captured image data, the target detection unit 104 advances the process to step S6. On the other hand, if the entire color reaction region 902 and the entire reference reaction color pattern sample 900 ′ are not included in the captured image data, the target detection unit 10 advances the process to step S7.

Step S6:
The object detection unit 104 causes the observation position specifying unit 105 to perform an estimation process of the imaging direction of the color reaction region 902 and the reference reaction color pattern sample 900 ′, that is, the observation angle and the observation position.
Thereby, the observation position specifying unit 105 stores the three-dimensional shape of each of the container of the analysis kit 900 and the reference reaction color pattern sample 900 ′ obtained from the captured image data in the imaging range of the imaging unit 101, and 3 stored in advance. The observation angle with respect to the imaging direction of the color reaction region 902 and the reference reaction color pattern sample 900 ′ is estimated by comparing the container of the analysis kit 900 and the three-dimensional shapes of the reference reaction color pattern sample 900 ′ in the two-dimensional coordinate system. To do. Here, the object detection unit 104 obtains the imaging direction of the captured image data in which the imaging unit 101 has captured each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ by the above comparison. Then, the object detection unit 104 has a surface to which the color reaction region 902 of the analysis kit 900 in the three-dimensional coordinate system is attached (any one of which the color reaction region 902 on the upper surface or the lower surface of the analysis kit 900 is attached). The angle formed by the normal of the surface) and the imaging direction of the imaging unit 101 is obtained as an observation angle.

Step S7:
The target detection unit 104 displays on the display screen of the display unit 111 that the imaging position captured by the imaging unit 101 is adjusted so that the entire color reaction region 902 is included in the imaging range of the imaging unit 101. This suggests that the user changes the imaging position.

Step S8:
The drop detection unit 106 calculates the initial color reaction color of the colorimetric reaction region in the first captured image data and the color reaction color of the color reaction region in the captured image data captured in time series thereafter. Find the difference value.
And the dripping detection part 106 determines whether the calculated | required difference value is more than a dripping threshold value. At this time, when the obtained difference value is equal to or greater than the dropping threshold value, the dropping detection unit 106 outputs a detection signal to the elapsed time detection unit 107, assuming that dropping of the test solution has been detected, and then performs processing. To step S9. As a result, the elapsed time detection unit 107 starts measuring the elapsed time for detecting the reaction time.
On the other hand, when the obtained difference value is less than the dropping threshold value, the dropping detection unit 106 repeats the process of step S8 on the assumption that the dropping of the test solution is not detected.

Step S9:
The elapsed time detection unit 107 determines whether the counted elapsed time is equal to or longer than the reaction time of any inspection item.
At this time, when the elapsed time becomes equal to or longer than the reaction time of any inspection item, the elapsed time detection unit 107 performs a color reaction based on the test solution in the colorimetric reaction region of the inspection item in the color reaction region. A control signal indicating that the color observation time has come is output to the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109.

Step S10:
Then, each of the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109 is displayed on the display screen of the display unit 111 at the time when the control signal including the information of the inspection item for which the reaction time has elapsed is supplied. In each of the color reaction region 902 and the reference reaction color pattern sample 900 ′, markings for the colorimetric reaction region and the reference colorimetric pattern corresponding to the inspection item are displayed, and the color reaction is observed, that is, for the inspection item. Prompt the user to make a determination.

Step S11:
The elapsed time detection unit 107 detects the presence / absence of other inspection items that the user has not observed, that is, the user is not prompted to observe the color reaction, in addition to the currently determined inspection item.
At this time, if there is another inspection item that does not prompt the observation of the color reaction, that is, if there remains an inspection item for which the reaction time is detected, the elapsed time detection unit 107 advances the process to step S9. . On the other hand, the elapsed time detection unit 107 ends the process when there is no other inspection item that does not prompt the observation of the color reaction, that is, when there is no inspection item for which the reaction time is detected.

FIG. 7 is a diagram illustrating another display in which the reference colorimetric part and the reference colorimetric pattern of the inspection item to be observed after the reaction time has elapsed are associated with each other.
In FIG. 7, the observation reaction region instruction unit 108 displays each of the colorimetric reaction regions 903 to 908 with colors captured in real time on the display screen 111 a of the display unit 111. Then, the reference reaction color pattern presenting unit 109 arranges the image of the colorimetric reaction region 905 of the inspection item to be observed on the display screen 111a of the display unit 111, and displays the image of the reference colorimetric pattern 905 ′ corresponding to the inspection item. indicate.

Thus, the user can simultaneously grasp the colorimetric reaction region 905 to be observed in the analysis kit 900 color reaction region 902 and the observation location in the reference colorimetric pattern 905 ′ corresponding to the colorimetric reaction region 905.
Therefore, similarly to the case of FIG. 5, even if the number of the colorimetric reaction region and the reference colorimetric pattern corresponding to this colorimetric reaction region is large, the user can easily observe the colorimetric region to be observed and inspect it. The item can be determined.

The display unit 111 is a display device such as a liquid crystal display, and the image of the analysis kit 900 and the image of the reference reaction color pattern specimen 900 ′ are controlled by the observation reaction region instruction unit 108 and the reference reaction color pattern presentation unit 109, respectively. Display, marking, etc.
Or you may comprise from the printing apparatuses, such as a printer, the printed matter which printed each of the colorimetric reaction area | region of the test | inspection item which passed reaction time, and each reference colorimetric pattern of this test | inspection item.

<Second Embodiment>
Next, a second embodiment of the present invention will be described with reference to the drawings.
The second embodiment has the same configuration as the configuration example of the color reaction observation assistance system according to the first embodiment shown in FIG. Hereinafter, operations different from those of the first embodiment will be described. In the second embodiment, the reference colorimetric patterns 903 ′, 904 ′, 905 ′, 906 ′,..., 907 ′ in the reference reaction color pattern sample 900 ′ under the environment of a standard light source (for example, D65 light source). , 908 ′ are captured as reference image data, and are written and stored in advance in the data storage unit 113.

  With the configuration of the second embodiment, in the observation of the color reaction for each inspection item using the imaged image data of the colorimetric reaction region to be observed, it is printed on the paper as in the first embodiment. Even if the patch pattern 909 ′ for color correction does not exist, the reference colorimetric pattern corresponding to the colorimetric reaction region for determining the inspection item is read from the data storage unit 113 and displayed in parallel with the colorimetric reaction region to be observed. By displaying on the display screen of the unit 111, the user can easily observe the color reaction.

In addition, in the relationship between the expressions (1) and (2) already described, when using the reference reaction color pattern sample 900 ′ that is imaged with a standard light source and stored in the data storage unit 113, a different light source is used. Since it becomes an environment, it is necessary to change the color.
The imaging control unit 103 reads the three types of color correction patches 1001 ′, 1002 ′, and 1003 ′ of the reference reaction color pattern sample 900 ′ from the data storage unit 113, and sets the respective colors to C _gt (R _gt , G _gt , _Bgt ). Further, the imaging control unit 103 calculates the three types of color correction patches 1001, 1002, and 1003 of the color correction patch area 909 in the captured image data as C _in (R _in , G _in , B _in ). . In this case, when the simplest color correction conversion formula is applied as an example between the tint C _gt and the tint C _in , the following formula (1) is established.

Then, the imaging control unit 103, the color _{C gt} and the color _{C in} (1) equation, as described above, color _{C gt} can see Reaction color pattern sample 900 imaged under standard light source It is (R _gt , G _gt , B _gt ) when described by the color component RGB.
On the other hand, the color tone C _in is the color tone of each color correction patch in the color correction patch region 909 in the captured image data, and is described as the color component RGB (R _in , G _in , B _in ).

  When displaying on the display screen 111a of the display unit 111, the color of each of the analysis kit 900 and the reference reaction color pattern sample 900 ′ is changed to three different color correction patches 1001, 1002, and 1003, and the color correction patch 1001. By using ', 1002', 1003 ', there are three or more pairs of combinations of the color of the reference reaction color pattern sample 900' imaged with a standard light source and the color of the analysis kit 900 in the captured image data. The matrix A used in equation (2) can be obtained.

Here, the imaging control unit 103 reads the color C _gt (R _gt , G _gt , B _gt ) of the three types of color correction patch patterns 909 ′ and the expression (2) from the data storage unit 113. Then, the imaging control unit 103 extracts the color C _in (R _in , G _in , B _in ) of the three types of color correction patches 909 from the captured image data, and the color of the color correction patch pattern 909 ′. Using C _gt (R _gt , G _gt , B _gt ), the matrix A is calculated using equation (2) corresponding to each of the three types of color correction patches. The imaging control unit 103 uses the calculated matrix A to convert the color imaged under a standard light source into the color of each pixel of the imaged image data imaged under an arbitrary light source, using equation (2). To do.

  As described above, by obtaining the matrix A and using the expression (2), the color imaged under a standard light source is converted to the color imaged image data imaged under an arbitrary light source. Is possible. As a result, the colorimetric reaction region observation result and the reference colorimetric pattern corresponding to the observation result can be compared with the illumination condition at the time of imaging. By this process, it is possible to eliminate the influence of the difference in illumination (under the light source environment) at the time of colorimetric reaction region observation and reference colorimetric pattern imaging, and to observe the color reaction with high accuracy.

1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by the computer system. By executing, a process for assisting observation of the color reaction in the color reaction detection may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices.
The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

DESCRIPTION OF SYMBOLS 1 ... Color reaction observation assistance system 101 ... Imaging part 101A ... Imaging direction 102 ... Exposure control part 103 ... Imaging control part 104 ... Object detection part 105 ... Observation position specification part 106 ... Drop detection part 107 ... Elapsed time detection part 108 ... Observation reaction region instructing unit 109: Reference reaction color pattern presentation unit 110 ... Reaction region image storage control unit 111 ... Display unit 112 ... Illumination unit 113 ... Data storage unit 400 ... Normal 900 ... Analysis kit 900 '... Reference reaction color pattern sample 901: Observation target surface 902: Color reaction region 903, 904, 905, 906, 907, 908 ... Colorimetric reaction region 903 ', 904', 905 ', 906', 907 ', 908' ... Reference color pattern 1001 , 1002, 1003, 1001 ′, 1002 ′, 1003 ′... Color correction patches

Claims (11)

In the colorimetric reaction region for each test item in the color reaction region of the analysis kit, the color reaction observation that assists the observation of the color reaction by the detection target substance contained in the test solution dropped in the color reaction region An auxiliary system,
An observation target surface detection unit that determines whether or not the captured image supplied from the imaging device includes an observation target surface including each of the colorimetric reaction regions;
An observation position specifying unit for specifying each position of the colorimetric reaction region on the observation target surface;
A drip detection unit that detects that the test solution has been dropped for each of the colorimetric reaction regions;
An elapsed time detection unit that detects whether or not a reaction time set in advance for each colorimetric reaction region has elapsed since the time when the dropping was performed;
An observation reaction region indicating unit that facilitates observation of the colorimetric reaction region after the reaction time has elapsed;
A color reaction observation assisting system, comprising: a reference reaction color pattern presentation unit that presents a reference reaction color pattern for determining a color reaction color in the colorimetric reaction region where the reaction time has elapsed. Colorimetric reaction region image storage control for writing and storing a colorimetric reaction region image, which is an image of a colorimetric reaction region in which the elapsed time detection unit has detected that the reaction time has passed, in the captured image Further comprising
The observation reaction region instruction unit reads the colorimetric reaction region image from the data storage unit and presents the image at the position of the colorimetric reaction region where the corresponding reaction time has elapsed on the observation target surface. Item 2. The color reaction observation assisting system according to Item 1. When the observation target surface is not included in the captured image, the observation target surface detection unit urges the imaging device to perform imaging so that the observation target surface is included in the captured image. The color reaction observation assistance system according to claim 1 or 2. The dripping detection unit is
2. It is detected that the test solution is dropped on each of the colorimetric reaction regions based on a change in coloration anti-dark color of each image of the colorimetric reaction region in the captured image. The color reaction observation assistance system according to any one of claims 1 to 3. The elapsed time detector is
The elapsed time from the time when the test solution is dropped is measured for each color reaction region, and it is detected whether or not the reaction time of the color reaction region has elapsed. The color reaction observation auxiliary system according to any one of claims 1 to 4. The observation reaction region instruction section is
6. The observation of the colorimetric reaction region is promoted by displaying an image indicating the colorimetric reaction region in which the reaction time has elapsed in the color reaction region. The color reaction observation auxiliary system according to claim 1. The reference reaction color pattern presentation section
When the observation reaction region instruction unit prompts observation of the colorimetric reaction region, the color reaction sample in which the reference reaction color pattern of the colorimetric reaction region of each of the inspection items is described is prompted for observation. The color reaction observation assisting system according to any one of claims 1 to 6, wherein the reference reaction color pattern corresponding to the colorimetric reaction region is indicated. The reference reaction color pattern presentation section
When the reference reaction color pattern is previously written and stored in the data storage unit, when the observation reaction region instruction unit prompts observation of the colorimetric reaction region, the reference reaction color pattern is made to correspond to the imaging condition of the captured image. The color reaction observation auxiliary system according to any one of claims 1 to 7, wherein the reference reaction color pattern subjected to color calibration is presented. As a medium in which the analysis kit detects the substance to be detected by a color reaction of the substance to be detected, at least a pH (pea etch) test paper, a urine test paper, and a simple test paper such as an oxidation degree measurement paper are used. The color reaction observation observation auxiliary system according to any one of claims 1 to 8, wherein In the colorimetric reaction region for each test item in the color reaction region of the analysis kit, the color reaction observation that assists the observation of the color reaction by the detection target substance contained in the test solution dropped in the color reaction region Is an auxiliary method,
An observation target surface detection unit that determines whether the observation target surface including each of the colorimetric reaction regions is included in the captured image supplied from the imaging device;
An observation position specifying unit for specifying each position of the colorimetric reaction region on the observation target surface;
A drip detection process in which a drip detection unit detects that the test solution has been dropped into each of the colorimetric reaction regions;
An elapsed time detection process in which an elapsed time detection unit detects whether or not a reaction time set in advance for each colorimetric reaction region has elapsed since the time when the dropping was performed,
An observation reaction region indicating unit for urging observation of the colorimetric reaction region after the reaction time has passed;
The reference reaction color pattern presentation unit includes a reference reaction color pattern presentation process for presenting a reference reaction color pattern for determining a color reaction color of the colorimetric reaction region where the reaction time has elapsed. Coloring reaction observation support method. In the color reaction region for each test item in the color reaction region of the analysis kit, the color reaction observation that assists the observation of the color reaction by the detection target substance contained in the test solution dropped in the colorimetric reaction region A program that causes a computer to execute the functions of an auxiliary system.
The computer,
An observation target surface detection unit for determining whether or not the captured image supplied from the imaging device includes an observation target surface including each of the colorimetric reaction regions;
An observation position specifying means for specifying the position of each of the colorimetric reaction regions on the observation target surface;
Drop detection means for detecting that the test solution is dropped on each of the colorimetric reaction regions;
Elapsed time detection means for detecting whether or not a reaction time set in advance for each colorimetric reaction region has elapsed since the time when the dropping was performed,
Observation reaction area indicating means for urging observation of the colorimetric reaction area after the reaction time has elapsed;
A program for functioning as a reference reaction color pattern presenting means for presenting a reference reaction color pattern for determining a color reaction color in the colorimetric reaction region where the reaction time has elapsed.

Images