JP7324485B2 - Droplet measurement system, droplet measurement method, and program - Google Patents

Description

The present invention relates to a droplet measuring system, a droplet measuring method, and a program for measuring the volume of a droplet dropped from a nozzle.

When instilling a liquid (infusion liquid) such as a drug solution or a nutrient, it is important to maintain a predetermined flow rate. Conventionally, drip flow rate control counts the number of droplets dripping into the drip cylinder to determine the number of drips per unit time, calculates the flow rate on the assumption that the droplet volume is constant, and determines the number of droplets. This was done by adjusting the dropping cycle (time interval for dropping).

However, in reality, the surface tension of droplets varies depending on conditions such as viscosity and environmental temperature, so the volume per droplet is not constant. In the medical field, the patient may change his/her posture during the infusion, and in this case, the head difference of the infusion liquid may change and the volume of the liquid droplet may fluctuate. Therefore, in the conventional method of controlling the flow rate only by the dropping period of droplets, it is difficult to control the flow rate with high accuracy.

As a solution to this problem, a technique is known in which the volume of a dropped droplet is measured and used for flow rate control. For example, in Patent Document 1, a transparent drip cylinder, a light emitting unit arranged on one side of the outside of the drip cylinder, and a two-dimensional image sensor arranged at a position facing the light emitting unit with the drip cylinder interposed therebetween. , and the field of view of the two-dimensional image sensor is set so as to include the tip of a dripping nozzle in a drip tube and a predetermined falling distance of droplets falling from the dripping nozzle.

Further, in Patent Document 2, an imaging device is installed so as to include the tip of a nozzle and the entire liquid hanging down from the nozzle in its field of view, and images in chronological order showing how droplets are dropped from the nozzle. Among them, the drop detection image, which is an image in which it is detected that the droplet has left the nozzle, is detected, and is generated a predetermined frame before the drop detection image, and is an image in which the liquid dripping down the nozzle is captured Based on the image and an image generated within a predetermined frame after the drop detection image and showing the nozzle after the liquid that was dripping onto the nozzle in the pre-drop image has been dropped, the liquid drops onto the nozzle in the pre-drop image. A droplet measurement system is disclosed that calculates the volume of a droplet deposited by a liquid that has been in contact with the object.

JP 2011-62371 A Japanese Patent No. 6406590

In Patent Document 1, since the volume of the droplet is calculated from the overall shape of the falling droplet, it has a large size that can capture the range from the tip of the dropping nozzle to the predetermined falling distance of the droplet. and a two-dimensional image sensor with a high frame rate. As a result, the amount of image data to be processed increases, requiring a high-performance processor capable of processing a large amount of arithmetic processing and memory operations at high speed.

On the other hand, in Patent Document 2, an image sensor of such a large size is used because it is sufficient that the range including at least the tip of the nozzle and the entire droplet hanging down from the nozzle can be captured in the image. No need. Therefore, the amount of data per image can be suppressed, and the droplet volume can be accurately calculated without using a high-performance processing device. However, in order to further improve the accuracy and real-time measurement of the droplet volume, there are processes such as detecting the timing at which the droplet leaves the nozzle and begins to fall, and extracting the area of the droplet captured in the image. Further improvement is desired.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a droplet measuring system, a droplet measuring method, and a program capable of improving the measurement accuracy and real-time performance of the volume of a droplet dropped from a nozzle.

In order to solve the above-described problems, a droplet measuring system according to one aspect of the present invention is an imaging device that captures an image of a subject and outputs image data in the droplet measuring system that measures the volume of a droplet dropped from a nozzle. An imaging device installed so as to include the tip of the nozzle and the entire droplet hanging down the nozzle within a field of view; and the droplet from the nozzle based on image data output from the imaging device. and an image depicting a state in which the droplet has separated from the nozzle and started to drip from the images generated in chronological order by the image generation unit. a dropping detection unit that detects the dropping start image, an image before a predetermined frame of the dropping start image, and an image after a predetermined frame of the dropping start image, the droplets reflected in the dropping start image a volume calculation unit configured to calculate a volume, wherein the droplet detection unit detects a position a predetermined distance below a position where the tip of the nozzle is captured in the images generated in chronological order by the image generation unit. A rectangular area having a predetermined size is set in the area, and the inside of the rectangular area is monitored to detect a drip standby image, which is an image showing a state in which a droplet is hanging at the tip of the nozzle. and a drip start determination unit that detects the drip start image by monitoring the rectangular region in the image generated by the image generator after the drip standby image is detected. It is.

The droplet measurement system further includes a storage unit including a first buffer storage area and a second buffer storage area, and the image generation unit performs predetermined processing on image data output from the imaging device. and stores the processed image data in the first buffer storage area, and the dropping detection unit stores the image data in the first buffer storage area when the dropping start image is detected. of the image data obtained, the image data of the image before the predetermined frame and the image after the predetermined frame of the dropping start image are copied to the second buffer storage area, and the volume calculation unit stores the image data in the second buffer The volume of the droplet may be calculated based on the image data stored in the storage area.

In the above-described droplet measurement system, the droplet standby determination unit determines that the image to be determined is the droplet standby image when detecting an area of images of droplets that are continuous in the vertical direction within the rectangular area. The dropping start determining unit may determine that the image to be determined is the dropping start image when an area of the image of the droplet divided in the vertical direction is detected in the rectangular area.

In the droplet measurement system, the droplet standby determination unit determines that the image to be determined is the droplet standby image when an image of droplets continuous in the vertical direction is detected in the rectangular area, and When the image of the liquid divided in the vertical direction within the rectangular area is detected over a plurality of continuous frames, the dripping start determination unit determines that the top image of the plurality of frames is the dripping start image. good.

In the above-described droplet measurement system, the dropping start determination unit starts monitoring the images generated in chronological order by the image generation unit after the dropping standby determination unit detects the dropping standby image, and detects the dropping standby image. After the drip start determination unit detects the drip start image, the determination unit may resume monitoring the images generated in chronological order by the image generation unit.

The droplet measurement system detects the position of the lower end of the image of the nozzle in the image generated by the image generation unit, and outputs an error signal when the position of the lower end is not within a predetermined range. a determination unit; a flow control unit that controls the start of dropping of droplets from the nozzle; and an operation input unit that inputs a signal corresponding to an external operation to the flow control unit, wherein the flow control unit When an error signal is output from the error determination unit, droplets are not dropped from the nozzle even when a signal instructing the start of droplet dropping is input from the operation input unit. It is also possible not to start it.

The droplet measurement system recognizes the image of the nozzle in the image generated by the image generation unit, and determines the number of droplets dropped from the nozzle based on at least one of the shape and thickness of the image of the nozzle. A flow rate estimation unit that estimates a standard flow rate range, a flow rate control unit that controls the flow rate of droplets dropped from the nozzle, and a flow rate of the droplets dropped from the nozzle according to an external operation. an operation input unit that inputs a signal representing a set value to the flow rate control unit, and the flow rate control unit determines that the set value is included in the standard flow rate range estimated by the flow rate estimation unit. If not, the droplet may not start to drop from the nozzle.

In the droplet measurement system described above, in the vertical direction of the image generated by the image generation unit, the upper end of the rectangular region is set below the tip of the image of the nozzle, and the lower end of the rectangular region is set below the droplet It may be set to be above the lower edge of the image of the liquid droplet captured in the image a predetermined frame before the start image.

In the droplet measurement system, when the conditions for dropping droplets from the nozzle are changed, the lower end portion of the image of the droplet adhering to the nozzle in the dropping start image and the falling droplet in the dropping start image changes, and when there is a common range between the gaps that change when the conditions are changed, the size of the rectangular area in the vertical direction is equal to the gap may be set so as to encompass a common range between them.

In the droplet measurement system, when the conditions for dropping droplets from the nozzle are changed, the lower end portion of the image of the droplet adhering to the nozzle in the dropping start image and the falling droplet in the dropping start image changes, and when there is no common range between the gaps that change when the condition is changed, the vertical size of the rectangular area is equal to the condition The uppermost position of the upper end when the is changed and the lowest position of the lower end when the condition is changed may be set to include a range between .

In the droplet measurement system, when the width of the droplet image in the droplet start image changes when the conditions for dropping droplets from the nozzle are changed, the horizontal size of the rectangular area is It may be set so as to include the largest width among the widths that change when conditions are changed.

In the above-described droplet measuring system, the droplet standby determination unit converts the gradation of pixels in at least the outline region of the droplet image to a first gradation by performing a binarization process on the rectangular region. and converting the gradation of the pixels in the background area of the droplet image into the second gradation, and searching the rectangular area to determine the row in which the pixel having the first gradation exists. It may be determined that the image to be determined is the dropping standby image when all the lines of the rectangular area are the droplet presence lines as a result of the search.

In the above-described droplet measuring system, the droplet standby determination unit converts the gradation of pixels in at least the outline region of the droplet image to the first gradation by performing binarization processing on the rectangular region. Then, by converting the gradation of the pixels in the background area of the droplet image to the second gradation and searching the rectangular area, a plurality of pixels having the first gradation are continuously generated. Existing rows may be extracted as droplet presence rows, and if all rows in the rectangular area are droplet presence rows as a result of the search, the image to be determined may be determined to be a drip standby image.

In the above-described droplet measurement system, the dropping start determination unit searches the binarized rectangular area, and if a line in which all pixels have the second gradation is detected, , the image to be determined may be determined to be the dropping start image.

In the above-described droplet measurement system, the dropping start determination unit searches the rectangular area subjected to the binarization process, and a plurality of rows in which all pixels have the second gradation are continuously arranged. It may be determined that the image to be determined is the dropping start image when the image is detected over the entire area.

In the above-described droplet measurement system, the dropping start determination unit determines that one or more lines that are continuous with the lower end of the rectangular area are detected even if a line in which all pixels have the second gradation is detected. However, when all the pixels are rows having the second gradation, it may be determined that the determination target image is not the dropping start image.

In the droplet measurement system, the dropping start determination unit counts the pixels having the first gradation in a plurality of rows that are continuous with the lower end of the rectangular area in the rectangular area that has been subjected to the binarization process. , the image to be determined may be determined to be the dropping start image when the total number of pixels is less than the total number in the frame immediately preceding the image to be determined by exceeding a predetermined threshold. .

A droplet measuring system according to another aspect of the present invention is an imaging device for imaging a subject and outputting image data in the droplet measuring system for measuring the volume of a droplet dropped from a nozzle, the imaging device comprising: An imaging device installed so that the tip and the entire droplet hanging down from the nozzle are within the field of view, and the appearance of the droplet dropping from the nozzle based on the image data output from the imaging device. An image generating unit that generates images in chronological order, and a dropping start image that is an image showing a state in which a droplet has separated from the nozzle and started to drop is detected from the images that are generated in chronological order by the image generating unit. and a volume calculation unit for calculating the volume of the droplet reflected in the drip start image based on the image before the predetermined frame of the drip start image and the image after the predetermined frame of the drip start image. and the volume calculating unit includes: an area extracting unit for extracting an area of the image of the liquid droplet captured in the image of the predetermined frame before; A circle area calculation unit that calculates the area of a circle having a diameter equal to the number of pixels for each row of the extracted region, and an integration unit that integrates the areas calculated by the circle area calculation unit, The region extraction unit generates a difference image between a first image that is an image before the predetermined frame and a second image that is an image after the predetermined frame, and performs binarization processing on the difference image. generating a third image by performing a filling process, generating a fourth image obtained by performing a hole-filling process on the third image, and generating the first image based on the third and fourth images; and extracting the overlapping droplet image portion by setting a region between a second image and applying the region to the first image to define the overlapping droplet image portion; By synthesizing the portion of the overlapping droplet image and the portion extracted from the fourth image, the entire area of the droplet image reflected in the first image is extracted.

In the droplet measuring system, the circular area calculator sets a rectangular frame circumscribing the portion extracted from the fourth image in the region extracted by the region extractor. Then, a center line extending in the vertical direction of the frame is set, and for each row of the portion extracted from the fourth image, the number of pixels on the left and right sides of the center line is twice the number of pixels on the side with the larger number of pixels. The area of a circle with a diameter of .

In the droplet measuring system described above, the circular area calculation unit calculates the portion of the area extracted by the area extraction unit, which is extracted from the fourth image, for the area of the nozzle reflected in the first image. A line passing through the center of the lower end of the image is set as a dividing line, and for each row extending in a direction perpendicular to the dividing line, a circle whose diameter is twice the number of pixels on the side with the larger number of pixels among both sides of the dividing line You may calculate the area of .

In the droplet measuring system, the circular area calculator may set a line passing through the center of the lower end of the nozzle image and parallel to the vertical direction of the first image as the dividing line. .

The droplet measurement system further includes an angle sensor that detects the inclination of the nozzle, and the circular area calculator passes through the center of the lower end of the image of the nozzle and is parallel to the vertical direction of the first image. A line obtained by correcting a line based on the detection result of the angle sensor may be set as the dividing line.

In the liquid droplet measuring system, the circular area calculator calculates the longest of the line segments connecting the center of the lower end of the nozzle image and each point on the contour of the portion extracted from the fourth image. A line segment may be used as the dividing line.

In the droplet measuring system described above, the circle area calculation unit calculates the width center point at the position where the width in the horizontal direction is the maximum in the portion extracted from the fourth image, and the center of the lower end of the nozzle image. may be used as the dividing line.

A droplet measuring system according to still another aspect of the present invention is an imaging device for imaging a subject and outputting image data in the droplet measuring system for measuring the volume of a droplet dropped from a nozzle, wherein the nozzle and an imaging device installed so as to include the entire droplet hanging down from the nozzle in the field of view, and the state of the droplet dropping from the nozzle based on the image data output from the imaging device. an image generation unit that generates the images in chronological order; and a dropping start image, which is an image showing a state in which a droplet has separated from the nozzle and started to drop, from the images that are generated in chronological order by the image generation unit. Volume calculation for calculating the volume of the droplet reflected in the dropping start image based on the dropping detection unit to detect, the image before the predetermined frame of the dropping start image, and the image after the predetermined frame of the dropping start image. and the volume calculation unit calculates an upper limit value and a lower limit value of the volume of the droplet based on the volume of the droplet measured within a predetermined time after the droplet starts dropping from the nozzle. and inserts a predetermined provisional value instead of the calculated droplet volume when the droplet volume calculated by the volume calculator is out of the upper and lower limits. It has a corrector.

In the above droplet measuring system, the provisional value may be an average value of droplet volumes measured within the predetermined time.

In the above-described droplet measurement system, the provisional value is an average value of the droplet volumes calculated during a predetermined number of previous times when the droplet volumes deviating from the upper limit and the lower limit were calculated. Also good.

In the droplet measuring system described above, the volume calculation unit may set the lower limit value according to a set value of the flow rate of droplets dropped from the nozzle.

In the above-described droplet measurement system, the volume calculation unit may output an average value of droplet volumes calculated over a plurality of times in the past as the droplet volume measurement value.

In the above droplet measuring system, the volume calculator may set the number of times of averaging the volume of the droplets according to a set value of the flow rate of the droplets dropped from the nozzle.

A droplet measuring method according to still another aspect of the present invention is a droplet measuring method executed in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle, in which an object is imaged and image data is output. An imaging device, wherein droplets are dropped from the nozzle based on image data output from an imaging device installed so as to include the tip of the nozzle and the entire droplet hanging down the nozzle within a field of view. an image generating step of generating images showing a state in chronological order; and dripping, which is an image showing a state in which a droplet has separated from the nozzle and started to drip from the images chronologically generated in the image generating step. Based on a drop detection step of detecting a start image, an image before a predetermined frame of the drop start image, and an image after a predetermined frame of the drop start image, the volume of the droplet reflected in the drop start image is calculated. and a volume calculation step, wherein the dropping detection step includes a predetermined position at a predetermined distance below the position where the tip of the nozzle is captured in the images generated in chronological order in the image generation step. A dropping standby determination step of setting a rectangular region having a size and monitoring the inside of the rectangular region to detect a dropping standby image, which is an image showing a state in which a droplet is hanging from the tip of the nozzle. and, after the dropping standby image is detected, the rectangular area is set for the image generated in the image generating step, and the inside of the rectangular area is monitored to detect the dropping start image. and a determination step.

A droplet measuring method according to still another aspect of the present invention is a droplet measuring method executed in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle, in which an object is imaged and image data is output. An imaging device, wherein droplets are dropped from the nozzle based on image data output from an imaging device installed so as to include the tip of the nozzle and the entire droplet hanging down the nozzle within a field of view. an image generating step of generating images showing a state in chronological order; and dripping, which is an image showing a state in which a droplet has separated from the nozzle and started to drip from the images chronologically generated in the image generating step. Based on a drop detection step of detecting a start image, an image before a predetermined frame of the drop start image, and an image after a predetermined frame of the drop start image, the volume of the droplet reflected in the drop start image is calculated. and a volume calculation step, wherein the volume calculation step includes a region extraction unit step of extracting a region of the droplet image captured in the image of the predetermined frame before; and rotating the region extracted in the region extraction step. a circle area calculating step of calculating an area corresponding to a circle obtained by slicing the body of revolution based on the number of pixels arranged in the extracted region, which is regarded as a projected image of the body; and an accumulating step of accumulating the area, wherein the area extracting step generates a difference image between a first image that is an image before the predetermined frame and a second image that is an image after the predetermined frame. and generating a third image by performing a binarization process on the difference image, and generating a fourth image obtained by performing a filling process on the third image, setting a region between the first and second images to define an overlapping drop image portion based on the third and fourth images, and applying the region to the first image; By extracting the portion of the overlapping droplet image by combining the portion of the overlapping droplet image and the portion extracted from the fourth image, the droplet reflected in the first image is to extract the area of the entire image.

A droplet measuring method according to still another aspect of the present invention is a droplet measuring method executed in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle, in which an object is imaged and image data is output. The imaging device is installed so that the tip of the nozzle and the entire droplet hanging down from the nozzle are within the field of view. an image generating step of generating the captured images in chronological order; and a dropping start image, which is an image representing a state in which a droplet has separated from the nozzle and started to drip from the images generated in chronological order in the image generating step. a drop detection step of detecting the drop start image, an image before a predetermined frame of the drop start image, and an image after a predetermined frame of the drop start image, and the volume of the droplet reflected in the drop start image is calculated. and a calculating step, wherein the volume calculating step calculates the upper limit and the lower limit of the volume of the droplet based on the volume of the droplet measured within a predetermined time after the droplet starts dropping from the nozzle. setting a value, and inserting a predetermined provisional value in place of the calculated droplet volume when the droplet volume calculated by the volume calculator is out of the upper and lower limits; By doing so, the error is corrected.

According to still another aspect of the present invention, there is provided a program for imaging a subject and outputting image data in a program executed by a computer in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle. , an image of a droplet dripping from the nozzle, based on image data output from an imaging device installed so as to include the tip of the nozzle and the entire droplet hanging down from the nozzle in a field of view; and detecting, from the images generated in chronological order in the image generating step, a dropping start image, which is an image showing a state in which the droplet has separated from the nozzle and started to drop. a dropping detection step; and a volume calculating step of calculating the volume of the droplet reflected in the dropping start image based on an image before the dropping start image by a predetermined frame and an image after the dropping start image by a predetermined frame. is executed, and the droplet detection step has a predetermined size at a position a predetermined distance below the position where the tip of the nozzle is captured with respect to the images generated in chronological order in the image generation step. a dropping standby determination step of setting a rectangular region and detecting a dropping standby image, which is an image showing a state in which a droplet is hanging from the tip of the nozzle, by monitoring the rectangular region; a dripping start determination step of detecting the dripping start image by setting the rectangular area for the image generated in the image generating step after the standby image is detected, and monitoring the rectangular area; includes.

According to still another aspect of the present invention, there is provided a program for imaging a subject and outputting image data in a program executed by a computer in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle. , an image of a droplet dripping from the nozzle, based on image data output from an imaging device installed so as to include the tip of the nozzle and the entire droplet hanging down from the nozzle in a field of view; and detecting, from the images generated in chronological order in the image generating step, a dropping start image, which is an image showing a state in which the droplet has separated from the nozzle and started to drop. a dropping detection step; and a volume calculating step of calculating the volume of the droplet reflected in the dropping start image based on an image before the dropping start image by a predetermined frame and an image after the dropping start image by a predetermined frame. , and the volume calculation step includes: a region extracting unit step of extracting a region of the image of the droplet captured in the image of the predetermined frame before; a circle area calculation step of calculating an area corresponding to a circle obtained by slicing the body of revolution based on the number of pixels arranged in the extracted region; and the area extraction step generates a difference image between a first image that is an image before the predetermined frame and a second image that is an image after the predetermined frame, and generating a third image by subjecting the image to binarization processing, and generating a fourth image obtained by subjecting the third image to fill-in-the-blank processing; 4, setting an area between the first and second images to define a portion of the overlapping droplet image, and applying the area to the first image to obtain the overlapping liquid. By extracting the droplet image portion and synthesizing the overlapping droplet image portion and the portion extracted from the fourth image, the entire droplet image reflected in the first image is obtained. It extracts regions.

According to still another aspect of the present invention, there is provided a program for imaging a subject and outputting image data in a program executed by a computer in a droplet measuring system for measuring the volume of a droplet dropped from a nozzle. , an image depicting the state of droplets dropping from the nozzles is displayed based on the image data output from the apparatus installed so as to include the tip of the nozzles and the entire droplets hanging from the nozzles in the field of view. an image generation step for generating images in sequential order; drip detection for detecting a drip start image, which is an image showing a state in which a droplet has separated from the nozzle and started dripping, from the images generated in chronological order in the image generation step. and a volume calculation step of calculating the volume of the droplet reflected in the drip start image based on the image before the predetermined frame of the drip start image and the image after the predetermined frame of the drip start image. In the volume calculation step, the upper limit and the lower limit of the droplet volume are set based on the volume of the droplet measured within a predetermined time after the start of dropping the droplet from the nozzle. , by inserting a predetermined provisional value instead of the calculated droplet volume when the droplet volume calculated by the volume calculator is out of the upper limit value and the lower limit value is corrected.

According to one aspect of the present invention, a rectangular area having a predetermined size is set at a predetermined position in the images generated in chronological order by the image generation unit, and the inside of the rectangular area is monitored. Since the detection of the drip standby image and the detection of the drip start image after the drip standby image is detected are alternately performed, the drip start image can be obtained from a series of images that are sequentially generated without requiring a large computational load. can be detected well. Therefore, it is possible to extract an image used for droplet volume calculation in real time.

Further, according to another aspect of the present invention, not only the area extracted based on the difference image between the image before the predetermined frame of the dropping start image and the image after the predetermined frame, but also in both images, Since the area that should be extracted as part of the image of the droplet can be extracted, the measurement accuracy of the volume of the droplet can be improved.

Further, according to still another aspect of the present invention, when the calculated droplet volume is out of the set range, a predetermined provisional value is inserted instead of the calculated value. Even if the liquid drops in such a state and an abnormal value is calculated, the influence of the abnormal value on a series of droplet volume measurements can be suppressed.

1 is a diagram showing a schematic configuration of a droplet measuring system according to an embodiment of the invention; FIG. 2 is a block diagram showing a schematic configuration of an information processing apparatus shown in FIG. 1; FIG. 4 is a flow chart that schematically illustrates the operation of a droplet measurement system according to an embodiment of the invention; 4 is a flowchart showing nozzle positioning processing. FIG. 5 is a schematic diagram for explaining nozzle positioning processing; FIG. 5 is a schematic diagram for explaining nozzle positioning processing; 6 is a flowchart showing nozzle detection processing; FIG. 5 is a schematic diagram for explaining nozzle detection processing; FIG. 5 is a schematic diagram for explaining nozzle detection processing; 4 is a flow chart showing a dripping state determination process according to the embodiment of the present invention. 4 is a flowchart showing droplet volume calculation processing in the embodiment of the present invention. 9 is a flow chart showing a modification of the dropping state determination process according to the embodiment of the present invention. FIG. 4 is a schematic diagram for explaining a method of setting a monitoring area; FIG. 4 is a schematic diagram for explaining a method of setting a monitoring area; FIG. 4 is a schematic diagram for explaining a method of setting a monitoring area; FIG. 4 is a schematic diagram for explaining a method of setting a monitoring area; 6 is a flow chart showing determination processing of a drip standby image in the embodiment of the present invention. It is a schematic diagram for demonstrating the drip standby determination process in embodiment of this invention. It is a schematic diagram for demonstrating the modification of the drip standby determination process in embodiment of this invention. 6 is a flow chart showing determination processing of a droplet image in the embodiment of the present invention. FIG. 10 is a schematic diagram for explaining the determination processing of the droplet image in the embodiment of the present invention; FIG. 11 is a flow chart showing a second modification of the droplet image determination process according to the embodiment of the present invention. FIG. FIG. 11 is a schematic diagram for explaining a second modification of the droplet image determination process according to the embodiment of the present invention; FIG. 11 is a flow chart showing a third modification of the droplet image determination process according to the embodiment of the present invention. FIG. FIG. 11 is a schematic diagram for explaining a third modification of the droplet image determination process according to the embodiment of the present invention; FIG. 11 is a schematic diagram for explaining a third modification of the droplet image determination process according to the embodiment of the present invention; FIG. 11 is a schematic diagram for explaining a third modification of the droplet image determination process according to the embodiment of the present invention; 5 is a flow chart showing extraction processing of a droplet image region in the embodiment of the present invention. FIG. 10 is a diagram showing images before and after a dropping start image generated in chronological order; FIG. 5 is a schematic diagram for explaining extraction processing of a droplet image region in the embodiment of the present invention. FIG. 5 is a schematic diagram for explaining extraction processing of a droplet image region in the embodiment of the present invention. FIG. 5 is a schematic diagram for explaining extraction processing of a droplet image region in the embodiment of the present invention. 4 is a flowchart showing processing for calculating the volume of an extracted region according to the embodiment of the present invention; FIG. 10 is a flowchart showing a first modified example of processing for calculating the volume of an extracted region according to the embodiment of the present invention; FIG. FIG. 10 is a diagram showing images before and after a dropping start image generated in chronological order; FIG. 11 is a schematic diagram for explaining a first modification of the process of calculating the volume of an extracted region; FIG. 11 is a schematic diagram for explaining a first modification of the process of calculating the volume of an extracted region; 10 is a flow chart showing a second modification of the process of calculating the volume of an extracted region according to the embodiment of the present invention; FIG. 12 is a schematic diagram for explaining a second modification of the process of calculating the volume of the extracted region; FIG. 11 is a schematic diagram for explaining a third modification of the process of calculating the volume of an extracted region; FIG. 14 is a schematic diagram for explaining a fourth modification of the process of calculating the volume of the extracted region; FIG. 14 is a schematic diagram for explaining a fourth modification of the process of calculating the volume of the extracted region; FIG. 5 is a schematic diagram for explaining an example of droplet volume calculation processing; FIG. 10 is a schematic diagram for explaining another example of droplet volume calculation processing; FIG. 5 is a diagram showing rectangular areas set when determining a dropping standby state and a dropping start state in Examples 1 and 2 and Comparative Example 1; FIG. 10 is a diagram showing rectangular areas set when determining a dropping standby state and a dropping start state in Examples 3 and 4 and Comparative Example 2; 4 is a graph showing experimental results of Examples 1 and 2 and Comparative Example 1. FIG. 4 is a graph showing experimental results of Examples 3 and 4 and Comparative Example 2. FIG.

A droplet measuring system, a droplet measuring method, and a program according to embodiments of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited by these embodiments. Also, in the description of each drawing, the same parts are indicated by the same reference numerals.

The drawings referred to in the following description only schematically show shapes, sizes, and positional relationships to the extent that the contents of the present invention can be understood. That is, the present invention is not limited only to the shapes, sizes, and positional relationships illustrated in each drawing. Moreover, even between the drawings, there are cases where portions having different dimensional relationships and ratios are included.

(Configuration of droplet measurement system)
FIG. 1 is a diagram showing a schematic configuration of a droplet measuring system according to one embodiment of the present invention. FIG. 1 shows an example in which a droplet measuring system 10 according to the present embodiment is applied to an infusion device 1 for dripping a liquid (infusion liquid) filled in an infusion bag 2 . The infusion device 1 includes an intermediate tube 3 connected to an infusion bag 2, an infusion tube 4, and an infusion tube 5. A nozzle 6 is attached in the infusion tube 4. As shown in FIG. The droplet measuring system 10 measures the volume of the droplet 7 dropped from the tip portion (hereinafter also referred to as the nozzle tip portion) 6a of the nozzle 6, and controls the drip flow rate based on the measured volume.

However, the droplet measurement system 10 according to the present embodiment is not limited to the drip device 1, and can be applied to various devices that drip droplets from the tip of a nozzle, such as a pipette that ejects a predetermined amount of liquid. It is possible.

As shown in FIG. 1, the infusion bag 2 is a container filled with an infusion liquid such as a drug solution or a nutrient, and is held by being suspended from a support base or the like during infusion. The intermediate tube 3 has one end connected to the drainage port 2 a of the infusion bag 2 and the other end connected to one end of a nozzle 6 attached to the upper lid 4 a of the drip tube 4 . The other end of this nozzle 6 is provided so as to protrude into the drip tube 4 .

The infusion tube 5 is made of an elastic material. A clamp 8 capable of radially pressing the infusion tube 5 and an actuator 9 for driving the clamp 8 are provided in the middle of the infusion tube 5 .

The actuator 9 changes the pressing force of the clamp 8 against the infusion tube 5 by driving the clamp 8 under electrical control. As a result, the inner diameter of the infusion tube 5 changes (opens and closes), and the flow rate of the infusion liquid flowing through the infusion tube 5 can be adjusted. Along with this, the internal pressure of the drip tube 4 changes, and the dropping cycle of the droplets 7 dropped from the nozzle 6, in other words, the flow rate of the liquid per unit time changes.

The droplet measurement system 10 includes a light source 11 that illuminates the vicinity of the nozzle tip 6a, a camera 12 that captures an image of the vicinity of the nozzle tip 6a to generate image data, and liquid based on the image data generated by the camera 12. and an information processing device 100 that calculates the volume of the droplet. Further, the droplet measurement system 10 may further include a display device 13 for displaying the calculation result of the droplet volume, etc., and an inclination sensor 14 attached to the drip tube 4 .

The light source 11 includes, for example, a light emitting element such as an LED (Light Emitted Diode), and an optical system such as a filter and a lens that controls light distribution so that light emitted from the light emitting element becomes parallel light. The light source 11 is installed so as to face the field of view of the camera 12, and illuminates the vicinity of the nozzle tip 6a from which the droplets are dropped from behind the droplets 7. As shown in FIG.

The camera 12 is an imaging device that has an imaging element 12a such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) and is capable of imaging moving images or still images at a predetermined imaging frame rate. The imaging device 12a receives light (object image) that is incident on the camera 12 and formed into an image by the optical system on its light receiving surface, and photoelectrically converts the light to generate an electric signal. The camera 12 generates and outputs image data by performing predetermined signal processing such as amplification and A/D conversion on the electrical signal.

The specifications of the camera 12 can be appropriately configured according to the drip device 1 to be measured. As an example, if the infusion device 1 is a device commonly used in the medical field, the camera module has an outer diameter of It is preferable to use a compact camera with a focal length of several mm to several tens of mm and a focal length of several mm to several tens of mm.

A general-purpose product having a total number of pixels of 500,000 pixels or less can be used as the image sensor 12a. Specifically, the number of pixels in the longitudinal direction should be about 480 to 800, the number of pixels in the width direction should be about 320 to 600, and the aspect ratio should be 1 or more. As will be described later, in the present embodiment, the camera 12 is installed such that the longitudinal direction of the imaging device 12a is the vertical direction and the short side direction is the horizontal direction. Therefore, hereinafter, the longitudinal direction of the imaging element 12a is referred to as the vertical direction, and the lateral direction thereof is referred to as the horizontal direction.

The imaging frame rate of the camera 12 may be in the range of about 30 to 120 fps (frame rate/second). A frame rate of about 30 fps may be used when low cost is emphasized. In this case, when the set flow rate of the drip is high (i.e., when the drip cycle is short), measurement may be difficult, but the flow rate set for general drips can be adequately handled. Yes, and the algorithm described below can cover the loss of accuracy due to low frame rates. In addition, the frame rate may be set to 60 fps or more when the accuracy is particularly important or when measuring a high-flow drip. In this case, the cost of the camera will be higher, but the higher the frame rate, the better the performance can be expected. Preferably, the imaging frame rate is variable within the above range.

The camera 12 is installed so as to cover the nozzle tip 6a and a predetermined range below the nozzle tip 6a. Specifically, it is sufficient that the entire droplet hanging down on the nozzle tip portion 6a immediately before being dropped is within the field of view. At this time, part or all of the falling liquid droplets separated from the nozzle tip portion 6a may be out of the field of view. Since the size of the droplets hanging down from the nozzle tip 6a changes depending on conditions such as the diameter of the nozzle, the viscosity of the liquid, and the dropping cycle, the work distance WD (the distance from the subject to the tip of the lens of the camera 12) is adjusted. Therefore, it is preferable to determine the range of the subject to be included in the field of view of the camera 12 . For example, when using a nozzle for adults (for 20 drops/mL) for dropping 1 mL in 20 drops, the dripping drops are relatively large, so it is preferable to lengthen the working distance WD. Also, when using a nozzle for children (for 60 drops/mL) for dropping 60 drops of 1 mL, the working distance WD may be short because the dropping drops are relatively small.

Preferably, the camera 12 may be provided with an object-side telecentric lens. Here, since the position and inclination of the nozzle 6 in the drip tube 4 may differ depending on the individual, even if the standard distance between the drip tube 4 and the camera 12 is determined, the actual working distance WD changes. It can happen. In such a case, if light is incident on the image sensor 12a through a normal condenser lens, the size of the subject image on the light receiving surface of the image sensor 12a will change, and an error will occur in the process of calculating the volume of the droplet 7. may occur. On the other hand, by allowing light to enter the image sensor 12a via a telecentric lens, it is possible to suppress the variation in the size of the subject image on the light-receiving surface even when the working distance WD changes. Note that the camera 12 may be provided with a telecentric lens on the imaging side, but it is not essential.

FIG. 2 is a block diagram showing a schematic configuration of the information processing apparatus 100. As shown in FIG. As the information processing device 100, a general-purpose information processing device such as a personal computer (PC) or a notebook PC can be used in addition to a device configured exclusively for the droplet measuring system 10. FIG. As shown in FIG. 2 , the information processing apparatus 100 includes an input/output unit 101 , an operation input unit 102 , a storage unit 103 and a calculation unit 104 .

The input/output unit 101 is an external interface that inputs/outputs image data and various signals to/from various external devices such as the camera 12 and the display device 13 .

The operation input unit 102 is configured by an input device such as an input button, switch, keyboard, mouse, touch panel, etc., and inputs a signal according to the operation performed by the user to the calculation unit 104 . Operations performed on the operation input unit 102 include an operation to start or end imaging by the camera 12, an operation to start or end dripping (dripping of droplets from the nozzle 6), and an operation to start or end dripping. An operation of inputting a flow rate set value, etc., can be mentioned.

The storage unit 103 is configured using a computer-readable storage medium such as a disk drive and semiconductor memory such as ROM and RAM. The storage unit 103 may be constructed by mapping multiple physical devices to one logical device, or may be constructed by mapping one physical device to multiple logical devices. The storage unit 103 stores an operating system program, a driver program, a program for causing the information processing apparatus 100 to execute a predetermined operation, various data and setting information used during execution of the program, and the like.

Specifically, the storage unit 103 includes a program storage unit 111 that stores a droplet measurement program for measuring the volume of droplets dropped from the nozzle 6, and a program storage unit 111 that is output from the camera 12 and used for droplet volume measurement. An image data storage unit 112 for storing image data of an image obtained by the measurement, a setting value storage unit 113 for storing various setting values used for droplet volume measurement, and a measurement value storage unit for storing measurement values of the droplet volume. 114, and a droplet detection image buffer 115 and a volume calculation image buffer 116 provided as buffer storage areas for temporarily storing image data. The dropping detection image buffer (first buffer storage area) 115 temporarily stores image data of an image output from the camera 12 and subjected to predetermined image processing by an image generation unit 131, which will be described later. A volume calculation image buffer (second buffer storage area) 116 temporarily stores image data of an image used for droplet volume measurement among the image data stored in the drop detection image buffer 115 .

The arithmetic unit 104 is configured using hardware such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor), and reads and executes a program stored in the program storage unit 111. As a result, data transfer and instructions to each unit of the information processing apparatus 100 are performed, and the operation of the information processing apparatus 100 is controlled as a whole. Further, the calculation unit 104 executes the droplet measurement program stored in the program storage unit 111 to perform calculation processing for measuring the volume of the droplet dropped from the nozzle 6 based on the image data acquired from the camera 12. to run. Specifically, the functional units implemented by the arithmetic unit 104 executing the droplet measurement program include an imaging control unit 121 , a flow control unit 122 , and an image processing unit 123 .

The imaging control unit 121 controls the start and end of imaging by the camera 12, and controls the operation of the camera 12 so as to perform imaging at a predetermined imaging frame rate. Further, the imaging control unit 121 may perform control to limit the imaging area of the imaging element 12a. For example, for the imaging device 12a having 800×600 pixels, the effective imaging area can be variably set within the range of (480 to 800)×(320 to 600) pixels, and Control is performed so that image signals are acquired only from the arranged pixels. By making the imaging area variable in this way, regardless of conditions such as the diameter of the nozzle, the viscosity of the liquid to be dripped, and the period of dripping, while keeping the entire droplet hanging down from the nozzle 6 within the field of view, Processing load can be reduced.

The flow control unit 122 controls the operation of the actuator 9 in accordance with a signal input from the operation input unit 102 to start and end the dropping of droplets from the nozzle 6, and the image processing unit 123 described later. Based on the calculated droplet volume, the operation of the actuator 9 is controlled so as to achieve a preset flow rate.

The image processing unit 123 includes an image generation unit 131, an error determination unit 132, a flow rate estimation unit 133, a drop detection unit 134, and a volume calculation unit 135, and performs image data input from the camera 12. A predetermined process is performed to calculate the volume of the droplet dropped from the nozzle 6 .

Specifically, the image generation unit 131 applies predetermined image processing such as demosaicing, white balance processing, and gamma correction to image data sequentially input from the camera 12 , thereby causing droplets to drop from the nozzles 6 . To generate images showing the state in chronological order.

The error determination unit 132 determines whether the position of the camera 12 relative to the nozzle 6 is appropriate based on the position of the image of the nozzle tip 6a in the image.

The flow rate estimator 133 estimates the standard flow rate of droplets dropped from the nozzle 6 based on at least one of the shape and thickness of the image of the nozzle tip portion 6a in the image. Here, the standard flow rate is a flow rate value (range) that can be set for each type of nozzle, such as 20 drops/mL and 60 drops/mL.

The droplet detection unit 134 determines the state of the droplet in the images generated in chronological order by the image generation unit 131, thereby detecting the state in which the droplet has left the nozzle 6 and started to drip. A dropping start image, which is a photographed image, is detected. Specifically, the drip detection unit 134 includes a drip standby determination unit 141 and a drip start determination unit 142 .

The drip standby determination unit 141 sets a rectangular area having a predetermined size at a position below the position where the nozzle tip 6a is captured by a predetermined distance in the images sequentially generated by the image generation unit 131, and By monitoring the inside of the rectangular area, a dropping standby image, which is an image showing a state (dropping standby state) in which a droplet hangs down on the nozzle tip portion 6a, is detected. After the dropping standby image is detected, the dropping start determination unit 142 monitors the rectangular region in the images sequentially generated by the image generating unit 131 to determine whether the droplets hanging down from the nozzle 6 are removed from the nozzle 6. A dropping start image, which is an image showing a state in which the droplet starts to fall apart, is detected. Hereinafter, the rectangular area set in the image by the dropping standby determination unit 141 and the dropping start determination unit 142 is referred to as a monitoring area.

The volume calculation unit 135 uses an image of a predetermined frame before the drip start image and an image after the predetermined frame of the drip start image among the images of a plurality of continuous frames including the drip start image. ) to calculate the droplet volume. Specifically, the volume calculating unit 135 regards the region extracting unit 143 that extracts the region of the image of the droplet captured in the image a predetermined frame before the dropping start image, and the extracted region as the projected image of the rotating body, It includes a circle area calculator 144 that calculates the area of a circle obtained by slicing the body of revolution, and an accumulator 145 that integrates the calculated areas of the circles. The volume calculation unit 135 includes an error correction unit 146 that corrects the calculated droplet volume, and a high flow rate adjustment unit 147 that adjusts the droplet volume calculation algorithm according to the set value of the droplet flow rate. May contain more.

Note that the hardware configuration of the computing unit 104 is not limited to that described above, and each functional configuration of the computing unit 104 may be realized using a circuit such as an FPGA (Field Programmable Gate Array).

The display device 13 is configured by a liquid crystal display, an organic EL display, or the like, and displays control signals output from the information processing device 100, images generated by the information processing device 100, and droplets under the control of the information processing device 100. display the measured values, etc.

The tilt sensor 14 is composed of, for example, a gyro sensor or an acceleration sensor, and detects the tilt of the drip tube 4 with respect to the vertical axis. Here, since the nozzle 6 is fixed to the drip tube 4, it can be considered that the inclination of the drip tube 4 is substantially equal to the inclination of the nozzle 6. FIG.

(Operation of droplet measurement system)
Next, the operation of droplet measurement system 10 will be described. FIG. 3 is a flow chart illustrating the operation of droplet measurement system 10 .
Prior to starting the drip, the user installs the light source 11 and the camera 12 near the drip tube 4 (see FIG. 1). At this time, the user displays an image on the display device 13 by operating the information processing device 100 to cause the camera 12 to perform imaging, and while viewing the image, the user operates the nozzle tip portion 6a and a predetermined portion below the nozzle tip portion 6a. It is preferable to adjust the positional relationship of the light source 11, the drip tube 4, and the camera 12 so that the range falls within the field of view of the imaging element 12a.

In step S<b>10 , the information processing apparatus 100 sets the flow rate of the droplets 7 to be dropped from the nozzles 6 based on the signal input from the operation input unit 102 .

In subsequent step S<b>20 , the information processing device 100 initializes the drip infusion device 1 . FIG. 4 is a flow chart showing the nozzle positioning process executed as one of the initial settings. 5 and 6 are schematic diagrams for explaining the nozzle positioning process.

Here, if the relative position between the camera 12 and the nozzle tip portion 6a (droplet discharge port) is indefinite, it becomes difficult to detect the timing at which the droplet 7 leaves the nozzle 6 and starts falling. There is a possibility that the problem that the measurement precision of the volume of 7 will fall will arise. Therefore, as shown in FIG. 5A, it is necessary to arrange the nozzle 6 so that the center of the droplet 7 is positioned substantially at the center of the angle of view of the camera 12. FIG. This is because, as shown in FIG. 5B, the end m12 of the nozzle seen from the front is reflected in the image m10, and the distortion of the shape of the droplet image m11 can be reduced as much as possible. On the other hand, as shown in FIG. 6(a), if the nozzle tip 6a is positioned at the end of the angle of view of the camera 12, the nozzle 6 is seen obliquely from above. It becomes impossible to clearly grasp the moment when the droplet 7 separates from the . In addition, as shown in FIG. 6B, in the image m13, the shape of the droplet image m14 is greatly distorted.

Therefore, in the present embodiment, as shown in FIG. 4, the information processing apparatus 100 causes the camera 12 to perform image capturing (step S101) after setting the drip device 1 and before starting the drip (step S101). Create an image with 6 in it.

In response, the error determination unit 132 detects the positions of the lower ends of the nozzles in the generated image (step S102). Subsequently, the error determination unit 132 determines whether or not the position of the lower end of the nozzle is within the predetermined range r1 of the image (step S103).

If the position of the lower end of the nozzle is within the predetermined range r1 (step S103: Yes, see FIG. 5), the error determination unit 132 permits the flow control unit 122 to start dripping (step S104). After that, the process returns to the main routine. In this case, when the user operates the operation input unit 102 to start dripping, the flow rate control unit 122 starts operating the actuator 9 at the set flow rate.

On the other hand, if the position of the lower end of the nozzle is not within the predetermined range r1 (step S103: No, see FIG. 6), the error determination unit 132 outputs an error signal (step S105), and then the process returns to step S101. In this case, the flow controller 122 does not start the operation of the actuator 9 even if the user performs an operation to start dripping. As a result, it is possible to prevent an error that the drip is started in a state where the positional relationship between the camera 12 and the nozzle tip portion 6a is inappropriate. In this case, the user has to adjust the relative positions of the camera 12 and the nozzle 6 .

FIG. 7 is a flow chart showing nozzle detection processing executed as another initial setting (step S20 in FIG. 3). 8 and 9 are schematic diagrams for explaining the nozzle detection process.

Here, there are two types of general drip devices, one for 20 drops/mL and one for 60 drops/mL. flow rate) is specified. The flow rate is usually manually set by the user, but if the set flow rate deviates from the specified range, it becomes difficult to accurately control the flow rate.

Therefore, in the present embodiment, the nozzle positioning process shown in FIG. 4 is executed, and after the positional relationship between the camera 12 and the nozzle tip portion 6a is appropriately adjusted, information processing is performed again as shown in FIG. The device 100 causes the camera 12 to take an image (step S111), and generates an image in which the nozzle 6 is captured.

In response to this, the error determination unit 132 detects the image of the nozzle from the generated image, and measures the thickness of the vicinity of the tip of the nozzle based on this image (step S112). Alternatively, instead of or in addition to the thickness near the tip of the nozzle, the shape near the tip may be detected. Subsequently, the error determination unit 132 acquires the flow rate range (specified range) specified for the drip device including the nozzle based on the thickness (and/or shape) of the vicinity of the tip of the nozzle (step S113).

For example, the nozzle image m21 in the image m20 shown in FIG. 8 represents a nozzle for 20 droplets/mL. A nozzle image m23 shown in the image m22 shown in FIG. 9 represents a nozzle for 60 droplets/mL. The nozzle for 20 droplets/mL and the nozzle for 60 droplets/mL are different in thickness and shape, the latter being thinner and straighter. Therefore, by measuring the thickness w1 to w4 of the nozzle images m21 and m23 in the images m20 and m22, the thickness and shape of the set nozzle 6 can be detected. use, etc.).

Subsequently, the error determination unit 132 determines whether or not the flow rate set in step S10 (see FIG. 3) is within the specified flow rate range obtained in step S113 (step S114). If the set value of the flow rate is within the specified range (step S114: Yes), the error determination unit 132 permits the flow control unit 122 to start dripping (step S115). After that, the process returns to the main routine. In this case, when the user operates the operation input unit 102 to start dripping, the flow rate control unit 122 starts operating the actuator 9 at the set flow rate.

On the other hand, if the set value of the flow rate is outside the specified range (step S114: No), the error determination unit 132 outputs an error signal (step S116), and then the process returns to step S111. In this case, the flow controller 122 does not start the operation of the actuator 9 even if the user performs an operation to start dripping. This prevents the error of starting an infusion with an inappropriate flow rate. In this case, the user needs to either reset the flow rate so that it falls within the specified flow rate range, or replace the infusion device 1 .

Referring to FIG. 3 again, in step S30 following step S20, the information processing apparatus 100 causes the camera 12 to start imaging. Accordingly, image data is sequentially input from the camera 12 to the information processing apparatus 100 . In the information processing apparatus 100 , the image generation unit 131 performs predetermined image processing on an image based on the input image data, and then temporarily stores the image data in the drop detection image buffer 115 .

In subsequent step S40, the information processing apparatus 100 starts dripping (infusion) by controlling the actuator 9 according to the signal input from the operation input unit 102. FIG.

In step S<b>50 , the information processing device 100 measures the volume of the droplet based on the image data sequentially input from the camera 12 . In step S<b>50 , the droplet dropping state determination processing by the dropping detection unit 134 and the volume calculation processing by the volume calculation unit 135 are executed in parallel.

FIG. 10 is a flowchart showing the dropping state determination process in the droplet volume measurement process (step S50 in FIG. 3). First, the drop detection unit 134 takes in image data from the drop detection image buffer 115 (step S131), and the drop standby determination unit 141 displays an image based on the taken image data in a state in which a droplet is hanging down the nozzle. (step S132). This determination is made by setting a rectangular monitoring area having a predetermined size at a position below the image of the nozzle tip by a predetermined distance from the image to be determined, and monitoring the monitoring area. done. Hereinafter, the state in which the droplets are hanging down from the nozzle is referred to as a drop standby state, and an image showing this state is also referred to as a drop standby image.

If the determination target image is not the drip standby image (step S133: No), the process returns to step S131. In this case, similar processing (steps S131 to S133) is performed on the image of the next frame of the image that has already been determined. On the other hand, if the image to be determined is the drip standby image (step S133: Yes), the drip detection unit 134 recognizes that the drip standby state is present (step S134).

Subsequently, the drip detection unit 134 acquires the image data of the next frame of the determination target image in step S132 from the drip detection image buffer 115 (step S135), and the drip start determination unit 142 detects the acquired image data. A determination is made as to whether or not the image is an image showing a state in which the liquid droplets hanging down on the nozzle are separated from the nozzle and are falling (step S136). This determination is also made by monitoring the set area in the same manner as in step S132. Hereinafter, an image showing a state in which droplets that have been hanging down from the nozzle are separated from the nozzle and are falling is also referred to as a droplet image. Further, the state in which the droplet that has been hanging down from the nozzle has separated from the nozzle and has begun to fall is called a dropping start state, and an image showing this state is also called a dropping start image.

If the determination target image is not the drip image (step S137: No), the process returns to step S135. In this case, similar processing (steps S135 to S137) is performed on the image of the next frame of the image that has already been determined. On the other hand, if the image to be determined is the dripping image (step S137: Yes), the dripping detection unit 134 recognizes that the dripping standby state has transitioned to the dripping start state (step S138), and converts the dripping image to the dripping start image. and

Subsequently, the drop detection unit 134 copies the image data of the image used for volume calculation among the image data stored in the drop detection image buffer 115 to the volume calculation image buffer 116 (step S139). Images used for volume calculation are images before and after a predetermined frame of the dropping start image. However, the drip detection unit 135 may copy the image data of the drip start image to the volume calculation image buffer 116 in addition to the image data of the image used for volume calculation, or copy the drip start image from the drip standby image. Image data of a series of images up to an image after a predetermined frame may be copied.

After that, the drip detection unit 134 determines whether or not the imaging has ended. Specifically, the image generated by the image generating unit 131 when the image of the nozzle 6 is no longer detected from the image generated by the image generating unit 131 is operated to instruct the operation input unit 102 to end imaging. When the droplet image is no longer detected continuously for a predetermined number of frames or more in the image captured, the droplet detection unit 134 determines that the imaging has ended. If it is not determined that the imaging has ended (step S140: No), the process returns to step S131. On the other hand, if it is determined that the imaging has ended (step S140: Yes), the process returns to the main routine.

FIG. 11 is a flow chart showing the volume calculation process in the droplet volume measurement process (step S50 in FIG. 3). First, the volume calculation unit 135 fetches image data from the volume calculation image buffer 116 (step S151), and extracts a droplet image region from an image based on the fetched image data (step S152). Subsequently, the volume calculation unit 135 calculates the volume of the region (body of revolution) by regarding the region extracted from the image as a projected image of the body of revolution (step S153).

Furthermore, the volume calculation unit 135 calculates the ratio of the number of pixels of the nozzle image in the image to the actual size of the nozzle, the magnification of the optical system in the camera 12, and the work distance (the distance between the nozzle 6 and the camera 12) WD. Based on this, the volume of the area extracted from the image is converted to the actual scale to calculate and output the volume of the droplet (step S154). At this time, the volume calculation unit 135 may output the calculated volume of the droplet as it is as the volume of the droplet reflected in the dropping start image. The average value of the calculated volume may be output as the volume of the droplet reflected in the dropping start image. The output droplet volume is stored in the measured value storage unit 114 and displayed on the display device 13 .

After that, the volume calculation unit 135 determines whether or not the imaging has ended (step S155). For example, the volume calculation unit 135 determines that imaging has ended when new image data is no longer stored in the volume calculation image buffer 116 . If it is not determined that the imaging has ended (step S155: No), the process returns to step S151. On the other hand, if it is determined that the imaging has ended (step S155: Yes), the process returns to the main routine.

Referring to FIG. 3 again, when the droplet volume measurement processing in step S50, that is, the droplet dropping state determination processing by the droplet detection unit 134 and the volume calculation processing by the volume calculation unit 135 are completed, droplet measurement is performed. System 10 ends operation.

As described above, according to the present embodiment, the nozzle positioning process is executed before starting the dropping of droplets, so the timing at which the droplet 7 leaves the nozzle 6 and begins to fall can be reliably detected. In addition, it becomes possible to improve the measurement accuracy of the droplet volume.

Further, according to the present embodiment, an error determination of the set value of the flow rate is performed before starting dripping of droplets, so it is possible to prevent a situation in which flow rate control becomes impossible after starting dripping.

Further, according to the present embodiment, the drip standby state is recognized based on the image data that is sequentially generated while the imaging is repeated, and the processing for determining the drip image is executed after the drip standby state is entered. It is possible to efficiently and accurately detect the timing at which the droplet is dropped.

Furthermore, according to the present embodiment, the droplet detection image buffer 115 and the volume calculation image buffer 116 are selectively used, and the droplet droplet droplet state determination processing and the droplet volume measurement processing are performed in parallel. It is possible to measure the volume of individual droplets that are dropped sequentially in real time at high speed.

(Modification of drip state determination processing)
FIG. 12 is a flow chart showing a modification of the dropping state determination process according to the embodiment of the present invention. The processing in steps S161 to S164 in FIG. 12 is the same as the processing in steps S131 to S134 in FIG. 10 described in the above embodiment. In the above embodiment, it is recognized that the state has transitioned to the dripping start state when the dripping image is detected for the first time. Recognize.

That is, following step S164, the drop detection unit 134 initializes (k=0) a counter k that indicates the number of times a drop image has been detected (step S165). Subsequently, the drip detection unit 134 acquires the image data of the next frame of the image to be determined in step S162 from the drip detection image buffer 115 (step S166), and the drip start determination unit 142 acquires image data based on the loaded image data. It is determined whether or not the image is a droplet image (step S167).

If the determination target image is not the drip image (step S168: No), the process returns to step S165. On the other hand, if the image to be determined is the drip image (step S168: Yes), the drip start determination unit 142 increments the counter k (step S169).

Subsequently, the dropping start determination unit 142 determines whether or not the counter k is equal to or greater than a predetermined threshold value _k0 (step S170). If the counter k is less than the threshold _k0 (step S170: No), the process returns to step S166. On the other hand, when the counter k is equal to or greater than the threshold value k ₀ (step S170: Yes), the dropping detection unit 134 recognizes that the state has changed to the dropping start state (step S171), and when the counter k = 0 (that is, the first ) The detected dripping image is set as the dripping start image. Subsequent steps S172-S173 are the same as steps S139-S140 in FIG.

According to this modified example, it is determined that the dropping start state has been entered when the dropping image is detected a plurality of times after entering the dropping standby state. .

(Monitoring area setting)
Next, the monitor area set when executing the drip standby image determination process (see step S132 in FIG. 10) and the drip image determination process (see step S136 in FIG. 10 and step S167 in FIG. 12) will be described in detail. explain. 13 to 16 are schematic diagrams for explaining how to set the monitoring area.

First, a method for setting the maximum range in the vertical direction of the monitoring area will be described. Images m31 to m33 shown in FIG. 13 are the images of the frames immediately before the dropping start image, that is, the last images in the dropping standby state. Between the images m31 to m33, since the conditions such as the type and flow rate of the liquid to be dropped are changed, the sizes (heights h1 to h3) of the images of the droplets hanging from the nozzles are also different.

The upper end of the monitoring region R is set below the tip position of the nozzle image positioned in step S20 (see FIG. 3). On the other hand, the lower end of the monitoring region R is set above the lower end of the image of the droplet hanging down the nozzle in the image of the frame immediately before the dropping start image. That is, at least immediately before the droplet is dropped, the image of the droplet covers the entire monitoring area R from the upper end to the lower end. Here, as shown in images m31 to m33, even if the same nozzle is used, the size of the droplet that hangs down from the nozzle changes when conditions such as the type of liquid and the flow rate are changed. Therefore, preferably, the height h1, h2, and h3 of the droplet image that can change when using the nozzle is the smallest, so that the common monitoring area R can be used even when the conditions are changed. It is preferable to set the upper end position and the lower end position of the monitoring area R so as not to exceed the height h1.

Next, a method for setting the minimum range in the vertical direction of the monitoring area will be described. Images m34 to m36 shown in FIG. 14 are drip start images. Between the images m34 to m36, conditions such as the type and flow rate of the liquid to be dropped are changed. have different gaps d1 to d3.

The monitoring area R is set to a range that includes a gap between the lower end of the image of the droplet adhering to the nozzle and the upper end of the image of the droplet falling away from the nozzle. However, as shown in images m34 to m36, even when the same nozzle is used, the above gap changes when the conditions are changed. Therefore, if there is a common area d among the gaps d1, d2, and d3 that can change when using the nozzle, so that the common monitoring area R can be used even when the conditions are changed. , the monitoring region R should be set so as to include the common region d. In FIG. 14, a gap d between the upper end of the gap d3 in the image m36 and the lower end of the gap d1 in the image m34 is the minimum range of the monitoring area R in the vertical direction. Therefore, it is preferable to set the upper end position and the lower end position of the monitoring region R so as to include the gap d.

Next, another method for setting the minimum range in the vertical direction of the monitoring area will be described. Images m37 to m39 shown in FIG. 15 are dropping start images. Between the images m37 to m39, the gap between the lower edge of the image of the droplet adhering to the nozzle and the upper edge of the image of the droplet falling away from the nozzle due to changes in conditions such as the type and flow rate of the liquid to be dropped. d4 to d6 are different. Between these gaps d4 to d6, there may be no common area depending on conditions. In such a case, in order to use the common monitoring area R even when the conditions are changed, the position where the lower end of the image of the droplet hanging down the nozzle is the lowest and the position where the droplet falling away from the nozzle It is preferable to set the monitoring region R so that the upper end of the image of .DELTA. In FIG. 15, the upper end position and lower end position of the monitoring area R are set so as to include the gap d' between the upper end of the gap d6 of the image m39 and the lower end of the gap d4 of the image 37. In FIG.

Next, a method for setting the minimum range in the horizontal direction of the monitoring area will be described. Images m40 to m42 shown in FIG. 16 are images of frames immediately before the dropping start image. Between the images m40 to m42, the widths w5 to w7 of the droplets hanging from the nozzles are different due to changes in conditions such as the type of liquid to be dropped and the flow rate.

The monitoring area R is set to a range that includes the width of the image of the droplet hanging down from the nozzle. However, as shown in images m40 to m42, even if the same nozzle is used, the width of the droplet changes when the conditions are changed. Therefore, so that the common monitoring region R can be used even when the conditions are changed, it is possible to include the largest width w7 among the widths w5, w6, and w7 that can change when using the nozzle. It is preferable to set the left end position and the right end position of the monitoring area R. However, if the width of the monitoring region R is too large, the amount of search for the monitoring region R increases.

As a specific example of the size of the monitoring region, when a nozzle for 20 droplets/mL is used, the distance from the tip of the nozzle image to the upper end of the monitoring region is 0.4 times to 2.0 times the width of the tip of the nozzle in the image. The width of the monitoring area in the horizontal direction should be about 0.7 to 3.0 times the same, and the length of the monitoring area in the vertical direction should be about 0.3 to 1.5 times the same. When using a nozzle for 60 droplets/mL, the distance from the tip of the nozzle image to the upper end of the monitoring area is about 0.4 to 2.0 times the width of the nozzle tip in the image. The width in the horizontal direction should be about 1.0 to 3.0 times the same, and the length in the vertical direction of the monitoring area should be about 1.0 to 3.0 times the same.

By setting the management area R as described above and monitoring the inside of the management area R of the images generated in chronological order, it is possible to reliably and quickly detect the dripping standby image and the dripping image.

(Determination processing of drip standby image)
Next, the determination processing of the drip standby image (see step S132 in FIG. 10) will be described in detail. FIG. 17 is a flow chart showing determination processing of a drip standby image in this embodiment. FIG. 18 is a schematic diagram for explaining the determination processing of the drip standby image in this embodiment. Images m50 to m53 shown in FIG. 18 represent images in which droplets hanging down from the nozzles gradually increase in size in chronological order. In images m50 to m52, areas within the monitoring area R that have already been searched are shaded in gray.

First, the drip standby determination unit 141 performs binarization processing on the monitoring region set in the determination target image (step S201). In this embodiment, the gradation of pixels is assumed to be 0 to 255 gradations, the gradation of pixels in at least the outline region of the droplet image is 0 gradation (that is, black), and the background of the droplet image is The threshold in the binarization process is set so that the gradation of the pixels in the area is 255 gradations (that is, white).

Subsequently, the drip standby determination unit 141 sets a monitoring region in the image to be determined, and sets a counter i (i=1 to i _max ) representing a row of pixels in the monitoring region to 1 (step S202). In this embodiment, the line at the top of the monitoring area is the first line.

Subsequently, the dropping standby determination unit 141 horizontally searches for pixels in the i-th row in the monitoring area (step S203), and determines whether or not there is a pixel of 0 gradation in the i-th row (step S204).

If there is no 0-tone pixel in the i-th row (step S204: No), that is, if there is no liquid image region in the row, it is determined that the image to be determined is not a drip standby image, and processing is performed. returns to the main routine. For example, in the image m50 and the image m51 shown in FIG. 18, the i-th row pixels are all white. Therefore, it is determined that the image m50 and the image m51 are not drip standby images.

On the other hand, if there is a pixel with gradation 0 in the i-th row (step S204: Yes), the dropping standby determination unit 141 determines that the i-th row is a droplet existing row (step S205).

Subsequently, the dropping standby determination unit 141 determines whether or not the line being searched (i-th line) reaches the last line (i _max line) of the monitoring area (step S206). If the last line has not been reached (step S206: No), the dropping standby determination unit 141 increments the counter i (step S207), and then the process returns to step S203.

On the other hand, when the line being searched reaches the last line (step S206: Yes), the drip standby determination unit 141 determines that the drip standby image has been detected (step S208), and the process returns to the main routine. For example, in the image m52, since all rows of the monitoring region R are droplet presence rows, it is determined to be a dropping standby image.

Next, a modified example of the determination processing of the drip standby image will be described. FIG. 19 is a schematic diagram for explaining a modification of the dropping standby image determination process according to the embodiment of the present invention.
Here, as shown in FIG. 1, when droplets are dropped from the nozzle 6 in the drip tube 4, the liquid accumulated in the drip tube 4 splashes and adheres to the inner wall of the drip tube 4, and the liquid adhered to the inner wall image may appear in the monitoring area R. In such a case, if a row in which at least one pixel of gradation 0 exists is determined as a droplet existing row (see step S204 in FIG. 17), depending on the position of the liquid image in the monitoring area R, the drop standby image may be displayed. False detection may occur.

For example, the images m50′ and m51′ shown in FIG. 19 are similar to the images m50 and m51 shown in FIG. An image m54 of the jumping liquid is shown. In this case, the image m51' is determined to be a drip standby image.

Therefore, in this modified example, when searching for the i-th row of the monitoring area R, a row in which a predetermined number or more of grayscale pixels are continuously present is determined as a droplet existing row. The number of continuous pixels is larger than the average size of the image of the liquid adhering to the inner wall of the drip tube 4 based on the type of liquid to be dropped, the working distance, the scale of the image, etc. It may be appropriately set so as to be smaller than the average size of the image.
According to this modified example, it is possible to suppress erroneous detection of the drip standby image.

(Determination processing of drip image)
Next, the droplet image determination processing (see step S136 in FIG. 10 and step S167 in FIG. 12) will be described in detail. FIG. 20 is a flow chart showing a droplet image determination process according to the embodiment of the present invention. FIG. 21 is a schematic diagram for explaining the droplet image determination process according to the embodiment of the present invention. Images m55 to m58 shown in FIG. 21 show the images after the drip standby image is detected in chronological order.

First, the dropping start determination unit 142 performs binarization processing on the monitoring region set in the determination target image (step S301). In this process as well, the gradation of pixels in at least the contour region of the droplet image is 0 gradation (that is, black), and the gradation of pixels in the background region of the droplet image is 255 gradation (that is, white). The threshold in the binarization process is set so that

Subsequently, the dropping start determination unit 142 sets a monitoring region in the image to be determined, and sets a counter i (i=1 to i _max ) representing a row of pixels in the monitoring region to 1 (step S302).

Subsequently, the dropping start determination unit 142 horizontally searches for pixels in the i-th row in the monitoring area (step S303), and determines whether or not all the pixels in the i-th row are 255-gradation pixels. (step S304). In other words, it is determined whether or not there is a 0-tone pixel in the i-th row.

If even one pixel with gradation 0 exists in the i-th row (step S304: No), then the dropping start determination unit 142 determines that the row being searched (i-th row) is the last row (i _max line) is determined (step S305). If the final line has not been reached (step S305: No), the dropping start determination unit 142 increments the counter i (step S306), and then the process returns to step S303. For example, since there is a 0-level pixel in the i-th row of the image m55 and the row being searched has not yet reached the final row, the next row is further searched.

On the other hand, if the line being searched reaches the last line of the monitoring area (step S305: Yes), the process returns to the main routine. In this case, it is determined that the image to be determined is not the drip image (step S137 in FIG. 10: No, step S168 in FIG. 12: No), and the drip image determination process is continuously performed on the image of the next frame. . For example, the image m55 and the image m56 are not droplet images because pixels of 0 gradation exist in all rows of the monitoring region R of the image m55 and the image m56.

Further, when all the pixels in the i-th row are gradation 255 (step S304: Yes), in other words, when there is not even one pixel of gradation 0, the drip start determination unit 142 detects the drip image. (step S307). After that, the process returns to the main routine. For example, in the image m57, a row in which all the pixels are white (liquid-out row) is generated between the image of the droplet remaining in the nozzle and the image of the falling droplet. When one such liquid-out line is detected, the determination target image is determined to be a drip image.

According to this embodiment, it is possible to detect, with high accuracy, an image that captures the moment when the droplet that has been hanging down from the nozzle breaks and begins to fall.

Next, a first modification of the droplet image determination process will be described.
In the above-described embodiment, when even one line of liquid-outage lines is detected, it is determined that the image to be determined is a dripping image (see steps S304 and S307 in FIG. 19). However, it may be determined that the image to be determined is the dripping image when the liquid shortage line is detected over a plurality of continuous lines. The number of rows (threshold value) of liquid-outage rows used as a determination criterion may be appropriately set based on the type of liquid to be dropped, the work distance, the scale of the image, and the like.
According to this modified example, it is possible to determine that the image is a dropped image when the droplets hanging down on the nozzle are surely cut off.

Next, a second modification of the droplet image determination process will be described. FIG. 22 is a flowchart showing a second modification of the droplet image determination process according to the embodiment of the present invention. FIG. 23 is a schematic diagram for explaining a second modification of the droplet image determination process according to the embodiment of the present invention. Images m60 to m63 shown in FIG. 23 show the images after the drip standby image is detected in chronological order. Note that the processing in steps S301 to S306 shown in FIG. 22 is the same as the processing described with reference to FIG. 20 in the above embodiment.

Here, since the droplets hanging down from the nozzle may vibrate, as shown in images m60 to m62, even if the images are acquired in chronological order, the position of the lower edge of the droplet image fluctuates up and down. sometimes. Therefore, once it is determined to be in a dripping standby state (see image m60), and after the dripping start determination unit 142 starts the dripping image determination processing, when the lower end of the droplet image moves upward in the monitoring region R ( (see image m61), the liquid shortage line may be erroneously detected.

Therefore, in this modification, when it is determined that all the pixels in the row being searched (row i) have 255 gradations (step S304: Yes), the dropping start determination unit 142 continues to monitor the i row. It is determined whether or not it is the bottom row (i _max row) of the region R (step S311). If the i line is the bottom line of the monitoring area R (step S311: Yes), the process directly returns to the main routine. In this case, it is determined that the determination target image is not the droplet image. On the other hand, if the i line is not the line at the lower end of the monitoring region R (step S311: No), the dropping start determination unit 142 determines that the dropping image has been detected (step S312). After that, the process returns to the main routine.

According to this modified example, it is possible to prevent erroneous detection of a droplet image due to vibration of a droplet. It should be noted that, in this modified example, similarly to the first modified example, it may be determined whether or not the out-of-liquid line is detected over a plurality of continuous lines. In this case, when a plurality of continuous liquid-out lines include the line at the bottom end of the monitoring region R, it is determined that the determination target image is not the drip image.

Next, a third modification of the droplet image determination process will be described. FIG. 24 is a flowchart showing a third modification of the droplet image determination process according to the embodiment of the present invention. 25 to 27 are schematic diagrams for explaining a third modification of the droplet image determination process according to the embodiment of the present invention. Images m64 to m67 shown in FIG. 25, images m68 to m71 shown in FIG. 26, and images m72 to m75 shown in FIG.

First, _the dropping start determination unit 142 determines a plurality of rows including the lower end of the monitoring region R set in the image to be determined, that is, the ₀ gradation (black ) is _counted (step S321).

Subsequently, the dropping start determination unit 142 determines the difference ΔN=N _k-1 −N _k between the counted number of pixels N _k and the number of pixels N _k−1 of 0 gradation pixels in the same region of the image of the previous frame. is greater than a predetermined threshold value Sh (step S322).

Here, after the drop standby state is entered, the droplet gradually increases in size while changing its shape in various ways and vibrates until the droplet that has been hanging down on the nozzle is dropped. Therefore, as shown in FIGS. 25 to 27, the droplet image area (that is, the number of pixels of 0 gradation) in the monitoring area R increases or decreases. On the other hand, when the droplet separates from the nozzle and falls, the area of the image of the droplet in the vicinity of the lower end of the monitoring area R (the area from the (i _max -x) to the i _max x rows) is decrease sharply. The threshold Sh used in step S322 is appropriately set based on the type of liquid to be dropped, the work distance, the scale of the image, etc., so as to catch the case where the number of pixels near the lower end is greatly reduced to near zero.

If the difference ΔN=N _k−1 −N _k is less than or equal to the predetermined threshold value Sh (step S322: No), the dropping start determination unit 142 saves the number of pixels N _k (step S323). After that, the process returns to the main routine. In this case, it is determined that the image to be determined is not the drip image (step S137 in FIG. 10: No, step S168 in FIG. 12: No), and the drip image determination process is continuously performed on the image of the next frame. .

On the other hand, if the difference ΔN=N _k−1 −N _k is greater than the predetermined threshold value Sh (step S322: Yes), that is, the number of 0 gradation pixels existing in the vicinity of the lower end of the monitoring region R has decreased sharply. In this case, the dripping start determination unit 142 determines that the determination target image is the dripping image (step S324). After that, the process returns to the main routine.

According to this modification, the change in the area of the droplet image near the lower end of the monitoring area R is quantitatively detected by counting the number of pixels of 0 gradation. It is possible to determine whether or not with high accuracy. Therefore, as shown in FIG. 26, for example, even if the area of the image of the droplet in the monitoring area R changes greatly due to vibration of the droplet, erroneous determination can be suppressed. In addition, as shown in FIG. 27, even if liquid adhering to the inner wall of the drip tube is captured in the image, the existence of such a liquid image hardly affects the determination. It is possible to improve the determination accuracy of the droplet image.

(Extraction processing of droplet image)
Next, a detailed description will be given of the process of extracting the image of the droplet in the image (see step S152 in FIG. 11). FIG. 28 is a flow chart showing a droplet image region extraction process executed by the region extraction unit 143 of the volume calculation unit 135 shown in FIG. FIG. 29 is a diagram showing images before and after the dropping start image generated in chronological order. 30 to 32 are schematic diagrams for explaining the process of extracting a droplet image region in this embodiment.

As shown in FIG. 29, the volume of the droplet falling in the dropping start image _M2 is hanging down to the nozzle in the frame immediately before the dropping start image _M2 , that is, in the last image _M1 in the dropping standby state. It can be said that it is almost equal to the volume of the droplet. As a method for extracting the image of the droplet reflected in the image _M2 , a difference image is generated by subtracting the image in which the nozzle is reflected from the image _M1 , and then binarization processing, hole filling processing, and blob processing are performed. can be considered.

Here, the droplet positioned between the light source 11 and the camera 12 (see FIG. 1), like a convex lens, transmits the light incident on the center of the droplet almost straight, and the peripheral edge of the droplet greatly refracts and scatters the light incident on the The area of the droplet image is extracted by detecting the contour formed by the refraction and scattering of light at the periphery of the droplet. As shown in FIG. 30, in the binary image BN1 obtained by subjecting the difference image DF1 to the binarization process, a region corresponding to a bright portion near the center of the droplet image is missing. Therefore, the binary image BN1 is subjected to hole-filling processing (isolated point removal processing) using a known method such as morphology processing, and further, by performing blob processing, as one area block, the image of the droplet. Corresponding regions can be extracted.

Specifically, the image to be subtracted from the image M ₁ includes an image before the start of dripping (for example, when nozzle positioning is completed) and an image after a predetermined frame of the dripping start image M ₂ (for example, after one frame). image M ₃ or image M ₄ after two frames).

However, when using the image before the start of drip, there is a problem that unnecessary images other than the droplet reflected in the image _M1 , such as the image of the liquid adhering to the inner wall of the drip cylinder, cannot be canceled. Further, the image _M3 in the frame immediately after the drop start image _M2 may be a combination of the image of the droplet in the image _M1 and the image of the falling droplet in the image _M3 , depending on conditions such as the frame rate. In this case, the part of the image of the droplet reflected in the image M ₁ is canceled. Therefore, although not limited, it can be said that it is more preferable to use the image _M4 two frames after the dropping start image _M2 .

On the other hand, when the flow rate of the droplets to be dropped is high (in other words, when the droplet dropping period is short), the growth of the droplets hanging down to the nozzle is also fast. Therefore, the following problem may occur when using images with a slightly wide frame interval, such as images M ₁ and M ₄ . That is, as shown in FIG. 30, in the difference image DF1 obtained by subtracting the image _M4 from the image _M1 , the area near the nozzle end of the droplet image in the image _M1 is the liquid that has begun to grow in the image _M4 . It will be canceled in the area of the image of the droplet. Therefore, when the difference image DF1 is binarized, and the binary image BN1 is subjected to hole-filling processing and blob processing, in the blob image BL1 obtained by this, the image of the droplet reflected in the image _M1 is Among them, the area near the nozzle end (see area Δy) that should be originally extracted is cut off. As a result, when the droplet volume is calculated based on the region extracted from the blob image BL1, the error becomes large.

Therefore, in the present embodiment, an image (binary image BN1) before the area to be extracted is cut and an image (blob image BL1) after the area is cut are used as the basis for the cut image. By specifying the cut area and re-extracting the cut area from the image M ₁ , the entire area of the image of the droplet reflected in the image M ₁ is reproduced.

Specifically, as shown in FIG. 28, the region extracting unit 143 first generates a difference image DF1 between the last image _M1 in the dripping standby state and the image _M4 two frames after the dripping start image (step S401). Subsequently, the region extracting unit 143 performs binarization processing on the difference image DF1 (step S402). Subsequently, the region extracting unit 143 performs hole-filling processing and blob processing on the binary image BN1 (step S403).

Subsequently, as shown in FIG. 31, the region extracting unit 143 sets a rectangular region R2 for the last image _M1 in the dropping standby state based on the binary image BN1 and the blob image BL1 (step S404). . This rectangular region R2 has the upper end of the region E1 of the droplets remaining in the binary image BN1 (in other words, the lower end of the nozzle in the image _M1 ), and the upper end of the region E2 extracted in the blob image BL1. , and has the same width as the maximum width of the region E2 extracted from the blob image BL1.

Subsequently, the region extracting unit 143 performs binarization processing on the rectangular region R2 set in the last image _M1 in the dropping standby state (step S405). In the binary image BN2 thus obtained, an area E3 that has been canceled by taking the difference from the image _M4 is extracted from the image of the droplet in the image _M1 .

Subsequently, as shown in FIG. 32, the area extracting unit 143 merges the blob image BL1 obtained in step S403 and the binary image BN2 obtained in step S405 (step S406), and further performs filling processing. and apply blobs (step S407). In the composite image ME1 obtained in this way, the entire area E4 of the image of the droplet reflected in the image _M1 is extracted. After that, the process returns to the main routine.

According to this embodiment, it is possible to sufficiently extract the area of the image of the droplet captured in the final image _M1 in the drop waiting state, so that the measurement error of the droplet volume can be reduced. becomes.

In the above embodiment, the droplet volume is calculated using the image of the frame immediately before the dropping start image. It is also possible to use an image several frames (for example, two frames) before. The same applies to the image after the dropping start image, and depending on the conditions, instead of the image two frames after the dropping start image, it is possible to use the image of the immediately following frame or the image three frames after.

(Calculation processing of volume of extracted region)
Next, the process of calculating the volume of the region extracted from the last image in the dropping standby state (see step S153 in FIG. 11) will be described in detail. FIG. 33 is a flow chart showing a process of calculating the volume of the extracted region, which is executed by the circular area calculator 144 and the integrator 145 of the volume calculator 135 shown in FIG.

In the present embodiment, the volume calculation unit 135 regards the extracted region (see region E4 in the composite image ME1 in FIG. 32) as a projected image of a body of revolution whose rotation axis is the vertical axis of the image. Calculate the volume of the body of revolution. That is, as shown in FIG. 33, the circle area calculation unit 144 calculates the area of a circle whose diameter is the number of pixels for each line of the extracted region (step S411). In subsequent step S412, the integration unit 145 integrates the calculated areas of the circles. After that, the process returns to the main routine.

(First Modification of Processing for Calculating Volume of Extracted Region)
Next, a first modified example of the processing for calculating the volume of the extracted region will be described. FIG. 34 is a flowchart showing a first modified example of processing for calculating the volume of an extracted region. FIG. 35 is a diagram showing images before and after the dropping start image generated in chronological order. 36 and 37 are schematic diagrams for explaining a first modification of the process of calculating the volume of the extracted region.

In this modification, it is assumed that the region E7 shown in FIG. 37(a) is extracted by the region extraction unit 143. FIG. An area E7 is an image M13 after a predetermined frame (here, two frames) of the drip start image _M11 from the image _M10 (the last image in the drip standby state) of the frame immediately before the drip start image _{M11 shown} in FIG. and a region E6 directly extracted by setting a rectangular region (see step S404 in FIG. 28) for the last image in the dropping standby state. It is what I did.

Here, when a certain amount of time elapses after starting drip infusion, the liquid accumulated in the drip tube may splash and adhere to the inner wall of the drip tube. In such a case, as shown in images M ₁₀ to M ₁₃ in FIG. 35, the image of the liquid adhering to the inner wall of the drip tube is reflected in the image in addition to the image of the nozzle and droplet. In such a state, subtracting an image after a predetermined frame (for example, image M ₁₃ ) of the drip start image M ₁₁ from _the image M ₁₀ (the last image in the drip standby state) of the frame immediately before the drip start image M 11 yields As shown in the difference image DF3 in FIG. 36, the portion of the droplet image in the image _M10 of the previous frame that overlaps with the background liquid image is canceled. If such a difference image DF3 is subjected to binarization processing, many defects will occur in the binary image BN3. It is difficult to make up for the missing part.

Therefore, as shown in (a) of FIG. 37, the circle area calculation unit 144 calculates a sets a rectangular area R4 circumscribing this area E5 (step S421).

Subsequently, as shown in FIG. 37(b), the circle area calculation unit 144 horizontally divides each row of the region E5 extracted based on the difference image DF3 by the center line L1 extending in the vertical direction of the rectangular region R4. Divide (step S422).

Subsequently, the circle area calculation unit 144 calculates the area of a circle whose diameter is twice the number of pixels on the left and right sides of the center line L1, and which has the larger number of pixels, for each row of the region E5 (step S423). ). For example, in the n-th row shown in FIG. 37, the number of pixels P _R on the right side of the center line L1 is greater than the number of pixels P _L on the left side. Therefore, for this row, the area of a circle with a diameter of 2×P _R pixels is calculated.

On the other hand, the circle area calculator 144 calculates the area of a circle having a diameter equal to the number of pixels for each line of the region E6 directly extracted from the last image _M10 in the drip standby state (step S424).

The integration unit 145 integrates the areas of the circles calculated in steps S423 and S424 (step S425). As a result, as shown in FIG. 37(c), it is possible to obtain the volume when the region E7' corresponding to the image of the droplet hanging down the nozzle in the image _M10 is regarded as a rotating body.

According to this modification, even if a part of the area is missing due to the image processing when extracting the droplet image area, the missing part is interpolated and the rotating body It is possible to calculate the volume of

(Second Modification of Processing for Calculating Volume of Extracted Region)
Next, a second modification of the process of calculating the volume of the extracted region will be described. FIG. 38 is a flow chart showing a second modification of the process of calculating the volume of the extracted region. FIG. 39 is a schematic diagram for explaining a second modification of the process of calculating the volume of the extracted region.

In this modified example, it is assumed that the region E7 shown in FIG. 39 is extracted by the region extraction unit 143. As in the first modification, the area E7 is the area E5 extracted based on the difference image between the last image of the dripping standby state and the image after a predetermined frame of the dripping start image, and the last image of the dripping standby state. It is synthesized with the region E6 directly extracted from the image. Although the nozzle image E8 is shown in FIG. 39 to facilitate understanding, the nozzle image E8 is not displayed in the image from which the regions E5 and E6 are extracted.

First, the circular area calculator 144 sets a dividing line based on a vertical line passing through the center of the lower end of the nozzle image captured in the last image in the drip standby state (step S431). Here, the lower end of the nozzle image can be considered to coincide with the upper end of the region E6 directly extracted from the last image in the drip standby state. Therefore, a vertical line passing through the center P1 of the upper end line of the region E6 is defined as a dividing line L2.

Subsequently, the circle area calculation unit 144 divides each row of the extracted region into left and right sides with the division line L2 as the center (step S432). Processing in subsequent steps S433 to S435 is the same as steps S423 to S425 in FIG.

(Third Modification of Processing for Calculating Volume of Extracted Region)
Next, a third modified example of the processing for calculating the volume of the extracted region will be described. FIG. 40 is a schematic diagram for explaining a third modification of the process of calculating the volume of the extracted region.

In this modified example, it is assumed that the region E11 shown in FIG. 40 is extracted by the region extraction unit 143. FIG. An area E11 is an area E9 extracted based on the difference image between the image of the frame immediately before the dripping start image (the last image of the dripping standby state) and the image after the predetermined frame of the dripping start image, and the last image of the dripping standby state. This image is combined with an area E10 directly extracted by setting a rectangular area (see step S404 in FIG. 28) for the image of . In FIG. 40(a), the nozzle image E12 is shown to aid understanding, but the nozzle image E12 is not displayed in the image from which the regions E9 and E10 are extracted.

Here, when the nozzle 6 (see FIG. 1) that drops droplets is tilted with respect to the vertical direction, the vertical line passing through the center of the lower end of the nozzle image does not necessarily pass through the central axis of the region E11. do not have. Therefore, in this modified example, as shown in FIG. Based on the detected value, a line passing through the center P2 and inclined from the vertical line L3 by an angle corresponding to the inclination of the nozzle 6 is set as the dividing line L4. Then, the area E11 is regarded as a projected image of the rotating body, and the area of the circle sliced in the direction perpendicular to the dividing line L4 is calculated and integrated.

At this time, as shown in (b) of FIG. 40, for the area E9, among the pixels arranged on both sides of the dividing line L4 in the direction orthogonal to the dividing line L4, two pixels having a larger number of pixels are displayed. Calculate the area of the circle with the diameter doubled. On the other hand, for the region E10, the area of a circle is calculated using the number of pixels arranged in the direction perpendicular to the dividing line L4 as the diameter. Then, the area calculated for the region E9 and the volume calculated for the region E10 are integrated. As a result, as shown in (c) of FIG. 40, the volume is obtained when the region E9′ corresponding to the image of the droplet hanging on the nozzle in the last image in the drop waiting state is regarded as the body of rotation. be able to.

According to this modification, it is possible to accurately correct the volume calculation error caused by the inclination of the nozzle 6 .

(Fourth Modification of Processing for Calculating Volume of Extracted Region)
Next, a fourth modification of the process of calculating the volume of the extracted region will be described. 41 and 42 are schematic diagrams for explaining a fourth modification of the process of calculating the volume of the extracted region. In the third modification described above, the dividing line L4 (see FIG. 40) of the area 11 is set based on the detection value output from the tilt sensor 14. FIG. However, if the tilt sensor 14 is not provided, it is also possible to set the dividing line based on the shape of the extracted area.

For example, as shown in FIG. 41, out of the distances from the center P2 of the bottom end of the nozzle image, that is, the center of the top line of the region E11 to each point on the contour of the region E11, the point P3 that is the farthest from A passing line L5 may be used as a dividing line. Alternatively, as shown in FIG. 42, a line L6 passing through the center P2 of the lower end of the nozzle image and the center P4 of the row having the largest number of pixels in the horizontal direction (that is, the widest width) in the region E11 is divided. It can be used as a line.

According to this modification, even if a sensor for detecting the inclination of the nozzle 6 is not provided, it is possible to accurately correct the volume calculation error caused by the inclination of the nozzle 6 .

(Calculation process of droplet volume)
Next, the process of calculating the droplet volume based on the volume calculated for the region extracted from the image (see step S154 in FIG. 11) will be described.

As described above, the volume calculation unit 135 calculates the volume of the region extracted from the image on the scale of the number of pixels (step S153), and converts the scale of the calculated value to obtain the actual droplet volume. (Step S154). At this time, the volume calculation unit 135 may output the scale-converted value (volume calculation value) as it is as the volume of the droplet, or accumulate the scale-converted value (volume calculation value) for a plurality of times and average them. The value may be output as the volume of the droplet.

FIG. 43 is a schematic diagram for explaining an example of droplet volume calculation processing. As shown in FIG. 43, the volume calculation unit 135 calculates volume calculation values v1, v2, . Furthermore, an average value of volume calculation values for a predetermined number of past times (five times in total in FIG. 43) is calculated, and this average value is output as the droplet volume. Specifically, when the volume calculation value v5 is calculated, the volume calculation unit 135 calculates the average value of the volume calculation values v1 to v5, and uses this average value as the droplet volume v _out 6 at the next volume calculation timing t6. Then, when the volume calculation value 6 is calculated, the average value of the volume calculation values v2 to v6 is calculated, and this average value is output as the droplet volume v _out 7 at the next volume calculation timing t7. are executed sequentially. Note that when the number of volume calculation values necessary for calculating the average value is not obtained immediately after the start of drip infusion, the calculated volume calculation values may be output as they are. For example, in the case of FIG. 43, at the volume calculation timings t1 to t5, the calculated volume values v1 to v5 are directly output as the droplet volumes v _out 1 to v _out 5 .

FIG. 44 is a schematic diagram for explaining another example of droplet volume calculation processing. The volume calculation unit 135 may calculate an average value of the volume calculation values v1, v2, . . . at regular intervals. For example, in the case of FIG. 44, when the volume calculation value v5 is calculated, the volume calculation unit 135 calculates the average value of the volume calculation values v1 to v5, and calculates the volume calculation values v6 to v10 at subsequent volume calculation timings t6 to t10. While outputting this average value as droplet volumes v _out 6 to v _out 10 and calculating the volume calculation value v10, the average value of the volume calculation values v6 to v10 is calculated, and the following volume calculation timings t11 to t15 , the volume calculation values v11 to v15 are calculated, and the average values are output as the droplet volumes v _out 11 to v _out 15. As in the case of FIG. 43, when the number of volume calculation values required for calculating the average value is not obtained immediately after the start of drip infusion, the calculated volume calculation value may be output as it is.

(Operation of error corrector)
Next, the operation of the error corrector 146 shown in FIG. 2 will be described.
When time elapses after the start of dripping, the liquid splashes in the drip tube 4 (see FIG. 1) and adheres to the nozzle 6, causing a measurement error in the droplet volume. This may result in an irregular state, resulting in unexpected measurement values being calculated. In preparation for the case where such an abnormal value is calculated, the error correction unit 146 uses the volume of the droplet measured in a predetermined time (for example, several minutes to ten and several minutes) after the start of drip as a reference value. Set the upper and lower limits of the volume range. Then, when an abnormal value outside the set range is calculated, the error correction unit 146 inserts a provisional value instead of this abnormal value. As the provisional value, for example, the reference value used when determining the setting range, the average value of the volume of droplets measured at this time, or the average volume of droplets measured within a predetermined time in the past values, etc. can be used.

According to this modification, it is possible to suppress the influence of droplet volume measurement errors and abnormal values calculated when a temporary irregular state occurs on a series of droplet volume measurements. .

(Operation of high flow rate adjustment unit)
Next, the operation of the high flow rate adjusting section 147 shown in FIG. 2 will be described.
As described above, the average value of volume calculation values 1, 2, . The high flow rate adjustment unit 147 adjusts the number of volume calculation values used when calculating this average value according to the flow rate set for the flow rate control unit 122 . Specifically, the high flow rate adjustment unit 147 increases the number of volume calculation values used to calculate the average value when the set value of the flow rate is equal to or greater than a predetermined value.

Here, when the flow rate of droplets increases, depending on the frame rate of the camera 12 and the imaging timing, measurement errors in droplet volume are more likely to occur, and variations in volume per droplet increase. Therefore, by increasing the number of times that the average is taken, it is possible to suppress fluctuations in droplet volume caused by variations in individual calculated volume values.

Further, the high flow rate adjusting section 147 may adjust the setting range used for the determination by the error correcting section 146 when the set flow rate is high. Specifically, when the set value of the flow rate is equal to or greater than a predetermined value, the range is narrowed by raising the lower limit of the set range. Here, when the flow rate of droplets is high, there are many measurement errors in which the droplet volume is calculated to be too small. Therefore, by raising the lower limit value of the setting and inserting a provisional value when an abnormal value below the lower limit value of the setting range is calculated, the influence of the abnormal value on a series of droplet volume measurements can be reduced. can be done.

(Example)
An experiment was conducted in which droplets were continuously dropped from the nozzle for one hour, and the error between the volume of the droplet calculated based on the image and the volume of the actually dropped droplet was calculated. The experimental conditions are as follows.

<Example 1>
Nozzle type: 20 drops/mL
Extraction method of the droplet image area: The first area extracted based on the difference image between the last image of the dripping standby state and the image after two frames of the dripping start image, and the last image of the dripping standby state A second area extracted by setting a rectangular area was synthesized (see FIG. 28).
Volume calculation method: The areas of circles having diameters corresponding to the number of pixels in each row of the first and second regions were calculated, and the areas of these circles were integrated (see FIG. 33).

<Example 2>
Nozzle type: 20 drops/mL
Extraction method of droplet image area: Same as in Example 1.
Volume calculation method: The first region was divided into left and right sides by the vertical center line of the circumscribed rectangular region, and the area of a circle whose diameter was twice the size of the side with the larger number of pixels was calculated and integrated. . For the second region, the area of a circle whose diameter is the number of pixels in each row of the extracted region was calculated and integrated (see FIG. 34).

<Example 3>
Nozzle type: 60 drops/mL
Extraction method of droplet image area: Same as in Example 1.
Volume calculation method: Same as in Example 1.

<Example 4>
Nozzle type: 60 drops/mL
Extraction method of droplet image area: Same as in Example 1.
Volume calculation method: Same as in Example 2.

<Comparative Example 1>
Nozzle type: 20 drops/mL
Extraction method of droplet image area: binarization processing and blob processing were performed on the difference image obtained by subtracting the image two frames after the drip start image from the final image in the drip standby state.
Volume calculation method: Same as in Example 1.

<Comparative Example 2>
Nozzle type: 60 drops/mL
Extraction method of droplet image area: Same as Comparative Example 1.
Volume calculation method: Same as in Example 1.

Moreover, in Examples 1 and 2 and Comparative Example 1, as shown in FIG. Determined the starting state. The actual size of the monitoring area R10 is as follows.
Width of nozzle tip in image: 125px
Distance from nozzle tip to monitoring area: 119px (approximately 0.95 times the width of the nozzle)
Width of monitoring area: 181px (about 1.45 times the nozzle width)
Vertical length of monitoring area: 81px (approximately 0.65 times the width of the nozzle)

Moreover, in Examples 3 and 4 and Comparative Example 2, as shown in FIG. Determined the starting state. The actual size of the monitoring area R11 is as follows.
Width of nozzle tip in image: 33px
Distance from nozzle tip to monitored area: 29px (approximately 0.9 times the width of the nozzle)
Width of monitoring area: 81px (approximately 2.45 times the width of the nozzle)
Vertical length of monitoring area: 81px (approximately 2.45 times the width of the nozzle)

47 is a graph showing experimental results of Examples 1 and 2 and Comparative Example 1. FIG. 48 is a graph showing experimental results of Examples 3 and 4 and Comparative Example 2. FIG. 47 and 48, the horizontal axis indicates the set value of the flow rate, and the vertical axis indicates the ratio of the error in the measurement value of the droplet volume based on the image (the ratio of the error value to the true value (=measured value−true value) ratio). Here, the true value of the droplet volume was obtained by measuring the weight of the droplet dropped from the nozzle with an electronic balance and converting this weight into volume based on the specific gravity of the droplet.

As shown in FIG. 47, in the nozzle for 20 droplets/mL, when the flow rate is low (about 30 mL/h, about 80 mL/h), the error is small in both Examples 1 and 2 and Comparative Example 1. There is not much difference between Examples 1 and 2 and Comparative Example 1. However, when the flow rate is high (approximately 630 mL/h), while the error in Example 2 is kept small, the error in Comparative Example 1 is greatly increased. Further, the error in Example 1 is larger than that in the case of low flow rate, but is significantly smaller than that in Comparative Example 1.

As shown in FIG. 48, in the nozzle for 60 droplets/mL, when the flow rate is low (about 10 mL/h, about 27 mL/h), the error is small in both Examples 3 and 4 and Comparative Example 2. There is not much difference between Examples 3 and 4 and Comparative Example 2. However, when the flow rate is high (approximately 140 mL/h), the error in Comparative Example 2 is greater than in Examples 3 and 4.

From the above, it was found that, in Examples 1 to 4, the droplet volume can be measured with high accuracy and little error. In addition, in Examples 1 to 4, even if the flow rate of the drip is increased, the error in the droplet volume is kept small compared to Comparative Examples 1 and 2, and the higher the flow rate of the drip, the more remarkable effect is obtained. I understand.

The present invention described above is not limited to the above embodiments and modifications, and various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments and modifications. can. For example, it may be formed by excluding some components from all the components shown in the above embodiments and modifications, or may be formed by appropriately combining the components shown in the above embodiments and modifications. good.

REFERENCE SIGNS LIST 1 drip device 2a drainage port 2 infusion bag 3 intermediate tube 4 drip cylinder 4a upper lid 5 infusion tube 6 nozzle 6a nozzle tip 7 droplet 8 clamp 9 actuator 10 droplet measurement system 11 light source 12 camera 12a imaging element 13 display device 14 sensor 100 information processing device 101 input/output unit 102 operation input unit 103 storage unit 104 calculation unit 111 program storage unit 112 set value storage unit 113 image data storage unit 114 measured value storage unit 115 dropping detection image buffer 116 volume calculation image Buffer 121 Imaging control unit 122 Flow control unit 123 Image processing unit 131 Image generation unit 132 Error determination unit 133 Flow rate estimation unit 134 Drop detection unit 135 Volume calculation unit 141 Drop standby determination unit 142 Drop start determination unit 143 Region extraction unit 144 Circle area Calculation unit 145 Integration unit 146 Error correction unit 147 High flow rate adjustment unit

Claims (21)

In a droplet measurement system that measures the volume of droplets dropped from a nozzle,
an imaging device for capturing an image of a subject and outputting image data, the imaging device being installed so as to include a tip portion of the nozzle and the entire droplet hanging down from the nozzle within a field of view;
an image generating unit that generates, in chronological order, images showing droplets dripping from the nozzle based on the image data output from the imaging device;
a dropping detection unit for detecting a dropping start image, which is an image showing a state in which a droplet has separated from the nozzle and started dropping, from the images generated in chronological order by the image generating unit;
An image before the dropping start image by a predetermined frame, in which the droplet hangs down on the nozzle, and an image after the dropping start image by a predetermined frame , after the droplet is separated. a volume calculation unit that calculates the volume of the droplet reflected in the drip start image based on the image in which the nozzle is reflected;
with
The dropping detection unit is
A rectangular area having a predetermined size is set at a position below the position where the tip of the nozzle is captured by a predetermined distance from the images generated in chronological order by the image generating unit, and the rectangular area is a dropping standby determination unit that detects a dropping standby image, which is an image showing a state in which a droplet hangs down from the tip of the nozzle, by monitoring;
After the drip standby image is detected, the drip start determination unit detects the drip start image by setting the rectangular area for the image generated by the image generation unit and monitoring the rectangular area. and,
has
The drip standby determination unit searches for an area of the droplet image that is continuous in the vertical direction within the rectangular area, and determines that the image to be determined is the drip standby image when the area is detected,
The droplet measurement system , wherein the droplet start determination unit searches for droplet image regions divided in the vertical direction within the rectangular region . further comprising a storage unit including a first buffer storage area and a second buffer storage area;
The image generation unit performs predetermined processing on the image data output from the imaging device, stores the processed image data in the first buffer storage area,
When the dripping start image is detected, the dripping detection unit detects an image before the predetermined frame of the dripping start image and an image after the predetermined frame of the image data stored in the first buffer storage area. copying image data for an image to the second buffer storage area;
2. The droplet measurement system according to claim 1, wherein said volume calculator calculates the volume of said droplet based on the image data stored in said second buffer storage area. The drop start determination unit determines that the image to be determined is the drop start image when detecting an area of the droplet image divided in the vertical direction in the rectangular area.
Droplet measurement system according to claim 1 or 2. When the image of the liquid divided in the vertical direction in the rectangular area is detected over a plurality of consecutive frames, the dripping start determination unit determines that the leading image of the plurality of frames is the dripping start image. ,
Droplet measurement system according to claim 1 or 2. After the drip standby determination unit detects the drip standby image, the drip start determination unit starts monitoring the images generated by the image generation unit in chronological order, and the drip standby determination unit stops monitoring. death,
After the drip start determination unit detects the drip start image, the drip standby determination unit resumes monitoring the images generated in chronological order by the image generation unit.
Droplet measurement system according to any one of claims 1-4. an error determination unit that detects the position of the lower end of the image of the nozzle in the image generated by the image generation unit and outputs an error signal when the position of the lower end is not within a predetermined range;
a flow control unit that controls the start of dropping droplets from the nozzle;
an operation input unit for inputting a signal corresponding to an external operation to the flow control unit;
further comprising
When an error signal is output from the error determination unit, the flow rate control unit controls the flow rate of the liquid from the nozzle even when a signal instructing the start of droplet dropping is input from the operation input unit. Droplet measurement system according to any one of claims 1 to 5, which does not initiate the dropping of drops. recognizing the image of the nozzle in the image generated by the image generating unit, and determining a standard flow rate range of droplets dropped from the nozzle based on at least one of the shape and thickness of the image of the nozzle; a flow rate estimator for estimating;
a flow rate control unit that controls the flow rate of droplets dropped from the nozzle;
an operation input unit for inputting a signal representing a set value of the flow rate of droplets to be dropped from the nozzle to the flow control unit according to an operation performed from the outside;
further comprising
6. The method according to any one of claims 1 to 5, wherein the flow rate control unit does not start dropping droplets from the nozzle when the set value is not within the range of the standard flow rate estimated by the flow rate estimation unit. A droplet measurement system according to any one of the preceding claims. In the vertical direction of the image generated by the image generator,
the upper end of the rectangular area is set below the tip of the image of the nozzle;
8. The lower end of the rectangular region according to any one of claims 1 to 7, wherein the lower end of the rectangular area is set to be higher than the lower end of the image of the droplet captured in the image of the predetermined frame before the drip start image. Droplet measurement system. When the conditions for dropping droplets from the nozzle are changed, the lower end of the image of the droplet adhering to the nozzle in the dropping start image and the upper end of the image of the falling droplet in the dropping start image. When the gap changes and there is a common range between the gaps that changes when the conditions are changed,
9. The droplet measurement system of claim 8, wherein the vertical size of the rectangular areas is set to encompass the common extent between the gaps. When the conditions for dropping droplets from the nozzle are changed, the lower end of the image of the droplet adhering to the nozzle in the dropping start image and the upper end of the image of the falling droplet in the dropping start image. When the gap changes and there is no common range between the gaps that change when the conditions are changed,
The size of the rectangular area in the vertical direction is between the uppermost position of the upper end when the condition is changed and the lowest position of the lower end when the condition is changed. 9. The droplet measurement system of claim 8, configured to encompass the range of . When the width of the image of the droplet in the droplet start image changes when the conditions for dropping the droplet from the nozzle are changed,
The liquid according to any one of claims 8 to 10, wherein the horizontal size of the rectangular area is set to include the largest width among the widths that change when the conditions are changed. Drop measurement system. The drip standby determination unit
By performing the binarization process on the rectangular area, the gradation of pixels in at least the outline area of the droplet image is converted to the first gradation, and the gradation of the pixels in the background area of the droplet image is converted to the first gradation. Convert to a second tone,
extracting rows in which pixels having the first gradation exist as droplet-existing rows by searching the rectangular region;
The droplet according to any one of claims 1 to 11, wherein when all rows of the rectangular area are droplet presence rows as a result of the search, the image to be determined is determined to be a drip standby image. measurement system. The drip standby determination unit
By subjecting the rectangular region to binarization processing, the gradation of pixels in at least the outline region of the droplet image is converted to the first gradation, and the gradation of pixels in the background region of the droplet image is converted to the first gradation. converting the tone to a second tone,
By searching the rectangular area, a row in which a plurality of pixels having the first gradation are continuously present is extracted as a droplet existing row;
The droplet according to any one of claims 1 to 11, wherein when all rows of the rectangular area are droplet presence rows as a result of the search, the image to be determined is determined to be a drip standby image. measurement system. The drip start determination unit searches the rectangular area subjected to the binarization process, and when a row in which all pixels have the second gradation is detected, the image to be determined is dripped. 14. Droplet measurement system according to claim 12 or 13, for determining a starting image. The dropping start determination unit searches the rectangular area subjected to the binarization process, and when lines in which all pixels have the second gradation are continuously detected over a plurality of lines, 14. The liquid droplet measurement system according to claim 12, wherein the determination target image is determined to be a dropping start image. Even when a row in which all pixels have the second gradation is detected, the dropping start determination unit determines that one or more rows continuous with the lower end of the rectangular area are determined to have all pixels 16. The droplet measurement system according to claim 14, wherein if the row has the second gradation, it is determined that the image to be determined is not the dropping start image. The dropping start determination unit counts the pixels having the first gradation in a plurality of lines that are continuous with the bottom end of the rectangular area in the rectangular area subjected to the binarization process, and the total number of the pixels is 14. The liquid according to claim 12 or 13, wherein the determination target image is determined to be the dropping start image when the total number in the frame immediately preceding the determination target image has decreased by more than a predetermined threshold. Drop measurement system. The droplet measurement system according to any one of claims 1 to 17, wherein the volume calculation unit outputs an average value of droplet volumes calculated over a plurality of past times as a droplet volume measurement value. . 19. The droplet measurement system according to claim 18, wherein said volume calculator sets the number of times of averaging the volume of said droplets according to a set value of the flow rate of droplets dropped from said nozzle. In a droplet measurement method performed in a droplet measurement system for measuring the volume of a droplet dropped from a nozzle,
An imaging device that captures an image of a subject and outputs image data, wherein the image data output from the imaging device is installed so that the tip of the nozzle and the entire droplet hanging down from the nozzle are included in the field of view. an image generating step of generating in chronological order an image showing a state in which droplets are dropped from the nozzle, based on the
a dropping detection step of detecting a dropping start image, which is an image showing a state in which a droplet has separated from the nozzle and started dropping, from the images generated in chronological order in the image generating step;
An image before the dropping start image by a predetermined frame, in which the droplet hangs down on the nozzle, and an image after the dropping start image by a predetermined frame , after the droplet is separated. a volume calculation step of calculating the volume of the droplet reflected in the drip start image based on the image in which the nozzle is reflected;
including
The dropping detection step includes
A rectangular region having a predetermined size is set at a position lower by a predetermined distance from the position where the tip of the nozzle is captured in the images generated in chronological order in the image generating step, and the rectangular region is a dropping standby determination step of detecting a dropping standby image, which is an image showing a state in which a droplet hangs down from the tip of the nozzle, by monitoring;
A dropping start determination step of detecting the dropping start image by setting the rectangular area for the image generated in the image generating step after the dropping standby image is detected and monitoring the rectangular area. and,
including
The dropping standby determination step searches for an area of images of droplets that are continuous in the vertical direction within the rectangular area, and determines that the image to be determined is the dropping standby image when the area is detected,
The droplet measurement method, wherein the droplet start determination step searches for a droplet image region divided in the vertical direction within the rectangular region . In a program to be executed by a computer in a droplet measurement system that measures the volume of droplets dropped from a nozzle,
An imaging device that captures an image of a subject and outputs image data, wherein the image data output from the imaging device is installed so that the tip of the nozzle and the entire droplet hanging down from the nozzle are included in the field of view. an image generating step of generating in chronological order an image showing a state in which droplets are dropped from the nozzle, based on the
a dropping detection step of detecting a dropping start image, which is an image showing a state in which a droplet has separated from the nozzle and started dropping, from the images generated in chronological order in the image generating step;
An image before the dropping start image by a predetermined frame, in which the droplet hangs down on the nozzle, and an image after the dropping start image by a predetermined frame , after the droplet is separated. a volume calculation step of calculating the volume of the droplet reflected in the drip start image based on the image in which the nozzle is reflected;
and
The dropping detection step includes
A rectangular region having a predetermined size is set at a position lower by a predetermined distance from the position where the tip of the nozzle is captured in the images generated in chronological order in the image generating step, and the rectangular region is a dropping standby determination step of detecting a dropping standby image, which is an image showing a state in which a droplet hangs down from the tip of the nozzle, by monitoring;
A dropping start determination step of detecting the dropping start image by setting the rectangular area for the image generated in the image generating step after the dropping standby image is detected and monitoring the rectangular area. and,
including
The dropping standby determination step searches for an area of images of droplets that are continuous in the vertical direction within the rectangular area, and determines that the image to be determined is the dropping standby image when the area is detected,
The program according to claim 1, wherein the dropping start determination step searches for a droplet image region divided in the vertical direction within the rectangular region .

Images