JP2018068863A - Gauze detection system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a gauze detection system capable of preventing a gauze from being left behind in surgery without subjecting the gauze to special processing.SOLUTION: A gauze detection system 100 includes an image input part 1 for inputting an image of a photographed surgical field, a determination part 2 for determining, with a region of a predetermined size in the input image as a determination object region, whether or not the region includes a feature of a gauze image by image processing, and a determination result output part 3 for notifying that a gauze is detected in the input image when there is a determination object region determined to include the feature of the gauze image by the determination part 2. The determination part 2 detects the presence or absence of the gauze region by, for example, binarizing the input image to classify it into a white image and a black image, and detecting a mesh portion of the gauze represented as a block of the black image of a predetermined area or less.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a gauze detection system for preventing misplacement of a gauze in a patient body by detecting features of the gauze image from an image obtained by imaging a surgical situation.

  A situation in which a surgical instrument or gauze is left in the patient's body during a surgical operation is a medical mistake that should not occur. In order to prevent such misplacement of gauze, a gauze counting system for confirming whether or not the number of gauze before use matches the number of gauze collected as used has been proposed ( Patent Document 1) below.

  In the conventional gauze counting system disclosed in Patent Document 1, an IC tag storing gauze identification information is attached to each gauze for operation, and the number and use of gauze to be used from now on by the tag reader. The number of gauze collected later is counted. Then, by checking whether the number of gauze before use matches the number of collected gauze, it is confirmed whether the gauze is left behind.

JP 2014-217547 A

  However, in the conventional gauze counting system described above, it is necessary to attach an IC tag to each gauze, and there is a problem that the price of the gauze becomes high and it takes time and effort to manage the gauze.

  Then, an object of this invention is to provide the gauze detection system which can prevent misplacement of gauze effectively, without carrying out the special process to gauze.

In order to solve the above problems, the gauze detection system according to the present invention is:
An image input unit for inputting a photographed image;
A determination unit configured to determine, by image processing, whether or not the region includes a feature of the gauze image for a region of a predetermined size in the input image;
A determination result output unit for notifying that the gauze is detected in the input image when there is an area determined to include the characteristic of the gauze image by the determination unit;

  According to said structure, the gauze detection system which can prevent misplacement of gauze effectively without performing special process to gauze can be provided.

FIG. 1 is a block diagram showing a functional schematic configuration of the gauze detection system according to the first embodiment. FIG. 2 is a flowchart showing a flow of image processing in the determination unit shown in FIG. FIG. 3 is a schematic diagram illustrating an example of a gauze image when binarized. FIG. 4 is a schematic diagram illustrating an example of output to a monitor. FIG. 5 is a block diagram showing a functional schematic configuration of the gauze detection system 101 according to the first modification. FIG. 6 is a block diagram showing a functional schematic configuration of the gauze detection system 102 according to the second modification. FIG. 7 is a block diagram showing a functional schematic configuration of the gauze detection system 103 according to the third modification. FIG. 8 is a block diagram showing a functional schematic configuration of the gauze detection system 104 according to the fourth modification. FIG. 9 is a schematic diagram illustrating an example of an image that is not a gauze image when binarized. FIG. 10 is a block diagram showing a functional schematic configuration of the gauze detection system 105 according to the fifth modification. FIG. 11 is an example of a histogram of the gauze image generated in the fifth modification. FIG. 12 is a block diagram showing a functional schematic configuration of the gauze detection system 200 according to the second embodiment. FIG. 13 is a block diagram showing a functional schematic configuration of a gauze detection system 300 according to the third embodiment. FIG. 14 is a flowchart showing the flow of the gauze entry / exit detection process in the gauze detection system 300.

The first configuration of the gauze detection system of the present invention is:
An image input unit for inputting a photographed image;
A determination unit that determines, by image processing, whether an area having a predetermined size in the input image is a determination target area and whether the area includes a feature of the gauze image;
A determination result output unit for notifying that the gauze is detected in the input image when there is a determination target region determined to include the characteristic of the gauze image by the determination unit;

  According to said 1st structure, since it can be judged by image processing whether the characteristic of a gauze image is included in the image | photographed operative field image, special things, such as attaching an IC tag etc. to gauze It is possible to effectively prevent the gauze from being misplaced without any special processing.

The second configuration is the first configuration,
The determination result output unit includes a display control unit that highlights a determination target region determined to include a gauze image feature in a captured image.

  According to the second configuration, for example, on the monitor in the operating room, the gauze region can be highlighted in the surgical field image. Thereby, it becomes possible to alert the surgeon and the assistant in an easy-to-understand manner that gauze is present in the surgical field.

The third configuration is the first or second configuration,
The determination unit
A binarization processing unit that performs binarization processing on the determination target region and classifies each pixel into a white pixel and a black pixel;
A block detection unit for detecting a black pixel block having a predetermined area or less in the determination target region,
When the number of black pixel blocks detected by the block detection unit in the determination target area is equal to or greater than a threshold value, it is determined that the determination target area includes a gauze image feature.

  According to the third configuration, by detecting a gauze mesh (opening portion) that appears as a black pixel block when binarization processing is performed, it is possible to accurately determine whether or not there is a gauze region in the operative field image. Can judge well.

A fourth configuration is any one of the first to third configurations,
A luminance difference detection unit for obtaining a luminance difference between adjacent pixels continuous in at least one direction in the determination target region;
When a periodic change is observed in the luminance difference obtained by the luminance difference detection unit, it is determined that the determination target region includes a feature of the gauze image.

  In the gauze image, the fibers and meshes (openings) are regularly arranged. Therefore, according to the fourth configuration, in the image of the operative field, the luminance difference between adjacent pixels continuous in at least one direction is obtained, When a periodic change is found in the obtained luminance difference, it can be determined that the region is a gauze region. This makes it possible to accurately determine whether or not there is a gauze region in the operative field image.

A fifth configuration is any one of the first to fourth configurations,
The determination unit
A white region detection unit for detecting a closed region of white pixels that is not continuous with a boundary line of the determination target region;
When the white area detection unit detects the closed area, it is determined that the determination target area does not include a feature of the gauze image.

  In the gauze image, the fiber portion that appears as a set of white pixels when binarized is an area continuous with at least one of the boundary lines of the determination target area, and in a closed area surrounded by black pixels. It is not considered. Therefore, in the fifth configuration, when a closed region of white pixels that is not continuous with the boundary line of the determination target region is detected, it is determined that the closed region is not a feature of the gauze image. This makes it possible to accurately determine whether or not there is a gauze region in the operative field image.

A sixth configuration is any one of the first to fifth configurations,
A histogram generator for generating a histogram normalized with respect to the luminance of the pixels in the determination target region;
A uniformity determining unit configured to determine that the determination target region includes a feature of the gauze image when the histogram is distributed with a predetermined uniformity.

  In the gauze image, since the fiber portion and the mesh portion are regularly arranged, when a histogram normalized with respect to the luminance of the pixel is generated, it is assumed that the distribution shows a predetermined uniformity. Therefore, according to the sixth configuration, it can be accurately determined whether or not there is a gauze region in the image of the operative field.

The seventh configuration is the first or second configuration,
The determination unit
It has a neural network that learns the feature pattern of gauze images,
A region having a predetermined size in the input image is set as a determination target region, and when the image in the region matches a feature pattern learned by the neural network, the determination target region is determined to include a gauze image feature. To do.

  According to the seventh configuration, if an appropriate learning material is used and the feature pattern of the gauze image is sufficiently learned, it is accurately determined whether or not the gauze region is present in the image of the operative field. Is possible.

An eighth configuration is any one of the first to seventh configurations,
For a plurality of temporally continuous input images, a motion detection unit that detects a moving direction of a region that is determined by the determination unit to include a gauze image feature;
A counting unit that counts the number of gauze that has moved from outside the range of the input image into the range and the number of gauze that has moved out of the range of the input image based on the detection result of the motion detection unit; Prepared,
If the number of gauze that has moved from the inside of the input image to the outside of the range of the input image is smaller than the number of the gauze that has moved from the outside of the input image to the inside of the range, the determination result output unit notifies the remaining gauze.

  According to the eighth configuration, by detecting the moving direction of the area determined to be a gauze image, the probability that the gauze remains in the operative field by counting the gauze in and out of the operative field (probability) Can be determined by image processing.

A computer program as one aspect of the present invention is:
A computer-readable program that causes a computer processor to perform gauze detection processing,
The image detection process
Image input processing for inputting the captured image;
A determination process in which an area of a predetermined size in the input image is set as a determination target area, and whether the determination target area includes a feature of the gauze image is determined by image processing;
A determination result output process for notifying that the gauze is detected in the input image when there is an area determined to include the characteristics of the gauze image by the determination process.

[Embodiment]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated. In addition, in order to make the explanation easy to understand, in the drawings referred to below, the configuration is shown in a simplified or schematic manner, or some components are omitted. Further, the dimensional ratio between the constituent members shown in each drawing does not necessarily indicate an actual dimensional ratio.

[First Embodiment]
(1. Overview of the first embodiment)
The gauze detection system according to the first embodiment reads an image from a camera that images a surgical field, performs image processing for each frame, and detects the characteristics of the gauze image from the captured image. As a result of this feature detection, if it is determined that there is a high probability that the features of the gauze image are included, the region showing the features of the gauze image in the captured image is highlighted, etc. On the other hand, it warns that gauze exists in the operative field.

  This gauze detection system can be implemented, for example, as a function of a surgical video recording / delivery system. In other words, a surgical video recording / recording system in which a camera attached to an endoscope, a surgical field camera fixed in the operating room, a server installed outside the operating room, and a monitor installed in the operating room are connected via a network. By introducing this gauze detection system into the distribution system, misplacement of gauze during surgery can be effectively prevented.

(2. Explanation of schematic configuration)
FIG. 1 is a block diagram showing a functional schematic configuration of a gauze detection system 100 according to the first embodiment. The block diagram shown in FIG. 1 conceptually shows the functions of the gauze detection system 100 classified into blocks, and each block actually needs to be implemented as individual hardware. It is not a thing. Each block can be realized by the CPU of the server or computer executing a predetermined program. The same applies to other block diagrams.

  As shown in FIG. 1, a gauze detection system 100 according to the first embodiment targets an image input unit 1 that inputs an image of an operative field taken by a camera, and an area of a predetermined size in the input image. If there is a determination unit 2 that determines whether or not the region includes a feature of the gauze image and a region that is determined by the determination unit 2 to include the feature of the gauze image, the gauze is present in the surgical field. And a judgment result output unit 3 for calling attention.

  The image input unit 1 inputs an image of the operative field from a camera attached to an endoscope, a fixed camera attached to, for example, a surgical light or the like in an operating room. The image of the operative field is generally input as a moving image, but may be a frame-advanced still image. The image input unit 1 extracts an image for one frame from the input image and sends it to the determination unit 2.

  The determination unit 2 performs image processing by dividing the image for one frame received from the image input unit 1 into small regions, and determines whether or not the features of the gauze image are included in the region. Therefore, the determination unit 2 of the present embodiment includes an image dividing unit 20, a binarization processing unit 21, and a block detection unit 22, as shown in FIG.

  The image dividing unit 20 divides the image for one frame received from the image input unit 1 into small regions that are processing units of the subsequent binarization processing and block detection processing. The binarization processing unit 21 performs binarization processing on one small region divided by the image dividing unit 20 and classifies each pixel into a white pixel and a black pixel. The block detection unit 22 detects whether or not a black pixel block having a predetermined area or less exists in the binarized small region. Based on the number of black pixel blocks detected by the block detection unit 22, the determination unit 2 determines whether or not the region includes a gauze image feature.

(3. Explanation of processing by the determination unit)
Here, the flow of image processing by the determination unit 2 will be described with reference to FIG.

  First, the image dividing unit 20 divides the image for one frame received from the image input unit 1 into the aforementioned small regions. In other words, the image dividing unit 20 determines a region (generally a rectangle) of a predetermined size from the image for one frame received from the image input unit 1 as a target region (determination target region) for subsequent determination processing. (Step S1).

  Next, the binarization processing unit 21 uses a discriminant analysis method for all the pixels included in the determination target region set in step S1 to threshold each pixel into a white pixel and a black pixel. Is set (step S2).

  Binarization by discriminant analysis is generally called "Otsu's binarization", and a threshold value that maximizes the degree of separation (ratio between class variance and class variance) is obtained. This is a method of performing classification according to the obtained threshold value.

Here, when the pixels in the determination target region are classified into two classes of black pixels and white pixels with a certain luminance t as a threshold, the number of black class pixels is n ₁ , the average is m ₁ , and the variance is σ _1. When the number of pixels of the white class is n ₂ , the average is m ₂ , and the variance is σ ₂ , the intra-class variance σ _W ² can be expressed by the following equation (1).
σ _W ² = (n ₁ σ ₁ ² + n ₂ σ ₂ ² ) / (n ₁ + n ₂ ) (1)

The interclass variance σ _b ² can be expressed by the following equation (2), where M is the average luminance value of the entire image.
σ _b ² = {n ₁ (m ₁ −M) ² + n ₂ (m ₂ −M) ² } / (n ₁ + n ₂ ) (2)

Then, the value of the luminance t that maximizes the degree of separation S, which is the ratio of the intra-class variance σ _W ² and the inter-class variance σ _b ² , is determined as a threshold value.

  The binarization processing unit 21 classifies the pixels in the determination target region into white pixels and black pixels using the luminance t thus determined as a threshold (step S3).

  Next, the block detection unit 22 detects a black pixel block corresponding to a “mesh portion” that is one characteristic of the gauze image in the determination target region (step S4). A black pixel block is an area composed of a plurality of black pixels. The black pixel block detected in step S4 is limited to a predetermined area or less. That is, only the black pixel block having an area equal to or smaller than the area of the gauze mesh is detected here. The determination unit 2 determines whether the determination target region is a gauze image according to the number of black pixel blocks detected by the block detection unit 22 (step S5).

  When the gauze image is a black and white binary image, as schematically shown in FIG. 3, a plurality of gauze mesh portions (opening portions) b are scattered in a black block shape in the white portion w. . For this reason, for example, in step S1, the determination unit 2 sets a region for X1 pixels in the horizontal direction and a region for Y1 pixels in the vertical direction as a determination target region. When a black pixel block is detected and a predetermined number or more of black pixel blocks are detected in the determination target area in step S5, it is determined that the determination target area is a gauze image.

  Note that the gauze placed in the patient's body is bent or wrinkled, unlike the state where the gauze is placed on a flat surface, so that the mesh is not necessarily rectangular as shown in FIG. For this reason, the black pixel block detected as a feature corresponding to the gauze mesh is a block having a predetermined area or less, so that even when the gauze mesh is crushed, the mesh can be accurately detected. it can.

  For example, when five or more black pixel blocks are detected in the determination target area, if the determination target area is determined to be a gauze image, in the example shown in FIG. There is a high probability that six black pixel blocks are detected and the gauze image is determined.

  Note that the determination result by the determination unit 2 may be an alternative of “is a gauze image” or “not a gauze image”, or is a numerical value (continuous value or discrete value) representing the probability of being a gauze image. It may be. In the latter case, for example, a plurality of threshold values are provided for the number of black pixel blocks detected by the block detection unit 22, and the probability value that is a gauze image is increased as the detected number increases. It ’s fine. Furthermore, as will be described later, a final determination as to whether or not the image is a gauze image may be made using elements other than the number of black pixel blocks.

  Further, the values of X1 and Y1 for the size of the determination target region may be arbitrarily determined in consideration of processing efficiency and the like. Further, the area threshold value of the black pixel block detected in step S4 as corresponding to the gauze mesh portion may be arbitrarily determined in consideration of the size of the gauze mesh used, the detection accuracy, and the like.

  After performing the processes of steps S2 to S5 on one determination target area, the determination unit 2 returns to step S1 to determine a new determination target area until the determination process is completed for the entire input image. It resets and the process of step S2-S5 is repeated (step S6).

  When the processing is completed for the entire input image (Yes in step S6), the determination unit 2 outputs the processing result to the determination result output unit 3 (step S7). That is, data representing a determination result as to whether or not each of the determination target areas having a size of X1 pixel × Y1 pixel in the input image is a gauze image is passed from the determination unit 2 to the determination result output unit 3. .

  The determination result output unit 3 generates display data for highlighting a portion determined to be a gauze image in the operative field image based on the determination result data passed in step S7, and the like in the operating room or the like. To the monitor (step S8).

  FIG. 4 is a schematic diagram illustrating an example of output to a monitor. In the example of FIG. 4, the screen of the monitor is divided into upper and lower parts, the actual image of the operative field is displayed on the upper screen, and the part that is determined to be a gauze image in the image of the operative field is displayed on the lower screen. Is highlighted. The highlighting method is arbitrary, but it is conceivable to adjust the brightness of the image so that the portion determined to be a gauze image is relatively bright and the other portions are relatively dark. In FIG. 4, the shape of a portion that is an image of an organ other than a gauze image and is displayed relatively dark is schematically indicated by a broken line. In this way, by highlighting the portion determined to be a gauze image, it is possible to alert the surgeon and the assistant that the gauze is present in the operative field in an easy-to-understand manner. It should be noted that the manner of highlighting the gauze portion is not limited to the example shown in FIG. 4 and can take various forms.

(5. Effects of the first embodiment)
As described above, the gauze detection system 100 according to the first embodiment converts a captured image into a binarized image, and detects a predetermined number or more of gauze meshes that appear as blocks of black pixels in the determination target region. Whether or not gauze is present in the operative field is determined depending on whether or not it is performed. And the presence of gauze is alerted to a surgeon and an assistant by highlighting on the monitor the location determined to be a gauze image.

  Thereby, it is possible to effectively prevent the gauze from being left in the patient body only by image processing without performing special processing such as attaching an IC tag to the gauze.

(6. Modifications)
Hereinafter, some modified examples of the gauze detection system 100 according to the present embodiment will be disclosed. In addition to the function of determining the presence or absence of gauze by detecting the gauze mesh that appears as a black pixel block in the binarized image, the following modification example has an auxiliary determination function for more accurately determining the presence or absence of gauze It is what has.

(6.1 First Modification: Determination Based on Cumulative Luminance Difference Value)
FIG. 5 is a block diagram showing a functional schematic configuration of the gauze detection system 101 according to the first modification. As shown in FIG. 5, the gauze detection system 101 further includes a luminance difference detection unit 23 in the determination unit 2 subsequent to the block detection unit 22.

  The luminance difference detection unit 23 detects a luminance difference in the determination target region for the determination target region determined by the block detection unit 22 to include the characteristics of the gauze image, and detects the luminance difference in the determination target region. Exclude non-images. That is, the binarization processing unit 21 classifies all pixels into white pixels and black pixels based on the threshold set by the discriminant analysis method in the determination target region. At this time, in an image with a small difference in luminance (so-called “smooth”), a mesh pattern may occur accidentally. For this reason, the luminance difference detection unit 23 is provided to exclude a determination target region with a small luminance difference so that such a portion of the superimposed image is not detected as a gauze image.

  The luminance difference detection unit 23 obtains a luminance difference between adjacent pixels continuous in at least one of the horizontal direction (X direction) and the vertical direction (Y direction) based on the grayscale image of the determination target region. The luminance difference detector 23 further detects whether or not a luminance change equal to or greater than a predetermined threshold is periodically observed in at least one of the horizontal direction (X direction) and the vertical direction (Y direction).

  That is, in the gauze image, since the fiber portion and the mesh portion appear periodically in the image plane, the luminance difference between adjacent pixels appears relatively large at the boundary portion between the fiber and the mesh. Therefore, in the determination target region including the gauze image, a portion where the luminance difference between adjacent pixels is larger than the threshold when viewed in at least one of the horizontal direction (X direction) and the vertical direction (Y direction) is periodic. Observed at. On the other hand, a determination target area where there is no portion where the luminance difference between adjacent pixels is larger than the threshold value can be excluded as having a high probability of not including a gauze image.

  As described above, the luminance difference detection unit 23 is supplementarily provided, and the presence / absence of the gauze image is comprehensively determined together with the processing result of the determination unit 2 based on the detection of the gauze mesh by the black pixel block in the binarized image. This makes it possible to more reliably determine the presence or absence of gauze in the operative field.

  In the example of FIG. 5, the luminance difference detection unit 23 is provided after the block detection unit 22. However, the luminance difference detection unit 23 is provided before the binarization processing unit 21, and a gauze image is generated by the luminance difference detection unit 23. Only a determination target area that is determined to have a high probability of inclusion may be a target of subsequent processing. It is also possible to adopt a configuration in which the presence or absence of gauze is determined based on only the detection result of the luminance difference detection unit 23. In other words, the binarization processing unit 21 and the block detection unit 22 may be omitted, and a determination target region in which the luminance difference between adjacent pixels periodically changes may be determined as a gauze image.

(6.2 Second Modification: Suppression of Light)
FIG. 6 is a block diagram showing a functional schematic configuration of the gauze detection system 102 according to the second modification. As shown in FIG. 6, the gauze detection system 102 further includes a shine suppression processing unit 24 in the determination unit 2 before the binarization processing unit 21.

  In the image of the operative field, there is a case where the scale is seen on the surface of the organ. Tekari is a state in which the brightness is partially increased by direct reflection of light. If there is such a light, when the binarization process is performed on the portion, a mesh pattern may occur accidentally. In addition, it is rare that a gauze image shows a light. Therefore, it is desirable to exclude the area where the light is seen from the target of the gauze image detection process on the assumption that it is not the gauze area.

  In addition, if there is a scale in the image of the determination target area, the high luminance of that portion becomes noise, which may adversely affect the entire image processing of the determination target area. For example, when the average value of the luminance of the entire determination target region is obtained, the average value may be calculated as a higher value than the original value due to the high luminance of the measuring region. Therefore, in this modification, the shine suppression processing unit 24 detects the shine area and excludes the determination target area including the shine area from the target of the subsequent image processing, thereby suppressing the influence of the shine.

  Based on the grayscale image of the determination target region, the shine suppression processing unit 24 determines a region in which a certain number of pixels having a luminance higher than a predetermined threshold exists as a photographic region. Alternatively, the shine suppression processing unit 24 may detect a pixel having a luminance higher than the average luminance of the region by a threshold or more, and determine a region where a certain number or more of such pixels exist as a photographic region.

  As described above, in the second modification example, by further including the shine suppression processing unit 24, the shine area is excluded from the determination target areas of the gauze image. Thereby, the detection process of a gauze image can be speeded up. Further, it is possible to eliminate an adverse effect on the entire image processing due to the luminance of the pixels in the margin area becoming noise. This makes it possible to detect the gauze image more reliably and at higher speed.

  Note that the shine suppression processing unit 24 may be provided after the block detection unit 22. However, by disposing a processing unit with a relatively small amount of calculation, such as the anti-lighting processing unit 24, in the preceding stage, the determination target area that is determined not to be gauze by the anti-lighting processing unit 24 is determined from the subsequent processing target. Exclusions and subsequent calculations can be omitted. Thereby, there exists an advantage that the process in the judgment part 2 can be sped up.

(6.3 Third Modification: Determination by Hue)
FIG. 7 is a block diagram showing a functional schematic configuration of the gauze detection system 103 according to the third modification. As shown in FIG. 7, the gauze detection system 103 further includes a hue determination unit 25 in the determination unit 2 before the binarization processing unit 21. Note that the hue determination unit 25 can also be arranged after the block detection unit 22. However, from the viewpoint of speeding up the processing, it is desirable to be in the preceding stage of the binarization processing unit 21.

  Based on the color image of the determination target region, the hue determination unit 25 excludes a region having a color that is clearly different from the gauze from the determination target region of the gauze image. For example, since the fat in the body is yellowish, it can be excluded from the determination target region of the gauze image, assuming that the yellow region is fat and not a gauze image.

  As described above, in the third modification, the hue determination unit 25 is further provided, so that an area having a color different from that of the gauze is excluded from the determination target area of the gauze image. Thereby, the detection process of a gauze image can be speeded up. Further, it is possible to eliminate an adverse effect on the entire image processing due to the luminance of the pixels other than the gauze becoming noise. This makes it possible to detect the gauze image more reliably and at higher speed.

(6.4 Fourth Modification: Determination by White Area)
FIG. 8 is a block diagram showing a functional schematic configuration of the gauze detection system 104 according to the fourth modification.

  As shown in FIG. 8, the gauze detection system 104 has a configuration in which a white area detection unit 26 is provided after the block detection unit 22. The white area detection unit 26 determines whether or not there is a white area corresponding to the fiber part of the gauze in the area of a predetermined size in the binarized input image. Here, the white region corresponding to the fiber portion of the gauze is determined based on the continuity of white pixels.

  That is, as shown in FIG. 3, in the gauze image in the binarized input image, a portion corresponding to the gauze fiber appears as a white region w. The gauze mesh portion (opening portion) appears as a black region b. The white area w corresponding to the gauze fiber is an area continuous with at least one of the boundary lines of the determination target area and is not a closed area surrounded by black pixels. That is, when there is a closed region of white pixels that is not continuous with the boundary line of the determination target region in the determination target region, it can be determined that the image of the determination target region is not a gauze image. For example, in the example illustrated in FIG. 9, among the white areas w <b> 1 and w <b> 2 formed of white pixels, the white area w <b> 2 is a closed area of white pixels that is not continuous with any boundary line of the determination target area. When such a closed region is included, it can be determined that the image of the determination target region is not a gauze image.

  On the other hand, as in the example shown in FIG. 3, when the white region w continuously reaches one boundary line to be judged from another boundary line, the white region w is The probability of being an image of the fiber portion is high.

  As described above, the white region detection unit 25 determines that the image of the determination target region is a gauze image depending on whether or not a closed region of white pixels that is not continuous with the boundary line of the determination target region is detected in the binarized image. It can be determined whether or not. Thus, by additionally performing the processing by the white region detection unit 25 on the determination target region determined to include the gauze image by the block detection unit 22, whether each determination target region includes the gauze image or not. The accuracy of such determination can be improved. The white area detection unit 25 is arranged after the binarization processing unit 21 so that the subsequent process is performed only on the determination target area other than the determination target area determined not to be a gauze image by the white area detection unit 25. Anyway.

(6.5 Fifth Modification: Determination by White Area)
FIG. 10 is a block diagram showing a functional schematic configuration of the gauze detection system 105 according to the fifth modification.

  As shown in FIG. 10, in the gauze detection system 105, the determination unit 2 is normalized with respect to the luminance of the pixels in the determination target region based on the grayscale image of the determination target region after the block detection unit 22. In this configuration, a histogram generation unit 27 that generates a histogram and a uniformity determination unit 28 are provided. The histogram generation unit 27 and the uniformity determination unit 28 perform processing on the determination target area determined by the block detection unit 22 to include the features of the gauze image, and the histogram generated by the histogram generation unit 27 is distributed with a predetermined uniformity. If it is determined, the determination target area is determined to include the feature of the gauze image. Note that the histogram generation unit 27 and the uniformity determination unit 28 may be arranged before the binarization processing unit 21.

  That is, in the gauze image, since the gauze fibers and the mesh (opening portion) are regularly arranged, it is assumed that a histogram as shown in FIG. 11 is ideally obtained when the luminance histogram is normalized. Is done. In the histogram of FIG. 11, among the pixels included in the determination target region, the maximum luminance is 1 and the minimum luminance is 0.

  Whether or not “distributed with a predetermined uniformity” is determined by the uniformity determining unit 28, for example, by dividing a normalized histogram into four regions as shown in FIG. This can be determined by determining whether or not the number (frequency) of pixels included in is within a predetermined numerical value.

  As described above, the gauze detection system 105 can determine whether or not the image of the determination target region is a gauze image using the normalized luminance histogram.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described. FIG. 12 is a block diagram showing a functional schematic configuration of the gauze detection system 200 according to the second embodiment.

  As shown in FIG. 12, in the gauze detection system 200, the determination unit 2 includes a neural network 29 that learns the feature pattern of the gauze image. As a learning method for the feature pattern of the gauze image, an arbitrary learning method can be adopted. For example, learning by deep learning is preferable.

  When the neural network 29 is made to learn the feature pattern of the gauze image by deep learning, a large number of gauze images placed in the body during the operation are used. If the sufficiently learned neural network 29 is used, the determination unit 2 targets an area of a predetermined size in the input image, and the image in the area matches the feature pattern learned by the neural network 29. In this case, it can be determined that the region includes the feature of the gauze image.

  As described above, in the second embodiment, it is possible to determine whether or not the image of the determination target region is a gauze image using a neural network that learns the characteristic pattern of the gauze image.

[Third Embodiment]
Hereinafter, a third embodiment of the present invention will be described. FIG. 13 is a block diagram showing a functional schematic configuration of a gauze detection system 300 according to the third embodiment.

  As shown in FIG. 13, the gauze detection system 300 is further provided with a motion detection unit 5 and a counting unit 6 in addition to the configuration described in the first embodiment, and detects the entry and exit of the gauze with respect to the operative field. This is different from the first embodiment.

  The motion detection unit 5 detects the moving direction of the region (gauze region) determined by the determination unit 2 as including the features of the gauze image between the images of a plurality of consecutive frames. Based on the detection result of the motion detection unit 5, the counting unit 6 counts the number of gauze that has moved from outside the range of the operative field image to the range and the number of gauze that has moved out of the range of the operative field image. And count.

  FIG. 14 is a flowchart showing the flow of the gauze entry / exit detection process in the gauze detection system 300.

  Step S11 in FIG. 14 corresponds to the processing of Steps S1 to S6 described in the first embodiment (determination processing by the determination unit 2). That is, the input image of one frame is divided into determination target areas of a predetermined size, and the input is performed by determining whether or not each of the determination target areas includes the characteristics of the gauze image. Detect gauze areas in the image. After that, the motion detection unit 5 obtains the position of the center of gravity of the region determined to include the feature of the gauze image in the operative field image (step S12).

  When the image of the next frame is input, the process returns to step S11 to determine whether or not the input image of the next frame includes the gauze image feature. Also for the image, the position of the center of gravity of the area determined to include the features of the gauze image is obtained (step S12).

  By repeating the processes of steps S11 and S12, the direction in which the position of the center of gravity of the region determined to include the gauze image features moves over a plurality of consecutive frames.

  If the center of gravity of the gauze region appears near one of the four sides of the operative field image in a certain frame and moves toward the inside of the image in the subsequent frame (Yes in step S13), a new gauze is obtained. Is added, the count number of gauze is increased by 1 (step S14).

  On the other hand, when the center of gravity of the gauze region disappears after moving from the inside of the image of the surgical field toward the four sides of the image over a plurality of consecutive frames (Yes in step S15), the gauze is removed from the surgical field. As a result, the count number of gauze is decreased by 1 (step S16).

  If the area containing the features of the gauze image disappears suddenly without the center of gravity moving toward the four sides of the image, the gauze is hidden behind the organ or surgical instrument or buried in the blood There is a high probability of being.

  Therefore, by detecting both the movement direction of the center of gravity position of the gauze region over a plurality of consecutive frames and the relationship between the center of gravity position and the four sides of the image, it is possible to accurately grasp the gauze entry and exit from the operative field. it can.

  And, when the number of gauze that has moved out of the range of the surgical field image is less than the number of gauze that has moved out of the range of the surgical field image into the range at the time of completion of the surgery, That is, when the count number of the gauze at the time when the operation is completed is 1 or more (Yes in Step S17), the determination result output unit 3 notifies that the gauze remains in the surgical field (Step S18). ).

  As described above, in the third embodiment, by obtaining the center of gravity position of the gauze region and the moving direction of the center of gravity position over a plurality of frames, the number of gauze that has entered the surgical field, the number of gauze that has been removed from the surgical field, Count. If the number of gauze entering the operative field and the number of gauze removed from the operative field do not match, alert the surgeon and assistant, for example, by displaying that fact on the monitor. I do. Thereby, misplacement of the gauze can be effectively prevented without performing special processing such as an IC tag on the gauze.

  In addition, in FIG. 13, the structure which added the motion detection part 5 and the counting part 6 to the structure (gauze detection system 100) of 1st Embodiment was illustrated. However, as a configuration in which the motion detection unit 5 and the counting unit 6 are added to the configuration described as the modification of the first embodiment (gauze detection systems 101 to 105) or the configuration of the second embodiment (gauze detection system 200). Also good.

  As mentioned above, although several embodiment of this invention was described, embodiment of this invention is not limited only to the above specific example, A various change is possible. Moreover, it is also possible to implement this invention combining the function demonstrated in each embodiment and the modification suitably.

  In the above description, the embodiment of the gauze detection system has been described as a computer system mounted on a server or a computer. However, a computer program for causing a general-purpose server or computer to realize the function of each block described above and a recording medium recording the computer program are also included in one embodiment of the present invention.

  DESCRIPTION OF SYMBOLS 1 ... Image input part, 2 ... Judgment part, 3 ... Judgment result output part, 5 ... Motion detection part, 6 ... Counting part, 20 ... Image division part, 21 ... Binarization processing part, 22 ... Block detection part, 23 ... Luminance difference detection unit 24. Defocusing processing unit 25. Hue determination unit 26. White region detection unit 27 27 Histogram generation unit 28 28 Uniformity determination unit 29 29 Neural network 100 to 105, 200 , 300 ... Gauze detection system

Claims (9)

An image input unit for inputting a photographed image;
A determination unit that determines, by image processing, whether an area having a predetermined size in the input image is a determination target area and whether the area includes a feature of the gauze image;
A gauze detection system comprising: a determination result output unit for notifying that a gauze has been detected in an input image when there is a determination target region determined to include a gauze image feature by the determination unit.   The gauze detection system according to claim 1, wherein the determination result output unit includes a display control unit that highlights a determination target region that is determined to include a feature of a gauze image in the captured image. The determination unit
A binarization processing unit that performs binarization processing on the determination target region and classifies each pixel into a white pixel and a black pixel;
A block detection unit for detecting a black pixel block having a predetermined area or less in the determination target region,
The gauze according to claim 1 or 2, wherein when the number of black pixel blocks detected by the block detection unit in the determination target region is equal to or greater than a threshold value, the determination target region is determined to include a gauze image feature. Detection system. The determination unit
A luminance difference detection unit for obtaining a luminance difference between adjacent pixels continuous in at least one direction in the determination target region;
4. The determination according to claim 1, wherein when the luminance difference obtained by the luminance difference detection unit is periodically changed, the determination target region is determined to include a gauze image feature. Gauze detection system. The determination unit
A white region detection unit for detecting a closed region of white pixels that is not continuous with a boundary line of the determination target region;
The gauze detection system according to any one of claims 1 to 4, wherein when the white area detection unit detects the closed area, the determination target area is determined not to include a feature of a gauze image. The determination unit
A histogram generator for generating a histogram normalized with respect to the luminance of the pixels in the determination target region;
6. The uniformity determination unit according to claim 1, further comprising: a uniformity determination unit that determines that the determination target region includes a feature of a gauze image when the histogram is distributed with a predetermined uniformity. Gauze detection system. The determination unit
It has a neural network that learns the feature pattern of gauze images,
A region having a predetermined size in the input image is set as a determination target region, and when the image in the region matches a feature pattern learned by the neural network, the determination target region is determined to include a gauze image feature. The gauze detection system according to claim 1 or 2. For a plurality of temporally continuous input images, a motion detection unit that detects a moving direction of a region that is determined by the determination unit to include a gauze image feature;
A counting unit that counts the number of gauze that has moved from outside the range of the input image into the range and the number of gauze that has moved out of the range of the input image based on the detection result of the motion detection unit; Prepared,
When the number of gauze that has moved out of the range of the input image is less than the number of gauze that has moved from outside the range of the input image into the range, the determination result output unit notifies the remaining gauze, The gauze detection system according to any one of claims 1 to 7. A computer-readable program that causes a computer processor to perform gauze detection processing,
The image detection process
Image input processing for inputting the captured image;
A determination process in which an area of a predetermined size in the input image is set as a determination target area, and whether the determination target area includes a feature of the gauze image is determined by image processing;
And a determination result output process for notifying that the gauze is detected in the input image when there is an area determined to include the characteristic of the gauze image by the determination process.

Images