WO2020070832A1

WO2020070832A1 - Endoscope and endoscopic system

Info

Publication number: WO2020070832A1
Application number: PCT/JP2018/037068
Authority: WO
Inventors: 秀範橋本
Original assignee: オリンパス株式会社
Priority date: 2018-10-03
Filing date: 2018-10-03
Publication date: 2020-04-09

Abstract

This endoscope 2 is provided with: an image pickup element 10 equipped with a plurality of pixels that are arranged in a two-dimensional matrix, that receive light from outside, and that generate and output an image pickup signal in accordance with the amount of light received; a comparison unit 20 which makes a comparison so as to determine whether or not output values of image pickup signals outputted from, among the plurality of pixels, a target pixel 10A and peripheral pixels 10B disposed around the target pixel 10A exceed a prescribed threshold value; and an assessment unit 21 which, on the basis of a comparison result of the comparison unit 20, determines whether or not the target pixel 10A is a white defective pixel. When the comparison unit 20 determines that the output value of the image pickup signal outputted from the target pixel 10A exceeds the prescribed threshold value and that the output values of the image pickup signals outputted from the peripheral pixels 10B are not more than the prescribed threshold value, the assessment unit 21 assesses that the target pixel 10A is a white defective pixel.

Description

Endoscope and endoscope system

The present invention relates to an endoscope and an endoscope system for effectively detecting a white defect pixel.

2. Description of the Related Art Conventionally, an endoscope that includes an endoscope that captures an image of a subject inside a subject, a processor that generates an observation image of the subject captured by the endoscope, and a monitor that displays the observation image generated by the processor Mirror systems are widely used in medical fields, industrial fields, and the like.

The endoscope has an image pickup device such as a CCD image sensor or a CMOS image sensor at the distal end of the insertion section. In an image sensor used for an endoscope, a defective pixel called a white defect pixel in which an offset is added to an output may occur due to a variation or deterioration during manufacturing.

For example, when white pixels are generated after the image sensor is mounted on the endoscope, the generated white pixels are detected, learned, and corrected from the image as disclosed in WO2018 / 020686. The technology is known. In this technique, the detection and learning of white spot pixels during imaging of a high-luminance subject are hindered, and thus high-luminance white spot pixels having a large output value are excluded from detection targets.

By the way, when an endoscope equipped with an image sensor that generates high-luminance white pixels is connected to a released processor that does not have a correction circuit for correcting defective pixels, for example, the endoscope is detected in the endoscope. And need to be corrected. When a correction circuit is mounted on an endoscope, a small-scale correction circuit is required because the internal space of the endoscope is limited.

Therefore, an object of the present invention is to provide an endoscope and an endoscope system that can detect a high-intensity white defect with a small-scale circuit.

An endoscope according to one embodiment of the present invention has an imaging element including a plurality of pixels which are arranged in a two-dimensional matrix and receive an external light to generate and output an imaging signal corresponding to a received light amount. A comparison unit that compares whether an output value of the imaging signal output from a pixel of interest and surrounding pixels disposed around the pixel of interest among the plurality of pixels exceeds a predetermined threshold value, A determination unit that determines whether the target pixel is a white defect pixel based on a comparison result of the target unit, wherein the determination unit determines whether the imaging signal is output from the target pixel by the comparison unit. If the output value is determined to exceed the predetermined threshold and the output value of the imaging signal output from the surrounding pixels is determined to be equal to or less than the predetermined threshold, the target pixel is determined to be the white defect pixel I do.

The endoscope system according to one embodiment of the present invention includes a plurality of pixels which are arranged in a two-dimensional matrix and receive light from the outside, generate an imaging signal corresponding to the amount of received light, and output the image signal. An image sensor, and a comparison unit that compares whether or not the output value of the imaging signal output from the pixel of interest and surrounding pixels arranged around the pixel of interest among the plurality of pixels exceeds a predetermined threshold. A determination unit that determines whether the target pixel is a white defect pixel based on a comparison result in the comparison unit, wherein the determination unit outputs the target pixel from the target pixel by the comparison unit. When it is determined that the output value of the imaging signal exceeds the predetermined threshold, and the output value of the imaging signal output from the surrounding pixels is determined to be equal to or less than the predetermined threshold, the target pixel is subjected to the white defect. An endoscope that determines pixels It is connected to the serial endoscope, having a signal processing apparatus for performing signal processing on the imaging signal output from the imaging element.

1 is a configuration diagram illustrating a configuration of an endoscope system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a detailed configuration of the endoscope. FIG. 3 is a diagram illustrating a positional relationship between a target pixel and surrounding pixels among a plurality of pixels of an image sensor. It is a flowchart for demonstrating an example of the flow of the detection processing of a white defect pixel.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a configuration diagram showing a configuration of an endoscope system according to an embodiment of the present invention. As shown in FIG. 1, an endoscope system 1 according to the present embodiment mainly includes an endoscope 2, a processor 3, and a monitor 4.

内 The endoscope 2 of the present embodiment has a configuration that can be introduced into a subject such as a human body and optically images a predetermined observation site in the subject. The subject into which the endoscope 2 is introduced is not limited to a human body, but may be another living body, or may be an artificial object such as a machine or a building.

撮像 The image signal captured by the endoscope 2 is transmitted to the processor 3. The processor 3 is configured to be connected to the endoscope 2, receives an imaging signal from the endoscope 2, performs predetermined signal processing on the received imaging signal, and observes the endoscope 2 on the monitor 4. Display an image.

The endoscope 2 includes an image sensor 10 including a plurality of pixels that are arranged in a two-dimensional matrix, receive light from outside, and generate and output an image signal according to an amount of received light. A defect correction circuit 11 for detecting a defective pixel and correcting an image signal relating to the defective pixel, and a transmission circuit 12 for transmitting the corrected image signal to the processor 3.

Further, the processor 3 as a signal processing device receives the imaging signal transmitted from the endoscope 2, performs predetermined signal processing on the imaging signal received by the receiving circuit 13, and outputs the signal to the monitor 4. An output signal processing circuit 14 is provided. Thereby, the processor 3 can display the optical image (endoscope image) captured by the imaging device 10 of the endoscope 2 on the monitor 4 as an image.

FIG. 2 is a block diagram illustrating a detailed configuration of the endoscope, and FIG. 3 is a diagram illustrating a positional relationship between a target pixel and surrounding pixels among a plurality of pixels of the image sensor.

As shown in FIG. 2, the defect correction circuit 11 includes a comparison unit 20, a determination unit 21, a storage control unit 22, a storage unit 23, and a defect correction unit 24.

{Circle around (3)} As shown in FIG. 3, a predetermined pixel among a plurality of pixels of the image sensor 10 is a target pixel 10A, and eight pixels around the target pixel 10A are peripheral pixels 10B. In the present embodiment, eight pixels around the target pixel 10A are set as the peripheral pixels 10B. However, the peripheral pixels 10B are not limited to the eight pixels around the target pixel 10A. The surrounding pixels 10B may be optimized according to the type.

For example, when a Bayer array RGB filter is arranged in front of each pixel of the image sensor 10, a pixel arranged in the vicinity of the target pixel 10A and having the same color as the target pixel 10A is defined as a peripheral pixel 10B.

The comparing unit 20 determines whether the output value of the imaging signal output from the target pixel 10A and the surrounding pixels 10B disposed around the target pixel 10A among the plurality of pixels of the image sensor 10 exceeds a predetermined threshold. judge. This predetermined threshold is a value for detecting whether or not the target pixel 10A and the surrounding pixels 10B have reached the saturation level.

When the RGB filters in the Bayer array are arranged in front of each pixel of the image sensor 10, different values may be set for the predetermined threshold for each color of the target pixel 10A and each color of the surrounding pixels 10B.

The determination unit 21 determines whether the target pixel 10A is a white defect pixel based on the comparison result of the comparison unit 20. More specifically, the determination unit 21 determines that the output value of the imaging signal output from the pixel of interest 10A exceeds the predetermined threshold by the comparison unit 20, and determines whether the imaging signal output from the surrounding pixels 10B When it is determined that the output value is equal to or less than the predetermined threshold, the target pixel 10A is determined to be a white pixel.

On the other hand, the determination unit 21 determines that the output value of the imaging signal output from the pixel of interest 10A exceeds the predetermined threshold by the comparison unit 20, and determines the output value of the imaging signal output from at least one of the surrounding pixels 10B. When it is determined that the output value exceeds the predetermined threshold, it is determined that the target pixel 10A is not a white pixel. When the output value of the imaging signal output from the target pixel 10A is equal to or smaller than a predetermined threshold, the determination unit 21 determines that the target pixel 10A is not a white defect pixel.

In general, the frequency of occurrence of the white spot pixel at the saturation level is low, so that if the target pixel 10A is a white spot pixel at the saturation level, a white spot pixel at the saturation level may occur adjacent to the surrounding pixel 10B. Poor. Therefore, when the target pixel 10A has reached the saturation level and all the surrounding pixels 10B have not reached the saturation level, the determination unit 21 determines that the target pixel 10A is a white defect pixel.

On the other hand, when the target pixel 10A has reached the saturation level and at least one of the surrounding pixels 10B has reached the saturation level, the determination unit 21 determines that the pixel has reached the saturation level due to the high-luminance subject. It is determined that the target pixel 10A is not a white defect pixel.

That is, even when the target pixel 10A has reached the saturation level, if there is a pixel that has reached the saturation level around the target pixel 10A, the target pixel 10A becomes saturated due to a high luminance pixel depending on the subject. The determination unit 21 determines that the pixel has reached the level, and determines that the target pixel 10A is not a white defect pixel.

(4) The storage unit 23 as the white defect storage unit stores information on the target pixel 10A determined as a white defect pixel, for example, coordinate information of the target pixel 10A determined as a white defect pixel.

The storage control unit 22 as a white defect storage control unit adds information of the target pixel to the storage unit 23 based on the determination result of the determination unit 21 and the information of the target pixel stored in the storage unit 23, and Control is performed to delete the information of the target pixel from the storage unit 23.

Specifically, the storage control unit 22 determines that the target pixel 10A is a white defect pixel when the determination unit 21 determines that the target pixel 10A is a white defect pixel and the information of the target pixel 10A is not stored in the storage unit 23. The information of the noticed pixel 10A is additionally stored in the storage unit 23.

Further, the storage control unit 22 determines that the target pixel 10A is a white defect pixel by the determination unit 21, and the number of target pixels stored in the storage unit 23 is equal to or less than a predetermined number and the storage unit 23 stores the target pixel 10A. If not, the target pixel 10 A is additionally stored in the storage unit 23.

Further, the storage control unit 22 determines that the determination unit 21 determines that the target pixel 10A is not a white defect pixel, and that the storage unit 23 stores information of the target pixel 10A that is determined not to be a white defect pixel. Then, the information of the target pixel 10 A determined to be not a white defect pixel is deleted from the storage unit 23.

The defect correction unit 24 corrects a pixel related to a white defect pixel (defective pixel) of the imaging signal based on the information on the white defect pixel stored in the storage unit 23 and outputs the pixel to the transmission circuit 12.

Next, a description will be given of a process for detecting white flaw pixels of the endoscope thus configured.

FIG. 4 is a flowchart illustrating an example of the flow of the white pixel detection process.

First, the comparison unit 20 compares the output values of the imaging signals of the target pixel 10A and the surrounding pixels 10B with a predetermined threshold (S1). The determination unit 21 determines whether the target pixel 10A is a white defect pixel based on the comparison result by the comparison unit 20 (S2). The determination unit 21 determines that the target pixel 10A is a white defect pixel when the output value of the target pixel 10A exceeds the predetermined threshold value and the output value of the surrounding pixel 10B is equal to or less than the predetermined threshold value. It is determined that the target pixel 10A is not a white defect pixel.

When the determination unit 21 determines that the target pixel 10A is a white defect pixel (S2: YES), the storage control unit 22 stores the information of the target pixel 10A determined as the white defect pixel in the storage unit 23. It is determined whether or not there is (S3).

(4) When the storage control unit 22 determines that the information of the target pixel 10A determined to be a white defect pixel is stored in the storage unit 23 (S3: YES), the processing ends. That is, when the information of the target pixel 10 A determined as the white defect pixel is already stored in the storage unit 23, the storage control unit 22 ends the processing without storing the information in the storage unit 23.

On the other hand, when the storage control unit 22 determines that the information of the target pixel 10 A determined as the white defect pixel is not stored in the storage unit 23 (S 3: NO), the storage control unit 22 determines whether the pixel of interest is stored in the storage unit 23. It is determined whether the number is equal to or more than a predetermined number (S4).

(4) When the storage control unit 22 determines that the number of target pixels stored in the storage unit 23 is equal to or greater than the predetermined number (S4: YES), the processing ends. That is, when the information of the white defect pixel corresponding to the storage capacity of the storage unit 23 is stored, the storage control unit 22 ends the process without storing the information in the storage unit 23.

On the other hand, when it is determined that the number of target pixels stored in the storage unit 23 is not equal to or more than the predetermined number (S4: NO), the storage control unit 22 stores information of the target pixel 10A determined to be a white pixel. The information is additionally stored in the unit 23 (S5), and the process ends.

On the other hand, when the determination unit 21 determines that the target pixel 10A is not a white defect pixel (S2: NO), the storage control unit 22 stores information on the target pixel 10A determined to be not a white defect pixel in the storage unit 23. It is determined whether or not it has been performed (S6).

(4) When the storage control unit 22 determines that the information of the target pixel 10A determined to be not a white defect pixel is not stored in the storage unit 23 (S6: NO), the processing ends. On the other hand, when the storage control unit 22 determines that the information of the target pixel 10A determined to be not a white defect pixel is stored in the storage unit 23 (S6: YES), the target pixel determined to be not the white defect pixel The information of 10A is deleted from the storage unit 23 (S7), and the process ends.

That is, when the information of the target pixel 10A determined to be not the white defect pixel is stored in the storage unit 23, the storage control unit 22 determines that the high-luminance subject has erroneously determined the white defect pixel before. Then, the information of the target pixel 10 A determined to be not the white defect pixel is released from the storage unit 23.

As described above, in the present embodiment, the white pixel is determined by comparing the target pixel 10A and the surrounding pixels 10B with the predetermined thresholds. As a result, the endoscope 2 can detect a high-brightness white flaw only by comparing the target pixel 10A and the surrounding pixels 10B with a predetermined threshold value, so that the high-brightness white flaw can be detected by a small-scale circuit.

In the present embodiment, the comparing unit 20 determines whether or not the output values of the imaging signals output from the target pixel 10A and the surrounding pixels 10B exceed a predetermined threshold, but is not limited thereto. is not. For example, the comparison unit 20 determines whether or not the output value of the imaging signal output from the target pixel 10A exceeds a first threshold, and determines whether the output value of the imaging signal output from the surrounding pixel 10B is the second value. It is determined whether or not the threshold is exceeded.

When there is a pixel that has reached the saturation level due to a high-luminance subject, for example, only the pixel of interest 10A has reached the saturation level, and the surrounding pixels 10B composed of eight pixels are very close to the saturation level. There are cases where the level has not been reached. In this case, the target pixel 10A that has reached the saturation level due to the high-luminance subject is erroneously determined as a white pixel.

Therefore, as described above, the comparison unit 20 determines whether or not the output value of the imaging signal output from the target pixel 10A exceeds the first threshold, and outputs the output of the imaging signal output from the surrounding pixels 10B. It is determined whether the value exceeds the second threshold.

The first threshold value is a value for detecting whether or not the target pixel 10A has reached the saturation level. The second threshold is a value smaller than the first threshold, for example, a value for detecting whether the surrounding pixel 10B has reached 80% to 90% of the saturation level.

Then, the determination unit 21 determines that the output value of the imaging signal output from the target pixel 10A by the comparison unit 20 exceeds the first threshold, and that the output value of the imaging signal output from the surrounding pixel 10B Is determined to be equal to or less than the second threshold, the target pixel 10A is determined to be a white defect pixel.

On the other hand, the determination unit 21 determines that the output value of the imaging signal output from the target pixel 10A by the comparison unit 20 exceeds the first threshold value, and that the imaging unit output from at least one of the surrounding pixels 10B. When it is determined that the output value of the signal exceeds the second threshold, it is determined that the target pixel 10A is not a white defect pixel.

As a result, it is possible to prevent the target pixel 10A reaching the saturation level due to the high-luminance subject from being erroneously determined as a white defect pixel.

Each step in the flowchart in this specification may be executed in a different order, and may be executed at the same time, or may be executed in a different order for each execution, as long as it does not violate its nature.

The present invention is not limited to the above-described embodiments and modified examples, and various changes and modifications can be made without departing from the gist of the present invention.

Claims

An image sensor having a plurality of pixels arranged in a two-dimensional matrix, and receiving light from the outside and generating and outputting an image signal according to the amount of received light,
Of the plurality of pixels, a comparison unit that compares whether the output value of the imaging signal output from the pixel of interest and surrounding pixels arranged around the pixel of interest exceeds a predetermined threshold,
A determining unit that determines whether the pixel of interest is a white defect pixel based on a comparison result of the comparing unit;
With
The determination unit determines that the output value of the imaging signal output from the pixel of interest by the comparison unit exceeds the predetermined threshold, and the output value of the imaging signal output from the surrounding pixels is The endoscope, wherein, when it is determined that the pixel is not more than the predetermined threshold, the pixel of interest is determined to be the white pixel.
A white scratch storage unit that stores information of the target pixel determined to be the white scratch pixel;
Based on the determination result of the determination unit and the information of the target pixel stored in the white defect storage unit, add the information of the target pixel to the white defect storage unit, and from the white defect storage unit A white scratch memory control unit for deleting information of a pixel of interest;
The endoscope according to claim 1, further comprising:
The white defect storage control unit determines that the target pixel is the white defect pixel when the determination unit determines that the target pixel is the white defect pixel, and that the information of the target pixel is not stored in the white defect storage unit. 3. The endoscope according to claim 2, wherein the information on the target pixel is additionally stored in the white flaw storage unit.
The white defect storage control unit determines that the target pixel is the white defect pixel by the determination unit, and the number of the target pixels stored in the white defect storage unit is equal to or less than a predetermined number, and the white defect storage unit 3. The endoscope according to claim 2, wherein when the information of the target pixel is not stored in the white spot storage unit, the information of the target pixel is additionally stored in the white flaw storage unit.
The white defect storage control unit determines that the target pixel is not the white defect pixel by the determination unit, and records information of the target pixel determined to be not the white defect pixel in the white defect storage unit. 3. The endoscope according to claim 2, wherein in the case where the white pixel is not present, information on the pixel of interest determined to be not the white pixel is deleted from the white pixel storage unit.
The comparison unit compares whether an output value of the imaging signal output from the pixel of interest exceeds a first threshold, and determines whether an output value of the imaging signal output from the surrounding pixels is equal to the first threshold value. 2. The endoscope according to claim 1, wherein it is determined whether or not a second threshold smaller than the threshold is exceeded. 3.
The endoscope according to claim 1, wherein the surrounding pixels have the same color as the target pixel.
The endoscope according to claim 1, wherein the predetermined threshold is set for each color of the pixel of interest.
The endoscope according to claim 1, wherein the predetermined threshold is set for each color of the surrounding pixels.
An endoscope according to any one of claims 1 to 8,
A signal processing device connected to the endoscope and performing signal processing on the imaging signal output from the imaging device,
An endoscope system comprising: