JP2020014808A - Endoscope apparatus, operation method for the same, and program for endoscope apparatus - Google Patents

Abstract

To provide an endoscope apparatus capable of detecting a position of a specific area formed on a subject by measured auxiliary light by appropriately controlling a quantity of light of the measured auxiliary light, an operation method for the same, and a program for the endoscope apparatus.SOLUTION: A position of a specific area formed by measured auxiliary light on a subject is identified on the basis of a first captured image obtained by capturing a subject illuminated by illumination light and the measured auxiliary light. A quantity of the measured auxiliary light is controlled on the basis of at least one or more of an observation distance showing a distance from the subject, a quantity of the illumination light, and the position of the specific area.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an endoscope apparatus for measuring the size of a subject, an operation method thereof, and an endoscope apparatus program.

  2. Description of the Related Art In an endoscope apparatus, a distance to an observation target or a size of the observation target is acquired. For example, in Patent Literature 1, measurement auxiliary light different from illumination light for illuminating a subject is irradiated to form a spot on the subject. The position of a spot is specified from a captured image obtained by capturing an image of a subject. An index figure indicating the actual size of the observation target included in the subject is set according to the position of the spot, and the set measurement marker is displayed on the captured image. By using the measurement marker displayed on the captured image, the size of the observation target can be measured.

WO2018 / 051680

  In Patent Literature 1, the measurement auxiliary light used for measuring a subject is irradiated with a constant light amount, and thus has the following problem. For example, if the distance to the subject is long, the spot light formed by the measurement auxiliary light is weak against reflected light of illumination light from surrounding mucous membranes, blood, residues, body fluids, and the like. In this case, since the contrast of the spot light portion on the captured image becomes small, it has been difficult to secure S / N for detecting the position of the spot. On the other hand, when the distance to the subject is short, the spot light is strongly reflected on the surface of the subject such as a living mucous membrane, so that halation occurs or the scattering component near the surface layer of the mucous membrane increases. As a result, the spot diameter becomes large, and it has been difficult to ensure the detection accuracy of the spot position.

  When measuring the size of a subject as in Patent Document 1, the measurement auxiliary light is illuminated while illuminating the illumination light. The amount of illumination light is controlled based on brightness information obtained from a captured image (AE (Auto Exposure)). Further, depending on the model, there is a case where the AE level can be changed or the amount of illumination light can be adjusted so that the brightness level is arbitrarily set by the user. Therefore, when the illumination light changes but the measurement auxiliary light is constant and does not change, the contrast between the spot light and the illumination light cannot be obtained, and the position of the spot cannot be detected in some cases.

  The present invention relates to an endoscope apparatus capable of detecting the position of a specific area formed on a subject by measurement auxiliary light by appropriately controlling the amount of measurement auxiliary light, and an operation method thereof and an endoscope. It is an object of the present invention to provide a mirror device program.

  The endoscope apparatus of the present invention is an illumination light source unit that emits illumination light for illuminating a subject, a measurement auxiliary light source unit that emits measurement auxiliary light used for measurement of the subject, and an imaging element that captures an image of the subject. A position specifying unit that specifies a position of a specific region formed by the measurement auxiliary light on the subject based on a first captured image obtained by imaging the subject illuminated by the illumination light and the measurement auxiliary light; A light emission control unit that controls the amount of measurement auxiliary light based on at least one of an observation distance indicating a distance, an amount of illumination light, or a position of a specific area, and measurement set according to the position of the specific area A display control unit that displays, on the display unit, a specific image displayed in the first captured image or a second captured image obtained by capturing an image of a subject illuminated by the illumination light.

  It is preferable that the light emission control unit controls the light quantity of the measurement auxiliary light based on a first light emission control table that stores a first relationship between the position of the specific area and the light quantity level of the measurement auxiliary light. The light emission control unit controls the amount of measurement auxiliary light based on a second light emission control table that stores a position of the specific area and a second relationship between the amount of illumination light and the amount of measurement auxiliary light. Is preferred. In the second light emission control table, it is preferable that the light amount level of the illumination light and the light amount level of the measurement auxiliary light are set to a ratio that allows the position specifying unit to specify the position of the specific area.

  It is preferable that the position specifying unit specifies the position of the specific area based on the plurality of first captured images obtained at different timings. It is preferable that the position of the specific region is obtained based on an arithmetic average or a moving average of coordinate information of the position of the specific region obtained from each of the plurality of first captured images. If the position specifying unit cannot specify the position of the specific area from the first captured image obtained at the Nth timing, the specific area is determined based on the first captured image obtained at a specific timing different from the first timing. Is preferably specified. The measurement marker is a first measurement marker indicating the actual size of the subject, or a first measurement marker including an intersection line on the subject formed by the measurement auxiliary light, and a scale on the intersection line serving as an index of the size of the subject. It is preferable to include two measurement markers.

  The present invention provides an illumination light source unit that emits illumination light for illuminating a subject, a measurement auxiliary light source unit that emits measurement auxiliary light used for measurement of a subject, and an endoscope device including an imaging element that captures an image of the subject. In the operation method, the position specifying unit specifies a position of a specific region formed by the measurement auxiliary light on the subject based on a first captured image obtained by imaging the subject illuminated by the illumination light and the measurement auxiliary light. And controlling the light quantity of the measurement auxiliary light based on at least one of the observation distance indicating the distance to the subject, the light quantity of the illumination light, or the position of the specific area. The control unit displays the measurement marker set in accordance with the position of the specific region in the first captured image or the second captured image obtained by capturing the subject illuminated by the illumination light. And a step of displaying an image on the display unit.

  The present invention provides an illumination light source unit that emits illumination light for illuminating a subject, a measurement auxiliary light source unit that emits measurement auxiliary light used for measurement of a subject, and an endoscope device including an imaging element that captures an image of the subject. In the installed endoscope program, the computer uses the measurement auxiliary light on the object based on the first captured image obtained by imaging the object illuminated by the illumination light and the measurement auxiliary light. The function of specifying the position of the specific region to be formed, and the light emission control unit performs measurement auxiliary light based on at least one of the observation distance indicating the distance to the subject, the amount of illumination light, or the position of the specific region. And a display control unit that captures a measurement marker set in accordance with the position of a specific area with a first captured image or a subject illuminated with illumination light Specific image displayed on the second captured image obtained Te to execute the function of displaying a.

  According to the present invention, by appropriately controlling the light amount of the measurement auxiliary light, it is possible to detect the position of the specific area formed on the subject by the measurement auxiliary light.

It is an external view of an endoscope apparatus. FIG. 2 is a plan view showing a distal end portion of the endoscope. FIG. 2 is a block diagram illustrating functions of the endoscope apparatus. It is a block diagram which shows a measurement auxiliary light emission part. It is explanatory drawing which shows the relationship between the front-end | tip part of an endoscope, the near end Px in the range Rx of observation distance, the center vicinity Py, and the far end Pz. FIG. 3 is a block diagram illustrating functions of a signal processing unit. FIG. 9 is an explanatory diagram showing a method of specifying the position of a spot SP by averaging based on a plurality of first captured images obtained at different timings. FIG. 9 is an explanatory diagram illustrating a method of specifying the position of a spot SP by a moving average based on a plurality of first captured images obtained at different timings. FIG. 9 is an explanatory diagram showing a method for specifying the position of a spot SP from the first captured image at the (N-1) th timing when the position of the spot SP cannot be specified from the first captured image at the Nth timing. FIG. 3 is a block diagram illustrating functions of a system control unit. 6 is a table showing a first light emission control table. FIG. 6 is an image diagram showing coordinate areas 1, 2, 3, 4, and 5 in a first captured image. It is a table | surface which shows the table for 2nd light emission control. 9 is a flowchart showing a flow of light quantity control of measurement auxiliary light in a length measurement mode. FIG. 7 is an image diagram showing a spot and a first measurement marker when an observation distance is a near end Px. FIG. 9 is an image diagram showing a spot and a first measurement marker when the observation distance is near the center Py. FIG. 9 is an image diagram showing a spot and a first measurement marker when an observation distance is a far end Pz. It is explanatory drawing which shows the 1st measurement marker of a scaled cross shape, a distorted cross shape, a circle and cross shape, and a measurement point cloud type. It is explanatory drawing which shows the chart-shaped chart used for the measurement of the relationship between the position of a spot and the size of the 1st measurement marker in case an observation distance is the near end Px. It is explanatory drawing which shows the graph-shaped chart used for measurement of the relationship between the position of a spot, and the magnitude | size of a 1st measurement marker when an observation distance is a far end Py. 9 is a graph illustrating a relationship between a pixel position in the X direction of the spot and the number of pixels in the X axis direction of the first measurement marker. It is a graph which shows the relationship between the Y direction pixel position of a spot, and the X-axis direction pixel number of the 1st measurement marker. 9 is a graph showing a relationship between a pixel position in the X direction of the spot and the number of pixels of the first measurement marker in the Y axis direction. It is a graph which shows the relationship between the Y direction pixel position of a spot, and the number of pixels of the 1st measurement marker in the Y direction. It is an image figure showing three concentric markers with the same color, respectively. FIG. 8 is an image diagram showing three concentric markers having different colors. FIG. 4 is an image diagram showing a distorted concentric marker. It is an image figure showing an intersection line and a scale.

  As shown in FIG. 1, the endoscope device 10 includes an endoscope 12, a light source device 14, a processor device 16, a monitor 18, and a user interface 19. The endoscope 12 is optically connected to the light source device 14 and is electrically connected to the processor device 16. The processor device 16 is electrically connected to a monitor 18 (display unit) that displays an image. The user interface 19 is connected to the processor device 16 and is used for various setting operations on the processor device 16. The user interface 19 includes a mouse and the like in addition to the illustrated keyboard.

  The endoscope 12 includes an insertion portion 12a inserted into a subject, an operation portion 12b provided at a base end portion of the insertion portion 12a, a bending portion 12c and a distal end portion 12d provided at a distal end side of the insertion portion 12a. have. By operating the angle knob 12e of the operation section 12b, the bending section 12c performs a bending operation. With this bending operation, the tip 12d is directed in a desired direction.

  The endoscope 12 has a normal mode and a length measurement mode, and these two modes are switched by a mode changeover switch 13a provided on the operation unit 12b of the endoscope 12. The normal mode is a mode in which an observation target is illuminated with illumination light. In the length measurement mode, illumination light or measurement auxiliary light is illuminated on the observation target, and a measurement marker used for measuring the size of the observation target is displayed on a captured image obtained by imaging the observation target. The measurement auxiliary light is light used for measuring a subject. In addition, in the endoscope 12, in addition to the normal mode and the length measurement mode, a special light mode for illuminating a subject with special light used to emphasize a specific part may be provided as illumination light.

  Further, the operation unit 12b of the endoscope 12 is provided with a freeze switch 13b for operating a still image acquisition instruction for instructing acquisition of a still image of a captured image. When the user operates the freeze switch 13b, the screen of the monitor 18 freeze-displays, and emits an alert sound (for example, “Pee”) to acquire a still image. Then, the still images of the captured images obtained before and after the operation timing of the freeze switch 13b are stored in the still image storage unit 42 (see FIG. 3) in the processor device 16.

  Note that the still image storage unit 42 is a storage unit such as a hard disk or a USB (Universal Serial Bus) memory. When the processor device 16 is connectable to a network, a still image of a captured image is stored in a still image storage server (not shown) connected to the network instead of or in addition to the still image storage unit 42. You may.

  Note that a still image acquisition instruction may be issued using an operating device other than the freeze switch 13b. For example, a foot pedal may be connected to the processor device 16 and a still image acquisition instruction may be issued when the user operates a foot pedal (not shown) with the foot. The mode switching may be performed by a foot pedal. In addition, a gesture recognition unit (not shown) for recognizing a gesture of the user is connected to the processor device 16, and when the gesture recognition unit recognizes a specific gesture performed by the user, a still image acquisition instruction is issued. It may be. Mode switching may also be performed using the gesture recognition unit.

  Also, a gaze input unit (not shown) provided near the monitor 18 is connected to the processor device 16, and the gaze input unit recognizes that the gaze of the user is within a predetermined area of the monitor 18 for a certain period of time. In this case, a still image acquisition instruction may be issued. Further, a voice recognition unit (not shown) may be connected to the processor device 16 so that the voice recognition unit issues a still image acquisition instruction when recognizing a specific voice generated by the user. The mode switching may also be performed using the voice recognition unit. Further, an operation panel (not shown) such as a touch panel may be connected to the processor device 16 and a still image acquisition instruction may be issued when a user performs a specific operation on the operation panel. Mode switching may also be performed using the operation panel.

  As shown in FIG. 2, the distal end portion of the endoscope 12 has a substantially circular shape, and the objective lens 21 located closest to the subject among the optical members constituting the imaging optical system of the endoscope 12, An illumination lens 22 for illuminating the object with illumination light, a measurement auxiliary lens 23 for illuminating the object with measurement auxiliary light, which will be described later, an opening 24 for projecting the treatment tool toward the object, An air supply / water supply nozzle 25 for supplying water is provided.

  The optical axis Ax (see FIG. 5) of the objective lens 21 extends in a direction perpendicular to the paper surface. The first vertical direction D1 is orthogonal to the optical axis Ax, and the second horizontal direction D2 is orthogonal to the optical axis Ax and the first direction D1. The objective lens 21 and the measurement auxiliary lens 23 are arranged along the first direction D1.

  As shown in FIG. 3, the light source device 14 includes a light source unit 26 and a light source control unit 27. The light source unit 26 (illumination light source unit) generates illumination light for illuminating a subject. Illumination light emitted from the light source unit 26 is incident on the light guide 28, passes through the illumination lens 22, and irradiates the subject. The light source unit 26 includes a white light source that emits white light, a plurality of light sources including a white light source and a light source that emits light of another color (for example, a blue light source that emits blue light), or the like, as a light source of the illumination light. Is preferably used. The light source controller 27 is connected to the system controller 41 of the processor device 16. The light source control unit 27 controls the light source unit 26 based on an instruction from the system control unit 41. The system control unit 41 (light emission control unit) gives an instruction regarding light source control to the light source control unit 27, and also controls the light source 30a (see FIG. 4) of the measurement auxiliary light emitting unit 30. Details of the light source control by the system control unit 41 will be described later.

  An illumination optical system 29a, an imaging optical system 29b, and a measurement auxiliary light emitting unit 30 are provided at a distal end portion 12d of the endoscope 12. The illumination optical system 29a has an illumination lens 22, through which light from a light guide 28 is applied to an observation target. The imaging optical system 29b has an objective lens 21 and an imaging element 32. The reflected light from the observation target enters the image sensor 32 via the objective lens 21. Thereby, a reflection image of the observation target is formed on the image sensor 32.

  The image sensor 32 is a color image sensor, and captures a reflected image of the subject and outputs an image signal. The image sensor 32 is preferably a charge coupled device (CCD) image sensor, a complementary metal-oxide semiconductor (CMOS) image sensor, or the like. The image sensor 32 used in the present invention is a color image sensor for obtaining RGB image signals of three colors of R (red), G (green), and B (blue). The imaging device 32 is controlled by the imaging control unit 33. Note that, as the image sensor 32, a complementary color image sensor provided with complementary color filters of C (cyan), M (magenta), Y (yellow), and G (green) may be used.

  The image signal output from the image sensor 32 is transmitted to the CDS / AGC circuit 34. The CDS / AGC circuit 34 performs correlated double sampling (CDS (Correlated Double Sampling)) and automatic gain control (AGC (Auto Gain Control)) on an image signal that is an analog signal. The image signal that has passed through the CDS / AGC circuit 34 is converted into a digital image signal by an A / D converter (A / D (Analog / Digital) converter) 35. The A / D-converted digital image signal is input to the processor device 16 via the communication I / F (Interface) 36.

  The processor device 16 includes a communication I / F (Interface) 38 connected to the communication I / F 36 of the endoscope 12, a signal processing unit 39, a display control unit 40, and a system control unit 41. The communication I / F receives the image signal transmitted from the communication I / F 36 of the endoscope 12 and transmits the image signal to the signal processing unit 39. The signal processing unit 39 has a built-in memory for temporarily storing an image signal received from the communication I / F 38, and processes an image signal group that is a set of image signals stored in the memory to generate a captured image. .

  When the length measurement mode is set, the signal processing unit 39 performs a structure emphasis process for emphasizing a structure such as a blood vessel on the captured image, and a process of emphasizing a normal part and a lesion part of the observation target. Color difference enhancement processing in which the color difference is extended may be performed. The signal processing unit 39 and the system control unit 41 are provided in an external processing device (not shown) connected to the processor device 16, and are associated with the signal processing unit 39 and the system control unit 41. Processing may be performed. For example, when the measurement mode is set, the first captured image including the spot SP is sent to an external processing device, and the position of the spot SP is specified from the first captured image by the external processing device. Good. Further, in the external processing device, a measurement marker may be set according to the position of the spot SP, and a specific image in which the set measurement marker is displayed in the first captured image may be generated. The generated specific image is transmitted to the processor device 16.

  The display control unit 40 displays the captured image generated by the signal processing unit 39 on the monitor 18. The system control unit 41 controls the imaging device 32 via an imaging control unit 33 provided in the endoscope 12. The imaging control unit 33 controls the CDS / AGC 34 and the A / D 35 in accordance with the control of the imaging element 32. The still image storage control unit 43 controls a still image of a captured image stored in the still image storage unit 42.

  As shown in FIG. 4, the measurement auxiliary light emitting unit 30 (measurement auxiliary light source unit) includes a light source 30a, a diffractive optical element DOE 30b (Diffractive Optical Element), a prism 30c, and a measurement auxiliary lens 23. The light source 30a emits light (specifically, visible light) of a color that can be detected by the pixels of the imaging device 32, and includes a light emitting device such as a laser light source LD (Laser Diode) or an LED (Light Emitting Diode). And a condenser lens for condensing the light emitted from the light emitting element.

  The wavelength of the light emitted from the light source 30a is preferably, for example, red light having a wavelength of 600 nm or more and 650 nm or less. Alternatively, green light of 495 nm or more and 570 nm or less may be used. The light source 30a is controlled by the system control unit 41, and emits light based on an instruction from the system control unit 41. The DOE 30b converts the light emitted from the light source into measurement auxiliary light used for measuring a subject.

  The prism 30c is an optical member for changing the traveling direction of the measurement auxiliary light converted by the DOE 30b. The prism 30c changes the traveling direction of the measurement auxiliary light so as to intersect with the field of view of the imaging optical system including the objective lens 21 and the lens group. The details of the traveling direction of the measurement auxiliary light will also be described later. The measurement auxiliary light Lm emitted from the prism 30c passes through the measurement auxiliary lens 23 and is applied to the subject. By irradiating the subject with the measurement auxiliary light, a spot SP is formed as a circular area (specific area) on the subject (see FIG. 7 and the like). The position of the spot SP is specified by the position specifying unit 50, and a measurement marker indicating the actual size is set according to the position of the spot SP. The set measurement marker is displayed on the captured image.

  Note that the measurement auxiliary light emitting unit 30 may be any unit that can emit the measurement auxiliary light toward the visual field of the imaging optical system. For example, the light source 30a may be provided in a light source device, and the light emitted from the light source 30a may be guided to the DOE 30b by an optical fiber. Alternatively, the measurement auxiliary light Lm may be emitted in a direction crossing the visual field of the imaging optical system by installing the light source 30a and the DOE 30b at an angle to the optical axis Ax without using the prism 30c.

  Regarding the traveling direction of the measurement auxiliary light, the measurement auxiliary light Lm is emitted in a state where the optical axis Lm of the measurement auxiliary light Lm intersects with the optical axis Ax of the objective lens 21, as shown in FIG. Assuming that observation is possible in the range Rx of the observation distance, at the near end Px, near the center Py, and the far end Pz of the range Rx, the measurement auxiliary light in the imaging range (indicated by arrows Qx, Qy, Qz) at each point. It can be seen that the position of the spot SP formed on the subject (the point where each arrow Qx, Qy, Qz intersects the optical axis Ax) differs depending on Lm. Note that the imaging angle of view of the imaging optical system is expressed in an area sandwiched by two solid lines 45, and measurement is performed in a central area of the imaging angle of view having less aberration (an area sandwiched by two dotted lines 46). ing.

  As described above, by emitting the measurement auxiliary light Lm in a state where the optical axis Lm of the measurement auxiliary light intersects with the optical axis Ax, the sensitivity of the movement of the spot position with respect to the change in the observation distance is high. The size can be measured with high accuracy. Then, by imaging the subject illuminated by the measurement auxiliary light with the image sensor 32, a captured image including the spot SP is obtained. In the captured image, the position of the spot SP differs depending on the relationship between the optical axis Ax of the objective lens 21 and the optical axis Lm of the measurement auxiliary light Lm and the observation distance, but if the observation distance is short, the same actual size ( For example, the number of pixels indicating 5 mm) increases, and the number of pixels decreases as the viewing distance increases.

  As shown in FIG. 6, the signal processing unit 39 of the processor device 16 converts the subject illuminated by the illumination light and the measurement auxiliary light to recognize the position of the spot SP (specific region) and set the measurement marker. A position specifying unit 50 for specifying the position of the spot SP in the first captured image obtained by capturing an image, a measurement marker is set based on the position of the spot SP, and the set measurement marker is displayed on the first captured image. A measurement marker setting unit that generates a specific image. The specific image is displayed on the monitor 18 by the display control unit 40. The specific image may be an image in which a measurement marker is set for a second captured image obtained by capturing an image of a subject illuminated by the illumination light.

  The position specifying unit 50 specifies the position of the spot SP based on the first captured image. Specifically, the coordinate information on the position of the spot SP is specified. The spot SP is displayed as a substantially circular red area including many components corresponding to the color of the measurement auxiliary light in the first captured image. Therefore, the position of the spot SP is specified from the substantially circular red area. As a method of specifying the position, for example, the captured image is binarized, and the center of gravity of a white portion (a pixel whose signal intensity is higher than the binarization threshold) in the binarized image is specified as the position of the spot SP. Details of the method of specifying the position will be described later.

  The measurement marker setting unit 52 sets a measurement marker based on the position of the spot SP in the first captured image. The measurement marker setting unit 52 calculates the size of the marker from the position of the spot SP with reference to the marker table 54 that stores the relationship between the position of the spot SP and the size of the measurement marker in the first captured image. . Then, the measurement marker setting section 52 sets a measurement marker corresponding to the size of the marker. Then, the measurement marker setting unit 52 generates a specific image in which the measurement marker is superimposed and displayed on the first captured image.

  The details of the method for specifying the position of the spot SP will be described below. It is preferable that the position specifying unit 50 specifies the position of the spot SP based on not only the first captured image of one frame but also a plurality of first captured images obtained at different timings. In this case, as shown in FIG. 7, the position of the spot SP is determined using the first captured images obtained at the Nth to N + Lth timings (N and L are natural numbers) as the plurality of first captured images. When specifying is made, the coordinate information of the spot SP is acquired from the first captured image at each timing. Here, the coordinate information of the spot SP obtained from the first captured image at the N-th timing, the N + 1-th timing, the N + 2-th timing,..., The N + L-th timing is (x (N), y (N)), (X (N + 1), y (N + 1)), (x (N + 2), y (N + 2)),..., (X (N + L), y (N + L)). Then, (x (N), y (N)), (x (N + 1), y (N + 1)), (x (N + 2), y (N + 2)), ..., (x (N + L) ), Y (N + L)) to obtain the average coordinate information (xk, yk) of the spot SP. The average coordinate information (xk, yk) of the spot SP is set as information on the position of the spot SP specified by the position specifying unit 50. Note that arithmetic averaging may be performed in addition to addition averaging.

  Alternatively, as shown in FIG. 8, for any one of the first captured images at the Nth to N + L timings, for example, for a spot SP (N) included in the first captured image at the Nth timing, The movement amount of the spot SP included in the first captured image at the timing of may be obtained. In this case, the movement amount of the spot SP (N + 1) included in the first captured image at the (N + 1) th timing with respect to the spot SP (N) is V1, and the spot SP ( The amount of movement of (N + 2) with respect to spot SP (N) is V2, and the amount of movement of spot SP (N + 3) included in the first captured image at the (N + 3) th time with respect to spot SP (N) is V3. Also, let VL be the amount of movement of the spot SP (N + 1) included in the first captured image at the (N + L) th timing with respect to the spot SP (N). Then, an average of the movement amounts V1, V2, V3,..., VL is obtained, and the coordinate information (x (N), y (N)) of the spot SP (N) is added to the obtained average. The coordinate information (xk, yk) is obtained. The obtained coordinate information (xk, yk) is set as information on the position of the spot SP specified by the position specifying unit 50.

  Further, in the case where the position of the spot SP is specified from the first captured image of one frame, the position specifying unit 50 specifies the position by the following processing procedure in case that the position specification is impossible. Is preferred. As shown in FIG. 9, first, the spot SP position identification processing is performed on the first captured image obtained at the N-th timing. In the position specifying process, when the position of the spot SP cannot be specified from the first captured image at the N-th timing, such as when there is no area exceeding the threshold for binarization, a specific timing different from the N-th timing Of the spot SP is performed on the first captured image. Here, as the first captured image at the specific timing, the position of the spot SP is specified from the first captured image obtained at the (N-1) th timing (specific timing) before the Nth timing. When the position of the spot SP is specified as a result of the position specifying process, the position of the spot SP is set as information on the position of the spot SP specified by the position specifying unit 50. Here, it is preferable that the plurality of first captured images obtained at the timing before the N-th timing be stored in a temporary storage memory (not shown) in the processor device 16. In this case, it is preferable to delete the first captured image stored in the temporary storage memory after a certain period of time.

  In FIG. 9, the arrow indicating the flow of the processing indicates that the lower the processing is, the later the processing is performed. When the position of the spot SP cannot be specified in the first captured image at the N-th timing, the position specifying process of the spot SP is performed from the first captured image at the (N + 1) -th timing (specific timing) after the N-th timing. It may be performed.

  The details of the light source control in the length measurement mode will be described below. As shown in FIG. 10, the system control unit 41 that performs overall light source control includes a brightness information calculation unit 58, an illumination light amount level setting unit 59, a first light emission control table 60, a second light emission control Table 62. The brightness information calculation unit 58 calculates brightness information regarding the brightness of the subject based on the captured image obtained in the normal mode or the first captured image obtained in the length measurement mode. The illumination light level setting unit 59 sets the illumination light level based on the brightness information. In the present embodiment, the light amount level of the illumination light is set to five levels of Level1, Level2, Level3, Level4, and Level5. The illumination light level set by the illumination light level setting section 59 is sent to the light source control section 27. The light source control unit 27 controls the light source unit 26 so that the amount of illumination light becomes equal to the received illumination light level.

  The first light emission control table 60 is used for light quantity control of the measurement auxiliary light, and stores a first relationship between coordinate information of the spot SP (the position of the specific area) and the light quantity level of the measurement auxiliary light. Specifically, as shown in FIG. 11, the light amount levels Level1, Level2, Level3, Level4, and Level5 of the measurement auxiliary light are defined for five coordinate areas to which the coordinate information of the spot SP belongs. The system control unit 41 refers to the first light emission control table 60 and specifies the light amount level corresponding to the coordinate area to which the position of the spot SP specified by the position specifying unit 50 belongs. The system control unit 41 controls the light source 30a to control the light amount of the measurement auxiliary light so as to reach the specified light amount level. It should be noted that which of the first light emission control table 60 and the second light emission control table 62 is used to control the amount of the measurement auxiliary light is appropriately set by operating the user interface 19.

  As shown in FIG. 12, the coordinate area 1 is an area set at the bottom in the first captured image, and when the spot SP belongs to the coordinate area 1, the observation distance is the closest. . Therefore, the smallest Level 1 is assigned to the coordinate area 1 as the light amount level of the measurement auxiliary light. The coordinate area 2 is an area set above the coordinate area 1, and when the spot SP belongs to the coordinate area 2, the observation distance is farther than in the coordinate area 1. In addition, Levl2, which is larger than Level1, is assigned as the light amount level of the measurement auxiliary light. Note that the moving direction of the spot SP changes according to the cross direction of the optical axis Ax of the objective lens and the optical axis of the measurement auxiliary light Lm. For example, when the observation distance is short, the spot SP is located at the lower left of the first captured image, and as the observation distance increases, the spot SP moves to the upper right. When the observation distance is long, the spot SP is located at the upper right of the first captured image.

  Similarly, the coordinate area 3 is provided above the coordinate area 2. When the spot SP belongs to the coordinate area 3, the observation distance is farther than in the coordinate area 2, and therefore, Level 3 larger than Level 2 is assigned as the light level of the measurement auxiliary light. The coordinate area 4 is provided above the coordinate area 3. When the spot SP belongs to the coordinate area 4, the observation distance is farther than in the coordinate area 3, and therefore, Level 4 larger than Level 3 is assigned as the light level of the measurement auxiliary light. The coordinate area 5 is an area set at the top. When the spot SP belongs to the coordinate area 5, the observation distance is located at the farthest position than in the other coordinate areas 1 to 4, so that the highest Level 5 is assigned as the light amount level of the measurement auxiliary light. I have.

  The second light emission control table 62 is used for light quantity control of the measurement auxiliary light, and stores coordinate information of the spot SP (the position of the specific area) and a second relationship between the light quantity level of the illumination light and the light quantity level of the measurement auxiliary light. ing. Specifically, as shown in FIG. 13, the light amount level of the measurement auxiliary light is different from the five coordinate areas to which the coordinate information of the spot SP belongs and the light amount levels Level1, Level2, Level3, Level4, and Level5 of the illumination light. Stipulated. For example, when the spot SP belongs to the coordinate area 1 and the light level of the illumination light set by the illumination light level setting unit 59 is Level 3, Level 3 is assigned as the light level of the measurement auxiliary light. .

  The system control unit 41 refers to the second light emission control table 62, and refers to the coordinate area to which the position of the spot SP specified by the position specifying unit 50 belongs, and the light intensity of the illumination light set by the illumination light intensity level setting unit 59. From the level, the light amount level of the measurement auxiliary light is specified. The system control unit 41 controls the light source 30a to control the light amount of the measurement auxiliary light so as to reach the specified light amount level.

  In the second light emission control table 62, the light amount level of the illumination light and the light amount level of the measurement auxiliary light are set to a ratio that allows the position specifying unit 50 to specify the position of the spot SP. This is because if the ratio between the light amount of the illumination light and the light amount of the measurement auxiliary light is not appropriate, the contrast of the spot SP becomes low, and it becomes difficult for the position specifying unit 50 to specify the position of the spot SP. .

  Next, a series of flows of light quantity control of the measurement auxiliary light in the length measurement mode will be described with reference to the flowchart of FIG. When the user switches to the length measurement mode by operating the mode change switch 13a, the subject is irradiated with the measurement auxiliary light and the illumination light. A first captured image is obtained by capturing an image of a subject illuminated by the illumination light and the measurement auxiliary light. The position specifying unit 50 specifies the position of the spot SP from the first captured image. When the position of the spot SP is specified, the light amount of the measurement auxiliary light is controlled based on at least one of the observation distance indicating the distance to the subject, the light amount of the illumination light, or the position of the spot SP. For example, the light amount of the measurement auxiliary light is controlled by referring to the first light emission control table 60 storing the first relationship between the coordinate area indicating the area to which the position of the spot SP belongs and the light amount level of the measurement auxiliary light. Further, a measurement marker corresponding to the position of the spot SP is set. The specified image is generated by superimposing and displaying the set measurement marker on the first captured image. The generated specific image is displayed on the monitor 18. The above flow is repeated until the length measurement mode is switched to another mode (for example, the normal mode).

  In the above embodiment, the first relationship between the position of the spot SP and the light amount level of the measurement auxiliary light or the second relationship between the position of the spot SP and the light amount level of the illumination light and the light amount level of the measurement auxiliary light is used. Although the light amount of the measurement auxiliary light is controlled, the control may be performed using other relations. For example, the light amount of the measurement auxiliary light may be controlled using the relationship between the observation distance indicating the distance to the subject and the light amount level of the measurement auxiliary light. Alternatively, the light amount of the measurement auxiliary light may be controlled using the relationship between the observation distance, the light amount level of the illumination light, and the light amount level of the measurement auxiliary light. The observation distance is calculated from the image signal in an observation distance calculation unit (not shown) provided in the processor device 16 in addition to the observation distance calculated from the position of the spot SP. The observation distance may be calculated based on the information.

  Note that the measurement markers include a plurality of types such as a first measurement marker and a second measurement marker, and the user's instruction as to which type of measurement marker is to be displayed on the captured image. Allows selection. As the user's instruction, for example, the user interface 19 is used.

  For example, a cross-shaped measurement marker Mx is used as the first measurement marker. In this case, as shown in FIG. 15, when the observation distance is close to the near end Px, the actual size is 5 mm (horizontal and vertical directions of the captured image) in accordance with the center of the spot SP1 formed on the tumor tm1 of the subject. Direction) is displayed. Since the tumor tm1 substantially matches the range defined by the cross-shaped marker Mx1, the tumor tm1 can be measured to be about 5 mm.

  Similarly, as shown in FIG. 16, when the observation distance is close to the vicinity Py of the center, the actual size is 5 mm (in the horizontal direction of the second captured image and the center of the spot SP2 formed on the tumor tm2 of the subject). (A vertical direction), a cross-shaped marker Mx2 is displayed. Also, as shown in FIG. 17, a cross-shaped marker Mx3 indicating the actual size of 5 mm (the horizontal direction and the vertical direction of the second captured image) is displayed at the center of the spot SP3 formed on the tumor tm3 of the subject. Is done. As described above, the position of the spot on the imaging surface of the image sensor 32 differs depending on the observation distance, and accordingly, the display position of the marker also differs. As shown in FIGS. 15 to 17, as the observation distance increases, the size of the first measurement marker corresponding to the same actual size of 5 mm decreases.

  In FIGS. 15 to 17, the center of the spot SP and the center of the marker are displayed so as to coincide with each other. However, if there is no problem in measurement accuracy, the first measurement marker is located at a position away from the spot SP. May be displayed. However, also in this case, it is preferable to display the first measurement marker near the spot. Further, instead of deforming and displaying the first measurement marker, the first measurement marker in a state where the distortion of the captured image is corrected and not deformed may be displayed on the corrected captured image. .

  15 to 17, the first measurement marker corresponding to the actual size of the subject of 5 mm is displayed. However, the actual size of the subject is an arbitrary value (for example, 2 mm) depending on the observation target and the observation purpose. , 3 mm, 10 mm, etc.). In FIGS. 15 to 17, the first measurement marker has a cross shape in which a vertical line and a horizontal line are perpendicular to each other. However, as shown in FIG. 18, at least one of the vertical line and the horizontal line has a scale. It may be a cruciform cross with Mt. Further, the first measurement marker may be a distorted cross shape in which at least one of a vertical line and a horizontal line is inclined. Further, the first measurement marker may be a circle and a cross, which are a combination of a cross and a circle. In addition, the first measurement marker may be a measurement point cloud type combining a plurality of measurement points EP corresponding to the actual size from the spot. The number of the first measurement markers may be one or more, and the color of the first measurement markers may be changed according to the actual size.

  A method for creating the marker table 54 will be described below. The relationship between the position of the spot and the size of the marker can be obtained by imaging a chart in which patterns of the actual size are regularly formed. For example, a spot-shaped measurement auxiliary light is emitted toward the chart, and the observation distance is changed to change the position of the spot, while changing the position of the spot to a ruled line (5 mm) or a ruled line (for example, 1 mm) smaller than the actual size. The chart is imaged, and the relationship between the spot position (pixel coordinates on the imaging surface of the image sensor 32) and the number of pixels corresponding to the actual size (how many pixels the actual size 5 mm is represented by) is acquired.

  As shown in FIG. 19, (x1, y1) is the pixel position in the X and Y directions of the spot SP4 on the imaging surface of the imaging element 32 (the upper left is the origin of the coordinate system). At the position (x1, y1) of the spot SP4, the number of pixels in the X direction corresponding to the actual size of 5 mm is Lx1, and the number of pixels in the Y direction is Ly1. Such measurement is repeated while changing the observation distance. FIG. 20 shows a state in which the same 5 mm ruled chart as in FIG. 19 is imaged. However, the shooting distance is closer to the far end than in the state of FIG. 19, and the rule interval is narrower. In the state of FIG. 20, the number of pixels in the X direction corresponding to the actual size of 5 mm at the position (x2, y2) of the spot SP5 on the imaging surface of the imaging element 32 is Lx2, and the number of pixels in the Y direction is Ly2. Then, the measurement as shown in FIGS. 19 and 20 is repeated while changing the observation distance, and the results are plotted. 19 and 20 are shown without considering the distortion of the objective lens 21.

  FIG. 21 shows the relationship between the X coordinate of the spot position and Lx (the number of pixels in the X direction of the first measurement marker), and FIG. 22 shows the relationship between the Y coordinate of the spot position and Lx. ing. Lx is expressed as Lx = g1 (x) as a function of the position in the X direction from the relationship shown in FIG. 21, and Ly is expressed as Ly = g2 (y) as a function of the position in the Y direction according to the relationship shown in FIG. Is done. g1 and g2 can be obtained, for example, by the least square method from the above-mentioned plot results.

  Note that the X coordinate and the Y coordinate of the spot have a one-to-one correspondence, and basically the same result (the same number of pixels for the same spot position) is obtained by using either of the functions g1 and g2. Therefore, when calculating the size of the first measurement marker, either of the functions may be used, and the function of g1 or g2 which has higher sensitivity to the change in the number of pixels with respect to the position change may be selected. If the values of g1 and g2 are significantly different, it may be determined that "the spot position could not be recognized".

  FIG. 23 illustrates the relationship between the X coordinate of the spot position and Ly (the number of pixels in the Y direction), and FIG. 24 illustrates the relationship between the Y coordinate of the spot position and Ly. According to the relationship in FIG. 23, Ly is represented as Ly = h1 (x) as the coordinate in the X direction, and from the relationship in FIG. 24, Ly is represented as Ly = h2 (y) as the coordinate in the Y direction. As for Ly, any of the functions h1 and h2 may be used as in Lx.

  The functions g1, g2, h1, and h2 obtained as described above are stored in the marker table in a look-up table format. The functions g1 and g2 may be stored in the marker table in a function format.

  In the second embodiment, as the first measurement marker, as shown in FIG. 25, three concentric markers M4A, M4B, and M4C having different sizes (the diameters are 2 mm, 5 mm, and 10 mm, respectively). ) May be displayed on the first captured image with the spot SP4 formed on the tumor tm4 as the center. The three concentric markers display a plurality of markers, so that the trouble of switching is eliminated, and measurement is possible even when the subject has a non-linear shape. When displaying a plurality of concentric markers around a spot, instead of specifying the size and color for each marker, a combination of a plurality of conditions is prepared in advance, and a selection can be made from the combination. It may be.

  In FIG. 25, all three concentric markers are displayed in the same color (black). However, when a plurality of concentric markers are displayed, a plurality of colored concentric markers of different colors depending on the markers may be used. Good. As shown in FIG. 26, the marker M5A is represented by a dotted line representing red, the marker M5B is represented by a solid line representing blue, and the marker M5C is represented by a dashed line representing white. By changing the color of the marker in this manner, the discriminability is improved, and the measurement can be easily performed.

  In addition, as the first measurement marker, as shown in FIG. 27, a plurality of distorted concentric markers obtained by distorting each concentric circle may be used in addition to a plurality of concentric markers. In this case, the distorted concentric markers M6A, M6B, and M6C are displayed in the first captured image centering on the spot SP5 formed on the tumor tm5.

  Note that, as the measurement auxiliary light, light formed as a spot when the object is irradiated is used, but other light may be used. For example, when the object is irradiated, as shown in FIG. 28, a planar measurement auxiliary light formed as an intersection line 80 on the object may be used. In this case, as a measurement marker, a second measurement marker including an intersection line 80 and a scale 82 serving as an index of the size (for example, polyp P) of the subject on the intersection line is generated. When using a planar measurement auxiliary light, the position specifying unit 50 specifies the position of the intersection line 80 (specific region). The lower the intersection line 80 is, the shorter the observation distance is. The higher the intersection line 80 is, the longer the observation distance is. Therefore, the interval between the graduations 82 increases as the intersection line 80 is positioned lower, and the interval between the graduations 82 decreases as the intersection line 80 is positioned higher.

  In the above embodiment, the hardware structure of a processing unit that executes various types of processing, such as the signal processing unit 39, the display control unit 40, and the system control unit 41, includes various types of processors (processors) as follows. ). For various processors, the circuit configuration is changed after the manufacture of CPUs (Central Processing Units) and FPGAs (Field Programmable Gate Arrays), which are general-purpose processors that execute software (programs) and function as various processing units. It includes a programmable logic device (PLD) that is a possible processor, a dedicated electric circuit that is a processor having a circuit configuration specifically designed to execute various processes, and the like.

  One processing unit may be configured by one of these various processors, or configured by a combination of two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of a CPU and an FPGA). May be done. Further, a plurality of processing units may be configured by one processor. As an example in which a plurality of processing units are configured by one processor, first, as represented by a computer such as a client or a server, one processor is configured by a combination of one or more CPUs and software, There is a form in which this processor functions as a plurality of processing units. Second, as represented by a system-on-chip (System On Chip: SoC), a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more of the above various processors as a hardware structure.

  Furthermore, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

DESCRIPTION OF SYMBOLS 10 Endoscope apparatus 12 Endoscope 12a Insertion part 12b Operation part 12c Bending part 12d Tip part 12e Angle knob 13a Mode changeover switch 13b Freeze switch 14 Light source device 16 Processor device 18 Monitor 19 User interface 21 Objective lens 22 Illumination lens 23 Measurement Auxiliary lens 24 Opening 25 Air / water supply nozzle 26 Light source unit 27 Light source control unit 28 Light guide 29a Illumination optical system 29b Imaging optical system 30 Measurement auxiliary light emitting unit 30a Light source 30c Prism 32 Image sensor 33 Image controller 34 CDS / AGC circuit 35 A / D circuit 36 Communication I / F (Interface)
38 Communication I / F (Interface)
39 signal processing unit 40 display control unit 41 system control unit 42 still image storage unit 43 still image storage control unit 45 solid line 46 dotted line 50 position specification unit 52 measurement marker setting unit 54 marker table 58 information calculation unit 59 illumination light quantity level Setting unit 60 First emission control table 62 Second emission control table 80 Intersecting line EP Measurement point Mt Scale Mx, M1, M2, M3 Cross-shaped marker My Concentric measurement marker tm, tm1, tm2, tm3, tm4, tm5 Tumor SP spots SP1, SP2, SP3, SP4, SP5 Spot Lx1, Lx2 Number of pixels in X direction Ly1, Ly2 Number of pixels in Y direction M4A, M4B, M4C, M5A, M5B, M5C Concentric markers M6A, M6B, M6C Distorted Concentric Marker P Polyp

Claims (10)

An illumination light source unit that emits illumination light for illuminating a subject;
A measurement auxiliary light source that emits measurement auxiliary light used for measurement of the subject,
An image sensor for imaging the subject;
A position specifying unit that specifies a position of a specific area formed by the measurement auxiliary light on the subject based on a first captured image obtained by capturing the object illuminated by the illumination light and the measurement auxiliary light. When,
A light emission control unit that controls the light amount of the measurement auxiliary light based on at least one of the observation distance indicating the distance to the subject, the light amount of the illumination light, or the position of the specific region,
The measurement marker set according to the position of the specific area, the specific image displayed on the first captured image or the second captured image obtained by capturing the subject illuminated by the illumination light is displayed on the display unit. An endoscope device comprising: a display control unit for displaying.   2. The light emission control unit controls the light intensity of the measurement auxiliary light based on a first light emission control table that stores a first relationship between a position of the specific area and a light intensity level of the measurement auxiliary light. The endoscope apparatus according to claim 1.   The light emission control unit is configured to control the position of the specific area and a second light emission control table that stores a second relationship between the light amount level of the illumination light and the light amount level of the measurement auxiliary light. The endoscope apparatus according to claim 1, wherein the endoscope apparatus controls a light amount.   In the second light emission control table, the light amount level of the illumination light and the light amount level of the measurement auxiliary light are set to a ratio for enabling the position specifying unit to specify the position of the specific area. The endoscope device according to claim 3.   The endoscope apparatus according to any one of claims 1 to 4, wherein the position specifying unit specifies the position of the specific area based on a plurality of first captured images obtained at different timings.   The endoscope apparatus according to claim 5, wherein the position of the specific region is obtained based on an arithmetic average or a moving average of coordinate information of the position of the specific region obtained from each of the plurality of first captured images.   If the position specifying unit cannot specify the position of the specific region from the first captured image obtained at the Nth timing, based on the first captured image obtained at a specific timing different from the first timing. 5. The endoscope apparatus according to claim 1, wherein the position of the specific area is specified.   The measurement marker is a first measurement marker indicating the actual size of the subject, or an intersection line on the subject formed by the measurement auxiliary light, and an index of the size of the subject on the intersection line. The endoscope apparatus according to any one of claims 1 to 7, further comprising a second measurement marker including a scale that becomes: An operation method of an endoscope apparatus including an illumination light source unit that emits illumination light for illuminating an object, a measurement auxiliary light source unit that emits measurement auxiliary light used for measurement of the object, and an image sensor that images the object At
Based on a first captured image obtained by capturing an image of the subject illuminated by the illumination light and the measurement auxiliary light, a position specifying unit determines a position of a specific area formed by the measurement auxiliary light on the subject. Identifying steps;
A step of controlling the light amount of the measurement auxiliary light based on at least one of an observation distance indicating the distance to the subject, the light amount of the illumination light, or the position of the specific region,
A display control unit configured to display a measurement marker set according to the position of the specific region in the first captured image or a second captured image obtained by capturing the subject illuminated by the illumination light; Displaying an image on a display unit. An illumination light source unit that emits illumination light for illuminating an object, a measurement auxiliary light source unit that emits measurement auxiliary light used for measurement of the object, and an endoscope device that is installed in an endoscope apparatus that includes an imaging element that images the object. Endoscope program,
On the computer
Based on a first captured image obtained by capturing an image of the subject illuminated by the illumination light and the measurement auxiliary light, a position specifying unit determines a position of a specific area formed by the measurement auxiliary light on the subject. Features to identify,
A function of controlling the light amount of the measurement auxiliary light based on at least one of an observation distance indicating the distance to the subject, the light amount of the illumination light, or the position of the specific region,
A display control unit configured to display a measurement marker set according to the position of the specific region in the first captured image or a second captured image obtained by capturing the subject illuminated by the illumination light; An endoscope program for executing a function of displaying an image on a display unit.

Images