JPWO2017104002A1 - Image processing apparatus, optical scanning observation system, and image processing method - Google Patents

Abstract

The image processing apparatus (2) includes a storage unit (14) for storing data in which a detection value and a detection position of observation light are associated with each other, a pixel region in a subject corresponding to a target pixel in the image, and at least one A calculation region setting unit (15) for setting a calculation region including two detection positions, a detection value acquisition unit (16) for acquiring a detection value associated with the detection position in the calculation region from the data, and A pixel value calculation unit (18) that calculates the pixel value of the target pixel based on the detection value, and an image formation unit (20) that forms an image by assigning the pixel value to the target pixel.

Description

  The present invention relates to an image processing apparatus, an optical scanning observation system, and an image processing method.

  2. Description of the Related Art Conventionally, there is known an optical scanning endoscope that spirally scans illumination light on a subject along a spiral trajectory (see, for example, Patent Document 1). The optical scanning endoscope detects observation light (for example, reflected light or fluorescence) generated in a subject by irradiation of illumination light, and the detection value of the observation light and the irradiation position of the illumination light at the time when the observation light is detected An image is formed by associating with (detection position). The array of detection positions on the subject and the array of pixels in the image are different from each other, and a plurality of detection positions can be included in a pixel area on the subject corresponding to one pixel. In Patent Document 1, an average value of detection values at a plurality of detection positions included in one pixel region is calculated as a pixel value of a corresponding pixel.

JP 2011-24624 A

  In spiral scanning in which illumination light is scanned at a constant angular velocity, the density of detection positions is reduced on the outer peripheral side of the scanning locus. When the density of the detection positions is smaller than the density of the pixel area, a pixel area that does not include the detection positions is generated. Similarly, even when shooting a scanning locus that is not parallel to the scanning line of the monitor display image or when scanning is performed at a scanning speed that is not constant, there is a possibility that a pixel area that does not include the detection position is generated as described above. In the case of Patent Document 1, the pixel value of the pixel corresponding to the pixel region not including the detection position is calculated as zero. An interpolation process is known in which the pixel value of such a defective pixel having a missing pixel value is estimated from the surrounding pixel values and interpolated.

  In an image acquired using an image sensor such as a CCD, missing pixels are sparsely generated, so that pixel values of the missing pixels can be interpolated by interpolation processing. However, the spiral scanning image has a problem that it is difficult to interpolate the pixel value of the defective pixel by the interpolation process because the number of the defective pixels is large.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide an image processing apparatus, an optical scanning observation system, and an image processing method capable of obtaining an image without missing pixels.

In order to achieve the above object, the present invention provides the following means.
According to a first aspect of the present invention, there is provided an image processing apparatus for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other, and storing the data A calculation area setting section for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image, and the detection included in the calculation area set by the calculation area setting section A detection value acquisition unit that selects a position and acquires the detection value associated with the selected detection position from the data, and the pixel of interest based on the detection value acquired by the detection value acquisition unit A pixel value calculation unit that calculates a pixel value of the image, and an image forming unit that allocates the pixel value calculated by the pixel value calculation unit to the target pixel and forms the image. Region setting unit, an image processing apparatus for setting the computational domain so as to include at least one of the detection positions in the calculated area.

  According to the present invention, the calculation region including the pixel region on the subject is set by the calculation region setting unit, and the detection value of the observation light at the detection position included in the calculation region is acquired from the data in the storage unit by the detection value acquisition unit. Then, the pixel value calculation unit calculates the pixel value of the target pixel constituting the image from the acquired detection value. A pixel value is calculated using each pixel in the image as a target pixel, and the calculated pixel value is assigned to a corresponding pixel by the image forming unit, thereby forming an image.

  In this case, the region including the pixel region is set as the calculation region by the calculation region setting unit so that at least one detection position is included in the calculation region. Therefore, the pixel of the pixel corresponding to the pixel region including the detection position Not only the value but also the pixel value of the pixel corresponding to the pixel region not including the detection position is calculated. As a result, an image without missing pixels can be obtained.

  According to a second aspect of the present invention, there is provided an image processing apparatus for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other, and storing the data A calculation area setting section for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image, and the detection included in the calculation area set by the calculation area setting section A detection value acquisition unit that selects a position and acquires the detection value associated with the selected detection position from the data, and a weighting unit that weights the detection value acquired by the detection value acquisition unit A pixel value calculation unit that calculates a pixel value of the target pixel based on the detection value weighted by the weighting unit, and an image calculated by the pixel value calculation unit. An image forming unit that assigns a value to the target pixel to form the image, and the calculation region setting unit sets the calculation region so that at least one detection position is included in the calculation region. It is a processing device.

In the second aspect, the weighting unit may give a greater weight to a detection position closer to the center of the pixel region.
By doing in this way, the detection value at the detection position closer to the center of the pixel area is reflected in the pixel value more strongly than the detection value at the detection position farther from the center of the calculation area. Thereby, the detection value of the observation light at the center position of the pixel region can be calculated as the pixel value.

In the second aspect, the pixel value calculation unit calculates the sum of the detection values weighted by the weighting unit and the sum of the weights attached to the detection values, and calculates the sum of the detection values. The pixel value of the target pixel may be calculated by dividing by the sum of the weights.
Since the number of detection positions included in the calculation area is not constant, the number of detection values used for calculating the pixel value is not constant. Therefore, the sum of the weighted detection values varies greatly depending on the number of detection values. Therefore, a normalized pixel value can be obtained by dividing the sum of the weighted detection values by the weight sum added to the detection values.

In the first and second aspects, the calculation area setting unit may set the calculation area wider than the pixel area.
In the first and second aspects, the spatial resolution of the detection position in the data may be higher than the spatial resolution of the pixel area in the subject.
By doing in this way, the calculation accuracy of a pixel value can be improved.

In the first and second aspects, the calculation region setting unit may be able to change the size of the calculation region in accordance with the number of the detection positions included in the calculation region.
In this way, when the number of pixel areas and the number of detection positions around the pixel area is small, it is possible to set a wider calculation area and improve the calculation accuracy of the pixel value. On the other hand, when the number of pixel areas and the number of detection positions around the pixel area is large, a narrower calculation area can be set to prevent a reduction in image resolution.

According to a third aspect of the present invention, an optical scanning unit that emits light toward an object while scanning illumination light;
An optical scanning observation system comprising: a light detection unit that detects observation light generated in the subject by irradiation of the illumination light and obtains a detection value of the observation light; and the image processing apparatus according to any one of the above.

  According to a fourth aspect of the present invention, there is provided an image processing method for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other, and storing the data Storage step, a calculation region setting step for setting a calculation region including a pixel region in the subject corresponding to the target pixel in the image, and the detection included in the calculation region set by the calculation region setting step A detection value acquisition step of selecting a position and acquiring the detection value associated with the selected detection position from the data, and the pixel of interest based on the detection value acquired by the detection value acquisition step A pixel value calculating step for calculating a pixel value of the pixel, and assigning the pixel value calculated by the pixel value calculating step to the target pixel And a image forming step of forming an image, in the calculation region setting step, an image processing method for setting the computational domain so as to include at least one of the detection positions in the calculated area.

  According to a fifth aspect of the present invention, there is provided an image processing method for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other, and storing the data Storage step, a calculation region setting step for setting a calculation region including a pixel region in the subject corresponding to the target pixel in the image, and the detection included in the calculation region set by the calculation region setting step A detection value acquisition step of selecting a position and acquiring the detection value associated with the selected detection position from the data, and a weighting step of weighting the detection value acquired by the detection value acquisition step And a pixel value calculation for calculating a pixel value of the target pixel based on the detection value weighted by the weighting step. And an image forming step of forming the image by assigning the pixel value calculated by the pixel value calculating step to the region of interest, and in the calculation region setting step, at least one of the detections in the calculation region In this image processing method, the calculation area is set so that a position is included.

  According to the present invention, there is an effect that an image without missing pixels can be obtained.

1 is an overall configuration diagram of an optical scanning observation system according to an embodiment of the present invention. It is a block diagram which shows the function of the image processing apparatus with which the optical scanning observation system of FIG. 1 is provided. It is a figure which shows the detection position of observation light and the data of a detection value which are memorize | stored in the frame memory for detection values. It is a figure which shows the two-dimensional data produced | generated from the data of FIG. It is a figure which shows the pixel area | region on a to-be-photographed object. It is a figure which shows the calculation area set by the calculation area setting part. It is a figure which shows an example of the detection value set acquired from data by the detection value acquisition part. It is a figure which shows an example of the weighting filter used by the weighting part. It is a figure which shows the detection value presence / absence set produced | generated from the detection value set of FIG. 3 is a flowchart illustrating an image processing method by the image processing apparatus of FIG. 2. It is a flowchart which shows the pixel value calculation routine in the flowchart of FIG.

Hereinafter, an optical scanning observation system 100 according to an embodiment of the present invention will be described with reference to the drawings.
As shown in FIG. 1, an optical scanning observation system 100 according to the present embodiment includes an optical scanning endoscope apparatus 1, an image processing apparatus 2 connected to the optical scanning endoscope apparatus 1, And a display device 3 connected to the image processing device 2.

  The optical scanning endoscope apparatus 1 includes a light scanning unit 4 that emits laser light (illumination light) L toward the subject A while scanning, and a light detection unit 5 that detects observation light L ′ from the subject A. And a control unit 6 for controlling the light scanning unit 4 and the light detection unit 5. Reference numeral 7 denotes an elongated insertion portion inserted into the body, and reference numeral 8 denotes a housing connected to the proximal end of the insertion portion 7.

The optical scanning unit 4 includes a light source 9 that is provided in the housing 8 and outputs a laser beam L, and an irradiation optical fiber 10 and an actuator 11 that are provided in the insertion unit 7.
The optical fiber 10 is disposed in the insertion portion 7 along the longitudinal direction. The optical fiber 10 guides the laser beam L supplied from the light source 9 and emits it from the tip.

  The actuator 11 is, for example, a piezoelectric actuator that includes a piezoelectric element, and is attached to the tip of the optical fiber 10. The actuator 11 vibrates the tip of the optical fiber 10 along a spiral trajectory in a substantially plane perpendicular to the longitudinal direction of the optical fiber 10 in accordance with a drive signal supplied from the control unit 6. As a result, the laser beam L emitted from the tip of the insertion portion 7 is scanned along the spiral scanning locus B on the subject A. Note that the scanning trajectory is not limited to a spiral shape, and may be another shape, for example, a raster shape or a Lissajous shape.

  The light detection unit 5 is connected to the proximal end of a light receiving optical fiber 12 disposed in the insertion unit 7. The distal end of the optical fiber 12 is disposed on the distal end surface of the insertion portion 7. Observation light L ′ (for example, reflected light of laser light L or fluorescence excited by laser light L) generated in subject A by irradiation with laser light L is received at the front end surface of optical fiber 12, and optical fiber 12. Is guided to the light detection unit 5. Although only one optical fiber 12 is illustrated in FIG. 1, a plurality of optical fibers 12 may be provided in the insertion portion 7.

  The light detection unit 5 photoelectrically converts the observation light L ′ received through the optical fiber 12 and samples the generated electrical signal in response to a sampling signal transmitted from the control unit 6 at regular time intervals. Then, a detection value indicating the intensity of the observation light L ′ is obtained by converting it into a digital value. The detection value is transmitted to the detection value frame memory 14 in the image processing apparatus 2.

  The controller 6 generates a drive signal for driving the actuator 11 and transmits the drive signal to the actuator 11. In addition, the control unit 6 generates a sampling signal at regular time intervals and transmits the sampling signal to the light detection unit 5. The control unit 6 also transmits the drive signal and the sampling signal to the detection position calculation unit 13 in the image processing apparatus 2.

  As shown in FIG. 2, the image processing apparatus 2 includes a detection position calculation unit 13 that calculates the detection position of the observation light L ′ based on the drive signal and the sampling signal, and data that associates the detection position with the detection value. Detection value frame memory (storage unit) 14, a calculation region setting unit 15 for setting a calculation region E used to calculate pixel values, and detection values corresponding to detection positions included in the calculation region E A detection value acquisition unit 16 that acquires from the data in the detection value frame memory 14, a weighting unit 17 that weights the detection values acquired from the data, and a pixel value that calculates a pixel value from the weighted detection values A calculation unit 18, a pixel value frame memory 19 for storing pixel values, and an image forming unit 20 for forming an image are provided.

  The detection position of the observation light L ′ is coordinates (X, Y) representing the irradiation position on the subject A of the laser light L at the time when the electrical signal of the observation light L ′ is sampled by the light detection unit 5. Since the vibration amplitude of the optical fiber 10 corresponds to the amplitude of the drive signal, the irradiation position of the laser light L can be calculated from the amplitude of the drive signal. The detection position calculation unit 13 receives the drive signal and the sampling signal from the control unit 6, and calculates the irradiation position of the laser light L at the output time of the sampling signal from the drive signal. Next, the detection position calculation unit 13 converts the calculated irradiation position into a two-dimensional position coordinate (X, Y), and transmits the coordinate (X, Y) to the detection value frame memory 14 as the detection position. .

  The detection value frame memory 14 receives the detection value of the observation light L ′ from the light detection unit 5, receives the detection position (X, Y) of the observation light L ′ from the detection position calculation unit 13, and receives the observation light L ′. Is detected in association with the detection position (X, Y) when the detection value is sampled. As a result, as shown in FIG. 3, data having the detection position (X, Y) as the address of each detection value is generated in the detection value frame memory 14.

  FIG. 4 is a two-dimensional image of the data. As shown in FIG. 4, the data is two-dimensional data composed of sections C arranged in the X direction (horizontal direction) and the Y direction (vertical direction), and each coordinate corresponds to one section C. In FIG. 4, a section C having a detection value is represented in black, and a section C having no detection value is represented in white.

  The calculation area setting unit 15 sets one of the pixels of the image as a target pixel and sets the calculation area E including the pixel area D. As shown in FIG. 5, the pixel area D is an area on the subject A corresponding to the target pixel. In FIG. 5, black circles indicate the illumination light irradiation positions at the detection value sampling time. The size of the calculation region E is set larger than the size of the pixel region D as shown in FIG. The size of the calculation area E is set so that at least one detection position is included in the calculation area E based on the minimum density of detection positions in the scanning trajectory B. The density of the detection positions in the scanning locus B varies depending on the position in the scanning locus B as will be described later, and can be obtained by calculation from the design value of the optical scanning endoscope apparatus 1.

Here, the detection position calculation unit 13 calculates the detection position (X, Y) so that the spatial resolution of the coordinates (X, Y) as the detection position is higher than the spatial resolution of the pixel region D. Therefore, a plurality of sections C are included in one pixel region D. In the example shown in FIG. 6, the spatial resolution of the coordinates (X, Y) is three times the spatial resolution of the pixel area D, and a 3 × 3 section C is included in one pixel area D.
The calculation area setting unit 15 sets an area including a predetermined number of sections C around the section C located at the center in the pixel area D as the calculation area E. In the example shown in FIG. 6, the calculation area E is composed of 7 × 7 sections C.

The detection value acquisition unit 16 selects the detection position (X, Y) included in the calculation region E set by the calculation region setting unit 15 from the data in the detection value frame memory 14, and the selected detection is performed. Detection value sets I ₀ , I ₁ ,..., I ₁₀ which are detection values corresponding to the position (X, Y) are acquired from the data. FIG. 7 shows an example in which the acquired detection value sets I ₀ , I ₁ ,..., I ₁₀ are converted into a two-dimensional image. The detection value acquisition unit 16 transmits the acquired detection value set to the weighting unit 17 and the pixel value calculation unit 18.

The weighting unit 17 has a weighting filter shown in FIG. The weighting filter is composed of weighting factors W _ij (i = 0, 1, 2,..., N, j = 0, 1, 2,..., M) having the same arrangement as that of the section C in the calculation area E. . In the example shown in FIG. 8, n = m = 6. The weighting unit 17 performs a convolution operation on the detected values I ₀ , I ₁ ,..., I ₁₀ using a weighting filter. The convolution operation is expressed by the following equation. In the following expression, I _ij represents a detection value at a position (i, j) in the calculation area E. For example, in the example shown in FIG. 7, I ₀ is represented as I ₀₇ .

Here, in the weighting filter, the weighting coefficients W _ij from the center toward the outside so as to be smaller in order, the weight coefficient W _ij is set. Therefore, a greater weight is assigned to the detection value closer to the center of the pixel region D.

By the convolution operation, the sum Si of the detected values I ₀ , I ₁ ,..., I ₁₀ to which the weights W _ij are attached is calculated. That is, in the example of FIG. 7 and FIG.
Si = I ₀ × W ₀₆ + I ₁ × W ₁₂ + I ₂ × W ₁₅ + I ₃ × W ₂₁ + I ₄ × W ₂₄ + I ₅ × W ₄₂
+ I ₆ × W ₄₅ + I ₇ × W ₅₁ + I ₈ × W ₅₄ + I ₉ × W ₆₀ + I ₁₀ × W ₆₆
It is. The weighting unit 17 transmits the calculated value of the sum Si to the pixel value calculating unit 18.

  The pixel value calculation unit 18 receives the detection position (X, Y) included in the calculation region E from the detection value acquisition unit 16. Next, the pixel value calculation unit 18 generates a detection value presence / absence set shown in FIG. 9 based on the received detection position (X, Y). The detection value presence / absence set includes values of “0” and “1” having the same arrangement as that of the section C in the calculation area E. In the detection value presence / absence set, the pixel value calculation unit 18 assigns “1” to a position corresponding to the detection position (X, Y) and assigns “0” to other positions.

Next, the pixel value calculation unit 18 performs a convolution operation on the detection value presence / absence set using a weighting filter. The weighting filter used in the convolution calculation here is the same as the weighting filter used in the weighting unit 17. By the convolution operation, the sum Sw of the same weight coefficients as the weight coefficients assigned to the detected values I ₀ , I ₁ ,..., I ₁₀ is calculated. That is, in the example of FIG. 8 and FIG.
Sw = W ₀₆ + W ₁₂ + W ₁₅ + W ₂₁ + W ₂₄ + W ₄₂ + W ₄₅ + W ₅₁ + W ₅₄ + W ₆₀ + W ₆₆
It is.

  Next, the pixel value calculation unit 18 calculates the pixel value Si / Sw of the target pixel by dividing the sum Si received from the weighting unit 17 by the sum Sw. The pixel value calculation unit 18 transmits the calculated pixel value to the pixel value frame memory 19.

  The pixel value frame memory 19 generates image data by storing the pixel value received from the pixel value calculation unit 18 in association with the number N of the target pixel. Here, the calculation area setting unit 15 sets all the pixels in the image as the target pixel in order. As a result, the above-described processing by the detection value acquisition unit 16, the weighting unit 17, and the pixel value calculation unit 18 is executed for all the pixels, and the pixel values of all the pixels are stored in the pixel value frame memory 19. The

  The image forming unit 20 reads the image data from the pixel value frame memory 19 and forms an image by assigning a pixel value corresponding to each pixel based on the image data. The image forming unit 20 transmits the formed image to the display device 3 and causes the display device 3 to display the image.

The image processing apparatus 2 includes, for example, a general purpose or dedicated computer including a central processing unit (CPU), a main storage device such as a RAM, and an auxiliary storage device. The auxiliary storage device is a non-temporary storage medium such as a hard disk or various memories, and stores an image processing program for causing the CPU to execute the above-described processing. When the image processing program is loaded from the auxiliary storage device to the main storage device and activated, the CPU performs detection position calculation unit 13, calculation region setting unit 15, detection value acquisition unit 16, weighting unit according to the image processing program. 17, the above-described processing of the pixel value calculation unit 18 and the image forming unit 20 is executed.
Alternatively, the image processing apparatus 2 may be configured using an integrated circuit such as a field programmable gate array (FPGA) instead of the CPU.

Next, the operation of the optical scanning observation system 100 configured as described above will be described.
In the optical scanning observation system 100 according to the present embodiment, when transmission of a drive signal and a sampling signal from the control unit 6 to the actuator 11 and the light detection unit 5 is started, the laser light L output from the light source 9 is The object A is irradiated through the optical fiber 10 and scanned along the spiral scanning locus B on the object A. The observation light L ′ generated at the irradiation position of the laser light L is received by the optical fiber 12, detected by the light detection unit 5, and the detection value of the observation light L ′ is transmitted to the image processing apparatus 2. On the other hand, the drive signal and the sampling signal are also transmitted from the control unit 6 to the image processing apparatus 2.

10 and 11 show an image processing process executed in the image processing apparatus 2.
In the image processing apparatus 2, first, as shown in FIG. 10, data in which the detection value of the observation light L ′ is associated with the detection position is generated and stored in the detection frame memory 14 (storage step S1). ). That is, the detection position calculation unit 13 receives the drive signal and the sampling signal from the control unit 6, calculates the irradiation position of the laser light L at the sampling time of the detection value from the drive signal, and further expresses the coordinates (X , Y) is calculated as the detection position, and the coordinates (X, Y) are stored in the detection frame memory 14 in association with the detection value from the light detection unit 5.

  Next, the pixel value of each pixel is calculated using the data in the detection frame memory 14 (step S2). That is, as shown in FIG. 11, first, the calculation region setting unit 15 sets a target pixel (step S21), and a calculation region E that includes the pixel region D corresponding to the target pixel and is wider than the pixel region D. It is set (calculation area setting step S22).

Next, the detection value acquisition unit 16 selects a detection position included in the calculation region E (step S23), and detection value sets I ₀ , I ₁ ,..., I ₁₀ corresponding to the selected detection position are detected. The data is read from the data in the frame memory 14 (detection value acquisition step S24).
Next, in the weighting unit 17, the read detection value sets I ₀ , I ₁ ,..., I ₁₀ are subjected to a convolution operation using a weighting filter (weighting step S25). Thereby, a greater weight is given to the detection value at the detection position closer to the center of the pixel region D, and the sum Si of the weighted detection values is calculated.

Next, the pixel value calculation unit 18 generates a detection value presence / absence set from the read detection value sets I ₀ , I ₁ ,..., I ₁₀ (step S26), and the detection value presence / absence set is used in step S25. The convolution operation is performed using the same weighting filter as the weighting filter (step S27). Thereby, the sum Sw of the weights attached to the detected values is calculated. Next, the pixel value Si / Sw of the target pixel is calculated by dividing the sum Si by the sum Sw (pixel value calculation step S28). The calculated pixel value is stored in the pixel value frame memory 19 in association with the number of the target pixel (step S29).

  Next, another pixel is set as the target pixel, and steps S21 to S29 are repeated until all the pixels in the image are set as the target pixel (NO in step S30, step S31). When the pixel values for all the pixels are stored in the pixel value frame memory 19 (YES in step S30), the image forming unit 20 reads the image data from the pixel value frame memory 19, and the number in the image data An image is formed by assigning the pixel value to the corresponding pixel based on (Image forming step S3). The formed image is displayed on the display device 3.

  In this case, in the optical scanning endoscope apparatus 1 that spirally scans the laser light L, the laser light L is scanned at a constant angular velocity by the optical scanning unit 4, and the observation light L ′ is scanned by the optical detection unit 5 for a certain period of time. Detected at intervals. Accordingly, in the scanning locus B, the density of the detection positions is increased on the center side, and the density of the detection positions is decreased on the outer peripheral side. As a result, a pixel region D that does not include the detection position may occur on the outer peripheral side of the scanning locus B.

  According to the present embodiment, the calculation area E is set so as to include at least one detection position, and the pixel value of the corresponding pixel is calculated from the detection value of the detection position included in the calculation area E. Accordingly, there is an advantage that an image in which all pixels have a pixel value can be formed without a defective pixel having no pixel value.

  Furthermore, in the calculation area E, the detection position is not necessarily located at the center of the pixel area D, and the distribution of detection positions may be biased in the pixel area D. In order to obtain an accurate pixel value of each pixel, it is necessary to calculate the detection value of the observation light L ′ at the center of the pixel region D in consideration of the distribution of detection values in the calculation region E. When the average of the detection values of the detection positions included in the calculation area E is calculated as the pixel value, a value corresponding to the detection value of the observation light L ′ at a position deviating from the center of the pixel area D is calculated. When the pixel value is used for forming an image, the image quality is deteriorated, for example, moire occurs in the image.

  According to the present embodiment, in the calculation of the pixel value, a greater weight is given to the detection value at the detection position close to the center of the pixel region D. Thereby, there is an advantage that an accurate pixel value of each pixel can be calculated and a high-quality image can be obtained.

Furthermore, as shown in FIG. 7 and FIG. 8, there is no corresponding detection value for some weighting factors W _ij . When the weighting filter is applied to the detection value set in which the detection value does not partially exist in this way, the convolution calculation result Si greatly varies depending on the number of detection values in the detection value set. According to the present embodiment, the convolution calculation result Si is normalized by being divided by the convolution calculation result Sw of the detection value presence / absence set and the weighting filter. Thereby, there exists an advantage that weighting of a detected value can be performed appropriately using a weighting filter.

In this embodiment, the weighting unit 17 weights the detection value, and then the pixel value calculation unit 18 standardizes the pixel value. Instead, the detection value is weighted. Instead, the pixel value calculation unit 18 may calculate an addition average of the detection value sets I ₀ , I ₁ ,..., I ₁₀ as a pixel value. This is synonymous with setting all weighting factors W _ij to 1 in the weighting unit 17 and the pixel value calculation unit 18.
Even in this case, it is possible to eliminate variation in the calculation result of the pixel value with respect to the detection value set in which the detection value does not partially exist.

In the present embodiment, the calculation area setting unit 15 sets the calculation area E having a fixed size, but instead, the dimension of the calculation area E can be changed according to the number of detection positions. May be.
For example, the calculation area setting unit 15 first sets a calculation area E having a small dimension (for example, a dimension equal to the pixel area D), and the number of detection positions included in the calculation area E is equal to or greater than a predetermined number. In this case, this small calculation area E is applied. On the other hand, when the number of detection positions included in the calculation area E having a small size is less than the predetermined number, the calculation area E until the number of detection positions included in the calculation area E becomes equal to or greater than the predetermined number. Gradually increase the dimensions of

  The larger the size of the calculation area E, the lower the resolution of the image. Therefore, by reducing the size of the calculation region E to the minimum size that can prevent the occurrence of defective pixels, it is possible to prevent a reduction in the resolution of the image.

DESCRIPTION OF SYMBOLS 1 Optical scanning type endoscope apparatus 2 Image processing apparatus 3 Display apparatus 4 Optical scanning part 5 Optical detection part 6 Control part 7 Insertion part 8 Case 9 Light source 10, 12 Optical fiber 11 Actuator 13 Detection position calculation part 14 For detection values Frame memory (storage unit)
DESCRIPTION OF SYMBOLS 15 Computation area setting part 16 Detection value acquisition part 17 Weighting part 18 Pixel value calculation part 19 Pixel value frame memory 20 Image formation part 100 Optical scanning type observation system A Subject B Scanning trajectory C Section D Pixel area E Calculation area L Laser light (Illumination light)
L 'Observation light

Claims (10)

An image processing apparatus for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other,
A storage unit for storing the data;
A calculation area setting unit for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image;
A detection value acquisition unit that selects the detection position included in the calculation region set by the calculation region setting unit, and acquires the detection value associated with the selected detection position from the data;
A pixel value calculation unit that calculates a pixel value of the target pixel based on the detection value acquired by the detection value acquisition unit;
An image forming unit that forms the image by allocating the pixel value calculated by the pixel value calculating unit to the target pixel;
The image processing apparatus, wherein the calculation area setting unit sets the calculation area so that at least one detection position is included in the calculation area. An image processing apparatus for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other,
A storage unit for storing the data;
A calculation area setting unit for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image;
A detection value acquisition unit that selects the detection position included in the calculation region set by the calculation region setting unit and acquires the detection value associated with the selected detection position from the data;
A weighting unit for weighting the detection value acquired by the detection value acquisition unit;
A pixel value calculation unit that calculates a pixel value of the target pixel based on the detection value weighted by the weighting unit;
An image forming unit that forms the image by allocating the pixel value calculated by the pixel value calculating unit to the target pixel;
The image processing apparatus, wherein the calculation area setting unit sets the calculation area so that at least one detection position is included in the calculation area.   The image processing apparatus according to claim 2, wherein the weighting unit assigns a greater weight to a detection position closer to the center of the pixel region.   The pixel value calculation unit calculates a sum of the detection values weighted by the weighting unit and a sum of weights attached to the detection values, and divides the sum of the detection values by the sum of the weights. The image processing apparatus according to claim 2, wherein a pixel value of the target pixel is calculated by the following.   The image processing apparatus according to claim 1, wherein the calculation area setting unit sets the calculation area wider than the pixel area.   The image processing apparatus according to claim 1, wherein a spatial resolution of the detection position in the data is higher than a spatial resolution of the pixel area in the subject.   The image processing apparatus according to claim 1, wherein the calculation area setting unit is capable of changing a dimension of the calculation area according to the number of the detection positions included in the calculation area. A light scanning unit that emits light toward the subject while scanning illumination light;
A light detection unit that detects observation light generated in the subject by irradiation of the illumination light and obtains a detection value of the observation light;
An optical scanning observation system comprising the image processing apparatus according to claim 1. An image processing method for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other,
A storage step for storing the data;
A calculation area setting step for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image;
A detection value acquisition step of selecting the detection position included in the calculation area set by the calculation area setting step, and acquiring the detection value associated with the selected detection position from the data;
A pixel value calculation step for calculating a pixel value of the target pixel based on the detection value acquired by the detection value acquisition step;
An image forming step of forming the image by assigning the pixel value calculated by the pixel value calculating step to the target pixel,
An image processing method for setting the calculation area so that at least one of the detection positions is included in the calculation area in the calculation area setting step. An image processing method for forming an image of a subject from data in which a detection value and a detection position of observation light generated in the subject by illumination light irradiation are associated with each other,
A storage step for storing the data;
A calculation area setting step for setting a calculation area including a pixel area in the subject corresponding to the target pixel in the image;
A detection value acquisition step of selecting the detection position included in the calculation area set by the calculation area setting step, and acquiring the detection value associated with the selected detection position from the data;
A weighting step of weighting the detection value acquired by the detection value acquisition step;
A pixel value calculating step for calculating a pixel value of the target pixel based on the detection value weighted by the weighting step;
An image forming step of forming the image by assigning the pixel value calculated by the pixel value calculating step to the region of interest,
An image processing method for setting the calculation area so that at least one of the detection positions is included in the calculation area in the calculation area setting step.

Images