JP6352673B2 - Endoscope apparatus and operation method of endoscope apparatus - Google Patents

Description

The present invention relates to an endoscope apparatus and an operation method of the endoscope apparatus, and more particularly to a technique for acquiring a high-quality endoscope image having a desired viewing angle.

  Conventionally, medical diagnosis using an endoscope has been performed in the medical field. For example, an electronic endoscope called an electronic scope is provided with an insertion portion that is inserted into a patient's body. An observation optical system and a solid-state imaging device that captures an optical image formed by the observation optical system are provided in the internal space of the insertion portion distal end, which is the distal end portion of the insertion portion.

  In the case of an endoscope for observing the shape of a lumen, if there are folds or projections in the lumen, it is difficult to observe the portion hidden by them even if the tip of the endoscope is curved. Therefore, the field of view of the endoscope's observation optical system is widened to widen the observation range, but the wide-angle observation optical system is prone to distortion and the image around the field of view is crushed. There is a problem of being visible.

  On the other hand, the endoscope image device described in Patent Document 1 sets the electronic magnification of the image in the radial direction and the concentric direction when moving and converting the position of the optical image formed on the imaging element. By operating independently, the distortion is corrected so that the peripheral portion can be seen better than the central portion.

  On the other hand, since the solid-state image sensor is attached and fixed to the solid-state image sensor frame in the distal end of the insertion section or is fixed with a sealing resin in the shield frame, the mounting position accuracy of the solid-state image sensor is low There is. In addition, the mounting position accuracy of the observation optical system in the distal end portion of the insertion portion may be lowered as in the case of the solid-state imaging device.

Japanese Patent No. 5198138

  By the way, when the viewing optical system is widened, a difference in viewing angle between the upper and lower ends or the left and right ends of the photographing screen is greatly caused by a slight tilt of the optical axis of the observation optical system. In this case, as in the invention described in Patent Document 1, even if distortion correction is performed, the difference in viewing angle cannot be corrected, and the periphery of the screen cannot be viewed optimally. Note that the appearance around the screen is important in an endoscope that observes the shape of the lumen.

The present invention has been made in view of such circumstances, and the viewing angle range of the display image is not affected when the mounting accuracy between the wide-angle observation optical system and the solid-state imaging device is low or the optical characteristics of the observation optical system are not uniform. It is an object of the present invention to provide an endoscope apparatus and an operation method of the endoscope apparatus that can make a desired viewing angle range.

  To achieve the above object, an endoscope apparatus according to one aspect of the present invention is positioned and fixed relative to an observation optical system and an observation optical system, and an optical image is formed by the observation optical system. A solid-state imaging device that corresponds to a display area that is displayed on a monitor from a solid-state imaging device in which a plurality of photoelectric conversion elements that photoelectrically convert an optical image are arrayed and an image that corresponds to a displayable pixel area of the solid-state imaging device A storage unit that stores image cutout information for specifying an image cutout region for cutting out an image, and the storage unit includes an angular difference between the upper and lower visual fields corresponding to the display area, and an angle of the left and right visual fields. Image cutout information in which the difference is within an allowable angle is stored.

  According to one aspect of the present invention, the image corresponding to the display area is cut out from the image corresponding to the displayable pixel area of the solid-state imaging device based on the image cut-out information stored in the storage unit. The angle difference between the visual fields at the upper and lower ends of the corresponding image and the angular difference between the visual fields at the left and right ends can be set within allowable angles. This makes it possible to display solid-state image sensors due to errors in the relative mounting position between the observation optical system and the solid-state image sensor, such as slight tilting of the optical axis of the observation optical system, and variations in the optical characteristics of the observation optical system. Even if the angle difference between the visual fields at the upper and lower ends of the image corresponding to the pixel area or the angular difference between the visual fields at the left and right ends is large, the image corresponding to the display area can be within the desired viewing angle range. Therefore, the viewing angles of the images having the same image height can be matched, or the viewing angle can be extremely increased, so that an image of a portion having a large distortion can be prevented from being displayed.

  In the endoscope apparatus according to another aspect of the present invention, the image cutout information stored in the storage unit is a cutout region having the same aspect ratio as the display area, and is cut out from an image corresponding to the displayable pixel area. It is preferable that the information indicating the cut-out area closest to the image size of the display area among the images in which the angle difference between the upper and lower visual fields and the angle difference between the left and right visual fields is within an allowable angle. .

  In other words, the image cutout information is a cutout area having the same aspect ratio as the display area, and the angle difference between the upper and lower visual fields of the image cut out by the cutout area and the angle difference between the left and right visual fields are within an allowable angle. It is the information which shows the cutout area which becomes. Furthermore, the image cutout information is preferably information indicating a cutout area having the same size as the image size of the display area. However, in order to satisfy the above-described angle difference condition of the visual field, the same size as the image size of the display area. If it is not possible to secure the cutout area, it is preferable to set the largest image size (that is, the cutout area closest to the image size of the display area). This is because the pixels in the displayable pixel area of the solid-state imaging device are effectively used.

In the endoscope apparatus according to still another aspect of the present invention, the storage unit sets the image size in the vertical direction and the horizontal direction of the image size of the display area to 2L _V and 2L _H, and sets the image size from the center of the image cut-out area. the top and the angle of the field of view of the lower end of the L _V and theta _U, and theta _D, the center left and right angles of the field of view of the image size L _H from the cutout region of an image and theta _L, and theta _R, the upper and lower ends of the field When the allowable angles of the angle difference | θ _U −θ _D | and the angle difference between the left and right visual fields | θ _L −θ _R | are Δθ _V and Δθ _H , respectively,
| Θ _U −θ _D | ≦ Δθ _V
| Θ _L −θ _R | ≦ Δθ _H (1)
It is preferable to store a cutout region of an image satisfying the above as image cutout information.

Thereby, the angular difference between the field of view of the angular difference and the left and right ends of the upper and lower ends of the field of view of the image of the cut-out region (region of 2L _V × 2L _H corresponding to the display area) may be within the acceptance angle.

In the endoscope apparatus according to still another aspect of the present invention, the storage unit stores the image sizes L _V and L from the center of the image cutout area when there is no image cutout area satisfying the expression (1). _It is preferable that _H is reduced at the same aspect ratio as that of the display area, and an image cut-out area when the expression (1) is satisfied by the reduced image sizes A _V and A _H is stored as image cut-out information.

That is, when the image clipping region is moved within the displayable pixel area of the solid-state imaging device so as to satisfy the expression (1), the image clipping region exceeds the displayable pixel area because vignetting occurs. Cannot be moved. In this case, the image sizes L _V and L _H of the image cut-out area are reduced with the same aspect ratio as that of the display area, thereby satisfying the expression (1).

  An endoscope apparatus according to still another aspect of the present invention is an observation optical system and a solid-state imaging device that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system. An image for cutting out an image corresponding to a display area to be displayed on a monitor from a solid-state image sensor in which a plurality of photoelectric conversion elements for photoelectrically converting an optical image are arranged and an image corresponding to a displayable pixel area of the solid-state image sensor A storage unit that stores image cutout information for specifying a cutout region, and the storage unit is an image in which the angles of the visual fields at the upper end, the lower end, the left end, and the right end of the image corresponding to the display area are each within an allowable angle. The cutout information is stored.

  According to still another aspect of the present invention, the image corresponding to the display area is cut out from the image corresponding to the displayable pixel area of the solid-state imaging device based on the image cut-out information stored in the storage unit. The angle of the visual field at the upper end, the lower end, the left end, and the right end of the image corresponding to can be set within an allowable angle. This makes it possible to display solid-state image sensors due to errors in the relative mounting position between the observation optical system and the solid-state image sensor, such as slight tilting of the optical axis of the observation optical system, and variations in the optical characteristics of the observation optical system. Even if the viewing angles at the upper end, lower end, left end, and right end of the image corresponding to the pixel area are large, the image corresponding to the display area can be within a desired viewing angle range. Therefore, the viewing angles of the images having the same image height can be matched, or the viewing angle can be extremely increased, so that an image of a portion having a large distortion can be prevented from being displayed.

  In the endoscope apparatus according to still another aspect of the present invention, the image cutout information stored in the storage unit is a cutout area having the same aspect ratio as the display area, and is cut out from an image corresponding to the displayable pixel area. It is preferable that the information indicates the cut-out area closest to the image size of the display area among the images in which the angles of the visual fields at the upper end, the lower end, the left end, and the right end of the image cut out by the area are within the allowable angle.

  That is, the image cutout information is a cutout area having the same aspect ratio as that of the display area, and a cutout area in which the upper, lower, left, and right field of view of the image cut out by the cutout area is within an allowable angle. It is information to show. Furthermore, the image cutout information is preferably information indicating a cutout area having the same size as the image size of the display area. However, in order to satisfy the above-described angle difference condition of the visual field, the same size as the image size of the display area. If it is not possible to secure the cutout area, it is preferable to set the largest image size (that is, the cutout area closest to the image size of the display area). This is because the pixels in the displayable pixel area of the solid-state imaging device are effectively used.

In the endoscope apparatus according to still another aspect of the present invention, the storage unit sets the image size in the vertical direction and the horizontal direction of the image size of the display area to 2L _V and 2L _H, and sets the image size from the center of the image cut-out area. the top and the angle of the field of view of the lower end of the L _V and theta _U, and theta _D, the center left and right angles of the field of view of the image size L _H from the cutout region of an image and θ _L, θ _R, the angle theta _U, theta _When the allowable angle ranges of _{D 1} , θ _L , and θ _R are Θ _U ± ΔU, Θ _D ± ΔD, Θ _L ± ΔL, and Θ _R ± ΔR, respectively,
| Θ _U −θ _U | ≦ ΔU
| Θ _D −θ _D | ≦ ΔD
| Θ _L −θ _L | ≦ ΔL
| Θ _R −θ _R | ≦ ΔR (2)
It is preferable to store a cutout region of an image satisfying the above as image cutout information.

Thus, the upper end of the image of the cut-out region (region of 2L _V × 2L _H that corresponds to the display area), the lower end, left end, and right end of the field of view angle _{(θ U, θ D, θ} L, θ R) of each It can be within an allowable angle.

In the endoscope apparatus according to still another aspect of the present invention, the storage unit stores the image sizes L _V and L from the center of the image cutout area when there is no image cutout area satisfying the expression (2). _It is preferable that _H is reduced at the same aspect ratio as that of the display area, and an image cut-out area when the expression (2) is satisfied by the reduced image sizes A _V and A _H is stored as image cut-out information.

That is, when the image cutout area is moved within the displayable pixel area of the solid-state imaging device so as to satisfy the expression (2), the image cutout area exceeds the displayable pixel area because vignetting occurs. Cannot be moved. In this case, the image sizes L _V and L _H of the image cut-out area are reduced with the same aspect ratio as that of the display area, thereby satisfying the expression (2).

In the endoscope apparatus according to still another aspect of the present invention, the storage unit is an electronic enlargement ratio calculated by the image size L _V or L _H and the reduced image size _AV or A _H , electronic magnification by dividing the image size L _V in image size a _V, or preferably stores electronic magnification obtained by dividing the image size L _H in image size a _H. The image size corresponding to the display area can be obtained by electronically enlarging the image cut out at the electronic enlargement ratio.

  In the endoscope apparatus according to still another aspect of the present invention, it is preferable that the electronic magnification rate stored by the storage unit is 2 or less. Since the resolution (image quality) decreases when the electronic enlargement ratio increases, the electronic enlargement ratio is limited so as not to exceed twice. In addition, when the error of the relative attachment position of an observation optical system and a solid-state image sensor is large, and an electronic magnification rate exceeds 2 times, it can be considered as a defect product.

  In the endoscope apparatus according to still another aspect of the present invention, when the image size of the image cutout area is smaller than the image size of the display area, the storage unit sets the image size of the image cutout area to the image of the display area. It is preferable to further memorize the electron enlargement ratio that expands to the size. This eliminates the need to calculate the electronic enlargement ratio based on the size of the image size of the cutout area of the image, and displays the image by electronically enlarging the image of the cutout area according to the stored electronic enlargement ratio. The image size of the area can be made constant.

  In the endoscope apparatus according to still another aspect of the present invention, the image cut-out information stored in the storage unit is two pieces of coordinate information of an image cut out from the displayable pixel area, or an image cut out from the displayable pixel area. Is one piece of image information and image size information at the center or four corners.

  The image cut out from the displayable pixel area can be specified by the image cut-out information.

  In an endoscope apparatus according to still another aspect of the present invention, an image input unit that inputs an image corresponding to an effective pixel area of a solid-state image sensor from a solid-state image sensor, and an effective pixel area input by the image input unit A cutout unit that cuts out an image corresponding to the display area from the image based on the image cutout information stored in the storage unit, and a cutout when the image size of the image cut out by the cutout unit is smaller than the image size of the display area An electronic enlargement processing unit that enlarges the image size of the image to the image size of the display area.

  According to still another aspect of the present invention, an image corresponding to the display area is cut out from an image corresponding to the effective pixel area input by the image input unit based on the image cut-out information stored in the storage unit. When the image size of the image is smaller than the image size of the display area, the image size of the clipped image is electronically enlarged to the image size of the display area. Thereby, the viewing angle range of the display image can be set to a desired viewing angle range while ensuring the image size of the display image.

  In the endoscope apparatus according to still another aspect of the present invention, it is preferable that the endoscope apparatus further includes a mask processing unit that masks the image cut out by the cutout unit in the same shape as the display area. As a result, when the image in the display area is other than a rectangular shape (for example, in the case of an image with four corners cut or a circular image), the cut-out image is masked to match the shape of the image in the display area. be able to.

Furthermore the invention according to another aspect of the present invention, an observation optical system, are relatively positioned and fixed with respect to the observation optical system, a solid-state imaging device of which optical image is formed by the observation optical system, an optical image An imaging unit including an imaging unit having a solid-state imaging device in which a plurality of photoelectric conversion elements for photoelectric conversion are arranged , and an image processing unit that inputs an image corresponding to an effective pixel area of the solid-state imaging device from the solid-state imaging device a method of operating a mirror device, the imaging unit includes the steps of imaging the chart with an index indicating the angle of view field, the image corresponding to the displayable pixel area of the solid-state imaging device, a display area to be displayed on the monitor A step of determining image cutout information for specifying an image cutout area for cutting out a corresponding image, and storing the determined image cutout information in a storage unit. The step of determining the information is based on the image acquired in the step of imaging the chart, and the angle difference between the upper and lower visual fields and the angular difference between the left and right visual fields corresponding to the display area are within an allowable angle. The image cutout information to be determined is determined.

Furthermore the invention according to another aspect of the present invention, an observation optical system, are relatively positioned and fixed with respect to the observation optical system, a solid-state imaging device of which optical image is formed by the observation optical system, an optical image An imaging unit including an imaging unit having a solid-state imaging device in which a plurality of photoelectric conversion elements for photoelectric conversion are arranged , and an image processing unit that inputs an image corresponding to an effective pixel area of the solid-state imaging device from the solid-state imaging device a method of operating a mirror device, the imaging unit includes the steps of imaging the chart with an index indicating the angle of view field, the image corresponding to the displayable pixel area of the solid-state imaging device, a display area to be displayed on the monitor A step of determining image cutout information for specifying an image cutout area for cutting out a corresponding image, and storing the determined image cutout information in a storage unit. The step of determining the information is based on the image acquired by the step of imaging the chart, and the image cutout information is such that the angles of the visual field at the upper end, the lower end, the left end, and the right end of the image corresponding to the display area are within the allowable angle. To decide.

In yet an operation method of the endoscope apparatus according to another aspect of the present invention, the imaging unit includes the steps of imaging the observation target, an image corresponding from the solid-state image sensor in the effective pixel area of the solid-state imaging device, image processing A step of inputting by the image processing unit, and a step of cutting out the image corresponding to the display area from the image corresponding to the input effective pixel area by the image clipping unit of the image processing unit based on the image clipping information stored in the storage unit. , Including. As a result, when the image corresponding to the display area displayed on the monitor is cut out from the image corresponding to the displayable pixel area of the solid-state imaging device, the viewing angle range of the display image to be cut out is set to the desired viewing angle range. Can do.

In the operation method of the endoscope apparatus according to still another aspect of the present invention, when the image size of the image cut out by the step of cutting out the image is smaller than the image size of the display area, the electronic enlargement processing unit of the image processing unit preferably further comprising the step of electronically magnify the image size of the clipped image to the image size of the display area. Thereby, a display image having the same size as the image size of the display area can be acquired.

  According to the present invention, when the mounting accuracy between the observation optical system and the solid-state imaging device is low or the optical characteristics of the observation optical system are not varied, the angle difference between the upper and lower visual fields of the display image, or the right and left visual fields. The angle difference or the angle of the visual field on the top, bottom, left, and right of the display image can be kept within the allowable angle, the display angle range of the display image can be set to the desired viewing angle range, and the desired image without vignetting A size image can be obtained.

1 is a perspective view of an endoscope apparatus according to the present invention. It is a block diagram which shows the electric constitution of an endoscope apparatus. It is a figure for demonstrating the positional relationship with the effective pixel area of a solid-state image sensor, the displayable pixel area, the image formation area where an optical image is imaged with an observation optical system, and the display area displayed on a monitor. It is a figure which shows the case where the image corresponding to a display area is cut out on the basis of the center of an imaging area in the state where the center of the displayable pixel area and the center of an imaging area have shifted | deviated. It is explanatory drawing for demonstrating an image cut-out area | region and image cut-out information. It is explanatory drawing for demonstrating an electronic expansion rate. FIG. 3 is a functional block diagram of an image processing unit illustrated in FIG. 2. It is explanatory drawing for demonstrating the mask process by a mask process part. It is a flowchart which shows the flow of the display process of the observation image containing the image process which concerns on the operating method of the endoscope apparatus which concerns on this invention. It is a figure for demonstrating the image processing which concerns on the operating method of the endoscope apparatus which concerns on this invention. It is a flowchart which shows the flow of the memory | storage process of the image cut-out information used by the image processing which concerns on the operating method of the endoscope apparatus which concerns on this invention, and an electronic magnification. It is a figure which shows an example of a test chart. It is a figure for demonstrating the image processing which concerns on the operating method of the endoscope apparatus which concerns on this invention, and is a figure regarding the case of the display area of other embodiment. It is the schematic of the capsule system used for the endoscope apparatus of this invention. It is the schematic of the capsule system of other embodiment.

Hereinafter, preferred embodiments of an endoscope apparatus and an operation method of the endoscope apparatus according to the present invention will be described with reference to the accompanying drawings.

[Entire configuration of endoscope apparatus]
FIG. 1 is an external perspective view of an endoscope apparatus (also referred to as an endoscope system) 10 according to the present invention. The endoscope apparatus 10 is roughly divided and includes an electronic endoscope 11, a light source device 12, a processor device 13, a monitor 14, and the like as a scope (here, a flexible endoscope) that images an observation target in a patient. Yes.

  The light source device 12 supplies illumination light for illuminating the observation target to the electronic endoscope 11. The processor device 13 corresponds to an embodiment of the image processing device of the present invention, and image data (hereinafter referred to as display data) of a display image displayed on the monitor 14 based on the image signal obtained by the electronic endoscope 11. Image data) and output to the monitor 14. The monitor 14 displays an observation image to be observed based on the image data input from the processor device 13.

  The electronic endoscope 11 is connected to a flexible insertion portion 16 to be inserted into a patient's body and a proximal end portion of the insertion portion 16, and is used for gripping the electronic endoscope 11 and operating the insertion portion 16. An operation unit 17 and a universal cord 18 that connects the operation unit 17 to the light source device 12 and the processor device 13 are provided.

  The insertion portion distal end portion 16a that is the distal end portion of the insertion portion 16 incorporates an observation optical system, a solid-state imaging device, and the like that are used for illumination of an observation target and photographing. A bendable bending portion 16b is connected to the rear end of the insertion portion distal end portion 16a. A flexible tube portion 16c having flexibility is connected to the rear end of the curved portion 16b.

  The operation unit 17 is provided with an angle knob 21, an operation button 22, a forceps inlet 23, and the like. The angle knob 21 is rotated when adjusting the bending direction and the bending amount of the bending portion 16b. The operation button 22 is used for various operations such as air / water supply and suction. The forceps inlet 23 communicates with a forceps channel in the insertion portion 16.

  The universal cord 18 incorporates an air / water channel, a signal cable, a light guide, and the like. The distal end portion of the universal cord 18 is provided with a connector portion 25a connected to the light source device 12 and a connector portion 25b connected to the processor device 13. Thereby, illumination light is supplied from the light source device 12 to the electronic endoscope 11 via the connector portion 25a, and an image signal obtained by the electronic endoscope 11 via the connector portion 25b is converted into the processor device 13. Is input.

[Electric configuration of endoscope apparatus]
FIG. 2 is a block diagram showing an electrical configuration of the endoscope apparatus 10. As shown in FIG. 2, the light source device 12 includes a light source central processing unit (light source CPU) 31, a light source 32, a light source driver 33, a diaphragm mechanism 34, and a condenser lens 35. The light source CPU 31 controls the light source driver 33 and the diaphragm mechanism 34. Further, the light source CPU 31 communicates with the processor CPU 61 of the processor device 13 to exchange various information.

  As the light source 32, a semiconductor light source such as a light emitting diode (LED) or a laser diode (LD) or a xenon lamp is used, and emission of illumination light is controlled by a light source driver 33. The diaphragm mechanism 34 is disposed on the light emission side of the light source 32 and increases or decreases the amount of illumination light incident on the condenser lens 35 from the light source 32. The condensing lens 35 condenses the illumination light that has passed through the diaphragm mechanism 34 and guides it to the incident end of the light guide 40 in the connector portion 25 a connected to the light source device 12.

  The electronic endoscope 11 is roughly divided into a light guide 40, an illumination window 42, an observation window 43, an observation optical system 44 and a solid-state image sensor 45 which are one form of an imaging unit, an endoscope CPU 47, and a ROM. (Read Only Memory) 48 and an endoscope storage unit 49 are provided.

  For the light guide 40, a large-diameter optical fiber, a bundle fiber, or the like is used. The light guide 40 has an incident end inserted into the light source device 12 via the connector portion 25a, and an emission end opposed to an illumination window 42 provided in the insertion portion distal end portion 16a through the insertion portion 16. ing. The illumination light supplied from the light source device 12 to the light guide 40 is irradiated to the observation object through the illumination window 42. The illumination light reflected / scattered by the observation object enters the observation optical system 44 through the observation window 43.

  The observation optical system 44 is disposed on the back side of the observation window 43. The observation optical system 44 forms reflected light or scattered light of illumination light that has entered through the observation window 43, that is, an optical image of the observation target, on the imaging surface of the solid-state imaging device 45.

  The solid-state imaging device 45 is a CMOS (complementary metal oxide semiconductor) type or CCD (charge coupled device) type imaging device, and is relatively positioned and fixed to the observation optical system 44 at a position farther from the observation optical system 44. ing. On the imaging surface of the solid-state imaging device 45, a plurality of pixels configured by a plurality of photoelectric conversion elements (photodiodes) that photoelectrically convert an optical image are two-dimensionally arranged. The solid-state imaging device 45 converts an optical image formed by the observation optical system 44 into an electrical image signal and outputs it to the processor device 13.

  When the solid-state image sensor 45 is a CMOS type, an A / D converter is built in, and a digital image signal is directly output from the solid-state image sensor 45 to the processor device 13. When the solid-state image sensor 45 is a CCD type, the image signal output from the solid-state image sensor 45 is converted into a digital image signal by an A / D converter (not shown) and then output to the processor device 13. Is done.

  The endoscope CPU 47 sequentially executes various programs and data read from the ROM 48 and the like, and mainly controls the driving of the solid-state image sensor 45. When the solid-state image sensor 45 is a CMOS type, the endoscope CPU 47 may be built in the solid-state image sensor 45.

  Further, the endoscope CPU 47 communicates with the processor CPU 61 of the processor device 13 to transmit information stored in the ROM 48 and the endoscope storage unit 49 to the processor device 13. The ROM 48 stores, for example, identification information for identifying the type of the electronic endoscope 11 as information transmitted to the processor device 13.

  The endoscope storage unit 49 corresponds to an embodiment of the storage unit of the present invention. In the endoscope storage unit 49, image cutout information 51 and an electronic enlargement ratio 53 described later in detail as information transmitted to the processor device 13 are stored in advance when the electronic endoscope 11 is manufactured.

  The processor device 13 includes a processor CPU 61, a ROM 62, a digital signal processing circuit (DSP) 63, a processor storage unit 64, an image processing unit 65, and a display driver 66.

  The processor CPU 61 controls each unit of the processor device 13 by reading out necessary programs and data from the ROM 62 and sequentially processing them. Further, the processor CPU 61 outputs the identification information transmitted from the endoscope CPU 47 to the DSP 63, and outputs the image cutout information 51 and the electronic magnification 53 to the image processing unit 65. Therefore, the processor CPU 61 corresponds to an embodiment of the information acquisition unit of the present invention.

  A DSP 63 that functions as one form of an image input unit that inputs an image signal output from the solid-state imaging device 45, with respect to an image signal for one frame input from the electronic endoscope 11 under the control of the processor CPU 61. Various signal processing such as color interpolation, color separation, color balance adjustment, gamma correction, and image enhancement processing is performed to generate image data for one frame. The DSP 63 performs various types of signal processing according to the type (model) of the electronic endoscope 11 based on the identification information input from the processor CPU 61. Then, the DSP 63 sequentially outputs the generated image data for each frame to the image processing unit 65.

  The processor storage unit 64 corresponds to an embodiment of the storage unit of the present invention. The processor storage unit 64 stores the image cutout information 51 and the electronic enlargement ratio 53 input from the processor CPU 61.

  As shown in FIG. 7, the image processing unit 65 includes an image cutout unit 81 for an image corresponding to the display area, an electronic enlargement processing unit (electronic enlargement processing unit) 82 for electronically enlarging the cutout image, and a part of the image. A mask processing unit 83 for covering the image data with a mask, and under the control of the processor CPU 61, based on the image cropping information 51 and the electronic enlargement ratio 53 acquired from the processor storage unit 64, the image cropping is performed on the image data input from the DSP 63. Display image data is generated by performing processing, electronic enlargement processing, and mask processing. Then, the image processing unit 65 outputs the display image data to the display driver 66.

  The display driver 66 displays an observation image to be observed on the monitor 14 based on the display image data input from the image processing unit 65.

[Image clipping information]
3 shows an effective pixel area 150, a displayable pixel area 155, an image formation area (image circle) 160 on which an optical image is formed by the observation optical system, and a display area (display) displayed on the monitor 14. It is explanatory drawing for demonstrating the positional relationship with (pixel area) 170.

  In FIG. 3, the optical axis of the observation optical system 44 and the center of the displayable pixel area 155 of the solid-state image sensor 45 coincide, and the optical axis direction is perpendicular to the surface of the displayable pixel area 155. This shows a state (ideal positioning state) in which the observation optical system 44 and the solid-state imaging device 45 are positioned with relatively high accuracy so that the optical axis is not tilted. In addition, the observation optical system 44 is an optical system having distortion aberration that is point-symmetric with respect to at least the center (optical axis) of the observation optical system.

  In FIG. 3, an effective pixel area 150 is a pixel area where a pixel signal is used as a signal actually captured on the imaging surface of the solid-state imaging device 45, and is a rectangular pixel area in the present embodiment.

  The effective pixel area 150 includes a displayable pixel area 155 that is an area that can be used for image display on the monitor 14. Specifically, the displayable pixel area 155 is an area obtained by removing a portion used for image processing and the like from the effective pixel area 150 and a margin of several pixels. The displayable pixel area 155 of this embodiment is a rectangular pixel area, and the number of pixels in the vertical and horizontal directions is “2V” and “2H”. In the present embodiment, the center of the effective pixel area 150 is matched with the center CG of the displayable pixel area 155 in order to prevent the drawing from becoming complicated. Note that the center of the effective pixel area 150 and the center CG of the displayable pixel area 155 may be shifted.

  The imaging area 160 is an area of an optical image formed on the imaging surface of the solid-state imaging device 45 by the observation optical system 44, and is a circular area in this embodiment. The plurality of circles indicate ranges of a plurality of viewing angles (for example, 90 °, 150 °, 170 °, etc.). In the example shown in FIG. 3, the observation optical system 44 has point-symmetric distortion, and the imaging area 160 and the circles indicating a plurality of viewing angles are respectively true circles around the optical axis.

The display area 170 is an observation target display area of an image displayed on the monitor 14 and subjected to mask processing described later. The display area 170 of this embodiment is an octagonal area with four corners cut, and the maximum number of pixels in the vertical and horizontal directions is “2L _V ” and “2L _H ”. The aspect ratio (aspect ratio (L _H : L _V )) of the display area 170 is, for example, 4: 3.

Further, in the ideal case shown in FIG. 3, (in the example shown in FIG. 3, the center CK of the imaging area 160) center CH of the display area 170 the upper end and the angle of the field of view of the lower end of the image size L _V from theta _U , theta is _D, the center left and right angles of the field of view of the image size L _H from the cutout region of an image theta _L, when the theta _R, is θ _{U =} θ _D, and the θ _{L =} θ _R.

  Next, the case where the positioning accuracy of the observation optical system 44 and the solid-state image sensor 45 is low (including tilting of the optical axis) and the distortion aberration of the observation optical system 44 is not point-symmetric will be described.

  FIG. 4 shows a case where an image corresponding to the display area 170 is cut out based on the center of the imaging area 160 in a state where the center of the displayable pixel area 155 and the center CK of the imaging area 160 are shifted. .

  In the example illustrated in FIG. 4, an image is cut out from the displayable pixel area 155 so that the center CK of the imaging area 160 is the center of the display area 170.

When an image corresponding to the display area 170 is cut out with the center of the imaging area 160 as a reference, the positional deviation between the observation optical system 44 and the solid-state imaging device 45 (that is, the center CK of the imaging area 160 and the displayable pixel area 155). And the astigmatism of the distortion aberration of the observation optical system 44, the left end visual field angle θ _L and the right end visual field angle θ _R are large. May be different. For example, the viewing angle θ _{L at} the left end of the display area 170 is smaller than 170 ° (for example, about 160 °), whereas the viewing angle θ _{R at the} right end is larger than 170 ° (for example, 175 °), the angle difference between the left-end visual field angle θ _L of the image corresponding to the display area 170 and the right-end visual field angle θ _R becomes large, and the image around the left edge of the display area 170, The difference in image quality (distortion aberration) from the image around the right end increases.

<First Embodiment>
In the first embodiment of the present invention, the angle difference between the visual fields at the upper and lower ends and the angular difference between the visual fields at the left and right ends of the image cut out from the image corresponding to the displayable pixel area 155 (the image corresponding to the display area 170) are acceptable. An image is cut out from the displayable pixel area 155 so as to be within an angle.

If a desired image size (2L _V × 2L _H ) cannot be cut out from the displayable pixel area 155, the image size is reduced and cut out.

As shown in FIG. 5, the displayable pixel area 155, it cuts out the image of 2L _V × 2L _H corresponding to the display area 170. Determining the cutout region of 2L _V × 2L _H is the angle of the upper and lower ends of the central vision (the center of the display area 170) images from CH size _{L V} of the cutout region of an image and theta _U, theta _D, clipping of the image The angle of the visual field at the left end and the right end of the image size L _H from the center of the area (center CH of the display area 170) is θ _L and θ _R, and the angle difference | θ _U −θ _D | If the permissible angles of the visual field angle difference | θ _L −θ _R | are Δθ _V and Δθ _H , respectively,
[Equation 1]
| Θ _U −θ _D | ≦ Δθ _V
| Θ _L −θ _R | ≦ Δθ _H (1)
To be satisfied.

The angle difference | θ _U −θ _D | between the upper and lower ends and the angle difference | θ _L −θ _R | between the left and right ends are preferably zero, but cannot be zero (displayable pixels). If the cutout range cannot be set beyond the range of the area 155 and cannot be set to zero, the cutout range is set within the allowable angles Δθ _V and Δθ _H. The allowable angles Δθ _V and Δθ _H can be set as appropriate values within 10 °, for example.

Thereby, the angular difference between the field of view of the angular difference and the left and right ends of the upper and lower ends of the field of view of the image of the cut-out region (region of 2L _V × 2L _H that corresponds to the display area) may be within respective allowable angle.

Further, when there is no image cutout area that satisfies the above expression (1), the image sizes L _V and L _H from the center of the image cutout area are reduced at the same aspect ratio as that of the display area 170, and after the reduction The image cutout area when the above equation (1) is satisfied is determined by the image sizes A _V and A _H of the image. Here, the reduced image sizes A _V and A _H are preferably the largest image size (the image size closest to the image size of the display area 170) within the range satisfying the above expression (1). This is because the pixels in the displayable pixel area 155 of the solid-state image sensor 45 are effectively used.

  In FIG. 5, a region 170A indicated by a dotted line indicates a cut-out region of the image after reduction.

The endoscope storage unit 49 stores image cutout information 51 that identifies the image cutout area determined as described above. The image clipping information 51 is a coordinate information of the displayable pixel area 155, the two coordinate information diagonal cut image _{((x 1, y 1)} , (x 2, y 2)), or can be displayed One piece of image information at the center or four corners of the image cut out from the pixel area 155 and image size information can be used.

  The image cutout unit 81 (FIG. 7) of the image processing unit 65 cuts out the display image data 73 corresponding to the image cutout region based on the image cutout information 51 stored in the endoscope storage unit 49.

Further, as shown in FIG. 6, in the endoscope storage unit 49, when the cutout area of the image is reduced, the image of the reduced area is displayed in the original size (image size corresponding to the display area 170). ) Is preferably stored with an electronic enlargement ratio 53 for electronic enlargement. The electronic enlargement ratio 53 is based on the image size L _V or L _H from the center of the image cut-out area and the reduced image sizes A _V and A _H reduced at the same aspect ratio as the display area 170. image size L _V value obtained by dividing the image size a _V, or the image size L _H value obtained by dividing the image size a _H, can be obtained as electronic magnification. Further, based on the reduction ratio when the cutout area of the image is reduced, the electronic enlargement ratio 53 can be obtained from the reciprocal of the reduction ratio.

  The electronic enlargement processing unit 82 (FIG. 7) of the image processing unit 65 electronically enlarges the image data 73 cut out based on the electronic enlargement rate 53 stored in the endoscope storage unit 49, and displays the display area. Display image data 73A corresponding to 170 is generated.

  Furthermore, it is preferable that the electron magnification 53 is not more than twice. In other words, since the resolution (image quality) decreases as the electronic magnification 53 increases, it is preferable to limit the electronic magnification 53 so that it does not exceed twice. In addition, when the error of the relative attachment position of the observation optical system 44 and the solid-state image sensor 45 is large, and an electronic magnification rate exceeds 2 times, it can be considered as inferior goods.

Second Embodiment
In the second embodiment of the present invention, the angles of the top, bottom, left, and right fields of view of an image cut out from an image corresponding to the displayable pixel area 155 (an image corresponding to the display area 170) are within allowable angles. Thus, an image is cut out from the displayable pixel area 155. That is, in the first embodiment, the image is cut out from the displayable pixel area 155 so that the angle difference between the upper and lower visual fields and the angle difference between the left and right visual fields are within allowable angles. The second embodiment is different in that the image is cut out from the displayable pixel area 155 so that the angles of the visual fields at the upper end, the lower end, the left end, and the right end of the cut out image are within allowable angles.

Further, when a desired image size (2L _V × 2L _H ) cannot be cut out from the displayable pixel area 155, the point that the image size is reduced and cut out is the same as in the first embodiment.

As shown in FIG. 5, the displayable pixel area 155, it cuts out the image of 2L _V × 2L _H corresponding to the display area 170. Determining the cutout region of 2L _V × 2L _H is the angle of the upper and lower ends of the central vision (the center of the display area 170) images from CH size _{L V} of the cutout region of an image and theta _U, theta _D, clipping of the image The angles of the visual field at the left end and the right end of the image size L _H from the center of the area (display area center CH) are θ _L and θ _R, and the allowable angle ranges of the angles θ _U , θ _D , θ _L , and θ _R are Assuming that Θ _U ± ΔU, Θ _D ± ΔD, Θ _L ± ΔL, and Θ _R ± ΔR, respectively,
[Equation 2]
| Θ _U −θ _U | ≦ ΔU
| Θ _D −θ _D | ≦ ΔD
| Θ _L −θ _L | ≦ ΔL
| Θ _R −θ _R | ≦ ΔR (2)
To be satisfied.

Incidentally, the angle difference _{| Θ U -θ U |, |} Θ D -θ D |, | Θ L -θ L |, and | Θ _R -θ _R |, is preferably a zero, zero When it is not possible (when the clipping range cannot be set beyond the range of the displayable pixel area 155 and cannot be set to zero), the angle difference is set within ΔU, ΔD, ΔL, and ΔR, respectively. ΔU, ΔD, ΔL, and ΔR can be set as appropriate values within about half of the allowable angles Δθ _V and Δθ _H of the first embodiment, for example, within 5 °.

Thus, the upper end of the image of the cut-out region (region of 2L _V × 2L _H that corresponds to the display area), the lower end, left end, and right end of the field of view angle _{(θ U, θ D, θ} L, θ R) of each It can be within an allowable angle.

Further, the (2) when the cut-out region of the image does not exist to satisfy the equation, the image size 2L _V and 2L _H of the cut-out region of the image, reduced in the same aspect ratio as the display area 170, the image size after reduction Based on A _V and A _H , an image cut-out region when the above expression (2) is satisfied is determined. Here, the reduced image sizes A _V and A _H are preferably the largest image size (the image size closest to the image size of the display area 170) within the range satisfying the expression (2). This is because the pixels in the displayable pixel area 150 of the solid-state image sensor 45 are effectively used.

  Then, similarly to the first embodiment, the image storage information 51 and the electronic magnification rate 53 are stored in the endoscope storage unit 49.

  Then, the image cutout unit 81 of the image processing unit 65 shown in FIG. 7 cuts out the display image data 73 corresponding to the image cutout region based on the image cutout information 51 stored in the endoscope storage unit 49 (FIG. 7). 5) The electronic enlargement processing unit 82 of the image processing unit 65 electronically enlarges the image data 73 cut out based on the electronic enlargement ratio 53 stored in the endoscope storage unit 49, and displays the display area 170. The display image data 73A corresponding to is generated (FIG. 6).

[Configuration of image processing unit]
FIG. 7 is a functional block diagram of the image processing unit 65. As illustrated in FIG. 7, the image processing unit 65 includes a frame memory 80, an image cutout unit 81, an electronic enlargement processing unit 82, and a mask processing unit 83.

  The frame memory 80 is a memory that temporarily stores image data 71 (image data of the displayable pixel area 155) for each frame input from the DSP 63, and simultaneously stores image data 71 for a plurality of frames. The new image data 71 input from the DSP 63 is overwritten on the oldest image data 71 stored in the frame memory 80.

  The image cutout unit 81 reads out the newly recorded image data 71 from the frame memory 80, and as shown in FIG. 5, the image cutout information 51 acquired from the processor storage unit 64 (or the endoscope storage unit 49). Based on the above, the display image data 73 is cut out from the image data 71. The image cutout unit 81 outputs the cutout display image data 73 to the electronic enlargement processing unit 82.

  As shown in FIG. 6, the electronic enlargement processing unit 82 electronically displays the display image data 73 input from the image cutout unit 81 based on the electronic enlargement rate 53 acquired from the processor storage unit 64. And display image data 73A is generated. Note that the electronic enlargement process is a process of interpolating the display image data 73 in accordance with the electronic enlargement ratio 53 to increase the number of pixels to obtain an image size corresponding to the display area 170. The electronic enlargement processing unit 82 outputs the generated display image data 73A to the mask processing unit 83.

  When the display image data 73 input from the image cutout unit 81 has an image size corresponding to the display area 170, the value of the electronic enlargement ratio 53 is 1. In this case, the electronic enlargement processing unit 82 outputs the input display image data 73 as it is without performing the electronic enlargement process.

  FIG. 8 is an explanatory diagram for explaining the mask processing by the mask processing unit 83. As shown in FIG. 8, the mask processing unit 83 performs mask processing for masking the display image data 73A (or the display image data 73) in the same shape as the display area 170 based on the mask image data 86, and performs display processing. Image data 73B is generated. The mask image data 86 is a rectangular image having the same size as the display image data 73A, and includes a non-mask portion 86a that exposes the display area 170 and a mask portion 86b.

  In the mask process, the display image data 73A is flowed pixel by pixel in the mask processing unit 83, and pixels corresponding to the display area 170 (pixels corresponding to the non-mask part 86a) are output as they are based on the mask image data 86. Then, the pixel corresponding to the outside of the display area 170 (the pixel corresponding to the mask portion 86b) is discarded and the mask pixel is output instead. The mask pixel is data corresponding to, for example, a low luminance (black) pixel. The mask processing unit 83 outputs the display image data 73B subjected to the mask process to the display driver 66. Thereby, the observation image of the observation target based on the display image data 73 </ b> B is displayed on the monitor 14 by the display driver 66.

[ Operation method of endoscope apparatus]
Next, an operation method of the endoscope apparatus 10 having the above configuration will be described with reference to FIGS. 9 to 12.

<Display processing of observation image>
Figure 9 is a flowchart showing a flow of display processing of the observation image including an image processing according to the operation method of the endoscope apparatus according to the present invention, FIG. 10 according to the operation method of the endoscope apparatus according to the present invention It is explanatory drawing for demonstrating image processing.

  As shown in FIG. 10, the activation of the endoscope apparatus 10 is first started. Specifically, the connector portions 25a and 25b of the electronic endoscope 11 are connected to the light source device 12 and the processor device 13, respectively, and the power sources of the light source device 12 and the processor device 13 are turned on. As a result, the light source device 12 and the processor device 13 are activated, and the electronic endoscope 11 is activated by power supply from the light source device 12 or the like.

  When starting up the electronic endoscope 11, the light source device 12, and the processor device 13, the endoscope CPU 47 converts the image cutout information 51 and the electronic magnification 53 stored in the endoscope storage unit 49 into the processor CPU 61. Send to. Thereby, the processor CPU 61 can acquire the image cutout information 51 and the electronic enlargement ratio 53. The processor CPU 61 stores the acquired image cutout information 51 and the electronic enlargement ratio 53 in the processor storage unit 64.

  Next, the image cutout unit 81 acquires the image cutout information 51 from the processor storage unit 64 (step S11), and the electronic enlargement processing unit 82 acquires the electronic enlargement rate 53 from the processor storage unit 64 (step S12).

  Further, other necessary information is exchanged among the light source CPU 31, the endoscope CPU 47, and the processor CPU 61, and the activation of the endoscope apparatus 10 is completed.

  After the start-up of the endoscope apparatus 10 is completed, the insertion unit 16 of the electronic endoscope 11 is inserted into the patient's body and imaging of the observation target is performed (step S13). Illumination light supplied from the light source device 12 passes through the light guide 40 and is emitted from the illumination window 42 toward the observation target. The illumination light reflected or scattered by the observation object is imaged on the imaging surface of the solid-state imaging device 45 as an optical image by the observation optical system 44 through the observation window 43. The solid-state imaging device 45 converts the optical image formed on the imaging surface into an image signal and outputs the image signal to the DSP 63 of the processor device 13.

  The DSP 63 performs various signal processing such as color interpolation, color separation, color balance adjustment, gamma correction, and image enhancement processing on the image signal for one frame input from the electronic endoscope 11, and the image for one frame. Data 71 is generated (step S14). The image data 71 is output from the DSP 63 to the image processing unit 65 and temporarily stored in the frame memory 80 of the image processing unit 65.

  Next, image processing corresponding to one form of the image processing step of the present invention is started by each unit of the image processing unit 65 on the image data 71 stored in the frame memory 80 (step S15). Specifically, an image cutout process (step S16), an image enlargement process (step S17), and a mask process (step S18) are sequentially performed on the image data 71.

  As shown in FIGS. 10A and 10B, the image cutout unit 81 reads the image data 71 from the frame memory 80, and based on the previously obtained image cutout information 51, the image data 71 is transferred to the image cutout area. Image cutout processing for cutting out the corresponding display image data 73 is performed. At this time, the visual angle difference between the upper and lower visual fields and the angular difference between the visual fields at the left and right edges of the display image data 73 (image corresponding to the display area) are within an allowable angle, or The display image data 73 is cut out from the image data 71 so that the visual field angles at the lower end, the left end, and the right end are within allowable angles. The clipped display image data 73 is output from the image clipping unit 81 to the electronic enlargement processing unit 82.

  As shown in FIG. 10C, the electronic enlargement processing unit 82 electronically enlarges the display image data 73 input from the image cutout unit 81 at the electronic enlargement rate 53 based on the previously acquired electronic enlargement rate 53. Perform image enlargement processing. Thereby, display image data 73A obtained by enlarging the display image data 73 to the image size of the display area 170 is obtained.

  The display image data 73A is output from the electronic enlargement processing unit 82 to the mask processing unit 83. When the value of the electronic enlargement ratio 53 is 1 (when the electronic enlargement process is not performed), the display image data 73 input from the image cutout unit 81 is output to the mask processing unit 83 as it is.

  As illustrated in FIG. 10D, the mask processing unit 83 performs mask processing on the display image data 73A to generate display image data 73B. The display image data 73B subjected to the mask processing is output from the mask processing unit 83 to the display driver 66.

  Returning to FIG. 9, the display driver 66 causes the monitor 14 to display an observation image to be observed based on the display image data 73B input from the mask processing unit 83 (step S19). Hereinafter, the above-described processing from step S13 to step S19 is repeatedly executed until imaging of the observation target is completed (step S20).

<Image cutout information and electronic magnification rate storage process>
FIG. 11 is a part of the manufacturing process of the electronic endoscope 11 and the flow of the storage process of the image cutout information 51 and the electronic magnification 53 used in the image processing according to the operation method of the endoscope apparatus according to the present invention. It is a flowchart which shows. FIG. 12 is a diagram illustrating an example of the test chart 100.

  As shown in FIG. 11, after the assembly of the imaging module including the observation optical system 44 and the solid-state imaging device 45 of the electronic endoscope 11 is completed (even after the assembly of the electronic endoscope 11 is completed), the imaging module The test chart 100 is imaged (step S30).

  As shown in FIG. 12, the test chart 100 is one in which, for example, a circle (concentric circle) indicating a number of viewing angles is drawn as an index indicating a viewing angle.

  At the time of imaging the test chart 100, the distance between the test chart 100 and the imaging module is adjusted accurately, and the center of the test chart 100 and the optical axis of the observation optical system 44 are adjusted to coincide. The optical axis of the observation optical system 44 can be obtained by analyzing an image obtained by imaging the test chart 100 and using, for example, a point symmetry point of distortion as the optical axis. The method for matching the center of the test chart 100 with the optical axis of the observation optical system 44 is not limited to this.

Subsequently, image cutout information for cutting out a required image from the effective pixel area 150 of the solid-state image sensor 45 is set (step S32). Image cropping information here, for example, the center coordinates of the image cut-out area _{C (x 0, y 0)} , the image size of the cut out image _(2L H, 2L _V) can be. The center coordinates C (x _0, y ₀ ) and the image size (2L _H , 2L _V ) can be set to preset values, and the center coordinates C (x ₀ , y ₀ ) (The center CG of the displayable pixel area 155).

Subsequently, the visual field angles (θ _U , θ _D , θ _L , θ _R ) of the upper end, the lower end, the left end, and the right end of the image cut out based on the currently set image cut-out information are measured (step S34). These angles (θ _U , θ _D , θ _L , θ _R ) can be read from the image of the test chart 100.

Next, based on the angles (θ _U , θ _D , θ _L , θ _R ) measured in step S34, the angle difference | θ _U −θ _D | of the upper and lower end visual fields and the angle difference of the left and right visual fields | θ _L −θ _R | is obtained, and it is determined whether or not these angle differences | θ _U −θ _D | and angle difference | θ _L −θ _R | are within allowable angles Δθ _V and Δθ _H , respectively. (Step S36).

When at least one of the angle difference | θU−θD | and the angle difference | θL−θR | is not within the allowable angles ΔθV and ΔθH (in the case of “No”), the center coordinates C (x _0, y ₀ ) can be corrected (that is, whether or not the image clipping region can be moved within the displayable pixel area 155) (step S38). Note that the center coordinates C (x _0, y ₀ ) of the image cut-out area are moved by the difference (allowance) between the image size of the displayable pixel area 155 and the image size (2L _V , 2L _H ) of the cut-out image. be able to.

When the center coordinates C (x _0, y ₀ ) of the image cutout region can be corrected (when “Yes”), the angle difference | θ _U −θ _D | and the angle difference | θ _L −θ _R | Correction is made in a decreasing direction (step S40), and the process proceeds to step S34. In step S34, the corrected angles (θ _U , θ _D , θ _L , θ _R ) of the image cutout region are measured again, and the processing from step S34 to step S40 is repeated.

On the other hand, in step S38, when the correction of the center coordinates C (x ₀ , y ₀ ) of the image cut-out area is impossible (in the case of “No”), the image size (2L _V , 2L _H ) of the cut-out image is Reduction is performed at a predetermined reduction rate α (for example, 0.9) (step S42). That is, the current image size (2L _V , 2L _H ) is multiplied by the reduction ratio α, and the multiplication result is set as a new image size.

After the image size is reduced, the process proceeds to step S34, and the processes from step S34 to step S40 are performed again. Note that, by reducing the image size of the image to be cut out, the range in which the center coordinates C (x _0, y ₀ ) of the image cut-out region can be corrected is expanded.

In step S36, when the angle difference | θ _U −θ _D | and the angle difference | θ _L −θ _R | are within the allowable angles Δθ _V and Δθ _H (when “Yes”), respectively. The currently set image cutout information and the electronic enlargement ratio (reciprocal of the reduction ratio) are stored in the endoscope storage unit 49 (step S44), and the process ends.

  The determination of the image cutout information indicating the image cutout area may be automatically performed, or the operator may appropriately determine the cutout information. In the latter case, it is preferable to provide a user interface that displays the position of the image cutout region and the image size of the cutout image superimposed on the image on which the test chart is captured.

  Further, the process of storing the image cutout information and the electronic enlargement ratio shown in FIG. 11 in the endoscope storage unit 49 corresponds to the first embodiment of the present invention, but similarly in the case of the second embodiment. It can be carried out.

[Other embodiments of display area]
FIG. 13 is an explanatory diagram for explaining image processing according to the operation method of the endoscope apparatus according to the present invention, and shows a case of a display area according to another embodiment.

  The display area of the embodiment shown in FIG. 13 is circular.

  As shown in FIGS. 13A and 13B, the image cutout unit 81 reads the image data 71 (image data of the effective pixel area 150) from the frame memory 80 and acquires the image from the endoscope storage unit 49. Based on the cutout information 51, a process of cutting out the display image data 73 corresponding to the image cutout area from the image data 71 is performed.

  In this case, the image cutout information 51 stored in the endoscope storage unit 49 is image cutout information indicating a square cutout area that circumscribes a circular display area, and display image data 73 corresponding to the square cutout area. The angle difference between the visual field at the upper and lower ends and the angle difference between the visual fields at the left and right ends are within the allowable angle, or the visual field angles at the upper, lower, left, and right ends of the display image data 73 are within the allowable angle. The cutout area is set so as to be.

  Display image data 73 cut out based on image cutout information indicating a square cutout area is output from the image cutout unit 81 to the electronic enlargement processing unit 82.

  As shown in FIG. 13C, the electronic enlargement processing unit 82 converts the display image data 73 input from the image clipping unit 81 based on the electronic enlargement rate 53 acquired from the endoscope storage unit 49 to an electronic enlargement rate. In 53, image enlargement processing for electronic enlargement is performed. Thereby, display image data 73A obtained by enlarging the display image data 73 to the image size of the display area 170 is obtained.

  The display image data 73A is output from the electronic enlargement processing unit 82 to the mask processing unit 83. When the electronic enlargement ratio is 1 (when the electronic enlargement process is not performed), the display image data 73 input from the image cutout unit 81 is output to the mask processing unit 83 as it is.

  As shown in FIG. 13D, the mask processing unit 83 performs mask processing on the display image data 73A to generate display image data 73B. The display image data 73B subjected to the mask processing is output from the mask processing unit 83 to the display driver 66.

  The shapes and sizes of the effective pixel area, the displayable pixel area, the imaging area, and the display area are not limited to the shapes and sizes shown in the above embodiments, and may be changed as appropriate. Also good. Further, when the display area is rectangular, the mask process can be omitted.

[Example of application to capsule system]
Examples of the electronic endoscope constituting the endoscope apparatus of the present invention include a flexible endoscope, a rigid endoscope, an industrial endoscope, a capsule system (also referred to as a capsule endoscope), and the like. Hereinafter, a capsule system will be described as an example and will be described in detail with reference to the drawings.

  As shown in FIG. 14, the capsule system 501 includes an illumination system 512 and a camera that includes an optical system 514 and an image sensor 516. The image captured by the image sensor 516 is processed by the image processor 518. The image processor 518 is implemented by software executed by a digital signal processor (DSP) or a central processing unit (CPU), by hardware, or by a combination of both software and hardware. be able to. The processed image is compressed by an image compression subsystem 519 (in some embodiments, implemented in software executed by the DSP of the image processor 518). The compressed data is stored in the archive memory system 520. The capsule system 501 includes a battery power source 521 and an output port 526. The capsule system 501 can advance through the digestive tract (GI tract) 500 by peristalsis.

  The illumination system 512 can also be configured with LEDs. In FIG. 14, the LEDs are arranged close to the camera opening, but other arrangements are possible. A light source may be provided, for example, behind the opening. Other light sources such as laser diodes may be used. Alternatively, a white light source or a combination of two or more narrow wavelength band light sources may be used. It is also possible to use a white LED with a phosphorescent material that is excited by the light of the LED to emit long wavelength light. The white LED may include a blue LED or a purple LED. The predetermined portion of the capsule housing for allowing light to pass is made from a biologically compatible glass or polymer.

  The optical system 514 corresponds to an embodiment of the observation optical system of the present invention, and allows the image sensor 516 to read an image of the wall of the lumen of the GI tube 500 or the like, and includes a plurality of refractive lenses. It may include an element, a diffractive lens element, or a reflective lens element.

  The image sensor 516 converts the received light intensity into a corresponding electrical signal and can be provided by a charge coupled device (CCD) or complementary metal oxide semiconductor (CMOS) type device. The image sensor 516 may be monochromatic, or may include a color filter array that can capture a color image (eg, using RGB or CYM representations). This image sensor 516 corresponds to an embodiment of the solid-state imaging device of the present invention.

  The analog signal from the image sensor 516 is preferably converted to digital form so that it can be processed in digital form. Such a conversion is performed using an analog-to-digital (A / D) converter provided in the sensor (in this embodiment) or in another part of the capsule housing 510. An A / D unit can also be provided between the image sensor 516 and other parts of the system. The LEDs of the illumination system 512 are synchronized with the operation of the image sensor 516. One of the functions of the control module (not shown) of the capsule system 501 is to control the LED during an image capture operation.

  The endoscope storage unit 527 corresponds to one form of the storage unit of the present invention, and corresponds to the mounting accuracy of the optical system 514 and the image sensor 516, similarly to the endoscope storage unit 49 of the above embodiment. The image cutout information 51 and the electronic magnification 53 are stored. The image cutout information 51 and the electronic magnification 53 stored in the endoscope storage unit 527 are transmitted to the processor device via the output port 526.

  FIG. 15 illustrates a swallowable capsule system 502 according to one embodiment of the present invention. The capsule system 502 can be configured substantially similar to the capsule system 501 of FIG. 14 except that the archive memory system 520 and the output port 526 are not required. The capsule system 502 also includes a communication protocol encoder 1320 and a transmitter 1326 used for wireless transmission. Of the elements of the capsule system 501 and the capsule system 502, substantially the same elements are given the same reference numerals. Therefore, their structure and function will not be described again here. The control module 522 performs overall control of the entire capsule system 502. The communication protocol encoder 1320 is implemented by software executed by a DSP or CPU, hardware, or a combination of software and hardware. The transmitter 1326 includes an antenna system for transmitting the captured digital image.

  A part of the ROM (not shown) of the control module 522 functions as the endoscope storage unit 528. The endoscope storage unit 528 corresponds to one form of the storage unit of the present invention, and corresponds to the mounting accuracy of the optical system 514 and the image sensor 516, similarly to the endoscope storage unit 49 of the above embodiment. The image cutout information 51 and the electronic magnification 53 are stored. The image cutout information 51 and the electronic magnification 53 stored in the endoscope storage unit 528 are transmitted to the processor device via the transmitter 1326.

  The processor device corresponding to the capsule system 501 and the capsule system 502 has basically the same configuration as the processor device of the above embodiment except that the processor device corresponds to the capsule system. Cutout processing, image enlargement processing based on the electronic enlargement ratio 53, and the like are performed. Thereby, the effect similar to the effect demonstrated by the said embodiment is acquired.

[Others]
In the above embodiment, the processor CPU 61 corresponding to the information acquisition unit of the present invention acquires the image cutout information 51 and the electronic magnification 53 from the electronic endoscope 11. The obtained image cutout information 51 and the electronic enlargement ratio 53 may be acquired, or the image cutout information 51 and the electronic enlargement ratio 53 may be obtained from a communication interface or a memory card.

  In addition, a plurality of types of image cutout information 51 and an electronic enlargement ratio 53 corresponding to a plurality of types of electronic endoscope 11 are stored in advance in the processor storage unit 64 and are input from the electronic endoscope 11 to the processor device 13. Based on the identification information of the electronic endoscope 11 or the like, the image cutout information 51 and the electronic magnification 53 corresponding to the type of the electronic endoscope 11 may be selected.

  In the above embodiment, the light source device 12 and the processor device 13 are provided separately, but both may be provided integrally.

  Further, since the electronic enlargement ratio can be calculated based on the image size specified by the image cutout information 51 and the image size of the display area, the electronic enlargement ratio does not necessarily have to be stored.

  It goes without saying that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

  DESCRIPTION OF SYMBOLS 10 ... Endoscope apparatus, 11 ... Electronic endoscope, 13 ... Processor apparatus, 14 ... Monitor, 44 ... Observation optical system, 45 ... Solid-state image sensor, 47 ... Endoscope CPU, 49 ... Endoscope memory | storage part, 51 ... Image cutout information, 53 ... Electronic magnification, 61 ... Processor CPU, 64 ... Processor storage unit, 65 ... Image processing unit, 73 ... Display image data, 73A, 73B ... Display image data, 81 ... Image cutout unit 82 ... Electronic enlargement processing unit, 83 ... Mask processing unit, 150 ... Effective pixel area, 155 ... Displayable pixel area, 160 ... Image forming area, 170 ... Display area

Claims (15)

An observation optical system;
A solid-state imaging device that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system, and in which a plurality of photoelectric conversion elements that photoelectrically convert the optical image are arranged An image sensor;
A storage unit for storing image cutout information for specifying an image cutout region for cutting out an image corresponding to a display area displayed on a monitor from an image corresponding to a displayable pixel area of the solid-state imaging element;
The said storage part is an endoscope apparatus which memorize | stores the said image cut-out information in which the angle difference of the visual field of the upper and lower ends of the image corresponding to the said display area, and the angle difference of the visual field of a right-and-left end are each within an allowable angle.   The image cutout information stored in the storage unit is a cutout area having the same aspect ratio as the display area, and upper and lower visual fields of an image cut out by the cutout area from an image corresponding to the displayable pixel area. The endoscope apparatus according to claim 1, which is information indicating a cutout region closest to the image size of the display area among images in which the difference in angle and the difference in angle between the left and right visual fields are within an allowable angle. The storage unit
Image size of vertical and horizontal image size of 2L V of the display _area, and 2L _H, the angle of the field of view of the upper and lower ends of the image size L _V and theta _U, theta _D from the center of the cutout region of the image , Θ _L and θ _R are the visual angles of the left end and the right end of the image size L _H from the center of the cutout region of the image, and the angle difference | θ _U −θ _D | When the allowable angles of the visual field angle difference | θ _L −θ _R | are Δθ _V and Δθ _H , respectively,
| Θ _U −θ _D | ≦ Δθ _V
| Θ _L −θ _R | ≦ Δθ _H (1)
The endoscope apparatus according to claim 1, wherein a cutout region of the image that satisfies the above is stored as the image cutout information. The storage unit sets the image sizes L _V and L _H from the center of the cutout area of the image to the same aspect ratio as the display area when there is no cutout area of the image that satisfies the expression (1). The endoscope apparatus according to claim 3, wherein the image cut-out area is stored as the image cut-out information when the image size is reduced by the image size A _V and A _{H and the} expression (1) is satisfied. An observation optical system;
A solid-state imaging device that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system, and in which a plurality of photoelectric conversion elements that photoelectrically convert the optical image are arranged An image sensor;
A storage unit for storing image cutout information for specifying an image cutout region for cutting out an image corresponding to a display area displayed on a monitor from an image corresponding to a displayable pixel area of the solid-state imaging element;
The storage unit stores the image cut-out information in which the angles of the visual field at the upper end, the lower end, the left end, and the right end of the image corresponding to the display area are each within an allowable angle,
The image cutout information stored in the storage unit is a cutout area having the same aspect ratio as the display area, and an upper end, a lower end, and a left end of an image cut out from the image corresponding to the displayable pixel area by the cutout area. An endoscope apparatus that is information indicating a cutout area closest to the image size of the display area among images in which the angle of the visual field at the right end is within an allowable angle. An observation optical system;
A solid-state imaging device that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system, and in which a plurality of photoelectric conversion elements that photoelectrically convert the optical image are arranged An image sensor;
A storage unit for storing image cutout information for specifying an image cutout region for cutting out an image corresponding to a display area displayed on a monitor from an image corresponding to a displayable pixel area of the solid-state imaging element;
The storage unit stores the image cutout information in which the angles of the visual field at the upper end, the lower end, the left end, and the right end of the image corresponding to the display area are each within an allowable angle, and the image of the display area the size of the vertical and horizontal image size and 2L _V, 2L _H, the upper end and the angle of the field of view of the lower end of the image size L _V and theta _U, theta _D from the center of the cutout region of the image, clipping of the image The viewing angles of the left end and the right end of the image size L _H from the center of the region are θ _L and θ _R, and the allowable angle ranges of the angles θ _U , θ _D , θ _L , and θ _R are respectively θ _U ± ΔU, If Θ _D ± ΔD, Θ _L ± ΔL, Θ _R ± ΔR,
| Θ _U −θ _U | ≦ ΔU
| Θ _D −θ _D | ≦ ΔD
| Θ _L −θ _L | ≦ ΔL
| Θ _R −θ _R | ≦ ΔR (2)
The endoscope apparatus which memorize | stores the cutout area | region of the said image which satisfy | fills as said image cutout information. When there is no cutout area of the image that satisfies the expression (2), the storage unit sets the image sizes L _V and L _H from the center of the cutout area of the image to the same aspect ratio as the display area. The endoscope apparatus according to claim 6, wherein the image cut-out area is stored as the image cut-out information when the image size is reduced by the image size A _V and A _{H and the} expression (2) is satisfied. The storage unit is an electronic enlargement ratio calculated by the image size L _V or L _H and the reduced image size _AV or A _H, and the image size L _V is the image size _AV . dividing the said electronic enlargement ratio, or endoscope apparatus according to the image size L _H to claim 4 or 7 to store the electronic magnification obtained by dividing by the image size a _H.   The endoscope apparatus according to claim 8, wherein the electronic magnification rate stored by the storage unit is 2 or less.   The storage unit further stores an electronic enlargement ratio for enlarging the image size of the image cutout area to the image size of the display area when the image size of the image cutout area is smaller than the image size of the display area. The endoscope apparatus according to any one of claims 1 to 9.   The image cut-out information stored in the storage unit is two pieces of coordinate information on the diagonal of the image cut out from the displayable pixel area, or one piece of image information and image at the center or four corners of the image cut out from the displayable pixel area. The endoscope apparatus according to any one of claims 1 to 10, which is size information. An image input unit for inputting an image corresponding to an effective pixel area of the solid-state image sensor from the solid-state image sensor;
A cutout unit that cuts out an image corresponding to the display area based on image cutout information stored in the storage unit from an image corresponding to the effective pixel area input by the image input unit;
An electronic enlargement processing unit for enlarging the image size of the cutout image to the image size of the display area when the image size of the image cut out by the cutout unit is smaller than the image size of the display area;
The endoscope apparatus according to claim 1, further comprising:   The endoscope apparatus according to claim 12, further comprising a mask processing unit that masks an image cut out by the cutout unit in the same shape as the display area. An observation optical system and a solid-state imaging element that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system, and a plurality of photoelectric conversion elements that photoelectrically convert the optical image Is an operation method of an endoscope apparatus including: an imaging unit including a solid-state imaging device in which an image is arranged; and an image processing unit that inputs an image corresponding to an effective pixel area of the solid-state imaging device from the solid-state imaging device. And
The imaging unit imaging a chart having an index indicating a viewing angle;
Determining image cutout information for specifying an image cutout area for cutting out an image corresponding to a display area to be displayed on a monitor from an image corresponding to a displayable pixel area of the solid-state imaging device;
Storing the determined image cutout information in a storage unit,
The step of determining the image cut-out information is based on the image acquired by the step of imaging the chart, and the angle difference between the visual fields at the upper and lower ends of the image corresponding to the display area, and the angle difference between the visual fields at the left and right ends, An operation method of an endoscope apparatus for determining the image cut-out information that is within an allowable angle. An observation optical system and a solid-state imaging element that is positioned and fixed relative to the observation optical system and forms an optical image by the observation optical system, and a plurality of photoelectric conversion elements that photoelectrically convert the optical image Is an operation method of an endoscope apparatus including: an imaging unit including a solid-state imaging device in which an image is arranged; and an image processing unit that inputs an image corresponding to an effective pixel area of the solid-state imaging device from the solid-state imaging device. And
The imaging unit imaging a chart having an index indicating a viewing angle;
Determining image cutout information for specifying an image cutout area for cutting out an image corresponding to a display area to be displayed on a monitor from an image corresponding to a displayable pixel area of the solid-state imaging device;
Storing the determined image cutout information in a storage unit,
In the step of determining the image cutout information, based on the image acquired in the step of capturing the chart, the angles of the visual field at the upper end, the lower end, the left end, and the right end of the image corresponding to the display area are each within an allowable angle. The operation method of the endoscope apparatus for determining the image cutout information.

Images