JPWO2018079387A1 - Image processing device - Google Patents

Abstract

An image processing apparatus according to the present invention is an image processing apparatus that generates a parallax image based on first and second images obtained by capturing the same subject from different directions, and based on the first and second images, A display image generation unit that generates a parallax image for display by performing at least zoom processing, and a recording that is recorded on an external recording medium based on the first and second images before zoom processing by the display image generation unit A recording image generating unit that generates an image for use.

Description

  The present invention relates to an image processing apparatus and an image processing system.

  2. Description of the Related Art In recent years, there has been known an image creation method for capturing a subject and acquiring a stereoscopic image from two image data for left eye and right eye having parallax with each other. In an endoscope system in which an endoscope used in the medical field or the like and a processing device (processor) are detachable, there is a demand for observing an observation target as a three-dimensional image in order to facilitate diagnosis and examination. is there. As a technique for meeting this requirement, an endoscope including two optical systems for the left eye and right eye and an image sensor that receives light from each optical system is known (for example, a patent). Reference 1). The processor to which the endoscope is attached receives two image data for the left eye and right eye from the attached endoscope, and generates a parallax image from the two image data.

  In an endoscope system, distance measurement from an endoscope to a subject may be performed using the above-described parallax image. At this time, parallax image data is recorded on a recording medium such as a USB memory from the processor, and another measurement device reads the data recorded on the USB memory and performs measurement processing.

JP 2014-140594 A

  However, in a conventional processor, measurement data is input via a USB memory in which data subjected to zoom processing or the like for display is recorded. The data recorded in the USB memory is subjected to enlargement and thinning processing by image processing such as zoom processing, and the amount of information is smaller than data output from the image sensor, for example, RAW data. Yes. When distance measurement processing is performed using such data, there is a problem that measurement accuracy is lowered.

  The present invention has been made in view of the above, and an image processing apparatus and an image processing system capable of suppressing a decrease in measurement accuracy when distance measurement processing is performed using data recorded on a recording medium. The purpose is to provide.

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention generates an image of parallax based on first and second images obtained by imaging the same subject from different directions. A display image generation unit that generates a parallax image for display by performing at least a zoom process based on the first and second images, and the zoom process before the zoom process by the display image generation unit A recording image generation unit configured to generate a recording image to be recorded on an external recording medium based on the first and second images.

  Further, in the image processing apparatus according to the present invention, in the above invention, when the parallax image is freeze-displayed and an instruction to record the freeze-displayed parallax image is received, each of the first and second images is displayed. The image processing apparatus further includes a freeze processing unit that selects the first and second images to be frozen from a plurality of sets of images.

  In the image processing apparatus according to the present invention, in the above invention, the recording image generation unit generates the recording image based on the first and second images selected by the freeze processing unit. It is characterized by.

  In the image processing apparatus according to the present invention as set forth in the invention described above, the recording image generation unit may be configured to store the recording image and information recorded in an endoscope that inputs the parallax image. It is recorded on a recording medium.

  The image processing apparatus according to the present invention further includes a preprocessing unit that performs at least OB clamping processing and white balance processing as preprocessing on the first and second images in the above invention, and the recording image. The generation unit generates the recording image based on the first and second images after the preprocessing by the preprocessing unit.

  In the image processing apparatus according to the present invention as set forth in the invention described above, the recording image generation unit records the image processing parameters relating to the recording image together with the recording image on the external recording medium. It is characterized by.

  In the image processing apparatus according to the present invention, in the above invention, the display image generation unit includes the side-by-side image in which the first and second images are arranged in a horizontal direction of the image sensor, or the first and second images. A line-by-line image in which the second image is alternately arranged for each line in the horizontal direction is generated as the parallax image for display.

  Further, in the image processing apparatus according to the present invention, in the above invention, the recording image generation unit includes a side-by-side image in which the first and second images are arranged in a horizontal direction of the imaging device, or the first and second images. A line-by-line image in which the second image is alternately arranged for each line in the horizontal direction is generated as the recording image.

  An image processing system according to the present invention is based on an imaging unit that generates image data including first and second images by imaging the same subject from different directions, and the first and second images. A display image generation unit that generates a parallax image for display by performing at least a zoom process, and an external recording medium based on the first and second images before the zoom process by the display image generation unit An image processing apparatus having a recording image generation unit that generates a recording image to be recorded.

  According to the present invention, when distance measurement processing is performed using data recorded on a recording medium, it is possible to suppress a decrease in measurement accuracy.

FIG. 1 is a diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a schematic configuration of the endoscope system according to the embodiment of the present invention. FIG. 3 is a diagram illustrating a parallax image generated by the endoscope system according to the embodiment of the present invention. FIG. 4 is a flowchart showing image processing performed by the endoscope system according to the embodiment of the present invention.

  Hereinafter, modes for carrying out the present invention (hereinafter referred to as “embodiments”) will be described. In the embodiment, as an example of an image processing system including an image processing apparatus according to the present invention, a medical endoscope system that captures and displays an image in a subject such as a patient will be described. Moreover, this invention is not limited by this embodiment. Furthermore, in the description of the drawings, the same portions will be described with the same reference numerals.

(Embodiment)
FIG. 1 is a diagram showing a schematic configuration of an endoscope system according to an embodiment of the present invention. FIG. 2 is a block diagram showing a schematic configuration of the endoscope system according to the embodiment of the present invention.

  An endoscope system 1 shown in FIGS. 1 and 2 includes an endoscope 2 that captures an in-vivo image of a subject by inserting a tip portion into the subject, and illumination light emitted from the tip of the endoscope 2. A processing device 3 (processor) that has a light source unit 3a that generates a signal, performs predetermined signal processing on the image signal captured by the endoscope 2, and comprehensively controls the operation of the entire endoscope system 1. And a display device 4 for displaying an in-vivo image generated by the signal processing of the processing device 3. For example, a recording medium 5 capable of recording data related to an image captured by the endoscope 2 is electrically connected to the processing device 3. The recording medium 5 is detachably connected to a measuring device 6 that is a device different from the processing device 3 and causes the measuring device 6 to read recorded data. The recording medium 5 is configured by a USB memory or the like that is detachably connected to the processing device 3. In FIG. 2, a solid arrow indicates transmission of an electric signal related to an image, and a broken arrow indicates transmission of an electric signal related to control.

  The endoscope 2 includes an insertion portion 21 having an elongated shape having flexibility, an operation portion 22 that is connected to a proximal end side of the insertion portion 21 and receives input of various operation signals, and an insertion portion from the operation portion 22. And a universal cord 23 that includes various cables that extend in a direction different from the direction in which 21 extends and are connected to the processing device 3 (including the light source unit 3a).

  The insertion unit 21 receives a light and performs photoelectric conversion to generate a signal to generate a signal. The insertion unit 21 includes an image pickup unit 244 in which pixels are arranged in a two-dimensional shape, and a bendable portion formed by a plurality of bending pieces. And a long flexible tube portion 26 connected to the proximal end side of the bending portion 25 and having flexibility. The insertion unit 21 is inserted into the body cavity of the subject, and the imaging unit 244 images a subject such as a living tissue in a position where external light does not reach.

  The tip portion 24 is configured by using a glass fiber or the like, and forms a light guide path for light emitted from the light source portion 3a, an illumination lens 242 provided at the tip of the light guide 241, and a condensing left The optical system 243a for the right eye, the optical system 243b for the right eye, the optical system 243a for the left eye, and the optical system 243b for the right eye are provided at the imaging positions, and the optical system 243a for the left eye and the optical system 243b for the right eye are collected. An imaging unit 244 that receives the emitted light, photoelectrically converts the light into an electrical signal, and performs predetermined signal processing.

  The left-eye optical system 243a is configured by using one or a plurality of lenses, and is provided in front of the imaging unit 244 to form an incident light from a subject. The left-eye optical system 243a may have an optical zoom function that changes the angle of view and a focus function that changes the focus.

  The right-eye optical system 243b is configured using one or a plurality of lenses, and is provided in front of the imaging unit 244 to form incident light from a subject with parallax from the left-eye optical system 243a. To do. The right-eye optical system 243b may have an optical zoom function that changes the angle of view and a focus function that changes the focus.

  The imaging unit 244 includes a left-eye imaging device 244a and a right-eye imaging device 244b.

  The left-eye image pickup device 244a photoelectrically converts light from the left-eye optical system 243a in accordance with the drive signal received from the processing device 3, and outputs an electrical signal for one frame (left-eye use) constituting one image. RAW data). Specifically, the imaging element 244a for the left eye includes a plurality of pixels each having a photodiode that accumulates charges according to the amount of light, a capacitor that converts charges transferred from the photodiodes to voltage levels, and the like in a matrix. Arranged, each pixel photoelectrically converts the light from the left-eye optical system 243a to generate an electrical signal, sequentially reading out the electrical signal generated by a pixel arbitrarily set as a readout target among a plurality of pixels, It outputs as an image signal which is RAW data. For example, a color filter is provided on the light-receiving surface of the left-eye image pickup device 244a, and each pixel has one of the wavelength bands of the color components of red (R), green (G), and blue (B). Receives light in the band.

  The right-eye image pickup device 244b photoelectrically converts light from the right-eye optical system 243b in accordance with the drive signal received from the processing device 3, and outputs an electrical signal (for the right eye) for one frame constituting one image. RAW data). Specifically, the right-eye image pickup device 244b includes a plurality of pixels each having a photodiode that accumulates charges according to the amount of light, a capacitor that converts charges transferred from the photodiodes to voltage levels, and the like in a matrix. Arranged, each pixel photoelectrically converts the light from the right-eye optical system 243b to generate an electrical signal, sequentially reading out the electrical signal generated by a pixel arbitrarily set as a readout target among the plurality of pixels, It outputs as an image signal which is RAW data. For example, a color filter is provided on the light receiving surface of the right-eye image pickup device 244b, and each pixel has a wavelength of any one of the wavelength bands of the red (R), green (G), and blue (B) color components. Receives light in the band.

  The left-eye imaging device 244a and the right-eye imaging device 244b are realized using, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The left-eye image sensor 244a and the right-eye image sensor 244b may each be configured using a single-plate image sensor, for example, using a plurality of image sensors such as a three-plate system. It may be configured.

  The left-eye image obtained by the left-eye image pickup device 244a and the right-eye image obtained by the right-eye image pickup device 244b are images of different fields of view in which a common subject is captured and have parallax. is there.

  The imaging unit 244 according to the present embodiment will be described as having two imaging elements respectively corresponding to the left-eye optical system 243a and the right-eye optical system 243b. An electrical signal for the frame is generated. In the present embodiment, the light imaged by the left-eye optical system 243a and the right-eye optical system 243b is transmitted by two image sensors in accordance with the left-eye optical system 243a and the right-eye optical system 243b. Although described as receiving light, the light receiving region may be divided and received by the same image sensor.

  The operation unit 22 includes a bending knob 221 that bends the bending unit 25 in the vertical direction and the left-right direction, a treatment tool insertion unit 222 that inserts a treatment tool such as a biopsy forceps, an electric knife, and a test probe into the body cavity of the subject. In addition to the processing device 3, it has air supply means, water supply means, and a plurality of switches 223 that are operation input units for inputting operation instruction signals for screen display control by freeze processing or the like. The treatment tool inserted from the treatment tool insertion portion 222 is exposed from the opening (not shown) via the treatment tool channel (not shown) of the distal end portion 24. The plurality of switches 223 are independently provided with a switch for inputting a release instruction (to be described later), a switch for inputting a freeze instruction, and the like.

  The universal cord 23 includes at least a light guide 241 and a collective cable 245 in which one or a plurality of signal lines are collected. The collective cable 245 is a signal line for transmitting an image signal, a signal line for transmitting a drive signal for driving the imaging unit 244, information including unique information regarding the endoscope 2 (imaging unit 244), and the like. Including a signal line for transmitting and receiving. In the present embodiment, the description will be made assuming that an electrical signal is transmitted using a signal line. However, an optical signal may be transmitted, or the endoscope 2 and the processing device 3 may be connected by wireless communication. A signal may be transmitted between them.

  The endoscope 2 includes a memory 224 that records information on the endoscope 2. In the memory 224, identification information indicating the type and model number of the endoscope 2, the types of the left-eye image sensor 244a, the right-eye image sensor 244b, and the like is recorded. The memory 224 includes images for image data captured by the left-eye image sensor 244a and the right-eye image sensor 244b, such as parameters for white balance (WB) adjustment and variation correction values at the time of manufacturing the endoscope 2. Various parameters for processing may be recorded.

  When the endoscope 2 is attached to the processing device 3, the information of the endoscope 2 described above is output to the processing device 3 through communication processing with the processing device 3. Alternatively, the connector is provided with a connection pin in accordance with a rule corresponding to the information of the endoscope 2, and the processing device 3 is configured such that when the endoscope 2 is attached, the connection pin on the processing device 3 side and the connection pin on the endoscope 2 side are connected. The connection of the endoscope 2 may be recognized based on the connection state.

  Next, the configuration of the processing device 3 will be described. The processing device 3 includes a preprocessing unit 31, a freeze processing unit 32, a frame memory 33, a display image generation unit 34, a recording image generation unit 35, an input unit 36, a control unit 37, and a storage unit. 38.

  The preprocessing unit 31 performs black level on the left-eye image data (analog) output from the left-eye image sensor 244a and the right-eye image data (analog) output from the right-eye image sensor 244b. OB clamping processing, white balance correction processing, gain adjustment processing, noise removal processing, A / D conversion processing, and demosaicing processing are performed. The preprocessing unit 31 outputs left-eye image data (digital) including a left-eye image to which RGB color components are added by the signal processing described above, and right-eye image data (digital) including a right-eye image. It is generated and output to the freeze processing unit 32 and the frame memory 33. Hereinafter, the image data for the left eye and the image data for the right eye may be collectively referred to as image data.

  When the freeze processing unit 32 receives an input of a release instruction signal by pressing the switch 223, the freeze processing unit 32 selects an image to be freeze-displayed on the display device 4 from the frame memory 33, and generates a display image generation unit 34 and a recording image generation unit. To the unit 35. When the freeze processing unit 32 receives an input of a freeze instruction signal by pressing the switch 223, the freeze processing unit 32 selects an image to be displayed on the display device 4 from the frame memory 33 and outputs the selected image to the display image generation unit 34. . When the freeze processing unit 32 does not accept the input of the release instruction signal or the freeze instruction signal, the freeze processing unit 32 outputs predetermined image data in the frame memory 33, for example, the latest image data in time series, to the display image generation unit 34. The freeze processing unit 32 performs only the process of outputting the processing signal input from the preprocessing unit 31 to the frame memory 33 within the still image display period by the freeze processing after selecting the image data to be frozen. The freeze processing unit 32 selects predetermined image data from the frame memory 33 including the image data newly input from the preprocessing unit 31 after the freeze processing is canceled. For this reason, the image displayed on the display device 4 after the release process or the freeze process is an image in which the latest image data in the still image display period by the freeze process has been lost and which is in chronological order before the freeze process. Become. For this reason, in the images displayed before and after the freeze process, the change in the subject image may be larger than when moving images of images that are adjacent in time series.

  The freeze processing unit 32 selects, as the best frame, an image stored in the frame memory 33, for example, a left-eye image with less blur from the left-eye image. The freeze processing unit 32 acquires the image for the left eye and the image for the right eye of the selected best frame from the frame memory 33 and outputs them to the display image generation unit 34 and the recording image processing unit 35. Specifically, when the release instruction signal or the freeze instruction signal is input, the freeze processing unit 32 calculates shakes of a plurality of left-eye images stored in the frame memory 33, and based on the calculated shakes. The best frame may be selected, and each time image data is input from the preprocessing unit 31, the blur of the left-eye image corresponding to the image data is calculated, and the calculated blur and the left-eye image are calculated. The frame number may be stored in the frame memory 33 in association with each other. The image blur is calculated using a known calculation method.

  The frame memory 33 stores the image data generated by the preprocessing unit 31 for a set frame. In the present embodiment, the frame memory 33 stores image data for several frames. When new image data is input, the frame memory 33 overwrites the oldest image data among the currently stored image data with new image data, so that several frames are sequentially added from the newest acquisition time. Minute image data is stored while being sequentially updated.

  The display image generation unit 34 generates a parallax image in which parallax is generated by shifting the left-eye image and the right-eye image with respect to the image data selected by the freeze processing unit 32. Image processing for display is generated by performing signal processing so that the signal can be displayed. Specifically, the display image generation unit 34 performs CMS (Color Management System), γ correction, zoom processing, enhancement processing, compression processing, and the like on the image signal including the left-eye image and the right-eye image. Thereafter, the right-eye image and the left-eye image are relatively shifted based on a preset shift amount to generate a display image signal. The CMS is a system for uniformly managing colors between different devices (here, between the endoscope 2 and the display device 4). The display image generation unit 34 generates, as a display image, a composite image in which image information including an endoscopic image corresponding to the image data input from the freeze processing unit 32 and character information related to the image information are superimposed. . The display image generation unit 34 transmits the generated image data of the display image to the display device 4.

Here, the parallax image generated by the display image generation unit 34 will be described with reference to FIG. FIG. 3 is a diagram illustrating a parallax image generated by the endoscope system according to the embodiment of the present invention. Display image generating unit 34, as shown in FIG. 3, the line image D _L horizontal line in the image IM _L for the left eye, and a line image D _R of the horizontal line of the right-eye image IM _R, is set The parallax images IM _D are generated by alternately arranging them while shifting according to the shift amount. Specifically, the display image generation unit 34, a line image D _L of the odd lines with the image IM _L for the left eye, and a line image D _R of the even lines with the right-eye image IM _R, is set They are arranged alternately by shifting according to the shift amount. Such parallax images IM _D is also referred to as line-by-line image. Here, the horizontal line corresponds to a line formed by pixels arranged along one arrangement direction in an imaging device in which a plurality of pixels are arranged in a matrix.

  In addition, when the image data input from the freeze processing unit 32 is a release target or a freeze target, the display image generation unit 34 has a preset period, for example, a period for displaying an image of several frames, The target image is displayed on the display device 4 as a still image.

  The recording image generation unit 35 generates a parallax image in which parallax is generated by shifting the left-eye image and the right-eye image with respect to the image data selected by the freeze processing unit 32, and this is recorded. Image data. Examples of the parallax image include the above-described line-by-line image and a side-by-side image in which the left-eye image and the right-eye image are arranged along the horizontal line direction. Unlike the above-described parallax image generated by the display image generation unit 34, the parallax image generated by the recording image generation unit 35 is an image after the preprocessing by the preprocessing unit 31, and display such as zoom processing. Thus, the image is not subjected to image processing. The recording image generation unit 35 records the generated image data on the recording medium 5 together with parameters such as zoom processing and unique information of the endoscope 2, for example. At this time, the parameters and the unique information of the endoscope 2 may be embedded in the header portion of the line-by-line image or recorded as a text file associated with the image data separately from the image data. You may make it do.

  The pre-processing unit 31, the freeze processing unit 32, the display image generating unit 34, and the recording image generating unit 35 can rewrite a general-purpose processor such as a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), and processing contents. It is configured using a dedicated processor such as various arithmetic circuits that execute specific functions such as a field programmable gate array (FPGA) that is a programmable logic device.

  The input unit 36 is realized using a keyboard, a mouse, a switch, and a touch panel, and receives input of various signals such as an operation instruction signal that instructs the operation of the endoscope system 1. The input unit 36 may include a portable terminal such as a switch provided in the operation unit 22 or an external tablet computer.

  The control unit 37 performs drive control of each component including the imaging unit 244 and the light source unit 3a, and input / output control of information with respect to each component. The control unit 37 refers to control information data (for example, readout timing) for imaging control stored in the storage unit 38, and uses the imaging unit as a drive signal via a predetermined signal line included in the collective cable 245. To 244. The control unit 37 is configured using a general-purpose processor such as a CPU, or a dedicated processor such as various arithmetic circuits that execute specific functions such as an ASIC or FPGA.

  The control unit 37 performs freeze control by the freeze processing unit 32 based on the instruction signal received by the input unit 36.

  The storage unit 38 stores various programs for operating the endoscope system 1 and data including various parameters necessary for the operation of the endoscope system 1. Further, the storage unit 38 stores identification information of the processing device 3. Here, the identification information includes unique information (ID) of the processing device 3, model year, specification information, and the like.

  The storage unit 38 stores various programs including an image acquisition processing program for executing the image acquisition processing method of the processing device 3. Various programs can be recorded on a computer-readable recording medium such as a hard disk, a flash memory, a CD-ROM, a DVD-ROM, or a flexible disk and widely distributed. The various programs described above can also be obtained by downloading via a communication network. The communication network here is realized by, for example, an existing public line network, a LAN (Local Area Network), a WAN (Wide Area Network) or the like, and may be wired or wireless.

  The storage unit 38 having the above configuration is realized by using a ROM (Read Only Memory) in which various programs are installed in advance, a RAM (Random Access Memory) storing a calculation parameter and data of each process, a hard disk, and the like. Is done.

  Next, the configuration of the light source unit 3a will be described. The light source unit 3 a includes an illumination unit 301 and an illumination control unit 302. Under the control of the illumination control unit 302, the illumination unit 301 sequentially switches and emits illumination light with different exposure amounts to the subject (subject). The illumination unit 301 includes a light source 301a and a light source driver 301b.

  The light source 301a is configured using an LED light source that emits white light, one or a plurality of lenses, and the like, and emits light (illumination light) by driving the LED light source. Illumination light generated by the light source 301 a is emitted from the tip of the tip 24 toward the subject via the light guide 241. In the first embodiment, in order to perform NBI observation suitable for observation in the urinary bladder, the light source 301a uses blue narrow band light (for example, 390 nm to 445 nm) and green narrow band light (for example, 530 nm to 550 nm). The narrow band light is emitted as illumination light. The light source 301a is realized using any one of an LED light source, a laser light source, a xenon lamp, a halogen lamp, and the like.

  The light source driver 301b causes the light source 301a to emit illumination light by supplying power to the light source 301a under the control of the illumination control unit 302.

  The illumination control unit 302 controls the amount of power supplied to the light source 301a and the drive timing of the light source 301a based on a control signal (dimming signal) from the control unit 37.

  The display device 4 displays a display image corresponding to the image signal received from the processing device 3 (display image generation unit 34) via the video cable. The display device 4 is configured using a monitor such as liquid crystal or organic EL (Electro Luminescence).

  The measuring device 6 reads data from the recording medium 5, acquires the parallax image generated by the recording image generation unit 35, displays the parallax image, and measures the distance from the endoscope 2 to the subject. The measurement device 6 measures the distance at the measurement point when the user instructs the displayed image via the input unit or the like to be measured. The measurement method can be measured by a known method such as stereo measurement (for example, triangulation).

  Next, image processing performed by the endoscope system 1 will be described. FIG. 4 is a flowchart showing image processing performed by the endoscope system according to the embodiment of the present invention. Hereinafter, description will be made assuming that each unit operates under the control of the control unit 37.

  First, the preprocessing unit 31 receives image data from the endoscope 2 (step S101). When receiving the image data, the preprocessing unit 31 performs the above-described preprocessing on the image data, and outputs the processed image data to the freeze processing unit 32 and the frame memory 33 (step S102).

  In step S <b> 103 following step S <b> 102, the control unit 37 determines whether a release instruction is input via the switch 223. When the control unit 37 determines that there is an input of a release instruction (step S103: Yes), the control unit 37 proceeds to step S104.

  In step S <b> 104, the freeze processing unit 32 selects the best frame from the images stored in the frame memory 33. After selecting the image by the freeze processing unit 32, the control unit 37 proceeds to step S105 and step S106. In step S104, the image of the best frame selected by the freeze processing unit 32 is output to the display image generating unit 34 and the recording image generating unit 35. The images selected in this case are a left-eye image and a right-eye image of the best frame.

  In step 105, the display image generation unit 34 generates display image data. The display image generation unit 34 generates a parallax image subjected to zoom processing or the like based on the left-eye image and the right-eye image of the best frame selected by the freeze processing unit 32.

  In parallel with step S105, the recording image generation unit 35 generates recording image data (step S106). The recording image generation unit 35 generates a parallax image that has not been subjected to zoom processing or the like based on the left-eye image and the right-eye image selected by the freeze processing unit 32.

  When the parallax images are respectively generated in step S105 and step S106, the recording image generation unit 35 performs a recording process on the recording medium 5 (step S107). The recording image generation unit 35 records the parallax image generated in step S106 on the recording medium 5 in association with parameters such as zoom processing and unique information of the endoscope 2. After recording on the recording medium 5, the control unit 37 proceeds to step S113.

  On the other hand, if the control part 37 judges that the release instruction | indication is not input (step S103: No), it will transfer to step S108.

  In step S <b> 108, the control unit 37 determines whether a freeze instruction is input via the switch 223. When the control unit 37 determines that there is an input of a freeze instruction (step S108: Yes), the control unit 37 proceeds to step S109.

  In step S <b> 109, the freeze processing unit 32 selects the best frame from the images stored in the frame memory 33. After selecting the image by the freeze processing unit 32, the control unit 37 proceeds to step S110. In step S <b> 109, the image for the left eye and the image for the right eye of the best frame selected by the freeze processing unit 32 are output to the display image generation unit 34.

  In step S110, the display image generation unit 34 generates display image data. The display image generation unit 34 generates a parallax image subjected to zoom processing or the like based on the left-eye image and the right-eye image of the best frame selected by the freeze processing unit 32. After generating the parallax image, the control unit 37 proceeds to step S113.

  On the other hand, if the control part 37 judges that there is no input of a freeze instruction (step S108: No), it will transfer to step S111.

  In step S <b> 111, the freeze processing unit 32 selects a predetermined frame from the image stored in the frame memory 33. The freeze processing unit 32 selects predetermined image data in the frame memory 33, for example, the latest frame in time series. After selecting the image by the freeze processing unit 32, the control unit 37 proceeds to step S112. In step S <b> 111, the image for the left eye and the image for the right eye of the latest frame selected by the freeze processing unit 32 are output to the display image generation unit 34.

  In step S112, the display image generation unit 34 generates display image data. The display image generation unit 34 generates a parallax image subjected to zoom processing or the like based on the left-eye image and the right-eye image of the latest frame selected by the freeze processing unit 32. After generating the parallax image, the control unit 37 proceeds to step S113.

  In step S113, under the control of the control unit 37, the display image generation unit 34 causes the display device 4 to output a display image including the generated parallax image.

  Note that the recording process in step S107 and the display process in step S113 may be performed first in step S113 or simultaneously.

  According to the embodiment of the present invention described above, a parallax image is generated by performing zoom processing or the like for display on an image selected by a freeze instruction, and is not subjected to zoom processing as an image to be recorded for measurement. A parallax image is generated and recorded on the recording medium 5. As a result, data including a parallax image that is not subjected to interpolation or the like by zoom processing or the like is recorded on the recording medium 5 instead of a parallax image that is subjected to interpolation or the like by zoom processing or the like and causes a decrease in measurement accuracy. It will be. As a result, according to the present embodiment, when the distance measuring process is performed using the data recorded on the recording medium 5 by the measuring device 6 which is a device different from the processing device 3, it is possible to suppress a decrease in measurement accuracy. Can do.

  Further, according to the present embodiment described above, the recording medium 5 records the parallax image generated by the recording image generation unit 35 based on the image data subjected to the signal processing by the preprocessing unit 31. Therefore, the amount of data required for one frame can be reduced as compared with the case where the left-eye RAW data and the right-eye RAW data output from the imaging unit 244 are recorded. When image processing such as zoom processing is performed in the measurement device 6, it is possible to suppress an increase in the amount of data by recording parameters in association with the image data.

  In the above-described embodiment, the image data after the freeze processing by the freeze processing unit 32 is extracted and recorded on the recording medium 5. However, the image data before the freeze processing by the freeze processing unit 32 is recorded. The image may be output to the image generation unit 35. If the image data before the image processing (such as zoom processing) by the display image generation unit 34 is used, it is possible to suppress the above-described decrease in measurement accuracy.

  In the above-described embodiment, the recording image generation unit 35 records the parallax image, the parameters such as the zoom process, and the unique information of the endoscope 2 in association with each other and records them on the recording medium 5. However, data having a parallax image and information of the endoscope 2 recorded in the memory 224, for example, unique information of the endoscope 2, may be used as recording data, or unique information of the endoscope 2 Data that does not include may be used as recording data. If the recording data has the unique information of the endoscope 2, the measurement device 6 can perform a measurement process taking into account variations in the unique information of the endoscope 2.

In the above-described embodiment, the parallax image is a line-by-line image as an example. However, the present invention is not limited to this, and an image having parallax, for example, the left-eye image IM _L and the right-eye image is used. an image IM _R may be a side-by-side images arranged along the direction of the horizontal line. Further, not limited to a single parallax image, for example, a left-eye image and a right-eye image may be alternately output and recorded on the recording medium 5 as in a frame sequential method.

  Further, in the above-described embodiment, the preprocessing unit 31 has been described as performing the OB clamping process, the white balance correction process, the gain adjustment process, the noise removal process, the A / D conversion process, and the demosaicing process. However, the pre-processing unit 31 may perform the correction processing, and the display image generation unit 34 may perform the processing after the gain adjustment processing. In other words, in the above-described embodiment, the image data before the freeze process is the image data after the demosaicing process. However, for example, the image data may have been subjected to the correction process.

  In the above-described embodiment, the light source unit 3a emits narrow band light, and the imaging unit 244 is described on the assumption that it is a simultaneous illumination / imaging method in which reflected light from illumination light is received. The light source unit 3a may sequentially emit narrowband light of each color component individually, and the imaging unit 244 may be a surface sequential illumination / imaging method in which the light of each color component is received.

  In the above-described embodiment, the light source unit 3a has been described as being configured separately from the endoscope 2. However, for example, a light source device such as a semiconductor light source is provided at the distal end of the endoscope 2. The structure provided in the endoscope 2 may be sufficient. Furthermore, the function of the processing device 3 may be given to the endoscope 2.

  Moreover, although the light source part 3a was demonstrated as what was integrated with the processing apparatus 3 in embodiment mentioned above, the light source part 3a and the processing apparatus 3 are separate bodies, for example, it illuminates the exterior of the processing apparatus 3. The part 301 and the illumination control part 302 may be provided. Further, the light source 301 a may be provided at the tip of the tip portion 24.

  In the above-described embodiment, the endoscope system according to the present invention is described as the endoscope system 1 using the flexible endoscope 2 whose observation target is a living tissue or the like in the subject. However, a camera head is attached to the eyepiece of an optical endoscope such as a rigid endoscope, an industrial endoscope that observes material characteristics, a capsule endoscope, a fiberscope, or an optical endoscope. Even an endoscope system using a connected one can be applied.

  In the above-described embodiment, the endoscope system has been described as an example. However, the present invention can also be applied to, for example, outputting an image to an EVF (Electronic View Finder) provided in a digital still camera or the like.

  As described above, the image processing apparatus and the image processing system according to the present invention are useful for suppressing a decrease in measurement accuracy when distance measurement processing is performed using data recorded on a recording medium.

DESCRIPTION OF SYMBOLS 1 Endoscope system 2 Endoscope 3 Processing apparatus 3a Light source part 4 Display apparatus 5 Recording medium 6 Measuring apparatus 21 Insertion part 22 Operation part 23 Universal code 24 Tip part 25 Bending part 26 Flexible pipe part 31 Preprocessing part 32 Freeze Processing unit 33 Frame memory 34 Display image generation unit 35 Recording image generation unit 36 Input unit 37 Control unit 38 Storage unit 301 Illumination unit 302 Illumination control unit

Claims (9)

An image processing device that generates a parallax image based on first and second images obtained by imaging the same subject from different directions,
A display image generation unit that generates a display parallax image by performing at least zoom processing based on the first and second images;
A recording image generation unit that generates a recording image to be recorded on an external recording medium based on the first and second images before the zoom processing by the display image generation unit;
An image processing apparatus comprising: When an instruction to record the parallax image that has been freeze-displayed is received while the parallax image is displayed in a freeze manner, the first subject to be frozen is selected from a plurality of sets each including the first and second images. And a freeze processing unit for selecting the second image,
The image processing apparatus according to claim 1, further comprising: The image processing apparatus according to claim 2, wherein the recording image generation unit generates the recording image based on the first and second images selected by the freeze processing unit. The recording image generation unit records the recording image and information recorded in an endoscope that inputs the parallax image on the external recording medium. Image processing apparatus. A preprocessing unit that performs at least OB clamping processing and white balance processing as preprocessing on the first and second images;
Further comprising
The image processing apparatus according to claim 1, wherein the recording image generation unit generates the recording image based on the first and second images after the preprocessing by the preprocessing unit. The image processing apparatus according to claim 1, wherein the recording image generation unit records image processing parameters relating to the recording image together with the recording image on the external recording medium. The display image generation unit alternately arranges the first and second images in a side-by-side image in which the first and second images are arranged in the horizontal direction of the image sensor or the first and second images for each line in the horizontal direction. The image processing apparatus according to claim 1, wherein the line-by-line image is generated as the parallax image for display. The recording image generating unit alternately arranges the first and second images in a side-by-side image in which the first and second images are arranged in the horizontal direction of the image sensor or the first and second images for each line in the horizontal direction. The image processing apparatus according to claim 1, wherein the line-by-line image is generated as the recording image. An imaging unit that generates image data including first and second images by imaging the same subject from different directions;
Based on the first and second images, at least a zoom process is performed to generate a display parallax image, and the display image generation unit before the zoom process by the display image generation unit. An image processing apparatus having a recording image generating unit that generates a recording image to be recorded on an external recording medium based on the two images;
An image processing system comprising:

Images