JP2015233261A - Imaging apparatus and verification system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of suppressing degradation in retrieval accuracy after an image with a shallow depth is photographed.SOLUTION: The image apparatus includes: an imaging device with a phase difference detection function; communication means for transmitting data required to restore a photographed image to a verifiable state to an external system; and reception means for receiving a collation result obtained by the external system.

Description

  The present invention relates to an imaging apparatus and a collation system.

  Conventionally, when trying to determine the type and name of a subject shown in an image taken by a user, the user himself has to collate a database such as a picture book with the image.

  In order to solve such a problem, a method has been proposed in which a photographed image is transmitted to a server having a database and a main subject is searched (see Patent Document 1). In Patent Literature 1, in order to improve the search accuracy of the main subject, the subject distance is calculated for each predetermined unit constituting one piece of image data, and a portion within a specific distance range is determined as the main subject. Cut this out. Then, an image obtained by cutting out only the main subject is output to a server having a database to search for the main subject.

JP 2009-260800 A

  However, Patent Document 1 uses a means of cutting out within a specific distance range, and when shooting an image with a shallow depth, such as when shooting a flower or the like with a macro lens, the main subject is accurately cut out. There is a problem that the search accuracy is lowered because collation is performed in a state that is not suitable for search or in a state that is not suitable for search.

  Accordingly, an object of the present invention is to provide an imaging apparatus that can reduce a decrease in search accuracy even when an image having a shallow depth is captured.

  In order to solve the above-described problem, an imaging device according to the present invention includes an imaging device having a phase difference detection function and a transmission unit that transmits data necessary for restoring a captured image to a state where the captured image can be collated And receiving means for receiving the collation result obtained by the external system.

  According to the present invention, it is possible to provide an imaging apparatus capable of reducing a decrease in search accuracy even when an image having a shallow depth is captured.

1 is a configuration diagram of a camera according to a preferred embodiment of the present invention. Circuit diagram of imaging device of the present invention Cross-sectional view of the pixel portion of the image sensor of the present invention The top view and sectional drawing of the pixel for imaging of the image pick-up element of this invention The top view and sectional drawing of a focus detection pixel of an image sensor of the present invention The top view and sectional drawing of the other pixel for focus detection of the image sensor of the present invention The main flow figure of the system of a 1st embodiment of the present invention. The flowchart of the external processing subroutine by the external arithmetic unit 131 of the 1st Embodiment of this invention Figure showing a display example of the results The main flow figure of the system of a 1st embodiment of the present invention. The flowchart of the external processing subroutine by the external arithmetic unit 131 of the 1st Embodiment of this invention

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram of a camera according to a preferred embodiment of the present invention, and shows an image pickup apparatus 100 in which a camera body having an image pickup device as an image pickup means and a photographing optical system are integrated. In the figure, reference numeral 101 denotes a first lens group disposed at the tip of a photographing optical system (imaging optical system), which is held so as to be movable back and forth in the optical axis direction. Reference numeral 102 denotes an aperture / shutter, which adjusts the light amount at the time of shooting by adjusting the aperture diameter, and also has a function as an exposure time adjustment shutter at the time of still image shooting. Reference numeral 103 denotes a second lens group. The diaphragm / shutter 102 and the second lens group 103 integrally move forward and backward in the optical axis direction, and perform a zooming function (zoom function) in conjunction with the forward and backward movement of the first lens group 101. Reference numeral 104 denotes a third lens group that performs focus adjustment by advancing and retreating in the optical axis direction.

  Reference numeral 105 denotes an optical low-pass filter, which is an optical element for reducing false colors and moire in a captured image. Reference numeral 106 denotes an image pickup device composed of a C-MOS sensor and its peripheral circuits. The image sensor is a two-dimensional single-plate color sensor in which a Bayer array primary color mosaic filter is formed on-chip on light receiving pixels of m pixels in the horizontal direction and n pixels in the vertical direction.

  Reference numeral 111 denotes a zoom actuator, which rotates a cam cylinder (not shown) to drive the first lens group 101 to the third lens group 103 forward and backward in the optical axis direction, thereby performing a zooming operation. Reference numeral 112 denotes an aperture shutter actuator that controls the aperture diameter of the aperture / shutter 102 to adjust the amount of photographing light, and controls the exposure time during still image photographing. Reference numeral 113 denotes a focus actuator that performs focus adjustment by driving the third lens group 104 forward and backward in the optical axis direction.

  Reference numeral 121 denotes an in-camera CPU that controls various controls of the camera body, and includes a calculation unit, ROM, RAM, A / D converter, D / A converter, communication interface circuit, and the like. The CPU 121 also drives various circuits included in the camera based on a predetermined program stored in the ROM, and executes a series of operations such as AF, photographing, image processing, and recording.

  Reference numeral 122 denotes an image sensor driving circuit that controls the imaging operation of the image sensor 106 and A / D-converts the acquired image signal and transmits it to the CPU 121. An image processing circuit 123 performs processing such as γ conversion, color interpolation, and JPEG compression of the image acquired by the image sensor 106.

  A focus driving circuit 124 controls the focus actuator 113 based on the focus detection result, and adjusts the focus by driving the third lens group 104 back and forth in the optical axis direction. An aperture shutter drive circuit 125 controls the aperture shutter actuator 112 and controls the aperture of the aperture / shutter 102. A zoom driving circuit 126 drives the zoom actuator 111 according to the zoom operation of the photographer.

  Reference numeral 127 denotes a display device such as an LCD, which displays information related to the shooting mode of the camera, a preview image before shooting and a confirmation image after shooting, a focus state display image at the time of focus detection, and the like. A group of operation switches 128 includes a power switch, a release (shooting trigger) switch, a zoom operation switch, a shooting mode selection switch, and the like. Reference numeral 129 denotes a detachable flash memory that records a photographed image. A communication unit 130 transmits / receives data to / from an external arithmetic unit 131 (not shown) described later. The communication unit 130 functions as a transmission unit and a reception unit in the claims.

  FIG. 2 is a schematic circuit diagram of the image pickup device of the present invention, and the technique disclosed in Japanese Patent Application Laid-Open No. 09-046596 by the applicant of the present application is suitable. The figure shows a range of 2 columns × 4 rows of pixels of a two-dimensional C-MOS area sensor. However, when used as an image sensor, a large number of pixels shown in FIG. Acquisition is possible. In the present embodiment, the description will be made on the assumption that the pixel pitch is 2 μm, the number of effective pixels is 3000 horizontal pixels × 2000 vertical rows = 6 million pixels, and the imaging screen size is 6 mm horizontal × 4 mm vertical.

  In FIG. 2, 1 is a photoelectric conversion portion of a photoelectric conversion element comprising a MOS transistor gate and a depletion layer under the gate, 2 is a photogate, 3 is a transfer switch MOS transistor, 4 is a reset MOS transistor, and 5 is a source follower amplifier MOS. Transistor, 6 is a horizontal selection switch MOS transistor, 7 is a source follower load MOS transistor, 8 is a dark output transfer MOS transistor, 9 is a light output transfer MOS transistor, 10 is a dark output storage capacitor CTN, and 11 is a light output storage capacitor CTS , 12 are horizontal transfer MOS transistors, 13 is a horizontal output line reset MOS transistor, 14 is a differential output amplifier, 15 is a horizontal scanning circuit, and 16 is a vertical scanning circuit.

  FIG. 3 shows a cross-sectional view of the pixel portion. In this figure, 17 is a P-type well, 18 is a gate oxide film, 19 is a first-layer poly-Si, 20 is a second-layer poly-Si, and 21 is an n + floating diffusion portion (FD). The FD 21 is connected to another photoelectric conversion unit via another transfer MOS transistor. In the figure, the drains of the two transfer MOS transistors 3 and the FD portion 21 are made common to improve the sensitivity by miniaturization and the capacity reduction of the FD portion 21, but the FD portion 21 may be connected by Al wiring. .

  4 to 5 are diagrams illustrating the structures of the imaging pixels and the focus detection pixels. In the present embodiment, out of 4 pixels of 2 rows × 2 columns, pixels having G (green) spectral sensitivity are arranged in 2 diagonal pixels, and R (red) and B (blue) are arranged in the other 2 pixels. A Bayer arrangement in which one pixel each having a spectral sensitivity of 1 is arranged is employed. The focus detection pixels are distributed and arranged in a predetermined rule between the Bayer arrays.

  FIG. 4 shows the arrangement and structure of the imaging pixels. FIG. 2A is a plan view of 2 × 2 imaging pixels. As is well known, in the Bayer array, G pixels are arranged diagonally, and R and B pixels are arranged in the other two pixels. The structure of 2 rows × 2 columns is repeatedly arranged.

  A cross section AA of FIG. 4A is shown in FIG. ML is an on-chip microlens disposed on the forefront of each pixel, CFR is an R (Red) color filter, and CFG is a G (Green) color filter. PD is a schematic diagram of the photoelectric conversion unit of the C-MOS sensor described with reference to FIG. 3, and CL is a wiring layer for forming signal lines for transmitting various signals in the C-MOS sensor. TL schematically shows the photographing optical system.

  Here, the on-chip microlens ML and the photoelectric conversion unit PD of the imaging pixel are configured to capture the light beam that has passed through the photographing optical system ML as effectively as possible. In other words, the exit pupil EP of the photographing optical system TL and the photoelectric conversion unit PD are conjugated with each other by the microlens ML, and the effective area of the photoelectric conversion unit is designed to be large. Further, in FIG. 5B, the incident light beam of the R pixel has been described, but the G pixel and the B (Blue) pixel have the same structure. Accordingly, the exit pupil EP corresponding to each of the RGB pixels for imaging has a large diameter, and the S / N of the image signal is improved by efficiently capturing the light flux from the subject.

  FIG. 5 shows the arrangement and structure of focus detection pixels for performing pupil division in the horizontal direction (lateral direction) of the photographic lens. FIG. 5A is a plan view of 2 × 2 pixels including focus detection pixels. When obtaining an imaging signal, the G pixel is a main component of luminance information. Since human image recognition characteristics are sensitive to luminance information, image quality degradation is likely to be recognized if G pixels are lost. On the other hand, the R or B pixel is a pixel that acquires color information. However, since humans are insensitive to color information, pixels that acquire color information are less likely to notice deterioration in image quality even if some loss occurs. Therefore, in the present embodiment, among the pixels of 2 rows × 2 columns, the G pixel is left as an imaging pixel, and the R and B pixels are used as focus detection pixels. This is indicated by SHA and SHB in FIG.

  A cross section AA of FIG. 4A is shown in FIG. The microlens ML and the photoelectric conversion unit PD have the same structure as the imaging pixel shown in FIG. In the present embodiment, since the signal of the focus detection pixel is not used for image creation, a transparent film CFW (White) is arranged instead of the color separation color filter. Moreover, since pupil division is performed by the image sensor, the opening of the wiring layer CL is biased in one direction with respect to the center line of the microlens ML. Specifically, since the opening SHA and the pixel SHA are biased to the right side, the light beam that has passed through the left exit pupil EPHA of the photographic lens TL is received. Similarly, since the opening OPHB of the pixel SHB is biased to the left side, the light beam that has passed through the right exit pupil EPHB of the photographic lens TL is received. Therefore, the pixels SHA are regularly arranged in the horizontal direction, and the subject image acquired by these pixel groups is defined as an A image. In addition, the pixels SHB are also regularly arranged in the horizontal direction, and the subject image acquired by these pixel groups is a B image. By detecting the relative position of the A image and the B image, the amount of defocus of the photographing lens 137 is detected. (Defocus value) can be detected.

  Here, the microlens ML is a pair of optical images of an A image composed of a light beam transmitted through the left exit pupil EPHA of the photographing lens TL and a B image composed of a light beam transmitted through the right exit pupil EPHAB of the photographing lens TL. Plays the function of a lens element that generates

  In the pixels SHA and SHB, focus detection is possible for an object having a luminance distribution in the horizontal direction of the photographing screen, for example, a vertical line, but focus detection is not possible for a horizontal line having a luminance distribution in the vertical direction. Therefore, in the present embodiment, pixels that perform pupil division are also provided in the vertical direction (longitudinal direction) of the shooting screen so that the focus can be detected in the latter case.

  FIG. 6 shows the arrangement and structure of focus detection pixels for performing pupil division in the vertical direction of the imaging screen. FIG. 6A is a plan view of pixels of 2 rows × 2 columns including focus detection pixels. As in FIG. 6A, the G pixel is left as an imaging pixel, and the R and B pixels are focused. Detection pixels are used. This is indicated by SVC and SVD in FIG.

  FIG. 6B shows a cross section A-A in FIG. 6A. The pixel in FIG. 5B has a structure in which the pupil is separated in the horizontal direction, whereas the pixel in FIG. The pupil separation direction is just the vertical direction, and the pixel structure does not change. That is, since the opening OPVC of the pixel SVC is biased downward, the light beam that has passed through the exit pupil EPVC on the upper side of the photographing lens TL is received. Similarly, since the opening OPVD of the pixel SVD is biased upward, the light beam that has passed through the lower exit pupil EPVD of the photographing lens TL is received. Therefore, the pixels SVC are regularly arranged in the vertical direction, and the subject image acquired by these pixel groups is defined as a C image. The pixels SVD are also regularly arranged in the vertical direction, and if the subject image acquired by these pixel groups is a D image, the subject having a luminance distribution in the vertical direction is detected by detecting the relative position of the C image and the D image. The amount of defocus (defocus value) of the image can be detected.

  7 and 8 are flowcharts showing the system according to the first embodiment of this invention. FIG. 7 is a main flow diagram of the system according to the first embodiment of this invention.

  In step S001, it is determined whether or not the release switch is pressed. The user sets the composition and shooting conditions (such as shutter speed and aperture value) until the release switch is pressed. At this time, it is not necessary to consider whether or not the condition is suitable for the collation process performed in the subsequent process (that is, the depth is sufficiently deep). When the release switch is pressed, the process proceeds to step S002.

  In step S002, an image is acquired by the image sensor 106. The acquired image signal is A / D converted by the image sensor driving circuit 122 and then transmitted to the CPU 121. The image signal transmitted to the CPU 121 is temporarily recorded as raw data, while the image processed by the image processing circuit 123 is sent to the flash memory 129 in order to save the captured image as a still image. . Thereafter, the process proceeds to step S003.

  In step S003, it is confirmed whether or not the shooting mode is the collation mode. If the shooting mode is the collation mode, the process proceeds to step S004. If the shooting mode is not the collation mode, the process proceeds to step S010, and this flow ends.

  In step S 004, data is transmitted to the external arithmetic device 131 by the communication unit 130. The external computing device 131 corresponds to the external system in the claims. The data to be transmitted includes raw data temporarily recorded in step S003, data necessary for defocus value calculation processing, blur repair processing, and database collation processing processed by the external arithmetic unit 131, such as model information of the photographing optical system. And zoom information, focus information, aperture value, and the like. When the data transmission is completed, the process proceeds to step S005.

  In step S005, the external processing device 131 executes an external processing subroutine. When the external processing subroutine is completed, the process proceeds to step S006.

  In step S006, the communication unit 130 receives the result transmitted from the external computing device 131. When the reception of the result is completed, the process proceeds to step S007.

  In step S 007, the result is displayed on the display device 127 based on the result received by the communication unit 130. FIG. 9 is a display example of the result. If a result is displayed, it will progress to Step S010 and this flow will be completed.

  FIG. 8 is a flowchart of an external processing subroutine by the external arithmetic unit 131 according to the first embodiment of this invention.

  In step S101, data transmitted from the communication unit 130 of the imaging apparatus 100 is received. When the reception of data is completed, the process proceeds to step S102.

  In step S102, the defocus value in each divided area is calculated. When the calculation is completed, the process proceeds to step S103.

  In step S <b> 103, blur correction processing is performed by the external arithmetic device 131. In general, blur restoration is performed by deconvolution of a point spread function into an image signal. The point spread function is derived from zoom information, focus information, aperture value, defocus value, image height, and the like, depending on the model of the photographing optical system. Therefore, the blur restoration is a point spread function derived according to the image height and the defocus value of each area of the image using the model information, zoom information, focus information, and aperture value of the photographing optical system received in step S101. Is deconvolved into the image received in step S101. When the blur is repaired to such an extent that the database collation processing performed on the main subject in the next step 104 is possible, the process proceeds to step S104.

  In step S104, collation processing is performed by the external arithmetic device 131. The collation process is performed by a known technique (for example, Japanese Patent Laid-Open No. 2000-113097), and the external arithmetic device 131 is connected to a device that stores a database (not shown) and collates with the blurred repair image signal created in step S103. Processing is performed. If a match between the database and the image signal is detected, the process proceeds to step S105.

  In step S105, the result detected in step S104 is transmitted from the external arithmetic unit 131 to the communication unit 130 of the imaging apparatus 100 of the present invention. When the transmission of the result is completed, the process proceeds to step S106, and this subroutine ends.

  Further, in this embodiment, the collation process is performed from the image information of the captured still image. However, in the case of live view shooting, the same process is performed using the image information obtained from the image sensor 106 during the live view. And the results can be displayed during live view.

  In the present embodiment, an example in which the model information of the photographing optical system and the point spread function derived from the zoom information, the focus information, the aperture value, the defocus value, and the image height are used as the blur restoration process is shown. The repair process does not depend on this method. For example, blur correction may be performed by using a function derived by using some parameters of zoom information, focus information, aperture value, defocus value, and image height for a preset basic function.

  As described above, according to the configuration of the present invention, it is possible to provide an imaging apparatus capable of reducing a decrease in search accuracy even when an image having a shallow depth is captured.

  In the first embodiment, the defocus value is calculated by the external arithmetic device 131. However, when the performance of the CPU 121 is sufficiently high, the defocus value may be calculated in the imaging device 100. The second embodiment will be described with reference to FIGS. 10 and 11. Since the configuration of the imaging apparatus 100 is the same as that of the first embodiment, description thereof is omitted, and only the system flow is described.

  10 and 11 are flowcharts showing the system according to the second embodiment of this invention. FIG. 10 is a main flow diagram of a system according to the second embodiment of this invention.

  In step S201, it is determined whether or not the release switch is pressed. The user sets the composition and shooting conditions (such as shutter speed and aperture value) until the release switch is pressed. At this time, it is not necessary to consider whether or not the condition is suitable for the collation process performed in the subsequent process (that is, the depth is sufficiently deep). When the release switch is pressed, the process proceeds to step S202.

  In step S202, an image is acquired by the image sensor 106. The acquired image signal is A / D converted by the image sensor driving circuit 122 and then transmitted to the CPU 121. The image signal transmitted to the CPU 121 is temporarily recorded as raw data, while the image processed by the image processing circuit 123 is sent to the flash memory 129 in order to save the captured image as a still image. . Thereafter, the process proceeds to step S203.

  In step S203, it is confirmed whether or not the shooting mode is the collation mode. If the shooting mode is the collation mode, the process proceeds to step S204. If the shooting mode is not the collation mode, the process proceeds to step S209, and this flow ends.

  In step S204, the CPU 121 calculates a defocus value in each divided area. When the calculation is completed, the process proceeds to step S205.

  In step S <b> 205, data is transmitted to the external arithmetic device 131 by the communication unit 130. The external computing device 131 corresponds to the external system in the claims. The data to be transmitted includes the raw data temporarily recorded in step S203, the defocus value calculated in step S204, and data necessary for blur repair processing and database collation processing processed by the external arithmetic device 131, for example, Zoom information, focus information, aperture value, and the like. When the data transmission is completed, the process proceeds to step S206.

  In step S206, an external processing subroutine is executed by the external arithmetic unit 131. When the external processing subroutine is completed, the process proceeds to step S207.

  In step S207, the communication unit 130 receives the result transmitted from the external computing device 131. When the reception of the result is completed, the process proceeds to step S208.

  In step S208, the result is displayed on the display 127 based on the result received by the communication means 130. If a result is displayed, it will progress to Step S209 and this flow will be completed.

  FIG. 11 is a flowchart of an external processing subroutine by the external arithmetic unit 131 according to the second embodiment of the present invention.

  In step S301, data transmitted from the communication unit 130 of the imaging apparatus 100 is received. When the reception of data is completed, the process proceeds to step S302.

  In step S <b> 302, blur correction processing is performed by the external arithmetic device 131. In general, blur restoration is performed by deconvolution of a point spread function into an image signal. The point spread function is derived from zoom information, focus information, aperture value, defocus value, image height, and the like, depending on the model of the photographing optical system. Therefore, the blur restoration is a point spread function derived in accordance with the image height and defocus value of each region of the image using the model information, zoom information, focus information, and aperture value of the photographing optical system received in step S301. Is deconvolved into the image received in step S301. When the blur of the main subject is repaired to the extent that the database collation process performed in the next step 303 is possible, the process proceeds to step S303.

  In step S303, collation processing is performed by the external arithmetic device 131. The collation process is performed by a known technique (for example, Japanese Patent Laid-Open No. 2000-113097), and the external arithmetic device 131 is connected to a device that stores a database (not shown) and collates with the blurred repair image signal created in step S103. Processing is performed. If a match between the database and the image signal is detected, the process proceeds to step S304.

  In step S304, the result detected in step S303 is transmitted from the external arithmetic unit 131 to the communication unit 130 of the imaging apparatus 100 of the present invention. When the transmission of the result is completed, the process proceeds to step S305, and this subroutine ends.

  Further, in this embodiment, the collation process is performed from the image information of the captured still image. However, in the case of live view shooting, the same process is performed using the image information obtained from the image sensor 106 during the live view. And the results can be displayed during live view.

  In the present embodiment, an example in which the model information of the photographing optical system and the point spread function derived from the zoom information, the focus information, the aperture value, the defocus value, and the image height are used as the blur restoration process is shown. The repair process does not depend on this method. For example, blur correction may be performed by using a function derived by using some parameters of zoom information, focus information, aperture value, defocus value, and image height for a preset basic function.

  As described above, according to the configuration of the present invention, it is possible to provide an imaging apparatus capable of reducing a decrease in search accuracy even when an image having a shallow depth is captured.

100 imaging device, 106 imaging device, 130 communication means,
131 External arithmetic unit

Claims (6)

An image sensor having a phase difference detection function;
Transmission means for transmitting data necessary to determine the type and name of the main subject of the captured image to an external system;
An image pickup apparatus comprising: a receiving unit that receives a collation result obtained by an external system. An optical system having a zoom function, a focus function, and an aperture function,
2. The imaging apparatus according to claim 1, wherein the data includes at least one of model information, zoom information, focus information, and aperture information of the optical system, and image information obtained by the imaging element. . The imaging apparatus according to claim 1 or 2, wherein the external system performs at least one of a defocus value calculation, a blurred image restoration process, and a collation process between an image and a database. Matching system. 2. The imaging apparatus according to claim 1, further comprising means for calculating a defocus value for each area based on a signal obtained by the imaging element. An optical system having a zoom function, a focus function, and an aperture function, and the data is obtained by the image sensor and at least one of model information of the optical system, zoom information, focus information, and aperture information. The image pickup apparatus according to claim 4, wherein the image information is the defocus value. 6. A collation system comprising the imaging apparatus according to claim 4 or 5, wherein the external system performs at least one of a blurred image restoration process and a collation process between an image and a database.