JP6083946B2 - Image processing apparatus and image processing apparatus control method - Google Patents

Description

  The present invention relates to an image processing apparatus that performs a peaking assist function for edge enhancement for focus adjustment, and a control method for the image processing apparatus.

  Conventional video cameras have a focus assist function for adding a peaking signal to an image in order to facilitate focus adjustment. The peaking signal is a signal generated by extracting and amplifying a high frequency component from the video signal. In Patent Document 1, a peaking signal is added to a video signal and displayed on a viewfinder or an output monitor so that a contour portion in the video is highlighted and the focus adjustment is performed by visually informing the user of the degree of focus. Can be easily performed.

JP 2009-231918 A

  However, Patent Document 1 employs a configuration in which enlargement / reduction is performed after the peaking signal generation unit in accordance with the size of the display device. Therefore, when enlargement processing is performed at the time of focus assist, the number of pixels to be peaked is small and accurate. There was a problem that peaking information could not be obtained.

  In view of the above problems, an object of the present invention is to provide an image processing apparatus and a control method for the image processing apparatus that can output smooth display video to a display device while generating peaking information more accurately. And

In order to solve the above problems, an image processing apparatus according to the present invention includes an interpolation unit that performs an interpolation process on an input image signal according to an enlargement ratio, and an image signal that has been subjected to the interpolation process by the interpolation unit. A peaking signal generating means for generating a peaking signal having a level corresponding to a change amount of a signal level in a predetermined direction in the image signal, and an image signal subjected to interpolation processing by the interpolation means are generated by the peaking signal generating means. the peaking signal are superimposed that, and display control means for displaying on the Viewing device has, the interpolation means comprises: a first interpolation processing on the image signal said input, said first interpolated It is possible to perform a second interpolation process that leaves a high-frequency component of the image signal to be processed as compared with the process, and the display control means performs the first interpolation process by the interpolation means. The obtained image signal, characterized by superimposing the peaking signal generated from an image signal obtained by performing a second interpolation processing in the interpolation means.

Further, the control method of the image processing apparatus according to the present invention includes an interpolation step in which an input image signal is subjected to an interpolation process in accordance with an enlargement magnification, and an image signal in which the interpolation process has been performed in the interpolation step. A peaking signal generating step for generating a peaking signal having a level corresponding to a change amount of the signal level in a predetermined direction, and the peaking generated in the peaking signal generating step on the image signal subjected to the interpolation processing in the interpolation step by superimposing the signals, and a display control step of displaying on the Viewing device has, the interpolation step includes a first interpolation processing on the image signal said input, compared to the first interpolation processing The second interpolation process that leaves the high-frequency component of the image signal to be processed can be performed, and in the display control step, the interpolation is performed. A peaking signal generated from the image signal obtained by performing the second interpolation process in the interpolation step is superimposed on the image signal obtained by performing the first interpolation process in step. .

  According to the present invention, it is possible to output a smooth display image to a display device while generating peaking information more accurately.

1 is a block diagram illustrating a configuration of an imaging apparatus to which the present invention is applied in a first embodiment. It is the schematic of a focus assist function. It is the schematic of a peaking signal generation circuit. It is the figure which showed the relationship between the input and output of a peaking production | generation circuit. It is the figure which showed the relationship between the input and output of a peaking production | generation circuit by the linear interpolation process at the time of expansion. It is the figure which showed the relationship between the input and output of a peaking production | generation circuit by the nearest neighbor interpolation process at the time of expansion. In 2nd Embodiment, it is a block diagram which shows the structure of the imaging device to which this invention is applied. It is a follow chart of a 1st embodiment. It is a follow chart of 2nd Embodiment.

(First embodiment)
FIG. 1 is a diagram illustrating a block of the entire system including a digital camera which is an imaging apparatus as an example of an image processing apparatus in the present embodiment. Note that part or all of each unit may be realized by hardware, or may be realized by software by firmware.

  Light from the subject is incident on the image sensor 101 via the optical system 100. In this embodiment, the image sensor 101 is a single plate and has an R, G, B Bayer array, but is not limited thereto. The optical system 100 includes a zoom lens capable of zoom driving and focus driving, a lens group including a focus lens, a lens holding member, a shutter, a diaphragm, and a driving device for driving them. The signal subjected to AD conversion by the image sensor 101 is subjected to camera signal processing by the digital signal processing unit 102. Camera signal processing includes synchronization processing, white balance processing, aperture correction processing, peripheral light amount drop correction processing, unsharp mask processing, color space conversion processing, gamma processing, and the like. In the present embodiment, the signal format of the image output from the digital signal processing unit 102 is in accordance with a preset format, such as an R, G, B three-plate format image signal or an R, G, B Bayer array. At least one image signal, Y, Cr, or Cb image signal format is selectively output.

  The frame memory 103 buffers the video output from the digital signal processing unit 102, and can cope with shooting sequences having different frame rates between the camera recording unit 106 and the display block 117 in the next stage. The video recording drive unit 104 converts the signal into a format suitable for the recording medium 105 and records the video on the recording medium 105.

  The display block 117 is a processing block for displaying an image on the display device 113.

  In the conventional general configuration, the interpolation processing for the display device is performed after the signal processing for the display device and the peaking signal are generated. However, in the present embodiment, the preceding stage of both blocks is set so that the image size (number of pixels) of the image signal input to the monitor signal processing unit 110 and the peaking signal generation unit 114 is constant corresponding to the display device 113. Is provided with a scaling circuit 107.

  The scaling circuit 107 is transmitted from the control unit 116 based on the image size of the recording image set in advance by the user via an interface (image size instruction means) (not shown), the scaling factor, and the number of pixels of the display device 113. The input image signal is enlarged and reduced. The scaling circuit 107 includes a first interpolation processing circuit 108 and a second interpolation processing circuit 109, and performs enlargement / reduction processing of the image signal output to the monitor signal processing circuit 110 and the peaking signal generation unit 114, respectively. The difference between the first interpolation processing by the first interpolation processing circuit 108 and the second interpolation processing by the second interpolation processing unit 109 will be described later.

  In the monitor signal processing unit 110, processing such as gamma conversion for the display device 113 is performed, and the image is converted into an image suitable for display on the display device 113. Thereafter, when the focus assist function is OFF, the SW 115 is turned OFF based on the control signal 121 from the display control unit 116. At this time, the signal output of the monitor signal processing unit 110 is directly input to the display element driving unit 112, converted into a signal format suitable for the display device 113 by the display element driving unit 112, and then output to the display device 113. When the focus assist function is ON, the SW 115 is turned ON based on the control signal 121 from the display control unit 116. At this time, the peaking signal generated by the peaking signal generation unit 114 is superimposed on the output from the monitor signal processing circuit 110 by the adder 111 and converted into a signal format suitable for the display device 113 by the display element driving unit 112. Thereafter, the data is output to the display device 113. Here, the display device may be a viewfinder built in the digital camera or an external normal monitor output. Or the structure provided with both is also considered.

  The control unit 200 controls the operation of each unit described above and performs calculations necessary for the processing of each unit. The control unit 200 realizes focusing by driving a focus lens included in the optical system 100.

  Next, an outline of the focus assist function will be described. FIG. 2 is a diagram showing an outline of the focus assist function. FIG. 2A shows the entire image stored in the frame memory 103. When the focus assist function is OFF, a signal having the same angle of view is input to the recording medium 105 and the display device 113. The entire image of FIG. 2A is displayed on the display device 113. Here, when the user wants to focus on a part of the subject, for example, when the range 201 is selected by turning on the focus assist function, the predetermined range is expanded as shown in FIG. Displayed on the display device 113. At this time, as indicated by 202, by displaying an image with a peaking signal (edge information) added to the subject, it is possible to visually inform the user of the degree of focus.

  FIG. 3 is a schematic diagram of the peaking signal generation unit 114. The peaking signal generation circuit is configured based on an FIR (Finite Impulse Response) Filter 302 and receives an input image signal Y_in 301 from the second interpolation processing circuit 109. In this embodiment, the luminance signal (Y signal) is inputted at the time of input to the FIFilter 302, but it may be an R, G, B signal. The gain adjustment signal 303 and the frequency adjustment signal 304 can be used to adjust the edge signal intensity and frequency. Although the peaking signal generation circuit is a known technique and will not be described in detail here, the image signal of the signal intensity and frequency set by the gain adjustment signal 303 and the frequency adjustment signal 304 is extracted by the FIRF filter 302, and the peaking signal and To do. The relationship between the input signal Y_in 301 and the output signal out305 will be described with reference to FIG.

  FIG. 4A shows a graph of the luminance signal at each x-coordinate with the horizontal coordinate of the image data as shown in FIG. 2 as the x-axis. The horizontal axis 402 represents the x coordinate, and the vertical axis 401 represents the luminance level. FIG. 4B shows the distribution of the output signal 305, that is, the peaking signal in the x-axis direction on the same horizontal line as in FIG. 4A, and the horizontal axis 404 is the same as the horizontal axis 402 in FIG. Correspond. Various methods for generating a peaking signal have been proposed, but the simplest outline thereof is shown in FIG. 4 in which “a change amount of an input signal, that is, a differential value is defined as a peaking level”. Reference numerals 410, 411, and 412 indicate peaking signal levels. The larger the amount of change in the signal in FIG. 4A, the larger the value of the peaking signal in FIG. 4B.

  Next, problems in enlarged display during focus assist and the contents of the present invention will be described. FIG. 5 shows the relationship between the luminance signal and the peaking signal when linear interpolation, which is representative of smooth interpolation at the time of enlargement interpolation, is used when the focus assist function is ON and the interpolation processing in the scaling circuit 107 is enlargement magnification. Show.

  FIG. 5A shows the luminance of each pixel when the signal of FIG. 4A is linearly interpolated twice. A circle indicates the original input signal 501 to the second interpolation processing circuit 109, and a square indicates the interpolation pixel 502 by the interpolation processing of the second interpolation processing 109. FIG. 5B shows a distribution of peaking signals generated by inputting these signals to the peaking signal generation unit 114. Also in FIG. 5, as in FIG. 4, the correspondence between the input luminance level and the peaking signal generated by the peaking signal generator is taken on the time axis.

  Since the interpolated pixels are created by linear interpolation, the image projected on the display device 113 has a smooth and beautiful image quality. On the other hand, as indicated by 510, 511, and 512 in FIG. 5B, the level of the peaking signal is about half that of 410, 411, and 412 in the case of FIG. This causes a problem that it is difficult to generate an effective peaking signal. Therefore, as a countermeasure for this problem, interpolation processing is performed as follows. The input signal to the monitor signal processing unit 110 that processes the image signal itself displayed on the display device 113 is smoothed by the first interpolation processing circuit 108 such as the first interpolation processing, that is, cubic interpolation processing or linear interpolation processing. Interpolation processing that produces a clear image. On the other hand, the edge information is left in the input signal to the peaking signal generation unit 114 that generates the peaking signal by the second interpolation processing circuit 109, that is, the nearest neighbor interpolation processing (nearest neighbor method). Send a signal in the state. FIG. 6 shows an input signal and an output signal of the peaking signal generator by nearest neighbor interpolation. A circle indicates the original signal 601 and a square indicates the interpolation pixel 602. Since the edge information is not lost even after interpolation because of the nearest neighbor interpolation processing, the peaking signal levels 610, 611, and 612 are equivalent to 410, 411, and 412.

  FIG. 8 is a flowchart showing the operation of image display processing on the display device by the image processing apparatus at the time of shooting. The following processing is executed in each unit according to instructions from the display control unit 116 and the display control unit 116. First, in S1000, it is determined whether the setting of the focus assist function set in advance by the user is ON or OFF. In the case of OFF, the scaling magnification is determined according to the recording image size set in advance by the user and the number of pixels of the display device 113 (S1004), and the output image signal that has undergone the above-described processing is output to the display device 113. (S1005). When ON, the scaling magnification is determined according to the recording image size, the number of pixels of the display device 113 and the focus assist magnification (S1001), and different interpolation processing is performed between the first interpolation processing and the second interpolation processing ( S1002). Thereafter, the peaking information generated by the peaking generation circuit 114 is superimposed (S1003) and output to the display device 113.

  As described above, in the present embodiment, when peaking display for confirming the edge portion of an image is performed on an enlarged image, smooth interpolation processing is performed on the display image, and the first image is generated on the image for generating the peaking signal. The enlargement process is performed by an interpolation process that leaves a high-frequency component (less likely to be lost) than the interpolation process. This makes it possible to output a smooth display image to the display device while generating peaking information more accurately.

  In the present embodiment, the embodiment related to the display of peaking information for confirming the degree of focus at the time of focus adjustment has been described. However, the present invention is not limited to this, and may be a peaking display accompanied by an image enlargement process. For example, the method of using it does not matter.

  In the present embodiment, the first interpolation process is a linear interpolation process and the second interpolation process is a nearest neighbor interpolation process, but the combination of interpolation processes is not limited to this. In other words, the second interpolation process may be an interpolation process in which edge information (high-frequency component information) is more likely to remain than the first interpolation process. For example, when the first interpolation process is a bicubic interpolation process and the second interpolation process is a nearest neighbor interpolation process, the first interpolation process is a linear interpolation process, and the second interpolation process is a bicubic interpolation process. Cases can be considered.

  In the present embodiment, the first interpolation processing circuit 108 is placed before the monitor signal processing unit 110, so that the image size (number of pixels) of the image signal input to the monitor signal processing unit 110 is displayed on the display device 113. It was configured to be consistent. As a result, the monitor signal processing unit 110 has a simple configuration. However, the present invention is not limited to this, and the object and effect of the present invention can also be achieved by a configuration in which the first interpolation processing circuit 108 is placed after the monitor signal processing unit 110.

(Second Embodiment)
FIG. 7 is a block diagram of a digital camera that is an imaging apparatus as an example of an image processing apparatus according to the second embodiment. The difference from the first embodiment is that a switch 701 is added. Whether the input signal to the monitor signal processing circuit 110 is an output from the first interpolation processing circuit 109 or an output from the first interpolation processing circuit 109 according to an enlargement ratio at the time of focus assist according to an instruction from the display control unit 116. Switch with.

  As a problem when the enlargement magnification is raised above a certain level at the time of focus assist, when interpolation is performed by a smooth interpolation process represented by bicubic in interpolation process 1, an image is displayed when the enlargement ratio becomes very large. There is a problem of being blurred. Therefore, if the magnification determination circuit 701 determines that the magnification is higher than the predetermined magnification, the same neighborhood interpolation processing as the interpolation processing 2 is used for the input signal to 110. Thus, by intentionally displaying an image that is not smooth, it is possible to prevent blurring of the display image when the enlargement ratio during focus assist is large, and to easily check the degree of focus.

  FIG. 9 is a flowchart showing an operation of image display processing on the display device by the image processing apparatus at the time of shooting in the second embodiment. S2000, S2001, S2004, S2005, and S2006 are the same as those in FIG.

  In S2002, it is determined whether or not the scaling magnification exceeds the threshold value. If the threshold value is not exceeded, the process branches to S2006, and the output from the first interpolation processing circuit 108 is selected by the switch 701 in accordance with an instruction from the display control unit 116. Is done. That is, the subsequent processing is the same as that of the first embodiment.

  On the other hand, when the scaling magnification exceeds the threshold value, the output from the second interpolation processing circuit 109 is selected by the switch 701 according to an instruction from the display control unit 116. That is, the image input to the monitor signal processing unit 110 is subjected to the same interpolation process (nearest neighbor interpolation process) as the image input to the peaking signal generation unit 114 (S2003), and the peaking signal is superimposed in S2005 to display the display device. It is output to 113.

  As described above, in the present embodiment, when the display image is enlarged (interpolated) larger than a predetermined magnification, the same interpolation as that of the peaking signal generation unit 114 is performed so that the display image does not become too smooth. The display image is enlarged by the processing. Thereby, even when the enlargement magnification is large, an image for performing appropriate focus confirmation can be obtained.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

(Other embodiments)
The object of the present invention can also be achieved as follows. That is, a storage medium in which a program code of software in which a procedure for realizing the functions of the above-described embodiments is described is recorded is supplied to the system or apparatus. The computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the novel function of the present invention, and the storage medium and program storing the program code constitute the present invention.

  Examples of the storage medium for supplying the program code include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, a CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-R, magnetic tape, nonvolatile memory card, ROM, or the like can also be used.

  Further, by making the program code read by the computer executable, the functions of the above-described embodiments are realized. Furthermore, when the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, the functions of the above-described embodiments are realized by the processing. Is also included.

  Furthermore, the following cases are also included. First, the program code read from the storage medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing.

DESCRIPTION OF SYMBOLS 100 Optical system 101 Image pick-up element 102 Digital signal processing part 103 Frame memory 107 Scaling circuit 108 1st interpolation processing circuit 109 2nd interpolation processing circuit 110 Monitor signal processing part 111 Peaking signal superimposition circuit 112 Display element drive part 113 Display device 114 Peaking signal generation unit 115 switch 116 display control unit 117 display block

Claims (8)

Interpolation means for performing interpolation processing according to the enlargement magnification on the input image signal;
Peaking signal generation means for generating a peaking signal having a level corresponding to a change amount of a signal level in a predetermined direction in the image signal from the image signal subjected to interpolation processing by the interpolation means;
The peaking signal by superimposing the generated by the peaking signal generating means to an image signal interpolating process has been performed by said interpolation means comprises display control means for displaying on the Viewing device, a
The interpolation means may perform a first interpolation process on the input image signal and a second interpolation process that leaves a high-frequency component of the image signal to be processed as compared to the first interpolation process. Is possible,
The display control means includes a peaking signal generated from an image signal obtained by performing the second interpolation processing by the interpolation means on an image signal obtained by performing the first interpolation processing by the interpolation means. An image processing apparatus characterized by superimposing the images.   The image processing apparatus according to claim 1, wherein the first interpolation process is a linear interpolation process or a bicubic interpolation process, and the second interpolation process is a nearest neighbor interpolation process.   The image processing apparatus according to claim 1, wherein the first interpolation process is a linear interpolation process, and the second interpolation process is a bicubic interpolation process. Wherein when the magnification indicated by the instructions means that instructs the magnification of the image displayed on the display device is larger than a predetermined threshold value, said display control means, said second interpolation processing by the interpolation means The image processing apparatus according to claim 3, wherein a peaking signal generated from the image signal obtained by performing the second interpolation process by the interpolation unit is superimposed on the image signal obtained by performing the interpolation. . An image processing apparatus according to any one of claims 1 to 4,
An optical system including a focus lens;
Imaging means for imaging light incident from the optical system;
An image pickup apparatus comprising: control means for controlling focusing of the image pickup means. An interpolation step for performing an interpolation process according to the enlargement magnification on the input image signal;
A peaking signal generation step of generating a peaking signal having a level corresponding to a change amount of a signal level in a predetermined direction in the image signal from the image signal subjected to the interpolation processing in the interpolation step;
The peaking signal by superimposing the generated by the peaking signal generating step to the image signal interpolation processing has been performed in the interpolation step, has a display control step of displaying on the Viewing device,
In the interpolation step, a first interpolation process and a second interpolation process that leaves a high-frequency component of the image signal to be processed compared to the first interpolation process are performed on the input image signal. Is possible,
In the display control step, a peaking signal generated from the image signal obtained by performing the second interpolation processing in the interpolation step on the image signal obtained by performing the first interpolation processing in the interpolation step. A method for controlling an image processing apparatus, wherein:   A computer-executable program in which a procedure of a control method for an image processing apparatus according to claim 6 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the image processing apparatus according to claim 7.

Images