JP5832078B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus and a control method thereof, and more particularly to a technique for displaying a degree of focus in the imaging apparatus.

  With the advance of automatic focus detection (autofocus: hereinafter referred to as “AF”) technology, AF is mainly used for focus detection in an imaging apparatus such as a still camera or a video camera. However, there are cases where it is necessary to perform imaging under conditions where focusing is difficult with AF, such as when it is necessary to focus precisely as in macro shooting, or when shooting an intentionally blurred image, etc. . In that case, photographing is often performed by manual focus adjustment (manual focus: hereinafter referred to as “MF”).

  When performing the MF operation, the operability and the focusing accuracy are greatly affected by the appearance of the optical viewfinder and the resolution of the electronic viewfinder (hereinafter referred to as “EVF”). However, the increase in size and resolution of the optical viewfinder and EVF leads to an increase in size and price of the image pickup apparatus.

  Accordingly, various auxiliary display technologies have been proposed that make it easier for the photographer to grasp the degree of focus (focused state) instead of increasing the size of the optical viewfinder or EVF or increasing the resolution. For example, highlighting (peaking display) of an in-focus edge portion in the captured image, partial enlarged display for checking the degree of focus, and the like can be given. In addition, a configuration has been proposed in which an integral value of a high frequency component of an image luminance signal is obtained as an “evaluation value” indicating the degree of focus, and a change in the size is synthesized and displayed on an EVF image using a bar graph (patent) Reference 1). In addition, a technique has been proposed in which a histogram of the amplitude of a high-frequency component of an imaging signal is generated as an “evaluation value” indicating the degree of focus, and is synthesized and displayed on an EVF image (see Patent Document 2).

JP-A-6-113184 Japanese Patent No. 4089675

  However, in the peaking display, it may be difficult to identify the highlighted display depending on the size and location of the focused edge portion. In the peaking display, a technique has been proposed in which only the focused edge portion of the EVF image is colored and the other portion is displayed in black and white. In this case, the entire screen excluding the edge portion is displayed in black and white. By changing to, the visibility of the photographic subject by EVF is lowered.

  Further, in the techniques described in Patent Documents 1 and 2, it is possible for the photographer to grasp which portion of the shooting angle of view the “evaluation value” indicating the degree of focusing indicates the degree of focusing. Not easy for. Furthermore, since the change in the magnitude of the evaluation value is expressed by the change in the height of the bar graph and the spread of the histogram, it is not easy to grasp the maximum point of the evaluation value. When the degree of focus for an individual subject is displayed on the same screen as the subject image by some method, the individual subject may have a different focus due to the positional relationship between subjects or the difference in the angle of view of the camera (for example, vertical / horizontal). There is a possibility that the display of the degree of focus on the subject may be difficult to understand.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide an imaging apparatus and a control method thereof that improve operability and visibility related to focusing.

An image pickup apparatus according to the present invention is an image pickup apparatus having a function of performing focus adjustment of a lens that forms a subject image on an image pickup element, and an extraction unit that extracts an edge component of an image picked up by the image pickup element; Generating means for generating a waveform image representing a relationship between a position and a size of the edge component extracted by the extracting means in a predetermined direction; display means for displaying the video imaged by the imaging device and the waveform image on a display device; And designation means for designating a plurality of areas for the video displayed on the display means, and determination means for determining overlap in a specific direction of the plurality of areas designated by the designation means. the generating means when said overlap in said plurality of regions in said particular direction Ri I is determined in the determination unit, the waveform direction other than the specific direction as the predetermined direction And generating an image.

  According to the imaging device and the control method thereof according to the present invention, it is possible to improve the operability and visibility related to focusing.

1 is a block diagram illustrating a schematic configuration of a digital video camera according to an embodiment of the present invention. It is a block diagram which shows the structural example which performs the waveform display process of an edge component in the video control part which the digital video camera concerning embodiment of this invention has. It is a figure which shows schematic structure of the edge extraction circuit which the digital video camera concerning embodiment of this invention has. It is a figure which shows the example of the frequency characteristic of the edge extraction circuit of FIG. It is a figure which shows typically the state which displayed the designation | designated area | region and the level of the edge component of captured image with the captured image on the 1st display part and / or 2nd display part which the digital video camera concerning embodiment of this invention has. . 4 is a flowchart of basic waveform display processing of an edge component during a manual focus operation in the digital video camera according to the embodiment of the present invention. It is a figure which shows typically the position determination method of two designated area | regions in the digital video camera which concerns on embodiment of this invention. 5 is a flowchart of edge component waveform display processing during a manual focus operation in the digital video camera according to the embodiment of the present invention. When the digital video camera according to the embodiment of the present invention is rotated, the state where the designated area and the level of the edge component of the captured image are displayed together with the captured image on the first display unit and / or the second display unit is schematically illustrated. FIG. It is a figure which shows typically the position determination method of two designated area | regions when rotating the digital video camera which concerns on embodiment of this invention. It is a flowchart of the waveform display process interlock | cooperated with rotation of a digital video camera at the time of the manual focus operation of the digital video camera concerning embodiment of this invention. It is a figure which shows typically the position determination method of three or more designation | designated area | regions in the digital video camera which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the present embodiment, a digital video camera having an MF operation function is taken as an example of an imaging apparatus. However, the imaging apparatus according to the present invention is not limited to this, and can be applied to an imaging apparatus having a focus adjustment function (regardless of AF or MF) and a display device that displays a captured image or a captured image. is there. That is, the imaging apparatus according to the present invention includes, for example, a digital (still) camera, a portable information terminal with a camera, a mobile phone with a camera, and the like.

<< First Embodiment >>
<Schematic configuration of digital video camera>
FIG. 1 is a block diagram showing a schematic configuration of a digital video camera according to an embodiment of the present invention. A lens unit 101 shown in FIG. 1 constitutes an optical system that forms a subject image on the imaging surface of the image sensor 102, and has a zoom function, a focus adjustment function, and an aperture adjustment function. The image sensor 102 has a configuration in which a large number of photoelectric conversion elements are two-dimensionally arranged, and converts the subject optical image formed by the lens unit 101 into a video signal in units of pixels. The image sensor 102 is, for example, a CMOS image sensor or a CCD image sensor. Note that the image sensor 102 has an electronic shutter function by adjusting the charge accumulation time of the photoelectric conversion element.

  The image sensor driving unit 103 drives and controls the image sensor 102 according to the control timing of the camera signal processing unit 106. The CDS / AGC unit 104 performs correlated double sampling (CDS) on the analog video signal output from the image sensor 102 to reduce noise, and performs signal level gain control (AGC) according to the control of the system control unit 111. The A / D converter 105 converts the analog video signal output from the CDS / AGC unit 104 into a digital video signal and supplies the digital video signal to the camera signal processing unit 106.

  The camera signal processing unit 106 controls the camera imaging system such as timing signal generation, automatic exposure (AE) control, gamma adjustment, and AF control in cooperation with the system control unit 111. For example, the camera signal processing unit 106 calculates an “AF evaluation value” that serves as a criterion for determining the degree of focus during AF control, and performs face detection processing in the captured video. It should be noted that the face position and its area of each subject are calculated by face detection processing (face feature detection process) in the captured image, and the central coordinates and area size indicating the face position are presented to the system control unit 111. It becomes possible. The face detection method is not particularly limited, and a known method can be used.

  The digital video camera includes a first storage unit 107, a second storage unit 116, a third storage unit 112, and a fourth storage unit 119 according to usage. In this embodiment, for convenience, the first storage unit 107 is for camera signal processing, the second storage unit 116 is for video control, the third storage unit 112 is for system control, and the fourth storage unit 119 is for codec (CODEC). Each is provided individually. Physically, these storage units may be realized by the same storage device. Each of these storage units is typically configured by a readable / writable semiconductor memory, but is not limited thereto, and one or a plurality of storage units may be replaced with another type of storage device [for example, a hard disk drive (HDD). ) Etc.].

  The first storage unit 107 is used by the camera signal processing unit 106 as a frame memory or the like when the captured video is signal-processed. The lens driving unit 108 drives a motor, an actuator (not shown) included in the lens unit 101 according to the control of the system control unit 111, and performs zoom magnification adjustment, AF control, and AE control. The lens driving unit 108 is controlled by the system control unit 111 based on the signal processing result in the camera signal processing unit 106. For example, at the time of AF control, the system control unit 111 controls the lens driving unit 108 based on the AF evaluation value obtained by the camera signal processing unit 106, and drives and controls the focus adjustment lens of the lens unit 101. 101 is focused on the subject.

  When the MF mode for manually adjusting the focus of the lens unit 101 is set, the system control unit 111 detects a focus adjustment operation by the photographer. Specifically, the system control unit 111 detects an operation of a focus ring adjustment switch and lever included in the input operation unit 113 and a focus ring provided on the outer periphery of the lens barrel of the lens unit 101. Then, the system control unit 111 controls the lens driving unit 108 based on the moving direction and moving amount of the focal point corresponding to the detected operation, and changes the focal length of the lens unit 101. However, when the focal length of the lens unit 101 can be mechanically changed by a switch or lever operated by the photographer, the system control unit 111 and the lens driving unit 108 are not necessary.

  The strobe 109 is used as an auxiliary light source at the time of still image shooting as required or according to the settings of the photographer. The microphone 110 is used when recording (recording) ambient sounds, and an audio signal from the microphone 110 is supplied to the camera signal processing unit 106. For example, when recording audio together with the captured video, the camera signal processing unit 106 matches the time axes of the imaging element 102 and the microphone 110 and supplies the video signal and the audio signal to the video control unit 115.

  The system control unit 111 is, for example, a CPU, and controls the overall operation of the digital video camera including an auxiliary display process during MF operation described later by executing a program stored in the third storage unit 112. . The third storage unit 112 includes a ROM and a RAM, and stores programs executed by the system control unit 111, parameters related to various settings, initial values, and the like. The third storage unit 112 is also used as a work area for the system control unit 111.

  The input operation unit 113 is a user interface for a photographer to give an instruction to the digital video camera, and includes input devices such as keys, buttons, and a touch panel. In the present embodiment, the input operation unit 113 specifically selects various functions such as on / off of enlarged display, on / off of zebra pattern display and peaking display, and on / off of auxiliary display during MF operation. It has a button and a decision button. The input operation unit 113 includes a shutter button for shooting a still image, an MF ring, a zoom switch, an aperture adjustment dial, and the like. The input operation unit 113 further includes a gravity sensor and a gyro sensor for detecting top-and-bottom information indicating an imaging posture of the digital video camera such as horizontal shooting / vertical shooting.

  The time measuring unit 114 includes a real time clock (RTC) and a backup battery, and returns date information in response to a request from the system control unit 111. The video control unit 115 performs display control including adjustment of hue, saturation, and lightness for the first display unit 122 and the second display unit 123, and also performs output control of the analog line output unit 124 and a digital data I / F unit. Control of output to 125, control of the recording / reproducing unit 120, and the like are performed. The video control unit 115 also converts the resolution of video signals for each video output system including the first display unit 122 and the second display unit 123, generates and superimposes zebra patterns and peaking signals, extracts edge components from the captured video, Also performs waveform generation. The video control unit 115 further performs OSD (On Screen Display) display control such as shooting information and a user setting menu. The edge component is one type of factor that represents the degree of focus.

  The second storage unit 116 is a storage unit for video control, and is used as a frame memory, a work memory, or the like when the video control unit 115 performs signal processing related to the video baseband signal. H. The H.264 codec unit 117 is an example of a moving image codec that performs moving image encoding / decoding processing. As an encoding / decoding format, for example, the MPEG-2 system is used, but other formats may be used. Similarly, the JPEG codec unit 118 is an example of a still image codec that performs encoding / decoding processing of a still image. For example, JPEG2000 is used as the encoding / decoding format, but other formats such as PNG may be used. In the present embodiment, H.264. The JPEG codec unit 118 is connected to the video control unit 115 in order to share a circuit with the H.264 codec unit 117 and to realize a still image shooting function (capture function) from a reproduced moving image. The camera signal processing unit 106 may be directly connected.

  The fourth storage unit 119 is a codec for H.264. The H.264 codec unit 117 and the JPEG codec unit 118 are used when the video signal is encoded / decoded. The recording / playback unit 120 includes a video control unit 115 and an H.264 recording medium. Recording data encoded by the H.264 codec unit 117 or the JPEG codec unit 118 and processed as a recording format is recorded on or read from the recording medium 121. As the recording medium 121, an optical disk such as a memory card or DVD, an HDD, or the like can be used, and a recording / reproducing system corresponding to each medium may be configured.

  The first display unit 122 and the second display unit 123 are display devices, and both can display the same information. In the present embodiment, the second display unit 123 is smaller than the first display unit 122 and is provided in the viewfinder. On the other hand, the first display unit 122 is a relatively large display device that can be opened and closed on, for example, a side surface of a casing of a digital video camera.

  In the imaging mode, auxiliary display such as a focus frame is displayed on the first display unit 122 and the second display unit 123 in addition to the input video and the enlarged video from the imaging device 102. Further, an electronic viewfinder (EVF) function is realized by sequentially displaying the input video from the image sensor 102 on the first display unit 122 and the second display unit 123. The auxiliary display includes not only zebra pattern display and peaking display, but also auxiliary display at the time of MF operation such as waveform display of the edge component of the captured image.

  In the playback mode, moving images and still images recorded on the recording medium 121 are displayed on the first display unit 122 and the second display unit 123. Further, input operation information by the photographer from the input operation unit 113, arbitrary image information (imaging information) in the recording medium 121, and the like can be displayed.

  The analog line output unit 124 is an interface group for analog component video output, S terminal output, composite video output, and the like. By connecting the analog line output unit 124 to an external monitor or the like, the video output from the digital video camera can be displayed on the external monitor or the like. The digital data I / F unit 125 has one or more digital interfaces such as a USB interface, an IEEE 1394 interface, and an HDMI, for example.

<Waveform display processing of edge component of captured image by video controller>
As described above, the video control unit 115 performs waveform display processing of the edge component of the captured video, that is, extraction of the edge component from the captured video (video signal) and waveform generation. FIG. 2 is a block diagram illustrating a configuration example for executing the waveform display processing of the edge component of the captured video in the video control unit 115.

  The preprocessing unit 201 illustrated in FIG. 2 includes a region designation circuit 202, an edge extraction circuit 203, a first memory controller 204, and a waveform control unit 205. The area designation circuit 202 has a function of notifying the first memory controller 204 of information on an area (designated area) designated through the system control unit 111 in the captured video as an area designation signal.

  Information on the designated area that can be designated by the system control unit 111 in the area designating circuit 202 includes the size (shape), number, and position of the area, but is not limited thereto. In this embodiment, as shown in FIG. 5A to be described later, a plurality of designations are made by the system control unit 111 based on the center coordinates and the area size obtained as a result of the face detection processing from the camera signal processing unit 106. An area (two rectangular designated areas 502 and 503) is set.

  In addition, it is good also as a structure which permits a photographer to set a designated area without being linked to the face detection function. In this case, as in the method used when specifying a rectangular area in an image, the size and position of a rectangular frame whose size and position are variable are adjusted with direction keys and set by pressing a decision button or the like Can be adopted.

  The GUI for setting the designated area, the frame representing the set designated area, and the like are displayed in the video composition circuit 210 (to be described later) as necessary in the same way as the waveform display image of the edge component of the captured video. And can be superimposed and displayed. Data relating to these displays is supplied from the system control unit 111 to the video composition circuit 210.

  The edge extraction circuit 203 extracts edge components of the captured video. In the present embodiment, the edge extraction circuit 203 extracts edge components using a FIR (Finite Impulse Response) type band pass filter, and details thereof will be described later.

  The first memory controller 204 performs bank switching control between the first waveform storage memory (Bank1) 207 and the second waveform storage memory (Bank2) 208 in the bank memory unit 206 for each field (or frame) of the video signal. Perform memory access control. Details of the first waveform storage memory 207 and the second waveform storage memory 208 will be described later. The first memory controller 204 considers the current coordinate information of the video signal and the level (amplitude) of the edge component of the captured video extracted by the edge extraction circuit 203 as information in the orthogonal coordinate system, and generates an address for memory access To do.

  For example, as shown in FIG. 5A, which will be described later, when the edge component of the captured image is horizontally displayed as a waveform, the address space of the memory is the horizontal coordinate of the video signal, and the level of the edge component. Access control is performed with a vertical axis as a two-dimensional space. On the other hand, as shown in FIG. 5B, which will be described later, when the edge component of the captured image is vertically displayed as a waveform, the memory address space is represented by the vertical coordinate of the video signal as the vertical axis and the level of the edge component. Access control may be performed by regarding the two-dimensional space with the horizontal axis. The process by the access control here is a process of first reading the data at the corresponding address and writing the data processed in the waveform control unit 205 described later back to the same address through the first memory controller 204 again.

  The waveform control unit 205 adds the data read from the first waveform storage memory 207 by the first memory controller 204 to the edge component data of the captured video extracted by the edge extraction circuit 203 and multiplied by an arbitrary gain. Then, it returns to the first memory controller 204 again. By controlling the gain by which the waveform control unit 205 multiplies the edge component data from the system control unit 111, it is possible to control the increase rate of the data according to the frequency and to control the visibility.

  When the data value of an address with a high access frequency (that is, the number of extractions of edge components of the same level from the captured video) is large by these processes and the synthesis process in the video synthesis circuit 210 described later, the display brightness is large ( Can be controlled to be brighter). At this time, when edge components of different levels are extracted at different frequencies in the same coordinates in the edge component extraction direction (horizontal direction or vertical direction in the present embodiment), the display luminance corresponding to the position is not constant and is detected. It changes at the position corresponding to the specified level. Thus, the photographer can easily grasp the edge component visually.

  Note that the initial value of each address is, for example, a value corresponding to the lowest luminance, and a predetermined value is written when accessed for the first time, and the above-described reading and writing back are performed from the second access. Further, by performing control using the access frequency, it is possible to obtain an effect of preventing a single noise that is not an edge component from being displayed as if it was an edge component and being recognized by a photographer.

  The first memory controller 204 adds flag information to the edge component extracted for the designated area in the captured image designated by the area designation signal from the area designation circuit 202, and the first waveform storage memory 207 or the second waveform Write to the storage memory 208. Then, the second memory controller 211 reads the edge component data written in the first waveform storage memory 207 or the second waveform storage memory 208 including the flag information.

  Based on the flag information read by the second memory controller 211, the video composition circuit 210 performs control to make the edge component display method (for example, display color) for the designated area different from the edge component display method for the outside of the designated area. . As a result, it is possible to easily visually grasp which designated region in the captured image corresponds to the edge component (that is, what degree of focus is achieved). After completing the processing in one field (or one frame), the video composition circuit 210 reads out data and flag information at each address from the second memory controller 211 to generate a waveform display image.

  The bank memory unit 206 includes a first waveform storage memory (Bank 1) 207 and a second waveform storage memory (Bank 2) 208. The second waveform storage memory 208 is equivalent to the first waveform storage memory and is used for bank control.

  The first waveform storage memory 207 and the second waveform storage memory 208 are switched so as to be accessed from either the first memory controller 204 or the second memory controller 211 for each field (or frame) of the video signal. In the present embodiment, when the first waveform storage memory 207 is accessed from the first memory controller 204, the second waveform storage memory 208 is controlled to be accessed from the second memory controller 211. In the next video signal field (or frame), the access relationship between the memory controller and the waveform storage memory is switched.

  The post-processing unit 209 includes a video composition circuit 210 and a second memory controller 211.

  The video synthesis circuit 210 synthesizes the captured video and the waveform display image based on the edge component data of the captured video read from the bank memory unit 206 by the second memory controller 211 in accordance with an instruction from the system control unit 111. The synthesized image is displayed on the first display unit 122 and / or the second display unit 123. As described above, when flag information is added to the read edge component data, the video composition circuit 210 visually recognizes the edge component having the flag information with respect to the edge component having no flag information. Display so that it can be identified.

  For example, FIG. 5A shows a state where the captured image is displayed on the full screen and the waveform display image of the edge component of the captured image is superimposed and displayed below the screen. Although FIG. 5A is a black and white drawing, the edge component waveform displays 504 and 505 extracted in the designated areas 502 and 503 are the edge component waveform displays 506 extracted outside the designated area. It is displayed in a different color. FIG. 5A schematically shows the state (color difference) with black and white shading.

  The colors of the edge component waveform displays 504 and 505 in the designated areas 502 and 503 may be configured to be designated by the system control unit 111. In the present embodiment, the waveform display corresponding to the edge component straddling the designated area and the outside of the designated area is displayed at the same position (same coordinates) in the horizontal direction. However, the present invention is not limited to this, and it is possible to visually recognize parts with different edge component levels, and to compose and display the parts with the same level in the video composition circuit 210 in the priority order according to the control from the system control unit 111. can do.

  Although not particularly shown in FIG. 5A, a background color (semi-transparent black or the like) may be displayed on the background of the waveform display image in order to make the waveform display image of the edge component of the captured image easy to see. Good. In that case, the visibility of the waveform display image can be improved by superimposing the background color on the captured image and then superimposing the waveform display image on the background color.

  The second memory controller 211 reads the edge component data written in the first waveform storage memory 207 or the second waveform storage memory 208 by the first memory controller 204 together with the flag information, and supplies it to the video composition circuit 210. As described above, the video synthesis circuit 210 generates a waveform display image representing the level of the edge component and the extracted position from the edge component data and the flag information, and combines and synthesizes it with the captured video. The information is displayed on the first display unit 122 and / or the second display unit 123.

  The system control unit 111 supervises various controls in the video control unit 115 although detailed connection wiring is not shown. For example, the system control unit 111 performs region designation based on the face detection result to the region designation circuit 202, control of the center frequency of the edge component extracted from the captured image by the edge extraction circuit 203, gain control, and gain in the waveform control unit 205. Control and so on. The system control unit 111 also performs the presence / absence of synthesis of the waveform display image of the edge component in the video synthesis circuit 210, the synthesis position control for the video signal, and the control of the waveform display method corresponding to the designated area and other areas.

<Schematic configuration and processing contents of edge extraction circuit>
The edge extraction circuit 203 is applied with a FIR type bandpass filter having a structure capable of adjusting the frequency characteristics and the pass gain for the luminance component in a predetermined direction (here, the horizontal direction) of the image. Extract edge components from. Note that the edge component may be extracted by other methods.

  FIG. 3 is a diagram showing a schematic configuration of the edge extraction circuit 203. An input video signal 301 is input to a D-FF (Delay type Flip Flop) 302. The D-FF 302 latches the input video signal 301 with the pixel clock of the video signal and passes it to the D-FF 303 at the next stage. Each of the D-FFs 303 to 309 has the same configuration as that of the D-FF 302, latches data from the previous stage with the pixel clock of the video signal, delays it by one pixel clock, and outputs it to the subsequent stage. Here, the pixel clock is a display clock of the first display unit 122 and depends on specifications (display resolution) of a liquid crystal display panel or the like used for the first display unit 122. In the present embodiment, the pixel clock is set in a range of approximately 13.5 to 33.75 MHz.

In the present embodiment, three taps are combined in accordance with the frequency of the edge component extracted from the captured video among the nine taps Y (-4) to Y (4) of the FIR type horizontal filter. Specifically, the center tap Y (0) is the same, changing the combination of the front and rear taps,
[Y (-1), Y (0), Y (1)],
[Y (-2), Y (0), Y (2)],
[Y (-3), Y (0), Y (3)],
Consider a combination of [Y (−4), Y (0), Y (4)]. In each combination, an unselected tap can be considered to be selected but the tap coefficient is set to zero. By controlling the combination of taps in this way, the frequency band extracted as an edge can be made variable.

The tap coefficient and DC gain of the three taps to be combined are as follows:
Center tap tap coefficient: 2α,
Tap coefficient of front and rear taps: -α,
DC gain: 2α−α−α = 0.

  The gain adjustment signal 310 is supplied from the system control unit 111 and controls the level of the emphasis frequency band of the FIR horizontal filter of the edge extraction circuit 203. Here, the gain adjustment signal 310 is a 4-bit signal, and the gain level to be emphasized can be switched to 16 levels. Specifically, α = 15 (maximum gain) to α = 0 (minimum gain (OFF)). When α = 0, the video signal is not processed (through) by the FIR horizontal filter.

  The frequency adjustment signal 311 shown in FIG. 3 is supplied from the system control unit 111, and thereby, the center frequency of the edge component extracted from the captured video by the edge extraction circuit 203 is controlled. The frequency adjustment signal 311 is a 2-bit signal, and the center frequency of the extracted edge component can be switched in four ways.

  The signal selector 312 is controlled by the frequency adjustment signal 311 and switches the center frequency extracted by the edge extraction circuit 203 by selecting and outputting one input corresponding to the frequency adjustment signal 311. The relationship between the 2-bit frequency adjustment signal 311 and the center frequency (emphasis center frequency) fc output from the edge extraction circuit 203 is specifically as follows.

Sampling frequency: f (MHz)
Frequency adjustment signal 311: Emphasized center frequency (fc)
00: fc = f / 2 (MHz)
(Use of center tap and front and rear taps adjacent to the center tap)
01: fc = f / 4 (MHz)
(Use the center tap and the tap that has been skipped forward and backward from the center tap)
10: fc = f / 6 (MHz)
(Using the center tap and two taps skipped forward and backward from the center tap)
11: fc = f / 8 (MHz)
(Use of the center tap and three taps skipped back and forth from the center tap)
The signal selected by the signal selector 312 is output to the first memory controller 204 as an output signal 313.

  FIG. 4 is a diagram illustrating an example of frequency characteristics of the edge extraction circuit 203 when the sampling frequency f is 13.5 MHz. 4A, 4B, 4C, and 4D correspond to the cases where the frequency adjustment signal 311 is 00, 01, 10, and 11, respectively, and the center frequency fc (MHz) is f / 2. f / 4, f / 6, f / 8 and their harmonic frequencies.

  A captured image signal is input to the edge extraction circuit 203 for each horizontal line, and edge components are extracted in the horizontal direction. The first memory controller 204 and the waveform control unit 205 perform the above-described memory access control according to the level of the edge component extracted by the edge extraction circuit 203. By repeating this operation for each line, a value reflecting the level of the edge component of each line in the horizontal direction for the video signal for one field (or one frame) is the first waveform storage memory 207 or the second waveform. It is written in the storage memory 208.

  For example, when a vertical line in the vertical direction exists in the captured image, the longer the vertical line, the higher the frequency with which edge components of the same level are extracted at the same horizontal position. When edge components are extracted a plurality of times at the same position and at the same level, the frequency of access to the same address of the waveform storage memory increases as the number of extractions increases, and the data value at that address increases by the above-described access control. As a result, the corresponding index is displayed with high luminance. On the other hand, the length of the index indicating the level (amplitude) of the edge component does not depend on the frequency. When the detected vertical display is performed, the same operation is performed for each vertical line.

<Display Form in the First Display Unit and / or the Second Display Unit>
FIG. 5 is a diagram schematically illustrating a state in which the first display unit 122 and / or the second display unit 123 display the specified region and the level of the edge component of the captured image along with the captured image along with the captured image. In the present embodiment, for example, when the shooting mode is set to the MF mode through the input operation unit 113, the waveform display image of the edge component is superimposed and displayed on the captured video as an auxiliary display of the degree of focus.

  Here, before describing FIG. 5 in detail, the waveform display processing of the edge component during the MF operation will be described with reference to FIG. FIG. 6 is a flowchart of basic edge component waveform display processing during MF operation.

  The system control unit 111 controls the lens driving unit 108 to change the focal length of the lens unit 101 based on the moving direction and moving amount of the focal point according to the focal length adjustment operation by the input operation unit 113. Through images (EVF images) are sequentially displayed on the first display unit 122 and / or the second display unit 123, and the first display unit 122 and / or the second display unit 123 function as an EVF. . In addition, it is assumed that at least one designated area is set in the captured image displayed on the first display unit 122 and / or the second display unit 123.

  The system control unit 111 instructs the video control unit 115 to start the auxiliary display processing of the MF operation, and at this time, notifies the region designation circuit 202 of information on the designated designated region. In response to the auxiliary display processing start instruction from the system control unit 111, the edge extraction circuit 203 starts extracting edge components from the captured video (step S601).

  The first memory controller 204 and the waveform control unit 205 access an address corresponding to the extraction result of the edge extraction circuit 203 in the waveform storage memory (here, the second waveform storage memory 208) in the bank memory unit 206, Waveform generation is performed (step S602). At that time, the first memory controller 204 adds flag information to the data of the address corresponding to the edge component extracted in the designated area notified by the area designation signal from the area designation circuit 202.

  Subsequently, it is determined whether or not the extraction of the edge component (step S601) and the waveform generation (step S602) from one field (or one frame) of the captured video is completed (step S603). If waveform generation is completed (“YES” in S603), the process proceeds to step S604. If waveform generation has not ended ("NO" in S603), the process returns to step S601.

  In step S <b> 604, the second memory controller 211 reads edge component data from the second waveform storage memory 208 and supplies it to the video composition circuit 210. The video synthesis circuit 210 generates a waveform display image, superimposes the generated waveform display image on the EVF image, and displays the waveform display image on the first display unit 122 and / or the second display unit 123. At that time, the video composition circuit 210 generates a waveform display image that can visually identify the edge component data to which the flag information is added from the edge component data without the flag information. The correspondence between the flag information and the display method can be determined in advance.

  After step S604, the first memory controller 204 switches the bank of the waveform storage memory used for the next field (or frame) (here, switching from the second waveform storage memory 208 to the first waveform storage memory 207). Is performed (step S605).

  The process in step S605 includes a series of processes of reading edge component data, generating and displaying a waveform display image by the second memory controller 211. The processing in step S605 also includes bank switching by the first memory controller 204 and edge component extraction processing for the next field (or frame).

  Thus, by displaying the waveform display image of the edge component representing the degree of focus at that time on the EVF image and displaying it, the photographer can easily focus on the MF while viewing the EVF. Moreover, since the part reflecting the edge component in the designated area in the waveform display image is displayed so as to be visually distinguishable from other parts, the degree of focusing in the designated area can be easily confirmed.

  Note that the waveform display image indicating the edge component of the captured video does not need to be superimposed on the captured video, and may be displayed around the captured video if the captured video is not a full screen display. Further, the focus level in the designated area can be easily identified even if the waveform display is performed based only on the edge component extracted in the designated area. A function may be assigned to the input operation unit 113 or the like so that the photographer can arbitrarily and dynamically switch how the display form is set.

  Next, the display form in the first display unit 122 and / or the second display unit 123 shown in FIG. 5 will be described in detail. Here, it is assumed that both the first display unit 122 and the second display unit 123 have a horizontally long display screen 501 having an aspect ratio of 16: 9. Further, in FIG. 5, it is assumed that the designated areas 502 and 503 detected by the face detection function are focused.

  In the horizontal display of FIG. 5A, an index (that is, a waveform) indicating the level of the edge component in the horizontal direction of the captured image is displayed at the lower end of the screen. The indicator may be displayed on the upper end portion of the screen. When the photographer performs the MF operation, when the subject comes into focus, the edge portion of the subject is brought into focus, so that the level of the edge component extracted at that position increases. Therefore, the index displayed in the waveform at the lower end of the screen is displayed high (the waveform is large). Therefore, the photographer easily focuses on the desired subject by performing the MF operation so that the index corresponding to the desired subject in the waveform display image is displayed highest while viewing the EVF. be able to.

  In the horizontal display of FIG. 5A, the waveform display image of the edge component of the captured video is superimposed and displayed on the captured video in the horizontal position, so that the waveform display image and the captured image are captured for each subject included in the captured video. The degree of focus can be confirmed while viewing the video in the vertical direction. In the present embodiment, display control is performed so that the luminance of the waveform display image increases as the level of the same edge component increases in the vertical direction of the captured image, thereby further facilitating confirmation of the degree of focus. ing.

  The vertical display in FIG. 5B is a display of a waveform display image indicating the level of the edge component in the vertical direction of the captured image at the right end of the screen. The waveform display may be performed at the left end of the screen. As in the case of the horizontal display in FIG. 5A, the photographer performs the MF operation so that the index corresponding to the desired subject is displayed highest in the waveform display image. The subject can be easily focused on. In the vertical display of FIG. 5B, since the waveform display image of the edge component of the captured video is superimposed and displayed with the vertical position aligned with the captured video, the waveform display image and the captured image are captured for each subject included in the captured video. The degree of focus can be confirmed while viewing the image in the horizontal direction.

  In the waveform display of FIG. 5A, the index 504 corresponding to the edge component of the designated area 502 and the index 505 corresponding to the edge component of the designated area 503 are different from the index 506 of the edge component extracted outside the designated area. The display is done. That is, in FIG. 5A, since the color difference cannot be expressed, the indicators 504 and 505 are represented by a collection of lines. However, in actuality, the display color is changed, the display is blinked, the luminance is changed, etc. Any display method that can be visually identified can be employed.

  For example, the indices 504 and 505 reflecting the edge components extracted in the designated areas 502 and 503 can be displayed in “red”, and the index 506 reflecting the edge components extracted outside the designated area can be displayed in “green”. . Thereby, it is possible to easily confirm the degree of focus of the designated areas 502 and 503 to be noted while confirming the degree of focus of the entire captured image. Further, the rectangular frames of the designated areas 502 and 503 can be not displayed. However, based on the center coordinates and area size of the face detected by the face detection function, the image composition circuit 210 unit displays a rectangular frame by the OSD on the captured image, so that the photographer can use it as a mark of where to focus. By presenting, the MF operability can be improved. These characteristics are the same for the index 510 corresponding to the edge component of the designated area 502 and the index 511 corresponding to the edge component of the designated area 503 as in the case of FIG.

  In the case of FIG. 5A, when the designated areas 502 and 503 by the face detection function are separated in the horizontal direction, the indicators 504 and 505 indicating the edge components of the designated areas 502 and 503 displayed horizontally at the lower end of the screen. Do not overlap each other. Therefore, the degree of focus of each of the designated areas 502 and 503 can be clearly grasped.

  On the other hand, as shown in FIG. 5C, the designated areas 502 and 503 may overlap at least partially in the horizontal direction. In this case, when the indices 507 and 508 indicating the edge components of the designated areas 502 and 503 are horizontally displayed at the lower end of the screen, an overlapping portion 509 is generated between the indices 507 and 508, and each of the designated areas 502 and 503 is focused. It may be difficult to grasp the degree. The display form that solves this problem is the display form shown in FIG. That is, the digital video camera according to the present embodiment switches from the horizontal display in FIG. 5C to the vertical display in FIG. 5B by identifying the positional relationship between the designated areas 502 and 503. In other words, the edge component extraction direction of the captured video is switched from the horizontal direction to the vertical direction orthogonal thereto, and the display of the indices 507 and 508 is switched to the display of the indices 510 and 511. Accordingly, it is possible to avoid the overlapping of the indexes 507 and 508 and to generate the overlapping portion 509, display the indexes 510 and 511 that do not overlap, and clearly grasp the degree of focus of the designated areas 502 and 503. Will be able to.

  Such display mode switching control from FIG. 5C to FIG. 5B will be described below with reference to FIGS. FIG. 7 is a diagram schematically showing a method for determining the positions of the two designated areas 502 and 503. FIG. 8 is a flowchart of the waveform display processing of the edge component of the captured image during the MF operation.

  In FIG. 7, paying attention to the designated areas 502 and 503, two-dimensional XY coordinates with the center of the designated area 502 as the origin are considered. In FIG. 7A, since the designated area 503 is located in the X direction of the designated area 502, the designated area 503 is located across the first quadrant and the fourth quadrant. In contrast, in FIG. 7B, the designated area 503 is in the first quadrant, and in FIG. 7C, the designated area 503 extends across the first and second quadrants. positioned.

  Here, before the designated area 503 in FIG. 7A starts a new image, a switching dead zone 701 is set, and the waveform display of the edge component of the captured image is changed from the horizontal display in FIG. 5A to the one in FIG. Do not switch to vertical display. When the position of the region 503 is changed as shown in FIG. 7C through FIG. 7A to FIG. 7B and the second quadrant is applied, the waveform display of the edge component is shown. Switching from 5 (a) to FIG. 5 (b). Conversely, even if the state of FIG. 7C is changed to that of FIG. 7B, the switching from FIG. 5B to FIG. 5A is not performed, and the designated area 503 is changed as shown in FIG. When the position is changed and another quadrant (fourth quadrant) is reached, the mode is switched from FIG. 5 (b) to FIG. 5 (a). Thereby, the indicators 504, 505, 510, and 511 are changed depending on the positional relationship between the designated areas 502 and 503, while preventing the waveform display of the edge component from being rapidly switched between the horizontal display and the vertical display. It can be avoided that identification becomes difficult.

  As shown in the flowchart of FIG. 8, in the display switching control described with reference to FIG. 7, first, face detection processing from a captured image is performed (step S801). Here, in the face detection process, center coordinates and area sizes of two faces are calculated, and designated areas 502 and 503 are set. Subsequently, the designated area 502 is set as a reference area, a two-dimensional XY coordinate with the center of the designated area 502 as the origin is set (step S802), and the designated area 503, which is the area of interest. Is detected (step S803). Then, it is determined whether or not the designated area 503 is switched and positioned within the dead zone 701 (step S804).

  If it is within the switching dead zone 701 (“YES” in S804), the process returns to step S801 because the display position of the waveform display image is not switched. If it is not within the switching dead zone 701 (“YES” in S804), it means that it is straddling another quadrant, and therefore processing for changing the display position of the waveform display image (FIGS. 5A and 5B). ) Is switched (step SS805). As a result, the indicators 504, 505, 510, and 511 indicating the degree of focus of the designated areas 502 and 503 are appropriately displayed according to the positions of the designated areas 502 and 503.

  As described above, according to the present embodiment, as an auxiliary display for supporting focusing during MF operation, the level of the edge component extracted from the captured image is displayed on the captured image display screen. Waveforms are displayed in correspondence with the subject position. Thereby, focusing by MF operation can be easily performed by performing focus adjustment so that the level at which the waveform is displayed becomes the highest. At this time, the level of the edge component is displayed at a position corresponding to the subject position, and the portion of the waveform display image that reflects the edge component in the designated region is displayed so as to be visually distinguishable from other portions. As a result, it is possible to easily recognize at which position (designated area) in the captured image the edge component is extracted, and to easily check the degree of focus in the designated area. Focusing can be performed easily. At the same time, the waveform display image of the edge component of the designated area in the captured image is displayed by switching between the horizontal direction and the vertical direction according to the positional relationship between the designated areas, thereby displaying the index indicating the degree of focus. Identification is secured. Thereby, the focusing operation at the time of MF operation is further facilitated.

<< Second Embodiment >>
Compared with the first embodiment, the second embodiment is different from the first embodiment only in the waveform display method of the edge component of the designated region, and the other points are the same as the first embodiment. The description will be omitted, and the differences will be described in detail.

  This embodiment has a feature in the waveform display method of the edge component of the designated region when the horizontal video shooting and the vertical video shooting are changed by rotating the digital video camera body. In other words, the present embodiment is characterized in that when the positional relationship of the designated area within the shooting angle of view changes with respect to the displayed waveform display, the change corresponds to the change.

  In FIG. 9, when the digital video camera is rotated, the designated area and the waveform display image of the edge component of the captured video are displayed on the first display unit 122 and / or the second display unit 123 together with the captured video. It is a figure which shows a state typically. FIG. 9A is a display form that does not belong to the embodiment of the present invention, and shows a state in which the digital video camera body is rotated from the state of FIG. That is, the waveform display image indicating the edge component of the captured video is displayed along the long side of the display screen 901. In FIG. 9A, the indices corresponding to the designated areas 502 and 503 in the display screen 901 overlap each other like the index 902. As a result, it is difficult to clearly identify each index. It has been.

  In order to solve this problem, in this embodiment, as shown in FIG. 9B, the XY coordinates are rotated in conjunction with the rotation of the digital video camera body, and the designated areas 502 and 503 are respectively The indicators 904 and 905 are displayed so as not to overlap. FIG. 10 is a diagram schematically showing a method for determining the positions of the designated areas 502 and 503 when the digital video camera is rotated. Note that information from a gravity sensor (not shown) or the like that determines the top-to-bottom direction included in the input operation unit 113 is used to detect rotation of the digital video camera body.

  The XY coordinate state of FIG. 10A is the same as the XY coordinate state of FIG. 5A, FIG. 7A, and FIG. Is the Y direction. For this reason, the position of the designated area 502 serving as a reference is not changed, and the quadrant in which the designated area 503 is located remains “first quadrant + fourth quadrant”. The display position is not switched and the display state shown in FIG.

  On the other hand, FIG. 10B shows the XY coordinate state of FIG. 9B, where the horizontal direction is the Y direction and the vertical direction is the X direction. That is, the coordinate system is rotated in conjunction with the rotation of the digital video camera body, thereby converting the XY coordinate state of FIG. 10A to the XY coordinate state of FIG. 10B. . When such a rotation of the coordinate system is performed, the position of the designated area 502 serving as a reference does not change, and the quadrant in which the designated area 503 is located is “first quadrant + fourth quadrant” in FIG. Is changed to “fourth quadrant + third quadrant” in FIG. Thus, since the designated area 503 extends over the third quadrant, which is a new quadrant, the display position of the waveform display image of the edge component is switched, and the indicators 903 and 904 for the designated areas 502 and 503 are not overlapped. The display state shown in FIG. 9B can be obtained.

  FIG. 11 is a flowchart of a waveform display process that is linked to the rotation of the digital video camera body during MF operation. Since the processes in steps S1101 and S1102 are the same as the processes in steps S801 and S802 shown in FIG. 8, a description thereof is omitted here. After step S1102, it is determined whether or not the digital video camera body has been rotated (changed between horizontal shooting and vertical shooting) (step S1103).

  If it is determined that it has been rotated (“YES” in S1103), the XY coordinates are rotated around the reference designated area (designated area 502) (step S1107), and then the process proceeds to step S1104. On the other hand, if it is determined that it is not rotating (“NO” in S1103), the process proceeds to step S1104. Since each process of step S1104 to S1106 is the same as each process of step S803 to S805 shown in FIG. 8, description here is abbreviate | omitted.

  As described above, according to the present embodiment, when the digital video camera is rotated to change the horizontal shooting and the vertical shooting, the waveform display image that is an index indicating the degree of focus is displayed at an appropriate position. Can be displayed. This makes it possible to improve the visibility of the degree of focus on each region corresponding to the subject position within the shooting angle of view without being aware of the subject position, how to hold the digital video camera, or the like.

<< Other Embodiments >>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  For example, in the above-described embodiment, only an image pickup apparatus including a lens (lens unit 101) that forms a subject image on the image pickup element 102 has been described. However, in the present invention, the lens is not essential, and the present invention is not limited to lens replacement. It can also be applied to a digital camera.

  Further, in the above embodiment, the waveform display images of the edge components of the designated areas 502 and 503 are displayed in correspondence with (matched with) the position of the designated area in the captured image, but it is possible to grasp the positional relationship between the two. For example, it is not always necessary to match the positions completely. For example, even if the waveform display image of the edge component of the captured image in the horizontal direction is reduced and displayed in a range shorter than the horizontal width of the captured image, the correspondence between the captured image and the waveform display image can be grasped. An effect can be obtained.

  Furthermore, in the above embodiment, only the case where the waveform display images of the edge components of the designated areas 502 and 503 are auxiliary displayed at the time of the MF operation has been described. However, even at the time of the AF operation, according to the settings and instructions of the photographer or always The waveform display of the edge component of the captured image may be performed. This makes it easier to check the degree of focus and the focus position during AF operation.

  In addition, when the plurality of regions are not two but three or more, the following processing can be performed. 12 is a diagram schematically showing a method for determining the position of three or more designated areas. FIG. 12A shows a case where there are three designated areas, and FIG. 12B shows a case where three designated areas are used. As many examples, there are shown cases where there are five designated areas.

In the case of FIG. 12A, the ratios occupied by the regions when viewed from the X and Y directions are compared. That is, as shown in FIG.
(A + B) ≧ (C + D) (1)
Is it?
(A + B) <(C + D) (2)
Judge whether it is. As a result, the display position of the waveform display image of the edge component is switched in parallel to the X axis when the relationship (1) is satisfied, and parallel to the Y axis when the relationship (2) is satisfied. . In the case of FIG. 12B, a threshold value is counted from the top of the size of the designated area, and the display color control in the area below the threshold value is stopped. Reduce overlap and improve visibility.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 111 System control part 115 Video control part 122 1st display part 201 Preprocessing part 202 Area designation circuit 203 Edge extraction circuit 204 1st memory controller 205 Waveform control part 206 Bank memory part 207 1st waveform preservation | save memory (Bank1)
208 Second waveform storage memory (Bank2)
209 Post-processing unit 210 Video composition circuit 211 Second memory controller 212 Area determination circuit

Claims (12)

An imaging apparatus having a function of adjusting the focus of a lens that forms a subject image on an imaging element,
Extracting means for extracting an edge component of a video imaged by the imaging device;
Generating means for generating a waveform image representing a relationship between a position and a size in a predetermined direction of the edge component extracted by the extracting means;
Display means for displaying the video imaged by the image sensor and the waveform image on a display device;
Designating means for designating a plurality of areas for video displayed on the display means;
Determining means for determining overlap in a specific direction of the plurality of areas designated by the designation means,
It said generating means, if the overlap of the plurality of regions in said particular direction Ri I is determined to be the determination means, to generate the waveform image direction other than the specific direction as the predetermined direction An imaging device that is characterized. The determination unit determines a relative positional relationship between the plurality of regions, and when it is determined that there is an overlap between a central coordinate of a region serving as a reference and another region in the specific direction, the plurality of regions The imaging apparatus according to claim 1, wherein it is determined that there is an overlap in the area. Having detection means for detecting the vertical direction of the image sensor;
Said generating means, said predetermined direction determined according to the direction detected by the detecting means, the image pickup apparatus according to claim 1 or 2, characterized in that generating the waveform images. The imaging device according to any one of claims 1 to 3 , wherein the display unit sequentially displays the video imaged by the imaging device and the waveform image on the display device. The display means, wherein the said waveform image and captured image by the imaging device, in any one of claims 1 to 4, characterized in that the position in the predetermined direction is superimposed to coincide Imaging device. The display means, the image pickup apparatus according to any one of claims 1 to 5, characterized in that display to visually identify the plurality of regions relative to other regions. The extraction means extracts edge components at a plurality of lines in the predetermined direction of the video,
The said generation means counts each signal of the said edge component extracted in each position in the said predetermined direction for every height of a signal level, The said waveform image according to the said count is produced | generated. The imaging device according to any one of 1 to 6 . The imaging device according to claim 7 , wherein the generation unit generates the waveform image so that the waveform image is displayed with higher luminance as the count increases. The generating means has memory control means for controlling access to the memory,
When the video signal from the image sensor is input, the memory control unit reads data stored in an address corresponding to the position of the video signal in the predetermined direction and the magnitude of the signal level of the memory, Write back the data plus a specified value to the same address,
Said generating means includes an address of the memory, the image pickup apparatus according to any one of claims 1 to 8 based on the data stored in the address and generates the waveform image. Subject detection means for detecting a specific subject from the video imaged by the imaging device;
It said specifying means, the imaging apparatus according to any one of claims 1 to 9, characterized in that designating a plurality of regions based on a result of detection by the object detection unit. An operation means for accepting a user operation for designating the plurality of areas;
It said specifying means, the imaging apparatus according to any one of claims 1 to 10, characterized in that designating a plurality of regions based on the operation result by the operation means. A control method executed by an imaging apparatus having a function of performing focus adjustment of a lens that forms a subject image on an imaging element,
An extraction step of extracting an edge component of a video imaged by the imaging device;
A generation step for generating a waveform image representing a relationship between a position and a size in a predetermined direction of the edge component extracted in the extraction step;
A display step of displaying the video imaged by the imaging device and the waveform image on a display device;
A designation step for designating a plurality of areas for the video displayed in the display step;
A determination step of determining overlap in a specific direction of the plurality of areas specified by the specifying step,
In the generating step, the determination if the duplicate to the plurality of regions in said particular direction Ri I is determined in step, to generate the waveform image direction other than the specific direction as the predetermined direction A control method for an imaging apparatus.

Images