JP2010114556A - Imaging device, image processing device, and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology capable of highlighting a subject edge adequately. <P>SOLUTION: An imaging device 1A includes: a sub-image pickup device 7 acquiring a pickup image of a subject image; an evaluated value acquiring portion 124 acquiring an evaluated value of a high-frequency component included in the pickup image per pixel; a threshold determining portion 125 determining an edge threshold which becomes a reference for specifying an edge pixel corresponding to an edge part of a subject among pixels composing the pickup image based on the evaluated value; an edge pixel specifying portion 126 specifying the edge pixel using the edge threshold; and a display control portion 127 displaying a preview image before picking up the image on a monitor 12 based on the pickup image sequentially acquired by the sub-image pickup device 7. The display control portion 127 highlights the edge pixel in the preview image. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image display technique in an imaging apparatus or an image processing apparatus.

  Some imaging devices such as digital cameras have a function of causing a display unit to display a subject image acquired by an imaging device as a live view image. In such an imaging apparatus, the composition is determined by looking at the live view image on the display unit, and focus adjustment is performed using an autofocus (AF) function or by manual focus (MF) operation.

  When focus adjustment is performed by manual focus operation, the focus (focus) state is confirmed by looking at the live view image. However, since the display resolution of the display unit is generally low, it is difficult to determine whether or not the live view image is in focus by viewing the live view image.

  Therefore, as an auxiliary function for improving the focusing accuracy by the manual focus operation, a technique called so-called peaking is proposed in which the contour of the subject is highlighted and displayed in the live view image (for example, Patent Document 1).

JP 2008-135812 A

  However, in the technique described in Patent Document 1, the high-frequency component included in the captured image is detected to identify the edge portion of the subject, and the edge portion is highlighted. However, the subject with low brightness and / or contrast is selected. In the case of photographing, it has been difficult to detect high frequency components.

  Therefore, an object of the present invention is to provide a technique capable of detecting a high frequency component without being affected by the luminance and / or contrast of a subject and appropriately highlighting the edge of the subject.

  A first aspect of the present invention is an imaging apparatus, an imaging element that acquires a captured image related to a subject image, an evaluation value acquisition unit that acquires an evaluation value of a high-frequency component included in the captured image for each pixel, A determination means for determining, based on the evaluation value, an edge threshold value for specifying an edge pixel corresponding to an edge portion of a subject among pixels constituting the photographed image; and Specifying means for specifying an edge pixel; and display control means for displaying a preview image before actual photographing on a display unit based on the photographed images sequentially acquired by the image sensor; The edge pixels are highlighted in the preview image.

  According to a second aspect of the present invention, there is provided an image processing apparatus, an evaluation value acquisition unit that acquires an evaluation value of a high-frequency component included in an input image for each pixel, and a pixel that constitutes the image. Determining means for determining an edge threshold value for specifying an edge pixel corresponding to the edge portion of the subject based on the evaluation value; specifying means for specifying the edge pixel using the edge threshold value; Display control means for displaying a moving image on a display unit based on the sequentially input images, and the display control means emphasizes and displays the edge pixels in the moving image.

  According to the present invention, it is possible to detect a high-frequency component from a captured image without being affected by the luminance and / or contrast of the subject, and to appropriately highlight the edge of the subject.

  Embodiments of the present invention will be described below with reference to the drawings.

<1. First Embodiment>
[1-1. Constitution]
1 and 2 are diagrams showing an external configuration of an imaging apparatus 1A according to the first embodiment of the present invention. Here, FIG. 1 is a front external view of the image pickup apparatus 1A, and FIG. 2 is a rear external view of the image pickup apparatus 1A. This imaging device 1A is configured as a lens interchangeable single-lens reflex digital camera.

  As shown in FIG. 1, the imaging device 1 </ b> A includes a camera body (camera body) 2. An interchangeable photographic lens unit (interchangeable lens) 3 can be attached to and detached from the camera body 2.

  The photographic lens unit 3 is mainly composed of a lens barrel 36, a lens group 37 (see FIG. 3) and an aperture (not shown) provided inside the lens barrel 36, and the like. The lens group 37 includes a focus lens that changes the focal position by moving in the optical axis direction.

  The taking lens unit 3 is provided with a focus ring 35 (see FIG. 3). The focus ring 35 is rotatable with respect to the lens barrel 36, and the rotation amount thereof is detected by a rotation amount detection sensor 34 (see FIG. 5) built in the photographing lens unit 3. Then, the lens position of the focus lens (more specifically, the lens position in the optical axis direction) is changed according to the rotation amount by the user's rotation operation. As described above, in the imaging apparatus 1A, the user adjusts the focus position (image formation position) of the subject image by manual operation (manual operation), that is, manual focusing (hereinafter referred to as “manual focus” or simply “MF”). Operation).

  In the imaging apparatus 1A, not only such manual focus operation but also automatic focusing (hereinafter referred to as “auto focus”) is performed as an operation for adjusting the focus position by changing the lens position of the focus lens, that is, the focus operation. Alternatively, an operation (also simply referred to as “AF”) can be performed. Details of the AF operation will be described later.

  Note that switching between the MF mode in which the focus adjustment is manually performed and the AF mode in which the focus adjustment is automatically performed can be performed by operating a switching button included in the setting button group 83 (described later), for example.

  The camera body 2 includes an annular mount Mt to which the photographing lens unit 3 is attached at the front center, and an attach / detach button 89 for attaching / detaching the photographing lens unit 3 near the annular mount Mt. Yes.

  Further, the camera body 2 is provided with a mode setting dial 82 in the upper left part of the front surface and a control value setting dial 86 in the upper right part of the front surface. When the mode setting dial 82 is operated, various camera modes (such as various shooting modes (people shooting mode, landscape shooting mode, etc.), playback modes for playing back captured images, and communication modes for communicating data with external devices, etc. Setting operation (switching operation) can be performed. Further, by operating the control value setting dial 86, it is possible to set control values in various shooting modes.

  Further, the camera body 2 includes a grip portion 14 for gripping the imaging device 1A at the front left end. A release button 11 for instructing the start of exposure is provided on the upper surface of the grip portion 14. A battery storage chamber and a card storage chamber are provided inside the grip portion 14. For example, four AA batteries are housed in the battery compartment as a power source for the camera, and a memory card 90 (see FIG. 5) for recording image data of a photographed image is detachable in the card compartment. Can be stored.

  The release button 11 is a two-stage detection button that can detect two states, a half-pressed state (S1 state) and a fully-pressed state (S2 state).

  When the release button 11 is half-pressed to enter the S1 state, a preparation operation (for example, an AF control operation and an AE control operation) for acquiring a recording still image (main captured image) related to the subject is performed.

  In addition, when the release button 11 is further pushed into the S2 state, a shooting operation of the actual captured image (also referred to as “main shooting operation”) is executed. In the main photographing operation, an exposure operation related to the subject image is performed using an image sensor (or also referred to as “main image sensor”) 5 (described later). Then, the actual captured image is acquired by performing predetermined image processing on the image signal acquired by the exposure operation.

  In FIG. 2, a monitor 12 is provided in the approximate center of the back surface of the camera body 2. The monitor 12 is configured as a color liquid crystal display (LCD), for example. The monitor 12 can display a menu screen for setting shooting conditions and the like, and can reproduce and display a captured image recorded in the memory card 90 in the reproduction mode.

  A finder window 10 is provided at the upper center of the back surface of the camera body 2. The subject image from the photographic lens unit 3 is guided to the finder window 10, and the photographer can view an image equivalent to the subject image acquired by the main imaging element 5 by looking into the finder window 10. . Specifically, the subject image incident on the photographing optical system is reflected upward by the mirror mechanism 6 (see FIG. 3) and viewed through the eyepiece lens 67. Thus, the photographer can determine the composition by looking through the finder window 10. In addition, when the photographing operation of the main photographed image is started by detecting the S2 state of the release button 11, the mirror mechanism 6 is retracted from the light path of the light that forms the subject image. As a result, light from the photographing lens unit 3 (light that forms a subject image) reaches the main image sensor 5, and a captured image (image data) relating to the subject is acquired by the main image sensor 5.

  An eyepiece detection sensor 13 is provided below the finder window 10. The eyepiece detection sensor 13 is a sensor that detects the presence or absence of a nearby object, and detects whether or not the finder is used by the photographer.

  A main switch 81 is provided at the upper left of the monitor 12. The main switch 81 is composed of two slide switches. When the contact is set to the left “OFF” position, the imaging apparatus 1A is turned off, and when the contact is set to the right “ON” position, the imaging is performed. The apparatus 1A is turned on.

  A direction selection key 84 and a display changeover switch 9 are provided on the right side of the monitor 12. The direction selection key 84 has a circular operation button, and the upper, lower, left, and right direction pressing operations of the operation button and the upper right, upper left, lower right, and lower left pressing directions can be detected. ing. In addition, the direction selection key 84 can also detect the pressing operation of the push button at the center, in addition to the pressing operation in the eight directions.

  The display changeover switch 9 is composed of three slide switches. When the contact of the display changeover switch 9 is set to the upper “optical” position, the OVF mode is selected, and the subject image is displayed in the optical finder field of view. Thus, the photographer can visually recognize the display in the optical finder field through the finder window 10 and can perform a composition determination operation (also referred to as “framing”) while viewing the display in the finder field. Become.

  Further, when the contact of the display changeover switch 9 is set at the lower “liquid crystal” position, the EVF mode (described later) is selected. In the EVF mode, a live view image (also referred to as “preview image” or “moving image”) related to the subject image is displayed on the monitor 12 in a moving image manner (also referred to as “live view display” or “preview display”). . Thus, the photographer can perform framing while viewing the live view display displayed on the monitor 12.

  Further, when the contact of the display changeover switch 9 is set to the middle “automatic” position, the display in the optical finder field of view and the live view display are automatically switched according to the presence or absence of the eyepiece on the finder window 10. . Thus, the photographer can perform framing by visually recognizing either the display in the field of view of the optical viewfinder or the live view display.

  On the left side of the monitor 12, a setting button group 83 including a plurality of buttons for setting a menu screen, deleting an image, and the like is provided.

  Next, the internal configuration of the imaging apparatus 1A will be described. 3 and 4 are cross-sectional views of the image pickup apparatus 1A according to the first embodiment.

  As shown in FIG. 3, in the imaging apparatus 1 </ b> A, a finder unit 102 (finder optical system), a mirror mechanism 6, a phase difference AF module (hereinafter also simply referred to as “AF module”) 20, a shutter 4, main imaging. An element 5 and a sub imaging element 7 are provided.

  The main image sensor (here, a CCD sensor (also simply referred to as a CCD)) 5 is disposed on a plane perpendicular to the optical axis L on the optical axis L of the lens group 37 provided in the photographing lens unit 3. The main imaging element 5 converts the subject image received on the imaging surface into an electrical signal by photoelectric conversion action, and generates an image signal related to the actual captured image.

  A shutter 4 is disposed immediately before the main image sensor 5. The shutter 4 is a mechanical focal plane shutter that includes a curtain body that moves in the vertical direction and performs an optical path opening operation and an optical path blocking operation of subject light guided to the main image sensor 5 along the optical axis L.

  On the optical axis L, a mirror mechanism 6 (reflecting plate) is disposed at a position where the subject light is reflected toward the finder unit 102. The subject light that has passed through the photographic lens unit 3 is reflected upward by a mirror mechanism 6 (a main mirror 61 described later) and forms an image on a focusing screen (focus glass) 63.

  The finder unit 102 includes a pentamirror 65, a half mirror 68, an eyepiece lens 67, a sub imaging device 7, and a finder window 10. The pentamirror 65 replaces the top and bottom of the subject image with reflection to make an upright image. The half mirror 68 is disposed between the pentamirror 65 and the eyepiece 67 and separates (branches) subject light. The separated subject light is guided to the eyepiece lens 67 and the sub imaging device 7 respectively. The eyepiece 67 guides the separated subject light to the outside of the finder window 10. As a result, the photographer can visually recognize the subject image through the finder window 10. In this way, the finder unit 102 functions as an optical finder (OVF) for confirming the object scene during shooting standby.

  Further, the sub imaging element 7 receives the other separated subject light, and sequentially acquires captured images related to the subject image. The acquired captured images are sequentially displayed on the monitor 12 in a moving image manner (live view display). In this way, the subject image that can be viewed from the finder window 10 is acquired by the sub-imaging device 7, and the photographer can view the live view image related to the subject image on the monitor 12.

  The mirror mechanism 6 includes a main mirror 61 and a sub mirror 62. The sub mirror 62 is rotatably provided on the back side of the main mirror 61 so that the sub mirror 62 is tilted toward the back of the main mirror 61. The main mirror 61 is configured as a half mirror, for example, and transmits part of the subject light. Part of the subject light transmitted through the main mirror 61 is reflected by the sub-mirror 62 and enters the AF module 20.

  The mirror mechanism 6 is configured as a so-called quick return mirror, which jumps upward at the time of exposure when the S2 state of the release button 11 is detected, and stops at a position below the focusing screen 63 (see FIG. 4). Thereby, the subject light from the photographic lens unit 3 is guided to the imaging surface of the main imaging element 5 without being blocked by the mirror mechanism 6, and the main imaging element 5 is exposed. When the exposure is completed, the mirror mechanism 6 returns to the original position (position shown in FIG. 3).

  The AF module 20 is configured as a so-called AF sensor including a line sensor that detects focus information of a subject. Specifically, the AF module 20 receives light from a subject in a distance measuring area (also referred to as “AF area”) set in the shooting area by a pair of line sensors for the distance measuring area. The AF module 20 has a function of outputting outputs from the pair of line sensors to the overall control unit 101 as phase difference detection signals.

[1-2. Function block]
Next, an outline of functions of the imaging apparatus 1A will be described. FIG. 5 is a block diagram illustrating a functional configuration of the imaging apparatus 1A according to the first embodiment.

  As illustrated in FIG. 5, the imaging apparatus 1 </ b> A includes a phase difference AF module 20, an operation unit 80, an overall control unit 101, a mirror mechanism 6, a shutter 4, a main imaging device 5, a sub imaging device 7, and an A / D conversion circuit 48. The image processing unit 50 and the image memory 56 are provided.

  The operation unit 80 includes various buttons and switches including the release button 11 (see FIG. 1). In response to a user input operation on the operation unit 80, the overall control unit 101 implements various operations.

  In response to a drive control signal input from a timing control circuit (not shown), the main image sensor 5 performs exposure (charge accumulation by photoelectric conversion) of a subject image formed on a light receiving surface (imaging surface), An image signal related to the subject image is generated.

  The sub-image sensor 7 basically has the same function as the main image sensor 5, exposes the subject image guided to the finder optical system, and acquires an image signal related to an image for live view display. .

  Image signals (analog signals) acquired by the main image sensor 5 and the sub image sensor 7 are converted into digital signals by the A / D conversion circuit 48, respectively. The image signal converted into the digital signal is input to the image processing unit 50.

  The image data input to the image processing unit 50 is temporarily stored in the image memory 56. Thereafter, the image data stored in the image memory 56 is accessed, and various processes of the image processing unit 50 are executed.

  Image data acquired by the main image sensor 5 at the time of the main photographing is appropriately subjected to image processing by the image processing unit 50 and then stored in the memory card 90.

  Further, the image data acquired by the sub imaging device 7 is subjected to predetermined processing in the A / D conversion circuit 48 and the image processing unit 50, temporarily stored in the image memory 56, and then displayed on the monitor 12.

  The image memory 56 is a high-speed accessible memory for temporarily storing generated image data, and has a capacity capable of storing image data for a plurality of frames.

  Here, various processes realized by the image processing unit 50 will be described in detail. The image processing unit 50 includes a white balance (WB) control unit 51, a pixel interpolation unit 52, a gamma correction unit 53, and a YCC conversion unit 54.

  The white balance control unit 51 performs white balance correction for converting the digital signal level of each color component of R (red), G (green), and B (blue). In this white balance correction, a portion that is originally white from a photographic subject is estimated from brightness, saturation data, and the like, and an average value, a G / R ratio, and a G / B ratio for each of R, G, and B are obtained. The level is corrected as R and B correction gains.

  The pixel interpolation unit 52 obtains a color component lacking in pixels in the imaging data stored in the image memory 56 by interpolation using color information of peripheral pixels adjacent to the pixel.

  The gamma correction unit 53 corrects the gradation characteristics of the image signal subjected to WB adjustment. Specifically, the gamma correction unit 53 uses a preset gamma correction table to nonlinearly convert the level of the image signal for each color component and perform offset adjustment.

  The YCC converter 54 converts the color space of the image data. Specifically, the YCC conversion unit 54 converts a color space having RGB primary color components into a color space having luminance components (Y) and color difference components (Cr, Cb) by matrix calculation. Hereinafter, a luminance component signal is also referred to as a luminance signal, and a color difference component signal is also referred to as a color difference signal.

  In the present embodiment, the color space of the image data (also referred to as “auxiliary image data”) acquired by the sub image sensor 7 is converted, and a luminance signal related to the auxiliary image data (more specifically, auxiliary image data is configured. A signal value of a luminance component of each pixel (also referred to as “pixel value of luminance component” or simply “luminance value”) is acquired.

  The overall control unit 101 is configured as a microcomputer and mainly includes a CPU, a RAM, a ROM, and the like. The overall control unit 101 implements various functions by reading a program stored in the ROM and executing the program by the CPU.

  The overall control unit 101 executes a phase difference AF control unit 121, a drive control unit 122, a normalization unit 123, an evaluation value acquisition unit 124, a threshold value determination unit 125, an edge pixel specification unit 126, a display control unit by executing the above-described program. 127 and the mirror control unit 128 are functionally realized.

  The phase difference AF control unit 121 performs execution control of an automatic focusing (phase difference AF) operation using a phase difference method. Specifically, the phase difference AF control unit 121 determines the position of the photographing lens (more specifically, the focus lens) at the time of focusing (lens focusing position) based on the phase difference detection signal output from the AF module 20. The lens in-focus position specifying operation for specifying is executed.

  In addition, the phase difference AF control unit 121 cooperates with the drive control unit 122 to execute a lens driving operation for moving the photographing lens (focus lens) to the lens focusing position.

  The drive control unit 122 has a function of controlling driving of the photographing lens, and actually drives the photographing lens in cooperation with the phase difference AF control unit 121.

  Specifically, the phase difference AF control unit 121 transmits a control signal to the lens side control unit 31 of the photographing lens unit 3 via the drive control unit 122 to drive the lens driving unit 38, and The focus lens included in the lens group 37 is moved in the optical axis direction. Further, the position of the focus lens is detected by the lens position detection unit 39 of the photographing lens unit 3, and data indicating the position of the focus lens is sent from the lens side control unit 31 to the overall control unit 101 on the main body side.

  The normalizing unit 123 detects the maximum luminance value and the minimum luminance value from the luminance signals of the auxiliary image data, and normalizes the luminance values constituting the auxiliary image data.

  The evaluation value acquisition unit 124 performs a filter operation using HPF (High Pass Filter) on the normalized auxiliary image data, and also performs a filter operation value (“filter result value” or simply “evaluation value”) for each pixel. ) To get. The filter calculation value, which is the output value of the HPF, is also expressed as an evaluation value of a high-frequency component that indicates how much high-frequency component is included at each pixel position, that is, the degree of high-frequency component content. The output value of HPF can also be expressed as representing the magnitude of contrast.

  The threshold value determination unit 125 determines a threshold value (also referred to as an “edge threshold value”) as a criterion for determining whether or not each pixel is a pixel constituting an edge (also referred to as an “edge pixel”). A method for determining the edge threshold will be described later.

  The edge pixel specifying unit 126 specifies an edge pixel (also referred to as an “edge pixel”) from among the pixels using an edge threshold value.

  The display control unit 127 controls display contents on a display unit such as the monitor 12. Specifically, the display control unit 127 displays a live view image on the monitor 12 based on a plurality of images continuously acquired by the sub imaging element 7.

  The display control unit 127 has a function of highlighting and displaying the edge pixel specified by the edge pixel specifying unit 126 in the live view image.

  The mirror control unit 128 controls state switching between a state in which the mirror mechanism 6 is retracted from the optical path (mirror up state) and a state in which the mirror mechanism 6 blocks the optical path (mirror down state). The mirror control unit 128 switches between a mirror-up state and a mirror-down state by driving and controlling a mirror switching motor (not shown).

[Operation]
Hereinafter, an operation of the imaging apparatus 1A when the MF mode is selected will be described. FIG. 6 is a flowchart of the edge display operation of the imaging apparatus 1A. 7, FIG. 8, and FIG. 9 are diagrams showing how pixel values are normalized. FIG. 10 is a diagram illustrating a histogram related to the filter calculation value of the auxiliary image data after normalization. FIG. 11 is a diagram illustrating a live view image having highlighted edges. FIG. 12 is a diagram illustrating a histogram relating to filter calculation values of two auxiliary image data acquired in different shooting environments.

  In the MF mode, the rotation amount of the focus ring 35 is detected by the rotation amount detection sensor 34, and the lens-side control unit 31 drives and controls the lens position of the focus lens according to the detected rotation amount.

  When performing framing using live view display in the MF mode, the edge of the subject is emphasized in the live view image displayed on the monitor 12. Specifically, the description will be made with reference to the flowchart of the edge display operation of FIG.

  As shown in FIG. 6, when the EVF mode is selected in the shooting mode, auxiliary image data is acquired by the sub image sensor 7 (step SP11), and live view display using the monitor 12 is started.

  In step SP12, it is determined whether or not the MF mode is selected. When the MF mode is not selected (that is, when the AF mode is selected), the process proceeds to step SP11, and normal live view display is continuously executed. On the other hand, if the MF mode is selected, the process proceeds to step SP13.

  In step SP13, it is determined whether or not the edge information display function for extracting the edge information included in the auxiliary image data and superimposing it on the live view image is enabled. If the edge information display function is not enabled (disabled), the process proceeds to step SP11, and the user performs a manual focus operation in a state where normal live view display is performed. On the other hand, if the edge information display function is enabled, the process proceeds to step SP14, and thereafter each process for displaying edge information (the processes from step SP14 to step SP19 are collectively referred to as “edge information display process”). Also called).

  The edge information display function may be validated / invalidated by a menu operation using a menu screen, or by a button operation of an edge information display button (not shown) provided in the imaging apparatus 1A. It is good.

  In step SP14, the color space of the auxiliary image data is converted by the YCC conversion unit 54, and based on the pixel value of the primary color component of each pixel constituting the auxiliary image data, the pixel value (brightness value) of each pixel. ) Is acquired.

  In step SP15, the normalization unit 123 normalizes the luminance values constituting the auxiliary image data.

  Specifically, the maximum luminance value (maximum luminance value) “Pma” and the minimum luminance value (minimum luminance value) “Pmi” in the auxiliary image data are detected (see FIG. 7). Then, the minimum luminance value “Pmi” is subtracted from the luminance value “Pj” of each pixel Gj (see FIG. 8), and the maximum luminance value “Pmb” among the subtracted luminance values is the maximum value as the pixel value. (For example, “255” for an 8-bit pixel) is amplified (see FIG. 9). By such normalization, the newly calculated luminance value “PNj” of each pixel is expressed as shown in Expression (1). Note that the subscript j represents the jth pixel.

  Note that, when the difference between the maximum luminance value “Pma” and the minimum luminance value “Pmi” is small, noise may be excessively amplified if normalization based on Expression (1) is performed. For this reason, when the difference between the maximum luminance value “Pma” and the minimum luminance value “Pmi” is smaller than a predetermined threshold, the amplification factor is limited.

  In step SP16, the evaluation value acquisition unit 124 performs filter calculation using HPF on the normalized auxiliary image data, and acquires a filter calculation value for each pixel.

  In step SP17, the threshold value determination unit 125 determines an edge threshold value that serves as a reference (index) for specifying the edge pixel corresponding to the edge portion of the subject. The edge threshold is determined every time new auxiliary image data is acquired in step SP11. Specifically, the edge threshold candidate value (also referred to as “threshold candidate value”) obtained from the newly acquired auxiliary image data is compared with the edge threshold before the auxiliary image data acquisition, and the larger one is the newer one. The edge threshold value is determined.

  The threshold candidate value is determined using the characteristic that the filter calculation value of the pixel corresponding to the edge portion of the subject in the image is relatively large. Specifically, a pixel whose filter calculation value is included in an upper predetermined ratio among all pixels is set as an edge candidate pixel, and a value for specifying such an edge candidate pixel is determined as a threshold candidate value. That is, the number of pixels is counted in order from the pixel having the large filter calculation value, and the filter calculation value given to the pixel when the total number of counted pixels becomes a predetermined ratio with respect to the total number of pixels is the threshold candidate value. Become.

  For example, in the situation in which the histogram of the filter calculation values related to the auxiliary image data is shown in FIG. In this case, the filter calculation value “FN” of the pixel when the number of pixels counted in order from the pixel having the maximum filter calculation value becomes 10% of the total number of pixels is the threshold value of the new auxiliary image data. Candidate value.

  Then, the threshold value candidate value of the new auxiliary image data is compared with the edge threshold value before the auxiliary image data is acquired, and the larger value becomes the new edge threshold value.

  As described above, the edge threshold value is determined every time auxiliary image data is acquired after the MF mode is selected. When a threshold candidate value higher than the current edge threshold value is acquired, the edge threshold value is updated. .

  Note that the edge threshold value determined by the threshold value determination unit 125 is reset after the actual photographing or after the MF mode is canceled. In addition, when auxiliary image data is acquired for the first time after the MF mode is selected, there is no current edge threshold value to be compared, and therefore a threshold candidate value for the auxiliary image data is determined as the edge threshold value.

  In step SP18, the edge pixel specifying unit 126 specifies an edge pixel from among the pixels using the edge threshold value. That is, a pixel having a filter calculation value larger than the edge threshold is specified as an edge pixel.

  In step SP19, the edge pixel is colored with a specific color (for example, red) by the display control unit 127 and displayed with emphasis on the monitor 12 (see FIG. 11).

  In step SP20, it is detected whether or not the release button 11 has been pressed down to a fully pressed state. When the fully-pressed state of the release button 11 is detected, the process proceeds to step SP21 and the actual photographing operation is executed.

  On the other hand, when the fully pressed state of the release button 11 is not detected, the process proceeds to step SP11, and the processes of steps SP11 to SP20 are repeatedly executed until the fully pressed state of the release button 11 is detected.

  When the focus ring 35 is operated by the user while the repetitive process is being performed, the focus state (also referred to as “focus state” or “focus state”) changes. At this time, in step SP17, the edge threshold value is determined according to the change in the focus state. Specifically, when the focus state is improved and the in-focus state is approached, the edge becomes clear and the number of pixels having a relatively large filter calculation value increases, so that the threshold candidate value increases and the edge threshold value also increases.

  On the other hand, for example, when the focus state is deteriorated from the in-focus state to the out-of-focus state, the edge becomes unclear and the filter calculation value of the edge portion becomes small. The edge threshold value is not changed by comparison with the edge threshold value. That is, the edge threshold value when the focus state is good is maintained.

  Thus, in a series of focus adjustment processes by manual operation of the focus ring 35, the edge threshold value is held at the maximum value acquired in the focus adjustment process, so that the focus state deteriorates and the filter calculation value of each pixel. As becomes smaller, the edge pixels highlighted are reduced. Accordingly, the user can grasp the focus state by checking the amount of edge pixels (edge information amount) highlighted in the live view image displayed on the monitor 12 while manually operating the focus ring 35. it can. For example, by specifying a state where the amount of edge information displayed on the monitor 12 is the largest, the user can recognize that an appropriate in-focus state has been realized.

  Further, since the edge threshold value is changed according to the luminance and / or contrast of the subject, the edge of the subject with low luminance or contrast can be appropriately highlighted.

  The histogram relating to the filter calculation value of the image data changes according to the luminance and / or contrast of the subject. Specifically, as in the graph GH1 indicated by the one-dot chain line in FIG. 12, the filter calculation value of each pixel is relatively small in the image data relating to the subject with low luminance and / or contrast. For this reason, when the luminance and / or contrast of the subject is low, the threshold candidate value that defines the pixel whose filter calculation value is included in the upper predetermined ratio among all the pixels is small, and thus the edge threshold value “EH1” is Get smaller.

  On the other hand, as shown in the graph GH2 indicated by the broken line in FIG. 12, in the image data related to the subject with high brightness and / or contrast, the filter calculation value of each pixel is relatively large. For this reason, when the luminance and / or contrast of the subject is high, the threshold candidate value increases, and the edge threshold “EH2” increases accordingly.

  Thus, since the edge threshold value is changed according to the luminance and / or contrast of the subject, when photographing a subject with low luminance and / or contrast, the edge threshold value is set to be relatively small, Appropriate edge highlighting according to contrast is possible.

  As described above, in the imaging apparatus 1A, the edge threshold value is determined using the evaluation value of the high-frequency component that changes according to the luminance and / or contrast of the subject. Therefore, the edge threshold value is determined according to the luminance and / or contrast of the subject. Can be determined to an appropriate value. If such an appropriate edge threshold is used, the edge pixel can be specified without being affected by the luminance and / or contrast of the subject, and the edge of the subject can be appropriately highlighted.

<2. Second Embodiment>
Next, a second embodiment of the present invention will be described. The imaging apparatus 1A according to the first embodiment uses the acquired auxiliary image data as it is to identify the edge pixel in the auxiliary image, but the imaging apparatus 1B according to the second embodiment acquires the acquired auxiliary image. The image is divided into a plurality of regions, and edge pixels are specified for each of the divided regions (also referred to as “divided regions”). FIG. 13 is a block diagram illustrating a functional configuration of the imaging apparatus 1B according to the second embodiment.

  Note that the imaging apparatus 1B according to the second embodiment has substantially the same structure and function (see FIGS. 1 to 4) as the imaging apparatus 1A, except that the image processing unit 50 includes a region dividing unit 55. The common parts are denoted by the same reference numerals and the description thereof is omitted.

  As illustrated in FIG. 13, the image processing unit 50 of the imaging apparatus 1 </ b> B further includes a region dividing unit 55 that divides the auxiliary image acquired by the sub imaging element 7 into a plurality of regions.

  There are a plurality of area division methods. Here, the operation of the imaging apparatus 1B when the auxiliary image is divided into a predetermined number of areas will be described. FIG. 14 is a flowchart of the edge display operation of the imaging apparatus 1B when the MF mode is selected. FIG. 15 is a diagram illustrating an example of area division of the auxiliary image.

  As shown in FIG. 14, in the imaging apparatus 1B, when the EVF mode is selected in the shooting mode, auxiliary image data is acquired by the sub imaging element 7 (step SP11), and live view display using the monitor 12 is started. Is done.

  When the MF mode is selected (step SP12) and the edge information display function is enabled (step SP13), the edge information display process (steps SP14 to SP19) is executed.

  In the edge information display process of the imaging apparatus 1B, after the color space of the auxiliary image data is converted by the YCC conversion unit 54 (step SP14), in the next step SP31, the auxiliary image is set in advance by the area dividing unit 55. Divided into a number of regions.

  Then, the normalization process (step SP15), the evaluation value acquisition process (step SP16), the edge threshold value determination process (step SP17), and the edge position specifying process (step SP18) described above are executed for each divided area. . For example, as shown in FIG. 15, when the region RH of the auxiliary image is divided into nine equal parts, the edge threshold value is determined for each of the divided regions BR1 to BR9, and the edge pixel is specified for each of the divided regions BR1 to BR9. Is done.

  As described above, in the imaging apparatus 1B, the edge threshold value corresponding to the luminance and contrast of each divided region is determined for each divided region, and edge emphasis display (step SP19) based on each edge threshold value is performed.

  According to this, since an edge is displayed for each subdivided area, even if the difference in luminance is large for each part in the auxiliary image (in short, luminance unevenness in the auxiliary image is large), the auxiliary image It is possible to effectively display the edge of a relatively low brightness object.

<3. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

  For example, in the second embodiment, the technique TE1 for dividing the auxiliary image into a fixed number of areas set in advance is used as a technique for dividing the auxiliary image area. However, the present invention is not limited to this. Specifically, the following methods TE2 and TE3 may be employed instead of the method TE1.

Region division method TE2: The number of divided regions is determined based on the maximum value of the filter calculation values. The region division method TE2 calculates the maximum filter calculation value among the filter calculation values of each pixel acquired by the evaluation value acquisition unit 124. The number of divided regions (also simply referred to as “number of divisions”) is determined using the maximum filter calculation value. FIG. 16 is a flowchart of the edge display operation when the area division method TE2 is employed. FIG. 17 is a diagram illustrating a correspondence relationship between the maximum value of the filter calculation value and the number of divided areas.

  Specifically, the same processing as that of the imaging device 1B is executed from step SP11 to step SP14.

  In step SP51, it is determined whether or not the auxiliary image data acquired in step SP11 is the first acquired after the edge information display function is activated.

  If it is the first auxiliary image data, the process proceeds to step SP15 and normalization is performed, and then the filter calculation values of all the pixels are acquired in step SP16.

  In step SP54, the maximum filter calculation value is specified from the filter calculation values of all pixels, and the number of sub-regions of the auxiliary image is determined based on the maximum filter calculation value. The number of divided areas is determined based on a predetermined correspondence relationship in which the number of divided areas increases as the maximum value of the filter calculation value increases as shown in FIG. For example, when the maximum value of the filter calculation value is “Fma”, the number of divided areas is determined to be “25 (5 × 5)” based on the correspondence relationship of FIG.

  On the other hand, if the auxiliary image data acquired in step SP11 is not the first acquired after the edge information display function is activated, the process proceeds to step SP52 (step SP51), and the area of the auxiliary image is determined in step SP54. It is divided into the number of divided areas. Then, edge information display processing is executed for each divided region.

  That is, in the edge display operation employing the region dividing method TE2, the number of divided regions is determined using the first auxiliary image data, and the auxiliary image data acquired after the second time depends on the number of divided regions. Edge information display processing is executed for each of the divided areas.

  As described above, according to the region division method TE2, the number of divided regions increases as the maximum value of the filter calculation value increases. Therefore, for a subject with high luminance and / or contrast, an edge is provided for each subdivided region. Is likely to be highlighted. According to this, when a subject with high brightness and / or contrast is taken as an object to be photographed, more detailed edge display is performed in a relatively fine area, so that it is easy to grasp the focus state.

Region division method TE3: Dividing on the basis of the pixel having the maximum filter calculation value The region division method TE3 has the maximum filter calculation value among the filter calculation values of each pixel acquired by the evaluation value acquisition unit 124. The pixel is specified as a reference pixel, and the size of the divided region is changed according to the distance from the reference pixel. FIG. 18 is a diagram illustrating an example of region division when the region RH of the auxiliary image is divided by the region division method TE3.

  More specifically, in the region division method TE3, the auxiliary image is divided so that the size of the divided region increases with distance from the reference pixel with the reference pixel as the center. For example, as shown in FIG. 18, when the reference pixel BP is detected near the center of the auxiliary image, a small divided region RS is set in a region that is relatively close to the reference pixel BP in the auxiliary image, and is compared with the reference pixel. A large divided region RB is set in a far region.

  According to such a region division method TE3, it is possible to realize appropriate region division according to the luminance and / or contrast in the auxiliary image, and thus it is possible to suppress an increase in the amount of calculation due to a wasteful increase in divided regions. . Further, according to the region dividing method TE3, in a region having a high luminance and / or contrast in the auxiliary image, since the edge is displayed in a relatively fine manner and detailed edges are displayed, it is easier to grasp the focus state.

  In addition, the imaging devices 1A and 1B according to the above embodiments may have a function of enlarging and displaying a part of the auxiliary image displayed on the monitor 12. FIG. 19 is a diagram illustrating a state in which a part of the auxiliary image is enlarged and displayed on the monitor 12.

  For example, as shown in FIG. 19, by enlarging and displaying a part of the auxiliary image, it becomes easy to confirm the focus state of the part to be focused.

  The enlarged display may be executed in response to a button operation of an enlarged display button (not shown) provided in the imaging devices 1A and 1B, and is executed when a non-operation state of the focus ring 35 has elapsed for a certain time. You may do it. Further, the portion to be enlarged and displayed may be specified by the user.

  In each of the above embodiments, the case where the edge information display process is executed in the imaging devices 1A and 1B is illustrated, but the present invention is not limited to this.

  Specifically, an edge information display process may be performed on an input image in an image processing apparatus that inputs an image acquired by an imaging element or the like from the outside.

1 is a diagram illustrating an external configuration of an imaging apparatus according to a first embodiment of the present invention. 1 is a diagram illustrating an external configuration of an imaging apparatus according to a first embodiment of the present invention. It is a longitudinal cross-sectional view of the imaging device which concerns on 1st Embodiment. It is a longitudinal cross-sectional view of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows the function structure of the imaging device which concerns on 1st Embodiment. It is a flowchart of the edge display operation | movement of the imaging device which concerns on 1st Embodiment. It is a figure which shows the mode of normalization of a pixel value. It is a figure which shows the mode of normalization of a pixel value. It is a figure which shows the mode of normalization of a pixel value. It is a figure which shows the histogram regarding the filter calculation value of the auxiliary image data after normalization. It is a figure which shows the live view image which has the highlighted edge. It is a figure which shows the histogram regarding the filter calculation value of auxiliary image data. It is a block diagram which shows the function structure of the imaging device which concerns on 2nd Embodiment. It is a flowchart of the edge display operation | movement of the imaging device which concerns on 2nd Embodiment. It is a figure which shows the area | region division example of an auxiliary image. It is a flowchart of the edge display operation | movement which concerns on a modification. It is a figure which shows the correspondence of the maximum value of a filter calculation value, and the number of division areas. It is a figure which shows the area | region division example of an auxiliary image. It is a figure which shows a mode that a part of auxiliary image is expanded and displayed on a monitor.

Explanation of symbols

1A, 1B Imaging device 7 Sub imaging device 34 Rotation amount detection sensor 35 Focus ring 50 Image processing unit 51 White balance control unit 52 Pixel interpolation unit 53 Gamma correction unit 54 YCC conversion unit 55 Region division unit 123 Normalization unit 124 Evaluation value acquisition Unit 125 threshold value determination unit 126 edge pixel specifying unit 127 display control unit

Claims (6)

An image sensor for acquiring a captured image related to the subject image;
Evaluation value acquisition means for acquiring an evaluation value of a high-frequency component contained in the captured image for each pixel;
A determining means for determining an edge threshold value that is a reference for specifying an edge pixel corresponding to an edge portion of a subject among pixels constituting the photographed image based on the evaluation value;
Identifying means for identifying the edge pixel using the edge threshold;
Display control means for displaying a preview image before actual photographing on a display unit based on the photographed images sequentially acquired by the image sensor;
With
The image pickup apparatus, wherein the display control means emphasizes and displays the edge pixel in the preview image. Focus adjusting means for adjusting the focus position of the subject image in accordance with a manual operation;
Further comprising
The imaging apparatus according to claim 1, wherein the determination unit holds a maximum value of the edge threshold acquired in a focus adjustment process using the focus adjustment unit, and uses the maximum value as an edge threshold. Area dividing means for dividing the captured image into a plurality of areas;
Further comprising
The determining means determines an edge threshold value for each of the plurality of regions,
The imaging device according to claim 2, wherein the specifying unit specifies the edge pixel for each of the plurality of regions using the edge threshold value determined for each of the plurality of regions.   The imaging apparatus according to claim 3, wherein the display control unit enlarges and displays a part of the preview image displayed with the edge pixels highlighted. Evaluation value acquisition means for acquiring an evaluation value of a high-frequency component included in the input image for each pixel;
A determination unit that determines an edge threshold value that is a reference for specifying an edge pixel corresponding to an edge portion of a subject among pixels constituting the image based on the evaluation value;
Identifying means for identifying the edge pixel using the edge threshold;
Display control means for displaying a moving image on a display unit based on the images sequentially input;
With
The display control means is an image processing device for emphasizing and displaying the edge pixels in the moving image. a) acquiring an image;
b) obtaining an evaluation value of a high-frequency component included in the image for each pixel;
c) determining, based on the evaluation value, an edge threshold value serving as a reference for specifying an edge pixel corresponding to an edge portion of a subject among pixels constituting the image;
d) identifying the edge pixel using the edge threshold;
e) displaying a moving image on a display unit based on the images sequentially acquired in the step a);
f) a step of highlighting and displaying the edge pixel in the moving image;
An image processing method comprising:

Images