JP5934844B2 - Image processing apparatus, imaging apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, an image processing method, and an image processing program.

  In addition to autofocus using a phase difference detection method and a contrast detection method, digital cameras having a so-called manual focus mode in which a user can manually perform focus adjustment are widely known.

  As a digital camera having a manual focus mode, a method using a split micro prism screen that displays a phase difference visually by providing a reflex mirror so that focus adjustment can be performed while checking a subject is known. ing. There is also known a digital camera that employs a method of visually confirming contrast.

  By the way, in a digital camera that omits a reflex mirror that has been widely used in recent years, there is no method for checking a subject image while displaying a phase difference because there is no reflex mirror, and it has been necessary to rely on a contrast detection method. However, in this case, it is impossible to display a contrast higher than the resolution of a display device such as an LCD (liquid crystal display), and it has been necessary to adopt a method of displaying the image by partially enlarging it.

  Therefore, in recent years, a split image is displayed in a live view image (also referred to as a through image) in order to facilitate a user (for example, a photographer) to focus on a subject in the manual focus mode. A split image is, for example, a divided image in which a display area is divided into a plurality of parts (for example, each image divided in the vertical direction). In the combined state, it indicates a divided image in which there is no deviation in the parallax generation direction. The user adjusts the focus by operating a manual focus ring (hereinafter referred to as “focus ring”) so that the split image (for example, each image divided in the vertical direction) is not displaced.

  Here, the principle of the split image will be described with reference to the imaging device described in Patent Document 1. The imaging apparatus described in Patent Literature 1 includes a first subject image and a second subject image that are photoelectrically converted from the first subject image and the second subject image formed by the light flux divided by the pupil splitting unit out of the light flux from the imaging optical system. An image and a second image are generated. Then, a split image is generated using these first and second images, and a third subject image formed by a light beam that is not split by the pupil splitting unit is photoelectrically converted to generate a third image. . Then, the third image is displayed on the display unit, and the generated split image is displayed in the third image. In addition, the imaging device described in Patent Literature 1 displays an enlarged image of an image included in a specified area of a split image.

  In Patent Document 2, a specific image among a plurality of images displayed on a touch panel display is instructed by an indicator (for example, a finger) via the touch panel, and the specified specific image is enlarged and displayed. Disclosed techniques are disclosed.

  In Patent Document 3, when an image displayed on a touch panel display is instructed by an indicator through the touch panel, an image overlapping the indicator in contact with the touch panel avoids the indication area of the indicator A technique for enlarging and displaying in the area is disclosed.

JP 2009-237214 A JP 2010-152827 A JP 2012-203644 A

  However, it is difficult for the user to grasp the degree of parallax even if any of the divided images included in the split image is indicated with an indicator via the touch panel and the indicated divided image is enlarged and displayed. Further, even if an image included in an area of the split image that overlaps the indication area of the indicator is enlarged and displayed in an area that avoids the indication area of the indicator, the enlarged image is an image of the parallax generation area. Otherwise, it is difficult for the user to grasp the degree of parallax. That it is difficult for the user to grasp the parallax from the enlarged image means that it is difficult to visually recognize whether or not the image is in focus.

  The present invention has been proposed in view of such a situation, and is an image processing device, an imaging device, an image processing method, and an image processing capable of easily recognizing whether or not it is in focus. The purpose is to provide a program.

  In order to achieve the above object, the image processing apparatus according to the first aspect of the present invention is configured such that the subject image that has passed through the first and second regions in the photographing lens is divided into pupils and formed respectively. 1st image of 1st and 2nd image based on 1st and 2nd image signal output from the image pick-up element which has the 1st and 2nd pixel group which outputs 1 and 2nd image signal Is divided into a first divided image obtained by dividing the second image in the dividing direction and a first divided image obtained by dividing the second image in the dividing direction. A generation unit that generates a display image used for focusing confirmation in which a divided region corresponding to the divided image and a second divided image corresponding to the adjacent divided region are arranged adjacent to each other in the dividing direction; And a touch panel on the surface of the display area A detected display unit, and a detection unit that detects an instruction operation that indicates the first or second divided image of the display image displayed in the display region by detecting an instruction region of the indicator on the touch panel; The display image generated by the generation unit is displayed in the display area, and when the instruction operation is detected by the detection unit in a state where the display image is displayed in the display area, the display image is selected from the first and second divided images. Display a partial image including at least a part of the instruction divided image instructed by the instruction operation and at least a part of the adjacent divided image adjacent to the instruction divided image of the first and second divided images in a predetermined direction. A control unit that extracts the partial image extracted from the work image and displays the extracted partial image in a region different from the indication region detected by the detection unit in the display region. Thereby, the image processing apparatus can make it easier to visually recognize whether or not the image processing apparatus is in focus as compared with the case where the present configuration is not provided.

  In the image processing apparatus according to the second aspect of the present invention, in the first aspect of the present invention, the control unit expands the parallax between the instruction divided image included in the partial image and the adjacent divided image. In this state, the display area is displayed in an area different from the instruction area detected by the detection unit. As a result, the image processing apparatus can make it easier to visually recognize whether or not the image processing apparatus is in focus as compared with the case where the present configuration is not provided.

  In the image processing apparatus according to the third aspect of the present invention, in the first aspect or the second aspect of the present invention, the control unit extracts the extracted partial image from the indication region detected by the detection unit in the display region. Are enlarged and displayed in different areas. As a result, the image processing apparatus can make it easier to visually recognize whether or not the image processing apparatus is in focus as compared with the case where the present configuration is not provided.

  In the image processing device according to the fourth aspect of the present invention, in any one of the first to third aspects of the present invention, the control unit detects the extracted partial image by the detection unit in the display area. Control for displaying in a different area from the designated instruction area is less than a predetermined value in the brightness difference in the image at a position corresponding to the instruction area detected by the detection unit in accordance with the instruction operation in the display image If not, do not. As a result, the image processing apparatus can display the extracted partial image in the designated area when it is difficult to visually recognize whether or not the extracted partial image is in focus even if it is displayed in the area avoiding the designated area. Can be suppressed from being displayed in the area where the problem is avoided.

  In the image processing device according to the fifth aspect of the present invention, in any one of the first to fourth aspects of the present invention, an area different from the designated area is an indication area detected by the detection unit. When included in the central portion of the display image in the parallax generation direction, it is an area other than the central portion of the display image in the parallax generation direction. As a result, the image processing apparatus has difficulty in visually recognizing the partial image displayed in the area avoiding the instruction area due to the blind spot formed by the indicator that is in contact with the area corresponding to the center of the display area on the touch panel. Can be suppressed.

  In the image processing device according to the sixth aspect of the present invention, in the fifth aspect of the present invention, the region other than the central portion in the parallax generation direction in the display image is a region other than the central portion in the parallax generation direction in the display region. It is. Thereby, the image processing apparatus can suppress that it becomes difficult to visually recognize the display image by displaying the partial image in the area avoiding the instruction area.

  In the image processing device according to the seventh aspect of the present invention, in any one of the first to sixth aspects of the present invention, an area different from the designated area is an indication area detected by the detection unit. When included in one end of both ends in the parallax generation direction in the display image, it is the end far from the pointing area in both ends in the parallax generation direction in the display area. As a result, the image processing apparatus is a portion displayed in the area that avoids the pointing area by the blind spot formed by the pointer that is in contact with the area corresponding to one of the two ends of the display area on the touch panel. It can suppress that an image becomes difficult to visually recognize.

In the image processing device according to the eighth aspect of the present invention, in any one of the first to seventh aspects of the present invention, the control unit converts an image included in the extracted partial image into a partial image. The adjustment is performed according to at least one of an enlargement ratio, a saturation, and an edge enhancement level determined according to the parallax between the instruction divided image and the adjacent divided image . Thereby, the image processing apparatus can improve the visibility of the partial image displayed in the area avoiding the designated area.

  In the image processing device according to the ninth aspect of the present invention, in any one of the first to eighth aspects of the present invention, the image sensor does not divide the subject image transmitted through the photographing lens into pupils. The image forming apparatus further includes a third pixel group that forms an image and outputs a third image signal, the generation unit further generates a captured image based on the third image signal, and the control unit uses the generation unit to display an image for display. When the captured image is generated, the captured image is displayed on the display unit, and the display image is displayed in the captured image. Thereby, the image processing apparatus can improve the image quality of the captured image as compared with the case where the captured image based on the third image signal is not generated.

  An imaging device according to a tenth aspect of the present invention is an image processing device according to any one of the first to ninth aspects of the present invention, an imaging element having first and second pixel groups, and imaging. A storage unit that stores an image generated based on an image signal output from the element. Thereby, the imaging apparatus can make it easier to visually recognize whether or not the imaging apparatus is in an in-focus state as compared to the case where the present configuration is not provided.

  In the imaging device according to the eleventh aspect of the present invention, in the tenth aspect of the present invention, the photographing lens has a focus lens that is movable along the optical axis direction, and the focus lens is moved in the optical axis direction. A first moving operation that further includes a moving unit, and moves the pointing area in a predetermined crossing direction that intersects the division direction in a state where the display image is displayed in the display area when the detecting unit detects the pointing operation; When the first movement operation is detected by the detection unit, the control unit controls the movement unit to move the focus lens in the optical axis direction according to the first movement operation. Thereby, the imaging device can easily move the focus lens to the in-focus position.

  In the imaging device according to the twelfth aspect of the present invention, in the eleventh aspect of the present invention, when the detection unit detects an instruction operation, the display area is displayed in the crossing direction with the display image displayed on the display area. When a movement operation equal to or greater than a predetermined threshold in a direction orthogonal to the second threshold value is detected as the second movement operation, the control unit performs control to maintain the extraction position of the partial image when the detection unit detects the second movement operation. As a result, the imaging apparatus can suppress the extraction of an image of a region other than the focus target region as a partial image due to the indication region being shifted in a direction not intended by the user.

  In the imaging device according to the thirteenth aspect of the present invention, in the eleventh aspect or the twelfth aspect of the present invention, the first image at a position corresponding to the indication region detected by the detection unit of the display image. And a presentation unit for presenting guidance information for guiding the moving direction of the designated area by the first movement operation based on the parallax between the first image and the second image. Thereby, the imaging apparatus can make the user easily recognize an operation necessary to move the focus lens to the in-focus position, compared to a case where the guide information is not displayed.

  In the image processing method according to the fourteenth aspect of the present invention, the subject image that has passed through the first and second regions in the photographic lens is divided into pupils to form the first and second image signals. The first image of the first and second images based on the first and second image signals output from the imaging device having the first and second pixel groups to be output is set in a predetermined division direction. A divided region corresponding to the first divided image from a plurality of divided images obtained by dividing the second image in the dividing direction and a part of the first divided image of the plurality of divided images obtained by dividing. And a second divided image corresponding to the adjacent divided area are generated adjacent to each other in the dividing direction to generate a display image used for in-focus confirmation, having a display area, and on the surface of the display area Touch panel of the display unit provided with a touch panel When the pointing operation for pointing to the first or second divided image of the display image displayed in the display area is detected by detecting the pointing area of the pointing body, and the pointing operation is detected, the first and first At least one part of the instruction divided image instructed by the instruction operation of the two divided images and at least one of the adjacent divided images adjacent to the instruction divided image of the first and second divided images in a predetermined direction. A partial image including a part is extracted from the display image, and the extracted partial image is displayed in an area different from the detected instruction area in the display area. Therefore, according to the image processing method, it is possible to make it easier to visually recognize whether or not the subject is in focus as compared with the case where the present configuration is not provided.

  An image processing program according to a fifteenth aspect of the present invention causes a computer to function as a generation unit, a detection unit, and a control unit in any one of the image processing apparatuses according to the first to ninth aspects of the present invention. belongs to. Therefore, according to the image processing program, it is possible to make it easier to visually recognize whether or not the in-focus state is achieved as compared with the case where this configuration is not provided.

  According to the present invention, it is possible to obtain an effect that it is possible to easily visually recognize whether or not the subject is in a focused state.

It is a perspective view which shows an example of the external appearance of the imaging device which is a lens interchangeable camera which concerns on 1st Embodiment. It is a rear view which shows the back side of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows an example of the principal part function which concerns on this invention of the imaging device which concerns on 1st Embodiment. It is a block diagram which shows an example of a structure of the electric system of the imaging device which concerns on 1st Embodiment. FIG. 3 is a schematic arrangement diagram illustrating an example of an arrangement of color filters provided in an imaging element included in the imaging apparatus according to the first embodiment and an arrangement of phase difference pixels with respect to the color filters. FIG. 3 is a schematic layout diagram illustrating an example of a layout of first pixels, second pixels, and normal pixels included in the image sensor according to the first embodiment. It is a schematic diagram which shows an example of each structure of the 1st pixel contained in the image pick-up element shown in FIG. 6, and a 2nd pixel. It is a block diagram which shows an example of the principal part function of the imaging device which concerns on 1st Embodiment. It is a schematic diagram with which it uses for description of the production | generation method of the split image produced | generated by the image process part contained in the imaging device which concerns on 1st Embodiment. It is a screen figure which shows an example of the split image and normal image which were displayed on the 1st display contained in the imaging device which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of the image generation process which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of the display control process which concerns on 1st Embodiment. It is an aspect figure which shows an example in the case where the instruction | indication area | region of the indicator with respect to a touch panel is detected in the state in which the live view image was displayed on the 1st display contained in the imaging device which concerns on 1st Embodiment. It is an aspect figure which shows an example of the aspect when the instruction | indication area | region of the indicator which is contacting the touch panel moves in the state by which the live view image was displayed on the 1st display contained in the imaging device which concerns on 1st Embodiment. Schematic for explaining a method for identifying a rectangular left eye image based on an instruction area of a pointer that is in contact with a touch panel from a left eye image generated by an image processing unit included in the imaging apparatus according to the first embodiment. FIG. Schematic for explaining a method for specifying a rectangular right eye image from a right eye image generated by an image processing unit included in the imaging apparatus according to the first embodiment based on an instruction area of an indicator that is in contact with the touch panel. FIG. A mode in which an indication area of an indicator is detected for an area corresponding to a position of an image having a brightness difference less than a threshold on the touch panel in a state where a live view image is displayed on the first display of the imaging apparatus according to the first embodiment. It is an aspect figure which shows an example. A mode diagram showing an example of a mode in a case where a parallax extended image is displayed in an area that avoids an instruction area in a display area in a state where a live view image is displayed on a first display included in the imaging device according to the first embodiment. It is. An example of an aspect in the case where the pointing area of the pointer for the area corresponding to the uppermost display parallax image on the touch panel is detected in a state where the split image is displayed on the first display of the imaging apparatus according to the first embodiment. FIG. It is an aspect figure which shows an example in the case where the arrow is displayed as guidance information in the split image of the out-of-focus state displayed on the 1st display of the imaging device which concerns on 1st Embodiment. It is an aspect figure which shows an example of the aspect of the split image of the focusing state displayed on the 1st display of the imaging device which concerns on 1st Embodiment. It is a flowchart which shows an example of the flow of the focusing control process which concerns on 1st Embodiment. It is a perspective view which shows an example of the external appearance of the smart phone which concerns on 2nd Embodiment. It is a block diagram which shows an example of the principal part structure of the electrical system of the smart phone which concerns on 2nd Embodiment. It is a modification of the split image which concerns on 1st and 2nd embodiment, Comprising: An example of the split image formed by dividing the 1st image and the 2nd image into the odd line and the even line, and arranging them alternately It is a schematic diagram shown. FIG. 10 is a schematic diagram showing an example of a split image according to the first and second embodiments, which is an example of a split image divided by oblique dividing lines inclined with respect to the row direction. FIG. 10 is a schematic diagram showing an example of a split image according to the first and second embodiments, which is an example of a split image divided by a grid-like dividing line. FIG. 6 is a schematic diagram showing an example of a split image formed in a checker pattern, which is a modification of the split image according to the first and second embodiments. It is a schematic arrangement | positioning figure which shows the modification of arrangement | positioning of the 1st pixel contained in an image sensor, and a 2nd pixel.

  Hereinafter, an example of an embodiment of an imaging device according to the present invention will be described with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a perspective view illustrating an example of an appearance of the imaging apparatus 100 according to the first embodiment, and FIG. 2 is a rear view of the imaging apparatus 100 illustrated in FIG.

  The imaging device 100 is a lens interchangeable camera. The imaging apparatus 100 is a digital camera that includes an imaging apparatus main body 200 and an interchangeable lens 300 that is interchangeably attached to the imaging apparatus main body 200, and a reflex mirror is omitted. The interchangeable lens 300 includes a photographing lens 16 (see FIG. 4) having a focus lens 302 that can be moved in the optical axis direction by a manual operation. The imaging apparatus main body 200 is provided with a hybrid finder (registered trademark) 220. The hybrid viewfinder 220 here refers to a viewfinder in which, for example, an optical viewfinder (hereinafter referred to as “OVF”) and an electronic viewfinder (hereinafter referred to as “EVF”) are selectively used.

  The interchangeable lens 300 is attached to the imaging apparatus main body 200 so as to be interchangeable. The lens barrel of the interchangeable lens 300 is provided with a focus ring 301 used in the manual focus mode. As the focus ring 301 is manually rotated, the focus lens 302 moves in the optical axis direction, and subject light is imaged on an imaging device 20 (see FIG. 3) described later at a focus position corresponding to the subject distance. .

  An OVF viewfinder window 241 included in the hybrid viewfinder 220 is provided on the front surface of the imaging apparatus main body 200. A finder switching lever (finder switching unit) 214 is provided on the front surface of the imaging apparatus main body 200. When the viewfinder switching lever 214 is rotated in the direction of the arrow SW, it switches between an optical image that can be viewed with OVF and an electronic image (live view image) that can be viewed with EVF (described later). The optical axis L2 of the OVF is an optical axis different from the optical axis L1 of the interchangeable lens 300. A release button 211 and a dial 212 for setting a shooting mode, a playback mode, and the like are mainly provided on the upper surface of the imaging apparatus main body 200.

  A release button 211 serving as a shooting preparation instruction unit and a shooting instruction unit is configured to detect a two-stage pressing operation of a shooting preparation instruction state and a shooting instruction state. The shooting preparation instruction state refers to, for example, a state where the image is pressed from the standby position to the intermediate position (half-pressed position), and the shooting instruction state refers to a state where the image is pressed to the final pressed position (full-pressed position) exceeding the intermediate position. Point to. In the following description, “a state where the button is pressed from the standby position to the half-pressed position” is referred to as “half-pressed state”, and “a state where the button is pressed from the standby position to the fully-pressed position” is referred to as “full-pressed state”.

  In the imaging apparatus 100 according to the first embodiment, the shooting mode and the playback mode are selectively set as the operation mode according to a user instruction. In the shooting mode, a manual focus mode and an autofocus mode are selectively set according to a user instruction. In the autofocus mode, the shooting conditions are adjusted by pressing the release button 211 halfway, and then exposure (shooting) is performed when the release button 211 is fully pressed. In other words, by setting the release button 211 half-pressed, the AE (Automatic Exposure) function works to set the exposure state, and then the AF (Auto-Focus) function works to control the focus. Shooting occurs when the button is pressed.

  A touch panel display 213, a cross key 222, a MENU / OK key 224, a BACK / DISP button 225, and an OVF viewfinder eyepiece 242 are provided on the rear surface of the imaging apparatus main body 200 shown in FIG.

  The touch panel display 213 includes a liquid crystal display (hereinafter referred to as “first display”) 215 and a touch panel 216.

  The first display 215 displays images and character information. The first display 215 is used to display a live view image (through image) that is an example of a continuous frame image obtained by capturing images in a continuous frame in the shooting mode. The first display 215 is also used to display a still image that is an example of a single frame image obtained by capturing a single frame when a still image shooting instruction is given. Further, the first display 215 is also used for displaying a reproduction image and a menu screen in the reproduction mode.

  The touch panel 216 is a transmissive touch panel and is overlaid on the surface of the display area of the first display 215. The touch panel 216 detects contact with an indicator (for example, a finger or a stylus pen). The touch panel 216 outputs detection result information indicating a detection result (whether or not an indicator touches the touch panel 216) to a predetermined output destination (for example, a later-described CPU 12 (see FIG. 4)) at a predetermined cycle (for example, 100 milliseconds). To do. The detection result information includes two-dimensional coordinates (hereinafter referred to as “coordinates”) that can specify the contact position of the indicator on the touch panel 216 when the touch panel 216 detects the contact of the indicator. If no contact is detected, no coordinates are included.

  The cross key 222 functions as a multifunction key for outputting various command signals such as selection of one or a plurality of menus, zooming, frame advance, and the like. The MENU / OK key 224 functions as a menu button for instructing to display one or more menus on the screen of the first display 215, and as an OK button for instructing confirmation and execution of the selection contents. This is an operation key that combines functions. The BACK / DISP button 225 is used for deleting a desired object such as a selection item, canceling a designated content, or returning to the previous operation state.

  FIG. 3 is a functional block diagram illustrating an example of main functions according to the present invention of the imaging apparatus 100 according to the first embodiment.

  As an example, as illustrated in FIG. 3, the imaging apparatus 100 includes a generation unit 100A, a display unit 100B, a detection unit 100C, a control unit 100D, a moving unit 100E, and a presentation unit 100F. The generation unit 100A generates a display image used for in-focus confirmation. Here, the display image refers to an image in which the first divided image and the second divided image are arranged adjacent to each other in a predetermined division direction (hereinafter referred to as “division direction”). The first divided image is the first and second output from the image sensor 20 (see FIG. 4) having first and second pixel groups (described later) on which the subject image is formed by pupil division. A partial image of a plurality of divided images obtained by dividing a first image of images (described later) in the dividing direction. In addition, the second divided image refers to a divided image extracted from a plurality of divided images obtained by dividing the second image in the dividing direction with respect to a divided region adjacent to the divided region corresponding to the first divided image. .

  The display unit 100B has a display area for displaying an image. A touch panel 216 is provided on the surface of the display area. In the first embodiment, the first display 215 (see FIGS. 1 and 4) and the second display 247 (see FIG. 4) are employed as an example of the display unit 100B shown in FIG.

  The detection unit 100 </ b> C detects the indication area of the indicator on the touch panel 216. The detection unit 100C detects an instruction operation for instructing the first or second divided image performed on the touch panel 216 in a state where the display image is displayed in the display area of the display unit 100B.

  When the display image is generated by the generation unit 100A, the control unit 100D causes the display unit 100B to display the display image in the display area. In addition, when an instruction operation is detected by the detection unit 100C in a state where the display image is displayed in the display area, the control unit 100D extracts a partial image from the display image. The partial image refers to an image including at least a part of the instruction divided image and at least a part of the adjacent divided image. The instruction divided image refers to a divided image instructed by an instruction operation from the first and second divided images. The adjacent divided image refers to a divided image that is adjacent to the instruction divided image of the first and second divided images in a predetermined direction (for example, a specific direction out of the directions orthogonal to the parallax generation direction). . The control unit 100D detects the partial image extracted from the display image by the detection unit 100C in the display area of the display unit 100B in a state where the parallax between the instruction divided image and the adjacent divided image included in the partial image is expanded. The indication area is displayed in an area that is different from the indication area detected by the detection unit 100C.

  When detecting the instruction operation, the detection unit 100C detects the first movement operation in a state where the display image is displayed in the display area of the display unit 100B. The first moving operation refers to an operation of moving the pointing area of the pointing device for the touch panel 216 in a predetermined crossing direction (hereinafter referred to as “crossing direction”) that crosses the division direction. Here, the crossing direction refers to, for example, a parallax generation direction. The moving unit 100E moves the focus lens 302 in the optical axis direction. When the first movement operation is detected by the detection unit 100C, the control unit 100D performs control for moving the focus lens 302 in the optical axis direction according to the first movement operation on the movement unit 100E.

  When detecting the instruction operation, the detection unit 100C detects the second movement operation in a state where the display image is displayed in the display area of the display unit 100B. The second movement operation refers to an operation for moving the pointing area of the pointer on the touch panel 216 in a direction other than the crossing direction. The control unit 100D performs control to maintain the extraction position of the partial image when the second movement operation is detected by the detection unit 100C.

  The presentation unit 100F moves the instruction area by the first movement operation based on the parallax between the first image and the second image at a position corresponding to the instruction area detected by the detection unit 100C in the display image. Presents guidance information to guide directions.

  FIG. 4 is an electric system block diagram illustrating an example of a hardware configuration of the imaging apparatus 100 according to the first embodiment.

  The imaging apparatus 100 includes a mount 256 provided in the imaging apparatus main body 200 and a mount 346 on the interchangeable lens 300 side corresponding to the mount 256. The interchangeable lens 300 is attached to the imaging apparatus main body 200 in a replaceable manner by coupling the mount 346 to the mount 256.

  The interchangeable lens 300 includes a slide mechanism 303 and a motor 304 which are an example of the moving unit 100E shown in FIG. The slide mechanism 303 moves the focus lens 302 along the optical axis L1 by operating the focus ring 301. A focus lens 302 is attached to the slide mechanism 303 so as to be slidable along the optical axis L1. A motor 304 is connected to the slide mechanism 303, and the slide mechanism 303 receives the power of the motor 304 and slides the focus lens 302 along the optical axis L1.

  The motor 304 is connected to the imaging apparatus main body 200 via mounts 256 and 346, and driving is controlled in accordance with a command from the imaging apparatus main body 200. In the first embodiment, a stepping motor is applied as an example of the motor 304. Accordingly, the motor 304 operates in synchronization with the pulse power in accordance with a command from the imaging apparatus main body 200.

  The imaging device 100 is a digital camera that records captured still images and moving images, and the operation of the entire camera is controlled by a CPU (central processing unit) 12. The imaging apparatus 100 includes an operation unit 14, an interface unit 24, a primary storage unit 25, a secondary storage unit 26, an image processing unit 28 (an example of the generation unit 100A illustrated in FIG. 3), a speaker 35, a display control unit 36, and eyepiece detection. Part 37 and an external interface (I / F) 39.

  The CPU 12, the operation unit 14, the interface unit 24, the primary storage unit 25, the secondary storage unit 26, the image processing unit 28, the speaker 35, the display control unit 36, the eyepiece detection unit 37, the external I / F 39, and the touch panel 216 are connected to the bus. 40 are connected to each other.

  The primary storage unit 25 means a volatile memory, for example, a RAM (Random Access Memory). The secondary storage unit 26 means a non-volatile memory, for example, a flash memory or an HDD (Hard Disk Drive).

  The secondary storage unit 26 stores a display control program 50 that is an example of an image processing program according to the present invention. The CPU 12 reads the display control program 50 from the secondary storage unit 26 and develops it in the primary storage unit 25, and executes each process included in the display control program 50. The display control program 50 includes a detection process 50A, a display control process 50B, and a presentation process 50C. The CPU 12 operates as the detection unit 100C illustrated in FIG. 3 by executing the detection process 50A. Further, the CPU 12 operates as the control unit 100D illustrated in FIG. 3 by executing the display control process 50B. Furthermore, the CPU 12 operates as the presentation unit 100F illustrated in FIG. 3 by executing the presentation process 50C.

  The secondary storage unit 26 stores a focus control program 52. The CPU 12 operates as the control unit 100D shown in FIG. 3 by reading the focusing control program 52 from the secondary storage unit 26, developing it in the primary storage unit 25, and executing the focusing control program 52.

  Here, the case where the display control program 50 and the focus control program 52 (hereinafter referred to as “program” when there is no need to distinguish between them) is read from the secondary storage unit 26 is illustrated, but it is not necessarily from the beginning. It is not necessary to store in the secondary storage unit 26. For example, the program may first be stored in an arbitrary portable storage medium such as an SSD (Solid State Drive), an IC card, a magneto-optical disk, or a CD-ROM that is connected to the imaging apparatus 100 and used. . Then, the CPU 12 may acquire a program from these portable storage media and execute it. In addition, the program may be stored in a storage unit of an external electronic computer such as a computer or a server device connected to the imaging apparatus 100 via a communication line. In this case, the CPU 12 acquires a program from the external electronic computer and executes it.

  In the imaging apparatus 100 according to the first embodiment, in the autofocus mode, the CPU 12 performs focusing control by driving and controlling the motor 304 so that the contrast value of the image obtained by imaging is maximized. . In the autofocus mode, the CPU 12 calculates AE information that is a physical quantity indicating the brightness of an image obtained by imaging. When the release button 211 is half-pressed, the CPU 12 derives the shutter speed and F value corresponding to the brightness of the image indicated by the AE information. Then, the exposure state is set by controlling each related part so that the derived shutter speed and F value are obtained.

  The operation unit 14 is a user interface operated by the user when giving various instructions to the imaging apparatus 100. Various instructions received by the operation unit 14 are output as operation signals to the CPU 12, and the CPU 12 executes processing according to the operation signals input from the operation unit 14.

  The operation unit 14 includes a release button 211, a dial 212 for selecting a shooting mode, a finder switching lever 214, a cross key 222, a MENU / OK key 224, and a BACK / DISP button 225.

  The imaging apparatus main body 200 includes a position detection unit 23. The position detection unit 23 is connected to the CPU 12. The position detection unit 23 is connected to the focus ring 301 via mounts 256 and 346, detects the rotation angle of the focus ring 301, and outputs rotation angle information indicating the rotation angle as a detection result to the CPU 12. The CPU 12 executes processing according to the rotation angle information input from the position detection unit 23.

  When the photographing mode is set, the image light indicating the subject is incident on the light receiving surface of the color image sensor (for example, a CMOS sensor) 20 via the photographing lens 16 including the focus lens 302 that can be moved manually and the shutter 18. Imaged. The signal charge accumulated in the image sensor 20 is sequentially read out as a digital signal corresponding to the signal charge (voltage) by a read signal applied from the device control unit 22. The imaging element 20 has a so-called electronic shutter function, and controls the charge accumulation time (shutter speed) of each photosensor according to the timing of the readout signal by using the electronic shutter function. The image sensor 20 according to the first embodiment is a CMOS image sensor, but is not limited thereto, and may be a CCD image sensor.

  The image sensor 20 includes a color filter 21 shown in FIG. 5 as an example. The color filter 21 includes a G filter G corresponding to G (green) that contributes most to obtain a luminance signal, an R filter R corresponding to R (red), and a B filter corresponding to B (blue). In the example shown in FIG. 5, “4896 × 3265” pixels are employed as an example of the number of pixels of the image sensor 20, and the G filter, R filter, and B filter are arranged in the row direction (horizontal direction) and these pixels. They are arranged with a predetermined periodicity in each of the column directions (vertical directions). Therefore, the imaging apparatus 100 can perform processing according to a repetitive pattern when performing synchronization (interpolation) processing of R, G, and B signals. The synchronization process is a process for calculating all color information for each pixel from a mosaic image corresponding to a color filter array of a single-plate color image sensor. For example, in the case of an image sensor made up of RGB color filters, the synchronization process means a process for calculating color information of all RGB for each pixel from a mosaic image made of RGB.

  The image sensor 20 includes a first pixel L, a second pixel R, and a normal pixel N (see FIG. 6). The imaging apparatus 100 operates the phase difference AF function to reduce the phase shift amount based on the signal output value (hereinafter referred to as pixel value) from the pixel of the first pixel L and the pixel value of the second pixel R. To detect. Then, the focal position of the photographing lens is adjusted based on the detected phase shift amount.

  As an example, as illustrated in FIG. 6, the imaging element 20 includes a first pixel row 150, a second pixel row 152, and a third pixel row 154. The first pixel row 150 includes a first pixel set in the same row, and the first pixel set includes a plurality of normal pixels N (four normal pixels N in the example shown in FIG. 6) in the row direction. Are arranged periodically. The first pixel group is adjacent in the row direction in a first arrangement (an arrangement in which the first pixels L are located on the left side in the front view and the second pixels R are located on the right side in the figure). A pair of first pixel L and second pixel R is indicated.

  The second pixel row 152 includes a second pixel set in the same row, and the second pixel set includes a plurality of normal pixels N (four normal pixels N in the example shown in FIG. 6) in the row direction. Are arranged periodically. The second pixel group refers to a pair of first pixels L and second pixels R that are adjacent in a second direction in the row direction (an arrangement opposite to the first arrangement). In the third pixel row 154, a plurality of normal pixels N are arranged adjacent to each other in the same row.

  The first pixel row 150 and the second pixel row 152 include a plurality of third pixel rows 154 in the column direction (in the first embodiment, third pixel rows having different numbers of rows in a predetermined cycle in the column direction). 154) are arranged alternately.

  As an example, as shown in FIG. 7, the first pixel L has a left half of the light receiving surface in the row direction by the light blocking member 20A (the left side when the subject faces the light receiving surface (in other words, when the light receiving surface faces the subject). The right side)) is a light-shielded pixel. As shown in FIG. 7 as an example, the second pixel R has a right half in the row direction on the light receiving surface (right side when facing the subject from the light receiving surface (in other words, when facing the light receiving surface from the subject). The left side)) is a light-shielded pixel. Hereinafter, when it is not necessary to distinguish between the first pixel L and the second pixel R, they are referred to as “phase difference pixels”.

  The light flux that passes through the exit pupil of the photographic lens 16 is roughly classified into left region passing light and right region passing light. The left region passing light refers to the left half of the light beam passing through the exit pupil of the photographing lens 16, and the right region passing light is the right half of the light beam passing through the exit pupil of the photographing lens 16. Refers to luminous flux. The light beam passing through the exit pupil of the photographing lens 16 is divided into left and right by the microlens 19 and the light shielding members 20A and 20B functioning as a pupil dividing unit, the first pixel L receives the left region passing light, and the second The pixel R receives right region passing light. As a result, the subject image corresponding to the left region passing light and the subject image corresponding to the right region passing light are acquired as parallax images (left eye image and right eye image described later) having different parallaxes. Hereinafter, when it is not necessary to distinguish between the light shielding members 20A and 20B, the light shielding members 20A and 20B are referred to as “light shielding members” without reference numerals.

  The image sensor 20 is classified into a first pixel group, a second pixel group, and a third pixel group. As shown in FIG. 6 as an example, the first pixel group refers to a plurality of first pixels L arranged in a matrix. As shown in FIG. 6 as an example, the second pixel group indicates a plurality of second pixels R arranged in a matrix. The third pixel group refers to a plurality of normal pixels N shown in FIG. 6 as an example. Here, the normal pixel N indicates a pixel other than the phase difference pixel (for example, a pixel in which the light shielding members 20A and 20B are not provided). In the following, the RAW image output from the first pixel group is referred to as a “first image”, the RAW image output from the second pixel group is referred to as a “second image”, and the third image The RAW image output from the pixel group is referred to as a “third image”. Hereinafter, the first to third images are referred to as “RAW images” when it is not necessary to distinguish between them.

  Returning to FIG. 4, the image sensor 20 outputs the first image (digital signal indicating the pixel value of each first pixel L) from the first pixel group, and the second image from the second pixel group. (Digital signal indicating the pixel value of each second pixel R) is output. Further, the image sensor 20 outputs a third image (a digital signal indicating the pixel value of each normal pixel) from the third pixel group. Note that the third image output from the third pixel group is a chromatic image, for example, a color image having the same color arrangement as the arrangement of the normal pixels N. The first image, the second image, and the third image output from the image sensor 20 are temporarily stored (overwritten) in a RAW image storage area (not shown) of the primary storage unit 25 via the interface unit 24. Is done.

  The image processing unit 28 performs various types of image processing on the first to third images stored in the primary storage unit 25. The image processing unit 28 is realized by an ASIC (Application Specific Integrated Circuit) that is an integrated circuit in which a plurality of functions related to image processing are integrated into one. However, the hardware configuration is not limited to this, and may be, for example, a programmable logic device or another hardware configuration such as a computer including a CPU, a ROM, and a RAM.

  The hybrid finder 220 includes a liquid crystal display (hereinafter referred to as “second display”) 247 that displays an electronic image.

  The display control unit 36 is connected to the first display 215 and the second display 247, and selectively controls the first display 215 and the second display 247 to control the first display 215 and the second display 247. Display images selectively. Hereinafter, the first display 215 and the second display 247 are referred to as “display devices” when it is not necessary to distinguish between them.

  Note that the imaging apparatus 100 according to the first embodiment is configured to be able to switch between a manual focus mode and an autofocus mode with a dial 212 (focus mode switching unit). When any one of the focus modes is selected, the display control unit 36 causes the display device to display a live view image obtained by combining the split images. When the autofocus mode is selected by the dial 212, the CPU 12 operates as a phase difference detection unit and an automatic focus adjustment unit. The phase difference detection unit detects a phase difference between the first image output from the first pixel group and the second image output from the second pixel group. The automatic focus adjustment unit controls the motor 304 from the device control unit 22 via the mounts 256 and 346 so that the defocus amount of the focus lens 302 is zero based on the detected phase difference, and controls the focus lens 302. Move to the in-focus position. Note that the above “defocus amount” refers to, for example, the amount of phase shift between the first image and the second image.

  The eyepiece detection unit 37 detects that a user (for example, a photographer) has looked into the viewfinder eyepiece unit 242, and outputs the detection result to the CPU 12. Therefore, the CPU 12 can grasp whether or not the finder eyepiece unit 242 is used based on the detection result of the eyepiece detection unit 37.

  The external I / F 39 is connected to a communication network such as a LAN (Local Area Network) or the Internet, and controls transmission / reception of various types of information between the external device (for example, a printer) and the CPU 12 via the communication network. Therefore, when a printer is connected as an external device, the imaging apparatus 100 can output a captured still image to the printer for printing. Further, when a display is connected as an external device, the imaging apparatus 100 can output and display a captured still image or live view image on the display.

  FIG. 8 is a functional block diagram illustrating an example of main functions of the imaging apparatus 100 according to the first embodiment. In addition, the same code | symbol is attached | subjected to the part which is common in the block diagram shown in FIG.

  The normal processing unit 30 and the split image processing unit 32 each have a WB gain unit, a gamma correction unit, and a synchronization processing unit (not shown), and the original digital signal (RAW image) temporarily stored in the primary storage unit 25. Each processing unit sequentially performs signal processing. That is, the WB gain unit executes white balance (WB) by adjusting the gains of the R, G, and B signals. The gamma correction unit performs gamma correction on each of the R, G, and B signals that have been subjected to WB by the WB gain unit. The synchronization processing unit performs color interpolation processing corresponding to the color filter array of the image sensor 20, and generates synchronized R, G, B signals. The normal processing unit 30 and the split image processing unit 32 perform image processing on the RAW image in parallel every time a RAW image for one screen is acquired by the image sensor 20.

  The normal processing unit 30 receives R, G, and B raw images from the interface unit 24, and converts the R, G, and B pixels of the third pixel group into the first pixel group and the second pixel group. A chromatic color normal image is generated by interpolating with peripheral pixels of the same color (for example, adjacent G pixels).

  On the other hand, the split image processing unit 32 extracts G signals (first and second images) of the first pixel group and the second pixel group from the RAW image once stored in the primary storage unit 25, and the first image An achromatic split image is generated based on the G signals of the pixel group and the second pixel group. The pixel group corresponding to each of the first pixel group and the second pixel group extracted from the RAW image is a pixel group including G filter pixels as described above. Therefore, the split image processing unit 32 converts the achromatic left parallax image and the achromatic right parallax image based on the G signal of the pixel group corresponding to each of the first pixel group and the second pixel group. Can be generated. Hereinafter, for convenience of explanation, the above “achromatic left parallax image” is referred to as a “left eye image”, and the above “achromatic right parallax image” is referred to as a “right eye image”.

  As an example, the split image is an image in which the display left-eye image and the display right-eye image are arranged adjacent to each other in a predetermined direction (here, a direction orthogonal to the parallax generation direction). . The left eye image for display is a partial image of four divided images obtained by dividing the left eye image into four in a predetermined direction (in the example shown in FIG. 9, the first and third images from the front view). Divided image). The right-eye image for display is a divided image extracted from four divided images obtained by dividing the right-eye image into four in a predetermined direction with respect to a divided region adjacent to the divided region corresponding to the left-eye image for display (see FIG. 9). In the example shown, it indicates the second and fourth divided images from the front view).

  As an example, as shown in FIG. 10, the split image is displayed in a rectangular frame at the center of the screen of the display device, and a normal image is displayed in the outer peripheral area of the split image. In the example illustrated in FIG. 10, a split image in which two display right-eye images and two display left-eye images are alternately arranged in a predetermined direction is illustrated. The left eye image for display and the right eye image for display included in the split image are shifted in the parallax generation direction according to the in-focus state. Further, the example shown in FIG. 10 shows a state in which a person's peripheral area (for example, a tree) is in focus and the person is not in focus. In the following, for convenience of explanation, when it is not necessary to distinguish between the display left-eye image and the display right-eye image, they are referred to as “display parallax images”.

  In the first embodiment, the split image is combined with the normal image by fitting the split image in place of a part of the normal image. However, the present invention is not limited to this. It is also possible to use a synthesis method in which a split image is superimposed on the image. In addition, when the split image is superimposed, a combining method may be used in which the transmittance of a part of the normal image on which the split image is superimposed and the split image are appropriately adjusted and superimposed. As a result, a live view image showing a subject image continuously shot is displayed on the screen of the display device, but the displayed live view image has a split image displayed in the normal image display area. It becomes an image.

  As an example, as shown in FIG. 8, the hybrid finder 220 includes an OVF 240 and an EVF 248. The OVF 240 is an inverse Galileo finder having an objective lens 244 and an eyepiece 246, and the EVF 248 has a second display 247, a prism 245, and an eyepiece 246.

  Further, a liquid crystal shutter 243 is disposed in front of the objective lens 244, and the liquid crystal shutter 243 shields the optical image from being incident on the objective lens 244 when the EVF 248 is used.

  The prism 245 reflects the electronic image or various information displayed on the second display 247 and guides it to the eyepiece 246, and also displays the optical image and information (electronic image and various information) displayed on the second display 247. And synthesize.

  Here, when the viewfinder switching lever 214 is rotated in the direction of the arrow SW shown in FIG. 1, an OVF mode in which an optical image can be visually recognized by the OVF 240 and an electronic image can be visually recognized by the EVF 248 each time it is rotated. The EVF mode is switched alternately.

  In the OVF mode, the display control unit 36 controls the liquid crystal shutter 243 to be in a non-light-shielding state so that an optical image can be visually recognized from the eyepiece unit. Further, only the split image is displayed on the second display 247. Thereby, a finder image in which a split image is superimposed on a part of the optical image can be displayed.

  In the EVF mode, the display control unit 36 controls the liquid crystal shutter 243 to be in a light shielding state so that only the electronic image displayed on the second display 247 can be visually recognized from the eyepiece unit. The second display 247 receives image data equivalent to the image data obtained by combining the split images output to the first display 215. Accordingly, the second display 247 can display an electronic image in which the split image is combined with a part of the normal image, like the first display 215.

  Next, the operation of the imaging device 100 according to the first embodiment will be described. First, image generation processing performed by the image processing unit 28 in the manual focus mode will be described with reference to FIG. In the following, the case where the image processing unit 28 performs image generation processing will be exemplified, but the present invention is not limited to this. For example, the CPU 12 executes the image generation processing program and the image capturing apparatus 100 performs image processing. Generation processing may be performed.

  In the image generation process illustrated in FIG. 11, first, in step 400, the image processing unit 28 acquires the first to third images from the RAW image storage area of the primary storage unit 25, and then proceeds to step 402.

  In step 402, the image processing unit 28 generates a left eye image and a right eye image based on the first and second images acquired in step 400, and the generated left eye image and right eye image are stored in the primary storage unit 25. Stored (overwritten) in a parallax image storage area (not shown). In the first embodiment, the left-eye image refers to an image that has been subjected to predetermined processing (for example, gain is adjusted) for each pixel in the first image, and the right-eye image is the second eye image. The second image in which a predetermined process is performed on each pixel in the image. Accordingly, the positions (coordinates) of all the pixels included in the left eye image are the same as the positions of all the pixels included in the first image, and the positions of all the pixels included in the right eye image are included in the second image. It is the same as the position of all pixels.

  In the next step 404, the image processing unit 28 generates a normal image (an example of a captured image of the present invention) based on the third image acquired in step 400, and the left eye image and right eye generated in step 402. A split image is generated based on the image. Then, the generated normal image and split image are output to the display control unit 36, and then the process proceeds to step 406. When the normal image and the split image are input, the display control unit 36 causes the display device to continuously display the normal image as a moving image, and continuously displays the split image as a moving image within the display area of the normal image. Control to display. In response to this, the display device displays a live view image as shown in FIG. 10 as an example.

  In step 406, the image processing unit 28 stores (overwrites) the split image generated in step 404 in the split image storage area (not shown) of the primary storage unit 25, and then ends the image generation processing.

  Next, display control processing performed by the imaging apparatus 100 when the CPU 12 executes the display control program 50 when detection result information is input from the touch panel 216 will be described with reference to FIG. In the examples illustrated in FIGS. 13 to 18 for explaining the display control process, a state in which the column image 80 and the background image 82 are displayed on the first display 215 as live view images will be described for convenience of explanation. The column image 80 refers to an image obtained by photographing a vertical column, which is a main subject having a color different from the background color (for example, green), from the front by the imaging apparatus 100, and the background image 82 is a background of the vertical column. An image showing (for example, a white background).

  First, in step 410, the detection unit 100C determines whether or not an indicator is in contact with the touch panel 216 based on the detection result information input from the touch panel 216. If the indicator is in contact with the touch panel 216 at step 410, the determination is affirmed and the routine proceeds to step 412. If the indicator is not in contact with the touch panel 216 at step 410, the determination is negative and the routine proceeds to step 436.

  In step 412, detection unit 100C determines whether or not an instruction operation flag indicating that an instruction operation has been performed on touch panel 216 is off. If it is determined in step 412 that the instruction operation flag is off, the determination is affirmed and the process proceeds to step 414. If the instruction operation flag is on in step 412, the determination is negative and the routine proceeds to step 420.

  Here, for example, as shown in FIG. 13, the instruction operation is a right eye image for display or display when the indicator touches the instruction target area 216A in a state where the split image is displayed in the display area 215A. This refers to an operation for instructing the left eye image as an instruction divided image. The display area 215A refers to an area in which a split image is displayed in the display area of the first display 215. The instruction target area 216A refers to an area corresponding to the display area 215A in the contact target area of the touch panel 216 (for example, a surface having sensitivity in the touch panel 216). In the example illustrated in FIG. 13, the lower display among the left-eye images for display in the front view and the front in the figure included in the split image in which four display parallax images are arranged adjacent to each other in a direction orthogonal to the parallax generation direction. A state in which the left eye image is designated as an instruction divided image by an instruction operation is shown. As an example, as shown in FIG. 13, when the indication area 62 of the indicator straddles a plurality of display parallax images (in the example shown in FIG. 13, both the display left-eye image and the display right-eye image) The display parallax image to which the center point 64 belongs is instructed as an instruction divided image. When the center point 64 exists on the boundary line between the display parallax images, the display divided image on the lower side of the front view in FIG. 13 with respect to the boundary line is designated as the instruction divided image. The center point 64 refers to the center point of the rectangular frame 68 that circumscribes the indication area 62.

  The case where the determination is negative in step 412 is, for example, the touch panel 216 following the instruction operation (without passing through a period in which the indicator is not in contact with the touch panel 216 after the instruction operation is performed). Means that some operation has been performed on. As an example of “some operation” mentioned here, the first movement operation or the second movement operation can be exemplified. As shown in FIG. 14 as an example, the first movement operation refers to an operation of moving the instruction area 62 in the parallax generation direction (in the example shown in FIG. 14, the direction of arrow A or the direction of arrow B). The second movement operation refers to an operation of moving the instruction area 62 in a direction other than the parallax generation direction (in the example illustrated in FIG. 14, the arrow C direction or the arrow D direction).

  In step 414, the detection unit 100 </ b> C determines whether an instruction operation has been performed on the touch panel 216 based on the detection result information input from the touch panel 216. In step 414, when an instruction operation is performed on the touch panel 216, the determination is affirmed and the process proceeds to step 416. In step 414, when the instruction operation is not performed on the touch panel 216, the determination is denied and the display control process is terminated.

  The case where the determination is negative in step 414 is, for example, instructing an area other than the instruction target area 216A shown in FIG. 14 in the contact target area of the touch panel 216 (for example, an area corresponding to the display area of the normal image). Refers to the case where the body touches.

  In step 416, the detection unit 100C turns on the instruction operation flag, and then proceeds to step 418.

  In step 418, the control unit 100D stores (overwrites) the designated coordinates in the designated coordinate storage area (not shown) of the primary storage unit 25, and then proceeds to step 420. Here, the designated coordinates indicate the coordinates of the center point 64 (see FIG. 13).

  In step 420, the control unit 100D obtains a pair of parallax calculation images from the left eye image and right eye image stored in the parallax image storage area of the primary storage unit 25, and then proceeds to step 422. Here, the pair of parallax calculation images indicates a rectangular left eye image 66A illustrated in FIG. 15A as an example, and a rectangular right eye image 66B illustrated in FIG. 15B as an example. Each of the rectangular left eye image 66A and the rectangular right eye image 66B is an image cut out by a rectangular frame 68 (a broken line rectangular frame shown in FIGS. 15A and 15B as an example). That is, the rectangular left-eye image 66A indicates an image of the entire region in the rectangular frame 68 in the left-eye image as shown in FIG. 15A as an example, and the rectangular right-eye image 66B is shown as an example in FIG. 15B. The image of the whole area in the rectangular frame 68 in the right eye image is indicated.

  In step 422, the control unit 100D calculates the inter-image parallax Δx, and then proceeds to step 424. Here, the inter-image parallax Δx refers to the parallax of the pair of parallax calculation images acquired in step 420.

  In step 424, the control unit 100D acquires the rectangular image 70 shown in FIG. 13 as an example from the split image stored in the split image storage area of the primary storage unit 25, and then proceeds to step 426. Here, the rectangular image 70 indicates an image cut out from the display parallax image by the rectangular frame 68 as shown in FIG.

  In step 426, the control unit 100 </ b> D calculates the light / dark difference of the rectangular image 70 acquired in step 424 (here, as an example, the standard deviation of pixel values in all pixels included in the rectangular image 70), and then proceeds to step 428. To do.

  In step 428, the control unit 100D determines whether or not the brightness difference calculated in step 426 is greater than or equal to a threshold value. In step 428, if the light / dark difference calculated in step 426 is greater than or equal to the threshold value, the determination is affirmed and the process proceeds to step 430. In step 428, when the contrast calculated in step 426 is less than the threshold value, the determination is negative and the display control process ends. The case where the contrast is less than the threshold is, for example, as shown in FIG. 16, in the state where the column image 80 and the background image 82 are displayed, the instruction area 62 is in the area corresponding to the background image 82 in the instruction target area 216A. If present.

  In step 430, the control unit 100D performs, as an example, a split image stored in the split image storage area of the primary storage unit 25 based on the indicated coordinates stored in the indicated coordinate storage area of the primary storage unit 25 as shown in FIG. And the partial image 72 shown in FIG. 18 is specified. Then, the identified partial image is extracted from the split image, and then the process proceeds to step 432.

  As an example, the partial image 72 indicates an image of the entire area in a rectangular frame 74 (broken-line rectangular frame shown in FIGS. 17 and 18) as shown in FIGS. 17 and 18. The rectangular frame 74 is a rectangular frame with the point 76 as the center. The length of one side of the rectangular frame 74 is twice the length of the short side of the display parallax image (the length of one side in the direction orthogonal to the parallax occurrence direction in the display parallax image). Point 76 is a leg of a perpendicular drawn from the center point 64 to a boundary line adjacent in a predetermined direction. Here, the boundary line refers to a boundary line between parallax images for display. As shown in FIG. 17 as an example, the predetermined direction is a display other than the display right-eye image in the uppermost front view in the figure among the four display parallax images arranged in the vertical direction in the front view in the figure. When the parallax image for use is designated as the designated divided image, it indicates the front view upward direction (arrow E direction) in the figure. In addition, the predetermined direction is, as shown in FIG. 18 as an example, in the case where the display parallax image at the top in the front view in the figure is designated as the instruction divided image among the four display parallax images. It refers to the front view downward direction (arrow F direction). Therefore, the partial image 72 refers to a part of the right eye image for display and a part of the left eye image for display included in the entire region within the rectangular frame 74 (in a state of contact via the boundary line). means. In the example shown in FIGS. 17 and 18, the case where the center point 64 is not on the boundary line is illustrated, but when the center point 64 exists on the boundary line, the center point 64 is replaced with the rectangular frame 74. The partial image 72 may be specified by a rectangular frame at the center (a rectangular frame having the same size as the rectangular frame 74).

  In step 432, the control unit 100 </ b> D displays the parallax extended image 75 in which the parallax of the partial image 72 extracted in step 430 is expanded in the specific area 76 avoiding the instruction area 62 (see FIG. 17 as an example) For example, the display is superimposed), and then the process proceeds to step 434.

  As an example, as illustrated in FIG. 17, the parallax extension image 75 is displayed in a rectangular window (a window having the same size as the rectangular frame 74), and includes a parallax extension instruction image 75A and a parallax extension adjacent image 75B. The parallax extension instruction image 75A is an image obtained by moving the instruction divided image included in the partial image 72 in the parallax generation direction (the direction in which the parallax with the adjacent divided image included in the partial image 72 is extended). In other words, the parallax extension instruction image 75A is an image division parallax calculated by kΔx (step 422) in the left direction of the front view of the instruction divided image (in the example shown in FIG. 17, the display left-eye image) included in the partial image 72. This is an image that has been moved by a value obtained by multiplying Δx by k. The parallax extended adjacent image 75B is an image obtained by moving the adjacent divided image included in the partial image 72 in the parallax generation direction (direction in which the parallax with the instruction divided image included in the partial image 72 is extended). That is, the parallax extended adjacent image 75B moves the adjacent divided image (the display right eye image adjacent to the display left eye image that is the instruction divided image) included in the partial image 72 by kΔx in the right direction of the front view in the figure. It is an image.

  The specific region 76 refers to one end portion of the display end portions 79 when the center point 64 is included in the central portion 78 (see FIG. 17) in the parallax generation direction in the display region 215A. The display end portions 79 refer to both end portions in the parallax generation direction in the display area of the first display 215 (areas on both sides of the display area 215A in the parallax generation direction). Further, the specific region 76 refers to an end portion far from the center point 64 of the display end portions 79 when the center point 64 is included in one end portion of the split image both ends 81. The split image both end portions 81 indicate both end portions in the parallax generation direction (regions on both sides of the central portion 78 in the parallax generation direction) in the display region 215A. In the example shown in FIG. 17, since the center point 64 belongs to the area on the left side of the display area 215 </ b> A in the front view of the display area 215 </ b> A in the parallax generation direction, the center point 64 belongs to the right side of the display area 215 </ b> A. A parallax extended image 75 is displayed in the normal image.

  In the example illustrated in FIG. 17, the contour of the image included in the parallax extension image 75 (for example, a region of a high frequency component equal to or greater than a predetermined value in the parallax extension instruction image 75A and the parallax extension adjacent image 75B) corresponds to the inter-image parallax Δx. It is highlighted with a degree of emphasis. Here, the highlighting refers to, for example, displaying an outline with a density equal to or higher than a predetermined value. The outline of the image included in the parallax extension image 75 refers to, for example, the outline of the column image 80 included in the parallax extension instruction image 75A and the outline of the column image 80 included in the parallax extension adjacent image 75B. The degree of enhancement according to the inter-image parallax Δx refers to, for example, the degree of enhancement (for example, the contour line density) that increases as the inter-image parallax Δx decreases. Note that the adjustment of the image included in the parallax extended image 75 is not limited to the contour emphasis display, and the enlargement ratio of the parallax extended image 75 according to the inter-image parallax Δx (for example, the smaller the inter-image parallax Δx is, It may be enlarged at a larger enlargement ratio). Further, the saturation of the parallax extended image 75 may be adjusted to the saturation corresponding to the inter-image parallax Δx (for example, the saturation that increases as the inter-image parallax Δx decreases). Further, at least two image adjustments may be combined: emphasis display of the outline of the image included in the parallax extension image 75, enlargement display of the parallax extension image 75, and adjustment of the saturation of the parallax extension image 75.

  In step 434, the presentation unit 100F displays the guidance information corresponding to the inter-image parallax calculated in step 422 on the display area 215A adjacent to the parallax extended image 75, and then ends the display control process.

  Here, the guidance information means information for guiding the direction in which the instruction area 62 is moved by the first movement operation in order to get out of the out-of-focus state (in order to make the in-focus state). The arrow 83 shown in FIG. An arrow 83 indicates a recommended direction as a direction in which the instruction area 62 is moved by the first movement operation. When the first movement operation is performed in the direction indicated by the arrow 83, the focus control process described later makes a transition from the out-of-focus state to the in-focus state, and as a result, the split image is displayed as an example shown in FIG. 19A. As an example, the display mode changes to that shown in FIG. 19B. In the example shown in FIG. 19A, the indicator 84 is in contact with an area corresponding to the display left-eye image in the instruction target area 216A (because the display left-eye image is instructed as an instruction divided image). The arrow 83 indicates the right direction in the front view in the figure, which is the direction opposite to the parallax generation direction. On the contrary, when the indicator 84 is in contact with the area corresponding to the display right eye image in the instruction target area 216A, an arrow (not shown) pointing in the opposite direction to the direction indicated by the arrow 83 is used as the guidance information. Is displayed.

  In step 436, the detection unit 100C determines whether or not the instruction operation flag is on. If the instruction operation flag is on in step 436, the determination is affirmed and the routine proceeds to step 438. If the instruction operation flag is off in step 436, the determination is negative and the display control process is terminated.

  In step 438, the detection unit 100C turns off the instruction operation flag, and then ends the display control process.

  Next, focusing control processing performed by the imaging apparatus 100 when the CPU 12 executes the focusing control program 52 when detection result information is input from the touch panel 216 will be described with reference to FIG.

  First, in step 450, the control unit 100D determines whether or not the instruction operation flag is on. If the instruction operation flag is on in step 450, the determination is affirmed and the routine proceeds to step 452. If the instruction operation flag is off in step 450, the determination is negative and the focus control process is terminated.

  In step 452, the control unit 100 </ b> D determines whether or not a first movement operation has been performed on the touch panel 216 based on the detection result information input from the touch panel 216. If the first movement operation is performed on the touch panel 216 in step 452, the determination is affirmed and the process proceeds to step 454. If it is determined in step 452 that the first movement operation is not performed on the touch panel 216, the determination is negative and the process proceeds to step 456.

  In step 454, the control unit 100D performs focusing control according to the first movement operation, and then proceeds to step 456. Here, the focus control is an example in which the center point 64 moves in the direction of arrow A or B shown in FIG. 14 to move the focus lens 302 in a direction corresponding to the moving direction of the center point 64 in the optical axis direction. In this case, the movement is controlled by a lens movement amount corresponding to the movement amount of the center point 64.

  In step 456, control unit 100D determines whether or not the instruction operation flag is off. If it is determined in step 456 that the instruction operation flag is on, the determination is negative, and the process proceeds to step 452. If the instruction operation flag is off in step 456, the determination is affirmed and the focusing control process is terminated.

  As described above, in the imaging device 100, when an instruction operation is detected in a state where the split image is displayed in the display area 215A, the partial image 72 (see FIG. 17) including the instruction divided image and the adjacent divided image is split. Extracted from the image. And the parallax expansion image 75 (refer FIG. 17) which is an image by which the parallax of the extracted partial image 72 was expanded is displayed on the specific area | region 76. FIG. Therefore, the imaging apparatus 100 can make it easier to visually recognize whether or not the parallax extended image 75 is in a focused state as compared to the case where the parallax extended image 75 is not displayed in the specific region 76.

  Further, in the imaging apparatus 100, when the instruction operation is detected in a state where the split image is displayed, the control for displaying the parallax extended image 75 is a rectangle at the position corresponding to the instruction region 62 detected in accordance with the instruction operation. It is not performed when the contrast in the image 70 is less than the threshold value. Therefore, the imaging apparatus 100 displays the parallax expansion image 75 in the specific area 76 when it is difficult to visually recognize whether or not it is in focus even if the parallax expansion image 75 is displayed in the specific area 76. This can be suppressed.

  In the imaging apparatus 100, when the center point 64 (see FIG. 17) is included in the central portion 78 of the display area 215A, the parallax extended image 75 is displayed at one end of the display both end portions 79 (see FIG. 17). Is displayed. Therefore, the imaging apparatus 100 can suppress the parallax expansion image 75 from becoming difficult to visually recognize due to the blind spot formed by the indicator that is in contact with the area corresponding to the central portion 78 of the display area 215A on the touch panel 216.

  Further, in the imaging device 100, when the center point 64 is included in one end of both ends in the parallax generation direction in the display region 215A, the identification is the end far from the center point 64 in the display both ends 79. A parallax extended image 75 is displayed in the area 76. Therefore, the imaging apparatus 100 is difficult to view the parallax expansion image 75 due to the blind spot formed by the indicator that is in contact with the area corresponding to one of the both ends of the display area 215A on the touch panel 216. Can be suppressed. In addition, since the display end portions 79 are regions other than the display region 215A, it is possible to prevent the split image from becoming difficult to visually recognize due to the display of the parallax extended image 75.

  Further, in the imaging apparatus 100, the parallax extended image 75 is adjusted according to at least one of the enlargement ratio, the saturation, and the edge enhancement level determined according to the inter-image parallax Δx. Therefore, the imaging apparatus 100 can improve the visibility of the parallax extension image 75.

  Further, in the imaging apparatus 100, when the first movement operation is detected, control is performed to move the focus lens 302 in the optical axis direction according to the first movement operation. Therefore, the imaging apparatus 100 can easily move the focus lens 302 to the in-focus position as compared with the case where this configuration is not provided.

  Further, in the imaging device 100, when the second movement operation is detected (when a negative determination is made in step 412 in FIG. 12), the extraction position of the partial image 72 is maintained. Therefore, the imaging apparatus 100 can suppress the image of the area other than the focus target area from being extracted as the partial image 72 due to the instruction area 62 being shifted in a direction not intended by the user.

  In the imaging apparatus 100, an arrow 83 (see FIG. 19A) is displayed as guidance information for guiding the movement direction of the instruction area 62 by the first movement operation based on the inter-image parallax Δx. Therefore, the imaging apparatus 100 can make the user easily recognize an operation necessary to move the focus lens 302 to the in-focus position, compared to a case where the guide information is not displayed.

  In the first embodiment, a part of the instruction divided image and a part of the adjacent divided image are set as the partial image 72. However, the present invention is not limited thereto, and the entire instruction divided image and the entire adjacent divided image are partially displayed. It may be an image.

  Moreover, although the case where the arrow 83 was displayed on the 1st display 215 was illustrated in the said 1st Embodiment, not only this but the audio | voice which shows the direction which the arrow 83 points out may be output to the speaker 35, for example. Further, the operation direction of the first movement operation may be guided only by the sound output from the speaker 35 without displaying the arrow 83 on the first display 215.

  In the first embodiment, the split image in which the four display parallax images are arranged adjacent to each other is illustrated. However, the present invention is not limited to this, and one display left-eye image and one display right-eye image are included. May be split images arranged adjacent to each other. Further, it may be a split image in which one display right eye image and two display left eye images are alternately arranged adjacent to each other. As described above, the split image may be an image in which the display left-eye image and the display right-eye image are arranged alternately and linearly adjacent to each other.

  In the first embodiment, when the center point 64 is included in the central portion 78 in the parallax generation direction in the display area 215A, the parallax extended image 75 is displayed at one end portion of the display both end portions 79. However, the present invention is not limited to this. For example, the parallax expansion image 75 may be displayed in an area other than the central part 78 in the display area 215A. Also in this case, it is possible to prevent the parallax expansion image 75 from becoming difficult to visually recognize due to the blind spot formed by the indicator that is in contact with the area corresponding to the central portion 78 of the display area 215A on the touch panel 216. The same effect as that of the first embodiment can be expected.

  Further, in the first embodiment, an example in which the parallax extended image 75 is displayed in the specific area 76 when the center point 64 is included in one end of both ends in the parallax generation direction in the display area 215A will be described. However, the present invention is not limited to this. For example, as shown in FIG. 17, when the center point 64 belongs to an area on the left side of the front view in the drawing with respect to the central portion 78, the parallax extended image is located on the right side of the drawing in the front view with respect to the central portion 78 in the display area 215 </ b> A. 75 may be displayed. Also in this case, it is possible to suppress the parallax expansion image 75 from becoming difficult to visually recognize due to the blind spot formed by the indicator that is in contact with the area corresponding to one of the both ends of the display area 215A on the touch panel 216. You can expect the effect that you can.

  In the first embodiment, as shown in FIG. 18 as an example, the display parallax image at the top of the front view included in the split image (the display left-eye image in the example shown in FIG. 18) is the instruction divided image. However, the present invention is not limited to this. For example, the display parallax image is instructed as the instruction divided image only when the indicator touches an area corresponding to the display parallax image other than the display parallax image at the topmost front view included in the split image on the touch panel 216. You may do it. In this way, only a specific display parallax image among a plurality of display parallax images included in the split image may be used as a display parallax image that can be designated as an instruction divided image.

  In the first embodiment, the case where one of the plurality of display parallax images included in the split image is instructed as an instruction divided image to be focused is described. However, the present invention is not limited to this. It is not something. For example, one of a plurality of display parallax images included in the split image may be instructed as an instruction divided image for image quality adjustment (for example, brightness adjustment or sharpness adjustment). Further, one of the plurality of display parallax images included in the split image may be designated as an enlarged divided display target, a reduced display target, a cut target, or a copy target instruction divided image.

  In addition, the flow of image generation processing (see FIG. 11), the flow of display control processing (see FIG. 12), and the focus control processing (see FIG. 20) described in the first embodiment are merely examples. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, and the processing order may be changed within a range not departing from the spirit. In addition, each process included in the image generation process described in the first embodiment may be realized by a software configuration using a computer by executing a program, or a combination of a hardware configuration and a software configuration. It may be realized by. In addition, each process included in the display control process and the focus control process described in the first embodiment may be realized by a hardware configuration such as an ASIC or a programmable logic device, or the hardware configuration and the software configuration. It may be realized by a combination.

When the image generation processing described in the first embodiment is realized by executing a program by a computer, the program may be stored in advance in a predetermined storage area (for example, the secondary storage unit 26). Note that it is not always necessary to store the data in the secondary storage unit 26 from the beginning. For example, SSD (Solid State Drive) connected to a computer and used
First, the program may be stored in an arbitrary portable storage medium such as a CD-ROM, a DVD disk, a magneto-optical disk, or an IC card. Then, the computer may acquire the program from these portable storage media and execute it. Also, each program may be stored in another computer or server device connected to the computer via the Internet, a LAN (Local Area Network), etc., and the computer may acquire and execute the program from these. Good.

  In addition, the imaging apparatus 100 described in the first embodiment may have a function of confirming the depth of field (depth of field confirmation function). In this case, for example, the imaging apparatus 100 has a depth-of-field confirmation key. The depth-of-field confirmation key may be a hard key or a soft key. In the case of an instruction by a hard key, for example, a momentary operation type switch (non-holding type switch) is preferably applied. The momentary operation type switch mentioned here refers to a switch that maintains a specific operation state in the imaging apparatus 100 only while being pushed into a predetermined position, for example. Here, when the depth-of-field confirmation key is pressed, the aperture value is changed. Further, while the depth-of-field confirmation key is continuously pressed (while being pushed into a predetermined position), the aperture value continues to change until reaching the limit value. As described above, while the depth-of-field confirmation key is pressed, the aperture value changes, and thus there may be a case where the phase difference necessary for obtaining the split image cannot be obtained. Thus, when the depth-of-field confirmation key is pressed while the split image is displayed, the split image may be changed to the normal live view display while the split image is being pressed. Further, the CPU 12 may switch the screen so that the split image is displayed again when the pressed state is released. In this example, a momentary operation type switch is applied as an example of the depth of field confirmation key. However, the present invention is not limited to this, and an alternative operation type switch (holding type switch) may be applied.

[Second Embodiment]
In the first embodiment, the imaging device 100 is illustrated, but examples of the mobile terminal device that is a modification of the imaging device 100 include a mobile phone and a smartphone having a camera function. In addition, PDA (Personal Digital Assistants), a portable game machine, etc. are mentioned. In the second embodiment, a smartphone will be described as an example, and will be described in detail with reference to the drawings.

  FIG. 21 is a perspective view illustrating an example of the appearance of the smartphone 500. A smartphone 500 illustrated in FIG. 21 includes a flat housing 502, and a display input in which a display panel 521 as a display unit and an operation panel 522 as an input unit are integrated on one surface of the housing 502. Part 520. The housing 502 includes a speaker 531, a microphone 532, an operation unit 540, and a camera unit 541. Note that the configuration of the housing 502 is not limited thereto, and for example, a configuration in which the display unit and the input unit are independent may be employed, or a configuration having a folding structure or a slide structure may be employed.

  FIG. 22 is a block diagram showing an example of the configuration of the smartphone 500 shown in FIG. As shown in FIG. 21, the main components of the smartphone 500 are a wireless communication unit 510, a display input unit 520, a communication unit 530, an operation unit 540, a camera unit 541, a storage unit 550, and an external input / output. Part 560. In addition, as a main component of the smartphone 500, a GPS (Global Positioning System) receiving unit 570, a motion sensor unit 580, a power supply unit 590, and a main control unit 501 are provided. In addition, as a main function of the smartphone 500, a wireless communication function for performing mobile wireless communication via the base station device BS and the mobile communication network NW is provided.

  The radio communication unit 510 performs radio communication with the base station apparatus BS accommodated in the mobile communication network NW according to an instruction from the main control unit 501. Using this wireless communication, transmission and reception of various file data such as audio data and image data, e-mail data, and reception of Web data and streaming data are performed.

  The display input unit 520 is a so-called touch panel, and includes a display panel 521 and an operation panel 522. For this reason, the display input unit 520 displays images (still images and moving images), character information, and the like visually by controlling the main control unit 501, and visually transmits information to the user. Is detected. Note that when viewing the generated 3D, the display panel 521 is preferably a 3D display panel.

  The display panel 521 uses an LCD, an OELD (Organic Electro-Luminescence Display), or the like as a display device. The operation panel 522 is a device that is placed so that an image displayed on the display surface of the display panel 521 is visible and detects one or a plurality of coordinates operated by a user's finger or stylus. When such a device is operated by a user's finger or stylus, a detection signal generated due to the operation is output to the main control unit 501. Next, the main control unit 501 detects an operation position (coordinates) on the display panel 521 based on the received detection signal.

  As shown in FIG. 21, the display panel 521 and the operation panel 522 of the smartphone 500 integrally form the display input unit 520, but the operation panel 522 is disposed so as to completely cover the display panel 521. ing. When this arrangement is adopted, the operation panel 522 may have a function of detecting a user operation even in an area outside the display panel 521. In other words, the operation panel 522 includes a detection area (hereinafter referred to as a display area) for an overlapping portion that overlaps the display panel 521 and a detection area (hereinafter, a non-display area) for an outer edge portion that does not overlap the other display panel 521. May be included).

  Note that the size of the display area and the size of the display panel 521 may be completely matched, but it is not always necessary to match the two. In addition, the operation panel 522 may include two sensitive regions of the outer edge portion and the other inner portion. Further, the width of the outer edge portion is appropriately designed according to the size of the housing 502 and the like. Furthermore, examples of the position detection method employed in the operation panel 522 include a matrix switch method, a resistance film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, and a capacitance method. You can also

  The communication unit 530 includes a speaker 531 and a microphone 532. The communication unit 530 converts the user's voice input through the microphone 532 into voice data that can be processed by the main control unit 501, and outputs the voice data to the main control unit 501. Further, the communication unit 530 decodes the audio data received by the wireless communication unit 510 or the external input / output unit 560 and outputs it from the speaker 531. In addition, as shown in FIG. 21, for example, the speaker 531 can be mounted on the same surface as the display input unit 520 and the microphone 532 can be mounted on the lower front portion of the housing 502.

  The operation unit 540 is a hardware key using a key switch or the like, and receives an instruction from the user. For example, as shown in FIG. 21, the operation unit 540 is mounted on the lower front portion of the housing 502 of the smartphone 500 and is turned on when pressed with a finger or the like, and is turned off by a restoring force such as a spring when the finger is released. This is a push button type switch.

  The storage unit 550 stores the control program and control data of the main control unit 501, application software, address data that associates the name and telephone number of the communication partner, and transmitted / received e-mail data. In addition, the storage unit 550 stores Web data downloaded by Web browsing and downloaded content data. The storage unit 550 temporarily stores streaming data and the like. In addition, the storage unit 550 includes an external storage unit 552 having an internal storage unit 551 built in the smartphone and a removable external memory slot. Each of the internal storage unit 551 and the external storage unit 552 constituting the storage unit 550 is realized using a storage medium such as a flash memory type or a hard disk type. Other storage media include multimedia card micro type, card type memory (for example, MicroSD (registered trademark) memory, etc.), RAM (Random Access Memory), ROM (Read Only Memory). ) Can be exemplified.

  The external input / output unit 560 serves as an interface with all external devices connected to the smartphone 500, and is used to connect directly or indirectly to other external devices through communication or the like or a network. is there. Examples of communication with other external devices include universal serial bus (USB), IEEE 1394, and the like. Examples of the network include the Internet, wireless LAN, Bluetooth (Bluetooth (registered trademark)), RFID (Radio Frequency Identification), and infrared communication (Infrared Data Association: IrDA (registered trademark)). Other examples of the network include UWB (Ultra Wideband (registered trademark)) and ZigBee (registered trademark).

Examples of the external device connected to the smartphone 500 include a memory card connected via a wired / wireless headset, a wired / wireless external charger, a wired / wireless data port, and a card socket. Other examples of external devices include SIM (Subscriber
Examples include an Identity Module Card (UIM) / UIM (User Identity Module Card) card and an external audio / video device connected via an audio / video I / O (Input / Output) terminal. In addition to the external audio / video device, an external audio / video device that is wirelessly connected can be used. Also, instead of external audio / video devices, for example, smartphones with wired / wireless connection, personal computers with wired / wireless connection, PDAs with wired / wireless connection, personal computers with wired / wireless connection, earphones, etc. also apply Is possible.

  The external input / output unit may transmit data received from such an external device to each component inside the smartphone 500, or may allow data inside the smartphone 500 to be transmitted to the external device. it can.

  The GPS receiving unit 570 receives GPS signals transmitted from the GPS satellites ST1 to STn in accordance with instructions from the main control unit 501, performs positioning calculation processing based on the received GPS signals, and calculates the latitude of the smartphone 500 Detect the position consisting of longitude and altitude. When the GPS reception unit 570 can acquire position information from the wireless communication unit 510 or the external input / output unit 560 (for example, a wireless LAN), the GPS reception unit 570 can also detect the position using the position information.

  The motion sensor unit 580 includes, for example, a three-axis acceleration sensor and detects the physical movement of the smartphone 500 in accordance with an instruction from the main control unit 501. By detecting the physical movement of the smartphone 500, the moving direction and acceleration of the smartphone 500 are detected. This detection result is output to the main control unit 501.

  The power supply unit 590 supplies power stored in a battery (not shown) to each unit of the smartphone 500 in accordance with an instruction from the main control unit 501.

  The main control unit 501 includes a microprocessor, operates according to a control program and control data stored in the storage unit 550, and controls each unit of the smartphone 500 in an integrated manner. Further, the main control unit 501 includes a mobile communication control function for controlling each unit of the communication system and an application processing function in order to perform voice communication and data communication through the wireless communication unit 510.

  The application processing function is realized by the main control unit 501 operating according to application software stored in the storage unit 550. Application processing functions include, for example, an infrared communication function that controls the external input / output unit 560 to perform data communication with the opposite device, an e-mail function that transmits and receives e-mails, and a web browsing function that browses web pages. .

  Further, the main control unit 501 has an image processing function such as displaying video on the display input unit 520 based on image data (still image or moving image data) such as received data or downloaded streaming data. The image processing function is a function in which the main control unit 501 decodes the image data, performs image processing on the decoding result, and displays an image on the display input unit 520.

  Further, the main control unit 501 executes display control for the display panel 521 and operation detection control for detecting a user operation through the operation unit 540 and the operation panel 522.

  By executing the display control, the main control unit 501 displays an icon for starting application software, a soft key such as a scroll bar, or a window for creating an e-mail. Note that the scroll bar refers to a soft key for accepting an instruction to move a display portion of an image such as a large image that cannot be accommodated in the display area of the display panel 521.

  Further, by executing the operation detection control, the main control unit 501 detects a user operation through the operation unit 540, or accepts an operation on the icon or an input of a character string in the input field of the window through the operation panel 522. Or In addition, by executing the operation detection control, the main control unit 501 accepts a display image scroll request through a scroll bar.

  Furthermore, by executing the operation detection control, the main control unit 501 causes the operation position with respect to the operation panel 522 to overlap with the display panel 521 (display area) or other outer edge part (non-display area) that does not overlap with the display panel 21. ) In response to the determination result, a touch panel control function for controlling the sensitive area of the operation panel 522 and the display position of the soft key is provided.

  The main control unit 501 can also detect a gesture operation on the operation panel 522 and execute a preset function in accordance with the detected gesture operation. Gesture operation is not a conventional simple touch operation, but an operation that draws a trajectory with a finger or the like, designates a plurality of positions at the same time, or combines these to draw a trajectory for at least one of a plurality of positions. means.

  The camera unit 541 is a digital camera that captures an image using an image sensor such as a CMOS or a CCD, and has the same function as the image capturing apparatus 100 illustrated in FIG.

  The camera unit 541 can switch between a manual focus mode and an autofocus mode. When the manual focus mode is selected, the photographing lens of the camera unit 541 can be focused by operating a focus icon button or the like displayed on the operation unit 540 or the display input unit 520. In the manual focus mode, a live view image obtained by combining the split images is displayed on the display panel 521 so that the in-focus state during manual focus can be confirmed. In addition, you may make it provide the smart phone 500 with the hybrid finder 220 shown in FIG.

  Further, the camera unit 541 converts image data obtained by imaging into compressed image data such as JPEG (Joint Photographic coding Experts Group) under the control of the main control unit 501. The converted image data can be recorded in the storage unit 550 or output through the input / output unit 560 or the wireless communication unit 510. In the smartphone 500 shown in FIG. 21, the camera unit 541 is mounted on the same surface as the display input unit 520, but the mounting position of the camera unit 541 is not limited to this and may be mounted on the back surface of the display input unit 520. Alternatively, a plurality of camera units 541 may be mounted. Note that when a plurality of camera units 541 are mounted, the camera unit 541 used for imaging may be switched and imaged alone, or a plurality of camera units 541 may be used simultaneously for imaging. it can.

  The camera unit 541 can be used for various functions of the smartphone 500. For example, an image acquired by the camera unit 541 can be displayed on the display panel 521, or the image of the camera unit 541 can be used as one of operation inputs of the operation panel 522. Further, when the GPS receiving unit 570 detects the position, the position can also be detected with reference to an image from the camera unit 541. Furthermore, referring to an image from the camera unit 541, the optical axis direction of the camera unit 541 of the smartphone 500 is determined without using the triaxial acceleration sensor or in combination with the triaxial acceleration sensor. It is also possible to determine the current usage environment. Of course, the image from the camera unit 541 can be used in the application software.

  In addition, various information can be added to still image or moving image data and recorded in the storage unit 550, or can be output through the input / output unit 560 or the wireless communication unit 510. Examples of the “various information” herein include, for example, position information acquired by the GPS receiving unit 570 and image information of the still image or moving image, audio information acquired by the microphone 532 (sound text conversion by the main control unit or the like). May be text information). In addition, posture information acquired by the motion sensor unit 580 may be used.

  In each of the above embodiments, the split image divided into two in the vertical direction is illustrated. However, the present invention is not limited to this, and an image divided into a plurality of parts in the horizontal direction or the diagonal direction may be applied as the split image.

  For example, the split image 66a shown in FIG. 23 is divided into odd and even lines by a plurality of dividing lines 63a parallel to the row direction. In the split image 66a, a line-like (eg, strip-like) phase difference image 66La generated based on the output signal outputted from the first pixel group is displayed on an odd line (even an even line is acceptable). In addition, a line-shaped (eg, strip-shaped) phase difference image 66Ra generated based on the output signal output from the second pixel group is displayed on even lines.

  Also, the split image 66b shown in FIG. 24 is divided into two by a dividing line 63b (for example, a diagonal line of the split image 66b) having an inclination angle in the row direction. In the split image 66b, the phase difference image 66Lb generated based on the output signal output from the first pixel group is displayed in one area. Further, the phase difference image 66Rb generated based on the output signal output from the second pixel group is displayed in the other region.

  Further, the split image 66c shown in FIGS. 25A and 25B is divided by grid-like dividing lines 63c parallel to the row direction and the column direction, respectively. In the split image 66c, the phase difference image 66Lc generated based on the output signal output from the first pixel group is displayed in a checker pattern. In addition, the phase difference image 66Rc generated based on the output signal output from the second pixel group is displayed in a checker pattern.

  In addition to the split image, another focus confirmation image may be generated from the two phase difference images, and the focus confirmation image may be displayed. For example, two phase difference images may be superimposed and displayed as a composite image. If the image is out of focus, the image may be displayed as a double image, and the image may be clearly displayed when the image is in focus.

  Moreover, in each said embodiment, although the image pick-up element 20 which has the 1st-3rd pixel group was illustrated, this invention is not limited to this, Only the 1st pixel group and the 2nd pixel group An image sensor made of A digital camera having this type of image sensor generates a three-dimensional image (3D image) based on the first image output from the first pixel group and the second image output from the second pixel group. 2D images (2D images) can also be generated. In this case, the generation of the two-dimensional image is realized, for example, by performing an interpolation process between pixels of the same color in the first image and the second image. Moreover, you may employ | adopt a 1st image or a 2nd image as a two-dimensional image, without performing an interpolation process.

  Further, in each of the above embodiments, when the first to third images are input to the image processing unit 28, an example in which both the normal image and the split image are simultaneously displayed on the same screen of the display device is illustrated. The present invention is not limited to this. For example, the display control unit 36 may be configured to suppress continuous display as a moving image of a normal image on the display device and to perform control for continuously displaying a split image as a moving image on the display device. Good. Here, “suppressing the display of a normal image” refers to not displaying a normal image on a display device, for example. Specifically, although a normal image is generated, the normal image is not displayed on the display device by not outputting the normal image to the display device, or the normal image is not displayed on the display device by not generating the normal image. Refers to that. The split image may be displayed using the entire screen of the display device, or as an example, the split image may be displayed using the entire display area of the split image shown in FIG. As used herein, “split image” refers to an image output from a phase difference image group (for example, a first image and a second image output from a first pixel group when a specific imaging device is used). A split image based on the second image output from the pixel group can be exemplified. Examples of “when using a specific image sensor” include a case where an image sensor consisting of only a phase difference pixel group (for example, a first pixel group and a second pixel group) is used. In addition to this, a case where an image sensor in which phase difference pixels (for example, a first pixel group and a second pixel group) are arranged at a predetermined ratio with respect to a normal pixel can be exemplified.

  Various conditions are conceivable as conditions for suppressing the display of the normal image and displaying the split image. For example, when the display instruction of the normal image is canceled while the display of the split image is instructed, the display control unit 36 performs control to display the split image without displaying the normal image on the display device. It may be. Further, for example, when the photographer looks into the hybrid viewfinder, the display control unit 36 may perform control to display the split image without displaying the normal image on the display device. Further, for example, when the release button 211 is pressed halfway, the display control unit 36 may perform control to display a split image without displaying a normal image on the display device. Further, for example, when the pressing operation is not performed on the release button 211, the display control unit 36 may perform control to display the split image without displaying the normal image on the display device. Further, for example, when the face detection function for detecting the face of the subject is activated, the display control unit 36 may perform control to display the split image without displaying the normal image on the display device. Here, a modification example has been described in which the display control unit 36 suppresses the display of the normal image. However, the present invention is not limited to this. For example, the display control unit 36 displays the split image of the full screen over the normal image. You may control.

  Further, in each of the above embodiments, as an example, as illustrated in FIG. 6, the imaging element 20 in which each of the first pixel L and the second pixel R is arranged in units of one pixel is illustrated. The imaging element 600 shown in FIG. 26 may be used as an example without being limited thereto. As an example, the image sensor 600 illustrated in FIG. 26 has a first pixel L (for example, a photodiode into which light for the left eye image is incident) and a second pixel R for each of all the pixels as compared to the image sensor 20. The difference is that a pair of photodiodes (for example, photodiodes to which light for the right eye image is incident) is arranged one by one. In the example shown in FIG. 26, each pixel has a microlens 602, and the first pixel L and the second pixel R in each pixel are arranged so that light enters through the microlens 602. Yes. Examples of pixel signal readout patterns in each pixel include first to fourth readout patterns. The first read pattern refers to a pattern in which a pixel signal is read from only the first pixel L. The second readout pattern refers to a pattern in which a pixel signal is read from only the second pixel R. The third readout pattern is a pattern in which pixel signals are simultaneously read from each of the first pixel L and the second pixel R via separate routes (for example, via individual signal lines provided for each pixel). Point to. The fourth readout pattern refers to a pattern in which the pixel signal is read from the first pixel L and the second pixel R as a pixel signal of a normal pixel through one signal line provided for each pixel. The pixel signal of the normal pixel is, for example, a pixel signal obtained by adding the pixel signal of the first pixel L and the pixel signal of the second pixel R per pixel (for example, the photo signal of the first pixel L). (A charge obtained by combining the charge of the diode and the charge of the photodiode of the second pixel R).

  In the example shown in FIG. 26, one first pixel L and one second pixel R are arranged for each pixel. However, the present invention is not limited to this, and at least one first pixel L and It is sufficient that at least one second pixel R is arranged in one pixel. In the example shown in FIG. 26, the first pixel L and the second pixel R are arranged in the horizontal direction in one pixel. However, the present invention is not limited to this, and the second pixel L and the second pixel R are not limited thereto. The pixels R may be arranged in the vertical direction, and the first pixels L and the second pixels R may be arranged in the oblique direction.

16 imaging lens 20, 600 imaging device 100 imaging device 100A generation unit 100B display unit 100C detection unit 100D control unit 100E moving unit 100F presentation unit 216 touch panel 302 focus lens

Claims (15)

An imaging element having first and second pixel groups that output first and second image signals by subjecting a subject image that has passed through the first and second regions in the photographic lens to pupil division and forming them respectively. One of a plurality of divided images obtained by dividing the first image of the first and second images based on the first and second image signals output from the first and second images in a predetermined division direction. A first divided image corresponding to the divided region adjacent to the divided region corresponding to the first divided image from a plurality of divided images obtained by dividing the second image in the dividing direction. A generating unit that generates a display image to be used for focusing confirmation in which the two divided images are arranged adjacent to each other in the dividing direction;
A display unit having a display area and a touch panel provided on the surface of the display area;
A detection unit that detects an instruction operation for indicating the first or second divided image of the display image displayed in the display area by detecting an instruction area of the indicator on the touch panel;
When the instruction operation is detected, at least a part of the instruction divided image instructed by the instruction operation of the first and second divided images and the instruction of the first and second divided images A partial image including at least a part of an adjacent divided image adjacent to the divided image in a predetermined direction is extracted from the display image, and the extracted partial image is detected by the detection unit in the display area. A control unit to display in a different area from the instruction area;
An image processing apparatus.   The control unit detects the instruction detected by the detection unit in the display area of the extracted partial image in a state where a parallax between the instruction divided image and the adjacent divided image included in the partial image is expanded. The image processing apparatus according to claim 1, wherein the image processing apparatus is displayed in an area different from the area.   The image processing apparatus according to claim 1, wherein the control unit displays the extracted partial image in an enlarged manner in a region different from the instruction region detected by the detection unit in the display region. .   The control unit controls the display of the extracted partial image in an area different from the instruction area detected by the detection unit in the display area, in accordance with the instruction operation in the display image. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus is not performed when a difference in brightness in an image at a position corresponding to the indication area detected by the detection unit is less than a predetermined value.   The area different from the instruction area is an area other than the central part in the parallax occurrence direction in the display image when the instruction area detected by the detection unit is included in the central part in the parallax generation direction in the display image. The image processing apparatus according to claim 1, which is:   The image processing apparatus according to claim 5, wherein the region other than the central portion in the parallax generation direction in the display image is a region other than the central portion in the parallax generation direction in the display region.   The area different from the instruction area is a parallax generation direction in the display area when the instruction area detected by the detection unit is included in one end of both ends of the parallax generation direction in the display image. The image processing apparatus according to any one of claims 1 to 6, wherein the image processing device is an end portion farther from the pointing area of the both end portions. The control unit is configured to convert an image included in the extracted partial image into an enlargement ratio, a saturation, and a contour enhancement degree determined according to parallax between the instruction divided image and the adjacent divided image included in the partial image. The image processing apparatus according to claim 1, wherein the image processing apparatus is adjusted according to at least one of the following. The image pickup device further includes a third pixel group that outputs a third image signal by forming a subject image that has passed through the photographing lens without being divided into pupils.
The generation unit further generates a captured image based on the third image signal,
The control unit causes the display unit to display the captured image and display the display image in the captured image when the generation unit generates the display image and the captured image. The image processing apparatus according to any one of claims 1 to 8. The image processing apparatus according to any one of claims 1 to 9,
An imaging device having the first and second pixel groups;
A storage unit for storing an image generated based on an image signal output from the image sensor;
An imaging apparatus including: The photographing lens has a focus lens movable along the optical axis direction,
A moving unit that moves the focus lens in the optical axis direction;
When the detection unit detects the instruction operation, the detection unit moves the instruction region in a predetermined intersecting direction that intersects the division direction in a state where the display image is displayed in the display region. Detect
The said control part performs control which moves the said focus lens to an optical axis direction according to a said 1st movement operation with respect to the said movement part, when the said 1st movement operation is detected by the said detection part. The imaging device according to 10. When the detection unit detects the instruction operation, the detection unit performs a second movement operation that is greater than or equal to a predetermined threshold with respect to a direction orthogonal to the intersecting direction in the indication area with the display image displayed in the display area Detected as an operation,
The imaging apparatus according to claim 11, wherein the control unit performs control to maintain an extraction position of the partial image when the second movement operation is detected by the detection unit.   Based on the parallax between the first image and the second image at a position corresponding to the indication region detected by the detection unit in the display image, the indication region of the indication region by the first movement operation is displayed. The imaging apparatus according to claim 11, further comprising a presentation unit that presents guidance information for guiding a moving direction. An imaging element having first and second pixel groups that output first and second image signals by subjecting a subject image that has passed through the first and second regions in the photographic lens to pupil division and forming them respectively. One of a plurality of divided images obtained by dividing the first image of the first and second images based on the first and second image signals output from the first and second images in a predetermined division direction. A first divided image corresponding to a divided region corresponding to the first divided image and a plurality of divided images obtained by dividing the second image in the dividing direction. Generating a display image used for focusing confirmation in which the two divided images are arranged adjacent to each other in the dividing direction;
The first display image displayed in the display area is detected by detecting an instruction area of an indicator with respect to the touch panel of a display unit having a display area and having a touch panel provided on a surface of the display area. Alternatively, an instruction operation for instructing the second divided image is detected,
When the instruction operation is detected, at least part of the instruction divided image instructed by the instruction operation of the first and second divided images and the instruction division of the first and second divided images A partial image including at least a part of an adjacent divided image adjacent to an image in a predetermined direction is extracted from the display image, and the extracted partial image is an area different from the detected instruction area in the display area An image processing method including displaying on a screen. Computer
The image processing program for functioning as the said production | generation part, the said detection part, and the said control part in the image processing apparatus of any one of Claims 1-9.

Images