JP2018133094A - Image processing apparatus, image display device, and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable a user to comfortably check a photographed image.SOLUTION: An example of an image processing apparatus of the present invention comprises: a reading part that reads image data from which at least two images photographed in different photographing conditions can be created; an image creation part that creates, from the image data, an image with a depth of field within a first range; and a display control part that displays, on a monitor, the image created by the image creation part. When a subject included in the image is specified, the image creation part creates, from the image data, an image with a depth of field within a second range smaller than the first range.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image display apparatus, and an imaging apparatus that process a captured image that can be set after an imaging condition.

  In recent years, cameras that acquire captured images that can be set after imaging conditions have been attracting attention. As such cameras, there are a light field camera (Patent Document 1), a camera capable of taking a focus bracket (Patent Documents 2 and 4), a camera array (Patent Document 3) and the like.

  According to the light field camera (Patent Document 1), it is possible to set the shooting distance (distance of the in-focus surface in the object field), the aperture value, and the viewpoint, and the camera capable of focus bracket shooting (Patent Document 2, According to 4), the shooting distance can be set later, and according to the camera array (Patent Document 3), the shooting distance and the viewpoint (or aperture value) can be set later.

  Japanese Patent Application Laid-Open No. 2004-228688 gives a priority to a plurality of in-focus images included in a group of captured images taken with focus bracketing, and lengthens the display time of the in-focus image with the highest priority when displaying a slide show. Is also disclosed.

  Patent Document 4 discloses a technique for adjusting a shooting distance of a shot image in accordance with an operation of a scroll bar that scrolls in the vertical direction, a technique for focusing a shot image on a target subject designated by a user with a pointer, and the like. Is also disclosed.

US Patent Publication No. 7936392 JP 2010-166519 A JP 2011-22796 A JP 2003-141506 A

  However, there is still room for improvement in order for the user to check the captured image comfortably.

  Accordingly, an object of the present invention is to allow a user to comfortably check a captured image.

  An example of the image processing apparatus of the present invention includes a reading unit that reads image data that can generate at least two images shot under different shooting conditions, and an image whose depth of field is within a first range from the image data. An image generation unit to generate, and a display control unit to display the image generated by the image generation unit on a monitor. When the subject included in the image is specified, the image generation unit From the data, an image having a second range in which the depth of field is smaller than the first range is generated.

  An example of the image display device of the present invention includes any one of the image processing devices of the present invention and the monitor that displays the image.

  An example of the imaging apparatus of the present invention includes an imaging unit that acquires the image data, and an example of the image display apparatus of the present invention.

  According to the present invention, a user can comfortably check a captured image.

It is a figure explaining a light field camera. It is an example of a structure of an image file and a priority table. It is a figure explaining the slide show reproduction mode of 1st Embodiment. It is a figure explaining the registration mode of 1st Embodiment. It is a flowchart of the light field camera in imaging | photography mode of 2nd Embodiment. It is a figure explaining the normal reproduction mode of 3rd Embodiment. It is a figure explaining the normal reproduction mode of 4th Embodiment. It is a figure explaining normal reproduction mode of a 5th embodiment. It is a figure explaining normal reproduction mode (continuation of Drawing 8) of a 5th embodiment.

[First Embodiment]
Hereinafter, a light field camera will be described as a first embodiment of the present invention.

  FIG. 1A is a configuration example of the light field camera 10, and FIG. 1B is an example of the light field camera 10 viewed from the back.

  As shown in FIG. 1A, the light field camera 10 includes an imaging unit 11, an image processing unit 15, a camera control unit 16, a system control unit 17, a recording interface 18, a display unit 19, an operation member 20, and the like. Among these, the imaging unit 11 includes a photographic lens 12, a microlens array 13, an imaging device 14, and the like, and a card memory 21 that is a storage medium is attached to the recording interface 18.

  The taking lens 12 is an imaging lens that forms an image of a light beam from the object scene. The actual photographing conditions of the photographing lens 12 may be adjustable, but here it is assumed that they are unchanged for simplicity. The photographic lens 12 may be interchangeable with other photographic lenses having different specifications. However, for the sake of simplicity, the photographic lens 12 whose photographing conditions are unchanged is always attached to the light field camera 10. Assuming that

  The actual photographing conditions of the photographing lens 12 include an actual photographing distance of the photographing lens 12 (a distance to the surface conjugate with the microlens array 13 with respect to the photographing lens 12), an actual aperture value of the photographing lens 12, and At least one of the actual angle of view of the photographing lens 12 is included.

  Incidentally, when the actual shooting distance of the photographing lens 12 changes, the range of the photographing distance that can be set later (refocus range) changes, and when the actual aperture value of the photographing lens 12 changes, the aperture value that can be set later is changed. When the range changes and the actual angle of view of the photographic lens 12 changes, the range of the viewpoint that can be set later changes, but each range is unambiguous from the actual shooting distance, the actual aperture value, and the actual angle of view. It is decided.

  The microlens array 13 is formed by two-dimensionally arranging a plurality of microlenses on a surface in the vicinity of the imaging surface of the photographing lens 12.

  The imaging element 14 is formed by two-dimensionally arranging a plurality of light receiving elements (pixels) behind the individual microlenses belonging to the microlens array 13. Note that the pixel arrangement pattern of the pixel group (partial imaging element 14a) located on the rear side of each microlens is common among the individual microlenses.

  Here, when attention is paid to the partial image sensor 14a located on the rear side of one microlens, between the light beams individually incident on the pixel coordinates a, b, c, d, and e of the partial image sensor 14a, The passing areas in the exit pupil of the photographing lens 12 are different from each other. This applies similarly to all the partial image sensors 14a.

  Therefore, the pixel coordinates arranged in one direction on the partial image sensor 14a are sequentially set as a, b, c, d, and e, and an image is obtained one by one with pixels having the same pixel coordinates among the plurality of partial image sensors 14a. Once generated, the image Ia generated from the pixel coordinate a, the image Ib generated from the pixel coordinate b, the image Ic generated from the pixel coordinate c, the image Id generated from the pixel coordinate d, the pixel The relationship with the image Ie generated from the coordinate e is the same as the relationship between the five parallax images taken from different viewpoints of the object scene. Therefore, hereinafter, the images Ia, Ib, Ic, Id, and Ie are referred to as “parallax images”.

  Note that the actual pixel arrangement direction of the partial image sensor 14a extends in two directions instead of one direction, and the actual number of pixels of the partial image sensor 14a is sufficiently larger than “5”, but in the following description, For the sake of simplicity, it is assumed that the pixel consists of only five pixel coordinates a, b, c, d, and e arranged in one direction.

  The image processing unit 15 takes in a light field image (consisting of a series of parallax images Ia, Ib, Ic, Id, and Ie) generated by the image sensor 14 and temporarily stores the light field image in a buffer memory (not shown). To do. Further, the image processing unit 15 performs various types of image processing on the light field image on the buffer memory in accordance with an instruction from the system control unit 17.

  For example, the image processing unit 15 shifts the positions of the five parallax images Ia, Ib, Ic, Id, and Ie that make up the light field image, and then performs weighting synthesis to obtain shooting conditions (shooting distance, aperture value, One shot image with a (viewpoint) set is created. Hereinafter, a photographed image set after photographing conditions is referred to as a “refocus image”.

  Note that the shooting distance of the refocus image is determined by the position shift amount of the parallax images Ia, Ib, Ic, Id, and Ie. For example, when the amount of position shift of the parallax images Ia, Ib, Ic, Id, and Ie is “zero”, the shooting distance of the refocus image is the same distance as the plane conjugate with the microlens array 13 with respect to the shooting lens 12. When the position shift amount is a value other than “zero”, the shooting distance of the refocus image is the same distance as the non-conjugated surface of the shooting lens 12 and the microlens array 13.

  The aperture value of the refocus image is determined by the weight ratio of the parallax images Ia, Ib, Ic, Id, and Ie. For example, when the weight ratio of the parallax images Ia, Ib, Ic, Id, and Ie is “0: 0: 1: 0: 0”, the aperture value of the refocus image is maximized, and the parallax images Ia, Ib, Ic, When the weight ratio of Id and Ie is “1: 1: 1: 1: 1”, the aperture value of the refocus image is minimum.

  The viewpoint of the refocus image is also determined by the weight ratio of the parallax images Ia, Ib, Ic, Id, and Ie. For example, when the weight ratio of the parallax images Ia, Ib, Ic, Id, and Ie is “1: 0: 0: 0: 0”, the viewpoint of the refocus image is one end point, and the parallax images Ia, Ib, and Ic , Id, and Ie have a weight ratio of “0: 0: 0: 0: 1”, the viewpoint of the refocus image is the other end point.

  Note that any known light field camera configuration can be applied to the configuration of the imaging unit 11 described above, and the operation of any known light field camera can be applied to the operation of the image processing unit 15 described above. Applicable. Examples of known light field cameras include those disclosed in Japanese Patent Application Laid-Open No. 2011-217411, US Pat. No. 7,936,392, and Japanese Patent Application Laid-Open No. 2011-259218.

  The camera control unit 16 includes a mechanism that drives the imaging unit 11 and a circuit that controls the mechanism. The camera control unit 16 drives the image sensor 14 in accordance with an instruction from the system control unit 17 and acquires a light field image.

  The card memory 21 can store a large number of light field image image files (described later) acquired by the imaging unit 11 and light field image image files acquired by other light field cameras. It is.

  The recording interface 18 serves as an interface between the card memory 21 attached to the light field camera 10 and the system control unit 17. The recording interface 18 writes a light field image image file to the card memory 21 and reads a light field image image file from the card memory 21 in accordance with an instruction from the system control unit 17.

  The display unit 19 includes a multi-touch panel and includes at least a monitor such as a liquid crystal color monitor and a multi-touch sensor that detects a user operation on the monitor. Among them, the multi-touch sensor has two or more multi-touch sensors in addition to user operations using one finger (touch (touch), slide (trace), tap (tap), double tap (tap twice)), etc. It is possible to detect user operations (such as pinching (pinching), pinching in (narrowing), pinching out (expanding), etc.) using the fingers at the same time. Such a display unit 19 displays various images in accordance with instructions from the system control unit 17, detects the contents of user operations on the images, and transmits them to the system control unit 17.

  In addition to a power button (not shown), the operation member 20 includes various input switches such as a release button 20a, a reproduction button 20b, and a cross button 20c shown in FIG. The user inputs various instructions such as a shooting instruction and a mode switching instruction to the system control unit 17 via the operation member 20. In the present embodiment, since the display unit 19 configured by a touch panel is used, the operation members 20 other than the power button can be omitted.

  The system control unit 17 is a computer that includes a ROM, a RAM, a CPU, and the like (not shown). The CPU controls each part of the light field camera 10 by executing a program stored in the ROM, and operates the light field camera 10 in various modes such as a photographing mode and a reproduction mode. Note that the RAM is a memory that temporarily stores data necessary during the operation of the CPU.

  Incidentally, when the light field camera 10 is in the shooting mode, the system control unit 17 drives the imaging unit 11 in accordance with the shooting instruction from the user to acquire the light field image, and creates an image file of the light field image. And stored in the card memory 21.

  When the light field camera 10 is in the playback mode, the system control unit 17 displays the image file on the card memory 21 as a thumbnail or displays and displays the image file on the card memory 21.

  FIG. 2A is a configuration example of an image file. As shown in FIG. 2A, this image file has an area storing a light field image and an area storing accompanying information of the light field image. The accompanying information includes, for example, subject information and thumbnail images.

  First, subject information is information that describes each of one or more subjects that appear in a light field image. The subject information of each subject includes attribute information that indicates subject attributes of the subject and the presence of the subject. Area information indicating the area, distance information indicating the subject distance of the subject, and priority order information indicating the priority order of the subject are included.

  Among these, the attribute information includes subject type information (human, building, tree, mountain, sea,...) Indicating the type of subject, and subject name information (Mr. A, Mr. B, etc.) for distinguishing subjects of the same type from each other. Building 1, Building 2, Tree 1, Tree 2, ...).

  Next, the thumbnail image is a size-reduced version of the refocus image created from the light field image. Here, it is assumed that a plurality of thumbnail images having different shooting conditions (shooting distance, aperture value, viewpoint combination) are stored in the image file. Among the plurality of thumbnail images, a thumbnail image of a pan focus image (= a thumbnail image of a refocus image having the maximum depth of field, that is, the maximum aperture value) is also included.

  Of the accompanying information described above, the subject information and the thumbnail image are extracted from the light field image by the image processing unit 15 when the image file is created or necessary. Alternatively, the image file is extracted from the light field image by another light field camera or a personal computer at the time when the image file is created or necessary.

  Incidentally, in order to extract a subject existing area and subject attributes from a light field image, a known subject extraction process such as template matching may be performed on the light field image. However, in order to extract the existence area and subject attributes of all subjects, a plurality of refocus images having different shooting distances are created from the light field image, and subject extraction is performed for each of the plurality of refocus images. It is necessary to perform processing. Alternatively, it is necessary to create a pan focus image from the light field image and subject extraction processing to the pan focus image. Note that the pan-focus image corresponds to a refocus image having the maximum depth of field (maximum aperture value). Further, in order to extract the subject distance of the subject from the light field image, it is necessary to perform distance calculation processing according to the coordinates of the subject on at least two parallax images constituting the light field image.

  FIG. 2B shows a priority table. As shown in FIG. 2B, the priority table stores priorities for each subject attribute. These priorities indicate relative priorities among the subject attributes. In FIG. 2B, the subject attributes that should be given higher priorities have lower priority values. This priority table is referred to when determining the priority order of the subject in the light field image.

  The priority table can be manually rewritten by the user as necessary. The priority table is automatically updated when the user manually registers priority order information of a certain image file (described later).

  Further, the storage destination of the above priority table is, for example, a database that collectively manages a large number of image files. The database storage destination is, for example, the card memory 21.

  In the above description, the storage location of the accompanying information of each image file is each image file. However, the storage location of some or all of the accompanying information of each image file may be a database. In that case, the accompanying information of each image file is associated with the file name of each image file.

  Hereinafter, the operation of the light field camera 10 in the shooting mode will be described.

  First, the system control unit 17 starts live view display of the object scene. In live view display, the system control unit 17 continuously drives the imaging unit 11 in the high-speed drive mode to repeatedly acquire through images, and sends the through images to the display unit 19 in the order of acquisition as live view images.

  Since the through image is not an image for storage, the resolution may be low. However, if the real-time property of the live view display is important, it is desirable that the through image be acquired with a frame period as short as possible. Therefore, for example, the imaging unit 11 in the high-speed drive mode performs “decimation readout” in which pixel coordinates to be read are limited to only the pixel coordinates c, and acquires a light field image including only the parallax image Ic as a through image. . This through image corresponds to a refocus image with the viewpoint being the front and the maximum aperture value (that is, the pan focus image with the viewpoint being the front).

  During the live view display described above, the system control unit 17 determines whether or not a shooting instruction has been input from the user, and if it has been input, starts shooting the storage light field image.

  In capturing the storage light field image, the system control unit 17 drives the imaging unit 11 in the detailed drive mode to acquire the storage light field image. The image processing unit 15 creates an after-view image based on the storage light field image, and the system control unit 17 performs the after-view display by sending the after-view image to the display unit 19.

  Note that since the storage light field image needs to include a lot of information, the imaging unit 11 in the detailed drive mode uses five types of pixel coordinates a, b, c, d, “All pixel readout” is performed for all of e, and a light field image composed of parallax images Ia, Ib, Ic, Id, and Ie is acquired as a storage light field image.

  Further, the shooting conditions of the after-view image are set to default (for example, the viewpoint is front and the aperture value is maximum).

  Further, the image processing unit 15 executes subject extraction processing on the storage light field image to make known the subject attribute and the existing area of each subject reflected in the storage light field image, and for each subject. By executing the distance calculation process, the subject distance of each subject is known. The image processing unit 15 also creates various thumbnail images from the storage light field image.

  In addition, when the system control unit 17 refers to the priority table (FIG. 2B) and recognizes the priority corresponding to each subject in the storage light field image, the system control unit 17 increases in order from the subject with the highest priority. By assigning priorities, the priorities of each subject are determined.

  However, if the required subject attribute priority is not written in the priority table (FIG. 2B), the system control unit 17 sets the priority of the subject attribute having the highest similarity to the subject attribute. It is regarded as the priority of the subject attribute.

  When two or more subjects having the same priority are shown in the storage light field image, the system control unit 17 determines the priority order of the subjects according to the subject distance of the subjects. For example, if the priority is common, the system control unit 17 may assign a higher priority to a subject with a shorter subject distance.

  Next, the system control unit 17 creates an image file (see FIG. 2A) of the storage light field image.

  In creating the image file, the system control unit 17 performs subject attribute, existence area, subject distance, priority order of each subject extracted from the storage light field image, and various thumbnail images extracted from the storage light field image. Are written in the image file.

  Next, the system control unit 17 sends the created image file to the recording interface 18, thereby saving the image file in the card memory 21 (the shooting mode).

  Therefore, in the shooting mode of the present embodiment, the priorities of a plurality of subjects in the light field image are automatically registered in the image file of the light field image.

  In this way, if the priority order registered in the image file is set to “subject priority order” instead of “distance priority order”, the convenience of the slide show playback mode or normal playback mode described later can be easily improved. it can.

  Hereinafter, the operation of the light field camera 10 in the slide show playback mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Here, as shown in FIG. 3A, the light field image includes Mr. A, Mr. B, Mr. C, and the tree 1, and Mr. A has the first priority, Mr. B has the second priority, Assume that Mr. C's priority is 3rd and Tree 1 is 4th.

  Next, the system control unit 17 refers to the accompanying information of the image file, and the subject distance L1 of the subject (Mr. A) having the first priority and the subject distance of the subject (Mr. B) having the second priority. L2, the subject distance L3 of the subject (Mr. C) with the third highest priority, and the subject distance L4 of the subject (tree 1) with the fourth highest priority are recognized. From the light field image, the image processing unit 15 includes a first refocus image (FIG. 3B) whose shooting distance is L1 and a second refocus image (FIG. 3C) whose shooting distance is L2. Then, a third refocus image (FIG. 3D) having a shooting distance L3 and a fourth refocus image (FIG. 3E) having a subject distance L4 are generated. Shooting conditions (viewpoint and aperture value) other than the shooting distance of these refocused images (FIGS. 3B, 3C, 3D, and 3E) are default (for example, the viewpoint is front and the aperture value is Minimum).

  Here, the first refocus image (FIG. 3B) is focused on Mr. A who has the first priority, and the second refocus image (FIG. 3C) has the priority. The third refocus image (FIG. 3D) is in focus on Mr. B who is in second place, and the fourth refocus image (FIG. 3D) is in focus on Mr. C who has the third priority. 3 (E)) focuses on the tree 1 having the fourth priority.

  Next, the system control unit 17 performs the first refocus image (FIG. 3B), the second refocus image (FIG. 3C), the third refocus image (FIG. 3D), and the fourth. Slide show reproduction is performed by sequentially sending refocused images (FIG. 3E) to the display unit 19.

  However, when the slide show is reproduced, the system control unit 17 sets the switching pattern between these refocused images according to the priority order of a plurality of subjects (A, B, C, and Tree 1).

  Specifically, the system control unit 17 makes the display timing earlier for a refocus image focused on a subject with higher priority, and lengthens the display time for a refocus image focused on a subject with higher priority (above, Slideshow playback mode).

  Therefore, in the slide show playback mode of the present embodiment, the shooting distance of the refocus image is switched in a pattern according to the priority order between the plurality of subjects and the subject distance of the plurality of subjects.

  Specifically, in the slide show playback mode of the present embodiment, a subject with a higher priority is focused at an earlier timing, and a subject with a higher priority is focused for a longer time.

  In the slide show playback mode of this embodiment, the slide show playback target is limited to only the refocus image focused on the subject, but the refocus image focused on a certain subject and the other subject focused. A refocus image focused on an intermediate distance between the subjects (refocus image not focused on the subject) may be inserted between the refocus images. However, it is desirable that the display time of the refocus image not focused on the subject is set shorter than the display time of the refocus image focused on the subject.

  Further, in the slide show playback mode of the present embodiment, it is assumed that the priority order information has been registered in the image file that is the target of the slide show playback, but if the priority order information has not been registered, Instead of performing slide show reproduction, pan-focus images focused on all subjects may be displayed continuously.

  In the slide show playback mode of the present embodiment, the light field camera 10 may be shifted to the manual registration mode (described later) when the priority order information is not registered in the image file that is the target of the slide show playback. .

  Hereinafter, the operation of the light field camera 10 in the normal playback mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Again, as shown in FIG. 3 (A), the light field image shows Mr. A, Mr. B, Mr. C, and the tree 1. Mr. A has the first priority, Mr. B has the second priority, Assume that Mr. C's priority is 3rd and Tree 1 is 4th.

  Next, the system control unit 17 refers to the accompanying information of the image file and recognizes the subject distance L1 of the subject (Mr. A) having the first priority. The image processing unit 15 generates a first refocus image whose shooting distance is L1 from the light field image. The shooting conditions (viewpoint and aperture value) other than the shooting distance of the refocus image are set to default (for example, the viewpoint is front and the aperture value is minimum). The system control unit 17 displays the refocused image on the display unit 19 (see FIG. 3B).

  When the user taps an arbitrary subject (for example, Mr. B) whose priority is 2nd or lower on the refocused image, the system control unit 17 causes the subject distance (L2) of the subject (Mr. B) located at the tap point. ), The shooting distance of the refocus image is switched to the same object distance (L2). Thereby, the focus destination of the refocus image is shifted from the subject (Mr. A) having the highest priority to the tapped subject (Mr. B), and the shooting conditions other than the shooting distance of the refocus image are as follows. (See FIG. 3 (B) → (C)).

  Thereafter, when a predetermined time (for example, 2 seconds) elapses, the system control unit 17 returns the shooting distance of the refocused image being displayed to the original value L1. As a result, the focus destination of the refocus image returns from the tapped subject (Mr. B) to the subject (Mr. A) with the highest priority, and the shooting conditions other than the shooting distance of the refocus image are as follows: Maintained (see FIG. 3 (C) → (B)).

  In addition, when the user taps another arbitrary subject (for example, Mr. C) whose priority is 2nd or lower on the refocus image, the system control unit 17 sets the subject (Mr. C) at the tap point. With reference to the subject distance (L3), the photographing distance of the refocus image is switched to the same subject distance (L3). As a result, the focus destination of the refocus image shifts from the subject with the highest priority (Mr. A) to the tapped subject (Mr. C), and the shooting conditions other than the shooting distance of the refocus image are as follows. (See FIG. 3 (B) → (D)).

  Thereafter, when a predetermined time (for example, 2 seconds) elapses, the system control unit 17 returns the shooting distance of the refocused image being displayed to the original value L1. As a result, the focus destination of the refocus image returns from the tapped subject (Mr. C) to the subject (Mr. A) with the highest priority, and the shooting conditions other than the shooting distance of the refocus image are as follows: Maintained (see FIG. 3D → B). The above is the normal playback mode.

  Therefore, in the normal playback mode of the present embodiment, the focus destination of the refocus image is maintained at the subject with the highest priority (Mr. A), and the subjects with the second priority or lower (Ms. B, C, Only when one of the trees 1) is tapped, shifts to the subject where the focus destination of the refocus image is tapped.

  In the normal playback mode of the present embodiment, the user operation for specifying the subject is a tap operation on the subject on the display unit 19, but may be another predetermined operation. For example, the user operates the cross button 20c to move the cursor 19a (see FIG. 1B) on the display unit 19 and presses the OK button when the cursor 19a is positioned on the subject. You may specify.

  In the normal playback mode of the present embodiment, the timing at which the shooting distance of the refocus image is changed and then restored is the timing at which a predetermined time (for example, 2 seconds) has elapsed since the user specified the subject. The timing when the user performs a predetermined operation after designating the subject may be used. For example, it may be the timing when the user taps an arbitrary point on the refocus image or the timing when the left direction key (return button) of the cross button 20c is pressed.

  Further, in the normal playback mode of the present embodiment, it is assumed that the priority information has been registered in the image file that is the target of normal playback, but if the priority information has not been registered, Instead of displaying a refocus image focused on the distance of the subject with the first priority, a pan focus image focused on all subjects may be displayed.

  Further, in the normal playback mode of the present embodiment, the light field camera 10 may be shifted to the manual registration mode (described later) when the priority order information is not registered in the image file that is the target of normal playback. .

  Hereinafter, the operation of the light field camera 10 in the manual registration mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Subsequently, the image processing unit 15 generates a refocus image from the light field image, and the system control unit 17 displays the refocus image on the display unit 19. The shooting conditions for this refocus image are set to default (for example, the viewpoint is front and the aperture value is maximum).

  In addition, when the system control unit 17 refers to the accompanying information of the image file and recognizes the existence area and priority of each subject in the light field image, the system control unit 17 near the subject in the refocus image on the display unit 19. A character image “rank: XX” indicating the priority order of the subject is superimposed and displayed (see FIG. 4A).

  Incidentally, FIG. 4 (A) shows A, B, C, and Tree 1 in the refocused image, with A's priority ranked first and B's priority ranked second. In this example, Mr. C's priority is 3rd and Tree 1 is 4th.

  Therefore, the user can check the current priority order of each subject on the display unit 19.

  If the priority information is not registered in the image file, the system control unit 17 displays a character image “rank: _rank” indicating that the priority of each subject is not registered in the vicinity of each subject. Display (not shown).

  When the user taps an arbitrary subject (for example, Mr. B) on the refocused image (see FIG. 4B), the system control unit 17 sets the priority of the subject (Mr. B) positioned at the tap point to 1. The character image on the display unit 19 and the priority information of the image file are updated so as to move up (see FIG. 4C), and the priority table is updated according to the updated priority information. Update.

  When the user taps another arbitrary subject (for example, Mr. C) on the refocus image (see FIG. 4D), the system control unit 17 sets the subject (Mr. C) at the tap point. The character image on the display unit 19 and the priority order information of the image file are updated so that the priority order increases to the first place (see FIG. 4E), and according to the updated priority order information. , Update the priority table (manual registration mode).

  Therefore, according to the manual registration mode of the present embodiment, the priority information registered in the image file can be manually rewritten by the user, or the user can manually input the priority information to an image file for which the priority information is not registered. You can also register at

  In the manual registration mode of the present embodiment, each time a subject is tapped, the priority of the subject is incremented to the first place. However, the user sequentially taps a plurality of subjects, and the subject tapped earlier. High priority may be given.

  Further, in the manual registration mode of the present embodiment, when the user designates an image file for which priority order information is not registered, the system control unit 17 automatically provisionally determines the priority order of each subject and temporarily determines the subject. The character image “rank: XX” indicating the priority order may be displayed in the vicinity of each subject. Thereby, the system control unit 17 can propose the temporarily determined priority order to the user. Note that the priority order provisional determination can be performed in the same manner as the priority order determination at the time of automatic registration (described above).

  In the manual registration mode of the present embodiment, the user operation for specifying the subject is a tap operation on the subject on the display unit 19, but may be another predetermined operation. For example, the user operates the cross button 20c to move the cursor 19a (see FIG. 1B) on the display unit 19 and presses the OK button when the cursor 19a is positioned on the subject. You may specify.

[Modification of First Embodiment]
The system control unit 17 according to the present embodiment sets the priority stored in the priority table only to the latest priority, but leaves the priority history in the priority table and determines the priority based on the history. Or you may make a temporary decision. In this way, it is possible to reflect not only the latest priority but also the past priority in the priority determination or provisional determination.

  Further, the system control unit 17 of the present embodiment may learn the tendency of manual registration and update the priority table every time manual registration is performed by the user. In this way, the tendency of manual registration by the user can be reflected in the priority order determination or provisional determination.

  In addition, the priority table of the present embodiment stores the priority for each subject attribute (Mr. A, Mr. B, Mr. C, Building 1, Building 2, Tree 1, Tree 2,...). Each person, building, tree, etc.) may be stored. However, in this case, if two or more subjects having a common subject type are reflected in the storage light field image, the priority is the same between the two or more subjects. It is desirable to determine the priority order of the subjects according to the subject distance of the subjects.

  In addition, the system control unit 17 according to the present embodiment performs the priority determination or provisional determination based on the priority table, but may also perform the priority determination based on the type of shooting scene (shooting theme) designated by the user before shooting. Good. For example, when the shooting scene designated by the user is “portrait”, the system control unit 17 may set the priority of the person higher than the priority of the subject other than the person. For example, when the shooting scene designated by the user is “macro shooting”, the system control unit 17 may set the priority of flowers higher than the priority of subjects other than flowers.

  Further, the system control unit 17 according to the present embodiment performs the priority order determination or provisional determination based on the priority table, but may instead perform the priority order based on the subject distance of the subject instead of the priority table. For example, the priority of a subject with a short subject distance may be set higher than the priority of a subject with a long subject distance.

  In addition, the system control unit 17 according to the present embodiment may cause the user to designate a subject before shooting, and may assign a higher priority to a subject similar to the designated subject.

  Further, the system control unit 17 of the present embodiment may determine the priority order of the subject according to the state of the object scene before shooting (for example, see the second embodiment).

[Second Embodiment]
Hereinafter, a light field camera will be described as a second embodiment of the present invention. Here, only differences from the first embodiment will be described. The difference is in the operation in the shooting mode. Specifically, the situation of the object scene before shooting is reflected in the priority order of the subject.

  FIG. 5 is an operation flowchart of the light field camera 10 in the shooting mode of the second embodiment. Hereinafter, each step will be described in order.

  Step S11: The system control unit 17 drives the imaging unit 11 in the high-speed drive mode to acquire a through image of the current frame, and sends the through image to the display unit 19 as a live view image.

  Step S12: The image processing unit 15 performs subject extraction processing on the through image of the current frame, thereby making the subject attribute and the existing area of each subject shown in the current frame known.

  Step S13: The system control unit 17 updates the object scene monitoring table based on the subject attribute of each subject in the current frame. However, in the first step S13, the object scene monitoring table is cleared before the update.

  Here, the scene monitoring table is a table for counting the number of appearances of the subject for each subject attribute. For example, “Mr. A”, “Mr. B”, “Mr. C”, “Tree” When “1” is shown, the number of appearances of “Mr. A”, the number of appearances of “Mr. B”, the number of appearances of “Mr. C”, and the number of appearances of “Tree 1” are incremented by 1 in this step. .

  Step S14: The system control unit 17 determines whether or not a shooting instruction has been input from the user. If not, the system control unit 17 proceeds to step S15. If not, the system control unit 17 proceeds to step S17.

  Step S15: The system control unit 17 determines whether or not a predetermined time (for example, 1 minute) has elapsed since the object scene monitoring table was cleared last time, and when it has elapsed, the system control unit 17 proceeds to Step S16. If not, the process returns to step S11.

  Step S16: The system control unit 17 clears the object scene monitoring table, and then returns to step S11.

  Step S17: The system control unit 17 drives the imaging unit 11 in the detailed drive mode to acquire a light field image for storing a new frame. The image processing unit 15 creates an after-view image based on the storage light field image.

  In addition, the image processing unit 15 performs subject extraction processing on the storage light field image, thereby knowing the subject attributes and the existing areas of the subjects shown in the storage light field image, and the distance to each subject. By executing the calculation process, the subject distance of each subject is known. The image processing unit 15 also creates various thumbnail images from the storage light field image.

  Step S18: The system control unit 17 performs after-view display by sending the after-view image created in the previous step to the display unit 19.

  Step S19: When the system control unit 17 refers to the scene monitoring table and recognizes the number of appearances of each subject appearing in the storage light field image, the system control unit 17 assigns a higher priority in order from the subject having the highest number of appearances. Determine the priority of each subject.

  Step S20: The system control unit 17 creates an image file (see FIG. 2) of the storage light field image.

  In creating the image file, the system control unit 17 creates the subject attribute of each subject, the existence area of each subject, the subject distance of each subject, and the storage light field image that are included in the storage light field image. Write various thumbnail images to the image file.

  Then, the system control unit 17 sends the created image file to the recording interface 18, thereby saving the image file in the card memory 21 (the shooting mode).

  Therefore, according to the shooting mode of the present embodiment, a subject that has appeared in the scene before shooting for a long time has a higher priority.

  Here, a subject that has appeared in the field before photographing for a long time can be regarded as a subject that has been noted by the user before photographing.

  Therefore, according to the shooting mode of the present embodiment, it is possible to increase the priority order of the subject that has received attention from the user before shooting.

  Further, in the shooting mode of the present embodiment, the situation of the object scene before shooting is monitored, so that the priority table (see FIG. 2B) need not be stored in the database.

  In the shooting mode of the present embodiment, the priority order of the subject is determined according to the appearance time of the subject in the scene, but may be determined according to the appearance size of the subject in the scene. When the subject is a person, it may be determined according to the appearance size of the person's face in the object scene.

  In the shooting mode of the present embodiment, the priority order of the subject is determined according to the appearance time of the subject in the scene, but is determined according to both the appearance time and the appearance size of the subject in the scene. Also good.

[Third Embodiment]
Hereinafter, a light field camera will be described as a third embodiment of the present invention. Here, only differences from the first embodiment will be described. The difference lies in the operation in the normal playback mode, specifically, the priority order information is not used.

  Hereinafter, the operation of the light field camera 10 in the normal playback mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Subsequently, the image processing unit 15 generates a pan focus image from the light field image, and the system control unit 17 displays the pan focus image on the display unit 19 (see FIG. 6A). Note that the viewpoint of the pan focus image is set to a default (for example, front).

  When the user taps an arbitrary subject (for example, Mr. B) on the pan focus image, the system control unit 17 recognizes the subject distance of the subject (Mr. B) located at the tap point, and captures the same subject distance. A refocus image having a distance and a non-maximum (for example, minimum) aperture value is displayed on the display unit 19 instead of the pan focus image. The viewpoint of this refocus image is the same as that of the pan focus image. As a result, the pan-focus image focused on all subjects is switched to a refocus image focused on some subjects (see FIGS. 6A to 6B).

  Thereafter, when a predetermined time (for example, 2 seconds) elapses, the system control unit 17 displays the pan focus image on the display unit 19 in place of the currently displayed refocus image. The viewpoint of this pan focus image is the same as that of the refocus image. As a result, the refocus image focused on some subjects switches to a pan focus image focused on all subjects (see FIGS. 6B to 6D).

  When the user taps another arbitrary subject (for example, Mr. A) on the pan focus image, the system control unit 17 recognizes the subject distance of the subject (Mr. A) located at the tap point, and the subject distance. A refocus image having the same shooting distance and a non-maximum (for example, minimum) aperture value is displayed on the display unit 19 instead of the pan focus image. The viewpoint of this refocus image is the same as that of the pan focus image. As a result, the pan-focus image focused on all subjects is switched to a refocus image focused on some subjects (see FIGS. 6A to 6C).

  Thereafter, when a predetermined time (for example, 2 seconds) elapses, the system control unit 17 displays the pan focus image on the display unit 19 in place of the currently displayed refocus image. The viewpoint of this pan focus image is the same as that of the refocus image. As a result, the refocus image focused on some subjects switches to a pan focus image focused on all subjects (see FIGS. 6C to 6D).

  Therefore, in the normal playback mode of the present embodiment, a pan-focus image focused on all subjects is continuously displayed, and a refocus image focused on the subject is panned only when an arbitrary subject is tapped. Displayed instead of the focus image.

  In the normal playback mode of the present embodiment, the user operation for specifying the subject is a tap operation on the subject on the display unit 19, but may be another predetermined operation. For example, the user operates the cross button 20c to move the cursor 19a (see FIG. 1B) on the display unit 19 and presses the OK button when the cursor 19a is positioned on the subject. You may specify.

[Fourth Embodiment]
Hereinafter, a light field camera will be described as a fourth embodiment of the present invention. Here, only differences from the third embodiment will be described. The difference is that the subject can be continuously specified.

  Hereinafter, the operation of the light field camera 10 in the normal playback mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Subsequently, the image processing unit 15 generates a pan focus image from the light field image, and the system control unit 17 displays the pan focus image on the display unit 19 (see FIG. 7A). Note that the viewpoint of the pan focus image is set to a default (for example, front).

  When the user taps an arbitrary subject (for example, Mr. B) on the pan focus image, the system control unit 17 recognizes the subject distance of the subject (Mr. B) located at the tap point, and captures the same subject distance. A refocus image having a distance and a non-maximum (for example, minimum) aperture value is displayed on the display unit 19 instead of the pan focus image. The viewpoint of this refocus image is the same as that of the pan focus image. As a result, the pan-focus image focused on all subjects switches to a refocus image focused on some subjects (see FIG. 7A → FIG. 7B).

  When the user taps another arbitrary subject (for example, Mr. A) on the refocus image, the system control unit 17 refers to the subject distance of the subject (Mr. A) located at the tap point, and the subject. A refocus image having the same shooting distance as the distance and having a non-maximum (for example, minimum) aperture value is displayed on the display unit 19 instead of the currently displayed refocus image. The viewpoint of this refocus image is the same as that of the pan focus image. As a result, the refocus image focused on a part of the subject is switched to the refocus image focused on the other part of the subject (see FIG. 7B → FIG. 7C).

  When the left arrow key (return button) of the cross button 20c is pressed during display of the refocus image, the system control unit 17 sends the pan focus image to the display unit 19 instead of the refocus image being displayed. Display again. The viewpoint of this pan focus image is the same as the viewpoint of the pan focus image described above. As a result, the refocus image focused on some subjects switches to a pan focus image focused on all subjects (see FIG. 7C → FIG. 7D).

  Therefore, according to the normal playback mode of the present embodiment, the user can switch the subject to which the refocus image is focused in any order, and can return the refocus image to the pan focus image with one touch when necessary. it can.

  In the normal playback mode of the present embodiment, the timing for switching from the refocus image focused on some subjects to the pan focus image focused on all subjects is the timing when the return button is pressed. The timing when the user performs another predetermined operation may be used. For example, the timing when the user taps an arbitrary point other than the subject or the timing when the user presses the cancel button may be used.

[Fifth Embodiment]
Hereinafter, a light field camera will be described as a fifth embodiment of the present invention. Here, only differences from the above-described embodiment will be described. The difference is that the shooting distance is adjusted according to the zoom operation.

  The light field camera of this embodiment is provided with a zoom lever. For example, the zoom lever is provided in the vicinity of the release button 20a. Usually, a function for adjusting the actual angle of view of the taking lens 12 (optical zoom function) is assigned to the zoom lever in the shooting mode, and a function for adjusting the display size of the image (electronic zoom) in the zoom lever in the playback mode. Function). However, in the light field camera of this embodiment, a function for adjusting the shooting distance of the refocus image is assigned to the zoom lever in the playback mode.

  Hereinafter, the operation of the light field camera 10 in the normal playback mode will be described.

  First, the system control unit 17 reads an image file designated by the user from among a plurality of image files stored in the card memory 21, and writes the light field image stored in the image file to the buffer memory.

  Subsequently, the image processing unit 15 generates a refocus image from the light field image, and the system control unit 17 displays the refocus image on the display unit 19 (see FIG. 8A). The shooting distance of the refocus image is set to be the same as the subject distance of the subject (Mr. B) located at an intermediate distance among the subjects shown in the refocus image, for example. The shooting conditions are set to default (for example, the viewpoint is front and the aperture value is minimum).

  When the zoom lever is operated once to the tele side during the display of the refocus image, the system control unit 17 causes the subject (Mr. A) located on the far side of the current focus target subject (Mr. B) of the refocus image. ) And the shooting distance of the refocus image is switched to the same distance as the subject distance. As a result, the focus destination of the refocus image is shifted from the original focus target subject (Mr. B) to the far subject (Mr. A), and the shooting conditions other than the shooting distance of the refocus image are maintained. (Refer to FIG. 8 (A) → (B)).

  Further, when the zoom lever is operated once to the wide side during the display of the refocus image, the system control unit 17 is positioned on the near side of the current in-focus subject (for example, Mr. B) of the refocus image. The subject distance of the subject (Mr. C) is recognized, and the shooting distance of the refocus image is switched to the same subject distance. As a result, the refocused image shifts from the original in-focus subject (Mr. B) to the nearer subject (Mr. C), and the shooting conditions other than the shooting distance of the refocused image are maintained ( (See FIG. 8 (A) → (C)).

  However, even when the zoom lever is operated to the tele side, if there is no subject on the far side of the current in-focus subject, the system control unit 17 causes the shooting distance of the refocus image to reach the far end. The pan focus image is displayed on the display unit 19 instead of the refocus image. The viewpoint of this pan focus image is set to be the same as the viewpoint of the refocus image. Thus, the user can recognize that the shooting distance of the refocus image has reached the far end (see FIGS. 8B to 8D).

Similarly, even when the zoom lever is operated to the wide side, if there is no subject on the near side of the current in-focus subject, the system control unit 17 causes the refocus image shooting distance to be near. The pan focus image is displayed on the display unit 19 instead of the refocus image. The viewpoint of this pan focus image is set to be the same as the viewpoint of the refocus image. Thus, the user can recognize that the shooting distance of the refocus image has reached the near end (see FIGS. 8C to 8E).
Therefore, according to the normal playback mode of the present embodiment, the user can enlarge or reduce the shooting distance of the refocus image only by operating the zoom lever, and the shooting distance of the refocus image is at the far end or near end. Can be recognized by the appearance of the pan focus image (= the depth of field of the refocus image is maximized).

  In the normal playback mode of the present embodiment, the shooting distance of the refocus image is continuously changed during the period when the zoom lever is continuously operated (while the zoom lever is moved to the non-reference position). It is desirable to make it.

  In the normal playback mode of the present embodiment, the refocus image that is the target of normal playback is limited to only the refocus image focused on the subject, but the refocus image focused on the subject distance of a certain subject A refocus image focused on the intermediate distance between them (refocus image not focused on the subject) may be inserted between the refocus images focused on the subject distance of other subjects. However, it is desirable that the display time of the refocus image not focused on the subject is set shorter than the display time of the refocus image focused on the subject.

  In the normal playback mode of this embodiment, the zoom lever is continuously operated when the zoom lever is operated once (when the zoom lever returns to the reference position within a predetermined time after changing to the non-reference position). The change pattern of the shooting distance may be different from the case where the zoom lever is moved to the non-reference position for a longer time than the predetermined time.

  For example, when the zoom lever is operated once, the shooting distance is skipped from the subject distance of one subject to the subject distance of the next subject, and when the zoom lever is continuously operated, the shooting distance is continuously changed (frame advance or (Fast forward).

  Further, in the normal playback mode of this embodiment, when the zoom lever is still operated to the tele side after the shooting distance of the refocused image reaches the far end, it is shown in FIG. 8 (D) → FIG. 9 (F). As described above, the display size of the pan focus image may be continuously (or stepped) enlarged. Further, at the time of enlargement, it is desirable to keep the center of the farthest subject (Mr. A) consistent with the center of the pan-focus image display area (for example, the entire area of the display unit 19). Incidentally, FIG. 9F illustrates a state in which the center of Mr. A coincides with the center of the display area. As a result, the electronic zoom-up is performed with the farthest subject (Mr. A) as the center.

  Further, in the normal playback mode of this embodiment, when the zoom lever is still operated to the wide side even after the shooting distance of the refocus image reaches the near end, FIG. 8 (E) → FIG. 9 (G). As shown, the display size of the pan focus image may be reduced continuously (or stepwise). In the reduction, it is desirable to keep the center of the subject (Mr. C) located closest to the center of the pan-focus image display area (for example, the entire area of the display unit 19). Incidentally, FIG. 9G depicts a state in which the center of Mr. C coincides with the center of the display area. As a result, electronic zoom back is performed with the subject (Mr. C) as the center.

  In addition, the system control unit 17 in the normal playback mode of the present embodiment regards “subject distance of the subject located farthest” as the far end of the photographing distance, and regards “subject distance of the subject located closest” as the photographing distance. Is considered as the near end of the shooting distance range, but the far end of the range of shooting distance that can be set later is regarded as the far end of the shooting distance, and the near end of range of shooting distance that can be set later is set near the shooting distance. It may be considered as the end.

  In the normal playback mode of the present embodiment, the user operation for increasing the shooting distance or display size of the refocus image is “teleoperation of the zoom lever”, and the shooting distance or display size of the refocus image is reduced. The user operation for the zoom lever is “wide operation of the zoom lever”, but the user operation for enlarging the shooting distance or display size of the refocus image is “wide operation of the zoom lever” and the shooting distance of the refocus image. Alternatively, the user operation for reducing the display size may be a “teleoperation of the zoom lever”.

  In the normal playback mode of this embodiment, the user operation for adjusting the shooting distance of the refocus image or the display size of the pan focus image is the zoom lever operation, but may be another predetermined operation. For example, it is good also as operation of the up-down direction key of the cross button 20c, or a finger slide operation with respect to the display part 19. FIG. Further, when the up / down direction key of the cross button 20c is used, for example, a function for increasing the shooting distance or display size may be assigned to the up direction key, and a function for reducing the shooting distance or display size may be assigned to the down direction key. .

  In the above description, the function for expanding / reducing the shooting distance of the refocus image and the function for increasing / decreasing the display size have been developed sequentially. Also good.

  For example, when the zoom lever is tele-operated, the shooting distance of the refocus image is enlarged and the display size is enlarged simultaneously, and when the zoom lever is operated wide, the shooting distance is reduced and the display size is reduced simultaneously. Also good.

[Modification of each embodiment]
In each of the embodiments described above, the shooting distance, the aperture, and the viewpoint are described as the shooting conditions that can be set later. However, in the shooting conditions that can be set later, the “posture or shape of the focal plane”, that is, “Shooting distance distribution in the refocus image” may be added.

  Further, the function of the light field camera 10 of the first embodiment, the function of the light field camera 10 of the second embodiment, the function of the light field camera 10 of the third embodiment, and the light field camera 10 of the fourth embodiment. A light field camera having both the above function and the function of the light field camera 10 of the fifth embodiment may be configured.

  Further, in each of the above-described embodiments, the light field camera 10 has been described. However, the present invention is also applied to other cameras that can acquire a photographed image that can be set after photographing conditions, for example, a camera that can perform focus bracket photographing, a camera array, and the like. The invention is applicable.

  Further, in each of the above-described embodiments, the apparatus having both the imaging function and the monitor function has been described. However, the present invention can also be applied to an apparatus that does not have at least one of the imaging function and the monitor function. For example, the present invention can be applied to a digital photo frame, a tablet PC, a smartphone, a desktop PC, a notebook PC, and the like.

  Moreover, you may provide the function of at least one part of the apparatus of any one of embodiment mentioned above with respect to the computer executable program or the storage medium which memorize | stored it.

[Effects of First Embodiment]
As described above, the image processing apparatus (the image processing unit 15, the system control unit 17, and the recording interface 18) according to the first embodiment can perform a photographic image (light field image) that can be set after the photographic conditions (photographing distance, aperture value, viewpoint). ) Reading means (system control unit 17, recording interface 18), display control means (system control unit 17) for displaying the captured image (refocus image) on a monitor, and in the captured image (light field image) Adjusting means (system control unit 17, image processing unit 15) for switching the shooting distance of the captured image (refocused image) with a pattern according to the priority order between the plurality of subjects existing in the image and the subject distance of the subjects With.

  Therefore, the switching of the shooting distance (refocus image) is performed effectively. For example, a high priority subject can be actively focused, a high priority subject can be focused at an early timing, or a high priority subject can be focused for a long time.

  Further, the adjusting means determines a priority order among the plurality of subjects based on a table in which relative priorities between subject attributes are stored in advance. Therefore, the adjusting means can always properly determine the priority order.

  In addition, the adjustment means maintains the subject distance of the subject only when the user designates another subject while maintaining the subject distance of the photographed image at the same distance as the subject distance of the subject having the highest priority. The shooting distance is shifted to the same distance.

  Therefore, the focus destination of the captured image (refocus image) is maintained at the subject with the highest priority, and the focus destination of the captured image (refocus image) is only specified when the subject with the low priority is designated. However, it shifts to the subject.

  The adjusting unit sets the aperture value of the captured image to the maximum when the priority order is not yet determined.

  Therefore, the user can recognize that the priority order has not been determined by the appearance of the captured image (pan focus image) having the maximum aperture value.

  The adjusting means determines the priority order according to an instruction from a user.

  Therefore, the user can manually determine the undecided priority order or manually rewrite the determined priority order.

[Effects of Second Embodiment]
As described above, in the image processing apparatus (the image processing unit 15, the system control unit 17, and the recording interface 18) according to the second embodiment, the adjustment unit (system control unit 17) is long in the field before the captured image is captured. A subject that appears over time has a higher priority.

  Further, the adjustment means (system control unit 17) increases the priority of the subject that appears in a large size in the object scene before the photographing of the photographed image.

  Therefore, according to the image processing apparatus of the second embodiment, a subject that has received attention from the user before shooting has a higher priority. Further, according to the image processing apparatus of the second embodiment, the table in which the priorities are stored in advance is not essential.

[Operational effects of the third embodiment or the fourth embodiment]
As described above, the image processing apparatus (the image processing unit 15, the system control unit 17, and the recording interface 18) according to the third embodiment or the fourth embodiment can perform shooting after setting shooting conditions (shooting distance, aperture value, viewpoint). Reading means for reading an image (system control unit 17, recording interface 18), display control means (system control unit 17) for displaying the photographed image (refocus image) on a monitor, and the photographed image (refocus image) Adjusting means (system control unit 17 and image processing unit 15) that maintains the aperture value at the maximum and shifts the aperture value of the captured image (refocused image) to a non-maximum value only when the user designates the subject; Prepare.

  Further, when the user designates a subject, the adjustment means shifts the photographing distance of the photographed image (refocus image) to the same distance as the subject distance of the subject.

  Therefore, according to the image processing apparatus of the third embodiment or the fourth embodiment, the photographed image (pan focus image) having the maximum aperture value is continuously displayed, and an arbitrary subject is designated or a predetermined operation is performed. Only when the (return operation) is performed, the captured image (refocus image) focused on the subject is displayed instead of the pan focus image.

[Effects of Fifth Embodiment]
As described above, the image processing apparatus (the image processing unit 15, the system control unit 17, the recording interface 18) according to the fifth embodiment reads the captured image that can be set after the shooting conditions (shooting distance, aperture value, viewpoint). (System control unit 17, recording interface 18), display control means (system control unit 17) for displaying the captured image (refocused image) on a monitor, and the captured image (refocused) in response to an enlargement / reduction instruction from the user. Adjustment means (system control unit 17, image processing) for setting the aperture value of the photographed image (refocused image) to the maximum when the photographing distance of the image) is enlarged or reduced and the photographing distance reaches the far end or the near end. Part 15).

  Therefore, the user can enlarge or reduce the shooting distance of the photographed image (refocus image) simply by inputting an enlargement / reduction instruction, and the shooting distance of the photographed image (refocus image) is far or near. The timing of reaching the end can be recognized by the appearance of the captured image (pan focus image) having the maximum aperture value.

  In addition, when an enlargement instruction is input from the user after the photographing distance has reached the far end, the adjustment unit enlarges a display magnification of the photographed image (pan focus image) in accordance with the enlargement instruction, When a reduction instruction is input from the user after the shooting distance has reached the near end, the display magnification of the shot image (pan focus image) is reduced according to the reduction instruction.

  Therefore, the user can sequentially increase the shooting distance of the captured image (refocus image) and increase the display magnification of the captured image (pan focus image) by simply inputting an enlargement instruction. It is possible to sequentially reduce the shooting distance of the captured image (refocus image) and the display magnification of the captured image (pan focus image) simply by inputting.

  In addition, when the display unit enlarges the display magnification of the captured image (pan focus image), the adjustment unit matches the center of the farthest subject with the center of the display area of the captured image, and When reducing the display magnification of the pan focus image), the center of the subject located closest is matched with the center of the display area of the captured image.

  Therefore, the user simply inputs an enlargement instruction to increase the shooting distance of the shot image (refocus image) and to electronically zoom up the shot image (pan focus image) toward the farthest subject. By simply inputting a reduction instruction, it is possible to reduce the shooting distance of the captured image (refocus image) and to perform an electronic zoom back of the captured image (pan focus image) based on the closest subject. Can be performed sequentially.

  Further, the adjusting unit simultaneously enlarges / reduces the shooting distance of the captured image (refocus image) and the display magnification of the captured image (refocus image) in accordance with an enlargement / reduction instruction from the user.

  Therefore, the user can simultaneously perform enlargement / reduction of the photographing distance of the photographed image (refocus image) and enlargement / reduction of the display magnification of the photographed image (refocus image) simply by inputting an enlargement / reduction instruction.

DESCRIPTION OF SYMBOLS 11 ... Imaging part, 15 ... Image processing part, 16 ... Camera control part, 17 ... System control part, 18 ... Recording interface, 19 ... Display part, 20 ... Operation member, 12 ... Shooting lens, 13 ... Micro lens array, 14 ... Image sensor, 21 ... Card memory

Claims (9)

A reading unit that reads image data that can generate at least two images shot under different shooting conditions;
An image generating unit that generates an image having a depth of field in a first range from the image data;
A display control unit for displaying the image generated by the image generation unit on a monitor;
With
When the subject included in the image is specified, the image generation unit generates an image in a second range having a depth of field smaller than the first range from the image data. In the image processing apparatus according to claim 1,
When the subject is specified, the image generation unit generates an image focused on the subject from the image data. The image processing apparatus according to claim 1 or 2,
The display control unit displays an image with a depth of field on the monitor when a predetermined time elapses after the image with the depth of field on the monitor is displayed on the monitor. An image processing apparatus. In the image processing device according to any one of claims 1 to 3,
The first range is an image processing apparatus in which a depth of field is maximized. In the image processing device according to any one of claims 1 to 4,
The second range is an image processing apparatus in which a depth of field is minimized. In the image processing device according to any one of claims 1 to 5,
The display control unit is an image processing device that displays an image having a depth of field of the first range on the monitor when a predetermined operation is performed. In the image processing device according to any one of claims 1 to 6,
The image processing apparatus, wherein the image data is a light field image. The image processing apparatus according to any one of claims 1 to 7,
The monitor for displaying the image;
An image display device comprising: An imaging unit for acquiring the image data;
An image display device according to claim 8,
An imaging apparatus comprising: