JP2017175401A - Image processing device, method for controlling the same, imaging apparatus, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device which enables a user to easily use distance information relating to an image for image processing, without depending on the display medium of an output destination and to provide a method for controlling the image processing device.SOLUTION: The image processing device acquires the information of a photographed image and the distance information of a subject image in the image. A division image generation part 108 generates a plurality of division images corresponding to the distance ranges of subjects respectively from the photographed image on the basis of the distance information and the information of the display medium. A system control part 103 receives the selection operation of the division images by an operation part 109, acquires the resolution information of the display medium of the output destination or the information of a display screen size, and instructs the division image generation part 108 to perform division processing. An image processing part 107 generates a display image from the division images whose number corresponds to the display medium on the basis of the plurality of division images divided by the division image generation part 108 and outputs the display image to the display medium.SELECTED DRAWING: Figure 1

Description

  The present invention relates to image processing technology using image information and distance information related to the image information.

  There is an imaging device that can acquire distance information of a subject in a captured image. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for presenting an overhead map that represents the positional relationship in the depth direction of a subject in a shooting scene and the current in-focus distance using distance information of each subject. The photographer can easily grasp which subject is currently focused. There is an imaging device that can be connected to an external display medium as well as a display unit such as a liquid crystal display provided in the camera body. There is also a technique in which a digital camera converts the resolution of image data according to a display medium.

JP 2010-177741 A JP 2015-146500 A

However, the distance information of the subject in the conventional apparatus is exclusively used for image processing performed in the apparatus at the time of shooting, and is not assumed to be used by the user after shooting. For example, when a user performs image editing on a captured image, a technique for easily performing image processing using distance information of a subject has not been proposed.
SUMMARY OF THE INVENTION An object of the present invention is to provide an image processing apparatus and a control method therefor, in which a user can easily use distance information related to an image for image processing regardless of an output destination display medium.

  An image processing apparatus according to an embodiment of the present invention has a distance range that is different from an acquisition unit that acquires information of a captured image and distance information related to the image, and the distance range is divided from the distance information. A generation unit that generates a plurality of divided images from the image, an operation unit that is used for the selection operation of the divided image, and when the divided image is selected by the operation unit, the number of the divisions determined for each display medium Control means for performing control for generating a display image from the image and outputting the display image to the display medium.

  According to the image processing apparatus of the present invention, the distance information related to the image can be easily used for image processing by the user regardless of the output destination display medium.

It is a block diagram which shows the structural example of the imaging device which concerns on embodiment of this invention. 6 is a block diagram illustrating a configuration example of a divided image generation unit according to Embodiment 1 and Embodiment 2. FIG. 3 is a flowchart of overall processing during image editing according to the first exemplary embodiment. 6 is a flowchart for describing divided image generation processing according to the first exemplary embodiment. 6 is a diagram illustrating image data and a histogram in Embodiment 1. FIG. It is a figure explaining the depth division | segmentation process in Example 1. FIG. FIG. 4 is a diagram for explaining image data corresponding to a display medium A. FIG. 4 is a diagram for explaining image data corresponding to a display medium B. It is a figure which shows each example of a display on the display media A and B. FIG. 10 is a flowchart of overall processing during image editing in Embodiment 2. 10 is a flowchart for explaining divided image generation processing according to the second embodiment. It is a figure explaining the depth division process with respect to the display medium. It is a figure explaining the depth division process with respect to the display medium.

Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.
[Example 1]
A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus. FIG. 2 is a block diagram illustrating a configuration example of the divided image generation unit 108.

  In the imaging apparatus 100, the imaging unit 101 includes a lens, a shutter, a diaphragm, and an imaging element. The imaging unit 101 performs a shooting operation according to an instruction from the system control unit 103, acquires captured image data, and stores the acquired image data in the memory 105. A bus 102 connects each block and transmits and receives data and control signals.

  The system control unit 103 controls the entire imaging apparatus 100. The system control unit 103 includes a CPU (Central Processing Unit), and realizes processing to be described later by reading and executing a program stored in the nonvolatile memory 106. Further, the system control unit 103 performs display control on a display medium connected to the imaging apparatus 100. For example, the system control unit 103 performs display switching processing for the display unit 110 and the communication unit 111 described later, and instructs the image processing unit 107 to generate display image data in accordance with the resolution of the display medium.

  The distance information acquisition unit 104 acquires subject distance information corresponding to the captured image data obtained by the imaging unit 101. The distance information corresponding to the captured image data is stored in the memory 105. For example, the distance information acquisition unit 104 acquires a plurality of captured image data captured by changing the in-focus position from the imaging unit 101. And the process which acquires distance information from the focusing area | region of each picked-up image data and the focusing position information of picked-up image data is performed. In addition, when the imaging device of the imaging unit 101 is configured by a pixel unit that is divided into pupils, distance information for each pixel can be acquired from a phase difference between signals of a pair of subject images (parallax images) ( Patent Document 2).

The present invention can be applied in various embodiments as information corresponding to the depth of a subject in an image. That is, the information (depth information) indicated by the data corresponding to the depth of the subject directly represents the subject distance from the imaging device to the subject in the image, or the subject distance and depth of the subject in the image. Any information indicating a relative relationship may be used.
Specifically, the imaging device can output a pair of light beams that pass through different pupil regions of the imaging optical system as optical images, as a pair of image data, from a plurality of photoelectric conversion units. . An image shift amount of each region is calculated by a correlation calculation between the paired images, and an image shift map representing a distribution of the image shift amounts is calculated. Alternatively, the image shift amount is further converted into a defocus amount, and a defocus map representing the distribution of the defocus amount (distribution on the two-dimensional plane of the captured image) is generated. When this defocus amount is converted into the subject distance based on the conditions of the imaging optical system and the imaging element, distance map data representing the distribution of the subject distance is obtained. As described above, in this embodiment, the imaging apparatus may acquire image shift map data, defocus map data, or distance map data of a subject distance converted from the defocus amount. Note that each piece of map data may be stored in units of blocks or in units of pixels. At this time, as in normal image data, the number of bits of about 8 bits may be assigned for each minimum unit, and image processing, display, recording, and the like may be performed as a distance image in the same manner as image processing.
In the present embodiment, the distance information has a format of a distance image in which the value of each pixel represents a distance, but the format is not limited. For example, the distance information may be information indicating the position and distance of a region of a specific subject (for example, a person) existing in the shooting scene. In addition, there is no restriction | limiting in particular about the method of producing | generating distance information by the distance information acquisition part 104, The various methods disclosed by patent document 1 etc. can be used.

  The memory 105 stores various data such as captured image data obtained by the imaging unit 101 and distance information related to the captured image data acquired by the distance information acquisition unit 104. The memory 105 has a storage capacity sufficient to store a predetermined number of photographed image data. The nonvolatile memory 106 is an electrically erasable and recordable memory. For example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) is used. The nonvolatile memory 106 stores constants, setting values, programs, and the like for operation of the system control unit 103. The program here is a program for executing processing of each flowchart described later.

  In accordance with an instruction from the system control unit 103, the image processing unit 107 performs predetermined resizing processing such as pixel interpolation and reduction, and color conversion processing on the captured image data read from the imaging unit 101 and the memory 105. Do. The image processing unit 107 performs image processing such as blurring processing, sharpness processing, and color conversion processing on each distance range calculated using the distance information.

  The divided image generation unit 108 acquires captured image data, distance information, and display medium information read from the memory 105, generates data of a predetermined image (depth divided image), and stores the data in the memory 105. A depth division image is an image obtained by extracting a subject image included in a predetermined distance range in captured image data. The data of the depth division image is hereinafter referred to as “division image data”. For example, it is assumed that the distance range is divided into three distance ranges in the depth direction (the direction along the shooting direction). The first distance range is the distance range closest to the imaging device, and the third distance range is the distance range farthest from the imaging device. The first divided image data is data of an image (an image of a foreground subject) including a subject image within the first distance range. The second divided image data is data of an image including a subject image within the second distance range (subject image at an intermediate distance). The third divided image data is data of an image including a subject image within the third distance range (an image of a distant subject). As described above, the divided image data is image data when the entire captured image is divided according to the distance range of the subject.

  With reference to FIG. 2, an internal configuration of the divided image generation unit 108 will be described. The divided image generation unit 108 includes a distance range division processing unit 201, a divided image generation mask data generation processing unit 202, and a divided image data generation processing unit 203.

  A distance range division processing unit (hereinafter referred to as a division processing unit) 201 divides the distance range of the subject into a plurality of pieces using the distance information acquired from the memory 105. By dividing the subject distance range, subject distance range division information is generated.

  A divided image generation mask data generation processing unit (hereinafter referred to as a mask generation processing unit) 202 acquires distance range division information of a subject from the division processing unit 201 and generates mask image data for generating a divided image. A divided image data generation processing unit (hereinafter referred to as a data generation processing unit) 203 generates a plurality of pieces of divided image data using captured image data acquired from the memory 105 and mask image data for generating divided images. The generated divided image data is stored in the memory 105 in association with the captured image data.

  An operation unit 109 in FIG. 1 is a user interface unit that converts user operation information into control information of the imaging apparatus 100 and notifies the system control unit 103 of the control information. The display unit 110 displays the image data acquired from the memory 105 on a display medium such as an LCD (liquid crystal display). The communication unit 111 is a communication interface unit for transmitting / receiving control commands and data to / from an external device including a display medium. For example, the communication unit 111 communicates with an external device through a wired connection based on a standard such as USB (Universal Serial Bus) or HDMI (registered trademark) (High-Definition Multimedia Interface). Alternatively, communication with an external apparatus may be performed by wireless connection based on a standard such as a wireless local area network (LAN) or BlueTooth (registered trademark). Further, the imaging apparatus 100 and the external apparatus may be directly connected, or may be connected via a network such as a server or the Internet. In the present embodiment, it is assumed that display medium information can be received and display image data related to a captured image can be transmitted to at least a display medium of an external device to which the communication unit 111 is connected.

The recording medium I / F unit 112 is an interface unit with the recording medium 113. The recording medium 113 is a medium for recording captured image data and distance information related to the captured image data, and includes a memory card using a semiconductor memory, a magnetic disk, or the like.
The configuration shown in FIGS. 1 and 2 is an example. Therefore, the configuration of the imaging apparatus 100 according to the present invention is not limited to the configurations illustrated in FIGS. 1 and 2 as long as the operations described below can be performed.

  The operation of this embodiment will be described below with reference to FIGS. FIG. 3 is a flowchart for explaining the overall processing flow. FIG. 4 is a flowchart for explaining the divided image generation processing. FIG. 5 and FIG. 6 are explanatory diagrams of the division processing. FIG. 5A illustrates captured image data, and FIG. 5B illustrates a histogram of pixel values. FIG. 6A shows an example in which the subject distance is divided by the maximum number of divided images (denoted as N). FIG. 6B is a diagram for explaining a depth-divided image.

  As an embodiment, there is a form in which image processing such as refocus processing using color information and color space conversion is performed on captured image data. The refocus processing is processing for generating image data in which the focus position is readjusted after recording the image data. The user can use the imaging device 100 to perform desired image processing on a subject at a specific distance, and to perform image editing on a captured image.

  FIG. 7 and FIG. 8 are diagrams for explaining the correspondence between depth-divided images, display image data, and display image data and divided image data corresponding to different display media. The different display media are, for example, a display medium A with a high display resolution and a display medium B with a low display resolution. The display medium A can ensure a sufficient display resolution for the purpose of performing detailed image editing by the user, such as a display of a PC (personal computer). On the other hand, the display medium B cannot provide a sufficient display resolution for the purpose of the user performing detailed image editing, like a display panel attached to the main body of the imaging apparatus. FIG. 7 illustrates a depth division image for the display medium A, and FIG. 8 illustrates a depth division image for the display medium B. FIG. 9A shows a display example on the display medium A, and FIG. 9B is a schematic diagram showing a display example on the display medium B.

  First, the overall processing at the time of image editing will be described with reference to the flowchart of FIG. In step S <b> 301, the system control unit 103 acquires captured image data and distance information for the captured image data from the memory 105. Further, the system control unit 103 acquires the maximum number of divided images N from the nonvolatile memory 106 and stores it in the memory 105. The maximum number of divided images N is the maximum number of divided images, and is a fixed value or a variable value. For example, the maximum number N of divided images is determined by user designation.

  In step S <b> 302, the system control unit 103 acquires display destination display medium information. The display medium information in this embodiment is information such as the display resolution of the display medium and the size (screen size) of the display panel. The display medium information is information related to a display medium that is stored in, for example, the nonvolatile memory 106 and to which the imaging apparatus 100 can be connected. In this case, the system control unit 103 acquires the display medium information of the display medium connected to the imaging device 100 from the nonvolatile memory 106. In addition, the system control unit 103 acquires display medium information from the display medium whose connection is detected via the communication unit 111.

  Subsequently, the system control unit 103 calculates display number information (denoted as M) for determining the number of divided images that can be sorted by the user on the display image data from the display medium information. In this embodiment, the display number information M is stored in the nonvolatile memory 106 corresponding to the display medium information. For example, when the display medium information indicates the display resolution, the value of the display number information M is larger as the display resolution is higher. When the display medium information indicates the size of the display medium (display screen size), the larger the display medium size, the larger the value of the display number information M.

  In step S <b> 303, the divided image generation unit 108 generates divided image data according to an instruction from the system control unit 103. Using the captured image data stored in the memory 105, the distance information, and the maximum number of divided images N, a process of generating N pieces of divided image data is executed, and the generated N pieces of divided image data are stored in the memory. 105 is stored. Details of the divided image generation processing of S303 will be described later with reference to the flowchart of FIG.

  In step S304, the system control unit 103 instructs the image processing unit 107 to generate display image data in accordance with the display medium from the display number information M stored in the memory 105 and the N pieces of divided image data. Details of the data processing will be described later. In step S <b> 305, the system control unit 103 performs control to display an image on the display medium according to the display image data read from the memory 105. A specific display example will be described later. In the present embodiment, when connection to a plurality of display media is detected simultaneously with respect to the imaging apparatus 100, only the display medium with the highest resolution is displayed. In this case, the system control unit 103 performs the process of S303 using display medium information having the highest display resolution among the plurality of display media. Display image data corresponding to each display medium may be generated by repeating the subsequent S304 and S305 as many times as the number of display media to be displayed.

  In step S306, the system control unit 103 determines whether the operation unit 109 has detected a user operation. If a user operation is detected, the process proceeds to S307. If no user operation is detected, the process returns to S305. In step S307, the system control unit 103 determines a user operation. If it is determined that the user operation is an image processing operation on the depth-division image, the process proceeds to S308. If the user operation is not an image processing operation for the depth-division image, the process proceeds to S309. In the image processing operation on the depth-divided image, at least a depth-divided image selection operation is performed by a user operation. This selection operation is performed, for example, by designating in-plane coordinates of display image data when the display medium is a touch panel format. When a touch operation by the user is detected, the system control unit 103 selects divided image data from the coordinates of the position touched on the display image, using the information generated in S304. In addition, there is a selection operation by key input of the operation unit 109. In this case, the system control unit 103 performs control to emphasize and display the divided image corresponding to the key input, and the user performs a key operation for selecting desired divided image data.

  In step S <b> 308, the image processing unit 107 executes designated image processing on the selected depth division image according to an instruction from the system control unit 103. The designated image processing is, for example, processing such as image blurring processing, sharpness processing, and color conversion processing for the distance range corresponding to the depth division image selected by the user. These image processes are image processes for the divided images that can be performed by the user selecting the divided images. Thereafter, the process returns to S303 to continue the processing.

  In step S309, the system control unit 103 determines a user operation. If the user operation is an end operation in the image editing mode, the process proceeds to S310. If the user operation is not an end operation in the image editing mode, the process returns to S305. In step S310, the system control unit 103 performs control to record the image processed data. The image data that has been subjected to image processing is stored in the recording medium 113 via the recording medium I / F unit 112, and a series of processing ends.

  Next, the divided image generation process will be described with reference to the flowchart of FIG. 4 and FIGS. 5 and 6. In this embodiment, the maximum number of divided images N is N = 5. That is, it is assumed that five depth division images 1 to 5 shown in FIG. 6B are generated from the captured image data of FIG.

  In S401 of FIG. 4, the value of the reference variable (denoted i) of the divided image data is initialized to zero (i = 0). In step S <b> 402, the division processing unit 201 calculates a distribution of distances where the subject exists in the depth direction from the distance information. The distance information is in the form of a distance image in which the value of each pixel represents the distance of the subject in the captured image. For this reason, the division processing unit 201 obtains a histogram of pixel values and handles it as a histogram by distance related to the subject. FIG. 5B is a histogram showing the distribution frequency of the subject distance corresponding to the captured image data of FIG. The horizontal axis in FIG. 5B represents the distance in the depth direction, and the vertical axis represents the distribution frequency.

  In step S403, the division processing unit 201 calculates distance range information for the i-th divided section from the histogram illustrated in FIG. 5B and the maximum number of divided images N. A specific example will be described with reference to FIG. This exemplifies a distance range divided into five in the horizontal axis direction in the distance histogram for the subject. When N = 5, the distance range information of division 1 to division 5 is calculated. For the distance range divided by the maximum number of divided images N, data is previously stored in the nonvolatile memory 106 as constants. Alternatively, the division processing unit 201 uses the histogram of FIG. 5B to determine N division ranges (divided distance ranges) in the order of distances where the subject presence frequency is low.

  In step S404, the mask generation processing unit 202 generates divided image generation mask data from the i-th distance range information. Specifically, using the distance range information for the i-th divided image calculated in S403, a process of binarizing the pixel value is executed for the distance image that is the distance information. That is, the process of setting the pixel value within the distance range to 1 and the pixel value outside the distance range to 0 is performed, and divided image generation mask data is generated.

In step S405, the data generation processing unit 203 generates i-th divided image data using the captured image data and the i-th divided image generation mask data. Specifically, a process of multiplying each pixel of the captured image data by the pixel value of the divided image generation mask data is executed. Thereby, only the region where the pixel value of the divided image generation mask data is 1 is extracted from the captured image data, and the data of the depth divided image in FIG. 6B is generated. Next, in S406, the reference variable i is incremented and the i value is increased by one. In S407, the value of the reference variable i is compared with the maximum number of divided images N, and it is determined whether or not “i> N”. If the value of the reference variable i is larger than the maximum number N of divided images, the divided image generation process ends. If the value of the reference variable i is equal to or less than the maximum number of divided images N, the process returns to S403 and continues.
By performing the above processing, the divided image generation unit 108 generates data of the depth divided image obtained by dividing the captured image data in the distance range of the maximum number of divided images N according to the display medium.

  Next, the divided image data and display image data corresponding to the display media A and B will be described with reference to FIGS. Regarding the display number information M, the display number information corresponding to the high resolution display medium A is M = 5, and the display number information corresponding to the low resolution display medium B is M = 3.

  FIG. 7A shows divided images 1 to 5 as divided images for the display medium A. FIG. FIG. 7B shows a display image on the display medium A. FIG. 7C is an explanatory diagram of the association between the display image and the depth-divided image with respect to the display medium A. In the case of the display medium A with high resolution, N = M = 5. Therefore, the N pieces of divided image data generated by the divided image generation unit 108 are the same as the divided image data shown in FIG. That is, as shown in FIG. 7C, all the divided images 1 to 5 are selected. The image processing unit 107 generates display image data based on the image data of the divided images 1 to 5 and stores the display image data in the memory 105.

  FIG. 8A shows divided images 1 to 3 as divided images for the display medium B. FIG. 8B shows a display image on the display medium B. FIG. 8C is an explanatory diagram of the association between the display image and the depth-divided image on the display medium B. In the case of the display medium B having a low resolution, N> M (= 3). Therefore, a process of combining the divided images having the adjacent distance ranges among the N divided image data generated by the divided image generation unit 108 is executed, and M divided image data is generated. Specifically, the divided image 1 and the divided image 2 in FIG. 6B are combined to generate the divided image 1 in FIG. The divided image 3 and the divided image 4 in FIG. 6B are combined to generate the divided image 2 in FIG. As shown in FIG. 8C, data of the divided images 1 to 3 generated by the synthesis process is selected. The image processing unit 107 generates display image data shown in FIG. 8B based on the three pieces of divided image data.

  The combining process performed to generate M image data less than N based on the N divided image data is not limited to the present embodiment, and the divided image data to be combined is a distance range. As long as they are adjacent. For example, the divided image data to be combined is selected based on the shooting conditions at the time of shooting and the analysis result of the shot image. Specifically, when the distance range focused on the captured image data can be acquired, the control unit does not use the divided image data included in the focused distance range for synthesis, and the other distance range. Control for preferentially selecting and combining the corresponding divided image data is performed. In addition, it is possible to acquire a distance range when a face is detected and a distance range when a person registered by personal authentication is detected. In this case, the control unit performs control so that the divided image data included in these distance ranges are not combined as much as possible. By appropriately selecting the divided image data to be combined, image processing in which the influence of the combining process is suppressed is performed on the face-detected image or the registered person image.

  Based on the divided image data generated according to the display number information M, the system control unit 103 controls the generation of display image data corresponding to the distance relationship between the divided images and the shooting angle of view. Specifically, for each divided image data, the image processing unit 107 performs resizing processing so that the closer the image is (the smaller the distance in the depth direction), the smaller the image size. By arranging the image data after the resizing process as display image data, a display image within the shooting angle of view is acquired. For example, as shown in FIGS. 7C and 8C, the depth division image corresponding to the closest distance is arranged in front, and the depth division image corresponding to the long distance is arranged in the back. At that time, display image data arranged so that at least a part of the image can be seen is generated for all the divided image data. The arrangement of the display image data is performed by the system control unit 103 and the image processing unit 107, but is not limited to such a display image data generation example. The user can select all the divided image data separately. The system control unit 103 performs control to arrange and display images in a direction in which the subject distance increases or decreases in accordance with a user operation instruction. For example, the image data can be arranged so that the distance increases from the right to the left instead of from the front to the back.

  Further, as shown in FIG. 7C or FIG. 8C, the system control unit 103 stores selection information in which the coordinates (X, Y) of the display image data and the selection destination of the divided image are associated with each other in the memory. It memorizes to 105. The coordinates (X, Y) are coordinates designated by the user, the X coordinate represents the horizontal coordinate, and the Y coordinate represents the vertical coordinate. FIG. 7C shows an example in which the user designates coordinates on the divided image 2. The system control unit 103 can specify a subject (main subject or the like) included in the divided image 2.

  FIG. 9 shows an example in which an image is displayed on the display screen by display image data corresponding to the display medium. As shown in FIG. 9A, the display medium A with high resolution displays an image using the display image data of FIG. 7B. Further, as shown in FIG. 9B, the display medium B with a low resolution displays an image using the display image data of FIG. 8B. That is, in the display medium B, compared with the display medium A, the number of displayed images is reduced and the image is displayed, so that the visibility of the user is improved.

  In this embodiment, the division method is determined based on shooting information, shooting conditions, and subject detection information. Specifically, when dividing the image of the main subject in focus and the images before and after the image, a plurality of divided image data is generated by setting the number of divisions according to the resolution before and after the main subject. The In the case of macro photography, a setting is made to increase the number of divisions with priority on the close range and decrease the number of divisions in the long range. In portrait photography, a process of dividing a subject (face) of the maximum size and images before and after the subject is performed. A number of divisions corresponding to the resolution is set before and after the subject of the maximum size, and a plurality of pieces of divided image data are generated. In the case of an imaging apparatus having a face detection function, it is divided into a main distance range in which a face of a predetermined size or more is detected and a range other than the distance range. Further, it is divided into a main distance range in which a subject registered by the user is detected and a range other than the distance range. The main distance range and the other distance ranges are divided, and the non-main distance range is divided according to the resolution.

  In this embodiment, the captured image is divided into a plurality of images according to the distance range of the subject using distance information related to the captured image data. Each image is presented in a state that can be selected by the user. By using the display medium information when generating the display image, it is possible to display an image corresponding to a desired display medium. Therefore, image processing data for a subject existing at a specific distance can be provided according to various display media.

[Example 2]
Next, Example 2 will be described. In the first embodiment, the example in which the system control unit 103 controls the display number based on the display medium information when generating the display image data has been described. In the second embodiment, an example in which the number of divided images is controlled based on display medium information when the divided image generation unit 108 generates divided image data will be described. In addition, about the structure similar to the case of Example 1, the detailed description is abbreviate | omitted by using the code | symbol already used, and a difference is mainly demonstrated.

  In the present embodiment, in FIG. 2, the divided image generation unit 108 generates divided image data using not only the distance information but also the display medium information. That is, the division processing unit 201 acquires the distance information acquired from the memory 105 and the display medium information, and divides the distance range of the subject into a plurality.

  The present embodiment will be described in detail with reference to the flowcharts of FIGS. 10 and 11 and FIGS. 1, 2, 5, 9, 12, and 13. FIG. FIG. 12 is a diagram for explaining an example in which the distance in the depth direction is divided by the maximum number N of divided images. FIG. 13 is a diagram for explaining an example in which the distance in the depth direction is divided by a division number smaller than the maximum division image number N. The captured image data and the histogram in this embodiment are as shown in FIG.

  FIG. 10 is a flowchart for explaining the overall processing flow during image editing. Since each process of S301, S302, S305 to S310 is the same as the process described in FIG. 3 in the first embodiment, a detailed description thereof is omitted. The process proceeds from S302 to S900.

  In S900, the divided image generation unit 108 uses the photographed image data stored in the memory 105, the distance information, the display medium information, and the maximum number N of divided images in accordance with an instruction from the system control unit 103. Split image data is generated. The generated N pieces of divided image data are stored in the memory 105. The divided image generation processing in S900 will be described later with reference to FIGS.

  In next step S <b> 901, the system control unit 103 instructs the image processing unit 107 to generate display image data from N pieces of divided image data read from the memory 105. The generated display image data is stored in the memory 105 and display control is performed. And the process after S305 is performed.

  The divided image generation processing will be described with reference to the flowchart of FIG. Since the processing of S401, S402, and S404 to S407 is the same as the processing described in FIG. 4 in the first embodiment, the description is omitted, and only S103 is described. The process proceeds from S402 to S103. In step S <b> 103, the division processing unit 201 calculates distance range information for the i-th divided section from the distance histogram and the display number information M. Then, the process proceeds to S404.

  In this embodiment, the maximum divided image number N of the display medium A is set to N = 5. From the captured image data of FIG. 5A, the data of the divided images 1 to 5 shown in FIG. Further, the maximum divided image number N of the display medium B is set to N = 3. From the captured image data of FIG. 5A, the data of the divided images 1 to 3 of FIG. 13B are generated.

  The system control unit 103 performs control to generate display image data from the divided image data shown in FIGS. 12B and 13B according to the distance relationship between the divided images and the shooting angle of view. For example, display image data is output such that a depth division image corresponding to a short distance is in front, a depth division image corresponding to a long distance is arranged in the back, and at least a part of all the division images can be seen ( FIG. 9). Further, selection information in which the coordinates of the display image data designated by the user are associated with the divided image for which the coordinates are selected is stored in the memory 105. As in the case of the first embodiment, in the display medium B in which sufficient display resolution cannot be ensured for performing detailed image editing, the number of displayed images is reduced compared to the display medium A, thereby improving the visibility of the user.

  In this embodiment, the number of depth division images is determined based on the display medium information. Therefore, the process of combining a plurality of divided image data and reducing the number of displayed images is unnecessary. According to the present embodiment, by using the display medium information when generating the depth-divided image, it is possible to generate a display image corresponding to the display medium from the divided image determined for each display medium.

[Other Examples]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Imaging part 103 System control part 104 Distance information acquisition part 107 Image processing part 108 Divided image generation part 109 Operation part 110 Display part 111 Communication part

Claims (12)

An acquisition means for acquiring information of a photographed image and distance information related to the image;
Generating means for generating a plurality of divided images having different distance ranges from the image by dividing the distance range from the distance information;
Operation means used for selecting the divided images;
Control means for generating a display image from the number of the divided images determined for each display medium and outputting to the display medium when the divided image is selected by the operation means. An image processing apparatus.   The image processing apparatus according to claim 1, wherein the control unit acquires information on the display medium and determines a maximum number of the divided images used for generating the display image.   The image processing apparatus according to claim 1, wherein the control unit acquires information about the display medium and determines the number of divisions when the generation unit generates the divided image for each display medium. .   4. The image processing apparatus according to claim 2, wherein the information on the display medium includes information on a display resolution or a display screen size.   The control means controls the first display medium to generate a first display image from the divided images generated by the generation means with the first division number, and the second display medium Control is performed to generate a second display image from a divided image having a second division number smaller than the first division number by combining the divided images generated by the generation unit with the first division number. The image processing apparatus according to claim 2.   The control means controls the first display medium to generate a first display image from the divided images generated by the generation means with the first division number, and the second display medium The image processing apparatus according to claim 3, wherein the display image is controlled to be generated from the divided image generated by the generation unit with a second division number smaller than the first division number.   The image processing apparatus according to claim 5, wherein the first display medium has a higher display resolution or a larger display screen size than the second display medium.   The control unit performs control to display a display image generated from the plurality of divided images on the screen of the display medium, and associates the coordinates designated by the operation unit with the selected divided image. The image processing apparatus according to claim 1, wherein the image processing apparatus is stored in a storage unit.   The display image is an image in which the plurality of divided images are arranged in the depth direction according to the distance information, and the first divided image corresponding to the first distance information is closer than the first distance information. The image processing apparatus according to claim 8, wherein the image processing apparatus is arranged behind the second divided image corresponding to the second distance information which is a distance.   An imaging device comprising the image processing device according to claim 1 and an imaging unit. A control method executed by an image processing apparatus that processes information of a captured image and distance information related to the image,
An acquisition step of acquiring the image information and the distance information;
Generating a plurality of divided images having different distance ranges from the image by dividing the distance range from the distance information;
And a control step of generating a display image from the number of the divided images determined for each display medium and outputting the display image to the display medium when the divided image is selected by an operation unit. Control method.   A program for causing a computer of an image processing apparatus to execute each step according to claim 11.