JP2012004716A - Imaging apparatus and electronic apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform scene determination by discriminating a foreground from a background.SOLUTION: The photographing image of subjects (a flower 311, a mountain 312, and a field 313) is acquired as a determination target image (300). Subject distance information (a distance image 320) expressing a subject distance of each subject is acquired. Then, the determination target image is divided into areas, based on the subject distance information. For example, the whole image area of the determination target image is segmented into a division image area (301) where image data of the subject positioned on a foreground side exists and a division image area (302) where image data of the subject positioned on a background side exists. Scene determination is performed by each division image area, so as to perform image processing corresponding to a scene determination result by each division image area.

Description

  The present invention relates to an imaging apparatus such as a digital camera. The present invention also relates to an electronic device such as an imaging device, an image reproducing device, and an information terminal device.

  When shooting with an imaging device such as a digital camera, there are optimum shooting conditions (shutter speed, aperture value, ISO sensitivity, etc.) according to the shooting scene in a specific shooting scene. However, in general, manual setting of shooting conditions is complicated. Considering this, an imaging apparatus is often equipped with an automatic scene determination function that automatically determines a shooting scene and automatically optimizes shooting conditions. With this function, the shooting scene is determined by determining the type of subject existing within the shooting range or detecting the brightness of the subject, and the optimal shooting is selected from multiple registered shooting modes based on the determination scene. Select a mode. Then, the shooting conditions are optimized by shooting under the shooting conditions according to the selected shooting mode.

  Thus, the automatic scene determination function is a useful function for the user. However, depending on the state of the subject, it may be difficult to obtain the scene determination result desired by the user. In this case, the user may be forced to adjust the shooting composition until the desired scene determination result is obtained. For example, as shown in FIG. 18A, a state is considered in which a flower 911 located on the foreground side, a mountain 912 and a meadow 913 located on the background side are taken as subjects in the photographing range. An image 900 in FIG. 18A is an image obtained by shooting in that state. In addition, it is assumed that the user considers the flower 911 as a main subject and desires that the determination scene by the automatic scene determination function is a flower scene suitable for shooting a flower.

  A scene determination unit (not shown) having an automatic scene determination function performs a scene determination process on the image 900. Since the image 900 includes a flower 911, a mountain 912, and a meadow 913, the shooting scene of the image 900 is estimated to be a flower scene suitable for shooting a flower or a landscape scene suitable for shooting a landscape. Is done. The scene determination unit selects one of the flower scene and the landscape scene in consideration of the position and size of each subject on the image 900, and presents the selected one to the user as a determination scene. FIG. 18B shows a display screen when a landscape scene is set as a determination scene. In FIG. 18B, a region filled with dots represents a housing portion of the display unit (the same applies to FIG. 19B described later).

  When the display as shown in FIG. 18B is made, the user makes a display as shown in FIG. 19B until the judgment scene becomes a flower scene, that is, the judgment scene is a flower scene. It is necessary to adjust the shooting composition until For example, it is necessary to arrange the flower 911 at the center of the shooting range or reduce the shooting angle of view so that the shot image 920 of FIG.

  On the other hand, in an electronic apparatus such as an imaging apparatus, a method for searching for an image that meets a search condition from a plurality of recorded images has been proposed.

  For example, consider a case where the user is searching for an image similar to the image 900 of FIG. At this time, it is assumed that the user pays attention to the flower 911 and is looking for an image including the flower as a subject. In this case, the user instructs the electronic device to search for an image similar to the image 900. When this instruction is received, the electronic device performs a search focusing on the landscape against the user's intention. Sometimes. That is, an image including a landscape corresponding to the mountain 912 and the meadow 913 (for example, the image 931 and the image 932 in FIG. 20) may be output as a search result image instead of including a flower.

  For example, consider a case where the user is searching for an image similar to the image 920 in FIG. At this time, it is assumed that the user is searching for an image including a landscape corresponding to the mountain 912 and the meadow 913 as a subject. In this case, the user instructs the electronic device to search for an image similar to the image 920, but when receiving this instruction, the electronic device performs a search focusing on flowers against the user's intention. There is. That is, instead of not including a landscape corresponding to the mountain 912 and the grassland 913, an image including a flower (for example, the image 941 and the image 942 in FIG. 21) may be output as a search result image.

  Patent Document 1 below proposes a method of identifying the type of a subject using a distance measurement result and setting a shooting mode based on the identified type of the subject.

JP 2009-272799 A

  In the conventional automatic scene determination function described above, various subjects such as a subject located on the foreground side and a subject located on the background side are always present in accordance with a predetermined algorithm even though there are various subjects within the shooting range. A single scene determination result is derived and displayed. For this reason, when a desired scene determination result cannot be obtained, adjustment of the shooting composition as described above is required. If a scene determination that distinguishes the foreground and the background can be performed, such adjustment of the shooting composition is unnecessary or reduced, and the convenience for the user is improved. In addition, the method of patent document 1 is not a method which contributes to such a convenience improvement.

  The same applies to image retrieval. That is, in the above-described conventional image search method, an image search focusing only on the foreground subject or an image search focusing only on the background subject may be performed, contrary to the user's intention. If an image search that distinguishes the foreground and the background according to the user's intention can be performed, a search that matches the user's intention can be performed.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging apparatus capable of performing scene determination such that a foreground and a background are distinguished. It is another object of the present invention to provide an electronic device that can perform an image search such that a foreground and a background are distinguished.

  An imaging device according to the present invention includes an imaging element that outputs image data of a subject, an area dividing unit that divides an entire image area of a determination target image based on an output of the imaging element into a plurality of divided image areas, and the plurality of the plurality of divided image areas. A scene determination unit configured to determine two or more divided image areas of the divided image areas as determination areas, and for each determination area, determine a scene of an image in the determination area based on image data in the determination area; It is characterized by that.

  Thereby, for example, it is possible to individually perform scene determination focused on the foreground subject and scene determination focused on the background subject. If image data is generated using the results of the scene determination, it is possible to individually optimize the image data of the foreground subject and the background subject.

  In addition, for example, a display unit that displays a result of scene determination for each determination region may be further provided in the imaging apparatus.

  Thereby, the user can recognize the result of each scene determination, and can confirm whether the result is a desired result.

  Then, for example, when a predetermined shooting instruction for instructing acquisition of a target image is given, the output of the image sensor is used to generate image data of the target image under conditions according to the result of scene determination for each determination region It is preferable to further provide an image processing unit for the imaging apparatus.

  As a result, the image data of the foreground subject and the background subject can be individually optimized.

  More specifically, for example, shooting parameters corresponding to the result of scene determination for the determination region are set for each determination region, and the image processing unit uses the corresponding shooting parameter for each determination region. The image data of the entire target image is generated by generating the image data of the image area in the target image corresponding to.

  An electronic apparatus according to the present invention includes an area dividing unit that divides an entire image area of a reference image that is an image of a subject into a plurality of divided image areas, and a search reference from the plurality of divided image areas according to a given selection instruction. A search processing unit that selects a region and searches a plurality of non-reference images for a non-reference image having an image feature corresponding to the image feature of the search reference region.

  Accordingly, for example, it is possible to distinguish between searching for an image similar to the foreground subject image and searching for an image similar to the background subject image.

  Further, for example, the search is a process of selecting a non-reference image that matches a search condition from the plurality of non-reference images, and the search processing unit includes subject distance information of the reference image and subjects of each non-reference image. The distance information may also be used to determine whether each non-reference image meets the search condition.

  Thus, for example, an advanced search such as searching for an image having “flower” on the foreground side or searching for an image having “flower” on the background side becomes possible.

  In addition, a search processing unit that searches for a search target image that satisfies a search condition from a plurality of search target images that are images of a subject, and a whole image area of the search target image for each search target image is divided into a plurality of divisions. You may make it form the other electronic device provided with the area | region division part divided | segmented into an image area | region. In the other electronic device, for each image to be searched, the search processing unit evaluates the suitability between each of the divided image regions of the image to be searched and the search condition, so that the image to be searched satisfies the search condition. It may be determined whether or not it conforms.

  As a result, for example, it is possible to search for an image in which the foreground subject matches the search condition, an image in which the background subject matches the search condition, etc., and depending on the search condition, these images are distinguished and detected. It is also possible.

  At this time, for example, the search processing unit may also determine whether each searched image meets the search condition by using subject distance information of each searched image.

  Thus, for example, an advanced search such as searching for an image having “flower” on the foreground side or searching for an image having “flower” on the background side becomes possible.

  According to the present invention, it is possible to provide an imaging apparatus capable of performing scene determination in which the foreground and the background are distinguished and an electronic apparatus capable of performing image search in which the foreground and the background are distinguished.

1 is a schematic overall block diagram of an imaging apparatus according to a first embodiment of the present invention. It is an internal block diagram of the imaging part shown by FIG. FIG. 2 is a block diagram of a part that contributes to automatic scene determination and imaging condition optimization included in the imaging apparatus of FIG. 1. It is a figure showing the relationship between an input image and a scene determination area | region. It is a flowchart showing the procedure of the imaging | photography operation | movement which concerns on 1st Embodiment of this invention. It is a figure which concerns on 1st Embodiment of this invention and shows the relationship between a determination target image and a target image. The figure (a) which shows a judgment object picture, the figure (b) which shows a distance picture concerning a 1st embodiment of the present invention, and the figure (c) showing signs that a judgment target picture is divided into areas based on a distance picture ). It is a figure which shows a mode that the result of a scene determination process is displayed concerning 1st Embodiment of this invention. It is a figure concerning a 1st embodiment of the present invention and showing signs that a target picture is generated by applying the 1st and 2nd specific image processing to the object input picture based on the output of an image sensor. FIG. 7A is a diagram illustrating a structure of an image file and FIG. 8B is a diagram illustrating a configuration of additional data according to the second embodiment of the present invention. It is a figure which shows a mode that the search process part is included in the main control part of FIG. 1 in connection with 2nd Embodiment of this invention. FIG. 9 is a diagram illustrating a plurality of image files stored in a recording medium according to the second embodiment of the present invention. It is a flowchart showing the procedure of the search operation | movement which concerns on 2nd Embodiment of this invention. It is a figure which concerns on 2nd Embodiment of this invention and shows the time change etc. of the display content of a display part. It is a figure which shows a mode that the reference | standard image is divided into areas according to 2nd Embodiment of this invention. It is a figure which concerns on 2nd Embodiment of this invention and shows the example of a display screen. It is a flowchart showing the procedure of the search operation | movement which concerns on 3rd Embodiment of this invention. FIG. 4A is a diagram illustrating an image that is a target of automatic scene determination according to the related art, and FIG. 5B is a diagram illustrating a state in which a result of the automatic scene determination is displayed. FIG. 4A is a diagram illustrating an image that is a target of automatic scene determination according to the related art, and FIG. 5B is a diagram illustrating a state in which a result of the automatic scene determination is displayed. It is a figure which shows the example of an image searched by the conventional image search method. It is a figure which shows the example of an image searched by the conventional image search method.

  Hereinafter, some embodiments of the present invention will be specifically described with reference to the drawings. In each of the drawings to be referred to, the same part is denoted by the same reference numeral, and redundant description regarding the same part is omitted in principle.

<< First Embodiment >>
A first embodiment of the present invention will be described. FIG. 1 is a schematic overall block diagram of an imaging apparatus 1 according to the first embodiment. The imaging device 1 is a digital still camera capable of capturing and recording still images, or a digital video camera capable of capturing and recording still images and moving images. The imaging device 1 may be mounted on a mobile terminal such as a mobile phone.

  The imaging device 1 includes an imaging unit 11, an AFE (Analog Front End) 12, a main control unit 13, an internal memory 14, a display unit 15, a recording medium 16, and an operation unit 17.

  FIG. 2 shows an internal configuration diagram of the imaging unit 11. The imaging unit 11 drives and controls the optical system 35, the diaphragm 32, the imaging element 33 such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and the optical system 35 or the diaphragm 32. And a driver 34. The optical system 35 is formed from a plurality of lenses including the zoom lens 30 and the focus lens 31. The zoom lens 30 and the focus lens 31 are movable in the optical axis direction. The driver 34 drives and controls the positions of the zoom lens 30 and the focus lens 31 and the opening degree of the diaphragm 32 based on the control signal from the main control unit 13, so that the focal length (view angle) and focus of the imaging unit 11 are controlled. The position and the amount of light incident on the image sensor 33 (in other words, the aperture value) are controlled.

  The image sensor 33 photoelectrically converts an optical image representing a subject incident through the optical system 35 and the diaphragm 32 and outputs an electrical signal obtained by the photoelectric conversion to the AFE 12. More specifically, the image sensor 33 includes a plurality of light receiving pixels arranged two-dimensionally in a matrix, and in each photographing, each light receiving pixel stores a signal charge having a charge amount corresponding to the exposure time. An analog signal from each light receiving pixel having a magnitude proportional to the amount of stored signal charge is sequentially output to the AFE 12 in accordance with a drive pulse generated in the imaging device 1.

  The AFE 12 amplifies the analog signal output from the imaging unit 11 (image sensor 33), and converts the amplified analog signal into a digital signal. The AFE 12 outputs this digital signal to the main control unit 13 as RAW data. The amplification degree of signal amplification in the AFE 12 is controlled by the main control unit 13.

  The main control unit 13 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. Based on the RAW data from the AFE 12, the main control unit 13 generates image data representing an image captured by the imaging unit 11 (hereinafter also referred to as a captured image). The image data generated here includes, for example, a luminance signal and a color difference signal. However, the RAW data itself is a kind of image data, and an analog signal output from the imaging unit 11 is also a kind of image data. The main control unit 13 also has a function as display control means for controlling the display content of the display unit 15, and performs control necessary for display on the display unit 15.

  The internal memory 14 is formed by SDRAM (Synchronous Dynamic Random Access Memory) or the like, and temporarily stores various data generated in the imaging device 1. The display unit 15 is a display device having a display screen such as a liquid crystal display panel, and displays captured images, images recorded on the recording medium 16, and the like under the control of the main control unit 13.

  A touch panel 19 is provided on the display unit 15, and the user can give a specific instruction to the imaging apparatus 1 by touching the display screen of the display unit 15 with an operating tool. An operation by touching the display screen of the display unit 15 with an operating body is referred to as a touch panel operation. A contact position between the operating tool and the display screen of the display unit 15 is referred to as a touch position. When the operating body touches the display screen of the display unit 15, coordinate values indicating the touched position (ie, touch position) are transmitted to the main control unit 13. The operation body is a finger, a pen, or the like. In the following, it is mainly assumed that the operation body is a finger. In this specification, when it is simply referred to as a display and a display screen, they refer to a display and a display screen in the display unit 15.

  The recording medium 16 is a non-volatile memory such as a card-like semiconductor memory or a magnetic disk, and stores a photographed image or the like under the control of the main control unit 13. The operation unit 17 includes a shutter button 20 that receives a still image shooting instruction, and receives various operations from the outside. The operation on the operation unit 17 is also referred to as a button operation in order to distinguish it from the touch panel operation. The content of the operation on the operation unit 17 is transmitted to the main control unit 13.

  The imaging apparatus 1 has a function of automatically determining a scene to be photographed by a user and automatically optimizing photographing conditions. In the first embodiment, the function will be mainly described below. FIG. 3 is a block diagram of a part particularly involved in realizing this function. The scene determination unit 51, the imaging control unit 52, the image processing unit 53, and the display control unit 54 are provided in the main control unit 13 of FIG. The imaging apparatus 1 is provided with a subject distance information generation unit 55. The subject distance information generation unit 55 may be provided outside the main control unit 13 as shown in FIG. 3 or may be provided inside the main control unit 13.

  The scene determination unit 51 is provided with image data of an input image. The input image refers to a two-dimensional image based on image data output from the imaging unit 11. The RAW data itself may be image data of the input image, or image data obtained by performing predetermined image processing (demosaicing processing, noise reduction processing, color correction processing, etc.) on the RAW data It may be image data. Since the imaging unit 11 can perform shooting at a predetermined frame rate, input images can also be sequentially obtained at a predetermined frame rate.

  The scene determination unit 51 sets a scene determination area in the input image, and performs a scene determination process based on image data in the scene determination area. The scene determination unit 51 can perform a scene determination process for each input image. The scene determination area can be set based on the subject distance information from the subject distance information generation unit 55, and the method will be described later.

  FIG. 4 shows the relationship between the input image and the scene determination area. In FIG. 4, reference numeral 200 represents an arbitrary input image, and reference numeral 201 represents a scene determination area set in the input image 200. The scene determination area 201 is the entire image area of the input image 200 or a part of the entire image area of the input image 200. In FIG. 4, it is assumed that the scene determination area 201 is a part of the entire image area of the input image 200. In the following description, as shown in FIG. 4, it is assumed that the shape of an arbitrary scene determination area represented by the scene determination area 201 is a rectangle. However, the shape of the scene determination area can be other than a rectangle.

  The scene determination process for the input image is performed using extraction of image features from the input image, detection of the subject of the input image, analysis of the hue of the input image, estimation of the light source state of the subject at the time of shooting the input image, and the like. Any known method (for example, the method described in JP2008-11289A or JP2009-71666A) can be used for the determination.

  A plurality of registered scenes are set in the scene determination unit 51 in advance. Among the registered scenes, for example, portrait scenes, which are shooting scenes that attract attention of people, landscape scenes, which are shooting scenes that attract attention of scenery, flower scenes, which are shooting scenes that attract attention of flowers, and autumn leaves Autumn scene, which is a shooting scene, animal scene, which is a shooting scene where animals are attracting attention, sea scene, which is a shooting scene where the sea is focused, daytime scenes that indicate shooting conditions during the day, and night scenes that indicate shooting conditions of the night view Scenes can be included. The scene determination unit 51 extracts an image feature amount useful for scene determination processing from the image data of the focused input image, thereby selecting a shooting scene of the focused input image from the plurality of registered scenes. Thus, the shooting scene of the focused input image is determined. The shooting scene determined by the scene determination unit 51 is called a determination scene. The scene determination unit 51 provides the shooting control unit 52 and the display control unit 54 with scene determination information indicating the determination scene and determination region information indicating the position and size of the scene determination region. The scene determination information and the determination area information can be given to any part (image processing unit 53 and the like) other than the imaging control unit 52 and the display control unit 54.

  The shooting control unit 52 sets a shooting mode for setting shooting conditions based on the scene determination information. The shooting conditions defined in the shooting mode include the shutter speed at the time of shooting the input image (that is, the length of the exposure time of the image sensor 33 for obtaining the image data of the input image from the image sensor 33), the input image. The aperture value at the time of shooting, the ISO sensitivity at the time of shooting the input image, the content of image processing (hereinafter referred to as specific image processing) to be performed on the input image by the image processing unit 53, and the like are included. The ISO sensitivity means sensitivity defined by ISO (International Organization for Standardization), and the brightness (luminance level) of the input image can be adjusted by adjusting the ISO sensitivity. Actually, the amplification factor of the signal amplification in the AFE 12 is determined according to the ISO sensitivity. After setting the shooting mode, the shooting control unit 52 controls the imaging unit 11 and the AFE 12 so that the image data of the input image can be obtained under the shooting conditions of the set shooting mode, and also controls the image processing unit 53. Note that the photographing control unit 52 can also execute camera control for optimizing the focus state and the exposure state.

  The image processing unit 53 generates an output image (that is, an input image after the specific image processing) by performing specific image processing on the input image. Depending on the shooting mode set by the shooting control unit 52, no specific image processing may be performed. In this case, the output image is the input image itself.

  For the sake of concrete explanation, it is assumed that there are N types of registered scenes (N is an integer of 2 or more). That is, assume that the number of the plurality of registered scenes is N. N types of registered scenes are referred to as first to Nth registered scenes. The shooting mode set by the shooting control unit 52 when the scene determined by the scene determination unit 51 is the i-th registered scene is referred to as the i-th shooting mode (i is an integer).

Regarding the first to Nth shooting modes, the shooting conditions defined by the i-th shooting mode and the shooting conditions defined by the j-th shooting mode are different from each other. This is usually true for different arbitrary integer i and j each other (however, i ≦ N and j ≦ N), imaging of the first. 1 to N _A number of shooting modes included in the shooting mode of the N Conditions can be the same as each other. N _A is an integer of less than N and 2 or more. For example, when N = 10, the shooting conditions for the first to ninth shooting modes are different from each other, but the shooting conditions for the ninth and tenth shooting modes may be the same (in this case). , N _A = 2).

  The subject distance information generation unit 55 (hereinafter sometimes abbreviated as the generation unit 55) detects the subject distance of each subject that is within the shooting range of the imaging unit 11, and thereby at each position on the input image. Object distance information representing the object distance of the object is generated. The subject distance for a certain subject refers to the distance in real space between the subject and the imaging device 1 (more specifically, the imaging device 33). In the present specification, the expression “the distance is large” and the expression “the distance is long” are synonymous, and the expression “the distance is small” and the expression “the distance is short” are synonymous. The subject distance information is a distance image in which each pixel value forming the subject has a subject distance measurement value. Any method including a known method (for example, a method described in Japanese Patent Application Laid-Open No. 2009-272799) can be used as a method for detecting a subject distance and a method for generating subject distance information.

  The subject distance information may be generated from the image data of the input image, or the subject distance information may be generated from information other than the image data of the input image. For example, subject distance information can be generated using a distance measuring sensor (not shown) that measures the subject distance of the subject at each position on the input image. As the distance measuring sensor, any known distance measuring sensor such as a distance measuring sensor based on the triangulation method can be used. Alternatively, for example, the distance image may be generated using a compound eye camera. Alternatively, for example, a contrast detection method may be used. In this case, for example, the image data is sequentially acquired from the AFE 12 while moving the position of the focus lens 31 by a predetermined amount, and the distance image can be generated from the high frequency component in the spatial frequency component of the acquired image data.

  Alternatively, for example, the imaging unit 11 may be formed so that the RAW data includes information representing the subject distance, and the subject distance information may be generated from the RAW data. In order to realize this, for example, a method called “Light Field Photography” (for example, a method described in International Publication No. 06/039486 pamphlet or Japanese Patent Application Laid-Open No. 2009-224982; hereinafter referred to as “Light Field method”) is used. May be. In the light field method, by using an imaging lens having an aperture stop and a microlens array, the image signal obtained from the imaging device has information on the light traveling direction in addition to the light intensity distribution on the light receiving surface of the imaging device. It comes to include. Therefore, although not shown in FIG. 2, when the light field method is used, an optical member necessary for realizing the light field method is provided in the imaging unit 11. The optical member includes a microlens array or the like, and incident light from the subject enters the light receiving surface (in other words, the imaging surface) of the image sensor 33 via the microlens array or the like. The microlens array is composed of a plurality of microlenses, and one microlens is assigned to one or a plurality of light receiving pixels on the image sensor 33. As a result, the output signal of the image sensor 33 includes information on the traveling direction of the incident light to the image sensor 33 in addition to the light intensity distribution on the light receiving surface of the image sensor 33.

  The imaging apparatus 1 can perform a characteristic shooting operation using subject distance information. With reference to FIG. 5, this photographing operation will be described. FIG. 5 is a flowchart illustrating the procedure of the shooting operation of the imaging apparatus 1.

  First, in step S11, the imaging apparatus 1 stands by for input of a shooting instruction by the user, and executes the processes in and after step S12 when there is a shooting instruction. The shooting instruction is, for example, a button operation by the user to press the shutter button 20 or a predetermined touch panel operation. The shooting instruction corresponds to an instruction for shooting and generating a target image.

  In step S <b> 12, the imaging control unit 52 executes camera control for optimizing the focus state, the exposure state, and the like of the imaging device 1. For example, the position of the focus lens 31 is adjusted so that the subject imaged in a specific area on the image sensor 33 is in focus, and the aperture value and the signal in the AFE 12 are optimized so that the brightness of the input image is optimized. Adjust the amplification. At this time, it is also possible to perform camera control while paying attention to the main subject within the imaging range of the imaging unit 11. That is, in the camera control in step S12, the position of the focus lens 31, the aperture value, and the like are adjusted so that the main subject is in focus and the brightness of the main subject image on the input image is optimized. Anyway. The user can also specify the main subject using a touch panel operation or the like.

  In subsequent step S13, the subject distance information generation unit 55 generates subject distance information by detecting the current subject distance of each subject. Next, in step S14, the main control unit 13 sets the current input image, which is the latest input image, as the determination target image, and sets a region for the determination target image based on the subject distance information obtained in step S13. Split processing is executed. The area division process may be performed by the scene determination unit 51 or may be performed by an area division unit (not shown) provided in the main control unit 13. Through the region division processing, the entire image region of the determination target image is divided into a plurality of divided image regions based on the subject distance information. In the region division processing, the entire image region of the determination target image is divided so that a set of pixels having similar feature amounts and spatially close together forms one unit. A known region unification method or the like can be used for forming the set and dividing the image area. In the region division processing based on the subject distance information, the subject distance is used as a feature amount.

  As shown in FIG. 6, the determination target image is represented by reference numeral 300, and a specific example of region division processing for the determination target image 300 will be described with reference to FIG. In FIG. 6, reference numeral 350 represents a target image to be obtained in accordance with the photographing instruction. A determination target image 300 illustrated in FIG. 7A is an image captured in a state where a flower 311 located on the foreground side, a mountain 312 and a meadow 313 located on the background side are included in the photographing range as subjects. Of course, in any image, the subject distance of the subject located on the foreground side is shorter than that of the subject located on the background side. Accordingly, in the determination target image 300, the subject distance of the flower 311 is shorter than the subject distances of the mountain 312 and the meadow 313.

A distance image 320 representing subject distance information for the determination target image 300 is shown in FIG. The arbitrary distance image is a gray scale image. In the distance image, each pixel has a pixel value closer to zero as the corresponding subject distance is smaller, and has a larger pixel value as the corresponding subject distance is larger. Further, in the drawing showing the distance image, a portion having a relatively small pixel value (that is, a portion having a relatively small subject distance) is shown in black, and a portion having a relatively large pixel value (that is, a portion having a relatively large subject distance). ) Is shown relatively white. In the distance image 320 corresponding to the determination target image 300, a portion where the image data of the flower 311 exists is illustrated in a relatively black color, and a portion where the image data of the mountain 312 and the meadow 313 exists is illustrated in a relatively white color. Actually, each subject in the determination target image can take various subject distances. However, for simplification of explanation, the distance image 320 includes only the pixel values PV _A and PV _B. Assume that the entire image area of the distance image 320 is divided into an image area 321 corresponding to the black part in FIG. 7B and an image area 322 corresponding to the white part in FIG. Consider a case in which the pixel values of each pixel in the image area 321 are all PV _A and the pixel values of each pixel in the image area 322 are all PV _B. Here, 0 ≦ PV _A <PV _B.

  In step S14, the determination target image 300 is divided into regions based on the distance image 320, whereby the entire image region of the determination target image 300 is divided into a plurality of divided image regions. In this example, since the entire image area of the distance image 320 is divided into two image areas 321 and 322, the entire image area of the determination target image 300 is also divided into two divided image areas 301 and 322 as shown in FIG. It is divided into 322. The divided image area 301 is an image area corresponding to the image area 321 where the image data of the flower 311 exists, and the divided image area 302 includes image data of the mountain 312 and the meadow 313 corresponding to the image area 322. This is an image area to be processed. The hatched area in FIG. 7C corresponds to the divided image area 302. Note that in the determination target image 300, it is assumed that the subject distance of the grassland portion actually located on the left and right sides of the base of the flower 311 is relatively small, and as a result, the grassland portion can also be included in the divided image region 302. In this example, for the sake of simplification of description, it is assumed that the grassland portion is also included in the divided image region 301.

  Please refer to FIG. 5 again. When the region division processing in step S14 is completed, in step S15, the main control unit 13 determines whether or not the number of divided image regions (hereinafter referred to as region division number) obtained by the region division processing in step S14 is two or more. Judging. When the number of area divisions is 2 or more, that is, when the entire image area of the determination target image 300 is divided into a plurality of divided image areas, the processes of steps S16 to S18 are executed thereafter to perform the division. If not, the processes in steps S19 to S21 are subsequently executed. When the divided image regions 301 and 302 are set in the determination target image 300 as in the examples of FIGS. 7A, 7B, and 7C, the processes of steps S16 to S18 are executed.

  In step S16, the scene determination unit 51 executes a scene determination process for each divided image area of the determination target image. The result of each scene determination process is displayed on the display unit 15 together with the determination target image under the control of the display control unit 54. When the determination target image is the determination target image 300 in FIG. 7C, the scene determination unit 51 performs a scene determination process on each of the divided image regions 301 and 302. Specifically, after setting the divided image area 301 as a scene determination area, scene determination of an image in the scene determination area is performed based on the image data of the divided image area 301 (that is, the divided image area 301 is set as a scene determination area). After setting, the above-mentioned judgment scene is determined). Separately, the divided image area 302 is set as a scene determination area, and scene determination of an image in the scene determination area is performed based on the image data of the divided image area 302 (that is, the divided image area 302 is set as a scene determination area). After setting, the above-mentioned judgment scene is determined).

  Assume that the determination scene determined by the scene determination process for the divided image region 301 is a flower scene, and the determination scene determined by the scene determination process for the divided image region 302 is a landscape scene. Then, in step S16, a display for informing the user of the contents of the determination scene for the divided image areas 301 and 302 is performed. For example, as shown in FIG. 8A, the determination target image 300 is displayed on the display unit 15 together with the index 331. In the figure showing the state of the display unit 15 including FIG. 8A, the region filled with dots represents the casing portion of the display unit 15. In the index 331, the wording “flower, landscape” corresponding to the flower scene and the landscape scene is written. Instead of words, “flowers” or “scenery” may be expressed by graphics or the like (the same applies to indices 332 and 333 described later). Alternatively, for example, as illustrated in FIG. 8B, the determination target image 300 may be displayed on the display unit 15 together with the indexes 332 and 333. In the indexes 332 and 333, the wording “flower” and the wording “landscape” corresponding to the flower scene and the landscape scene are written, respectively. The index 332 may be arranged in the divided image area 301 or in the vicinity of the divided image area 301, and the index 333 may be arranged in the divided image area 302 or in the vicinity of the divided image area 302. Furthermore, as shown in FIG. 8B, the boundary line 334 between the divided image regions 301 and 302 may be displayed in a superimposed manner on the determination target image 300 (display of the boundary line 334 is not essential).

  In step S17, the shooting control unit 52 sets a shooting mode for each divided image area based on the result of the scene determination process in step S16. However, the shooting mode set for each divided image region defines only the content of the specific image processing in the image processing unit 53. Therefore, in the following, it is considered that what is set for each divided image region in step S17 is an image processing parameter that defines the content of the specific image processing.

  On the other hand, the scene determination process is performed after setting either the entire image area of the determination target image or the divided image area of the determination target image as the scene determination area. Based on the result of the scene determination process, the shooting control unit 52 Thus, shooting conditions other than the image processing parameters are set. In step S18, one input image is shot as a target input image under the set shooting conditions (shooting conditions other than image processing parameters). Image data of the target image is generated by performing specific image processing according to the image processing parameters on the target input image.

  For example, consider a case where the target input image is the image 340 of FIG. In addition, the specific image processing according to the image processing parameter set for the divided image region 301 is referred to as first specific image processing, and the specific image processing according to the image processing parameter set for the divided image region 302 is referred to as the second specific image. This is called processing. In this case, the image processing unit 53 sets the divided image areas 341 and 342 having the same position and size as the divided image areas 301 and 302 set in the determination target image 300 as the target input image 340. In FIG. 9, the hatched area corresponds to the divided image area 342, and the non-hatched area corresponds to the divided image area 341. Then, the image processing unit 53 performs the first specific image processing on the image data of the divided image area 341 of the target input image 340, and the image data after the first specific image processing is stored in the divided image area 341 of the target image 350. Output as image data. Similarly, the second specific image processing is performed on the image data in the divided image region 342 of the target input image 340, and the image data after the second specific image processing is output as image data in the divided image region 342 of the target image 350. To do. By combining the image data in the divided image area 341 of the target image 350 and the image data in the divided image area 342 of the target image 350, image data of the entire image area of the target image 350 is generated.

  If the judgment scenes are different among the plurality of divided image areas, basically, the image processing parameters set for the plurality of divided image areas are different from each other, and the image processing parameters set for the plurality of divided image areas are different from each other. For example, the content of the specific image processing performed in each divided image region is different between the plurality of divided image regions. In the case of the above example, the first specific image processing corresponding to the flower scene and the second specific image processing corresponding to the landscape scene are different from each other. Specifically, the first specific image processing corresponding to the flower scene can include saturation improvement processing, and the second specific image processing corresponding to the landscape scene can be excluded from saturation processing. The saturation enhancement process is a process for increasing the saturation of an image in an image area that is a target of specific image processing. Further, for example, while including noise reduction processing and color correction processing in both the first and second specific image processing, the noise reduction intensity by the noise reduction processing or the color correction content by the color correction processing is changed to the first and second specific images. It is also possible to vary between processes. In any case, based on the result of the scene determination process for each divided image region, the optimum specific image processing is executed for each divided image region to generate a target image. The target image obtained in step S18 is displayed on the display unit 15, and the image data of the target image is recorded on the recording medium 16.

  In step S15, when the subject range of all the subjects in the determination target image is approximately the same, such as when the shooting range includes only a sufficiently large subject distance, a plurality of divided image regions cannot be obtained, As a result, steps S19 to S21 are executed. In steps S19 to S21, the target image is shot through a general scene determination process. That is, in step S19, a scene determination process is performed after setting the entire image area or a partial area of the determination target image as a scene determination area. In subsequent step S20, based on the result of the scene determination process, the shooting control unit The shooting mode is set by 52. At this time, an image area including the main subject noted in the camera control in step S12 may be set as the scene determination area. All shooting conditions including the contents of the specific image processing are set by setting the shooting mode in step S20. In subsequent step S21, one input image is photographed as a target input image under the photographing conditions set in step S20, and specific image processing is performed on the target input image (specific image processing set in step S20). As a result, image data of the target image is generated. Similar to step S18, the target image obtained in step S21 is also displayed on the display unit 15 and recorded on the recording medium 16.

In the above example, for simple description, the pixel values in the image region 321 of the distance image 320 have stated that all of the P _A, contains the pixel value other than P _A in the image area 321 May be. The same applies to the image area 322. In this case, each pixel value in the image region 321 is the pixel value in the image region 322 with falls within certain numerical range, based on the pixel value P _A is within a certain value range based on the pixel value P _B (The former numerical range and the latter numerical range are different from each other.) At this time, a known region unification method or the like can be used to separate the image region 321 and the image region 322.

  The scene determination information generated by the scene determination unit 51 includes information representing a determination scene for each scene determination region, and the determination region information generated by the scene determination unit 51 includes each scene determination region. The information indicating the position and size of is included. The scene determination information and the determination area information can be recorded on the recording medium 16 together with the image data of the target image, or the subject distance information can be recorded together with the image data of the target image in addition to the scene determination information and the determination area information. The scene determination information, determination area information, and subject distance information generated in the process of acquiring the target image is information regarding the determination target image and information regarding the target image.

Moreover, although the example in the case where the number of area divisions is 2 has been described above, the number of area divisions can be 3 or more. For example, when the distance image 320 includes pixel values PV _A , PV _B, and PV _C , the entire image area of the distance image 320 includes an image area 321 a composed of a set of pixels having the pixel value PV _A , and a pixel an image area 322a comprising a set of pixels having values PV _B, can be divided and an image area 323a comprising a set of pixels having a pixel value PV _C, (not shown for the image area 321a~323a). When such division is performed, the entire image area of the determination target image 300 is divided into three divided image areas, that is, a divided image area 321b corresponding to the image area 321a, a divided image area 322b corresponding to the image area 322a, and The image is divided into divided image areas 323b corresponding to the image areas 323a (the image areas 321b to 323b are not shown).

Even when the number of area divisions is M _A, each of the M _A divided image areas may be set as a scene determination area (M _A is an integer of 3 or more). In this case, for each of M _A number of divided image areas, the scene determination process, result display of the scene determining process, so that the setting shooting mode (picture parameter set), and the specific image processing is performed. However, if the number of divided regions is M _A number, select M _B-number of divided image areas from among M _A number of divided image areas, to set only the M _B-number of divided image areas as a scene determination area Anyway. M _B is an integer of 2 or more, and M _A > M _B. The above selection can be made based on various indices such as the position and size of each divided image area, the subject distance of the subject in each divided image area, and the type of subject in each divided image area.

  As a simple example, for example, when the entire image area of the determination target image 300 is divided into the divided image areas 321b to 323b, the size of the divided image area 323b is sufficiently smaller than the sizes of the image areas 321b and 322b. Only the image area divided image areas 321b and 322b may be set as the scene determination area. Alternatively, for example, the spatial distance between the center position of the divided image areas 321b, 322b, and 323b on the determination target image 300 and the center of the determination target image 300 is obtained, and the spatial distance obtained for the divided image area 323b is the divided image area. When longer than those of 321b and 322b, only the image area divided image areas 321b and 322b may be set as the scene determination area.

  As described above, in the present embodiment, scene determination (scene determination processing) is performed for each divided image region. Since the region division processing for obtaining the divided image region is performed based on the subject distance information, for example, it is possible to separately perform scene determination focusing on the foreground subject and scene determination focusing on the background subject. It becomes. Since the image data of the target image is generated using the result of the scene determination performed individually, for example, the image data of the foreground subject and the background subject can be individually optimized. .

<< Second Embodiment >>
A second embodiment of the present invention will be described. The second and other embodiments to be described later are embodiments based on the first embodiment, and matters not specifically described in the second embodiment and each of the embodiments to be described later are the same as those of the first embodiment unless otherwise contradicted. The description is also applied to the second embodiment and the following embodiments.

  First, the structure of the image file stored in the recording medium 16 will be described. In the imaging apparatus 1, image data of an arbitrary image can be recorded in the recording medium 16 after being stored in an image file. FIG. 10A shows the structure of one image file FL. One image file FL can be created for one still image or moving image. The structure of the image file FL can be made to conform to an arbitrary standard. The image file FL includes a main body area in which image data itself of the recording target image or its compressed data is to be stored, and a header area in which additional data is to be stored. The image data in the main body area and the additional data in the header area in the common image file FL are associated with each other. In the second embodiment, it is assumed that the recording target image is the target image described in the first embodiment.

  As shown in FIG. 10B, the additional data for the recording target image includes subject distance information, scene determination information, and determination area information (see FIG. 3). In addition to this, arbitrary data (shooting of the recording target image) is included. Date, thumbnail, etc.). When the recording target image is the target image 350, subject distance information, scene determination information, and determination region information in the additional data of the recording target image are subject distance information, scene determination information, and determination region generated for the target image 350. The information includes subject distance information, scene determination information, and determination area information generated for the determination target image 300 (see FIGS. 6 and 9).

In the second embodiment, a search operation for a similar image of a reference image by the imaging apparatus 1 will be described. This search operation is realized using the search processing unit 60 included in the main control unit 13 (see FIG. 11). The reference image is referred to by the symbol I _REF . The reference image I _REF is an arbitrary image, but here, for the purpose of concrete description, a case where the target image 350 (see FIG. 9), which is one of the recording target images, is the reference image I _REF is taken as an example. . Each recording target image recorded on the recording medium 16 other than the reference image I _REF functions as a non-reference image. A non-reference image can also be called an image to be searched. As shown in FIG. 12, it is assumed that there are Q non-reference images (Q is an integer equal to or greater than 2), and the first to Q-th non-reference images are referred to by symbols I [1] to I [Q], respectively. The image files FL storing the image data of the non-reference images I [1] to I [Q] are referred to by the symbols FL [1] to FL [Q], respectively. It is assumed that each of the non-reference images I [1] to I [Q] is a target image (which differs from the target image 350) obtained according to the method described in the first embodiment. Accordingly, the subject distance information, scene determination information, and determination area information generated for the non-reference image I [i] are stored in the header area of the image file FL [i] (i is an integer). Also, the image file FL storing the image data of the reference image I _REF referred to by numeral FL _REF.

  FIG. 13 is a flowchart of the search operation of the imaging apparatus 1 according to the second embodiment. FIG. 14A corresponds to the flowchart of FIG. 13 and shows a change over time in the display content of the display unit 15. The same applies to FIG.

First, when a user selects one recording target image from a plurality of recording target images stored in the recording medium 16 by a predetermined touch panel operation or button operation, the selected recording target image is set as the reference image I _REF . In step S31, the reference image I _REF is displayed on the display unit 15. When the reference image I _REF is set, subject distance information of the reference image I _REF is acquired in step S32. By reading the object distance information of the reference image I _REF from the image file FL _REF, it is possible to get it. Area division processing for the reference image I _REF based on the object distance information of the reference image I _REF is performed in further step S33. The region dividing process for the reference image I _REF based on the subject distance information is the same as the region dividing process for the determination target image based on the subject distance information described in the first embodiment. In step S33, it is possible to actually perform the region division processing based on the subject distance information. However, since the determination region information of the image file FL _REF includes the result of region division processing for the reference image I _REF , if the determination region information of the image file FL _REF is read, the region division processing is actually performed Similar results can be obtained.

By the processing in steps S32 and S33, the entire image area of the reference image I _REF is divided into divided image areas 401 and 402 having the same position and size as the divided image areas 301 and 302 in FIG. (See FIG. 15). In FIG. 15, the hatched area corresponds to the divided image area 402, and the non-hatched area corresponds to the divided image area 401. In the divided image area 401, image data of a flower 311 exists, and in the divided image area 402, image data of a mountain 312 and a meadow 313 exist (see also FIG. 7A).

  Thereafter, in step S34, the imaging apparatus 1 accepts a search specifying operation by the user. In subsequent step S35, the search processing unit 60 sets a search reference area based on the content of the search designation operation. Although the search designation operation may be realized by a button operation, it is assumed here that the search designation operation is realized by a touch panel operation.

The search specification operation by the user is, for example, a touch panel operation in which a specific point on the display screen displaying the reference image I _REF is touched with a finger. In this case, the search processing unit 60 sets a divided image area including a specific point as a search reference area from among a plurality of divided image areas set in the reference image I _REF . Therefore, for example, when the specified specific point is a point in the divided image area 401 in the reference image I _REF (see also FIG. 15) as in the example corresponding to FIG. When the region 401 is set as the search reference region and the specified specific point is a point in the divided image region 402 in the reference image I _{REF as} in the example corresponding to FIG. 14B (see also FIG. 15). In step S35, the divided image area 402 is set as a search reference area.
Alternatively, for example, a touch panel operation surrounding a region to be set as a search reference region with a finger while maintaining a contact state between the display screen and the finger may be a search specifying operation. In this case, for example, when all or most of the area surrounded by the touch panel operation is included in the divided image area 401, or when the area surrounded by the touch panel operation includes all or most of the divided image area 401, the divided image area 401 may be set as the search reference area.
As described above, the search specifying operation functions as a selection instruction for selecting a search reference area from among a plurality of divided image areas set in the reference image I _REF .

Further, during a period in which accepts search for the specified operation, as shown in FIG. 16, while displaying a reference image I _REF, for each divided image areas of the reference image I _REF to the image feature of the divided image areas of the reference image I _REF A corresponding index may be displayed. As a result, the search specifying operation by the user is supported, and it becomes easy to specify a desired search condition. Since it is assumed that the reference image I _REF is the target image 350, the determination scenes corresponding to the divided image regions 401 and 402 are “flower scene” and “landscape scene”, respectively. Therefore, in the example of FIG. 16, the reference image I _REF is displayed on the display unit 15 together with the indexes 332 and 333 during the period when the search designation operation is accepted, as in FIG. The index 332 is an index corresponding to the image feature of the divided image area 401 of the reference image I _REF , and the index 333 is an index corresponding to the image feature of the divided image area 402 of the reference image I _REF . The index 332 may be arranged in the divided image area 401 or in the vicinity of the divided image area 401, and the index 333 may be arranged in the divided image area 402 or in the vicinity of the divided image area 402 (see also FIG. 15). Furthermore, the boundary line 334 between the divided image areas 401 and 402 may be displayed in a superimposed manner on the reference image I _REF . Formation and display of the indicators 332 and 333 and the boundary line 334 can be performed using the scene determination information and the determination area information in the image file FL _REF .

  After setting the search reference area, in step S36, the search processing unit 60 searches the non-reference images I [1] to I [Q] for non-reference images that meet the search conditions. The non-reference image that matches the search condition is a non-reference image having an image feature similar to the image feature in the search reference region. After setting the search reference region, each image feature of the non-reference images I [1] to I [Q] is analyzed, and a non-reference having an image feature similar to the image feature in the search reference region based on the analysis result. It is also possible to search for images. However, when the number of non-reference images (that is, the value of Q) is large, if such an analysis is performed after setting the search reference area, the time until the search result is obtained becomes considerably long. Therefore, the search in step S36 may be performed based on the additional data of the image files FL [1] to FL [Q].

  For example, when the divided image area 401 corresponding to “flower scene” is set as the search reference area, “flower scene” is selected from Q scene determination information for the image files FL [1] to FL [Q]. It is preferable to extract the scene determination information including the determination scene “” and determine that the non-reference image corresponding to the extracted scene determination information is a non-reference image that meets the search condition. In this case, if the determination scene of “flower scene” is included only in the scene determination information for the image files FL [1] and FL [2] among the image files FL [1] to FL [Q]. Only the non-reference images I [1] and I [2] are detected as non-reference images that meet the search condition. The same applies when the divided image area 402 is set as the search reference area. That is, when the divided image area 402 corresponding to “landscape scene” is set as the search reference area, “landscape scene” is selected from Q scene determination information for the image files FL [1] to FL [Q]. It is preferable to extract the scene determination information including the determination scene “” and determine that the non-reference image corresponding to the extracted scene determination information is a non-reference image that meets the search condition.

  Note that the content of the search condition may be notified to the user during the search period of the non-reference image that matches the search condition. The notification method is arbitrary, and any method that appeals to the human senses (sight, hearing, etc.) can be adopted. For example, when the divided image area 401 corresponding to “flower scene” is set as the search reference area, the display unit 15 displays that the image obtained by photographing the flower is searched, and corresponds to “landscape scene”. When the divided image area 402 is set as a search reference area, it is preferable to display on the display unit 15 that an image obtained by capturing a landscape is being searched. In the above example, it is assumed that the scene determination information is stored in advance in the image file FL [i]. However, when the scene determination information is not stored in the image file FL [i], In step S36, scene determination is performed for each scene determination area of the non-reference image I [i] based on the image data of the non-reference image I [i], thereby obtaining scene determination information for the non-reference image I [i]. You may make it produce | generate.

  When the search in step S36 ends, in step S37, the search processing unit 60 displays the search result in step S36 on the display unit 15 (see FIG. 14A or FIG. 14B). As long as the user can recognize the information of the non-reference image that matches the search condition, this display method is arbitrary. For example, the display unit 15 may display the non-reference image itself that matches the search condition, the thumbnail of the non-reference image that matches the search condition, or the file name of the non-reference image that matches the search condition. If there are multiple non-reference images that meet the search criteria, and if you cannot display all of the non-reference images that meet the search criteria or all of the non-reference image thumbnails that meet the search criteria at the same time, like a slideshow These may be displayed in a plurality of times. Note that if there is no non-reference image that matches the search condition, it is sufficient to notify the user to that effect.

  When it is desired to perform an image search focusing on the foreground subject, the user only needs to perform a search specifying operation so that the foreground subject corresponds to the search reference area, and conversely, an image focusing on the background subject. When it is desired to perform a search, the user may perform a search specifying operation so that the subject on the background side corresponds to the search reference area. As described above, according to the second embodiment, it is possible to execute an image search in which the foreground and the background are distinguished.

Hereinafter, several application search methods applicable to the second embodiment will be described. It is also possible to implement a combination of a plurality of application search methods shown below.
[First applied search method]
The first applied search method will be described. In the above description, it is determined whether or not each non-reference image meets the search condition based on the image feature. Furthermore, the subject distance information of the reference image I _REF and each non-reference image I [i This determination may be made using the subject distance information. For the sake of concrete explanation, this method will be described in detail assuming that the divided image area 401 in FIG. 15 is set as the search reference area.

  As described above, the image data of the flower 311 exists in the divided image area 401 as the search reference area, and as a result, the determination scene corresponding to the divided image area 401 is “flower scene”. In this case, the search processing unit 60 extracts scene determination information including a determination scene of “flower scene” from a total of Q scene determination information for the image files FL [1] to FL [Q]. The non-reference image corresponding to the scene determination information extracted here is handled as a non-reference image candidate that matches the search condition. Now, it is assumed that only the non-reference images I [1] and I [2] are handled as non-reference image candidates that meet the search condition.

As described above, in the reference image I _REF , the flower 311 that is the subject in the search reference region is the foreground subject. Now, the subject in the scene determination area corresponding to the determination scene of “flower scene” in the non-reference image I [1] is the foreground subject, and “flower scene” in the non-reference image I [2]. Assume that the subject in the scene determination area corresponding to the determination scene “” is the subject on the background side. In this case, among the non-reference image candidates that match the search condition, only the non-reference image having the subject (that is, the flower) for the “flower scene” as the foreground side subject, that is, only the non-reference image I [1], A non-reference image that matches the search condition is detected. According to this method, it is possible to search for a non-reference image in which “flower” is located on the foreground side or a non-reference image in which “flower” is located on the background side. It becomes.

Can any subject in the reference image I _REF of the object distance reference image from the information I _REF to determine whether it is the object of the foreground side and the background side, the object distance information of non-reference image I [i] It can be determined whether an arbitrary subject on the non-reference image I [i] is a foreground side subject or a background side subject. For example, when two types of subjects are included in an arbitrary image of interest, a subject with a shorter subject distance can be regarded as a subject on the foreground side, and a subject with a longer subject distance is placed on the background side. It can be seen as a subject. For example,
In any target image, an average of the subject distances of all subjects in the target image can be obtained, and a subject having a subject distance shorter than the average can be regarded as a foreground subject, and a subject having a subject distance longer than the average Can be regarded as a subject on the background side.

[Second application search method]
The second applied search method will be described. In the above description, only one search reference area is set for the reference image I _REF , but the user may set a plurality of search reference areas for the reference image I _REF by a search specifying operation. it can. For example, when the entire image area of the reference image I _REF is divided into first to fifth divided image areas and a determination scene is obtained for each divided image area, the first operation for specifying the search is performed by the user. Consider the case where the second divided image region is set as the search reference region.

In this case, the search processing unit 60 determines whether the scene determination information of the non-reference image I [i] satisfies the following first and second conditions.
The first condition is a condition that the scene determination information of the non-reference image I [i] includes the same determination scene as the determination scene for the first divided image area of the reference image _IREF .
The second condition is a condition that the scene determination information of the non-reference image I [i] includes the same determination scene as the determination scene for the second divided image area of the reference image I _REF .

  The search processing unit 60 then matches the non-reference image I [i] with the search condition only when the scene determination information of the non-reference image I [i] satisfies both the first and second conditions. Judge that Alternatively, only when the scene determination information of the non-reference image I [i] satisfies at least one of the first and second conditions, it is determined that the non-reference image I [i] meets the search condition. You may do it.

[Third applied search method]
A third applied search method will be described. After detecting the non-reference image that matches the search condition based on the scene determination information or based on the scene determination information and the subject distance information, the non-reference image that matches the search condition is further narrowed down using image analysis or the like. May be. For example, when the divided image region 401 corresponding to “flower scene” is set as the search reference region, the determination scene of “flower scene” is included in the scene determination information for the image files FL [1] and FL [2]. When it is, the non-reference images I [1] and I [2] are once detected as non-reference image candidates that meet the search conditions. Thereafter, based on the image data in the divided image region 401 of the reference image I _REF and the image data of the non-reference images I [1] and I [2], the image in the divided image region 401 of the reference image I _REF and the non-reference image The similarity between the image corresponding to the “flower scene” in I [1] and the image in the divided image area 401 of the reference image I _REF and the “flower scene in the non-reference image I [2] The non-reference image I [1] is a non-reference image that meets the search condition only when the similarity between the image and the portion corresponding to "is obtained and the former similarity is higher than the predetermined reference similarity. Similarly, it may be determined that the non-reference image I [2] is a non-reference image that meets the search condition only when the similarity of the latter is higher than a predetermined reference similarity. You may make it do. Thereby, for example, when the flower in the divided image area 401 of the reference image I _REF is “sunflower”, it is determined that the non-reference image obtained by photographing the “sunflower” flower satisfies the search condition, and the “hydrangea” flower is selected. It is determined that the captured non-reference image does not meet the search condition.

<< Third Embodiment >>
A third embodiment of the present invention will be described. The third embodiment is an embodiment based on the first and second embodiments. For matters not specifically described in the third embodiment, the description of the first and second embodiments is the third embodiment as long as there is no contradiction. Also applies to form. In the third embodiment, as in the second embodiment, an image search operation that matches the search condition will be described. However, in the search method according to the third embodiment, the reference image I _REF is not necessary. A search operation according to the third embodiment will be described with reference to FIG. FIG. 17 is a flowchart showing the procedure of this search operation.

  First, in step S51, the imaging apparatus 1 accepts input of search conditions by the user. The user can input a search condition using a touch panel operation or a button operation. In subsequent step S52, the search processing unit 60 or an area division unit (not shown) reads out subject distance information of each non-reference image from the recording medium 16, and performs area division processing on each non-reference image based on the subject distance information of each non-reference image. Execute. Thereby, a plurality of divided image areas are set in the non-reference image I [i] as the search target image. However, the position and size of each divided image area of the non-reference image I [i] to be set by the area division processing for the non-reference image I [i] is basically the image of the non-reference image I [i]. It is defined by the determination area information of the file FL [i]. Therefore, by reading the determination area information from the image file FL [i], the area division process for the non-reference image I [i] can be completed.

  After that, in step S53, the search processing unit 60 evaluates the suitability of the search condition for each non-reference image and for each divided image region, thereby determining whether each non-reference image meets the search condition. to decide. This determination can be made using the scene determination information of each non-reference image. Finally, in step S54, the search processing unit 60 causes the display unit 15 to display the search result of step S53. This display method is the same as that in step S37 of the second embodiment. As long as the user can recognize the information of the non-reference image that matches the search condition, this display method is arbitrary.

  A specific example is given. For example, when the user is searching for “a non-reference image in which the shooting scene is a flower scene”, for example, “a non-reference image including a flower as a subject” or “a non-reference image taken by focusing on flowers”, the user A word indicating that is directly input to the imaging apparatus 1 or an item indicating that is selected from a plurality of items provided in advance. Thereby, a search condition corresponding to the input wording or the selection item is set. After setting the search conditions, it is also possible to analyze the image features of each of the non-reference images I [1] to I [Q] and search for a non-reference image that meets the search conditions based on the analysis result. However, when the number of non-reference images (that is, the value of Q) is large, if such an analysis is performed after setting the search conditions, the time until the search result is obtained becomes considerably long.

  Therefore, when the additional data in the recording medium 16 is read and the scene determination information of the non-reference image I [1] includes “flower scene” (that is, any divided image of the non-reference image I [1]). When the determination scene of the region is “flower scene”), it is determined that the non-reference image I [1] satisfies the search condition, and the scene determination information of the non-reference image I [1] includes “flower scene”. If not, it may be determined that the non-reference image I [1] does not meet the search condition. The same applies to the non-reference images I [2] to I [Q].

  Note that, as in the first applied search method described above, a condition related to the subject distance may be added to the search condition. In this case, the suitability / nonconformity of the search condition for each non-reference image I [i] is determined using the subject distance information of each non-reference image I [i]. For example, the user can specify “the shooting scene is a flower scene and the flower is a foreground subject” as a search condition. In this case, even if the scene determination information of the non-reference image I [1] includes “flower scene”, the image in the divided image region corresponding to the “flower scene” of the non-reference image I [1]. When the flower is the subject on the background side, it is determined that the non-reference image I [1] does not meet the search condition, and the image is within the divided image region corresponding to the “flower scene” of the non-reference image I [1]. Only when the flower is a foreground subject, it is determined that the non-reference image I [1] meets the search condition. The same applies to the non-reference images I [2] to I [Q]. According to this method, it is possible to search for a non-reference image in which “flower” is located on the foreground side or a non-reference image in which “flower” is located on the background side. It becomes.

The user can specify various types of search conditions.
For example, “include flowers and animals as subjects” can be specified as a search condition. When this specification is made, the scene determination information of the non-reference image I [1] is “flower scene” and “animal scene”. Only when both of "" are included, it is determined that the non-reference image I [1] meets the search condition. In other cases, the non-reference image I [1] does not meet the search condition. It is good to judge. The same applies to the non-reference images I [2] to I [Q].
Further, for example, “include animals as foreground subjects and sea as background subjects” can be specified as a search condition. When this designation is made, the scene determination information of the non-reference image I [1] includes both the “animal scene” and the “sea scene”, and the “animal scene” of the non-reference image I [1] Only when the animal in the divided image region corresponding to “” is the foreground subject and the sea in the divided image region corresponding to the “sea scene” of the non-reference image I [1] is the background subject. It may be determined that the non-reference image I [1] matches the search condition, and otherwise, it is determined that the non-reference image I [1] does not match the search condition. The same applies to the non-reference images I [2] to I [Q].

  As can be understood from the above description, according to the third embodiment, for example, it is possible to search for an image in which the foreground subject matches the search condition, an image in which the background subject matches the search condition, and the like. Thus, depending on the search condition, it is possible to distinguish and detect these images.

<< Deformation, etc. >>
The embodiments of the present invention can be appropriately modified in various ways within the scope of the technical idea shown in the claims. The above embodiment is merely an embodiment of the present invention, and the meaning of the term of the present invention or each constituent element is not limited to that described in the above embodiment. The specific numerical values shown in the above description are merely examples, and as a matter of course, they can be changed to various numerical values. As annotations applicable to the above-described embodiment, annotation 1 and annotation 2 are described below. The contents described in each comment can be arbitrarily combined as long as there is no contradiction.

[Note 1]
The search operation according to the second or third embodiment can also be executed in an electronic device (not shown) different from the imaging device 1. The electronic device is, for example, an image reproduction device such as a personal computer or an information terminal device such as a PDA (Personal Digital Assistant). An imaging device is also a type of electronic device. When the search operation according to the second or third embodiment is executed in an electronic device different from the imaging device 1, a part (search processing unit 60) provided in the imaging device 1 and necessary for the operation of the second or third embodiment. , The display unit 15, the region dividing unit, and the like) may be provided in the armature.

[Note 2]
The imaging apparatus 1 in FIG. 1 can be configured by hardware or a combination of hardware and software. When the imaging apparatus 1 is configured using software, a block diagram of a part realized by software represents a functional block diagram of the part. A function realized using software may be described as a program, and the function may be realized by executing the program on a program execution device (for example, a computer).

DESCRIPTION OF SYMBOLS 1 Imaging device 11 Imaging part 15 Display part 19 Touch panel 51 Scene determination part 52 Shooting control part 53 Image processing part 54 Display control part 55 Subject distance information generation part

Claims (6)

An image sensor that outputs image data of a subject;
An area dividing unit that divides the entire image area of the determination target image based on the output of the imaging element into a plurality of divided image areas;
A scene determination unit configured to determine two or more divided image areas among the plurality of divided image areas as determination areas, and for each determination area, determine a scene of an image in the determination area based on image data in the determination area; An imaging apparatus comprising: The imaging apparatus according to claim 1, further comprising a display unit that displays a result of scene determination for each determination region. Image processing for generating image data of the target image under a condition corresponding to a result of scene determination for each determination region using an output of the image sensor when a predetermined shooting instruction for instructing acquisition of the target image is given The imaging apparatus according to claim 1, further comprising a unit. For each determination area, shooting parameters are set according to the scene determination result for the determination area,
The image processing unit generates image data of the entire target image by generating image data of the image area in the target image corresponding to the determination area using the corresponding shooting parameter for each determination area. The imaging apparatus according to claim 3, wherein: An area dividing unit that divides the entire image area of the reference image that is the image of the subject into a plurality of divided image areas;
A search reference area is selected from the plurality of divided image areas in accordance with a given selection instruction, and a non-reference image having an image feature corresponding to the image feature of the search reference area is searched from the plurality of non-reference images. An electronic apparatus comprising: a search processing unit. The search is a process of selecting a non-reference image that matches a search condition from the plurality of non-reference images,
The search processing unit also uses subject distance information of the reference image and subject distance information of each non-reference image to determine whether each non-reference image meets the search condition. The electronic device according to claim 5.

Images