JP2013229696A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of easily associating, with a captured image, objects not included within the composition though the objects exist in the periphery of the imaging place.SOLUTION: The imaging apparatus comprises an imaging device, a recognition part, a photographing controller, and a recording processing part. The imaging device captures an image of a subject. The recognition part analyzes a first image obtained in time series by the imaging device, and recognizes objects included in the first image. The photographing controller photographs a first image and photographs a second image for recording. The recording processing part associates the second image with information on the objects not included in at least the second image recognized by the recognition part and records them.

Description

  The present invention relates to an imaging apparatus.

  2. Description of the Related Art Conventionally, an imaging apparatus that identifies a person's face from a through image for composition confirmation is known (for example, see Patent Document 1).

JP 2010-87898 A

  By the way, for example, it is assumed that a group of five friends travels and a photographer in the group photographs two persons in the group. In this case, the other two persons (excluding the photographer) are not photographed in the photographed image even though they exist around the photographing place. Therefore, when the photographer confirms the captured image at a later date (for example, several years later), the following problem may occur. That is, the photographer may not be able to remember the entire group who went to the trip.

  However, since the imaging apparatus of Patent Document 1 is premised on identifying a person to be photographed, the above problem cannot be solved.

  Therefore, in view of the above circumstances, the present invention provides an imaging apparatus capable of easily associating information on a target (object) that does not fit in the composition even though it exists in the vicinity of the shooting location with the shot image. The purpose is to do.

  An imaging apparatus according to a first invention includes an imaging element, a recognition unit, a shooting control unit, and a recording processing unit. The imaging element captures an image of a subject. The recognizing unit analyzes the first image obtained in time series by the imaging element, and recognizes an object included in the first image. The shooting control unit performs shooting of the first image and shooting of the second image for recording. The recording processing unit records the second image and information of an object not included in at least the second image recognized by the recognition unit in association with each other.

  According to a second aspect, in the first aspect, the display device further includes a display unit for displaying an image and a display control unit. The display control unit controls the display mode of the display unit. The display control unit displays the first image and the object information on the display unit. The recording processing unit records the object information displayed on the display unit together with the first image in association with the second image.

  According to a third invention, in the second invention, a selection input unit is further provided. The selection input unit selects an object by user input for the first image displayed on the display unit. The recognition unit recognizes the object selected by the selection input unit by user input.

  In a fourth aspect based on the third aspect, the selection input unit is a touch panel for inputting information by contact of a contact body. A recognition part recognizes the object which the contact body instruct | indicated through the touch panel with respect to the object in the 1st image currently displayed on the display part.

  In a fifth aspect based on any one of the first to fourth aspects, the recording processing unit synthesizes object information in a partial area of the second image, or adds object information to tag information of the second image. By adding, the second image and the object information are recorded in association with each other.

  According to a sixth invention, in any one of the first to fifth inventions, the database further includes a database and a first verification unit. In the database, information on objects for collation is registered in advance. The first collating unit collates the partial image included in the first image with the information of the collating object registered in the database. The recognition unit recognizes the object according to the collation result of the first collation unit.

  According to a seventh invention, in any one of the first to sixth inventions, the communication device and the second verification unit are further provided. The communication unit communicates with an external server having a database in which information on objects for verification is registered in advance. The second collation unit collates the partial image included in the first image with the information on the collation object registered in the database via the communication unit. The recognition unit recognizes the object according to the collation result of the second collation unit.

  In an eighth invention according to the sixth or seventh invention, the information of the object registered in the database includes information related to the object. The recording processing unit records the second image and information related to the object in association with each other according to the collation result of the first collation unit or the second collation unit.

  According to a ninth invention, in any one of the sixth to eighth inventions, a management unit is further provided. The management unit manages object information registered by the recording processing unit for each predetermined condition.

  In a tenth aspect based on the ninth aspect, the management unit manages information of the plurality of objects as a group.

  In an eleventh aspect based on the tenth aspect, the recording processing unit registers the object information recognized by the recognition unit as a group in the database for each photographing, every time, every date, or every photographing event.

  In a twelfth aspect based on the tenth or eleventh aspect, the management unit reads a specific group designated by a user input or a specific group including an object currently recognized by the recognition unit. The display control unit causes the display unit to display information on objects of a specific group.

  The imaging apparatus according to the present invention can easily associate information of a target (object) that does not fit in the composition with a captured image even though it exists in the vicinity of the shooting location.

The block diagram which shows the structural example of the electronic camera 1 Rear view of the housing of the electronic camera 1 A flowchart showing an example of the operation of the electronic camera 1 The figure explaining an example of operation | movement of the electronic camera 1 The figure which shows an example of the photographed image The figure which shows an example of the image file of Exif format The flowchart which shows an example of operation | movement of the electronic camera 1 in 2nd Embodiment. The figure explaining an example of operation | movement of the electronic camera 1 in 2nd Embodiment. The figure explaining an example of operation | movement of the electronic camera 1 in 3rd Embodiment. The figure which shows an example of the picked-up image in 3rd Embodiment. The figure explaining an example of operation | movement of the electronic camera 1 in 4th Embodiment. The figure explaining the structural example of the collation system of 5th Embodiment. Conceptual diagram showing an example of a group registered in the database 15a The figure which shows an example of the group displayed on the display monitor 17 The figure which illustrated other display modes of a photography picture The figure explaining the reproduction process of the picked-up image in an azimuth detection mode

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the first embodiment, an electronic camera 1 according to an embodiment of the present invention will be described. In the second to sixth embodiments, since the configuration of the electronic camera 1 is common, the description of the configuration of the electronic camera 1 is omitted in the second to sixth embodiments.

  FIG. 1 is a block diagram illustrating a configuration example of the electronic camera 1. Here, the electronic camera 1 is a photographing mode (hereinafter referred to as “association mode”) for easily associating information on a target (object) that does not fit in the composition even though it exists in the vicinity of the photographing location with the photographed image. I have). Note that the object information in the present embodiment is information relating to a subject that does not fit in the composition, and may include, for example, a subject image (such as a human face image), a subject name, and the like. In the following description, for the sake of simplicity, a human face image or the like may be simply referred to as an object.

  The electronic camera 1 includes a photographing optical system 10, an image sensor 11, a signal processing unit 12, a RAM (Random Access Memory) 13, an image processing unit 14, a flash memory 15, and a recording interface unit (hereinafter referred to as "recording I"). / F unit ") 16, display monitor 17, display interface unit (hereinafter referred to as" display I / F unit ") 18, touch panel 19, operation unit 20, release button 21, and communication interface unit. (Hereinafter referred to as “communication I / F unit”) 22, antenna 22 a, direction sensor 23, CPU (Central Processing Unit) 24, and bus 25.

  Among these, the signal processing unit 12, the RAM 13, the image processing unit 14, the flash memory 15, the recording I / F unit 16, the display I / F unit 18, and the CPU 24 are connected to each other via a bus 25.

  The photographic optical system 10 includes a plurality of lens groups including a zoom lens that adjusts the focal length and a focus lens that adjusts the imaging position on the imaging surface of the imaging element 11. A lens driving unit (not shown) adjusts the lens positions of the zoom lens and the focus lens in the photographic optical system 10 in the direction of the optical axis in accordance with an instruction from the CPU 24. For the sake of simplicity, FIG. 1 shows the photographing optical system 10 as a single lens.

  The image sensor 11 captures an image of a subject and generates an image (analog image signal). The analog image signal output from the image sensor 11 is input to the signal processing unit 12. Note that three types of color filters of R (red), G (green), and B (blue) are arranged on the imaging surface of the imaging device 11 in, for example, a Bayer array. Further, the charge accumulation time of the image sensor 11 and the reading of the image signal are controlled by a timing generator (not shown). The image sensor 11 is a CCD (Charge Coupled Device) type or CMOS (Complementary Metal-Oxide Semiconductor) type color image sensor.

  The signal processing unit 12 converts an analog image signal output from the image sensor 11 into a digital signal (image data) and outputs the digital signal to the bus 25.

  The RAM 13 is a volatile memory (buffer memory). The RAM 13 temporarily records image data via the bus 25 under the control of the CPU 24. The image processing unit 14 reads the image data recorded in the RAM 13 and performs various image processing (gradation conversion processing, contour enhancement processing, white balance processing, etc.).

  The flash memory 15 is a rewritable nonvolatile semiconductor memory. A program for controlling the electronic camera 1 is recorded in the flash memory 15 in advance. In accordance with this program, the CPU 24 executes processing of flowcharts shown in FIGS. 3 and 7 (to be referred to as “flow processing” hereinafter) as an example.

  Further, the flash memory 15 has a database 15a in which information on objects for verification is registered in advance. The database 15a will be described in a fourth embodiment.

  The recording I / F unit 16 is formed with a connector (not shown) for connecting a detachable recording medium 30. Then, the recording I / F unit 16 accesses the recording medium 30 connected to the connector and performs recording processing of a recorded image for recording. The recording medium 30 is, for example, a card type nonvolatile memory card. FIG. 1 shows the recording medium 30 after being connected to the connector.

  The display monitor 17 is composed of, for example, a liquid crystal display medium. The display monitor 17 displays a through image for composition confirmation obtained by the image sensor 11 in time series, a captured image, an operation menu of the electronic camera 1, and the like.

  The display I / F unit 18 provides an interface for displaying an image or the like on the display monitor 17. Specifically, the display I / F unit 18 displays an operation menu of the electronic camera 1 and a captured image on the display monitor 17 in response to an instruction from a display control unit 24b described later.

  In addition, the display I / F unit 18 displays the through image on the display monitor 17 for live view display during shooting standby. Specifically, the CPU 24 sets the resolution of the image output by the image sensor 11 to be lower than that of the captured image recorded on the recording medium 30 during standby for shooting. As a result, the image sensor 11 outputs a through image at 30 frames per second (30 fps), for example, according to an instruction from the timing generator. Then, the display I / F unit 18 causes the display monitor 17 to display the through image as a through image for live view display according to an instruction from the display control unit 24b.

  The touch panel 19 inputs information by contact of a contact body. Specifically, the touch panel 19 is a pressure-sensitive (resistive film) type touch panel, for example, and detects the position of a contact body (for example, a fingertip) on the surface of the touch panel provided on the display monitor 17. Then, the touch panel 19 transmits a signal (position information) indicating the detected position to the CPU 24. Thereby, the touch panel 19 can select object information by a user input with respect to the through image displayed on the display monitor 17.

  The touch panel 19 is not limited to a pressure-sensitive (resistive film) type, and may be a touch panel of another type (for example, a capacitance type). Further, the touch panel 19 may detect the position of, for example, a pressure-sensitive pen-type input device (such as a pointing device) as a contact body.

  The operation unit 20 has a plurality of buttons for receiving a photographer's operation, and receives an instruction input for operating the electronic camera 1. The release button 21 receives a half-press operation (start of shooting preparation operations such as autofocus (AF) and automatic exposure (AE) before shooting) and a shooting instruction input for full-press operation (start of shooting operation).

  FIG. 2 is a rear view of the housing of the electronic camera 1. On the back of the housing of the electronic camera 1, as an example, a display monitor 17, an operation unit 20, a wide-angle zoom button 20a, a telephoto zoom button 20b, a mode button 20c, a menu button 20d, a cross button 20e, Is provided. A release button 21 and a power button 20 f as the operation unit 20 are provided on the upper surface of the casing of the electronic camera 1.

  The wide-angle zoom button 20a changes the focal length to the wide-angle side, and the telephoto zoom button 20b changes the focal length to the telephoto side. The mode button 20c receives an operation for switching to the shooting mode or the playback mode. The menu button 20d accepts an operation for starting up the menu screen from the user. The cross button 20e receives selection or determination of setting conditions and the like in the operation menu of the electronic camera 1 from the user.

  Here, the cross button 20e is an operation button for selecting or determining a setting condition of an operation menu of the electronic camera 1. The cross button 20e receives, for example, selection input of items such as setting conditions from the user by operating buttons in four directions, up, down, left and right. The cross button 20e receives an input for determining the selected setting condition from the user when a circular operation button arranged at the center is pressed. In addition, the cross button 20e selects object information by a user input for the through image displayed on the display monitor 17. The power button 20f accepts turning on or off of the power source (not shown) of the electronic camera 1.

  Returning to the description of FIG. 1 again, the communication I / F unit 22 provides an interface for wireless communication. Specifically, the communication I / F unit 22 is connected to an electric communication line (for example, the Internet network) by wireless communication via the antenna 22a, and communicates with an external server 50 described later with reference to FIG. The direction sensor 23 is an electronic compass, for example, and detects geomagnetism and outputs it to the CPU 24.

  The CPU 24 is equipped with an application specific integrated circuit (ASIC) having a high-speed image processing function, and performs various operations as a microprocessor and overall control of the electronic camera 1. The CPU 24 controls each part of the electronic camera 1 by executing the above program.

  Further, the CPU 24 recognizes the recognition unit 24a, the display control unit 24b, the imaging control unit 24c, the recording processing unit 24d, the first verification unit 24e, the second verification unit 24f, the replacement unit 24g, and the management unit 24h. Also works.

  The recognition unit 24a analyzes the through image and recognizes an object (for example, a face image) included in the through image. Specifically, the recognizing unit 24a reads through-image image data from the RAM 13, and extracts a human face area (face image) as a subject from the image data. In this case, the extraction of the human face area can be performed by a known face extraction process.

  More specifically, the recognition unit 24a extracts a face area from the image data of the through image based on a luminance signal such as a skin color, and recognizes the face image as an object. Then, the face image data is sequentially temporarily stored in the RAM 13. Here, the recognition unit 24a may cut out and recognize a rectangular area of the face frame including the face as a face image. The recognition unit 24a may be configured to issue an instruction to the image processing unit 14 and cause the image processing unit 14 to recognize an object.

  Alternatively, the recognition unit 24a may recognize the contour shape of an object such as a person as an object by using a known graph cut method. As an example, the graph cut method is based on a mathematical measure of “probability” for determining whether adjacent pixel values can be regarded as pixel values in the same region and the background of the target region (for example, the contour shape of a person). In this method, the target area is determined by dividing the area.

  Alternatively, the recognizing unit 24a may recognize the object by user selection via the touch panel 19 or the cross button 20e.

  The display control unit 24 b controls the display mode of the display monitor 17. Specifically, the display control unit 24 b causes the display monitor 17 to display an object superimposed on a through image, for example, via the display I / F unit 18. Here, as an example, when the photographer changes the orientation of the electronic camera 1 and takes a through image, the recognition unit 24 a can recognize the object according to the orientation of the electronic camera 1. Alternatively, as an example, when the photographer changes the focal length to the wide-angle side or the telephoto side and takes a through image, the recognition unit 24a can recognize the object according to the focal length.

  Then, if the display control unit 24b displays, for example, an arbitrary object that is not currently displayed in the through image on the display monitor 17, the photographer can intuitively recognize an object that does not fit in the composition. The display control unit 24b causes the display monitor 17 to sequentially display the objects recognized by the recognition unit 24a in default settings (values set in advance when the user does not specify otherwise).

  The shooting control unit 24c performs shooting of a through image and shooting of a shot image. Specifically, when shooting the through image, the shooting control unit 24c causes the imaging device 11 to output the through image at, for example, 30 fps via a timing generator (not shown). When the release button 21 receives an instruction input for shooting (half-pressing operation, full-pressing operation) at a predetermined timing, the shooting control unit 24c takes a shot image.

  The recording processing unit 24d records the captured image and the information of the object that is not included in the captured image recognized by the recognition unit 24a in association with each other. Details of the processing of the recording processing unit 24d will be described later with reference to FIGS. The first collation unit 24e collates the partial image included in the captured image with the information on the collation object registered in the database 15a. The recognition unit 24a recognizes the object according to the collation result of the first collation unit 24e.

  Further, the second collation unit 24f collates the partial image included in the captured image with the information on the collation object registered in the database 50a via the communication I / F unit 22. The recognizing unit 24a recognizes the object according to the collation result of the second collating unit 24f.

  The replacement unit 24g replaces the object with a name corresponding to the object (for example, a character string of text data) according to the collation result of the first collation unit 24e or the second collation unit 24f.

  The management unit 24h manages object information. For example, the management unit 24h manages information of at least one object registered by the recording processing unit 24d. Details of the first verification unit 24e and the replacement unit 24g will be described in the fourth embodiment. Details of the second verification unit 24f will be described in the fifth embodiment and the like. Details of the management unit 24h will be described in the third and sixth embodiments.

  Next, an example of operation in the association mode of the electronic camera 1 will be described.

  FIG. 3 is a flowchart showing an example of the operation of the electronic camera 1. FIG. 4 is a diagram for explaining an example of the operation of the electronic camera 1. 4 (1) to 4 (4) schematically show the photographing procedure. 4 (5) to (8) show through images displayed on the display monitor 17 in accordance with the orientation of the electronic camera 1 in the photographing area surrounded by the dotted frame 31 in FIGS. 4 (1) to (4). They are drawn in association with each other. That is, FIGS. 4 (1) and (5), FIGS. 4 (2) and (6), FIGS. 4 (3) and (7), and FIGS. 4 (4) and (8) are in pairs.

  In order to make the explanation easy to understand, in the present embodiment, a scene in which a trip with five friends (A, B, C, D, E) is taken is assumed. In FIG. 4, E is a photographer of the electronic camera 1. Here, it is assumed that the photographer E is thinking about photographing friends B and C as main subjects. The photographer E also wants to keep information on the friends A and D who are in a position that does not fit in the composition of the photographed image.

  In the following operation example, after the power is turned on, when the CPU 24 receives a selection input of the association mode on the menu screen, the shooting control unit 24c of the CPU 24 starts shooting a through image. And CPU24 starts the process of the flow shown in FIG.

  Step S101: The display control unit 24b of the CPU 24 starts live view display. Specifically, the display control unit 24 b causes the display monitor 17 to display a through image output from the image sensor 11. In FIG. 4 (1), the photographer E represents a state in which the electronic camera 1 is directed toward friends B and C. An arrow (an arrow superimposed on the electronic camera 1) in the figure represents the direction of the electronic camera 1. Therefore, friends B and C are shown as a through image on the display monitor 17 (see FIG. 4 (5)).

  Step S102: The recognition unit 24a of the CPU 24 performs object recognition processing. Specifically, the recognition unit 24a performs face recognition processing, for example, and recognizes the faces of friends B and C as objects from the through image. Note that the recognition unit 24a may cut out the friends B and C from the through image by, for example, a method using graph cut theory. In this case, the recognition unit 24a recognizes the entire area of the friends B and C as an object. That is, in the present embodiment, the object may be a person's face (face image), the entire person (person image), or other object (for example, an image of a tree shown in the through image). In the description of FIG. 4, for the sake of simplicity, the recognition unit 24 a recognizes a person's face (face image) as an object.

  Step S103: The recognizing unit 24a determines whether or not the recognized object is new. Here, in the association mode, for example, since the photographer E captures a through image while changing the orientation of the electronic camera 1, it is assumed that the recognition unit 24a recognizes the face of the same person again. Therefore, in the case of a new object, the CPU 24 performs the process of step S104. Therefore, when the recognized object is not new (step S103: No), the CPU 24 proceeds to the process of step S105. When the recognized object is new (step S103: Yes), the CPU 24 proceeds to the process of step S104.

  Step S104: The CPU 24 performs an object acquisition process recognized by the recognition unit 24a. Specifically, the CPU 24 temporarily records the face images B1 and C1 of the friends B and C in the RAM 13.

  Step S105: The photographing control unit 24c of the CPU 24 determines whether or not a photographing instruction is input. Specifically, when the release button 21 receives an instruction input for shooting (step S105: Yes), the CPU 24 proceeds to the process of step S106. If the release button 21 has not received a shooting instruction input (step S105: No), the CPU 24 returns to the process of step S102. Then, the CPU 24 repeats the processing from step S102 to step S105 until the release button 21 accepts a shooting instruction input.

  Here, it is assumed that the photographer E turns the electronic camera 1 toward the friend A as shown in FIG. Then, the display control unit 24b of the CPU 24 superimposes and displays the face images B1 and C1 recognized by the recognition unit 24a on the through image showing the friend A on the screen of the display monitor 17 (see FIG. 4 (6)). . The display control unit 24b preferably displays the face images B1 and C1 so as not to overlap the main subject (friend C) of the through image. Subsequently, the recognizing unit 24a recognizes the face of the friend A as an object from the through image (step S102). Since friend A is a new object (step S103: Yes), the CPU 24 performs an object acquisition process (step S104).

  Subsequently, it is assumed that the photographer E changes the direction of the electronic camera 1 and faces the friend D as shown in FIG. Then, the display control unit 24b of the CPU 24 superimposes and displays the face images B1, C1, and A1 recognized by the recognition unit 24a on the through image showing the friend D on the screen of the display monitor 17 (FIG. 4 (7)). reference). Subsequently, the recognizing unit 24a recognizes the face of the friend D as an object from the through image (step S102). Since the friend D is a new object (step S103: Yes), the CPU 24 performs an object acquisition process (step S104).

  Next, it is assumed that the photographer E points the electronic camera 1 toward the friends B and C again as shown in FIG. Then, the display control unit 24b of the CPU 24 superimposes and displays the face images B1, C1, A1, and D1 recognized by the recognition unit 24a on the through image that shows the friends B and C on the screen of the display monitor 17. Here, since the recognition unit 24a has already recognized the friends B and C as objects, the CPU 24 skips the process of step S104 and proceeds to the process of step S105. When the release button 21 receives an instruction input for shooting (step S105: Yes), the CPU 24 proceeds to the process of step S106.

  Step S106: The photographing control unit 24c performs a photographing process. Specifically, the shooting control unit 24c receives shooting parameters such as an aperture value, a charge accumulation time (shutter speed) in the image sensor 11 and a gain (ISO sensitivity) of the signal processing unit 12 in response to an instruction input for a half-press operation. Set each. Thereafter, the CPU 24 drives the image sensor 11 to generate an analog image signal in response to a full-press operation instruction input. The signal processing unit 12 performs A / D conversion of an analog image signal. The digital image signal output from the signal processing unit 12 is temporarily recorded as image data for recording in the RAM 13. Then, the image processing unit 14 reads the image data recorded in the RAM 13 and performs various image processing (for example, gradation conversion processing, contour enhancement processing, white balance processing, etc.) as necessary.

  Step S107: The recording processing unit 24d of the CPU 24 performs a recording process. Specifically, the recording processing unit 24d records the captured image subjected to the process of step S106 and the object in association with each other. In this case, the recording processing unit 24d performs recording by incorporating object information into a partial area of the captured image. Specifically, the recording processing unit 24d records the object image and the object information in association with each other by combining the object information with a partial area of the imaged image or adding the object information to the tag information of the imaged image. To do.

  FIG. 5 is a diagram illustrating an example of a captured image. Of the face images A1, B1, C1, and D1 shown in FIG. 4 (8), the face images B1 and C1 are objects to be photographed. Therefore, the recording processing unit 24d instructs the image processing unit 14 to synthesize the face images A1 and D1 excluding the face images B1 and C1 with the captured image (see FIG. 5A). In this case, the image processing unit 14 combines the face images A1 and D1 so as not to overlap the main subject (friends B and C) of the captured image.

  Then, the CPU 24 records the captured image on the recording medium 30 via the recording I / F unit 16. Thereby, the object information is associated with the captured image. Therefore, if the photographed image is as shown in FIG. 5A, the photographer can easily visually remember the friend who went together.

  Note that the recording processing unit 24d may instruct the image processing unit 14 to combine the face images B1, C1, A1, and D1 with the captured image (see FIG. 5B).

  Alternatively, when the captured image is an image file in Exif (Exchangeable Image File Format) format, the recording processing unit 24d may record the object in the image file.

  FIG. 6 is a diagram illustrating an example of an Exif format image file. FIG. 6 shows an example in which an object (for example, a face image) is registered in an Exif format image file. As shown in FIG. 6, the Exif format image file includes, for example, areas such as the number of pixels, compression mode type, shooting date and time, model name,..., Manufacturer note, thumbnail, shot image, and object (for example, face image). Is configured. The thumbnail is a reduced image (160 × 120 pixels as an example) having a smaller data amount than the main image. The captured image is a recording image generated by the image sensor 11. The object is an image associated with the captured image by the recording processing unit 24d. Thus, if the image file is in an Exif format as shown in FIG. 6, the display control unit 24b displays a captured image in which no object is incorporated at the time of image reproduction according to the selection input of the photographer, The incorporated captured image (see FIG. 5A) can be synthesized and displayed.

  CPU24 complete | finishes the process of the flow shown in FIG. 3, after complete | finishing the process of step S107.

  As described above, the electronic camera 1 analyzes the through image and recognizes an object included in the through image. Therefore, in the electronic camera 1, for example, the electronic camera 1 is photographed by analyzing a through image with respect to information on an object (object) that does not fit in the composition although it exists in the vicinity of a photographing place different from the direction in which the photographed image is taken. Can be easily associated with an image. Therefore, when the photographer confirms a photographed image photographed in the association mode at a later date (for example, several years later), the photographer can easily remember who the group has traveled.

  If friends A and D move and friends A, B, C, and D fit in the composition in the state of FIG. 4D, there is no object that does not fit in the composition. The unit 24b may stop performing the superimposed display of the face image.

(Second Embodiment)
Next, a second embodiment will be described. In the first embodiment, the recognition unit 24a analyzes the through image and recognizes the object. In the second embodiment, a function for selecting and recognizing an object by user input is further added. Thereby, the recognition unit 24a can expand the object recognition means.

  FIG. 7 is a flowchart showing an example of the operation of the electronic camera 1 in the second embodiment. FIG. 8 is a diagram for explaining an example of the operation of the electronic camera 1 in the second embodiment. 8 (1) to 8 (5) schematically show the photographing procedure as in FIG. FIGS. 8 (6) to (10) depict the through images displayed on the display monitor 17 in association with the orientation of the electronic camera 1, as in FIG.

  In order to make the explanation easy to understand, it is assumed that the user has gone on a trip with five friends (A, B, C, D, E) as in the first embodiment. However, it is assumed that another person X who is not related to the five friends is in the shooting area 31. Here, the photographer E wants to photograph the friends B and C. Further, it is desired to keep information on the tree T located near the friends A and D that do not fit in the composition of the photographed image. The photographer E thinks that the friend D does not have to leave any information. In addition, the photographer E thinks that he does not want to leave information on another person X next to the friend A. In other words, in this embodiment, it is possible to select information that the user wants to leave from the viewpoint of the photographer, instead of leaving all friends that do not fit in the composition of the photographed image as information.

  In the following operation example, as in the first embodiment, when the CPU 24 receives a selection input of the association mode on the menu screen, the shooting control unit 24c starts shooting a through image. Then, the CPU 24 starts processing of the flow shown in FIG.

  Step S201: The display control unit 24b starts live view display. And CPU24 transfers to the process of step S202. In FIG. 8 (1), the photographer E represents a state in which the electronic camera 1 is directed to friends B and C. Therefore, friends B and C are shown as a through image on the display monitor 17 (see FIG. 8 (6)). Here, it is assumed that the photographer E changes the direction of the electronic camera 1 toward the friend A (see FIG. 8B).

  Step S202: The CPU 24 determines whether or not an instruction for selecting an object has been input. Here, the photographer E can select peripheral objects that do not fit in the composition of the friends B and C to be photographed by operating the touch panel 19. When an object selection instruction input has not been received (step S202: No), the CPU 24 repeats the process of step S202, and enters a state of waiting for an object selection instruction input. On the other hand, when an instruction input for selecting an object is received (step S202: Yes), the CPU 24 proceeds to the process of step S203.

  Step S203: The recognition unit 24a performs object recognition processing. Here, as shown in FIG. 8 (7), the friend A and the other person X are displayed on the display monitor 17. Under this state, the CPU 24 accepts an object selection input when the photographer E's finger F touches the touch panel 19.

  The CPU 24 receives a signal from the touch panel 19 and detects the fingertip position information in correspondence with the position in the through image. The photographer E may surround the area of the object to be selected with the finger F with the finger.

  Based on the fingertip position information, the recognition unit 24a analyzes the through image and recognizes the object. Or the recognition part 24a recognizes the area | region of the range enclosed by the fingertip as an object. Then, the CPU 24 proceeds to the process of step S204.

  Step S204: The recording processing unit 24d performs object registration processing. Specifically, first, the CPU 24 extracts an object (for example, a person image) of the friend A recognized by the recognition unit 24a via the touch panel 19 from the through image. Subsequently, the recording processing unit 24 d records the object in the database 15 a of the flash memory 15. As a result, the object is temporarily registered in the database 15a. And CPU24 transfers to the process of step S205.

  Step S205: The display control unit 24b performs object display processing. Specifically, as shown in FIG. 8 (8), the display control unit 24b superimposes and displays a friend A's person image (reduced image) A2 in a through image area corresponding to the lower right of the screen of the display monitor 17. And CPU24 transfers to the process of step S206.

Step S206: The CPU 24 determines whether or not there is an instruction input for ending the object selection. When the instruction input for ending the object selection has not been received (step S206: No), the CPU 24 returns to the process of step S202, shifts to the waiting state for the instruction selection for the object selection, and again performs the process from step S202 to step S206. repeat.
Then, the CPU 24 repeats the processing from step S202 to step S206 until receiving an instruction input for ending the object selection.

  Here, as shown in FIG. 8 (4), it is assumed that the photographer E changes the direction of the electronic camera 1 and faces the tree T. And CPU24 receives selection input of tree T as an object when photographer E's finger F touches touch panel 19 (Step S202: Yes). Then, the recognition unit 24a performs a tree T recognition process (step S203). Subsequently, the recording processing unit 24d performs a registration process for the tree T (step S204). Further, the display control unit 24b performs display processing of the image T1 of the tree T (step S205). Specifically, as shown in FIG. 8 (9), the display control unit 24b superimposes and displays the person A image of the friend A and the image T1 of the tree T on the through image area corresponding to the lower right of the screen of the display monitor 17. Let

  If an instruction input for ending the selection of the object is received (step S206: Yes), the CPU 24 proceeds to the process of step S207.

  Step S207: The CPU 24 determines whether or not a shooting instruction is input. Specifically, when the release button 21 has not received a shooting instruction input (step S207: No), the CPU 24 repeats the process of step S207 to enter a shooting instruction input waiting state. FIG. 8 (5) shows a state in which the photographer E changes the orientation of the electronic camera 1 and points the electronic camera 1 toward the friends B and C that are the photographing objects. At this time, the display control unit 24b superimposes and displays the person A image of the friend A and the image T1 of the tree T on the through image area corresponding to the right side of the screen of the display monitor 17.

  In this state, when the release button 21 receives an instruction input for photographing (step S207: Yes), the CPU 24 proceeds to the process of step S208.

  Step S208: The photographing control unit 24c performs a photographing process. Here, the imaging control unit 24c performs the imaging process similar to step S106 in FIG. Then, the process proceeds to step S209.

  Step S209: The recording processing unit 24d performs a recording process. Specifically, the recording processing unit 24d associates the photographed image subjected to the process of step S208 with the object (the person image A2 and the tree T image T1 shown in FIG. 8 (10)) registered in the database 15a. Record. Here, for example, as described in step S107 of FIG. 3 of the first embodiment, the recording processing unit 24d records the object information in association with the image file in the Exif format of the captured image. In other words, the recording processing unit 24d in the second embodiment registers an object in the database 15a and records the object in a captured image. And CPU24 will complete | finish the process of the flow shown in FIG. 7, after complete | finishing the process of step S209.

  As described above, since the electronic camera 1 can selectively register an object by user input, a specific target (object) that does not fit in the composition can be selectively associated with a captured image. In addition, the electronic camera 1 can easily register objects other than a person (for example, a tree) by user input. That is, in the second embodiment, only the information of the object intended by the photographer can be recorded in association with the photographed image as necessary. In the second embodiment, an object is selected in real time by user input. However, a list of object information may be displayed immediately after shooting a captured image, and may be selected by user input.

(Third embodiment)
Next, a third embodiment will be described. In the third embodiment, a function for managing information on objects already registered is further added. For example, there may be a case where the photographer wants to continuously shoot in the association mode after shooting in the association mode. In this case, if an object (for example, a face image) is registered again at the time of shooting a through image, there is a problem that it takes twice. Therefore, the third embodiment provides a means for solving this problem.

  FIG. 9 is a diagram for explaining an example of the operation of the electronic camera 1 in the third embodiment. FIG. 10 is a diagram illustrating an example of a captured image in the third embodiment.

  Here, when the photographer E points the electronic camera 1 as shown in FIG. 4 (4), the display control unit 24b converts the through image on the screen of the display monitor 17 as shown in FIG. 4 (8). The face images A1, B1, C1, and D1 are superimposed and displayed. In this state, the recording processing unit 24d of the CPU 24 registers the face images A1, B1, C1, and D1 in the database 15a. When the photographer E presses the release button 21, the shooting control unit 24c records the shot image.

  Here, in order to make the explanation easy to understand, FIG. 9A shows a through image after being shot with the composition shown in FIG. In FIG. 9, compared with FIG. 4, the illustration in the imaging area 31 is omitted, and the direction of the electronic camera 1 is indicated by a notation in which an arrow is superimposed on the electronic camera 1.

  In FIG. 9 (1), the display control unit 24 b causes the face images A 1, B 1, C 1, and D 1 to be superimposed and displayed on the through image region corresponding to the right side of the display monitor 17 via the display I / F unit 18. . The recognizing unit 24a recognizes the face in the current through image by face recognition processing. Then, as shown in FIG. 9A, the display control unit 24b further superimposes and displays the face images B1 and C1 currently shown in the through image in a specific color (for example, light gray) (hereinafter referred to as “gray-out display”). .) Thereby, the photographer E can know that the face images A1 and D1 are associated with the captured image when the captured image is captured with this composition.

  In other words, the display control unit 24b reads the face images A1, B1, C1, and D1 from the database 15a and displays them superimposed on the through image. Thereby, the recording processing unit 24d can save the trouble of re-registering the object.

  When the photographer E points the electronic camera 1 toward the friend A, the display control unit 24b displays the face image A1 of the friend A in gray on the display monitor 17. When the photographing control unit 24c receives a photographing instruction input from the photographer E with this composition, the photographing control unit 24c performs a photographing process. The photographed image shown in FIG. 10A indicates that the photograph was taken with the composition shown in FIG. 9B, and the face image A1 displayed in gray out is not incorporated into the photographed image.

  When the photographer E points the electronic camera 1 at the friend D, the display control unit 24b causes the display monitor 17 to gray-out the face image D1 of the friend D. When the photographing control unit 24c receives a photographing instruction input from the photographer E with this composition, the photographing control unit 24c performs a photographing process. The photographed image shown in FIG. 10B indicates that the photograph was taken with the composition shown in FIG. 9C, and the face image D1 displayed in gray out is not incorporated in the photographed image.

  In this way, the display control unit 24b performs gray-out display, so that the photographer E can easily visually understand the presence of a person who does not exist in the composition even though the photographer E exists in the vicinity of the shooting location. Become. Further, in the present embodiment, it is not necessary to register an object again at the time of shooting a through image, and for example, only a person who does not exist in the composition can be registered in association with the image file of the captured image.

  Next, a case where the photographer E deletes a specific person who does not exist in the composition will be described as a function for managing object information. For example, when the photographer E wants to delete the registration of the friend D, the photographer E may touch the area of the face image D1 on the screen of the display monitor 17 with the finger F after setting the deletion mode. Thereby, the CPU 24 receives an instruction input for confirming deletion of the object (face image D1) via the touch panel 19 (see FIG. 9 (4)).

  Then, the display control unit 24b causes the display monitor 17 to display a message such as “Do you want to delete?” As an example, using the hop-up menu. When the photographer E touches “OK” on the screen with the finger F, the CPU 24 receives an instruction to delete the object (face image D1) via the touch panel 19 (see FIG. 9 (5)).

  Thereby, the management unit 24h deletes the friend D's object (face image D1) registered in the database 15a. Then, the display control unit 24b superimposes and displays the face images A1, B1, and C1 on the through image area corresponding to the right side of the display monitor 17 (see FIG. 9 (6)). However, since friends B and C exist in the composition, the display control unit 24b displays the face images B1 and C1 in gray out. When the photographing control unit 24c receives a photographing instruction input from the photographer E with this composition, the photographing control unit 24c performs a photographing process. The captured image shown in FIG. 10 (c) indicates that the photograph was taken with the composition shown in FIG. 9 (6), and the face images B1 and C1 displayed in gray out are not incorporated into the photographed image.

  As described above, by using the function for managing objects, the photographer E can easily select an object to be associated with the captured image.

(Fourth embodiment)
Next, a fourth embodiment will be described. In the fourth embodiment, collation object information is registered in advance in the database 15 a in the flash memory 15 of the electronic camera 1. In the fourth embodiment, matching is performed using a through image or a partial image (for example, a face image) included in the through image and information on a matching object registered in the database 15a. Here, the recognizing unit 24a recognizes that the through image or the partial image included in the through image is an object to be recognized according to the collation result of the first collating unit 24e.

  Then, the recording processing unit 24d associates the object information recognized by the recognition unit 24a with the captured image. In the fourth embodiment, only specific object information can be associated with a captured image.

  More specifically, when an object included in a partial image (for example, a face image) is registered in the database 15a, for example, as a method for recognizing the object, a feature amount indicating object information is registered in the database 15a. The first matching unit 24e performs matching between the feature amount and the partial image included in the through image.

  As a result of the comparison, if they match by a certain threshold or more, the recognition unit 24a recognizes the partial image included in the through image as an object to be recognized. The recording processing unit 24d associates the information of the object recognized by the recognition unit 24a with the captured image.

  Here, the first collating unit 24e may collate the feature amount registered in the database 15a with the through image in the entire through image, or may collate with only a part of the through image. That is, a face image is previously extracted from the through image using a known technique such as face detection, and the first collation unit 24e can collate the face image with the face image registered in the database 15a. good. By extracting a part of the through image in this way, the processing load of the collation process can be reduced.

  On the other hand, when the object included in the through image or the partial image (for example, the face image) is not registered in the database 15a (as a result of the comparison, the recognition unit 24a does not match a certain threshold or more), the recognition unit 24a recognizes the object. Absent. That is, the recording processing unit 24d does not associate the captured image with the object information.

  Therefore, in the fourth embodiment, information on a specific object that does not require user input and does not fit in the composition can be selectively associated with a captured image.

  In the database 15a, names corresponding to the objects may be registered in advance. The replacement unit 24g replaces the object with a name (character string) corresponding to the object according to the collation result of the first collation unit 24e.

  FIG. 11 is a diagram illustrating an example of the operation of the electronic camera 1 in the fourth embodiment. FIG. 11A shows a state where a through image having the composition shown in FIG. 9A is displayed on the display monitor 17, for example. In the above embodiment, in live view display, an object is superimposed and displayed in a through image, but the display mode by the display control unit 24b is not limited to such superimposed display. As shown in FIG. 11A, the display control unit 24b may display the object in an area different from the display portion of the through image.

  Here, after the photographer E points the electronic camera 1 at the friend A, the photographer E operates the cross button 20e to display the face image as the display mode of the object and the name corresponding to the face image. An instruction to replace the display with (character string) may be input.

  In this case, when the CPU 24 accepts the instruction input, the first collation unit 24e collates the names of the face images A1, B1, C1, and D1 from the database 15a. When the first collation unit 24e can collate each name, the replacement unit 24g replaces each of the face images A1, B1, C1, and D1 with a corresponding name.

  Subsequently, the display control unit 24b causes the display monitor 17 to display a name corresponding to each face image, as shown in FIG. In this case, since the friend A is reflected in the through image, the display control unit 24b displays the character string “Mr. A” in gray out. As a result, the recording processing unit 24d can record the photographed images in association with the names of the friends B, C, and D.

  When the captured image is an image file in the Exif format, the recording processing unit 24d may associate the name of the friend with the face image and register the text data of the name in the maker note area. The recording processing unit 24d may also record a face image.

  As described above, in the fourth embodiment, the first collation unit 24e collates the object information in the database 15a, so that the object information can be selectively associated with the photographed image without requiring the user input, or the user input The display mode of the object can be replaced from the image to a character string according to the above.

(Fifth embodiment)
Next, a fifth embodiment will be described. In the fourth embodiment described above, information on the object for verification is registered in advance in the database 15a in the flash memory 15 of the electronic camera 1, but in the fifth embodiment, information on the object for verification is stored in the external server. The difference is that it is registered in the database and used. Therefore, in the fifth embodiment, a collation system using an external server is adopted.

  FIG. 12 is a diagram illustrating a configuration example of a collation system according to the fifth embodiment. The verification system includes an electronic camera 1, a wireless local area network (LAN) device 40, an Internet network 41, and an external server 50.

  The external server 50 is applicable to a general server computer, and includes a CPU, a RAM for data processing work, a flash memory for recording an OS (operating system) program, and a storage device for storing various data (for example, a hard disk drive ( (Hard Disc Drive) apparatus), communication means for communicating with the electronic camera 1 via the Internet network 41, etc. (not shown).

  Further, in the external server 50, the database 50a of this embodiment is installed in the storage device. The database 50a has the same configuration as the database 15a in the electronic camera 1. In the external server 50 shown in FIG. 12, only the database 50a is drawn for convenience of explanation. Note that the CPU of the external server 50 performs data processing of the database 50a based on an instruction from the electronic camera 1 side.

  The communication I / F unit 22 of the electronic camera 1 is connected to the Internet network 41 via the wireless LAN device 40 by wireless communication via the antenna 22a and communicates with the external server 50.

  As described above, the second collation unit 24f collates the partial image included in the captured image with the information on the collation object registered in the database 50a via the communication I / F unit 22. The recognizing unit 24a recognizes the object according to the collation result of the second collating unit 24f.

  As in the fourth embodiment, the collation is performed when an object included in a partial image (for example, a face image) is registered in the database 50a. For example, as a method for recognizing an object, object information is indicated in the database 50a. The feature amount is registered, and the second matching unit 24f performs matching between the feature amount and the partial image included in the through image.

  As a result of the comparison, if the values match by a certain threshold value or more, the recognition unit 24a recognizes the partial image included in the through image as an object to be recognized, as in the fourth embodiment. The recording processing unit 24d associates the information of the object recognized by the recognition unit 24a with the captured image.

  Here, the second collating unit 24f may collate the feature amount registered in the database 50a with the through image in the entire through image, or collate with only a part of the through image. Note that details are the same as the processing of the first verification unit 24e described above, and a description thereof will be omitted.

  On the other hand, when the object included in the through image or the partial image (for example, the face image) is not registered in the database 50a (as a result of the comparison, the recognition unit 24a recognizes the object). Absent. That is, the recording processing unit 24d does not associate the captured image with the object information.

  Therefore, in the fifth embodiment, by using the external server 50, information of a specific object that does not require user input and does not fit in the composition can be selectively associated with the captured image.

  In the database 50a, names corresponding to objects (for example, face images) may be registered in advance in the same manner as the database 15a. The replacement unit 24g replaces the object with a name (character string) corresponding to the object according to the collation result of the second collation unit 24f. Thereby, the recording processing unit 24d can record the captured image and the name in association with each other.

  As described above, in the fifth embodiment, the second collation unit 24f collates object information using the database 50a of the external server 50. Thus, in the fifth embodiment, object information can be selectively associated with a captured image without requiring user input, or the display mode of object information is replaced from an image to a character string in accordance with user input. can do.

(Sixth embodiment)
Next, a sixth embodiment will be described. In the sixth embodiment, the management unit 24h of the CPU 24 further includes a management function for managing information on at least one object registered by the recording processing unit 24d. For example, the management unit 24h can manage object information as a group as described below.

  FIG. 13 is a conceptual diagram showing an example of a group registered in the database 15a. Since the recording processing unit 24d registers the objects recognized by the recognition unit 24a as a group, as an example, the recording processing unit 24d records the recognized objects as a group in the database 15a. Thereby, the recording processing unit 24d can register a group of objects in association with the shooting date and time. Here, in order to efficiently use the recording area, the recording processing unit 24d may cancel the current registration when the object of the group registered this time is the same as the object of the group registered last time.

  In the present embodiment, the registered object may be transferred to the database 50a of the external server 50.

  Alternatively, the unit of the group is a certain event (XX trip, YY dinner party, etc.), and the user may create the group manually (manual operation) via the touch panel 19. Specifically, when the user selects group registration in the camera menu, the display control unit 24b causes the display monitor 17 to display a message such as “Do you want to register the currently recognized object as a group?”. Here, when the user selects “YES”, the recording processing unit 24d registers the currently recognized objects as a group in the database 15a.

  When a group is automatically created, the electronic camera 1 may automatically recognize the start / end of a certain event and group the currently recognized objects at the end of the event.

  In order to make the explanation easy to understand, for example, in the case of the composition shown in FIG. 11A, the management unit 24h automatically reads a specific group including the object currently recognized by the recognition unit 24a from the database 15a. In this case, since the friends B and C are shown in the through image, the management unit 24h reads the group including the friends B and C from the database 15a. The management unit 24h may read a specific group designated by user input from the database 15a.

  Then, the display control unit 24b temporarily switches the screen display so that, for example, the first to third group objects are displayed on the display monitor 17. Since the display control unit 24b displays a specific group of objects on the display monitor 17, the photographer E can easily find an object to be associated with.

  FIG. 14 is a diagram illustrating an example of a group displayed on the display monitor 17. Here, if the photographer E wants to associate the friends A and D with the photographed image, the photographer E may select the first group.

  As described above, in the sixth embodiment, object information can be associated with a captured image in accordance with the intention of the photographer E by using the group management function. Note that the management unit 24h may create, read, edit, or delete a group in response to a user input.

(Supplementary items of the embodiment)
(1) In the above-described embodiment, the compact electronic camera is illustrated as an imaging apparatus to which the present invention is applicable. However, the present invention is not limited to this electronic camera. The imaging apparatus to which the present invention is applicable includes, for example, a single-lens reflex electronic camera and a mirrorless single-lens electronic camera. The imaging apparatus to which the present invention can be applied also includes a mobile phone having a camera function and a personal digital assistant (PDA). In the case of a mobile phone having an electronic camera function, communication with the external server 50 may be configured to connect to the Internet network 41 via a mobile phone base station.

  (2) In the above embodiment, as an example of the captured image, as shown in FIG. 5 and FIG. 10, the object (face image) is superimposed on the composition of the captured image. Not limited to this. Hereinafter, in FIG. 15, other display modes of the captured image will be exemplified.

  FIG. 15 is a diagram illustrating another display mode of a captured image. FIG. 15 shows a state where an Exif format image file shot and recorded in the association mode is reproduced on the display monitor 17. In FIG. 15A, registered objects (face images) are combined in a different area from the photographed image. In FIG. 15B, names (character strings) corresponding to objects registered in a different area from the photographed image are synthesized. In FIG. 15C, names (character strings) corresponding to registered objects are superimposed and displayed in a partial area in the captured image.

  As described above, the display mode of the captured image can be variously changed by, for example, the photographer selecting it on the menu screen.

  (3) In addition to the above embodiment, in the association mode, a captured image may be captured after capturing a through image while detecting an azimuth angle. Here, an azimuth angle detection mode is added as a setting mode subordinate to the association mode.

  Specifically, in the azimuth angle detection mode, the CPU 24 detects the direction (azimuth angle) from the electronic camera 1 (the optical axis of the photographing optical system 10) to the object based on the geomagnetic data output from the azimuth sensor 23. To do. Then, the recording processing unit 24d records the captured image and the information of the object in the through image in association with each other. Further, the recording processing unit 24d adds and records the azimuth information to the object information. Thereby, the positional relationship between the object in the composition and the object that does not fit in the composition can be known. Accordingly, the display control unit 24b can change the way of displaying the object based on the azimuth information when, for example, reproducing a captured image. Hereinafter, a specific example will be described with reference to FIG.

  FIG. 16 is a diagram for explaining a captured image reproduction process in the azimuth angle detection mode. For example, the photographer E first shoots a through image and then shoots a captured image in the same manner as in the first embodiment. At this time, the imaging control unit 24c captures a through image while detecting the azimuth angle.

  Subsequently, the imaging control unit 24c captures a captured image. The recording processing unit 24d records the captured image and the object information in association with each other, and records the object information with azimuth information added thereto.

  FIG. 16A shows a state where the captured image is displayed on the display monitor 17. Here, when the photographer E moves the finger F to the left or right by operating the touch panel 19, the display control unit 24 b displays the object associated with the captured image with an azimuth angle according to the operation of the touch panel 19. Based on the information, it is read out and displayed on the display monitor 17 (see FIGS. 16B and 16C). As a result, the photographer E can see the positional relationship between the friends A and D who did not fit in the photographed image as seen from the photographed image.

  Further, when displaying the object in the captured image, the display control unit 24b may change the position to be displayed in the captured image based on the azimuth information. For example, in FIG. 16A, the display control unit 24b superimposes and displays the face images D1 and A1 on the lower right side of the photographed image. However, the face image D1 of the photographed image is displayed based on the azimuth information. The face image A1 may be superimposed and displayed on the lower left side of the captured image on the lower right side. As a result, the photographer E knows the positional relationship between the friends A and D that did not fit in the captured image in the captured image.

  Alternatively, in FIGS. 16B and 16C, it is assumed that the distance between the friend B and the friend A is shorter than the distance between the friend B and the friend D. Here, the CPU 24 may estimate the distance based on, for example, azimuth information. The display control unit 24b, for example, according to the distance between the friend B and the friend A and the distance between the friend B and the friend D, information on the object of the friend A in the virtual space that is out of the composition of the photographed image, and the friend Information on the object D may be arranged on the display monitor 17. In this case, the display control unit 24b displays the finger slide distance necessary for displaying the information of the friend A object while displaying the captured image, and the finger slide necessary for displaying the information of the friend D object. Make it shorter than the distance.

  That is, when the user traces with the finger on the touch panel 19, the sliding distance of the finger is associated with, for example, the distance between the friend B and the friend A or the distance between the friend B and the friend D. Thereby, for example, when the user performs a scrolling operation twice with a finger in the right direction on the shooting screen, the display control unit 24b displays the object of the friend A on the display monitor 17, and the user moves the shooting screen. When the scroll operation is performed three times with the finger in the left direction, the display control unit 24b may display the object of the friend D on the display monitor 17.

  Alternatively, when the display control unit 24b slides in the object information and displays it on the display monitor 17, the display control unit 24b changes the time taken for the object information to be displayed at a predetermined position on the display monitor 17 according to the distance. You may do it. For example, it is assumed that the distance between the friend B and the friend A is shorter than the distance between the friend B and the friend D. While the captured image is being displayed (FIG. 16A), when the button on the left side of the cross button 20e shown in FIG. 2 is pressed by a user input, the display control unit 24b slides in the friend D on the display monitor 17. At a predetermined position (for example, the position of the image D1 displayed on the display monitor 17 of FIG. 16). On the other hand, when the photographed image is being displayed (FIG. 16A), when the button on the right side of the cross button 20e shown in FIG. 2 is pressed by the user input, the display control unit 24b slides in the friend A to display monitor. 17 is displayed. In the display by the slide-in, since the distance between the friend B and the friend A is shorter than the distance between the friend B and the friend D, the display control unit 24b displays the friend A after the button on the right side of the cross button 20e is pressed. Is displayed at a predetermined position on the display monitor 17 than when the friend D is displayed at a predetermined position on the display monitor 17 after the button on the left side of the cross button 20e is pressed. You can do it quickly.

  (4) In the above embodiment, an Exif format image file is exemplified, but the image file is not limited to an Exif format image file. In the above embodiment, an image file of another format that can record a captured image and an object in association with each other may be used.

DESCRIPTION OF SYMBOLS 1 ... Electronic camera, 11 ... Imaging device, 24a ... Recognition part, 24c ... Shooting control part, 24d ... Recording control part

Claims (12)

An image sensor for capturing an image of a subject;
A recognition unit that analyzes a first image obtained in time series by the imaging device and recognizes an object included in the first image;
A shooting control unit for shooting the first image and shooting a second image for recording;
A recording processing unit that records the second image and information of an object not included in at least the second image recognized by the recognition unit in association with each other;
An imaging apparatus comprising: The imaging device according to claim 1,
A display unit for displaying images;
A display control unit for controlling the display mode of the display unit,
The display control unit causes the display unit to display the first image and the information of the object,
The recording processing unit records information on the object displayed on the display unit together with the first image in association with the second image. The imaging device according to claim 2,
A selection input unit that selects the object by user input with respect to the first image displayed on the display unit,
The recognizing unit recognizes the object selected by the selection input unit by the user input. The imaging device according to claim 3.
The selection input unit is a touch panel for inputting information by contact of a contact body,
The said recognition part recognizes the said object which the said contact body instruct | indicated via the said touch panel with respect to the said object in the said 1st image currently displayed on the said display part. In the imaging device according to any one of claims 1 to 4,
The recording processing unit combines the object information with a part of the second image, or adds the object information to the tag information of the second image, thereby combining the second image and the object information. An imaging apparatus characterized by recording in association with each other. In the imaging device according to any one of claims 1 to 5,
A database in which object information for matching is registered in advance;
A first collation unit that collates a partial image included in the first image and information on the object for collation registered in the database;
The recognizing unit recognizes the object according to a collation result of the first collating unit. The imaging apparatus according to any one of claims 1 to 6,
A communication unit that communicates with an external server having a database in which object information for verification is registered in advance;
A second collation unit that collates the partial image included in the first image and the information of the object for collation registered in the database via the communication unit;
The recognizing unit recognizes the object according to a collation result of the second collating unit. In the imaging device according to claim 6 or 7,
The information of the object registered in the database includes information related to the object, and the recording processing unit is configured to output the second image according to a collation result of the first collation unit or the second collation unit. And an information relating to the object and recorded in association with each other. The imaging device according to any one of claims 6 to 8,
An imaging apparatus, further comprising: a management unit that manages information on the object registered by the recording processing unit for each predetermined condition. The imaging device according to claim 9,
The image capturing apparatus, wherein the management unit manages information of the plurality of objects as a group. The imaging device according to claim 10.
The image processing apparatus, wherein the recording processing unit registers the information of the object recognized by the recognition unit as a group in the database for each photographing, every time, every date, or every photographing event. In the imaging device according to claim 10 or 11,
The management unit reads a specific group specified by user input or a specific group including an object that is currently recognized by the recognition unit,
The display control unit causes the display unit to display information on the objects of the specific group.