JP2015159511A - Photographing apparatus and image recording method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a photographing apparatus and an image recording method with which information desired by a user can be added to a photographed image and a resulting image can be recorded.SOLUTION: A photographing apparatus 10 includes a recording and reproduction control unit 12 that can record a photographed image of a subject as a main image, a position and object determination unit 13 that sets at least one of a relevant position and a relevant object in the main image, and an association control unit 14 that acquires sub information on the relevant position and relevant object set by the position and object determination unit, and controls the recording and reproduction control unit to record the main image and the sub information in association with each other.

Description

  The present invention relates to a photographing apparatus and an image recording method for recording auxiliary data in an image file.

  2. Description of the Related Art In recent years, portable devices (photographing devices) with a photographing function such as digital cameras have various photographing functions by making full use of image processing. Some photographing devices have a communication function, and an image obtained by photographing can be transmitted to another device or a network.

  By the way, conventionally, in an image recording device, auxiliary data other than an image such as information on the shooting date and time and a shooting location is recorded as a file together with the image. That is, the image file is composed of image data and auxiliary data, and the image file can be edited after recording.

  Furthermore, as disclosed in Patent Document 1, an apparatus for discriminating an object included in a recorded original image and extracting the discriminated object by assigning a priority has been developed. For example, by setting a character area as an object, it is possible to extract characters included in the image and record them as auxiliary data of the image data.

Japanese Patent No. 4519531

  However, the conventional technique only records information extracted from an image and information related to image capturing as auxiliary data, and can always present information desired by the user in a form desired by the user. There was no problem.

  An object of the present invention is to provide a photographing apparatus and an image recording method capable of adding and recording information desired by a user to a captured image with a simple operation.

  An imaging device according to the present invention includes a recording control unit capable of recording a captured image of a subject as a main image, a position and target determination unit for setting at least one of a related position and a related target in the main image, and the position And an association control unit that acquires sub information related to the related position and the related object set by the target determination unit, controls the recording control unit, and records the main image and the sub information in association with each other. It comprises.

  The image recording method according to the present invention includes a step of acquiring a captured image of a subject as a main image, a position and target determination step for setting at least one of a related position and a related object in the main image, and the position And an association step of acquiring sub-information related to the related position and the related object set in the object determination step, and associating and recording the main image and the sub-information.

  According to the present invention, there is an effect that information desired by a user can be added to a captured image and recorded by a simple operation.

1 is a block diagram showing a circuit configuration of a photographing apparatus according to a first embodiment of the present invention. Explanatory drawing for demonstrating imaging | photography and recording of a main image and a sub image. Explanatory drawing which shows the mode of imaging | photography of the sub image in correlation mode. Explanatory drawing for demonstrating the image file recorded at the time of correlation mode. Explanatory drawing which shows an example of sub information. The flowchart for demonstrating operation | movement of 1st Embodiment. The flowchart for demonstrating the operation | movement in reproduction | regeneration mode. The block diagram which shows the 2nd Embodiment of this invention. Explanatory drawing which shows the external appearance of the imaging device which concerns on 2nd Embodiment. Explanatory drawing for demonstrating the example which downloads sub information from the internet. Explanatory drawing for demonstrating the example which downloads sub information from the internet. Explanatory drawing for demonstrating the image file recorded at the time of correlation mode. Explanatory drawing which shows an example of sub information. Explanatory drawing for demonstrating the mode at the time of reproduction | regeneration. The flowchart for demonstrating operation | movement of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a block diagram showing a circuit configuration of the photographing apparatus according to the first embodiment of the present invention. In the present embodiment, a sub image (sub information) related to an arbitrary position or an object in a main image obtained by shooting by a user is obtained, and the main image and the sub image (sub information) are recorded in association with each other. To do. For example, the sub image is obtained by a user's photographing operation.

  2 to 5 are explanatory diagrams for explaining the shooting and recording of the main image and the sub image.

  FIG. 2 shows how the user acquires the main image by shooting. As shown in FIG. 2, the user 21 is photographing a person 31 as a subject while holding the housing 22 in which the circuit block of the photographing apparatus 10 shown in FIG. 1 is stored with the right hand 21 a. A person 31 stands by a stone monument 33 engraved with a name 33a of a plateau where the person 31 is located against a mountain range 32 as a background. A display screen 23 is provided on the back side of the housing 22 of the photographing apparatus 10, and a through image 24 that is an image being captured is displayed on the display screen 23. FIG. 2 shows that a captured image including a person 31, a mountain range 32, and a monument 33 within the shooting range is displayed as a through image 24.

  In this state, when the user 21 presses the shutter button 22a with the right hand 21a, the captured image displayed as the through image 24 is recorded. Now, for example, it is assumed that the captured image recorded by the user 21 is the main image.

  In the present embodiment, an operation in an association mode in which a main image and a sub image (sub information) are recorded in association with each other is possible. In the association mode, the user 21 can specify which position in the image (hereinafter referred to as a related position) or an object (hereinafter referred to as a related object) to capture a sub image. FIG. 2 shows that the user 21 performs an operation of touching the image 25 corresponding to the monument 33 in the through image 24 with the finger 21b of the left hand in the association mode. By this touch operation, the position of the image 25 corresponding to the monument 33 in the main image is set as the related position. In this way, it is possible to give meaning to the position of the image and to read other information from the position on the image based on the captured image. As a result, the image appreciation is not only superficial, but it can be a key for further information reading, and the viewer's pursuit experience can be further enriched. For example, if you are worried about being written on a signboard or poster that cannot be read, touching it will allow you to play more information and satisfy your curiosity. Information that satisfies this curiosity is added as follows.

  FIG. 3 shows a state of taking a sub image in the association mode. The user 21 shoots a sub image corresponding to the monument 33 designated as an association target by a touch operation while holding the housing 22 of the photographic device 10 with the right hand 21a. It is possible to shoot after designation, or shoot while designation. When photographing after designation, as shown in FIG. 3, for example, the user 21 performs a zoom operation or changes the photographing position, and the monument 33 is photographed to fill the display range of the display screen 23. Adjust the angle of view so that FIG. 3 shows this state, and the image 43 corresponding to the monument 33 is displayed in the full display range. In addition, on the upper part of the display screen 23, a display 44 of “related shooting” indicating that the sub image is being shot in the association mode is displayed. This display is important. Without this display, the user cannot know whether the main image is taken as a photographic work or an information image. In addition, when shooting a work, naturalness, hue, gradation expression, etc. are important. However, when shooting text, etc. when shooting information, it is important to be able to read the text, it may be black and white, and the contrast is unnatural. You can make it easier to read. The shooting control includes exposure control, aperture control, and the like. However, this may be different from the main shooting to make it easier to read characters or to reduce blur. In addition, black and white and edge enhancement are image processing, and it is better to change the representation as auxiliary information from the main image by performing one or both of shooting control and image processing. Such a control may be changed to the main image not only at the time of recording but also at the time of a through image. In this state, when the user 21 presses the shutter button 22a, the captured image displayed as the through image 24 is recorded as a sub image. Prior to this shooting, a portion of the viewer's curiosity may be displayed as a guide during main shooting. For example, a face or a character, a mountain characteristic of the background scenery, or the like may be determined as an image recapture candidate by image determination, and advice display may be performed so as to acquire information regarding this portion. For example, if you specify this part with a finger in a shooting confirmation image, that part will be in the through image state, and while shooting that while viewing the through image, displaying that it needs more information in that part Anyway. It is such a shooting method that it is possible to shoot while specifying.

  FIG. 4 is an explanatory diagram for explaining an image file recorded in the association mode. 4A illustrates a state where only the main image is recorded, and FIG. 4B illustrates a state where the main image and the sub image are recorded.

  As shown in FIGS. 4A and 4B, the image file includes image data 45 of the main image and tag information 46 which is auxiliary data. The tag information 46 includes information relating to the main image, information relating to the shooting of the main image, and the like, for example, subject face information, shooting location, shooting time information and the like. Further, in the present embodiment, the tag information is information indicating that the sub-image related to the main image is recorded in association with the shooting in the association mode and is recorded. Some sub-image assumption information is also included. Furthermore, in the present embodiment, the image file also includes sub information 47 recorded in the association mode.

  FIG. 5 is an explanatory diagram showing an example of sub information. FIG. 5 corresponds to the photographing in FIGS. 2 and 3. In the present embodiment, the sub information is registered for each related position or related object specified by the user. The example of FIG. 5 shows an example in which the screen is divided into five areas: center, upper right, lower right, upper left, and lower left, and sub information is registered for each area. Personal names are registered corresponding to the area in the center of the screen. This sub-information indicates information related to the name of a person (person 31 in FIGS. 2 and 3) photographed approximately at the center of the imaging range. Further, FIG. 5 shows that a sub-image acquired by photographing shown in FIG. 3 is recorded in correspondence with the lower left of the screen. Further, FIG. 5 shows that character information of “Δ plateau” acquired by character recognition for the character string in the sub-image is also recorded. In order to perform such character recognition, the sub image shooting is controlled differently from the main image shooting (including image processing).

  In FIG. 1, an imaging device 10 has an imaging unit 11 constituted by an imaging element such as a CCD or a CMOS sensor. The imaging unit 11 captures a subject and outputs a captured image. The captured image is given to the associating unit 20. The association control unit 14 of the association unit 20 includes a CPU and a memory (not shown), and performs camera control according to a program stored in the memory. The association control unit 14 controls the imaging unit 11 (not shown), and performs predetermined signal processing, such as color adjustment processing, matrix conversion processing, noise removal processing, and the like, on the captured image output by the imaging unit 11. Perform various signal processing.

  Further, the association control unit 14 can execute various processes related to display. The association control unit 14 can give the captured image after the signal processing to the display and operation unit 17. The display and operation unit 17 has a display screen such as an LCD, and displays an image given from the association control unit 14. Further, the association control unit 14 can display various menu displays and the like on the display and display screen 23 (see FIG. 2) of the operation unit 17.

  The display and operation unit 17 includes an operation unit (not shown) that receives user operations. As this operation unit, the display and operation unit 17 may include, for example, a touch panel configured on the display screen 23. The touch panel can generate an operation signal corresponding to the position on the display screen pointed by the user with a finger. This operation signal is supplied to the association control unit 14. Thus, when the user touches or slides on the display screen, the association control unit 14 arrives by the user's touch position, an operation for closing and separating the finger (pinch operation), a slide operation or a slide operation. Various operations such as a position, a sliding direction, a touch period, and the like can be detected, and processing corresponding to a user operation can be executed.

  The recording / playback control unit 12 can perform processing related to recording and playback of captured images. For example, the recording / playback control unit 12 can compress the captured image after signal processing, and can apply the compressed image to the recording unit 16 for recording. As the recording unit 16, for example, a card interface can be adopted, and the recording unit 16 can record image information, audio information, and the like on a recording medium such as a memory card.

  The recording / playback control unit 12 can read out the captured image recorded in the recording unit 16 and perform expansion processing. The recording / reproducing control unit 12 can reproduce the recorded image by giving the expanded captured image to the display / operation unit 17.

  Note that the display screen 23 of the display and operation unit 17 is arranged so as to occupy substantially the entire rear surface of the photographing apparatus 10, for example, and the photographer displays the display screen 23 on the display and operation unit 17 during photographing. The photographed operation can be performed while confirming the through image.

  In the present embodiment, the association control unit 14 can acquire the main image and the sub information and record them in the recording unit 16 in the association mode. When the position in the image is designated by an operation of the display and operation unit 17, for example, a touch operation on the touch panel, the position and target determination unit 13 sets the position as a related position. Further, when an object in the image is designated by an operation of the display and operation unit 17, for example, a touch operation on the touch panel, the position and object determination unit 13 sets the object as a related object. Note that the position and target determination unit 13 can set the related position and the related target object not only for the captured image from the imaging unit 11 but also for the reproduced image reproduced from the recording unit 16. Target determination functions include face detection and character detection based on image patterns, and landscapes are far distances, so distance information etc. is reflected to determine objects such as mountains by sky or shape by color. May be. What can be determined in this way may be considered to be of interest to the user, and advice for prompting re-shooting may be given to the determined portion.

  In the related information storage unit 15, sub information is recorded. The sub information recorded in the related information storage unit 15 includes information in which the related position and the related object are already specified, and information in which the related position and the related object are not specified yet.

  In the association mode, the association control unit 14 processes the captured image from the imaging unit 11 as a main image or a sub image. This sub-image may be recorded in the related information storage unit 15 and used as sub-information. The association control unit 14 can also acquire various types of information obtained from the captured image from the imaging unit 11, such as character information, as sub-information. The association control unit 14 may cause the related information storage unit 15 to record the acquired sub information.

  The association control unit 14 can assign sub-information to each related position or related object in the main image and cause the recording / playback control unit 12 to record the information on the recording unit 16.

  Next, the operation of the embodiment configured as described above will be described with reference to FIGS. FIG. 6 is a flowchart for explaining the operation of the embodiment. FIG. 6 shows camera control.

  In step S1 of FIG. 6, the association control unit 14 determines whether or not the shooting mode is set. Assume that the shooting mode is instructed. In this case, the association control unit 14 controls the imaging unit 11 to start imaging in step S <b> 2 and captures a captured image from the imaging unit 11. The association control unit 14 determines a subject in the captured image that has been captured.

  In the next step S3, the association control unit 14 determines whether or not the association mode is set. When the association mode is not set, the association control unit 14 performs a shooting operation in the normal mode. That is, the association control unit 14 displays a through image in step S4, and determines whether or not shooting has been performed in step S5. When the user presses the shutter button 22a, the association control unit 14 performs imaging in step S6, performs predetermined signal processing on the captured image from the imaging unit 11, and then performs a captured image by the recording / playback control unit 12. Is recorded in the recording unit 16.

  Now, it is assumed that the association mode is designated by a user operation. In this case, the association control unit 14 determines in step S11 whether the main image shooting is specified or the sub-image shooting is specified. In the present embodiment, the main image may be taken first, and the sub image may be taken first.

  That is, in the present embodiment, a main image has already been captured and recorded, and a mode in which sub-images are recorded in association with the recorded main image for each related position or related object (hereinafter referred to as main-). Sub-association mode) and a mode in which a sub-image has already been captured and recorded, and the recorded sub-image is recorded in association with each associated position of the captured main image and each related object (hereinafter referred to as sub-main association). Mode). Steps S23 to S30 in FIG. 6 show the operation in the main-sub association mode, and steps S12 to S20 and S31 to S37 show the operation in the sub-main association mode.

  Now, assume that the user specifies the main-sub association mode. That is, it is assumed that the main image has already been photographed and recorded, and the user designates photographing of the sub image so that the sub image is associated with the recorded main image. In this case, the association control unit 14 shifts the processing from step S11 to step S21, reads the main image that has already been recorded from the recording unit 16, and provides it to the display and operation unit 17 for thumbnail display for a predetermined time. Let In the next step S22, the association control unit 14 determines whether or not the main image has been selected and operated.

  If the main image is not selected even after a predetermined time has elapsed, the association control unit 14 tries to take a sub image in the sub-main association mode, that is, to take a sub image without associating the user with the main image. Therefore, the process proceeds from step S22 to step S31.

  Now, since the user is trying to associate the sub image with the already recorded main image, the user performs an operation of selecting the main image. The association control unit 14 receives a touch position designation operation in step S23. For example, the user touches the display position of the image 25 in FIG. Thereby, the position and object determination unit 13 sets, for example, the lower left of the screen as the related position. Next, the association control unit 14 displays the captured image currently being captured by the imaging unit 11 and displays a through image on the display screen of the operation unit 17 (step S24). The user performs shooting by performing a shooting position and a zoom operation so that the related position and the related object in the main image are within the visual field range.

  The association control unit 14 determines whether or not a sub-image shooting operation has been performed in step S25. When the shooting operation is performed, the association control unit 14 controls the imaging unit 11 to perform shooting (step S26). This can be expressed as work shooting, and naturalness, hue, gradation expression, etc. are important. It is important to be able to read characters when shooting information etc. in sub-shooting, which will be described later, but here it is important to reflect the user's preference by skipping exposure or changing colors. Shooting is important. Of course, in order to perform character recognition or the like, a step of determining which part is characteristic during a through image or immediately after shooting may be provided. Such control may be changed to a sub-image to be described later not only at the time of recording but also at the time of the through image in S24. Next, the association control unit 14 determines whether or not the characters in the related position of the captured image can be recognized in step S27. If the character is recognizable, the association control unit 14 reads the character information to record the character as sub-information (step S28), and converts the read character into sub-information (step S29).

  Next, the association control unit 14 records the sub information including the captured sub image and the read characters in the image file of the selected main image together with the information on the related position and the related object (step S30). Further, not only one of the related position and the related object, but also the related position and the related object at the position may be determined, and both the related position and the related object may be recorded. For example, the center of the screen is set as the related position, and the fact that the object at that position is a real face is recorded. Thus, the image file shown in FIG. 4B is recorded in the recording unit 16.

  The sub-image shooting process in the sub-main association mode is performed in steps S31 to S37. The processes in steps S31 to S36 are the same as those in steps S24 to S29, except that the related position and the related object are not designated. In step S <b> 37, the association control unit 14 records the captured sub image in the related information storage unit 15.

  Next, it is assumed that the main image is shot in the sub-main association mode. In this case, the association control unit 14 proceeds from step S11 to step S12. In order to shoot the main image, the user performs shooting after setting the angle of view. For example, when the monument 33 in FIG. 3 has already been photographed as a sub-image, the main image is photographed at an angle of view including the monument 33, for example, as shown in FIG.

  The association control unit 14 determines whether or not a main image photographing operation has been performed in step S13. When the shooting operation is performed, the association control unit 14 controls the imaging unit 11 to perform shooting (step S14). In this shooting, it is important to make it easier to read characters and to detect faces, etc., because of the naturalness, hue, and gradation expression that were important at the time of main shooting (work shooting). You can make it higher. The shooting control includes exposure control, aperture control, and the like. However, this may be different from the main shooting to make it easier to read characters or to reduce blur. In addition, black and white and edge enhancement are image processing, and it is better to change the representation as auxiliary information from the main image by performing one or both of shooting control and image processing. Of course, such control may be changed to the main image not only at the time of recording but also at the through image of S12. Next, in step S15, the association control unit 14 displays a confirmation display (rec view) of the captured image on the display screen 23, and accepts designation of a related position and a related object (step S16). At this time, as described above, an image portion having a specific feature may be highlighted. In other words, face recognition and character recognition, whose detection technology has advanced, are representative of what the user is worried about, and these parts are displayed as determined, and the user is surrounded by a frame or color as a re-shooting candidate. Change the display or indicate with an arrow to display advice. If distance data can be acquired, the background may be detected to determine the sky, mountains, etc., and it may be interactively acquired such as which mountain the mountain is. At this time, in addition to a signpost and a destination display board, a hand-held map or the like may be photographed, or a guidebook or the like may be photographed so that only the region can be displayed. The user may not be able to think of shooting a guidebook, and may display advice such as “shoot a map or guidebook on hand”. Such display information may be recorded in the recording unit 16. The user may designate at which position in the main image the sub-image to be associated with the main image that has just been captured. For example, when the sub image is the image 43 of the monument 33 in FIG. 3 and the main image shown in FIG. 2 is captured, the user touches the lower left position on the screen. Thereby, the association control unit 14 specifies the related position and the related object.

  In step S <b> 17, the association control unit 14 displays an icon indicating a list of sub information stored in the related information storage unit 15 on the display and display screen 23 of the operation unit 17. The user designates the sub information by referring to this display.

The association control unit 14 determines whether or not the sub information is specified in step S18. If specified, the association control unit 14 adds the sub information to the image file of the main image and records it in step S19. If the sub information is not specified within the predetermined time, only the main image for which the related position or related object is specified is recorded (step S20).
FIG. 7 is a flowchart for explaining the operation in the reproduction mode.

  The association control unit 14 determines whether or not the playback mode is designated in step S41 of FIG. When the reproduction mode is designated, the association control unit 14 reads the main image that has already been recorded from the recording unit 16 and gives it to the display and operation unit 17 to display a list for a predetermined time in step S42. The association control unit 14 determines whether or not an image has been selected in the next step S43. When the image is selected, the association control unit 14 displays the selected image on the full screen (enlarged display) in step S44.

  In the present embodiment, the association control unit 14 determines whether or not the association shooting has been changed in step S45. In the present embodiment, the sub image can be taken even in the playback mode. When association shooting is instructed by the user, the association control unit 14 proceeds to step S46 and determines whether or not a touch position has been designated. The determination in step S46 is the same as the determination in step S23 in FIG. 6, and the process thereafter proceeds to step S24 in FIG.

  Thus, in the present embodiment, even in the main image playback mode, it is possible to specify a related position and a related object, and to newly shoot and store a sub-image.

  When the change to the association shooting is not instructed, the association control unit 14 shifts the process from step S45 to step S51, and determines whether the user has performed the process of returning from the reproduction mode to another mode. .

  Next, the association control unit 14 determines whether or not a touch operation for designating a related position or a related object has been performed in step S52 if the change to the related shooting is not performed in the playback mode. When the touch operation is performed, the sub information related to the related position and the related object specified by the touch operation is read from the related information storage unit 15 and given to the display and operation unit 17 for display.

  In this way, the user can display the sub-information on the related position and the related object in the main image on the display screen 23 of the display and operation unit 17. In addition, as in the example of FIG. 5, a plurality of sub information may be recorded for a predetermined related position or related object. In this case, the user can delete the sub information determined to be unnecessary. In step S54, the association control unit 14 receives selection of sub information to be deleted. When the sub information to be deleted is designated, the association control unit 14 deletes the sub information corresponding to the deletion operation from the related information storage unit 15.

  As described above, in the present embodiment, it is possible to specify the related position and the related object in the main image and assign and record the sub information. As the sub information, a captured image obtained by photographing or information acquired from the captured image can be recorded, and information desired by the user can be added and recorded in the main image in a form desired by the user.

(Second Embodiment)
FIG. 8 is a block diagram showing a second embodiment of the present invention. Moreover, FIG. 9 is explanatory drawing which shows the external appearance of the imaging device which concerns on 2nd Embodiment.

  This embodiment is applied to a tablet PC (personal computer) equipped with a lens type camera. A smartphone or a mobile phone may be employed instead of the tablet PC. As will be described later, an image captured by the lens-type camera 51 is displayed on the display unit 68 of the tablet PC 61.

  In FIG. 8, the tablet PC 61 is provided with a camera communication unit 62, and the lens type camera 51 is provided with a communication unit 55, and the tablet PC 61 and the lens type camera 51 include these communication units 62 and 55. Via the network.

  As shown in FIG. 9, an attachment device 71 is detachably attached to the housing 61 a of the tablet PC 61, and the attachment device 71 is provided with an attachment portion 72 for attaching the lens camera 51. Yes. The lens-type camera 51 is provided with a mounting portion 51b on the base end side, and the mounting portion 51b is attached to the mounting portion 72 of the mounting device 71 by fitting or screwing.

  In the present embodiment, the tablet PC 61 has a function of controlling the photographing and recording of the main image and the sub-image by controlling the lens-type camera 51. These functions may be provided on the lens-type camera 51 side.

  The lens-type camera 51 is provided with an imaging unit 52 having an optical system 52a. The imaging unit 52 is provided with an imaging element (not shown) configured by a CCD, a CMOS sensor, or the like, and a subject image is guided to the imaging surface of the imaging element by an optical system 52a. The optical system 52a includes a focus lens that is movable to set a focus (in-focus) state by focusing within the lens barrel 51a, a zoom lens that changes magnification in the focus state, and the like. The optical system 52a has a mechanism (not shown) that drives these lenses and the diaphragm. The control unit 53 controls the mechanism unit of the optical system 52a to drive and control these focus lens, zoom lens and diaphragm.

  The lens-type camera 51 is provided with an operation unit 54 such as an operation ring 54a for setting operation of parameters related to shooting by the user, for example, parameters related to focus, zoom, and aperture. The control unit 53 detects a user operation on the operation unit 54 and performs control based on the detection result. The control unit 53 includes a CPU and the like, and controls each unit of the lens-type camera 51 based on a user operation on the operation unit 54 and a signal from a tablet PC 61 described later.

  The imaging control unit 53c generates a focus signal, a zoom signal, and an aperture control signal, and drives and controls the focus, zoom, and aperture of the optical system 52a. In addition, the shooting control unit 53c supplies a drive signal to the image sensor to control the shooting of the subject.

  The image processing unit 53b is provided with a captured image from the image sensor and performs predetermined image signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing, and then the recording unit 56 Can be given and recorded. As the recording unit 56, for example, an IC memory can be employed. Further, the image processing unit 53 b can transfer the captured image to the tablet PC 61 via the communication unit 55.

  The lens information processing unit 53a can acquire information about the lens state such as the zoom lens and the focus lens, and the lens state such as the aperture state, and can transfer the information to the tablet PC 61 via the communication unit 55. Information on the lens includes information on the distance in the optical axis direction such as the focus position and the range of depth of field.

  The communication unit 55 can communicate with the camera communication unit 62 provided in the tablet PC 61 via a predetermined transmission path. As the transmission path, various wired and wireless transmission paths, for example, a wireless LAN transmission path such as a USB (Universal Serial Bus) cable or WiFi (Wireless Fidelity) can be employed. When the communication with the tablet PC 61 is established, the control unit 53 controls the shooting according to the control unit 65 of the tablet PC 61 and can transfer the captured image and information about the lens to the tablet PC 61. .

  The control unit 65 of the tablet PC 61 is configured by a CPU or the like, for example, and controls each unit of the tablet PC 61. The control unit 65 outputs a signal for driving the imaging unit 52 of the lens type camera 51 and receives a captured image from the lens type camera 51. The control unit 65 performs predetermined signal processing, for example, color adjustment processing, matrix conversion processing, noise removal processing, and other various signal processing on the received captured image.

  An operation unit 66 is also provided on the tablet PC 61. The operation unit 66 includes various operation units such as switches, keys, and software keyboards (not shown) provided in the tablet PC 61. The operation unit 66 generates operation signals based on user operations and outputs the operation signals to the control unit 65. Yes. The control unit 65 controls each unit based on the operation signal.

  The control unit 65 can perform processing related to recording and reproduction of captured images. For example, the control unit 65 can compress the captured image after the signal processing and give the compressed image to the recording unit 64 for recording. As the recording unit 64, for example, various recording media such as an IC memory can be adopted, and the recording unit 64 can record image information, audio information, and the like on the recording medium.

  The display control unit 67 executes various processes related to display. The display control unit 67 can receive the captured image after the signal processing from the control unit 65 and can provide it to the display unit 68. The display unit 68 has a display screen 68a (see FIG. 10) such as an LCD, and displays an image given from the display control unit 67. The display control unit 67 can also display various menu displays on the display screen 68a of the display unit 68. The control unit 65 can read out the captured image recorded in the recording unit 64 and perform expansion processing. The display control unit 67 can reproduce the recorded image by giving the expanded captured image to the display unit 68.

  On the display screen 68a of the display unit 68, a touch panel (not shown) is provided. The touch panel can generate an operation signal according to the position on the display screen 68a pointed by the user with a finger. This operation signal is supplied to the control unit 65. Thereby, when the user touches or slides on the display screen 68a, the control unit 65 arrives by the touch position of the user, the operation of closing and separating the finger (pinch operation), the slide operation or the slide operation. Various operations such as a position, a sliding direction, a touch period, and the like can be detected, and processing corresponding to a user operation can be executed. With this touch panel, it is possible to specify a related position or a related object.

  Note that the display unit 68 is arranged so as to occupy substantially the entire front surface of the tablet PC 61, for example, and the user captures a captured image displayed on the display screen 68 a of the display unit 68 when shooting with the lens type camera 51. It is possible to confirm, and a photographing operation can be performed while confirming the captured image.

  Furthermore, in the present embodiment, the tablet PC 61 includes a net connection unit 63. The network connection unit 63 is controlled by the control unit 65 and can be connected to an Internet line (not shown) wirelessly or by wire. The net connection unit 63 can access the site in accordance with a user operation of the operation unit 66, and send and receive data in accordance with the user operation to download data desired by the user. In the present embodiment, the data to be downloaded can be automatically acquired by the control unit 65 as well as designated directly by the user.

  The control unit 65 has the same functions as both the position and object determination unit 13 and the association control unit 14 in the first embodiment, and can operate in the association mode. For example, the control unit 65 sets the related position and the related object based on the touch operation of the user, and associates the lens with the set related position and the related object, as in the first embodiment. The sub information can be acquired from the captured image of the mold camera 51.

  Further, in the present embodiment, the information of the address book registered by the user may be recorded in the recording unit 64, and the control unit 65 is recorded in the information of the address book and the other recording unit 64. Can be used as sub-information.

  Further, in the present embodiment, the control unit 65 can record the data acquired by the network connection unit 63 in the recording unit 64 in association with the related position or the related object as sub information.

  10 and 11 are explanatory diagrams for explaining an example of downloading sub-information from the Internet.

  FIG. 10 shows how the user acquires the main image by shooting. As shown in FIG. 10, the user 81 takes a picture of a person 91 as a subject with the lens-type camera 51 while holding the housing 61a in which the circuit block of the tablet PC 61 of FIG. . The person 91 stands by the sign 93 displayed as “Performance” with the wall 92 as the background. A display screen 68a is provided on the back side of the housing 61a, and a through image 85 that is an image being captured is displayed on the display screen 68a. FIG. 10 shows that a captured image including a person 91, a wall 92, and a signboard 93 is displayed as a through image 85 within the shooting range.

  In this state, when the user 81 presses the shutter button 61b with the right hand 81a, the captured image displayed as the through image 85 is recorded. Now, for example, it is assumed that the captured image recorded by the user 81 is a main image.

  Also in this embodiment, it is possible to operate in the association mode in which the main image and the sub image are recorded in association with each other. In the association mode, the user 81 can designate that a sub-image is to be taken by designating a related position or related object in the image. Furthermore, as described above, in the association mode, it is also possible to acquire the sub information corresponding to the related position and the related object set in the image via the Internet line.

  Now, it is assumed that the user 81 acquires sub information from the Internet line in the association mode. For example, it is assumed that the user 81 performs an operation of touching the image 87 in the through image 85 corresponding to the signboard 93 with the left hand 81b. By this touch operation, the position of the image 87 corresponding to the signboard 93 in the main image is set as a related position, and the process proceeds to a sub information acquisition mode for acquiring sub information related to the related position.

  FIG. 11 shows how the sub information is acquired in the association mode. In the sub information acquisition mode, the character information of the related position is acquired by character recognition processing or the like for the image of the related position. Further, in the sub information acquisition mode, connection to the Internet line is made, and the information acquisition screen 95 is displayed on the display screen 68a. The user 81 holds the tablet PC 61 with the left hand 81b and touches the search button 96 displayed on the information acquisition screen 95 by the touch operation of the right hand 81a. Thus, a search is performed using the character information acquired from the related position as a key, and information related to the character information is acquired.

  For example, when the name of a performer is acquired as character information, profile information about the performer can be acquired as sub-information. Further, when the performance song name is acquired as the character information, the sound data of the performance song name or the like can be acquired as the sub information.

  FIG. 12 is an explanatory diagram for explaining an image file recorded in the association mode. As shown in FIG. 12, the image file includes image data 45 of the main image and tag information 46 that is auxiliary data. The tag information 46 includes information relating to the main image, information relating to the shooting of the main image, and the like, for example, subject face information, shooting location, shooting time information, and the like. The tag information includes sub-image assumption information which is information indicating that the sub-image related to the main image is recorded in association with the main image and is recorded in association mode. The image file also includes sub information 47 recorded in the association mode. As the sub information 47, each location information corresponding to the related position and the related object is recorded. For example, information such as URL information and related file names based on character information acquired from related positions and related objects is included.

  FIG. 13 is an explanatory diagram showing an example of sub information. FIG. 13 corresponds to FIG. 10 and FIG. Also in the present embodiment, the sub information is registered for each related position or related object specified by the user. The example of FIG. 13 shows an example in which sub-information is registered for each person, a person's face, and characters for a subject that is an object on the screen. A personal name is registered for the person. In addition, for a human face, an image of a face with a different expression is recorded. For characters, voice information, URL information, file names, and the like are recorded. The file name is for designating an audio file recorded in the recording unit 64. This audio file may be recorded in the recording unit 64 in advance, or may be acquired in the sub information acquisition mode described above.

  The user can present sub-information by touching the relevant position when reproducing the main image. FIG. 14 is an explanatory diagram for explaining this situation. As shown in FIG. 14, for example, when the sub information shown in FIG. 13 is recorded in the recording unit 64, the user 81 determines the position of the signboard image 98 in the reproduced image 97 displayed on the display screen 68a. By touching, it is possible to reproduce audio data which is sub-information with the position of the signboard image 97 as a related position. FIG. 14 shows that an acoustic output 99 based on the audio data is output.

  Next, the operation of the embodiment configured as described above will be described with reference to FIG. FIG. 15 is a flowchart for explaining the operation of the second embodiment. In FIG. 15, the same steps as those in FIG.

  Steps S23 and S71 to S77 in FIG. 15 show the operation in the main-sub association mode, and steps S12 to S20 and S81 to S87 show the operation in the sub-main association mode.

  Assume that the user specifies the main-sub association mode. That is, it is assumed that the main image has already been captured and recorded, and the user designates acquisition of the sub information so as to associate the sub information with the recorded main image. In this case, the control unit 65 shifts the process from step S11 to step S21, reads the main image that has already been recorded from the recording unit 16, and provides the display and operation unit 17 with thumbnail display for a predetermined time. . In the next step S22, the control unit 65 determines whether or not a main image has been selected and operated.

  If the main image is not selected even after a predetermined time has elapsed, the control unit 65 acquires the sub information in the sub-main association mode, that is, the user tries to acquire the sub information without associating it with the main image. It judges that it is a thing and transfers a process to step S81 from step S22.

  Now, since the user is trying to associate the sub information with the already recorded main image, the user performs an operation of selecting the main image. In step S23, the control unit 65 receives a touch position designation operation. For example, the user touches the display position of the image 87 in FIG. Accordingly, the control unit 65 sets the signboard image 87 as the related object. Next, the control unit 65 determines whether or not the sub information acquisition mode is designated (step S71). When the sub information acquisition mode is designated, the control unit 65 displays the information acquisition screen 95 shown in FIG. 11 in step S72.

  The user performs a search by operating a search button 96 on the information acquisition screen 95. The search key may be input by the user himself or may be set using a character string acquired by the control unit 65 from the image of the related object. The control unit 65 determines whether or not a search input has occurred in step S73, and if there is a search input, acquires the content and URL corresponding to the search input (step S74). That is, the control unit 65 controls the network connection unit 63 to connect to the Internet line, and downloads information from the corresponding site. The control unit 65 records the acquired sub-information together with the information on the related position and related object in the image file of the selected main image (step S77).

  In addition, when no net input occurs in step S73, the control unit 65 displays the captured image from the lens-type camera 51 as a through image as a sub-image as sub-information (step S75). In S76, processing such as shooting is performed. In this sub-image shooting, it is important that characters can be read, such as when shooting characters at the time of information shooting, and it may be black and white, and the contrast may be unnatural and easy to read. The shooting control includes exposure control, aperture control, and the like. However, this may be different from the main shooting to make it easier to read characters or to reduce blur. In addition, black and white and edge enhancement are image processing, and it is better to change the representation as auxiliary information from the main image by performing one or both of shooting control and image processing. Such control may be changed to the main image not only at the time of recording but also at the time of the through image in S75. Thus, in this case, the control unit 65 records the acquired sub-image together with information on the related position and related object in the image file of the selected main image (step S77).

  The sub information acquisition process in the sub-main association mode is performed in steps S81 to S87. The processing in steps S81 to S86 is the same as that in steps S71 to S76. In step S87, the control unit 65 records the acquired sub information in the recording unit 64.

  Other operations are the same as those in the first embodiment. In this embodiment, when sub-information is acquired, it is not limited to image information. Therefore, it is better to record music, it is better to record place names, or information acquisition according to image determination may be displayed as an advice. good. If the result of character recognition in the image is “concert” or “concert”, music recording may be prompted. This can be actually recorded or downloaded. When a unique mountain or river, sea shape or color, or wave motion is detected in the landscape portion, the sub-information may be used as the place name. In the case of a place name, map information linked to GPS may be acquired. Flowers and animals may be advised to work with the picture book. In these cases, advice is given by displaying text on the display unit or by voice.

  As described above, in this embodiment, the same effects as those in the first embodiment can be obtained, and not only the image to be taken and the information already recorded are used as sub-information, but also the information acquired from the Internet line is used. Sub-information can be used, and more effective information can be recorded.

  Furthermore, in each embodiment of the present invention, a digital camera has been described as an apparatus for photographing. However, the camera may be a lens type camera, a digital single-lens reflex camera, or a compact digital camera, a video camera, A camera for moving images such as a movie camera may be used, and a camera built in a personal digital assistant (PDA) such as a mobile phone or a smartphone may of course be used. In addition, it may be an industrial or medical optical device such as an endoscope or a microscope, such as a surveillance camera, an on-vehicle camera, a stationary camera, for example, a camera attached to a television receiver or a personal computer. There may be.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  It should be noted that even if the operation flow in the claims, the description, and the drawings is described using “first,” “next,” etc. for convenience, it is essential to carry out in this order. It doesn't mean. In addition, it goes without saying that the steps constituting these operation flows can be omitted as appropriate for portions that do not affect the essence of the invention.

  Of the technologies described here, many of the controls and functions mainly described in the flowcharts can be set by a program, and the above-described controls and functions can be realized by a computer reading and executing the program. it can. As a computer program product, the program may be recorded or stored in whole or in part on a portable medium such as a non-volatile memory such as a flexible disk or a CD-ROM, or a storage medium such as a hard disk or a volatile memory. It can be distributed or provided at the time of product shipment or via a portable medium or communication line. The user can easily realize the photographing apparatus of the present embodiment by downloading the program via a communication network and installing it on a computer, or installing the program on a computer from a recording medium.

    DESCRIPTION OF SYMBOLS 11 ... Imaging part, 12 ... Recording / reproducing control part, 13 ... Position and object determination part, 14 ... Association control part, 15 ... Related information storage part, 16 ... Recording part, 17 ... Display and operation part, 20 ... Association part.

Claims (8)

A recording control unit capable of recording a captured image of a subject as a main image;
A position and target determination unit for setting at least one of the related position and the related object in the main image;
Association control for acquiring sub information related to the related position and related object set by the position and target determination unit, controlling the recording control unit, and recording the main image and the sub information in association with each other An imaging device characterized by comprising: The association control unit controls the imaging unit to acquire an image related to the related position and the related object by performing image processing using a method different from the main image shooting control, as the sub information. The photographing apparatus according to claim 1. A display control unit for displaying the main image;
The photographing apparatus according to claim 1, wherein the position and target determination unit sets a position on the displayed main image and sets at least one of the related position and the related target object. . The photographing apparatus according to claim 3, wherein the display control unit displays a sub-image photographing advice during the main image photographing. The imaging device according to any one of claims 1 to 4, wherein the association control unit records the sub information in association with each of the related positions or the related objects. The imaging apparatus according to claim 1, wherein the association control unit acquires the sub information with respect to the main image that has already been captured. The imaging device according to any one of claims 1 to 5, wherein the association control unit assigns the already acquired sub-information to the main image obtained by imaging. Acquiring a captured image of a subject as a main image;
A position and target determination step for setting at least one of a related position and a related object in the main image;
A step of acquiring sub-information related to the related position and the related object set in the position and target determination step, and associating and recording the main image and the sub-information. Image recording method.