JP2013146025A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that, since it is necessary to reproduce a motion picture at least once or more in order to obtain data for retrieving a desired scene from massive motion picture data, when watching captured images for the first time, the desired scene cannot be retrieved.SOLUTION: An imaging apparatus comprising first imaging means and second imaging means is characterized in that information based on an image captured by the second imaging means which performs imaging simultaneously, is added to a captured image which is captured by the first imaging means. Furthermore, the added information is used for reproducing images captured by the first imaging means. Moreover, the information to be added is based on expressions of a photographer.

Description

  The present invention relates to an imaging apparatus capable of recording information relating to a photographer's facial expression together with an image, and capable of scene search using this information.

  Due to the increased capacity of recording media in imaging devices such as digital cameras and video cameras, it has become possible to shoot movies for long periods of time, making it difficult to search for specific images from a large number of recorded images. It has become. In particular, in order to find a desired scene when playing back a moving image, it is necessary to search while repeating fast forward and reverse, which is a difficult and time-consuming work.

  For this reason, in the prior art, there has been a moving image search support apparatus that determines the priority order of image data based on an operation history (see Patent Document 1). In this method, when reproducing a moving image, history data weighted by a reproduction method (Slow Scan, Play, Fast Scan, etc.) is created and used for search support. Therefore, it has been necessary to reproduce the moving image at least once in order to obtain data to be searched.

JP-A-5-54085

  However, in this method, since it is necessary to reproduce a moving image at least once in order to obtain data to be searched, when a photographed image is first viewed, a desired scene cannot be searched. . In addition, since the data is weighted depending on the reproduction method, a scene that is difficult to see that has been slowly reproduced many times may be set higher.

  If information for scene search can be added to a captured image at the time of shooting, it is not necessary to reproduce a moving image before performing a scene search, and time-consuming work is unnecessary.

  In addition, if information for scene search can be added according to the photographer's behavior, such as the photographer's facial expression, search information that takes human feelings into consideration, such as interesting content from the viewpoint of the photographer, is added to the photographed image. As a result, it becomes information that is easily understood by the person who searches.

  Therefore, in the present invention, in the imaging device having the first imaging means and the second imaging means, the photographed image photographed by the first imaging means was photographed by the second imaging means that photographed simultaneously. It is characterized by adding information based on an image. As a result, when searching for a desired scene from the captured images captured by the first image capturing unit, the photographer can search the memory based on the surrounding conditions captured by the second image capturing unit. This makes it easy to find a desired scene in a sensuous manner.

  Further, by using the added information at the time of reproduction of the image captured by the first imaging unit, the first imaging is performed when searching for a desired scene from the captured images captured by the first imaging unit. Although it cannot be photographed by the means, it is possible to easily find a desired scene by remembering the characteristic change in the surroundings photographed by the second imaging means.

  Furthermore, since the subject to be imaged by the second imaging means is a photographer, it is possible to search for a scene from the behavior of the photographer at the time of photographing. When it is desired to see a fun scene, it is possible to find a fun scene by searching for a scene whose attached photographer information is “behavior when fun”. Thus, the player can easily find a desired scene by specifying and searching for a fun scene, a sad scene, or the like. Further, since the information to be added is information based on the photographer's facial expression, the photographer can search for a desired scene by remembering what he / she felt at the time of photographing.

  In this way, even when the photographed image is first viewed, an operation of searching for a desired scene can be performed sensuously according to the player's desire for what kind of scene he / she wants to see.

  As described above, according to the present invention, it is possible to easily find a desired scene when searching for a desired scene from a captured image. Since the photographer can easily find the desired scene by remembering what he / she felt, the time required to search for the desired scene while repeating fast forward and reverse operations as before. Takes up and eliminates the need to spend boring time. Also, a player who first views a photographed image can easily find and reproduce the scene he / she wants to see by specifying and searching for a fun scene, a sad scene, or the like. For this reason, it is not necessary to look through all of the images, and only a desired scene can be viewed in a short time. In this way, a desired scene can be easily and intuitively searched, and the operability of the apparatus is improved.

The block diagram which shows the structure of the video camera in the 1st Embodiment of this invention. FIG. 2 is an external perspective view of the video camera according to the first embodiment of the present invention as viewed from the front side (subject side) right side. The external appearance rear view seen from the back (photographer side) of the video camera in the 1st Embodiment of this invention Schematic showing the mode at the time of imaging | photography with the video camera in the 1st Embodiment of this invention. The flowchart which shows the process sequence at the time of the video recording of the video camera in the 1st Embodiment of this invention. Flowchart of additional information generation processing of the video camera in the first embodiment of the present invention Table showing additional information of the video camera according to the first embodiment of the present invention The flowchart which shows the process at the time of the image reproduction of the video camera in the 1st Embodiment of this invention Flowchart of playback mode screen creation processing of the video camera in the first embodiment of the present invention Playback mode screen of the video camera in the first embodiment of the present invention The selection unit is highlighted on the playback mode screen of the video camera according to the first embodiment of the present invention. The figure showing an example of the file format of the recording information in the 1st Embodiment of this invention

  Hereinafter, embodiments for carrying out the present invention will be described.

<First Embodiment>
FIG. 1 is a block diagram showing a configuration of a video camera according to an embodiment of the present invention. 2 is an external perspective view of the video camera according to the first embodiment of the present invention as viewed from the front (subject side) right side, and FIG. 3 is an external rear view of the video camera as viewed from the rear (photographer side). 4 is a schematic diagram showing a state during photographing.

  As shown in FIG. 1, the video camera main body 100 includes an optical lens group including an optical control system such as focus and zoom, a lens 101 including a diaphragm, a sub lens 124, an imaging unit 102 including an imaging element such as a CCD, and a sub imaging. The main lens 101 corresponds to the main imaging unit 102, and the sub lens 124 corresponds to the sub imaging unit 125.

  In addition, the video camera 100 includes a camera signal processing unit 103, a compression / decompression processing unit 104, a display unit 105, a recording / playback processing unit 106, a card control unit 107, a tape control unit 108, a microphone 109, an audio signal processing unit 110, a memory. 111, a memory control unit 112, a CPU 113, an operation unit 114, a digital I / F 115, an input / output unit 116, and a power supply control unit 117.

  The functional blocks are connected to each other via a data bus 118 so that data communication is possible.

  In such a video camera main body 100, the memory 111 has a program storage area and a data work area, and is used by time sharing in each functional block via the data bus 118, and is controlled by the memory control unit 112. It is managed.

  The CPU 113 is a central processing circuit that performs overall control of the entire video camera main body 100 by executing a predetermined program.

  The operation unit 114 is an operation key such as 114a, 114b, 114c, 114d, 114e, 114f, 114g, 114h, 114i, 114j, 114k, 114l, 114m, 114n, 114o, 114p, 114q, 114r in FIGS. And has a function of instructing the CPU 113 to perform an operation based on a key operation from the user.

  Here, the function of each operation key will be described with reference to FIGS.

  114a, 114b, 114c, and 114d are playback operation buttons, 114a is a reverse button, 114b is a fast forward button, 114c is a playback / pause button, and 114d is a stop button.

  114e is a digital effect enable / disable button, 114f is a continuous shooting function setting button, and 114g is a self-timer setting button.

  114h is a zoom key for performing a zoom operation to change the angle of view of the lens to the wide side or the tele side, and 114i is a still image shooting button for recording a still image, which is a two-stage switch that can be pressed halfway and fully It is.

  114j is a menu button for switching the menu display ON / OFF, 114k is a menu key for selecting and determining a menu, 114l is a white balance setting button for setting white balance at the time of shooting, and 114m is auto / auto at the time of shooting. A focus setting button 114 n for setting manual focus selection, and an exposure setting button 114 n for setting exposure at the time of shooting.

  Reference numeral 114o denotes a mode dial. By rotating the dial, it is possible to set program automatic exposure in accordance with a photographed image such as a fully automatic mode, a shutter speed priority mode, and an aperture priority mode.

  Reference numeral 114p denotes a power dial, which can be switched between power-off, shooting mode, and playback mode by rotating.

  Reference numeral 114q denotes a moving image shooting button, which can start / stop moving image shooting by pressing.

  A recording medium switching lever 114r can select a recording medium for recording an image by rotating.

  The digital I / F 115 sends and receives signals in accordance with standards such as IEEE 1394 and USB.

  The memory card 119 is a detachable flash memory in which shooting data and audio data are recorded.

  The power supply control unit 117 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like, and detects whether or not a battery is installed, the type of battery, and the remaining battery level. The battery 123 is a lithium ion battery or the like, and supplies power to the video camera body 100.

  The additional information generation processing unit 126 includes a facial expression detection unit, a facial expression identification unit, and a facial expression information storage unit. Based on a method disclosed in Japanese Patent Laid-Open No. 6-76058, the additional information generation processing unit 126 identifies the facial expression of a person who has passed through the sub imaging unit 125. , Generate additional information. The facial expression detection means extracts facial feature parts from the photographed image and extracts facial expression elements. The facial expression identification means calculates a facial expression element code from the extracted facial expression element, and identifies the facial expression from the facial expression element code. An identification code corresponding to each facial expression is stored in the facial expression information storage unit, and an identification code corresponding to the identified facial expression is added to the captured image as additional information.

  Next, the shooting operation, recording operation, and playback operation of the video camera in the first embodiment of the present invention will be described. For example, when a photographing operation and a recording operation are instructed by the operation unit 114, light from the subject of the main imaging unit 102 becomes an optical signal whose predetermined brightness, angle of view, focus, and the like are controlled by the lens 101. Are input to the imaging unit 102. The imaging unit 102 converts the input optical signal into an electrical signal and outputs it to the camera signal processing unit 103. The camera signal processing unit 103 converts this electric signal into a digital image signal. The image signal is subjected to signal processing such as color separation, gradation correction, and white balance adjustment, and then output. The audio signal processor 110 outputs the audio signal as audio data by gain-controlling the audio signal input from the microphone 109 to a predetermined level and digitizing it.

  On the other hand, in the sub imaging unit 125, the photographer's image is input to the sub imaging unit 125 as an optical signal in which predetermined brightness, field angle, focus, and the like are controlled by the sub lens 124. At the time of shooting, as shown in FIG. 4, the photographer 401 grips the imaging apparatus main body 100 and shoots the subject 402 with the main lens. In such a case, the sub lens 124 is arranged at a place where the photographer 401 can be photographed.

  The sub imaging unit 125 converts the input optical signal into an electrical signal and outputs it to the camera signal processing unit 103. The camera signal processing unit 103 converts this electric signal into a digital image signal. If there is a human face in the image signal, the additional information generation processing unit 126 extracts facial feature parts, extracts facial expression elements, calculates facial expression element codes, identifies facial expressions, An identification code corresponding to the identified facial expression is output as additional information.

  The compression / decompression processing unit 104 compresses and encodes the image data output from the camera signal processing unit 103 and the audio data output from the audio signal processing unit 110 using a predetermined compression encoding method. The display unit 105 includes a liquid crystal display unit 105a and an EVF 105b shown in FIGS. 2 and 3, and the compression / decompression processing unit 104 displays shooting data to be compressed and encoded. The display unit 105 displays the operation status of the video camera body 100 as on-screen display information as necessary.

  The recording / playback processing unit 106 performs error correction coding processing, modulation processing, and the like on the data after compression coding obtained by the compression / decompression processing unit 104, and further adds data such as synchronization and ID to record. Then, the data is recorded on a recording medium such as the tape 120 or the memory card 119. At this time, the tape control unit 108 performs recording control of the tape 120, and the card control unit 107 performs recording control of the memory card 119.

  On the other hand, when a reproduction operation is instructed by the operation unit 114, the recording / reproduction processing unit 106 detects original digital data from data read from the tape 120 or the memory card 119, and performs processing such as error correction and demodulation. Further, a clock synchronized with the reproduction data is generated by a PLL circuit.

  The compression / decompression processing unit 104 performs decompression processing corresponding to a predetermined compression encoding method on the data output from the recording / playback processing unit 106, and performs shooting data and audio data before compression encoding and additional information generation processing The additional information generated by the unit 126 is acquired. The display unit 105 displays the reproduction data obtained by the compression / decompression processing unit 104.

  When the operation unit 114 instructs to record input data from the digital I / F 115, the CPU 113 inputs image data, audio data, or additional information from the external device 121 in a digital signal state by the digital I / F 115. Then, the data is sent to the recording / reproducing processor 106. At this time, it is assumed that the image data and audio data from the external device 121 have already been encoded, and the recording / playback processing unit 106 performs processing necessary for recording on the input data as described above to perform the tape 120 or the like. Recording is performed on a recording medium such as a memory card 119.

  When the operation unit 114 instructs to record an input signal from the input / output unit 116, the CPU 113 controls the input / output unit 116 and inputs an image and audio signal output from the external device 122. The input / output unit 116 converts the image and audio signals output from the external device 122 into digital data and outputs the digital data to the memory 111. The CPU 113 outputs the image, audio data, and additional information stored in the memory 111 to the compression / decompression processing unit 104, compression-codes them, and sends them to the recording / reproduction processing unit 106. As described above, the recording / playback processing unit 106 performs processing necessary for recording on the encoded data and records it on a recording medium such as the tape 120 or the memory card 119.

  Next, additional information addition processing at the time of shooting and use of additional information at the time of image reproduction, which are features of the first embodiment of the present invention, will be described with reference to the drawings. In this imaging apparatus, information can be added to the image captured by the main imaging unit 102 based on the image captured by the sub imaging unit 125, and reproduction can be performed based on this additional information during reproduction. .

  FIG. 5 is a flowchart showing a processing procedure during moving image shooting according to the first embodiment of the present invention.

  First, when the user shoots a moving image, the shooting mode is selected by the power dial 114p.

  As described above, when the shooting mode is selected, the real-time through image of the captured moving image is displayed on the liquid crystal display unit 105a or the EVF 105b in step S501, and the process proceeds to step S502.

  In step S502, it is determined whether or not the moving image shooting button 114q has been pressed. If it is determined that the moving image shooting button 114q has been pressed, the process proceeds to step S503.

  In step S503, the recording process is started, and the process proceeds to step S504.

  In step S504, additional information is generated based on the information obtained from the sub imaging unit 125.

  Here, the additional information generation process will be described in detail. The additional information generation processing unit identifies the facial expression of the person imaged by the sub imaging unit 125 based on a method as disclosed in JP-A-6-76058, and uses it as additional information. FIG. 6 is a flowchart for explaining the additional information generation processing in the first embodiment of the present invention.

  First, it is determined whether or not there is a face in the image information obtained from the sub imaging unit 125 in step S601. If there is, the process proceeds to step S602.

  In step S602, a characteristic part is extracted from the detected face, and the process proceeds to step S603.

  In step S603, facial expression elements are extracted from the extracted feature parts, and the process proceeds to step S604.

  In step S604, a facial expression is determined based on the facial expression elements extracted in step S603, and the process proceeds to step S605.

  In step S605, additional information corresponding to the facial expression determined in step S604 is generated. At this time, if no face is detected in step S601, the process advances to step S605 to generate additional information corresponding to “no person”. The information to be added is classified into several facial expressions as shown in FIG. 7, and facial expression ID 701 is used as the additional information.

  If additional information is generated in step S605, the process proceeds to step S505.

  In step S505, recording information is created based on the additional information generated in step S504 and the information obtained from the main imaging unit 102.

  In step S506, it is determined whether or not the moving image shooting button 114q is pressed. If the moving image shooting button 114q is not pressed, the process returns to step S504. On the other hand, if it is determined that the moving image shooting button 114q has been pressed, the process proceeds to step S507.

  In step S507, the moving image recording ends, and the present process ends.

  As shown in FIG. 12, the recorded information has a header, a thumbnail, a file number, and other additional information stored therein, and is combined with an image or audio file.

  Next, processing during image reproduction in the first embodiment of the present invention will be described with reference to the drawings.

  First, at the time of image reproduction, the reproduction mode is selected by the power dial 114q. As described above, when the reproduction mode is selected, a reproduction mode screen is created on the liquid crystal display unit 105a or the EVF 105b in step S801. Here, the reproduction mode screen creation S801 will be described in detail. FIG. 9 is a flowchart for explaining the playback mode screen creation processing in the first embodiment of the present invention.

  First, in step S901, it is determined whether a facial expression ID 701, which is additional information, is added to the image. If the facial expression ID 701 is added, the process proceeds to step S902.

  In step S902, the facial expression ID 701 added to the image is read, and the process proceeds to step S903.

  In step S903, the display character 702 or the display indicator 703 corresponding to the facial expression ID 701 is read, and the process proceeds to step S904.

  In step S904, a thumbnail image is read if the image is a still image, and a time code is read if the image is a moving image, and the process advances to step S905. If the expression ID 701 is not added to the image in step S901, the process proceeds directly to step S904.

  In step S905, a playback mode screen is created from the display character 702 or display indicator 703 read in step S903 and the thumbnail image or time code read in step S904, and the process returns to step S801.

  In step S802, the reproduction mode screen created in step S801 is displayed.

  The playback mode screen has a form as shown in FIG. 10, for example, a still image is represented by a thumbnail image 1001 and display characters 702 as shown in FIG. 10A, or a still image is displayed as a thumbnail as shown in FIG. 10B. It may be represented by an image 1001 and a display indicator 703. When the image is a moving image, additional information such as a display character 702 and a display indicator 703 is displayed in accordance with the time code 1002 of the moving image, as shown in FIG. Although the additional information may be displayed at regular intervals, if the additional information is changed as shown in FIG. 10C, the display does not have to be complicated.

  In step S803, the user selects an image to be reproduced using the operation unit 114k. At this time, as shown in FIG. 11, it is easy to understand if the display character 702 and the display indicator 703 that are selected are distinguished from each other like the highlight portion 1101.

  When the image or scene is selected in step S803, the process proceeds to step S804. When the playback button is pressed, the process proceeds to step S805, where the selected image or scene is reproduced, and this process ends.

  As described above, in the imaging apparatus according to the present invention, information can be added to the image captured by the main imaging unit based on the image captured by the sub imaging unit, and reproduction is performed based on this additional information during reproduction. It is possible. Thus, the photographer can select a scene to be reproduced sensuously by remembering what he / she felt at the time of shooting. For example, “At that time, I was laughing when I saw a child playing in the mud.” “After that, an unknown person crossed the front and got angry.” It is possible to replay it when you remember what you felt. If you want to replay a scene that was playing in the muddy area, you can search for a place where the additional information of “Laughter” arrives and then the additional information of “anger”. On the other hand, if an operator who plays an image for the first time wants to see only a fun scene, he / she only needs to select and play the “laugh” or “lol” scene. With this method, it is possible to select the location you want to watch from long videos by reducing the complicated and time-consuming operations of searching for the desired location while repeating fast-forwarding and rewinding in a long movie. It becomes possible to feel it, and the operability of the imaging apparatus is improved.

DESCRIPTION OF SYMBOLS 100 Video camera main body 101 Main lens 102 Main imaging part 103 Camera signal processing part 104 Compression / decompression processing part 105 Display part 106 Recording / reproduction processing part 107 Card control part 108 Tape control part 109 Microphone 110 Audio | voice signal processing part 111 Memory 112 Memory control part 113 CPU
114 Operation unit 115 Digital I / F
116 Input / output unit 117 Power supply control unit 118 Bus 124 Sub lens 125 Sub imaging unit 126 Additional information generation processing unit

Claims (4)

  In an imaging apparatus having a first imaging means and a second imaging means, information based on an image photographed by the second imaging means that was photographed at the same time as a photographed image photographed by the first imaging means. An imaging device characterized by adding   2. The imaging apparatus according to claim 1, wherein the added information is used at the time of reproducing an image photographed by the first imaging means.   The imaging apparatus according to claim 1, wherein an imaging target of the second imaging unit is a photographer.   The imaging apparatus according to claim 3, wherein the information to be added is information based on a photographer's facial expression.

Images