JP6351260B2 - Imaging apparatus and control method thereof - Google Patents

Description

  The present invention relates to an imaging apparatus having a plurality of imaging means and a control method thereof.

  Like a mobile phone or a smart phone, it has a photographing unit (hereinafter referred to as an in-camera) that is directed to the user side such as a cellular phone in the opposite direction to an imaging unit for normal photographing (hereinafter referred to as an out-camera). There is an imaging device. In such an imaging device, a single release operation calls the out camera and simultaneously releases the in-camera shutter to capture multiple images, and associates the in-camera image with the out-camera image. It is possible to record. (See Patent Document 1)

JP-A-2005-94741

  In the case of an imaging apparatus as described in Patent Document 1, shooting with an out-camera and shooting with an in-camera are performed simultaneously according to a release operation. Therefore, only the photographer's facial expression at the moment of photographing the subject could be left. Also, only a still image is taken with the in-camera according to the release operation. For this reason, changes in the photographer's facial expression and reaction at the time of photographing, and the photographer's reaction, facial expression, and emotion when viewing the actually photographed image could not be recorded.

  Therefore, an imaging apparatus having a plurality of imaging units is provided that can record a situation at the time of still image shooting, for example, a change or reaction of a person other than a subject of a still image such as a photographer. With the goal.

In order to solve the above problems, the imaging apparatus of the present invention
A first imaging means for photographing a subject; a second imaging means for photographing a photographer;
A shooting instruction means for instructing shooting of a still image by the first image pickup means, and a still image shot by the first imaging means in response to an instruction for shooting of a still image by the first image pickup means by the shooting instruction means. A moving image shot by the first imaging unit before shooting a still image, a moving image shot by the second imaging unit before shooting a still image, and a second shot after shooting the still image. And a control unit that controls to record a moving image shot by the imaging unit on a recording medium and controls not to record a moving image by the first imaging unit after shooting a still image .

  According to the present invention, in the imaging apparatus having the first imaging unit and the second imaging unit, when a still image shooting instruction is given by the first imaging unit, a moving image shot by the second imaging unit is displayed. Since it is recorded, it is possible to record changes in the situation at the time of photographing, changes in facial expressions other than the subject of the first imaging means such as a photographer, and reactions.

It is a rear view of a digital camera. It is a block diagram which shows the structure of a digital camera. It is a flowchart which shows the imaging | photography process of still image and moving image simultaneous imaging | photography mode. It is a flowchart which shows the imaging | photography process of still image and moving image simultaneous imaging | photography mode. It is a figure which shows the example of the image image | photographed and displayed in a still image and moving image simultaneous imaging mode. It is a figure for demonstrating a moving image recording pattern. It is a figure for demonstrating the relationship between various settings and a moving image recording pattern.

  Preferred embodiments of the present invention will be described below with reference to the drawings.

(First embodiment)
In this embodiment, a digital camera 100 will be described as an example of an imaging apparatus according to the present invention.

<Appearance of digital camera 100>
FIG. 1 is a rear view of a digital camera 100 as an example of an imaging apparatus according to the present invention.

  The display unit 28 is a display unit that displays images and various types of information. The shutter button 60 is an operation unit for issuing a still image shooting instruction. The moving image shooting button 70 is an operation member for giving a moving image shooting instruction. The operation unit 62 is an operation unit including operation members such as various switches and buttons for receiving various operations from the user. The operation unit 62 includes a power switch for switching the power of the digital camera 100 on and off, a mode switch for switching the operation mode of the digital camera 100, a menu button for setting menus, a cross key button for instructing menus and settings, and the like. ing. The recording medium 91 is a recording medium such as a memory card or a hard disk. A lid 93 is a lid for storing the recording medium 91.

  Reference numeral 104 denotes a camera module for photographing a photographer who is photographing a subject or photographing a viewer who is viewing an image displayed on the display unit 28. Etc. Hereinafter, this camera is referred to as an in-camera.

  The in-camera 104 is arranged to photograph the side of the digital camera 100 where the display unit 28 is provided in order to photograph the viewer or the photographer. 1 is an external view of the back side of the digital camera provided with the display unit 28. On the front side surface of the digital camera 100, an out camera that is an imaging unit for photographing the subject side is provided. ing. The optical axis (imaging direction) of the out camera and the sub camera is substantially parallel to each other, and the optical axis (imaging direction) of the out camera is set to be opposite to the optical axis (imaging direction) of the sub camera.

<About the configuration of the digital camera 100>
FIG. 2 is a block diagram illustrating a configuration of the digital camera 100 according to the embodiment of the present invention.

  A protection unit 10 is a barrier that prevents the imaging unit from being soiled or damaged by covering the imaging unit including the photographing lens 11 of the digital camera 100. 11 is a photographing lens, 12 is a shutter having a diaphragm function, 13 is an image sensor composed of a CCD or CMOS element for converting an optical image into an electric signal, and 15 is an analog signal output of the image sensor 13 for converting it into a digital signal. An A / D converter for acquiring image data. The barrier 10, the taking lens 11, the shutter 12, and the image sensor 13 constitute an out camera.

  Similarly, the photographic lens 101 is a lens group including a zoom lens and a focus lens that are installed in a direction in which a photographer who uses the digital camera 100 can be photographed separately from the photographic lens 11. Reference numeral 102 denotes a shutter having an aperture function. Reference numeral 103 denotes an image pickup device composed of a CCD or CMOS device that converts an optical image on the photographer side into an electric signal. The photographing lens 101, the shutter 102, and the image sensor 103 constitute an in-camera.

  A timing generation circuit 14 supplies a clock signal and a control signal to the imaging elements 13 and 103, the A / D converter 15, and the D / A converter 21, and is controlled by the memory control circuit 22 and the system control unit 50.

  For the in-camera, it is possible to set whether to enable or disable the in-camera shooting function. When the in-camera shooting function is set to be effective, an image generated based on the data read from the image sensor 103, that is, an image shot by the in-camera is recorded in the external recording medium 91. When the in-camera shooting function is disabled, the image sensor 103 is not driven, that is, shooting with the in-camera is not performed. Alternatively, even if imaging is performed by driving the imaging element 103, data is not read from the imaging element 103, and recording of an image based on the output of the imaging element 103 to the external recording medium 91 is not performed.

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 15 or the data from the memory control circuit 22.

  Further, the image processing circuit 20 performs predetermined arithmetic processing using the captured image data. Then, based on the obtained calculation result, the system control unit 50 controls the exposure control unit 40 and the distance measurement control unit 41 to perform AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash pre-processing). (Emission) processing.

  The image processing circuit 20 performs predetermined arithmetic processing using the captured image data, and also performs AWB (auto white balance) processing based on the obtained arithmetic result.

  Further, the image processing circuit 20 reads an image stored in the memory 25 to read the JPEG format or MPEG-4 AVC / H. H.264 compression or expansion processing is performed, and the processed data is written into the memory 25.

  A memory control circuit 22 controls the A / D converter 15, the timing generation circuit 14, the image processing circuit 20, the image display memory 24, the D / A converter 21, and the memory 25.

  The data of the A / D converter 15 is written to the image display memory 24 or the memory 25 via the image processing circuit 20 and the memory control circuit 22, or the data of the A / D converter 15 is directly written via the memory control circuit 22. It is.

  Reference numeral 24 is an image display memory, and 21 is a D / A converter. Reference numeral 23 denotes a display unit including a TFT, an LCD, and the like, and performs display based on display image data written in the image display memory 24 via the D / A converter 21. If the image data captured using the display unit 28 is sequentially displayed, an electronic viewfinder function for displaying live video can be realized. As described above, the display unit 28 is also used to display a captured image and a graphic user interface for changing menus and settings. The display unit 28 also includes a terminal for outputting a video signal to an external display device such as a TV. The system control unit 50 may not only perform display control on the display unit 28 but also display control so that a captured image or a graphic user interface is displayed on the external display device.

  Reference numeral 25 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity to store a predetermined number of still images and a predetermined time of moving images. The memory 25 can also be used as a work area for the system control unit 50.

  Reference numeral 40 denotes exposure control means for individually controlling the shutters 12 and 102 having an aperture function.

  Reference numeral 41 denotes a distance measuring control means for individually controlling the focusing of the photographing lenses 11 and 101, reference numeral 42 denotes a zoom control means for individually controlling the zooming of the photographing lenses 11 and 101, and reference numeral 43 denotes a barrier control means for controlling the operation of the barrier 10. is there.

  A system control circuit 50 controls the entire digital camera.

  Reference numeral 51 denotes an electrically erasable / recordable nonvolatile memory such as an EEPROM. The nonvolatile memory 51 stores programs, various setting information, and the like.

  Reference numeral 60 denotes a shutter button. During the operation of the shutter button 60 (half-pressed state), the shutter switch SW1 is turned on, and the SW1 signal is input to the system control unit 50. When the SW1 signal is input, the system control unit 50 starts shooting preparation operations such as AF (auto focus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, and EF (flash pre-flash) processing. Instruct. When the operation of the shutter button 60 is completed (fully pressed), the shutter switch SW2 is turned on and the SW2 signal is input to the system control unit 50. When the SW2 signal is input, the system control unit 50 instructs the start of a still image shooting and recording operation by the out camera. When the start of still image shooting and recording operation is instructed, first, an exposure process is performed in which a signal read from the image sensor 13 is written to the memory 25 via the A / D converter 15 and the memory control circuit 22. Is called. Then, the image data written in the memory 25 is subjected to a development process using a calculation in the image processing circuit 20 or the memory control circuit 22, and the image data subjected to the development process is processed by the image processing circuit 20. The data is compressed and stored in the memory 25. Thereafter, the compressed image data is read from the memory 25, and the image data is written to the external recording medium 91 via the card controller 90. In this way, a series of still image shooting and still image recording operations of shooting and recording of still image data are performed in response to the input of the SW2 signal.

  An operation unit 62 includes various buttons and a touch panel for inputting various operation instructions of the system control unit 50. A power button, a menu button, a mode changeover switch for switching between a shooting mode / playback mode / other special shooting modes, a cross key, a set button, and the like. Also included are a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and the like. Furthermore, menu movement + (plus) button, menu movement-(minus) button, playback image movement + (plus) button, playback image-(minus) button, shooting image quality selection button, exposure compensation button, date / time setting button, etc. Is also included. Further, a voice recognition device or the like may be used, and the above buttons and touch panel are configured by a single or a plurality of combinations.

  Reference numeral 70 denotes a moving image shooting button. When the moving image shooting button is operated (pressed), the system control unit instructs to start shooting a moving image by the out-camera. When the start of moving image shooting is instructed, a signal is read from the image sensor 13, the signal is converted by the A / D converter 15 to acquire image data, and the moving image compression processing is performed by the image processing circuit 20, and the external recording medium 91. To remember. The moving image is recorded on the external recording medium 91 by periodically performing the moving image compression process and the recording process. This moving image compression processing and recording processing are executed until the moving image shooting button 70 is pressed again. When the moving image shooting button 70 is pressed again, the moving image compression process is terminated, and the moving image recorded so far is recorded in the external recording medium 91 as one moving image file.

  Reference numeral 81 denotes power supply control means.

  Reference numeral 80 denotes a power source including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter.

  A card controller 90 transmits and receives data to and from an external recording medium such as a memory card. Reference numeral 91 denotes an external recording medium such as a memory card.

<Still image / video recording simultaneous recording mode>
The digital camera 100 has a still image / moving image recording simultaneous recording mode as a still image shooting mode. In the still image / video shooting simultaneous recording mode, a still image is shot and recorded by the out-camera in response to the shutter button 60 being fully pressed, and a moving image shot by the out-camera is also recorded on the external recording medium. 91. In the still image / video recording simultaneous recording mode, a video (short movie) is automatically recorded every time a still image is shot, and a video (short movie) recorded in the still image / video simultaneous recording mode is recorded each time. Recorded in the same movie file. Therefore, it is possible to automatically create a video that captures the situation at the time of still image shooting without any user operation, and to continuously create a video (short movie) obtained by multiple still image shootings. It is possible to obtain a single digest video that can be appreciated. Hereinafter, a moving image file in which a moving image (short movie) is recorded in the still image / moving image recording simultaneous recording mode is referred to as a digest moving image file. The still image / moving image recording simultaneous recording mode can be set by the user switching the still image shooting mode setting with the mode switch included in the operation unit 62.

  In the still image / moving image recording simultaneous recording mode, the in-camera shooting function can be set to be valid or invalid as described above. By enabling the in-camera shooting function, it is possible to superimpose and record an image shot by the in-camera on an image shot by the out-camera. The setting of the in-camera shooting function can be set to be valid or invalid by the user operating the operation unit. Enables the user to set on / off display of images taken with the in-camera instead of setting the in-camera shooting function, and enables the in-camera shooting function when the in-camera image display is set to on The in-camera shooting function may be disabled when the display is off.

  Further, in the still image / moving image simultaneous recording mode, the user can set stop motion on / off by operating the operation unit. When the stop motion setting is set to on, a stop motion is created by converting the shot still image to a movie, and the created stop motion is recorded in the movie file for the still image / movie shooting simultaneous recording mode. The That is, a stop motion is recorded following a moving image (short movie) before still image shooting. If the stop motion setting is set to off, no stop motion is generated or recorded. In addition, when the stop motion setting is set to on and the in-camera shooting function is enabled, the combined motion picture shot with the in-camera is combined with the stop motion of the shot still image. A movie is recorded.

  When the still image / movie shooting simultaneous recording mode is set and the movie shooting button 70 is pressed, movie shooting and recording processing by the out-camera is started, and still image shooting is not performed. I will not. If the in-camera shooting function setting is enabled, the movie recorded at this time will be combined and recorded when the in-camera shooting function setting is disabled. Only videos shot with the camera are recorded.

  That is, when the still image / movie shooting simultaneous recording mode is set, when a still image shooting instruction is issued by the shutter button 60, a still image and a movie are recorded, but a movie shooting instruction is issued by the movie shooting button 70. Even if there is a movie, the movie is recorded but the still image is not recorded.

<Shooting processing in the simultaneous recording mode for still images and moving images>
FIGS. 3 and 4 describe the shooting operation of the digital camera 100 in the still image / moving image recording simultaneous recording mode. Note that these processes are realized by performing control of each unit, arithmetic processing, determination, and the like based on a program read from the nonvolatile memory 51 by the system control unit 50.

  First, when the still image / moving image recording simultaneous recording mode is selected in the digital camera 100, a display image is generated based on the output of the image sensor 13 with proper exposure controlled by the exposure control 40 in step S301. That is, the display image is generated based on the main image taken by the out camera. Hereinafter, an image created from the output of the imaging device 13 of the out-camera is referred to as a main image. The generated display image is temporarily stored in the memory 25.

  In step S302, a moving image recording image (main moving image) is generated based on the main image that is the output of the imaging device 13 of the out-camera. Hereinafter, a moving image created from the output of the imaging device 13 of the out-camera is referred to as a main moving image. The generated moving image recording image is temporarily stored in the memory 25.

  In step S303, the settings related to the in-camera set in the digital camera 100 are confirmed. If the in-camera shooting function is set to be valid, the process proceeds to step S304, and the processes of steps S304 to S308 are performed. If the in-camera shooting function is set to be invalid, the process proceeds to step S309 without performing steps S304 to S308.

  In step S <b> 304, settings related to the display position and display size of the in-camera set in the digital camera 100 are acquired from the nonvolatile memory 51. In the present embodiment, the display position and display size of the in-camera can be designated by the user using a menu or the like, and information on the set display position and display size is stored in the nonvolatile memory 51.

  In step S305, an in-camera image for display is generated based on the output of the image sensor 103 with proper exposure controlled by the exposure control 40. The size of the generated image is generated based on the size acquired in step S304.

  In step S306, the display image of the main image created in step S301 and the display in-camera image created in step S305 are combined to generate a display combined image. The image created at this time is an image in which the in-camera image is superimposed and synthesized on the main image, for example, as shown in FIG. As shown in FIG. 5, when the display in-camera image 501 is arranged so as to be superimposed on the display main image 500, the position where the display in-camera image 501 is superimposed follows the in-camera display position information acquired in step S304. . The display composite image generated in step 306 is temporarily stored in the memory 25 in the form of overwriting the display image stored in the memory 25 in step S301. In other words, the display image stored in the memory 25 is replaced with the composite image generated in step S306 from the main image generated in step S301. In this embodiment, as shown in FIG. 5B, the in-camera image is superimposed on the main image, but a small main image may be superimposed on the in-camera image.

  In step S <b> 307, an in-camera moving image for moving image recording is generated based on the output of the image sensor 103 with proper exposure controlled by the exposure control 40. Hereinafter, a moving image created from the output of the imaging device 103 of the in-camera is referred to as an in-camera moving image. The size of the generated in-camera moving image is generated based on the size acquired in step S304.

  In step S308, the moving image recording main moving image generated in step S302 and the moving image recording in-camera moving image generated in step S307 are combined to generate a combined moving image as a moving image recording moving image. In order to synthesize the in-camera moving image at the same position as the display image, the in-camera moving image for video recording is superimposed on the main moving image recording according to the in-camera display position information as in step S306. To create a composite video. That is, the display image generated in S301 or S306 and the moving image recording image size generated in S302 or S308 are different for both the main image and the in-camera image. However, the display image and the moving image recording image are created so that the ratio of the size of the main image and the in-camera image is the same, and the displayed image and the recorded image are displayed in the same manner. Create images for movies and videos. The synthesized moving image for moving image recording generated in 308 is temporarily recorded in the memory 25 as a moving image recording image. That is, the moving image recording image recorded in the memory 25 is rewritten from the main moving image generated in step S302 to the synthesized moving image generated in step S308.

  In step S 309, the display image temporarily stored in the memory 25 is displayed on the display unit 28. That is, if the in-camera shooting function is set to be valid, a composite image generated in step S306, for example, as shown in FIG. 5B is displayed, and if the in-camera shooting function is disabled, step S301 is performed. The generated main image as shown in FIG. 5A is displayed.

  Next, in step S <b> 310, the moving image recording image is read from the memory 25, the moving image compression processing is performed by the image processing circuit 20, the moving image data is compressed, and the moving image compression processing of the moving image recording image stored in the memory 25 is started. To do. If the in-camera shooting function is enabled, moving image data is generated based on the composite image generated in step S308. If the in-camera shooting function is disabled, moving image data is generated based on the main image generated in step S302. And stored in the memory 25. In addition to the compression of the moving image data, the audio data input by a microphone (not shown) is also compressed using the audio compression processing included in the system control unit 50 and stored in the memory 25. The processing in steps S301 to S310 is basically repeated during the still image / moving image recording simultaneous recording mode. Therefore, here, the compressed video data for a maximum of 4 seconds in the ring buffer format is held in the memory 25.

  As described above, in steps S301 to S310, display image generation and display processing, moving image recording image generation and moving image compression processing (temporary recording in the memory 25) are performed. These processes are not limited to the above order. The generation and display processing of the display image and the generation and temporary recording of the moving image recording image may be executed in parallel.

  Next, in step S311, it is determined whether the mode change from the still image / moving image simultaneous recording mode has been performed. The mode change includes a change of another shooting mode by the mode change switch, a shift to the playback mode, a shift to the setting menu by the menu button, a power OFF by the power switch, and the like. If it is determined in step S311 that the mode has been changed, the process proceeds to step S312 to end the moving image compression process, discard the moving image data and audio data stored in the memory 25, and record the still image and moving image simultaneously. Exit mode. If it is determined in step S311 that the mode has not been changed, the process proceeds to step S313.

  In step S313, it is confirmed whether the shooting setting has been changed. Shooting setting items to check whether they have been changed include white balance, exposure compensation values, focus method switching such as MF / macro shooting, and still image switching such as image size and compression rate according to the subject and situation. This includes shooting conditions that require. These shooting settings are displayed on the GUI screen for the user to change the shooting settings in response to a user operation on an assigned shooting setting change button included in the operation unit 62 or a touch panel provided in the display unit 28. To be temporarily superimposed on the screen. The shooting setting can be changed by operating the operation unit 62 or the touch panel on the GUI screen. If it is determined in step S313 that the shooting setting has been changed, the process proceeds to step S314. If the shooting setting has not been changed, the process proceeds to step S315.

  In step S315, it is confirmed whether the setting of the in-camera has been changed. There are items that can be set in accordance with the user's operation for setting the in-camera. Items that can be set include the display position, the display size, and switching between enabling and disabling the shooting function of the in-camera. Regarding the setting items related to these in-cameras, the user can change the settings in accordance with a user operation on the touch panel provided in the operation unit 62 or the display unit 28.

  If it is determined in step S315 that the setting of the in-camera has been changed, the process proceeds to step S316, where the moving image compression process is stopped and the moving image data and audio data stored in the memory 25 are discarded. The purpose of this is to prevent the deterioration of the quality of the moving image data by recording the switching of the in-camera display between the moving image data recorded so far and the moving image data to be shot from now on when the setting of the in-camera display is changed. Next, in step S317, it is confirmed whether the changed setting of the in-camera is an in-camera shooting function valid / invalid setting. If it is determined in step S317 that the in-camera shooting function is enabled / disabled, in step S318, the in-camera setting is enabled / disabled according to a user instruction. And it returns to step S301 and based on the changed setting, the process from step S301 is performed again. If it is determined that the setting of the in-camera changed in step S317 is not the validity / invalidity of the in-camera shooting function but the position and size of the in-camera display, the position of the in-camera display is determined in step S319 according to the user's instruction. , Change the size respectively. Then, it progresses to step S301 and performs the process from step S301 again according to the newly set content.

  If the in-camera setting has not been changed, the process proceeds from step S315 to step S320. In step S320, it is confirmed whether the shutter button 60 is half-pressed and the SW1 signal is input. If the SW1 signal is not input, the process proceeds to step S322 without performing the process in step S321. When SW1 is input, the process proceeds to step S321, and focusing / exposure processing according to the focus mode and exposure mode set in the digital camera 100, that is, the above-described shooting preparation operation is performed.

  In step S322, it is confirmed whether the shutter button 60 is fully pressed and the SW2 signal is input. If the SW2 signal is not input, the process returns to step S01 and the process is repeated. If it is determined in step S322 that the SW2 signal has been input, in step S323, the image display on the display unit 28 is temporarily stopped and the moving image compression process started in step S310 is temporarily stopped. Here, the moving image data and audio data already stored in the memory 25 are retained.

  Next, in step S324, recording still image data (main still image) is captured based on the output of the imaging device 13 of the out-camera. In step S325, a review image for image confirmation is generated from the still image data (main still image) captured in step S324. In step S326, the still image compression processing is performed on the still image data (main still image) captured in step S324 by the image processing circuit 20 to generate still image data (JPEG data) for recording. To store. Here, in the still image / moving image recording simultaneous recording mode, only the main image in which the in-camera image is not synthesized is recorded on the external recording medium 91 as a still image regardless of the setting of the in-camera shooting function. Therefore, still image compression processing is performed on the main still image.

  Then, it progresses to step S401 shown in FIG.

  In step S401, the still image data (JPEG data) stored in the memory 25 in step S326 is recorded on the external recording medium 91 as one still image file (JPEG file).

  Next, in step S402, it is determined whether or not the moving image recording time (shooting time) of moving image data for moving image recording stored in the memory 25 is longer than a predetermined time () t0. In this embodiment, t0 = 2 seconds. If it is determined in step S402 that the moving image recording time is 2 seconds or longer, the process proceeds to step S403. If it is determined that the moving image recording time is shorter than 2 seconds, the process proceeds to step S404.

  In step S <b> 403, the moving image data and audio data stored in the memory 25 are recorded on the external recording medium 91. Here, the moving image data and the audio data are recorded so as to be added to the moving image data and the audio data of the digest moving image file recorded in the external recording medium 91. Each moving image is divided into chapters and recorded in a digest moving image file so that moving images corresponding to still image shooting can be distinguished. Then, still image specifying information that can specify a still image file corresponding to each chapter is recorded as chapter information, and the still image and the moving image are associated with each other. As the still image specifying information, the file name or shooting date and time of the corresponding still image file may be used. The chapter information describes such still image specifying information, and the start position (start frame) and time (number of frames) or end position (end frame) of each chapter in the moving image data. The header of the digest moving image file It is good to record in the section. Accordingly, it is possible to determine a moving image corresponding to a still image file included in the digest moving image file without analyzing the entire moving image data. If there is no digest movie file on the external recording medium 91, a new digest movie file is created and the movie data and audio data read from the memory 25 are recorded. In addition, when the shooting date is switched or when the size of the moving image data is larger than a predetermined size, a new digest moving image file may be created and recorded. In step S404, the moving image data and audio data stored in the memory 25 are discarded. In other words, if the moving image recording time (shooting time) is shorter than 2 seconds, the moving image data and audio data shot before shooting a still image in step S325 and temporarily stored in the memory 25 are stored in the external recording medium 91. Is not recorded or saved. This is because if the video is too short, the situation at the time of shooting cannot be confirmed. Therefore, in the present embodiment, if it is shorter than 2 seconds, it is not recorded on the external recording medium 91.

  In step S405, the review image created in step S325 is displayed on the display unit 28. With this display, the user can confirm the captured still image.

  In step S406, it is determined whether the shooting function of the in-camera is valid or invalid. If it is determined that the in-camera shooting function is valid, the process proceeds to step S407, and a process for generating a moving image after shooting a still image by combining the in-camera images in steps S407 to S414 is performed. If the in-camera setting is invalid, the process proceeds to step S418.

  In step S407, a stop motion image for moving image recording is generated from the main still image taken by the out-camera in step S324. That is, the main still image is converted into moving image data. Here, when the aspect ratio of the still image recorded on the external recording medium 91 is different from that of the moving image, the main still image is resized so that the entire main still image is included in the stop motion image. Generate a stop motion image in black. For example, when the still image has an aspect ratio of 4: 3 and the moving image has an aspect ratio of 16: 9, a stop motion image in which the left and right areas are painted black is generated.

  Next, in step S408, an in-camera image for display is generated based on the output of the image sensor 103, as in step S305.

  In step S409, similarly to step S306, a display composite image is generated. However, in step S409, a composite image for display is generated by combining the in-camera image generated in step S408 with the main still image captured by the out-camera in step S324.

  Next, in step S410, as in step S307, an in-camera moving image for moving image recording is generated based on the output of the image sensor 103.

  In step S411, the display composite image generated in step S409 is displayed on the display unit 28. Here, an image photographed by the in-camera is displayed as a moving image on a still image photographed by the out-camera and recorded on the external recording medium 91.

  Next, in step S412, similarly to step S308, a combined moving image is generated by combining the in-camera moving image as a moving image recording image. However, in step S412, the in-camera moving image generated in step S410 is combined with the stop motion image generated in step S407 to generate a moving image recording combined image.

  In step S413, as in step S310, moving image compression processing is performed on the synthesized moving image generated in step S412 to generate moving image data, which is stored in the memory 25 as still image shooting moving image data. In addition to the compression of the moving image data, the shutter sound data stored in advance in the nonvolatile memory 51 and the audio data input by a microphone (not shown) or the like are also used by using the audio compression processing included in the system control unit 50. Compressed and stored in the memory 25.

  In step S414, it is determined whether the moving image recording time (shooting time) of the moving image data after shooting the still image generated in step S413 and stored in the memory 25 is equal to or longer than the predetermined time t1. In the present embodiment, t1 = 2 seconds. If it is determined that the moving image recording time of the moving image data is 2 seconds or longer, the process proceeds to step S420. If it is determined that the moving image recording time of the moving image data is shorter than 2 seconds, the process proceeds to step S408, and the processes of steps S408 to S414 are repeated. In other words, when the in-camera shooting function is enabled, after the still image review display is started, the in-camera is shot until the moving image recording time of the moving image data after the still image shooting is 2 seconds. . Then, until the moving image recording time of the moving image data after taking the still image becomes 2 seconds, the display of the image obtained by combining the stop motion image and the in-camera moving image and the moving image compression processing of the combined image are repeated.

  In step S411, the moving image is shot by the in-camera while the still image shot by the out-camera and recorded on the external recording medium 91 is displayed. Therefore, the facial expression of the user (photographer) confirming the still image captured by the display unit 28 can be captured by the in-camera and recorded together with the still image.

  In step S415, it is determined whether or not the moving image recording time (shooting time) of the moving image data for moving image recording taken before taking a still image is t0 or more. Instead of determining the moving image recording time, it may be determined in step S404 whether or not the moving image data has been canceled. If the canceling process is performed because the video recording request for the moving image before still image shooting is shorter than t0, the process returns to step S301, and the process is repeated from the beginning. If it is determined that the moving image recording request for the moving image before still image shooting is t0 or more and the cancel processing has not been performed, the process proceeds to step S416.

  In step S416, it is determined whether the stop motion setting is on or off. If the stop motion setting is on, the process proceeds to step S417. If the stop motion setting is off, the process returns to step S301.

  In step S417, a stop motion image is generated as in step S407.

  Next, in step S418, moving image compression processing is performed on the stop motion image generated in step S417, and the still image shooting moving image data is stored in the memory 25. The audio data to be compressed here is shutter sound data and silence data stored in advance in the nonvolatile memory 51.

  In step S419, moving image compression processing is performed in step S418, and it is determined whether or not the moving image recording time (shooting time) of the moving image data after shooting the still image temporarily stored in the memory 25 is equal to or longer than the predetermined time t2. . In this embodiment, t2 = 1 second. The moving image compression processing of the stop motion image in S418 is repeated until it is determined in step S419 that the moving image recording time of the moving image data after still image shooting is 1 second or longer.

  In step S420, the still image-captured moving image data created by the moving image compression process in step S413 or step S419 is recorded on the external recording medium 91. When recording, the recording is performed by adding to the moving image data and audio data of the digest moving image file in the same manner as in step S403. That is, when moving image data before still image shooting is recorded in step S403, moving image data after still image shooting is recorded following the moving image data before still image shooting.

  In step S421, it is determined whether or not the recording of moving image data after still image shooting in step S420 is completed. If it is determined that the recording of the moving image data after the still image shooting is completed, the moving image data temporarily stored in the memory 25 is deleted, and the process returns to S301.

<Various settings and movie recording pattern of recorded movie>
Next, moving image data recorded on the external recording medium 91 when the above-described shooting process is performed in the still image / moving image simultaneous recording mode will be described with reference to FIGS. 6 and 7.

  FIG. 6 is a diagram for explaining a moving image recording pattern. FIG. 7 shows a moving image pattern recorded on the external recording medium 91 in the in-camera shooting function setting, the recording time (shooting time) of moving image data before shooting still images, the stop motion setting, and the still image / moving image shooting simultaneous recording mode. It is a figure which shows the relationship. Here, the recording time of moving image data before still image shooting means moving image recording time (shooting time) of moving image data temporarily recorded in the memory 25 before shooting a still image.

  Each moving image recording pattern will be described with reference to FIG.

  In the moving image recording pattern 1, a moving image before 2 to 4 seconds still image shooting and a moving image after 2 seconds still image shooting are recorded. In the moving image recording pattern 1, a combined moving image in which an in-camera moving image captured by an in-camera is combined with a main moving image captured by an out-camera is recorded as a moving image before still image shooting. Then, a combined moving image in which the in-camera moving image captured by the in-camera is combined with the stop motion generated from the still image captured by the out-camera is recorded as a moving image after capturing the still image. Also, as the sound data of the moving image before shooting the still image, the environmental sound input with a microphone or the like is recorded, and the moving image sound data after shooting the still image is first recorded with the shutter sound data, and then the environmental sound is recorded. Is done.

  In the moving image recording pattern 2, a moving image before still image shooting is not recorded, and only a moving image after shooting a still image of 2 seconds is recorded. In the moving image recording pattern 2, a combined moving image in which an in-camera moving image captured by an in-camera is combined with a stop motion generated from a still image captured by an out-camera is recorded as a moving image after still image capturing. In addition, as for the sound data of the moving image after still image shooting, shutter sound data is recorded first, and then environmental sound is recorded. That is, in the moving image recording pattern 2, moving image data and audio data of a moving image after still image shooting are recorded in the same format as the moving image after shooting still image recorded in the moving image recording pattern 1.

  In the moving image recording pattern 3, a moving image before 2 to 4 seconds still image shooting and a moving image after 1 second still image shooting are recorded. In the moving image recording pattern 3, the main moving image shot by the out camera in which the in-camera moving image is not combined is recorded as the moving image before still image shooting. Then, stop motion generated from the still image captured by the out-camera is recorded as a moving image after still image capturing.

  Also, as the sound data of the moving image before shooting the still image, the environmental sound input by the microphone or the like is recorded, and as the moving image sound data after shooting the still image, only the shutter sound data is recorded and then recorded as silence. Is done.

  In the moving image recording pattern 4, only the moving image before the still image shooting for 2 to 4 seconds is recorded, and the moving image after the still image shooting is not recorded. In the moving image recording pattern 4, the main moving image shot by the out camera in which the in-camera moving image is not combined is recorded as a moving image before shooting a still image. In addition, as sound data of a moving image before still image shooting, environmental sound input with a microphone or the like is recorded. That is, in the moving image recording pattern 4, moving image data and audio data of a moving image after still image shooting recorded in the same format as the moving image before still image shooting recorded in the moving image recording pattern 3.

  Here, since the moving image data temporarily stored in the memory 25 in the processing of steps S301 to S310 is set to a maximum of 4 seconds, the moving image before still image shooting is 4 seconds or less. Furthermore, in steps S402 to S404, the moving image before still image shooting is recorded on the external recording medium 91 only when the recording time of the moving image data before shooting still image stored in the memory 25 is 2 seconds or more. The pre-shooting video is 2-4 seconds.

  In the case of moving image recording patterns 1 and 2, a moving image of 2 seconds is recorded as a moving image after shooting a still image, whereas moving image recording pattern 3 records a moving image of 1 second as a moving image after shooting a still image. This is because, in the case of moving image recording patterns 1 and 2, an in-camera moving image is combined with a stop motion to record a moving combined moving image, whereas in moving image recording pattern 3, a stop motion without movement is recorded. The recording time is shortened. In this embodiment, a moving image to be recorded when a moving synthetic moving image is recorded as in moving image recording patterns 1 and 2 is set to 2 seconds (t1), and a moving image is recorded as in moving image recording pattern 3 When it does, it is set as 1 second (t2).

  Next, the relationship between each setting and the moving image recording pattern will be described with reference to FIG.

  First, when the in-camera shooting function is set to be valid, the moving image recording pattern 1 or 2 is set. In other words, when the in-camera shooting function is enabled, a combined video in which the in-camera video is combined with the stop motion is always recorded as a video after shooting a still image regardless of the recording time of the video data before shooting the still image and the stop motion settings Is done. Therefore, it is possible to record the photographer's facial expression and reaction while viewing a still image. When the in-camera shooting function is enabled, a composite image in which the in-camera video is combined with the main video as a video before still image shooting depending on whether the recording time of the video data before still image shooting is 2 seconds or longer The recording will change. That is, whether to record a composite image changes depending on whether or not it is a predetermined t0 time or longer. If the recording time of the moving image data before still image shooting is 2 seconds (t0) or longer, the moving image data before shooting the still image is recorded. Therefore, it is possible to record the subject immediately before the still image shooting, the photographer's situation and facial expression. If the recording time of the moving image data before still image shooting is less than 2 seconds (t0), the moving image before shooting still image is not recorded.

  When the in-camera shooting function is disabled and the recording time of moving image data before shooting still images is less than 2 seconds (t0), the moving images (moving images before shooting still images and still images are not affected) regardless of the stop motion setting. (Movie after shooting) is not recorded. In this case, since the recording time of the moving image data before still image shooting is less than 2 seconds, the moving image before still image shooting is not recorded as described in steps S402 to S404. For movies after still image shooting, stop motion can be recorded as still image shooting video depending on the stop motion setting, but if the in-camera shooting function is disabled, the in-camera movie will be combined. Stop motion is not performed. If the in-camera video is not combined, it will be a motionless video. Therefore, if the in-camera shooting function is disabled and the recording time of the video data before still image recording is less than 2 seconds, do not record the video after still image shooting regardless of the stop motion setting. Yes.

  When the in-camera shooting function is disabled and the recording time of moving image data before still image shooting is 2 seconds (t0) or longer, the moving image recording pattern 3 or 4 is set, and the moving image before shooting still image is recorded. . Therefore, it is possible to record the situation immediately before the still image shooting, the change in the facial expression of the subject, and the like. Whether to record is determined according to the setting of the stop motion. When the stop motion setting is on, the moving image recording pattern 3 is set, and a stop motion in which the in-camera moving image is not combined is recorded as a moving image after still image shooting. When the stop motion setting is OFF, the moving image recording pattern 4 is set and no moving image is recorded after the still image is shot. For the moving image recording pattern 3, stop motion without motion is recorded as a moving image after taking a still image. This is because the stop motion is recorded together with the pre-shooting moving image, so that both the still image before shooting and the image at the time of shooting can be confirmed.

  When the recording time of moving image data before still image shooting is 2 seconds (t0) or longer, the moving image recording pattern is 1, 3 or 4, and the moving image before shooting still image is always recorded regardless of other settings. If the recording time of the moving image data before still image shooting is less than 2 seconds (t0), the moving image recording pattern 2 or the moving image is not recorded. Normally, if the recording time of the video data before still image shooting is less than 2 seconds, the movie (the movie before still image shooting and the movie after still image shooting) will not be recorded, but if the in-camera shooting setting is enabled, In-camera video can be acquired after still image shooting. Therefore, as in the moving image recording pattern 2, a combined moving image obtained by combining the in-camera moving image with the stop motion image is recorded as a moving image after shooting a still image.

  In this way, by performing the shooting process as described above in the still image / moving image simultaneous recording mode, it is possible to record a moving image with a suitable moving image recording pattern according to the setting shown in FIG. In the still image / moving image simultaneous recording mode, for moving images, the moving image recording pattern and whether or not to record are changed according to the settings shown in FIG. 7, but for still images, the setting shown in FIG. Instead, recording is performed according to the set recording size and compression rate.

(Other embodiments)
Although the present invention has been described in detail based on the preferred embodiments thereof, the present invention is not limited to these embodiments, and various forms within the scope of the present invention are also included in the present invention. It is. In the above-described embodiment, the imaging apparatus to which the present invention is applied has been described as an example. However, the present invention may be implemented by an electronic device such as a camera-equipped mobile phone or a portable game machine. A system in which a plurality of devices, for example, an imaging device having a plurality of imaging units, a display device having a display unit, and a control device having a control unit are connected without realizing the functions of the above-described embodiments with a single device. May be implemented. Moreover, you may implement combining a part of above-mentioned embodiment suitably.

  Also, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed Are also included in the present invention. Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS. As a recording medium for supplying the program, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a nonvolatile semiconductor memory may be used. As a program supply method, a computer program that forms the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

Claims (11)

First imaging means for photographing a subject ;
A second imaging means for photographing the photographer ;
Photographing instruction means for instructing photographing of a still image by the first imaging means;
In response to an instruction to shoot a still image by the first image pickup means by the shooting instruction means, the still image shot by the first image pickup means is recorded on a recording medium and before the still image is shot. and videos taken by the first imaging means, the videos taken by the second imaging means before recording the still image, and a video taken by the second imaging means after shooting of the still image An image pickup apparatus comprising: a control unit that controls to record on the recording medium and controls not to record a moving image by the first image pickup unit after shooting the still image. The control means, in response to the still image shooting by the first imaging means is instructed by the imaging instruction means, with a still image captured by the first imaging means for recording on a recording medium, wherein A moving image obtained by combining the moving image shot by the second imaging unit before shooting the still image with the moving image shot by the first imaging unit before shooting the still image, and the still image after shooting the still image the imaging apparatus according to claim 1, characterized in that the videos form if the videos captured by the second imaging means, recorded on the recording medium. The control unit includes a moving image obtained by combining the moving image captured by the second imaging unit with the moving image captured by the first imaging unit before capturing the still image, and the still image captured by the first imaging unit. The imaging apparatus according to claim 2 , wherein the moving image obtained by combining the moving image captured by the second imaging unit with the image is recorded as one continuous moving image. Display control means for controlling to display a still image taken by the first image pickup means on the display means in response to an instruction to take a still image by the first image pickup means by the shooting instruction means. Have
The control means, wherein the video movies shot by pre Symbol second imaging means, taken by the second imaging means after shooting of the still image while displaying the still image on the display means the imaging apparatus according to any one of claims 1 and controls to record on the recording medium 3. Furthermore, it has a display means for displaying the image image | photographed with the said 1st imaging means,
The second imaging means, the imaging apparatus according to any one of claims 1, characterized in that to shoot a side on which the display unit is provided 4 of the imaging device. Setting means for switching settings relating to whether to record an image captured by the second imaging means;
If it is set to be recorded by said setting means, the control means, any one of claims 1 to 5, wherein the controller controls so as to photograph a moving image by the second imaging means The imaging device described in 1. When it is set by the setting means not to record an image taken by the second imaging means, no image is taken by the second imaging means, or an image is taken by the second imaging means. The imaging apparatus according to claim 6 , wherein even if an image is captured, the image is not recorded on the recording medium. The control unit associates the still image captured by the first imaging unit with the moving image captured by the second imaging unit in response to an instruction to capture a still image by the imaging instruction unit, and records the recording. the imaging apparatus according to any one of claims 1 to 7, characterized in that controlling so as to record on the medium. A control method of a control device for controlling an imaging device having a first imaging means for photographing a subject and a second imaging means for photographing a photographer ,
A shooting instruction step of receiving a still image shooting instruction by the first imaging means;
In response to receiving an instruction to shoot a still image from the first imaging unit, control is performed to record the still image captured by the first imaging unit on a recording medium, and before the still image is captured. Recording the moving image shot by the first imaging unit , the moving image shot by the second imaging unit before shooting the still image, and the moving image shot by the second imaging unit after shooting the still image And a control step of controlling to record a moving image by the first imaging unit after shooting the still image, so as to record on a medium. The program for functioning a computer as each means of the imaging device of any one of Claim 1 thru | or 9 . A computer-readable storage medium storing a program for causing a computer to function as each unit of the imaging apparatus according to any one of claims 1 to 9 .

Images