JP2013168710A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To record a sound expressing an atmosphere in photographing together with a slow motion moving image.SOLUTION: The imaging device comprises: an imaging element for imaging a subject; a storage section for sequentially storing plural frame images from the imaging element; a moving image data creation section for creating data of a slow motion moving image reproduced at a second frame rate smaller than a first frame rate indicating the number of frame images stored in the storage section for a unit time; a raw sound data recording section for recording raw sound data associated with the slow motion moving image; a sound data creation section for processing the raw sound data and creating sound data reproduced in reproducing the slow motion moving image; an image reproduction section for reproducing the slow motion moving image based on data of the slow motion moving image; and a sound reproduction section for reproducing the sound data created by the sound creation section in reproducing the slow motion moving image.

Description

  The present invention relates to an imaging apparatus capable of recording moving images and audio.

  2. Description of the Related Art Conventionally, when reproducing a still image shot during moving image shooting, a playback device that outputs audio using audio data of moving image data corresponding to a predetermined time range including the time point when the still image was shot is provided. Exists (see, for example, Patent Document 1). According to this playback device, it is possible to play the sound near the point in time when the still image is shot like BGM.

JP 2005-117083 A

  However, Patent Document 1 does not disclose the recording of sound when shooting a slow motion moving image in which the subject moves more slowly than the actual movement, and the technique disclosed in Patent Document 1 described above is not disclosed. When used, since the length of the recorded voice is shortened with respect to the time of the slow motion moving image to be reproduced, the atmosphere at the time of shooting the slow motion moving image cannot be expressed.

  The objective of this invention is providing the imaging device which can record the audio | voice expressing the atmosphere at the time of imaging | photography with a slow motion moving image.

  The present invention solves the above problems by the following means. In addition, in order to make an understanding easy, it attaches | subjects and demonstrates the code | symbol corresponding to embodiment of this invention, but this invention is not limited to this.

  An image pickup apparatus according to the present invention includes an image pickup element (6) that picks up light from a subject, a storage unit (8) that sequentially stores a plurality of frame images based on image pickup signals from the image pickup element, and a predetermined time. Based on the plurality of frame images stored in the storage unit, a slow motion video that is played back at a second frame rate less than a first frame rate indicating the number of frame images stored in the storage unit per unit time A moving image data generating unit (5b) for generating image data, a raw audio data recording unit (17) for recording raw audio data related to the slow motion moving image, and processing the raw audio data to perform the slow sound An audio data generation unit (4) that generates audio data to be played back when the motion video is played back, and the slow motion video based on the slow motion video data. An image playback unit (12) that plays back a moving image, and an audio playback unit (16) that plays back the audio data generated by the audio data generation unit when the slow motion video is played back. It is characterized by that.

  According to the imaging apparatus of the present invention, it is possible to record a sound that expresses the atmosphere at the time of shooting together with a slow motion moving image.

It is a block diagram which shows the system configuration | structure of the electronic camera which concerns on embodiment. It is a time chart which shows the capture operation | movement of the frame image and raw audio | voice data of the electronic camera which concerns on embodiment. It is a figure which shows the production | generation process of the slow motion moving image data and audio | voice data of the electronic camera which concerns on embodiment. It is a time chart which shows the reproduction | regeneration operation | movement of the slow motion moving image of the electronic camera which concerns on embodiment, a still image, and an audio | voice.

  Hereinafter, an electronic camera as an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a system configuration of an electronic camera according to an embodiment. As shown in FIG. 1, the electronic camera 2 is configured by a microprocessor or the like, and includes a control unit 4 that comprehensively controls each unit of the electronic camera 2. The control unit 4 includes a video signal processing circuit 5, and the video signal processing circuit 5 generates a through image generation unit 5 a that generates through image data to be displayed on a display unit 12 described later, and a moving image that generates moving image data. It has the production | generation part 5b and the production | generation image generation part 5c which produces | generates still image data. The control unit 4 is connected with a buffer memory 8, a recording medium 10, a display unit 12, an operation unit 14, a speaker 16, an audio data storage unit 17, and an amplifier / A / D conversion unit 18.

  The image sensor 6 is constituted by a CCD or CMOS, and images light from a subject through a photographing lens (not shown). The buffer memory 8 is output from the image sensor 6 and passes through an A / D converter (not shown). Thus, the image data based on the imaging signal converted from the analog signal to the digital signal is temporarily stored.

  In addition, the buffer memory 8 starts from the image sensor 6 when the electronic camera 2 is set to a slow motion moving image capturing mode for capturing, for example, a slow motion moving image (hereinafter referred to as a slow moving image) described later. A plurality of image data based on imaging signals output in synchronization with a moving image cycle (60 fps (frame / second) or the like), that is, a plurality of frame images constituting a slow moving image are sequentially stored. In this embodiment, the buffer memory 8 stores a frame image having the number of pixels of the full high definition (Full High Definition: 1920 × 1080 pixels), that is, a high-resolution frame image in a FIFO (first in first out) system. Are sequentially stored.

  The recording medium 10 is a portable recording medium that is detachably mounted in a card slot (not shown) provided in the electronic camera 2, and for example, a CF card, an SD card, smart media, or the like is used. In the recording medium 10, each frame image stored in the buffer memory 8 is resized to a moving image resolution (from a large resolution to a small resolution) by the moving image generation unit 5 b in the control unit 4. Moving image data and slow motion moving image data (hereinafter referred to as slow moving image data) created by performing image processing for video and compression processing for moving images are recorded. Further, the recording medium 10 is created by subjecting the frame image stored in the buffer memory 8 to image processing for still images and compression processing for still images by the still image generating unit 5c in the control unit 4. Audio data created by applying predetermined processing to the still image data and the raw audio data stored in the audio data storage unit 17 by the control unit 4, information relating to photographing, and the like are recorded.

  The display unit 12 is configured by a monitor or EVF configured by an LCD or the like disposed on the back side of the electronic camera 2, and a through image generated by the through image generation unit 5 a and a moving image recorded on the recording medium 10. A moving image based on the image data, a slow moving image based on the slow motion moving image data, a still image based on the still image data, information relating to shooting, and the like are displayed.

  The operation unit 14 is a command for setting a shooting mode such as a power switch for turning on / off the power of the electronic camera 2, a moving image shooting mode for shooting a moving image, and a slow motion moving image shooting mode for shooting a slow moving image. Dial, release button for instructing start of shooting of moving image and still image, menu button for displaying menu etc. on display unit 12, cross key for selecting menu item and various settings, menu The screen includes an OK button for selecting items and confirming various settings, a playback button for displaying moving image data and still image data recorded on the recording medium 10 on the display unit 12, and the like. Yes.

  The speaker 16 outputs sound based on the sound data recorded on the recording medium 10. The amplifier / A / D converter 18 converts the analog signal of the sound around the electronic camera 2 collected by the microphone 20 into a digital signal, and outputs the converted digital signal of the sound to the control unit 4. . The control unit 4 creates raw audio data from the audio digital signal output from the amplifier / A / D conversion unit 18. The voice data storage unit 17 temporarily stores raw voice data.

  In the electronic camera 2 according to this embodiment, a slow moving image that is output from the image sensor 6 and reproduced at a second frame rate that is less than the first frame rate indicating the number of frame images stored in the buffer memory 8 per unit time. A recording medium that generates data and generates still image data based on at least one frame image of a plurality of frame images forming the slow moving image data, and associates the generated slow moving image data and still image data with each other Record in 10. In addition, audio data created by processing the raw audio data collected by the microphone 20 and generated by the amplifier / A / D converter 18 is recorded on the recording medium 10 in association with the slow moving image data and the still image data.

  Hereinafter, with reference to the drawings, processing in the case where the electronic camera 2 according to the present embodiment performs slow moving image shooting, still image shooting, and sound recording will be described.

  In this embodiment, a plurality of frame images stored in the buffer memory 8 and a frame image serving as still image data between the input of the still image shooting instruction and the first predetermined time before are stored in the buffer memory 8. Slow moving image data is generated on the basis of a plurality of frame images stored in the buffer memory 8 between the stored time and after the second predetermined time. That is, slow moving image data is generated based on a plurality of frame images stored in the buffer memory 8 during a predetermined time (first predetermined time + second predetermined time), and at the end of the first predetermined time (second predetermined time). Still image data is generated based on the frame image stored in the buffer memory 8 at the time start). The predetermined time (for example, 1 second), the first predetermined time (for example, 0.6 second), and the second predetermined time (for example, 0.4 second) are set in advance and stored in a memory (not shown). It is configured to be changeable.

  In this embodiment, the audio and still image data stored in the audio data storage unit 17 from the time when the frame image serving as the still image data is stored in the buffer memory 8 to the third predetermined time before Based on the audio stored in the audio data storage unit 17 from when the frame image is stored in the buffer memory 8 until after the fourth predetermined time, audio data of audio to be reproduced at the time of slow moving image reproduction is generated. . That is, sound data to be played back during slow moving image playback is generated based on the sound data stored in the sound data storage unit 17 for a predetermined time (third predetermined time + fourth predetermined time). The third predetermined time (for example, 9.6 seconds) and the fourth predetermined time (for example, 0.4 seconds) are preset and stored in a memory (not shown) or the like so as to be changeable.

  When the user sets the shooting mode to the slow motion moving image shooting mode by operating the command dial, the control unit 4 captures the slow moving image, the still image related to the slow moving image, and the slow moving image. Transition to slow motion video shooting mode for recording audio. Next, the control unit 4 determines whether or not the release button is half-pressed by the user. As shown in FIG. 2, when the release button is half-pressed (S1), the control unit 4 prepares to capture a slow moving image and a still image, that is, focuses on the main subject, and captures and throws a slow moving image. In order to record sound related to the moving image, pre-capture of the frame image in the buffer memory 8 and sound buffering to the sound data storage unit 17 are started.

  In this embodiment, the time from when the user fully presses the release button (S2) to the time before the first predetermined time and the time from when the release button is fully pressed (S2) to the time after the second predetermined time. The slow moving image data is generated based on the frame image stored in the buffer memory 8. Therefore, during the pre-capture operation shown in FIG. 2, when the buffer memory 8 has a storage area capable of storing, for example, a frame image for 1 second, the frame image for 1 second is stored in the buffer memory 8. After that, every time a frame image is generated until the release button is fully pressed (S2), the oldest frame image in the buffer memory 8 is deleted, and the generated frame image (latest The frame image is stored in the buffer memory 8 repeatedly.

  Further, when the audio data storage unit 17 has a storage area capable of storing, for example, 10 seconds of audio, after the audio data storage unit 17 stores 10 seconds of audio, the release button The oldest voice data in the voice data storage unit 17 is deleted every time voice data is output from the amplifier / AD knitting unit 18 until the full pressing operation (S2) is performed, and the output voice data (the latest The process of storing the audio data) in the audio data storage unit 17 by the FIFO method is repeated.

  Next, the control unit 4 determines whether or not the release button full-press operation (S2) has been performed while the frame image in the buffer memory 8 and the audio data in the audio data storage unit 17 are being repeatedly stored. Determine. When the release button is fully pressed, the control unit 4 determines that a still image shooting instruction for instructing generation of still image data has been input, ends the first predetermined time and the third predetermined time, and ends the second predetermined time. And a fourth predetermined time is started.

  The control unit 4 deletes the oldest frame image in the buffer memory 8 every time a frame image is generated until the second predetermined time elapses, and the generated frame image (latest frame image) is stored in the buffer memory. The process stored in 8 is repeated. The control unit 4 deletes the oldest audio data in the audio data storage unit 17 every time audio data is output until the fourth predetermined time elapses, and outputs the output audio data (latest audio data) as a FIFO. The processing stored in the voice data storage unit 17 is repeated by the method.

  When the second predetermined time and the fourth predetermined time have elapsed, the control unit 4 ends the buffering of the frame image in the buffer memory 8 and the buffering of the audio data in the audio data storage unit 17.

  FIG. 2 shows a case where the first predetermined time is 0.6 seconds, the third predetermined time is 9.6 seconds, and the second predetermined time and the fourth predetermined time are each 0.4 seconds. When the fourth predetermined time is longer than the second predetermined time, the control unit 4 notifies the user that the buffering of the audio data has not ended when the second predetermined time has elapsed. Here, the notification to the user may be performed by displaying an image or a character indicating that the audio data is being buffered on the display unit 12, or the audio data buffering elapsed time may be displayed on the display unit 12. You may carry out by displaying the indicator etc. which show. Further, the remaining time until the end of buffering may be displayed on the display unit 12 to notify the user that the buffering of audio data has not ended.

  When the pre-capture of the frame image and the audio buffering are completed, the control unit 4 generates slow moving image data, still image data, and audio data to be reproduced when reproducing the slow moving image. First, the process of generating slow moving image data will be described. As shown in FIG. 2, the control unit 4 includes a plurality of buffers buffered in the buffer memory 8 between the time when the release button is fully pressed and before the first predetermined time (0.6 seconds in this embodiment). Based on the frame image and a plurality of frame images buffered in the buffer memory 8 between the time when the release button is fully pressed and after the second predetermined time (0.4 second in this embodiment). Generate moving image data. Specifically, as shown in FIG. 3, frame images (see FIG. 3) stored in the buffer memory 8 at a first frame rate (for example, 60 fps) equivalent to the frame rate (shooting frame rate) output from the image sensor 6. 3) is read from the buffer memory 8 at a second frame rate lower than the first frame rate (for example, 24 fps which is 1 / 2.5 of the first frame rate), and the read frame image is controlled. The moving image generating unit 5b in the unit 4 performs the resizing to the resolution for moving images (from large resolution to small) and the image processing for moving images (including the moving image compression process), so that the slow moving image Image data (frame rate conversion data shown in FIG. 3) is created. Therefore, the reproduction time of the slow moving image created in this embodiment is 2.5 seconds as shown in FIG.

  In addition, when the release button is fully pressed (S2), the control unit 4 generates still image data based on the frame image based on the imaging signal output from the imaging element 6. Specifically, when the release button is fully pressed (S2), the frame image stored in the buffer memory 8 is read from the buffer memory 8, and the still image generation in the control unit 4 is generated for the read frame image. Still image data is generated by performing still image processing (including still image compression processing) by the unit 5c.

  Further, the control unit 4 stores the audio data stored in the audio data storage unit 17 in the audio data storage unit 17 within 0.6 seconds before the release button is fully pressed (S2). Buffered audio data and audio data (raw audio data) buffered in the audio data storage unit 17 from when the release button is fully pressed (S2) to 0.4 seconds later, that is, a slow movie The audio data buffered in the audio data storage unit 17 is extracted during the period in which the frame image used for image generation is stored in the buffer memory 8, and the extracted audio data is processed and reproduced when the slow moving image is reproduced. Audio data to be generated. That is, as shown in FIG. 3, the extracted voice data for one second is overlapped by 0.5 seconds and repeatedly connected four times to generate voice data for 2.5 seconds.

  Next, the control unit 4 records the generated slow moving image data, the generated still image data, and the generated audio data in the recording medium 18 in association with each other.

  Next, with reference to a time chart shown in FIG. 4, processing when playing back a slow moving image, a still image, and sound in the electronic camera 2 according to the embodiment will be described. First, when the user operates the command dial to set the playback mode, the control unit 4 is based on the slow moving image based on the slow moving image data recorded on the recording medium 10, the still image based on the still image data, and the audio data. Transition to a playback mode for playing back audio. And it is discriminate | determined whether the reproduction | regeneration instruction | indication of the slow moving image was made through the operation part 14 by the user.

  When an instruction to reproduce the slow moving image is issued, the control unit 4 records the slow moving image data of the slow moving image for which the reproduction instruction has been issued, and still image data and audio data recorded in association with the slow moving image data. Reading from the medium 10 starts reproduction of slow moving images and audio.

  Here, in order to impress that the slow moving image is reproduced, the control unit 4 fades in the slow moving image over a predetermined time (0.5 seconds in this embodiment) from the start of the reproduction of the slow moving image. Perform transition effect processing. That is, as shown in FIG. 4, playback of the slow moving image is started while fading in the slow moving image. Further, the control unit 4 performs processing on the sound such as the sound fading in over a predetermined time (0.5 seconds in this embodiment) from the start of the sound reproduction.

  In FIG. 4, the time for performing the transition effect on the slow moving image and the time for processing the sound at the start of playback of the slow moving image and sound are the same, but they are different lengths. May be.

  Then, the control unit 4 continues to reproduce the slow moving image and the sound, and from the end of the slow moving image reproduction to a predetermined time (0.5 seconds in this embodiment) before the end of the slow moving image reproduction. Transition effect processing such as fading out slow moving images is performed. Thereafter, the still image is reproduced for 7.5 seconds, but transition effect processing such as fading in the still image is performed over a predetermined time (0.5 seconds in this embodiment) from the start of reproduction of the still image. Also. Transition effect processing such as fading out of a still image is performed over a predetermined time (in this embodiment, 0.5 seconds) from the end of reproduction of the still image. Here, the reproduction of the sound continues until the end of the reproduction of the still image. That is, by reproducing the sound for 2.5 seconds four times, the sound is reproduced for 10 seconds which is the reproduction time of the slow moving image and the still image. Note that the audio reproduction may be terminated at the end of reproduction of the slow moving image. Here, when the user gives an instruction to stop displaying the still image via the operation unit 14 during reproduction of the still image, the control unit 4 stops displaying the still image. In this case, the reproduction of the sound is stopped when the display of the still image is stopped.

  According to the electronic camera according to this embodiment, it is possible to record a sound representing the atmosphere at the time of shooting together with the slow moving image. In addition, when reproducing this slow moving image, it is possible to reproduce sound expressing the atmosphere at the time of shooting.

  In the above-described embodiment, the raw audio data buffered in the period in which the frame image used for generating the slow moving image is stored is extracted, and the extracted raw audio data is partially overlapped and repeatedly connected. In addition, audio data to be played back when playing slow moving images and still images is generated, but audio data to be played back when playing slow moving images and still images by repeatedly connecting the extracted raw audio data without overlapping them. May be generated.

  Further, in the above-described embodiment, the playback time of the slow moving image and the still image is 10 seconds. However, since the audio data storage unit 17 buffers the live audio data for 10 seconds, The live audio data for 10 seconds may be reproduced as it is when the image and the still image are reproduced.

  In the above-described embodiment, the live sound data from the time when the release button is fully pressed (S2) to the third predetermined time before and the live sound data from the time when the release button is fully pressed (S2) to the fourth predetermined time after. The audio data to be reproduced at the time of reproduction of the slow moving image and the still image is generated using the data, but the slow moving image and the still image are used by using an arbitrary part of the raw audio data stored in the audio data storage unit 17. Audio data to be reproduced may be generated at the time of reproduction.

  Further, in the above-described embodiment, the raw audio data buffered in the period in which the frame image used for the generation of the slow moving image is stored is extracted, and the extracted raw audio data is repeatedly connected to the slow motion image. Audio data to be reproduced at the time of playback of moving images and still images is generated. A plurality of audio data having different frequencies are created from the extracted raw audio data, and the audio data having different frequencies are overlapped to create a slow moving image and You may make it produce | generate the audio data reproduced | regenerated at the time of reproduction | regeneration of a still image.

  In the above-described embodiment, when the raw audio data is superimposed, the degree of superimposition may be determined by image information recognized by face recognition or the like. For example, even when the control unit 4 performs face recognition processing on the frame image and determines the degree of overlaying the raw voice data (the width to be superposed) and the number of raw voice data to be superposed according to the number of recognized faces. Good. As a result, when a plurality of persons are shown in the frame image constituting the slow moving image data, the atmosphere at the time of shooting can be reflected in the audio data.

  In addition, when superimposing the raw audio data, the raw audio data to be superimposed are processed so as to have different pitches, or the raw audio data to be superimposed are processed to have different playback speeds, and a plurality of processed The raw voice data may be superimposed. Further, each raw voice data may be processed so as to have a different volume, and a plurality of processed raw voice data may be superimposed. Further, by superimposing a plurality of raw audio data while being shifted little by little, sound that seems to be echoed may be reproduced when reproducing a slow moving image and a still image.

  Further, in the above-described embodiment, a portion of the raw voice data including a characteristic sound corresponding to the shooting scene is extracted from the raw voice data stored in the voice data storage unit 17, and the extracted raw voice data is processed. Then, audio data to be reproduced when reproducing slow moving images and still images may be generated. In this case, the shooting scene may be recognized based on the set shooting mode, or the shooting scene may be recognized by analyzing the raw voice data stored in the voice data storage unit 17.

  In the above-described embodiment, a part of the raw voice data including the peak value of the voice level is extracted from the raw voice data stored in the voice data storage unit 17, and a plurality of the extracted raw voice data are connected. Alternatively, audio data to be reproduced when reproducing slow moving images and still images may be generated by performing processing such as superimposing a plurality of images.

  In the above-described embodiment, when a part of the buffered audio is used in the audio data storage unit 17, a plurality of parts of the buffered raw audio data are extracted, and the extracted The audio data may be processed to generate audio data to be played back when the slow moving image and the still image are played back.

DESCRIPTION OF SYMBOLS 2 ... Electronic camera, 4 ... Control part, 5 ... Video signal processing circuit, 5a ... Through image generation part, 5b ... Moving image generation part, 5c ... Still image generation part, 6 ... Imaging device, 8 ... Buffer memory, 10 ... Recording medium 12 ... Display unit 14 ... Operation unit 16 ... Speaker 17 ... Audio data storage unit 18 ... Amplifier / A / D conversion unit 20 ... Microphone

Claims (9)

An image sensor for imaging light from a subject;
A storage unit for sequentially storing a plurality of frame images based on imaging signals from the imaging element;
Based on the plurality of frame images stored in the storage unit during a predetermined time, playback at a second frame rate less than a first frame rate indicating the number of frame images stored in the storage unit per unit time A moving image data generation unit for generating data of the slow motion moving image to be generated;
A raw voice data recording unit for recording raw voice data related to the slow motion moving image;
An audio data generation unit that processes the raw audio data and generates audio data to be reproduced when the slow motion video is reproduced;
An image reproduction unit for reproducing the slow motion video based on the data of the slow motion video;
An imaging apparatus comprising: an audio reproduction unit that reproduces the audio data generated by the audio data generation unit when reproducing the slow motion moving image.   2. The raw voice data recording unit records the raw voice data during the predetermined time at at least one time before the predetermined time and after the predetermined time. The imaging device described. A still image data generation unit that generates data of at least one still image related to the plurality of frame images stored in the storage unit during the predetermined time;
The image reproduction unit reproduces the at least one still image following reproduction of the slow motion moving image,
The imaging apparatus according to claim 1, wherein the audio reproduction unit reproduces the audio data generated by the audio data generation unit when reproducing the at least one still image.   The said audio | voice data production | generation part produces | generates the said audio | voice data by connecting repeatedly the raw audio | voice data recorded by the said raw audio | voice data recording part, The Claim 1 characterized by the above-mentioned. Imaging device.   The said audio | voice data production | generation part produces | generates the said audio | voice data by repeatedly connecting at least one part the raw audio | voice data recorded by the said raw audio | voice data recording part repeatedly. The imaging device according to item.   The imaging apparatus according to claim 5, wherein the audio data generation unit generates the audio data by superimposing a plurality of audio data having different frequencies created from the raw audio data.   The imaging device according to claim 5 or 6, wherein the audio data generation unit changes a degree of superimposition of the audio data based on image information of the slow motion moving image.   8. The imaging apparatus according to claim 1, wherein the audio data processed by the audio data generation unit is audio data including a characteristic sound corresponding to a shooting mode. .   The imaging apparatus according to any one of claims 1 to 7, wherein the audio data processed by the audio data generation unit is audio data including a peak value.

Images