JP2018152766A - File generation device, file generating method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to easily reproduce a sound corresponding to a still image when reproducing the still image by a general reproduction apparatus.SOLUTION: The file generation device includes: first acquisition means for acquiring still image data; second acquisition means for acquiring voice data; and generation means for generating one moving image file from the still image data and the voice data so that a still image on the basis of the still image data is reproduced during a period in which a sound on the basis of the voice data is reproduced.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a file generation device, a file generation method, and a program.

  There is an imaging device (such as a digital video camera) that has a function of extracting one frame of image data of recorded moving image data (moving image data representing a subject image generated by moving image shooting) from moving image data as still image data. To do. By using this function, it is possible to extract still image data from moving image data even when an operation for recording still image data is not performed during moving image shooting. Can be easily obtained.

  Further, as a technique for leaving the atmosphere at the time of moving image shooting clearer, a technique has been proposed in which desired audio data is also extracted from audio data of moving image with sound when moving still image data from moving image data of moving image with sound. (Patent Document 1). When reproducing a still image based on the cut out still image data, in order to easily reproduce sound based on the audio data cut out together with the still image data, it is necessary to associate the still image data with the audio data.

  As a method for the association, a method for generating one still image file from still image data and audio data has been proposed. In addition, a method has been proposed in which a still image data file and an audio data file are generated separately, and management information for associating these data files is managed by a management file.

JP 2006-295575 A

  However, in the method of generating one still image file from still image data and audio data, a still image file having a special file structure is generated. Therefore, in order to reproduce a still image and sound from a still image file, a dedicated reproduction device capable of interpreting a special file structure is required.

  Also, when still image data and audio data files are generated separately, and management information for associating those data files is managed in the management file, still images and audio are played back. In order to do this, a dedicated playback device is required. Specifically, a dedicated playback device that can interpret the management file is required.

  As described above, in the related art, unless a dedicated playback device is used, when playing back a still image, it is not possible to easily play back sound corresponding to the still image. That is, when a general reproduction apparatus is used, when reproducing a still image, it is not possible to easily reproduce the sound corresponding to the still image.

  It is an object of the present invention to provide a technique that enables a sound corresponding to a still image to be easily played back by a general playback device when the still image is played back.

The first aspect of the present invention is:
First acquisition means for acquiring still image data;
Second acquisition means for acquiring audio data;
Generating means for generating one moving image file from the still image data and the audio data so that a still image based on the still image data is reproduced during a period in which sound based on the audio data is reproduced;
A file generation device characterized by comprising:

The second aspect of the present invention is:
Acquiring still image data;
Obtaining audio data;
Generating one moving image file from the still image data and the audio data so that a still image based on the still image data is reproduced during a period in which sound based on the audio data is reproduced;
A file generation method characterized by comprising:

  A third aspect of the present invention is a program for causing a computer to function as each unit of the file generation apparatus described above.

  According to the present invention, when a still image is reproduced, sound corresponding to the still image can be easily reproduced by a general reproduction device.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to the present embodiment. 6 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the first embodiment. The figure which shows an example of the audio | voice acquisition period which concerns on Example 1. FIG. 10 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a specific example of processing of the imaging apparatus according to the second embodiment. 10 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the third embodiment. FIG. 10 is a diagram illustrating a specific example of processing of the imaging apparatus according to the third embodiment.

  Hereinafter, embodiments of the present invention will be described. FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus according to the present embodiment.

  The photographing lens 100 guides light from the subject to the image sensor 101. As the photographing lens 100, a single focus lens, a zoom lens, or the like can be used.

  The image sensor 101 converts a subject image (subject image; optical image) formed on the image sensor 101 by the photographing lens 100 into an analog signal, and outputs the analog signal. As the image sensor 101, a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) sensor, or the like can be used.

  The A / D converter 102 converts the analog signal output from the image sensor 101 into a digital signal (photographed image data; image data representing a subject image). Then, the A / D converter 102 outputs captured image data.

The microphone 103 collects sound to generate an analog signal representing the collected sound. Then, the microphone 103 outputs the generated analog signal.

  The A / D converter 104 converts the analog signal output from the microphone 103 into a digital signal (sound data representing collected sound). Then, the A / D converter 104 outputs audio data.

  The microcomputer 105 controls processing of each functional unit of the imaging apparatus.

  The volatile memory 106 temporarily stores captured image data output from the A / D converter 102, audio data output from the A / D converter 104, and the like. The volatile memory 106 temporarily stores data (still image data, moving image data, audio data, etc.) extracted from the data file (moving image file, still image file, etc.).

  In the nonvolatile memory 111, a program executed by the microcomputer 105 is recorded in advance.

  The still image codec 107 performs compression encoding (compression conversion) of still image data recorded in the volatile memory 106. For example, the still image codec 107 performs compression encoding that converts the data format of still image data into a JPEG format.

  The moving image codec 108 performs compression encoding of moving image data recorded in the volatile memory 106, decompression decoding (decompression conversion) of moving image data recorded in the volatile memory 106, and the like. For example, the moving image codec 108 changes the data format of moving image data to H.264. H.264 is the compression encoding for converting to H.264 format, and the data format of moving image data is H.264. For example, decompression / combining to convert from H.264 format is performed.

  The audio codec 109 performs compression encoding of the audio data recorded in the volatile memory 106, decompression decoding of the audio data recorded in the volatile memory 106, and the like. For example, the audio codec 109 performs compression encoding for converting the data format of the audio data to the AAC format, decompression decoding for converting the data format of the audio data from the AAC format, and the like.

  The file generation unit 110 generates a still image file by generating header information for still image data after compression encoding of the still image codec 107. For example, the file generation unit 110 generates a still image file in JPEG format. In addition, the file generation unit 110 multiplexes the moving image data after the compression encoding of the moving image codec 108 and the audio data after the compression encoding of the audio codec 109. Then, the file generation unit 110 generates a moving image file by generating header information for the multiplexed data. For example, the file generation unit 110 generates a moving image file in MP4 format. Here, the moving image file is a data file of a moving image with sound.

  The external memory 112 stores data files (still image files, moving image files, etc.) generated by the file generation unit 110. For example, the microcomputer 105 records the data file generated by the file generation unit 110 in the external memory 112. As the external memory 112, a compact flash, smart media, a hard disk drive, or the like can be used. The external memory 112 is detachable from the imaging device. Note that a storage unit built in the imaging apparatus may be used instead of the external memory 112.

The display member 113 displays images (moving images, still images, etc.) on the screen. For example, display member 1
13 displays a moving image (moving image with sound) based on a moving image file recorded in the external memory 112, a still image based on a still image file recorded in the external memory 112, and the like (reproduction processing). The imaging device is provided with a speaker (not shown), and sound is emitted from the speaker. A liquid crystal monitor or the like can be used as the display member 113.

  The operation member 114 receives a user operation on the imaging apparatus. For example, a user operation for taking a still image, a user operation for taking a moving image, a user operation for switching the operation mode of the imaging apparatus, a user operation for changing a shooting parameter, and the like are performed. Still image shooting is shooting for obtaining still image data as captured image data. Moving image shooting is shooting for obtaining moving image data as captured image data. As the operation member 114, a switch, a button, a touch panel, a controller, or the like can be used. The touch panel may or may not be provided on the screen of the display member 113.

<Example 1>
Embodiment 1 of the present invention will be described below. The imaging apparatus according to the present embodiment has the configuration shown in FIG. FIG. 2 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

  First, in S <b> 201, the microcomputer 105 reproduces a moving image with sound based on a moving image file recorded in the external memory 112. For example, the microcomputer 105 selects one of a plurality of moving image files recorded in the external memory 112 in response to a user operation on the imaging device. Then, the microcomputer 105 reproduces the moving image with sound based on the selected moving image file. The sound of the moving image with sound is displayed on the screen of the display member 113, and the sound of the moving image with sound is emitted from a speaker (not shown). The microcomputer 105 selects one frame of the reproduced moving image with sound in response to a user operation on the imaging apparatus. Hereinafter, the moving image file in which the moving image with sound is reproduced in S201 is referred to as “reproduced moving image file”, and the frame selected in S201 is referred to as “selected frame”.

  Next, in S202, the microcomputer 105 determines whether or not recompression coding is necessary to acquire (extract; cut out) the image data of the selected frame as still image data from the reproduced moving image file. If it is determined that recompression encoding is not necessary, the process proceeds to S203, and if it is determined that recompression encoding is necessary, the process proceeds to S210.

  For example, in the playback video file, if the image data of the selected frame is image data of intra-frame compression, the image data of the selected frame is acquired from the playback video file as still image data without performing recompression encoding. can do. Therefore, in such a case, it is determined that recompression encoding is not necessary. On the other hand, in the playback video file, if the image data of the selected frame is inter-frame compressed image data, the image data of the selected frame is acquired from the playback video file as still image data unless recompression encoding is performed. Can not do it. Therefore, in such a case, it is determined that recompression encoding is necessary.

  Note that a method for determining whether or not recompression encoding is necessary is not particularly limited. For example, when the data format (compression format) of the video data included in the playback video file is JPEG format, it is determined that recompression encoding is not necessary, and in other cases, recompression encoding is required. It may be determined that

In S <b> 203, the microcomputer 105 acquires the image data of the selected frame as still image data from the reproduced moving image file (moving image data included in the reproduced moving image file), and records the acquired still image data in the volatile memory 106. To do. Then, the process proceeds to S204.

  In S210, the moving image codec 108 performs decompression decoding of the moving image data included in the reproduced moving image file. The microcomputer 105 acquires the image data of the selected frame as still image data from the moving image data after being decompressed and combined. The still image codec 107 performs compression encoding of still image data acquired by the microcomputer 105. The microcomputer 105 records the still image data after compression encoding in the volatile memory 106. Thereafter, the process proceeds to S204.

  In S204, microcomputer 105 determines the period of audio data acquired from the playback video file (audio data included in the playback video file). Hereinafter, the period determined in S204 is referred to as “voice acquisition period”. In the present embodiment, at least a part of the period of the moving image with sound based on the reproduced moving image file is determined as the sound acquisition period.

  In S205, the microcomputer 105 determines (calculates) the length (time) of the voice acquisition period. Then, the microcomputer 105 determines whether or not the length of the voice acquisition period is equal to or greater than a threshold value. If it is determined that the length of the voice acquisition period is equal to or greater than the threshold, the process proceeds to S207. If it is determined that the length of the voice acquisition period is less than the threshold, the process proceeds to S211. It is done. The threshold value compared with the length of the voice acquisition period may be a fixed value determined in advance by the manufacturer, or may not be so. For example, the threshold value compared with the length of the voice acquisition period may be a value specified by the user.

  In S211, the microcomputer 105 gives a predetermined notification to the user. For example, the microcomputer 105 performs control to display a message indicating that the length of the voice acquisition period is insufficient on the screen of the display member 113. Then, the process returns to S204, and the voice acquisition period is determined again. The predetermined notification is not limited to displaying a predetermined message. As the predetermined notification, display of a predetermined icon, output of a predetermined sound, lighting of a predetermined lamp, lighting of a lamp with a predetermined lighting pattern, and the like may be performed.

  In S207, the microcomputer 105 acquires the audio data in the audio acquisition period from the reproduction moving image file (audio data included in the reproduction moving image file), and records the acquired audio data in the volatile memory 106.

  Next, in S208 and S209, the still image data and audio of the selected frame are reproduced so that the still image based on the still image data of the selected frame is reproduced during the period in which the audio based on the audio data of the audio acquisition period is reproduced. One moving image file is generated from the audio data in the acquisition period. Audio data and still image data are read from the volatile memory 106 and used.

  Specifically, in S208, the microcomputer 105 determines (calculates) the frame rate of the moving image data included in the moving image file based on the length of the audio acquisition period. In this embodiment, the moving image file is generated so that the moving image data includes only one frame corresponding to the still image data of the selected frame. Therefore, the microcomputer 105 determines a frame rate corresponding to the still image reproduction time based on the still image data. For example, when the length of the voice acquisition period is 5 seconds, the frame rate is determined so that the playback time of one frame is 5 seconds. Specifically, a frame rate of 0.2 (= 1/5) fps is determined.

In step S <b> 209, the microcomputer 105 reads out still image data of the selected frame and audio data during the audio acquisition period from the volatile memory 106. Then, the file generation unit 110 multiplexes the read data, and adds header information for controlling the frame rate of the moving image data to the frame rate determined in S208 for the multiplexed data. Generate. As a result, a moving image file is generated in which the moving image data includes only one frame corresponding to the still image data of the selected frame, and the playback time of one frame is equal to the length of the audio acquisition period. Thereafter, the microcomputer 105 records the generated moving image file in the external memory 112.

  Note that the processing flow of the imaging apparatus according to the present embodiment is not limited to the above processing flow. For example, the processes of S206 and S211 may be omitted. The process may proceed from S205 to S207.

  FIG. 3 shows an example of a voice acquisition period. In FIG. 3, the direction of the arrow on the time axis is the direction of time passage. FIG. 3 shows an image of each frame of the reproduced moving image file, an audio waveform of the reproduced moving image file, and an audio acquisition period. The frame rate of the playback moving image file is, for example, 30 fps. In FIG. 3, an image n is an image of a selected frame.

  The microcomputer 105 can automatically determine the period corresponding to the time position of the selected frame as the voice acquisition period. For example, the microcomputer 105 can automatically determine a period s1 from the time position before the first time to the time position of the selected frame with respect to the time position of the selected frame as the voice acquisition period. In the example of FIG. 3, the period s1 includes the time position of the selected frame, but the period up to the time position of the selected frame may not include the time position of the selected frame. The first time may or may not be a fixed time predetermined by the manufacturer. For example, the first time may be a time specified by the user.

  The microcomputer 105 can also automatically determine the period s2 from the time position of the selected frame to the time position after the second time with respect to the time position of the selected frame as the voice acquisition period. In the example of FIG. 3, the period s2 does not include the time position of the selected frame, but the period from the time position of the selected frame may include the time position of the selected frame. The second time may or may not be a fixed time predetermined by the manufacturer. For example, the second time may be a time specified by the user.

  The microcomputer 105 can also automatically determine the period s3 straddling the time position of the selected frame as the voice acquisition period. The period s3 is a period from the time position before the third time with respect to the time position of the selected frame to the time position after the fourth time with respect to the time position of the selected frame. In the example of FIG. 3, the time position at the center of the period s3 matches the time position of the selected frame. That is, the third time is equal to the fourth time. Note that the time position at the center of the voice acquisition period that straddles the time position of the selected frame does not have to coincide with the time position of the selected frame. That is, the third time may be different from the fourth time. The third time may be longer than the fourth time, or the third time may be shorter than the fourth time. Each of the third time and the fourth time may or may not be a fixed time predetermined by the manufacturer. For example, at least one of the third time and the fourth time may be a time designated by the user.

  The microcomputer 105 can also determine the period s4 designated by the user as the voice acquisition period. For example, the user designates the period s4 using the operation member 114 while confirming the moving image with sound displayed on the screen of the display member 113. The period s4 is an arbitrary period. The period s4 may be a period related to the selected frame, or may not be a period related to the selected frame.

  As described above, according to the present embodiment, from the data so that the still image based on the still image data of the selected frame is reproduced during the period in which the audio based on the audio data in the audio acquisition period is reproduced. One moving image file having a general file structure is generated. A moving image with sound based on a moving image file having a general file structure can be easily reproduced by a general reproduction device. Therefore, according to the present embodiment, when playing back a still image, the sound corresponding to the still image can be easily played back by a general playback device.

<Example 2>
Embodiment 2 of the present invention will be described below. In the following, points (configuration, processing, etc.) different from those in the first embodiment will be described in detail, and descriptions of the same points as in the first embodiment will be omitted. The imaging apparatus according to the present embodiment has the configuration shown in FIG. FIG. 4 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

  First, in S401, buffering of audio data is started. In this embodiment, the buffering of the audio data is started at the timing when the state of the imaging apparatus becomes ready for photographing. Note that the timing at which audio data buffering is started is not particularly limited. For example, buffering of audio data may be started in response to a user operation on the imaging apparatus.

  The audio data buffering will be specifically described. The microphone 103 collects sound to generate an analog signal representing the collected sound. The A / D converter 104 converts the analog signal generated by the microphone 103 into audio data that is a digital signal. The microcomputer 105 records the audio data generated by the A / D converter 104 in the volatile memory 106. The microcomputer 105 reads the audio data generated by the A / D converter 104 from the volatile memory 106 and supplies the read audio data to the audio codec 109. The audio codec 109 performs compression encoding of the supplied audio data. The microcomputer 105 records the audio data after compression encoding in the volatile memory 106.

  In the imaging apparatus, a maximum time (upper limit time; maximum holding time) of audio data that can be held in the volatile memory 106 is set in advance. Here, consider a case where audio data having a time longer than the maximum holding time is recorded in the volatile memory 106 by buffering of the audio data. In this case, the old audio data is deleted from the volatile memory 106 so that the audio data of the maximum holding time until the current timing is held by the volatile memory 106. Note that the maximum holding time may or may not be a fixed time predetermined by the manufacturer. For example, the maximum holding time may be a time specified by the user.

  Next, in S <b> 402, the user performs a user operation for taking a still image using the operation member 114. In step S403, the audio data buffering is stopped in response to a user operation for taking a still image. Next, in S404, still image shooting is performed in response to a user operation for performing still image shooting. Thereby, still image data is acquired.

The still image shooting will be specifically described. The image sensor 101 converts the subject image formed on the image sensor 101 by the photographing lens 100 into an analog signal. The A / D converter 102 converts the analog signal generated by the image sensor 101 into still image data that is a digital signal and is captured image data. The microcomputer 105 records the still image data generated by the A / D converter 102 in the volatile memory 106. The microcomputer 105 reads the still image data generated by the A / D converter 102 from the volatile memory 106 and supplies the read still image data to the still image codec 107.
The still image codec 107 performs compression encoding of the supplied still image data. Then, the microcomputer 105 records the still image data after compression encoding in the volatile memory 106.

  In step S <b> 405, the microcomputer 105 determines whether the time of the audio data held in the volatile memory 106 is equal to or longer than the minimum recording time. Here, the sound data held in the volatile memory 106 is sound data recorded in the volatile memory 106 by sound data buffering. If it is determined that the time of the audio data held in the volatile memory 106 is equal to or longer than the minimum recording time, the process proceeds to S406. If it is determined that the time of the audio data held in the volatile memory 106 is less than the minimum recording time, the process proceeds to S411. Note that the minimum recording time may or may not be a fixed time determined in advance by the manufacturer. For example, the minimum recording time may be a time specified by the user.

  In S411, a still image file is generated from the still image data recorded in the volatile memory 106 by the still image shooting in S404. Specifically, the microcomputer 105 reads out the still image data obtained by the still image shooting in S <b> 404 from the volatile memory 106. Then, the file generation unit 110 generates a still image file from the read still image data. Thereafter, the microcomputer 105 records the generated still image file in the external memory 112.

  In step S406, the microcomputer 105 determines whether the time of the audio data held in the volatile memory 106 is equal to or longer than the maximum recording time. Here, the sound data held in the volatile memory 106 is sound data recorded in the volatile memory 106 by sound data buffering. If it is determined that the time of the audio data held in the volatile memory 106 is equal to or longer than the maximum recording time, the process proceeds to S407. If it is determined that the time of the audio data held in the volatile memory 106 is less than the maximum recording time, the process proceeds to S412. The maximum recording time is longer than the minimum recording time and not longer than the maximum holding time. Note that the maximum recording time may or may not be a fixed time predetermined by the manufacturer. For example, the maximum recording time may be a time specified by the user.

  In step S <b> 412, the microcomputer 105 reads the audio data (entire) recorded in the volatile memory 106 by the audio data buffering from the volatile memory 106. Thereafter, the process proceeds to S408.

  In step S <b> 407, the microcomputer 105 reads, from the volatile memory 106, the audio data for the maximum recording time recorded in the volatile memory 106 by the buffering of the audio data until the timing when the buffering is completed. Thereafter, the process proceeds to S408.

  In S408 to S410, a moving image file is generated. In the present embodiment, in order to reproduce the still image based on the still image data obtained by the still image shooting of S404 during the period in which the audio based on the audio data read in S407 or S412 is reproduced. One moving image file is generated from the data.

  Specifically, in S408, the microcomputer 105 determines (calculates) the time of the audio data read in S407 or S412.

In S409, the microcomputer 105 determines based on the time determined in S408.
Determine (calculate) the frame rate. In S409, the frame rate is determined by a process similar to the process of S208 in FIG.

  In S410, the microcomputer 105 reads out the still image data obtained by the still image shooting in S404 from the volatile memory 106. The file generation unit 110 multiplexes the read still image data and the audio data read in S407 or S412. Then, the file generation unit 110 generates header information for the multiplexed data to control the frame rate of the moving image data to the frame rate determined in S409. As a result, a moving image file is generated in which the moving image data includes only one frame corresponding to the still image data obtained by the still image shooting in S404 and the reproduction time of one frame is equal to the time determined in S408. Thereafter, the microcomputer 105 records the generated moving image file in the external memory 112.

  Note that the processing flow of the imaging apparatus according to the present embodiment is not limited to the above processing flow. For example, at least a part of the processing of S405, S406, S407, and S411 may be omitted. Processing may proceed from S404 to S412. If it is determined that the time of the audio data held in the volatile memory 106 is equal to or longer than the minimum recording time, the process may proceed from S405 to S412.

  FIG. 5 shows a specific example of processing of the imaging apparatus according to the present embodiment. In FIG. 5, the direction of the arrow on the time axis is the elapsed time direction. FIG. 5 shows a waveform of buffered audio data, a still image based on still image data acquired by still image shooting, various periods, and various timings.

  At timing t1, buffering of audio data is started (S401). At timing t2, a user operation for still image shooting is performed (S402), buffering of audio data is stopped (S403), and still image shooting (acquisition of still image data) is performed (S404).

  The audio data is buffered in the period s5 from the timing t1 to the timing t2, but the period s5 is longer than the maximum holding time. Therefore, in audio data buffering, old audio data is deleted from the volatile memory 106 so that the audio data having the maximum retention time up to the current timing is held by the volatile memory 106. As a result, at the timing t2, the volatile memory 106 holds the audio data in the period s5 'up to the timing t2. The time of the period s5 'is equal to the maximum holding time.

  The minimum recording time is shorter than the maximum holding time, and the maximum recording time is less than the maximum holding time. Therefore, the process proceeds from S405 to S406, and the process proceeds from S406 to S407. Here, it is assumed that the maximum recording time is equal to the maximum holding time. Therefore, the audio data of the period s5 'is acquired from the volatile memory 106 (S407), the time of the period s5' is determined (S408), and the frame rate is determined based on the time of the period s5 '(S409). Then, one moving image file is generated from the still image data acquired by still image shooting, the audio data acquired from the volatile memory 106, and the determined frame rate (S410).

As described above, according to the present embodiment, still image data is acquired by performing still image shooting, and audio data is acquired by collecting sound during a period corresponding to the timing at which still image shooting is performed. To be acquired. Then, one moving image file having a general file structure is generated from the data so that a still image based on the acquired still image data is reproduced during a period in which the sound based on the acquired audio data is reproduced. Is generated. Thereby, when reproducing a still image, the sound corresponding to the still image can be easily reproduced by a general reproduction apparatus.

<Example 3>
Embodiment 3 of the present invention will be described below. In the following, differences (configuration, processing, etc.) from the first and second embodiments will be described in detail, and the description of the same points as the first and second embodiments will be omitted. The imaging apparatus according to the present embodiment has the configuration shown in FIG. FIG. 6 is a flowchart illustrating an example of a processing flow of the imaging apparatus according to the present embodiment.

  First, in S <b> 601, the user performs a user operation for taking a still image using the operation member 114. Next, in S602, still image shooting is performed in response to a user operation for performing still image shooting. Thereby, still image data is acquired. In step S <b> 603, buffering of audio data is started in response to a user operation for performing still image shooting. Next, in S604, audio data is buffered.

  In step S <b> 605, the microcomputer 105 determines whether the time of the audio data held in the volatile memory 106 has reached the necessary recording time. Here, the sound data held in the volatile memory 106 is sound data recorded in the volatile memory 106 by sound data buffering. If it is determined that the time of the audio data held in the volatile memory 106 has not reached the required recording time, the process returns to S604. The audio data buffering is continued until the time of the audio data held in the volatile memory 106 reaches the required recording time. When the time of the audio data held in the volatile memory 106 reaches the required recording time, the process proceeds from S605 to S606. The required recording time may be a fixed time predetermined by the manufacturer or may not be so. For example, the necessary recording time may be a time specified by the user.

  In S606, buffering of audio data is stopped. In step S <b> 607, the microcomputer 105 reads out the audio data recorded in the volatile memory 106 by the audio data buffering from the volatile memory 106.

  Then, in S608 to S610, a moving image file is generated. In the present embodiment, during the period in which the sound based on the sound data read out in S607 is reproduced, the still image based on the still image data obtained by the still image shooting in S602 is reproduced from the data. One moving image file is generated.

  Specifically, in S608, the microcomputer 105 determines (calculates) the time of the audio data read out in S607.

  In S609, the microcomputer 105 determines (calculates) the frame rate based on the time determined in S608.

  In step S <b> 610, the microcomputer 105 reads out the still image data obtained by the still image shooting in step S <b> 602 from the volatile memory 106. The file generation unit 110 multiplexes the read still image data and the audio data read in S607. Then, the file generation unit 110 generates header information for controlling the frame rate of the moving image data to the frame rate determined in S609 for the multiplexed data. Thereby, the moving image data includes only one frame corresponding to the still image data obtained by the still image shooting in S602, and the moving image file having the same playback time as the time determined in S608 is generated. Thereafter, the microcomputer 105 records the generated moving image file in the external memory 112.

  Note that the processing flow of the imaging apparatus according to the present embodiment is not limited to the above processing flow. For example, the process of S605 may be omitted. Processing may proceed from S604 to S606.

  FIG. 7 shows a specific example of processing of the imaging apparatus according to the present embodiment. In FIG. 7, the direction of the arrow on the time axis is the direction of time passage. FIG. 7 shows still images based on still image data acquired by still image shooting, waveforms of buffered audio data, various periods, and various timings.

  At timing t3, a user operation for still image shooting is performed (S601), and still image shooting (acquisition of still image data) is performed (S602). Then, buffering of audio data is started at timing t4 when still image shooting is completed (S603), and buffering of audio data is stopped at timing t5 (S606). Therefore, audio data is buffered in a period s6 from timing t4 to timing t5 (S603 to S606). The time of the period s6 is equal to the necessary recording time.

  Thereafter, the audio data of the period s6 is acquired from the volatile memory 106 (S607), the time of the period s6 is determined (S608), and the frame rate is determined based on the time of the period s6 (S609). Then, one moving image file is generated from the still image data acquired by still image shooting, the audio data acquired from the volatile memory 106, and the determined frame rate (S610).

  As described above, according to the present embodiment, still image data is acquired by performing still image shooting, and audio data is acquired by collecting sound during a period corresponding to the timing at which still image shooting is performed. To be acquired. Then, one moving image file having a general file structure is generated from the data so that a still image based on the acquired still image data is reproduced during a period in which the sound based on the acquired audio data is reproduced. Is generated. Thereby, when reproducing a still image, the sound corresponding to the still image can be easily reproduced by a general reproduction apparatus.

  Each functional unit in FIG. 1 may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Each functional unit may be realized by hardware or not. For example, the apparatus may include a processor and a memory in which a control program is stored. The functions of at least some of the functional units included in the apparatus may be realized by the processor reading and executing the control program from the memory.

  In addition, Examples 1-3 are an example to the last, and the structure obtained by changing suitably and changing the structure of Examples 1-3 within the range of the summary of this invention is also contained in this invention. Configurations obtained by appropriately combining the configurations of Examples 1 to 3 are also included in the present invention.

  In the first to third embodiments, an example in which a file generation device that generates a moving image file is provided in the imaging device has been described. However, the file generation device may be a separate device from the imaging device. For example, the file generation device may be provided in a personal computer (PC), a tablet terminal, a smartphone, or the like.

In the first embodiment, an example in which still image data is acquired from moving image data that is captured image data has been described. For example, the moving image data may be broadcast program data, movie data, or the like. In the second and third embodiments, an example in which still image data that is captured image data is acquired by still image capturing has been described. However, still image data is not limited to captured image data. For example, the still image data may be illustration data.

  In the first embodiment, an example in which still image data and audio data are acquired from a moving image with audio has been described. In the second and third embodiments, the example in which audio data is acquired by collecting sound during a period in which still image shooting is performed has been described. However, the acquisition method of audio data is not limited to these. For example, a part of arbitrary music data may be acquired as audio data.

  In the second embodiment, the example in which the period up to the timing at which the still image shooting is performed is used as the period according to the timing at which the still image shooting is performed, that is, the period during which the sound collection is performed. In the third embodiment, the example in which the period from the timing at which the still image shooting is performed is used as the period according to the timing at which the still image shooting is performed. However, the period corresponding to the timing at which still image shooting is performed is not limited to these. For example, as a period corresponding to the timing at which still image shooting is performed, a period across the timing at which still image shooting is performed may be used. Specifically, the processing in S401 and S402 in FIG. 4 may be performed, the processing in S403 may be omitted, the processing in S404 may be performed, and the processing may proceed from S404 to S504 in FIG. By doing so, it is possible to use a period straddling the timing at which the still image shooting is performed as the period according to the timing at which the still image shooting is performed.

  Depending on the type of still image shooting, the period corresponding to the timing when still image shooting is performed, that is, the period during which sound collection is performed may be switched. Here, consider a case where still image shooting using a mechanical shutter and still image shooting using an electronic shutter can be performed. In still image shooting using a mechanical shutter, the shutter is opened by opening and closing a light blocking mechanism that physically blocks light. In still image shooting using an electronic shutter, the physical movement of the shutter is reproduced by controlling the processing of the image sensor. In this case, when still image shooting using a mechanical shutter is performed, even if sound collection is performed during the period up to the timing when the still image shooting is performed or during the period from when the still image shooting is performed. Good. When still image shooting using an electronic shutter is performed, sound collection may be performed during a period across the timing at which still image shooting is performed. The correspondence between the type of still image shooting and the period of sound collection is not particularly limited.

  In the first to third embodiments, the example in which the moving image file including only one frame corresponding to the obtained still image data has been described, but the moving image file includes a plurality of frames each corresponding to the same still image data. But you can. For example, when the time of the audio data is 5 seconds, a moving image file including 150 (= 30 × 5) frames each corresponding to the same still image data is generated as a moving image file having a frame rate of 30 fps. Also good.

  A user operation for deleting the audio data from the moving image file generated from the still image data and the audio data may be performed. When an instruction to delete audio data from the moving image file is issued by the user, the file generation unit 110 may generate a still image file from the still image data used for generating the moving image file. Then, the microcomputer 105 may replace the moving image file recorded in the external memory 112 with the generated still image file. As a result, a still image file having a data size smaller than that of the moving image file can be obtained, so that the storage capacity of the external memory 112 can be used efficiently. Note that the still image data may be acquired from a moving image file, or may be held by the volatile memory 106, the external memory 112, or the like.

The user may be able to select whether to generate a moving image file from still image data and audio data. When the user has selected to generate a moving image file from still image data and audio data, the file generation unit 110 may generate a moving image file from still image data and audio data. Then, when the user has selected not to generate a moving image file from still image data and audio data, the file generation unit 110 may generate a still image file from still image data.

<Other examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100: Shooting lens 101: Image pick-up element 102: A / D converter 103: Microphone 104: A / D converter 105: Microcomputer 106: Volatile memory 107: Still image codec 108: Movie codec 109: Audio codec 110: File Generation unit 111: nonvolatile memory 112: external memory 113: display member 114: operation member

Claims (15)

First acquisition means for acquiring still image data;
Second acquisition means for acquiring audio data;
Generating means for generating one moving image file from the still image data and the audio data so that a still image based on the still image data is reproduced during a period in which sound based on the audio data is reproduced;
A file generation apparatus comprising: The file generation apparatus according to claim 1, wherein the first acquisition unit acquires image data of one frame of a moving image as the still image data. The video is a video with sound,
The file generation apparatus according to claim 2, wherein the second acquisition unit acquires audio data in at least a part of the moving image with audio. The file generation apparatus according to claim 3, wherein the second acquisition unit acquires audio data in a period corresponding to a time position of a frame of the still image data of the moving image with audio. The period corresponding to the time position of the frame of the still image data is a period until the time position of the frame of the still image data, a period from the time position of the frame of the still image data, or the frame of the still image data The file generation apparatus according to claim 4, wherein the file generation apparatus is a period that crosses the time position. The file generation apparatus according to claim 3, wherein the second acquisition unit acquires audio data of the video with audio for a period specified by a user. The file generation apparatus according to claim 1, wherein the first acquisition unit acquires still image data representing a subject image by performing still image shooting. The file generation apparatus according to claim 7, wherein the second acquisition unit acquires the audio data by collecting sound during a period corresponding to a timing at which the still image shooting is performed. The period corresponding to the timing at which the still image shooting is performed includes a period until the timing at which the still image shooting is performed, a period from the timing at which the still image shooting is performed, or a timing at which the still image shooting is performed. The file generation apparatus according to claim 8, wherein the file generation apparatus is a straddling period. The first acquisition means can execute still image shooting using a mechanical shutter and still image shooting using an electronic shutter.
When still image shooting using the mechanical shutter is performed, the second acquisition unit collects sound during a period until the timing when the still image shooting is performed or during a period from the timing when the still image shooting is performed. And
The said 2nd acquisition means collects sound in the period over the timing when the said still image photography is performed, when the still image photography using the said electronic shutter is performed. File generator. Recording means for recording the moving image file in a storage unit;
When an instruction to delete the audio data from the video file is given by the user,
The generation unit generates a still image file from the still image data,
The file generation apparatus according to claim 1, wherein the recording unit replaces the moving image file recorded in the storage unit with the still image file. The user can select whether to generate the video file,
The said production | generation means produces | generates a still image file from the said still image data, when the user has chosen not to produce | generate the said moving image file, The any one of Claims 1-11 characterized by the above-mentioned. File generator. The said production | generation means produces | generates a still image file from the said still image data, when the time of the audio | voice based on the said audio data is less than a threshold value, The Claim 1 characterized by the above-mentioned. File generator. Acquiring still image data;
Obtaining audio data;
Generating one moving image file from the still image data and the audio data so that a still image based on the still image data is reproduced during a period in which sound based on the audio data is reproduced;
A file generation method characterized by comprising:   The program for functioning a computer as each means of the file generation apparatus of any one of Claims 1-13.

Images