JP2003348549A - Apparatus and method for generating moving image data with voice - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make compact and light in weight an apparatus for generating moving image data with voice. <P>SOLUTION: In an image acquiring part 10, image data associated with an object are acquired and in a voice acquiring part 20, voice data associated with voice uttered by a user of the like are acquired. Then, the image data and the voice data are inputted to a data generating part 40 and in the data generating part 40, the image data and the voice data are compression-coded in the time division manner to generate the moving image data with voice in which the compression-coded image data and the compression-coded voice data are multiplexed. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for generating moving image data with sound in a portable communication device or the like.

[0002]

2. Description of the Related Art Conventionally, a portable communication device such as a cellular phone obtains moving image data with sound by acquiring image data from an imaging unit having a CCD and acquiring sound data from a microphone. Video data with sound can be transmitted and received.

[0003] For example, in a moving image data with sound generator disclosed in Japanese Patent Application Laid-Open No. 9-55935, first, image data and sound data are acquired. Then, the compression encoding of the image data and the compression encoding of the audio data are respectively performed in separate compression encoding devices, and thereafter, the compression encoded image data and the compression encoded audio data are multiplexed. The video data with sound can be generated by multiplexing in the plex device.

[0004]

SUMMARY OF THE INVENTION However, the above-described moving picture data with audio data generating apparatus is provided with a device for compressing and encoding image data and a device for compressing and encoding audio data separately. Since a device for multiplexing the compression-encoded image data and the compression-encoded audio data is also provided, the circuit scale of the moving image data generation device with audio becomes large.

[0005] Therefore, the circuit size of the entire portable communication device equipped with the above-described moving image data generating device with sound also becomes large, and in order to reduce the size and weight constantly required for the portable communication device. It is an evil.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and a first object of the present invention is to reduce the size and weight of a moving image data generating apparatus with sound.

A second object of the present invention is to reduce the size and weight of a moving picture data generating apparatus with sound and to suppress a break in the sound generated at the time of reproduction on the communication partner side. It is to provide.

[0008]

According to an aspect of the present invention, there is provided an image acquiring apparatus for acquiring image data, audio acquiring means for acquiring audio data, and compression of the image data. Data that generates moving image data with sound by multiplexing compression-coded image data and compression-coded sound data by performing coding and compression coding of the sound data in a time-division manner. Generating means.

According to a second aspect of the present invention, there is provided the moving picture data with sound according to the first aspect, wherein a compression load and a predetermined threshold value for the compression encoding of the image data in the data generating means are set. It is characterized by further comprising load detecting means for comparison, and storage means for temporarily storing the audio data as temporary storage data when the compression load is larger than the predetermined threshold.

According to a third aspect of the present invention, in the moving image data generating apparatus with audio according to the second aspect, when the compression load is smaller than the predetermined threshold, the temporary storage data is stored in the storage means. If stored, the data generating means performs compression encoding of the temporary storage data,
Control means for controlling the data generation means such that the data generation means performs compression encoding of the audio data newly acquired by the audio acquisition means if the temporary storage data is not stored in the storage means; , Is further provided.

According to a fourth aspect of the present invention, there is provided the moving image data generating apparatus with audio according to the second aspect, wherein the storage means always stores the audio data acquired by the audio acquisition means in the temporary storage data. When the compression load is smaller than the predetermined threshold, the moving image data with audio data reads out the temporary storage data in the order stored in the storage unit and performs compression encoding. And a control unit for controlling the data generation unit.

According to a fifth aspect of the present invention, in the moving image data generating apparatus with audio according to the first aspect, a compression load and a predetermined threshold value for the compression encoding of the image data in the data generating means are set. Load detecting means for comparing, and when the compression load is greater than a predetermined threshold, a compression rate for controlling the data generating means so as to increase a compression rate in the compression encoding of the image data to be higher than a predetermined reference compression rate. And a control unit.

According to a sixth aspect of the present invention, there is provided the moving image data generating apparatus with sound according to any one of the second to fifth aspects, wherein the load detecting means is configured to output the image data from the image obtaining means. And acquiring the compression load based on the acquired information.

According to a seventh aspect of the present invention, there is provided the moving image data generating apparatus with sound according to the first aspect, wherein a compression ratio in the compression encoding of the image data is set to a predetermined standard based on a user operation. The image processing apparatus further includes a compression ratio control unit that controls the data generation unit so as to increase the compression ratio higher than the compression ratio.

[0015] Further, according to the invention of claim 8, a step of obtaining image data, a step of obtaining audio data,
By performing the compression encoding of the image data and the compression encoding of the audio data in a time-division manner, a moving image with audio in which the compression-encoded image data and the compression-encoded audio data are multiplexed. Generating data.

[0016]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of a moving image data generating apparatus with sound to which the present invention is applied.

<Embodiment 1>><1-1. Functional Block of Data Generation Apparatus 1> FIG.
FIG. 2 is a functional block diagram of a moving image data with sound generation device (hereinafter, referred to as a “data generation device”) 1 that generates moving image data with sound according to the first embodiment;

As shown in FIG. 1, the data generating device 1
The image acquisition unit 10, the audio acquisition unit 20, the audio data control unit 30 that controls the output of audio data from the audio acquisition unit 20 to the data generation unit 40, and an audio unit by performing compression encoding of image data and audio data. It is configured by a data generation unit 40 that generates moving image data. In addition,
Although not shown, an operation unit is provided outside the data generation device 1, and when the user operates the operation unit, the image data is acquired by the image acquisition unit 10 and the audio data is acquired by the audio acquisition unit 20. Start or end acquisition of. The data generating device 1 can be used by being incorporated in a mobile device such as a mobile phone, for example, and can also be used in other electronic devices.

The image acquiring section 10 constitutes an image acquiring means for acquiring image data of a subject, and includes an image pickup section 101 and an A / D converter 102.

The imaging unit 101 includes a lens and a CCD, and the CCD captures an optical image of a subject formed by the lens.
The signal is photoelectrically converted into an analog image signal of RGB color components and output. The A / D converter 102 obtains digital color full-color image data by converting each pixel signal of the analog image signal output from the imaging unit 101 into a digital signal, and outputs the digital data. Here,
Since the CCD outputs an analog image signal for 30 frames per second, the image acquisition unit 10 acquires and outputs image data for 30 frames per second.

The voice obtaining section 20 constitutes voice obtaining means for obtaining voice data relating to a voice uttered by a user or the like, and includes a microphone 103 and an A / D converter 104.

The microphone 103 converts a voice uttered by a user or the like into an analog voice signal and outputs it. The A / D converter 104 obtains digital audio data by converting an analog audio signal output from the microphone 103 into a digital signal, and outputs the digital audio data.

The voice data control unit 30 includes the load detection unit 10
6. Memory 107 for temporarily storing audio data, memory detection unit 108 for detecting the volume of audio data stored in memory 107, and switch (SW) for switching the output destination of audio data output from audio acquisition unit 20 Circuit 10
9 and an output control unit 110 for controlling the switching of the output destination of the audio data in the SW circuit 109.

The function of each part of the audio data control unit 30 except for the memory 107 may be realized by a dedicated electronic circuit or realized by a microcomputer (CP).
It may be realized by a program in U).

The load detecting section 106 constitutes load detecting means for comparing a load (hereinafter, referred to as a "compression load") for compressing and encoding image data and audio data in the data generating section 40 with a predetermined threshold value. If the compression load for the compression encoding of the image data (hereinafter, referred to as “image compression load”) is larger than a predetermined threshold, a signal indicating the fact (hereinafter, referred to as “high load signal”) is output. Output to control unit 110. When the image compression load is the same as the predetermined threshold, the image compression load may be included in any of the determination branches as to whether to output the high load signal (the same applies to the following embodiments). Here, the image data and the audio data can be distinguished from each other based on the data amount for generating one packet described later.

The detection of the compression load by the load detection unit 106 is performed, for example, for each image data for generating one image packet or audio packet, which will be described later, by the data amount before compression encoding in the data generation unit 40 and the compression code. This is performed by determining the ratio to the data amount after the conversion.

More specifically, when image data is compression-encoded according to compression-encoding rules such as MPEG1 and MPEG2, if the movement of a subject increases, the amount of data before compression-encoding is reduced. The data amount after encoding relatively increases. And (the amount of data after compression encoding)
When the value of / (data amount before compression encoding) is larger than a predetermined threshold, the load detection unit 106 outputs a high load signal to the output control unit 110.

If the load detection unit 106 does not detect any compression load for a certain period of time after the start of the compression encoding of the image data and the audio data in the data generation unit 40,
A signal indicating that compression encoding of image data and audio data for generating a series of moving image data with audio has been completed (hereinafter, “encoding completion signal”) is output to output control section 110.

When the image compression load detected by the load detection unit 106 is larger than a predetermined threshold, the memory 107 stores the audio acquired by the audio acquisition unit 20 every predetermined period (here, 0.5 seconds). It constitutes storage means for temporarily storing all or a part of the data as temporary storage data.

The memory detecting section 108 detects the capacity of the temporarily stored data stored in the memory 107,
If the temporary storage data is stored in the output control unit 110, a signal indicating this (hereinafter, referred to as a “data presence signal”) is output to the output control unit 110.

The SW (switch) circuit 109 receives audio data from the audio acquisition unit 20 and, under the control of the output control unit 110, converts the received audio data into the data generation unit 40.
Alternatively, the data is output to the memory 107. Also, the SW circuit 109
Under the control of the output control unit 110, the temporary storage data stored in the memory 107 is sequentially read out from the one stored earlier and output to the data generation unit 40.

The output control unit 110 determines the SW circuit 109 based on the high load signal input from the load detection unit 106 and the data presence signal input from the memory detection unit 108.
The switching of the output destination of the audio data in is controlled.

Specifically, when the image compression load of the data generation unit 40 is larger than a predetermined threshold, the output control unit 110 responds to the high load signal input from the load detection unit 106 by using the SW circuit. The control unit 109 controls the audio data to be output to the memory 107 without outputting the audio data to the data generation unit 40.

The output control section 110 compares the image compression load of the data generation section 40 with a predetermined threshold value. If the image compression load is smaller than the threshold value, the memory 107 stores temporary storage data. If so, the memory detection unit 108
Circuit 1 corresponding to the data presence signal input from
09 reads out the temporarily stored data from the memory 107 and outputs it to the data generation unit 40, and controls so that the voice data newly input to the SW circuit 109 from the voice acquisition unit 20 is output to the memory 107. On the other hand, if the temporary storage data is not stored in the memory 107,
The circuit 109 controls the audio data newly input to the SW circuit 109 from the audio acquisition unit 20 to be output to the data generation unit 40.

When generating the moving image data with sound and receiving the encoding completion signal from the load detecting unit 106 at the time of generating the moving image data with sound, the output control unit 110 determines that the generation of the series of moving image data with sound is completed. Then, the output control of the audio data ends.

Therefore, when the image compression load is larger than the predetermined threshold, the audio data input from the audio acquisition unit 20 to the SW circuit 109 is not output from the SW circuit 109 to the data generation unit 40 but is output to the memory 107. Then, the output control unit 110 controls the memory 107 to temporarily store the audio data as temporary storage data.

When the image compression load is smaller than the predetermined threshold, if the temporarily stored data is stored in the memory 107, the SW circuit 109 reads the temporarily stored data from the memory 107 and outputs it to the data generator 40. Then, the output control unit 110 controls the data generation unit 40 to perform compression encoding of the temporarily stored data. On the other hand, memory 1
If the temporary storage data is not stored in 07, the SW circuit 109 is newly acquired by the voice
The voice data input to the W circuit 109 is output to the data generation unit 4
0, and the output control unit 110 controls the data generation unit 40 to compress and encode the audio data.

For details of the timing of the output control of the audio data from the audio data control unit 30 to the data generation unit 40, see the description of the generation of the moving image data with audio and the output control of the audio data to the data generation unit 40. This will be described in detail when describing the flow.

The data generator 40 is not shown, but
CPU, image memory for temporarily storing image data when performing compression encoding of image data, data generation unit 40
RAM for temporarily storing audio data input to the RAM and adjusting the timing of compression encoding, and a data generation unit 4
And a ROM for storing a program for comprehensively controlling various processes in the O.0, and is configured as data generation means for generating audio-added moving image data multiplexed by time division multiplexing.

The data generating section 40 performs compression coding on image data based on compression coding rules such as MPEG1 and MPEG2, and performs compression coding on audio data according to the MPEG recommendation. The moving image data with sound is created, and the moving image data with sound is transmitted to the portable communication device or the like of the communication partner via the transmission unit 130.

Further, the data generation unit 40 performs the compression encoding of the second and subsequent image data 0.5 × N seconds later (where N is an arbitrary number) based on the start time of the first compression encoding of the image data. (Natural number), and outputs the audio data stored in the RAM to the CPU to start compression encoding of the audio data stored in the RAM after 0.4 + 0.5 × N seconds (N is an integer of 0 or more). Is controlled. Generation of moving image data with sound will be described later in detail.

<1-2. Generation of Moving Image Data with Sound><1-2-1. Data Structure of Moving Picture Data with Sound> FIG. 2 is a schematic diagram showing a data structure of moving picture data with sound generated by the data generation device 1 and obtained by multiplexing image data and sound data by time division multiplexing. Yes,
The data generation unit 40 generates the moving image data with sound shown in FIG. 2 in order from the front (in the figure, from the left).

As shown in FIG. 2, moving image data with sound is composed of a group of packs (Pk1, Pk2, Pk3,...).
・) One pack includes a sequence header SH and a plurality of packets Pt1, Pt2,.
, PtN, and one packet is composed of a packet header PH and an elementary stream ES obtained by dividing an individual image or audio stream.

The sequence header SH stores information such as a reference time (SRC) for synchronizing image and sound reproduction, a screen size, a screen aspect ratio, an image display cycle, and the like. In the packet header PH, time information PTS, DTS for synchronizing image and sound reproduction,
The characteristic information of the data in the elementary stream ES is stored.

Then, one elementary stream ES stores data relating to only images or only audio. That is, one packet is a packet in which data relating to only image data is stored (hereinafter, referred to as “image packet”) or a packet in which data relating to only audio data is stored (hereinafter, referred to as “audio packet”). .

The communication partner apparatus separates image data and audio data from audio-added moving image data in which image data and audio data are multiplexed as shown in FIG. The reference time (SRC) stored in the sequence header SH and the time information stored in the packet header PH are stored while storing a certain amount of data in each of the image data storage unit and the audio data storage unit. Based on the PTS and DTS, synchronous playback of a moving image and audio is performed from synchronized image data and audio data.

<1-2-2. Timing of compression encoding>
Next, the timing of compression encoding of image data and audio data in the data generation unit 40 will be described.

The data generating section 40 compresses and encodes image data while temporarily storing image data for 15 frames corresponding to image data acquired for 0.5 second in the image memory. On the other hand, during the compression encoding of the image data, the audio data input to the data generation unit 40 is stored in the RAM, and the RAM is stored between the compression encoding of the image data so that the moving image is not interrupted. Is configured to perform compression encoding of the audio data stored in the.

FIG. 3 is a timing chart for explaining the timing of compression encoding of image data and audio data in the data generation section 40. Here, the horizontal axis indicates time, and time is given with the start time of the time zone P1 of the first compression encoding of image data as a reference (0 [sec]).

As described above, the data generation device 1
In the above, the output control of the audio data to the data generation unit 40 is controlled depending on whether the image compression load for the compression encoding of the image data in the data generation unit 40 is smaller than a predetermined threshold and when the image compression load is larger than the predetermined threshold. Since they differ, the timing of compression encoding in the data generation unit 40 also differs.

FIG. 3A shows a timing chart when the image compression load for the compression encoding of the image data in the data generator 40 is smaller than a predetermined threshold value, and FIG. FIG. 3C is a timing chart illustrating a problem in a case where the image compression load is larger than a predetermined threshold, and FIG. 3C is a timing chart in a case where the image compression load is larger than a predetermined threshold.

FIG. 4 is a view for explaining the data structure of moving image data with sound. FIG. 4A shows an arrangement of packets generated when the image compression load is smaller than a predetermined threshold. FIG. 4B shows an arrangement of packets generated when the image compression load includes a case where the image compression load is larger than a predetermined threshold.

Hereinafter, the generation of moving image data with sound when the image compression load is smaller than the predetermined threshold value and when the image compression load includes a case where the image compression load is larger than the predetermined threshold value will be described with reference to FIGS. 3 and 4. explain.

<1-2-3. When the image compression load is smaller than the predetermined threshold> As described above, the data generation unit 40 sets the start time of the time zone P1 of the compression encoding of the first image data as a reference, The compression encoding is started after 0.5 × N seconds (N is an arbitrary natural number), and the compression encoding of the audio data is started after 0.4 + 0.5 × N seconds (N is an integer of 0 or more). Is controlled. Therefore, here, as shown in FIG.
The compression encoding of the image data and the compression encoding of the audio data are alternately performed in a time division manner.

More specifically, as shown in FIG. 3A, in each of the time zones P1 to P4 of the compression encoding of the image data, the image data acquired by the One image packet is generated by compression-encoding the corresponding 15 frames of image data. On the other hand, in each of the time zones S1 to S4 of the audio data compression encoding, the audio data acquired by the audio acquisition unit 20 is subjected to the compression encoding for 0.5 second to generate one audio packet.

Here, the audio data to be compression-encoded in the time zones S1 to S4 respectively correspond to the time zones S1 to S4.
0.5 obtained by the voice obtaining unit 20 immediately before 1 to S4
The voice data is equivalent to the voice of seconds.

In other words, the data generation section 40 selects different time zones P1 out of the different time zones P1 to P4 and S1 to S4.
In each of P4 to P4, image data that is one of compression encoding of image data input from the image acquisition unit 10 and compression encoding of audio data input from the audio acquisition unit 20 via the SW circuit 109 Is selectively performed. In each of the different time zones S1 to S4, compression encoding of the image data input from the image acquisition unit 10 and the SW circuit 1
Only the compression encoding of the audio data which is the other of the compression encoding of the audio data input via the input terminal 09 is selectively performed.

Therefore, here, for example, FIG.
As shown in the figure, the compression-encoded image data and audio data are multiplexed, and the image packet PP1, audio packet SP1, image packet PP2, audio packet SP2, image packet PP3, audio packet SP3, image packet PP4, audio The moving image data with sound in which the image packet and the sound packet are alternately arranged in the order of the packets SP4,.

<1-2-4. The case where the image compression load is greater than a predetermined threshold> In the compression encoding according to the compression encoding rules such as MPEG1 and MPEG2, when the motion of the subject is extremely large, the data generation unit 40 compresses the image data. The load on encoding increases. Therefore, if the image data output from the image acquisition unit 10 and the audio data output from the audio acquisition unit 20 are directly input to the data generation unit 40, for example, When the load on the compression encoding is extremely large, as shown in FIG. 3B, the time zone P2 of the compression encoding of the image data becomes long, and the time zone P2 compresses the audio data output from the RAM. The state overlaps with the time zone S2 to be encoded.

In the state shown in FIG. 3B, the data generation unit 40 generates moving image data with sound so that the moving image is not interrupted. Even if the image data is output from the RAM at .9 seconds, since the image data is compressed and encoded in the time zone P2, the audio data to be compression-encoded in the time zone S2 disappears without being compressed and encoded. The moving image data with sound that generates a break in the sound during reproduction on the communication partner side is generated.

However, in the data generating apparatus 1 according to the first embodiment of the present invention, when generating a certain image packet,
When the image compression load exceeds a predetermined threshold, under the control of the output control unit 110, audio data output from the audio acquisition unit 20 to the data generation unit 40 to generate an audio packet next to the image packet being generated is output. The audio data is not output from the SW circuit 109 to the data generation unit 40, but is output to the memory 10
7 and temporarily stored in the memory 107 as temporary storage data. Then, when an image packet is subsequently generated, if the image compression load becomes smaller than a predetermined threshold, the temporary storage data temporarily stored in the memory 107 is output to the data generation unit 40.

Specifically, for example, as shown in FIG. 3C, when the time zone P2 of the compression encoding of the image data becomes too long, the image compression load is set to a predetermined value in the time zone P2. Since the value becomes larger than the threshold value, audio data to be compression-encoded in the time zone S2 is temporarily stored in the memory 107 as temporary storage data.

Thereafter, after the time period P3 during which the image compression load has become smaller than the predetermined threshold, the audio data stored in the memory 107 is transferred to the data generator 40 via the SW circuit 109.
To perform compression encoding of audio data to be compression-encoded in the time zone S2 in the time zone SA2. At this time, audio data to be compression-encoded in the time zone S3 is temporarily stored in the memory 107 as temporary storage data, and after the time zone P4, the temporarily stored data stored in the memory 107 is transferred via the SW circuit 109. The data is output to the data generation unit 40, and compression encoding of audio data to be compression-coded in the time zone S3 is performed in the time zone SA3.

Here, the output control unit 110 controls the audio data to be output from the SW circuit 109 to the memory 107 after the image compression load actually increases in the time zone P2. Of the audio data to be compression-encoded, the audio data is
The audio data already output to the data generation unit 40 is lost before the control is performed to output the data to the memory 107. However, since the lost part is relatively small, the loss of the audio data can be reduced. .

Therefore, here, the data generation unit 40
As shown in FIG. 4B, the video data with audio output from the
1, image packet PP2, image packet PP3, audio packet SP2, image packet PP4, audio packet SP
The packets are arranged in the order of 3,.

<1-3. Data generator 40 for audio data
Output Control Flow> Next, an output control flow of audio data to the data generation unit 40 will be described.

FIG. 5 is a flowchart for explaining the output control of the audio data to the data generator 40 in the data generator 1. Here, the output of the audio data to the data generator 40 is controlled based on the control by the output controller 110.

First, in order to generate the moving image data with sound, the acquisition of the image data by the image acquisition unit 10 and the acquisition of the sound data by the audio acquisition unit 20 are started based on the operation of the user. The output control of the audio data in the data control unit 30, that is, the output control unit 110 is started, and the process proceeds to step ST1. In addition,
Here, when the output control of the audio data in the output control unit 110 is started, the audio data newly input from the audio acquisition unit 20 to the SW circuit 109 is output to the data generation unit 4.
It is assumed that it is controlled to input 0.

In step ST1, it is determined whether or not the load of the compression encoding in the data generation unit 40 is high. Here, a high load signal is output from the load detection unit 106 to the output control unit 1.
If it has been input to the control unit 10, it is determined that the load of the compression encoding in the data generation unit 40 is high, and the process proceeds to step ST <b> 2, and the high load signal from the load detection unit 106 must be input to the output control unit 110. For example, the data generation unit 40
Is determined to be low, and the process proceeds to step ST3.

In step ST2, the audio acquisition unit 20 generates an audio packet after the image packet currently being generated.
From the memory 1 to the SW 1
07 is stored as temporary storage data, and the process proceeds to step ST6 immediately before audio data is input from the audio acquisition unit 20 to the SW circuit 109 to generate the next audio packet.

In step ST3, it is determined whether or not temporary storage data is stored in memory 107. Here, when the data presence signal is input from the memory detection unit 108 to the output control unit 110, it is determined that the temporarily stored data is stored in the memory 107, and the process proceeds to step S
The process proceeds to T4, and if a data presence signal has not been input from the memory detection unit 108 to the output control unit 110, it is determined that temporary storage data is not stored in the memory 107, and the process proceeds to step ST5.

In step ST 4, among the temporary storage data stored in the memory 107, the temporary storage data stored at the earliest time is output to the data generator 40 via the SW circuit 109, and Control is performed so that newly input voice data is stored in the memory 107, and the process proceeds to step ST6. Note that the temporarily stored data output here corresponds to audio data for generating one audio packet.

In step ST5, control is performed so that voice data newly input from the voice obtaining unit 20 to the SW circuit 109 is output to the data generating unit 40, and the process proceeds to step ST6.

At step ST6, it is determined whether or not the generation of a series of moving image data with sound has been completed. Here, when the encoding completion signal has been input from the load detection unit 106 to the output control unit 110, it is determined that the generation of a series of moving image data with audio has been completed, and the output control of the audio data ends. If the encoding completion signal has not been input from the load detection unit 106 to the output control unit 110,
It is determined that the generation of a series of moving image data with sound has not been completed, and the process returns to step ST1 to return to step ST1.
And the data processing of step ST6 is repeated.

As described above, in the data generating apparatus 1 according to the first embodiment, one data generating unit 40 performs the compression encoding of the image data and the compression encoding of the audio data in a time-division manner, so that Generate image data. Therefore, the circuit for generating the moving image data with sound can be reduced in size, so that the size of the moving image data generating device with sound can be reduced in size and weight.

If the image compression load on the data generator 40 is larger than a predetermined threshold, the audio data is
It is not output from the W circuit 109 to the data generator 40 but is temporarily stored in the memory 107 as temporary storage data. Therefore, even if the image compression load is increased, the loss of audio data can be prevented, and the size of the moving image data generation device with audio can be reduced in size and weight, and the interruption of audio during reproduction on the communication partner side can be suppressed. can do.

When the image compression load in the data generation unit 40 is smaller than a predetermined threshold, if the temporarily stored data is stored in the memory 107, the SW circuit 109
Reads the temporarily stored data from the memory 107 and outputs it to the data generator 40. Then, the data generation unit 40 performs compression encoding of the temporarily stored data. On the other hand, if the temporary storage data is not stored in the memory 107, the SW circuit 109 outputs the audio data newly acquired by the audio acquisition unit 20 to the data generation unit 40. Then, the data generator 40 performs compression encoding of the audio data. Therefore, compression encoding of image data and audio data can be performed efficiently.

<Embodiment 2>><2-1. Functional Block of Data Generation Device 2> FIG.
FIG. 13 is a functional block diagram of a data generation device 2 that generates moving image data with sound according to the second embodiment. Embodiment 2
In the data generation device 2 according to the first embodiment, the memory 107 and the memory detection unit 10 in the data generation device 1 according to the first embodiment are used.
8, and the SW circuit 109 are replaced with a FIFO type stack memory (hereinafter abbreviated as “FIFO”) 111,
The output control unit 110 is replaced with an output control unit 110F, and the circuit scale of the audio data control unit 30 is reduced. Then, with the change of the configuration, the data generation device 1 and the data generation unit 4
The output control flow of audio data to 0 is different.

Here, the function of each section of the audio data control section 30 except for the FIFO 111 may be realized by a dedicated electronic circuit or realized by a program in the CPU. May be.

Hereinafter, only points different from the data generating apparatus 1 according to the first embodiment will be described, and the same parts as those of the data generating apparatus 1 will be denoted by the same reference numerals and the description will be omitted.

As shown in FIG. 6, audio data output from the audio acquisition unit 20 is input to the FIFO 111.

The FIFO 111 functions as a storage unit, and always stores the audio data input from the audio acquisition unit 20 temporarily as temporary storage data.
Of the stored temporary storage data, that is, the FIFO 11
1 in the order in which they are stored in the data generation unit 40.
Output to

The output control unit 110F is connected to the load detection unit 106
FIFO 111 based on the high load signal input from
The output of audio data to the data generation unit 40 is controlled.

More specifically, when the image compression load of the data generation unit 40 is larger than a predetermined threshold, the output control unit 110F prevents the output control unit 110F from outputting audio data from the FIFO 111 to the data generation unit 40. Controlling the audio data as temporary storage data in the FIFO 111
Is temporarily stored.

On the other hand, if the image compression load is smaller than the predetermined threshold value, the output control unit 110F outputs the temporarily stored data to the data generation unit 40 in the order in which they were stored from the FIFO 111, and the data generation unit 40 performs compression encoding. Is controlled to be performed.

<2-2. Data Processing in Data Generating Apparatus 2> FIG. 7 is a flowchart illustrating output control of audio data in the data generating apparatus 2. Here, the output of the audio data to the data generation unit 40 is controlled based on the control by the output control unit 110F.

First, in the same manner as in the first embodiment, in order to generate moving image data with audio, based on a user operation,
When the acquisition of the image data by the image acquisition unit 10 and the acquisition of the audio data by the audio acquisition unit 20 are started, the output control of the audio data by the output control unit 110F is started, and the process proceeds to step ST21.

At step ST21, the data generation unit 40
It is determined whether or not the load of the compression encoding is high. Here, the audio data corresponding to the audio for 0.5 seconds for generating one audio packet is stored in the FIFO 11.
It is performed every time immediately before input to 1. Specifically, at the time of determination, a high load signal is output from the load detection unit 106 to the output control unit 1.
If the data is input to 10F, it is determined that the load of the compression encoding in the data generation unit 40 is high, and step ST2
If the high load signal is not input from the load detection unit 106 to the output control unit 110F at the time of the determination, the load of the compression encoding in the data generation unit 40 is determined to be low, and the process proceeds to step ST23.

In step ST22, the temporary storage data, which is the audio data that is input from the audio acquisition unit 20 to the FIFO 111 and temporarily stored, is not output to the data generation unit 40, and the process proceeds to step ST24.

In step ST23, the oldest temporary storage data among the temporary storage data input from the voice acquisition unit 20 to the FIFO 111 and temporarily stored is set to be output to the data generation unit 40. Step S
The temporary storage data set to be output for each output setting of T23 corresponds to audio data corresponding to 0.5 seconds of audio for forming one audio packet.

In step ST24, it is determined whether or not the generation of a series of moving image data with sound has been completed.
Here, when the encoding completion signal is input from the load detection unit 106 to the output control unit 110F, it is determined that the generation of a series of moving image data with sound has been completed, and the output control of the sound data ends. If the encoding completion signal has not been input from the load detection unit 106 to the output control unit 110F, it is determined that the generation of a series of moving image data with sound has not been completed, and the process returns to step ST21 and returns to step ST21. And the data processing of step ST24 is repeated.

As described above, in the data generating apparatus 2 according to the second embodiment, the audio data acquired by the audio
The data is temporarily stored in the IFO 111 as temporary storage data. Then, the image compression load in the data generation unit 40 is compared with a predetermined threshold value. If the load value is smaller than the threshold value, the temporary storage data is read out from the FIFO 111 in the order stored from the FIFO 111 under the control of the output control unit 110F. In the generation unit 40, compression encoding of the temporarily stored data is performed. Therefore, the downsizing of the circuit scale of the audio data control unit 30 can further reduce the size and weight of the moving image data generation device with audio. Further, loss of audio data can be prevented, and interruption of audio during reproduction on the communication partner side can be suppressed.

<Embodiment 3> FIG. 8 shows the third embodiment.
FIG. 2 is a functional block diagram of a data generation device 3 that generates moving image data with sound according to the first embodiment. In the data generation device 1 according to the first embodiment, the load detection unit 106 directly detects the image compression load of the data generation unit 40 from the data generation unit 40. However, in the data generation device 3 according to the third embodiment, Acquires information about image data from the image acquisition unit 10 and predicts the image compression load of the data generation unit 40 in advance.

Hereinafter, only points different from the data generating apparatus 1 according to the first embodiment will be described, and the same parts as those of the data generating apparatus 1 will be denoted by the same reference numerals and the description thereof will be omitted.

As shown in FIG. 8, the A /
The D converter 102 outputs image data to the data generator 40 as in the first embodiment, and also outputs the same image data to the load detector 106P.

The load detector 106P is provided with the A / D converter 10
2, the motion compensation inter-frame prediction is performed in the same manner as the compression encoding based on the compression encoding rules such as MPEG1 and MPEG2 in the data generation unit 40. Then, by calculating the difference between the frames to be subjected to the compression encoding, the image compression load when the image data for which the difference is to be calculated is compression-encoded by the data generation unit 40 is predicted. That is, the load detection unit 106
P constitutes a load detecting means (load estimating means) for predictively detecting the image compression load in advance, and compares the image compression load detected predictively with a predetermined threshold value to determine the image compression load. When the load becomes larger than the predetermined threshold, a high load signal indicating that fact is output to the output control unit 110P.

The output control section 110P is substantially the same as the output control section 110 in the first embodiment.
Also, the flowchart regarding the output control flow of the audio data to the data generation unit 40 in the data generation device 3 is substantially the same as the flowchart for explaining the output control of the audio data in the data generation device 1 shown in FIG.

However, in the determination in step ST6 in the flowchart shown in FIG.
It is determined whether or not the generation of a series of moving image data with sound has been completed based on whether or not the encoding completion signal has been input to the output control unit 110 from here, but here, no sound data is stored in the memory 107. Therefore, when the data presence signal is not input from the memory detection unit 108 and the audio data is not acquired by the audio acquisition unit 20 based on the operation of the user, the output control unit 110P outputs a series of moving image data with audio. It is determined that the generation has been completed.

Here, as in the first embodiment,
The functions of each unit of the audio data control unit 30 except for the memory 107 may be realized by being constituted by a dedicated electronic circuit, or may be realized by a program in a CPU.

As described above, in the data generation device 3 according to the third embodiment, the load detection unit 106P
Image data is acquired from the D converter 102 as information relating to the image data. Then, the load detection unit 106P predicts an image compression load when the image data is compression-encoded in the data generation unit 40, and detects in advance whether the image compression load becomes larger than a predetermined threshold. Therefore, when the image compression load on the data generation unit 40 becomes high, the audio data can be temporarily stored in the memory 107 as the temporary storage data without delay, so that the audio is interrupted during reproduction on the communication partner side. The control for suppressing the noise can be promptly performed, and the interruption of the sound can be further reduced.

<Embodiment 4>><4-1. Functional Block of Data Generation Apparatus 4> FIG.
FIG. 14 is a functional block diagram of a data generation device 4 that generates moving image data with sound according to Embodiment 4. Embodiment 1
In the data generation device 1 according to the above, the output control of the audio data to the data generation unit 40 is performed, and when the image compression load on the data generation unit 40 is high, the audio data is
By temporarily storing the audio data,
Although the interruption of audio during reproduction on the communication partner side was suppressed while miniaturizing the circuit scale, the data generation device 4 according to the fourth embodiment uses the data generation unit 4 for audio data.
Without providing the audio data control unit 30 for controlling the output to 0,
When the image compression load on the data generation unit 40 is high,
By increasing the compression ratio in the compression encoding of the image data in the data generation unit 40 from a predetermined reference compression ratio,
The circuit size is reduced while preventing the loss of audio data.

Hereinafter, only points different from the data generating apparatus 1 according to the first embodiment will be described, and the same parts as those of the data generating apparatus 1 will be denoted by the same reference numerals and the description thereof will be omitted.

As shown in FIG. 9, audio data output from the audio acquisition unit 20 is directly input to the data generation unit 40, and is temporarily stored in a RAM for adjusting the timing of compression encoding. Then, the data generation unit 40 performs the compression encoding of the image data and the audio data at the timing shown in FIG.

The load detecting section 106D constitutes a load detecting means for comparing the image compression load in the data generating section 40 with a predetermined threshold value, similarly to the load detecting section 106 of the first embodiment. If the load is larger than the predetermined threshold, a high load signal indicating that fact is sent to the compression ratio controller 11.
4 is output.

Further, similarly to the load detection unit 106 of the first embodiment, the load detection unit 106D does not detect any compression load for a certain period of time after the data generation unit 40 starts compression encoding of image data and audio data. In this case, an encoding completion signal indicating that the compression encoding of the video data and audio data for generating a series of moving image data with audio is completed is output to the compression ratio control unit 114.

The compression ratio control unit 114 is
Based on the high load signal input from D, switching of the compression ratio in the compression encoding of the image data in the data generation unit 40 is controlled.

Specifically, when the image compression load of the data generation unit 40 is larger than a predetermined threshold, the load detection unit 10
A high load signal is input from 6D to the compression ratio control unit 114,
The compression rate control unit 114 controls the compression rate in the compression encoding in the data generation unit 40 to be higher than a predetermined reference compression rate, and when the image compression load of the data generation unit 40 is smaller than a predetermined threshold value. Since the high load signal is not input from the load detection unit 106D to the compression ratio control unit 114, the compression ratio control unit 114 does not perform control to increase the compression ratio in the compression encoding in the data generation unit 40.

When the encoding completion signal is input from load detection section 106D, compression rate control section 114 ends the control of switching the compression rate.

Next, the contents and effects of switching the compression ratio in the compression encoding in the data generation section 40 will be described more specifically.

For example, in the case where the compression ratio is a predetermined reference compression ratio that is not high, the data generation unit 40 sets all the pixels of the image data input from the image acquisition unit 10 as compression encoding targets. If the compression ratio is higher than a predetermined reference compression ratio, the data generation unit 40 causes the image acquisition unit 1
0 in the horizontal and vertical directions in the pixel matrix arrangement of all the pixel data of the input image data.
Pixel data decimated by / 2 is to be subjected to compression encoding.

As the compression rate is increased as described above, the quality of the image is reduced, but the amount of image data to be compression-coded is reduced.
0 can increase the processing speed of the compression encoding of the image data.

Therefore, when the movement of the subject is extremely large, the load on the data generator 40 for compressing and encoding the image data increases, and the time required for compressing and encoding the image data tends to increase. When the load on the compression encoding of the image data in the data generation unit 40 is high, the compression rate in the compression encoding of the image data in the data generation unit 40 is increased, and as shown in FIG. In order to prevent the time zone of data compression and encoding from becoming too long, the quality of the image is reduced, but the loss of audio data can be prevented.

Here, the functions of the load detection unit 106D and the compression ratio control unit 114 may be implemented by a dedicated electronic circuit, or may be realized by separate or one CPU by a program. What is realized may be sufficient. Further, here, the functions of the data generation unit 40, the load detection unit 106D, and the compression ratio control unit 114 may be realized by a single CPU by a program.

<4-2. FIG. 10 is a flowchart showing a control flow of the compression ratio of the data generation device 4 based on the control of the compression ratio control unit 114.

First, in the same manner as in the first embodiment, in order to generate moving image data with sound, based on a user operation,
When the acquisition of image data by the image acquisition unit 10 and the acquisition of audio data by the audio acquisition unit 20 are started, control of the compression ratio by the compression ratio control unit 114 is started, and the process proceeds to step ST41.

In step ST41, the data generation unit 40
It is determined whether or not the load of the compression encoding is high. Here, for example, the determination in step ST41 is performed at 0.1 second intervals, and when a high load signal is input from the load detection unit 106D to the compression ratio control unit 114 at the time of the determination, the compression encoding in the data generation unit 40 is performed. It is determined that the load is high,
The process proceeds to step ST42, and the load detector 1
If a high-load signal has not been input to the compression rate control unit 114 from 06D, it is determined that the load of compression encoding in the data generation unit 40 is low, and the process proceeds to step ST43.

In step ST42, the data generation unit 40
In step ST44, the compression ratio in the compression encoding of the image data is set higher than the reference compression ratio.

In step ST43, the data generation unit 40
Is set to a low compression ratio corresponding to the reference compression ratio, and the process proceeds to step ST44.

In step ST44, it is determined whether or not a series of generation of moving image data with sound has been completed.
Here, from the load detection unit 106D to the compression ratio control unit 114
When the encoding completion signal has been input to the CPU, it is determined that the generation of a series of moving image data with sound has been completed, and the control of the compression ratio ends. If the encoding completion signal has not been input from the load detection unit 106D to the compression ratio control unit 114, it is determined that the generation of a series of moving image data with sound has not been completed, and the process returns to step ST41. The data processing from ST41 to ST44 is repeated.

As described above, in the data generation device 4 according to the fourth embodiment, when the image compression load on the data generation unit 40 is larger than a predetermined threshold, the compression ratio control unit 114
The compression ratio in the compression encoding of the image data in the data generation unit 40 is made higher than a predetermined reference compression ratio. Therefore,
Even without providing a storage means for temporarily storing the audio data, it is possible to prevent the loss of the audio data,
It is possible to further reduce the size and weight of the moving image data generating apparatus with sound and suppress interruption of sound during reproduction on the communication partner side.

<Embodiment 5> FIG. 11 is a functional block diagram of a data generation device 5 for generating moving image data with sound according to the fifth embodiment. In the data generation device 4 according to the fourth embodiment, when the image compression load on the data generation unit 40 is high, under the control of the compression ratio control unit 114,
Although the compression rate in the compression encoding of the image data in the data generation unit 40 has been increased, the data generation device 5 according to the fifth embodiment responds to a use situation where the image compression load on the data generation unit 40 increases. Then, the user sets the compression ratio in the compression encoding of the image data in the data generation unit 40 to be higher than the reference compression ratio.

Hereinafter, only points different from the data generating apparatus 4 according to the fourth embodiment will be described, and the same parts as those of the data generating apparatus 4 will be denoted by the same reference numerals and the description thereof will be omitted.

As shown in FIG. 11, the user can perform various operations outside the data generator 5 to change the compression ratio in the compression encoding of image data in the data generator 40. 115 are provided.

The operation section 115 allows the user to perform various operations to obtain a "TV phone mode" for photographing a scene with a small movement of the subject, and a "sports mode" for photographing a scene with a large movement of the subject. A shooting mode can be selected, and a signal for specifying the shooting mode selected by the operation unit 115 (hereinafter, referred to as a “mode specifying signal”) is output to the compression ratio control unit 114Y.

The compression ratio control unit 114Y controls switching of the compression ratio in the compression coding of the image data in the data generation unit 40 based on the mode specifying signal input from the operation unit 115.

More specifically, when the user operates the operation unit 115 to select “sports mode” as the shooting mode, the operation unit 115 indicates that “sports mode” is selected as the shooting mode. The specific signal is output to compression ratio control section 114Y. At this time, the compression ratio control unit 114Y controls to increase the compression ratio in the compression encoding of the image data in the data generation unit 40. That is, the compression ratio control unit 1 is operated based on a user operation or selection.
14Y controls the compression ratio in the compression encoding of the image data in the data generation unit 40 to be higher than the reference compression ratio.

On the other hand, when the user operates the operation unit 115 to select “TV phone mode” as the shooting mode, the mode identification indicating that “TV phone mode” is selected as the shooting mode from the operation unit 115 The signal is output to compression ratio control section 114Y. At this time, the compression ratio control unit 1
14Y does not perform control to increase the compression ratio in the compression encoding of image data in the data generation unit 40.

Here, the function of the compression ratio control section 114Y may be realized by a dedicated electronic circuit, or may be realized by a program in a CPU. Further, here, the functions of the data generation unit 40 and the compression ratio control unit 114Y are 1
It may be realized by a program in one CPU.

The content and effect of switching the compression ratio in the compression encoding in the data generation unit 40 are the same as those of the data generation device 4 according to the fourth embodiment, and therefore the description is omitted here.

As described above, in the data generating apparatus 5 according to the fifth embodiment, the compression rate control unit 11
4Y increases the compression ratio in the compression encoding of the image data in the data generation unit 40 from a predetermined reference compression ratio. That is, loss of audio data is prevented by increasing the compression ratio in the compression encoding of image data in the data generation unit 40 before the image compression load in the data generation unit 40 increases. Therefore, by performing the setting in advance according to the use situation by the user, it is possible to reliably avoid the interruption of the sound during the reproduction on the communication partner side. Further, the size of the moving image data generation device with sound can be reduced in size and weight by further reducing the circuit size.

In this case, it is possible to reliably avoid interruption of the sound at the time of reproduction on the communication partner side in accordance with the use condition based on the user's setting. Since the quality of the image and the sound is determined, for example, in a state where the “sports mode” is set as the shooting mode, even if the movement of the subject becomes small, the compression ratio is maintained in a high state. The quality of the image is not good to some extent.

Therefore, from the viewpoint of ensuring the quality of the image as much as possible, it can be said that the data generation device 4 according to the fourth embodiment, in which the compression ratio is switched according to the movement of the subject, is superior, From the viewpoint of reliably avoiding interruption, the data generation unit 40 can switch the compression ratio after the image compression load actually increases, compared to the data generation device 4 according to the fourth embodiment.
It can be said that the data generation device 5 according to the fifth embodiment is superior.

<Modifications> The embodiments of the present invention have been described above, but the present invention is not limited to the above-described contents.

For example, in the above-described first to third embodiments, the function of the audio data control unit 30 except for the memory 107 and the FIFO 111 and the function of the data generation unit 40 are realized by separate configurations. The functions of the audio data control unit 30 except for the memory 107 and the FIFO 111 may be realized by a program in the CPU in the data generation unit 40.

In the second embodiment, the load detector 106 directly detects the compression load from the data generator 40. However, the present invention is not limited to this. Like the load detection unit 106P of the device 3, image data may be acquired from the A / D converter 102 of the image acquisition unit 10 to predictably detect the image compression load.

In the fourth embodiment, the load detector 106D directly detects the compression load from the data generator 40. However, the present invention is not limited to this. Like the load detection unit 106P of the device 3, image data may be acquired from the A / D converter 102 of the image acquisition unit 10 to predictably detect the image compression load.

In the above-described first to third embodiments, once the image compression load on the data generation unit 40 increases, the audio data is temporarily stored as temporary storage data, and the image compression load decreases. When the image compression load increases several times during the generation of a series of moving image data with audio, the compression encoding of the audio data is accumulated multiple times. It is postponed, and the transmission of audio data will be too slow, but this is not limited to this, and when the image compression load becomes low, compression encoding of audio data that is postponed due to accumulation of compression encoding May be performed collectively as much as possible. With such a configuration, it is possible to suppress a state in which transmission of audio data is too slow.

In the above-described embodiment, the image data and the audio data for generating one image packet and the audio packet are acquired in 0.5 seconds, but the present invention is not limited to this. But, for example, 1
For example, one image packet and one audio packet may be generated from image data and audio data acquired in / 6 seconds. With such a configuration, when the image compression load on the data generation unit 40 becomes high, the time during which the compression encoding of the audio data is postponed becomes short, so that the transmission of the audio data becomes too slow. More can be avoided.

[0139]

As described above, according to the first aspect of the present invention, the moving picture data with sound is generated by performing the compression coding of the image data and the compression coding of the sound data in a time division manner. With such a configuration, it is possible to reduce the size and weight of the moving image data generation device with sound.

According to the second aspect of the present invention, when the load on the compression encoding of image data is larger than a predetermined threshold, the audio data is temporarily stored. Since it is possible to prevent the disappearance of the audio data, it is possible to reduce the size and weight of the moving image data generating apparatus with sound and to suppress interruption of sound during reproduction on the communication partner side.

According to the third aspect of the present invention, when the load of the compression encoding of the image data is smaller than the predetermined threshold, if the audio data is temporarily stored in the storage means, By performing a compression encoding of audio data which is newly acquired by the audio acquisition unit if the audio data is not temporarily stored in the storage unit, Data and audio data can be efficiently compressed and encoded.

According to the fourth aspect of the present invention, the audio data acquired by the audio acquisition means is always temporarily stored in the storage means, and when the load on the compression encoding of the image data is smaller than a predetermined threshold value. The configuration in which the compression encoding of the audio data is performed in the order stored in the storage means makes it possible for the communication partner side to further reduce the size and weight of the audio-equipped moving image data generation device by reducing the circuit size. Can be prevented from being interrupted at the time of reproduction of.

According to the fifth aspect of the present invention, when the load on the compression encoding of the image data is larger than the predetermined threshold, the compression ratio in the compression encoding of the image data is made higher than the predetermined reference compression ratio. With such a configuration, it is possible to prevent the loss of the audio data without providing a storage unit or the like for temporarily storing the audio data. At the same time, it is possible to suppress breaks in audio during reproduction on the communication partner side.

According to the sixth aspect of the present invention, the information regarding the image data is acquired from the image acquiring means, and the load on the compression encoding of the image data is predicted. Detecting an increase in the load on data compression and encoding, it is possible to quickly perform control to suppress breaks in audio during playback on the communication partner side, so that breaks in audio can be further reduced. it can.

According to the seventh aspect of the present invention, the compression ratio in the compression encoding of the image data is made higher than a predetermined reference compression ratio based on the operation of the user. Before the load on the compression encoding becomes high, the user can increase the compression ratio in the compression encoding of the image data, so that it is possible to reliably avoid the interruption of the sound at the time of reproduction on the communication partner side, It is possible to reduce the size and weight of the moving image data generation device with sound by further reducing the circuit size.

Further, according to the invention of claim 8, claim 1
The same effect as the invention described in (1) can be obtained.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a data generation device according to a first embodiment.

FIG. 2 is a schematic diagram showing a data structure of moving image data with sound.

FIG. 3 is a timing chart illustrating the timing of compression encoding.

FIG. 4 is a diagram illustrating a data structure of moving image data with sound.

FIG. 5 is a flowchart illustrating output control of audio data.

FIG. 6 is a functional block diagram of a data generation device according to a second embodiment.

FIG. 7 is a flowchart illustrating output control of audio data.

FIG. 8 is a functional block diagram of a data generation device according to a third embodiment.

FIG. 9 is a functional block diagram of a data generation device according to a fourth embodiment.

FIG. 10 is a flowchart illustrating a control flow of a compression ratio.

FIG. 11 is a functional block diagram of a data generation device according to a fifth embodiment.

[Explanation of symbols]

1, 2, 3, 4, 5 moving image data generation device with sound (data generation device), 10 image acquisition unit, 20 sound acquisition unit, 30 sound data control unit, 40 data generation unit,
101 imaging unit, 103 microphone, 106, 106D,
106P load detector, 107 memory, 108 memory detector, 109 switch (SW) circuit, 110, 1
10F, 110P output control unit, 111 FIFO, 1
14, 114Y Compression ratio control unit, 115 operation unit.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/24 (72) Inventor Hidenori Ishida 2-3-2 Marunouchi, Chiyoda-ku, Tokyo Mitsubishi Electric Corporation F-term (reference) 5C022 AA12 AC69 AC72 5C059 MA00 PP04 RA08 RB02 RC04 RC32 RE03 SS07 SS30 TA06 TA46 TC18 TC38 TD05 TD12 UA02 UA32 UA38 5C063 AB07 AC01 AC05 CA11 CA20 DA05 DA13 5J064 AA04 BC01 BC02 BC06 BC12 BC02 BC06 NN18 SS07 SS08 TT06 UU19

Claims (8)

[Claims] An image acquisition unit for acquiring image data; an audio acquisition unit for acquiring audio data; and a compression encoding of the image data and a compression encoding of the audio data being performed in a time-division manner. A moving image data with sound generation apparatus, comprising: data generating means for generating moving image data with sound in which encoded image data and compression-encoded sound data are multiplexed. 2. The video data generating apparatus with sound according to claim 1, wherein: a load detecting unit that compares a compression load for the compression encoding of the image data in the data generating unit with a predetermined threshold value; A storage unit for temporarily storing the audio data as temporary storage data when the compression load is larger than the predetermined threshold value; 3. The video data generating apparatus with sound according to claim 2, wherein the temporary storage data is stored in the storage unit when the compression load is smaller than the predetermined threshold. A compression encoding of the temporary storage data is performed by a data generation unit, and the compression encoding of the audio data newly acquired by the audio acquisition unit by the data generation unit if the temporary storage data is not stored in the storage unit. And a control unit for controlling the data generation unit so as to perform the conversion. 4. The moving image data generation device with sound according to claim 2, wherein the storage unit temporarily stores the audio data acquired by the audio acquisition unit as temporary storage data. The audio-moving image data generating device, when the compression load is smaller than the predetermined threshold, reads out the temporary storage data in the order stored in the storage unit and performs compression encoding. A moving image data generating apparatus with sound, further comprising control means for controlling. 5. The moving image data with sound generation apparatus according to claim 1, wherein: a load detection unit that compares a compression load for compression encoding of the image data in the data generation unit with a predetermined threshold value; A compression ratio control unit that controls the data generation unit so that a compression ratio in the compression encoding of the image data is higher than a predetermined reference compression ratio when the compression load is larger than a predetermined threshold. A moving image data generation device with sound, characterized in that: 6. The moving image data with sound generation device according to claim 2, wherein the load detecting means acquires information on the image data from the image acquiring means, and A video data generating apparatus with sound, wherein the compression load is predicted based on information. 7. The moving image data with sound generation apparatus according to claim 1, wherein a compression ratio in the compression encoding of the image data is set higher than a predetermined reference compression ratio based on a user operation. A moving image data generation device with sound, further comprising a compression ratio control unit for controlling the data generation unit. 8. A method of obtaining image data, a step of obtaining audio data, and a step of performing compression encoding of the image data and compression encoding of the audio data in a time division manner. A method for generating moving image data with sound, comprising: generating moving image data with sound in which image data and sound data compressed and encoded are multiplexed.