JP2003259287A - Multimedia communication equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable editing of recorded information and to enable temporary stop of recording while performing encoding processing. <P>SOLUTION: A recording processing part 16 generates a recording format frame comprising time stamp information, attribute information to which at least a recording temporary stop flag and a recording restart flag can be set, and an information contents part of encoded information being a recording object when recording the encoded information. A reproducing processing part 17 sets a reproducing timer by a time stamp value in the recording format frame to reproduce the encoded information in the recording format frame. When the recording restart flag is set in the recording format frame being the reproducing object, the reproducing processing part 17 sets the reproducing timer by the time stamp value in the recording format frame, where the recording restart flag is set, to continue reproducing from encoded information in the frame. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a multimedia information communication function for transmitting and receiving a plurality of types of multimedia information such as voices and moving images to and from a partner terminal by encoding each type.
A recording processing function of multiplexing and recording the encoded information generated by the own terminal or the encoded information received from the partner terminal,
The present invention relates to a multimedia communication device having a reproduction processing function for reproducing recorded encoded information, and more specifically, temporarily suspends recording while executing encoding processing and edits recorded encoded information. It relates to a multimedia communication device capable.

[0002]

2. Description of the Related Art At present, multimedia communication has been realized in which a plurality of types of multimedia information such as voices and moving images are transmitted and received in an environment in which a communication path exists in the middle.

In this communication, multimedia communication is carried out by transmitting / receiving coded information obtained by code-processing voice signals and moving image information for each type.

In the multimedia information, the characteristics of the coded information that has been code-processed differ depending on the information type, but a plurality of pieces of information having different characteristics are multiplexed and transmitted on a communication path.
Further, by receiving the information from the communication path and separating it, the multimedia information such as voice and moving image generated in the partner terminal can be viewed in the partner terminal.

For example, when the moving image information is coded, the coding that occurs within a certain time depends on the amount of information of a certain moving image frame and the degree of temporal change of the moving image frame. The amount of information varies.

[0006] Further, in the encoding system such as MPEG4,
Normally, when a certain image frame is encoded, it is encoded by generating inter-frame predictive encoded information by performing inter-frame predictive encoding processing that encodes only the difference information based on the information of the previous image frame. Although the amount of information is reduced, intra-frame prediction is performed by performing an intra-frame encoding process that encodes using only one image frame information without using temporal difference information due to a request from the system. Coded information (I frame) may be generated and inserted, and in this case, the amount of coded information becomes large.

[0007] As a method for controlling such fluctuations in the amount of coded information, storage type rate control and low delay type rate control are used.

In the low-delay type rate control, it is assumed that encoded information is transmitted and received via a line having a fixed band,
Control is performed so as to reduce fluctuations in the amount of encoded information that occur during a fixed time.

That is, when the image to be coded changes greatly in an instant and the temporal fluctuation of the moving image frame becomes large, the amount of coded information at that point becomes large.
After that, the amount of encoded information is reduced by dropping frames or the image quality for a while, and control is performed so that the average amount of encoded information remains constant.

On the other hand, in the accumulation type rate control, the variation in the coded information amount is allowed as compared with the low delay type rate control, so that the deterioration of the image quality is suppressed.

Generally, in a system for transmitting and receiving multimedia information by using a communication path of a fixed band, delay variation of information is suppressed to a small level, and information having different characteristics such as voice and moving image is temporally received at a receiving terminal. It is common to use a low-delay type rate control at the time of encoding in order to prevent the data from being decoded with a shift.

When the receiving terminal decodes the encoded information sent from the transmitting terminal, the transmitting terminal always encodes the encoded information and the time until the encoded information reaches the receiving terminal through the communication path is always constant. The received encoded information is immediately decoded, assuming that it is almost constant.

Further, among the multimedia communication terminals used in the multimedia communication system using the low-delay type rate control as described above, the multimedia information received from the partner terminal or the multimedia coded by the own terminal is included. Some have a function of recording information.

In such a multimedia communication terminal,
When recording the multimedia information received from the partner terminal, the information received from the partner terminal is recorded with the amount of encoded information according to the size of the band, while the multimedia information encoded by the own terminal is recorded. In this case, the information is recorded at a speed according to the size of the band on the assumption that the recorded information is sent to the communication path.

In this case, the moving image information of the multimedia information to be recorded is information obtained by encoding only the difference between the previous image frames (inter-frame predictive encoding information).

In a conventional multimedia communication terminal which records moving image information through such image processing, since the moving image information in the recorded information consists of information on the difference between the previous image frames, this recording is performed. It was not possible to edit such as cutting and pasting using the information.

Further, in this type of communication terminal, during execution of multimedia communication, for example, while watching a moving image sent from a partner terminal, for a moving image that does not make sense to record, only the relevant section is recorded. There is also a demand for temporary suspension.

In response to such a request, consider the case where recording is temporarily stopped while the encoding process is continued, and then the temporary stop is released and recording is restarted. A recording processing image of moving image information related to the recording operation in this case is as shown in FIG.

That is, in this case, as the moving image coding information (input information) to be recorded, as shown in FIG. 8A, except for the image frame to be input first, information on the difference between the previous image frames is provided. Are input continuously.

For this moving picture coded information, for example,
When recording processing is performed such that recording is temporarily stopped at time T1 and recording is temporarily stopped at time T2, the information recorded in this case has a form as shown in FIG. 8B.

That is, in the moving picture coded information stored in this case [see FIG. 8 (b)], the moving picture coded information for the recording pause period (time T1 to T2) is lost, and the recording is temporarily stopped. The moving image coding information at the time of releasing the stop (the same time T2) also starts from the temporal frame difference information as described above.

When the information recorded in such a form is reproduced, it is not possible to grasp the temporal connection of the moving image coded information before and after the recording pause section (time T1 to T2), and moreover, the information is recorded. Since the moving image coding information that is first input after the restart is composed of the difference information that cannot be reproduced by itself, the moving image cannot be reproduced after the recording suspension time.

As a result, in the conventional communication device of this type, the recording cannot be temporarily stopped without stopping the encoding process in response to the request from the user.

[0024]

As described above, in the above-mentioned conventional multimedia communication terminal, as for the moving image information, the interframe predictive coding information in which only the temporal frame difference information is coded is recorded. Therefore, there is a problem in that the recorded information cannot be edited such as cut and pasted, and the recording cannot be temporarily stopped without stopping the encoding process in response to a request from the user. .

The present invention eliminates the above-mentioned problems and provides a multimedia communication apparatus capable of performing editing such as cutting and pasting recorded information, and also capable of temporarily stopping recording while performing encoding processing. The purpose is to do.

[0026]

In order to achieve the above object, the invention according to claim 1 encodes multimedia information of a plurality of types by an encoding processing unit for each type to generate encoded information. , A multimedia information communication function of multiplexing the coded information and transmitting the coded information to the partner terminal, separating the multiplexed information received from the partner terminal into coded information for each type, and reproducing the same. A recording processing function capable of multiplexing and recording the encoded information received or the encoded information received from the partner terminal, and temporarily stopping and resuming the recording, and a reproduction processing function of reproducing the recorded encoded information. A multimedia communication device including: time stamp information indicating an elapsed time from the start of recording the encoded information, and a flag indicating that the recording is restarted after the recording is temporarily stopped. A frame generating means for generating a recording format frame including settable attribute information; and a reproducing timer which, when reproducing the encoded information, matches the encoded information in the frame with the time stamp value in the recording format frame. When the flag is set in the recording format frame to be reproduced, the reproduction timer is continued to reproduce the encoded information in accordance with the time stamp value of the recording format frame in which the flag is set. And a reproduction control means for controlling the reproduction.

According to a second aspect of the present invention, a plurality of types of multimedia information are coded by each type of coding processing unit to generate coded information, and the coded information is multiplexed and transmitted to a partner terminal. Along with, a multimedia information communication function that separates and reproduces the multiplexed information received from the partner terminal into the coded information for each type, and the coded information generated by the own terminal or the coded information received from the partner terminal A multimedia communication device having a recording processing function capable of multiplexing and recording and recording and temporarily stopping and resuming the recording, and a reproduction processing function of reproducing the recorded encoded information, wherein the encoded information The time stamp information indicating the elapsed time from the start of recording, the recording pause flag indicating that recording was paused, and the recording restart flag indicating that recording was restarted can be set. Frame generating means for generating a recording format frame including attribute information; and a recording format frame from the position where the recording pause flag is set to the position where the recording restart flag is set among the generated recording format frames. Frame processing means for deleting and adjusting the time stamp value recorded in the recording format frame to be continuous, and a reproduction timer at the time of reproducing the encoded information, in accordance with the time stamp value in the recording format frame. And a reproduction control means for sequentially reproducing the encoded information.

According to a third aspect of the present invention, a plurality of types of multimedia information are encoded by each type of encoding processing unit to generate encoded information, and the encoded information is multiplexed and transmitted to the partner terminal. Along with, a multimedia information communication function that separates and reproduces the multiplexed information received from the partner terminal into the coded information for each type, and the coded information generated by the own terminal or the coded information received from the partner terminal A multimedia communication device having a recording processing function capable of multiplexing and recording and recording and temporarily stopping and resuming the recording, and a reproduction processing function of reproducing the recorded encoded information, wherein the encoded information Generates a recording format frame including time stamp information indicating the elapsed time from the start of recording and attribute information for which a recording restart flag indicating that recording has been restarted can be set. Intra-frame predictive coding is performed on the coding processing unit of the own terminal or the coding processing unit of the partner terminal, which is the transmission source of the coding information to be recorded, by inputting the frame generation means and the recording restart command. Instructing means for instructing insertion of information, recording is restarted at the time when the intraframe predictive coding information arrives after the instruction, and generation processing of the recording format frame is sequentially performed from the intraframe predictive coding information. The recording control means for continuing the operation is provided.

According to a fourth aspect of the present invention, in the above-mentioned third aspect, when the recording format frame of the intraframe predictive coding information is generated, the intraframe predictive coding information is included in the recording format frame. It is characterized by setting a flag indicating that there is.

According to a fifth aspect of the present invention, a plurality of types of multimedia information are encoded by each type of encoding processing unit to generate encoded information, and the encoded information is multiplexed and transmitted to the partner terminal. Along with, a multimedia information communication function that separates and reproduces the multiplexed information received from the partner terminal into the coded information for each type, and the coded information generated by the own terminal or the coded information received from the partner terminal A multimedia communication device having a recording processing function of multiplexing and recording the encoded information and a reproduction processing function of reproducing the recorded encoded information, the timestamp showing the elapsed time from the start of recording the encoded information. Frame generating means for generating a recording format frame including information and attribute information indicating the attribute of the encoding information to be recorded, and encoding the recording object when recording the encoding information Instructing means for instructing the encoding processing unit of the own terminal or the encoding processing unit of the partner terminal, which is the sender of the information, to insert the intraframe prediction encoding information, and the intraframe prediction code after the instruction. Recording control means for continuing the generation processing of the recording format frame in order from the intraframe predictive coding information when the encoded information arrives.

According to a sixth aspect of the present invention, in the above-mentioned fifth aspect of the present invention, there is provided clocking means, and the instructing means periodically inserts the intraframe prediction coded information based on the timing of the clocking means. It is characterized by instructing to do.

According to a seventh aspect of the invention, in the above-mentioned fifth aspect of the invention, there is provided a variation recognizing means for recognizing that one or more types of encoded information have changed when the encoded information is recorded, The instructing means may instruct to insert the intra-frame predictive coding information when the variation is recognized by the variation recognizing means.

The invention according to claim 8 is the invention according to claim 7, wherein the fluctuation recognizing means recognizes that the fluctuation of the moving picture information is larger than a preset magnitude. It is characterized by consisting of.

According to a ninth aspect of the present invention, in the above seventh aspect of the invention, the variation recognition means recognizes that the voice signal has changed from a silent state to a voiced state. It is characterized by being formed.

According to a tenth aspect of the present invention, in the above-mentioned fifth aspect of the invention, there is provided input means for inputting a command for inserting the intraframe prediction coded information, and the instruction means is input from the input means. It is instructed to insert the intra-frame prediction coded information based on the command.

[0036]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

FIG. 1 is a block diagram showing the overall configuration of a multimedia communication terminal 100 according to the present invention.

The communication terminal 100 includes a communication interface (I / F) unit 11 which controls an interface with a communication path.
Multiplexing / separating processing unit 12 including a multiplexing unit 121 that multiplexes coded information of multiple types of multimedia information and a separation unit 122 that separates multiplexed coded information, and a moving image that is one of multimedia information Moving picture coding / decoding processing section 1 including moving picture coding section 131 for coding information and moving picture decoding section 132 for decoding coded moving picture information.
3. Speech code / decoding processing section 14 including speech coding section 141 for coding speech signal which is another multimedia information and speech decoding section 142 for decoding encoded speech signal, multiplexing / demultiplexing The processing unit 12, the moving image coding / decoding processing unit 13, the audio coding / decoding processing unit 14, the route selecting unit 15 having a switch for selecting an information route between the recording processing unit 16 and the reproduction processing unit 17, which will be described later, and its own terminal. The recording processing unit 16 that performs a process of multiplexing and recording the generated encoded information or the encoded information received from the partner terminal, the reproduction processing unit 17 that reproduces the recorded information, and the information to be recorded temporarily. A buffer 18 for storing data in the communication terminal, a host interface (I / F) section 19 for controlling the interface between each section in the communication terminal and the system control section 20, and a system for controlling the entire communication terminal. System control unit 20, recording medium 21 for recording information accumulated in buffer 18, input / operation unit 22 for inputting various commands related to recording and reproducing operations using ten keys and various function keys, and voice as audio signals. A microphone unit 23 for converting and inputting, a speaker unit 24 for reproducing and outputting an audio signal, a camera unit 25 for imaging a subject and inputting moving image information, and a display unit 26 for displaying various information such as moving images.
It is configured to include.

FIG. 2 is a diagram showing a connection form of each unit in the communication terminal 100 of FIG.

In FIG. 2, the switch (SW) 151,
Reference numerals 152 and 153 are components of the route selection unit 15 in FIG. 1, and perform route selection of encoded information according to a switching instruction from the system control unit 20.

Specifically, the switch 151 includes an information path from the moving picture coding unit (V Enc) 131, the audio coding unit (A Enc) 141 to the multiplexing unit (MUX) 121, and a reproduction processing unit (PLAY). The information path from 17 to the multiplexer 121 is selected.

The switch 152 is used for the moving picture coding unit 13.
1. The information path from the audio encoding unit 141 to the recording processing unit (REC) 16 and the information path from the separating unit (DMX) 122 to the recording processing unit 16 are selected.

The switch 153 includes a moving image decoding unit (V Dec) 132, a voice decoding unit (A Dec) 1 from the separation unit 122.
An information path to the information path 42 and an information path from the reproduction processing unit 17 to the moving image decoding unit 132 and the audio decoding unit 142 are selected.

Further, the A / D converter (A / D) 133 and D
The A / A (D / A) conversion unit 134 is provided in the moving image coding / decoding processing unit 13 in correspondence with the moving image coding unit 131 and the moving image decoding unit 132, respectively, and performs A / D conversion. The section (A / D) 143 and the D / A (D / A) conversion section 144
The speech coding / decoding processing unit 14 is provided corresponding to the speech coding unit 141 and the speech decoding unit 142, respectively.

The general operation of the communication terminal 100 according to the present invention will be described with reference to FIG.

First, when the multimedia information is transmitted from the communication terminal 100 to the partner terminal via the communication path, the moving image information input from the camera unit 25 is converted into the A / D conversion unit 13.
After being converted from an analog signal to a digital signal by 3, the moving picture coding unit 131 codes the moving picture coding information and inputs it to the switches 151 and 152.

The voice signal input from the microphone unit 23 is converted from an analog signal to a digital signal by the A / D conversion unit 143 and then encoded by the voice encoding unit 141 to be voice encoded information. It is input to the switches 151 and 152.

At this time, the system control unit 20 causes the switch 151 so that the moving image coding information and the audio coding information are input to the multiplexing unit 121 through the switch 151.
Is given a switching instruction.

The multiplexing section 121 multiplexes the moving picture coded information and / or the voice coded information inputted through the switch 151 into a form capable of being sent to the communication path.

The coded information multiplexed by the multiplexing unit 121 is sent to the communication path via the communication I / F unit 11 and transmitted to the partner terminal.

Also, multimedia information (multiplexed coded information) sent from the partner terminal via the communication path.
When receiving the information, the communication terminal 100 first receives the multimedia information through the communication I / F unit 11, and then the separating unit 1 receives the multimedia information.
Enter in 22.

The separating unit 122 separates the moving image coding information and the audio coding information from the multimedia information input from the communication I / F unit 11 for each information type and transfers them to the switches 152 and 153.

At this time, the system control unit 20 has the separation unit 1
The switch 153 so that the moving picture coding information and the sound coding information separated in 22 are input to the moving picture decoding unit 132 and the sound decoding unit 142 corresponding to the respective information types through the switch 153. Is given a route selection instruction.

The moving picture decoding unit 132 has a switch 153.
It decodes and outputs the moving image coded information input from. The moving image information decoded by the moving image decoding unit 132 is converted from a digital signal to an analog signal by the D / A conversion unit 134, and then output to the display unit 26 and displayed as a moving image.

Further, the voice decoding unit 142 is provided with the switch 1
The audio encoded information input from 53 is decoded and output. The audio signal decoded by the audio decoding unit 142 is D
After being converted from a digital signal to an analog signal by the / A converter 144, the analog signal is output to the speaker 24 and reproduced as voice.

In this communication terminal 100, when the multimedia information generated by the terminal itself is transmitted to the partner terminal, the multimedia information can be recorded by the terminal itself.

In this case, in addition to the switching instruction of the switch 151 described above, the system control unit 20 encodes the moving image coding information coded by the moving image coding unit 131 and the speech coding unit 141. A switch instruction is given to the switch 152 so that the audio coded information is input to the recording process 16 through the switch 152.

The recording processing unit 16 multiplexes the moving image coding information and the audio coding information input from the switch 152 and sends them to the system control unit 20 via the buffer 18.

The system controller 20 records the information temporarily accumulated in the buffer 18 on the recording medium 21.

More specifically, in the recording processing section 16, the encoding information of the moving image encoding information and / or the audio encoding information inputted from the switch 152, or either one of them is determined in advance as recording format information (recording described later). After being converted to a format frame), it is sent to the system control unit 20 via the buffer 18, and the system control unit 20 records the recording format information on the recording medium 21.

Further, in this communication terminal 100, when reproducing the multimedia information received from the partner terminal, the multimedia information can be recorded in the own terminal.

In this case, in addition to the switching instruction of the switch 153 described above, the system control unit 20 adds the multiplexing unit 12
A switching instruction is given to the switch 152 so that the moving image coding information and the audio coding information separated in 2 are input to the recording processing unit 16 through the switch 152.

In the recording processing section 16, as in the case of recording the coded information generated by the own terminal described above, either or both of the moving picture coded information and the audio coded information inputted from the switch 152 are coded. The converted information is converted into predetermined recording format information and then sent to the system control unit 20 via the buffer 18. System control unit 20
Then, the recording format information is recorded on the recording medium 21.

Further, in this communication terminal 100, it is also possible to reproduce the record information previously stored in the recording medium 21 in the own terminal.

In this case, the system control unit 20 reads the recording information (recording format information) from the recording medium 21 based on the reproduction start command input from the input / operation unit 22 by the user operation, and the reproduction processing unit via the buffer 18. 1
Pass to 7.

The reproduction processing section 17 separates the moving picture coding information and the audio coding information from the recording format information passed from the system control section 20 for each information type and transfers them to the switch 151 and the switch 153.

At this time, the system control unit 20 causes the moving picture decoding unit 132 in which the moving picture coding information and the sound coding information separated by the reproduction processing unit 17 correspond to respective information types through the switch 153. The switch instruction is given to the switch 153 so as to be input to the voice decoding unit 142.

As a result, the moving image coding information input to the moving image decoding unit 132 through the switch 153 is decoded by the moving image decoding unit 132, and then the D / A conversion unit 134 outputs a digital signal. Is converted into an analog signal and then output to the display unit 26 and displayed as a moving image.

The speech coded information input to the speech decoding unit 142 through the switch 153 is decoded by the speech decoding unit 142, and then the D / A conversion unit 144 converts the digital signal into an analog signal. After the conversion, it is output to the speaker unit 24 and reproduced as a sound.

Further, in this communication terminal 100, the recorded information stored in the recording medium 21 in advance can be reproduced by the reproduction processing section 17 of the own terminal, and at the same time, the reproduced information can be transmitted to the partner terminal.

In this case, the system control unit 20 inputs the moving picture coding information and the sound coding information separated by the reproduction processing unit 17 to the decoding unit (132, 142) for each information type through the switch 153. As described above, while giving a switching instruction to the switch 153, the moving image coding information and the audio coding information are further input so as to be input to the multiplexing unit 121 through the switch 151.
A switching instruction is given to 1.

As a result, the moving image coding information and the audio coding information input to the multiplexing unit 121 through the switch 151 are multiplexed by the multiplexing unit 121 so that they can be transmitted to the communication path, and the communication I / It is transmitted to the partner terminal via the F unit 11.

As described above, in this communication terminal 100,
Switches 151 and 15 are provided in the input unit of the multiplexing unit 121, the input unit of the recording processing unit 16, and the output unit of the separating unit 122, respectively.
2, 153 are connected to these switches 151, 15
Reference numerals 2 and 153 select an information path according to a switching instruction from the system control unit 20.

Specifically, the switch 151 connected to the input section of the multiplexing section 121 sends to the communication path either the coded information generated by the terminal itself or the coded information reproduced by the reproduction processing section 16. Select whether to do.

The switch 152 connected to the input unit of the recording processing unit 16 stores the encoded information generated by the terminal itself and the encoded information received from the partner terminal and separated by the separating unit 122 for each information type. Select which one to record at your terminal.

The switch 153 provided in the output unit of the separation unit 122 receives the encoding information received from the partner terminal and separated by the separation unit 122 for each information type, and the encoding information sent from the reproduction processing unit 16. Select which of these to play back on your terminal.

On the other hand, the system control unit 20 of this communication terminal 100 uses the switches 1 for selecting the above-mentioned information paths.
In addition to issuing a switching instruction to 51, 152, 153, processing for reading the recording format information from the buffer 18 and writing it in the recording medium 21, reading information to be reproduced from the recording medium 21 and passing it to the reproduction processing unit 17 via the buffer 18. Processing, processing for setting the type of information handled by each processing unit in each processing unit in response to a request from the user, and the like are performed.

In the communication terminal 100 according to the present invention, by the above-described configuration and control, for example, while transmitting the coded information of the same type to the partner terminal through the communication path, the self terminal simultaneously records or the partner terminal records. It is also possible to record the received encoded information at the same time while reproducing the received encoded information.

Next, the operation for recording the coded information and the operation for reproducing the recorded information in the communication terminal 100 according to the present invention will be described in more detail with reference to each embodiment.

First Embodiment: The communication terminal 100 according to the first embodiment has the configuration shown in FIGS. 1 and 2, and as described in the description of the outline operation above, plural kinds of multimedia information. The encoding processing unit for each type (the moving image encoding unit 1
31, the voice coding unit 141) generates coded information, multiplexes the coded information and transmits the coded information to the partner terminal, and codes the multiplexed information received from the partner terminal for each type. A multimedia information communication function that separates and reproduces information, a recording processing function that multiplexes and records the encoded information generated by the terminal itself or the encoded information received from the partner terminal, and the recorded encoding information It has a reproduction processing function of reproducing.

Particularly, regarding the recording processing function, the recording can be temporarily stopped and released.

In the communication terminal 100 of this embodiment, for example, when recording the coding information generated by the terminal itself, the coding information coded by the moving picture coding unit 131 and the voice coding processing unit 141 is Is passed to the recording processing unit 16 through the switch 152 by the above-described information path selection control.

The recording processing section 16 fetches the coded information of a size in consideration of the capacity of the communication path for each predetermined cycle, and records the coded information in the recording format frame (according to the format of the preset record information. (For each information type) is generated.

The generated recording format frame is written in the buffer 18 and the system controller 20 is notified that recording information has been generated.

FIG. 3 is a diagram showing an example of the structure of a recording format frame according to the present invention.

This recording format frame is composed of a time stamp, attribute information of encoded information to be recorded, length information and an information part.

Here, the time stamp indicates the time from the start of recording the encoded information. Recording processing unit 16
Then, the internal timer is operated from the time when the recording process of the encoded information is started, and the value of the internal timer at the time when the recording format frame is generated is written in the recording format frame as a time stamp.

The attribute information is an information type indicating the type of information stored in the information section of the recording format frame (contents of the encoding information to be recorded), and a screen break flag indicating a break in one screen of moving image information. , A packet break flag indicating a break of a packet of various information, a recording pause flag indicating a first recording format frame in which recording pause is started (corresponding to fg1 described later),
A recording resumption flag (corresponding to fg2) indicating that it is the first recording format frame in which the recording suspension is released (recording is restarted), and an I frame flag indicating that it is an I frame of moving image information. (Same as fg3).

Next, description will be made on the recording process when the recording is temporarily stopped and restarted in the communication terminal 100 of this embodiment.

As described above, in the communication terminal 100 according to the present embodiment, it is necessary to assume that the self-terminal simultaneously records the encoded information of the same type to the other terminal through the communication path. Therefore, it is necessary to suspend only the recording process while operating the encoding unit for audio and moving images.

In the communication terminal 100 according to the present embodiment, when the recording process is temporarily stopped or the recording suspension is released during the normal recording process, the user operates the input / operation unit 22. The recording pause operation or the recording pause release operation is performed respectively.

When the recording suspension command is input from the input / operation unit 22 by the recording suspension operation, the system control unit 20 sends a recording suspension instruction to the recording processing unit 16.

When the recording processing section 16 receives the above-mentioned instruction to pause recording, the moving picture coding section 131 and the audio coding section 1
The normal recording process is performed by the above-described method until the division of the coded information sent from 41 is found.

Then, when the recording format frame of the coded information that arrives after the recording pause instruction is received and before the information delimiter is found, in the attribute information area of the recording format frame, The recording suspension flag fg1 is set to.

The delimiter of the coded information is, for example, a frame delimiter which is an information unit generated by the voice encoding unit 141 in the case of voice information, and a frame delimiter in the case of moving image information. It is used as a delimiter for a certain VOP frame.

When the recording processing section 16 receives the above-mentioned recording suspension instruction and then finds the above-mentioned information break,
The information of that type is controlled not to be recorded.

After that, when a recording pause cancel command is input from the input / operation unit 22 to the system control unit 20 by the recording pause cancel operation, the system control unit 20 instructs the recording processing unit 16 to restart recording. Send an instruction.

After receiving the instruction to restart recording, the recording processing unit 16 receives the moving image coding unit 131 and the audio coding unit 141.
Recording is restarted from the time when the break of the encoded information sent from is found.

At this time, the delimiter of the coded information is a frame delimiter which is an information unit generated by the audio coding unit 141 in the case of voice information, and is the VOP frame in the case of moving image information as in the case of temporary stop of recording. It's a break.

When the recording processing section 16 recognizes the above-mentioned division of the encoded information, the recording processing is restarted in order from the encoded information corresponding to this division. When the recording process is restarted, the recording restart flag fg2 is set in the attribute information area in the recording format frame.

FIG. 4 is a diagram for explaining the flow of moving image information processing relating to recording and reproducing operations in the communication terminal 100 of the first embodiment.

Here, FIG. 4A conceptually shows the temporal transition of the input information (moving image coded information of the recording target) to the recording processing unit 16.

This input information is, for example, either the moving picture coding information generated by the moving picture coding unit 131 of its own apparatus or the moving picture coding information received from the partner terminal and separated by the separating unit 122. It may be.

FIG. 4B conceptually shows the temporal transition of the recording format frame generated by the recording processing unit 16 when recording the input information.

The recording processing section 16 generates a recording format frame composed of the time stamp value tn, attribute information and information section based on the encoding information input in the order shown in FIG. Send to.

Here, t11 is added as the time stamp value corresponding to the first input coded information, and t12, t13, t14, ... Are sequentially added as the time stamp values corresponding to the subsequent coded information. Is added.

Further, the coding information input first is an I frame (intra-frame predictive coding information) obtained by coding using only the information of one image frame without using temporal difference information. Since the coding information input after the I frame is inter-frame prediction coding information obtained by coding only the difference information based on the information of the previous image frame, here, the coding information is input first. The I frame flag fg3 is set only in the attribute information area corresponding to the encoding.

During this recording processing operation, it is assumed that an instruction to temporarily stop recording is input at time T1 and an instruction to cancel temporary recording presentation is input at time T2 based on a user operation on the input / operation unit 22. .

In this case, in the recording processing section 16, the time T1
The generation of the recording format frame is continued for the encoded information to be input, including during the recording stop period up to time T2.

However, in this case, the recording processing section 16 has the attribute information area of the recording format frame (with the time stamp value t14) corresponding to the coded information that is input first after the recording pause command (after the time T1). A recording pause flag fg1 is set in the field.

Further, in the recording processing unit 16, within the attribute information area of the recording format frame (with the time stamp value t19) corresponding to the coded information which is input first after the recording temporary stop cancellation command (after the time T2). Is set to the recording restart flag fg2.

As a result, the buffer 18 shown in FIG.
The recording format frame sequence as shown in (b) is accumulated, and then the recording format sequence accumulated in the buffer 18 is stored in the recording medium 21 by the system control unit 20.

FIG. 4C is a reproduction processing unit read from the recording medium 21 by the system control unit 20 when reproducing the information recorded through the recording process of FIG. 17 conceptually shows the temporal transition of the recording format frame passed to the recording medium 17.

In this case, the reproduction processing section 17 refers to the time stamp value of the recording format frame passed from the system control section 20 in accordance with the above array, extracts the information section from the frame, and decodes the corresponding information type. The data is passed to the digitizing unit (132, 142) and the reproducing process is performed.

Since the time stamp value is given at the time of recording, the reproduction processing is performed in that order while comparing this time stamp value with the internal timer (reproduction timer) held by the reproduction processing section 17 at the time of reproduction. As a result, reproduction can be performed at the same speed as during recording and while maintaining synchronization between information types.

During this period, the recording format frame having the time stamp value t14 is received, and the recording pause flag fg is received.
When it is detected that 1 is set, the reproduction processing unit 17
Sequentially searches the time stamp values of the subsequent recording format frames, and when it detects that the recording restart flag fg2 is set in the attribute information area of the recording format frame of the time stamp value t19, the internal timer is turned on. The reproduction processing of the information portion of the recording format frame after the time stamp value t19 is continued after adjusting to the time stamp value t19 at the time of resuming the recording.

The above is the control for temporarily stopping the recording of the encoded information generated by the own terminal. However, when the encoded information received / separated from the partner terminal via the communication path is recorded. Also, the same processing can realize the temporary stop control of the encoded information.

As described above, the communication terminal 10 of the first embodiment.
At 0, at the time of recording the encoded information, the time stamp information indicating the elapsed time from the start of recording, at least the recording suspension flag indicating that the recording suspension has started, and the suspension of recording have been released. Attribute information that can set a recording restart flag indicating (recording has been cut off),
A recording format frame composed of the information content part of the encoded information to be recorded is generated, and at the time of reproducing the encoded information, the reproduction timer is adjusted to the time stamp value in the recording format frame and the encoded information in the frame is adjusted. And the recording restart frame is set in the recording format frame to be played back, the playback timer is set to match the time stamp value of the recording format frame in which the recording restart flag is set. The reproduction is continued from the encoded information of 1.

According to this structure, the recording and the reproduction of the coded information can be paused while the coding processing section (the moving picture coding section 131 and the audio coding section 141) is operated.

Second Embodiment: The multimedia communication terminal 100 according to the second embodiment has the first major component structure.
It is similar to that of the first embodiment, and is configured by including the components as shown in FIGS.

That is, the communication terminal 100 of the second embodiment.
Also, a plurality of types of multimedia information are encoded for each type (a moving image encoding unit 131, a voice encoding unit 141).
To generate encoded information by multiplexing the encoded information, transmit the multiplexed information to the partner terminal, and separate the multiplexed information received from the partner terminal into encoded information for each type to reproduce. A media information communication function, a recording processing function capable of multiplexing and recording the encoded information generated by the terminal itself or the encoded information received from the partner terminal, and temporarily stopping and releasing the recording, and recording. It has a reproduction processing function of reproducing the encoded information.

Regarding the recording processing function, as in the case of the first embodiment, at the time of recording the multimedia encoded information, the time stamp information and at least the attribute information for which the recording pause flag and the recording restart flag can be set. And a frame generation function for generating a recording format frame composed of the information content part of the encoded information to be recorded.

However, the communication terminal 100 of the second embodiment differs from that of the first embodiment in the processing until the recording format frame is recorded as recording information after the generation of the recording format frame. The reproduction process is also different from that of the first embodiment.

FIG. 5 is a diagram for explaining the flow of moving image information processing relating to the recording operation in the communication terminal 100 of the second embodiment.

Here, FIG. 5A conceptually shows the temporal transition of the input information (moving image coded information to be recorded) to the recording processing section 16. This input information is
This is the same as the one described in the embodiment (see FIG. 4A).

FIG. 5B conceptually shows the temporal transition of the recording format frame generated by the recording processing unit 16 when recording the input information.

The recording format frame generation processing is also similar to the recording format frame generation processing described in the first embodiment [see FIG. 4 (b)].

That is, also in this case, t11 is added as the time stamp value of the recording format frame corresponding to the first input coded information, and the time stamp of the recording format frame corresponding to the subsequent input coded information is added. The values are t12, t13, t1 in order.
4, ... Are added.

Since the first input coded information consists of I frames, the I frame flag fg3 is set in the attribute information area corresponding to the coded information.

Further, during the recording process, in response to the input of the recording temporary stop command at the time T1 and the input of the recording temporary presentation cancellation at the time T2, after the recording temporary stop command (after the time T1). In the attribute information area of the recording format frame (with the time stamp value t14) corresponding to the encoded information first input to the recording pause flag fg1
Is set, and the recording restart flag fg2 is set in the attribute information area of the recording format frame (with the time stamp value t19) corresponding to the encoded information that is input first after the recording suspension cancellation command (after time T2). To be done.

Furthermore, the communication terminal 10 according to the second embodiment.
In 0, the recording format frame is generated by the processing of FIG. 5B and accumulated in the buffer 18, and then the recording format frame is processed and passed to the system control unit 20 and stored in the recording medium 21.

FIG. 5 (c) conceptually shows the temporal transition of the actual recording information obtained by processing the recording format frame accumulated in the buffer 18 by the processing of FIG. 5 (b). .

That is, in the communication terminal 100 of the present embodiment, the recording processing section 16 restarts recording from the position where the recording pause flag fg1 is set in the recording format frame generated by the processing of FIG. 5B. Flag fg
The recording format frames up to the position where 2 is set are deleted, and the time stamp values recorded in the remaining recording format frames are adjusted so as to be continuous.

Specifically, in this embodiment, the recording format frame having the time stamp value of t15 to the recording format frame having the time stamp value of t18 shown in FIG. 5B is deleted and the time stamp of t14 is deleted. T1 after recording format frame with value
After connecting the recording format frames having the time stamp value of 9, the recording format frames after the recording format frame having the time stamp value of t19 have time stamp values t19, t20, t21, ... The time stamp values t15 and t1 are set so that the serial numbers after the time stamp value t14 of the first recording format frame are generated.
6, t17, ...

Next, when reproducing this recording information, the reproduction processing section 17 refers to the time stamp value of the recording format frame passed from the system control section 20 according to the array shown in FIG. The information part of the corresponding information type is extracted from the decoding part (13
2, 142) to perform a reproduction process.

In this case, since the time stamp value during the transition from the recording pause position to the recording pause release position is a continuous value, the reproduction processing section 17 determines that the first
The process of searching the time stamp value at the time of releasing the recording pause from the time stamp value at the time of temporarily stopping the recording as in the case of the above embodiment is unnecessary, and the reproducing process can be simplified.

As described above, the communication terminal 10 of the second embodiment.
At 0, at the time of recording the encoded information, the time stamp information indicating the elapsed time from the start of recording, at least the recording pause flag indicating that the recording pause is started, and the fact that the recording pause is released A recording format frame composed of attribute information that can set a recording restart flag and an information content part of the encoding information of the recording target is generated, and among the generated recording format frames,
The recording format frames from the position where the recording pause flag is set to the position where the recording restart flag is set are deleted, and the time stamp values recorded in the remaining recording format frames are adjusted so that they are continuous. ing.

According to this structure, when the coded information is reproduced, the reproduction timer is made to sequentially reproduce the coded information in the recording format frame in accordance with the time stamp value in the recording format frame. Image coding unit 1
31. It is possible to temporarily stop the recording of the encoded information and reproduce it while moving the audio encoding unit 141).

Third Embodiment: In the multimedia communication terminal 100 according to the third embodiment, the structure of the main part is the first.
The second embodiment is similar to that of the second embodiment, and is configured by including the constituent elements as shown in FIGS.

That is, the communication terminal 100 of the third embodiment.
Also, a plurality of types of multimedia information are encoded for each type (a moving image encoding unit 131, a voice encoding unit 141).
To generate encoded information by multiplexing the encoded information, transmit the multiplexed information to the partner terminal, and separate the multiplexed information received from the partner terminal into encoded information for each type to reproduce. A media information communication function, a recording processing function capable of multiplexing and recording the encoded information generated by the terminal itself or the encoded information received from the partner terminal, and temporarily stopping and releasing the recording, and recording. It has a reproduction processing function of reproducing the encoded information.

Regarding the recording processing function, as in the above-mentioned respective embodiments, at the time of recording the multimedia encoded information, the time stamp information and at least the attribute information for which the recording pause flag and the recording restart flag can be set. , And a frame generation function for generating a recording format frame composed of an information content part of encoded information to be recorded.

However, in the communication terminal 100 of the third embodiment, the recording format frame generation function is different from that of each of the above embodiments.

As described above, in each of the above-described embodiments, the recording processing unit 16 that receives the instruction to restart recording (cancels recording pause) has the encoding processing unit (moving image encoding unit 13).
1. The recording is restarted from the time when the boundary of the encoded information sent from the audio encoding unit 141) is found.

Here, the information delimiter is a frame delimiter, which is an information unit generated by the speech encoding unit 141 in the case of voice information, and a VOP frame delimiter in the case of moving image information, which is the same as the pause of recording. I am trying.

However, in the case of moving picture information, if the VOP frame is separated as in the case of the temporary stop of recording, a problem occurs when reproducing the recorded information.

That is, since the normal VOP frame is encoded based on the temporal difference information between the frames, there is a possibility that the normal reproduction cannot be performed due to the loss of the information of the recording pause period.

The third embodiment is made in view of this point, and the recording is restarted from the I frame to be encoded without using the temporal difference information.

In order to realize this, in the communication terminal 100 of the third embodiment, when the recording restart is instructed by the user in the recording suspended state, the system control unit 20 causes the encoded information generated in the own terminal. Is recorded, the I-frame is transmitted to the encoding processing unit of the own terminal, and to the encoding processing unit of the other terminal when encoding information received from the other terminal is recorded. Give instructions to insert.

According to this instruction, the encoding processing unit of the instruction destination generates and outputs the I frame of the encoding information of the moving image at that time.

The recording processing section 16 recognizes the time when the corresponding I frame arrives after the above instruction as a delimiter, and restarts the recording processing from that time. When the recording process is restarted, the recording restart flag f is added to the attribute information area in the recording format frame corresponding to the moving image information to be recorded.
Set g2.

FIG. 6 is a diagram for explaining the flow of moving image information processing relating to the recording operation in the communication terminal 100 of the third embodiment.

Here, FIG. 6A conceptually shows the temporal transition of the input information (moving image coded information of the recording object) to the recording processing section 16.

This input information is, for example, the moving picture coding information generated by the moving picture coding unit 131 of the own device or the moving picture coding information separated by the separating unit 122 received from the partner terminal. May be

FIG. 6B conceptually shows the temporal transition of the recording format frame generated by the recording processing unit 16 when recording the input information.

In the communication terminal 100 of this embodiment, the recording processing section 16 is composed of a time stamp value tn, attribute information, an information section, etc. based on the encoded information input in the order shown in FIG. 6 (a). A recording format frame is generated and sent to the buffer 18.

Here, t11 is added as the time stamp value corresponding to the first input coded information, and t12, t13, t14, ... Are sequentially added as the time stamp values corresponding to the subsequent coded information. Is added. Further, since the first input coded information is composed of an I frame, the I frame flag fg3 is set in the attribute information area corresponding to the coded information.

During this recording processing operation, it is assumed that a recording suspension command is input at time T1 and a recording temporary presentation cancellation command is input at time T2 based on a user operation on the input / operation unit 22. .

In this case, in the recording processing unit 16, the time T1
The generation of the recording format frame is stopped for the encoded information input during the recording stop period up to time T2.

At this time, the recording processing section 16 causes the recording format frame (time stamp value t) corresponding to the coded information first input after the recording pause command (after time T1).
The recording suspension flag fg1 is set in the attribute information area of the No. 14).

Thereafter, at time T2, the input / operation unit 22
When the recording temporary stop canceling operation is performed, a recording temporary stop canceling command is input from the input / operation section 22 to the system control section 20.

When the recording pause cancel command is input, the system control unit 20 causes the encoding processing unit (moving image encoding unit 131) of the own terminal which is the transmission source of the encoding information to be recorded at this time. Alternatively, I
Instruct to insert a frame.

On the other hand, in the recording processing section 16, when the relevant I frame arrives from the encoding processing section of the transmission source after the above I frame insertion instruction, the temporary stop of the recording is canceled and the recording is restarted, and the arrival concerned. The generation process of the recording format frame is continued from the I frame.

At this time, the recording processing unit 16 records the recording format frame (with the time stamp value t15) corresponding to the encoded information (I frame) that is input first after the recording temporary stop cancellation command (after the time T2). A recording restart flag fg2 is set in the attribute information area of. Also, the first input coded information is recognized as an I frame, and the I frame fg3 is also set in the attribute information area of the recording format frame.

As a result, the buffer 18 shown in FIG.
The recording format frame sequence as shown in (b) is accumulated, and then the recording format sequence accumulated in the buffer 18 is stored in the recording medium 21 by the system control unit 20.

When reproducing the information recorded on this recording medium, the reproduction processing section 17 is controlled by the system control section 20 shown in FIG.
While referring to the time stamp value of each recording format frame passed according to the array shown in (b), the information part from each frame is extracted and passed to the decoding part (132, 142) of the corresponding information type for reproduction. Perform processing.

During this period, the recording format frame having the time stamp value t14 is received, and the recording pause flag fg is received.
When it is detected that 1 is set, the reproduction processing unit 17
Prepares to reproduce the I frame, and then detects that the recording restart flag fg2 and the I frame flag fg3 are set in the attribute information area of the recording format frame of the time stamp value t15 to be input, The I frame is reproduced, and then the normal difference information is reproduced, and the encoded information (information portion) in the subsequent recording format frame is arranged in the order of the time stamp value (t16, t17, ...). Continue the process of reproducing.

In this way, the communication terminal 10 of the third embodiment
At 0, at the time of recording the encoded information, time stamp information indicating the elapsed time from the start of recording, attribute information for which at least the recording restart flag indicating that the temporary stop of recording has been released, and the recording target While generating a recording format frame consisting of the information content part of the encoded information,
Inserting an I frame into the encoding processing unit of the own terminal or the encoding processing unit of the other terminal, which is the transmission source of the encoding information of the recording target, by giving a command to cancel the pause of recording When the I frame arrives after this instruction, the temporary stop of the recording is released, and the generation process of the recording format frame is continued in order from the arrived I frame.

According to this structure, when the recording pause is released and the recording is resumed, the I frame (encoded by using only the information of one image frame without using the difference information) is always included in the recording format frame. Since the recording information is inserted, it is possible to suppress the disturbance of the moving image even when the recording information is paused and the recorded information is interrupted in the middle.

Fourth Embodiment: In the multimedia communication terminal 100 according to the fourth embodiment, the structure of the main part is the first.
The third embodiment is similar to the third embodiment, and is configured by including the components shown in FIGS. 1 and 2.

That is, the communication terminal 100 of the fourth embodiment.
Also, a plurality of types of multimedia information are encoded for each type (a moving image encoding unit 131, a voice encoding unit 141).
To generate coded information, multiplex the coded information and transmit the coded information to the partner terminal, and separate the multiplexed information received from the partner terminal into coded information for each type to reproduce. It has a media information communication function, a recording processing function for multiplexing and recording the encoded information generated by the terminal itself or the encoded information received from the partner terminal, and a reproduction processing function for reproducing the recorded encoded information. It was done.

However, the communication terminal 100 of the fourth embodiment is based on the premise that the recording processing function does not have a function of temporarily stopping recording and then canceling the temporary recording stop.

Further, in the communication terminal 100 of the fourth embodiment, the recording format frame generating function at the time of recording the encoded information is different from that of each of the above embodiments.

FIG. 7 is a diagram for explaining the flow of moving image information processing relating to the recording operation in the communication terminal 100 of the fourth embodiment.

Here, FIG. 7A conceptually shows the temporal transition of the input information to the recording processing unit 16 (moving image coded information to be recorded).

This input information is, for example, either the moving picture coding information generated by the moving picture coding unit 131 of its own apparatus or the moving picture coding information received from the partner terminal and separated by the separating unit 122. It may be.

In the recording processing unit 16 of the communication terminal 100 of the fourth embodiment, at the time of recording the encoded information, the encoding processing unit (moving image encoding unit) of the own terminal which is the transmission source of the encoded information to be recorded. 131) or instructing the encoding processing unit of the partner terminal to insert an I frame, and when the corresponding I frame arrives after the instruction, the generation process of the recording format frame is continued from the I frame in order. .

As the timing for instructing the insertion of the I frame, for example, it can be considered to be periodically performed based on the time counting by the timer (not shown) of the own device.

In this case, the system control unit 20 causes the encoding processing unit (moving image code) of its own terminal, which is the transmission source of the encoding information to be recorded at this time, every time the above-mentioned timer measures the constant time t0. The encoding unit 131) or the encoding processing unit of the partner terminal is instructed to insert the I frame [see FIG. 7 (a)].

On the other hand, the recording processing unit 16 continues the generation processing of the recording format frame in order from the arrived I frame when the corresponding I frame arrives from the transmission source encoding processing unit after the above I frame insertion instruction. .

FIG. 7B conceptually shows the temporal transition of the recording format frame generated by the recording processing unit 16 when recording the input information.

That is, the recording processing section 16 is shown in FIG.
A recording format frame composed of a time stamp value tn, attribute information, an information section and the like is generated based on the encoded information input in the order shown in (1) and sent to the buffer 18.

Here, t11 is added as the time stamp value corresponding to the first input coded information, and t12, t13, t14, ... Are sequentially added as the time stamp values corresponding to the subsequent coded information. Is added.

Since the first input coded information consists of I frames, the I frame flag fg3 is set in the attribute information area corresponding to the coded information.

Further, in the communication terminal 100 of the present embodiment, the I / O frame coding information is periodically input by the processing shown in FIG.
The I frame flag fg3 is also set in each of the attribute information areas of the recording format frames (time stamp values t14, t17, t20, ...) Corresponding to the frame.

As a result, the buffer 18 shown in FIG.
The recording format frame sequence as shown in (b) is accumulated, and then the recording format sequence accumulated in the buffer 18 is stored in the recording medium 21 by the system control unit 20.

In this case, the information recorded on the recording medium 21 is a recording format frame sequence in which I frames are periodically inserted.

When reproducing the record information recorded on the recording medium 21 in this manner, the reproduction processing section 17 causes the system control section 20 to pass the time stamp value of each recording format frame passed in accordance with the arrangement shown in FIG. 7B. Referring to, extracting the information part from each frame and passing it to the decoding part (132, 142) of the corresponding information type,
Perform playback processing.

In this case, the recording format frame sequence delivered from the system control unit 20 is such that I frames are periodically inserted [see FIG. 7 (b)].

As described above, the I frame is generated by encoding using only the information of one image frame without using the temporal difference information, and is suitable for editing such as cutting and pasting. It can bear.

Therefore, the communication terminal 1 of the fourth embodiment in which such an I frame is periodically inserted in the recorded information.
At 00, the record information can be edited at an arbitrary position where the I frame is inserted.

In the above description, an example of instructing to periodically insert an I frame based on the timing of the timer is given, but another example is input / input.
Input a command to insert an I frame from the operation unit 22,
System control unit 2 recognizing that this command has been input
The configuration may be such that the recording processing unit 16 instructs to insert an I frame via 0.

Further, when recording the encoded information, a variation recognition means for recognizing that one or more types of encoded information has changed is provided, and when the variation is recognized by this variation recognition means, the system control is performed. The recording processing unit 16 may instruct the insertion of the I frame via the unit 20.

As the fluctuation recognizing means in this case, for example, a moving picture information fluctuation recognizing means for recognizing that the fluctuation of the moving picture information becomes larger than a preset size due to a scene change or the like can be considered.

As another example of the fluctuation recognizing means, a voice information fluctuation recognizing means for recognizing that the voice signal has changed from the silent state to the voiced state can be considered.

In this case, the I-frame can be inserted into a portion where there is a scene change or a characteristic portion such as a change from silence to a sound through the variation recognition means, whereby the recorded image is recorded at these characteristic portions. Can be edited.

As described above, in the communication device 100 according to the fourth embodiment, at the time of recording the encoded information, the time stamp information indicating the elapsed time from the start of recording and the attribute of the encoded information to be recorded are shown. A recording format frame composed of attribute information and an information content part of the encoded information is generated, and the encoding processing part of the own terminal which is the transmission source of the encoded information to be recorded or the other party while recording the encoded information. The encoding processing unit of the terminal is instructed to insert an I frame, and when the I frame arrives after the instruction, the generation processing of the recording format frame is continued from the I frame. .

According to this structure, for example, by periodically instructing to insert an I frame,
It is possible to obtain recorded information in which frames are periodically inserted, and an image can be edited at an arbitrary position where an I frame in this recorded information is inserted.

Further, according to the fourth embodiment, the fluctuation of the moving image information becomes large at the time when the command is input by the user, the scene change, or the like, or the sound changes from silence to sound. At the time of recognition, it may be instructed to insert each I frame, and in this case, there is a feature such as a command from the user, a scene change, or a change from silence to voice. It is possible to obtain a recorded image in which an I frame is inserted at a certain position, and it is possible to edit the image at each of the above positions in the recorded image where the I frame is inserted.

The present invention is not limited to the embodiments described above and shown in the drawings, but can be appropriately modified and carried out within the scope of the invention.

For example, as a modification of each of the third and fourth embodiments described above, when the insertion of the I frame is instructed, the CI
It is also possible to instruct to insert information and generate a recording format frame in which the relevant I frame and CI information are inserted when the corresponding I frame and CI information arrive based on these instructions.

Further, it is also possible to combine the respective embodiments, such as applying the technique for periodically inserting the I frame and CI information according to the fourth embodiment to the third embodiment.

Further, in FIG. 1, a communication terminal 1 for transmitting / receiving multimedia information via a wired communication path.
However, the present invention can also be applied to a portable communication terminal that wirelessly transmits and receives multimedia information.

[0203]

As described above, according to the first aspect of the present invention, when the encoded information is recorded, the time stamp information and the flag indicating that the recording is restarted after the recording is temporarily stopped are set. In addition to generating the recording format frame including the encoded information, when reproducing the encoded information, the reproduction timer adjusts the encoded information in the frame according to the time stamp value in the recording format frame, and in the recording format frame to be reproduced. If the above flag is set in, the reproduction timer is made to continue the reproduction of the encoded information in accordance with the time stamp value of the recording format frame in which the flag is set. On the other hand, the recording of the encoded information can be paused and reproduced.

Further, in the second invention, at the time of recording the encoded information, the recording format frame including the time stamp information, the attribute information which can set the recording pause flag and the recording restart flag is generated, and the generated recording is performed. Of the format frames, delete the recording format frame from the position where the recording pause flag is set to the position where the recording restart flag is set, and adjust so that the time stamp values recorded in the recording format frame are continuous. Therefore, at the time of reproducing the encoded information, the reproduction timer is simply made to reproduce the encoded information in the frame according to the time stamp value in the recording format frame, and the encoded information is operated while the encoding processing unit is operating. You can pause recording and playback.

Further, according to the third invention, at the time of recording the encoded information, the recording format frame including the time stamp information and the attribute information for which the recording restart flag can be set is generated, and the recording restart command is given. By doing so, it is possible to insert the intraframe predictive coding information (I frame) into the coding processing unit of the own terminal or the coding processing unit of the partner terminal which is the transmission source of the coding information to be recorded. Since the generation of the recording format frame is continued from the I frame when the I frame arrives after the instruction, the moving image information recorded first after the recording is restarted always starts from the I frame. Therefore, it is possible to suppress the disturbance of the moving image even when the recording is paused and the recorded information is interrupted in the middle.

Further, according to the fourth invention, at the time of recording the encoded information, while performing the processing of generating the recording format frame including the time stamp information and the attribute information indicating the attribute of the encoded information to be recorded, A time point at which an I frame arrives after the instruction is given to the encoding processing unit of the own terminal or the encoding processing unit of the partner terminal that is the transmission source of the encoding information to be recorded Since the generation process of the recording format frame is continued from the I frame in sequence, for example, by periodically instructing to insert the I frame,
It is possible to obtain recorded information in which frames are periodically inserted, and an image can be edited at an arbitrary position where an I frame in this recorded information is inserted.

Further, according to the fourth aspect of the invention, the fluctuation of the moving image information becomes large when the command is input by the user, the scene change, or the like, or the sound changes from silence to voice. It may be instructed to insert an I frame at each point in time, and in this case, there is a feature such as a command from the user, a scene change, or a silence to a voice. A recorded image in which an I frame is inserted can be obtained, and the image can be edited at each of the above positions in the recorded image where an I frame is inserted.

[Brief description of drawings]

FIG. 1 is a block diagram showing the overall configuration of a multimedia communication terminal according to the present invention.

FIG. 2 is a diagram showing a connection form of each unit in the communication terminal of FIG.

FIG. 3 is a diagram showing a configuration example of a recording format frame according to the present invention.

FIG. 4 is a diagram for explaining the flow of moving image information processing relating to recording and reproducing operations in the communication terminal of the first embodiment.

FIG. 5 is a diagram for explaining the flow of moving image information processing relating to a recording operation in the communication terminal of the second embodiment.

FIG. 6 is a diagram for explaining the flow of moving image information processing relating to a recording operation in the communication terminal of the third embodiment.

FIG. 7 is a diagram for explaining the flow of moving image information processing relating to a recording operation in the communication terminal of the fourth embodiment.

FIG. 8 is a diagram for explaining the flow of moving image information processing relating to the recording operation in the conventional apparatus.

[Explanation of symbols]

100 multimedia communication terminals 11 Communication interface (I / F) section 12 Multiplexing / demultiplexing processing unit 121 Multiplexer (MUX) 122 Separator (DMX) 13 Video coding / decoding processing unit 131 Video Coding Unit (V Enc) 132 Video Decoding Unit (V Dec) 14 Voice Encoding / Decoding Processing Unit 141 Speech Encoding Unit (A Enc) 142 Speech Decoding Unit (A Dec) 133,143 A / D converter (A / D) 134, 144 D / A converter (D / A) 15 Route selector 151,152,153 switch (SW) 16 Recording processing unit (REC) 17 Playback processing unit (PLAY) 18 buffers 19 Host interface (I / F) section 20 System control unit 21 storage medium 22 Input / operation section 23 Microphone 24 speaker 25 Camera section 26 Display

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C053 FA12 FA14 GB04 GB06 GB11                       GB37 HA26 HA33 JA07 JA22                       JA23 KA08 LA05 LA06 LA11                       LA15                 5C059 KK36 KK40 PP05 PP06 PP07                       RC04 RC26 RF14 SS06 SS16                       TA25 TA74 TC21 TC41 TC42                       TC43                 5K028 CC02 DD06 EE07 KK01 KK03                       KK12 LL12 MM05 MM08 RR02                       SS05 SS15

Claims (10)

[Claims] 1. A plurality of types of multimedia information are coded by a coding processing unit for each type to generate coded information, the coded information is multiplexed and transmitted to a partner terminal, and received from the partner terminal. A multimedia information communication function for separating the reproduced multiplexed information into encoded information for each type and reproducing the encoded information,
The coded information generated by the terminal itself or the coded information received from the partner terminal is multiplexed and recorded, and a recording processing function capable of pausing and resuming the recording and reproducing the recorded coded information. A multimedia communication device having a reproduction processing function, the time stamp information indicating an elapsed time from the start of recording the encoded information, and indicating that the recording is restarted after the recording is temporarily stopped. Frame generating means for generating a recording format frame including attribute information for which a flag can be set; and, when reproducing the encoding information, a reproduction timer that encodes the recording format frame according to a time stamp value in the recording format frame. When the information is reproduced and the flag is set in the recording format frame to be reproduced, the reproduction timer is set to the flag. Multimedia communication apparatus characterized by comprising a reproduction control means for continuing the reproduction of the coded information in accordance with the time-stamp value of the recording format frame grayed is set. 2. A plurality of types of multimedia information are encoded by each type of encoding processing unit to generate encoded information, and the encoded information is multiplexed and transmitted to a partner terminal and received from the partner terminal. A multimedia information communication function for separating the reproduced multiplexed information into encoded information for each type and reproducing the encoded information,
The coded information generated by the terminal itself or the coded information received from the partner terminal is multiplexed and recorded, and a recording processing function capable of pausing and resuming the recording and reproducing the recorded coded information. A multimedia communication device having a reproduction processing function, comprising: time stamp information indicating an elapsed time from the start of recording the encoded information, a recording pause flag indicating that recording has been paused, and recording restarted. A frame generation means for generating a recording format frame including attribute information capable of setting a recording restart flag indicating that the recording is restarted, and the recording restart from the position where the recording pause flag is set in the generated recording format frame. The recording format frame up to the position where the flag is set is deleted and recorded in the recording format frame Frame processing means for adjusting the time stamp value so that the time stamp values are continuous; and reproduction control means for sequentially reproducing the coded information in accordance with the time stamp value in the recording format frame when reproducing the coded information. A multimedia communication device comprising: 3. A plurality of types of multimedia information are coded by a coding processing unit for each type to generate coded information, the coded information is multiplexed and transmitted to a partner terminal, and received from the partner terminal. A multimedia information communication function for separating the reproduced multiplexed information into encoded information for each type and reproducing the encoded information,
The coded information generated by the terminal itself or the coded information received from the partner terminal is multiplexed and recorded, and a recording processing function capable of pausing and resuming the recording and reproducing the recorded coded information. A multimedia communication device having a reproduction processing function, wherein time stamp information indicating an elapsed time from the start of recording the encoded information, and attribute information capable of setting a recording restart flag indicating that recording has been restarted A frame generation means for generating a recording format frame including a frame and a recording restart command are input, so that the encoding processing unit of the own terminal or the encoding processing unit of the partner terminal which is the transmission source of the encoding information of the recording target And an instruction means for instructing to insert the intraframe predictive coding information, and at the time when the intraframe predictive coding information arrives after the instruction. Resume recording, multimedia communication apparatus characterized by comprising a recording control means for continuing the process of generating the recording format frames from the intra-frame prediction coding information in order. 4. When generating a recording format frame of the intraframe predictive coding information, a flag indicating that it is intraframe predictive coding information is set in the recording format frame. 3. The multimedia communication device described in 3. 5. A plurality of types of multimedia information are coded by a coding processing unit for each type to generate coded information, the coded information is multiplexed and transmitted to a partner terminal, and received from the partner terminal. A multimedia information communication function for separating the reproduced multiplexed information into encoded information for each type and reproducing the encoded information,
A recording processing function of multiplexing and recording the encoded information generated by the own terminal or the encoded information received from the partner terminal,
A multimedia communication device having a reproduction processing function for reproducing recorded encoded information, wherein time stamp information indicating an elapsed time from the start of recording the encoded information and an attribute of the encoded information to be recorded Frame generating means for generating a recording format frame including attribute information indicating the encoding information, and an encoding processing unit of the own terminal which is the transmission source of the encoding information of the recording target or an encoding of the partner terminal when recording the encoding information. Instructing means for instructing the processing unit to insert the intraframe predictive coding information, and the recording format in order from the intraframe predictive coding information when the intraframe predictive coding information arrives after the instruction. A multimedia communication device, comprising: a recording control unit for continuing a frame generation process. 6. The method according to claim 5, further comprising a clocking unit, wherein the instructing unit instructs to regularly insert the intraframe prediction coded information based on the clocking of the clocking unit. Multimedia communication device. 7. When recording the coded information, a fluctuation recognition means is provided for recognizing that one or more kinds of coded information has changed, and the instruction means recognizes the fluctuation by the fluctuation recognition means. The multimedia communication device according to claim 5, wherein the multimedia communication device is instructed to insert the intraframe prediction coded information. 8. The multimedia communication according to claim 7, wherein the fluctuation recognizing unit comprises a moving image information fluctuation recognizing unit that recognizes that the fluctuation of the moving image information is larger than a preset size. apparatus. 9. The multimedia communication device according to claim 7, wherein the fluctuation recognizing means comprises a voice information fluctuation recognizing means for recognizing that the voice signal has changed from a silent state to a voiced state. . 10. An input unit for inputting a command to insert the intraframe predictive coding information, wherein the instructing unit includes:
The multimedia communication device according to claim 5, wherein the multimedia communication device is instructed to insert the intraframe prediction coded information based on the command input from the input means.