JP2003111036A - Video and audio compression method, video and audio compression apparatus, and supervisory video recording apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video and audio compression storage method that can attach audio to video at all times with an easy configuration even when the video is intermittently compressed and stored. SOLUTION: An HDD recorder 1 is connected to many digital cameras 2a, 2b, etc., (incorporating a microphone), and includes; a changeover device 3; an MPEG encoder 4; a video intermittent/audio continuous compression section 5; an HDD/IF 6; an HDD 7; an MPEG encoder 8; and a CPU 9. The video intermittent/audio continuous compression section 5 extracts frames of the number on the basis of an instruction of the CPU 9 from the MPEG stream outputted from the MPEG encoder 4, inserts an audio stream of the frame not extracted between the extracted frames to form a new MPEG stream.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video / audio compression method capable of always adding audio to a video with a simple structure even if the video is intermittently compressed and stored.

Further, the present invention relates to a monitoring video recording apparatus capable of always adding audio to a video with a simple structure and storing the video even if the video is compressed and stored intermittently.

[0003]

2. Description of the Related Art In recent years, high-efficiency coding (data compression) technology for images and audio has been used to increase the number of channels of television broadcasting.
Video on Demand (VOD: Video on Demand) allows you to watch your favorite programs whenever you want
d), video and audio storage on computer (HD
D) and replay are possible.

For example, in a recording device for monitoring, a VTR that stores analog data and a digital HDD have come to be used.

Such a digital monitoring recording device (H
A DD recorder) is a JPEG (Joint Photograph) that sequentially switches a plurality of surveillance cameras and compresses one image independently to store intermittently.
The mainstream is the one using the Phics Experts Group) and audio signal compression.

That is, JPEG is mainly a still image compression method, and since there is no regulation of audio signal encoding or synchronous multiplex method, a still image is compressed in synchronization with an audio signal by combining other methods. In order to do so, complicated processing was performed internally.

For example, using another coding for voice,
It was managed separately from the video and managed to synchronize the video with the audio. In addition, in order to match the timing of the image and the sound at the time of reproduction, the reproduction was performed so that the time stamps of both were matched by looking at each management division method.

On the other hand, MPEG (Moving Pict)
ure Expects Group Phase)
Is a method of collectively compressing a plurality of files (video, audio). That is, it is not a prescribed method for compressing a still image.

[0009]

However, the conventional HDD recorder as described above is a system that uses JPEG and audio signal compression.
There is a problem in that the apparatus itself becomes expensive because complicated processing is required to match the time stamps of G and the compression of the audio signal.

Further, when the surveillance recording device is used for surveillance of an entrance where there is almost no traffic, the same image is continuous, so one frame (still image) is compressed and stored in a plurality of frames. Is preferred.

Further, it is preferable that, for example, a voice when a door is opened is added to a still image and stored when the sound is detected.

However, for this reason, the conventional method is JP
The EG coding and the audio compression coding are provided to intermittently compress the image to compress the audio.

The MPEG system is also a system which defines image compression, audio compression, and synchronous multiplexing, and compression can be performed with an arbitrary number of frames depending on the control method.

However, since the time stamp of the audio is not managed and compressed, the MPEG system must always add the audio to the compressed data and save even if the image is compressed and stored intermittently. I can't.

That is, for example, when the surveillance recording device is used for surveillance of an entrance where there is almost no traffic, in order to obtain a video signal added with a voice or the like, video and audio are continuously compressed. Since it has to be saved, there is a problem that the memory capacity is consumed enormously.

The present invention has been made to solve the above problems, and provides a video / audio compression / storing method capable of always adding audio to a video with a simple structure even if the video is intermittently compressed and stored. With the goal.

[0017]

According to the video / audio compression method of the present invention, an MPEG stream in which an image and audio are multiplexed is input, and the MPEG stream is set in accordance with a preset extraction parameter of once for n frames. Extracting frame data from a stream, and extracting the audio packet when the audio packet exists in frame data other than the extracted frame data,
The gist includes the step of synthesizing the extracted frame data and the voice packet and sending the synthesized stream to a subsequent stage.

The video / audio compression device is a video / audio compression device connected and used in a subsequent stage of an MPEG encoder for transmitting an MPEG stream in which an image and audio are multiplexed.

Means for extracting the frame data from the MPEG stream and inputting the MPEG stream, and analyzing the frame data other than the extracted frame data according to a preset extraction parameter of once per n frames. Then, when the voice packet exists in the frame data, a means for extracting the voice packet and a new stream in which the voice packet is added to the frame data along with the output of the extracted frame data are provided. The gist is that it has means for sending out.

Further, with the input of field screen data, an image compression means for obtaining and transmitting an image stream consisting of image packets compressed by high efficiency coding for each field, and when an audio signal is input in the field An audio compression unit for obtaining an audio stream composed of the compressed audio packet, and a frame extraction for inputting the image stream and extracting and outputting a frame from the image stream according to a preset extraction parameter of once per n frames. Means for analyzing frame data other than the frame data to be extracted, and when the voice data packet exists in the frame data, voice extracting means for extracting the voice packet and output of the extracted frame data The voice packet is added to the frame data. Further comprising a synthesizing means for sending a new stream pressurized and gist.

Further, the video recording device for monitoring is connected to a large number of video cameras and audio acquisition devices, inputs video signals and audio signals from the video cameras and audio acquisition devices according to the switching instruction, and outputs these signals. In a monitoring video recording apparatus for storing a compressed and multiplexed stream on a disc, the MPEG stream is input, and a frame is extracted from the MPEG stream according to a preset sampling parameter of once for n frames. Means for extracting and outputting, and analyzing frame data other than the extracted frame data, and when there is the voice packet in the frame data, outputting means for extracting the voice packet and outputting the extracted frame data. Accordingly, a new stream in which the voice packet is added to the frame data is added. And summarized in that and means for delivering over arm.

[0022]

BEST MODE FOR CARRYING OUT THE INVENTION <First Embodiment> FIG. 1 is a schematic configuration diagram of a video / audio compression / storing system according to a first embodiment. This embodiment will be described using a monitoring recording device (HDD recorder). The present embodiment is a HDD recorder 1 for monitoring.
Uses the MPEG method as a compression technique for reducing the amount of data.

The HDD recorder 1 of FIG. 1 is connected to a large number of digital cameras 2a, 2b, ...
MPEG encoder 4 and video intermittent / audio continuous compression unit 5
, HDD / IF6, HDD7, MPEG DEC
An MPEG ENCO equipped with an ORDER8 and a CPU
The video intermittent / audio continuous compression unit 5 extracts a number of frames (frames) from the MPEG stream output from the RDER 4 based on the instruction of the CPU 9, and inserts the audio stream of the frames not extracted between the extracted frames. To form a new MPEG stream.

The above-mentioned switch 3 takes in the video signal from the camera of the camera number according to the instruction from the CPU 9 inside.

In the MPEG ENCODER 4, the video signal from the switch 3 is a DCT (Discrete Cos).
In Transform), compression is performed using not only correlation between frames but also correlation between frames using a technique called motion compensation. It also compresses the audio signal.

Then, the MPEG which is obtained by compressing the video signal and the audio signal and combining the respective compressed data.
Send the stream.

The video intermittent / audio continuous compression unit 5 is an MPEG
A number of frames (frames) based on the instruction of the CPU 9 are extracted from the MPEG stream output from the ENCODER 4, and the audio stream of the frames that have not been extracted is inserted between the extracted frames to create a new M
It is a PEG stream.

The HDD / IF 6 is provided with a new MPEG stream (hereinafter referred to as a new MPE) from the video intermittent / audio continuous compression unit 5.
HDD 7 after converting the G stream) into a predetermined format.
, Or send to MPEG DECORDER 8 for decoding.

FIG. 2 is a schematic block diagram of the video intermittent / audio continuous compression unit 5 of the first embodiment. As shown in FIG. 2, the frame extracting means 12 and the audio detecting means 13
, Audio extraction means 14, and synthesis instruction means 15
And a synthesizer 16. The frame extracting means 12 extracts the MPEG stream Hi stored in the buffer 11 and stores it in the buffer 15 according to the extraction parameter ei set by the CPU 9 once for every f pieces.

The audio detection means 13 searches for a frame other than the sampling frame by the sampling parameter ei, and detects an audio signal in the searched frame.

For example, when the extraction parameter ei is set once for three frames (frames), "2, 3", "5, 6", ... Frames are extracted every time the frame is extracted by the frame extraction means 12. To detect if audio is present in this frame. Since the images are intermittent, MPEG encoding is an e-picture o
Perform with nly.

In the MPEG stream Hi, as shown in FIG. 3, one frame data (which becomes GOP in the case of a picture only called a GOP unit) becomes "Pack".
"Header", "PES VIDEOOA" and "Pa
ck Header ”and“ Pac
k Header ”,“ PES VIDEO BOB ”and“ P
ack Header ”and“ pa
or "Packing".
"Header", "PES VIDEOOA" and "Pac
"K Header" and "Pack
Header "," PES VIDEOB "and" Pa
ck Header ”and“ Pac
k Header ”,“ PES AUDIO ”and“ pa
Dding ”.

The above-mentioned "PES AUDIO" indicates that it is present in the frame 3 (marked with a triangle).

That is, VIDEO and AUDIO indicate that the code systems are different, and audio is detected by the difference in the code system.

The audio extracting unit 14 extracts the audio from the frame using the number of frames and the packet number notified when the audio detecting unit 13 detects the audio, and stores it in the buffer 16. At this time, the packets not extracted are deleted from the buffer 11.

The synthesizing instructing means 17 causes the frame of the buffer 15 to be sent out and the audio signal of the buffer 16 to be sent out every time the frame extracting means 12 extracts a frame.

The synthesizer 18 includes a buffer 15 and a buffer 1.
6 and the output of 6 are combined. That is, a new MPEG stream Hp is generated by combining the extracted frame and the audio signal.

The video / audio compression / storing system configured as described above will be described below. In this embodiment,
It is assumed that MPEG ENCODER4 uses MPEG2. In MPEG2, as shown in FIG. 3, image and audio data are packet-multiplexed into one bit stream, and management on the HDD is performed for one stream.

In addition, since the normal MPEG2 is a method of predicting between frames within a GOP, if the frames are intermittently extracted after sequential switching input or encoding, the image will be corrupted, so I (intra)
Only the picture (screen encoded only from the information of the frame) is encoded.

The intermittent extraction of 10 frames per second and the intermittent extraction of 2 frames per second will be described below as an example.

(10 frames per second) The CPU 9 determines the sampling parameter e as 10 frames intermittently per second (one frame out of 3 frames).
Set i to each means.

As shown in FIG. 4, the MP of the surveillance camera 2a
The EG stream Hi is generated, input to the video intermittent / audio continuous compression unit 5, and stored in the buffer 11.

Generally, the audio signal of the MPEG stream Hi is much smaller than the image data. Also, there is not an audio packet for each frame, but it is multiplexed at a random location. FIG. 4A shows an MPEG stream of 30 frames for I pictures, and indicates that there is audio in frame 3 and frame 7.

Next, the frame extracting unit 12 extracts one frame from the MPEG stream Hi in the buffer 11 by 4 frames.
Frame, 7 frame, 10 frame, 13 frame,
16 frames, 19 frames, 22 frames, 25 frames and 28 frames are extracted and stored in the buffer 15 for each extraction.

On the other hand, when the MPEG stream Hi is stored in the buffer 11 and the frame 1 is extracted according to the extraction parameter ei, the audio detecting means 13 determines whether or not there are audio packets in the frames 2 and 3 which are not extracted. Detect. In the present embodiment, an audio packet exists in frame 3 of FIG.

When there is an audio packet in the frame 3, the audio detecting means 13 informs the audio extracting means 14 of the number of frames and the position of the audio packet (packet number), and causes the audio extracting means 14 to extract the packet. To save.

When the frame extracting unit 12 is notified of the extraction, the composition instructing unit 17 immediately sends the frame of the buffer 15 to the combiner 18 and the audio packet of the buffer 16. Synthesizer 1
8 synthesizes audio packets after the frame.

That is, as shown in FIG. 4B, when an audio packet exists in the frame 3 which is not extracted by the frame extracting means 12, the audio packet of the frame 3 is added after the frame 1 and then To form a new MPEG stream Hp to which the frame 4 is added.

Further, since the frame 7 of the MPEG stream Hi is extracted by the frame extracting means 12, it means that an audio packet exists in the frame 7 as shown in FIG. 4B.

That is, 1 out of every 3 frames from 1
If only the frame is extracted, the audio packet of frame 7 can be extracted, but the audio packet of frame 3 cannot be extracted. Even if such a stream is decoded, the sound of frame 3 will be destroyed (erased).

In order to avoid this, the audio detecting means 13 and the audio extracting means 14 cause the audio packet of the frame 3 ("Pack Head Header").
r "" PES Audio Padding ").

Therefore, as shown in FIG. 4, M frames of 30 frames per second
Since the PEG stream Hi is stored in the HDD as a new MPEG stream Hp containing 10 frames of audio and video per second, the capacity of the HDD is not consumed and the audio always exists with the video.

(2 frames intermittently per second) For example, 2 frames intermittently per second and C
Set using PU. That is, the intermittent interval is set to be long.

The frame extracting means 12 is the buffer 1
Frame 1, frame 16, frame 31, ... Are extracted from one MPEG stream Hi, and are stored in the buffer 15 for each extraction.

On the other hand, when the MPEG stream Hi is stored in the buffer 11 and the frame 1 is extracted according to the extraction parameter ei, the audio detecting means 13 has audio packets in the frames 2, 3, ... It is detected whether or not the position is detected, and the position is notified to the audio extracting unit 14.

The audio extracting unit 14 extracts all the audio packets existing in the frames 3 to 15 and stores them in the buffer 16.

When the frame extracting unit 12 is notified of the extraction, the composition instructing unit 17 immediately sends the frame of the buffer 15 to the combiner 18 and the audio packet of the buffer 16. Synthesizer 1
8 synthesizes audio packets after the frame.

That is, as shown in FIG. 4 (c), when audio packets exist in the frames 3 ... 15 which have not been extracted by the frame extracting means 12, the audio packets of the frames 3 ... 15 follow the frame 1. It becomes a new MPEG stream Hp to which the frame 16 is added after that.

That is, in the case of 2 frames per second, only one frame is extracted every 15 frames, but when there are 8 audio packets between the extracted frames, a new one shown in FIG. MPEG stream H
p.

In MPEG2, each of the image and audio packets has a time stamp, and the packets are decoded so that the time stamps match each other. Therefore,
When the above stream is decoded, the sound is reproduced linearly and the image is reproduced intermittently.

When such a technique is applied to a monitoring recorder, images can be recorded intermittently and audio can be recorded linearly for a long time.

The video intermittent / audio continuous compression unit 5 of the first embodiment has a frame extracting means 12, an audio detecting means 13, an audio extracting means 14,
An IC device in which the synthesis instruction unit 15, the synthesizer 16, and each buffer are integrated may be constructed.

<Second Embodiment> FIG. 5 is a schematic configuration diagram of an IC device having a video intermittent / audio continuous compression function according to a second embodiment.

As shown in FIG. 5, the external memory controller 21, the video data control unit 22, the video encoder 23, the audio encoder 24, the audio data memory control unit 25, and the video data memory control unit 27. 1, a DMA controller 29, a multiplexer unit 30, a memory 26, a memory 28, a memory 31 and the like. These are connected to the system bus 33.

The external memory controller 21 arbitrates access requests to the external memory and internal blocks,
Controls Read / Write to external memory.

The video data control unit 22 temporarily stores the video data input from the outside in the external memory (not shown) using the external memory controller 21, and then inputs the video data to the video encoder 23.

The video encoder 23 encodes the video data input from the video data control section to create a video stream.

This video stream includes "Pack Header" and "PES VIDEO" in GOP units.
A packet including "A" and "Pack Header", and "Pack Header" and "PES VIDE"
It is composed of a packet composed of "OB" and "Pack Header", and "padding".

The audio encoder 24 encodes audio data input from the outside and generates an audio stream.

This stream is "Pack Head
er ”,“ PES AUDIO ”and“ padding ”
Composed of and.

The audio data memory control unit 25
The audio stream generated by the audio encoder 24 is temporarily stored in the memory 26, and the multiplexer unit 3
Send to 0.

The video data memory control unit 27 temporarily stores the video stream generated by the video encoder 23 in the memory 28 and sends it to the multiplexer unit 30.

The DMA controller 29 controls the DMA transfer between each block inside the IC and the external memory.

The multiplexer unit 30 outputs the video and audio streams generated by the video encoder and the audio encoder to the IC controller 3
The stream multiplexed based on the instruction 2 is output to the outside.

The IC controller sets the extraction parameter ei from the HOST computer (not shown), and the multiplexer unit 30 intermittently extracts the video stream stored in the memory 31. If an audio stream exists, this is set. A new stream is generated by synthesizing an audio stream with a video stream from which the audio stream has been extracted.

The operation of the IC device having the video intermittent / audio continuous compression function configured as described above will be described below.

When the video data is input from the video data input terminal, the video data control unit 22 temporarily stores it in the external memory using the external memory controller 21, and the video encoder 23 uses the external memory controller 21 to store the video data. A video stream that has been taken out and subjected to compression processing (MPEG) is generated. This video stream is stored in the memory 28 by the video data memory control unit 27.
Is once stored in.

The audio encoder 24 performs compression processing (MPEG1) when an audio signal is input.
To generate an audio stream. The audio stream is temporarily stored in the memory 26 by the audio memory control unit 25.

Then, the IC controller 32
Each packet is sent to the multiplexer unit 30 at the timing requested by the OST.

At this time, the IC controller 32
When the video is set to intermittent recording and the audio is set to continuous recording, the processing shown in FIG. 6 is performed.

In the present embodiment, the video will be described as once for every three frames. As shown in FIG. 6, the IC controller 32 sets the extraction parameter ei from HOST (S1). Next, the multiplexer unit 3 determines whether the video stream hi is input from the video encoder 23.
It is determined to be 0 (S2).

When it is determined in step S2 that the video stream hi is input, the sampling parameter ei
, The video stream hi is extracted (S
3).

For example, in the case of once every three frames, the multiplexer unit 30 extracts the frame 1, the frame 4, ... From the memory 31.

Next, the audio is transmitted to the multiplexer unit 30.
It is determined whether or not it has been output (S4).

When it is determined in step S4 that audio is not output, that is, when voice input is within,
The frame of the video stream extracted in step S3 is transmitted (S6).

When it is determined in step S4 that the audio has been output to the multiplexer unit 30,
The audio packet is added to the extracted video frame and output (S5).

Therefore, as described with reference to FIG. 4, when an audio packet exists in the frame 3 that has not been extracted, the audio packet of the frame 3 is added after the frame 1, and the frame 4 is added after that. It becomes a new MPEG stream Hp.

[0088]

As described above, according to the present invention, MPEG
A frame is extracted from a stream according to a preset extraction parameter of once for n frames, and when a voice packet exists in a frame other than the extracted frame, the voice packet is extracted and synthesized, and the synthesized stream Thus, the effect that the sound can be easily continuous to the extracted image is obtained.

Therefore, even if the data is stored in the HDD, the capacity is not consumed, so that the effect that the voice and the video can be stored for a long time is obtained.

In particular, when it is used for an HDD recorder, it is possible to obtain the effect that required audio and video can be stored for a long time even if the number of cameras is large.

[Brief description of drawings]

FIG. 1 is a schematic configuration diagram of a video / audio compression storage system.

FIG. 2 is a schematic configuration diagram of a video intermittent / audio continuous compression unit 5.

FIG. 3 is an explanatory diagram illustrating an MPEG stream.

FIG. 4 is an explanatory diagram illustrating a process of generating a new MPEG stream according to the present embodiment.

FIG. 5 is a schematic configuration diagram of an IC device having a video intermittent / audio continuous compression function according to a second embodiment.

FIG. 6 is a flowchart illustrating an operation of an IC device having a video intermittent / audio continuous compression function according to the second embodiment.

[Explanation of symbols]

1 HDD recorder 3 switch 4 MPEG encoder 5 Video intermittent / audio continuous compression unit 7 HDD 12 Frame removal means 13 Audio detection means 14 Audio extraction means 15 Synthesis instruction means 16 synthesizer

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/081 H04N 5/92 H 7/18 E 7/32 (72) Inventor Takeo Murayama Yokohama City, Kanagawa Prefecture 3-12 Moriya-cho, Kanagawa-ku F-term (reference) within Victor Company of Japan, Ltd. 5C053 FA11 FA23 GA11 GB11 GB22 GB30 GB38 JA05 KA04 LA01 5C054 AA01 AA05 CA04 CC03 CH01 DA01 EA01 EA03 EA07 EG04 EG06 EH07 GA01 GD09 HA07 MA0059 MA23 PP04 RB01 RC32 SS11 UA02 5C063 AB07 AC01 AC05 CA11 CA20 DA05 DA13 5D044 AB05 AB08 BC01 CC05 DE32 DE96 EF03 EF05 EF07 GK04 GK08 GK12

Claims (5)

[Claims] 1. A step of inputting an MPEG stream in which video and audio are multiplexed and extracting frame data from the MPEG stream according to a preset extraction parameter of once per n frames, and the extracted frame. When the voice packet exists in the frame data other than the data, a step of extracting the voice packet, a step of combining the extracted frame data and the voice packet, and sending the combined stream to a subsequent stage A video-audio compression method comprising: 2. A video / audio compression apparatus, which is used after being connected to an MPEG encoder for transmitting an MPEG stream in which an image and audio are multiplexed, wherein the MPEG stream is input and preset n frames are input. According to the sampling parameter that is once
A unit for extracting and outputting frame data from a PEG stream; a unit for analyzing frame data other than the frame data to be extracted; a unit for extracting a voice packet when the voice packet exists in the frame data; And a unit for transmitting a new stream in which the audio packet is added to the frame data when the frame data is output. 3. With the input of field screen data,
Image compression means for obtaining and transmitting an image stream composed of image packets compressed by high efficiency encoding for each field, and audio compression means for obtaining an audio stream composed of the compressed audio packets when an audio signal is input in the field A frame extracting unit for inputting the image stream and extracting and outputting a frame from the image stream according to a preset extraction parameter for once per n frames; and analyzing frame data other than the extracted frame data. Then, when the voice packet is present in the frame data, a voice extracting means for extracting the voice packet, and a new voice packet adding the voice packet to the frame data along with the output of the extracted frame data And a synthesizing means for transmitting a stream A video / audio compression device characterized by the above. 4. The frame data extraction instruction, the voice packet extraction instruction, and the synthesis instruction are executed by a controller, and the frame data extraction, the audio extraction, and the synthesis are executed by a multiplexer. 3. The video / audio compression device described in 3. 5. A video signal and an audio signal, which are connected to a large number of video cameras and audio acquisition units, and which follow the switching instruction, are input from these video cameras and audio acquisition units, and these signals are compressed and multiplexed. In a surveillance video recording apparatus for storing a stream in a disc, the MPEG stream is input, and the M stream is recorded according to a preset extraction parameter of once for n frames.
A unit for extracting and outputting a frame from a PEG stream; a unit for analyzing frame data other than the frame data to be extracted; and a unit for extracting a voice packet when the voice packet exists in the frame data; A video recording apparatus for monitoring, comprising means for transmitting a new stream in which the voice packet is added to the frame data in accordance with the output of the frame data.