JP2006262205A - Encoder, codec method, and network transmission system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an encoder, CODEC method and network transmission system for realizing a CODEC in a form suitable for a requested video in accordance with the type of the video by taking into consideration occurrence of image collapse caused by occurrence of a packet loss and a delay without securing a fixed line band since a line band fluctuates to cause deterioration or the like of a communication band when 1111047964827_1. html to communicate with and traffic increase in a line that does not guarantee the line band of a network line. <P>SOLUTION: The adoption of a DSP (digital signal processor) detects the optimization of multi-CODEC software for simultaneously executing CODEC processing of a plurality of formats and a band state of a best effort circuit to perform an adaptive shaping function for controlling a video distribution rate. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a network transmission system, and more particularly to a video and audio encoder and codec method.

Conventionally, a codec method of JPEG (Joint Photographic Experts Group) format has been developed. The encoder using this JPEG codec has been downsized and built into the camera as a network camera. In addition, an encoder using a JPEG codec is built into the unit, and it is also used as a WEB encoder to attach externally to the camera.
In addition to JPEG codecs, products that use MPEG (Moving Pictures Experts Group) format (for example, MPEG-2 and MPEG-4) codecs in network cameras and web encoders are used. (For example, refer to Patent Document 1). A codec is a combination of compression and decompression, or coding and decoding, and there are several formats depending on how the codec is coded.

JP 2004-295835 A

Conventionally, a plurality of formats of codecs are properly used according to the type of video, and a codec of a format suitable for the type of video has not been used.
An object of the present invention is to provide an encoder, a codec method, and a network transmission system that realize a codec of a format suitable for a requested video according to the type of video.
Another object of the present invention is an encoder, a codec method, and a network transmission system that simultaneously execute a plurality of types of codec processes, and further, a distribution method and distribution for detecting a transmission state and controlling an optimal video distribution rate. To provide a system.
Furthermore, another object of the present invention is to provide an encoder and codec method and a network transmission system capable of simultaneously compressing voice and performing two-way voice communication.

  In order to achieve the above object, the encoder of the present invention provides a priority according to the type of data to be distributed, and executes a codec in a format suitable for the data requested for distribution according to the priority.

In order to achieve the above object, the codec method of the present invention provides a priority according to the type of data to be distributed, and executes a codec in a format suitable for the data requested to be distributed according to the priority.
In order to achieve the above object, the network transmission system of the present invention provides priority according to the type of data to be distributed, and executes a codec in a format suitable for the data requested for distribution according to the priority.

  According to the present invention, an encoder, a codec method, and a network transmission system that simultaneously execute a plurality of types of codec processes can be realized. Furthermore, a distribution method and a distribution system for detecting a transmission state and controlling an optimal video distribution rate have been realized.

This section describes network cameras and web encoders that have both JPEG codec and MPEG-4 codec functions for network cameras and web encoders that process multiple formats of codecs.
Multi-codec is a network-type surveillance system with simultaneous compression and audio coding functions in both MPEG-4 format with high compression efficiency suitable for video surveillance and JPEG format suitable for still image recording in real time. This is the WEB encoder function to be realized.

  The JPEG codec and the MPEG-4 codec can be realized by, for example, SH-4 (RISC processor) software. However, by adopting DSP (Digital Signal Processor) with higher processing capability, simultaneous processing of MPEG-4 and JPEG is realized in real time, and a web encoder that supports both systems is realized by a multi-codec board.

  In particular, the present invention realizes an adaptive shaping function for controlling the video distribution rate by optimizing multi-codec software that simultaneously executes a plurality of types of codec processing by the DSP and detecting the bandwidth situation of the best effort line.

  Due to the expansion of high-speed network infrastructure and the progress of ubiquitous society, the transition to network-type systems is also progressing in the surveillance system market and the security system market. In addition to the JPEG format, there is a growing need for an MPEG-4 format that has higher compression efficiency and is suitable for monitoring video in surveillance systems and security systems.

  FIG. 1 is a block diagram showing a configuration of a network type monitoring system according to an embodiment of the present invention. 11 is a camera, 12 is a microphone / speaker unit, 14 is a web encoder, 15 is a recording device, 16 is a network line, and 17 is a monitoring center. The camera 11 is a two-dimensional image sensor such as a video camera or a TV camera. The recording device 15 is a network server equipped with, for example, a DVD (Digital Versatile Disc) and an HD (Hard Disk). The network line 16 is, for example, ADSL (Asymmetric Digital Subscriber Line).

  The camera 11 captures an image obtained by capturing an image within a visual field range such as a road and outputs the image as a video signal to the WEB encoder 14. The microphone of the microphone / speaker unit 12 collects ambient sounds, converts them into electric signals, and outputs them to the web encoder 14. The web encoder 14 converts the video signal input from the camera 11 into video data of a plurality of codecs and outputs the video data. In addition, the electric signal of the sound input from the microphone of the microphone / speaker unit 12 is converted into sound data in a format to be distributed to the network line 16 and output. The converted video data converted into a plurality of codec formats and the sound data converted into a format to be distributed to the network line 16 are output to the recording device 15 and the network line 16. The recording device 15 stores the input data.

The monitoring center 17 has a function of capturing video data and sound data from the network line 16, decoding and outputting (video playback and sound playback), and at least one terminal that outputs data from a desired camera or microphone. have.
The monitoring center 17 plays the live video and sound acquired by the camera. When necessary, it sends a distribution request to the WEB encoder 14 via the network line 16 and acquires video data and sound data from the recording device 15. You can also get live video and sound. The speaker of the microphone / speaker unit 12 converts an electrical signal of sound input from the network line 16 via the WEB encoder 14 into sound and outputs the sound.
Further, the display screen of the terminal can be divided into a plurality of screens, and switching can be made so that a plurality of live images from a plurality of cameras and a plurality of images from a recording device are simultaneously output.

  As shown in FIG. 1, for example, as a function necessary for a network type monitoring system in road monitoring, the WEB encoder and WEB decoder of the present invention have a monitoring function for monitoring with a moving image, a function for periodically recording road conditions, Three sound functions for sound and loud sound are required. Based on these market trends, simultaneous MPEG-4 format suitable for moving image transmission and JPEG format suitable for still image recording such as easy search and audio encoding (audio compression) function are installed. The main functions of the surveillance system are the interactive voice call function, the image distribution function to mobile phones, various camera controls and alarm functions.

  In the present invention, in order to realize the platform of the codec board that enables the series development of products, the size of the board of the codec board is, for example, a small business card size (55 mm × 88 mm). In addition, a digital signal processor (DSP) was mounted on a business card-sized board, and the software was made compact by realizing simultaneous compression processing.

  However, in order to realize simultaneous compression processing of MPEG-4, JPEG, and audio in real time with a DSP with limited capabilities, it is necessary to optimize software that can efficiently use DSP capabilities.

On the other hand, in the network type monitoring system, video is distributed through all IP (Internet protocol) network lines such as the Internet, an intranet, and a dedicated line. However, it communicates on the best-effort line that does not guarantee the bandwidth of the network line.
1111047964827_0.html
When traffic increases, the line bandwidth fluctuates and the communication bandwidth decreases. For this reason, a fixed line bandwidth cannot be secured, packet loss and delay occur, and image collapse occurs. Therefore, it is necessary to have an adaptive shaping function that detects packet loss and dynamically controls the distribution data rate to be adapted to the fluctuating line bandwidth.

  In FIG. 1, a web encoder 14 converts an analog video signal input from the camera 11 and an audio signal input from the microphone / speaker unit 12 into a digital signal, and performs MPEG-4, JPEG and audio compression processing. (Encoding process) is performed simultaneously. Each compressed data is packetized by a network protocol and distributed to the network line 16. Further, it outputs to the recording device 15 and accumulates video data and audio data for a long time. Furthermore, for simple storage, recording to a removable memory (detachable memory) that can hold data even without a built-in power supply is possible (described later).

Further, in the monitoring center 17 of FIG. 1, a WEB decoder is connected from the network line 16 via an interface, and the video data and audio data distributed via the network line 16 are decompressed and output to a monitor or a speaker.
The monitor displays the video of the distributed video data, and the speaker converts the distributed audio data into sound and outputs it.
That is, the WEB decoder decompresses each compressed data received from the network (video data compressed in a plurality of formats and compressed audio data), and converts the digital signal into an analog video signal and audio signal.

The multi-codec of the present invention has the following features.
(1) Realizing moving image monitoring and high-definition image recording simultaneously.
(2) Two-way voice communication is possible.
(3) It is possible to minimize the occurrence of freeze of received images and the occurrence of block noise even on a best effort line that does not guarantee the line bandwidth.
(4) As a backup in case of a line failure, a detachable memory capable of recording still images and holding data even when there is no power source is provided in the apparatus, and simple recording is possible.
(5) Images can be recorded by external alarm and scheduled capture.
(6) Still image distribution to mobile phones is possible.

An example of the main equipment configuration of the present invention is shown in FIG. 13, and a configuration example of the network type monitoring system of the present invention is shown in FIG. FIG. 13 is a diagram for explaining an embodiment of the main device configuration of the present invention. FIG. 2 is a block diagram showing the configuration of an embodiment of the network type monitoring system of the present invention.
The video distribution system (site side) 210 has a web encoder 214. The web encoder 214 is connected to the electric pan head camera 211. In addition to the electric camera platform 211, although not shown, a video distribution device such as an analog camera or a recording device can be connected. A network camera with a built-in web encoder may also be used.

Further, the web encoder 214 is coupled with a microphone / speaker unit 212 having at least a microphone, a speaker, and an amplifier unit, and is coupled with a microphone / speaker unit 233 on the video display system 230 side, and the video distribution system 210 side and the video display system. A two-way voice call is made with the 230 side via the network line 220.
In addition, since the mobile phone 222 can be connected to the network line 220, the video distribution system 210 side and the video display system 230 side can talk to the mobile phone 222 at any time, and from the video distribution system 210 side. The video information can be distributed on the mobile phone 222. The video information to be distributed is, for example, a still image, but a frame-moving moving image or the like is also possible depending on the transmission capability.

  The video display system (monitoring side) 230 includes a web decoder 232 and a client PC (personal computer) 231. The web decoder 232 is used when reliability is required from the client PC 231 such as display on an analog general monitor, simple start-up just by turning on the power, or continuous monitoring. The client PC 231 displays the monitoring image on its own display.

An example of the configuration and basic processing of the WEB encoder 214 of the present invention will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of the WEB encoder of the present invention.
In FIG. 3, the web encoder 214 includes a multi-codec board unit 300 that performs compression processing and a network distribution board unit 350 that distributes compressed data to the network line 220.

The multi-codec board unit 300 includes an analog video input unit 310, an analog audio input / output unit 320, a multi-codec processing unit 330, and a compressed data memory 340.
The analog video input unit 310 receives an analog video signal from the motorized camera platform 211, passes through an LPF (low-pass filter) and an amplifier, and then converts it to a digital video signal at the A / D conversion unit. The signal is stored in the frame memory.

The analog audio input / output unit 320 converts the analog audio signal output from the microphone / speaker unit 212 into a digital audio signal by the A / D conversion unit, and outputs the digital audio signal to the multi-codec processing unit 330. It also supports voice (two-way) received from the network line 220.
In the multi-codec processing unit 330, the image capturing / conversion unit reads the digital video signal from the frame memory of the analog video input unit 310, receives the digital audio signal from the analog audio input / output unit 320, MPEG-4, JPEG, and audio signal Simultaneous compression processing is performed. That is, the video signal from the image capture conversion unit is compressed by the MPEG-4 compression unit and the JPEG compression unit, respectively, and output to the compressed data memory 340, respectively. The audio signal is compressed by the audio compression unit and output to the compressed data memory 340.
The compressed data memory 340 stores the compressed data after compression.

The network distribution board unit 350 includes a protocol processing unit 351 and a memory control unit 352.
The protocol processing unit 351 reads each compressed data from the compressed data memory 340 of the multi-codec board unit 300, and distributes it to the network line 220 by a network protocol.
The memory control unit 352 performs processing of writing JPEG into the removable memory 353 by an alarm signal from a sensor or the like.

  The specifications of one embodiment of the WEB encoder of the present invention are shown in FIG. The following describes (1) software optimization and (2) implementation of an adaptive shaping function for simultaneous video distribution in both MPEG-4 format suitable for moving image transmission and JPEG format suitable for still image recording.

(1) Software optimization
Optimizes multi-sk processing that simultaneously executes MPEG-4, JPEG, and audio compression processing, which is a feature of WEB encoders.
The multi-codec board unit (hereinafter referred to as codec board) performs video and audio compression processing by DSP software processing and outputs compressed data to an externally connected network distribution board unit (hereinafter referred to as distribution board) .

The DSP for embedded use is smaller than the processor used on a PC and is suitable for built-in equipment because it can reduce the number of parts used, but the processing power is low.
In order to efficiently execute a plurality of compression processes with such a low-processing DSP, a multi-task configuration of multi-codec software is used. The configuration will be described with reference to FIG. FIG. 4 is a schematic diagram for explaining an outline of an embodiment of the software processing of the codec board having the multitask configuration of the present invention.

In setting priorities for executing each task, the basic method is to increase the priority on the compressed data output processing side. Thereby, the delay time can be minimized without accumulating data in the buffer between tasks. Also, from the highest priority of each compression method, audio compression, JPEG compression, and MPEG-4 compression are performed in this order. The audio compression processing time is about 1 msec for audio data processing equivalent to 30 milliseconds (msec), which is shorter than image compression processing such as JPEG compression and MPEG-4 compression. Also, since JPEG compression is used for recording, missing frames in the image are not allowed. Therefore, the priority is set next to the voice. Normally, all the compression processing is completed within the required time. However, if complicated image input continues and MPEG-4 format compression processing time increases and DSP processing time is insufficient, MPEG-4 The compression process is postponed to allow temporary frame loss.
Furthermore, the priority of the image capture conversion process is set higher than the JPEG and MPEG-4 related processes so that the captured image data is not lost.
Based on the above basic idea, the final task priority was determined.

The above is one embodiment of the multi-task configuration of the present invention, but the part that has taken measures against the shortage of processing amount due to the increase in the compression processing time of the MPEG-4 format will be described in more detail.
In the image data capture conversion process, for example, it is necessary to capture image data (30 fps) input from the outside at intervals of 33 msec. However, the MPEG-4 compression processing time varies depending on the complexity of the input image, so that the next image data capture conversion process may not be started and image data may be lost.

FIG. 15 shows the relationship between the complexity of the image data and the task processing time obtained by acquiring the task processing time measured by the experiment.
Fig. 15 (a) shows the relationship between the road monitoring image and the processing time of Diva with Noise, which is the ITU-R evaluation image, Fig. 15 (b) is the road monitoring image, and Fig. 15 (c) is the ITU-R evaluation. Images are shown.

  The road surveillance image will be processed within 33 msec of one frame period when MPEG-4 format and JPEG format compression are executed simultaneously. However, Diva with Noise, which is an ITU-R evaluation image, does not finish within 33 msec when compression processing is executed simultaneously. For this reason, frame loss occurs.

The details of the image data loss will be described with reference to FIG. FIG. 5 is a block diagram showing the configuration of an embodiment of the WEB encoder of the present invention.
The MPEG-4 and JPEG processing tasks are common to the image capture conversion processing unit and the compression processing unit, respectively.
The MPEG-4 image capture conversion processing captures image data and executes filter processing such as image size conversion. The MPEG-4 compression processing unit compresses and outputs the image data. The output (OUT) of the interrupt processing / DMA transfer unit in FIG. 5 is image data transfer by DMA transfer triggered by Vsync input. This process is performed at regular intervals of 33 msec period. Therefore, if the image data is not read within 33 msec after the DMA transfer, the next image data will be overwritten.

Since the MPEG-4 image capture conversion processor and the MPEG-4 compression processor are the same task, the two are executed sequentially. For this reason, when the compression processing time increases, there is a problem that image data cannot be captured and a frame is lost.
The JPEG processing task has the same configuration as the above MPEG-4 processing task.
Here, the breakdown of the processing time is as shown in FIG. 16, for example.

As shown in FIG. 16, the MPEG-4 and JPEG image capture conversion processing is constant regardless of the complexity of the image. The difference in the image capture and conversion processing time between MPEG-4 and JPEG is because the image size is different between MPEG-4 and JPEG and the processing contents are different, as shown in the web encoder specification in Fig. 14. .
Therefore, frame loss occurs because the image capture conversion process cannot be executed at a fixed period of 33 msec, and the cause is that the MPEG-4 compression processing time increases due to the complexity of the image. On the other hand, the image capture conversion processing time is not affected by the complexity of the image.

The operation when an increase in MPEG-4 compression processing time causes other compression processing time to be insufficient and a JPEG frame loss occurs will be described with reference to the time chart of FIG. The horizontal axis shows the passage of time.
(a) is a diagram showing the timing at which the Vsync signal is input, and the drawn down arrows indicate the timing at which the Vsync signals Vsync A, Vsync B, Vsync C, Vsync D, and Vsync E are input. (b) shows the timing at which the frame memory image is stored, and the framed images are shown by the Vsync A, Vsync B, Vsync C, Vsync D, and Vsync E triggers of the Vsync signal, respectively. It is a part storing MA, MB, MC, MD, and ME, respectively. (c) shows the timing when the MPEG-4 processing task is executed. MPA, MPB, MPC, and MPD surrounded by a frame are the corresponding frame memory images MA, MB, MC, and MD, respectively. This is the timing part of executing a -4 format codec. (d) shows the timing when the JPEG processing task is executed. JPB and JPC surrounded by a frame are the timing parts where the corresponding frame memory images MB and MC are executed with the JPEG codec, respectively. It is.

  5 and 6, Vsync B is input, and the image data is stored as image data MB in the frame memory image. As a result, the MPEG-4 processing task MPB is executed. Here, the JPEG processing task is executed next, but if the image of the frame memory image MB is complex and the processing time of the MPEG-4 processing task MPB increases, the image data MB of the frame memory is changed during the JPEG processing task. The image data MC is overwritten and JPEG compression of the frame memory image MB cannot be performed. For this reason, JPEG frame loss occurs.

  Therefore, the image data capturing process is made independent as a high priority task. In addition, an image capture conversion process at a fixed interval of 33 msec, which is the Vsync input interval, and a compression process whose processing time varies are connected in a queue. FIG. 7 shows a task configuration diagram in which this image capture conversion processing is made independent.

FIG. 7 is a diagram showing a task configuration in which image capture conversion processing according to another embodiment of the present invention is made independent.
As shown in FIG. 7, by making the image capture conversion processing independent as one task, it is possible to absorb temporal fluctuations caused by the compression processing and prevent frame loss.
Also, by placing a queue for absorbing the fluctuation time between the image capture conversion process and the compression process, memory for storing reduced images such as 320 × 240 pixels whose image size has been changed by the image capture conversion process Improved usage efficiency.
In this way, task switching is triggered by Vsync and image capture conversion processing is executed at 33 msec intervals to prevent missing frames.

Furthermore, an embodiment for optimizing task processing switching will be described.
JPEG is used for recording and missing frames are not allowed. On the other hand, MPEG-4 used for moving image monitoring has a high frame rate, so even if a temporary frame loss occurs, there is little influence on grasping the monitoring target. Therefore, JPEG compression processing task priority is set higher than MPEG-4, and JPEG frame loss is prevented by switching tasks when inputting image data even during MPEG-4 compression processing.

The operation will be described with reference to the time chart of FIG. 8 for JPEG and MPEG-4. FIG. 8 is a diagram for explaining an operation by task switching when image data is input. The horizontal axis shows the passage of time.
(a) is a diagram showing the timing at which the Vsync signal is input, and the drawn down arrows indicate the timing at which the Vsync signals Vsync A, Vsync B, Vsync C, Vsync D, and Vsync E are input. (b) shows the timing at which the frame memory image is stored, and the framed images are shown by the Vsync A, Vsync B, Vsync C, Vsync D, and Vsync E triggers of the Vsync signal, respectively. It is a part storing MA, MB, MC, MD, and ME, respectively. (c) shows the timing when JPEG image capture processing is executed. The frame memory image MA is captured at the JWA box surrounded by a frame. (d) shows the timing when the MPEG-4 image capture processing is executed. The MWA, MWB, MWC, and MWD enclosed in a frame are the corresponding frame memory images MA, MB, MC, and MD. It is taken in.
(e) shows the timing when the JPEG processing task is executed. The JDA framed by the frame is the timing part when the corresponding frame memory image MA is executed by the JPEG codec. (f) shows the timing at which the MPEG-4 processing task is executed. The MDA, MDB, MDC, and MDD enclosed in a frame are the corresponding frame memory images MA, MB, MC, and MD, respectively. This is the timing part of executing a -4 format codec.

  7 and 8, JPEG image data import / conversion processing JWA and MPEG-4 image data import / conversion processing MWA for frame memory image MA are executed, then JPEG compression processing JDA with high priority is executed, then MPEG- 4 Execute compression processing MDA.

As in FIG. 5 and FIG. 6, it is assumed that the image input at this time is complicated and the MPEG-4 compression processing time has increased. When the frame memory image MB is input to the image capture conversion processing unit, all processing is interrupted and processing with higher priority is executed by task switching.
In this case, since the priority order of the MPEG-4 image capture conversion processing MWB is high, the MPEG-4 compression processing MDA is interrupted, and the image capture conversion processing MWB for the frame memory image MB is executed. After that, MPEG-4 compression processing MDA is continued in the remaining time. The captured image data is stored in a queue, and after the MPEG-4 compression process MDA is completed, the frame memory image MB is compressed.

  In this way, task switching is triggered by the input of frame memory images via Vsync, image capture conversion processing is executed at 33 msec intervals, and JPEG processing is performed preferentially to prevent missing frames in JPEG images. be able to.

  FIG. 17 shows an example of the processing time of each task in the setting according to the multi-codec specification of the present invention. For the input image, we selected “Diva with Noise”, which is a complex image in which the MPEG-4 compression processing time increases extremely. The total processing time per second is 884.7 msec, and the margin of processing time is 12%.

Next, (2) Realization of the adaptive shaping function Realization of the adaptive shaping function is to perform video distribution and distribution rate control adapted to fluctuations in the network line bandwidth.

  The WEB encoder can distribute video (images) over all IP network lines such as the Internet, intranet, and dedicated lines. However, when stream (compressed data) distribution is performed, frame loss may occur on the WEB decoder side. This is because the MPEG-4 compressed data stream fluctuates in the amount of compressed data for each frame, and even if the average rate is within the line bandwidth, the data amount greatly fluctuates in a burst direction in the time direction. Is temporarily exceeded.

Even if the average rate is within the line bandwidth, the burstiness of the data may exceed the line bandwidth when the amount of data increases. As a result, stream data cannot be transmitted, UDP packets cause packet loss, and the decoder causes display image corruption.
As a countermeasure, the present invention uses a transmission method generally called shaping as shown in FIG. FIG. 9 is a diagram for explaining an embodiment of stream transmission by shaping according to the present invention.

As shown in FIG. 9, shaping is a method for preventing packet loss on the line by adjusting the packet output interval of bursty data to make it constant in the time axis direction. However, as a shaping issue, in the best effort type line such as the Internet or an intranet where the line band fluctuates, the line band temporarily falls below the amount of data to be transmitted even within the line band where the average rate is guaranteed. Sometimes packet loss occurs. Therefore, when used on a best effort line, image corruption occurs due to packet loss even if shaping is performed.
Therefore, in the present invention, adaptive shaping is performed. This adaptive shaping function is a function that detects the bandwidth state of the best effort line and dynamically controls the video distribution rate. An outline of an embodiment of the adaptive shaping function of the present invention will be described with reference to FIG.

FIG. 10 is a diagram for explaining the outline of an embodiment of the adaptive shaping function of the present invention.
When a web encoder delivers packets at T1 second (T1 = T2 ÷ 2) intervals for a best effort line that can only pass one packet per T2 seconds, half of the delivered packets are discarded within the line. To the web decoder.
Therefore, the WEB decoder detects whether there is a packet that has not arrived according to the sequence number assigned to each received packet.

  When detecting a packet that has not arrived, the WEB decoder notifies the WEB encoder of packet loss information. As a result, the WEB encoder can detect that the packet sent to the WEB decoder has not arrived correctly. The WEB encoder adjusts the packet distribution interval from T1 seconds to T2 seconds based on this information, so that the WEB decoder can receive all the distributed packets.

Next, a method for adjusting the packet delivery interval will be described with reference to FIG.
FIG. 11 is a diagram for explaining the state of an embodiment of an MPEG-4 stream according to the packet delivery interval adjustment method of the present invention.
In FIG. 11, the WEB encoder sequentially writes the stream to the ring buffer after the encoding process. As the ring buffer goes around, older streams are overwritten sequentially.

  The writing unit is MPEG-4 stream VOP (Video Object Plane) unit. The VOP represents one frame image in the MPEG-4 stream, and there are two types of reference image I-VOP (Intra-Video Object Plane) and difference image P-VOP (Predictive-Video Object Plane). The reference image I-VOP can decode an image of one frame only with this data. In addition, the difference image P-VOP is difference data with the information of the previous image, and this data alone cannot be decoded correctly.

FIG. 11 represents an MPEG-4 stream, and one square represents one frame. The square written I represents I-VOP, and the square written P represents P-VOP.
The time axis advances from the back to the front, and the front is the newest frame. A dotted frame in FIG. 11 represents a lost frame. The P-VOP difference data after this lost frame cannot be correctly decoded and the image becomes corrupted, and all frames up to the next I-VOP cannot be displayed correctly.

However, the P-VOP immediately before the I-VOP does not affect other frames even if it becomes a lost frame. Utilizing this feature, when discarding data that could not be sent due to fluctuations in the line bandwidth, discarding the image from the frame immediately before the I-VOP prevents the delivery of corrupted images.
In this way, the WEB encoder performs adaptive shaping to control the distribution data rate adapted to the fluctuation of the line bandwidth, and after the packet loss occurs, it is possible to minimize image corruption of the WEB decoder received video.

  An example of the appearance of the WEB encoder and WEB decoder according to the present invention is shown in FIG. A business card size multi-codec board is installed in each of the WEB encoder and WEB decoder.

  As described above, according to the present invention, by software optimization, a multi-codec that can efficiently perform simultaneous MPEG-4, JPEG, and audio compression processing on a DSP with limited processing capability in real time is applied. By implementing a shaping function, video distribution that minimizes image collapse is possible even on a best-effort line where the line bandwidth varies.

  In the above embodiments, MPEG-4, JPEG, and audio codecs have been described. However, the compression format, the number thereof, and the combination thereof are not limited to the above embodiments. Further, in the above system, a software codec that combines at least one of encryption, multi-stream distribution, or image recognition functions, and a more efficient H.264 standard. A H.264 codec may be used.

The block diagram which shows the structure of the network type monitoring system of one Example of this invention. The block diagram which shows the structure of one Example of the network type monitoring system of this invention. The block diagram which shows the structure of one Example of the WEB encoder of this invention. The schematic diagram for demonstrating the outline | summary of one Example of the software processing of the codec board of the multitask structure of this invention. The block diagram which shows the structure of one Example of the WEB encoder of this invention. The time chart for demonstrating one Example of this invention. The block diagram which shows the structure of one Example of the WEB encoder of this invention. The time chart for demonstrating one Example of this invention. The figure for demonstrating one Example of the stream transmission by shaping of this invention. The figure for demonstrating the outline | summary of one Example of the adaptive shaping function of this invention. The figure for demonstrating the state of one Example of the MPEG-4 stream by the adjustment method of the packet delivery interval of this invention. The figure which shows one Example of the external appearance of the WEB encoder and WEB decoder by this invention. The figure for demonstrating one Example of the main apparatus structures of this invention. The figure which shows the specification of one Example of the WEB encoder of this invention. The figure for demonstrating the relationship between the complexity of image data, and task processing time. The figure for demonstrating the relationship between the complexity of image data, and task processing time. The figure which shows one Example of the processing time of each task in the setting according to the specification of the multi codec of this invention.

Explanation of symbols

  11: Camera, 12: Microphone / speaker unit, 14: Web encoder, 15: Recording device, 16: Network line, 17: Monitoring center, 210: Video distribution system, 211: Electric pan head camera, 212: Microphone / speaker , 214: Web encoder, 215: Detachable memory, 220: Network line, 221: Broadband network, 222: Mobile phone, 230: Video display system, 231: Client PC, 232: Web decoder, 233: Microphone / speaker 236: Monitor, 300: Multi-codec board unit, 310: Analog video input unit, 320: Analog audio input / output unit, 330: Multi-codec processing unit, 340: Compression data memory, 350: Network distribution board unit, 351: Protocol processor, 352: Memory controller.

Claims (3)

An encoder characterized by providing a priority according to the type of data to be distributed and executing a codec in a format suitable for the data requested for distribution according to the priority. A codec method characterized by providing a priority according to the type of data to be distributed and executing a codec in a format suitable for the data requested for distribution according to the priority. A network transmission system characterized in that a priority is set according to the type of data to be distributed, and a codec in a format suitable for the data requested for distribution is executed according to the priority.