JP2008219070A - Video encoding system, method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video encoding system capable of converting the frame rate of input video data to a frame rate suitable for video encoding. <P>SOLUTION: The video encoding system 100 is provided with a frame rate adjusting part 102 for converting the frame rate of the input video data to a specified frame rate and a video encoding part 103 for encoding the video data subjected to frame rate conversion in the frame rate adjusting part 102. The frame rate adjusting part 102 is provided with a frame buffer where the input video data are stored by a frame unit, and reads the video data stored in the frame buffer at a timing determined by the specified frame rate, which independent of a timing determined by the frame rate of the input video data. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video encoding system, and more particularly, to an encoding system that receives video from a camera device.

  With advances in terminal technologies such as personal computers (PCs), personal digital assistants (PDAs), mobile phones, and network technologies such as ADSL (Asymmetric Digital Subscriber Line), wireless LAN (Local Area Network), and third-generation mobile phone networks, Communication means combining streaming media such as video and audio are widely used.

  An example of a video encoding system is described in Patent Document 1. In this video encoding system, a video signal acquired from a video camera is compressed by an encoder, packetized, and then sent out to a playback side terminal using UDP (User Datagram Protocol).

  This type of video encoding system includes a camera device, a video encoding device, a packetizing device, and a transmission device. The operation will be briefly described below.

  First, the camera device is set. The camera device is, for example, a digital video camera or a video capture card, and settings such as an image size, an image format, and a frame rate are performed.

  After setting the camera device, start capturing video. When the start of capture is instructed, the camera device periodically captures images and performs video frame digitization processing. The camera device outputs a video signal at a set frame rate. The video signal output from the camera device is supplied to the video encoding device.

  In the video encoding device, settings such as image quality and bit rate are set in advance, and compression encoding processing is performed on the video signal supplied from the camera device. The compressed and encoded video signal is output from the video encoding device as a bit stream. The bit stream output from the video encoding device is supplied to the packetizing device.

The packetizing apparatus divides the bit stream supplied from the video encoding apparatus into a size suitable for the transmission medium and packetizes it. The packetized video data is supplied to the transmission device. The transmitting device sends the video data supplied from the packetizing device to the playback terminal via the network.
JP 2005-277950 A (page 38, Fig. 5)

  There are various types of camera devices, and there are differences in output frame rates. When a video encoding system is constructed using a PC, general-purpose device control means such as WDM (Windows Driver Model) and DirectShow, which are device driver architectures of Windows OS, are provided. An inexpensive camera device such as a connected Web camera is often used. However, depending on the type of Web camera, there are some that only support several frame rates.

  Some camera devices do not manage the frame rate with software that performs video acquisition processing such as firmware and device drivers. In such a camera device, even if the output frame rate is set, the video may not be output at the specified frame rate.

  When the camera device does not support the desired frame rate or when the camera device outputs a video signal at a frame rate different from the setting, the frame rate of the video signal output by the camera device is determined by the video encoding device. A situation occurs that is different from the requested frame rate. In general, a video encoding apparatus has a mechanism for controlling an output code amount per unit time, and requests video input time information for the control. For this reason, when a video signal is input at a frame rate different from the encoding setting, the encoding processing operation is hindered, an excessive bit stream exceeding the capacity of the transmission path is output, and image quality due to packet loss is output. The problem of deterioration occurs.

  In addition, when the video signal is not output from the camera device at the specified frame rate and an abnormality occurs in the frame rate of the input video signal, the user or application program is not notified of this, so the user There is also a problem that it is impossible to grasp the occurrence of an abnormal frame rate. This is because the video encoding system and the camera operation framework target a normal camera, and therefore do not have a frame rate monitoring function and a notification function.

  An object of the present invention is to solve the above problems and provide a video encoding system capable of converting the frame rate of input video data to a frame rate suitable for video encoding.

  It is a further object of the present invention to provide a video encoding system that can notify a user of an abnormality in the frame rate of input video data.

In order to achieve the above object, the video encoding system of the present invention includes:
A frame rate adjustment unit that converts the frame rate of the input video data to a specified frame rate;
A video encoding unit that encodes the video data that has been converted to the designated frame rate by the frame rate adjustment unit;
The frame rate adjustment unit includes a frame buffer in which the input video data is stored in units of frames, and the timing determined by the designated frame rate is independent of the timing determined by the frame rate of the input video data. The video data stored in the frame buffer is read out.

  According to the above configuration, in the frame rate adjustment unit, the writing and reading of the video data in the frame buffer are performed at independent timings. For example, when the frame rate of the input video data is lower than the specified frame rate, On the contrary, when the frame rate of the input video data is higher than the designated frame rate, the frame is discarded. As a result, the frame rate of the input video data is converted to the designated frame rate. As described above, when video data is input from the camera device at a frame rate different from the frame rate required by the video encoding unit (specified frame rate), the frame rate adjustment unit specifies the frame rate of the input video data. Since the frame rate is converted, the video data is always supplied to the video encoding unit at the designated frame rate. Therefore, it is not necessary to change the setting parameters of the video encoding unit, and there is no problem that the encoding process is stopped.

In the video encoding system,
The frame buffer is
A video data area in which video data for one frame is stored;
A frame number area in which a frame number of video data stored in the video data area is stored;
The frame rate adjustment unit
An input frame counter whose count value is incremented by one each time the input video data is supplied to the frame rate adjustment unit;
Writing the input video data in the video data area at a timing determined by the frame rate of the video data, and writing a count value of the input frame counter in the frame number area as a frame number of the video data;
A data acquisition unit that reads out the video data written in the video data area and the frame number written in the frame number area from the frame buffer at a timing determined by the designated frame rate;
When the data acquisition unit reads the video data stored in the video data area and the frame number stored in the frame number area from the frame buffer, the read frame number is read by the data acquisition unit next time. An output frame counter to hold until done,
For the video data read by the data acquisition unit, the frame number of the video data read by the data acquisition unit this time and the frame number of the video data read by the data acquisition unit previously held in the output frame counter A frame discard / duplication determination unit that determines whether or not a frame is discarded or duplicated based on the difference between
And an event notification unit for notifying event information indicating the fact to the outside when the frame discard / duplication determination unit determines that the data is after the frame has been discarded or duplicated. May be.

  According to the above configuration, video data is input from the camera device at a frame rate different from the frame rate (designated frame rate) required by the video encoding unit, and the frame rate adjustment unit discards or duplicates the frame. Then, the frame discard / duplication determination unit detects that the frame is discarded or duplicated, and the event notification unit notifies the event information indicating the fact to the outside. By notification of this event information, the user can grasp the abnormality of the frame rate of the input video data.

  According to the present invention, video data can be supplied to the video encoding unit at a specified frame rate regardless of the frame rate of the input video data. Therefore, the operation of the encoding process in the video encoding unit is not hindered, and the deterioration of the image quality due to the output of an excessive bit stream exceeding the capacity of the transmission path and the packet loss can be suppressed.

  Also, if the frame rate on the camera device side is different from the specified frame rate, or if the camera device did not output video data at the specified frame rate, event information indicating that the frame was discarded or duplicated Therefore, the user can grasp the abnormality of the frame rate based on the notified event information.

  Next, embodiments of the present invention will be described with reference to the drawings.

  In the frame rate adjustment process, the video data writing and reading timings in the frame buffer are made independent to achieve frame discard and duplication, thereby converting the frame rate of the input video data to the specified frame rate. It is characterized by the fact that frame discard or duplication is detected and notified to the outside. Hereinafter, the embodiment will be described in detail.

  FIG. 1 is a block diagram showing the main part of a video encoding system according to the first embodiment of the present invention.

  Referring to FIG. 1, a video encoding system 100 is a video encoding system that operates on a computer system to which a camera device is connected, and its main parts are a video acquisition unit 101, a frame rate adjustment unit 102, a video code, and the like. And a packetizing unit 104 and a transmitting unit 105. Computer systems include storage devices that store programs and data, input devices such as keyboards and mice, display devices such as CRTs and LCDs, communication devices such as modems that communicate with the outside, output devices such as printers, and input devices. It is comprised from the control apparatus which receives the input from and controls operation | movement of a communication apparatus, an output device, and a display apparatus. The user activates the video encoding system or gives instructions and settings necessary for operating the video encoding system through the input device.

  The video acquisition unit 101 is a functional block that acquires video data (moving images) from a camera device. The video acquisition unit 101 has a function of acquiring video data from the camera device and outputting it in units of frames. The format of the video frame is usually the YUV bitmap format, but an RGB bitmap format may be used instead. The YUV bitmap format is a format in which a color is represented by three pieces of information: a luminance signal (Y), a difference (U) between the luminance signal and the red component, and a difference (V) between the luminance signal and the blue component.

  As the camera device, a digital camera device connected to an external bus such as USB or IEEE1394, an NTSC format analog video camera, or the like can be used. When using an analog video camera, it is necessary to connect to a video capture card with a video transmission cable such as a coaxial cable.

  When the start of video acquisition is instructed, the video acquisition unit 101 autonomously starts moving image acquisition in accordance with the video size (number of vertical pixels and horizontal pixels), video format, and frame rate specified in advance. The video data output from the video acquisition unit 101 is passed to the frame rate adjustment unit 102.

  The frame rate adjustment unit 102 is a functional block that monitors the frame rate of the video data input from the video acquisition unit 101 and detects and adjusts the difference between the frame rate and the designated frame rate. The video data that has been subjected to the frame rate adjustment by the frame rate adjustment unit 102 is passed to the video encoding unit 103.

  The video encoding unit 103 is a functional block that performs compression encoding on the video data after the frame rate adjustment input from the frame rate adjustment unit 102 and converts the video data into a bit stream. Examples of compression encoding methods include MPEG-4 and H.264. Intra-frame and inter-frame coding schemes for compressing moving images such as H.264 can be used.

  Parameters set in the video encoding unit 103 are the frame rate and bit rate of the input video. The video encoding unit 103 performs quantization scale control for suppressing the output bit stream to a specified bit rate or less, and performs input video data encoding processing. As an encoding method, a method applying a still image compression technique that performs only intra-frame encoding such as Motion JPEG can be used. The output bit stream of the video encoding unit 103 is passed to the packetizing unit 104.

  The packetizing unit 104 is a functional block that divides the bit stream output from the video encoding unit 103 into an optimal size for each transmission medium and adds a header. In packetization processing in video distribution, RTP (RealTime Transport Protocol) defined by IETF RFC3550 is usually used. RTP can add information necessary for real-time streaming such as payload type, sequence number, sender identifier, and time information to the header. More detailed packet division methods and header definitions are standardized for each video encoding format. In the case of transmitting MPEG-4 format video, the RTP packet segmentation method according to IETF RFC3640 is referred to as H.264. In the H.264 format, IETF RFC3984 can be used, and in the Motion JPEG format, IETF RFC2435 can be used. The video stream divided into RTP packets by the packetizing unit 104 is passed to the transmitting unit 105.

  The transmission unit 105 is a functional block that transmits the video stream divided into RTP packets from the packetizing unit 104 to the video decoding system via the IP network. The RTP packet is transmitted using a UDP / IP protocol and a socket.

  Next, a detailed internal configuration of the frame rate adjustment unit 102 will be described.

  FIG. 2 is a block diagram showing a specific configuration of the frame rate adjustment unit 102. Referring to FIG. 2, the frame rate adjustment unit 102 includes a video input unit 201, an input frame counter 202, a frame buffer 203, a data acquisition unit 204, a timer 205, an output frame counter 206, a frame discard / duplication determination unit 207, and an event notification unit. 208 and a video output unit 209.

  The user designates the number of vertical and horizontal pixels of the video, the video format, and the frame rate to the video acquisition unit 101 before giving a distribution start instruction to the video encoding system 100. The frame rate setting set for the video acquisition unit 101 is also applied to the frame rate adjustment unit 102 and the video encoding unit 103. Based on this set value, the frame rate adjustment unit 102 sets the write / read timing in the frame buffer 203 and the interval of the timer 205.

  The video input unit 201 is called at the video data output timing of the video acquisition unit 101 (this is determined by the frame rate of the video data input from the camera device). The video input unit 201 performs a process of writing the video data received from the video acquisition unit 101 into the frame buffer 203 together with the frame number acquired from the input frame counter 202.

  The input frame counter 202 counts the number of frames of video data input into the frame rate adjustment unit 102, and the count value is incremented by one each time video data is input. The input frame counter 202 holds the count value L as a frame number. The initial value of the count value L is 1, and when requested by the video input unit 201, the counter value L held internally is passed to the video input unit 201.

  The frame buffer 203 is a memory having a video data area that stores one video frame and an area that stores an input frame number. FIG. 3 schematically shows the memory area of the frame buffer 203. In this example, the frame buffer 203 has a memory area including a frame number area 501, a video data area 502, and a synchronization flag 503.

  The video input unit 201 overwrites video data input in frame units in the video data area 502 and overwrites the frame number in the frame number area 501. The frame buffer 203 holds the same data until overwritten. The synchronization flag 503 is used to exclusively control access to the frame number area 501 and the video data area 502. By enabling the synchronization flag 503 when the video input unit 201 operates the frame buffer 203, it is possible to make the operation for the frame buffer 203 of the data acquisition unit 204 stand by. On the other hand, when the data acquisition unit 204 operates the frame buffer 203, the synchronization flag 503 is validated, so that the operation of the video input unit 201 on the frame buffer 203 can be put on standby.

  The data acquisition unit 204 performs processing for copying video data from the video data area 502 and copying the frame number from the frame number area 501. Data acquisition by the data acquisition unit 204 is periodically instructed by the timer 205, and is performed at a timing independent of the timing of writing to the frame buffer 203 (video input timing). The data acquisition unit 204 activates and locks the synchronization flag 503 in order to perform exclusive control of the buffer before performing video data read processing from the frame buffer 203. Thereafter, the data acquisition unit 204 acquires a frame number and video data from the frame number area 501 and the video data area 502. After the acquisition of the frame number and video data is completed, the data acquisition unit 204 invalidates the synchronization flag 503 and releases the lock.

  When the frame rate adjustment unit 102 is implemented as software, the data acquisition unit 204 is operated as a process or thread different from the video acquisition unit 101, and the synchronization flag 503 is realized as a synchronization object such as a mutex or a semaphore.

  The timer 205 is an interval timer that determines the timing for performing the data acquisition process in the data acquisition unit 204. The timer interval is determined by the frame rate specified by the user.

  The output frame counter 206 holds the frame number of the video data acquired from the frame buffer 203 by the data acquisition unit 204 until the next reading by the data acquisition unit 204.

  For the video data acquired by the data acquisition unit 204, the frame discard duplication determination unit 207 includes the frame number M acquired by the data acquisition unit 204 and the frame number held by the output frame counter 206 (the frame number previously acquired by the data acquisition unit 204). ) And a process for determining whether or not the frame is discarded or duplicated based on the difference. When the difference between the previously extracted frame number and the currently acquired frame number is 1, the frame discard duplication determination unit 207 determines that the frame has been acquired normally. If the difference between the previously extracted frame number and the currently acquired frame number is not 1, the frame discard / duplication determination unit 207 determines that the frame is discarded or duplicated. Specifically, if the difference between the previously extracted frame number and the currently acquired frame number is 0, it is determined that the frame has been duplicated. When the difference between the previously extracted frame number and the currently acquired frame number is 2 or more, it is determined that the frame has been discarded.

  When the frame discard / duplication determination unit 207 detects that the frame is discarded or duplicated, the event notification unit 208 performs a process of notifying the user of that fact. As an event notification means, message transmission, packet transmission, callback function call, or the like is used to display an error message or play a warning sound to the user.

  The video output unit 209 supplies the video data supplied from the data acquisition unit 204 to the video encoding unit 103.

  Next, the operation of the video encoding system will be described.

(Video frame input processing)
FIG. 4 is a flowchart showing video frame input processing performed by the frame rate adjustment unit 102. Hereinafter, the video frame input process will be described in detail with reference to FIG. 1, FIG. 2, and FIG.

  First, the count value L of the input frame counter 202 is initialized to 1 by a distribution start instruction (step S301). Thereafter, the video input unit 201 stands by until video data is input from the video acquisition unit 101 (step S302).

  When the video data is supplied to the video input unit 201, the video input unit 201 checks whether the synchronization flag 503 of the frame buffer 203 is activated by the data acquisition unit 204 (step S303). When the synchronization flag 503 is enabled, the video input unit 201 waits until the synchronization flag 503 is disabled because the frame buffer 203 is locked by the data acquisition unit 204.

  After confirming that the synchronization flag 503 of the frame buffer 203 has been invalidated, the video input unit 201 enables the synchronization flag 503 to lock the frame buffer 203 prior to accessing the frame buffer 203 associated with video input processing. (Step S304). As a result, the frame buffer 203 is locked by the video input unit 201.

  After locking the frame buffer 203, the video input unit 201 overwrites the frame number area of the frame buffer 203 with the counter value L acquired from the input frame counter 202, and also supplies the supplied video data to the video data area of the frame buffer 203. (Steps S305 and S306).

  Subsequently, the video input unit 201 invalidates the synchronization flag 503 of the frame buffer 203 that has been locked for the video input process (step S307). As a result, the frame buffer 203 is unlocked.

  After the frame buffer 203 is unlocked, the video input unit 201 adds 1 to the count value L of the input frame counter 202 (step S308). Then, the video input unit 201 determines whether or not a distribution end instruction has been made (step S309). If there is no distribution end instruction, the process returns to step S302 and waits until the next video data input.

(Video frame output processing)
FIG. 5 is a flowchart showing a video frame output process performed by the frame rate adjustment unit 102. Hereinafter, the video frame output process will be described in detail with reference to FIGS. 1, 2, and 5.

  In response to the distribution start instruction, the count value (frame number N) of the output frame counter 206 is initialized to 0 (step S401). The timer 205 issues an event at an interval set by the request frame rate from the user. The data acquisition unit 204 waits until an event is issued from the timer 205 (step S402).

  When the event is issued from the timer 205 and the standby is released, the data acquisition unit 204 checks whether the synchronization flag 503 of the frame buffer 203 is enabled by the video input unit 201 (step S403). If the synchronization flag 503 is enabled, the data acquisition unit 204 waits until the synchronization flag 503 is disabled because the frame buffer 203 is locked by the video input unit 201.

  After confirming that the synchronization flag 503 of the frame buffer 203 has been invalidated, the data acquisition unit 204 enables the synchronization flag 503 to lock the frame buffer 203 prior to access associated with the video frame output processing ( Step 404). As a result, the frame buffer 203 is locked by the data acquisition unit 204.

  After locking the frame buffer 203, the data acquisition unit 204 reads the frame number from the frame number area of the frame buffer 203 (step S405), and reads the video data from the video data area of the frame buffer 203 (step S406). In the following description, the frame number read from this frame number area is referred to as frame number M.

  Subsequently, the data acquisition unit 204 invalidates the synchronization flag 503 to unlock the frame buffer 203 that has been locked for the video frame output process (step S407).

  After unlocking the frame buffer 203, the data acquisition unit 204 determines whether the acquisition of the video data from the frame buffer 203 is the first video acquisition (step S408). In the case of the first video acquisition, the data acquisition unit 204 substitutes M−1 for the count value (frame number N) of the output frame counter 206 (step S409). Thereafter, the process proceeds to step S410. If it is not the first video acquisition, the process proceeds to step S410 without performing the process of step S409.

  In step S410, the frame discard duplication determination unit 207 determines the frame number M acquired by the data acquisition unit 204 (the frame number acquired this time) and the frame number N held in the output frame counter 206 (the frame number acquired last time or the first time The difference X of the time frame number) is calculated (step S410).

  When the frame number difference X is 0, the frame discard / duplication determination unit 207 determines that the video data stored in the frame buffer 203 has been duplicated (frame duplication has been performed) (steps S411 and S412). . Frame duplication means that the data acquisition unit 204 acquires the video data again before the video input unit 201 updates the video data on the frame buffer 203. As a result, the timing difference between the frame input and output is adjusted by one frame duplication. This frame duplication is executed when the frame rate output from the video acquisition unit 101 is smaller than the frame rate specified by the user.

  When the frame number difference X is 2 or more, the frame discard duplication determination unit 207 determines that the video data stored in the frame buffer 203 has been discarded (frame discard has been performed) (step S411, S413). The frame discard means that the video input unit 201 updates the video data on the frame buffer 203 before the data acquisition unit 204 acquires the video data from the frame buffer 203. As a result, the frame input / output timing difference is adjusted by performing frame discard. When the frame rate output from the video acquisition unit 101 is larger than the frame rate specified by the user, this frame discarding is executed. Note that the number of discarded frames can be obtained by X-1. The number of discarded frames can be notified to the user by event notification.

  When the frame number difference X is 1, the frame discard / duplication determination unit 207 determines that the video data stored in the frame buffer 203 has been output as it is without being discarded or duplicated (steps S411 and S414). . This state corresponds to a case where the frame rate output from the video acquisition unit 101 is the same as the frame rate requested by the user.

  If it is determined in step S412 that the frame has been duplicated or if it is determined in step S413 that the frame has been discarded, the event notification unit 208 notifies the user of the event (step S415). Event notification methods include alarm sound, message sound reproduction, and message display on a monitor. By this event notification, the user recognizes that the camera does not operate at the requested frame rate, and the video distribution is being performed by discarding or duplicating.

  The video data acquired by the data acquisition unit 204 is passed to the video encoding unit 103 by the video output unit 209 (step S416). Also, the data acquisition unit 204 updates the count value (output frame frame number N) of the output frame counter 206 with the acquired frame number M (step S417).

  After updating the count value, the data acquisition unit 204 determines whether or not a distribution end instruction has been issued (step S418). If there is no distribution end instruction, the process returns to step S402 and waits until the timer 205 generates the next event.

  According to the video encoding system of the present embodiment, the frame rate adjusting unit 102 executes video data writing and reading in the frame buffer 203 at independent timings. For example, the frame rate of the input video data is designated. If it is lower than the frame rate, the frame is duplicated. Conversely, if the frame rate of the input video data is higher than the specified frame rate, the frame is discarded. As a result, the frame rate of the input video data is converted to the designated frame rate. As described above, when video data is input from the camera device at a frame rate different from the frame rate required by the video encoding unit 103 (specified frame rate), the frame rate adjustment unit 102 determines the frame rate of the input video data. Therefore, the video encoding unit 103 is always supplied with video data at the specified frame rate. Therefore, it is not necessary to change the setting parameters of the video encoding unit 103, and there is no problem that the encoding process is stopped. In addition, it is possible to suppress the occurrence of problems such as output of an excessive bit stream exceeding the capacity of the transmission path and image quality degradation due to packet loss.

  In addition, when using the video encoding system of the present embodiment, the range of camera device selection can be expanded.

  Furthermore, when video data is input from the camera device at a frame rate different from the frame rate requested by the video encoding unit 103 (specified frame rate), and the frame rate adjusting unit 103 discards or duplicates the frame, The frame discard / duplication determination unit 207 detects the discard of the frame or the execution of the duplication, and the event notification unit 208 notifies the event information indicating the fact to the outside. By notification of this event information, the user can grasp the abnormality of the frame rate of the input video data.

  In general, MPEG-4 and H.264. In a system using H.264 or the like, the video encoding unit needs to be reset when switching video encoding parameters including a frame rate. This reset process temporarily stops video distribution. On the video decoding system side, after resuming distribution, video decoding cannot be started until an I picture or IDR picture is received. For this reason, usually, when the camera is switched, a section in which video reproduction is interrupted occurs. Here, a video encoding system capable of solving the problems associated with such video distribution stop will be described.

  FIG. 6 is a block diagram showing the main part of a video encoding system according to the second embodiment of the present invention.

  Referring to FIG. 6, a video encoding system 600 is a video encoding system that operates on a computer system to which a camera device is connected, and includes a plurality of video acquisition units 601 to 603 and a camera selection unit 604. , A frame rate adjusting unit 102, a video encoding unit 103, a packetizing unit 104, and a transmitting unit 105. The video encoding system 600 includes a camera selection function for selectively acquiring video data from a plurality of video acquisition units in addition to the function of the video encoding system 100 of the first embodiment. The video acquisition unit can be switched by an instruction.

  The video acquisition unit 601 has a capability of acquiring a moving image of 3 frames per second. The video acquisition unit 602 has a capability of acquiring a moving image of 17 frames per second. The video acquisition unit 603 has a capability of acquiring a moving image of 30 frames per second. Note that the number of image acquisition units and the frame rate are not limited to this, and can be changed as appropriate.

  Each of the video acquisition units 601, 602, and 603 is connected to the frame rate adjustment unit 102 via the camera selection unit 604. The camera selection unit 604 has a switching function of outputting the video frame output from the video acquisition unit selected by the user among the video acquisition units 601, 602, and 603 to the frame rate adjustment unit 102. The camera selection unit 604 discards the video frame of the video acquisition unit that has not been selected. The user can select a desired video acquisition unit from the video acquisition units 601, 602, and 603 through the input device of the computer system.

  The frame rate adjustment unit 102 performs a frame rate adjustment process on the video frame supplied from the camera selection unit 604. The user can arbitrarily set the frame rate value of the video data to be encoded by the video encoding unit 103. Here, the frame rate setting value is 15 frames per second. The frame rate value set by the user is supplied to the frame rate adjustment unit 102 and the video encoding unit 103.

  In the frame rate adjustment processing by the frame rate adjustment unit 102, the video frame is discarded or duplicated at a timing independent of the video frame supplied from the camera selection unit 604. This frame rate adjustment process is basically the same as the video frame input process (FIG. 4) and the video frame output process (FIG. 5) described in the first embodiment, but the video acquisition unit 601 is selected. If the video acquisition unit 602 or the video acquisition unit 603 is selected, the video frame is discarded. Thereby, the frame rate of the video data supplied from the camera selection unit 604 can be converted into a designated frame rate (15 frames per second).

  The video encoding unit 103 performs a process of compressing and encoding the video data converted to the frame rate specified by the frame rate adjusting unit 102 and converting it into a bit stream. As the compression encoding method, the above-mentioned MPEG-4 and H.264 are used. H.264 or the like is used.

  The video frame input to the video encoding unit 103 is adjusted to a frame rate of 15 frames per second regardless of the switching of the camera device, and thus does not need to be reset. Therefore, the video encoding unit 103 can normally perform the encoding process.

  The packetizing unit 104 divides the bit stream output from the video encoding unit 103 into an optimum size according to the transmission medium. Packetization processing is determined by the video encoding format and the type of transmission media. In the case of transmitting MPEG-4 format video, the RTP packet segmentation method according to IETF RFC3640 is used. In the case of transmitting H.264 format video, it is common to use IETF RFC3984.

  The transmission unit 105 has a function of transmitting the video stream divided into RTP packets to the video decoding system via the IP network. The RTP packet is transmitted using a UDP / IP protocol and a socket interface.

  According to the video encoding system 600 of this embodiment, since the frame rate adjustment unit 102 outputs video at a specified frame rate (for example, 15 frames per second) regardless of camera switching, the video encoding unit 103 Therefore, it is not necessary to change the video encoding parameter. Therefore, in the video encoding system 600, it is possible to continue encoding video at a designated frame rate (15 frames per second) regardless of camera switching. As a result, in the video decoding system, even when the camera is switched, it is possible to continuously reproduce the switched video without delay.

  FIG. 7 is a block diagram showing the main part of a video encoding system according to the third embodiment of the present invention.

  Referring to FIG. 7, a video encoding system 700 is a video encoding system that operates on a computer system to which a camera device is connected, and its main parts are a video acquisition unit 101, a frame rate adjustment unit 701, a video code. And a packetizing unit 104 and a transmitting unit 105. In addition to the functions of the video encoding system 100 of the first embodiment, the video encoding system 700 notifies not only the user but also the event notification when the frame rate is adjusted to the video decoding system side. A function for synthesizing the video data to be transmitted is provided.

  The video acquisition unit 101 is a functional block that acquires a moving image, and has a function of outputting video data in units of frames. The camera device used is a digital camera device or an analog video camera video capture card. When the start of video acquisition is instructed, the video acquisition unit 101 autonomously starts moving image acquisition in accordance with the video size (number of vertical pixels and horizontal pixels), video format, and frame rate specified in advance. The video data output from the video acquisition unit 101 is passed to the frame rate adjustment unit 701.

  The frame rate adjustment unit 701 adjusts the frame rate of the video data input from the video acquisition unit 101 to the specified frame rate, and sends an event notification indicating the discard or duplication of the frame at the time of the frame rate adjustment. It is a functional block having a function of notifying a user who uses the distribution system 700 and the video decoding system.

  The video encoding unit 103 is a functional block that receives video data subjected to frame rate adjustment from the frame rate adjustment unit 701, compresses and encodes the video data, and converts it into a bit stream. Examples of compression encoding methods include MPEG-4, H.264, and the like. H.264, Motion JPEG, etc. are used.

  The packetizing unit 104 has a function of dividing the bit stream output from the video encoding unit 103 into an optimal size for each transmission medium and adding a header. The packetization process is standardized according to the video encoding format and the type of transmission media. MPEG-4, H.264 using IP network. When transmitting video in the H.264, Motion JPEG format, RTP packet segmentation methods in the IETF RFC3640, 3984, and 2435 formats are used, respectively.

  The transmission unit 105 has a function of transmitting the video stream divided into RTP packets to the video decoding system via the IP network. The RTP packet is transmitted using a UDP / IP protocol and a socket.

  Next, a detailed internal configuration of the frame rate adjustment unit 701 will be described.

  FIG. 8 is a block diagram illustrating a configuration of the frame rate adjustment unit 701. Referring to FIG. 8, the frame rate adjustment unit 701 includes a video input unit 201, an input frame counter 202, a frame buffer 203, a data acquisition unit 204, a timer 205, an output frame counter 206, a frame discard / duplication determination unit 207, and an event notification unit. 208, a message image composition unit 801, and a video output unit 209.

  The operation of the frame rate adjustment unit 701 is the same as that of the first embodiment up to the process of determining whether the video frame is discarded or duplicated. Hereinafter, with respect to the details of the processing of the frame rate adjustment unit 701, the operation after the video frame discard / duplication determination processing will be described.

  The frame discard / duplication determination unit 207 calculates the difference between the frame number M acquired by the data acquisition unit 204 from the frame number area 501 of the frame buffer 203 and the frame number N held by the output frame counter 206, and discards / duplicates the frame. A process for determining whether or not it has been performed is performed.

  If the difference between the previously extracted frame number and the currently acquired frame number is 1, it indicates that the frame has been acquired normally. When both frame differences are 0, it indicates that frame duplication is being performed. If the difference between the two frames is 2 or more, it indicates that the frame is being discarded.

  When the frame discard / duplication determination unit 207 determines that the frame is discarded or duplicated, the event notification unit 208 performs a process of notifying the user of the fact. Further, the event notification unit 208 supplies the determination result (normal, discard, duplication) in the frame discard / duplication determination unit 207 to the message image synthesis unit 801. Event notification means is realized by sending a packet to the network, calling a callback function, or the like. By sending a packet to the network, an event can be notified to an external terminal. The system that has received the event notification notifies the user that the frame has been discarded and duplicated by displaying an error message or playing a warning sound.

  The video data acquired by the data acquisition unit 204 from the frame number area 502 is passed to the message image composition unit 801. At the same time, the event notification unit 208 notifies the message image composition unit 801 of the determination result (normal, discarded, or duplicated) regarding the video data.

  If the video data supplied from the data acquisition unit 204 is normally acquired video data, the message image synthesis unit 801 supplies the video data to the video output unit 209 as it is. When the video data supplied from the data acquisition unit 204 is video data that has been discarded or duplicated, the message image composition unit 801 adds information indicating that the video data has been discarded or duplicated. The message-attached video data is supplied to the video output unit 209.

  The video output unit 209 supplies the video data supplied from the message image synthesis unit 801 or video data with a message to the video encoding unit 103.

  Next, the configuration of the message image composition unit 801 will be specifically described.

  FIG. 9 is a block diagram showing the configuration of the message composition unit 801. Referring to FIG. 9, the message image synthesis unit 801 includes a video input unit 901, a video output unit 902, a video frame buffer 903, and an image synthesis unit 905.

  The video data output from the data acquisition unit 204 is supplied to the input of the video input unit 901. The video input unit 901 supplies the video data output from the data acquisition unit 204 to the image composition unit 905. An event notification 904 from the event notification unit 208 is supplied to the image composition unit 905. An event notification 904 indicates a determination result (normal, discarded, duplicated) regarding the video data acquired by the data acquisition unit 204.

  When the event notification 904 indicating that the video data supplied from the video input unit 901 is normally acquired is supplied from the event notification unit 208 to the image synthesis unit 905, the image synthesis unit 905 Without performing the synthesis process, the video data supplied from the video input unit 901 is output to the video output unit 209 as it is.

  When the event notification 904 indicating that the video data supplied from the video input unit 901 is video data after the video frame is discarded or duplicated is supplied from the event notification unit 208 to the image composition unit 905 The image composition unit 905 generates an event graphic 906 indicating an abnormal frame rate on the frame buffer 903 and synthesizes the event graphic 906 and video data supplied from the video input unit 901. Then, the image synthesis unit 905 supplies the synthesized video data to the video output unit 902.

  The event graphic 906 is composed of, for example, a character string, a graphic, and an image. Note that the event graphic 906 can be stored in advance on the frame buffer 903 or can be sequentially generated according to the event content. The event graphic 906 is not necessarily combined with the input video data, but may be transmitted by another protocol and displayed on the video decoding system side.

  According to the video encoding system 700 of the present embodiment, when the frame rate adjustment is performed by the frame rate adjustment unit 701, only a user who uses the video encoding system 700 performs image synthesis on the input video. In addition, it is possible to notify the user who views the video with the video decoding system of the execution of the frame rate adjustment.

  In the video encoding system of each embodiment described above, the operation in each configuration (functional block) can be realized by the program, and thus the computer executing the operation corresponding to the functional block. The program may be installed in a computer system in advance or may be stored and supplied in a recording medium represented by a CD-ROM or DVD. The program may be provided through a network represented by the Internet.

  The present invention can be applied to all systems that acquire and encode video frame data from a camera device such as a video recording system and a video surveillance system. It can also be used for video input means other than camera devices.

It is a block diagram which shows the principal part of the video coding system which is the 1st Example of this invention. It is a block diagram which shows the specific structure of the frame rate adjustment part shown in FIG. FIG. 3 is a schematic diagram showing a memory area of the frame buffer shown in FIG. 2. It is a flowchart which shows the video frame input process performed in the frame rate adjustment part shown in FIG. It is a flowchart which shows the video frame output process performed in the frame rate adjustment part shown in FIG. It is a block diagram which shows the principal part of the video coding system which is the 2nd Example of this invention. It is a block diagram which shows the principal part of the video coding system which is the 3rd Example of this invention. It is a block diagram which shows the structure of the frame rate adjustment part shown in FIG. It is a block diagram which shows the structure of the message synthetic | combination part shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Video encoding system 101 Video acquisition part 102 Frame rate adjustment part 103 Video encoding part 104 Packetization part 105 Transmission part 201 Video input part 202 Input frame counter 203 Frame buffer 204 Data acquisition part 205 Timer 206 Output frame counter 207 Frame discard Copy determination unit 208 Event notification unit 209 Video output unit

Claims (10)

A frame rate adjustment unit that converts the frame rate of the input video data to a specified frame rate;
A video encoding unit that encodes the video data that has been converted to the designated frame rate by the frame rate adjustment unit;
The frame rate adjustment unit includes a frame buffer in which the input video data is stored in units of frames, and the timing determined by the designated frame rate is independent of the timing determined by the frame rate of the input video data. A video encoding system for reading video data stored in a frame buffer.   The video encoding system according to claim 1, further comprising a video acquisition unit that acquires video data from an external camera device and supplies the acquired video data to the frame rate adjustment unit as the input video data. A plurality of video acquisition units for acquiring video data from an external camera device;
The apparatus further comprises: a selection unit that selects any one of the plurality of video acquisition units and supplies the video data acquired by the selected video acquisition unit to the frame rate adjustment unit as the input video data. The video encoding system described. The frame buffer is
A video data area in which video data for one frame is stored;
A frame number area in which a frame number of video data stored in the video data area is stored;
The frame rate adjustment unit
An input frame counter whose count value is incremented by one each time the input video data is supplied to the frame rate adjustment unit;
Writing the input video data in the video data area at a timing determined by the frame rate of the video data, and writing a count value of the input frame counter in the frame number area as a frame number of the video data;
A data acquisition unit that reads out the video data written in the video data area and the frame number written in the frame number area from the frame buffer at a timing determined by the designated frame rate;
When the data acquisition unit reads the video data stored in the video data area and the frame number stored in the frame number area from the frame buffer, the read frame number is read by the data acquisition unit next time. An output frame counter to hold until done,
For the video data read by the data acquisition unit, the frame number of the video data read by the data acquisition unit this time and the frame number of the video data read by the data acquisition unit previously held in the output frame counter A frame discard / duplication determination unit that determines whether or not a frame is discarded or duplicated based on the difference between
An event notification unit for notifying event information indicating the fact to the outside when the frame discard / duplication determination unit determines that the data is after the frame has been discarded or copied. Item 4. The video encoding system according to any one of Items 1 to 3. The frame rate adjustment unit further includes a timer for determining the read timing of video data from the frame buffer based on the designated frame rate,
5. The video according to claim 4, wherein the data acquisition unit reads the video data written in the video data area and the frame number written in the frame number area from the frame buffer at a read timing determined by the timer. Encoding system. The frame rate adjustment unit combines the video data and the image data indicating the event information with respect to the video data notified of the event information by the event notification unit among the video data read by the data acquisition unit. It further has a synthesis unit,
The video encoding system according to claim 4, wherein the video encoding unit encodes the video data combined by the image combining unit. A frame rate adjustment step for converting the frame rate of the input video data to a specified frame rate;
A video encoding step that encodes the video data that has been converted to the designated frame rate in the frame rate adjustment step;
The frame rate adjustment step includes:
Storing the input video data in a frame buffer in units of frames;
And a step of reading the video data stored in the frame buffer at a timing determined by the designated frame rate independent of a timing determined by the frame rate of the input video data. The frame rate adjustment step includes:
Incrementing the count value of the input frame counter by one each time the input video data is supplied;
The input video data is written into the video data area of the frame buffer at a timing determined by the frame rate of the video data, and the count value of the input frame counter is stored in the frame number area of the frame buffer as the frame number of the video data. Writing step;
Reading the video data written in the video data area and the frame number written in the frame number area from the frame buffer at a timing determined by the designated frame rate;
When the video data stored in the video data area and the frame number stored in the frame number area are read from the frame buffer, the read frame number is held in an output frame counter until the next reading is performed. When,
For the video data read from the video data area, the frame is discarded based on the difference between the frame number of the video data acquired this time and the frame number of the video data acquired last time held in the output frame counter. Or determining whether duplication has been performed;
The video encoding method according to claim 7, further comprising a step of notifying the outside of event information indicating that when it is determined that the frame is discarded or duplicated. A frame rate adjustment process for converting the frame rate of the input video data to a specified frame rate;
A program for causing a computer to execute video encoding processing for encoding video data that has been converted to the specified frame rate in the frame rate adjustment processing,
In the frame rate adjustment process,
Storing the input video data in a frame buffer in units of frames;
A program for causing the computer to execute a process of reading video data stored in the frame buffer at a timing determined by the designated frame rate independent of a timing determined by a frame rate of the input video data. In the frame rate adjustment process,
A process of incrementing the count value of the input frame counter by one each time the input video data is written to the video data area;
The input video data is written into the video data area of the frame buffer at a timing determined by the frame rate of the video data, and the count value of the input frame counter is stored in the frame number area of the frame buffer as the frame number of the video data. Writing process,
A process of reading the video data written in the video data area and the frame number written in the frame number area from the frame buffer at a timing determined by the designated frame rate;
When the video data stored in the video data area and the frame number stored in the frame number area are read from the frame buffer, the read frame number is held in the output frame counter until the next reading is performed. When,
For the video data read from the video data area, based on the difference between the frame number of the video data acquired this time and the frame number of the video data acquired last time held in the output frame counter, A process for determining whether or not duplication has been performed;
The program according to claim 9, wherein when it is determined that the frame is discarded or duplicated, the computer further executes a process of notifying event information indicating the fact to the outside.

Images