TWI430663B - Multi-stream video system, video monitoring device and multi-stream video transmission method - Google Patents

Description

Multi-stream video system, video monitoring device and multi-stream video transmission method

The present disclosure relates to video surveillance, and more particularly to a video surveillance system and method having multiple streams.

The safety of people and property has always been an important issue in society. Recently, various security surveillance systems have become popular. Among them, due to the advantages of easy installation and clear picture quality, video capture devices transmitted over the network have become the mainstream of security surveillance systems, such as IP Camera. With the development of electronic technology, providing higher resolution has become a trend of IP Camera manufacturers. Generally, the video capture device cooperates with the corresponding video monitoring device to form a monitoring system, such as a network video recorder (NVR). Currently, in a group of NVR systems, a number of IP cameras with Megapixel image resolution can be installed.

In general, the system side of the NVR (such as a video surveillance device) performs various application operations or processing on data transmitted from a video capture device (such as an IP camera), for example, storing image information, previewing an instant picture according to user requirements, Playback of video data or instant image discrimination (such as motion detection, face detection) on the video surveillance device. In the case of real-time image discrimination, the stream of the video compression format (MPEG4 or H.264) transmitted by the video capture device must be further decoded to obtain the RAW data.

These actions add to the load of the video decoder (decoder), such as the decoder chip, the processor chip, or the processor. In particular, when the video resolution transmitted by the video capture device is higher, the video quality is higher, and the video compression degree is larger, the computing load of the decoding unit of the video monitoring device is relatively increased.

A common problem is that the performance of the video decoding unit is insufficient or the load is too high, resulting in a decrease in the quality of the actual received image data, data delay, or slow response of the system operation interface. The more serious situation is that not only the receiving data may be missing or incomplete, but also the operation performance of the user interface is greatly affected, such as a pause or a crash situation.

In order to solve the above problems, the present invention provides a multi-stream video surveillance mechanism, in which each group of video capture devices is used to simultaneously generate multiple video streams with different image specifications, and the video monitoring device can be applied according to the current application. The operation dynamically selects the desired stream for optimal monitoring quality and execution efficiency.

Therefore, one aspect of the present invention provides a multi-stream video system including at least one video capture device and a video surveillance device. Each of the video capture devices is configured to generate a plurality of video streams at the same time, and the video streams respectively have different image specifications. The video monitoring device forms a communication connection with each video capturing device. When the video monitoring device performs an application operation, the video monitoring device selects and receives at least one video stream from each video capturing device according to the image operation of the application operation.

According to an embodiment of the present invention, the image specifications that are different between the video streams are selected from at least one image specification selected from the group consisting of image resolution, image quality, and image frame rate.

According to another embodiment of the present disclosure, the application operation is selected from at least one of a group operation consisting of a multi-segment display, a partial enlargement display, a video replay, a motion detection, a video recording, and a remote transmission. When the video monitoring device performs the remote transmission, the video monitoring device further forms a communication connection with a terminal device, and the video monitoring device further selects at least one video stream from the equal video stream according to the image requirement of the terminal device, and forwards the image stream to the terminal. Device.

According to still another embodiment of the present invention, the video monitoring device selects and receives at least two video streams from the video streams of each video capture device, and includes one of the high-standard first video streams and And a second image stream of the low specification, the video monitoring device respectively performing a first application operation on the first image stream and performing a second application operation on the second image stream.

Another aspect of the present invention provides a video monitoring device including a communication module, a processing module, and a determining unit. The communication module is configured to form a communication connection with at least one video capture device, wherein each of the video capture devices is configured to generate a plurality of video streams at the same time, and the video streams respectively have different image specifications. The processing module is configured to perform an application operation. The determining unit selects and receives at least one video stream from the video stream of each video capturing device according to the image demand control communication function of the application operation.

According to an embodiment of the invention, the video monitoring device further includes a decoding unit for decoding the video stream, wherein the image requirement is related to a decoding processing capability or a decoding load state of the decoding unit.

According to another embodiment of the present disclosure, a video monitoring device further includes a display module for displaying a video stream, wherein the image requirement is related to a display size or a display resolution of the display module.

According to still another embodiment of the present invention, the video monitoring device further includes a storage module for storing the video stream, wherein the image requirement is related to a storage capacity size or a writing rate of the storage module.

According to still another embodiment of the present invention, the image specifications that are different between the video streams are selected from at least one image specification selected from the group consisting of image resolution, image quality, and image frame rate.

According to still another embodiment of the present disclosure, the application operation is selected from the group consisting of: a multi-segment display, a partial enlargement display, a video replay, a motion detection, a video recording, and a remote transmission. When the video monitoring device performs the remote transmission, the video monitoring device forms a communication connection with a terminal device through the communication module, and the determining unit further selects at least one video stream from the image streams according to the image requirements of the terminal device. And transferred to the terminal device.

According to still another embodiment of the present invention, the video monitoring device selects and receives at least two video streams from the video streams of each video capture device, and includes one of the high-standard first video streams and And a second image stream of the low specification, the video monitoring device respectively performing a first application operation on the first image stream and performing a second application operation on the second image stream.

Another aspect of the present invention provides a multi-stream video transmission method, including the steps of: establishing a communication connection between a video monitoring device and at least one video capturing device; each video capturing device is configured to generate simultaneously a plurality of image streams each having a different image specification; the video monitoring device analyzing the image specifications of the image streams; the video monitoring device performing an application operation; and, according to the application Operating the image request, the video surveillance device selects and receives at least one video stream from the video streams of each of the video capture devices.

According to an embodiment of the present invention, the multi-stream video transmission method further includes the following steps: the video monitoring device decodes the video stream, wherein the image request and one of the video monitoring devices decode processing capability or decode The load status is related.

According to another embodiment of the present invention, the multi-stream video transmission method further includes the steps of: displaying the video stream on the video monitoring device, wherein the image requirement and one of the video monitoring devices display a size or a Display resolution is related.

According to still another embodiment of the present invention, the multi-stream video transmission method further includes the steps of: storing the video stream in the video monitoring device, wherein the image demand and the video monitoring device are one of a storage capacity or a The write rate is related.

According to still another embodiment of the present disclosure, the image specification that is different between the image streams is selected from at least one of a group of image resolution, an image quality, and an image frame rate. specification.

According to still another embodiment of the present invention, the application operation is selected from the group consisting of: a multi-segment display, a partial zoom display, an image replay, a motion detection, an image recording, and a remote transmission. At least one of the application operations.

According to still another embodiment of the present invention, the video monitoring device receives the video stream from each of the video capturing devices, and further includes the following steps: the video monitoring device from each of the video capturing devices Selecting and receiving at least two types of video streams in the video stream, including one of the high-standard first video stream and one of the low-standard second video streams; performing the first video stream An application operation; and performing a second application operation on the second image stream.

The present invention utilizes a video capture device to provide a plurality of streams of different image specifications (image resolution, image quality, image compression ratio, or image frame rate), thereby being able to select an appropriate video surveillance device according to different application operation requirements. Video streaming to achieve the highest processing efficiency. For example, low-resolution streaming can be used for basic image analysis and fast preview, and high-resolution streaming is stored in the storage medium for later image replay, thereby reducing the monitoring server (video monitoring device). Operational load. In the present invention, the most suitable video stream can be selected by the judging unit according to the image demand of the application operation, instead of always selecting the video stream of the fixed image specification. The detailed multi-stream video system is as follows.

Referring to FIG. 1, a schematic diagram of a multi-stream video system 100 in accordance with an embodiment of the present invention is shown. As shown in FIG. 1 , the multi-stream video system 100 includes a plurality of video capture devices (in this example, a video capture device 120a, a video capture device 120b, and a video capture device 120c), and a video monitoring device 140. The monitoring device 140 can form a communication connection with the video capturing devices 120a-120c, for example, through a wired/wireless internet network, a wired/wireless area network, and a physical line direct connection. In this example, the video monitoring device 140 primarily forms a communication connection with the video capture devices 120a-120c via the Internet 200. In practical applications, the multi-stream video system 100 can have more sets of video capture devices, and the present invention is not limited to three groups.

In the early days, the traditional analog camera had various image output specifications such as 320*240 pixels (pixel) of QVGA specification, 352*288 pixels of CIF specification, and 640*480 pixels of VGA specification. However, the image specifications of new video capture devices (such as IP Cameras) have recently increased. For example, it may support 1280*960 pixels, 1600*1200, 2048*1536 pixels, 2560*1920. Pixels have even higher resolution. That is to say, most of the video images transmitted by the video capture device are already having a resolution of several megapixels or more.

In general, the plurality of sets of video capture devices 120a-120c in the multi-stream video system 100 can be distributed in various corners, and the returned images are integrated and displayed on the video monitoring device 140 on the server side. The video monitoring device 140 can further analyze, replay, enlarge or store the returned video data.

It should be noted that in the multi-stream video system 100 of the present invention, each video capture device (120a-120c) can simultaneously generate multiple video streams with different image specifications.

For example, as shown in FIG. 1, the video capture device 120a generates three sets of video streams st1 to st3, wherein the image resolution of the video stream st1 is 1600*1200, and the video stream st2 is image-resolved. The image is 640*480 and the frame rate is 60 frames/sec (fps), the image resolution of the video stream st3 is 640*480 and the image frame rate is 24 fps; the video capture device 120b generates two groups. The video stream is st4~st5, the image resolution of the video stream st4 is 1280*1024, the image resolution of the video stream st5 is 720*480, and the video capturing device 120c generates two sets of video streams st6~st7, the video string. The image resolution of the stream st6 is 2560*1920, and the image resolution of the video stream st7 is 1280*1024.

In the above example, each video capture device (120a-120c) can simultaneously generate a different image size between video streams. The image resolution is mainly taken as an example. However, the present invention is not limited thereto. The image specification may be selected from at least one of image resolution, video quality, and image frame rate. In other words, the video capture device can generate the same 640*480 image resolution between different video streams, but have different image frame rates, such as 60 fps, 40 fps, and 24 fps, for example, the above example. The image resolutions of the video stream st2 and the video stream st3 are both 640*480, but the image frame rates are 60 fps and 24 fps, respectively.

On the other hand, some video streams can have different image quality. The actual image quality application here can be determined by the compression ratio, color depth, Peak Signal to Noise Ratio (PSNR) or other image parameters of the image.

Please refer to FIG. 2, which is a functional block diagram of a video monitoring device 140 according to an embodiment of the present invention. The video monitoring device 140 includes a communication module 142, a processing module 144, a determining unit 146, a decoding unit 147, a display module 148, and a storage module 149.

The video monitoring device 140 can be connected to the Internet 200 through the communication module 142 to form a communication connection with the video capturing devices 120a to 120c.

In general, when the video capture device 120 transmits image information (video streams st1 to st7) to the video monitoring device 140 through the network, most of the original data (RAW data) is first encoded, such as content encryption or For data compression or the like, the decoding unit 147 is configured to decode the video stream (st1 to st7). Generally, higher resolution, higher quality, or more complex encoded streams require longer decoding time and require better decoding processing power and result in a heavier decoding load. The decoding unit 147 has an operation decoding time that is substantially proportional to the total pixel size of the image. Most of the time, however, the video capture device 120 only needs to quickly view the image. In this example, the decoding unit 147 can be an independent decoder chip, but the invention is not limited thereto. In an actual application, the decoding unit 147 can also be integrated into the same processor with the processing module 144, or implement the decoding unit 147 in a software manner and executed by the processing module 144.

The processing module 144 can be a main computing and data processing center. In practice, the processing module 144 can be a processor, a processing chip, or a system on a chip (SoC). The processing module 144 can be used according to the usage. The instructions of the person perform various application operations.

For example, when the processing module 144 of the video monitoring device 140 can perform multi-segment display (1*2, 2*2, 4*4, etc. split screen) according to user instructions, partial zoom display (for a specified area zoom or for Application operations such as face enlargement, image playback, motion detection, recording, and remote transmission.

The processing module 144 in the video monitoring device 140 can process the video stream and transmit it to other corresponding modules or components according to different application operations. For example, the display module 148 is used to display video streams (single display, split display, partial zoom display, etc.), and the storage module 149 is used to store video streams for image recording or future image replay. Use.

At present, the resolution of the display module 148 on the video monitoring device 140 may be 1600*1200. Therefore, when the video monitoring device 140 is displayed in multiple images, the image stream having a resolution greater than or equal to the resolution of the divided window may be received according to the resolution of the plurality of divided windows of the display module 148, that is, lower. The resolution is displayed on the display module 148 and a good effect can be seen. For example, when the 2*2 split window simultaneously displays the images returned by the four sets of video capture devices, the four sets of video capture devices that receive 800*600 resolution or more are required according to the resolution of 800*600 or more for each split window. The streaming can be performed without necessarily requiring the highest image resolution. For example, the display module 148 has five divided windows, and one of the divided windows has a resolution greater than the other four divided windows, and is also based on the divided windows. Resolution to receive image streams greater than or equal to the resolution of each split window

In addition, common image analysis algorithms, such as motion detection, do not require too high resolution. Therefore, usually high-resolution images have their advantages only when image playback, biometric recognition (face recognition or iris recognition), and local subtle features.

Therefore, the determining unit 146 of the present invention selects and receives at least one video stream from the plurality of video streams of each of the video capturing devices (120a to 120c) according to the image demand control communication module 142 of the application operation. In this example, the determining unit 146 can be an independent decoding chip, but the invention is not limited thereto. In the actual application, the determining unit 146 can also be integrated with the processing module 144 in the same processor, or implement the determining unit 146 in a software manner and executed by the processing module 144.

The determining unit 146 describes the following according to the criteria for determining the image demand:

(1) When the user only needs to preview the screens of the video capture devices 120a-120c in a rough manner, and the resolution of only 400*300 is required for each divided screen on the display module 148: the determination unit 146 selects and The video stream st3 (640*480, 24 fps) of the video capture device 120a, the video stream st5 (720*480) of the video capture device 120b, and the video stream st7 of the video capture device 120c (1280*1024) .

(2) When the user only needs to preview the screens of the video capture devices 120a-120c in a rough manner, and the resolution of the display module 148 is only 800*600 for each divided screen: the analysis of the video stream The degree should be at least greater than the resolution of the divided picture on the display module 148. At this time, the determining unit 146 can select and receive the video stream st1 (1600*1200) of the video capturing device 120a and the video stream st4 of the video capturing device 120b. (1280*1024) and the video stream st7 (1280*1024) of the video capture device 120c.

(3) If the screen of the video capture device 120a is to be partially enlarged: the determination unit 146 can switch to the video stream st1 (1600*1200) of the video capture device 120a.

(4) If only the image analysis for changing the content of the screen is performed, the determining unit 146 may select the video stream st2 (640*480, 60 fps) of the video capturing device 120a and the video stream st5 of the video capturing device 120b (720). *480) and the video stream st7 (1280*1024) of the video capture device 120c, wherein the video stream st2 has a higher image frame rate for dynamic detection.

The example of determining the image requirement by the determining unit 146 mainly considers the display of the display module 148 and the size or display resolution of the split window, but the invention is not limited thereto.

In other embodiments, the determining unit 146 determines that the image requirement of the application operation may further consider the decoding processing capability or the decoding load state of the decoding unit 147, or the storage capacity size or the writing rate of the storage module 148. In addition, the determining unit 146 does not limit selection of only a single video stream from the single video capturing device (120a, 120b or 120c), and may select more than two video streams at the same time. The following is an example to illustrate:

(5) It is assumed that only a brief preview of the screens of the video capture devices 120a-120c is required and the screen is saved for later image replay, and the user wants to have the best image when the image is replayed. As the anti-theft or malicious use: the judging unit 146 receives the two-phase video stream (st1 and st3) from the video capturing device 120a at the same time, and simultaneously receives the two-phase video stream (st4 and st5) from the video capturing device 120b and The video capture device 120c simultaneously receives the two-phase video stream (st6 and st7). The high-standard video stream (st1, st4, st6) is not decoded by the decoding unit 147, and the video monitoring device directly stores the high-standard video stream (st1, st4, st6) in the storage module 148. . The low-standard video stream (st3, st5, st7) is decoded by the decoding unit 147 and displayed on the display module 148. When image replay is required, the high-definition video stream (st1, st4, st6) of the specified time period is read out by the storage module 148 and decoded by the decoding unit 147. In this way, the degree of picture clarity when the image replay requirement is required can be balanced, and the burden on the decoding unit 147 can be reduced during the preview of the general picture, so that the smoothness of the display and operation of the video monitoring device 140 can be improved.

The video monitoring device 140 can perform the remote transmission function according to the user operation. When performing the remote transmission, the video monitoring device 140 can forward the received video stream to other terminal devices (such as a mobile phone, a PDA, a laptop, a tablet). The video monitoring device 140 can form a communication connection with another terminal device (not shown) through the communication module 142, and the determining unit 146 further according to the image requirements of the terminal device (screen) Size, processor capability, etc.) Select the appropriate video stream from the video stream and forward it to the terminal device. For the method of selecting the video stream, refer to the previous embodiment, and no further details are provided herein.

Please refer to FIG. 3, which is a flowchart of a method for a multi-stream video transmission method according to another embodiment of the present invention. The multi-stream video transmission method in this embodiment can be used with a multi-stream video system, and the multi-stream video system includes a video monitoring device and at least one video capturing device, and the hardware architecture of the multi-stream video system is For internal details, reference is made to the description of the relevant paragraphs of the previous embodiments of the present invention.

As shown in FIG. 3, the multi-stream video transmission method of the present invention first performs step S100 to establish a communication connection between a video monitoring device and at least one video capturing device. Then, in step S102, each video capture device is configured to generate a plurality of video streams at the same time, and the video streams respectively have different image specifications, and the image specifications may be selected from image resolution, image quality, and At least one of the image frame rates.

Next, in step S104, the video monitoring device analyzes the image specifications of the video streams. Then, step S106 is executed to perform a specific application operation by the video monitoring device according to the needs of the user, wherein the application operation is selected from the group consisting of multi-segment display, partial zoom display, image replay, motion detection, video recording, and remote transmission. An image stream.

Then, in step S108, the video monitoring device selects and receives at least one video stream from the video streams of each video capture device according to the image requirements of the application operation. The image requirement of the application operation may be related to at least one of various factors such as a decoding processing capability, a decoding load state, a display size, a display resolution, a storage capacity size, or a write rate of the video monitoring device.

According to still another embodiment of the present invention, the application operation is selected from the group consisting of: a multi-segment display, a partial zoom display, an image replay, a motion detection, an image recording, and a remote transmission. At least one of the application operations.

In addition, in step S108, in which the video monitoring device receives the video streams from each of the video capturing devices, the video monitoring device may select and receive two types of video streams from each of the video capturing devices, including high specifications. Video streaming and low-profile video streaming. Then, the video monitoring device can perform different application operations on the high and low specification video streams respectively.

In summary, the present invention utilizes a video capture device to provide a plurality of streams of different image specifications (image resolution, image quality, image compression ratio, or image frame rate), thereby corresponding to different application operation requirements, and video communication. The monitoring device selects the appropriate video stream to achieve the highest processing efficiency.

The present disclosure has been disclosed in the above embodiments, but it is not intended to limit the disclosure, and any person skilled in the art can make various changes and refinements without departing from the spirit and scope of the disclosure. The scope of protection of the disclosure is subject to the definition of the scope of the patent application.

100. . . Multi-stream video system

120a. . . Video capture device

120b. . . Video capture device

120c. . . Video capture device

140. . . Video monitoring device

200. . . Internet

St1. . . Video stream

St2. . . Video stream

St3. . . Video stream

St4. . . Video stream

St5. . . Video stream

St6. . . Video stream

St7. . . Video stream

142. . . Communication module

144. . . Processing module

146. . . Judging unit

147. . . Decoding unit

148. . . Display module

149. . . Storage module

S100. . . step

S102. . . step

S104. . . step

S106. . . step

S108. . . step

The above and other objects, features, advantages and embodiments of the present disclosure will become more apparent and understood.

1 is a schematic diagram of a multi-stream video system according to an embodiment of the invention;

2 is a functional block diagram of a video monitoring device according to an embodiment of the present invention;

FIG. 3 is a flow chart of a method for a multi-stream video transmission method according to another embodiment of the present invention.

100. . . Multi-stream video system

120a. . . Video capture device

120b. . . Video capture device

120c. . . Video capture device

140. . . Video monitoring device

200. . . Internet

St1. . . Video stream

St2. . . Video stream

St3. . . Video stream

St4. . . Video stream

St5. . . Video stream

St6. . . Video stream

St7. . . Video stream

Claims (15)

A multi-stream video system includes: at least one video capture device, each of the video capture devices simultaneously generating a plurality of video streams, the video streams having different image specifications; and a video The monitoring device forms a communication connection with each of the video capture devices, and when the video surveillance device performs an application operation, the video surveillance device requests the images from each of the video capture devices according to an image of the application operation At least one video stream is selected and received in the stream. The multi-stream video system of claim 1, wherein the application operation is selected from the group consisting of a multi-segment display, a partial zoom display, an image replay, a motion detection, an image recording, and a remote transmission. At least one of the application operations in one of the groups. The multi-stream video system of claim 2, wherein when the video monitoring device performs the remote transmission, the video monitoring device further forms a communication connection with a terminal device, the video monitoring device further according to the terminal One of the devices is required to select at least one of the video streams from the video streams and forward them to the terminal device. The multi-stream video system of claim 1, wherein the video monitoring device selects and receives at least two video streams from the video streams of each of the video capture devices, wherein the video stream comprises a high specification. a first image stream and a second image stream of a low specification, the video monitoring device respectively performing a first application operation on the first image stream and performing a second application operation on the second image stream . A video monitoring device includes: a communication module for forming a communication connection with at least one video capture device, wherein each of the video capture devices is configured to simultaneously generate a plurality of video streams, wherein the video streams respectively have a different image specification; and a processing module for performing an application operation; and a determining unit that controls the communication module from each of the video capturing devices according to an image demand of the application operation At least one video stream is selected and received in the video streams. The video surveillance device of claim 5, further comprising: a display module for displaying the video streams, wherein the image requirements and one of the display modules display a size or a display resolution Or the resolution of the plurality of split windows it has. The video surveillance device of claim 5, further comprising: a storage module for storing the video stream, wherein the image requirement and a storage capacity of the storage module or a write rate related. The video monitoring device of claim 5, wherein the application operation is selected from the group consisting of: a multi-segment display, a partial zoom display, an image replay, a motion detection, an image recording, and a remote transmission. At least one of the application operations in one of the groups. The video monitoring device of claim 8, wherein when the video monitoring device performs the remote transmission, the video monitoring device forms a communication connection with a terminal device through the communication module, and the determining unit further determines One of the image devices of the terminal device selects at least one video stream from the video streams and forwards the image stream to the terminal device. The video surveillance device of claim 5, wherein the video surveillance device selects and receives at least two of the video streams from the video streams of each of the video capture devices, including one of the high specifications. The first video stream and the second video stream of the low specification, the video monitoring device respectively performing a first application operation on the first video stream and performing a second application operation on the second video stream. A multi-stream video transmission method includes: establishing a communication connection between a video surveillance device and at least one video capture device; each of the video capture devices is configured to simultaneously generate a plurality of video streams, the video streams Each of the image monitoring devices has a different image specification; the video monitoring device analyzes the image specifications of the image streams; the video monitoring device performs an application operation; and the video monitoring device operates according to one of the application requirements At least one video stream is selected and received in the video streams of the video capture device. The multi-stream video transmission method of claim 11, further comprising the steps of: displaying the video stream on the video monitoring device, wherein the image requirement and one of the video monitoring devices display a size or A display resolution or its resolution of a plurality of split windows. The multi-stream video transmission method of claim 11, further comprising the steps of: storing the video stream in the video monitoring device, wherein the image demand and the video monitoring device are one of a storage capacity or A write rate is related. The multi-stream video transmission method according to claim 11, wherein the application operation is selected from the group consisting of a multi-segment display, a partial enlargement display, an image replay, a motion detection, an image recording, and a remote end. Transmitting at least one of the application operations in one of the groups. The multi-stream video transmission method of claim 11, wherein the video monitoring device receives the video stream from each of the video capturing devices, further comprising the following steps: the video monitoring device Selecting and receiving at least two types of video streams in each of the video streams of the video capture device, including one of the high-standard first video stream and one of the low-standard second video streams; An image stream performs a first application operation; and a second application operation is performed on the second image stream.