TW201806381A - Server applied to video surveillance system and associated video display method - Google Patents

Abstract

A server applied in a video surveillance system includes a device interface and a processor, wherein the device interface is arranged to receive surveillance images from a plurality of cameras, and the processor is arranged to generate a layout of the surveillance images, and to transmit the layout of the surveillance images to a display via the device interface to be simultaneously displayed thereon in a default status. In addition the processor is further arranged to adjust the layout of the surveillance images according to importance information of the surveillance images, to adjust display sizes of the surveillance images.

Description

Server applied to image monitoring system and related monitoring image display method

The invention relates to an image monitoring system, in particular to an image monitoring system capable of instantly adjusting the size of a monitor image.

Generally, the monitoring system receives the image screen transmitted by the supported monitor and displays all the received image screens in different areas of the display. For example, if the monitoring system supports 16 monitors, the monitoring system monitors The system's display screen is divided into 16 areas (4*4), and each area is used to view the live image picture transmitted by a monitor. However, if the monitoring system needs to support more monitors, the display screen will need to be divided into more areas, such as 64 areas (8*8), so that each monitor displays the image on the display. The picture will be greatly reduced, which will make it easy for users to ignore important information in the picture.

Accordingly, it is an object of the present invention to provide an image monitoring system that can instantly adjust the size of an image displayed on a display by a portion of a monitor based on importance information to solve the problems of the prior art.

An embodiment of the present invention discloses a server applied to an image monitoring system, which includes a device interface and a processor, wherein the device interface is configured to receive multiple monitor images from multiple monitors, and the processor is used by the processor. Layout a plurality of monitor images, and in a preset state, the plurality of monitor images after the layout are transmitted to the display through the device interface for simultaneous display; and the processor is further configured according to the plurality of monitor images The importance information adjusts the layout of the plurality of monitor images to adjust the size of the screen when at least one of the plurality of monitor images is displayed on the display.

Another embodiment of the present invention discloses a monitoring image display method, which includes: acquiring a complex array frame sequence from a plurality of input monitor image streams; and analyzing a degree of change of front and rear frames in each group of frame sequences. Generating a plurality of first importance parameters; generating a plurality of importance information according to the plurality of first importance parameters; and adjusting the plurality of monitor image streams to be displayed on the display according to the plurality of importance information The size of the picture.

Please refer to FIG. 1, which is a schematic diagram of an image monitoring system 100 in accordance with an embodiment of the present invention. As shown in FIG. 1, the image monitoring system 100 includes a server 110, a plurality of monitors 102_1 102102_M, and a display 104. The server 110 includes at least a processor 120, a decoder 130, and a device interface 140, and the processor The 120 includes an importance calculation module 122, a layout setting module 124, a play module 126, and a live view layout module 128. In this embodiment, the plurality of monitors 102_1~102_M may be disposed at different locations in the environment, such as in different rooms in the home, or any location in the building that needs to be monitored, and the server 110 may pass through The connection line or the network receives a plurality of images from the plurality of monitors 102_1~102_M, and lays out a plurality of images from the plurality of monitors 102_1~102_M in a preset state, that is, The plurality of images from the plurality of monitors 102_1 to 102_M are reduced in the same image frame/frame and transmitted to the display 104 for simultaneous display.

In one embodiment, the importance calculation module 122, the layout setting module 124, the play module 126, and the live view layout module 128 can communicate with each other or exchange data by one or more communication paths. In addition, in an embodiment, the importance calculation module 122, the layout setting module 124, the play module 126, and the live view layout module 128 can be implemented by software, wherein the software can be stored in a temporary or non-transitory computer. Readable on a storage medium and executed by one or more processing units. In other embodiments, the importance calculation module 122, the layout setting module 124, the play module 126, and the live view layout module 128 may be implemented by hardware, such as a circuit module in a processor, or software and hard. Realized by a combination of bodies. In another embodiment, the importance calculation module 122, the layout setting module 124, the playback module 126, and the live view layout module 128 can be implemented as separate applications integrated into another application (eg, web browsing). Program). In some embodiments, the importance calculation module 122, the layout setting module 124, the play module 126, and the live view layout module 128 can be implemented in an operating system, such as a network attached storage (NAS). Server operating system.

In the operation of the server 110, first, the decoder 130 receives the plurality of monitor image data streams from the plurality of monitors 102_1 102 102_M through the device interface 140, and decodes the plurality of monitor image data streams. A plurality of sets of frame sequences that are continuously transformed over time are obtained, and then the plurality of sets of frame sequences are transmitted to the importance calculation module 122 and the play module 126. The importance calculation module 122 calculates the importance information of each of the monitors 102_1~102_M according to one or more detection results. For example, the one or more detection results can be detected by the following detection operations. At least one of the following is generated: the processor 120 performs motion detection on multiple monitor images, or event information on the surrounding sound sources of the monitors 102_1~102_M, the destruction behavior of the monitors 102_1~102_M, the direction of the user's eyeball, and the like. Detection.

In an embodiment, the processor 120 can perform motion detection on multiple monitor images. For example, the processor 120 may determine a degree of change of the image content from each of the monitors 102_1 102 102_M to determine a first importance parameter, wherein a greater degree of change in the image content indicates that the monitored area may be invaded by a person. Therefore, the importance parameter is also higher. In some embodiments, the degree of image change can be calculated by comparing the degree of difference between the frames before and after, for example, first subtracting each pixel value in the front and back frames to obtain the amount of change of each pixel. Then, the amount of change of each pixel is added to obtain the difference degree of the front and back frames, and the larger the value, the greater the degree of change. In another embodiment, the degree of image change can be differentiated by the image to find the amount of change in the slope (eg, grayscale change), and the greater the amount of change, the greater the degree of change in the image content.

In some embodiments, the processor 120 can also detect multiple event information, such as the surrounding sound source of the monitors 102_1~102_M, the destruction behavior of the monitors 102_1~102_M, and the detection of the direction of the user's eyeball, but The invention is not limited to this. In the embodiment of detecting the surrounding sound sources of the monitors 102_1~102_M, each of the monitors 102_1~102_M may include a sound monitoring device, and the sound monitoring device continuously detects the sound around the monitor and detects the detected sound. The sound or the detected sound volume result is transmitted to the server 110, and the importance calculation module 122 in the processor 120 can determine a second according to the degree of change of the sound volume monitored by the sound monitoring device. The importance parameter, in which the greater the degree of change in the volume of the sound, means that the monitored area may have human intrusion or other destructive behavior, so the importance parameter is also higher. In another embodiment, each of the monitors 102_1~102_M may include a sound collection device, the monitors 102_1~102_M may directly transmit a plurality of sound streams to the server 110, and the processor 120 may analyze the volume of the sound stream. The size, and importance calculation module 122 can determine the second importance parameter based on the volume level or the degree of change in the volume of the sound.

In the embodiment of detecting the destruction behavior of the monitors 102_1~102_M, each of the monitors 102_1~102_M may include, for example, a destruction detecting component, and the importance computing module 122 in the processor 120 may be based on the destruction detecting component. The detected damage behavior determines a third importance parameter, wherein if the damage behavior is detected, the value of the third importance parameter is higher, wherein the above destruction behavior may be a monitor out of focus, a monitor Obscured, or monitor displacement...etc. In addition, in the embodiment of detecting the direction of the user's eyeball, the server 110 and the display 104 may also be provided with an eyeball detecting device, which can detect the direction of the user's eyeball to determine that the user is watching the screen of the display 104. The area of the area and the fourth importance parameter of the monitor corresponding to the area viewed by the user. In one embodiment, the eyeball detecting device is disposed on the display 1004 in the monitoring center, and the closer the monitoring personnel are to the display screen, the higher the accuracy.

In an embodiment, the importance calculation module 122 transmits the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter of each of the monitors 102_1 102102_M to the layout. The module 124 is set, and the layout setting module 124 determines each of the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter of each of the monitors 102_1 102102_M. The importance information of a monitor image, for example, weighting and adding the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter to generate a value as the importance information, and The higher the value in this embodiment, the higher the importance of representing the monitor image.

In another embodiment, each of the detection results has a corresponding threshold value, and the threshold value can be adjusted by the user to determine whether to enlarge the image frame of the monitor. In detail, the layout setting module 124 compares the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter with their corresponding critical values, respectively, if there is a specific monitor When any of the importance parameters is greater than the threshold, the layout setting module 124 determines the size of the image to be enlarged when the image transmitted by the particular monitor is displayed on the display 104, and the image transmitted by the specific monitor The display module 126 and the live view layout module 128 and the device interface 140 are displayed on the display 104 in an enlarged manner.

Please refer to FIG. 2 to FIG. 5 , which is a schematic diagram of the server 110 adjusting and adjusting the layout of the plurality of images according to the importance of the plurality of images transmitted by each of the monitors 102_1 102 102_M. First, please refer to FIG. 2 first. In the default situation, assuming that the server 110 supports 64 monitors, the screen of the display 104 will be divided into 64 blocks (8*8), and each one. The block is used to display image data from a monitor. At this time, if the processor 120 determines that the importance of a specific image of a particular monitor exceeds a critical value, the processor 120 determines which direction the image is to be enlarged according to the location of the specific image, and in an embodiment. In particular, the particular image is magnified toward the center of the display screen, for example toward the center or center of the display screen. In detail, referring to FIG. 2, if a specific image whose importance exceeds the critical value is located in the first quadrant, the specific image will be magnified 4 times to the lower left corner, that is, the space covering 4 blocks; if the importance is exceeded If the specific image of the threshold is in the second quadrant, the specific image will be magnified 4 times in the lower right corner; if the specific image whose importance exceeds the critical value is in the third quadrant, the specific image will be magnified 4 times in the upper right corner; If the specific image whose importance exceeds the critical value is in the fourth quadrant, the specific image will be magnified 4 times in the upper left corner. It should be noted that the above image magnifications are for illustrative purposes only and are not intended to be limiting of the invention.

After determining the direction and area in which a particular image is to be enlarged, since the enlarged specific image will occupy the display area of other monitor images, some of the monitor image needs to be removed from the screen. In one embodiment, the processor 120 determines the three most important monitor images according to different directions of the zoomed area as images that need to be removed from the screen. In detail, as shown in FIG. 3, the enlarged area is first divided into four areas, and the processor 120 is in the upper right area and the lower right except for the area corresponding to the specific image to be enlarged (upper left area). The area and the lower left area respectively find a monitor image of the least importance. In an embodiment, the least important monitor image may be determined by the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter of each of the monitors 102_1 102102_M. For example, after weighting and adding the first importance parameter, the second importance parameter, the third importance parameter, and the fourth importance parameter, the monitor image having the smallest value is selected as the monitor image with the lowest importance.

Next, referring to FIG. 4, the processor 120 determines an adjustment path between the image in the enlarged area and the image with the lowest importance as the direction when the subsequent monitor image moves, wherein in one embodiment, the adjustment path may be It is the shortest path between the image in the enlarged area and the image with the lowest importance, and the shortest path means that the image in the enlarged area can reach the important position with the minimum number of steps to move (the display area of the monitor image at least) The least sensitive image, such as the one shown in Figure 4, is obtained by moving two straight lines to get the shortest path.

Finally, referring to Figure 5, the specific image is magnified 4 times, and the image on the shortest path shown in Figure 4 is also moved along the direction of the shortest path, wherein each image on the shortest path moves only one. The grid (one area) and the three most important monitor images are also removed from the screen.

Through the process shown in the above 2~5, the specific image whose importance exceeds the critical value is enlarged to allow the user to clearly see the content of the specific image, and the image with the lowest importance in the process of zooming in the specific image. It will be removed, and each monitor image that needs to be moved needs to be moved by one (one area), so the relative position of each monitor image can be maintained as much as possible.

If other monitor images are judged to have higher importance and need to be enlarged at the same time, the process can be performed according to the above steps 2 to 5, and FIG. 6 shows three monitor images. An example of being enlarged because of its high importance. In addition, in the layout of the image, it can be defined that only a certain number of monitor images can be enlarged, for example, only four enlarged specific images are allowed, and the order of image enlargement takes precedence with the higher the importance value.

In an embodiment, if the monitor image to be enlarged and the already enlarged monitor image overlap, it is necessary to move the enlarged monitor image first, and then perform the image enlargement operation, as shown in FIG.

In addition, the update of the above image layout can be triggered by two situations, the first is a fixed time update, for example, every 10 seconds, and the second is to instantly determine the importance information of each monitor image. To trigger. For example, please refer to FIG. 8. At about the third second, the server 110 enlarges one of the monitor images according to the importance information of the monitor image (the detailed steps are shown in FIGS. 2~5); Then, at the 10th second, the server 110 performs a fixed-time layout update, and at this time, the processor 120 determines the importance information of the monitor image again to re-adjust the screen size of the monitor image, in this embodiment. Since the importance of each monitor image is lower than the critical value at this time, the processor 120 returns the image layout to a preset state, and the previously replaced/removed monitor image is also re-displayed on the screen of the display 104. Displayed above; then, at approximately the 18th second, the server 110 amplifies the two monitor images based on the importance information of the monitor image, ..., and so on.

Please refer to FIG. 9 , which is a flowchart of a method for displaying a monitoring image according to an embodiment of the present invention. Referring to the above disclosure, the flow is as follows:

Step 900: The process begins.

Step 902: Receive multiple images from multiple monitors.

Step 904: Layout the plurality of images, and transmit the plurality of images after the layout to a display for display.

Step 906: Adjust a layout of the plurality of images according to importance information of the plurality of images to adjust a screen size when the plurality of images are displayed on the display.

In the above step 904, a plurality of images from a plurality of monitors can be simultaneously transmitted to the display for display in a preset state. In addition, in some embodiments, the foregoing step 906 may further include the following process: Step S1: Acquire a complex array frame sequence from the input plurality of monitor video streams. In step S2, the degree of change of the front and rear frames in each group of frame sequences is analyzed, and a plurality of first importance parameters are generated. Step S3, generating a plurality of importance information according to the plurality of first importance parameters. In step S4, according to the plurality of importance information, the screen size when the plurality of monitor video streams are displayed on the display is adjusted. It should be understood that the above steps S1 to S4 are merely examples, and the present invention is not limited thereto. In other embodiments, the foregoing step 904 may further receive a plurality of event information corresponding to the plurality of monitor image streams to generate corresponding importance parameters, and simultaneously according to the first importance parameter and other event information. The corresponding importance parameter generated generates a plurality of importance information. In an embodiment, the event information may be, for example, an event such as detecting a surrounding sound source of the monitor, detecting a destruction behavior of the monitor, and detecting a direction of the eye of the user.

Please refer to FIG. 10, which is a schematic diagram of an image monitoring system 1000 in accordance with an embodiment of the present invention. As shown in FIG. 10, the image monitoring system 1000 includes a server 1010, a plurality of monitors 1002_1~1002_M, and a display 1004. The server 1010 includes at least a processor 1020, a decoder 1030, and a device interface 1040, and the processor 1020 includes an importance calculation module 1022, a hierarchical layout module 1024, and a playback module 1026. In this embodiment, the plurality of monitors 1002_1~1002_M may be disposed at different locations in the environment, such as different rooms in the home, or any location in the building that needs to be monitored, and the server 1010 may pass through The connection line or the network receives a plurality of images from the plurality of monitors 1002_1 to 1002_M, and arranges a plurality of images from the plurality of monitors 1002_1 to 1002_M, and transmits them to the display 1004 for simultaneous display.

The device interface 1040, the decoder 1030, the importance calculation module 1022, and the play module 1026 shown in FIG. 10 are similar to the device interface 140, the decoder 130, the importance calculation module 122, and the device shown in FIG. The module 126 is played, so the relevant details are not described herein, and the following disclosure mainly describes the operation content of the hierarchical layout module 1024 of the image monitoring system 1000.

Please refer to FIG. 11. FIG. 11 is a schematic diagram of the hierarchical layout module 1024 and the display layout level input by the user according to the present invention. As shown in FIG. 11 , the hierarchical layout module 1024 includes a basic layout module 1102, a sequencing module 1104, a monitor image assignment module 1106, and an output circuit 1108, and the hierarchical layout module 1024 can be based on the user. The level of output determines that several monitor images are to be output to display 1004 for display. In this embodiment, the user can define three different levels. In the first level, the display 1004 can display 64 monitor images, and in the second level, the display 1004 can display 16 monitor images. The three-level display 1004 can only display four monitor images.

In the detailed operation of the hierarchical layout module 1024, first, the basic layout module 1102 generates a basic/preset layout information into the sequencing module 1104. In the embodiment, the basic layout module 1102 generates the basics. / The preset layout information may be layout information showing all supported monitor images, that is, when the server 1010 supports 64 monitors, the basic/preset layout information is an image showing 64 monitors. Then, the sorting module 1104 sorts at least a part of the monitor images according to the importance information of each monitor image provided by the importance computing module 1022, for example, according to the importance information of each monitor image. Each monitor image is labeled with a number that represents the importance level, for example, the lower the number represents the higher importance. Then, the monitor image assignment module 1106 determines which of the most important monitor images to be output as the output according to the level input by the user. For example, please refer to FIG. 12, assuming that the user The input is to display the second level of 16 monitor images, then the monitor image assignment module 1106 assigns the top 16 monitor images of importance as the output; please refer to Figure 13, assuming that the user inputs When the third level of the four monitor images is displayed, the monitor image assignment module 1106 assigns the top 4 monitor images of importance as the output. Finally, the output circuit 1108 transmits the monitor image assigned by the monitor image assignment module 1106 to the display 1040 for simultaneous display, and the monitor image assigned by the monitor image assignment module 1106 is not displayed on the screen.

In the embodiment shown in FIGS. 10-13, the server 1010 can determine the content of several monitor images to be output according to the level defined by the user, and the output monitor image is automatically based on the server 1010. The importance is automatically determined. Therefore, the method used in this embodiment can avoid the problem that the monitor image is too small.

Briefly summarized in the present invention, in the image monitoring system and the monitoring image display method of the present invention, it is possible to instantly adjust the size of the image displayed on the display by the partial monitor based on the importance information, thereby making the importance of the image high. The image picture is clearer for the user to view; further, in one embodiment, the image of the least important image is removed in response to the enlargement of part of the image, and each monitor image that needs to be moved only needs to be moved one. (one area), so the relative position of each monitor image can be maintained as much as possible. In addition, in another embodiment, the content of several monitor images to be output may be determined according to the level defined by the user, and the output monitor image is automatically determined by the server according to the importance, so that The user can more freely control the size of the monitor image and the display content of the display. The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.

100, 1010‧‧‧ image surveillance system
102_1~102_M, 1002_1~1002_M‧‧‧ monitor
104, 1004‧‧‧ display
110, 1010‧‧‧ server
120, 1020‧‧‧ processor
122, 1022‧‧‧Importance Calculation Module
124‧‧‧Layout setting module
126, 1026‧‧‧ play module
128‧‧‧Instant view layout module
130, 1030‧‧‧ decoder
140, 1040‧‧‧ device interface
900~906‧‧‧Steps
1024‧‧‧ Hierarchical layout module
1102‧‧‧Basic layout module
1104‧‧‧Sorting module
1106‧‧‧Monitor Image Assignment Module
1108‧‧‧Output circuit

1 is a schematic diagram of an image monitoring system in accordance with an embodiment of the present invention. Figure 2~5 is a schematic diagram of the server adjusting and adjusting the layout of the plurality of monitor images according to the importance of the plurality of monitor images transmitted by each monitor. Figure 6 shows an example of three monitor images being magnified because of their high importance. Figure 7 is a schematic diagram of the image enlargement operation when the monitor image to be enlarged and the monitor image that has been enlarged are overlapped. Figure 8 is a schematic diagram of triggering image layout update. FIG. 9 is a flow chart of a method for displaying a monitoring image according to an embodiment of the invention. Figure 10 is a schematic diagram of an image monitoring system in accordance with another embodiment of the present invention. Figure 11 is a schematic diagram of a hierarchical layout module and a display layout level input by a user according to the present invention. FIG. 12 is a schematic diagram of adjusting a layout of a plurality of monitor images according to a hierarchy set by a user according to an embodiment of the invention. Figure 13 is a diagram showing the layout of adjusting a plurality of monitor images according to a hierarchy set by a user according to another embodiment of the present invention.

100‧‧‧Image surveillance system

102_1~102_M‧‧‧Monitor

104‧‧‧ display

110‧‧‧Server

120‧‧‧ processor

122‧‧‧Importance Calculation Module

124‧‧‧Layout setting module

126‧‧‧Playing module

128‧‧‧Instant view layout module

130‧‧‧Decoder

140‧‧‧ device interface

Claims (11)

A server for an image monitoring system, comprising: a device interface for receiving a plurality of monitor images respectively from a plurality of monitors; and a processor coupled to the device interface for The plurality of monitor images are arranged, and in a preset state, the plurality of monitor images after the layout are transmitted to the display through the device interface for display; wherein the processor is further configured according to the plurality of The importance information of the monitor image adjusts the layout of the plurality of monitor images to adjust the screen size of the plurality of monitor images when displayed on the display. The server of claim 1, wherein the importance information is determined based on at least one detection result, wherein the at least one detection result is generated by at least one of the following detection operations: The processor performs motion detection on the plurality of monitor images, surrounding sound source detection of the monitor, detection of destruction behavior of the monitor, or detection of the direction of the user's eye. The server of claim 1, wherein when a specific monitor image of the plurality of monitor images is judged to have a higher importance, the processor controls the display to enlarge the specific monitor image And stop the display to display some of the other monitor images in the plurality of monitor images. The server of claim 3, wherein the importance information is determined based on a plurality of detection results, and the processor further determines, according to the multiple detection results, corresponding to the multiple detections. Measuring a plurality of importance parameters of the result, and then weighting the plurality of importance parameters to obtain a value; wherein the monitor image having the lowest value is the least important monitor image, and is stopped at the display The other monitor images displayed in the section contain at least the monitor image of the least importance. The server of claim 3, wherein when the particular monitor image is judged to have a higher importance, the processor magnifies the particular monitor image toward the center of the display screen. The server of claim 3, wherein the processor further determines at least one adjustment path between the specific monitor image and the portion of the other monitor images, and moves on the at least one adjustment path. Monitor image. A monitoring image display method includes: obtaining a complex array frame sequence from a plurality of input monitor image streams; analyzing a degree of change of front and rear frames in each group of frame sequences, generating a plurality of first important a plurality of importance information is generated according to the plurality of first importance parameters; and the screen size of the plurality of monitor video streams displayed on the display is adjusted according to the plurality of importance information. The method for displaying a surveillance image according to the seventh aspect of the invention, further comprising: receiving a plurality of volume information corresponding to the plurality of monitor image streams to generate a plurality of second importance parameters; wherein, according to the plurality The generating, by the first importance parameter, the plurality of importance information further includes: generating the plurality of importance information according to the plurality of first importance parameters and the plurality of second importance parameters. The method for displaying a surveillance image according to the seventh aspect of the invention, further comprising: receiving a plurality of event information corresponding to the plurality of monitor image streams to generate a plurality of third importance parameters; wherein the plurality The generating, by the first importance parameter, the plurality of importance information further includes: generating the plurality of importance information according to the plurality of first importance parameters and the plurality of third importance parameters. The method for displaying a surveillance image according to claim 7 , further comprising: establishing a first layout level and a second layout level, wherein the number of images of the first layout level disposed on the display is less than the plurality of monitoring The number of video streams that the second layout level has on the display is smaller than the number of images that are laid out on the display at the first layout level. The method for displaying a surveillance image according to claim 10, wherein the image of the second layout level disposed on the display is greater than the importance of the image of the first layout level disposed on the display.