CN116996749A

CN116996749A - Remote target object tracking system and method under multiple monitoring pictures

Info

Publication number: CN116996749A
Application number: CN202311260039.2A
Authority: CN
Inventors: 项士锋; 王象; 请求不公布姓名
Original assignee: Beijing Zen Ai Technology Co ltd
Current assignee: Beijing Zen Ai Technology Co ltd
Priority date: 2023-09-27
Filing date: 2023-09-27
Publication date: 2023-11-03
Anticipated expiration: 2043-09-27
Also published as: CN116996749B

Abstract

The invention relates to the technical field of image monitoring, in particular to a remote target object tracking system and method under multiple monitoring pictures, wherein the method comprises the following steps: shooting a display screen to collect a screen picture and converting the screen picture into a gray scale map; clipping the gray scale map into a plurality of sub-images; calculating a luminance value of each sub-image based on a gray value of a predetermined edge region of each sub-image; collecting time-series data of luminance values of each sub-image; calculating the correlation degree between the brightness value change of the sub-image where the target object is and the brightness value change of other sub-images, and determining the monitoring picture corresponding to one or more other sub-images with the correlation degree meeting the requirement as a target monitoring picture; and only respectively encoding the monitoring picture where the target object is positioned in the second screen picture and the target monitoring picture into a network video stream, and then transmitting the network video stream to the remote monitoring terminal through a network. The invention can solve the problems of difficult target tracking and the like under mass monitoring pictures and improve the tracking efficiency.

Description

Remote target object tracking system and method under multiple monitoring pictures

Technical Field

The application relates to the technical field of image monitoring, in particular to a remote target object tracking system and method under multiple monitoring pictures.

Background

In the prior art, although a monitoring center device can acquire a large number of monitoring pictures corresponding to a large number of monitoring cameras through a network, because the network capacity is limited, picture blocking is easy to occur when a large number of pictures are transmitted, therefore, when the monitoring pictures need to be checked to track the moving track of a certain target object, a plurality of people are usually required to be gathered before the monitoring center device, each person is focused on a plurality of monitoring pictures at the same time so as to determine which monitoring picture the target object moves to next, and therefore, the whole tracking process consumes very labor and time, and the tracking efficiency is very low.

Disclosure of Invention

In view of the above problems, the present application proposes, on the one hand, a remote target object tracking method under multiple monitoring pictures, including:

shooting a display screen on monitoring center equipment to acquire a first screen picture on the display screen, encoding the first screen picture into a network video stream, and then sending the network video stream to a remote monitoring end through a network for decoding and displaying by the remote monitoring end, wherein the first screen picture comprises at least two monitoring pictures;

Receiving a target object tracking instruction from the remote monitoring end and responding, wherein the target object tracking instruction comprises monitoring picture information of a target object, and the responding comprises the following steps:

s1, shooting the display screen to acquire a second screen picture on the display screen, and converting the second screen picture into a gray level image;

s2, cutting the gray level map into a plurality of sub-images, wherein each sub-image corresponds to one monitoring picture;

s3, calculating the brightness value of each sub-image based on the gray value of the preset edge area of each sub-image;

s4, collecting time series data of brightness values of each sub-image;

s5, calculating the correlation degree between the brightness value change of the sub-image where the target object is and the brightness value change of other sub-images, and determining the monitoring picture corresponding to one or more other sub-images with the correlation degree meeting the requirements as a target monitoring picture corresponding to the entering of the target object;

s6, only the monitoring picture where the target object is located in the second screen picture and the target monitoring picture where the target object is entering are respectively encoded into a network video stream, and then the network video stream is transmitted to the remote monitoring terminal through a network for decoding and displaying by the remote monitoring terminal;

S7, when the brightness value of the sub-image where the target object is located is determined not to change, encoding the target monitoring picture which the target object is entering into a network video stream, and then sending the network video stream to the remote monitoring terminal through a network for decoding and displaying by the remote monitoring terminal.

According to some embodiments of the invention, calculating the luminance value of each sub-image based on the gray value of the predetermined edge region of each sub-image includes: the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the sum of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image, or the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the average value of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image.

According to some embodiments of the invention, the correlation compliance comprises: the correlation is greater than a preset threshold or the correlation is greater than all other correlations.

According to some embodiments of the invention, the calculating the correlation between the brightness value change of the sub-image where the target object is located and the brightness value change of other sub-images includes:

s51, collecting time series data of brightness values of each sub-image, and ensuring that the time series data are matched correspondingly at corresponding time points;

s52, calculating an average value of time series data of the brightness value of each sub-image:

s53, calculating covariance of time series data of brightness values of each sub-image;

s54, calculating standard deviation of time series data of brightness values of each sub-image;

s55, calculating the pearson correlation coefficient of the time series data of the sub-image where the target object is and the time series data of other sub-images based on the covariance and the standard deviation.

According to some embodiments of the present invention, S6 further includes generating a first black image forming instruction and sending the first black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image according to the first black image forming instruction in a monitoring image where the target object is located in a second screen image and other monitoring image areas outside the target monitoring image; and

And S7, generating a second black image forming instruction and sending the second black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image according to the second black image forming instruction in other monitoring image areas outside the target monitoring image in a second screen image.

The application also provides a remote target object tracking system under the multi-monitoring picture, which comprises: the system comprises monitoring center equipment, at least two cameras, image acquisition and processing equipment and a remote monitoring end;

the monitoring center equipment is connected with each camera and used for converging monitoring pictures shot by each camera and outputting the monitoring pictures to the display screen;

the image acquisition and processing equipment is used for shooting a display screen on the monitoring center equipment to acquire a first screen picture on the display screen, encoding the first screen picture into a network video stream, and then sending the network video stream to a remote monitoring end through a network for decoding and displaying by the remote monitoring end, wherein the first screen picture comprises at least two monitoring pictures;

the image acquisition and processing device is further configured to receive a target object tracking instruction from the remote monitoring end and respond, where the target object tracking instruction includes monitoring picture information where a target object is located, and the responding includes:

s4, collecting time series data of brightness values of each sub-image;

s7, when the brightness value of the sub-image where the target object is located is determined not to be changed, encoding the target monitoring picture which the target object is entering into a network video stream, and then sending the network video stream to the remote monitoring terminal through a network for decoding and displaying by the remote monitoring terminal;

The remote monitoring end is used for receiving the network video stream, decoding and displaying the network video stream, and sending the target object tracking instruction to the graph acquisition and processing equipment.

According to some embodiments of the present invention, the remote monitoring end is further configured to store and splice the network video stream received in S7 according to the receiving time.

According to the embodiment of the invention, one monitoring person can quickly lock the next monitoring picture into which the target object enters before a large number of monitoring pictures, so that the situation that the target object is lost when the target object spans across a plurality of monitoring pictures is avoided. Compared with the monitoring work which can be completed only by a plurality of people staring at a plurality of screens at the same time, the invention greatly reduces the labor cost and the time cost and improves the target tracking and monitoring efficiency. Because the invention only carries out coding transmission on specific monitoring pictures, but not all pictures, the invention can realize the acquisition of effective monitoring content in the monitoring center through a network in different places, and the problems of picture blocking, lag and the like existing in the process of transmitting a large number of monitoring pictures originally are avoided.

Drawings

Fig. 1 illustrates a schematic diagram of a remote target object tracking system under a multi-monitor screen according to some embodiments of the invention.

Fig. 2 shows a screen shot by the image capturing and processing apparatus, when the target object is located at the monitoring screen B.

Fig. 3 shows a screen shot by the image capturing and processing apparatus, when the target object is located at the boundary of the monitor images a and B.

Fig. 4 shows a screen shot by the image capturing and processing apparatus, when the target object is located at the monitoring screen a.

Fig. 5 illustrates a partial process flow diagram of an image acquisition and processing device according to some embodiments of the application.

Fig. 6 shows a case where the target object moves from the monitoring screen a to the monitoring screen C.

Fig. 7 shows a schematic diagram of the main internal modules of an image acquisition and processing device according to some embodiments of the application.

Fig. 8 illustrates a schematic block diagram of a remote target object tracking device under a multi-monitor screen according to some embodiments of the present application.

Detailed Description

In the present application, the network is used to implement, and the essence is to cover the wired or wireless network connection implemented by the necessary firmware or software of the switch, router, etc., and also cover the wired or wireless network connection implemented by some service end or other computer as medium. In the present application, the networks involved may include Wi-fi networks, bluetooth networks, private Area Networks (PANs), local Area Networks (LANs), wide Area Networks (WANs), IEEE 802.1x, intranets, the internet, extranets, and combinations thereof. The network may also include a digital cellular telephone network, which may include Global System for Mobile communications (GSM), general Packet Radio Service (GPRS), cdmaOne, CDMA2000, evolution-data optimized (EV-DO), enhanced data rates for GSM evolution (EDGE), universal Mobile Telecommunications System (UMTS), digital Enhanced Cordless Telecommunications (DECT), digital AMPS (IS-136/TDMA), integrated Digital Enhanced Network (iDEN), wiMAX, LTE advanced, mobile Broadband Wireless Access (MBWA), IEEE 802.20. The network may be public access, private, virtual private, e.g., VPN.

The present application will be described by way of example with reference to the accompanying drawings in conjunction with the embodiments, and it should be noted that the embodiments of the present application and features of the embodiments may be combined with each other without conflict. In addition, the described embodiments are some, but not all, embodiments of the application.

In the application, in order to distinguish the screen images shot and collected by the image acquisition and processing device before and after the remote monitoring end sends the target object tracking instruction, the screen images shot and collected by the image acquisition and processing device before the remote monitoring end sends the target object tracking instruction are called as a first screen image, the screen images shot and collected by the image acquisition and processing device after the remote monitoring end sends the target object tracking instruction are called as a second screen image, and the first screen image and the second screen image are only distinguished in terms of expression.

Fig. 1 illustrates a schematic diagram of a remote target object tracking system under a multi-monitor screen according to some embodiments of the application.

The system comprises an image acquisition and processing device 6 comprising an image capturing device and a data processing device. The system may also comprise a plurality of cameras (cameras 1, 2, 3 and 4 are shown in the figure) and a monitoring center device 5. The monitoring center apparatus 5 includes a display screen. The image acquisition and processing device 6 and the remote monitoring terminal 7 establish a communication connection through a network.

The monitoring center device 5 is configured to aggregate the monitoring pictures of the respective cameras and output the monitoring pictures onto a display screen, which may include a display. According to other embodiments of the present invention, the monitoring center device may project the assembled pictures onto a projection screen by means of projection, where the display screen comprises a projection screen (curtain). Referring to fig. 1, a monitoring center apparatus 5 connects cameras (1, 2,3, 4) to acquire a picture of each camera. The monitoring center device 5 may be connected to each camera through an HDMI line or each camera through a network, for example. Four monitoring pictures A, B, C and D can be displayed on the monitoring center equipment, wherein each monitoring picture corresponds to pictures shot on cameras 1,2,3 and 4 respectively.

The monitoring center apparatus 5 may display pictures photographed by the respective cameras on the display screen in a distributed manner, and the monitoring center apparatus 5 may display four pictures in a manner of all longitudinal directions or all transverse directions in addition to those shown in the drawings.

The image collecting and processing device 6 is used for shooting a display screen of the monitoring center device 5, so as to collect a first screen picture on the display screen, encode the first screen picture into a network video stream, and then send the network video stream to the remote monitoring end 7 through a network, so that the remote monitoring end 7 decodes and displays the network video stream, and the remote monitoring end 7 can view all monitoring pictures from the decoded and displayed first screen picture.

The picture acquired by the image acquisition and processing device in fig. 2 coincides with the screen displayed on the monitoring center device of fig. 1, e.g. the picture acquired by the image acquisition and processing device is larger than the screen displayed on the monitoring center device, the image acquisition and processing device may crop the acquired picture so that it contains only the screen portion and no other background environment.

When the monitoring person at the remote monitoring end 7 finds that the received first screen image from the image capturing and processing apparatus contains the target object, and that the target object is moving toward the monitoring screen edge, a target object tracking instruction is input to the remote monitoring end (or the target object tracking instruction may be automatically generated when the remote monitoring end 7 detects that the target object is moving toward the monitoring screen edge or crosses a predetermined range), and the remote monitoring end 7 transmits the instruction to the image capturing and processing apparatus 6. The target object tracking instruction includes monitoring screen information on which the target object is located, for example, which monitoring screen the target object is currently located. The image acquisition and processing device 6 receives the target object tracking instruction from the remote monitoring end and responds, and the responding includes (as shown in fig. 5):

S1, shooting the display screen to acquire a second screen picture on the display screen, and converting the second screen picture into a gray level image.

S2, cutting the gray level map into a plurality of sub-images, wherein each sub-image corresponds to one monitoring picture.

The position of each monitoring screen on the screen to be photographed by the image pickup and processing device 6 may be stored in advance so that the sub-image can be cut out according to each monitoring screen position. The right-angle boundary of each monitoring picture in the gray level image can be identified, and the gray level image is cut into a plurality of sub-images according to the identification result, so that each sub-image corresponds to one monitoring picture. Here cropping may include actually cropping into a plurality of sub-frames, as well as merely distinguishing the boundaries of each sub-image.

S3, calculating the brightness value of each sub-image based on the gray value of the preset edge area of each sub-image.

The predetermined edge area is shown in dashed boxes in fig. 2,3, and 4, and may be located at the boundary position of each monitoring screen. According to some embodiments of the present invention, the predetermined edge region may also be set as a black filled portion shown in the monitoring screen D of fig. 2, which corresponds to an edge region of the entire circumference of the monitoring screen, so that a target object that suddenly moves into the monitoring line of sight (in the monitoring screen) from various directions can be covered. The size of the predetermined edge area may be adaptively adjusted according to the size, moving speed and camera shooting frame rate of the actual target object so that the activity of the target object at the edge may be captured within the response time of the camera. For example, when the target object is known to be an automobile, the predetermined edge area width may be a, and when the target object is known to be a pedestrian, the predetermined edge area width may be b, where a is greater than b, in order to provide the camera with sufficient time to capture the corresponding picture.

The calculating the brightness value of each sub-image based on the gray value of the predetermined edge region of each sub-image may employ the following formula:

luminance value=gray value/maximum gray value×maximum luminance value

Wherein the gray value is an integer value between 0 and 255 representing the gray level of the pixel. The maximum gray value is typically 255, representing a completely white color. The maximum luminance value is a maximum value representing luminance, and can be set as needed.

Assuming that the maximum luminance value is selected to be 100, the formula for calculating the luminance value will become:

luminance value=gray value/255×100

If the gray value of a certain pixel is 150, the luminance value calculated according to the above formula will be:

luminance value=150/255×100= 58.82

Therefore, the luminance value of the pixel is about 58.82.

The luminance value of each pixel of the edge region of each sub-image may be calculated based on the gray value of each pixel of the edge region of each sub-image and the sum of the luminance values of each pixel of the edge region of each sub-image may be taken as the luminance value of each sub-image, or the luminance value of each pixel of the edge region of each sub-image may be calculated based on the gray value of each pixel of the edge region of each sub-image and the average of the luminance values of each pixel of the edge region of each sub-image may be taken as the luminance value of each sub-image.

S4, collecting time series data of brightness values of each sub-image.

S5, calculating the correlation degree between the brightness value change of the sub-image (set as sub-image B) where the target object is located and the brightness value change of other sub-images, and determining the monitoring picture corresponding to the other sub-image or sub-images with the correlation degree meeting the requirement as the target monitoring picture corresponding to the entering of the target object.

The sub-image where the target object is located may be determined from a target object tracking instruction, where the instruction includes sub-image information where the target object is located.

The correlation meeting requirements comprises: the correlation is greater than a preset threshold or the correlation is greater than all other correlations.

The following calculates the correlation between a (t) and B (t) using pearson correlation coefficient, wherein B (t) is time-series data of luminance values of a monitor screen where a target object is located, and a (t) is time-series data of luminance values of one other sub-image, the calculation including S51-S55:

s51, collecting time series data (A (t), B (t)) of brightness values of each sub-image, and ensuring that the time series data and the B (t) are matched correspondingly at corresponding time points.

It is ensured here that the corresponding time points match, for example, the first time point of a (t), B (t) corresponds to the same time. According to the invention, all monitoring pictures are shot through one camera, and then the unified shooting result is subjected to image processing, so that the correspondence of the first time point can be well ensured.

S52, calculating the average value of time series data of the brightness value of each sub-image.

An average value of time-series data a (t) and B (t) of sub-images corresponding to the photographed monitor screen A, B is calculated:

mean value μa= (1/N) ×Σa (t)

Average μb= (1/N) ×Σb (t)

Where N represents the number of data points.

S53, calculating covariance of time series data of the brightness value of each sub-image.

Covariance cov (a (t), B (t)) of a (t) and B (t) is calculated:

cov(A(t),B(t))=(1/N)*Σ((A(t)-μA)*(B(t)-μB))

s54, calculating standard deviation of time series data of brightness values of each sub-image.

Standard deviations σa and σb of a (t) and B (t) are calculated:

σA=sqrt((1/N)*Σ((A(t)-μA)^2))

σB=sqrt((1/N)*Σ((B(t)-μB)^2))

s55, calculating the pearson correlation coefficient of the time series data of the sub-image (set as the sub-image B) where the target object is located and the time series data of other sub-images based on the covariance and the standard deviation:

for example, the pearson correlation coefficient between a (t), B (t) is calculated:

corr(A(t),B(t))=cov(A(t),B(t))/(σA*σB)

through the above steps, the pearson correlation coefficient between the variables a (t) and B (t) can be calculated.

When the pearson correlation coefficient is greater than a preset threshold, the monitor screen a may be determined as the target monitor screen into which the moving destination is entering.

For example, when one target object moves from the monitor screen B shown in fig. 2 toward the position of the monitor screen a (as shown in fig. 3 and 4), the luminance value of the left edge of the monitor screen B will change with time, and likewise, the luminance value of the right edge of the monitor screen a will change with time, and when such a change has a strong correlation, it is judged that the luminance change is caused by the same target object, and the luminance value of the B screen darkens to cause the luminance value of the a screen to lighten, or vice versa. This correlation will no longer exist when the target object has entered the monitoring screen completely.

For the case of only the monitor screen a and the monitor screen B, it can be determined whether the target object enters the monitor screen a in the above manner.

For the case where the monitor pictures C and D are also included, the pearson correlation coefficients of B (t) and C (t), and B (t) and D (t) are also calculated in the above manner. At this time, plural pearson correlation coefficients are compared, a sub-image having the greatest correlation with the change in the luminance value of the sub-image in which the target object is located is determined as a target sub-image, and a monitor screen corresponding to the target sub-image is determined as a target monitor screen in which the target object is entering. According to some embodiments of the present application, a threshold may also be set, and when the pearson correlation coefficients are greater than the threshold, the relevant monitoring picture is determined to include the target object.

S6, only the monitoring picture where the target object is located in the second screen picture and the target monitoring picture where the target object is entering are respectively encoded into a network video stream, and then the network video stream is transmitted to the remote monitoring terminal through the network for decoding and displaying by the remote monitoring terminal.

After determining the target monitoring picture that the target object is entering, the image acquisition and processing device 6 encodes only the monitoring picture in which the target object is located in the screen picture and the target monitoring picture that the target object is entering into a network video stream respectively, and sends the network video stream to the remote monitoring terminal through the network. As shown in fig. 3, when it is determined that the area where the change occurs is at the edge of the monitor pictures a and B, the image acquisition and processing apparatus 6 then independently encodes only the monitor pictures a and B photographed by it, respectively, encodes it to form a network video stream, and transmits the network video stream to the remote monitor terminal 7 through the network. In this way, compression encoding of the entire second screen can be avoided. According to other embodiments of the present application, S6 further includes generating a first black image forming instruction and sending the first black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image according to the first black image forming instruction in a monitoring image where the target object is located in the second screen image and other monitoring image areas outside the target monitoring image. The instruction may be, for example, "drawing a certain coordinate area black". Since the transmission amount of the instruction is far less than the transmission amount of the image, by adopting the method and combining the method, only partial images in the screen picture are cut and then respectively transmitted (namely, the monitoring picture where the target object is located and the target monitoring picture are respectively encoded into network video streams), instead of processing and transmitting the whole screen image, a large amount of image encoding and image transmission can be avoided. For example, for a PNG picture of 1080p black (pure black in the present application), the size is several KB even after being highly compressed, and the larger the number of black pictures, the larger the transmission amount. And for a black image forming instruction, the size may be only tens of bytes, and when the more black images are, only the black image generating position parameter needs to be introduced in the instruction. Therefore, the transmission amount is still small. When the image acquisition and processing device 6 encodes only the monitor pictures a and B it takes, respectively, and transmits it to the remote monitoring terminal 7, the remote monitoring terminal 7 will also receive the monitor pictures a and B taken. Therefore, the remote monitoring end can view the target object without other interference pictures, and the target can be rapidly positioned.

S7, when the brightness value of the sub-image where the target object is determined to be zero, encoding the target monitoring picture which the target object is entering into a network video stream, and then sending the network video stream to the remote monitoring terminal through a network for decoding and displaying by the remote monitoring terminal.

When the target object crosses the boundary area, only the new monitoring picture into which the target object has entered may be encoded to form a network video stream and the network video stream may be transmitted to the remote monitoring terminal 7 through the network. At this time, the remote monitoring end only receives a new monitoring picture where the target object is, so that the remote monitoring end is more convenient to centralize all monitoring on one monitoring picture, and the monitoring efficiency is improved.

As shown in fig. 4, when the target object moves to the position shown in fig. 4, the brightness value of the sub-image corresponding to the monitoring picture B changes to zero, and at this time, only the monitoring picture a (target monitoring picture) is encoded into a network video stream and then sent to the remote monitoring terminal through the network. Since S6 performs video transmission in such a manner that two videos are separately encoded and transmitted independently, it is only necessary to stop encoding and transmitting the monitoring picture B when the situation shown in fig. 4 is entered. Thereby, the amount of video data to be transferred to the remote monitoring end can be further reduced and also the object tracking is made easier and more efficient.

S7 may further include generating a second black image forming instruction and transmitting the second black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image in a region of the second screen image outside the target monitoring image according to the second black image forming instruction.

Since the monitoring picture in which the target object is located is changed from the first monitoring picture B to the monitoring picture a, the information of the old monitoring picture B in which the target object is located is replaced by the information of the new monitoring picture a in which the target object is located, when the target object enters another monitoring picture C (as shown in fig. 6) from the monitoring picture a again, S1-S7 are entered again, and a target object tracking instruction may also be reinitiated by the remote monitoring terminal to execute S1-S7.

The system may further comprise the remote monitoring terminal 7. The remote monitoring end 7 is used for receiving the network video stream and decoding and displaying the network video stream; the remote monitoring end 7 is further configured to send the target object tracking instruction to the graphics acquisition and processing device.

The remote monitoring terminal 7 may be further configured to store and splice the network video stream received in S7 according to the receiving time. For example, when the video stream of the monitoring picture B is received at the time t1, the video stream of the monitoring picture A is received at the time t2, and the video stream of the monitoring picture C is received at the time t3, the remote monitoring end splices the video streams of the B, the A and the C together in time sequence, so that how the target object moves from the A to the B and from the B to the C can be reproduced in the whole course, and the remote monitoring end directly skips the monitoring tracking of the monitoring picture D because the monitoring picture D is not involved, therefore, in this way, on one hand, a complete target object moving track map can be obtained quickly, and on the other hand, the problem that all monitoring personnel need to track the target object in the whole course, and the labor and the time consumption are extremely high is avoided, and meanwhile, the follow-up other people can conveniently review.

Fig. 7 shows a schematic diagram of the main internal modules of the image acquisition and processing device 6 according to some embodiments of the invention. The image acquisition and processing apparatus 6 includes an image acquisition module 6000, an image conversion module 6001, an image clipping module 6002, a luminance calculation module 6003, a correlation calculation module 6004, and a video encoding module 6005 and a video transmission module 6006.

The image acquisition module 6000 is used for acquiring screen pictures including at least two monitoring pictures (A, B, C and D);

The image conversion module 6001 is configured to convert a screen into a grayscale image.

The image clipping module 6002 is configured to clip the gray-scale image into a plurality of sub-images, where each sub-image corresponds to one of the monitoring frames.

The luminance calculation module 6003 is configured to calculate a luminance value (A1, B1, C1, D1) of each sub-image based on a gradation value of a predetermined edge region of each sub-image.

The correlation calculation module 6004 is configured to collect time-series data of luminance values of each sub-image, calculate a correlation between a luminance value (time-dependent) change of a sub-image where a target object is located and a luminance value change of other sub-images, and determine one or more other sub-images, in which the correlation meets a requirement, as a target monitoring picture in which the target object is entering. The correlation calculation module may calculate the pearson correlation coefficient through the aforementioned S51-S55.

The video encoding module 6005 is configured to encode, as a network video stream, only a monitored picture in which a captured target object is located and a target monitored picture in which the target object is entering, respectively.

The video sending module 6006 is configured to send a network video stream to a remote monitoring end through a network.

In addition, the image acquisition and processing device can also be used for receiving a target object tracking instruction from a remote monitoring end. The image acquisition and processing device may further include an instruction generation and transmission module (not shown in the figure) for generating a black image forming instruction, and transmitting the black image forming instruction to the remote monitoring terminal through a network, so that the remote monitoring terminal draws a black image in a monitoring image area other than the target monitoring image according to the black image forming instruction.

In addition, the respective functions of the image acquisition and processing apparatus described above with reference to fig. 1 to 6 may be implemented by adding more sub-modules, which will not be described here.

The application also provides a remote target object tracking method under the multi-monitoring picture, which comprises the following steps:

S4, collecting time series data of brightness values of each sub-image;

According to some embodiments of the invention, capturing a display screen on a monitoring center device to capture a first screen on the display screen includes capturing the display screen with a camera to capture a screen.

According to some embodiments of the present invention, S6 further includes generating a first black image forming instruction and sending the first black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image according to the first black image forming instruction in a monitoring image where the target object is located in a second screen image and other monitoring image areas outside the target monitoring image; and S7, generating a second black image forming instruction and sending the second black image forming instruction to the remote monitoring end through a network, so that the remote monitoring end draws a black image according to the second black image forming instruction in other monitoring image areas outside the target monitoring image in a second screen image.

According to some embodiments of the present invention, the method may further include the remote monitoring end storing and splicing the network video stream received in S7 according to the receiving time. For example, when the video stream of the monitoring picture B is received at the time t1, the video stream of the monitoring picture A is received at the time t2, and the video stream of the monitoring picture C is received at the time t3, the remote monitoring end splices the video streams of the B, the A and the C together in time sequence, so that how the target object moves from the A to the B and from the B to the C can be reproduced in the whole course, and the remote monitoring end directly skips the monitoring tracking of the monitoring picture D because the monitoring picture D is not involved, therefore, in this way, on one hand, a complete target object moving track map can be obtained quickly, and on the other hand, the problem that all monitoring personnel need to track the target object in the whole course, and the labor and the time consumption are extremely high is avoided, and meanwhile, the follow-up other people can conveniently review.

The method for tracking a remote target object under multiple monitoring frames may further comprise the respective detailed description of the method described above with reference to fig. 1-6, as performed by the image acquisition and processing device. For the sake of simplicity, the description is omitted here.

Fig. 8 illustrates a schematic block diagram of a remote target object tracking device 800 under a multi-monitor screen according to some embodiments of the invention. As shown in fig. 8, the device includes a processor 81, a memory 82, and a bus 83.

In some examples, the devices may also include an input device 801, an input port 802, an output port 803, and an output device 804. The input port 802, the processor 81, the memory 82, and the output port 803 are connected to each other through the bus 83, and the input device 801 and the output device 804 are connected to the bus 83 through the input port 802 and the output port 803, respectively, and further connected to other components of the device. The output interface and the input interface may be represented by I/O interfaces. Specifically, the input device 801 receives input information from the outside and transmits the input information to the processor 81 through the input port 802; processor 81 processes the input information based on computer executable instructions stored in memory 82 to generate output information, temporarily or permanently stores the output information in memory 82, and then communicates the output information to output device 804 via output port 803; the output device 804 outputs the output information to the outside of the device. The input device 801 may be, for example, an image acquisition module. The output device may be, for example, a video transmission module or the like.

The memory 82 includes mass storage for data or instructions. By way of example, and not limitation, memory 82 may comprise an HDD, floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 82 may include removable or non-removable (or fixed) media, where appropriate. The memory 82 may be internal or external to the device, where appropriate. In a particular embodiment, the memory 82 is a non-volatile solid state memory. In particular embodiments, memory 82 includes Read Only Memory (ROM). The ROM may be mask programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically Erasable PROM (EEPROM), electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these, where appropriate.

The bus 83 includes hardware, software, or both, coupling the various components to one another. By way of example, and not limitation, bus 83 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an infiniband interconnect, a Low Pin Count (LPC) bus, a memory bus, a micro channel architecture (MCa) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a video electronics standards association local (VLB) bus, or other suitable bus, or a combination of two or more of the above. Although embodiments of the invention have been described and illustrated with respect to a particular bus, the invention contemplates any suitable bus or interconnect.

The processor 81 performs the following actions based on a computer program stored in the memory 82:

s4, collecting time series data of brightness values of each sub-image;

In addition, the processor 81 may also execute the foregoing operations of the image processing and acquisition device 6 and the related remote target object tracking method under multiple monitoring frames based on the computer program stored in the memory 82, which are not described in detail for the sake of simplicity.

According to further embodiments of the present invention, the computer program may be divided into one or more units in various ways, stored in the memory, and executed by the processor to accomplish the present invention. The one or more elements may be a series of computer program instruction segments capable of performing the specified functions, which instruction segments describe the execution of the computer program in the device. The computer program may be split into a plurality of modules according to the module functions described above with reference to fig. 7. For simplicity, this will not be repeated here.

The processor may be a central processing unit (Central Processing Unit, CPU), other general purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), off-the-shelf programmable gate arrays (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like that is a control center of the device, connecting the various parts of the overall device using various interfaces and lines. The device may be a computing device such as a desktop computer, a notebook computer, a palm computer, a cloud server, or a portion thereof. The device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the schematic is merely an example of a device and is not meant to be limiting.

The respective detailed descriptions made above with reference to fig. 1-6 are also incorporated herein by reference and will not be repeated here.

The application also proposes a computer readable storage medium storing a computer program which, when executed by a processor, performs the following actions:

s4, collecting time series data of brightness values of each sub-image;

Further actions of the image processing and acquisition device 6 described above with respect to fig. 1-6 may be implemented when the computer program is executed by a processor, and are not described in detail for the sake of simplicity.

The computer program comprises computer program code which may be in source code form, object code form, executable file or in some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.

The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention. It should be noted that although the structure of the apparatus of the present invention and the method of operation thereof are depicted in the accompanying drawings in a particular order, this does not require or imply that the operations be performed in that particular order, or that all of the illustrated operations be performed, to achieve desirable results. Rather, the steps depicted in the flowcharts may change the order of execution. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step to perform, and/or one step decomposed into multiple steps to perform.

Claims

1. A method for tracking a remote target object in a multi-monitor screen, comprising:

s4, collecting time series data of brightness values of each sub-image;

2. The method of claim 1, wherein calculating the luminance value of each sub-image based on the gray value of the predetermined edge region of each sub-image comprises: the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the sum of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image, or the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the average value of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image.

3. The method for tracking a remote target object under a multi-monitor screen according to claim 1, wherein the correlation compliance requirement comprises: the correlation is greater than a preset threshold or the correlation is greater than all other correlations.

4. The method for tracking a remote target object under a multi-monitor screen according to claim 1, wherein calculating the correlation between the brightness value change of the sub-image where the target object is located and the brightness value change of other sub-images comprises:

5. The method according to one of claims 1 to 4, wherein S6 further comprises generating a first black image forming instruction and transmitting the first black image forming instruction to the remote monitoring terminal through a network, so that the remote monitoring terminal draws a black image according to the first black image forming instruction in a monitoring image of the target object in a second screen image and other monitoring image areas outside the target monitoring image; and

6. A remote target object tracking system under multiple monitoring pictures, comprising: the system comprises image acquisition and processing equipment, at least two cameras, monitoring center equipment and a remote monitoring end;

the monitoring center equipment is connected with each camera and used for converging monitoring pictures shot by each camera tuxk head and outputting the monitoring pictures to the display screen;

s4, collecting time series data of brightness values of each sub-image;

The remote monitoring end is used for receiving the network video stream, decoding and displaying the network video stream, and sending the target object tracking instruction to the image acquisition and processing equipment.

7. The system of claim 6, wherein calculating the luminance value of each sub-image based on the gray value of the predetermined edge region of each sub-image comprises: the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the sum of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image, or the luminance value of each pixel of the edge region of each sub-image is calculated based on the gray value of each pixel of the edge region of each sub-image and the average value of the luminance values of each pixel of the edge region of each sub-image is taken as the luminance value of each sub-image.

8. The system for tracking a remote object under multiple monitoring pictures according to claim 6, wherein the calculating the correlation between the brightness value change of the sub-image where the object is located and the brightness value change of other sub-images comprises:

9. The system according to any one of claims 6 to 8, wherein S6 further comprises generating a first black image forming instruction and transmitting the first black image forming instruction to the remote monitoring terminal via a network, so that the remote monitoring terminal draws a black image according to the first black image forming instruction in a monitoring image of the target object in a second screen and other monitoring image areas outside the target monitoring image; and

10. The remote object tracking system under multiple monitoring frames according to any one of claims 6-8, wherein the remote monitoring terminal is further configured to store and splice the network video stream received in S7 according to the receiving time.