TWI706676B - Multi- camera monitoring and tracking system and method thereof - Google Patents

Abstract

A multi-camera monitoring and tracking system and method, comprising: capturing a plurality of target images for a target character, and obtaining a plurality of target feature values according to the target images; respectively capturing a video by the plurality of image input devices; when judging that there is a humanoid image in each video, capturing an identification value from the humanoid image; obtained a trust value by comparing the identification value with the plurality of target feature value; judging the trust coefficient whether falls within a confidence value interval, and if the trust coefficient falls within the confidence interval, select the video with the smallest trust coefficient among the videos to be the specified video; and the specified video corresponding to the plurality of different time intervals is integrated into the monitoring and tracking video.

Description

Multi-camera monitoring and tracking system and method

This case is about the field of image monitoring, especially a multi-camera monitoring and tracking system and method.

Nowadays, the demand for video surveillance in public places is very popular, but because public places usually include many different areas and channels used to connect different areas, it is often necessary to install a monitoring system to monitor different spaces and channels with many monitors. In order to achieve the function of image monitoring.

However, such a monitoring system usually needs to display the pictures taken by each monitor side by side on the same screen or display them individually on different screens. However, this type of display method is because the monitoring system presents too many screens to the user at the same time, which makes it difficult for the user to focus on the target person in the screen, especially when the target person appears on different screens at the same time.

In addition, in order to facilitate users to monitor the movement of people in public places, some monitoring systems overlap the monitoring areas of the monitors to ensure that there are no monitoring loopholes. However, in practice, such a design often requires too many monitors, and it is difficult to avoid monitoring blind spots, so it still causes great inconvenience in actual use of the monitoring system.

In view of this, this case proposes a monitoring system and method.

According to some embodiments, a multi-camera monitoring and tracking method includes: capturing a plurality of target pictures of a target person, and obtaining a plurality of target feature values according to the target pictures; capturing videos by a plurality of image input devices; when determining that the video exists In the case of a humanoid image, the identification value of the humanoid image is captured; the identification value is compared with the target characteristic value to obtain the trust coefficient; it is judged whether the trust factor falls within the trust value interval, and if the trust factor falls within the trust interval, select the film with the smallest value A movie with a reliability coefficient of is used as a designated movie; and, multiple designated movies corresponding to different time intervals are integrated into a surveillance tracking movie.

According to some embodiments, the multi-camera monitoring and tracking system includes: a target feature value capturing device, a plurality of image input devices, a controller, and an image output device. The controller includes: an identification value acquisition module, a reliability coefficient calculation module, a video designation module, a monitoring and tracking video integration module, and a character continuous tracking module. Wherein, the target feature value capturing device is used to capture multiple target pictures of the target person, and obtain multiple target feature values according to the target picture. The image input device is used to capture video. The controller is used to receive the target feature value and the movie. The identification value acquisition module is used for the identification value acquisition process to detect the humanoid image in the video to acquire the identification value. The reliability coefficient calculation module is used to compare the identification value with the target characteristic value to obtain the reliability coefficient. The video specification module is used to determine whether the trust factor falls within the trust value interval. If the reliability coefficient falls within the confidence interval, the video specifying module selects the video with the smallest reliability coefficient as the specified video. Conversely, if the reliability factor does not fall within the confidence interval, the video specification module selects the default video in the video as the specified video. The monitoring and tracking video integration module is used to integrate multiple specified videos corresponding to different time intervals into monitoring and tracking videos. The image output device is used to output monitoring and tracking videos.

In summary, the multi-camera monitoring and tracking system and method thereof in some embodiments of this case can monitor the target person that may appear in each movie by comparing the target feature value of the target person with the identification value of the human image in each movie. In some embodiments, the multi-camera monitoring and tracking system and method thereof can also integrate multiple designated videos corresponding to different time intervals into monitoring and tracking videos, so that the user can continuously monitor the target person on a single screen.

FIG. 1 is a schematic diagram of a multi-camera monitoring and tracking system 10 according to some embodiments of the present application. 1, in some embodiments, the multi-camera monitoring and tracking system 10 includes a plurality of image input devices 100, a controller 200, an image output device 300, a target feature value capturing device 400, and a storage device (not shown in the figure) . The image input device 100, the image output device 300, the target feature value capturing device 400 and the storage device are coupled to the controller 200. In some embodiments, the image input device 100, the image output device 300, and the target feature value capturing device 400 are also coupled to the storage device.

It should be particularly noted that, in the multi-camera monitoring and tracking system 10, the signal transmission between the image input device 100, the controller 200, the image output device 300, the feature extraction device 400, and the storage device is not limited to electrical connections. Wired transmission, or by wireless cloud transmission. Moreover, the file access between the image input device 100, the controller 200, the image output device 300, the feature extraction device 400, and the storage device is not limited to the storage device, or directly through the built-in storage function of each device. In some embodiments, the image input device 100 is, for example, but not limited to, a fixed-point camera, a mobile camera with real-time image transmission function, or an aerial camera. The image output device 300 is, for example, but not limited to, a computer screen, a mobile phone screen, a car screen, or a screen with an image display function on other electronic devices.

In some embodiments, the controller 200 includes an identification value acquisition module 210, a reliability coefficient calculation module 220, a video specification module 230, a monitoring and tracking video integration module 240, and a character continuous tracking module 250. Among them, the identification value acquisition module 210, the reliability coefficient calculation module 220, the video specification module 230, the monitoring and tracking video integration module 240, and the character continuous tracking module 250 are, for example, but not limited to, each implemented by a single chip, and any combination is One chip is implemented, or the controller 200 integrates the identification value acquisition module 210, the reliability coefficient calculation module 220, the video specification module 230, the monitoring and tracking video integration module 240, and the character continuous tracking module 250 into the Implementation in a single chip, this case is not limited to this.

FIG. 2 is a schematic diagram of a multi-camera monitoring and tracking method according to some embodiments of this case. Please refer to Figure 1 and Figure 2 at the same time. In some embodiments, the multi-camera monitoring and tracking method includes the following steps: a target feature value capturing step (step S210); a video capturing step (step S220); an identification value capturing step (step S230); a reliability factor calculation step (step S230) S240); movie specifying step (step S250); monitoring and tracking movie integration step (step S260); and, character continuous tracking step (step S270).

The target feature value capturing step (step S210) includes: capturing multiple target pictures of the target person, and obtaining multiple target feature values based on the target pictures. Specifically, the target person is the target to be monitored and tracked by the multi-camera monitoring and tracking method. Multiple target pictures can be obtained by taking pictures of a target person through the target feature value capturing device 400. The target image is usually an image of the target person in different aspects to highlight the characteristics of the target person from various angles, such as the front image, the silhouette image or the back image of the target person. And by analyzing these target pictures, the target feature value extraction device 400 can obtain the target feature value corresponding to each target picture, that is, the color feature and texture feature of the target person at the corresponding angle (the detailed mechanism will be described later) Paragraph description). According to some embodiments, the feature value capturing device 400 performs circular photography of the target person to capture the target pictures. In practice, ring photography can be achieved by evenly distributing the angles corresponding to each target picture in 360 degrees. For example, when the target picture to be shot is 8 pictures, you can choose to target the target person in eight directions (0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees, 315 degrees, two-phase The angle between the adjacent directions is 45 degrees.) Take pictures to obtain characteristic images in these eight directions. In some embodiments, in the multi-camera monitoring and tracking system 10, the target feature value capturing device 400 outputs the target feature value, and the controller 200 receives the target feature value output by the target feature value capturing device 400.

The video capturing step (step S220) includes: capturing videos from a plurality of image input devices 100 respectively. Specifically, the image input device 100 usually includes at least one lens, and the video, that is, the image input device 100 captures a dynamic image corresponding to the area seen by the lens field of view. The monitoring areas corresponding to the respective image input devices 100 may or may not overlap, and this case is not limited to this. For example, the number of image input devices 100 can be three, which are the first camera, the second camera, and the third camera. The first camera corresponds to the first shooting area, the second camera corresponds to the second shooting area, and the third camera corresponds to the third shooting area. There is no overlap between the first shooting area and the second shooting area, there is no overlap between the first shooting area and the third shooting area, but there is an overlap between the second shooting area and the third shooting area. . In some embodiments, in the multi-camera monitoring and tracking system 10, the image input device 100 outputs a video, and the controller 200 receives the video output from the image input device 100.

The identification value capturing step (step S230) includes: when it is determined that there is a humanoid image in each movie, the identification value is captured from the humanoid image. Specifically, when the identification value acquisition module 210 detects the presence of a humanoid image in the video, the identification value acquisition module 210 obtains an identification value from the humanoid image according to the identification value extraction process. That is, when there is a humanoid image in any movie, the identification value capturing module 210 will obtain the identification value corresponding to the humanoid image, and it does not need to have a humanoid image in every movie to operate. According to some embodiments, the identification value capturing module 210 can determine whether there is a human image in the video by using the OpenPose algorithm. For the convenience of explanation, this manual only takes as an example that each movie has at most one humanoid image. In practice, multiple humanoid images can appear in the movie at the same time, and only need to repeat this step (step S230) to get the same movie. The identification value corresponding to each humanoid image of.

Next, the detailed steps of capturing the identification value of the humanoid image are further explained. FIG. 3 shows a flowchart of the identification value acquisition procedure of some embodiments of this case, FIG. 4 shows a schematic diagram of body position points of some embodiments of this case, and FIGS. 5 to 8 show diagrams of a scoring area of some embodiments of this case. Please also refer to Figure 1 to Figure 8. In some embodiments, the identification value capturing process includes the following steps: detecting a human figure image to obtain skeleton information (step S310); obtaining a plurality of body position points according to the skeleton information (step S320), and capturing a human figure according to the body position points The image is a recognized humanoid picture (step S330); the recognized humanoid picture is divided into multiple scoring regions; (step S340); and the color feature value corresponding to each scoring region and the texture feature value corresponding to each scoring region are obtained according to The identification value of the humanoid image (step S350).

Detect a humanoid image to obtain skeleton information (step S310). Specifically, in some embodiments, the identification value capturing module 210 can obtain skeleton information through human image operations by using the OpenPose algorithm.

Obtain multiple body position points according to the skeleton information (step S320). Specifically, in some embodiments, the skeleton information obtained by the recognition value capturing module 210 through the OpenPose algorithm includes multiple body position points of the human image. As shown in Figure 4, in some embodiments, the body position points include head vertices K0, neck joint points K1, shoulder joint points K2, K5, elbow joint points K3, K6, wrist joint points K4, K7, hip joint points K8, K11, knee joint points K9, K12, ankle joint points K10, K13, etc.

According to these body position points, the human figure image is captured as the human figure picture 900 (step S330). Specifically, based on the information of these body position points, the identification value capturing module 210 can locate the position of the humanoid image at any time in the video. Therefore, the identification value capturing module 210 can capture the humanoid image of the video at the corresponding time point as the humanoid image 900, that is, the static humanoid image at the corresponding time point, and use the humanoid image 900 for further feature analysis.

The recognizable humanoid picture is divided into multiple scoring regions (step S340). Specifically, the recognition value capturing module 210 divides a partial area in the recognized human figure picture 900 into a scoring area according to body position points. As shown in Figures 5 to 8, in some embodiments, there are 4 scoring regions, including a first scoring region A1 (Figure 5), a second scoring region A2 (Figure 6), and a third scoring region A3 (Figure 7). ) And the fourth scoring area A4 (Figure 8). The first scoring area A1 is a quadrilateral, with the distance from the head vertex K0 to the neck joint point K1 as the length, the half distance from the head vertex K0 to the neck joint point K1 as the width, and the neck joint point K1 as the lower midpoint. The second scoring area A2 is a quadrilateral surrounded by the shoulder joint points K2, K5 and the elbow joint points K3, K6. The third scoring area A3 is a quadrilateral surrounded by shoulder joint points K2, K5 and hip joint points K8, K11. The fourth scoring area A4 is a quadrilateral surrounded by hip joint points K8 and K11 and knee joint points K9 and K12.

The identification value of the human image is obtained according to the color feature value corresponding to each scoring area and the texture feature value corresponding to each scoring area (step S350). Specifically, in some embodiments, the color feature value is the identification value capturing module 210 using color moments to calculate the first scoring area A1, the second scoring area A2, the third scoring area A3, and the The four scoring area A4 can be obtained, where the color moment method is to calculate the first moment (mean), second moment (standard) and third moment (standard) corresponding to the area using the HSV color model (hue H, saturation S, lightness V) order moment (Skewness) derived nine values _{(mean h, mean s, mean} v,, std h, std s, std v, skewness h, skewness s, skewness v), and therefore the color characteristics of each area corresponding to a rating The value is the set of nine values, but this case is not limited to this. According to some embodiments, the texture feature value is the identification value acquisition module 210 using the conventional Gabor filter formula to calculate the first scoring area A1, the second scoring area A2, the third scoring area A3, and the fourth scoring area A4. Result, but this case is not limited by this. Because the identification value of the human image is the set of the color feature value corresponding to each scoring area and the texture feature value corresponding to each scoring area. Therefore, the identification value acquisition module 210 can obtain the identification value of the human image based on the color feature value corresponding to each scoring area and the texture feature value corresponding to each scoring area.

It should be noted that, in some embodiments, the identification value of the humanoid image does not include the color feature value of the first scoring area A1. Because the first scoring area A1 is located on the head of the humanoid image, and the head is usually not covered by clothing and presents skin color, resulting in unobvious color features, the identification value of the humanoid image does not use the color feature value of the first scoring area A1. Therefore, the identification value of the humanoid image of any movie at any point in time can be regarded as "the texture feature value of the first scoring area A1, the color feature value and texture feature value of the second scoring area A2, and the color feature of the third scoring area A3. Value and texture feature value, and the color feature value and texture feature value of the fourth scoring area A4", and can be represented by {V _TA , V _CB , V _TB , V _CC , V _TC , V _CD , V _TD }.

In the same way, the target feature value of the target person can also be obtained by analyzing the target person in the target picture by the target feature value extraction device 400 according to the identification value extraction process. Firstly, each scoring area of the target image of the target person in each direction is segmented, and then the color feature value and texture feature value of each scoring area are further obtained, so finally the target feature value can be obtained, that is, the color feature value of each scoring area And texture feature values". According to some embodiments, the target feature value does not include the color feature value of the first scoring area A1, so the target feature value can be represented by {V _X1 , V _X2 , V _X3 , V _X4 , V _X5 , V _X6 , V _X7 }, And the target feature values {V _X1 , V _X2 , V _X3 , V _X4 , V _X5 , V _X6 , V _X7 } correspond to the identification values {V _TA , V _CB , V _TB , V _CC , V _TC in sequence , V _CD , V _TD }. In some embodiments, since each target feature value is obtained from images taken in 8 directions, there are 8 target feature values in total.

Please continue to refer to Figure 1 and Figure 2. The reliability coefficient calculation step (step S240) includes: comparing the identification value with these target characteristic values to obtain the reliability coefficient. Specifically, the reliability coefficient calculation module 220 is used to compare the difference between the identification value corresponding to each movie and each target feature value, and the difference between the identification value and each target feature value can be an estimated value. Said. The reliability coefficient calculation module 220 can determine the size of these estimated values and select the minimum value of these estimated values as the reliability coefficient. According to some embodiments, the identification value {V _TA , V _CB , V _TB , V _CC , V _TC , V _CD , V _TD } and the target characteristic value {V _X1 , V _X2 , V _X3 , V _X4 , V _X5 , V The difference between _X6 and V _X7 } is obtained through a single-dimensional estimation formula, which is listed as follows: E _x =0.3*ED(v _x1 ,V _TA )+0.4*ED({v _x2 ,v _x4 ,v _x6 },{V _CB ,V _CC ,V _CD }) +0.3*ED({v _x3 ,v _x5 ,v _x7 },{V _TB ,V _TC ,V _TD }). Among them, E _x is the estimated value of the x-th direction, and the x-th direction corresponds to the shooting direction of the target picture. ED(v _x1 ,V _TA ) represents the Euclidean Distance between v _x1 and V _TA , ED({v _x2 ,v _x4 ,v _x6 },{V _CB ,V _CC ,V _CD }) represents Euclidean distance between {v _x2 ,v _x4 ,v _x6 } and {V _CB ,V _CC ,V _CD }, ED({v _x3 ,v _x5 ,v _x7 },{V _TB ,V _TC ,V _TD }) represents the Euclidean distance between {v _x3 ,v _x5 ,v _x7 } and {V _TB ,V _TC ,V _TD }.

In conclusion, the formula for obtaining the trust factor is T=Min(E _x ), where x=1~n, and T is the trust factor. The representative reliability coefficient is the minimum value among the estimated values in the first direction to the nth direction. Therefore, in an example where the target feature value is obtained through a target picture taken in 8 directions, the reliability coefficient is T=Min(E _x ), where x=1~8. The representative reliability coefficient is the minimum value among the estimated values in the first direction to the eighth direction.

In some embodiments, when the fourth scoring area A4 in the humanoid image does not exist, the single-oriented estimation formula may not calculate the difference between the color feature values and texture feature values of the fourth scoring area A4, and the single-oriented estimation formula at this time Listed as follows: E _x =0.3*ED(v _x1 ,V _TA )+0.4*ED({v _x2 ,v _x4 },{V _CB ,V _CC })+0.3*ED({v _x3 ,v _x5 } ,{V _TB ,V _TC }). Among them, ED({v _x2 ,v _x4 },{V _CB ,V _CC }) represents the Euclidean distance between {v _x2 ,v _x4 } and {V _CB ,V _CC }, ED({v _x3 ,v _x5 },{V _TB ,V _TC }) represents the Euclidean distance between {v _x3 ,v _x5 } and {V _TB ,V _TC }.

The film specifying step (step S250) includes: determining whether the reliability coefficient falls within the confidence interval, wherein, if the reliability coefficient falls within the confidence interval, the movie with the smallest reliability coefficient among these films is selected as the specified movie, otherwise, if the reliability is When the coefficient does not fall within the confidence interval, select the default video among these videos as the designated video. Specifically, the film specifying module 230 is used to determine whether the reliability coefficient falls within the confidence value interval. If any one of the reliability coefficients falls within the confidence interval, the film specifying module 230 selects the smallest reliability coefficient among the reliability coefficients falling within the confidence interval, and the movie specifying module 230 selects the one with the smallest reliability coefficient The video is the designated video. Because the smaller the value of the reliability coefficient, the smaller the difference between the identification value of the film and the target feature value, that is, the more likely the target person is in the film. Moreover, the reliability coefficient corresponding to the specified movie also needs to fall into the confidence interval. Conversely, if none of these reliability coefficients fall within the confidence interval, the video specifying module 230 selects the default video among these videos as the specified video. That is, if the current reliability coefficient corresponding to each video is not in the confidence interval, the video specification module 230 selects the video taken by the preset image input device 100 as the specified video, such as the first camera mentioned above. . According to some embodiments, the confidence interval is 0.814-1.65. When the reliability coefficients are all greater than 1.65, it means that there is no target person in each movie. Therefore, the movie specifying module 230 selects the preset movie of the image input device 100 as the specified movie. Conversely, when the trust factor is less than 0.814, it means that the difference between the identification value of the film and the target feature value is extremely small, that is, the target person exists in the film. Since it is relatively rare that the confidence coefficient is less than 0.814, it can be excluded from the confidence interval.

The monitoring and tracking video integration step (step S260) includes: integrating a plurality of designated videos corresponding to different time intervals into a monitoring and tracking video. Specifically, although there is only one designated movie in each time interval, the designated movies between different time intervals are not necessarily the same. Therefore, the monitoring and tracking video integration module 240 can cut multiple video clips (that is, multiple specified videos corresponding to different time intervals) into a single monitoring and tracking video. In some embodiments, the monitoring and tracking video integration module 240 detects the current designated video in real time, and transfers the designated video as a part of the monitoring and tracking video.

It should be noted that, in some embodiments, when the difference in the trust factor of two videos at the same time point is very small, that is, when both videos are qualified as designated videos, if the background information in the two videos The overlap ratio (except for the human image in the film) is higher than a set value, and the monitoring and tracking video integration module 240 can also stitch the frames of the two videos as a monitoring and tracking video.

The step of continuously tracking the character (step S270) includes: obtaining a first tracking value according to the first humanoid picture at the first time point in the specified movie; obtaining the second humanoid picture according to the second humanoid picture at a second time point in the specified movie Tracking value, where the second time point is later than the first time point; the first tracking value and the second tracking value are compared to obtain the tracking comparison value. When the tracking comparison value is greater than or equal to the tracking threshold, the first human figure is determined The picture matches the second humanoid picture. On the contrary, when the tracking comparison value is less than the tracking threshold, the first humanoid picture is determined to be inconsistent with the second humanoid picture, and the specified video is canceled; and, combining the first tracking value and the second The tracking value is set to update the tracking value. Since the person continuous tracking step (step S270) in the multi-camera monitoring and tracking method corresponds to the person continuous tracking procedure in the multi-camera monitoring and tracking system 10, please refer to the next paragraph for detailed description.

FIG. 9 shows a flowchart of a continuous tracking procedure of a person in some embodiments of this case. Please refer to Figure 2 and Figure 9 at the same time. In some embodiments, the person continuous tracking module 250 detects the human figure in the specified movie according to the person continuous tracking process to capture the tracking comparison value, and the person continuous tracking module 250 further determines the tracking ratio according to the person continuous tracking process. Whether the value is greater than or equal to the tracking threshold to determine whether the first figure picture matches the second figure picture. In some embodiments, the character continuous tracking procedure includes the following steps: obtaining a first tracking value according to the first humanoid picture at the first time point in the specified movie (step S910); according to the second time point in the specified movie The humanoid picture obtains a second tracking value, where the second time point is later than the first time point (step S920); the first tracking value and the second tracking value are compared to obtain the tracking comparison value (step S930); A tracking value and a second tracking value are set to update the tracking value (step S940) to determine whether the tracking comparison value is greater than or equal to the tracking threshold (step S950); when the tracking comparison value is greater than or equal to the tracking threshold, the first figure picture is determined It is consistent with the second humanoid picture (step S960); otherwise, when the tracking comparison value is less than the tracking threshold, it is determined that the first humanoid picture does not match the second humanoid picture, and the designated video is canceled (step S970).

According to the first humanoid picture at the first time point in the designated movie, a first tracking value is obtained (step S910). According to the second humanoid picture at the second time point in the specified movie, a second tracking value is obtained, where the second time point is later than the first time point (step S920). Specifically, through the aforementioned steps S310 to S330, the character continuous tracking module 250 can obtain the first humanoid picture at the first time point in the specified movie and the second humanoid picture at the second time point in the specified movie. In some embodiments, by using the OpenPose algorithm, the skeleton information can be obtained through the calculation of the humanoid image in the specified video, and the skeleton information includes multiple body position points of the humanoid image. Based on the information of these body position points, the humanoid image can be located at any time in the film. Therefore, the character continuous tracking module 250 can capture the recognized humanoid picture 900 at the corresponding time point in the specified movie, that is, the first humanoid picture at the first time point and the second humanoid picture at the second time point.

In conclusion, in some embodiments, the person continuous tracking module 250 can obtain the first tracking value from the first humanoid picture and the second tracking value from the second humanoid picture through the ORB algorithm. The first tracking value corresponds to the feature of the first humanoid picture, and the second tracking value corresponds to the feature of the second humanoid picture. It should be noted that, in some embodiments, the first time point is the time point when the character continuous tracking module 250 performs step S910, and the second time point is the time point when the character continuous tracking module 250 performs step S920. Therefore, The second time point is later than the first time point. And according to some embodiments, the first time point represents the start time point of the specified movie, and the second time point is one unit time later than the first time point.

The first tracking value and the second tracking value are compared to obtain a tracking comparison value (step S930). Specifically, according to some embodiments, the person continuous tracking module 250 obtains the tracking comparison value by Hamming match between the first tracking value and the second tracking value. The tracking comparison value is the difference between the first tracking value and the second tracking value, that is, the difference between the characteristics of the first humanoid picture and the characteristics of the second humanoid picture.

The first tracking value and the second tracking value are combined to set the updated tracking value (step S940). Specifically, the updated tracking value is the value of the first tracking value plus the second tracking value, so the updated tracking value can represent the specified video at the first time point and the second time point at the same time, and further follow the next time point Comparison (for example, the third point in time). In some embodiments, the person continuous tracking module 250 can further use the updated tracking value to perform a Hamming comparison with the third tracking value to obtain another tracking comparison value, where the third tracking value corresponds to the third tracking value in the specified movie. The third person picture at the time point, the third time point is one unit time later than the second time point. Therefore, the other tracking comparison value can simultaneously represent the difference between the third tracking value and the first tracking value and the difference between the third tracking value and the second tracking value. That is, the difference between the characteristics of the third humanoid picture and the characteristics of the first humanoid picture, and the difference between the characteristics of the third humanoid picture and the characteristics of the second humanoid picture. By analogy, by continuously updating the setting and updating tracking value of the character continuous tracking module 250, the character continuous tracking module 250 can continuously track the target person in the specified movie.

It is determined whether the tracking comparison value is greater than or equal to the tracking threshold (step S950); when the tracking comparison value is greater than or equal to the tracking threshold, it is determined that the first figure picture matches the second figure picture (step S960); otherwise, when the tracking comparison value is When the value is less than the tracking threshold, it is determined that the first humanoid picture does not match the second humanoid picture, and the designated video is canceled (step S970). Specifically, the person continuous tracking module 250 is used to determine whether the tracking comparison value is greater than or equal to the tracking threshold. The tracking threshold corresponds to the number of matching features between the first figure picture and the second figure picture, that is, the larger the number of matching features, the closer the features of the first figure picture and the second figure picture are. When the tracking comparison value is greater than or equal to the tracking threshold, the person continuous tracking module 250 determines that the first figure picture matches the second figure picture. That is, the character continuous tracking module 250 determines that there is still a target character in the specified movie, and the difference from the target character in the starting time point of the specified movie is limited. When the tracking comparison value is less than the tracking threshold, the person continuous tracking module 250 determines that the first humanoid picture does not match the second humanoid picture. That is, the character continuous tracking module 250 determines that there is a significant difference between the target character in the specified movie and the target character in the starting time point of the specified movie. Therefore, the character continuous tracking module 250 cancels the setting of the designated video, and the multi-camera monitoring and tracking system 10 must reselect another designated video to monitor the target person.

According to some embodiments, the feature threshold value is 15, which means that the tracking comparison value is divided by 15 matching features. If the tracking comparison value is less than 15, it is deemed that the target person cannot be identified or disappeared in the current specified video. Therefore, the person continuous tracking module 250 no longer executes the person continuous tracking process, and the multi-camera monitoring and tracking system 10 needs to search again Target person. Conversely, if the tracking comparison value is greater than or equal to 15, it is deemed that the target person still exists in the current designated movie, and therefore the person continuous tracking module 250 continues to execute the person continuous tracking process.

In some embodiments, when the controller 200 is performing the person continuous tracking step (step S270), since it only needs to analyze the specified movie and not every movie, the calculation burden for the multi-camera monitoring and tracking system 10 is relatively low. Conversely, when the controller 200 is performing the video capturing step (step S220) to the video specifying step (step S250), since the controller 200 needs to analyze each video to obtain the identification value, it is therefore a multi-camera monitoring and tracking system The calculation burden of 10 is greater. According to some embodiments, the controller 200 includes a central processing unit (CPU) and a graphics processing unit (GPU). When the controller 200 executes the video capturing step (step S220) to the video specifying step (step S250), the controller 200 will allocate part of the computing burden of the CPU to the graphics processor, which has reduced the utilization rate of the CPU . In contrast, when the controller 200 executes the person continuous tracking step (step S270), the controller 200 only uses the central processing unit as the calculation, because the calculation complexity is low at this time. Therefore, the multi-camera monitoring and tracking system 10 has the advantage of low CPU usage.

In summary, the multi-camera monitoring and tracking system and method thereof in some embodiments of this case can monitor the target person that may appear in each movie by comparing the target feature value of the target person with the identification value of the human image in each movie. In some embodiments, the multi-camera monitoring and tracking system and method thereof can also integrate multiple designated videos corresponding to different time intervals into monitoring and tracking videos, so that the user can continuously monitor the target person on a single screen. In some embodiments, the multi-camera monitoring and tracking system and the method thereof can detect the tracking comparison value of the human image in the specified movie to determine whether to reselect the specified movie, and therefore has the advantage of low computational complexity.

10: Monitoring system 100: Video input device 200: Controller 210: Identification value acquisition module 220: Reliability coefficient calculation module 230: Video designated module 240: Monitoring and tracking video integration module 250: Character Continuous Tracking Module 300: Video output device 400: Target feature value extraction device 900: Recognize humanoid pictures K0: head vertex K1: neck joint point K2, K5: Shoulder joint points K3, K6: elbow joint point K4, K7: wrist joint points K8, K11: hip joint points K9, K12: knee joint points K10, K13: Ankle joint point A1: The first scoring area A2: Second scoring area A3: The third scoring area A4: The fourth scoring area S210-S270: steps S310-S350: steps S910-S970: steps

Figure 1 shows a schematic diagram of a multi-camera monitoring and tracking system according to some embodiments of the present case. FIG. 2 is a schematic diagram of a multi-camera monitoring and tracking method according to some embodiments of this case. FIG. 3 shows a flowchart of the identification value acquisition procedure of some embodiments of the present application. FIG. 4 is a schematic diagram of the body position points of some embodiments of this case. FIG. 5 is a schematic diagram of the first scoring area in some embodiments of this case. FIG. 6 is a schematic diagram of the second scoring area in some embodiments of this case. FIG. 7 is a schematic diagram of the third scoring area in some embodiments of this case. FIG. 8 is a schematic diagram of the fourth scoring area in some embodiments of this case. FIG. 9 shows a flowchart of a continuous tracking procedure of a person in some embodiments of this case.

10: Multi-camera monitoring and tracking system

100: Video input device

200: Controller

210: Identification value acquisition module

220: Reliability coefficient calculation module

230: Video designated module

240: Monitoring and tracking video integration module

250: Character Continuous Tracking Module

300: Video output device

400: Target feature value extraction device

Claims (10)

A multi-camera monitoring and tracking method includes: A target feature value capturing step, capturing multiple target pictures of a target person, and obtaining multiple target feature values according to the target pictures; In a video capturing step, a video is captured by a plurality of image input devices; An identification value capturing step, when it is determined that there is a humanoid image in each movie, an identification value is captured for the humanoid image; A step of calculating a trust factor, comparing the identification value with the target characteristic values to obtain a trust factor; A film designation step to determine whether the trust factor falls within a confidence interval, and if the trust factor falls within the trust interval, select the movie with the smallest trust factor among the movies as a designated movie; and A monitoring and tracking video integration step is to integrate the designated videos corresponding to a plurality of different time intervals into a monitoring and tracking video. The multi-camera monitoring and tracking method according to claim 1, wherein the reliability coefficient calculation step includes, if the reliability coefficient does not fall within the confidence interval, selecting a preset film among the films as the designated film. The multi-camera monitoring and tracking method described in claim 2 further includes a step of continuously tracking a person, including: Obtaining a first tracking value according to a first humanoid picture at a first time point in the designated movie; Obtaining a second tracking value according to a second humanoid picture at a second time point in the designated movie, wherein the second time point is later than the first time point; Compare the first tracking value and the second tracking value to obtain a tracking comparison value, and when the tracking comparison value is greater than or equal to a tracking threshold, it is determined that the first figure picture matches the second figure picture; and Combine the first tracking value and the second tracking value to set an updated tracking value. The multi-camera monitoring and tracking method according to claim 3, wherein the continuous tracking step of the person includes: when the tracking comparison value is less than the tracking threshold, determining that the first humanoid picture does not match the second humanoid picture, And unset the specified video. The multi-camera monitoring and tracking method according to claim 1, wherein the identification value capturing step includes: Detecting the humanoid image to obtain a skeleton information; Obtain multiple body position points according to the skeleton information; Capturing the humanoid image according to the body position points as a recognizable humanoid image; Segment the recognizable humanoid image into multiple scoring areas; and The identification value of the human image is obtained according to a color feature value corresponding to each scoring area and a texture feature value corresponding to each scoring area. A multi-camera monitoring and tracking system includes: A target feature value capturing device for capturing multiple target pictures of a target person, and obtaining multiple target feature values according to the target pictures; A plurality of image input devices are respectively used to capture a video; A controller for receiving the target feature values and the films, the controller includes: An identification value acquisition module for detecting a humanoid image in each video to acquire an identification value according to an identification value acquisition process; A trust factor calculation module for comparing the identification value with the target characteristic values to obtain a trust factor; A film designation module for determining whether the reliability coefficient falls within a confidence interval, wherein, if the reliability coefficient falls within the confidence interval, the film designation module selects the film with the smallest reliability coefficient among the films The video is regarded as a designated video. On the contrary, if the reliability factor does not fall within the confidence interval, the video designation module selects a default video among the videos as the designated video; and A monitoring and tracking video integration module for integrating the designated videos corresponding to multiple different time intervals into a monitoring and tracking video; and An image output device for outputting the monitoring and tracking video. The multi-camera monitoring and tracking system according to claim 6, wherein the controller further includes a character continuous tracking module, and the character continuous tracking module detects the humanoid image in the specified movie according to a character continuous tracking procedure. A tracking comparison value is captured, and the person continuous tracking module determines whether the tracking comparison value is greater than or equal to a tracking threshold to determine whether the first human figure picture matches the second human figure picture, wherein, if the tracking comparison When the value is greater than or equal to a tracking threshold, the person continuous tracking module determines that the first humanoid picture matches the second humanoid image; otherwise, if the tracking comparison value is less than the tracking threshold, the person continuous tracking module It is determined that the first humanoid picture does not match the second humanoid picture, and the designated video is canceled. The multi-camera monitoring and tracking system according to claim 7, wherein the continuous tracking program for the person includes: Obtaining a first tracking value according to a first humanoid picture at a first time point in the designated movie; Obtaining a second tracking value according to a second humanoid picture at a second time point in the designated movie, wherein the second time point is later than the first time point; Compare the first tracking value and the second tracking value to obtain a tracking comparison value; and Combine the first tracking value and the second tracking value to set an updated tracking value. The multi-camera monitoring and tracking system according to claim 6, wherein the identification value acquisition step includes: Detecting the humanoid image to obtain a skeleton information; Obtain multiple body position points according to the skeleton information; Capturing the humanoid image according to the body position points as a recognizable humanoid image; Segment the recognizable humanoid image into multiple scoring areas; and The identification value of the human image is obtained according to a color feature value corresponding to each scoring area and a texture feature value corresponding to each scoring area. The multi-camera monitoring and tracking system according to claim 6, wherein the target feature value capturing device captures the target pictures by performing ring photography on the target person.

Images