CN110008831A - A kind of Intellectualized monitoring emerging system based on computer vision analysis - Google Patents

Abstract

The invention discloses a kind of Intellectualized monitoring emerging system based on computer vision analysis, including video monitoring system, face identification system, video structure analyzing system, Multisensor video fusion system.Video monitoring system is responsible for acquiring video information and plant area's important area temperature information；Face identification system is responsible for identifying plant area worker；Video structure analyzing system can determine whether whether safe wearing cap, vehicle enter unauthorized area, whether, employee consistent with its working region carries package to employee work clothes to employee；Multisensor video fusion system is responsible for being managed collectively the various information of video monitoring system, face identification system and video structure analyzing system.The system can be monitored and analyze to plant area in real time, find plant area's abnormal conditions and early warning in time, to provide safeguard for plant area's normal operation.

Description

Intelligent monitoring fusion system based on computer vision analysis

Technical Field

The invention relates to a factory running state monitoring system, in particular to an intelligent monitoring fusion system based on computer vision analysis, and belongs to the field of safety monitoring.

Background

Safety is always placed first in the production run of a plant. However, in the daily production of a factory, there are many safety hazards, such as the temperature of a critical area is too high, a worker enters a working area without wearing a safety helmet, a vehicle enters an area without permission to enter, a worker carries objects into a critical area where the objects are prohibited, and the worker goes wrong in the working area, and the like, which seriously threatens the normal operation of the factory.

Disclosure of Invention

In order to solve the problems, the invention provides an intelligent monitoring and fusion system based on computer vision analysis, which comprises a video monitoring system, a face recognition system, a video structural analysis system and a video fusion system.

The video monitoring system consists of a front-end camera, a thermal infrared imager and a video monitoring platform; the front-end camera is responsible for collecting video information; the thermal infrared imager is responsible for acquiring temperature information of important areas of a plant area; the video monitoring platform receives video information collected by the front-end camera and temperature information of important areas of the plant, collected by the thermal infrared imager, displays the video information, transmits the video information to the video structural analysis system, and transmits the video information and the temperature information of the important areas of the plant to the video fusion system.

The face recognition system consists of a face snapshot device, a face recognition module and an employee identity database; the face snapshot device collects images of staff entering and leaving each key area of the factory area and transmits the images to the face recognition module; the employee identity database stores the identity information and facial features of each employee; the face recognition module receives the employee image collected by the face snapshot device, performs face recognition, determines identity information of the employee through comparison with the employee identity database, and transmits a recognition result to the video fusion system.

The extraction process of the face features of the staff in the face recognition system is as follows: acquiring a face image of each employee in the factory, wherein the size of the image is MxN, and K face images can be obtained if K employees are in common; carrying out gray scale linear transformation processing on K human face images and3 x 3 median filtering to obtain K face gray level images; connecting the gray values of the pixels of each line of each face gray image, forming a line vector with dimension D being M multiplied by N for each face gray image, and recording the formed line vector x for the ith face gray image_iAnd calculating an average faceArranging K row vectors formed by K face gray level images to form a K multiplied by D dimensional matrix, and performing K-L transformation on the matrix to obtain a characteristic face space: w ═ u₁，u₂，u₃，…，u_p) In the formula: p is the set dimensionality after dimensionality reduction; calculating the facial feature of each employee, and for the jth employee, calculating the facial feature omega of each employee_jThe calculation formula of (2) is as follows: omega_j＝w^T(x_j-Ψ)。

The human face recognition module in the human face recognition system performs the human face recognition process as follows: acquiring a face image captured by a face capturing device and entering and exiting each key area of a factory area, performing gray linear transformation on the face image to obtain a face gray image, and connecting gray values of pixels of each line of the face gray image to obtain a line vector y; calculating the facial feature omega of the face image_j＝w^T(x_j- Ψ), wherein: w is the characteristic face space determined in claim 2: w ═ u₁，u₂，u₃，…，u_p) Ψ represents an average face obtained in claim 2; calculate Ω_ySelecting the Euclidean distance from the facial features stored in the employee identity database to omega_yAnd the staff identity corresponding to the facial feature is the face recognition result.

The video structured analysis system is composed of a safety helmet identification module, a license plate identification module, a work clothes identification module and a carrying object identification module, receives video information collected by a video monitoring system, judges whether a worker wears a safety helmet through the safety helmet identification module, judges whether a vehicle enters an unauthorized area through the license plate identification module, judges whether a worker work clothes is consistent with a work area through the work clothes identification module, judges whether the worker carries a package through the carrying object identification module, and transmits an analysis result to the video fusion system.

The process that the safety helmet identification module judges whether the staff wear the safety helmet is as follows:

reading video information collected by a camera in a detection area in real time, extracting a previous frame and a current frame, and carrying out gray processing on the previous frame and the current frame to obtain gray images of the previous frame and the current frame; processing the gray level images of the previous frame and the current frame by using an interframe difference method to obtain a moving object image; constructing one or more moving object estimation rectangular frames to extract all mutually independent parts in the moving object image, and if K mutually independent parts exist in the image, obtaining K moving object estimation rectangular frames; extracting the corresponding part of the gray level image and the color image of each moving object estimation rectangular frame in the current frame to obtain K moving object estimation gray level images and K moving object estimation color images; carrying out edge detection on the K moving object estimated gray level images one by using a Canny operator to obtain K moving object edge images, and constructing a moving object rectangular frame in each moving object edge image to contain all edges in the image to obtain K moving object rectangular frames; confirming a human body rectangular frame in the moving object rectangular frame according to the length-width ratio of the human body, supposing that k human body rectangular frames are judged, extracting the image of the corresponding part of each human body rectangular frame in the moving object estimated color image to obtain k human body color images; for each human body rectangular frame, positioning the head of a person in the human body rectangular frame according to the upper half-body characteristics of the human body, and positioning the safety helmet estimation areas according to the head proportion to obtain k safety helmet estimation areas; extracting the image of the corresponding part of each safety helmet estimation area in the corresponding human body color image to obtain k safety helmet area color images, and judging the safety helmet area color images one by one according to the color; the method comprises the following specific steps:

step1, extracting a previous frame and a current frame, and performing graying processing on the previous frame and the current frame by adopting a weighted average method, namely distributing different weights to three components of R, G and B of each pixel point, wherein the specific formula is as follows: f (x, y) ═ 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), where R (x, y), G (x, y), B (x, y) refer to the R, G, B components of the pixel with coordinates (x, y), respectively, and f (x, y) refers to the gray-scale value after conversion of the pixel with coordinates (x, y);

step2, processing the gray level images of the previous frame and the current frame by using an interframe difference method to obtain a moving object image, wherein the process is as follows: subtracting the gray values of the pixels corresponding to the two frames of images, and taking the absolute value of the gray values to obtain a difference image D_n(x，y)＝|f_n(x，y)-f_n-1(x, y) |, wherein f_n(x, y) denotes a gray value of a pixel having coordinates (x, y) in the current frame, f_n-1(x, y) refers to the gray value of the pixel with the coordinate (x, y) in the previous frame; setting a threshold T_aIf D is_n(x，y)＞T，R_n255 if D_n(x，y)＜T，R_nObtaining a moving object graph R as 0; r is a binary image, wherein a white area represents a moving object area;

and 3, constructing K moving object estimation rectangular frames, extracting all mutually independent parts in the moving object image, acquiring the position information of each moving object estimation rectangular frame, and storing the position information in a matrix LO, wherein the specific process is as follows:

step 1: taking the lower left corner of the moving object image R obtained in the step2 as an origin, establishing a coordinate axis, wherein the moving object image R is in a first quadrant; setting a straight line x as h, wherein the initial value of h is 0;

step 2: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 3: if the pixel value of each pixel on the straight line x-h is 0, returning to Step 2; if a pixel having a pixel value other than 0 exists on the straight line x-h, let h be x_min；

Step 4: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 5: if the pixel with the pixel value not being 0 exists on the straight line x ═ h, recording the ordinate value corresponding to the uppermost non-0 pixel and the ordinate value corresponding to the lowermost non-0 pixel on the straight line, storing the ordinate values in the array Y, and returning to Step 4; if the pixel value of each pixel on the straight line x-h is 0, let h be x_max；

Step 6: comparing the sizes of the elements in the array Y to obtain the maximum value Y in the Y_maxAnd the minimum value y_min(ii) a With a straight line y ═ y_maxAs the upper edge, a straight line y ═ y_minAs the lower edge, a straight line x ═ x_minAs the left edge, the straight line x ═ x_maxConstructing a motion estimation area rectangular frame as a right edge; zero clearing of array Y and recording LO_k＝(y_max，y_min，x_min，x_maxK) and storing in a matrix LO, wherein the initial value of k is 1, and the value of k is added with 1 after the rectangular frame of the motion estimation area is constructed each time;

step 7: repeating the steps 2 to 6 until the straight line x ═ h moves to the rightmost side of the binary image, and ending;

step4, extracting the corresponding part image of each moving object estimation rectangular frame in the gray level image and the color image of the current frame to obtain K moving object estimation gray level images G₁，...，G_KAnd K moving objects estimation color map P₁，…，P_K(ii) a The extraction process is as follows: for the ith moving object estimation rectangular frame, reading the corresponding position information LO from the matrix LO obtained in step3_i＝(y_max，y_min，x_min，x_maxI), confirming the specific position of the moving object estimation rectangular frame by using the upper edge, the lower edge, the left edge and the right edge so as to extract;

step5, using Canny operator to estimate gray level G of moving object one by one₁To G_KPerforming edge detection to obtain K edge images, comparing all edge pixels in each edge image with pixels at corresponding positions in the moving object image, and deleting the edge pixels at the positions corresponding to the moving object imageObtaining final edge images of the moving objects by edge pixels with unequal pixel values; constructing a moving object rectangular frame in each moving object edge image to contain all edges in the image, and obtaining K moving object rectangular frames M₁，…，M_K；

The process of constructing the rectangular frame of the moving object is as follows: firstly, traversing the moving object edge map from bottom to top to obtain the leftmost edge pixel X of each line_le＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lmAnd the abscissa X of the rightmost edge pixel_re＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rmIn which x_liRepresents the abscissa, x, of the leftmost edge pixel of the ith row_riThe abscissa of the rightmost edge pixel of the ith row is represented, and m represents the number of rows of the edge map of the moving object; then, traversing the moving object edge image from left to right to obtain the uppermost edge pixel Y of each column_he＝{y_h1，y_h2，y_h3，y_h4，y_h5，...，y_hnAnd the ordinate Y of the lowermost edge pixel_le＝{y_l1，y_l2，y_l3，y_l4，y_l5，...，y_lnIn which y_hiRepresenting the ordinate, y, of the uppermost edge pixel of the ith column_liRepresenting the ordinate of the lowermost edge pixel of the ith column, and n represents the edge image column number of the moving object; finally, with X_leWith the smallest element in (1) as the left edge, X_reMiddle largest element as right edge, Y_heWith the largest element in (3) as the upper edge, Y_leThe minimum element in the middle is used as a lower edge to construct a rectangular frame of the moving object;

step6, according to the human body proportion, the rectangular frame M of the moving object is processed₁To M_KJudging to obtain human body rectangular frames, if k human body rectangular frames are judged, extracting the corresponding part of each human body rectangular frame in the moving object estimation color image corresponding to the human body rectangular frameObtaining k human body color images;

recording the length of a rectangular frame of the moving object as L, the width as W and the lambda as W/L, setting thresholds α and β, if the length of the rectangular frame of the moving object is more than α and the lambda is less than β, regarding the rectangular frame as a rectangular frame of the human body, otherwise, regarding the rectangular frame as other objects;

step7, for each human body rectangular frame, positioning the head of a person in the human body rectangular frame according to the upper body characteristics of the human body, and positioning the safety helmet estimation areas according to the head proportion to obtain k safety helmet estimation areas;

the head positioning process is as follows: firstly, a rectangular frame of a human body is cut to obtain a head area, which specifically comprises the following steps: taking the upper edge of the human body rectangular frame as the upper edge of the head area, taking the positions of the upper edge and the lower edge 1/3 of the human body rectangular frame close to the upper edge as the lower edge of the head area, and taking the left edge and the right edge of the human body rectangular frame as the left edge and the right edge of the head area; second, traverse the head region from bottom to top to obtain the abscissa, X, of the leftmost and rightmost edge points of each row_ld＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lgAnd X_rd＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rgIn which x_liRepresents the abscissa, x, of the leftmost edge point of the ith row_riRepresenting the coordinate of the rightmost edge point of the ith row, and g representing the row number of the head area; then, the distance D ═ X from the leftmost edge to the rightmost edge is calculated_rd-X_ldObtaining D ═ D₁，D₂，D₃，D₄，D₅，…，D_gD, calculating the difference D between the next item and the previous item of the element in D_Δ＝{D₂-D₁，D₃-D₂，D₄-D₃，D₅-D₄，…，D_g-D_g-1}; finally find D_ΔUpdating the lower edge of the head region to be the p-th line from the 11 th element to the line number p corresponding to the largest element in the last element;

the process of constructing the rectangular frame of the safety helmet comprises the following steps: respectively find out X_ldFrom x_lpTo x_lgSmallest element x_mlAnd X_rdFrom x_rpTo x_rgMaximum element x of_mrX is equal to x_mlAs the precise head region left edge, x ═ x_mrAs a right edge, taking the upper edge and the lower edge of the head region as an upper edge and a lower edge of a precise head to obtain a precise head region; then, the precise head area is further segmented to obtain a safety helmet area, which is as follows: the upper, left and right edges of the helmet region coincide with the precision head region, with the positions of the precision head region upper and lower edges 1/3 near the upper edge as the lower edge of the helmet region;

step 8, extracting the image of the rectangular frame of the safety helmet at the corresponding position of the colorful picture of the human body to obtain a colorful picture of a safety helmet area, and judging the colorful picture of the safety helmet area one by one according to the color;

the judgment process is as follows: the color map of the safety helmet area is converted into an HSV image, and as the safety helmet in a common factory has red, blue, white and yellow, the red, blue, white and yellow are taken as judgment standards, and the value ranges of hue H, saturation S and lightness V corresponding to the four colors are as follows:

red: h:0-10 and 156-180; s, 43-255; v46-255

Blue color: h100-124; s, 43-255; v46-255

White: h is 0 to 180; s is 0 to 30; v221-

Yellow: 26-34 parts of H; s, 43-255; v46-255

The color of each pixel is classified as follows: if the H, S and V values of the pixel to be classified all meet the value range of a certain color, judging that the color of the pixel belongs to the color; after the classification is finished, calculating the proportion of the red, blue, white and yellow pixels to all the pixels to obtain a red proportion T_rBlue color ratio T_bWhite ratio ofExample T_wAnd yellow ratio T_y(ii) a Setting a threshold T₁If T is_r、T_b、T_w、T_yIn which there is a value exceeding T₁If yes, the safety helmet is judged to be worn; if T is_r、T_b、T_w、T_yAre all less than T₁And judging that the safety helmet is not worn.

The license plate recognition module judges whether the module vehicle enters an unauthorized area or not according to the following steps:

extracting a current frame, and performing graying and gray stretching treatment on the current frame by using an average value method to obtain a gray image of the current frame; performing edge extraction on the gray image of the current frame by using a Canny operator to obtain an edge image; performing four times of expansion, four times of corrosion and median filtering on the edge map to eliminate noise; determining a straight line in the edge image by using Hough transformation, determining the position of the license plate according to the proportion characteristics of the upper and lower parallels, the left and right parallels and the length and width of the license plate, and constructing a license plate recognition area; projecting the image of the license plate recognition area in the vertical direction, and segmenting according to the characteristics of the peak value group to obtain each character image; identifying each character image by using a BP neural network to obtain each character so as to identify the license plate number; after the license plate number is identified, the database is searched to determine the authorization area corresponding to the license plate, the authorization area is compared with the area where the current camera is located, and if the area where the camera is located does not belong to the license plate authorization area, warning is carried out.

The process of judging whether the employee work clothes are consistent with the work areas of the employee by the work clothes identification module is as follows:

reading video information acquired by a camera in a detection area in real time, extracting a current frame and a few frames before, and performing graying processing on the current frame and the few frames before to obtain grayscale images of the current frame and the few frames before; modeling, training and updating the gray level images of the previous frames by using a Gaussian mixture model to obtain a final model; judging each pixel point of the gray level image of the current frame by using the model to obtain a moving object image; constructing one or more moving object rectangular frames to extract all mutually independent parts in the moving object image, and if k mutually independent parts exist in the image, obtaining k moving object rectangular frames; confirming a human body rectangular frame in the object rectangular frame according to the length-width ratio of the human body, supposing that l human body rectangular frames are judged, extracting the image of each human body rectangular frame at the corresponding position of the current frame color picture to obtain l human body color pictures; for each human body rectangular frame, determining the position of the upper body according to the human body proportion, taking the position of the upper body as a working clothes estimation area, and extracting the image of the corresponding part of the working clothes estimation area in the corresponding human body color image to obtain l working clothes color images; judging the color image of each piece of work clothes according to the color, which comprises the following steps: converting the color image of the working clothes into an HSV image, classifying each pixel, if the H, S and V values of the pixel meet the value range of a certain color, classifying the pixel into the color, calculating the proportion of each color pixel to all pixels, setting a threshold value T to be 0.6, if the proportion of the certain color exceeds the threshold value T, judging the working clothes into the color, inquiring a database, obtaining the color of the working clothes in a working area corresponding to the position of the current camera, comparing the color with the judged color, and if the color is not consistent, warning; and if the proportion of all the colors does not exceed the threshold value T, judging that the staff does not wear the work clothes, and warning.

The process of judging whether the employee carries the package by the object carrying identification module is as follows:

reading video information acquired by a camera in a detection area in real time, extracting a current frame and previous frames, and carrying out gray processing on the current frame and the previous frames to obtain gray maps of the current frame and the previous frames; modeling, training and updating the gray level images of the previous frames by using a vibe algorithm to obtain a final model; judging each pixel point of the gray image of the current frame by using the model to obtain a motion area, and constructing a human body boundary rectangle containing all the motion areas; constructing an object estimation rectangle according to the human body proportion; judging whether the object is carried or not by taking the symmetry of the object in the object estimation rectangle as a standard; the method comprises the following specific steps:

step1, reading video information collected by a camera, and extracting previous frames and current frames; carrying out graying processing on each frame, and carrying out graying on the image by adopting a weighted average method: carrying out weighting operation on each pixel point, and distributing different weights to the R, G and B components, wherein f (x, y) is 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), and x and y refer to pixel point coordinates;

step2, carrying out modeling training and updating on the gray level images of the previous frames by using a vibe algorithm to obtain a final model;

step3, judging each pixel point of the gray image of the current frame by using the final model to obtain a motion area, and constructing a human body boundary rectangle containing all the motion areas;

step4, forming an object estimation rectangle according to the human body proportion, specifically as follows: the position 1/5, close to the upper edge, of the upper edge and the lower edge of the human body boundary rectangle is used as the upper edge of the object estimation rectangle, the position 2/5, close to the lower edge, between the upper edge and the lower edge of the human body boundary rectangle is used as the lower edge of the object estimation rectangle, and the left edge and the right edge of the human body boundary rectangle are respectively used as the left edge and the right edge of the object estimation rectangle to form the object estimation rectangle;

step5, judging whether the object is carried or not by taking the symmetry of the object estimation area as a standard, wherein the process is as follows: establishing a coordinate system by taking the lower left corner of the object estimation rectangle as an origin, wherein the object estimation rectangle is in the first quadrant, the object is taken to estimate the midpoint m of the object part at the upper edge of the rectangle, and the abscissa x of the object part is obtained_mX is equal to x_mCalculating the area of the object in the left region of the symmetry axis S1, the area of the object in the right region of the symmetry axis S2, and the degree of symmetry for the symmetry axisSetting threshold values th and tl, if tl is less than β and less than th, judging that the object is not carried, otherwise, judging that the object is carried.

The video fusion system is characterized in that the video monitoring system, the face recognition system and the video structural analysis system are fused to form a user-oriented video application system, a series of functions such as video viewing, video alarm prompting, video playback, alarm recording, user management, role management and authority management are realized in one software, and unified management of video application is facilitated.

The invention has the beneficial effects that: the system can monitor and analyze the working condition of a factory in real time, and can warn and check in time once dangerous conditions that the staff do not carry a safety helmet, the staff carry objects to enter an object carrying forbidding area, a vehicle enters an unauthorized area, the staff goes wrong with a working area, the temperature of a key area is too high and the like are found.

Drawings

Fig. 1 is a structural diagram of an intelligent monitoring and fusion system based on computer vision analysis according to the present invention.

Detailed Description

The structure of the intelligent monitoring fusion system based on computer vision analysis is shown in FIG. 1, and a video monitoring system consists of a front-end camera, a thermal infrared imager and a video monitoring platform; the front-end camera is responsible for collecting video information; the thermal infrared imager is responsible for acquiring temperature information of important areas of a plant area; the video monitoring platform receives video information collected by the front-end camera and temperature information of important areas of the plant, collected by the thermal infrared imager, displays the video information, transmits the video information to the video structural analysis system, and transmits the video information and the temperature information of the important areas of the plant to the video fusion system.

The face recognition system consists of a face snapshot device, a face recognition module and an employee identity database; the face snapshot device collects images of staff entering and leaving each key area of the factory area and transmits the images to the face recognition module; the employee identity database stores the identity information and facial features of each employee; the face recognition module receives the employee image collected by the face snapshot device, performs face recognition, determines identity information of the employee through comparison with the employee identity database, and transmits a recognition result to the video fusion system.

The video structured analysis system is composed of a safety helmet identification module, a license plate identification module, a work clothes identification module and an object carrying identification module, receives video information collected by a video monitoring system, judges whether a worker wears a safety helmet through the safety helmet identification module, judges whether a vehicle enters an unauthorized area through the license plate identification module, judges whether a work clothes of the worker is consistent with a work area through the work clothes identification module, judges whether the worker carries a package through the object carrying identification module, and transmits an analysis result to the video fusion system.

The video fusion system comprises a video alarm prompt module, a video playback module, an alarm recording module, a user management module, a role management module and a permission management module.

The process that the safety helmet identification module judges whether the staff wear the safety helmet is as follows:

step1, extracting a previous frame and a current frame, and performing graying processing on the previous frame and the current frame by adopting a weighted average method, namely distributing different weights to three components of R, G and B of each pixel point, wherein the specific formula is as follows: f (x, y) ═ 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), where R (x, y), G (x, y), B (x, y) refer to the R, G, B components of the pixel with coordinates (x, y), respectively, and f (x, y) refers to the gray-scale value after conversion of the pixel with coordinates (x, y);

step2, processing the gray level images of the previous frame and the current frame by using an interframe difference method to obtain a moving object image, wherein the process is as follows: subtracting the gray values of the pixels corresponding to the two frames of images, and taking the absolute value of the gray values to obtain a difference image D_n(x，y)＝|f_n(x，y)-f_n-1(x, y) |, wherein f_n(x, y) denotes a gray value of a pixel having coordinates (x, y) in the current frame, f_n-1(x, y) refers to the gray value of the pixel with the coordinate (x, y) in the previous frame; setting a threshold T_aIn this example, take T_aThe value being 1/10, i.e. T, of the current frame pixel_a＝f_n(x, y)/10; if D is_n(x，y)＞T，R_n255 if D_n(x，y)＜T，R_nObtaining a moving object graph R as 0; r is a binary image, wherein a white area represents a moving object area;

and 3, constructing k moving object estimation rectangular frames, extracting all mutually independent parts in the moving object image, acquiring the position information of each moving object estimation rectangular frame, and storing the position information in a matrix LO, wherein the specific process is as follows:

step 1: taking the lower left corner of the moving object image R obtained in the step2 as an origin, establishing a coordinate axis, wherein the moving object image R is in a first quadrant; setting a straight line x as h, wherein the initial value of h is 0;

step 2: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 3: if the pixel value of each pixel on the straight line x-h is 0, returning to Step 2; if a pixel having a pixel value other than 0 exists on the straight line x-h, let h be x_min；

Step 4: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 5: if the pixel with the pixel value not being 0 exists on the straight line x ═ h, recording the ordinate value corresponding to the uppermost non-0 pixel and the ordinate value corresponding to the lowermost non-0 pixel on the straight line, storing the ordinate values in the array Y, and returning to Step 4; if the pixel value of each pixel on the straight line x-h is 0, let h be x_max；

Step 6: comparing the sizes of the elements in the array Y to obtain the maximum value Y in the Y_maxAnd the minimum value y_min(ii) a With a straight line y ═ y_maxAs the upper edge, a straight line y ═ y_minAs the lower edge, a straight line x ═ x_minAs the left edge, the straight line x ═ x_maxConstructing a motion estimation area rectangular frame as a right edge; zero clearing of array Y and recording LO_k＝(y_max，y_min，x_min，x_maxK) and storing in a matrix LO, wherein the initial value of k is 1, and the value of k is added with 1 after the rectangular frame of the motion estimation area is constructed each time;

step 7: repeating the steps 2 to 6 until the straight line x ═ h moves to the rightmost side of the binary image, and ending;

step4, extracting the corresponding part image of each moving object estimation rectangular frame in the gray level image and the color image of the current frame to obtain K moving object estimation gray level images G₁，…，G_KAnd K moving objects estimation color map P₁，…，P_K(ii) a The extraction process is as follows: for the ith moving object estimation rectangular frame, reading the corresponding position information LO from the matrix LO obtained in step3_i＝(y_max，y_min，x_min，x_maxI), confirming the specific position of the moving object estimation rectangular frame by using the upper edge, the lower edge, the left edge and the right edge so as to extract;

step5, using Canny operator to estimate gray level G of moving object one by one₁To G_KPerforming edge detection to obtain K edge images, comparing all edge pixels in each edge image with pixels at corresponding positions in the moving object image, and deleting edge pixels with unequal pixel values at corresponding positions in the moving object image from the edge pixels to obtain a final moving object edge image; constructing a moving object rectangular frame in each moving object edge image to contain all edges in the image, and obtaining K moving object rectangular frames M₁，…，M_K；

The process of constructing the rectangular frame of the moving object is as follows: firstly, traversing the moving object edge map from bottom to top to obtain the leftmost edge pixel X of each line_le＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lmAnd the abscissa X of the rightmost edge pixel_re＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rmIn which x_liRepresents the leftmost side of the ith rowAbscissa, x, of the edge pixel_riThe abscissa of the rightmost edge pixel of the ith row is represented, and m represents the number of rows of the edge map of the moving object; then, traversing the moving object edge image from left to right to obtain the uppermost edge pixel Y of each column_he＝{y_h1，y_h2，y_h3，y_h4，y_h5，...，y_hnAnd the ordinate Y of the lowermost edge pixel_le＝{y_l1，y_l2，y_l3，y_l4，y_l5，…，y_lnIn which y_hiRepresenting the ordinate, y, of the uppermost edge pixel of the ith column_liRepresenting the ordinate of the lowermost edge pixel of the ith column, and n represents the edge image column number of the moving object; finally, with X_leWith the smallest element in (1) as the left edge, X_reMiddle largest element as right edge, Y_heWith the largest element in (3) as the upper edge, Y_leThe minimum element in the middle is used as a lower edge to construct a rectangular frame of the moving object;

step6, according to the human body proportion, the rectangular frame M of the moving object is processed₁To M_KJudging to obtain human body rectangular frames, supposing that k human body rectangular frames are judged, extracting images of corresponding parts of each human body rectangular frame in the moving object estimation color image corresponding to the human body rectangular frame to obtain k human body color images;

recording the length of a rectangular frame of the moving object as L, the width as W and lambda as W/L, setting thresholds α and β, wherein α and β respectively take 3 and 5 in the example, if α < lambda < β, the rectangular frame is regarded as a rectangular frame of the human body, otherwise, the rectangular frame is regarded as other objects;

step7, for each human body rectangular frame, positioning the head of a person in the human body rectangular frame according to the upper body characteristics of the human body, and positioning the safety helmet estimation areas according to the head proportion to obtain k safety helmet estimation areas;

the head positioning process is as follows: firstly, a rectangular frame of a human body is cut to obtain a head area, which specifically comprises the following steps: taking the upper edge of the rectangular frame of the human body as the headThe positions of the upper edge of the area, the upper edge of the human body rectangular frame and the position of the lower edge 1/3 close to the upper edge are used as the lower edge of the head area, and the left edge and the right edge of the human body rectangular frame are used as the left edge and the right edge of the head area; second, traverse the head region from bottom to top to obtain the abscissa, X, of the leftmost and rightmost edge points of each row_ld＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lgAnd X_rd＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rgIn which x_liRepresents the abscissa, x, of the leftmost edge point of the ith row_riRepresenting the coordinate of the rightmost edge point of the ith row, and g representing the row number of the head area; then, the distance D ═ X from the leftmost edge to the rightmost edge is calculated_rd-X_ldObtaining D ═ D₁，D₂，D₃，D₄，D₅，…，D_gD, calculating the difference D between the next item and the previous item of the element in D_Δ＝{D₂-D₁，D₃-D₂，D₄-D₃，D₅-D₄，…，D_g-D_g-1}; finally find D_ΔUpdating the lower edge of the head region to be the p-th line from the 11 th element to the line number p corresponding to the largest element in the last element;

the process of constructing the rectangular frame of the safety helmet comprises the following steps: respectively find out X_ldFrom x_lpTo x_lgSmallest element x_mlAnd X_rdFrom x_rpTo x_rgMaximum element x of_mrX is equal to x_mlAs the precise head region left edge, x ═ x_mrAs a right edge, taking the upper edge and the lower edge of the head region as an upper edge and a lower edge of a precise head to obtain a precise head region; then, the precise head area is further segmented to obtain a safety helmet area, which is as follows: the upper, left and right edges of the helmet region coincide with the precision head region, with the positions of the precision head region upper and lower edges 1/3 near the upper edge as the lower edge of the helmet region;

step 8, extracting the image of the rectangular frame of the safety helmet at the corresponding position of the colorful picture of the human body to obtain a colorful picture of a safety helmet area, and judging the colorful picture of the safety helmet area one by one according to the color;

the judgment process is as follows: the color map of the safety helmet area is converted into an HSV image, and as the safety helmet in a common factory has red, blue, white and yellow, the red, blue, white and yellow are taken as judgment standards, and the value ranges of hue H, saturation S and lightness V corresponding to the four colors are as follows:

red: h:0-10 and 156-180; s, 43-255; v46-255

Blue color: h100-124; s, 43-255; v46-255

White: h is 0 to 180; s is 0 to 30; v221-

Yellow: 26-34 parts of H; s, 43-255; v46-255

The color of each pixel is classified as follows: if the H, S and V values of the pixel to be classified all meet the value range of a certain color, judging that the color of the pixel belongs to the color; after the classification is finished, calculating the proportion of the red, blue, white and yellow pixels to all the pixels to obtain a red proportion T_rBlue color ratio T_bWhite ratio T_wAnd yellow ratio T_y(ii) a Setting a threshold T₁Example T₁Taking 0.5; if T is_r、T_b、T_w、T_yIn which there is a value exceeding T₁If yes, the safety helmet is judged to be worn; if T is_r、T_b、T_w、T_yAre all less than T₁And judging that the safety helmet is not worn.

The process of judging whether the employee carries the package by the object carrying identification module is as follows:

step1, reading video information collected by a camera, and extracting previous frames and current frames; carrying out graying processing on each frame, and carrying out graying on the image by adopting a weighted average method: carrying out weighting operation on each pixel point, and distributing different weights to the R, G and B components, wherein f (x, y) is 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), and x and y refer to pixel point coordinates;

step2, carrying out modeling training and updating on the gray level images of the previous frames by using a vibe algorithm to obtain a final model;

step3, judging each pixel point of the gray image of the current frame by using the final model to obtain a motion area, and constructing a human body boundary rectangle containing all the motion areas;

step4, forming an object estimation rectangle according to the human body proportion, specifically as follows: the position 1/5, close to the upper edge, of the upper edge and the lower edge of the human body boundary rectangle is used as the upper edge of the object estimation rectangle, the position 2/5, close to the lower edge, between the upper edge and the lower edge of the human body boundary rectangle is used as the lower edge of the object estimation rectangle, and the left edge and the right edge of the human body boundary rectangle are respectively used as the left edge and the right edge of the object estimation rectangle to form the object estimation rectangle;

step5, judging whether the object is carried or not by taking the symmetry of the object estimation area as a standard, wherein the process is as follows: establishing a coordinate system by taking the lower left corner of the object estimation rectangle as an origin, wherein the object estimation rectangle is in the first quadrant, the object is taken to estimate the midpoint m of the object part at the upper edge of the rectangle, and the abscissa x of the object part is obtained_mX is equal to x_mCalculating the area of the object in the left region of the symmetry axis S1, the area of the object in the right region of the symmetry axis S2, and the degree of symmetry for the symmetry axisSetting threshold values th and tl, wherein th takes a value of 0.75 and tl takes a value of 1.33, if tl is less than β and less than th, judging that the object is not carried, otherwise, judging that the object is carried.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be conceived by those skilled in the art within the technical scope of the present invention will be covered by the scope of the present invention.

Claims (7)

1. An intelligent monitoring and fusion system based on computer vision analysis is characterized by comprising a video monitoring system, a face recognition system, a video structural analysis system and a video fusion system; wherein,

the video monitoring system consists of a front-end camera, a thermal infrared imager and a video monitoring platform; the front-end camera is responsible for collecting video information; the thermal infrared imager is responsible for acquiring temperature information of important areas of a plant area; the video monitoring platform receives video information acquired by a front-end camera and temperature information of important areas of a plant acquired by a thermal infrared imager, displays the video information, transmits the video information to the video structural analysis system, and transmits the video information and the temperature information of the important areas of the plant to the video fusion system;

the face recognition system consists of a face snapshot device, a face recognition module and an employee identity database; the face snapshot device collects images of staff entering and leaving each key area of the factory area and transmits the images to the face recognition module; the employee identity database stores the identity information and facial features of each employee; the face recognition module receives the employee image collected by the face snapshot device, performs face recognition, determines identity information of the employee through comparison with an employee identity database, and transmits a recognition result to the video fusion system;

the video structured analysis system is composed of a safety helmet identification module, a license plate identification module, a work clothes identification module and an object carrying identification module, receives video information collected by a video monitoring system, judges whether a worker wears a safety helmet through the safety helmet identification module, judges whether a vehicle enters an unauthorized area through the license plate identification judgment module, judges whether a worker work clothes is consistent with a work area of the worker through the work clothes identification module, judges whether the worker carries a package through the object carrying identification module, and transmits an analysis result to the video fusion system;

the video fusion system comprises a video alarm prompt module, a video playback module, an alarm recording module, a user management module, a role management module and a permission management module.

2. An intelligent monitoring fusion system based on computer vision analysis as claimed in claim 1, wherein the face feature of each employee stored in the employee identity database in the face recognition system is calculated as follows:

acquiring a face image of each employee in the factory, wherein the size of the image is MxN, and K face images can be obtained if K employees are in common; carrying out gray linear transformation processing and 3 x 3 median filtering processing on the K face images to obtain K face gray images; the gray value of each line of pixels of each face gray image is comparedAnd if the face gray level image is not in the original image, and the face gray level image is not in the original image, so that the face gray level image is not in the original image_iAnd calculating an average faceArranging K row vectors formed by K face gray level images to form a K multiplied by D dimensional matrix, and performing K-L transformation on the matrix to obtain a characteristic face space: w ═ u₁，u₂，u₃，…，u_p) In the formula: p is the set dimensionality after dimensionality reduction; calculating the facial feature of each employee, and for the jth employee, calculating the facial feature omega of each employee_jThe calculation formula of (2) is as follows: omega_j＝w^T(x_j-Ψ)。

3. The intelligent monitoring fusion system based on computer vision analysis as claimed in claim 1, wherein the face recognition module in the face recognition system performs face recognition by the following steps:

acquiring a face image captured by a face capturing device and entering and exiting each key area of a factory area, performing gray linear transformation on the face image to obtain a face gray image, and connecting gray values of pixels of each line of the face gray image to obtain a line vector y; calculating the facial feature omega of the face image_j＝w^T(x_j- Ψ), wherein: w is the characteristic face space determined in claim 2: w ═ u₁，u₂，u₃，…，u_p) Ψ represents an average face obtained in claim 2; calculate Ω_ySelecting the Euclidean distance from the facial features stored in the employee identity database to omega_yAnd the staff identity corresponding to the facial feature is the face recognition result.

4. The intelligent monitoring fusion system based on computer vision analysis as claimed in claim 1, wherein the process of the helmet identification module in the video structural analysis system determining whether the employee has worn the helmet is as follows:

reading video information collected by a camera in a detection area in real time, extracting a previous frame and a current frame, and carrying out gray processing on the previous frame and the current frame to obtain gray images of the previous frame and the current frame; processing the gray level images of the previous frame and the current frame by using an interframe difference method to obtain a moving object image; constructing one or more moving object estimation rectangular frames to extract all mutually independent parts in the moving object image, and if K mutually independent parts exist in the image, obtaining K moving object estimation rectangular frames; extracting the corresponding part of the gray level image and the color image of each moving object estimation rectangular frame in the current frame to obtain K moving object estimation gray level images and K moving object estimation color images; carrying out edge detection on the K moving object estimated gray level images one by using a Canny operator to obtain K moving object edge images, and constructing a moving object rectangular frame in each moving object edge image to contain all edges in the image to obtain K moving object rectangular frames; confirming a human body rectangular frame in the moving object rectangular frame according to the length-width ratio of the human body, supposing that k human body rectangular frames are judged, extracting the image of the corresponding part of each human body rectangular frame in the moving object estimated color image to obtain k human body color images; for each human body rectangular frame, positioning the head of a person in the human body rectangular frame according to the upper half-body characteristics of the human body, and positioning the safety helmet estimation areas according to the head proportion to obtain k safety helmet estimation areas; extracting the image of the corresponding part of each safety helmet estimation area in the corresponding human body color image to obtain k safety helmet area color images, and judging the safety helmet area color images one by one according to the color; the method comprises the following specific steps:

step1, extracting a previous frame and a current frame, and performing graying processing on the previous frame and the current frame by adopting a weighted average method, namely distributing different weights to three components of R, G and B of each pixel point, wherein the specific formula is as follows: f (x, y) ═ 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), where R (x, y), G (x, y), B (x, y) refer to the R, G, B components of the pixel with coordinates (x, y), respectively, and f (x, y) refers to the gray-scale value after conversion of the pixel with coordinates (x, y);

step2, using the frameThe inter-difference method processes the gray level images of the previous frame and the current frame to obtain a moving object image, and the process is as follows: subtracting the gray values of the pixels corresponding to the two frames of images, and taking the absolute value of the gray values to obtain a difference image D_n(x，y)＝|f_n(x，y)-f_n-1(x, y) |, wherein f_n(x, y) denotes a gray value of a pixel having coordinates (x, y) in the current frame, f_n-1(x, y) refers to the gray value of the pixel with the coordinate (x, y) in the previous frame; setting a threshold T_aIf D is_n(x，y)＞T，R_n255 if D_n(x，y)＜T，R_nObtaining a moving object graph R as 0; r is a binary image, wherein a white area represents a moving object area;

and 3, constructing K moving object estimation rectangular frames, extracting all mutually independent parts in the moving object image, acquiring the position information of each moving object estimation rectangular frame, and storing the position information in a matrix LO, wherein the specific process is as follows:

step 1: taking the lower left corner of the moving object image R obtained in the step2 as an origin, establishing a coordinate axis, wherein the moving object image R is in a first quadrant; setting a straight line x as h, wherein the initial value of h is 0;

step 2: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 3: if the pixel value of each pixel on the straight line x-h is 0, returning to Step 2; if a pixel having a pixel value other than 0 exists on the straight line x-h, let h be x_min；

Step 4: shifting the straight line x h by one unit to the right, and judging the pixel value of each pixel on the straight line x h;

step 5: if the pixel with the pixel value not being 0 exists on the straight line x ═ h, recording the ordinate value corresponding to the uppermost non-0 pixel and the ordinate value corresponding to the lowermost non-0 pixel on the straight line, storing the ordinate values in the array Y, and returning to Step 4; if the pixel value of each pixel on the straight line x-h is 0, let h be x_max；

Step 6: comparing the sizes of the elements in the array Y to obtain the maximum value Y in the Y_maxAnd the minimum value y_min(ii) a With a straight line y ═ y_maxAs upper edge, straight line y＝y_minAs the lower edge, a straight line x ═ x_minAs the left edge, the straight line x ═ x_maxConstructing a motion estimation area rectangular frame as a right edge; zero clearing of array Y and recording LO_k＝(y_max，y_min，x_min，x_maxK) and storing in a matrix LO, wherein the initial value of k is 1, and the value of k is added with 1 after the rectangular frame of the motion estimation area is constructed each time;

step 7: repeating the steps 2 to 6 until the straight line x ═ h moves to the rightmost side of the binary image, and ending;

step4, extracting the corresponding part image of each moving object estimation rectangular frame in the gray level image and the color image of the current frame to obtain K moving object estimation gray level images G₁，…，G_KAnd K moving objects estimation color map P₁，…，P_K(ii) a The extraction process is as follows: for the ith moving object estimation rectangular frame, reading the corresponding position information LO from the matrix LO obtained in step3_i＝(y_max，y_min，x_min，x_maxI), confirming the specific position of the moving object estimation rectangular frame by using the upper edge, the lower edge, the left edge and the right edge so as to extract;

step5, using Canny operator to estimate gray level G of moving object one by one₁To G_KPerforming edge detection to obtain K edge images, comparing all edge pixels in each edge image with pixels at corresponding positions in the moving object image, and deleting edge pixels with unequal pixel values at corresponding positions in the moving object image from the edge pixels to obtain a final moving object edge image; constructing a moving object rectangular frame in each moving object edge image to contain all edges in the image, and obtaining K moving object rectangular frames M₁，…，M_K；

The process of constructing the rectangular frame of the moving object is as follows: firstly, traversing the moving object edge map from bottom to top to obtain the leftmost edge pixel X of each line_le＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lmAnd the abscissa X of the rightmost edge pixel_re＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rmIn which x_liRepresents the abscissa, x, of the leftmost edge pixel of the ith row_riThe abscissa of the rightmost edge pixel of the ith row is represented, and m represents the number of rows of the edge map of the moving object; then, traversing the moving object edge image from left to right to obtain the uppermost edge pixel Y of each column_he＝{y_h1，y_h2，y_h3，y_h4，y_h5，...，y_hnAnd the ordinate Y of the lowermost edge pixel_le＝{y_l1，y_l2，y_l3，y_l4，y_l5，...，y_lnIn which y_hiRepresenting the ordinate, y, of the uppermost edge pixel of the ith column_liRepresenting the ordinate of the lowermost edge pixel of the ith column, and n represents the edge image column number of the moving object; finally, with X_leWith the smallest element in (1) as the left edge, X_reMiddle largest element as right edge, Y_heWith the largest element in (3) as the upper edge, Y_leThe minimum element in the middle is used as a lower edge to construct a rectangular frame of the moving object;

step6, according to the human body proportion, the rectangular frame M of the moving object is processed₁To M_KJudging to obtain human body rectangular frames, supposing that k human body rectangular frames are judged, extracting images of corresponding parts of each human body rectangular frame in the moving object estimation color image corresponding to the human body rectangular frame to obtain k human body color images;

recording the length of a rectangular frame of the moving object as L, the width as W and the lambda as W/L, setting thresholds α and β, if the length of the rectangular frame of the moving object is more than α and the lambda is less than β, regarding the rectangular frame as a rectangular frame of the human body, otherwise, regarding the rectangular frame as other objects;

step7, for each human body rectangular frame, positioning the head of a person in the human body rectangular frame according to the upper body characteristics of the human body, and positioning the safety helmet estimation areas according to the head proportion to obtain k safety helmet estimation areas;

the head positioning process is as follows: firstly, a rectangular frame of a human body is cut to obtain a head area, which specifically comprises the following steps: get human body rectangleThe upper edge of the frame is used as the upper edge of the head area, the positions of the upper edge and the lower edge 1/3 of the human body rectangular frame, which are close to the upper edge, are used as the lower edge of the head area, and the left edge and the right edge of the human body rectangular frame are used as the left edge and the right edge of the head area; second, traverse the head region from bottom to top to obtain the abscissa, X, of the leftmost and rightmost edge points of each row_ld＝{x_l1，x_l2，x_l3，x_l4，x_l5，…，x_lgAnd X_rd＝{x_r1，x_r2，x_r3，x_r4，x_r5，…，x_rgIn which x_liRepresents the abscissa, x, of the leftmost edge point of the ith row_riRepresenting the coordinate of the rightmost edge point of the ith row, and g representing the row number of the head area; then, the distance D ═ X from the leftmost edge to the rightmost edge is calculated_rd-X_ldObtaining D ═ D₁，D₂，D₃，D₄，D₅，…，D_gD, calculating the difference D between the next item and the previous item of the element in D_Δ＝{D₂-D₁，D₃-D₂，D₄-D₃，D₅-D₄，…，D_g-D_g-1}; finally find D_ΔUpdating the lower edge of the head region to be the p-th line from the 11 th element to the line number p corresponding to the largest element in the last element;

the process of constructing the rectangular frame of the safety helmet comprises the following steps: respectively find out X_ldFrom x_lpTo x_lgSmallest element x_mlAnd X_rdFrom x_rpTo x_rgMaximum element x of_mrX is equal to x_mlAs the precise head region left edge, x ═ x_mrAs a right edge, taking the upper edge and the lower edge of the head region as an upper edge and a lower edge of a precise head to obtain a precise head region; then, the precise head area is further segmented to obtain a safety helmet area, which is as follows: the upper, left and right edges of the helmet region coincide with the precision head region, with the positions of the precision head region upper and lower edges 1/3 near the upper edge as the lower edge of the helmet region;

step 8, extracting the image of the rectangular frame of the safety helmet at the corresponding position of the colorful picture of the human body to obtain a colorful picture of a safety helmet area, and judging the colorful picture of the safety helmet area one by one according to the color;

the judgment process is as follows: the color map of the safety helmet area is converted into an HSV image, and as the safety helmet in a common factory has red, blue, white and yellow, the red, blue, white and yellow are taken as judgment standards, and the value ranges of hue H, saturation S and lightness V corresponding to the four colors are as follows:

red: h:0-10 and 156-180; s, 43-255; v46-255

Blue color: h100-124; s, 43-255; v46-255

White: h is 0 to 180; s is 0 to 30; v221-

Yellow: 26-34 parts of H; s, 43-255; v46-255

The color of each pixel is classified as follows: if the H, S and V values of the pixel to be classified all meet the value range of a certain color, judging that the color of the pixel belongs to the color; after the classification is finished, calculating the proportion of the red, blue, white and yellow pixels to all the pixels to obtain a red proportion T_rBlue color ratio T_bWhite ratio T_wAnd yellow ratio T_y(ii) a Setting a threshold T₁If T is_r、T_b、T_w、T_yIn which there is a value exceeding T₁If yes, the safety helmet is judged to be worn; if T is_r、T_b、T_w、T_yAre all less than T₁And judging that the safety helmet is not worn.

5. An intelligent monitoring and fusion system based on computer vision analysis as claimed in claim 1, wherein the step of judging whether the module vehicle enters the unauthorized area by license plate recognition in the video structural analysis system is:

extracting a current frame, and performing graying and gray stretching treatment on the current frame by using an average value method to obtain a gray image of the current frame; performing edge extraction on the gray image of the current frame by using a Canny operator to obtain an edge image; performing four times of expansion, four times of corrosion and median filtering on the edge map to eliminate noise; determining a straight line in the edge image by using Hough transformation, determining the position of the license plate according to the proportion characteristics of the upper and lower parallels, the left and right parallels and the length and width of the license plate, and constructing a license plate recognition area; projecting the image of the license plate recognition area in the vertical direction, and segmenting according to the characteristics of the peak value group to obtain each character image; identifying each character image by using a BP neural network to obtain each character so as to identify the license plate number; after the license plate number is identified, the database is searched to determine the authorization area corresponding to the license plate, the authorization area is compared with the area where the current camera is located, and if the area where the camera is located does not belong to the license plate authorization area, warning is carried out.

6. The intelligent monitoring fusion system based on computer vision analysis as claimed in claim 1, wherein the step of the staff work clothes recognition module in the video structured analysis system judging whether the staff work clothes are consistent with the work area is as follows:

reading video information acquired by a camera in a detection area in real time, extracting a current frame and a few frames before, and performing graying processing on the current frame and the few frames before to obtain grayscale images of the current frame and the few frames before; modeling, training and updating the gray level images of the previous frames by using a Gaussian mixture model to obtain a final model; judging each pixel point of the gray level image of the current frame by using the model to obtain a moving object image; constructing one or more moving object rectangular frames to extract all mutually independent parts in the moving object image, and if k mutually independent parts exist in the image, obtaining k moving object rectangular frames; confirming a human body rectangular frame in the object rectangular frame according to the length-width ratio of the human body, supposing that l human body rectangular frames are judged, extracting the image of each human body rectangular frame at the corresponding position of the current frame color picture to obtain l human body color pictures; for each human body rectangular frame, determining the position of the upper body according to the human body proportion, taking the position of the upper body as a working clothes estimation area, and extracting the image of the corresponding part of the working clothes estimation area in the corresponding human body color image to obtain l working clothes color images; judging the color image of each piece of work clothes according to the color, which comprises the following steps: converting the color image of the working clothes into an HSV image, classifying each pixel, if the H, S and V values of the pixel meet the value range of a certain color, classifying the pixel into the color, calculating the proportion of each color pixel to all pixels, setting a threshold value T to be 0.6, if the proportion of the certain color exceeds the threshold value T, judging the working clothes into the color, inquiring a database, obtaining the color of the working clothes in a working area corresponding to the position of the current camera, comparing the color with the judged color, and if the color is not consistent, warning; and if the proportion of all the colors does not exceed the threshold value T, judging that the staff does not wear the work clothes, and warning.

7. The intelligent monitoring and fusion system based on computer vision analysis as claimed in claim 1, wherein the step of judging whether the employee carries the package by the object carrying identification module in the video structural analysis system is as follows:

reading video information acquired by a camera in a detection area in real time, extracting a current frame and previous frames, and carrying out gray processing on the current frame and the previous frames to obtain gray maps of the current frame and the previous frames; modeling, training and updating the gray level images of the previous frames by using a vibe algorithm to obtain a final model; judging each pixel point of the gray image of the current frame by using the model to obtain a motion area, and constructing a human body boundary rectangle containing all the motion areas; constructing an object estimation rectangle according to the human body proportion; judging whether the object is carried or not by taking the symmetry of the object in the object estimation rectangle as a standard; the method comprises the following specific steps:

step1, reading video information collected by a camera, and extracting previous frames and current frames; carrying out graying processing on each frame, and carrying out graying on the image by adopting a weighted average method: carrying out weighting operation on each pixel point, and distributing different weights to the R, G and B components, wherein f (x, y) is 0.3R (x, y) +0.59G (x, y) +0.11B (x, y), and x and y refer to pixel point coordinates;

step2, carrying out modeling training and updating on the gray level images of the previous frames by using a vibe algorithm to obtain a final model;

step3, judging each pixel point of the gray image of the current frame by using the final model to obtain a motion area, and constructing a human body boundary rectangle containing all the motion areas;

step4, forming an object estimation rectangle according to the human body proportion, specifically as follows: the position 1/5, close to the upper edge, of the upper edge and the lower edge of the human body boundary rectangle is used as the upper edge of the object estimation rectangle, the position 2/5, close to the lower edge, between the upper edge and the lower edge of the human body boundary rectangle is used as the lower edge of the object estimation rectangle, and the left edge and the right edge of the human body boundary rectangle are respectively used as the left edge and the right edge of the object estimation rectangle to form the object estimation rectangle;

step5, judging whether the object is carried or not by taking the symmetry of the object estimation area as a standard, wherein the process is as follows: establishing a coordinate system by taking the lower left corner of the object estimation rectangle as an origin, wherein the object estimation rectangle is in the first quadrant, the object is taken to estimate the midpoint m of the object part at the upper edge of the rectangle, and the abscissa x of the object part is obtained_mX is equal to x_mCalculating the area of the object in the left region of the symmetry axis S1, the area of the object in the right region of the symmetry axis S2, and the degree of symmetry for the symmetry axisSetting threshold values th and tl, if tl is less than β and less than th, judging that the object is not carried, otherwise, judging that the object is carried.