CN112966593A - Enterprise safety standardized operation method and system based on artificial intelligence and big data - Google Patents
Enterprise safety standardized operation method and system based on artificial intelligence and big data Download PDFInfo
- Publication number
- CN112966593A CN112966593A CN202110236578.7A CN202110236578A CN112966593A CN 112966593 A CN112966593 A CN 112966593A CN 202110236578 A CN202110236578 A CN 202110236578A CN 112966593 A CN112966593 A CN 112966593A
- Authority
- CN
- China
- Prior art keywords
- action
- acquiring
- matrix
- change
- action change
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 26
- 230000008859 change Effects 0.000 claims abstract description 166
- 230000009471 action Effects 0.000 claims abstract description 162
- 239000011159 matrix material Substances 0.000 claims abstract description 102
- 230000000977 initiatory effect Effects 0.000 claims abstract description 28
- 238000010586 diagram Methods 0.000 claims abstract description 13
- 238000012216 screening Methods 0.000 claims abstract description 7
- 210000003414 extremity Anatomy 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 8
- 238000009826 distribution Methods 0.000 claims description 8
- 210000003127 knee Anatomy 0.000 claims description 8
- 230000002159 abnormal effect Effects 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 5
- 230000007613 environmental effect Effects 0.000 abstract description 5
- 230000000875 corresponding effect Effects 0.000 description 13
- 230000033001 locomotion Effects 0.000 description 8
- 210000002683 foot Anatomy 0.000 description 7
- 238000012549 training Methods 0.000 description 4
- 230000005856 abnormality Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000000544 articulatio talocruralis Anatomy 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 210000000629 knee joint Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 230000003313 weakening effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/28—Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/42—Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Multimedia (AREA)
- Tourism & Hospitality (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Primary Health Care (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Psychiatry (AREA)
- Manufacturing & Machinery (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Computer Security & Cryptography (AREA)
- Development Economics (AREA)
- Educational Administration (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an enterprise safety standardized operation method and system based on artificial intelligence and big data, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring a reference action type of an operator according to the scene image and the depth image, constructing an action change diagram, and acquiring an initial action change matrix; extracting a background area of the scene image, dividing the background area into a plurality of sub-areas, and establishing a bipartite graph between the sub-areas and behavior change information of operators; screening out an influence action change area and a secondary environment area, correcting the bipartite graph to obtain an environment influence adjacency matrix, and then obtaining an optimized action change matrix by combining the initial action change matrix; and then, judging whether the operation abnormity occurs to the operator or not by combining the standard action change matrix. Therefore, interference caused by environmental factors is eliminated according to the environment change condition and the relation between the environment change condition and the behavior and the action of the operating personnel, and the judgment accuracy of operation safety standardization is improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to an enterprise safety standardized operation method and system based on artificial intelligence and big data.
Background
In industrial safety production management, the safety of field workers is particularly important, but due to reasons of improper management, poor safety consciousness and the like, the workers often break rules and regulations in the actual operation process, and safety accidents caused by the rules and regulations are also frequent. Normally, operators have a lucky psychology, and can not find operation problems in real time by detecting whether the operators violate operations through managers.
In the conventional image monitoring technology, actually, with changes in environments such as different scenes and updating of operation equipment, the operation mode is affected to a certain degree, the safety operation standard is changed to a certain degree, and the operator can cast shadows in the surrounding environment, which greatly affects the existing image analysis and processing, thereby affecting the accuracy of behavior determination of the operator, and causing false detection and false alarm.
Disclosure of Invention
In order to solve the technical problems, the invention aims to provide an enterprise safety standardized operation method and system based on artificial intelligence and big data, and the adopted technical scheme is as follows:
in one aspect, an embodiment of the present invention provides an enterprise security standardized operation method based on artificial intelligence and big data, including the following steps:
acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to a time sequence mark, extracting key points of operators, and acquiring three-dimensional coordinates of the key points by using the depth images;
when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold, taking a scene image corresponding to the current moment as a converted image, and inputting the key points contained in the converted image into a full-connection network to obtain a reference action category;
acquiring a side weight value and a direction according to the time sequence mark, constructing an action change diagram for the reference action category, and acquiring an initial action change matrix;
acquiring a background area of the transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among reference action types based on the initial action change matrix, marking and distinguishing to obtain change information; constructing a bipartite graph between the sub-regions and the change information by taking the similarity degree of pixel values in the same sub-region between the transformed images as a side weight value;
screening out an influence action change area and a secondary environment area for the sub-area based on the optimal distribution and combining with the edge weight of the bipartite graph, correcting the bipartite graph to obtain a corrected environment influence graph, and acquiring an environment influence adjacency matrix;
acquiring a first feature tensor of the initial action change matrix and a second feature tensor of the environment influence adjacent matrix, and inputting the first feature tensor and the second feature tensor into a neural network to acquire an optimized action change matrix;
and acquiring the Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation of the operator is abnormal or not, and generating prompt information.
Preferably, the key points of the operator are divided according to the trunk and the limbs; the trunk part comprises a shoulder key point, a hip key point and a trunk center point, and the limb part comprises an elbow key point, a hand key point, a knee key point and a foot key point.
Preferably, the three-dimensional coordinate variation of the key points is a vector value of angular variation of a degree of freedom preset between the key points in a three-dimensional coordinate system.
Preferably, the method for obtaining the edge weight and the direction according to the time sequence mark and constructing the action change diagram for the reference action category includes:
connecting the vertex of the current moment to the vertex of the next moment according to a time sequence by taking the reference action category as the vertex to form a directed edge, and taking the change duration between the two moments as an edge weight;
when the same two vertexes are repeatedly changed in the time sequence, the average value of the changing duration is obtained and used as the edge weight of the two vertexes.
Preferably, the reference action category includes a standard action category and other action categories; the standard action change matrix is obtained by utilizing a plurality of groups of standard action categories and the environment influence adjacency matrix.
In a second aspect, another embodiment of the present invention provides an enterprise security standardized operation system based on artificial intelligence and big data, including:
the image acquisition module is used for acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to the time sequence marks, extracting key points of operators and acquiring three-dimensional coordinates of the key points by using the depth images;
the reference action acquisition module is used for taking a scene image corresponding to the current moment as a transformation image and inputting the key points contained in the transformation image into a full-connection network to acquire a reference action type when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold;
the action change matrix acquisition module is used for acquiring a side weight and a direction according to the time sequence mark, constructing an action change diagram for the reference action category and acquiring an initial action change matrix;
the bipartite graph acquisition module is used for acquiring a background area of the transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among reference action types based on the initial action change matrix, marking and distinguishing to obtain change information; constructing a bipartite graph between the sub-regions and the change information by taking the similarity degree of pixel values in the same sub-region between the transformed images as a side weight value;
the environment influence matrix module is used for screening out an influence action change area and a secondary environment area for the sub-area based on optimal distribution and combined with the edge weight of the bipartite graph, correcting the bipartite graph to obtain a corrected environment influence graph and obtain an environment influence adjacency matrix;
the optimized action change matrix module is used for acquiring a first feature tensor of the initial action change matrix and a second feature tensor of the environment influence adjacent matrix, and inputting the first feature tensor and the second feature tensor into a neural network to acquire an optimized action change matrix;
and the abnormity determining module is used for acquiring the Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation abnormity occurs to the operator, and generating prompt information.
Preferably, the key points of the operator are divided according to the trunk and the limbs; the trunk part comprises a shoulder key point, a hip key point and a trunk center point, and the limb part comprises an elbow key point, a hand key point, a knee key point and a foot key point.
Preferably, the three-dimensional coordinate variation of the key points is a vector value of angular variation of a degree of freedom preset between the key points in a three-dimensional coordinate system.
Preferably, the action change matrix obtaining module includes:
the directed graph acquiring unit is used for connecting the vertex of the current moment to the vertex of the next moment according to a time sequence by taking the reference action category as the vertex to form a directed edge, and taking the change duration between the two moments as an edge weight;
and the weight value optimization unit is used for acquiring the mean value of the change duration as the edge weight value of the two vertexes when the two identical vertexes are repeatedly changed in the time sequence.
Preferably, the reference action category includes a standard action category and other action categories; the standard action change matrix is obtained by utilizing a plurality of groups of the standard action categories and the environment influence adjacency matrix.
The embodiment of the invention at least has the following beneficial effects:
the method comprises the steps of obtaining a reference action type of an operator by obtaining a continuous scene image and a depth image, constructing an action change diagram by combining a time sequence mark, and obtaining an initial action change matrix; then extracting a background area of the scene image, dividing the background area into a plurality of sub-areas, and establishing two graphs between the sub-areas and the behavior change information of the operators according to the pixel change between the fixed sub-areas; changing an area and a secondary environment area according to the influence action, correcting the bipartite graph to obtain a corrected environment influence graph, and acquiring an environment influence adjacency matrix; and acquiring an optimized action change matrix according to the environment influence adjacent matrix and the initial action change matrix, judging whether the operation of the operator is abnormal or not by combining the Euclidean distance between the prestored standard action change matrices, and generating prompt information. Therefore, the safety operation standard is dynamically generated according to the environment change condition and the relation between the environment change condition and the behavior and the action of the operator, the interference caused by the environmental factors is removed, and the judgment accuracy of the operation safety standardization is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions and advantages of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a flowchart of a method for enterprise security standardization operation based on artificial intelligence and big data according to an embodiment of the present invention;
fig. 2 is a block diagram of an enterprise security standardized operation system based on artificial intelligence and big data according to an embodiment of the present invention.
Detailed Description
In order to further explain the technical means and effects of the present invention adopted to achieve the predetermined objects, the following detailed description, with reference to the accompanying drawings and preferred embodiments, describes specific embodiments, structures, features and effects of an enterprise security standardized operation method and system based on artificial intelligence and big data according to the present invention. In the following description, different "one embodiment" or "another embodiment" refers to not necessarily the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
The following describes a specific scheme of the enterprise security standardization operation method and system based on artificial intelligence and big data in detail with reference to the accompanying drawings.
Referring to fig. 1, a flowchart of an enterprise security standardization operation method based on artificial intelligence and big data according to an embodiment of the present invention is shown.
The enterprise security standardization operation method based on artificial intelligence and big data has a flow chart shown in fig. 1, and comprises the following steps:
step 1: acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to the time sequence marks, extracting key points of operators, and acquiring three-dimensional coordinates of the key points by using the depth images;
step 2: when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold, taking the scene image corresponding to the current moment as a transformation image, and inputting the key points contained in the transformation image into a full-connection network to obtain a reference action category;
and step 3: acquiring a side weight and a direction according to the time sequence mark, constructing an action change diagram for the reference action category, and acquiring an initial action change matrix;
and 4, step 4: acquiring a background area of a transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among reference action categories based on the initial action change matrix, and marking and distinguishing to obtain change information; constructing a bipartite graph between the subareas and the change information by taking the similarity degree of pixel values in the same subarea between the transformed images as a side weight value;
and 5: screening out an influence action change area and a secondary environment area for the sub-areas based on the optimal distribution and combining with the edge weight value of the bipartite graph, correcting the bipartite graph to obtain a corrected bipartite graph, and acquiring an environment influence adjacency matrix;
step 6: acquiring a first characteristic tensor of the initial action change matrix and a second characteristic tensor of the environment influence adjacent matrix, and inputting the first characteristic tensor and the second characteristic tensor into a neural network to acquire an optimized action change matrix;
and 7: and acquiring the Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation of an operator is abnormal or not, and generating prompt information.
In summary, the embodiment of the present invention provides an enterprise security standardized operation method based on artificial intelligence and big data, which includes acquiring a reference action category of an operator by acquiring a continuous scene image and a depth image, constructing an action change diagram by combining a time sequence marker, and acquiring an initial action change matrix; then extracting a background area of the scene image, dividing the background area into a plurality of sub-areas, and establishing two graphs between the sub-areas and the behavior change information of the operators according to the pixel change between the fixed sub-areas; changing the area and the secondary environment area according to the influence action, correcting the bipartite graph to obtain a corrected environment influence graph, and acquiring an environment influence adjacency matrix; and acquiring an optimized action change matrix according to the environment influence adjacent matrix and the initial action change matrix, judging whether the operation of the operator is abnormal or not by combining the Euclidean distance between the prestored standard action change matrices, and generating prompt information. Therefore, the safety operation standard is dynamically generated according to the environment change condition and the relation between the environment change condition and the behavior and the action of the operator, the interference caused by the environmental factors is removed, and the judgment accuracy of the operation safety standardization is improved.
Specifically, in the embodiment of the present invention, taking a multi-chemical enterprise job scenario as an example, and the job type is known, each job scenario is deployed with an RGB-D camera for information acquisition.
Specifically, in step 1 of this embodiment, the key points are extracted from the network, and the operator key points are classified according to the trunk and the limbs; the body part comprises shoulder key points, hip key points and a body central point, and the limb part comprises elbow key points, hand key points, knee key points and foot key points; wherein, the shoulder key points, the elbow key points, the hand key points, the knee key points and the foot key points comprise two types of key points, namely left and right key points.
Specifically, in step 2 of this embodiment, since the trunk portion of the human body can be regarded as a rigid body, the key points on the trunk portion can be regarded as key points of the rigid body, and when analyzing the key points of the rigid body, only the change of the central point of the trunk portion needs to be analyzed, so as to obtain the motion information of the trunk portion. Therefore, in the embodiment, three rotational degrees of freedom exist from the shoulder key point to the elbow key point in the four-limb key points, and two rotational degrees of freedom exist from the elbow key point to the hand key point; there are three degrees of freedom from the hip key point to the knee key point and one degree of freedom from the knee joint to the foot key point. Wherein, the key point of the foot is specifically the ankle joint position.
Furthermore, an initial three-dimensional coordinate system of each key point is constructed by the fact that the two arms of an operator naturally droop and normally stand, and the vector change between the two groups of key points is used as a judgment standard. Specifically, the rotation angle between key points is determined based on an initial coordinate system, the movement trend is determined according to the angle change between two continuous frames, and the trend is divided into clockwise and anticlockwise. In this embodiment, a time when a motion trend changes and a scene image corresponding to the time are obtained, where a time from the previous motion trend to the current time is a first time period, and a time from the current time to the next motion trend is a second time period. Acquiring a first vector value of angle change of the degree of freedom between the key points in the three-dimensional coordinate system in a first time period, and similarly acquiring a second vector value according to the first time period; taking the difference between the first vector value and the second vector value as a variation; and when the variation is larger than the variation threshold, taking the scene image at the current moment as a variation image. And then, forming a sequence by using the three-dimensional coordinates of the key points contained in the change image, and outputting the sequence as a reference action type by using the input of the full-connection network.
It should be noted that training of the fully-connected network is common knowledge, and is not described in detail in this embodiment, and the reference action category may be set by the implementer according to an actual scene during training.
Specifically, in step 3 of this embodiment, the reference action category is used as a vertex, a directed edge is formed by connecting the vertex at the current time to the vertex at the next time according to a time sequence, and the change duration between the two times is used as an edge weight; when the same two vertexes are repeatedly changed in the time sequence, the average value of the changing duration is obtained and used as the edge weight of the two vertexes. Specifically, the method comprises the following steps:
and (3) allocating serial numbers to the reference action types obtained in the step (2), wherein the serial numbers start from 1 and end from N, and N is the number of the reference action types. Then, each reference action type is taken as a vertex, the reference action generated later is pointed according to the time sequence, the side weight of each side is taken as the direction of the side, and the time length required for changing from the reference action generated earlier to the reference action generated later is taken as the change time length, so that the action change graph is obtained and is a directed graph. It should be noted that there may be multiple changes repeated between two reference action categories in a time sequence, and at this time, an average value of multiple change durations is obtained as an edge weight of two vertices. Specifically, the adjacent matrix is obtained according to the action change diagram, the size is N × N, and the diagonals are all 0, that is, the initial action change matrix.
Furthermore, for the same type of task, the benchmark actions of each enterprise are consistent, and in order to avoid overlarge influence of a single person in a single site on the initial action change matrix, the standard action change matrix is obtained, in the step, the point-by-point average value can be obtained through the initial action change matrix of the action data of multiple persons in multiple sites, and a reference action change matrix is obtained and used for obtaining action change characteristics and weakening the influence of special actions of the single person on the action change characteristics; the reference motion changes are graphically represented and analyzed, and the correlation between the reference motions is more intuitively considered.
Specifically, in step 4 of this embodiment, in order to improve the accuracy of the motion change analysis, a gaussian background modeling is used to divide pixels in the image into a foreground and a background, the background is divided into K regions based on the number of the pixels, the regions are numbered, and the number of the selected pixels is determined by an implementer according to actual conditions.
And for the element with the element value not being 0 in the initial action change matrix, acquiring the data change between the two elements and marking the data change, namely the change information between the two corresponding reference actions. And then, acquiring the transformation images corresponding to the two reference actions, and acquiring the similarity of pixel values in the same sub-area through Euclidean distance. And constructing a bipartite graph between the subareas and the change information by taking the similarity degree of pixel values in the same subarea among the transformed images as an edge weight value. Specifically, the method comprises the following steps:
constructing a bipartite graph by taking the change information between two reference actions as a first node and taking a sub-region as a second node, and when the similarity of the sub-region in the corresponding transformed image of the two reference actions is smaller than a similarity threshold value M3If so, the edge weight value of the corresponding change information to the sub-region is 1, and the change information is reflected to have potential association with the sub-region; otherwise, the edge weight is 0, indicating no potential association. It should be noted that, because the scene images are not consistent due to the position of the enterprise camera, the embodiment utilizes a plurality of bipartite graphs to perform superposition averaging, and eliminates the influence of abnormal conditions on the weight;
specifically, in step 5 of the present embodiment, the change information between the reference actions is represented by one vertex set in the bipartite graph, and the number of vertices in the vertex set is statistically d; and selecting top-d vertexes with the largest single edge weight value in the other vertex set, wherein the number of vertexes in the two vertex sets of the screened bipartite graph is d, the edge weight values are only the maximum edge weight values of all the environment regions, complete matching is carried out at the moment, and optimal distribution is obtained through a KM algorithm.
The comparison of the optimal distribution result with the initially screened bipartite graph can be divided into two cases: the edge between two vertexes is reserved, and the vertex of the corresponding sub-area in the edge is a main area influencing the action change in the actual operation scene, such as a conveying device, a conveying belt tail end, a control device and the like, and is used as an action change influencing area; and eliminating the edge between the two vertexes, wherein the vertex of the corresponding sub-area in the edge is in a subordinate relation or a cooperative relation with the influence action change area and serves as a secondary environment area.
It should be noted that establishing the relationship between the influence action change area and the secondary environment area is beneficial to improving the accuracy of generating the subsequent optimization action change matrix.
Further, the two maps obtained in step 4 are corrected based on the obtained influence action change area and the secondary environment area. Specifically, the correction mode is that the secondary environment region vertex selects an influence action change region connected with the same change information vertex, and an edge between two vertexes is constructed; comparing the edge weight of the secondary environment area vertex and the connected change information vertex with the edge weight of the action change area vertex and the connected change information vertex, wherein the ratio is the edge weight between the secondary environment area vertex and the corresponding action change area; and obtaining a corrected environment influence graph after correction, and more accurately representing the relation between the action change and the environment area. And then obtaining an environment influence adjacency matrix according to the corrected environment influence graph.
Specifically, in step 6 of this embodiment, an encoder is used to obtain a first feature tensor and a second feature tensor, then the two feature tensors are subjected to a locate operation to obtain a third feature tensor, and the third feature tensor is input to a neural network to obtain an optimized action change matrix;
the specific training details of the neural network encoder and decoder are as follows: a plurality of groups of corrected environment influence graph adjacency matrixes and reference action change matrixes are used as training data sets; marking a reference optimization action change matrix manually; the loss function adopts a mean square error loss function;
specifically, in step 7 of the present embodiment, the standard action change matrix is obtained by using a plurality of sets of standard action types and environment influence adjacency matrices. Specifically, when the standard action change matrix is obtained, the standard action category needs to be obtained, and the standard action category includes the standard action category and other action categories. The standard action change matrix is acquired by using and optimizing the action change matrix.
Specifically, when judging whether the operation abnormality occurs to the operator, a corresponding abnormality threshold value M is set4Obtaining the operation action abnormity judgment result by comparing the metric value with the threshold valueIf the abnormal condition occurs, the system generates safety warning information. The method realizes the safe standardization and the digital operation of enterprises and improves the accuracy of the digital operation.
Based on the same inventive concept as the method embodiment, the embodiment of the invention also provides an enterprise safety standardized operation system based on artificial intelligence and big data.
Referring to fig. 2, the enterprise security standardized operation system 100 based on artificial intelligence and big data includes an image acquisition module 101, a reference action acquisition module 102, an action change matrix acquisition module 103, a bipartite graph acquisition module 104, an environmental impact matrix module 105, an optimized action change matrix module 106, and an anomaly determination module 107.
Specifically, the image acquisition module is used for acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to the time sequence marks, extracting key points of operators, and acquiring three-dimensional coordinates of the key points by using the depth images. And the reference action acquisition module is used for taking the scene image corresponding to the current moment as a conversion image and inputting the contained key points into the full-connection network to obtain the reference action category when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold. The action change matrix acquisition module is used for acquiring the side weight and the direction according to the time sequence mark, constructing an action change diagram for the reference action category and acquiring an initial action change matrix. The bipartite graph acquisition module is used for acquiring a background area of the transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among the reference action categories based on the initial action change matrix, and marking and distinguishing to obtain change information; and constructing a bipartite graph between the sub-regions and the change information by taking the similarity degree of pixel values in the same sub-region between the transformed images as a side weight. The environment influence matrix module is used for screening out an influence action change area and a secondary environment area for the sub-area based on the optimal distribution and combining the edge weight value of the bipartite graph, correcting the bipartite graph to obtain a corrected environment influence graph and obtain an environment influence adjacency matrix. And the optimized action change matrix module is used for acquiring a first characteristic tensor of the initial action change matrix and a second characteristic tensor of the environment influence adjacent matrix, and inputting the first characteristic tensor and the second characteristic tensor into the neural network to acquire the optimized action change matrix. And the abnormity judging module is used for acquiring the Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation abnormity occurs to the operator, and generating prompt information.
Further, key points of the operators are divided according to the trunk and the limbs; the trunk part comprises a shoulder key point, a hip key point and a trunk center point, and the limb part comprises an elbow key point, a hand key point, a knee key point and a foot key point.
Further, the three-dimensional coordinate variation of the key points is a vector value of angle variation of preset degrees of freedom between the key points in the three-dimensional coordinate system.
Further, the action change matrix obtaining module comprises a directed graph obtaining unit and a value optimizing unit.
Specifically, the directed graph obtaining unit is configured to connect a vertex at a current time to a vertex at a next time according to a time sequence by using the reference action category as the vertex to form a directed edge, and use a change duration between the two times as an edge weight. The weight optimization unit is used for acquiring the mean value of the change duration as the edge weight of the two vertexes when the same two vertexes are repeatedly changed in the time sequence.
Further, the reference action category comprises a standard action category and other action categories; the standard action change matrix of (2) is obtained using a plurality of sets of standard action categories and environmental impact adjacency matrices.
It should be noted that: the precedence order of the above embodiments of the present invention is only for description, and does not represent the merits of the embodiments. And specific embodiments thereof have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, but rather as the subject matter of the invention is to be construed as broadly as the appended claims.
Claims (10)
1. An enterprise safety standardization operation method based on artificial intelligence and big data is characterized by comprising the following steps:
acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to a time sequence mark, extracting key points of operators, and acquiring three-dimensional coordinates of the key points by using the depth images;
when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold, taking a scene image corresponding to the current moment as a transformation image, and inputting the key points contained in the transformation image into the full-connection network to obtain a reference action category;
acquiring a side weight value and a direction according to the time sequence mark, constructing an action change diagram for the reference action category, and acquiring an initial action change matrix;
acquiring a background area of the transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among reference action types based on the initial action change matrix, marking and distinguishing to obtain change information; constructing a bipartite graph between the sub-regions and the change information by taking the similarity degree of pixel values in the same sub-region between the transformed images as a side weight value;
screening out an influence action change area and a secondary environment area for the sub-area based on the optimal distribution and combining with the edge weight of the bipartite graph, correcting the bipartite graph to obtain a corrected environment influence graph, and acquiring an environment influence adjacency matrix;
acquiring a first feature tensor of the initial action change matrix and a second feature tensor of the environment influence adjacent matrix, and inputting the first feature tensor and the second feature tensor into a neural network to acquire an optimized action change matrix;
and acquiring a Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation of the operator is abnormal or not, and generating prompt information.
2. The enterprise safety standardized operation method based on artificial intelligence and big data as claimed in claim 1, wherein the operator key points are divided according to trunk and limbs; the trunk part comprises a shoulder key point, a hip key point and a trunk center point, and the limb part comprises an elbow key point, a hand key point, a knee key point and a foot key point.
3. The enterprise safety standardized operation method based on artificial intelligence and big data as claimed in claim 2, wherein the three-dimensional coordinate variation of the key points is a vector value of angle variation of preset degrees of freedom between the key points in a three-dimensional coordinate system.
4. The enterprise safety standardization operation method based on artificial intelligence and big data according to claim 1, wherein the method for obtaining the edge weight and the direction according to the time sequence mark and constructing the action change diagram for the reference action category comprises the following steps:
connecting the vertex of the current moment to the vertex of the next moment according to a time sequence by taking the reference action category as the vertex to form a directed edge, and taking the change duration between the two moments as an edge weight;
when the same two vertexes are repeatedly changed in the time sequence, the average value of the changing duration is obtained and used as the edge weight of the two vertexes.
5. The method for enterprise security standardized operation based on artificial intelligence and big data as claimed in any one of claims 1 to 4, wherein the reference action category comprises a standard action category and other action categories; the standard action change matrix is obtained by utilizing a plurality of groups of the standard action categories and the environment influence adjacency matrix.
6. Enterprise safety standardization operation system based on artificial intelligence and big data is characterized by comprising:
the image acquisition module is used for acquiring continuous scene images and depth images, inputting the scene images into a key point extraction network according to the time sequence marks, extracting key points of operators and acquiring three-dimensional coordinates of the key points by using the depth images;
the reference action acquisition module is used for taking a scene image corresponding to the current moment as a transformation image and inputting the key points contained in the transformation image into a full-connection network to acquire a reference action category when the variation of the three-dimensional coordinates of the key points is larger than a variation threshold;
the action change matrix acquisition module is used for acquiring a side weight and a direction according to the time sequence mark, constructing an action change diagram for the reference action category and acquiring an initial action change matrix;
the bipartite graph acquisition module is used for acquiring a background area of the transformed image and dividing the background area into a plurality of sub-areas; acquiring data change among reference action types based on the initial action change matrix, marking and distinguishing to obtain change information; constructing a bipartite graph between the sub-regions and the change information by taking the similarity degree of pixel values in the same sub-region between the transformed images as a side weight value;
the environment influence matrix module is used for screening out an influence action change area and a secondary environment area for the sub-area based on optimal distribution and combined with the edge weight of the bipartite graph, correcting the bipartite graph to obtain a corrected environment influence graph and obtain an environment influence adjacency matrix;
the optimized action change matrix module is used for acquiring a first feature tensor of the initial action change matrix and a second feature tensor of the environment influence adjacent matrix, and inputting the first feature tensor and the second feature tensor into a neural network to acquire an optimized action change matrix;
and the abnormity determining module is used for acquiring the Euclidean distance between the optimized action change matrix and a pre-stored standard action change matrix, judging whether the operation abnormity occurs to the operator, and generating prompt information.
7. The artificial intelligence and big data based enterprise safety standardized operation system of claim 6, wherein the worker key points are divided according to trunk and limbs; the trunk part comprises a shoulder key point, a hip key point and a trunk center point, and the limb part comprises an elbow key point, a hand key point, a knee key point and a foot key point.
8. The artificial intelligence and big data based enterprise safety standardized operation system as claimed in claim 7, wherein the three-dimensional coordinate variation of the key points is a vector value of angle variation of preset degrees of freedom between the key points in a three-dimensional coordinate system.
9. The system according to claim 6, wherein the action change matrix obtaining module comprises:
the directed graph acquiring unit is used for connecting the vertex of the current moment to the vertex of the next moment according to a time sequence by taking the reference action category as the vertex to form a directed edge, and taking the change duration between the two moments as an edge weight;
and the weight value optimization unit is used for acquiring the mean value of the change duration as the edge weight value of the two vertexes when the two identical vertexes are repeatedly changed in the time sequence.
10. The system according to any one of claims 6 to 9, wherein the reference action categories include a standard action category and other action categories; the standard action change matrix is obtained by utilizing a plurality of groups of the standard action categories and the environment influence adjacency matrix.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236578.7A CN112966593B (en) | 2021-03-03 | 2021-03-03 | Enterprise safety standardized operation method and system based on artificial intelligence and big data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236578.7A CN112966593B (en) | 2021-03-03 | 2021-03-03 | Enterprise safety standardized operation method and system based on artificial intelligence and big data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112966593A true CN112966593A (en) | 2021-06-15 |
| CN112966593B CN112966593B (en) | 2024-03-15 |
Family
ID=76276873
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110236578.7A Active CN112966593B (en) | 2021-03-03 | 2021-03-03 | Enterprise safety standardized operation method and system based on artificial intelligence and big data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112966593B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113435794A (en) * | 2021-08-26 | 2021-09-24 | 山东大拇指喷雾设备有限公司 | Intelligent monitoring method for post-processing of nozzle casting based on image processing |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102737381A (en) * | 2012-06-13 | 2012-10-17 | 西安电子科技大学 | Image partitioning method based on mixed bipartite graph clustering integration |
| US20120278021A1 (en) * | 2011-04-26 | 2012-11-01 | International Business Machines Corporation | Method and system for detecting anomalies in a bipartite graph |
| EP3333771A1 (en) * | 2016-12-09 | 2018-06-13 | Fujitsu Limited | Method, program, and apparatus for comparing data hypergraphs |
| CN110348335A (en) * | 2019-06-25 | 2019-10-18 | 平安科技（深圳）有限公司 | Method, apparatus, terminal device and the storage medium of Activity recognition |
| CN110580446A (en) * | 2019-07-16 | 2019-12-17 | 上海交通大学 | Behavior semantic subdivision understanding method, system, computer device and medium |
| CN111507277A (en) * | 2020-04-20 | 2020-08-07 | 国网安徽省电力有限公司 | Construction method and application of operation flow normative recognition model |
| CN111597974A (en) * | 2020-05-14 | 2020-08-28 | 哈工大机器人（合肥）国际创新研究院 | Monitoring method and system based on TOF camera for personnel activities in carriage |
| CN112329560A (en) * | 2020-10-23 | 2021-02-05 | 岭东核电有限公司 | Illegal behavior recognition method and device for nuclear power operating personnel and computer equipment |
| CN112364785A (en) * | 2020-11-13 | 2021-02-12 | 中移雄安信息通信科技有限公司 | Exercise training guidance method, device, equipment and computer storage medium |
-
2021
- 2021-03-03 CN CN202110236578.7A patent/CN112966593B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120278021A1 (en) * | 2011-04-26 | 2012-11-01 | International Business Machines Corporation | Method and system for detecting anomalies in a bipartite graph |
| CN102737381A (en) * | 2012-06-13 | 2012-10-17 | 西安电子科技大学 | Image partitioning method based on mixed bipartite graph clustering integration |
| EP3333771A1 (en) * | 2016-12-09 | 2018-06-13 | Fujitsu Limited | Method, program, and apparatus for comparing data hypergraphs |
| CN110348335A (en) * | 2019-06-25 | 2019-10-18 | 平安科技（深圳）有限公司 | Method, apparatus, terminal device and the storage medium of Activity recognition |
| WO2020259213A1 (en) * | 2019-06-25 | 2020-12-30 | 平安科技（深圳）有限公司 | Behavior recognition method and apparatus, terminal device, and storage medium |
| CN110580446A (en) * | 2019-07-16 | 2019-12-17 | 上海交通大学 | Behavior semantic subdivision understanding method, system, computer device and medium |
| CN111507277A (en) * | 2020-04-20 | 2020-08-07 | 国网安徽省电力有限公司 | Construction method and application of operation flow normative recognition model |
| CN111597974A (en) * | 2020-05-14 | 2020-08-28 | 哈工大机器人（合肥）国际创新研究院 | Monitoring method and system based on TOF camera for personnel activities in carriage |
| CN112329560A (en) * | 2020-10-23 | 2021-02-05 | 岭东核电有限公司 | Illegal behavior recognition method and device for nuclear power operating personnel and computer equipment |
| CN112364785A (en) * | 2020-11-13 | 2021-02-12 | 中移雄安信息通信科技有限公司 | Exercise training guidance method, device, equipment and computer storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 陈汉武;李文骞;刘志昊;赵生妹;: "完全图上结构异常的搜索算法――融入量子计算思维的经典算法探讨", 东南大学学报(自然科学版), no. 05 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113435794A (en) * | 2021-08-26 | 2021-09-24 | 山东大拇指喷雾设备有限公司 | Intelligent monitoring method for post-processing of nozzle casting based on image processing |
| CN113435794B (en) * | 2021-08-26 | 2021-11-19 | 山东大拇指喷雾设备有限公司 | Intelligent monitoring method for post-processing of nozzle casting based on image processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112966593B (en) | 2024-03-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112347916B (en) | Video image analysis-based power field operation safety monitoring method and device | |
| JP6286474B2 (en) | Image processing apparatus and area tracking program | |
| CN111553229B (en) | Worker action identification method and device based on three-dimensional skeleton and LSTM | |
| JP6773829B2 (en) | Object recognition device, object recognition method, and object recognition program | |
| CN110889376A (en) | Safety helmet wearing detection system and method based on deep learning | |
| Yusoff et al. | Real-time hevea leaves diseases identification using Sobel edge algorithm on FPGA: A preliminary study | |
| JPH06266840A (en) | Status detector for moving object | |
| CN110414400A (en) | A kind of construction site safety cap wearing automatic testing method and system | |
| CN112184773A (en) | Helmet wearing detection method and system based on deep learning | |
| CN112966593A (en) | Enterprise safety standardized operation method and system based on artificial intelligence and big data | |
| CN113989711A (en) | Power distribution construction safety tool use identification method and system | |
| CN114155492A (en) | High-altitude operation safety belt hanging rope high-hanging low-hanging use identification method and device and electronic equipment | |
| JPH07282235A (en) | Operation recognition device | |
| CN112233770B (en) | Gymnasium intelligent management decision-making system based on visual perception | |
| CN109740527A (en) | Image processing method in a kind of video frame | |
| KR102366396B1 (en) | RGB-D Data and Deep Learning Based 3D Instance Segmentation Method and System | |
| KR102144474B1 (en) | Method for detecting injurious insect in image | |
| CN112949606B (en) | Method and device for detecting wearing state of work clothes, storage medium and electronic device | |
| CN116229502A (en) | Image-based tumbling behavior identification method and equipment | |
| CN115880641A (en) | Method and device for automatically detecting unsafe behaviors of workers based on machine vision | |
| CN113762115B (en) | Distribution network operator behavior detection method based on key point detection | |
| KR20230099225A (en) | An apparatus for monitoring safety status of construction workers in an image and operating method thereof | |
| CN113971838B (en) | Detection method and device based on security behavior portraits of identification personnel | |
| CN111274888A (en) | Intelligent identification method for safety helmet and work clothes based on wearable mobile glasses | |
| CN116909405B (en) | Instruction control method based on artificial intelligence action recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |