CN115424349A - Violation behavior recognition method and violation prediction model training method and system - Google Patents

Abstract

The embodiment of the application discloses a violation behavior identification method, a violation prediction model training method and a violation behavior identification system. The violation behavior identification method comprises the following steps: a terminal acquires an image to be identified; the terminal inputs an image to be identified into a front-end processing layer of the violation prediction model to obtain intermediate characteristics and sends the intermediate characteristics to the cloud; and after the cloud end obtains the intermediate features, inputting the intermediate features into a rear-end processing layer of the violation prediction model so as to identify and obtain violation results. Therefore, the terminal can send the acquired image to be recognized to the cloud after the acquired image to be recognized obtains the intermediate characteristic with smaller data volume through processing, so that the cloud finally obtains the violation result through processing the intermediate characteristic, the data transmission quantity between the terminal and the cloud is reduced, the operation quantity required by the violation result is obtained by the cloud, the violation result is accurately obtained while the processing speed is accelerated, and the pressure of the cloud is reduced.

Description

Violation behavior recognition method and violation prediction model training method and system

Technical Field

The application belongs to the technical field of violation behavior recognition systems, and particularly relates to a violation behavior recognition method, a violation prediction model training method and a violation behavior recognition system.

Background

At present, two common ways for carrying out violation processing in the industry are as follows: 1. a manual remote real-time monitoring mode; 2. information representing the operation condition is collected and uploaded to a server, and the server utilizes the built model to automatically identify and monitor. With the popularization of the data model technology, the second method has gradually become the mainstream in the industry.

However, the information representing the job condition, which is collected and uploaded to the server, is usually an image to be recognized with a large data volume. The server, whether receiving or processing, may require a large amount of computation. And as the data volume increases, the pressure on the cloud server also increases.

The foregoing description is provided for general background information and does not necessarily constitute prior art.

Disclosure of Invention

Therefore, in order to solve the above problems, it is necessary to provide a violation behavior recognition method, a violation prediction model training method, and a violation behavior recognition system.

The technical problem to be solved by the application is realized by adopting the following technical scheme:

the application provides a violation behavior identification method, which is applied to a terminal and comprises the following steps: acquiring an image to be identified; inputting an image to be identified into a front-end processing layer of a violation prediction model to obtain an intermediate feature; and sending the intermediate characteristic to a cloud end so that the cloud end can identify the violation result according to the intermediate characteristic.

In an optional embodiment of the present application, before inputting the image to be recognized into the front-end processing layer of the trained violation prediction model, the method further includes: when the image to be identified is obtained, audio information which is collected at the same time as the image to be identified is obtained; inputting an image to be recognized into a sound prediction model to obtain predicted audio information; judging whether the image to be identified is a real image or not according to the audio information and the predicted audio information; and when the image to be recognized is a real image, inputting the image to be recognized into the front-end processing layer.

In an optional embodiment of the present application, the method further comprises: extracting facial features in the image to be recognized, and determining identity information corresponding to the image to be recognized according to the facial features; matching a corresponding private key according to the identity information, and signing the intermediate characteristic by using the private key to obtain a signed characteristic; and packaging and sending the signature adding feature and the intermediate feature to a cloud.

The application also provides a violation behavior identification method, which is applied to the cloud and comprises the following steps: acquiring intermediate features sent by a terminal, wherein the intermediate features are acquired by processing an image to be identified by the terminal; and inputting the intermediate features into a back-end processing layer of the violation prediction model to identify the violation result.

In an optional embodiment of the present application, before obtaining the intermediate feature sent by the terminal, the method further includes: acquiring identity information of an operator; acquiring a public key and a private key matched with the identity information according to the identity information; sending the identity information, the private key and the matching relation between the identity information and the private key to the terminal so that the terminal adds the label to the intermediate feature according to the identity information to obtain a signature adding feature; and receiving the signature adding feature sent by the terminal.

In an optional embodiment of the present application, before inputting the intermediate features into a back-end processing layer of the trained violation prediction model, the method further comprises: the public key is used for carrying out label release on the signature adding characteristic to obtain a label release characteristic; when the de-tagged feature and the intermediate feature are consistent, the intermediate feature is input into a back-end processing layer.

The application also provides a violation behavior identification method, which comprises the following steps: a terminal acquires an image to be identified; the terminal inputs an image to be identified into a front-end processing layer of the violation prediction model to obtain intermediate characteristics and sends the intermediate characteristics to the cloud; and after the cloud end obtains the intermediate features, inputting the intermediate features into a rear-end processing layer of the violation prediction model so as to identify and obtain violation results.

The application also provides a violation prediction model training method, which comprises the following steps: acquiring sample information, wherein the sample information comprises violation state attributes; inputting the sample information into an initial violation prediction model for processing to obtain a prediction violation result, wherein the prediction violation result comprises a prediction violation state attribute and a violation probability; acquiring a target deviation between a prediction violation result and sample information; and performing iterative parameter adjustment on the initial violation prediction model according to the target deviation until the target deviation is smaller than a preset deviation threshold value, so as to obtain the violation prediction model.

In an optional embodiment of the present application, the method further comprises: acquiring output data quantity of each processing layer of the violation prediction model, and arranging the processing layers according to the size of the output data quantity; acquiring a dynamic output data volume threshold according to the output data volume; taking the processing layer with the output data volume larger than the dynamic output data volume threshold value as a front-end processing layer and distributing the front-end processing layer to the terminal; and taking the processing layer with the output data volume less than or equal to the dynamic output data volume threshold value as a back-end processing layer, and distributing the back-end processing layer to the cloud.

The application also provides a system for identifying the violation behaviors, which comprises a terminal and a cloud end; the terminal is used for executing the steps of the violation behavior recognition method of the terminal as applied in the foregoing; the cloud is used for executing the steps of the violation identification method applied to the cloud as described above.

By adopting the embodiment of the application, the following beneficial effects are achieved:

the method and the device enable the terminal to process the acquired image to be recognized to obtain the intermediate features with smaller data volume and then send the intermediate features to the cloud so that the cloud processes to finally obtain violation results; therefore, data transmission quantity between the terminal and the cloud end is reduced, the operation quantity required by the violation result is obtained by the cloud end, the violation result is accurately obtained, the processing speed is increased, and the pressure of the cloud end is reduced.

The foregoing description is only an overview of the technical solutions of the present application, and in order to make the technical means of the present application more clearly understood, the present application may be implemented in accordance with the content of the description, and in order to make the above and other objects, features, and advantages of the present application more clearly understood, the following preferred embodiments are specifically described in detail with reference to the accompanying drawings. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Wherein:

fig. 1 is a schematic flow chart of a violation behavior identification method applied to a terminal according to an embodiment;

fig. 2 is a schematic flow chart of a violation behavior identification method applied to a cloud according to the second embodiment;

fig. 3 is a schematic flow chart of a violation behavior identification method provided in the third embodiment;

fig. 4 is a schematic flow chart of a method for training a violation prediction model according to the fourth embodiment;

fig. 5 is a block diagram illustrating a structure of a violation behavior recognition system according to the fifth embodiment.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Example one

Fig. 1 is a schematic flow chart of a violation behavior identification method applied to a terminal according to an embodiment. For clearly describing the violation behavior identification method applied to the terminal provided by the embodiment, please refer to fig. 1.

Step S110: and acquiring an image to be identified.

In an embodiment, the violation behavior recognition method provided by the embodiment is applied to a terminal, wherein the terminal can be a camera device with an image acquisition function and simple data processing and transmission functions in a better embodiment; or it may be only a data processing device, and other acquisition devices provide data of the image to be recognized for it, and the specific form is not limited. Therefore, the image to be identified can be obtained, and corresponding processing is carried out in the subsequent steps, and the transmission and the like are carried out. Specifically, the specific form of the terminal may be a monitoring device such as a ball placement monitor. Further, the practical implementation scenario for the practical application of the embodiment may include, but is not limited to, monitoring whether a pedestrian or a driver has a violation in public transportation, and may also be, for example, whether a monitoring staff has a violation in working at work or at a production site. Therefore, the image to be recognized acquired by the terminal, that is, the image captured by the monitoring device terminal in the implementation scene exemplified by this embodiment, is used as the image to be recognized. The reason is to detect the violation behaviors of the personnel, namely, the corresponding personnel need to exist in the image to be identified.

Step S120: and inputting the image to be recognized into a front-end processing layer of the violation prediction model to obtain the intermediate features.

In one embodiment, the front-end processing layer of the violation prediction model is a processing layer of a deep learning model used by the terminal for processing the image to be identified, and the front-end processing layer also comprises a back-end processing layer for determining whether the result of the violation is a violation result. The specific structure of the violation prediction model will be described in detail later, and the functions implemented by the two data processing layers are determined. In a preferred embodiment, the number of the images to be recognized input into the front-end processing layer is at least two, and the images can also be directly video information, so that the front-end processing layer can determine a recognition object in the images to be recognized and behavior information of the recognition object through the dynamic images to be recognized, and process the behavior information into the intermediate features. The intermediate features are abstract data without direct meaning, have the characteristic of small data volume, and can be identified by a back-end processing layer. For example, in the front-end processing layer, the limb and joint nodes of the object are identified, the limb movement of the human body is determined according to the position relationship of the plurality of key nodes, the limb movement is converted into vector data, so that an intermediate feature which can be in a multi-dimensional vector form and has a smaller volume is generated, and the deviation of the intermediate feature before and after adjustment can be the similarity of the multi-dimensional vectors. Further, for the implementation of identifying the joint node, the key node may be shielded by other objects in the movement process of the operator, so that information is lost, and the identification of the node is affected, so that the identification of the violation result is affected finally. Therefore, the image to be recognized may not be limited to the image captured at one viewing angle, and may be a panoramic image including the recognition object, that is, an image captured by another capturing device at another viewing angle may also be acquired as the image to be recognized.

In one embodiment, in step S120: before the image to be recognized is input into the front-end processing layer of the trained violation prediction model, the method further comprises the following steps: when the image to be identified is obtained, audio information which is collected at the same time as the image to be identified is obtained; inputting the image to be identified into a sound prediction model to obtain predicted audio information; judging whether the image to be identified is a real image or not according to the audio information and the predicted audio information; and when the image to be recognized is a real image, inputting the image to be recognized into the front-end processing layer.

In one embodiment, in the violation identification method described above, the identification basis is the image to be identified, i.e. only the image information. However, in real life, especially for the scenes of work and production, since the manufacturers need to perform security and desensitization processing on their production lines, the credibility of the obtained images to be identified is not necessarily high. Therefore, in an implementation scenario where the terminal is only a data processing device and the image to be recognized is provided by another capturing device or a storage device, it is preferable that the audio information captured at the same time as the image to be recognized may be obtained when the image to be recognized is obtained, for example, the original static image to be recognized may be changed into dynamic video information, or multiple static images to be recognized and the audio information obtained at the same time may be obtained. And inputting the image to be recognized into the sound prediction model to obtain predicted audio information. For the sound prediction model, the sound prediction model may be constructed and trained according to a preset method, for example, a sound label corresponding to the sound prediction model may be set in a trained image, so as to train the sound prediction model, so that the sound prediction model can obtain predicted audio information according to image information. Judging whether the image to be recognized is a real image or not according to the audio information and the predicted audio information, for example, calculating similarity between the audio information and the predicted audio information which are synchronously acquired, if the similarity exceeds a preset threshold, determining that the acquired image to be recognized is a real image, and executing subsequent steps, such as inputting the image to be recognized into a front-end processing layer.

In an embodiment, in the process of verifying whether the image to be recognized is a real image, timestamp information may be extracted from the image to be recognized, the timestamp information is subjected to hash processing to be added to the content of the leaf node, the sequentiality of the timestamp hash result in the leaf node is verified, and if the sequentiality verification is passed, it is determined that the image is acquired in real time, that is, the image is a real image. In addition, after the facial features are extracted, the facial features can be compared with identity information of operators to verify whether the current operators in the production plan correspond to the recognized identities, and after various kinds of verification are passed, illegal actions can be recognized, so that the behaviors of counterfeiting by personnel and video counterfeiting are avoided. Therefore, the authenticity of the acquired image to be recognized is ensured, and the reliability and the accuracy of the recognition result are improved.

Step S130: and sending the intermediate characteristics to a cloud so that the cloud can identify violation results according to the intermediate characteristics.

In one embodiment, the terminal sends the intermediate feature to the cloud, that is, a connection is established between the terminal and the cloud, and the connection may include, but is not limited to, a wired communication manner or a wireless communication manner. Among the wireless communication technologies, there may be included, but not limited to: global System for Mobile Communication (GSM), enhanced Mobile Communication (EDGE), wideband Code division multiple Access (W-CDMA), code Division Multiple Access (CDMA), time Division Multiple Access (TDMA), bluetooth, wireless Fidelity (WiFi) (such as IEEE802.11 a, IEEE802.11 b, IEEE802.1 g, and/or IEEE802.11 n), voice over internet protocol (VoIP), worldwide Interoperability for Microwave Access (Wi-Max), other suitable protocols for email, instant messaging, and short messages, and any other suitable protocols, including those currently under development. Therefore, the terminal can send corresponding data information to the cloud.

In an implementation manner, the method provided in this embodiment further includes: extracting facial features in the image to be recognized, and determining identity information corresponding to the image to be recognized according to the facial features; matching a corresponding private key according to the identity information, and signing the intermediate characteristic by using the private key to obtain a signed characteristic; and packaging and sending the signature adding feature and the intermediate feature to a cloud.

In an embodiment, it can be understood that in the process of implementing violation identification, a large amount of data is transmitted between the cloud and the terminal, and even if a specific image to be identified is processed into an intermediate feature with a smaller data amount and a more abstract data amount, in the transmission process, a risk of data leakage still exists, and data safety is directly influenced. Therefore, the intermediate features can be encrypted, and data security is protected. The encryption needs a key, but it can be understood that the application aims at identifying the violation behaviors of the personnel, and the violation behaviors are identified, and the violation personnel are necessarily identified. Therefore, when the front-end processing layer of the terminal processes the image to be identified, the facial features of the violator in the image to be identified can be extracted, and the identity information of the violator can be determined according to the facial features. The identity information of the offenders is bound with the private key used for encryption, the binding relationship and the private key are prepared and provided in advance before the method process provided by the application is executed, and the specific generation mode is generated in advance at the cloud end, so that the specific generation process is detailed at the cloud end part. It is only necessary to know that the terminal stores the identity information and the corresponding private key. After the terminal matches the private key corresponding to the identity information, the intermediate feature can be signed to obtain a signed feature, and the signed feature and the intermediate feature are packaged and sent to the cloud. Even if data leakage occurs, the attacker cannot decrypt the data, so that the signature adding characteristic and the intermediate characteristic cannot be distinguished, the data security is protected, and the data security is improved. Meanwhile, as the concrete and clear image to be identified is processed into the abstract and meaningless intermediate features, the intermediate features cannot be identified and distinguished without a corresponding rear-end processing layer on an attacker, thereby providing effective guarantee for data security.

Therefore, the terminal can process the acquired image to be identified to obtain the intermediate features with smaller data volume and then send the intermediate features to the cloud, so that the violation results are finally obtained through cloud processing; therefore, data transmission quantity between the terminal and the cloud is reduced, the operation quantity required by the violation result is obtained by the cloud, the violation result is accurately obtained, the processing speed is increased, and the pressure of the cloud is reduced.

Example two

Fig. 2 is a schematic flow chart of the violation behavior recognition method applied to the cloud according to the second embodiment. For clearly describing the method for identifying the violation applied to the cloud according to the present embodiment, please refer to fig. 1 and fig. 2.

Step S210: and acquiring intermediate features sent by the terminal, wherein the intermediate features are acquired by processing the image to be recognized by the terminal.

In one embodiment, a connection is established between the terminal and the cloud, and the connection mode may include, but is not limited to, a wired communication mode or a wireless communication mode. The intermediate features are abstract data which are obtained by processing the image to be recognized in the front-end processing layer by the terminal and have no direct meaning, have the characteristic of small data volume, and can be recognized by the back-end processing layer. The embodiment of how to obtain the intermediate features through the to-be-recognized image acquisition is described in detail in the foregoing, and details are not repeated here, and reference may be made to the foregoing specifically.

In one embodiment, in step S210: acquiring the intermediate characteristic sent by the terminal, wherein the method provided by the embodiment further comprises the following steps: acquiring identity information of an operator; acquiring a public key and a private key matched with the identity information according to the identity information; sending the identity information, the private key and the matching relation between the identity information and the private key to the terminal so that the terminal adds the label to the intermediate feature according to the identity information to obtain a signature adding feature; and receiving the signature feature sent by the terminal.

In an embodiment, in the first embodiment, it is mentioned that there is a risk of data leakage during transmission, which directly affects data security. Therefore, the intermediate features can be encrypted, and data security is protected. In the former embodiment, it is required to generate a private key corresponding to the identity information by the cloud. Specifically, the identity information of the operator is obtained by the cloud, wherein the identity information includes facial features of the operator, and a digital summary is generated according to the facial features for registration, so that a public key for decryption and a private key for encryption are obtained, wherein the private key is bound with the identity information corresponding to the facial features. And then the cloud sends the identity information, the private key and the matching relation between the identity information and the private key to the terminal, so that the terminal can sign the intermediate features according to the identity information to obtain the signature characteristic. The specific tagging process has been described in detail in the first embodiment of the present application, and therefore, detailed description thereof is omitted, and reference may be made to the foregoing specifically. Therefore, while the intermediate features are obtained, the cloud end can also obtain the corresponding signature adding features and obtain the two types of feature data, so that even if data leakage occurs, an attacker cannot decrypt the data, the signature adding features and the intermediate features cannot be distinguished, data safety is protected, and data safety is improved.

Step S220: and inputting the intermediate features into a back-end processing layer of the violation prediction model to identify violation results.

In one embodiment, the intermediate features obtained by the cloud can be input into a back-end processing layer of the violation prediction model to identify the violation results. Wherein the back-end processing layer of the violation prediction model is a data processing layer that is trained to recognize violations. The intermediate features as described above may be vector data of the behavior of the limb, so the process of determining the violation result may be: and restoring the limb action of the recognition object according to the intermediate characteristic, and restoring the limb action of the recognition object when the recognition object is collected according to the limb action, wherein the limb action can be a production line production operation behavior, such as a processing action or a processing action combination. And identifying whether the limb action is a violation action by using a preset violation action identification rule. For example, a key node combination at each time can be determined, the current limb posture is determined according to the key node combination, and whether the limb postures at multiple times are violated is determined according to the change amplitude condition and the change speed condition of the limb postures at multiple times. In other embodiments, the data of the intermediate feature may be directly identified according to a back-end processing layer of the violation prediction model, for example, the intermediate feature may be in a form of a multidimensional vector, that is, the multidimensional vector may be directly identified according to a preset violation action identification rule, so as to determine whether the violation action is a violation. The final output violation result may include whether the violation is recognized, the violation probability, the violation type attribute, and the like.

In an embodiment, the cloud end can also identify the authenticity of the image to be identified, which is acquired by the terminal. For example, an image hash result obtained by the terminal performing hash processing on the image to be recognized is obtained. The cloud end combines the random number, the block node address corresponding to the terminal and the image hash result to create leaf nodes of the Mercker hash tree, and generates path information according to the positions of the leaf nodes in the Mercker hash tree to serve as index information of the image hash result. When the authenticity of the image to be verified is verified, the path information is used for inquiring in the Mercker hash tree to obtain leaf nodes, the image to be verified is subjected to hash processing and is compared with the leaf node information, if the path information is consistent with the leaf node information, the verification is passed, the image to be recognized corresponding to the intermediate features is proved to be a real image, and the subsequent steps are executed. The actions of personnel impersonation and video impersonation are avoided, so that the authenticity of the acquired image to be identified is ensured, and the reliability and the accuracy of an identification result are improved.

In one embodiment, in step S220: before the intermediate features are input into the back-end processing layer of the trained violation prediction model, the method provided by this embodiment further includes: the public key is used for carrying out label release on the signature adding characteristic to obtain a label release characteristic; when the de-signed feature and the intermediate feature are consistent, the intermediate feature is input into a back-end processing layer.

In an embodiment, the cloud may actually obtain the intermediate feature and the signature feature after the security processing, and it is impossible to determine which is valid data before decryption fails, so that the data needs to be decrypted. The decryption process may specifically be: and acquiring a public key obtained by registration, and utilizing the public key to perform label-added signature release on the labeled features subjected to label addition through the private key to obtain label-released features. And comparing the label-removing characteristic with the intermediate characteristic, and when the label-removing characteristic is consistent with the intermediate characteristic, if the corresponding intermediate characteristic is correctly obtained, executing the steps which should be executed originally, namely inputting the intermediate characteristic into a rear-end processing layer to obtain a violation result.

Therefore, the violation result can be identified by inputting the intermediate features into the pre-trained rear-end processing layer of the violation prediction model, so that the cloud end obtains the intermediate features with smaller data volume, the data transmission and operation pressure of the cloud end is reduced, the operation efficiency of the cloud end is improved, and the resources are saved.

EXAMPLE III

Fig. 3 is a schematic flow chart of a violation behavior identification method provided in the third embodiment. For clearly describing the method for identifying a violation, please refer to fig. 1 to fig. 3.

Step S310: and the terminal acquires an image to be identified.

In one embodiment, the terminal may be a camera device with image capturing function and simple data processing and transmission function in a preferred embodiment; or may be only a data processing device, and the data of the image to be recognized is provided for it by other acquisition devices, and the specific form is not limited. Therefore, the image to be identified can be obtained, and corresponding processing is carried out in the subsequent steps, and the transmission and the like are carried out. Specifically, the specific form of the terminal may be a monitoring device such as a ball placement monitor. Further, the practical implementation scenario applied in the embodiment may include, but is not limited to, public transportation for monitoring whether there is a violation behavior of a pedestrian or a driver, and may also be, for example, whether there is a violation behavior of a monitoring staff during work at work or a production site. Therefore, the image to be recognized acquired by the terminal, that is, the image captured by the monitoring device terminal in the implementation scene exemplified by this embodiment, is used as the image to be recognized. The reason is to detect the violation behaviors of the personnel, namely, the corresponding personnel need to exist in the image to be identified.

Step S320: and the terminal inputs the image to be recognized into a front-end processing layer of the violation prediction model to obtain intermediate characteristics and sends the intermediate characteristics to the cloud.

In one embodiment, the front-end processing layer of the violation prediction model is a processing layer of a deep learning model used by the terminal for processing the image to be identified, and the front-end processing layer also comprises a back-end processing layer for determining whether the result of the violation is a violation result. The specific structure of the violation prediction model will be described in detail later, and only the functions implemented by the two data processing layers need to be determined here. In a preferred embodiment, the number of the images to be recognized input into the front-end processing layer is at least two, and the images can also be directly video information, so that the front-end processing layer can determine a recognition object in the images to be recognized and behavior information of the recognition object through the dynamic images to be recognized, and process the behavior information into the intermediate features. The intermediate features are abstract data without direct meaning, have the characteristic of small data volume, and can be identified by a back-end processing layer. For example, in the front-end processing layer, the limb and joint nodes of the object are identified, the limb movement of the human body is determined according to the position relationship of the key nodes, the limb movement is converted into vector data, so that an intermediate feature which can be in a multi-dimensional vector form and has a smaller volume is generated, and the deviation of the intermediate feature before and after adjustment can be the similarity of the multi-dimensional vectors.

Step S330: and after the cloud end obtains the intermediate features, inputting the intermediate features into a rear-end processing layer of the violation prediction model so as to identify and obtain violation results.

In one embodiment, the intermediate features are obtained by the cloud and then input into a rear-end processing layer of the violation prediction model to identify violation results. Wherein the back-end processing layer of the violation prediction model is the data processing layer that is used to train for recognizing violations. The intermediate features as described above may be vector data of the behavior of the limb, so the process of determining the violation result may be: and restoring the limb action of the recognition object according to the intermediate characteristic, and restoring the limb action of the recognition object when the recognition object is collected according to the limb action, wherein the limb action can be a production line production operation behavior, such as a processing action or a processing action combination. And identifying whether the limb action is a violation action by using a preset violation action identification rule. For example, a key node combination at each time can be determined, the current limb posture is determined according to the key node combination, and whether the limb postures at multiple times are violated is determined according to the change amplitude condition and the change speed condition of the limb postures at multiple times. In other embodiments, the data of the intermediate feature may be directly identified according to a back-end processing layer of the violation prediction model, for example, the intermediate feature may be in a form of a multidimensional vector, that is, the multidimensional vector may be directly identified according to a preset violation action identification rule, so as to determine whether the violation action is a violation. The final output violation result may include whether the violation result is identified, the violation probability, the violation type attribute, and the like.

Therefore, the terminal can process the acquired image to be identified to obtain the intermediate features with smaller data volume and then send the intermediate features to the cloud, so that the violation results are finally obtained through cloud processing; therefore, data transmission quantity between the terminal and the cloud end is reduced, the operation quantity required by the violation result is obtained by the cloud end, the violation result is accurately obtained, the processing speed is increased, and the pressure of the cloud end is reduced.

Example four

Fig. 4 is a schematic flow chart of a violation prediction model training method provided in the fourth embodiment. For clearly describing the method for training the violation prediction model provided by the embodiment, please refer to fig. 4.

In an embodiment, the violation prediction model provided in this embodiment includes a front-end processing layer and a back-end processing layer, where the front-end processing layer is configured to perform feature extraction on input information, so as to obtain an intermediate feature; the back-end processing layer is used for processing the intermediate features to obtain violation results. It is worth noting that the violation prediction model training method provided by the embodiment is mainly used for training a back-end processing layer of the violation prediction model, namely, adjusting parameters in the back-end processing layer to obtain a more accurate violation result.

Step S410: and acquiring sample information, wherein the sample information comprises violation state attributes.

Step S420: and inputting the sample information into the initial violation prediction model for processing to obtain a prediction violation result, wherein the prediction violation result comprises the attribute of the prediction violation state and the violation probability.

In an embodiment, the sample information is a determination result, including a violation state attribute, and the violation attribute information may specifically be a determination result of whether the violation is a violation, a violation category attribute, or the like. The specific content form can be the same as the intermediate feature, and is a multi-dimensional vector form. In other embodiments, the sample information may be input into the front-end processing layer to obtain the predicted intermediate feature, and then the process of inputting the predicted intermediate feature into the initial violation prediction model may be to input the predicted intermediate feature into the back-end processing layer of the initial violation prediction model. The sample information is input into the initial violation prediction model for processing, and the obtained violation prediction result is the same as the method for obtaining the violation result through the intermediate feature in the foregoing, and the description can be referred to in the foregoing, and is not repeated herein.

Step S430: and acquiring the target deviation of the prediction violation result and the sample information.

Step S440: and performing iterative parameter adjustment on the initial violation prediction model according to the target deviation until the target deviation is smaller than a preset deviation threshold value, so as to obtain the violation prediction model.

In an embodiment, the sample information is input into the initial violation prediction model for processing, and the obtained violation prediction result is the same as the method for obtaining the violation result through the intermediate feature in the foregoing, which can refer to the foregoing description and is not described herein again. Wherein the obtained predicted violation results include the predicted violation status attribute and the violation probability. Because the same content is contained, the target deviation of the violation prediction result and the sample information can be obtained, so that the model parameters and the intermediate characteristics of the rear-end processing layer can be adjusted according to the target deviation, and iteration is repeated until the finally obtained target deviation is smaller than the threshold value. And feeding back the adjusted intermediate characteristics to the front-end processing layer, adjusting the model parameters of the front-end processing layer according to the deviation of the intermediate characteristics before and after adjustment, and performing iteration until the prediction accuracy of the initial violation prediction model meets the preset condition. Wherein the model parameter may be a threshold parameter or a slope parameter. Through joint training, the decoupled front-end processing layer and the rear-end processing layer can form a violation prediction model.

In one embodiment, it is mentioned that the image to be recognized may be a panoramic image including the recognition object under a plurality of viewing angles. Therefore, the model training of the panoramic image can be carried out for the training method of the violation prediction model. Specifically, the first training sample is collected at the dominant visual angle, the second training sample is collected at the recessive visual angle, the camera visual angle of the violation detection is carried out by the production line when the dominant visual angle is the operation, and the relative visual angle of the camera of the violation detection is carried out by the production line when the recessive visual angle is the operation. And constructing an initial violation prediction model taking the first training sample as an input item and a reference model taking the first training sample and the second training sample as input items, wherein the initial violation prediction model and the reference model have the same output layer. And training the reference model by using the first training sample and the second training sample, training a violation prediction model by using the first training sample, correcting the violation prediction model by using the target deviation during training the reference model, and performing iterative training. The specific training process is the same as the training mode described above, that is, the first training sample and/or the second training sample is processed by the front-end processing layer to obtain an intermediate feature, and then the intermediate feature is processed by the back-end processing layer to obtain a violation result. The violation prediction model is corrected by utilizing the target deviation in training the reference model, so that the violation prediction model can learn the hidden characteristics under the invisible visual angle, the prediction can be carried out under the panoramic visual angle only with the input of the local visible visual angle, and the identification accuracy of the violation result is improved.

In an embodiment, the method provided in this embodiment further includes: acquiring output data quantity of each processing layer of the violation prediction model, and arranging the processing layers according to the size of the output data quantity; acquiring a dynamic output data volume threshold according to the output data volume; taking the processing layer with the output data volume larger than the dynamic output data volume threshold value as a front-end processing layer and distributing the front-end processing layer to the terminal; and taking the processing layer with the output data volume less than or equal to the dynamic output data volume threshold value as a back-end processing layer, and distributing the back-end processing layer to the cloud.

In one embodiment, it is understood that the violation prediction model includes multiple processing layers, and therefore the front-end processing layer and the back-end processing layer are reasonably configured among the multiple processing layers to reduce the data volume of the back-end processing layer in the cloud. Specifically, the data amount output by each processing layer is calculated in the order of the processing layers from front to back, until the data amount is smaller than the data amount threshold of the output feature, the processing layer smaller than the threshold is used as the back-end processing layer, and the processing layer larger than the threshold is used as the front-end processing layer. The data volume threshold of the output characteristic can be a dynamic threshold, the data volume threshold of the output characteristic can be adjusted according to the load state of the server, the base model is transferred between the front end processing layer and the rear end processing layer, and then dynamic deployment of the violation identification model is achieved. Specifically, when the load state of the cloud increases, in order to maintain the reduced load state, the base model deployed in the first layer processing layer of the back-end processing layer may be moved to the front end.

Therefore, the dynamic deployment can be carried out on the multiple processing layers of the violation prediction model so as to reduce the operation pressure of the cloud. And the initial violation prediction model is decoupled into a front-end processing layer and a rear-end processing layer by a joint training mode and is trained respectively, and parameters are adjusted iteratively to finally obtain the violation prediction model meeting the preset conditions, so that the efficiency of data processing and the accuracy of finally obtaining violation results are improved.

EXAMPLE five

Fig. 5 is a block diagram illustrating a structure of a violation behavior recognition system according to the fifth embodiment. For clarity of description of the violation identification system provided in this embodiment, please refer to fig. 1-3 and 5.

The violation behavior recognition system 50 provided by this embodiment includes a terminal a510 and a cloud a520.

In one embodiment, terminal a510 includes at least one processor, and at least one memory. Wherein the at least one processor may be referred to as a processing unit and the at least one memory may be referred to as a memory unit. In particular, the storage unit stores a computer program which, when executed by the processing unit, causes the terminal provided by the present embodiment to implement the steps of the violation behavior recognition method of the terminal as applied in the foregoing.

In one embodiment, cloud a520 includes at least one processor, and at least one memory. Wherein the at least one processor may be referred to as a processing unit and the at least one memory may be referred to as a memory unit. Specifically, the storage unit stores a computer program, and when the computer program is executed by the processing unit, the terminal provided by this embodiment implements the steps of the violation behavior recognition method applied in the cloud as described above.

In one embodiment, the present application proposes a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the method as described in the first, second, third or fourth embodiment.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include non-volatile and/or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims (10)

1. A violation behavior identification method is applied to a terminal and is characterized by comprising the following steps:

acquiring an image to be identified;

inputting the image to be recognized into a front-end processing layer of a violation prediction model to obtain an intermediate characteristic;

and sending the intermediate characteristic to a cloud end so that the cloud end can identify a violation result according to the intermediate characteristic.

2. The violation behavior recognition method of claim 1 wherein said image to be recognized is input before being input into a front-end processing layer of a trained violation prediction model, said method further comprising:

when the image to be identified is obtained, audio information which is collected at the same time as the image to be identified is obtained;

inputting the image to be recognized into a sound prediction model to obtain predicted audio information;

judging whether the image to be identified is a real image or not according to the audio information and the predicted audio information;

and when the image to be recognized is a real image, inputting the image to be recognized into the front-end processing layer.

3. The violation identification method of claim 1 wherein said method further comprises:

extracting facial features in the image to be recognized, and determining identity information corresponding to the image to be recognized according to the facial features;

matching a corresponding private key according to the identity information, and signing the intermediate feature by using the private key to obtain a signed feature;

and packaging and sending the signed features and the intermediate features to the cloud.

4. A violation behavior identification method is applied to a cloud, and is characterized by comprising the following steps:

acquiring intermediate features sent by a terminal, wherein the intermediate features are acquired by processing an image to be recognized by the terminal;

and inputting the intermediate features into a rear-end processing layer of the violation prediction model to identify and obtain a violation result.

5. The violation identification method of claim 4 wherein prior to obtaining the intermediate signature transmitted by the terminal, the method further comprises:

acquiring identity information of an operator;

acquiring a public key and a private key matched with the identity information according to the identity information;

sending the identity information, the private key and the matching relation between the identity information and the private key to the terminal, so that the terminal signs the intermediate feature according to the identity information to obtain a signed feature;

and receiving the signature adding feature sent by the terminal.

6. The violation behavior recognition method of claim 5 wherein prior to entering said intermediate features into a back-end processing layer of a trained violation prediction model, said method further comprises:

the public key is used for carrying out label release on the signature adding characteristic to obtain a label release characteristic;

when the de-signing feature and the intermediate feature are consistent, inputting the intermediate feature into the back-end processing layer.

7. A method for identifying violation behaviors is characterized by comprising the following steps:

a terminal acquires an image to be identified;

the terminal inputs the image to be recognized into a front-end processing layer of a violation prediction model to obtain intermediate characteristics and sends the intermediate characteristics to a cloud end;

and after the cloud end acquires the intermediate features, inputting the intermediate features into a rear-end processing layer of the violation prediction model so as to identify and obtain violation results.

8. A violation prediction model training method is characterized by comprising the following steps:

acquiring sample information, wherein the sample information comprises violation state attributes;

inputting the sample information into an initial violation prediction model for processing to obtain a prediction violation result, wherein the prediction violation result comprises a prediction violation state attribute and a violation probability;

acquiring a target deviation between the predicted violation result and the sample information; and iteratively adjusting parameters of the initial violation prediction model according to the target deviation until the target deviation is smaller than a preset deviation threshold value, so as to obtain a violation prediction model.

9. The method of violation prediction model training of claim 8 wherein the method further comprises:

acquiring output data quantity of each processing layer of the violation prediction model, and arranging the processing layers according to the size of the output data quantity;

acquiring a dynamic output data volume threshold according to the output data volume;

taking the processing layer with the output data volume larger than the dynamic output data volume threshold value as a front-end processing layer and distributing the front-end processing layer to the terminal; and taking the processing layer with the output data volume less than or equal to the dynamic output data volume threshold value as a back-end processing layer, and distributing the back-end processing layer to the cloud.

10. A violation behavior recognition system is characterized by comprising a terminal and a cloud end;

the terminal is configured to perform the steps of the method according to any one of claims 1 to 3;

the cloud is used for executing the steps of the method according to any one of claims 4 to 6.

Images