CN110163084A - Operator action measure of supervision, device and electronic equipment - Google Patents

Abstract

The invention discloses a kind of Operator action measure of supervision, device and electronic equipments, the Operator action measure of supervision is executed by electronic equipment, the described method includes: obtaining the image of operator, movement when described image is to operator work carries out shooting generation；Based on key position detection model, movement when working operator described in described image carries out key position detection, obtains trace image, and the trace image acts the movement locus formed in described image when being used to indicate operator work；The trace image input trajectory identification model is subjected to track identification, obtains track identification result；If the track identification result indicates that the movement when operator works is unqualified, alarm information is generated.Solve the problems, such as that Operator action supervision in the prior art is realized dependent on artificial using Operator action measure of supervision, device and electronic equipment provided by the present invention.

Description

Operator action measure of supervision, device and electronic equipment

Technical field

The present invention relates to field of computer technology more particularly to a kind of Operator action measure of supervision, device and electronics to set It is standby.

Background technique

In industrial circle, manufacturer, in order to avoid generating weight huge economic loss, needs to before delivery to customer product Product is detected.

Currently, the detection of product is mainly completed by operator, specifically, operator according to regulation motion detection product, Occur excessive defect to avoid product, and causes the yield of product too low.Work habit due to each operator etc. because Element, it is possible that the case where product is non-defective unit or substandard products can not really be distinguished, therefore, when working for each operator Movement, also by rely on special messenger verify, it is whether qualified with the movement of this monitor operator.

However, inventors realized that aforesaid operations person act supervision still rely on artificial realization, it is time-consuming and laborious, still have effect The low problem of rate.

Summary of the invention

In order to solve the problems, such as that the supervision of Operator action present in the relevant technologies is realized dependent on artificial, each reality of the present invention It applies example and a kind of Operator action measure of supervision, device, electronic equipment and storage medium is provided.

Wherein, the technical scheme adopted by the invention is as follows:

A kind of one side according to an embodiment of the present invention, Operator action measure of supervision, is executed by electronic equipment, the side Method includes: the image for obtaining operator, and movement when described image is to operator work carries out shooting generation；It is based on Key position detection model, movement when working operator described in described image carry out key position detection, obtain track Image, the trace image act the movement locus formed in described image when being used to indicate operator work；By institute It states trace image input trajectory identification model and carries out track identification, obtain track identification result；If the track identification result Movement when indicating operator work is unqualified, then generates alarm information.

One side according to an embodiment of the present invention, a kind of Operator action monitoring apparatus are deployed in electronic equipment, the dress Setting includes: image collection module, for obtaining the image of operator, movement when described image is to operator work into Row shooting generates；Key position detection module, for being based on key position detection model, to operator described in described image Movement when work carries out key position detection, obtains the trace image corresponding to the movement；Track identification module is used In the trace image input trajectory identification model is carried out track anomalous identification, track identification result is obtained；Alarm module is used If the movement when track identification result indicates operator work is unqualified, alarm information is generated.

One side according to an embodiment of the present invention, a kind of electronic equipment, including processor and memory, on the memory It is stored with computer-readable instruction, the computer-readable instruction realizes operator as described above when being executed by the processor Act measure of supervision.

One side according to an embodiment of the present invention, a kind of storage medium are stored thereon with computer program, the computer Operator action measure of supervision as described above is realized when program is executed by processor.

Movement when in the above-mentioned technical solutions, by working operator carries out the image of shooting generation, first basis Key position detection model, movement when working operator in the image carry out key position detection, obtain trace image, then According to track identification model, track identification is carried out to the trace image, obtains track identification as a result, if track identification result refers to Movement when showing operator's work is unqualified, then generates alarm information, it is thus achieved that the automatic supervision of Operator action, effectively Ground solves the problems, such as that Operator action supervision is realized dependent on artificial in the prior art.

It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not It can the limitation present invention.

Detailed description of the invention

The drawings herein are incorporated into the specification and forms part of this specification, and shows and meets implementation of the invention Example, and in specification together principle for explaining the present invention.

Fig. 1 is a kind of hardware block diagram of server shown according to an exemplary embodiment.

Fig. 2 is a kind of flow chart of Operator action measure of supervision shown according to an exemplary embodiment.

Fig. 3 is the method flow diagram of model generating process in Fig. 2 corresponding embodiment.

Fig. 4 be in Fig. 2 corresponding embodiment step 230 in the flow chart of one embodiment.

Fig. 5 be in Fig. 4 corresponding embodiment step 233 in the flow chart of one embodiment.

Fig. 6 be in Fig. 2 corresponding embodiment step 250 in the flow chart of one embodiment.

Fig. 7 be in Fig. 6 corresponding embodiment step 253 in the flow chart of one embodiment.

Fig. 8 be in Fig. 6 corresponding embodiment step 253 in the flow chart of another embodiment.

Fig. 9 is a kind of realization schematic diagram of Operator action measure of supervision shown according to an application scenarios.

Figure 10 is a kind of block diagram of Operator action monitoring apparatus shown according to an exemplary embodiment.

Figure 11 is Operator action monitoring apparatus each step when carrying out Operator action supervision in Figure 10 corresponding embodiment Realization schematic diagram.

Figure 12 is the structural block diagram of a kind of electronic equipment shown according to an exemplary embodiment.

Through the above attached drawings, it has been shown that the specific embodiment of the present invention will be hereinafter described in more detail, these attached drawings It is not intended to limit the scope of the inventive concept in any manner with verbal description, but is by referring to specific embodiments Those skilled in the art illustrate idea of the invention.

Specific embodiment

Here will the description is performed on the exemplary embodiment in detail, the example is illustrated in the accompanying drawings.Following description is related to When attached drawing, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements.Following exemplary embodiment Described in embodiment do not represent all embodiments consistented with the present invention.On the contrary, they be only with it is such as appended The example of device and method being described in detail in claims, some aspects of the invention are consistent.

As previously mentioned, Operator action supervision realizes that be primarily present two schemes: the first scheme is needle dependent on artificial Operator is giveed training, so that the movement of each operator is qualified；Second scheme is made by recording operation employee When video, recycle special messenger exercise supervision to the video, whether the movement to judge operator qualified.

From the foregoing, it will be observed that it is not only time-consuming and laborious dependent on the Operator action supervision manually realized, but also can not be timely Mistake when operator's work is pointed out on ground, haves the defects that inefficiency.

For this purpose, spy of the present invention proposes a kind of Operator action measure of supervision, monitor operator can act automatically, effectively Ground improves efficiency, and correspondingly, is matched with a kind of Operator action monitoring apparatus of this kind of Operator action measure of supervision, is deployed in Has the electronic equipment of von Neumann architecture, for example, electronic equipment is personal computer, server etc., to realize operation Member's movement measure of supervision.

Fig. 1 is a kind of hardware block diagram of server shown according to an exemplary embodiment.This kind of server can be held Row Operator action measure of supervision.

It should be noted that this kind of server, which is one, adapts to example of the invention, it must not believe that there is provided right Any restrictions of use scope of the invention.This kind of server can not be construed to need to rely on or must have in Fig. 1 One or more component in illustrative server 100 shown.

The hardware configuration of server 100 can generate biggish difference due to the difference of configuration or performance, as shown in Figure 1, Server 100 include: power supply 110, interface 130, at least a memory 150 and an at least central processing unit (CPU, Central Processing Units)170。

Specifically, power supply 110 is used to provide operating voltage for each hardware device on server 100.

Interface 130 includes an at least wired or wireless network interface, for interacting with external equipment.For example, carrying out Fig. 1 institute Interaction in implementation environment between control node 110 and calculate node 130 is shown.

Certainly, in the example that remaining present invention is adapted to, interface 130 can further include an at least serioparallel exchange and connect 133, at least one input/output interface 135 of mouth and at least usb 1 37 etc., as shown in Figure 1, herein not to this composition It is specific to limit.

The carrier that memory 150 is stored as resource, can be read-only memory, random access memory, disk or CD Deng the resource stored thereon includes operating system 151, application program 153 and data 155 etc., and storage mode can be of short duration It stores or permanently stores.

Wherein, operating system 151 be used for manage and control server 100 on each hardware device and application program 153, To realize operation and processing of the central processing unit 170 to mass data 155 in memory 150.

Application program 153 is the computer program based at least one of completion particular job on operating system 151, can To include that an at least module (not shown in figure 1), each module can separately include the series of computation to server 100 Machine readable instruction.For example, Operator action monitoring apparatus can be considered the application program 153 for being deployed in server 100.

Data 155 can be image, trace image etc., be stored in memory 150.

Central processing unit 170 may include the processor of one or more or more, and be set as total by least one communication Line is communicated with memory 150, to read the computer-readable instruction stored in memory 150, and then is realized in memory 150 The operation and processing of mass data 155.For example, reading the series of computation stored in memory 150 by central processing unit 170 The form of machine readable instruction completes data distributing method.

In addition, also can equally realize the present invention by hardware circuit or hardware circuit combination software, therefore, this hair is realized The bright combination for being not limited to any specific hardware circuit, software and the two.

Referring to Fig. 2, in one exemplary embodiment, a kind of Operator action measure of supervision is suitable for electronic equipment, example Such as, electronic equipment is server, the structure of the server can be as shown in Figure 1.

This kind of Operator action measure of supervision can be executed by electronic equipment, may comprise steps of:

Step 210, the image of operator is obtained.

Wherein, movement when described image is to operator work carries out shooting generation.

That is, place working region is laid with the camera shooting with image collecting function and sets in operator's work It is standby, for example, video camera, video recorder, monitor, the smart phone for being even mounted with camera etc., then, when operator works Movement will be shot and be acquired by picture pick-up device.

About the acquisition of image, the image that picture pick-up device captured in real-time acquires and is uploaded to electronic equipment can be, it can also To be pre-stored image in electronic equipment, i.e., a historical time section is set by imaging in the memory by reading electronic equipment The standby image for shooting and acquiring, the present embodiment are limited not to this.

It should be noted that image, can refer to one section of video that continuity shooting generates, can also refer to noncontinuity Several pictures generated are shot, in other words, in the present embodiment, the image that electronic equipment carries out Operator action supervision can be with It is one section of video, is also possible to several pictures.

It further remarks additionally, no matter image is one section of video or several pictures, and electronic equipment is being grasped It is to be carried out as unit of picture frame, for example, picture frame is the frame video figure in one section of video when work person acts supervision Picture, alternatively, the picture in several pictures.

Step 230, it is based on key position detection model, movement when working operator described in described image is closed Key location detection, obtains trace image.

As previously mentioned, the detection of product is completed by operator, and compulsory exercise when operator's work is for each Product to be detected, in other words, it is specified that movement is different for each product to be detected, correspondingly, crucial portion Position will be also different.

For example, the components of some product include screw, screen etc., at this point, operator to some product into When row detection, need to be detected for screw, screen and product appearance etc. respectively, i.e., operator needs to move according to the rules It is screwed respectively, screen inspection, product packaging etc..

Based on this, when being screwed, key position is the hand of operator；It is crucial when carrying out screen inspection Position is the head of operator；When carrying out product packaging, key position is also the hand of operator.

In the present embodiment, the detection of key position is realized based on key position detection model.

The key position detection model is to carry out backpropagation training life by parameter of the sample to be learned to basic model At.Wherein, sample to be learned is the image for having carried out key position mark.

That is, the key position detection model, constructs the mapping relations between image and key position, so as to base Corresponding key position is obtained by image prediction in the mapping relations.

The generating process of key position detection model is illustrated below.

In the realization of an embodiment, as shown in figure 3, method as described above can with the following steps are included:

Step 310, the sample to be learned is obtained.

Wherein, sample to be learned is the image for having carried out key position mark.

Step 330, according to the sample to be learned, the parameter of backpropagation training basic model.

Backpropagation training, is to be iterated update by parameter of the sample to be learned to basic model, so that basic Model convergence.

Wherein, basic model, including but not limited to: logistic regression, support vector machines, random forest, neural network etc..

Step 350, it if the parameter of the basic model restrains the basic model, is restrained by the basic model Obtain the machine learning model.

Convergence refers to and meets the condition of convergence by the algorithmic function of the parameter building of basic model.Wherein, which can To be greatest hope function, loss function etc. can also be.

Below based on loss function, the process of backpropagation training is described.

The parameter of random initializtion basic model calculates the parameter by random initializtion according to when previous sample to be learned The penalty values of the loss function of building.

If the penalty values of loss function are not up to minimum, the parameter of basic model is updated, and wait learning according to the latter Practise the penalty values for the loss function that sample is calculated by the parameter building updated.

Such iteration is considered as loss function and meets the condition of convergence until the penalty values of loss function reach minimum, this When, basic model is also considered as convergence, then stops iteration.

Otherwise, iteration updates the parameter of basic model, and the parameter by updating is iterated to calculate according to remaining sample to be learned The penalty values of the loss function of building, until loss function meets the condition of convergence.

It is noted that if the number of iterations has reached setting iteration before loss function meets the condition of convergence Threshold value will also stop iteration, guarantee the efficiency of backpropagation training with this.

When basic model convergence, indicate that backpropagation training is completed, and then can be restrained and be closed by basic model Key location detection model, which is provided with key position detectability, by operator in image Movement when work carries out key position detection and obtains trace image.

Wherein, the movement rail formed in described image is acted when the trace image is used to indicate operator work Mark.

Step 250, the trace image input trajectory identification model is subjected to track identification, obtains track identification result.

Track identification is acted the movement locus formed in the picture when being operator's work for identification, is judged with this Whether movement when operator works is qualified.

In the present embodiment, track identification is based on track identification model realization.

Similarly in key position detection model, which is the ginseng by sample to be learned to basic model Number carries out what backpropagation training generated.Wherein, sample to be learned is to have carried out movement locus mark and/or abnormal attribute mark The image of note.

Herein it should be noted that the movement locus marked in image can be regular event track, i.e., operator according to Compulsory exercise carries out product testing, can also be abnormal operation track, i.e., movement does not carry out product testing to operator according to the rules, Correspondingly, when the movement locus marked in the picture is abnormal operation track, abnormal attribute will be also marked in image, will be described with this The reason of forming the abnormal operation track.

It further illustrates, which can be indicated by character strings such as number, letters.For example, abnormal belong to Property be 1, indicate that abnormal operation track is partially short, be because operator is perfunctory to rapidly testing product to promote workload and causes Movement range is less than normal；Abnormal attribute is 2, indicate abnormal operation track missing, be because operator stay away from work without leave or take liberty with one's job and Lack part to detect；Abnormal attribute is 3, indicates that abnormal operation track is on the high side and deviates regular event track, is because of operation Member is careless, half-hearted etc. and movement is caused to repeat, and the present embodiment not constitutes specific restriction to this.

Based on this, track identification model substantially constructs mapping relations between image and movement locus and abnormal dynamic Make the mapping relations between track and abnormal attribute, to be predicted to obtain corresponding rail by trace image based on those mapping relations Mark recognition result.

The generating process of track identification model and the generating process of key position detection model are almost the same, no longer heavy herein Multiple description.

Wherein, track identification is as a result, whether the movement being used to indicate when operator's work is qualified.

If track identification result indicates that movement when operator works is unqualified, jumps and execute step 270.

, whereas if the movement when instruction operator's work of track identification result is qualified, then to operator's work when after Continuous movement continues to supervise.

Step 270, if the track identification result indicates that the movement when operator works is unqualified, announcement is generated Alert message.

Based on alarm information, mistake when operator's work, and then raising efficiency can be pointed out in time.

Optionally, acting underproof track identification result and alarm information when instruction operator work will store to electronics In equipment, in order to trace problem, perfect management, summary process, improvement production technology etc. during subsequent production, with further Ground promotes work quality and efficiency.

Optionally, in order to ensure the reliability of Operator action supervision, only when the instruction operator's work of track identification result When the underproof number of movement be more than given threshold when, just generation alarm information.Certainly, which can be according to difference Application scenarios neatly adjust, for example, given threshold is larger in the scene more demanding to work quality, not structure herein It is limited at specific.

By process as described above, the automatic supervision acted when operator's work is realized, is avoided relying in artificial real It is existing, work quality and efficiency is effectively promoted.

Referring to Fig. 4, in one exemplary embodiment, step 230 may comprise steps of:

Step 231, the movement corresponding working attributes information when determining operator work, according to the work category Property acquisition of information be it is described movement setting key position information.

As described in above-mentioned example, for components include some product of screw, screen, operator need according to Compulsory exercise is screwed respectively, screen inspection, product packaging etc., completes the detection to some product with this.

Herein, the screwed lock, screen inspection, product packaging, that is, act corresponding working attributes when being considered as operator's work Information, it is understood that be working attributes information, for accurately describe operator work when movement, can by number, The character strings such as letter are indicated.For example, working attributes information is A1, movement when operator is screwed is indicated；Work Making attribute information is A2, indicates movement when operator carries out screen detection；Working attributes information is A3, indicates that operator carries out Movement when product packaging, the present embodiment not constitute specific limit to this.

Still as described in previous example, when being screwed, key position is the hand of operator；Carrying out screen When inspection, key position is the head of operator；When carrying out product packaging, key position is also the hand of operator.

Herein, hand, head, hand, that is to say, be the key position of the corresponding movement setting of different operating attribute information Information, then, key position information acts corresponding key position when being used to indicate operator's work.

Movement, key position information based on this, for electronic equipment, when working attributes information, operator work Between substantially there is corresponding relationship, then, when working attributes information determine, can be known by working attributes information operator work When movement, movement when further being worked by operator knows corresponding key position, i.e. key position information.

Step 233, the key position information and described image are inputted into the key position detection model, output obtains The trace image.

Specifically, as shown in figure 5, step 233 may comprise steps of in the realization of an embodiment:

Step 2331, each picture frame for including for described image is based on the key position detection model, to this Pixel in picture frame carries out regression forecasting, obtains the response that pixel in the picture frame is directed to the key position information.

Regression forecasting is substantially to calculate pixel in picture frame by key position detection model and belong to key position information The response of the key position of instruction.Response is bigger, then the pixel belongs to the key that key position information indicates in picture frame The probability at position is bigger.

Step 2333, classified according to the response to the pixel in the picture frame, obtain the key position letter Cease key position region of the key position of instruction in the picture frame.

As an example it is assumed that the key position of key position information instruction is head, then the classification of pixel includes header area Domain classification and non-head area classification.

It is further assumed that it is P1 that pixel, which is directed to the response of the key position information, in picture frame, and set for head Response lag be P2.

If P1 > P2, then it represents that the classification of the pixel is head zone classification in picture frame, be that is to say, which is located at Head zone in picture frame.

, whereas if P1 < P2, then it represents that the classification of the pixel is non-head area classification in picture frame, be that is to say, the picture Element is located at the non-head region in picture frame.So, classify for each pixel in picture frame, can determine and belong to head Which the pixel of portion's area classification has, it is understood that is to belong to the pixel of head zone classification to constitute in picture frame Head zone, i.e., the key position region in picture frame.

Remark additionally herein, key position region refers to position of the key position in picture frame, be by coordinate into What row indicated.

Step 2335, the trace image is spliced to form by the key position region in all picture frames.

After having obtained the key position region in each picture frame, according to the time sequencing of picture frame, it can spell It connects to form corresponding trace image.In other words, from timing, trace image is substantially indicated by key position in institute The track for having the position in picture frame to be formed, the track act corresponding movement locus when being operator's work.

Under the action of above-described embodiment, foundation is provided for subsequent track identification, so that monitor operation employee Whether qualification is achieved for movement when making.

Referring to Fig. 6, in one exemplary embodiment, step 250 may comprise steps of:

Step 251, it is based on the track identification model, classification prediction is carried out to the trace image, obtains the track The classification of image.

Classification prediction is substantially the probability for calculating trace image and belonging to a different category.

In the present embodiment, the classification of trace image includes: normal category and abnormal class.The normal category refers to trajectory diagram The movement locus of picture instruction belongs to regular event track, which refers to that the movement locus of trace image instruction is abnormal dynamic Make track.

As an example it is assumed that it is P3 that the trace image, which is calculated, to belong to the probability of normal category, and the trace image category In abnormal class probability be P4.

If P3 > P4, then it represents that the trace image belongs to normal category, i.e. the movement locus of trace image instruction belongs to just Normal movement locus.

, whereas if P3 < P4, then it represents that the trace image belongs to abnormal class, the i.e. movement locus of trace image instruction Belong to abnormal operation track.

Step 253, according to the classification of the trace image, the track identification result is generated.

If the classification of trace image is normal category, generates and indicate to act the described of qualification when operator work Track identification result.

, whereas if the classification of trace image is abnormal class, then generates movement when indicating operator work and do not conform to The track identification result of lattice.

Separately below based on different classes of trace image, the generating process of track identification result is illustrated.

As shown in fig. 7, step 253 may comprise steps of in the realization of an embodiment:

Step 2531, if the classification of the trace image is normal category, it is determined that described in when the operator works Corresponding working attributes information is acted, is the standard trajectory image of the movement setting according to the working attributes acquisition of information.

Wherein, the movement rail formed when the standard trajectory image is used to indicate the operator based on compulsory exercise work Mark.

As described in above-mentioned example, for components include some product of screw, screen, operator need according to Compulsory exercise is screwed respectively, screen inspection, product packaging etc., completes the detection to some product with this.Accordingly Ground, standard operation trace image be used to indicate operator act according to the rules be screwed respectively, screen examine, product packet The movement locus formed when dress.

The standard trajectory image in the picture can work to operator based on compulsory exercise from artificial mark When the movement locus that is formed be labeled.

As previously mentioned, working attributes information, accurately describes movement when operator's work, then, in working attributes After information determines, compulsory exercise when operator's work can be known, by working attributes information further to know operation The movement locus formed when member is based on compulsory exercise work, i.e. standard trajectory image.

Step 2533, the similarity operation between the trace image of normal category and the standard trajectory image, root are carried out Track score is set according to the trace image that similarity operation result is normal category.

Similarity operation is substantially the movement locus and standard trajectory figure indicated for the trace image of normal category For the movement locus of picture instruction, it is understood that be, for calculating the similarity between above-mentioned two movement locus, in order to Corresponding track score is set for the trace image of normal category.

For example, setting method is equal proportion conversion, that is, if similarity is 0.8, track score is set as 0.8 × 100 =80 points；If similarity is 0.99, track score is set as 0.99 × 100=99 points.

Step 2535, when the track score is more than setting score, generates instruction operator work, movement is qualified The track identification result.

According to setting score, it can determine that movement when operator's work is by the track identification result that track score generates No qualification.

As an example it is assumed that score is set as 95, then, track score is 80 timesharing, the instruction operation of track identification result Movement when employee makees is unqualified.And track score is 99 timesharing, track identification result indicates that movement when operator's work is closed Lattice.

Certainly, consider the treatment effeciency of electronic equipment, in other embodiments, track identification result can also be directly by phase It is generated like degree, for example, setting similarity threshold as 0.95, then when similarity is 0.8, the instruction operator's work of track identification result When movement it is unqualified.And similarity is when being 0.99, movement when track identification result indicates operator's work is qualified, herein simultaneously Non- composition is specific to be limited.

The trace image based on normal category is completed as a result, generates the process of track identification result.

In addition, also further being carried out after identifying that the movement locus of trace image instruction is normal movement locus Similarity operation, that is to say, further identify whether the regular event track belongs to standard operation track, fully ensure that The accuracy of track identification, and then fully ensured the accuracy of Operator action supervision.

As shown in figure 8, step 253 may comprise steps of in the realization of another embodiment:

Step 2532, it if the classification of the trace image is abnormal class, is provided based on the track identification model Abnormal attribute, obtain the corresponding abnormal attribute of trace image of abnormal class.

It is to have carried out movement locus mark and/or different as previously mentioned, corresponding to the sample to be learned of track identification model The image of normal attribute labeling, track identification model as a result, can not only identify trace image instruction movement locus whether be Abnormal operation track, and can be identified after the movement locus for recognizing trace image instruction is abnormal operation track The reason of forming the abnormal operation track.

Step 2534, according to the trace image of abnormal class and its abnormal attribute, when generating instruction operator work Act the underproof track identification result.

The trace image based on abnormal class is completed as a result, generates the process of track identification result.

By the above process, based on track identification as a result, it is possible to judge whether movement when operator's work closes in time Lattice avoid relying on and realize in artificial, effectively improve efficiency.

In addition, based on different classes of trace image different track identifications will be generated as a result, being conducive to improve operator Act enhanced scalability and the flexibility of supervision.

In one exemplary embodiment, before step 2534, method as described above can with the following steps are included:

The equipment state of working equipment when obtaining operator work.

Wherein, equipment state includes normal operating condition and abnormal operating condition.The normal operating condition is used to indicate work Make equipment normal operation in operator's work, it is abnormal in operator's work which is used to indicate working equipment Operation, for example, the equipment state of working equipment is abnormal operating condition if working equipment failure or crash.

If the equipment state is normal operating condition, that is, indicate the working equipment when the operator works just Often operation then jumps and executes step 2534.

Under the cooperation of above-described embodiment, the reliability of Operator action supervision is sufficiently ensured, has avoided because of working equipment Misoperation and the erroneous judgement that acts when causing to work to operator.

One of various embodiments of the present invention Operator action measure of supervision is suitable for different application scenarios, and this kind is grasped Each step that work person acts in measure of supervision can follow different application scenarios correspondingly to be adjusted.

Referring to Fig. 9, for a kind of Operator action supervision side set in an application scenarios for some product line Method, each step in this kind of Operator action measure of supervision are as shown in Figure 9.

Specifically, for the video of the operator acquired in real time, it is first determined whether can be detected based on key position Model obtains trace image.

If it is not, then recording this supervision as a result, and continuing to acquire the video of operator and carrying out subsequent supervision.

Conversely, the movement locus then based on trace image instruction, carries out closing rule judgement according to track identification model, with basis Track identification result judges whether movement when operator works is qualified.

If qualified, this supervision is recorded as a result, and continuing to acquire the video of operator and carrying out subsequent supervision.

Conversely, then carrying out irregularity analysis based on the abnormal operation track recognized, correlation is reminded to generate alarm information Personnel make the unqualified movement of its timely correction for example, related personnel can be operator, while recording this supervision as a result, simultaneously The video for continuing to acquire operator carries out subsequent supervision.

Following is apparatus of the present invention embodiment, can be used for executing Operator action measure of supervision according to the present invention. For undisclosed details in apparatus of the present invention embodiment, the side of Operator action measure of supervision according to the present invention is please referred to Method embodiment.

Referring to Fig. 10, in one exemplary embodiment, a kind of Operator action monitoring apparatus 900 is deployed in electronics and sets Standby, which includes but is not limited to: image collection module 910, key position detection module 930, track identification module 950 With alarm module 970.

Wherein, image collection module 910, for obtaining the image of operator, described image is worked the operator When movement carry out shooting generation.

Key position detection module 930, for being based on key position detection model, to operating staff described in described image Movement when making carries out key position detection, obtains the trace image corresponding to the movement.

Track identification module 950 is obtained for the trace image input trajectory identification model to be carried out track anomalous identification To track identification result.

Alarm module 970, if indicating that movement when operator work is unqualified for the track identification result, Then generate alarm information.

In one exemplary embodiment, the key position detection module 930 includes but is not limited to: information acquisition unit and Image output unit.

Wherein, information acquisition unit, it is described when for determining operator work to act corresponding working attributes information, It is the key position information of the movement setting according to the working attributes acquisition of information.

Image output unit detects mould for the key position information and described image to be inputted the key position Type, output obtain the trace image.

In one exemplary embodiment, described image output unit includes but is not limited to: response predicts subelement, classification Subelement and splicing subelement.

Wherein, response predicts subelement, each picture frame for including for described image is based on the key Location detection model carries out regression forecasting to the pixel in the picture frame, obtains pixel in the picture frame and is directed to the key portion The response of position information.

Classification subelement obtains the key for classifying according to the response to the pixel in the picture frame Key position region of the key position of location information instruction in the picture frame.

Splice subelement, for being spliced to form the trace image by the key position region in all picture frames.

In one exemplary embodiment, the track identification module 950 includes but is not limited to: classification predicting unit and result Generation unit.

Wherein, predicting unit of classifying classify to the trace image pre- for being based on the track identification model It surveys, obtains the classification of the trace image.

As a result generation unit generates the track identification result for the classification according to the trace image.

In one exemplary embodiment, the result generation unit includes but is not limited to: image obtains subelement, similarity Operation subelement and the first result generate subelement.

Wherein, image obtains subelement, if the classification for the trace image is normal category, it is determined that the behaviour It is described when making as employee to act corresponding working attributes information, it is the movement setting according to the working attributes acquisition of information Standard trajectory image, the movement rail that the standard trajectory image is formed when being used to indicate the operator based on compulsory exercise work Mark.

Similarity operation subelement, for carrying out the phase between the trace image of normal category and the standard trajectory image Like degree operation, track score is set according to the trace image that similarity operation result is normal category.

First result generates subelement, for being more than setting score when the track score, generates and indicates the operator The qualified track identification result is acted when work.

In a further exemplary embodiment, the result generation unit includes but is not limited to: abnormal attribute obtains subelement Subelement is generated with the second result.

Wherein, abnormal attribute obtains subelement, if the classification for the trace image is abnormal class, is based on institute The abnormal attribute for stating the offer of track identification model, obtains the corresponding abnormal attribute of trace image of abnormal class.

Second result generates subelement, for the trace image and its abnormal attribute according to abnormal class, generates instruction institute The underproof track identification result is acted when stating operator's work.

In one exemplary embodiment, the result generation unit further includes but is not limited to: equipment state obtains module.

Wherein, equipment state obtains module, the equipment state of working equipment when for obtaining operator work.

If the equipment state indicates that the working equipment is operated normally when the operator works, jumps and hold Row: the trace image and its abnormal attribute according to abnormal class acts unqualified when generating instruction operator work The track identification result.

In one exemplary embodiment, the machine learning model includes the key position detection model, the track Identification model.

Sample to be learned include corresponding to the key position detection model the first sample to be learned, correspond to the rail The sample to be learned of the second of mark identification model.

First sample to be learned is the image for having carried out key position mark.

Second sample to be learned be carried out movement locus mark, and/or, carried out abnormal attribute mark figure Picture.

Described device 900 further includes but is not limited to: sample acquisition module, parameter training module and model restrain module.

Wherein, sample acquisition module, for obtaining the sample to be learned.

Parameter training module, for according to the sample to be learned, the parameter of backpropagation training basic model.

Model restrains module, if the parameter for the basic model restrains the basic model, by the base Plinth model restrains to obtain the machine learning model.

It should be noted that Operator action monitoring apparatus provided by above-described embodiment is carrying out Operator action supervision When, each step as shown in figure 11, only the example of the division of the above functional modules, in practical application, Ke Yigen Above-mentioned function distribution is completed by different functional modules according to needs, i.e., the internal structure of Operator action monitoring apparatus will be drawn It is divided into different functional modules, to complete all or part of the functions described above.

In addition, the embodiment of Operator action monitoring apparatus and Operator action measure of supervision provided by above-described embodiment Belonging to same design, the concrete mode that wherein modules execute operation is described in detail in embodiment of the method, Details are not described herein again.

Figure 12 is please referred to, in one exemplary embodiment, a kind of electronic equipment 1000, including but not limited at least one processing Device 1001, at least a memory 1002 and at least a communication bus 1003.

Wherein, computer-readable instruction is stored on memory 1002, processor 1001 is read by communication bus 1003 The computer-readable instruction stored in memory 1002.

Realize that the Operator action in the various embodiments described above is supervised when the computer-readable instruction is executed by processor 1001 Method.

In one exemplary embodiment, a kind of storage medium, is stored thereon with computer program, which is located Manage the Operator action measure of supervision realized in the various embodiments described above when device executes.

Above content, preferable examples embodiment only of the invention, is not intended to limit embodiment of the present invention, this Field those of ordinary skill central scope according to the present invention and spirit can be carried out very easily corresponding flexible or repaired Change, therefore protection scope of the present invention should be subject to protection scope required by claims.

Claims (10)

1. a kind of Operator action measure of supervision, which is characterized in that the method is executed by electronic equipment, which comprises

The image of operator is obtained, movement when described image is to operator work carries out shooting generation；

Based on key position detection model, movement when working operator described in described image carries out key position detection, Trace image is obtained, the trace image acts the movement rail formed in described image when being used to indicate operator work Mark；

The trace image input trajectory identification model is subjected to track identification, obtains track identification result；

If the track identification result indicates that the movement when operator works is unqualified, alarm information is generated.

2. the method as described in claim 1, which is characterized in that it is described to be based on key position detection model, in described image The movement when operator works carries out key position detection, obtains trace image, comprising:

The movement corresponding working attributes information when determining operator work is according to the working attributes acquisition of information The key position information of the movement setting；

The key position information and described image are inputted into the key position detection model, output obtains the trajectory diagram Picture.

3. method according to claim 2, which is characterized in that described that the key position information and described image are inputted institute Key position detection model is stated, output obtains the trace image, comprising:

For each picture frame that described image includes, it is based on the key position detection model, to the picture in the picture frame Element carries out regression forecasting, obtains the response that pixel in the picture frame is directed to the key position information；

Classified according to the response to the pixel in the picture frame, obtains the crucial portion of the key position information instruction Key position region of the position in the picture frame；

The trace image is spliced to form by the key position region in all picture frames.

4. the method as described in claim 1, which is characterized in that described to carry out the trace image input trajectory identification model Track identification obtains track identification result, comprising:

Based on the track identification model, classification prediction is carried out to the trace image, obtains the classification of the trace image；

According to the classification of the trace image, the track identification result is generated.

5. method as claimed in claim 4, which is characterized in that the classification according to the trace image generates the rail Mark recognition result, comprising:

If the classification of the trace image is normal category, it is determined that the operator is described when working to act corresponding work Attribute information is the standard trajectory image of the movement setting, the standard trajectory figure according to the working attributes acquisition of information As the movement locus formed when being used to indicate the operator based on compulsory exercise work；

The similarity operation between the trace image and the standard trajectory image of normal category is carried out, according to similarity operation knot Fruit is that the trace image of normal category sets track score；

The qualified track identification of movement when the track score is more than setting score, generates instruction operator work As a result.

6. method as claimed in claim 4, which is characterized in that the classification according to the trace image generates the rail Mark recognition result, comprising:

If the classification of the trace image is abnormal class, based on the abnormal attribute that the track identification model provides, obtain To the corresponding abnormal attribute of trace image of abnormal class；

According to the trace image and its abnormal attribute of abnormal class, generates and indicate to act underproof institute when operator work State track identification result.

7. method as claimed in claim 6, which is characterized in that the classification according to the trace image generates the rail Mark recognition result, further includes:

The equipment state of working equipment when obtaining operator work；

If the equipment state indicates that the working equipment is operated normally when the operator works, execution is jumped: institute The trace image and its abnormal attribute according to abnormal class are stated, is acted when generating instruction operator work underproof described Track identification result.

8. method as described in any one of claim 1 to 7, which is characterized in that the machine learning model includes the key Location detection model, the track identification model；

Sample to be learned include corresponding to the key position detection model the first sample to be learned, correspond to the track and know Second sample to be learned of other model；

First sample to be learned is the image for having carried out key position mark；

Second sample to be learned be carried out movement locus mark, and/or, carried out abnormal attribute mark image；

The method also includes:

Obtain the sample to be learned；

According to the sample to be learned, the parameter of backpropagation training basic model；

If the parameter of the basic model restrains the basic model, restrained to obtain the machine by the basic model Learning model.

9. a kind of Operator action monitoring apparatus, which is characterized in that described device is deployed in electronic equipment, and described device includes:

Image collection module, for obtaining the image of operator, movement when described image is to operator work is carried out What shooting generated；

Key position detection module, for being based on key position detection model, when working operator described in described image The movement carries out key position detection, obtains the trace image corresponding to the movement；

Track identification module obtains track for the trace image input trajectory identification model to be carried out track anomalous identification Recognition result；

Alarm module generates if indicating that movement when operator work is unqualified for the track identification result Alarm information.

10. a kind of electronic equipment characterized by comprising

Processor；And

Memory is stored with computer-readable instruction on the memory, and the computer-readable instruction is held by the processor Such as Operator action measure of supervision described in any item of the claim 1 to 8 is realized when row.