CN118094253A

CN118094253A - Worker state estimation method and system based on foot and hand motion monitoring

Info

Publication number: CN118094253A
Application number: CN202410517138.2A
Authority: CN
Inventors: 沃天斌; 王旭; 洪磊; 严增尧; 薛泰进
Original assignee: Ningbo Bincube Technologies Co ltd
Current assignee: Ningbo Bincube Technologies Co ltd
Priority date: 2024-04-28
Filing date: 2024-04-28
Publication date: 2024-05-28

Abstract

The application belongs to the technical field of clothing production, and discloses a worker state estimation method and a worker state estimation system based on foot and hand motion monitoring, wherein the comprehensiveness of the normal working state and the data dimension of a production line worker can be effectively ensured by acquiring operation data corresponding to the foot and the hand of the production line worker; the corresponding standard part actions are determined by combining the two operation data, so that the difficulty of hand action recognition is greatly simplified, the algorithm complexity is greatly reduced, and the accuracy is improved; and secondly, the state result of the production line workers can be estimated by utilizing a pre-trained machine learning model according to all standard part actions and corresponding working time periods after sequencing treatment, and compared with manual calculation, the accuracy and reliability of the state result can be effectively ensured, so that effective data support is provided for subsequent production management and decision making.

Description

Worker state estimation method and system based on foot and hand motion monitoring

Technical Field

The application belongs to the technical field of clothing production, and particularly relates to a worker state estimation method and system based on foot and hand motion monitoring.

Background

The current clothing production line mainly comprises a production line worker operating equipment to complete the sewing of the cut pieces, and the production line worker usually performs the cooperation of hands and feet in the process of sewing clothing, wherein the hands are used for controlling the cut pieces of clothing, and the feet are used for controlling a sewing machine, so that if the actions of the hands and the feet can be captured simultaneously, the state of the worker can be estimated.

The traditional worker state detection means generally obtains the productivity state of workers through the means such as counting device is pressed to production line workers' staff or code scanning device is installed additional, and not only the problems of missing pressing, multiple pressing, missing scanning, multiple scanning or wrong scanning exist easily, but also the production efficiency of production line workers is seriously affected due to the fact that manual operation is added.

Disclosure of Invention

The application provides a worker state estimation method and a system based on foot and hand motion monitoring, which are used for solving the technical problems that the productivity state of a worker is obtained by means of a counting device or a code scanning device, etc. by the labor of the worker in a production line, not only the problems of missing pressing, multiple pressing, missing scanning, multiple scanning or wrong scanning, etc. are easy to exist, but also the production efficiency of the worker in the production line is easy to be seriously influenced by adding manual operation, etc., and the technical scheme is as follows:

in a first aspect, an embodiment of the present application provides a method for estimating a worker state based on foot and hand motion monitoring, including:

acquiring foot operation data acquired by a foot acquisition device in a preset period of time and hand operation data acquired by a hand acquisition device in the preset period of time;

dividing two groups of action data according to foot operation data and hand operation data, and inquiring standard actions corresponding to each group of action data based on a preset action library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period;

And sequencing all the standard actions corresponding to the action data according to a preset production line procedure, and inputting all the sequenced standard actions and working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result.

In an alternative of the first aspect, dividing the two sets of motion data according to the foot operation data and the hand operation data includes:

determining at least two peak intervals in foot operation data, and taking sub-part data corresponding to all the peak intervals and a first working period corresponding to the sub-part data in a preset period as first action data;

Determining a second working period in a preset period according to the first working period corresponding to all peak intervals, and taking the sub-part data corresponding to the second working period in the hand operation data and the second working period as second action data; the preset time period consists of a first working time period and a second working time period;

the first motion data and the second motion data are used as two groups of motion data.

In a further alternative of the first aspect, the dividing the two sets of motion data according to the foot operation data and the hand operation data further includes:

Removing sub-part data which is consistent with a preset peak value threshold value in the hand operation data, and inquiring initial standard actions corresponding to the processed hand operation data based on a preset action library;

screening the initial standard actions based on a preset key action library to obtain key standard actions, and taking sub-part data corresponding to the key standard actions in the hand operation data and a third working period corresponding to the sub-part data in a preset period as third action data;

Determining a fourth working period in a preset period according to the third working period, and taking the sub-part data corresponding to the fourth working period in the foot operation data and the fourth working period as fourth action data; wherein the preset period is composed of a third working period and a fourth working period;

The third motion data and the fourth motion data are taken as two groups of motion data.

In still another alternative of the first aspect, after sorting all the standard actions corresponding to the action data according to a preset production line procedure, and inputting all the standard actions after the sorting and the working time periods corresponding to each standard action into a preset machine learning model, obtaining a worker status result, the method further includes:

when the worker state results corresponding to the first action data and the second action data are detected to be lower than a preset result threshold value, corresponding worker state results are generated based on the third action data and the fourth action data, and the worker state results corresponding to the third action data and the fourth action data are taken as target results.

When the worker state results corresponding to the first action data and the second action data are detected to be greater than or equal to a preset result threshold, taking the worker state results corresponding to the first action data and the second action data as target results; or (b)

And when detecting that the worker state results corresponding to the first action data and the second action data and the worker state results corresponding to the third action data and the fourth action data are both greater than or equal to a preset result threshold, determining a target result according to the two worker state results.

Determining a corresponding calculation formula according to the received user input instruction, and processing a worker state result based on the calculation formula;

Transmitting the processed worker state result to a foot acquisition device so that the foot acquisition device displays the processed worker state result; or (b)

And sending the processed worker state result to a hand acquisition device so that the hand acquisition device displays the processed worker state result.

In a further alternative of the first aspect, before dividing the two sets of motion data according to the foot operation data and the hand operation data, the method further includes:

respectively carrying out filtering processing on foot operation data and hand operation data;

respectively translating the foot operation data after the filtering treatment and the hand operation data after the filtering treatment;

dividing two groups of action data according to the foot operation data and the hand operation data, wherein the two groups of action data comprise:

and dividing two groups of action data according to the foot operation data after the translation processing and the hand operation data after the translation processing.

In a second aspect, an embodiment of the present application provides a worker state estimation system based on foot and hand motion monitoring, including:

The data acquisition module is used for acquiring foot operation data acquired by the foot acquisition device in a preset time period and hand operation data acquired by the hand acquisition device in the preset time period;

the action determining module is used for dividing two groups of action data according to foot operation data and hand operation data, and inquiring standard actions corresponding to each group of action data based on a preset action library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period;

the result generation module is used for carrying out sequencing processing on the standard actions corresponding to all the action data according to a preset production line procedure, and inputting all the standard actions after the sequencing processing and the working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result.

In a third aspect, the embodiment of the application also provides a worker state estimation system based on the monitoring of the actions of the feet and the hands, which comprises a processor and a memory;

The processor is connected with the memory;

A memory for storing executable program code;

The processor executes a program corresponding to the executable program code by reading the executable program code stored in the memory, for implementing the method for estimating a state of a worker based on the monitoring of the motions of the foot and the hand provided in the first aspect of the embodiment or any implementation of the first aspect of the application.

In a fourth aspect, an embodiment of the present application provides a computer storage medium, where a computer program is stored, where the computer program includes program instructions, where the program instructions, when executed by a processor, implement the method for estimating a state of a worker based on monitoring of movements of the foot and the hand provided in the first aspect of the embodiment of the present application or any implementation manner of the first aspect of the present application.

In the embodiment of the application, when estimating the state of the production line worker, the foot operation data acquired by the foot acquisition device in a preset time period and the hand operation data acquired by the hand acquisition device in the preset time period can be acquired; dividing two groups of action data according to foot operation data and hand operation data, and inquiring standard actions corresponding to each group of action data based on a preset action library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period; and sequencing all the standard actions corresponding to the action data according to a preset production line procedure, and inputting all the sequenced standard actions and working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result. The foot and the hand of the line worker are obtained to correspond to the operation data when in work, so that the normal working state and the comprehensiveness of the data dimension of the line worker can be effectively ensured; the two operation data are combined to determine the corresponding standard part movements, so that the difficulty of hand movement identification is greatly simplified, the algorithm complexity is greatly reduced, the accuracy is improved, the problems of higher complexity, low movement detection accuracy and the like caused by overlarge movement sets can be effectively avoided compared with the independent analysis of the hand operation data, and the problems of poor suitability, low movement detection accuracy and the like caused by the change of the clothes size can be effectively avoided compared with the independent analysis of the foot operation data; and secondly, the state result of the production line workers can be estimated by utilizing a pre-trained machine learning model according to all standard part actions and corresponding working time periods, and compared with manual calculation, the accuracy and reliability of the state result can be effectively ensured, so that effective data support is provided for subsequent production management and decision making.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is an overall flowchart of a worker state estimation method based on foot and hand motion monitoring according to an embodiment of the present application;

fig. 2 is a schematic diagram of a system architecture of a worker state estimation method based on foot and hand motion monitoring according to an embodiment of the present application;

FIG. 3 is a flowchart illustrating another method for estimating a worker's state based on foot and hand motion monitoring according to an embodiment of the application;

FIG. 4 is a flowchart illustrating another method for estimating the state of a worker based on the monitoring of the actions of the foot and the hand according to an embodiment of the application;

fig. 5 is a schematic structural diagram of a worker state estimation system based on foot and hand motion monitoring according to an embodiment of the present application;

Fig. 6 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application.

In the following description, the terms "first," "second," and "first," are used for descriptive purposes only and are not to be construed as indicating or implying relative importance. The following description provides various embodiments of the application that may be substituted or combined between different embodiments, and thus the application is also to be considered as embracing all possible combinations of the same and/or different embodiments described. Thus, if one embodiment includes feature A, B, C and another embodiment includes feature B, D, then the application should also be seen as embracing one or more of all other possible combinations of one or more of A, B, C, D, although such an embodiment may not be explicitly recited in the following.

The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made in the function and arrangement of elements described without departing from the scope of the application. Various examples may omit, replace, or add various procedures or components as appropriate. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

Referring to fig. 1, fig. 1 is an overall flowchart of a worker state estimation method based on foot and hand motion monitoring according to an embodiment of the application.

As shown in fig. 1, the method for estimating the state of a worker based on the monitoring of the actions of the foot and the hand at least comprises the following steps:

step 102, acquiring foot operation data acquired by a foot acquisition device in a preset period of time and hand operation data acquired by a hand acquisition device in the preset period of time.

In the embodiment of the application, the method for estimating the state of the worker based on the monitoring of the actions of the foot and the hand can be applied to a server corresponding to the clothing production line, and the server can be respectively connected with the foot acquisition device and the hand acquisition device in a communication way so as to receive position data generated by different body parts of the workers in the clothing production line under the working state. The foot collecting device may be, but not limited to, a sensor device for collecting data generated by the foot of the line producing worker, and the foot collecting device may be a current sensor connected to a power supply of the sewing machine to use current change data generated by the sewing machine when the line producing worker works as position data generated by the foot of the line producing worker, because the line producing worker needs to control the foot to tread the pedal of the sewing machine when the line producing worker performs the sewing work. It is understood that the foot collecting device may also be, but not limited to, a pressure sensor, a rotation speed sensor, or a detecting device including any of at least one of the above mentioned sensor types, and the detecting device may include, but not limited to, a power source, a processor, a touch screen, and the like, in addition to any of the above mentioned at least one of the sensor types.

In this regard, taking the foot collecting device as the above-mentioned detecting device including the current sensor as an example, the collected foot operation data may be understood as a period of irregular current waveform data that varies with time, and since the current waveform data may be doped with noise signals or current data with different frequencies, in order to improve the subsequent data processing efficiency of the server, the detecting device including the current sensor may further process the current waveform data through a series of data processing methods, for example, but not limited to, any one of quantization processing, filtering processing, conversion processing and translation processing, and may store the processed current waveform data according to a specified format, so as to upload the processed current waveform data to the server when receiving a data acquisition request sent by the server.

The hand collecting device may be, but not limited to, a sensing device for collecting data generated by the hands of the line worker, and since the line worker needs to handle, cut, align, push, pull, put, transfer, etc. the hand collecting device may be a hand wearing device (at least one type of sensor such as an acceleration sensor, a gyroscope, or a geomagnetic sensor is disposed inside) worn on the hand of the line worker, for example, because the line worker also needs to handle, cut, align, push, pull, put, transfer, etc. the hand cutting piece by the hand when performing the sewing work, the hand collecting device may be, but not limited to, a specific wearing position and a combination manner, where the acceleration change data or the pose change data collected during the hand movement process are used as the position data generated by the hands of the line worker. It is understood that the hand capturing device may also be, but not limited to, a detection device including any one of a camera, an acceleration sensor, a gyroscope, a geomagnetic sensor, and a positioning sensor, where the detection device may include a power source, a processor, a touch screen, and other structures besides any one of the above mentioned types of sensors, and when the detection device includes a camera, the hand motion track video captured by the camera may be directly used as the location data.

Here, taking the hand acquisition device as the above-mentioned detection device including the acceleration sensor as an example, the acquired hand operation data may be understood as a period of irregular acceleration waveform data that varies with time, and since the acceleration waveform data may be doped with noise signals or acceleration data with different frequencies, in order to improve the subsequent data processing efficiency of the server, the detection device including the acceleration sensor may process the acceleration waveform data through a series of data processing manners after acquiring the acceleration waveform data, for example, but not limited to, any one of quantization processing, filtering processing, conversion processing and translation processing, and may store the processed acceleration waveform data according to a specified format, so as to upload the processed acceleration waveform data to the server when receiving a data acquisition request sent by the server.

Of course, since the line workers in the sewing workshop mainly include foot motions and hand motions in the working state, the above-mentioned foot collecting device and hand collecting device can be understood as two main position data collecting devices corresponding to the sewing workshop of the clothing production line, and when the line workers in other workshops of the clothing production line need to collect and process position data, the same number of collecting devices can collect and process position data of corresponding positions, or other numbers of collecting devices can collect and process position data of corresponding positions, which is not limited.

Referring to fig. 2, a schematic system architecture of a method for estimating a worker's state based on foot and hand motion monitoring according to an embodiment of the present application may be shown, and as shown in fig. 2, the system architecture may include a server, a current sensor in communication with the server, and a bracelet in communication with the server. The current sensor can be connected to a power supply of the sewing machine in the sewing workshop to acquire current change data generated by the sewing machine when a line production worker works according to a specified time period; the bracelet can also be sleeved on the wrist of the production line worker to acquire acceleration change data or pose change data corresponding to the finger movement process according to the same appointed time period; the server can estimate the state result of the line worker in the sewing workshop according to the current sensor and the position data uploaded by the bracelet, such as, but not limited to, the time required for the line worker to complete a garment by performing sewing operation, or state scoring for representing the time required for completing a garment.

It can be further understood that the server can effectively ensure the normal working state and the comprehensiveness of the data dimension of the line worker by acquiring the operation data corresponding to the feet and the hands of the line worker during working; the corresponding standard part actions are determined by combining the two operation data, so that the difficulty of hand action recognition is greatly simplified, the algorithm complexity is greatly reduced, and the accuracy is improved; and secondly, the state result of the production line workers can be estimated by utilizing a pre-trained machine learning model according to all standard part actions and corresponding working time periods, and compared with manual calculation, the accuracy and reliability of the state result can be effectively ensured, so that effective data support is provided for subsequent production management and decision making.

Specifically, when estimating the state of the line worker, the server may, but is not limited to, send the data acquisition request to the foot acquisition device and the hand acquisition device in the clothing production line when detecting that the line worker is at a station and performing clothing production, so that the foot acquisition device acquires foot operation data corresponding to the line worker in a preset period after receiving the data acquisition request, and may also cause the hand acquisition device to acquire hand operation data corresponding to the line worker in the same preset period after receiving the data acquisition request. The foot collecting device can be, but not limited to, a current sensor connected to a power supply of a sewing machine in a sewing workshop, so as to take current change data generated by a line production worker during working in a preset time period as foot operation data, and upload the foot operation data to a server in a communication mode such as Ethernet, bluetooth, wireless network or mobile network; the hand acquisition device can be, but is not limited to, a bracelet sleeved at the wrist of the line worker so as to change acceleration data or pose change data generated in the finger movement process of the line worker in a preset period.

It can be understood that the above-mentioned foot collecting device and hand collecting device can be provided with independent processors to process the collected position data, and can store the processed position data according to a specified format, so as to effectively improve the processing efficiency of the server.

As an option of the embodiment of the present application, before dividing the two sets of motion data according to the foot operation data and the hand operation data, the method further includes:

Specifically, after the server receives the foot operation data collected by the foot collecting device and the hand operation data collected by the hand collecting device, in order to improve accuracy of subsequent data processing, the server may, but is not limited to, perform quantization processing on the foot operation data and the hand operation data, and perform filtering processing on the foot operation data and the hand operation data after the quantization processing, so as to effectively filter noise data in the position data. The quantization processing and the filtering processing are understood as conventional technical means in the art, and will not be repeated.

Then, after obtaining the foot operation data and the hand operation data after the filtering processing, the server may perform conversion processing on the foot operation data and the hand operation data after the filtering processing, and perform translation processing on the foot operation data and the hand operation data after the conversion processing, so as to effectively ensure the subsequent data processing efficiency of the server. The conversion process and the translation process are understood as conventional technical means in the art, and will not be repeated.

It is further understood that the above-mentioned quantization processing, filtering processing, conversion processing, and translation processing may further be performed on the respective collected portion data by the foot collecting device and the hand collecting device, and after the foot collecting device obtains the foot operation data after the translation processing and the hand collecting device obtains the hand operation data after the translation processing, the foot collecting device stores the foot operation data in a specified format, and the hand collecting device stores the hand operation data in a specified format, so that when the server sends a data acquisition request, the foot operation data and the hand operation data in the specified format are uploaded to the server, respectively.

Step 104, dividing two groups of motion data according to the foot operation data and the hand operation data, and inquiring standard motions corresponding to each group of motion data based on a preset motion library.

Specifically, after obtaining the foot operation data and the hand operation data, the server may, but not limited to, extract, from the foot operation data, partial portion data including a standard motion performed by the foot of the line worker in the working state, and extract, from the hand operation data, partial portion data including a standard motion performed by the hand of the line worker in the working state, and the partial portion data may further include a working period corresponding to a preset period, for example, the server may extract, from the foot operation data, partial portion data including a standard motion of foot stepping by the line worker when stepping on the sewing machine, and extract, within the preset period, a working period corresponding to the standard motion of foot stepping, wherein the remaining portion data in the foot operation data may be understood as portion data corresponding to a case where the foot of the line worker does not perform the standard motion of foot stepping. Of course, the server may extract part of the position data corresponding to the standard actions of the line worker in handling the cut-parts, such as holding, cutting, aligning, pushing, pulling, releasing, and transferring, from the hand operation data, and extract the working time periods corresponding to the standard actions of the holding, cutting, aligning, pushing, pulling, releasing, and transferring, respectively, within the preset time periods, where the rest of the position data in the hand operation data may be understood as the position data corresponding to the hand of the line worker when no standard action is performed, and in the embodiment of the present application, the present application may not be limited to the above-mentioned standard actions of holding, cutting, aligning, pushing, pulling, releasing, and transferring.

It should be understood that the part data extracted from the above hand operation data and including the standard actions such as picking, cutting, aligning, pushing, pulling, placing and transmitting by the production line worker when the cut-parts are operated may be, but not limited to, the server may be used as a set of action data according to each type of standard action, or may be used as a set of action data according to all types of standard actions.

Further, after determining the motion data corresponding to the foot operation data and the motion data corresponding to the hand operation data, the server may query the standard motion corresponding to each group of motion data based on a preset motion library, where the preset motion library may include, but is not limited to, waveform data corresponding to standard motions such as foot stepping, and waveform data corresponding to standard motions such as holding, clipping, aligning, pushing, pulling, placing, and transmitting of the hand (not limited to all waveform data corresponding to the foot and the hand), so as to determine all standard motions included in the motion data corresponding to the foot operation data and all standard motions included in the motion data corresponding to the hand operation data through similarity comparison between different types of waveform data and the motion data.

Here, the preset action library may, but not limited to, be recorded with all standard actions corresponding to different standard procedures of the clothing production line manually, and standard waveform data corresponding to each standard action, where the standard waveform data may be obtained by experiment manually using a corresponding position acquisition device, and may further update standard actions and corresponding standard waveforms included in the preset action library according to a preset time interval.

And 106, sorting the standard actions corresponding to all the action data according to a preset production line procedure, and inputting all the standard actions after sorting and the working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result.

Specifically, after determining the standard actions corresponding to the foot operation data and the standard actions corresponding to the hand operation data, the server may, but is not limited to, perform the sorting process on the standard actions corresponding to all the action data according to the sorting manner of all the standard actions in the preset line production process corresponding to the sewing workshop, so that all the standard actions after the sorting process conform to the normal action flow and normalization of the line production worker in the line production process.

Further, after obtaining all the standard actions after the sorting process, the server may also, but is not limited to, simultaneously input all the standard actions and corresponding working periods into a preset machine learning model to predict a worker status result of the line worker through the preset machine learning model, for example, but not limited to, a time required for the line worker to complete a garment in a garment production line (or a status score for representing a time required for completing a garment). Here, the preset machine learning model may be, but is not limited to, a convolutional neural network structure known in the art, which is obtained by training all standard actions corresponding to each standard task, and manually noted working time corresponding to each standard action, and standard CT time (i.e., standard time required for completing a piece of clothing in a clothing production line).

It should be noted that, in addition to the time required for the line worker to complete a piece of clothing in the clothing production line, the worker status result in the embodiment of the present application may be, but not limited to, any one or more types of data including skill proficiency, efficiency fluctuation data, work fatigue, work pressure data, work concentration, skill adaptability, skill learning curve, innovation ability data, etc. of the line worker in the clothing production line, each type of data may be generated by combining all standard actions with a corresponding set machine learning model or a preset calculation formula, for example, when the worker status result is skill proficiency of the line worker in the clothing production line, the worker status result may be, but not limited to, inputting all standard actions and corresponding working time periods after the sorting treatment into a preset skill proficiency calculation formula, so that the calculation result may be used as skill proficiency of the line worker in the clothing production line, and is not limited thereto.

As still another alternative of the embodiment of the present application, after performing a sorting process on standard actions corresponding to all action data according to a preset production line procedure, and inputting all the standard actions after the sorting process and a working period corresponding to each standard action into a preset machine learning model, a worker status result is obtained, the method further includes:

Specifically, when it is detected that the server performs subsequent processing on the two sets of motion data corresponding to the first motion data and the second motion data, and the predicted worker state result is lower than a preset result threshold, it is indicated that the worker state result may deviate, and in order to ensure accuracy of the worker state result and overcome the adaptability defect of the foot motion data, the server may further perform subsequent processing on the two sets of motion data corresponding to the third motion data and the fourth motion data, so as to use the predicted worker state result as a final target result of the line worker.

It can be understood that when it is detected that the server performs the subsequent processing on the two sets of motion data corresponding to the first motion data and the second motion data, the predicted worker state result is greater than or equal to the preset result threshold, which indicates that the accuracy of the worker state result is higher, and then the worker state result can be directly used as the final target result of the line worker.

It will be further understood that, in order to further ensure accuracy and reliability of the final result, when it is detected that the predicted worker state result is obtained by performing the subsequent processing on the two sets of motion data corresponding to the first motion data and the second motion data and the predicted worker state result is greater than or equal to the preset result threshold by performing the subsequent processing on the two sets of motion data corresponding to the third motion data and the fourth motion data, the calculation result may be, but is not limited to, a manner of performing the mean value calculation on the two worker state results, or a manner of obtaining the final target result by using any one worker state result as the final target result by the line worker, and embodiments of the present application are not limited to the manner of obtaining the target result by any one of the above-mentioned worker state results.

Specifically, after the worker status result is obtained, in order to improve the interactive experience of the user, the server may determine a corresponding calculation formula according to the instruction input by the production line worker on the touch screen of the foot acquisition device or the touch screen of the hand acquisition device, for example, when the user input instruction is for calculating salary (may also be of a type such as a worker skill, a production line balance rate or a bottleneck process), the server may call the calculation formula for calculating salary, and the unknown quantity in the calculation formula is the determined worker status result.

Then, after a calculation formula corresponding to the user input instruction is called, the server can input the above mentioned worker state result into the calculation formula, and can feed back the calculation result to the foot acquisition device or the hand acquisition device, so that the production line worker can check the corresponding calculation result in time, and effectively adjust the current working state, thereby improving the productivity efficiency of the production line worker.

Referring to fig. 3, fig. 3 is an overall flowchart of still another method for estimating a worker's state based on monitoring of the actions of the foot and the hand according to an embodiment of the application. As shown in fig. 3, the method for estimating the state of a worker based on the monitoring of the actions of the foot and the hand at least comprises the following steps:

step 302, acquiring foot operation data acquired by a foot acquisition device in a preset period of time and hand operation data acquired by a hand acquisition device in the preset period of time.

Specifically, step 302 may refer to step 102 described above, and will not be described herein.

Step 304, determining at least two peak intervals in the foot operation data, and taking the sub-part data corresponding to all the peak intervals and the first working time period corresponding to the sub-part data in the preset time period as first action data.

Step 306, determining a second working period in a preset period according to the first working periods corresponding to all the peak intervals, and taking the sub-part data corresponding to the second working period and the second working period in the hand operation data as second action data.

When the operation data in the hand operation data are analyzed independently, the processing efficiency and the processing precision are not high due to the fact that the data collection is too large, and in the actual working state of the production line worker, the foot treading operation (namely, the sewing operation) and the foot non-treading operation (namely, the non-sewing operation) can be generally performed when the sewing machine treading operation is performed, when the production line worker does not perform the treading operation on the foot, the production line worker is indicated to be a hand operation cutting piece at the moment, and further, in order to improve the extraction accuracy and the extraction efficiency of the operation data, the foot operation data in the foot operation data and the hand operation data in the hand operation data can be respectively extracted according to whether the foot performs the treading operation or not.

Specifically, when the server divides two sets of motion data according to the foot operation data and the hand operation data, a peak section corresponding to at least two peaks may be determined in the foot operation data, and since the corresponding position data of the foot may change upward (i.e. form a peak) when the foot performs the stepping motion, and the corresponding position data may change downward (consistent with the data corresponding to the standby state of the sewing machine) when the foot does not perform the stepping motion, in other words, each peak section may correspond to one foot stepping motion of the line worker, and may use sub-position data corresponding to all the peak sections in the foot operation data, and a first working period corresponding to the sub-position data in a preset period as the first motion data.

Then, according to the above-mentioned description, when the worker performs no stepping action on the foot, it indicates that the worker performs a hand operation with a high probability at the moment, and the server may further use all remaining time periods except for the first time period in the preset time period as a second time period, that is, all corresponding time periods when the worker performs the hand operation on the hand operation, according to the first time period corresponding to all peak time periods in the preset time period, and may select all sub-portion data corresponding to the second time period from the hand operation data, and use the second time period as the second action data.

Step 308, taking the first action data and the second action data as two groups of action data, and inquiring the standard action corresponding to each group of action data based on a preset action library.

Specifically, step 308 can refer to step 104 described above, and will not be described herein.

Step 310, sorting all standard actions corresponding to the action data according to a preset production line procedure, and inputting all standard actions after sorting and working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result.

Specifically, step 310 may refer to step 106 described above, and will not be described herein.

Referring to fig. 4, fig. 4 is an overall flowchart of still another method for estimating a worker's state based on the monitoring of the actions of the foot and the hand according to an embodiment of the application. As shown in fig. 4, the method for estimating the state of a worker based on the monitoring of the actions of the foot and the hand at least comprises the following steps:

Step 402, acquiring foot operation data acquired by the foot acquisition device in a preset period of time, and hand operation data acquired by the hand acquisition device in the preset period of time.

Specifically, step 402 may refer to step 102 described above, and will not be described herein.

And step 404, removing sub-part data which is consistent with a preset peak value threshold value in the hand operation data, and inquiring initial standard actions corresponding to the processed hand operation data based on a preset action library.

Step 406, screening the initial standard actions based on a preset key action library to obtain key standard actions, and taking the sub-part data corresponding to the key standard actions in the hand operation data and a third working period corresponding to the sub-part data in a preset period as third action data.

Step 408, determining a fourth working period in a preset period according to the third working period, and taking the sub-part data corresponding to the fourth working period and the fourth working period in the foot operation data as fourth action data.

Because the change of the peak interval length is also possible due to the change of the clothes size when the motion data in the foot operation data are analyzed independently, the suitability and the accuracy of the foot motion data cannot be effectively ensured. In order to improve the adaptability and accuracy of the foot motion data, corresponding hand motion data can be determined according to the hand key motion identified in the hand operation data, and the foot motion data can be determined in the foot operation data according to the working time period corresponding to the hand key motion.

Specifically, when the server divides two groups of motion data according to the foot operation data and the hand operation data, partial sub-position data consistent with a preset peak threshold value can be determined in the hand operation data, the partial sub-position data can be understood as data corresponding to a line worker when the hand operation cut-parts are not executed, and the partial sub-position data can be removed from the hand operation data so as to keep the data corresponding to the rest line workers when the hand operation cut-parts are executed.

Then, the server can also combine a preset action library to perform matching processing on the data corresponding to the production line worker when executing the hand operation cut-parts so as to identify all hand standard actions contained in the data, and combine a preset key action library to perform screening processing on all the hand standard actions so as to obtain all the key standard actions corresponding to the hand operation cut-parts when executing the hand operation cut-parts. Here, the standard hand motion may be, but not limited to, a standard motion such as twisting or swinging, which is used as a non-critical standard motion of the hand, and the above-mentioned preset critical motion library may be updated periodically according to the manual requirement, so as to further ensure the accuracy of extracting the motion data.

Then, after determining the hand key actions, the server may, but not limited to, obtain standard action waveform data corresponding to all hand key actions by combining with a preset action library, so as to determine sub-part data corresponding to the standard action waveform data in the hand operation data, and may use a third working period corresponding to the sub-part data and the word part data in a preset period as third action data.

Then, according to the above-mentioned description, when the worker in the production line does not perform the stepping action on the foot, it is indicated that the worker in the production line is the hand operation cut-parts with a high probability at the moment, and the server may further use all the remaining time periods excluding the third working time period in the preset time period as the fourth working time period according to the third time period, that is, all the working time periods corresponding to the foot stepping action piece, and may screen out all the sub-part data corresponding to the fourth working time period from the foot operation data, and combine with the fourth working time period as the fourth action data.

Step 410, using the third motion data and the fourth motion data as two sets of motion data, and querying a standard motion corresponding to each set of motion data based on a preset motion library.

Specifically, step 410 may refer to step 104 described above, and will not be described herein.

Step 412, sorting all the standard actions corresponding to the action data according to the preset production line procedure, and inputting all the standard actions after sorting and the working time period corresponding to each standard action into a preset machine learning model to obtain a worker state result.

Specifically, step 412 may refer to step 106 described above, and will not be described herein.

Referring to fig. 5, fig. 5 is a schematic structural diagram of a worker state estimation system based on foot and hand motion monitoring according to an embodiment of the application.

As shown in fig. 5, the worker state estimation system based on foot and hand motion monitoring may at least include a data acquisition module 501, a motion determination module 502, and a result generation module 503, wherein:

the data acquisition module 501 is configured to acquire foot operation data acquired by the foot acquisition device in a preset period, and hand operation data acquired by the hand acquisition device in the preset period;

The action determining module 502 is configured to divide two sets of action data according to foot operation data and hand operation data, and query standard actions corresponding to each set of action data based on a preset action library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period;

The result generating module 503 is configured to perform sorting processing on standard actions corresponding to all action data according to a preset production line procedure, and input all the standard actions after the sorting processing and a working period corresponding to each standard action into a preset machine learning model to obtain a worker status result.

In some possible embodiments, dividing the two sets of motion data according to the foot operation data and the hand operation data includes:

In some possible embodiments, the dividing the two sets of motion data according to the foot operation data and the hand operation data further includes:

In some possible embodiments, after sorting the standard actions corresponding to all the action data according to a preset production line procedure, inputting all the standard actions after the sorting and the working time periods corresponding to each standard action into a preset machine learning model to obtain a worker status result, the method further includes:

When the worker state results corresponding to the first action data and the second action data are detected to be lower than a preset result threshold value, corresponding worker state results are generated based on the third action data and the fourth action data, and the worker state results corresponding to the third action data and the fourth action data are taken as target results. In some possible embodiments, after sorting the standard actions corresponding to all the action data according to a preset production line procedure, inputting all the standard actions after the sorting and the working time periods corresponding to each standard action into a preset machine learning model to obtain a worker status result, the method further includes:

In some possible embodiments, before dividing the two sets of motion data according to the foot operation data and the hand operation data, the method further includes:

It will be clear to those skilled in the art that the technical solutions of the embodiments of the present application may be implemented by means of software and/or hardware. "unit" and "module" in this specification refer to software and/or hardware capable of performing a particular function, either alone or in combination with other components, such as Field-Programmable gate arrays (Field-Programmable GATE ARRAY, FPGA), integrated circuits (INTEGRATED CIRCUIT, ICs), and the like.

Referring to fig. 6, fig. 6 shows a schematic structural diagram of a server according to an embodiment of the present application.

As shown in fig. 6, the server 600 may include at least one processor 601, at least one network interface 604, a user interface 603, a memory 605, and at least one communication bus 602.

Wherein the communication bus 602 may be used to enable connectivity communication for the various components described above.

The user interface 603 may include keys, and the optional user interface may also include a standard wired interface, a wireless interface, among others.

The network interface 604 may include, but is not limited to, a bluetooth module, an NFC module, a Wi-Fi module, etc.

Wherein the processor 601 may include one or more processing cores. The processor 601 performs various functions of the routing server 600 and processes data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 605 and invoking data stored in the memory 605 using various interfaces and lines connecting the various components within the server 600. Alternatively, the processor 601 may be implemented in at least one hardware form of DSP, FPGA, PLA. The processor 601 may integrate one or a combination of several of a CPU, GPU, modem, and the like. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for rendering and drawing the content required to be displayed by the display screen; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 601 and may be implemented by a single chip.

The memory 605 may include RAM or ROM. Optionally, the memory 605 includes a non-transitory computer readable medium. Memory 605 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 605 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, etc.; the storage data area may store data or the like referred to in the above respective method embodiments. The memory 605 may also optionally be at least one storage device located remotely from the processor 601. As shown in fig. 6, an operating system, a network communication module, a user interface module, and a worker state estimation application based on foot and hand motion monitoring may be included in a memory 605, which is one type of computer storage medium.

In particular, the processor 601 may be configured to invoke a worker state estimation application stored in the memory 605 based on foot and hand motion monitoring, and to specifically perform the following operations:

The present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the above method. The computer-readable storage medium may include, among other things, any type of disk including floppy disks, optical disks, DVDs, CD-ROMs, micro-drives, and magneto-optical disks, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of media or device suitable for storing instructions and/or data.

It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present application is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present application. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present application.

Claims

1. The method for estimating the state of the worker based on the monitoring of the actions of the foot and the hand is characterized by comprising the following steps:

Acquiring foot operation data acquired by a foot acquisition device within a preset period of time and hand operation data acquired by a hand acquisition device within the preset period of time;

Dividing two groups of motion data according to the foot operation data and the hand operation data, and inquiring standard motions corresponding to each group of motion data based on a preset motion library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period;

2. The method of claim 1, wherein the dividing the two sets of motion data from the foot operation data and the hand operation data comprises:

determining at least two peak intervals in the foot operation data, and taking all sub-part data corresponding to the peak intervals and a first working period corresponding to the sub-part data in the preset period as first action data;

Determining a second working period in the preset period according to the first working period corresponding to all the peak intervals, and taking the sub-part data corresponding to the second working period in the hand operation data and the second working period as second action data; wherein the preset period of time is comprised of the first working period of time and the second working period of time;

And taking the first action data and the second action data as two groups of action data.

3. The method of claim 2, wherein the dividing the two sets of motion data from the foot operation data and the hand operation data further comprises:

screening the initial standard actions based on a preset key action library to obtain key standard actions, and taking sub-part data corresponding to the key standard actions in the hand operation data and a third working period corresponding to the sub-part data in the preset period as third action data;

Determining a fourth working period in the preset period according to the third working period, and taking the sub-part data corresponding to the fourth working period in the foot operation data and the fourth working period as fourth action data; wherein the preset period of time is comprised of the third operating period of time and the fourth operating period of time;

and taking the third action data and the fourth action data as two groups of action data.

4. The method according to claim 3, wherein after the sorting process is performed on the standard actions corresponding to all the action data according to the preset production line procedure, and all the standard actions after the sorting process and the working time periods corresponding to each standard action are input into a preset machine learning model, the method further comprises:

When detecting that the worker state results corresponding to the first action data and the second action data are lower than a preset result threshold, generating corresponding worker state results based on the third action data and the fourth action data, and taking the worker state results corresponding to the third action data and the fourth action data as target results.

5. The method according to claim 4, wherein after the sorting process is performed on the standard actions corresponding to all the action data according to the preset production line procedure, and all the standard actions after the sorting process and the working time periods corresponding to each standard action are input into a preset machine learning model, the method further comprises:

When the worker state result corresponding to the first action data and the second action data is detected to be greater than or equal to the preset result threshold, taking the worker state result corresponding to the first action data and the second action data as a target result; or (b)

And when detecting that the worker state results corresponding to the first action data and the second action data and the worker state results corresponding to the third action data and the fourth action data are both greater than or equal to the preset result threshold, determining a target result according to the two worker state results.

6. The method according to claim 1, wherein after the sorting process is performed on the standard actions corresponding to all the action data according to the preset production line procedure, and all the standard actions after the sorting process and the working time periods corresponding to each standard action are input into a preset machine learning model, the method further comprises:

Determining a corresponding calculation formula according to the received user input instruction, and processing the worker state result based on the calculation formula;

Transmitting the processed worker state result to the foot collecting device so that the processed worker state result is displayed by the foot collecting device; or (b)

And sending the processed worker state result to the hand acquisition device so that the processed worker state result is displayed by the hand acquisition device.

7. The method of claim 1, further comprising, prior to said dividing the two sets of motion data from the foot operation data and the hand operation data:

respectively carrying out filtering processing on the foot operation data and the hand operation data;

respectively translating the foot operation data subjected to the filtering processing and the hand operation data subjected to the filtering processing;

the dividing the two sets of motion data according to the foot operation data and the hand operation data includes:

8. A worker state estimation system based on foot and hand motion monitoring, comprising:

The action determining module is used for dividing two groups of action data according to the foot operation data and the hand operation data, and inquiring standard actions corresponding to each group of action data based on a preset action library; wherein, each group of action data comprises sub-part data corresponding to at least one standard action and a working period;

The result generation module is used for carrying out sorting processing on the standard actions corresponding to all the action data according to a preset production line procedure, and inputting all the standard actions after the sorting processing and the working time periods corresponding to each standard action into a preset machine learning model to obtain a worker state result.

9. The worker state estimation system based on the foot and hand motion monitoring is characterized by comprising a processor and a memory;

The processor is connected with the memory;

the memory is used for storing executable program codes;

the processor runs a program corresponding to executable program code stored in the memory by reading the executable program code for performing the steps of the method according to any of claims 1-7.

10. A computer readable storage medium having stored thereon a computer program, characterized in that the computer readable storage medium has stored therein instructions which, when run on a computer or a processor, cause the computer or the processor to perform the steps of the method according to any of claims 1-7.