AU2024423859A1 - Working face human-machine collaborative operation safety monitoring and early warning system and method - Google Patents

Abstract

The present invention belongs to the field of multi-equipment safety management and control for personnel and equipment clusters at an underground coal mine driving working face, and solves the problem of safety hazards caused by human-machine collaborative operations as the mechanization level of underground equipment increases. Provided are a system and method for safety monitoring and early warning of human-machine collaborative operation at a working face. The system comprises a safety management and control system for mobile equipment at a working face, which is used for performing in real time position, orientation and environmental sensing of individual equipment and behavior analysis of drivers; working-face collaborative intelligent wearable devices, which are used for environmental sensing of operating personnel and individual equipment and acquiring in real time the motion postures of the operating personnel; and a working face output control subsystem, which is used for identification and management of operating personnel entry and exit, position monitoring of personnel and individual equipment, operating personnel behavior identification and monitoring and working environment monitoring, and generating corresponding alarm instruction information. The present invention can realize human-equipment and equipment-equipment safety management and control during multi-equipment collaborative operation at a driving working face.

Description

WORKINGFACE WORKING FACEHUMAN-MACHINE HUMAN-MACHINE COLLABORATIVE COLLABORATIVE OPERATION OPERATION SAFETY SAFETY MONITORING ANDEARLY MONITORING AND EARLYWARNING WARNING SYSTEM SYSTEM AND AND METHOD METHOD TECHNICALFIELD TECHNICAL FIELD

[0001] Thepresent

[0001] The presentdisclosure disclosurebelongs belongsto tothethefield fieldofofmulti-equipment multi-equipment safety safety management management and and control for personnel control for andequipment personnel and equipment clusters clusters in in coal coal mine mine underground underground heading heading faces, faces, and inand in

particular, totoa aworking particular, working face face human-machine collaborativeoperation human-machine collaborative operationsafety safetymonitoring monitoringandand early early

warningsystem warning systemand andmethod. method.

BACKGROUND BACKGROUND

[0002] Withthe

[0002] With thecontinuous continuousimprovement improvement in the in the mechanization mechanization levellevel of coal of coal mining mining equipment, equipment,

the number the numberofofequipment equipment operating operating in ainheading a heading face face has been has been increasing. increasing. DuringDuring production production

operations, the operations, the equipment frequentlyshuttles equipment frequently shuttlesand andalternates alternatesinin cycles, cycles, posing posingsignificant significant safety safety hazards to personnel at the heading face. In particular, during production, high dust concentration, hazards to personnel at the heading face. In particular, during production, high dust concentration,

poor visibility, poor visibility, and noisy environments and noisy environments result result in in large large visual visual blind blind spots spots during during equipment equipment

movement,which movement, which makes makes it highly it highly likely likely for crushing for crushing or collision or collision accidents accidents to occur to occur between between

equipment andpersonnel. equipment and personnel. Furthermore, Furthermore, due due to the to the large large number number of workers of workers and and the theself- poor poor self- discipline ofsome discipline of some individuals, individuals, safety safety accidents accidents often often occur occur as as a of a result result of inadequate inadequate supervision. supervision.

[0003] At present,

[0003] At present, personnel safety protection personnel safety protection in inheading heading faces faces mainly mainly relies relieson on single singleequipment equipment

unit protection, with relatively limited technical means. Typically, only one type of technology is unit protection, with relatively limited technical means. Typically, only one type of technology is

adopted, such adopted, suchasasultrasonic ultrasonicradar, radar,infrared infraredpyroelectric pyroelectricsensors, sensors,infrared infraredranging ranging sensors, sensors, or or millimeter-wave radar. millimeter-wave radar. The The safety safety protection protection effect effect is far is farideal, from fromand ideal, and collaborative collaborative operations operations

betweenequipment between equipment cannot cannot be be achieved. achieved.

[0004] For the

[0004] For the collaborative collaborative operations operations between betweenpersonnel personnelandand equipment equipment in heading in heading faces, faces, it is it is

first first necessary toknow necessary to knowthethe relative relative positional positional relationships relationships therebetween. therebetween. Inyears, In recent recentwith years, the with the

rise of rise of UWB technology, UWB technology, some some minesmines have installed have installed underground underground precise precise positioning positioning systems systems based on based on UWB. UWB. However, However, the the systems systems cannot cannot achieve achieve two-dimensional two-dimensional precise precise positioning positioning

undergroundand underground andareareunable unable toto calculatethe calculate therelative relative positions positions between betweenpersonnel personneland andequipment equipment or betweenequipment. or between equipment. Therefore, Therefore, suchsuch systems systems cannotcannot meet meet the the technical technical requirements requirements for for collaborative operations collaborative operations between betweenpersonnel personnel andand equipment equipment in underground in underground workingworking faces. Infaces. In addition, underground addition, precise positioning underground precise positioning systems systemsare are generally generally applied applied only only in in formed roadways. formed roadways.

As aa heading As headingface facecontinuously continuouslyadvances, advances,ititbecomes becomes difficultto difficult to deploy deploypositioning positioningbase basestations stations timely, resulting in failure to update positioning data of the heading face in real time, and even timely, resulting in failure to update positioning data of the heading face in real time, and even leading tolarge leading to largedeviations deviationsin in positioning positioning data. data. Consequently, Consequently, it is impossible it is impossible to the to obtain obtain the accurate accurate and relativepositions and relative positionsof of personnel personnel and and equipment, equipment, which which makes makes it to it difficult difficult to ensure ensure the the safety of safety of personnel and personnel andequipment equipmentduring during operations operations atatthe theworking working face. face.

[0005] The present

[0005] The present disclosure disclosure establishes establishes aasafety safetyearly-warning early-warningsystem system and and corresponding control corresponding control

logic logic for for equipment-to-equipment andequipment-to-personnel equipment-to-equipment and equipment-to-personnel in heading in heading faces, faces, aiming aiming to enable to enable

the equipment the equipmentto toactively actively avoid avoid hazards hazards when when danger danger occurs, occurs, thereby thereby reducing reducing injuries injuries to to personnel caused personnel causedbybythetheequipment. equipment. Meanwhile, Meanwhile, based based on theon the relative relative positional positional relationships relationships

betweenpersonnel between personnelandand equipment equipment as well as well as between as between different different equipment, equipment, safe collaborative safe collaborative

operations betweenequipment operations between equipmentandand between between personnel personnel and and equipment equipment is achieved, is achieved, so asso toasensure to ensure orderly orderly operation operation at at the theworking working face face while while guaranteeing guaranteeing personnel safety, and personnel safety, and ultimately ultimately improve improve

undergroundwork underground work efficiency. efficiency.

SUMMARY SUMMARY

[0006] The

[0006] The present present disclosure disclosure is intended is intended to solve to solve at least at least one one of theof the technical technical problems problems in the prior in the prior

art, art, and and provides provides a a working facehuman-machine working face human-machine collaborative collaborative operation operation safety safety monitoring monitoring and and

early early warning systemand warning system andmethod. method.

[0007] Thepresent

[0007] The present disclosure disclosure is is implemented implemented bybythe thefollowing followingtechnical technicalsolution: solution: aa working workingface face human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring andand early early warning warning system, system, comprising: comprising:

aa working working face face mobile mobileequipment equipmentsafety safetymanagement management and and control control system, system, working working face face

collaborative collaborative intelligent intelligentwearable wearable devices, devices, an an information sharing platform, information sharing platform, and anda aworking working face face

output control subsystem; output control subsystem;

the working the workingface facemobile mobile equipment equipment safety safety management management and system and control controlissystem is configured configured to to performa areal-time perform real-timeperception perception of of a position a position and and orientation orientation of each of each single single equipment equipment unit, unit, perception of perception of an an operational operational status status of of each single equipment each single unit, perception equipment unit, perceptionofof an anenvironment environment within aa preset within preset range rangesurrounding surrounding each each single single equipment equipment unit, unit, and behavior and behavior acquisition acquisition and and analysis foraadriver analysis for driverofofeach each single single equipment equipment unit; unit;

each workingface each working facecollaborative collaborative intelligentwearable intelligent wearable device device comprises comprises a portable a portable positioning positioning

terminal and terminal and aa motion motionmonitoring monitoring device device that that areareattached attached to to a a worker; worker; thethe portable portable positioning positioning

terminal is configured to perceive relative positions and relative orientations between the worker terminal is configured to perceive relative positions and relative orientations between the worker

and each single and each single equipment unit and equipment unit and aa position position of of aaworking working face, face,and and the themotion motion monitoring monitoring device device

is is configured configured toto acquire acquire motion motion postures postures of theof the worker worker in real in real time; time; the information the sharing platform information sharing platform is is configured to collect configured to collect information information from each single from each single equipment equipment unit and unit eachworking and each working face face collaborative collaborative intelligentwearable intelligent wearable device device and and achieve achieve information information sharing; and sharing; and the working the faceoutput working face outputcontrol controlsubsystem subsystemis isconfigured configured to to receiveshared receive shared information information from from the the information sharingplatform, information sharing platform,perform perform entry entry and and exit exit identification identification management management of workers, of workers, position recognition position recognition and andmonitoring monitoringof of thethe workers workers and and eacheach single single equipment equipment unit, behavior unit, behavior recognition, monitoring recognition, andprediction monitoring and prediction of of the the workers workersand andworking working environment environment monitoring, monitoring, and and generate correspondingalarm generate corresponding alarminstruction instructioninformation, information,wherein whereinthe theworking workingenvironments environments include include gas gas concentrations concentrations of of gas, gas, carbon carbon monoxide, andoxygen, monoxide, and oxygen,roof roofdisplacement, displacement,wind wind speed speed andand wind wind pressure. pressure.

[0008] Preferably, the

[0008] Preferably, the working workingface facemobile mobile equipment equipment safety safety management management and control and control system system

comprises comprises aawireless wirelesspositioning positioningterminal, terminal,aa camera cameradevice, device,a adriving drivingbehavior behavior analysis analysis device, device,

and a multi-source sensor that are provided on each single equipment unit; the wireless positioning and a multi-source sensor that are provided on each single equipment unit; the wireless positioning

terminal is terminal is configured to perform configured to performreal-time real-timeperception perceptionofofa aposition positionand andanan orientationofofeach orientation each single single equipment unit; the equipment unit; the camera deviceisis configured camera device configuredto to capture capture environmental environmentalvideo videoimages images in in

real time real time within within aapreset presetrange rangesurrounding surrounding each each single singleequipment equipment unit unit and and perform environmental perform environmental

perception; the perception; the driving driving behavior behavioranalysis analysis device deviceisis configured configuredtotocapture capturedriver driverbehavior behaviorvideo video images in real images in real time time and andanalyze analyzedriver driverbehaviors; behaviors;andand thethe multi-source multi-source sensor sensor is configured is configured to to

acquire an operational status of each single equipment unit in real time. acquire an operational status of each single equipment unit in real time.

[0009] Preferably, the

[0009] Preferably, the working workingface faceoutput outputcontrol controlsubsystem subsystem comprises comprises a wireless a wireless positioning positioning

base station, base station, aawireless wirelessnetwork network transmission device, an transmission device, an equipment controldevice, equipment control device,aa voice voicealarm alarm device, anda acentralized device, and centralized control control platform; platform;

the wireless the wireless positioning positioning base basestation station isis configured configuredtotocooperate cooperate with with thethe wireless wireless positioning positioning

terminal of terminal of each each single single equipment equipmentunit unitand andthetheportable portablepositioning positioningterminal terminal of of each each worker worker to to perform perception of the positions and orientations of each single equipment unit and each worker; perform perception of the positions and orientations of each single equipment unit and each worker;

the wireless the wireless network networktransmission transmissiondevice device is is configured configured to enable to enable communication communication between between the the single single equipment unitsfor equipment units forcollaborative collaborativeoperations, operations, the the wireless wireless positioning positioningterminals, terminals, and andthe the workingface working facecollaborative collaborativeintelligent intelligent wearable devicesand wearable devices andthe theinformation informationsharing sharing platform platform in in all working faces; all working faces;

the equipment control device is configured to determine a hazardous state of each single equipment the equipment control device is configured to determine a hazardous state of each single equipment

unit and unit eachworker and each workerby by analyzing analyzing and and processing processing data,data, perform perform entry entry andidentification and exit exit identification management management of of thethe workers, workers, position position recognition recognition and and monitoring monitoring of each of each worker worker and and each each single single

4 equipment unit,behavior equipment unit, behaviorrecognition, recognition,monitoring, monitoring, andand prediction prediction of the of the workers workers and working and working

environment monitoring, environment monitoring, output output corresponding corresponding control control signals signals based based on control on control strategies strategies to to control each control device to each device to perform performcorresponding correspondingrisk riskavoidance avoidance actions,and actions, andtransmit transmitthethegenerated generated alarm instruction information to the centralized control platform; alarm instruction information to the centralized control platform;

the voice alarm device is configured to, upon receiving the alarm instruction information, perform the voice alarm device is configured to, upon receiving the alarm instruction information, perform

alarms ofofcorresponding alarms corresponding categories categories based based on different on different categories categories of the of theinstruction alarm alarm instruction information; and information; and

the centralized the centralized control control platform platformis isconfigured configured to, to, based based onshared on the the shared information information of the of the information sharing platform, information sharing platform,perform performentry entryand andexit exitidentification identification management management of the of the workers, workers,

position recognition position recognition and andmonitoring monitoringof ofeach each worker worker and and each each single single equipment equipment unit, behavior unit, behavior

recognition, monitoring, recognition, monitoring, and prediction of and prediction of the the workers, workers, and and working environmentmonitoring, working environment monitoring, and and

generate correspondingalarm generate corresponding alarminstruction instructioninformation. information.

[0010] Preferably,

[0010] Preferably, modes modes for the for the worker worker entry entry into into and andfrom exit exitthefrom the working working face face include include a normal a normal

operation mode,a ashift operation mode, shift handover handovermode, mode, a linkage a linkage control control mode, mode, and and a temporary a temporary authorization authorization

mode. mode.

[0011] Preferably, the

[0011] Preferably, the normal normal operation operation mode is as mode is as follows: follows:

A current-shift A current-shift manager managerinputs inputsidentities identities ofofcurrent-shift current-shift workers workersinto intoa acontrol controlsystem systemof of thethe

workingface working faceoutput outputcontrol control subsystem subsystemand andsets setsaamaximum maximum allowable allowable number number of current-shift of the the current-shift workersat workers at the the working faceas working face as N; Na ;

an entry and an entry exit management and exit device management device at at thetheworking working face face mobile mobile equipment equipment safety safety management management

and control and control system systemperforms performsidentity identityrecognition recognitionfor foraacurrent current nath nath worker, worker,and andrecords recordsananentry entry time TTinand time andananexit exittime timeT Tof of the out the worker worker intointo andand from from the the working working face;face;

in in aa case case where where nna N, ≤ Nthe a, the current current worker worker is permitted is permitted to enter to enter thethe working working face; face; and and in aincase a case wherenn>a >N,Nthe where a, the current current worker worker is is prohibited prohibited from from entering entering thethe working working face, face, and and overstaffing overstaffing

alarm instruction information is generated, where n is an integer greater than 0; and a alarm instruction information is generated, where n is an integer greater than 0; and

in a case in a casewhere wherean an entry entry duration duration Tw of Tthe w of the current current worker worker at at theface the working working facethan is greater is greater Ty, than Ty, timeout instruction information for the worker is generated, where T is a preset working duration, y timeout instruction information for the worker is generated, where Ty is a preset working duration,

and Tw == Tout- and Tw Tout- TT.in.

[0012] Preferably, the

[0012] Preferably, the handover modeisisas handover mode as follows: follows:

Thecurrent-shift The current-shift manager managerandand a next-shift a next-shift manager manager perform perform handover handover confirmation confirmation via the via the workingface working facecollaborative collaborativeintelligent intelligentwearable wearable devices devices and and the the working working face output face output controlcontrol

subsystem; and subsystem; and

the next-shift manager inputs identities of next-shift workers into the control system of the working the next-shift manager inputs identities of next-shift workers into the control system of the working

face face output output control control subsystem andidentifies subsystem and identifies the the workers at the workers at the working face, and working face, and in in aa case case where where

the current the current worker does not worker does not belong belongtoto the the next-shift next-shift workers, workers, identity identity mismatch alarminstruction mismatch alarm instruction information information is is generated. generated.

[0013] Preferably,

[0013] Preferably, thethe linkage linkage control control mode mode is as follows: is as follows:

Whenthe When thecamera camera device device of of thethe working working faceface mobile mobile equipment equipment safetysafety management management and and control control system detects that system detects that aa person person not not belonging to the belonging to the current current working face intends working face intends to to intrude intrude without without

authorization, authorization,climb climb over over or or bypass the entry bypass the entry and and exit exit management management device, device, continuous continuous

unauthorizedentry unauthorized entry alarm alarminstruction instruction information information is is generated; generated;

whenaaworker when workerpasses passesthrough throughthe theentry entryand andexit exit management management device, device, identitycross-verification identity cross-verificationisis performedbased performed basedononthe theworking working face face collaborative collaborative intelligentwearable intelligent wearabledevice deviceofofthe theworker workerandand visual recognition visual recognition information fromthe information from theworking workingface facemobile mobile equipment equipment safety safety management management and and control system,and control system, andin ina case a case where where the identity the identity cross-verification cross-verification fails,fails, alarmalarm instruction instruction

information indicating worker information indicating workeridentity identity mismatch mismatch isisgenerated; generated;

when a manager intends to access the system, identity cross-verification is performed based on the when a manager intends to access the system, identity cross-verification is performed based on the

workingface working facecollaborative collaborativeintelligent intelligent wearable wearabledevice device of of thethe manager manager and visual and visual recognition recognition

information fromthe information from the working workingface facemobile mobileequipment equipment safety safety management management and control and control system, system, and and

in a case in a case where wherethetheidentity identitycross-verification cross-verificationfails, fails, alarm alarminstruction instructioninformation informationindicating indicating manageridentity manager identity mismatch mismatchisisgenerated; generated;

whena aworker when workerfrom from anyany working working faceface passes passes through through the entry the entry and and exit exit management management device, device, all all equipment atthe equipment at the working workingface faceisis powered poweredoff; off;

whena aworker when workerfrom from anyany working working face face enters enters the the working working face,face, the working the working face output face output control control

subsystem performsworking subsystem performs working environment environment monitoring; monitoring; and and

whenenvironmental when environmental parameters parameters fail fail to meet to meet safety safety requirements requirements for thefor the worker worker to to perform perform production operations, production operations, the the entry entry and and exit exitmanagement deviceopens management device opensananemergency emergency escape escape passage. passage.

[0014] Preferably, the

[0014] Preferably, the temporary authorization mode temporary authorization modeisis as as follows: follows:

whena atemporary when temporary worker worker notnot belonging belonging to the to the current current working working face face needsneeds to enter to enter the working the working

face, face, the themanager manager performs temporaryauthorization performs temporary authorizationvia viathe the working faceoutput working face output control control subsystem subsystem

and defines aa temporary and defines entry duration temporary entry duration as as t; tn ;

in in a a case whereananentry case where entry duration t' of𝑡 ′theofcurrent duration the current temporary temporary worker atworker at theface the working working face is greater is greater 𝑛

than t, than tn, timeout timeout alarm alarminstruction instruction information informationfor forthe thetemporary temporary worker worker is generated, is generated, where where t' 𝑡𝑛′ represents the entry duration of the current temporary worker at the working face; and represents the entry duration of the current temporary worker at the working face; and

a total a total number oftemporary number of temporary workers workers and and current current workers workers entering entering the working the working facenot face does does not exceedN, exceed Na,where whereN N isa is a maximum a maximum allowable allowable number number of current-shift of current-shift workers workers at working at the the working face.face.

[0015] Preferably, aa safety

[0015] Preferably, safety control control strategy strategy of of the the working workingface facehuman-machine human-machine collaborative collaborative

operation safetymonitoring operation safety monitoring and and earlyearly warning warning system system includes includes a normalcontrol a normal operation operation control strategy, strategy,

aa human-machine mutual human-machine mutual avoidance avoidance control control strategy, strategy, a wireless a wireless shutdown shutdown control control strategy, strategy, a post a post

monitoringcontrol monitoring control strategy, strategy, and and aa special specialpersonnel personnel management strategy. management strategy.

[0016] Preferably,

[0016] Preferably, thethe normal normal operation operation control control strategy strategy includes: includes:

an electronic fence an electronic fence range is set, range is set,including includinga aworking working area, area, an an alarm alarm area, area, aashutdown area, and shutdown area, and aa

safety area,and safety area, andthethestates states of of thethe workers workers located located in different in different ranges ranges area set are set as as astate, working working a state, a hazardous state, an off-duty state, and a safe state, respectively; hazardous state, an off-duty state, and a safe state, respectively;

whenother when otherworkers workers approach approach the the working working area area of single of the the single equipment equipment unit, unit, or workers or workers of theof the single equipmentunit single equipment unitapproach approach thethe working working area area of other of other singlesingle equipment equipment units, imminent units, imminent

proximity hazard proximity hazardalarm alarminstruction instruction information informationisis generated; generated;

whenother when otherworkers workersenter enterthe theworking working areaofofthe area thesingle singleequipment equipment unit,ororworkers unit, workersofof thesingle the single equipment unitenter equipment unit enter the the working area of working area of other other single single equipment units, hazardous equipment units, area entry hazardous area entry alarm alarm

instruction information instruction information is generated; is generated; and and

the working the faceoutput working face outputcontrol controlsubsystem subsystem performs performs behavior behavior identification identification of aofworker a worker in in the the workingarea, working area,and andin ina case a case where where the worker the worker extends extends theand/or the head head limbs and/orbeyond limbsa beyond cab a cab protection area, protection area, or or exhibits exhibits fatigue fatigueor oryawn, yawn, abnormal behavioralarm abnormal behavior alarminstruction instructioninformation informationisis generated. generated.

[0017] Preferably, the

[0017] Preferably, the human-machine mutual human-machine mutual avoidance avoidance control control strategy strategy includes: includes:

aa worker not belonging worker not belongingtotothe thecurrent currentsingle single equipment equipmentunit unitisisdefined definedasasaapedestrian, pedestrian, aa worker worker controlling the controlling the current current single single equipment unit to equipment unit to move moveisisdefined definedasasa adriver, driver,and andother otherworkers workers operating thecurrent operating the current single single equipment equipment unitdefined unit are are defined as operators; as operators;

whenthe when thepedestrian pedestrianneeds needstotoenter entera aprotection protectionrange rangeofofthethecurrent currentsingle singleequipment equipment unit, unit, thethe

working face collaborative intelligent wearable device of the pedestrian sends an entry request to working face collaborative intelligent wearable device of the pedestrian sends an entry request to

the working the face mobile working face mobileequipment equipment safetymanagement safety management and control and control system system of driver; of the the driver;

uponreceiving upon receivingthe therequest, request,the thedriver driverpauses pausesa apersonnel personnel proximity proximity protection protection function function of the of the

equipment; and equipment; and

in in a a case whereananactual case where actualprotection protectionfunction functionpause pause time time tr isgreater tr is greaterthan than a preset a preset protection protection

function pause time function pause timetr, tR, or or the the working face output working face outputcontrol controlsubsystem subsystemdetects detectsthat thatthe thepedestrian pedestrian has safely passed, the single equipment unit automatically resumes a protection function. has safely passed, the single equipment unit automatically resumes a protection function.

[0018] Preferably,

[0018] Preferably, thethe wireless wireless shutdown shutdown controlcontrol strategystrategy includes: includes:

whenthe when theworking working face face output output control control subsystem subsystem or worker or the the worker detects detects an emergency an emergency state, state, an an emergency shutdown emergency shutdown signal signal is sent is sent viawireless via a a wireless terminal, terminal, andtheallwireless and all the wireless positioning positioning

terminals within terminals within aa signal signal coverage coveragearea areaimmediately immediately control control each each single single equipment equipment unitunit to shut to shut

down; and down; and

in a case in a casewhere wherean an actual actual shutdown shutdown duration duration twq is greater twq is greater than ashutdown than a preset preset duration shutdown tws,duration or tws, or uponinspecting upon inspectingaaworking workingcondition condition and and confirming confirming the the safety safety of of each each single single equipment equipment unit, unit, a a designated managerresumes designated manager resumes a working a working state state of of each each singleequipment single equipment unit. unit.

[0019] Preferably,

[0019] Preferably, thethe post post monitoring monitoring control control strategy strategy includes: includes:

in in aa case case where the number where the number ofofoperators operatorsfor foraa single single equipment equipmentunit unitexceeds exceedsa apreset presetnumber, number,allall equipment atthe equipment at the working workingface faceisis prohibited prohibited from frombeing beingstarted; started;

in in a case where a case wherethethe current current single single equipment equipment unit aenters unit enters a protection protection range range of a of a designated designated single single equipment unitfor equipment unit for collaborative collaborative operations, operations, neither neither of of the the two single equipment two single unitsgenerates equipment units generates any alarm instruction any alarm instruction information; information; and and

in in aa case case where oneof where one of the the operators operators of of the the current current single single equipment unit leaves equipment unit leaves the the range range of of the the single equipmentunit single equipment unitandand thenthen re-approaches re-approaches the single the single equipment equipment unit,instruction unit, alarm alarm instruction information information is is generated generated and and the alarm the alarm devicedevice is controlled is controlled to issueto a issue a corresponding corresponding alarm. alarm.

[0020] Preferably, the

[0020] Preferably, the special specialpersonnel personnel management strategyincludes: management strategy includes:

whena asingle when singleequipment equipment unitrequires unit requiresmaintenance, maintenance,thethe manager manager inputs inputs a maintenance a maintenance task task into into

the system via the working face collaborative intelligent wearable device, and the single equipment the system via the working face collaborative intelligent wearable device, and the single equipment

unit to be maintained stops outputting alarm and control signals; unit to be maintained stops outputting alarm and control signals;

the system the sets aa maintenance system sets duration, and maintenance duration, and when whenmaintenance maintenance personnel personnel enter enter a protection a protection range range

of the of the single singleequipment unit to equipment unit to be be maintained, maintained, the the system system outputs outputs related relatedvoice voiceprompts prompts to to remind remind

the maintenance the personneltotopay maintenance personnel payattention; attention; and and

whenthe when themaintenance maintenance duration duration exceeds exceeds a preset a preset maintenance maintenance duration, duration, or manager or the the manager sends sends a a maintenancecompletion maintenance completion instruction instruction to to thethe system, system, thethe single single equipment equipment unit unit tomaintained to be be maintained resumesoutputting resumes outputtingalarm alarmand andcontrol controlsignals. signals.

[0021] Thepresent

[0021] The presentdisclosure disclosurefurther further provides providesananimage imagerecognition recognition method method for for a working a working face face

human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early warning warning system, system, including including

the following steps: the following steps:

acquiring acquiring video video images capturedby images captured bythe the camera cameradevice deviceatat the the working face mobile working face mobileequipment equipmentsafety safety managementand management andcontrol controlsystem systemasastraining training data, data, and using worker and using worker images images annotated annotated with with personnel information as training data labels; personnel information as training data labels;

performing unsupervised performing unsupervised training trainingand anddeep deeplearning ononimproved learning improvedYOLOV5 YOLOV5 bybymeans means of of the the

training data and the corresponding training data labels to obtain an image recognition model; and training data and the corresponding training data labels to obtain an image recognition model; and

performingvideo performing videoimage imagerecognition recognitionbybymeans means of of thethe trainedimage trained image recognition recognition model model to to complete complete

identity identity recognition, recognition, target targettracking, tracking,behavior behaviorprediction predictionfor forworkers, workers, and and working environment working environment

recognition. recognition.

[0022] Preferably, the

[0022] Preferably, the training training data datalabels labelsinclude includeposture postureimages images of workers of workers with different with different

postures, at different shooting angles, and under different shooting categories, and facial images postures, at different shooting angles, and under different shooting categories, and facial images

of the workers in different states; of the workers in different states;

the postures include standing, squatting, bending down, left-side standing, and right-side standing; the postures include standing, squatting, bending down, left-side standing, and right-side standing;

the shooting the shootingangles anglesinclude include top-down top-down long-range long-range shooting, shooting, top-down top-down short-range short-range shooting, shooting,

bottom-uplong-range bottom-up long-rangeshooting, shooting,bottom-up bottom-up short-range short-range shooting, shooting, frontallong-range frontal long-range shooting,andand shooting,

frontal short-rangeshooting; frontal short-range shooting;

the shooting the shootingcategories categories include: include: visible visible light light high-definition high-definition shooting, shooting, low-illumination low-illumination

supplementary lightingshooting, supplementary lighting shooting,and andthermal thermalimager imagershooting; shooting;and and the different the different states statesof ofthe theworkers workers include: include: an an unoccluded state, aa state unoccluded state, state of of wearing wearing aa protective protective mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing earmuffs, mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing earmuffs, aa state state of of wearing wearing a a protective protective mask andaasafety mask and safety helmet, helmet,aa state state of of wearing wearing aa safety safety helmet helmetand and earplugs, earplugs, aastate stateofofwearing wearing a safety a safety helmet helmet and earmuffs, and earmuffs, a stateaof state of wearing wearing a protective a protective mask and mask and earplugs, earplugs, a astate stateofofwearing wearing a protective a protective mask mask and earmuffs, and earmuffs, a state a state of of awearing wearing a protective protective mask, mask, aa safety helmetand safety helmet and earplugs, earplugs, and and a state a state of wearing of wearing a protective a protective mask, mask, a safety a safety helmet helmet and and earmuffs. earmuffs.

[0023] Preferably, the

[0023] Preferably, the image imagerecognition recognitionmodel model adjusts adjusts anchor anchor boxes boxes by means by means of a K-means of a K-means

clustering algorithm, clustering algorithm, andand optimal optimal cluster cluster centers centers of theof the K-means K-means clustering clustering algorithm algorithm are obtainedare obtained

by partitioning by partitioning training training samples samplesandand calculating calculating fitness fitness by means by means of a intelligence of a swarm swarm intelligence optimization method. optimization method.

[0024] Preferably, the

[0024] Preferably, the swarm intelligence optimization swarm intelligence methodincludes: optimization method includes:

initializing initializinga particle swarm a particle and swarm andrandomly randomly generating generating N N particles, particles,where where11<N<n, < N < n, < nn is is the thenumber number

of anchorboxes of anchor boxes to classified; to be be classified; and partitioning and partitioning the n anchor the n anchor boxes by boxes taking by the taking the N N particles as particles as

cluster centers; cluster centers;

calculating and normalizing calculating and normalizing distances distances fromfrom each each samplesample datatopoint data point to the centers, the cluster cluster centers, calculating andnormalizing calculating and normalizing a distance a distance between between any two any twocenters, cluster clusterand centers, and calculating calculating fitness of fitness of the sample the data; and sample data; and

performingoptimization performing optimizationbased basedononthethefitness fitnessofofthe the sample sampledata databybymeans meansof of a PSO a PSO algorithm algorithm to to obtain theoptimal obtain the optimal cluster cluster centers. centers.

[0025] Preferably,

[0025] Preferably, calculating calculating and and normalizing normalizing distances distances from from each each sample sample data point data to thepoint to the cluster cluster

centers includes: centers includes:

K

where D represents a distance from uth sample data A in a jth cluster ω of a sample dataset to a in where D represents a distance from uth sample data Au in a jth ucluster Wj of a sample j dataset to a

cluster centerC,Cjand cluster center , and K the K is is the number number of cluster of cluster centers; centers;

where D′inrepresents whereD'i representsnormalized normalized Din, max(D D, max(Din) andin)min(Din) and min(D in) represent represent a maximum a maximum value andvalue a and a minimum minimum value value among among all all D, D in, respectively. respectively.

10

[0026] Preferably,

[0026] Preferably, calculating calculating and and normalizing normalizing a distance a distance between between any two any two cluster cluster centers centers includes: includes:

D = d(w,w)

whereDDrepresents where out represents a distance a distance between between any any two cluster two cluster centers, centers, Ci Cand C and areCcluster j are cluster centers centers of of an ith cluster an ith cluster Wωand i and thethe jth jth cluster cluster ωj, respectively, w, respectively, N isnumber N is the the number of particles of particles in the initialized in the initialized

particle swarm, and D(ω , ω ) represents an absolute value of a difference value between the cluster particle swarm, and D(w, w) irepresents j an absolute value of a difference value between the cluster

centers centers C andC; C i and Cj;and andDout' Dout’is is normalized Dout, and normalized Dout, max(Dout)and and max(Dout) andmin(Dout) min(Dout)represent represent aa maximum maximum value and value and aa minimum value minimum value among among all all Doutrespectively. Dout, , respectively.

[0027] Preferably,

[0027] Preferably, thethe fitness fitness of of thethe sample sample data data is as is as follows: follows:

F=Din'+ 1 where where F F represents represents the the fitness fitness of the of the sample sample data. data.

[0028] Preferably, after

[0028] Preferably, after performing imagerecognition performing image recognitionononeach eachframe frame of of a video a video captured captured by by the the

image recognitionmodel, image recognition model,the themethod method furtherincludes: further includes:

generating generating a a video video trajectory trajectory for for eacheach worker worker in the in the image; image; and and

performing worker trajectory tracking and prediction based on the video trajectory, so as to enable performing worker trajectory tracking and prediction based on the video trajectory, so as to enable

risk determination under partially or fully occluded states of the worker. risk determination under partially or fully occluded states of the worker.

[0029] Thepresent

[0029] The presentdisclosure disclosurefurther further provides provides aa wireless wireless positioning positioning method methodfor fora aworking working face face

human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early warning warning system, system, including including

the following steps: the following steps:

acquiring acquiring measurement databetween measurement data between thewireless the wirelesspositioning positioningterminal terminaland andthe thewireless wireless positioning positioning base station at a time t, setting the wireless positioning base station as an original point to define base station at a time t, setting the wireless positioning base station as an original point to define

the position the position of of the the wireless wireless positioning positioning terminal terminal within within a a wireless wireless coverage area of coverage area of the the wireless wireless positioning base station as (x , y ), and obtaining distance data s between the wireless positioning t tand obtaining distance data S between the wireless positioning positioning base station as (x, yt),

terminal and the wireless positioning base station; terminal and the wireless positioning base station;

11

performing differential operations on the distance data s for a preset number of times to obtain a performing differential operations on the distance data S for a preset number of times to obtain a

velocity v and an acceleration a of the wireless positioning terminal; velocity v and an acceleration a of the wireless positioning terminal;

comparing and comparing and determining determining aavelocity velocityvt and an acceleration V and at at the an acceleration time a at t with the timea t velocityv with a t-1 and and

an accelerationat-1 an acceleration at-1atatthe thetime timet-1: t-1:

when the variations in the velocity v and the acceleration a are within preset threshold ranges, the when the variations in the velocity V and t the acceleration at are within t preset threshold ranges, the

position (x , y ) of the current wireless positioning terminal is valid; and position (x, t y) tof the current wireless positioning terminal is valid; and

otherwise, based otherwise, based on on the the velocity velocity vt-1the V-1 and andacceleration the acceleration at-1 at athe t-1 at thet-1time time and t-1 and a velocity a velocity Vt-2 and vt-2 and

′ predicted acceleration an accelerationat-2 an acceleration at-2atatthe thetimet-2, timet-2,predicting predicting a predicted a predicted velocity velocity 𝑡 and a predicted acceleration v't and𝑣 a

′ at the time t, and based on the predicted velocity v't and 𝑎 t at a't the time t, and based on the predicted velocity 𝑣𝑡′ and the the predicted predicted a', 𝑎𝑡′ , acceleration acceleration

′ ′ the wireless positioning terminal at the time t. calculating calculating a apredicted predicted position position (x't(𝑥 𝑡 ,𝑦𝑡 ) of the wireless positioning terminal at the time t. ,y') of

[0030] Compared

[0030] Compared with with the prior the prior art,present art, the the present disclosure disclosure has the has the beneficial beneficial effects: effects:

Theworking The workingface facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and warning and early early warning system proposedininthe system proposed thepresent presentdisclosure disclosure proposes proposesa asafety safetycontrol control logic logic and and strategy strategy developed developed to ensure to ensure the the safety safety of of personnel personnel and and equipment duringhuman-machine equipment during human-machine collaborative collaborative operations operations

in in working faces. Compared working faces. withtraditional Compared with traditional underground undergroundpersonnel personnel safetyprotection safety protectionsystems, systems,the the present disclosure is not limited to personnel safety protection of a single equipment unit or the present disclosure is not limited to personnel safety protection of a single equipment unit or the

safety protectionofofoperations, safety protection operations, but but instead instead starts starts fromfrom the mining the mining process process of an equipment of an equipment cluster cluster at at the theworking working face, face,and, and,by bycombining human-machine combining human-machine collaboration collaboration andand equipment equipment linkage linkage logic logic

control strategies,establishes control strategies, establishes a safety a safety early early warning warning device device and logic and control control logicthat thereof thereof match that match

the underground the undergroundworking working faceface production production process, process, thereby thereby achieving achieving dynamicdynamic integration integration of of personnel, equipment, personnel, equipment,and andprocess processininthe the underground undergroundworking working face. face.

[0031]

[0031] According to the According to the image imagerecognition recognition method methodfor forthe theworking workingface facehuman-machine human-machine collaborative operation safety collaborative operation safety monitoring monitoringandand early early warning warning system system provided provided by the by the present present

disclosure, disclosure, the the K-means clusteringalgorithm K-means clustering algorithm in in YOLOV5 YOLOV5 is improved, is improved, a swarm a swarm intelligence intelligence

optimization method optimization methodisisadopted adoptedtotoautomatically automaticallydetermine determinethethe optimal optimal K value, K value, so so as as to to achieve achieve

an optimal clustering an optimal clustering effect, effect, thereby thereby improving the accuracy improving the accuracyand andprecision precisionofofimage imagerecognition recognition and providingananeffective and providing effectiveimprovement improvement direction direction for for the the human-machine human-machine collaborative collaborative safety safety

management management andand control control of of thesystem. the system.

[0032] Thewireless

[0032] The wirelesspositioning positioning method method for working for the the working face human-machine face human-machine collaborative collaborative

operation operation safety safety monitoring and early monitoring and early warning systemprovided warning system providedbybythe thepresent presentdisclosure disclosureachieves achieves single-base-station single-base-station positioning positioning of of underground mobile underground mobile equipment, equipment, avoiding avoiding the the situation situation where where

12

positioning data of the heading face cannot be updated timely due to the fact that the positioning positioning data of the heading face cannot be updated timely due to the fact that the positioning

base station base station does not timely does not timely update updatelocation locationinformation, information,which whichleads leadstotolarge largedeviations deviationsininthe the positioning data. This method realizes real-time acquisition of the precise and relative positions of positioning data. This method realizes real-time acquisition of the precise and relative positions of

personnel and personnel and equipment, equipment, thereby thereby achieving achieving real-time real-time monitoring monitoring of of workers workersaround aroundthethe equipment, enablingrefined equipment, enabling refinedcontrol control of of the the equipment basedononthe equipment based thepositions positionsof of the the personnel, personnel, and and

ensuring personnel ensuring personnel safety safety protection protection while while maintaining maintaining efficientefficient production. production.

[0033] Theworking

[0033] The working face face human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning system provided provided by present by the the present disclosure disclosure achieves achieves safety safety early warning early warning and risk and risk management management andand control control during during human-machine human-machine collaborative collaborative operations operations at theatworking the working face face by by means of wireless positioning, video surveillance, intelligent vision processing, and other technical means of wireless positioning, video surveillance, intelligent vision processing, and other technical

methods,thereby methods, therebyimproving improving the the false false alarm alarm rate rate of safety of the the safety earlyearly warning warning devicedevice and theand the accuracy ofofpersonnel accuracy personnelrecognition. recognition. By By means means of theofintelligent the intelligent personnel personnel wearable wearable devices,devices,

environmental monitoringdevices, environmental monitoring devices,entry entryand andexit exitmanagement management devices, devices, and and other other equipment, equipment, the the

system further monitors system further monitorsthe thephysical physicalhealth healthofofworkers workers at the at the working working face face and operating and the the operating environmentininreal environment real time, time, thereby achieving safety thereby achieving safety management management andand control control of of theworkers. the workers.

DESCRIPTION DESCRIPTION OF OF THE THE DRAWINGS DRAWINGS

[0034]

[0034] InInorder order to to illustrate illustrate thethe technical technical solutions solutions ofembodiments of the the embodiments of the of the present present disclosure disclosure

moreclearly, more clearly, the the accompanying drawings accompanying drawings required required forfor describing describing theembodiments the embodiments of the of the present present

disclosure will be disclosure will briefly introduced be briefly below.ItIt is introduced below. is clear clear that that the the accompanying drawings accompanying drawings in the in the

description below description showonly below show onlysome some embodiments embodiments ofpresent of the the present disclosure, disclosure, and and those those of ordinary of ordinary

skill skillininthe artart the cancan obtain other obtain accompanying other drawings accompanying drawingsaccording according to tothese theseaccompanying drawings accompanying drawings

without creative efforts. without creative efforts.

[0035] FIG.11isis aa schematic

[0035] FIG. schematicdiagram diagram of of thethe composition composition of the of the working working face human-machine face human-machine

collaborative operation safety collaborative operation safety monitoring monitoringandand early early warning warning system system according according to theto the present present

disclosure; disclosure;

FIG. 22 is FIG. is aa schematic diagramofofthe schematic diagram thecomposition compositionofofa asingle singleequipment equipment unit unit atatthe theworking working face face

human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring andand early early warning warning system system according according to to the present disclosure; the present disclosure;

FIG. 33 is FIG. is aa schematic schematic diagram of the diagram of the composition ofan composition of an intelligent intelligent wearable wearable device device at at the the working working

face human-machinecollaborative face human-machine collaborative operation operation safety safety monitoring monitoring and and early early warning warningsystem system according to the present disclosure; according to the present disclosure;

13

FIG. 44 is FIG. is aa schematic diagramofofaacontrol schematic diagram control strategy strategy of of an an entry entry and exit management and exit and management and control control

device at the device at the working facehuman-machine working face human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto thepresent the presentdisclosure; disclosure;

FIG. 55 isis aa schematic FIG. schematicdiagram diagram of of a human-machine a human-machine mutual mutual avoidance avoidance control strategy control strategy of the of the workingface working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring andand early early warning warning system system

according according toto thepresent the present disclosure; disclosure;

FIG. 66 is FIG. is aa schematic diagramofofaawireless schematic diagram wirelessemergency emergency stop stop control control strategyofofthe strategy theworking working face face

human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring andand early early warning warning system system according according to to the present disclosure; the present disclosure;

FIG. 77 is FIG. is aa schematic diagramofofnormal schematic diagram normalcollaborative collaborativeoperation operationcontrol controlamong among equipment equipment at the at the

workingface working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and and early early warning warning system system

according according toto thepresent the present disclosure; disclosure;

FIG. 88 is FIG. is aa schematic diagramofofhazardous schematic diagram hazardous operation operation control control among among equipment equipment at theatworking the working face human-machinecollaborative face human-machine collaborative operation operation safety safety monitoring monitoring and and early early warning warningsystem system according according toto thepresent the present disclosure; disclosure; and and

FIG. 9 is a schematic diagram of a dynamic change of the electronic fence size during the swinging FIG. 9 is a schematic diagram of a dynamic change of the electronic fence size during the swinging

of of aa tail tailsection at at section thethe working face working human-machine face collaborative operation human-machine collaborative operation safety safety monitoring monitoring and and

early early warning systemaccording warning system accordingtotothe thepresent present disclosure. disclosure.

[0036] In the

[0036] In the figures: figures: 11-working faceenvironment 11-working face environment monitoring monitoring system; system; 12-working 12-working face video face video

surveillance surveillance system; 13-workingface system; 13-working facecollaborative collaborativeintelligent intelligent wearable wearabledevice; device;14-turnstile; 14-turnstile; 15- 15- centralized control platform; centralized control platform;16-working 16-workingfaceface entrance entrance videovideo surveillance surveillance system; system; 21-video 21-video

camera; 22-intelligent camera; 22-intelligent visual visual analysis analysis system; system; 23-data 23-data transmission transmission device; 24-multi-source device; 24-multi-source sensor; sensor; 25-driving behavior 25-driving behavioranalysis analysisdevice; device;26-wireless 26-wirelesspositioning positioning terminal; terminal; 27-thermal 27-thermal imager; imager; 31- 31- portable positioning portable positioning terminal; terminal;32-instant 32-instantmessaging messaging system; system; 33-smart 33-smart camera; camera; 34-intelligent 34-intelligent

terminal device; terminal device; 35-smart wristband; 36-motion 35-smart wristband; 36-motionmonitoring monitoring device. device.

DETAILED DESCRIPTION DETAILED DESCRIPTION OF OF EMBODIMENTS EMBODIMENTS

[0037] Thepresent

[0037] The presentdisclosure disclosureisistotobebedescribed describedfurther furtherinindetail detailbelow below with with reference reference to the to the

drawings andembodiments. drawings and embodiments.It It cancan be be understood understood that that specific specific embodiments embodiments described described herein herein are are

intended merelytotoillustrate intended merely illustrate the the related related disclosure disclosure and are not and are not to to be be construed construedasaslimiting limitingthe the present disclosure. present disclosure. It It should be additionally should be additionally noted notedthat, that, for for ease ease ofofdescription, description, only onlyportions portions relevant to relevant to the therelated relateddisclosure areareshown disclosure shownin inthe theaccompanying drawings. accompanying drawings.

14

[0038] It should

[0038] It shouldbebe noted noted that, that, without without conflict, conflict, the embodiments the embodiments of thedisclosure of the present present disclosure and the and the features features in in the the embodiments can embodiments can be be combined combined with with each other. each other. The present The present disclosure disclosure will be will be

described in detail described in detail below with reference below with reference to to the the accompanying accompanying drawings drawings and and in conjunction in conjunction withwith

the embodiments. the embodiments.

[0039] Thepresent

[0039] The presentdisclosure disclosureprovides providesa aworking working face face human-machine human-machine collaborative collaborative operation operation

safety safety monitoring andearly monitoring and earlywarning warningsystem, system, including including a working a working faceface mobile mobile equipment equipment safetysafety

management management andand control control system, system, working working face collaborative face collaborative intelligent intelligent wearable wearable devices, devices, an an information sharingplatform information sharing platformand anda working a working faceface output output control control subsystem. subsystem. The working The working face face mobileequipment mobile equipment safety safety management management and control and control system system is configured is configured to real-time to perform perform real-time perception ofof aaposition perception positionand andan an orientation orientation of of each each single single equipment equipment unit, unit, perception perception of an of an operational status of operational status of each single equipment each single equipmentunit, unit,perception perceptionofofananenvironment environment within within a preset a preset

range surrounding range surroundingeach eachsingle singleequipment equipment unit,and unit, andbehavior behavior acquisitionandand acquisition analysis analysis fora adriver for driver of of each single equipment each single equipmentunit. unit.Each Each working working faceface collaborative collaborative intelligent intelligent wearable wearable device device 13 13

includes includes a aportable portablepositioning positioning terminal terminal 31a and 31 and a motion motion monitoring monitoring device device 36 that are 36 that are attached to attached to

aa worker, wherethe worker, where theportable portablepositioning positioningterminal terminal3131isis configured configuredtotoperceive perceiverelative relative positions positions and relative orientations and relative orientations between the worker between the workerand andeach eachsingle singleequipment equipment unit unit andand a position a position of of a a

workingface, working face, and and the the motion monitoringdevice motion monitoring device3636isisconfigured configuredtoto acquire acquire motion motionpostures posturesofof the the workerinin real worker real time. time. The informationsharing The information sharingplatform platformisisconfigured configuredtotocollect collectinformation informationfrom from each single equipment each single unit and equipment unit and each each working workingface facecollaborative collaborativeintelligent intelligent wearable wearable device device 13 13 and and

achieve information achieve informationsharing. sharing. The Theworking working faceoutput face outputcontrol controlsubsystem subsystem is is configured configured to to receive receive

shared informationfrom shared information fromthe theinformation information sharing sharing platform, platform, perform perform entry entry and and exitexit identification identification

management management of of workers, workers, position position recognition recognition and monitoring and monitoring of theof the workers workers and and each each single single equipment unit,behavior equipment unit, behaviorrecognition, recognition,monitoring monitoring and and prediction prediction of workers of the the workers and working and working

environmentmonitoring, environment monitoring, and and generate generate corresponding corresponding alarm instruction alarm instruction information, information, where where workingenvironments working environments include include gas gas concentrations concentrations of gas, of gas, carbon carbon monoxide, monoxide, and oxygen, and oxygen, roof roof displacement, windspeed displacement, wind speedand andwind wind pressure. pressure.

[0040] In order

[0040] In order to to more more clearly clearly describe describe the theworking working face face human-machine collaborative human-machine collaborative operation operation

safety safety monitoring andearly monitoring and earlywarning warning system system of the of the present present disclosure, disclosure, thethe embodiments embodiments of theof the

present disclosure are described in detail below with reference to FIG. 1. The working face in the present disclosure are described in detail below with reference to FIG. 1. The working face in the

present disclosure present disclosure specifically specifically refers referstotoa a heading headingface. face.When When safety safety pre-approval and control pre-approval and control of of human-machinecollaborative human-machine collaborative operations operations are are performed performedatatother otherworking workingfaces, faces,adaptive adaptive adjustmentsmay adjustments mayalso alsobebemade made based based on on specific specific application application scenarios,and scenarios, andthethepresent presentdisclosure disclosure is is not not limited thereto. limited thereto.

15

[0041] Theworking

[0041] The working face face human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warning system of the present disclosure is described in detail as follows: warning system of the present disclosure is described in detail as follows:

As shown As shownininFIG. FIG.2,2,the theworking working face face mobile mobile equipment equipment safety safety management management and control and control system system

includes a wireless includes a wireless positioning positioning terminal 26, aa camera terminal 26, device, aa driving camera device, driving behavior behavioranalysis analysisdevice device 25, a multi-source sensor 24, an intelligent visual analysis system 22 and a data transmission device 25, a multi-source sensor 24, an intelligent visual analysis system 22 and a data transmission device

23 that 23 that are are provided providedononeach each single single equipment equipment unit.unit. The wireless The wireless positioning positioning terminal terminal 26 is 26 is configured toto perform configured performreal-time real-timeperception perception of of thethe position position andand the the orientation orientation of each of each single single

equipment unit. The equipment unit. Thecamera camera device device includes includes a video a video camera camera 21 aand 21 and a thermal thermal imagerimager 27 and27 is and is

configured to configured to collect collect environmental video images environmental video imagesininreal real time time within within the the preset preset range range surrounding surrounding

each single equipment each single unitand equipment unit andperform performenvironmental environmental perception. perception. TheThe driving driving behavior behavior analysis analysis

device 25 is device 25 is configured configuredtoto collect collect driver driver behavior videoimages behavior video imagesininreal realtime timeand andanalyze analyze driver driver

behaviors. The multi-source sensor 24 is configured to acquire the operational status of each single behaviors. The multi-source sensor 24 is configured to acquire the operational status of each single

equipment unit equipment unit in in real real time. time.

[0042] Asshown

[0042] As shown in FIG. in FIG. 3, each 3, each working working face collaborative face collaborative intelligent intelligent wearable wearable device 13 device 13

includes a portable includes a portable positioning positioning terminal terminal 31, 31, an an instant instant messaging system32, messaging system 32,a asmart smartcamera camera33,33,

an an intelligent intelligentterminal terminaldevice device34, 34,a asmart smartwristband wristband35 35 and and aa motion motion monitoring device36. monitoring device 36.

[0043] Theworking

[0043] The workingface faceoutput outputcontrol controlsubsystem subsystem includes includes a wireless a wireless positioning positioning base base station,a a station,

wireless network wireless networktransmission transmissiondevice, device,ananequipment equipment control control device, device, a voice a voice alarm alarm device device and and a a centralized centralized control control platform platform 15. 15. The wireless positioning The wireless positioning base base station station is is configured to cooperate configured to cooperate

with the with the wireless wirelesspositioning positioningterminal terminal 26 each 26 of of each singlesingle equipment equipment unit andunit the and the portable portable positioning terminal positioning terminal 31 31 of of each workertoto perform each worker performperception perceptionofofthe thepositions positionsand andorientations orientations of of each single equipment each single equipment unit unit and and each each worker. worker. The wireless The wireless network network transmission transmission device is device is

configured to configured to enable enablecommunication communication between between the working the working face collaborative face collaborative singlesingle equipment equipment

units, the wireless positioning terminals 26, and the working face collaborative intelligent wearable units, the wireless positioning terminals 26, and the working face collaborative intelligent wearable

devices (13) and devices (13) the information and the sharing platform. information sharing platform. The Theequipment equipmentcontrol controldevice deviceisisconfigured configuredtoto determine determine aahazardous hazardousstate stateofofeach eachsingle singleequipment equipment unitunit and and each each worker worker by analyzing by analyzing and and processing data, processing data, perform performentry entry andand exitexit identification identification management management of theof the workers, workers, positionposition

recognition and recognition and monitoring monitoringofofeach eachworker workerand and each each singleequipment single equipment unit, unit, behavior behavior recognition, recognition,

monitoring, and monitoring, prediction of and prediction the workers of the workers and andworking workingenvironment environment monitoring, monitoring, output output

corresponding controlsignals corresponding control signalsbased basedononcontrol controlstrategies strategiestoto control control each eachpiece pieceofofequipment equipmentto to

performcorresponding perform corresponding risk risk avoidance avoidance actions, actions, and transmit and transmit the generated the generated alarm instruction alarm instruction

information to the information to the centralized centralized control control platform platform 15. 15. The The voice voice alarm device is alarm device is configured to, upon configured to, upon

receiving the receiving the alarm alarm instruction instruction information, information, perform perform alarms of corresponding alarms of categoriesbased corresponding categories basedonon

16

different different categories categories of of the the alarm alarm instruction instruction information. information. The centralized control The centralized control platform 15 is platform 15 is configured to, configured to, based based on on the the shared shared information of the information of the information information sharing sharing platform, platform, perform entry perform entry

and exit identification and exit identificationmanagement management ofofthe theworkers, workers,position positionrecognition recognitionand and monitoring monitoring of each of each

workerand worker andeach eachsingle singleequipment equipment unit, unit, behavior behavior recognition, recognition, monitoring, monitoring, and and prediction prediction of the of the

workers, and workers, andworking working environment environment monitoring, monitoring, and generate and generate corresponding corresponding alarm instruction alarm instruction

information. information.

[0044]

[0044] A managementprocess A management processandand controlstrategies control strategies of of the the working workingface facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring andearly monitoring and early warning warningsystem systemofof thepresent the presentdisclosure disclosureare are illustrated illustratedbelow below through through several several examples. examples.

[0045] Example1.1.working

[0045] Example working face face entryand entry and exitmanagement exit management process: process:

A turnstile1414provided A turnstile provided withwith an identity an identity recognition recognition device device (such as (such an irisas an iris recognition recognition device, a device, a facial facial recognition device, recognition device, or or another another identity identity recognition recognition device), device), or personnel or other other personnel entry andentry exit and exit

management management devices devices (collectivelyreferred (collectively referredtotoasasan an"entry “entryand andexit exit management management device”), device"), is is used used

to perform to entry and perform entry and exit exit management forworkers management for workers arrivingatatananentrance arriving entranceofofthe theworking workingface, face,soso as to prevent as to preventforeign foreignpersonnel personnel from from intruding intruding into into a working a working area without area without authorization. authorization.

Meanwhile,a aworking Meanwhile, working duration duration of the of the worker worker at the at the working working face face is is recorded recorded and included and included in in attendance management. attendance management. A working A working faceface entrance entrance videovideo surveillance surveillance system system 16 is 16 is configured configured to to monitor and monitor and manage managethe theentry entry and andexit exit management managementdevice, device,sosoasastotoprevent prevent unauthorized unauthorized personnel from personnel fromclimbing climbingover overrailings railings and andintruding intruding into into the the working area. When working area. When ananunauthorized unauthorized person intrudes into a safety alarm range of the equipment, the system triggers an alarm and sends person intrudes into a safety alarm range of the equipment, the system triggers an alarm and sends

the alarm to the intelligent terminal device 34 of a manager. the alarm to the intelligent terminal device 34 of a manager.

[0046] Modesfor

[0046] Modes forthe the worker workerentry entryinto into and and exit exit from from the the working face include: working face include: aa normal normal operation operation

mode,aashift mode, shift handover mode,a alinkage handover mode, linkagecontrol controlmode, mode,and anda atemporary temporary authorization authorization mode. mode.

[0047] Thenormal

[0047] The normaloperation operationmode modeis is asas follows: follows:

Management personnel Management personnel (generally (generally a shift aleader) shift leader) enter identities enter identities of current-shift of current-shift workers permitted workers permitted

to enter to enter into intothe thesystem system and and set set aamaximum allowable maximum allowable number number of the of the current-shift current-shift workers workers at at thethe

workingface working faceasasN.NaUpon . Upon the the entry entry of of a worker a worker intointo thethe working working face, face, the the worker worker is required is required to to first firstpass passthrough through the theentry entryand and exit exitmanagement device.The management device. Theentry entryand andexit exitmanagement management device device

performs identity recognition of the worker. The worker with an authorized identities is permitted performs identity recognition of the worker. The worker with an authorized identities is permitted

to enter, to enter,and and the the entry entry and and exit exit management device management device records records thethe currenttime current time T; Tpersonnel in; personnel withwith

unauthorized identities are unauthorized identities areprohibited prohibitedfrom frompassing passingthrough through the theentry entryand andexit exitmanagement device. management device.

Uponmultiple Upon multipleunauthorized unauthorizedentry entryattempts attemptsbybypersonnel, personnel,thethesystem system outputs outputs anan alarm alarm signal. signal.

17

[0048] Whenthe

[0048] When thenumber number na personnel n of of personnel passing passing through through is less is less than than or or equal equal toto N,Nthe a, the entryand entry and exit management exit device management device permits permits continued continued entry; entry; when when n >nN, a >entry Na, entry is prohibited. is prohibited.

[0049] Whena aworker

[0049] When worker needs needs to to leave leave thethe working working face, face, upon upon successful successful identity identity recognition, recognition, thethe

turnstile 14 turnstile 14 opens opens to to allow allow the the worker to exit, worker to exit,and and when the worker when the workerleaves leavesthe theworking workingface, face,the the time TTout time is is recorded. recorded.

[0050] Theworking

[0050] The workingduration durationofofthe theworker's worker’sentry entryinto intothe the working workingface faceisis calculated calculated as as T Tww= = T T out

-- TT,in, where whereT Tisin is determined determined based based on the on the earliest earliest time time thethe worker worker passes passes through through the entry the entry and and

exit management exit device,and management device, andTout Toutis is determined basedononthe determined based thelatest latest time time the the worker worker passes passes through through

the entry the entry and exit management and exit management device. device. TheThe timetime Twnot Tw is is not allowed allowed to exceed to exceed a preset a preset working working

duration Ty.InInthe duration Ty. thepresent present disclosure, disclosure, the the preset preset working working duration duration Tytois 24 Ty is set seth.toWhen 24 h. Tw When > Ty, Tw > Ty,

the system outputs an alarm signal and notifies a super administrator (typically a system manager). the system outputs an alarm signal and notifies a super administrator (typically a system manager).

[0051] Thesystem

[0051] The systemrecords recordsthe theentry entryand andexit exittimes timesand andthe thenumber numberof of passages passages forfor each each worker. worker.

Whenthe When thesystem system entersthe enters thenormal normal operation operation mode, mode, any any entry entry intointo or or exit exit from from thethe working working faceface

by aa worker by workershall shallbebeauthorized authorizedbyby thethe manager. manager. Information Information may may be be entered entered via thevia the portable portable

intelligent terminal intelligent terminaldevice device 34 34 or or other otherinformation information input input devices. devices. Information Information about the manager about the manager

is is entered bythe entered by thesuper super manager. manager. Atbeginning At the the beginning of eachof eachonly shift, shift, oneonly oneismanager manager is be allowed to allowed to be on duty. on duty. During shift handover, During shift the handover handover, the handoverisispermitted permittedonly onlywhen when both both managers managers are are on duty on duty

simultaneously. simultaneously.

[0052] Theshift

[0052] The shift handover modeisisasasfollows: handover mode follows:

During shift handover, During shift the current-shift handover, the current-shiftmanager manager and and the the next-shift next-shiftmanager manager perform the handover perform the handover and confirm and confirmwhether whether thethe conditions conditions for for handover handover between between theshifts the two two are shifts are satisfied. satisfied. If theIf the handover conditions are satisfied, both shift managers confirm the activation of the handover mode handover conditions are satisfied, both shift managers confirm the activation of the handover mode

via the via the intelligent intelligent terminal terminal devices 34. Upon devices 34. Uponactivation activationofofthethehandover handover mode, mode, the next-shift the next-shift

manager enters the information of the current-shift workers into the device. At this time, workers manager enters the information of the current-shift workers into the device. At this time, workers

with authorized with authorized identities identities are are allowed allowed to to pass pass through through the the entry entry and and exit exit management deviceandand management device

enter the working enter the working face. face.

[0053] Uponauthorization

[0053] Upon authorizationandand confirmation confirmation by by the the manager manager afterafter the personnel the personnel at working at the the working face face complete the handover, complete the handover,the the system systementers enters the the normal normaloperation operationmode. mode.

[0054] Uponcompletion

[0054] Upon completion of the of the handover handover mode,mode, the system the system performs performs a statistical a statistical count count of the of the

personnel at the working face. For any worker not belonging to the current shift, the system issues personnel at the working face. For any worker not belonging to the current shift, the system issues

an alarm notification an alarm notification until until the worker exits. the worker exits. Relevant Relevant alarm alarmcommand command information information is is

18

simultaneously sent simultaneously sent to the to the intelligent intelligent terminal terminal device device 34 of 34 theof the manager manager and the centralized and the centralized control control platform 15. platform 15.

[0055] Thelinkage

[0055] The linkagecontrol control mode modeisisasas follows: follows:

Theworking The workingface faceentrance entrancevideo videosurveillance surveillancesystem system1616monitors monitorsthe theentrance entranceofof the the working face. working face.

Whenthe When theworker workernotnotbelonging belonging to to thecurrent the currentworking workingface faceintrudes intrudesinto intothe the working workingface facewithout without authorization, authorization, ororintends intendsto to climb climb over over or bypass or bypass the entry the entry andmanagement and exit exit management device, thedevice, system the system

continuously issues alarm signals. continuously issues alarm signals.

[0056] Whenthethenumber

[0056] When number of personnel of personnel exceeds exceeds a predefined a predefined maximum maximum allowable allowable number Nnumber for Na for the working the face, the working face, the system issues aa “working system issues facefull" "working face full” or or other other alarms alarms and prohibits personnel and prohibits personnel

from entering the from entering the working face. working face.

[0057] Whenallallpersonnel

[0057] When personnelpass passthrough through theentry the entryandand exitmanagement exit management device, device, all all equipment equipment at at

the working the workingface facewill willbebepowered powered off off to stop to stop operating, operating, thereby thereby ensuring ensuring the safety the safety of of shift shift handoverworkers handover workerswhen when entering entering andand exiting exiting thethe working working face. face. Specifically,after Specifically, afterpersonnel personnelenter enter the working the workingface, face,the theworking working faceface environment environment monitoring monitoring system system 11 periodically 11 periodically acquires acquires

environmental parametersofofthetheworking environmental parameters working face. face. When When the the environmental environmental parameters parameters failmeet fail to to meet the safe the safe production production conditions conditionsrequired requiredfor forpersonnel, personnel,andand thethe personnel personnel need need to evacuate to evacuate the the workingface working faceinin case case of of an an emergency, emergency,the theentry entry and andexit exit management management device device is is capable capable of of timely timely

opening anemergency opening an emergency escape escape passage passage to to ensure ensure thethe orderly orderly and and rapid rapid evacuation evacuation of of personnel. personnel.

[0058] When

[0058] When a worker a worker enters enters the the working working face,face, the working the working face entrance face entrance video surveillance video surveillance

system 16performs system 16 performsintelligent intelligent identity identity recognition recognition of of the the worker andobtains worker and obtainsinformation informationabout about the worker the workerfrom fromthethe intelligentterminal intelligent terminal device device 34 for 34 for cross-verification, cross-verification, thereby thereby preventing preventing

situations situationswhere where “worker identity mismatch "worker identity with card-based mismatch with card-based access" access” and and preventing preventing non- non- management management personnel personnel from from operating operating management management terminals terminals or configuring or configuring the system. the system. All alarm All alarm

signals signals can only be can only be cleared cleared upon uponconfirmation confirmationbyby thethe manager. manager. AllAll alarm-related alarm-related information information is is

simultaneously sent simultaneously sent to the to the intelligent intelligent terminal terminal device device 34 of 34 theof the manager manager and the centralized and the centralized control control platform 15. platform 15.

[0059] Thetemporary

[0059] The temporaryauthorization authorizationmode modeis is asas follows: follows:

when a temporary worker (e.g., a gas inspector, a patrol inspector, etc.) who is not assigned to the when a temporary worker (e.g., a gas inspector, a patrol inspector, etc.) who is not assigned to the

current current working faceneeds working face needstotoenter enterthetheworking working face, face, temporary temporary access access may may be be granted granted upon upon

approval by the approval by the manager, manager,permitting permittingentry entrywithin withinaa specified specified temporary temporaryduration durationt.tn.

19

[0060] Thetotal

[0060] The total number numberofofpersonnel personnel (including (including temporary temporary workers) workers) entering entering the working the working face face

shall shall not not exceed exceed a a maximum allowable maximum allowable number number N of N a of workers workers allowed allowed for thefor the current current shift shift at theat the workingface. working face. An Anentry timet'𝑡𝑛′ofofthe entrytime thetemporary temporary workers workers into into thethe working working face face is is greaterthan greater thant,tn, the system the issues an system issues an alarm to prompt alarm to the workers prompt the workerstoto evacuate. evacuate.

[0061] Themode

[0061] The modeis is linked linked with with a self-check a self-check function function of wireless of the the wireless positioning positioning terminal terminal 26 26 installed installed on on the theequipment at the equipment at the working face. During working face. the shift During the shift handover mode,each handover mode, eachworker worker forfor

the equipment the at the equipment at the working workingface faceisis required required to to perform perform aa self-check self-check on onthe the wireless wireless positioning positioning terminal 26 terminal 26 on on the the corresponding correspondingequipment. equipment.Only Only afterthethemanager after manager receives receives completion completion signals signals

for for all all equipment self-checks equipment self-checks can can the the system system switchswitch to an operation to an operation mode. mode. When Whentransport auxiliary auxiliary transport vehicles need to enter the working face, drivers are permitted to drive the vehicles into the working vehicles need to enter the working face, drivers are permitted to drive the vehicles into the working

face face upon authorization from upon authorization fromthe the manager. manager.

[0062] Example2.2.Safety

[0062] Example Safetycontrol controlstrategy: strategy:

A safety A safety control controlstrategy strategyofofthe theworking working face face human-machine human-machine collaborative collaborative operation operation safety safety monitoringand monitoring andearly earlywarning warning system system includes includes a normal a normal operation operation control control strategy, strategy, a human- a human-

machinemutual machine mutual avoidance avoidance control control strategy, strategy, a wireless a wireless shutdown shutdown control strategy, control strategy, a post a post monitoringcontrol monitoring control strategy, strategy, and and aa special specialpersonnel personnel management strategy. management strategy.

[0063] After aa worker

[0063] After enters the worker enters the working face, wireless working face, wireless communication communication isisestablished established between betweenthe the wireless positioning wireless positioning terminal terminal 26 arrangedon 26 arranged onaamobile mobilesingle singleequipment equipment unitatatthe unit theworking working face face

and theportable and the portable positioning positioning terminal terminal 31 (such 31 (such as an identification as an identification card, an card, an intelligent intelligent mine lamp,mine lamp,

or otherpositioning or other positioning terminal terminal devices) devices) worn worn by the by the worker, worker, so as totheacquire so as to acquire relativethe relative positions positions

and orientations and orientations between the worker between the andequipment worker and equipmentand andbetween between thethe equipment equipment and and the the equipment equipment

in in real real time. time.

[0064] The positioning

[0064] The positioning technology technology may mayadopt adopt UWBUWB high-precision high-precision wireless wireless positioning positioning

technologyororother technology otherhigh-precision high-precisionwireless wirelesspositioning positioningtechnologies. technologies. TheThe portable portable positioning positioning

terminal 31 has a sleep function; when no effective communication is established with the wireless terminal 31 has a sleep function; when no effective communication is established with the wireless

positioning terminal positioning terminal 26 26 for for aa long long time, time, the the portable portable positioning positioning terminal terminal 31 31 enters enters aasleep sleepmode mode

to extend the service life thereof. to extend the service life thereof.

[0065] Whena aworker

[0065] When workerenters entersananalarm alarmand andshutdown shutdown range range of of thetheequipment equipment in in operation, operation, anan alarm alarm

device of the equipment issues alarm signals such as voice and light alarms to alert the worker. A device of the equipment issues alarm signals such as voice and light alarms to alert the worker. A

control control signal signal is is output output to to control control the the equipment toavoid equipment to avoidhazards hazardsactively, actively,thereby therebypreventing preventing hazardousincidents. hazardous incidents.The Theportable portable positioning positioning terminal terminal 31 emits 31 emits alarms, alarms, vibrations, vibrations, or other or other

20

promptstotoalert prompts alert the the worker. worker. The Thecontrol controlsignals signalscan canprecisely preciselycontrol controlmechanical mechanical structures structures of of

different functionalparts different functional partsofofthethe equipment, equipment, such such as a cutting as a cutting part part and and a traveling a traveling part. part.

[0066] Theintelligent

[0066] The intelligent visual visual analysis analysis system 22 deployed system 22 deployedononthe theequipment equipment continuously continuously tracks tracks

pedestrians and pedestrians and workers workersdynamically dynamically and and performs performs real-time real-time behavior behavior trajectory trajectory analysis analysis and and prediction. This enables real-time safety management and control of personnel at the working face. prediction. This enables real-time safety management and control of personnel at the working face.

Trajectory information, Trajectory information, alarm alarmevents, events,andand other other related related information information are are stored stored locally locally on on the the current current equipment. When equipment. When communication communication is established is established between between the centralized the centralized control control platform platform

15 andthe 15 and theequipment equipmentat at thethe working working face, face, the the system system uploads uploads data data of the of the equipment equipment to the to the

centralized controlplatform centralized control platform 15 the 15 via via data the data transmission transmission device device 23 23 for facilitating for storage, storage, facilitating future future traceability. traceability.

[0067] As shown

[0067] As shownininFIG. FIG.4,4,the thenormal normaloperation operationcontrol controlstrategy strategyincludes: includes:

Thewireless The wirelesspositioning positioningterminal terminal 26 26 on equipment on the the equipment performs performs ranging ranging with the with the portable portable positioning terminal positioning terminal 31 31 and the wireless and the wireless positioning positioning terminals terminals 26 26 on on other other equipment via wireless equipment via wireless communication technology. communication technology. The wireless The wireless positioning positioning terminal terminal 26 26 is is capable of capable obtainingof a obtaining distance, a distance,

orientation, andrelative orientation, and relativepositions positions with with thethe portable portable positioning positioning terminals terminals 31 the 31 around around the equipment. equipment.

The wireless positioning terminal 26 may be configured with an electronic fence, i.e., a protection The wireless positioning terminal 26 may be configured with an electronic fence, i.e., a protection

range of range of the the equipment. When equipment. When thethe portable portable positioning positioning terminal terminal 31 31 enters enters differentranges different rangesofofthe the wireless positioning terminal 26, the wireless positioning terminal outputs different control signals. wireless positioning terminal 26, the wireless positioning terminal outputs different control signals.

[0068] Thesystem

[0068] The systemcan canachieve achieve adaptive adaptive adjustment adjustment of of thethe protection protection range range by by acquiring acquiring data data on on

dynamic changes dynamic changes of of various various mechanical mechanical structures structures of the of the equipment, equipment, including including a conveyor a conveyor tail tail

section. section. For For example, bycollecting example, by collecting angle anglevariation variation information informationofofthe theconveyor conveyortail tailsection, section, the the size of the size of theelectronic electronicfence fence is is adjusted adjusted in response in response to swing to swing motion motion of of section. the tail the tail As section. shown As shown

in in FIG. 9,ananinner FIG. 9, innerboxbox illustrates illustrates a contour a contour of the of the equipment, equipment, while while an area an area the between between the inner box inner box

and anouter and an outerboxbox represents represents the adaptive the adaptive electronic electronic fence outside fence outside theofcontour the contour of the It the equipment. equipment. It can beseen can be seenthat thatthetherange range of of thethe electronic electronic fence fence changes changes correspondingly correspondingly withvariation with an angle an angle variation of the tail of the tail section. section.

[0069] Thewireless

[0069] The wireless positioning positioning terminal terminal 26 26 classifies classifies personnel personnel at at the theworking working face face into into workers workers

(including (including workers onall workers on all equipment, suchasas drivers equipment, such drivers and and operators operators of of the the equipment), pedestrians equipment), pedestrians

(i.e., (i.e.,all workers all workersnot not associated associated with with the the current current equipment), andspecial equipment), and specialpersonnel personnel(including (including maintenance personnel, maintenance personnel, cable cable dragging dragging personnel, personnel, and and other other special special personnel) personnel) based on based on

parameters. Different parameters. Different categories categories of of personnel personnel are are managed managed andand controlled controlled according according to different to different

strategies. strategies.

21

[0070] Thewireless

[0070] The wirelesspositioning positioningterminal terminal26 26isis configured configuredtoto set set working rangesfor working ranges forthe the workers. workers. Different ranges Different of the ranges of the workers are divided workers are divided into into different different areas areas such such as as aa working area, an working area, an alarm alarm

area, area, aa shutdown area,and shutdown area, anda asafety safetyarea. area. The Theworkers workersareare in in differentstatuses different statusescorrespondingly correspondingly within different within different working ranges. The working ranges. The workers workersare areinin an an on-duty on-dutystatus status within within the the working workingarea, area, in in an off-dutystatus an off-duty statuswithin within thethe shutdown shutdown area, area, and inand in a safety a safety status the status within within thearea. safety safety area.

[0071] Thewireless

[0071] The wireless positioning positioning terminal terminal 26 26shares shares data data with with other other wireless wireless positioning positioning terminals terminals 26 on 26 onsurrounding surroundingequipment equipment and and wireless wireless positioning positioning base base stations stations via data via the the data transmission transmission

device 23, device 23, enabling enablingreal-time real-timeacquisition acquisitionofofworking working statesofofthetheworkers states workers from from eacheach wireless wireless

positioning terminal positioning terminal 26. 26. The Thewireless wirelesspositioning positioningterminal terminal2626isislinked linkedtotoananalarm alarmsystem system andand

outputs various voice outputs various voice and andvisual visualalarm alarmsignals signalstotoalert alert the the workers workersbased based on on differentworking different working statuses of the statuses of thesystem. system.TheThe control control signals signals from from the wireless the wireless positioning positioning terminal terminal 26 can precisely 26 can precisely

control mechanical control mechanical structures structures with with different different functions, functions, such assuch as a cutting a cutting part, a chain part, a scraper scraper chain part, part,

a conveyor tail section, a traveling part, and a shovel plate, thereby achieving accurate control of a conveyor tail section, a traveling part, and a shovel plate, thereby achieving accurate control of

the equipment. the equipment.

[0072] Theportable

[0072] The portablepositioning positioningterminal terminal3131ofofthe thesame same worker worker is configured is configured to correspond to correspond to to

workers linked with different wireless positioning terminals 26. workers linked with different wireless positioning terminals 26.

[0073] Theintelligent

[0073] The intelligent visual visual analysis analysissystem system 22 22 on the equipment on the monitorsoperating equipment monitors operatingconditions conditions aroundthe around the equipment equipmentininreal realtime timeand andisiscapable capableofofperforming performing ranging, ranging, behavior behavior analysis, analysis, andand

behavior trajectory behavior trajectory prediction prediction ononpersonnel personnel around around the the equipment. equipment. When personnel When personnel perform perform violation operations violation operations or or unsafe unsafe behaviors, the system behaviors, the outputscorresponding system outputs correspondingalarm alarm andand shutdown shutdown

signals. signals.

[0074] A behavior

[0074] A behavioranalysis analysis camera cameraofofthe thedriving drivingbehavior behavioranalysis analysisdevice device2525monitors monitorsabnormal abnormal driving behaviors driving of the behaviors of the driver. driver.When the behavior When the behavioranalysis analysis camera cameradetects detectsthat that the the head head or or other other limbs ofthe limbs of thedriver driverextend extend outout of the of the cab cab or a or a designated designated range, range, the system the system is of is capable capable of outputting outputting

alarm and alarm and control control signals. signals. When the behavior When the behavioranalysis analysis camera cameradetects detectsabnormal abnormal drivingbehaviors driving behaviors of the of the driver driver such such as as fatigued fatigued driving driving or or yawning, the system yawning, the systemis is capable capable of of outputting outputting alarm alarmand and control signals. control signals.

[0075] Asshown

[0075] As shownin in FIG. FIG. 5, 5, when when a worker a worker not not belonging belonging to current to the the current equipment equipment (hereinafter (hereinafter

collectively referred collectively referred to to as as aa pedestrian) pedestrian) temporarily temporarily passes throughthe passes through theprotection protectionrange rangeofofthe the equipment, thepersonnel equipment, the personnelprotection protectionfunction functionofofthe theequipment equipmentis is disabled.The disabled. The human-machine human-machine

mutualavoidance mutual avoidancecontrol controlstrategy strategyincludes: includes:

22

Whena apedestrian When pedestrianneeds needstotoenter enterthe theequipment's equipment’s safetyprotection safety protectionzone, zone,a arequest requestsignal signalisis sent sent to the wireless positioning terminal 26 via the identification card, miner’s lamp, or the button of to the wireless positioning terminal 26 via the identification card, miner's lamp, or the button of

another positioning another positioning terminal terminaldevice devicetotoalert alertthe theworkers workers(or(oroperators, operators,hereinafter hereinaftercollectively collectively referred to as drivers). Upon receiving a passage request signal, the driver presses a confirmation referred to as drivers). Upon receiving a passage request signal, the driver presses a confirmation

button to button to disable disable the thepersonnel personnel protection protection function. function.The The system system then then stops stops all allmotion motion mechanisms mechanisms

of the of the equipment equipmentthat thatmaymay cause cause injury, injury, allowing allowing the pedestrian the pedestrian to through. to pass pass through. After After the the confirmation button confirmation button is is pressed, pressed, timing timing begins. begins. When tr When tr ≤tr, tR, the the human-machine mutual human-machine mutual avoidance avoidance

function is started. function is started.When tr >> ttr, When tr R, the thehuman-machine mutualavoidance human-machine mutual avoidance function function is is disabled,andand disabled,

the personnel the personnel protection protection function function is isresumed. resumed. The The system outputs corresponding system outputs correspondingalarm alarmand andcontrol control signals signals according to the according to the area area where the personnel where the personnelare arelocated. located. When Whenthethe system system detects detects that that thethe

pedestrian leaves pedestrian leaves the the protection protection range range of of the the equipment, equipment,the thehuman-machine human-machine mutual mutual avoidance avoidance

function function isisautomatically automatically canceled. canceled. If pedestrian If the the pedestrian needs needs to enterto enter the theprotection safety safety protection range of range of the equipment the again, the equipment again, the above abovesteps steps are are repeated. repeated.

[0076] To improve

[0076] To improvethe theproduction productionefficiency efficiencyofofthe theequipment, equipment,the thesystem systemcontinuously continuously monitors monitors

relative positions relative positions between the equipment between the equipmentand andthethepedestrian. pedestrian.Based Basedon on thethe relativeposition, relative position,the the system determinesa aspecific system determines specificorientation orientationofof the the pedestrian pedestrianaround aroundthe theequipment equipment andand controls controls a a

portion of portion of motion motionmechanisms mechanisms of equipment of the the equipment to stoptooperation stop operation accordingly accordingly based onbased the on the orientation ofthe orientation of thepedestrian. pedestrian.

[0077] For example,

[0077] For example,after after aa pedestrian pedestrian passes passes through throughthe theprotection protection range rangeofofthe the equipment equipmentand and activates activates the thehuman-machine mutual human-machine mutual avoidance avoidance mode, mode, the the wireless wireless positioning positioning terminal terminal 26 26 obtains obtains

the relative the relativepositions positionsbetween between the the pedestrian pedestrian and and the the equipment. equipment. The systemthen The system thencontrols controlsrelated related mechanicalmechanisms mechanical mechanismsof of thethe equipment equipment to stop to stop operation operation based based on on a movement a movement trajectory trajectory of the of the

pedestrian. When pedestrian. thepedestrian When the pedestrianpasses passes by by oneone sideside of the of the equipment, equipment, the system the system controls controls the the mechanical structures on a pedestrian’s side to stop operation while mechanical structures in other mechanical structures on a pedestrian's side to stop operation while mechanical structures in other

orientations of orientations of the theequipment are capable equipment are capable of of operating operating normally. normally.

[0078] Uponreceiving

[0078] Upon receivingthe thepassage passagerequest requestsignal, signal, the the driver driver determines whetherconditions determines whether conditionsallow allow the pedestrian to pass based on the operational status of the equipment. If the conditions for passage the pedestrian to pass based on the operational status of the equipment. If the conditions for passage

are are not not met, met, the thedriver driverdoes doesnot notneed needtoto take anyany take action., andand action., the the human-machine human-machinemutual mutual avoidance avoidance

function failstotobebeactivated. function fails activated.

[0079] Thefunction

[0079] The functionis is canceled canceled upon uponmeeting meetinganyany of of thefollowing the following conditions: conditions: 1.1.The The pedestrian pedestrian

leaves the protection leaves the protection range range of of the the equipment; equipment;2.2.When Whenthe the elapsed elapsed timetime exceeds exceeds tr, the tr, the system system

automatically deactivates the automatically deactivates thefunction. function.Specifically, Specifically, the thefunction functionisistime-limited time-limitedandand will will be be

23

automatically deactivated automatically deactivated after after remaining remaining activeactive for a duration for a duration tr. Ifpedestrian tr. If the the pedestrian staysthe stays within within the protection range protection range beyond beyondthetheallowed allowed time, time, the the system system will will cancel cancel the human-machine the human-machine mutual mutual avoidance functionfor avoidance function for the the pedestrian pedestrian and andoutput outputcorresponding corresponding alarm alarm andand control control signals signals based based

on thearea on the areawhere wherethethe pedestrian pedestrian is located. is located. Forpedestrian, For each each pedestrian, tr is calculated tr is calculated from the from momentthe moment

the human-machine the mutual human-machine mutual avoidance avoidance function function is activated. is activated.

[0080] Thewireless

[0080] The wireless shutdown shutdowncontrol controlstrategy strategyincludes: includes:

In In the the event of an event of an emergency, emergency,a aworker worker located located outside outside a protection a protection area area of of thethe equipment equipment can can

activate activate aa wireless wirelessshutdown shutdown function function by pressing by pressing a button a button on the portable on the portable positioning positioning terminal 31, terminal 31,

whichsends which sendsaashutdown shutdowncontrol controlsignal signaltotothe the wireless wireless positioning positioning terminal terminal 26. 26. Upon receiving the Upon receiving the signal, signal, the the wireless wirelesspositioning positioningterminal terminal26 26 outputs outputs aa control controlsignal signaltotoshut shutdown down the the equipment, equipment,

thereby preventing injury accidents. thereby preventing injury accidents.

[0081] For example,

[0081] For example,when when a worker a worker performs performs operations operations at the at the working working face, face, a distance a distance between between

the worker the andnearby worker and nearbymoving movingor or operating operating equipment equipment is less is less than than a safetydistance; a safety distance;ororwhen whenthethe

workerperforms worker performsoperations operationsinina aconfined confinedworking working space space andand is exposed is exposed to atosafety a safety hazard hazard while while

an an equipment operatorfails equipment operator fails to to notice noticethe theworker worker in inhazard, hazard,other othernearby nearbyworkers workers can can continuously continuously

press an press an emergency stopbutton emergency stop buttonononthe theportable portablepositioning positioningterminal terminal31 31to to send send aa shutdown shutdownsignal signal to stop to stop the the equipment. equipment.

[0082] Thewireless

[0082] The wirelessshutdown shutdown function function hashighest has the the highest controlcontrol priority. priority. After After the the portable portable

positioning terminal 31 issues a wireless emergency stop signal, all wireless positioning terminals positioning terminal 31 issues a wireless emergency stop signal, all wireless positioning terminals

26 within 26 within aa signal signal coverage coverage range immediatelyoutput range immediately outputcontrol controlsignals signals to to stop stop the the equipment. Once equipment. Once

the wireless the wireless shutdown shutdownfunction functionis istriggered, triggered,the thefunction functionmust must be be deactivated deactivated by abydesignated a designated manager(for manager (forexample, example,a aworking working facemanager, face manager, a shiftleader, a shift leader, or or authorized authorized drivers drivers and operators and operators

for for the the equipment). equipment).

[0083] A time

[0083] A time interval interval between activation and between activation anddeactivation deactivation of of the the function function must must be begreater greater than than aa preset shutdown preset shutdownduration duration tws.After tws. Afterthetheequipment equipment is shut is shut down, down, the manager the manager (or authorized (or authorized

operators) operators) must inspect the must inspect the operating conditions of operating conditions of the the equipment andthe equipment and thesurrounding surrounding operating operating

environment. Theequipment environment. The equipmentcancan only only be be restartedafter restarted afterconfirming confirmingthe theequipment equipmentis issafe. safe.In In order order to prevent to prevent the the operator operator from fromdirectly directlyrestarting restarting the the equipment equipment without without inspection inspection and and causing causing

hazardous situations. The shutdown function is deactivated after the portable positioning terminal hazardous situations. The shutdown function is deactivated after the portable positioning terminal

31 of the 31 of the manager (orananauthorized manager (or authorizedoperator) operator)moves moves around around the the equipment equipment and remains and remains nearbynearby

for a duration for a longer duration longer than than a specified a specified time. time.

24

[0084] Asshown

[0084] As shownin in FIG. FIG. 6, the 6, the specific specific control control process process of the of the wireless wireless shutdown shutdown function function is is described described asas follows: follows:

Whena apedestrian When pedestriandetects detectsa adangerous dangerous situation,the situation, thepedestrian pedestrianpresses pressesaa combination combinationbutton buttononon the portable the portable positioning positioningterminal terminal31. 31.TheThe portable portable positioning positioning terminal terminal 31 sends 31 sends a wireless a wireless

shutdown command; shutdown command; the the wireless wireless positioning positioning terminals terminals 26 26 of equipment of equipment within within a signal a signal coverage coverage

of the of the portable portable positioning positioning terminal terminal 31 31 receive receive the thewireless wirelessshutdown command shutdown command andand immediately immediately

output control signals output control signals to to stop stop the the equipment. Simultaneously,the equipment. Simultaneously, thesystem systemstarts startstiming timingtwq. twq. After After the equipment the stopsoperating, equipment stops operating, the the driver driver inspects inspects the theoperating operatingconditions conditionsaround around the the equipment. equipment.

Whentwq When twq ≤tws, tws, the the wireless wireless positioning positioning terminal terminal 26 26 continuously outputs shutdown continuously outputs signalstoto stop shutdown signals stop the equipment. When tws, twq ≤the the equipment. When twq twswireless , the wireless positioning positioning terminal terminal 26 allows 26 theallows thedeactivate driver to driver to deactivate the wireless the wireless emergency emergency stop stop function function and stops and stops outputting outputting the shutdown the shutdown signal. signal. When the When the operating conditionsaround operating conditions aroundthetheequipment equipment meet meet the normal the normal operating operating conditions, conditions, the driver the driver

deactivates deactivates the the wireless wireless emergency stop function emergency stop function and andrestarts restarts the the equipment. equipment.

[0085] Thepost

[0085] The post monitoring monitoringcontrol controlstrategy strategy includes: includes:

When thedriver When the driverononthe theequipment equipment operates operates thethe equipment equipment for for production production operations, operations, the the system system

monitors the monitors the positions positions of of personnel, personnel, thereby therebyenabling enablingthe themonitoring monitoringof of absence absence from from the the post, post,

excessive personnel excessive personnel at aatpost, a post, and and driving driving behaviors behaviors of the of the driver, driver, to ensure to ensure theofsafety the safety of the driver the driver

and the equipment and the duringproduction equipment during productionoperations. operations.

[0086] When

[0086] When thethe driver driver operates operates the the equipment, equipment, the wireless the wireless positioning positioning terminal terminal 26 on the 26 on the

equipment outputscorresponding equipment outputs corresponding controlsignals control signalsaccording accordingtotothe theposition position of of the the driver. driver.When the When the

driver is within driver is withinthe theworking working range range orsafety or the the safety area, area, the the system system does not does outputnot an output an alarm signal; alarm signal;

whenthe when thedriver driverenters entersthe thealarm alarmrange rangesetsetbybythethewireless wirelesspositioning positioning terminal terminal 26,26, thethe system system

outputs an alarm outputs an alarm signal; signal; when thedriver when the driver enters enters aa hazardous area, the hazardous area, the system systemoutputs outputsaashutdown shutdown signal. signal.

[0087] Whenthethenumber

[0087] When numberof of on-duty on-duty drivers drivers exceeds exceeds the the number number of operators of operators allowed allowed to work to work on on the equipment, the the system equipment, the systemoutputs outputscorresponding correspondingalarm alarm and and control control signals. signals.

For example, For example,only onlyone onedriver driverisisallowed allowedtotobebeononduty dutywithin withinthe theworking working range range of of a shuttlecar. a shuttle car. When two or more workers are detected within the working range, the system fails to start normally. When two or more workers are detected within the working range, the system fails to start normally.

Whenthethedriver When driveroperates operatesthe theequipment equipment andand enters enters the the protection protection range range of other of other equipment equipment for for collaborative collaborative operations, operations,the theequipment equipment continues continues to tooperate operatenormally, normally,and andthe thealarm alarmand andshutdown shutdown

functions of aapersonnel functions of personnelpositioning positioning system system are triggered. are not not triggered. When When the leaves the driver driver the leaves the

25

equipment andenters equipment and entersthetheprotection protectionrange range of of other other equipment, equipment, the the personnel personnel safety safety protection protection

function function isisactivated activatedtotoensure ensure personnel personnel safety. safety.

[0088] Whenspecial

[0088] When specialpersonnel personnel(such (suchasasbolting boltingpersonnel personnelof of aa heading andbolting heading and bolting machine) machine)leave leave the equipment, the whenother equipment, when otherpersonnel personnelare arestill still operating operating the the equipment andthe equipment and the equipment equipmentisisnot not in in aa moving state,thethe moving state, system system outputs outputs an alarm an alarm signal signal but doesbut notdoes nota output output a control control signal signal to stop the to stop the

equipment. equipment.

[0089]

[0089] When bolting personnel When bolting personnel leave leave working workingpositions positions thereof thereof and moveaway and move awayfrom from thethe

equipment, the equipment equipment, the equipmentoutputs outputsa acontrol controlsignal signaltotostop stopmoving moving and and outputs outputs an an alarm alarm signal signal to to

alert alert other other personnel. Whenthethe personnel. When personnel personnel approach approach the equipment, the equipment, the equipment the equipment outputs outputs a a corresponding alarmsignal. corresponding alarm signal.AtAtthethesame same time, time, the the system system controls controls the motion the motion mechanisms mechanisms at at corresponding positions of corresponding positions of the the equipment to stop equipment to stop actions actions based on an based on an orientation orientation of of the theequipment equipment

whenpersonnel when personnelintrude intrudeinto intothe the working workingface. face.

[0090] Thespecial

[0090] The special personnel personnel management management strategy strategy includes: includes:

Special personnelrefer Special personnel refer to to workers workersperforming performing special special operations operations at at thethe working working face, face, suchsuch as as

maintenancepersonnel maintenance personneland andworkers workers forfor auxiliaryoperations auxiliary operations(e.g., (e.g., those those dragging draggingcables). cables).

[0091] Maintenancepersonnel:

[0091] Maintenance personnel:The The system system exempts exempts maintenance maintenance personnel personnel from alarms from alarms during during a a maintenanceperiod. maintenance period.DueDue to operation to the the operation particularity particularity of theofmaintenance the maintenance personnel, personnel, the the maintenancepersonnel maintenance personnelneed need to to frequentlyadjust frequently adjustand andoperate operateequipment. equipment. During During the the maintenance maintenance

process, the process, the system systemdisables disables alarms alarms for for maintenance maintenance personnel. personnel. After After the the enters system systema enters a maintenancemode, maintenance mode, themaintenance the maintenance personnel personnel areare capable capable of of performing performing maintenance maintenance operations. operations.

However, when the system detects that an operating space is smaller than a safety space or distance, However, when the system detects that an operating space is smaller than a safety space or distance,

the system the issues aa control system issues control signal signal to to stop stopthe themotion motion mechanisms mechanisms ofofthe theequipment equipmentto to ensure ensure thethe

operational safety operational safety of of the themaintenance personnel. maintenance personnel.

[0092] Thecontrol

[0092] The control process process for for the the maintenance personnelisisdescribed maintenance personnel describedasas follows: follows:

At the At the start start ofofmaintenance operations, the maintenance operations, the manager inputsaamaintenance manager inputs maintenance taskviaviathe task theintelligent intelligent terminal device 34 or in other manners. After receiving relevant information, the system activates terminal device 34 or in other manners. After receiving relevant information, the system activates

the equipment the maintenance equipment maintenance mode. mode. When When the maintenance the maintenance personnel personnel enterprotection enter the the protection range range of of the equipment, the the system equipment, the systemblocks blocksthe thecorresponding correspondingcontrol controlsignals signalsand andsimultaneously simultaneously outputs outputs a a voice prompt voice promptsuch suchasas"Equipment “Equipment under under maintenance” maintenance" or other or other relevant relevant voice voice alerts. alerts. TheThe system system

sets sets aa maintenance duration. When maintenance duration. Whenthethe time time exceeds exceeds the the specified specified duration, duration, thethe system system resumes resumes

the normal the outputofofalarm normal output alarmand andcontrol controlsignals. signals.When When other other specialpersonnel special personnel (including (including drivers drivers

26 and other equipment and other equipmentoperators) operators)enter enterananalarm alarm range range of of thethe equipment, equipment, the the wireless wireless positioning positioning

terminal 26 outputs alarm and control signals. terminal 26 outputs alarm and control signals.

[0093] Duringthe

[0093] During themaintenance maintenance process, process, thevisual the visualanalysis analysissystem system (including (including various various intelligent intelligent

visual analysis visual analysis systems systems2222deployed deployed on the on the working working facethe face and andequipment) the equipment) supervises supervises the the maintenancepersonnel maintenance personnel andand performs performs behavior behavior analysis analysis and prediction. and prediction. When When the maintenance the maintenance

personnel perform personnel performviolation violationoperations operationsor or exhibit exhibit improper improper operational operational behaviors, behaviors, the system the system

outputs outputs an an alarm. alarm.

[0094] Thepositioning

[0094] The positioning system systemand andthe thevideo videosurveillance surveillancesystem systemmonitor monitor in in realtime real timethe therelative relative positions among positions theequipment, among the equipment,a acoal coalwall, wall, and andthe the maintenance maintenancepersonnel personnel toto assessthe assess thesafety safety of of the maintenance the personnel.When maintenance personnel. When the the working working spacespace ofmaintenance of the the maintenance personnel, personnel, that that is, theis, the distance between distance theequipment, between the equipment,thethecoal coalwall, wall,and andthethemaintenance maintenance personnel personnel are are lessless thanthan the the

safety distance,the safety distance, thesystem system shuts shuts down down the equipment. the equipment. The positioning The positioning systemtheincludes system includes wirelessthe wireless

positioning terminal positioning terminal 26, 26, the the portable portable positioning positioning terminal terminal 31, 31, and andthe thewireless wirelesspositioning positioningbase base station. station.The The video video surveillance surveillancesystem system includes includes the theworking working face face video video surveillance surveillance system system 12 12 and and

the camera the deviceinstalled camera device installed at at aaposition positionwhere where the theequipment and the equipment and the workers workersare are located. located.

[0095] Auxiliary operating

[0095] Auxiliary operating personnel: personnel:

Auxiliary operating Auxiliary operating personnel personnelrefer refer to to temporary workersfor temporary workers forcollaborative collaborativeoperations operationsaround aroundthe the equipment during equipment during the the reversal reversal or retreat or retreat operations operations of the of the equipment. equipment.

For example, since a continuous miner lacks a cable reel device, auxiliary operating personnel are For example, since a continuous miner lacks a cable reel device, auxiliary operating personnel are

required to required to assist assist in in dragging dragging a a cable cable during retreat operations during retreat operations to to prevent the cable prevent the frombeing cable from being damaged. damaged.

[0096] When

[0096] When theworking the working face face equipment equipment performs performs retreat retreat operations, operations, otherother auxiliary auxiliary operating operating

personnel are required to cooperate in dragging the cable. At the same time, the positioning system personnel are required to cooperate in dragging the cable. At the same time, the positioning system

and the and the video videosurveillance surveillancesystem system continuously continuously supervise supervise the personnel the personnel dragging dragging the the cable, cable, ensuring the safety ensuring the safety of of the thepersonnel personnel dragging dragging the the cable cable by by means of aa machine means of machinevisual visualrecognition recognition technologyand technology andthe thepositioning positioning system. system.

[0097] Whenthe

[0097] When theworker worker drags drags thecable, the cable,the thevisual visual analysis analysis system performsbehavior system performs behavioranalysis analysisonon the surrounding the workers.When surrounding workers. Whenit it isisdetermined determined thatthe that theworker workeris is dragging dragging thethe cable,the cable, thevisual visual analysis system analysis system identifies identifies andand verifies verifies identity identity information information of theof the worker worker and integrates and integrates the identity the identity

information with information from the positioning system to trigger the auxiliary operation control information with information from the positioning system to trigger the auxiliary operation control

strategy and strategy and monitor operating conditions monitor operating conditions of of the the equipment. When equipment. When theequipment the equipment is is inin a areversing reversing

27 or retreating mode, or retreating the safety mode, the safety protection protection range rangefor forthe theworker workerisisadjusted, adjusted,and andthethepositioning positioning system temporarilyexempts system temporarily exemptsthe theworker workerfrom from equipment equipment shutdown. shutdown. ThatThat is, is, whenwhen the the worker worker enters enters

the protection the protection area area of of the the equipment, the equipment equipment, the equipmentissues issuesananalarm alarmbutbutdoes does notnot output output control control

signals. signals. However, whenthe However, when theworker workerenters entersthe theshutdown shutdown areaofofthe area theequipment equipmentor or isisin inaa hazardous hazardous state, state, the the system outputs system outputs control control signals signals to shut to shut down down the equipment. the equipment.

[0098] Thesystem

[0098] The systemevaluates evaluates thethe safetyofofanan safety operation operation environment environment of auxiliary of the the auxiliary operating operating

personnel for personnel for dragging draggingthe thecable cablebased based on on thethe distance distance and and positioning positioning direction direction between between the the equipment and equipment and thethe personnel personnel and and issues issues corresponding corresponding alarm alarm and and signals control control when signals the when the

auxiliary auxiliary operating operating personnel are within personnel are within different different protection protection ranges ranges of of the the equipment. When equipment. When thethe

auxiliary auxiliary operating personnelare operating personnel arewithin withinthethealarm alarm area, area, thethe wireless wireless positioning positioning terminal terminal 26 26

controls theconveyor controls the conveyor tail tail section section of of thethe equipment equipment to swinging, to stop stop swinging, and the and the equipment equipment can operatecan operate

normally. When the auxiliary operating personnel enter the shutdown area, the wireless positioning normally. When the auxiliary operating personnel enter the shutdown area, the wireless positioning

terminal 26 terminal 26 controls controls an an oil oil pump pumptotostop stopoperating, operating,thereby therebyensuring ensuringthethesafety safetyofofthe theauxiliary auxiliary operating personnel. operating personnel.

[0099] Example3.3.Collaborative

[0099] Example Collaborativeoperations operationsbetween between working working faceface equipment: equipment:

Throughdata Through datasharing, sharing, thethe system system is capable is capable of obtaining of obtaining in real in real time time relative relative positional positional

relationships between relationships between equipment and equipment equipment and equipmentandand operationalstatuses operational statusesofofsurrounding surrounding equipment, therebyenabling equipment, thereby enabling collaborative collaborative control control operations operations among among clusters clusters of working of working face face

equipment. Thisincludes equipment. This includesexemption exemption from from alarms alarms whenwhen equipment equipment approaches approaches each other, each other, linked linked

control control between equipment,etc. between equipment, etc.

[0100] Exemption

[0100] Exemption from from alarms alarms when when equipment equipment approaches approaches eachmeans each other other that means that working working face face equipment needstotoperform equipment needs perform collaborative collaborative operations, operations, andand it is it is necessary necessary to to ensure ensure thatthat during during

collaborative operations and collaborative operations andmutual mutual proximity proximity between between equipment, equipment, drivers drivers approaching approaching the the protection range protection range of of other other equipment do not equipment do not trigger trigger equipment shutdown.ByBymeans equipment shutdown. means of of data data sharing sharing

betweenequipment, between equipment,thethe positioning positioning system system and and the the video video surveillance surveillance system system are combined are combined to to achieve linked achieve linked control control between equipment. between equipment.

[0101] Duringcollaborative

[0101] During collaborative operations operations between between equipment, equipment, the relative the relative positions positions between between

equipment arerelatively equipment are relatively fixed. fixed. For example,aadischarge For example, dischargehopper hopperofofa ashuttle shuttlecar carand anda areceiving receiving hopper of a crusher have a collaborative operation relationship, whereas a receiving hopper of the hopper of a crusher have a collaborative operation relationship, whereas a receiving hopper of the

shuttle car and shuttle car andthe thecrusher crusher do do not not havehave a collaborative a collaborative operation operation relationship. relationship.

28

[0102] 1. When

[0102] 1. equipment When equipment approaches approaches eacheach other, other, the the relative relative positions positions between between equipment equipment are are

analyzed andpredicted analyzed and predictedbased basedonon shared shared data data between between the equipment the equipment to determine to determine whetherwhether the the conditions for conditions for collaborative collaborative operations operations between the equipment between the aremet. equipment are met.

[0103] 2. If

[0103] 2. If the conditions for the conditions for collaborative collaborativeoperations operationsbetween betweenequipment equipment are are met and met and

collaborative collaborative operations operations are are possible, possible,the theequipment equipment approaching eachother approaching each otherwill will be be exempt exemptfrom from alarms, andrelated alarms, and relatedmoving moving components components for collaborative for collaborative operations operations are started are started for linkage for linkage

operations; if the operations; if the relative relative orientations orientations between betweenthethe equipment equipment domeet do not notconditions meet conditions for for collaborative operations collaborative operations and and whenwhen the distance the distance is too is tooand short short and a collision a collision risk the risk arises, arises, the intrusion intrusion

of equipmentinto of equipment intothe theprotection protectionrange range causes causes thethe output output of corresponding of corresponding alarmalarm and control and control

signals. signals.

[0104] 3. The

[0104] 3. Thecollaborative collaborativecontrol controlstrategy strategybetween between equipment equipment is shown is shown in FIG.in7.FIG. FIG. 7. 8 FIG. 8

illustrates illustratesa hazardous a hazardousoperation operationbetween between equipment. Takingthe equipment. Taking the collaborative collaborative operations operations between between the shuttle the shuttle car carand and the theheading heading and and bolting bolting machine as an machine as an example, example,the thecontrol controllogic logic between betweenthe the equipment during equipment during collaborative collaborative operations operations is specifically is specifically described described as follows: as follows:

Whenthetheshuttle When shuttlecarcarandand thethe heading heading and bolting and bolting machine machine approach approach eachthe each other, other, the system system determines whetherthetheconditions determines whether conditions forfor collaborative collaborative operations operations are are met met basedbased onrelative on the the relative positions and positions and orientations orientations between the equipment. between the equipment.

[0105] When

[0105] When the the conditions conditions for collaborative for collaborative operations operations are met, are met, i.e., i.e., the receiving the receiving hopper of the hopper of the

shuttle shuttle car car overlaps overlaps with the discharge with the discharge hopper hopperofofthe theheading headingandand bolting bolting machine, machine, the the system system

controls controls the the relevant relevantmoving components moving components toto realizeequipment realize equipmentlinkage, linkage,automatically automaticallystarting startingand and stopping the relevant stopping the relevant components components totoperform performthe theoperations. operations.

Thesystem The systemuses usesthe theintelligent intelligent visual visual analysis analysissystem system 22 22 and and aa weighing systemtoto analyze weighing system analyzeaa coal coal piling volume, thereby controlling the scraper chain of the shuttle car and the heading and bolting piling volume, thereby controlling the scraper chain of the shuttle car and the heading and bolting

machinetotomove machine moveto to ensure ensure normal normal coal coal unloading unloading operations operations and prevent and prevent excessive excessive coal piling; coal piling;

whenthe when thecoal coalreceiving receivingamount amountof of theshuttle the shuttlecar carreaches reachesa amaximum maximum value, value, the scraper the scraper chains chains

stop running. stop running. The systemissues The system issues an an alarm alarm signal signal to to prompt the driver prompt the driver to tomove move the the equipment away. equipment away.

[0106] When

[0106] When thethe relative relative positions positions andand orientations orientations between between the equipment the equipment do not do notthemeet the meet

conditions for conditions for collaborative collaborative operations, operations, and and aa driving driving direction direction and and aa posture postureofofthe the shuttle shuttle car car approach the continuous approach the continuousminer, miner,the theequipment equipment issues issues corresponding corresponding alarm alarm and and shutdown shutdown signals signals

to initiate hazard avoidance actions, thereby ensuring equipment safety. to initiate hazard avoidance actions, thereby ensuring equipment safety.

29

[0107] Example4.4.Control

[0107] Example Controlstrategy strategyofofaa heading headingand andbolting boltingworking working face: face:

Taking aaheading Taking headingface face of of a heading a heading and and bolting bolting team team as an as an example, example, the control the control strategystrategy is is elaborated elaborated inindetail: detail:

Theheading The headingand andbolting boltingteam teamisisequipped equipped with with a heading a heading andand bolting bolting machine, machine, a shuttle a shuttle car,andand car,

a continuous a haulage vehicle. continuous haulage vehicle. The The heading and bolting heading and bolting machine is operated machine is operated by by 55 workers, workers, including including

one driverandand one driver twotwo bolting bolting workers workers on eachon of each of left left and rightand right sides; thesides; the shuttle carshuttle car isbyoperated by is operated

one driver; the one driver; the continuous continuoushaulage haulage vehicle vehicle is unmanned is unmanned during during crushing crushing and transportation and transportation

operations and is operations and is operated operated by by one driver when one driver traveling. when traveling.

[0108] Theequipment

[0108] The equipmentatatthe theworking workingface faceisisequipped equippedwith withseveral severalwireless wirelesspositioning positioningterminals terminals 26, a camera device, a driving behavior analysis device 25, a multi-source sensor 24, an intelligent 26, a camera device, a driving behavior analysis device 25, a multi-source sensor 24, an intelligent

visual analysis visual analysis system system22, 22,and anda adata datatransmission transmission device device 23. 23. The The system system is connected is connected to an to an electrical electricalcontrol controlbox boxofof thethe equipment equipmentand andcontrols controlsmotion motionmechanisms suchasas an mechanisms such an oil oil pump motor, pump motor,

a conveyor tail section, a cutting motor, and a bolting drill arm, to precisely control the equipment a conveyor tail section, a cutting motor, and a bolting drill arm, to precisely control the equipment

operation. operation. The videocamera The video camera21 21 andand thethe thermal thermal imager imager 27 monitor 27 monitor the operating the operating environment environment

around the equipment. around the equipment. The Theintelligent intelligent visual visual analysis analysis system 22 analyzes system 22 analyzes the the operating operating environment environment ofofthe the equipment, equipment,and, and,inincombination combinationwith withthe thepositioning positioningsystem, system,comprehensively comprehensively determines andoutputs determines and outputs alarm alarm and and control control signals. signals. The multi-source The multi-source sensor sensor 24 monitors 24 monitors the the operational status operational status of of each piece of each piece of equipment, equipment,and andrelevant relevantdata dataofofeach each piece piece of of equipment equipment is is shared viathe shared via thedata datatransmission transmission device device 23. 23.

[0109] Control logic

[0109] Control logic of of each each piece piece of of equipment: equipment:

1. 1. Heading andbolting Heading and bolting machine: machine:the thepositioning positioningbase basestation station of of the the heading headingand andbolting boltingmachine machine monitorsdrivers monitors drivers and andbolting boltingworkers. workers.Only Only oneone driver driver is is allowed allowed to operate to operate at aatdriver's a driver’s post. post.

Whenthe When thenumber numberof of drivers drivers exceeds exceeds a permitted a permitted limit, limit, thethesystem system outputs outputs an an alarm alarm and and controls controls

the equipment the equipment totostop. stop. The Theworking working range range of of thethe bolting bolting workers workers and and an operating an operating range range of of the the drivers aredifferent. drivers are different.The Thesystem system controls controls related related motion motion mechanisms mechanisms to stop to stop within thewithin the alarm area. alarm area.

Whenthe When theequipment equipment enters enters thetheshutdown shutdown area, area, thethe system system controls controls an an oiloil pump pump to shut to shut down down and and stop operation. stop operation.

[0110] Whenthetheheading

[0110] When heading andand bolting bolting machine machine performs performs cutting cutting operations, operations, the driver the driver and the and the

bolting workers must remain at the respective operating posts. If they leave the working area, the bolting workers must remain at the respective operating posts. If they leave the working area, the

equipment sendsalarm equipment sends alarm andand shutdown shutdown signals signals to shut to shut down down the equipment, the equipment, ensuringensuring personnel personnel

safety. safety.

30

[0111]

[0111] When theheading When the headingand andbolting boltingmachine machine performs performs bolting bolting operations, operations, thethe system system only only issues issues

an an alarm but does alarm but does not not shut shut down. down. The headingand The heading andbolting bolting machine machinecan cancontinue continuebolting boltingoperations. operations. Thesystem The systemcontrols controls other other motion motion mechanisms mechanisms (including (including tail swing, tail swing, cutting cutting head, platform head, platform

extension, etc.) to extension, etc.) to stop stop operations. operations. When thebolting When the boltingworkers workers leave leave thethe operating operating range range of the of the

equipment equipment to to prepare prepare materials materials or perform or perform other the other tasks, tasks, theonly system system only issues an issues an alarm alarm signal, and signal, and

bolting operations bolting operations of of the the heading heading and and bolting bolting machine are continued machine are continuednormally. normally.

[0112] 2.Shuttle

[0112] 2. Shuttlecar: car:TheThe positioning positioning base base station station of theof the shuttle shuttle car monitors car monitors the Only the driver. driver. one Only one

driver is allowed driver is within allowed within thethe working working rangerange of theof the driver driver of the of the shuttle shuttle car to car to operate operate the equipment. the equipment.

Whentwo When twoorormore more workers workers areare detectedwithin detected withinthe theworking working range,the range, thesystem systemcannot cannot startnormally. start normally. Whenthethedriver When driverleaves leavesthe thecab, cab,the thesystem systemcontrols controlsthe theequipment equipmentto to stop stop running. running. TheThe driving driving

behavior analysis behavior analysis device device2525analyzes analyzes behaviors behaviors of of thethe driver. driver. When When the driver's the driver's headhead or body or body

extends outside the extends outside the cab cabofofthe theshuttle shuttle car, car, the the system outputsananalarm system outputs alarmsignal signalandand controls controls thethe

equipment toshut equipment to shut down. down.

[0113] 3. Continuous

[0113] 3. haulagevehicle: Continuous haulage vehicle:When When the the continuous continuous haulage haulage vehicle vehicle performs performs crushing crushing

and transportation and transportation operations, operations, the thesystem systemonly only issues issues an alarm an alarm and not and does doesshut notdown shutthe down the equipment. When equipment. When thethe video video surveillance surveillance system system detects detects a person a person entering entering thereceiving the receivinghopper hopper of of

the equipment and stepping onto a belt, the system outputs a control signal to stop the equipment. the equipment and stepping onto a belt, the system outputs a control signal to stop the equipment.

Whenthe When thecontinuous continuous haulage haulage vehicle vehicle moves, moves, the the equipment equipment is operated is operated by driver. by the the driver. TheThe driver driver

post allows only one person to operate the equipment. If the post is overstaffed, the system outputs post allows only one person to operate the equipment. If the post is overstaffed, the system outputs

an alarmsignal an alarm signalandand a control a control signal, signal, preventing preventing the equipment the equipment from from being beingWhen started. started. When the driver the driver

leaves thecab, leaves the cab,the thesystem system controls controls the equipment the equipment to stop to stop running. running.

[0114] Pedestrian

[0114] Pedestrian control control strategy: strategy:

Whena apedestrian When pedestrianintrudes intrudesinto intothe theprotection protection range rangeofofthe the wireless wireless positioning positioning terminal terminal 26, 26, the the system outputs an system outputs an alarm alarmoror shutdown shutdownsignal signaltotocontrol controlthe the equipment equipmenttotostop stoprunning. running.

[0115] Whena apedestrian

[0115] When pedestriantemporarily temporarily passes passes through through thethe protection protection range range of of thethe equipment equipment and and

passes by passes bya atemporary temporary human-machine human-machine mutual mutual avoidance avoidance button onbutton on the positioning the portable portable positioning terminal 31, the portable positioning terminal 31 sends a request signal. After the driver receives terminal 31, the portable positioning terminal 31 sends a request signal. After the driver receives

the signal and presses an enter button, the system activates a temporary mutual avoidance function, the signal and presses an enter button, the system activates a temporary mutual avoidance function,

allowing the allowing the pedestrian pedestrian to to pass pass through the protection through the protection range range of of the the equipment withouttriggering equipment without triggering the equipment the to stop. equipment to stop.

[0116] After the

[0116] After the temporary temporaryman-machine man-machine avoidance avoidance function function is activated is activated by pedestrian, by the the pedestrian, the the

system detects aa direction system detects direction and and position position of of the the pedestrian pedestrian passing passingthrough throughthe theequipment equipmentin in real real

31

time. When time. personnelenter When personnel enterthe theprotection protectionrange rangeof of the the equipment, equipment,the the system systemcontrols controlsthe the motion motion mechanism mechanism at at thecorresponding the corresponding position position of of thethe equipment equipment based based on pedestrian's on the the pedestrian’s movement movement

trajectory totostop trajectory stopmovement andensure movement and ensurepersonal personalsafety. safety.

[0117]

[0117] Wireless shutdownfunction: Wireless shutdown function:When When a pedestrian a pedestrian discovers discovers thata aperson that personisisinin the the hazardous hazardous area area surrounding the equipment., surrounding the equipment.,the thepedestrian pedestrianpresses pressesaa wireless wireless emergency emergency stopbutton stop button on on thethe

portable positioning terminal 31. The system outputs a control signal to stop the equipment. portable positioning terminal 31. The system outputs a control signal to stop the equipment.

[0118] Collaborative control

[0118] Collaborative control strategy strategy between equipment: between equipment:

1. 1. No No alarm whenpieces alarm when piecesofofequipment equipment approach approach each each other: other:

Whentwo When two pieces pieces ofof equipment equipment approach approach eacheach other, other, the the system system doesdoes not not output output an alarm an alarm control control

signal. Forexample, signal. For example, when when a shuttle a shuttle car driver car driver drivesdrives the shuttle the shuttle carthe car into into the protection protection range ofrange the of the headingand heading andbolting boltingmachine, machine,no no alarm alarm or shutdown or shutdown functions functions are triggered are triggered for either for either piecepiece of of equipment equipment bybythe theshuttle shuttle car car driver driver and the driver and the driver of of the the heading heading and bolting machine. and bolting However, machine. However,

when the shuttle car driver exits the car and enters the protection range of the heading and bolting when the shuttle car driver exits the car and enters the protection range of the heading and bolting

machine alone,an machine alone, analarm alarmand andstop stopsignal signalof of the the equipment equipmentisis triggered triggered to to stop stop the the equipment. equipment.

[0119] 2. Heading

[0119] 2. andbolting Heading and boltingmachine machineand andshuttle shuttlecar: car:

Whenthetheheading When heading and and bolting bolting machine machine performs performs cuttingcutting operations, operations, a large aamount large of amount coal of coal accumulates onthe accumulates on the scraper scraper chain chain of of the the heading heading and and bolting bolting machine. machine. When themonitoring When the monitoringsystem system of the heading and bolting machine detects that the receiving hopper of the shuttle car approaches of the heading and bolting machine detects that the receiving hopper of the shuttle car approaches

the conveyor the tail section conveyor tail section of of the the heading headingand andbolting boltingmachine, machine,thethe equipment equipment doesdoes not issue not issue an an alarm. alarm. At the same At the time, when same time, whenthe theconveyor conveyor tailsection tail sectionofofthe theheading headingand andbolting boltingmachine machine andand

the receiving hopper of the shuttle car are connected, the scraper chain starts to unload coal. the receiving hopper of the shuttle car are connected, the scraper chain starts to unload coal.

[0120] The

[0120] The intelligent intelligent visual visual analysis analysis system system 22 installed 22 installed on the on the shuttle shuttle car monitors car monitors and analyzes and analyzes

the coal load capacity in the receiving hopper of the shuttle car in real time, controls the movement the coal load capacity in the receiving hopper of the shuttle car in real time, controls the movement

of of the the scraper chainofofthe scraper chain theshuttle shuttlecar, car,and and completes completes the automatic the automatic coal unloading coal unloading operation. operation. When When the coal the coal load load capacity capacity of of the the shuttle shuttlecar carreaches reaches aamaximum value,the maximum value, thesystem system outputs outputs a control a control

alarm signaltotoalert alarm signal alertthe thedriver. driver.

[0121] 3. Shuttle

[0121] 3. Shuttle car carand and continuous continuous haulage vehicle: haulage vehicle:

When the conveyor tail section of the shuttle car approaches the receiving hopper of the continuous When the conveyor tail section of the shuttle car approaches the receiving hopper of the continuous

haulage vehicle and a traveling direction of the shuttle car meets the conditions for collaborative haulage vehicle and a traveling direction of the shuttle car meets the conditions for collaborative

operations with the operations with the continuous continuoushaulage haulagevehicle, vehicle,the thesystem system does does notnot output output an an alarm alarm signal signal but but

outputs a control outputs a control signal signal to to start starta aconveyor conveyor of of the thecontinuous continuous haulage vehicle. When haulage vehicle. thesystem When the system

32

detects thatthe detects that thecontinuous continuous haulage haulage vehicle vehicle of theof the shuttle shuttle car is car is connected connected to the receiving to the receiving hopper hopper of the continuous of the continuous haulage haulage vehicle, vehicle, the scraper the scraper chain chain of the of the shuttle shuttle car automatically car automatically unloads coal. unloads coal.

[0122] Maintenancepersonnel

[0122] Maintenance personnel management: management:

Whenequipment When equipmentis isunder undermaintenance, maintenance,thetheshift shiftleader leader inputs inputs aa maintenance maintenancetask taskand anda a maintenance duration via the intelligent terminal device 34. After the system receives the relevant maintenance duration via the intelligent terminal device 34. After the system receives the relevant

information, aa maintenance information, maintenancemode modeof of thethe equipment equipment is activated. is activated. At At thistime, this time,when when maintenance maintenance

personnel enter personnel enter the theprotection protectionrange rangeofofthetheequipment, equipment, The The system system exempts exempts the maintenance the maintenance

personnel from personnel fromananalarm. alarm.However, However, when when a driver a driver or or a worker a worker intrudes intrudes into into thethe alarm alarm area area of of thethe

equipment, the wireless equipment, the wireless positioning positioning terminal terminal 26 26 outputs outputs corresponding correspondingalarm alarmand andcontrol controlsignals. signals.

[0123] Whenthe

[0123] When themaintenance maintenance task task is is completed, completed, themanager the manager cancels cancels thethe maintenance maintenance mode, mode, and and

the system the resumesoutputting system resumes outputtingnormal normal alarm alarm andand control control signals.When signals. Whenthe the maintenance maintenance duration duration

exceeds exceeds aaspecified specifiedduration, duration,the thesystem system automatically automatically resumes resumes outputting outputting normal normal alarm and alarm and

control signals.AtAtthis control signals. thistime, time,ititisis necessary necessarytotocontact contact thethe manager manager to re-enter to re-enter the maintenance the maintenance task task and activate and activate the the maintenance mode. maintenance mode.

[0124] Shift handover

[0124] Shift mode: handover mode:

Whenthe When thesystem system is is ininshift shifthandover handovermode, mode, allall equipment equipment cannot cannot be started. be started. TheThe worker worker of of the the equipment runs equipment runs a self-check a self-check function function of theof the system system via the via the intelligent intelligent terminalterminal device device 34. 34. After the After the

self-check function self-check function is is completed, completed, the intelligent the intelligent terminal terminal device device 34 sends 34 sendsinformation relevant relevant information to to aa management management andand control control platform. platform. When When the manager the manager receives receives completion completion signalssignals of the of the self- self-

check functions of check functions of all all equipment, the system equipment, the enters an system enters an operating operating mode, andthe mode, and theequipment equipmentisisable able to start operation. to start operation.

[0125] Example5:5:Worker

[0125] Example Worker statusmonitoring: status monitoring:

A gas A gas concentration concentrationsensors sensorssuch suchasasgas gasconcentration concentrationsensors sensorsand andoxygen oxygen concentration concentration sensors, sensors,

roof displacement roof monitors, etc. displacement monitors, etc. for forthe theworking working face faceare areused usedtoto monitor monitorthe working the workingenvironment environment

at at the the working faceininreal working face real time. time. When When a working a working face face environment environment becomes becomes unfavorable unfavorable for for personnel operations, personnel operations,such suchas aswhen when a gas concentration a gas concentration exceeds exceeds aa limit, limit,when when an oxygen an oxygen

concentration decreases, ororwhen concentration decreases, when a roof a roof displacement displacement exceeds exceeds a specified a specified value,value, indicating indicating a a change in the change in the working working environment, environment, the the system systemwill will emit emitananalarm alarmtone. tone.Relevant Relevantalarm alarm information ofthe information of thesystem systemis isshared shared viavia a communication a communication network network to the to the intelligent intelligent terminal terminal

devices 34 used devices 34 usedbybyworkers. workers.Upon Upon receiving receiving thethe information, information, thethe intelligentterminal intelligent terminaldevices devices3434 emit an emit an alarm, alarm, notifying notifying workers to evacuate workers to evacuatethe the working workingface faceimmediately. immediately.The The system system is is linked linked

33

with the with the turnstile turnstile 14 14 at at the the working face. When working face. thesystem When the system issues issues an an alarm, alarm, allall channels channels of of thethe

turnstile 14 at the working face are opened; at the same time, the system outputs a shutdown signal turnstile 14 at the working face are opened; at the same time, the system outputs a shutdown signal

to stop all equipment at the working face. to stop all equipment at the working face.

[0126] Example6:6:Health

[0126] Example HealthStatus StatusMonitoring: Monitoring:

The smart wristband 35 monitors vital signs of the workers in real time, including heart rate, blood The smart wristband 35 monitors vital signs of the workers in real time, including heart rate, blood

pressure, and pressure, blood oxygen and blood oxygensaturation saturationindex, index,totoanalyze analyzehealth healthindexes indexesandand fatiguelevels fatigue levelsofofthe the workers. When workers. Whenthethevital vital signs signs of of the the workers exceednormal workers exceed normalranges, ranges,ororwhen when thehealth the healthindexes indexesofof the workers the decreaseand workers decrease andthe theworkers workers exhibitsigns exhibit signsofoffatigue, fatigue,the the intelligent intelligent terminal terminal device device 34 34

issues analarm issues an alarmto toremind remind the the workers workers to payto pay attention attention to theirtophysical their physical conditions. conditions.

[0127] After the

[0127] After the equipment issuesan equipment issues analarm, alarm,the the worker workermust muststop stopworking. working. The The worker worker shall shall rest rest

and recoverin ina safety and recover a safety area area thatthat ensures ensures personal personal safety. safety. Onlyresting Only after after resting for more for more than than 20 min, 20 min,

as as recorded bythe recorded by thesmart smartwristband wristband35,35, maymay the the worker worker resume resume work. work. If the If the worker worker continues continues

working without rest, the intelligent terminal device 34 will continue to issue alarm signals. If the working without rest, the intelligent terminal device 34 will continue to issue alarm signals. If the

alarm lasts for alarm lasts for more than 10 more than 10min, min,the thesystem systemwill willpush push thealarm the alarm instructioninformation instruction information to to thethe

intelligent intelligentterminal terminaldevice device34 34of ofthe themanager. manager.When the vital When the vital signs signs abnormally exceedthe abnormally exceed the normal normal ranges, it indicates that vital sign indicators of the workers reach a dangerous range, the system ranges, it indicates that vital sign indicators of the workers reach a dangerous range, the system

continuously pushes the relevant information of the worker to the intelligent terminal device 34 of continuously pushes the relevant information of the worker to the intelligent terminal device 34 of

the manager. the manager.

[0128] Themotion

[0128] The motionmonitoring monitoring device device 3636 uses uses motion motion sensors sensors (MEMS (MEMS sensors sensors or other or other sensors) sensors) to to

measuremotion measure motionpostures posturesofofthe theworker worker and and detect detect information information such such as as gaitandand gait abnormal abnormal motion motion

postures of postures of the the body. body. When When a aworker worker exhibits exhibits abnormal abnormal postures postures suchsuch as falling as falling or or high-altitude high-altitude

drops, the system drops, the issues the system issues the corresponding correspondingalarm alarm instructioninformation instruction information andand pushes pushes the the alarm alarm

instruction information instruction information to the to the intelligent intelligent terminal terminal device device 34 of 34 the of the manager. manager.

[0129] Afterthe

[0129] After theintelligent intelligentterminal terminaldevice device 34 34 of manager of the the manager receives receives continuous continuous alarm alarm instruction instruction information from the information from the system, system,the themanager manager checks checks physical physical conditions conditions of the of the worker. worker.

Thesystem The systemstops stopsthe thealarm alarmonly onlywhen when thedistance the distancebetween between thethe manager manager and and the the worker worker satisfies satisfies

the conditions: the conditions: L L1 ≥ 0.5m m 0.5 andand T =T3=min, 3 min, where where L isLthe 1 is distance the distance between between the manager the manager and and the the workerand worker andTTisis aa duration duration when thedistance when the distance between betweenthe themanager managerandand thethe worker worker is is lessthan less than0.5 0.5 m. The m. managertakes The manager takescorresponding corresponding measures measures based based on the on the physical physical conditions conditions of of thethe worker worker andand

handling suggestions handling suggestionsof of the the system. system.

[0130] Therelevant

[0130] The relevantalarm alarm instruction instruction information information ofsystem of the the system is with is thewith the underground underground

centralized control centralized control platform platform1515 andand a ground a ground dispatching dispatching platformplatform through data through wireless wireless data

34

transmission. AAdispatcher transmission. dispatcherinitiates initiatesrelated relatedemergency emergency response response plans plans based based on on the the alarm alarm instruction instruction information fromthethe information from centralized centralized control control platform platform 15 (e.g., 15 (e.g., dispatching dispatching medical medical

personnel for on-site rescue, contacting local hospitals, etc.), thereby ensuring the safety of the personnel for on-site rescue, contacting local hospitals, etc.), thereby ensuring the safety of the

workers. workers.

[0131]

[0131] An imagerecognition An image recognitionmethod method forthe for theworking working facehuman-machine face human-machine collaborative collaborative operation operation

safety safety monitoring andearly monitoring and earlywarning warning system system according according to the to the present present disclosure disclosure adopts adopts a deep a deep

learning-based object learning-based object recognition recognition technology technology to segment, to segment, process, process, and intelligently and intelligently analyze images analyze images

captured by cameras, captured by cameras,thereby therebyacquiring acquiringthethemotion motion postures, postures, identityinformation, identity information, andand behavior behavior

trajectories ofofworkers. trajectories workers. The The method further enables method further enableslinkage linkagecontrol controlwith witha apositioning positioningsystem, system,a a centralized centralized control control platform platform 15, 15, an an entry entryand and exit exitmanagement device,and management device, andthe thelike. like. However, due However, due

to the to the high highdust dustlevels levelsatatthetheheading heading face, face, the the lacklack of environmental of environmental background background texture texture information, extensiveuse information, extensive useofofpersonal personal protective protective equipment equipment by personnel, by personnel, and and the thethat fact fact that workers carry tools for operating, facial features and body posture features are often indistinct, workers carry tools for operating, facial features and body posture features are often indistinct,

leading to leading to poor poorimage imagerecognition recognition effect.ToTo effect. solve solve thethe problem, problem, an improved an improved YOLOV5YOLOV5 deep deep learning algorithm isis adopted learning algorithm adoptedtotoachieve achievepersonnel personnel identificationat atthetheheading identification heading face, face, thereby thereby

enabling identity authentication, enabling identity authentication, behavior behavior and posture recognition, and posture recognition, target target tracking, tracking, and and behavior behavior

trajectory prediction, trajectory prediction, ultimately ultimately realizing realizing supervision over personnel supervision over personnelatatthe theworking working face. face. By By

improving improving a atraditional traditional YOLOV5 YOLOV5 algorithm, algorithm, the generalization the generalization capability capability of theofalgorithm the algorithm is is enhanced, therebyimproving enhanced, thereby improvingthetheaccuracy accuracyofof human human target target recognition recognition by by thethe algorithm. algorithm.

[0132] The image

[0132] The imagerecognition recognition method methodfor forthe theworking workingface facehuman-machine human-machine collaborative collaborative

operation safetymonitoring operation safety monitoring and and earlyearly warning warning system system accordingaccording to the to the present present disclosure disclosure includes: includes: acquiring acquiring video video images capturedby images captured bythe the camera cameradevice deviceatat the the working face mobile working face mobileequipment equipmentsafety safety managementand management andcontrol controlsystem systemasastraining training data, data, and using worker and using worker images images annotated annotated with with personnel information as training data labels; personnel information as training data labels;

Thetraining The training data data labels labels include include posture posture images imagesofofworkers workers with with different different postures, postures, at at different different

shooting angles, and shooting angles, andunder underdifferent differentshooting shootingcategories, categories,andand facialimages facial images of the of the workers workers in in

different states; different states;

the postures include standing, squatting, bending down, left-side standing, and right-side standing; the postures include standing, squatting, bending down, left-side standing, and right-side standing;

the shooting the shootingangles anglesinclude include top-down top-down long-range long-range shooting, shooting, top-down top-down short-range short-range shooting, shooting,

bottom-uplong-range bottom-up long-rangeshooting, shooting,bottom-up bottom-up short-range short-range shooting, shooting, frontallong-range frontal long-range shooting, shooting, and and

frontal short-rangeshooting; frontal short-range shooting;

35

Theshooting The shooting categories categories include: include: visible visible light light high-definition high-definition shooting, shooting, low-illumination low-illumination

supplementary lightingshooting, supplementary lighting shooting,and andthermal thermalimager imager2727shooting; shooting;and and

the different the different states statesof ofthe theworkers workers include: include: an an unoccluded state, aa state unoccluded state, state of of wearing wearing aa protective protective mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing earmuffs, mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing earmuffs,

aa state state of of wearing wearing a a protective protective mask andaasafety mask and safety helmet, helmet,aa state state of of wearing wearing aa safety safety helmet helmetand and earplugs, earplugs, aastate stateofofwearing wearing a safety a safety helmet helmet and earmuffs, and earmuffs, a stateaof state of wearing wearing a protective a protective mask and mask and

earplugs, earplugs, a astate stateofofwearing wearing a protective a protective mask mask and earmuffs, and earmuffs, a state a state of of awearing wearing a protective protective mask, mask, aa safety helmetand safety helmet and earplugs, earplugs, and and a state a state of wearing of wearing a protective a protective mask, mask, a safety a safety helmet helmet and and earmuffs. earmuffs.

[0133] A collected

[0133] A collected training training set set isismanually manually annotated annotated and divided. Rectangular and divided. Rectangularboxes boxesare areused usedtoto label label persons, identity information, persons, identity and behavior information, and behaviorpostures posturesininimages, images, establishing establishing a personnel a personnel

image training set image training set against against the the complex background complex background ofof theunderground the underground heading heading face. face. Leveraging Leveraging

the training the training data dataand and corresponding training data corresponding training data labels, labels,an animproved improved YOLOV5 model YOLOV5 model undergoes undergoes

unsupervisedtraining unsupervised training and and deep deeplearning learning to to acquire acquire an an image recognition model. image recognition model.The Thetrained trained image image recognition model recognition modelthen thenperforms performs video video image image recognition recognition to accomplish to accomplish identity identity recognition, recognition,

target tracking, target tracking,behavior behavior prediction, prediction,and andworking working environment recognitionfor environment recognition for the the workers. workers.

[0134] First,the

[0134] First, theimages imagesin in thethe training training set set are are preprocessed: preprocessed: Mosaic Mosaic data augmentation data augmentation is used to is used to

enlarge, shrink, and rotate the input training set, which is then used as an input end of the algorithm. enlarge, shrink, and rotate the input training set, which is then used as an input end of the algorithm.

[0135] YOLOV5

[0135] YOLOV5 adjusts adjusts anchor anchor boxesboxes by means by means of a K-means of a K-means clustering clustering algorithm, algorithm, where a where K aK value needs to be manually set in the application process. Due to excessive reliance on the K value value needs to be manually set in the application process. Due to excessive reliance on the K value

under initial under initial conditions, conditions, the results are the results are prone to falling prone to falling into into local local optima, optima,resulting resultingininweak weak generalization ability of generalization ability of the the output output of of the the algorithm. algorithm. Therefore, Therefore, the the present present disclosure disclosure adopts adopts aa

PSOalgorithm PSO algorithmtotooptimize optimizethetheK-means K-means algorithm, algorithm, so to so as as improve to improve the the clustering clustering accuracy accuracy and and stability stabilityofofthe algorithm. the The algorithm. Theimproved improved PSO-K-means algorithm PSO-K-means algorithm is used is used to to adaptively adaptively determine determine

optimal anchor optimal anchor box box values values for different for different training training sets, thereby sets, thereby ensuringensuring the clustering the clustering accuracy ofaccuracy of

the system. the system.

[0136] TheK-means

[0136] The K-means algorithm algorithm performs performs clustering clustering analysis analysis on aon a set set {A, {A A,1,...An} A2, …Aofn} n of n anchor anchor

boxes toto bebeclassified, boxes classified, where wheren ndenotes denotesthethenumber number of the of the anchor anchor boxesboxes to be to be classified. classified. The The algorithm includes: selecting K points as initial cluster centers; calculating Euclidean distances algorithm includes: selecting K points as initial cluster centers; calculating Euclidean distances

between each sample and the K cluster centers, where K refers to the number of the cluster centers between each sample and the K cluster centers, where K refers to the number of the cluster centers

and is consistent with K in the K value; finding the nearest cluster center to the point and assigning and is consistent with K in the K value; finding the nearest cluster center to the point and assigning

the cluster center to the corresponding cluster; and calculating a centroid of each cluster to obtain the cluster center to the corresponding cluster; and calculating a centroid of each cluster to obtain

newcluster new cluster centers. centers. When thesum When the sum of of squared squared distances distances from from eacheach sample sample tocluster to the the cluster center center

36

reaches the minimum, the cluster is considered the cluster center. As can be seen, the classification reaches the minimum, the cluster is considered the cluster center. As can be seen, the classification

result is result is highly dependentonon highly dependent thethe choice choice ofinitial of an an initial value value of cluster of the the cluster centers, centers, and anand an inappropriate inappropriate K K value value tends tends to lead to lead to a to a local local optima. optima.

[0137] Therefore, by

[0137] Therefore, by employing employing a swarm a swarm intelligence intelligence algorithm algorithm to select to select an an appropriate appropriate K value K value

for clustering division, for clustering division, the the present presentdisclosure disclosureadopts adopts a particle a particle swarm swarm optimization optimization (PSO) (PSO)

algorithm algorithm toto classifysample classify sample setssets and and determine determine an optimal an optimal K value K value for for clustering K-means K-means division. clustering division. The specific steps are as follows: The specific steps are as follows:

Initializing Initializinga a particle swarm particle swarmand andrandomly randomly generating generating N N particles, particles,where where11<N<n, < N < n, < nn is is the thenumber number

of of anchor boxestotobebeclassified; anchor boxes classified; and anddividing dividingthe then nanchor anchorboxes boxes by by taking taking the the N particles N particles as as

cluster centers. cluster centers.

[0138] Calculating and

[0138] Calculating andnormalizing normalizingdistances distancesfrom from each each sample sample data data point point to to thethe clustercenters, cluster centers, calculating andnormalizing calculating and normalizing a distance a distance between between any two any twocenters, cluster clusterand centers, and calculating calculating fitness of fitness of the sample the data: sample data:

calculating andnormalizing calculating and normalizing distances distances fromsample from each eachdata sample pointdata point to the to the cluster cluster center center includes: includes:

K D = whereDDrepresents where in represents a distancefrom a distance from uthuth sample sample data data Au Au in in a jthcluster a jth cluster wωof j ofaasample sampledataset datasettoto aa cluster centerCj, cluster center Cj,and andK K is is thethe number number of cluster of cluster centers; centers;

whered'𝑑 ′represents where 𝑖𝑛 represents normalized normalized Din, max(D D, max(Din) ) and min(D and inmin(Din) in) represent represent a maximum a maximum value and value a and a minimum minimum value value among among all all D, D in, respectively; respectively;

37

calculating calculating and and normalizing normalizing aa distance distance between betweenany anytwo twocluster clustercenters centersincludes: includes:

D = d(w,w) 1i, jN

where where DDrepresents out represents a distance a distance between between any any two cluster two cluster centers, centers, Ci Cand C and areCcluster j are cluster centers centers of of

an ith cluster an ith cluster wiωiand andthethe jthjth cluster cluster wj,ωj, respectively, respectively, N isN is number the the number of particles of particles in the initialized in the initialized

particle swarm, and d(ω , ω ) represents an absolute value of a difference value between the cluster particle swarm, and d(w, w) i represents j an absolute value of a difference value between the cluster

centers centers C andC;Cjand C i and ; and Doutis Dout' ’ isnormalized normalizedDout, Dout’, andand max(Doutmin max(Dout), ), min (Doutrepresent (Dout) ) represent a maximum a maximum

value and value and aa minimum value minimum value among among all all D, D out, respectively; respectively;

The fitness of the sample data is as follows: The fitness of the sample data is as follows:

where F represents the fitness of the sample data. where F represents the fitness of the sample data.

[0139] Performingoptimization

[0139] Performing optimization based based on fitness on the the fitness of the of the sample sample data data by by means means of a PSOof a PSO

algorithm algorithm toto obtain obtain thethe optimal optimal cluster cluster centers: centers:

inputting inputting aa feature featuremap map into into aa Neck network,using Neck network, usingan anFPN+PAN FPN+PAN structure structure to further to further enhance enhance the the

extraction offeature extraction of featureinformation informationand and to perform to perform fusionfusion processing, processing, and outputting and outputting a detection a detection map; map;

inputting inputting the the detection detection map into aa Head map into networktotoultimately Head network ultimately obtain obtain information informationsuch suchasasthe the size size and theposition and the positionofofthetheworker, worker, comparing comparing theand the size size theand the position position information information of the of the worker withworker with

the size the size and the position and the position information of an information of an annotation annotationbox, box,and andusing usinga CloU_Loss a CIoU_Loss function function to to measurethe measure theaccuracy accuracybetween between a targetbounding a target bounding box box andand thethe detection detection bounding bounding box:box:

FCIOU_Loss

38

whereAs, where As, BBsrespectively respectivelyrepresent representthe the detection detection bounding boundingbox boxandand thetarget the targetbounding bounding box, box, area area

(A s) represents (As) representsthethearea area of of thethe detection detection bounding bounding box, box, area (B)area (Bs) represents represents the area of the the area targetof the target

boundingbox, bounding box,FloU_Loss FIoU_Loss represent represent aa ratio ratio of of an an overlapping area between overlapping area betweenthe thedetection detectionbounding bounding box and box andthethetarget targetbounding bounding box,box, FIoU_Loss FloU_Loss is expression is an an expression ofCloU_Loss of the the CIoU_Loss function,function, d d0 represents aa Euclidean represents distance between Euclidean distance betweenthe thecenter centerofof the the target target bounding boxand bounding box anda acenter centerpoint point of a predicted bounding box, and d represents a diagonal length of the target bounding box. c of a predicted bounding box, and dc represents a diagonal length of the target bounding box.

[0140] V is isa aparameter

[0140] V parameter used used to to measure measure an an aspect aspect ratio,which ratio, whichis isdefined definedasasfollows: follows:

2

V who where

and gt h correspond and hgt correspond whereWwand wp and and wp

[0141] hp correspond andhp

TheCloU_Loss

[0141] The

takes into takes CIoU_Loss into account accountthe to the to the

correspondtotothe

function function

theoverlapping width width and and

the width widthand

is used is used

overlapping area area height height

andheight of aofground

heightof

as evaluation as an a ground of the

an evaluation between between truth truth

the predicted

metric metric

the detection the detection bounding bounding

predicted bounding box,box,

boundingbox,

for output, for the the output, bounding bounding box andbox respectively, respectively,

box,respectively. respectively.

which which

the and the target not only not only

target bounding box, but also incorporates factors such as the overlapping area, the distance between the bounding box, but also incorporates factors such as the overlapping area, the distance between the

center points,and center points, andthe theaspect aspect ratio.This ratio. This enables enables the the effective effective detection detection of overlapping of overlapping target objects. target objects.

[0142] Theconventional

[0142] The conventionalYOLOV5 YOLOV5 algorithm algorithm adopts adopts a standard a standard non-maximum non-maximum suppression suppression (NMS) (NMS) method.During method. Duringthe theobject objectdetection detectionprocess, process, aa large large number ofpredicted number of predictedbounding boundingboxes boxes maymay be be generated. generated. Among them, Among them, detectionbounding detection bounding boxes boxes corresponding corresponding to the to the samesame target target are are filteredtoto filtered

obtain non-overlappingtarget obtain non-overlapping targetdetection detectionbounding bounding boxes. boxes. TheThe predicted predicted bounding bounding boxthe box with with the highest score highest score for for the the same target is same target is compared withother compared with otherbounding bounding boxes boxes in in terms terms of of IoUIoU values values

and and aa preset presetthreshold. threshold.Pre-filtered Pre-filtered target target bounding bounding boxesboxes with with IoU IoUhigher values valuesthan higher than the threshold the threshold

are are considered highly overlapping considered highly overlappingand andare arediscarded. discarded.Finally, Finally, aa non-overlapping non-overlappingtarget targetdetection detection boundingbox bounding boxisisobtained. obtained. During Duringthe theexecution, execution,the the threshold threshold value value needs needs to to be be manually set, and manually set, and

the value of the threshold affects the accuracy of target detection. the value of the threshold affects the accuracy of target detection.

[0143] Duetotothe

[0143] Due thelarge large size size of of underground equipment underground equipment andand thethe relativelynarrow relatively narrow working working space space

for for personnel, personnel, the the standing standing positions positions of of personnel personnel tend tend to to be be dense. dense. As As aa result, result, using using IoU-NMS IoU-NMS

tends to cause target overlaps and leads to the elimination of detection bounding boxes. Therefore, tends to cause target overlaps and leads to the elimination of detection bounding boxes. Therefore,

the present the present disclosure disclosureadopts adoptsa DIoU-NMS a DIoU-NMS algorithm algorithm with an threshold with an adaptive adaptive to threshold filter to filter redundancybounding redundancy bounding boxes. boxes.

39

[0144] All predicted

[0144] All predictedbounding bounding boxes boxes of same of the the same targettarget are ranked are ranked in descending in descending order of order of

confidencescores. confidence scores. The Thetarget target bounding boundingboxes boxesare arefiltered filtered by by means meansofofthe theDIoU-NMS DIoU-NMS algorithm algorithm

to remove to redundantpredicted remove redundant predictedbounding bounding boxes, boxes, retaining retaining thebest the besttarget target predicted predicted bounding boundingbox: box:

(0, IoU - (M,B) S = (S, IoU - < p²(b¹,

RDIU = IoU = n U b²|

b1 and b¹ b2 represent and b² represent the the centers centers of of pre-filtered pre-filteredcandidate candidatebounding bounding box box 11 and andcandidate candidatebounding bounding box 2, box 2, respectively, respectively,ε represents represents an an NMS threshold,MMrepresents NMS threshold, representsthe thepredicted predictedbounding boundingbox box with with

the highest score, IoU represents a ratio of an intersection set to a union set of the pre-filtered the highest score, IoU represents a ratio of an intersection set to a union set of the pre-filtered

candidate target candidate target bounding box11 and bounding box and candidate candidatetarget target bounding box2,2, BBprepresents bounding box representsaapth pth detection detection boundingbox, bounding box,S Srepresents p represents a confidence a confidence score score of pth of the the detection pth detection bounding bounding box,isRDIoU box, RDIU a is a penalty factor penalty factor of of DIoU-NMS, P represents DIoU-NMS, P represents a Euclidean a Euclidean distance distance between between two two detection detection bounding bounding

boxes, and boxes, and CC12 denotes denotes a diagonal a diagonal lengthofofa aminimum length minimum enclosing enclosing areaarea of of thethe pre-filteredcandidate pre-filtered candidate boundingbox bounding box1 1and andcandidate candidatebounding bounding boxbox 2. 2.

[0145]

[0145] εrepresents representsthe theNMS NMS threshold, threshold, andand thethe value value of ofisε automatically is automatically adjusted adjusted by by means means of of

an an adaptive algorithm. When adaptive algorithm. thedistance When the distancebetween between targetobjects target objectsisis large large and personnel are and personnel are dense, dense, εisis smaller; smaller;when whenthe the distance distance is small is small and personnel and personnel are fewer, are fewer, ε isthereby is larger, larger,improving thereby the improving the accuracy of accuracy of long-distance long-distance personnel personnelrecognition. recognition.

[0146]

[0146] AAhigh-definition high-definition camera cameraatatthe theworking working face face captures captures images images of surrounding of surrounding personnel. personnel.

Animproved An improved YOLOV5 YOLOV5 algorithm algorithm is to is used used to perform perform targettarget detection detection on continuous on continuous single-frame single-frame

images of real-time images of real-time video streams, thereby video streams, detecting personnel thereby detecting and safety personnel and safety helmets on the helmets on the working working face to achieve face to achievesafety safety monitoring monitoring ofpersonnel of the the personnel at the at the working working face. face.

[0147] Inananimage

[0147] In image captured captured bycamera, by the the camera, an electronic an electronic fence fence area area isinto is divided divided into different different safety safety areas areas such as safe such as safe areas, areas, alarm alarm areas, areas,and and shutdown areas according shutdown areas accordingtotohazard hazardlevels levels of of aa camera camera monitoringarea. monitoring area.Decoded Decoded continuous continuous single-frame single-frame images images from from the the real-time real-time streams streams of the of the camera serveasasinput camera serve inputto tothethe algorithm. algorithm. A target A target detection detection algorithm algorithm is used is used to detect to detect mine mine

40 personnel and personnel andsafety safetyhelmets helmets in in thethe images images and and detect detect whether whether personnel personnel are wearing are wearing safety safety helmets. The identity of personnel triggering an alarm is retrieved based on the portable positioning helmets. The identity of personnel triggering an alarm is retrieved based on the portable positioning

terminal 31. terminal 31.

[0148] Personnelatatthe

[0148] Personnel the working workingface facearearesupervised supervised through through the the visual visual analysis analysis system. system. WhenWhen

personnel enter a hazardous area or fail to wear safety helmets correctly, the system recognizes the personnel enter a hazardous area or fail to wear safety helmets correctly, the system recognizes the

personnel triggering personnel triggering the the alarm. alarm.Upon Upon personnel personnel intrusion, intrusion, the the system system issues issues an alarm an alarm and, and, if if necessary, outputs necessary, outputs control control signals signals to to shut shut down downthethe equipment. equipment. Recognition Recognition results results and alarm and alarm

signals of the signals of thesystem system are are transmitted transmitted to the to the centralized centralized control control platform platform 15,the 15, where where the alarm current current alarm event videoisisrecorded, event video recorded, saved saved and sent and sent to thetointelligent the intelligent terminal terminal device device 34 of the34 of the manager. manager.

[0149] After the

[0149] After the image imagerecognition recognitionmodel model performs performs recognition recognition on each on each frameframe of theofcaptured the captured video, the method further includes: video, the method further includes:

Processing an Processing an input input video video by by frame to achieve frame to achieve pedestrian pedestrian detection detection and and tracking tracking based based on on complete complete

trajectories formed by each worker at the working face formed in videos of the camera. This solves trajectories formed by each worker at the working face formed in videos of the camera. This solves

the problem of temporary loss of personnel (such as workers, drivers, pedestrians, etc.) caused by the problem of temporary loss of personnel (such as workers, drivers, pedestrians, etc.) caused by

occlusion fromheading occlusion from heading equipment equipment in heading in the the heading face, otherwise face, which which otherwise prevents prevents effective effective

monitoringand monitoring andsafety safety protection protection of of the the personnel. personnel. The The system processes the system processes the input input video video by means by means

of aa DeepSort of DeepSortalgorithm algorithm andand an Osnet an Osnet pedestrian pedestrian re-identification re-identification network network model model to achieve to achieve

workertracking, worker tracking, generate generate tracking tracking trajectories, trajectories, and andperform perform trajectory trajectoryprediction. prediction.When When aa worker worker

is is partially partiallyor orfully fullyoccluded occluded by by equipment equipment ororother otherpedestrians, pedestrians,the thesystem systemcancan pre-determine pre-determine

hazardousbehaviors hazardous behaviorsbased based on on trajectoryinformation trajectory information of of thethe worker worker and and output output a control a control signal signal

according to the according to the danger degree of danger degree of the the worker. worker.

[0150] Duetotothe

[0150] Due thehigh highdust dustconcentration concentration andand low low illumination illumination intensity intensity in the in the heading heading face,face,

ordinary cameras ordinary produceunclear cameras produce unclearimages imagesand and failto fail to obtain obtain effective effective personnel personnel image image information. information.

The present disclosure uses a dual-light fusion video camera 21 and a thermal imager 27 to acquire The present disclosure uses a dual-light fusion video camera 21 and a thermal imager 27 to acquire

video frames video framesas as inputs inputs for for an an image algorithm of image algorithm of the the system. system. The specific implementation The specific process implementation process

includes: includes:

Step Step S10: S10: using using a a dust dust monitor monitor to to collect collecta adust dustconcentration concentrationunder undermine mine working conditions, and working conditions, and

performingdetermination performing determination andand execution: execution: if dust if the the dust concentration concentration under under theworking the mine mine working conditions is conditions is lower lowerthan thana apreset presetthreshold, threshold,capturing capturingvideo video images images under under the working the mine mine working conditions via the conditions via the video videocamera camera 21, 21, and and proceeding proceeding to S20; to Step Stepotherwise, S20; otherwise, synchronously synchronously

capturing capturing video video images underthe images under the mine mineworking workingconditions conditionsvia viathe thethermal thermalimager imager2727and andthe thevideo video camera 21, and camera 21, andproceeding proceedingtotoStep StepS30. S30.

41

[0151] StepS20:

[0151] Step S20:using usinga deep a deep convolutional convolutional neural neural network network to extract to extract high-level high-level multi-scale multi-scale

feature feature map groupsandand map groups low-level low-level multi-scale multi-scale feature feature map map groups groups from images from images capturedcaptured by a by a visible light visible light imager, imager, performing iterative feedback performing iterative feedbackfusion fusionofofthethefeature featuremapmap groups, groups, and and by by leveraging complementary leveraging complementary relationships relationships between between features features in theinfused the feature fused feature map map groups, groups, obtaining positions of obtaining positions of the the target target detection detection bounding boxesininthe bounding boxes theimages, images,and andidentifying identifyingtarget target detection results. detection results.

[0152] Performingiterative

[0152] Performing iterative feedback fusion of feedback fusion of the the feature feature map groupsinin Step map groups Step S20 S20includes: includes:

constructing Gaussian constructing pyramidsand Gaussian pyramids andLaplacian Laplacian pyramids pyramids corresponding corresponding to the to the feature feature maps maps in the in the

high-level multi-scale high-level multi-scale feature feature map groups, and map groups, andfusing fusing the the Gaussian Gaussianand andLaplacian Laplacian pyramids pyramids with with

the corresponding the low-levelmulti-scale corresponding low-level multi-scale feature feature maps alongaafeature maps along feature channel channeldimension dimensiontotoobtain obtain a multi-scale a multi-scale fused fused feature feature map map group. group.

[0153] StepS30:

[0153] Step S30:extracting extractingthermal thermal imaging imaging contours contours of images of the the images captured captured by the by the thermal thermal

imager 27and imager 27 andpositions positionsofof the the thermal thermal imaging imagingcontours contoursininthe theimages imagestotoobtain obtaincandidate candidatetarget target detection results: detection results:

extracting extracting the thethermal thermal imaging imaging contours contours of of the the images images captured captured by by the the thermal thermal imager imager 27 27 by by means means

of an of an adaptive adaptiveedge edgedetection detection method, method, and and calculating calculating the positions the positions ofthermal of the the thermal imagingimaging

contours in the contours in the images; images; and andrespectively respectivelycalculating calculatingsimilarities similarities between theextracted between the extractedcontours contours and target images and target pre-stored in images pre-stored in an an infrared infrared thermal thermal imaging database, and imaging database, andusing usingcategories categoriesof of the the target images target withsimilarities images with similaritiesgreater greaterthan thana preset a preset value, value, along along withwith the positions the positions of of the the corresponding thermal imaging contours, as the obtained candidate target detection results. corresponding thermal imaging contours, as the obtained candidate target detection results.

[0154] Step S40:

[0154] Step S40: constructing constructing aa multi-scale multi-scale pyramid poolingresidual pyramid pooling residual network, network,extracting extracting features features of of the the images images captured captured by by the the video video camera camera 21 21 corresponding to the corresponding to the images captured by images captured by the the thermal thermal

imager 27,andand imager 27, filtering filtering thethe candidate candidate target target detection detection results results through through a fullyaconnected fully connected network to network to

obtain obtain aafinal finaltarget targetdetection detectionresult. result.

[0155] Constructingaa multi-scale

[0155] Constructing multi-scale pyramid pyramidpooling poolingresidual residualnetwork networkininstep stepS40 S40includes: includes:

replacing convolutional layers of the deep convolutional neural network with dilated convolutional replacing convolutional layers of the deep convolutional neural network with dilated convolutional

layers layers of of aa preset preset scale, scale,and and replacing replacing aa final finalaverage average pooling pooling layer layer with a multi-scale with a multi-scale pyramid pyramid pooling layer. pooling layer.

[0156] In one

[0156] In embodiment one embodiment of of thepresent the presentdisclosure, disclosure,the the multi-scale multi-scale pyramid pyramidpooling poolinglayer layeradopts adopts three scales. The features transmitted from a previous-layer network are divided into three scales: three scales. The features transmitted from a previous-layer network are divided into three scales:

the first the first scale scale corresponds to aa feature corresponds to feature map mapidentical identicaltotoananoriginal originalfeature; feature;the thesecond second scale scale

42

involves dividing an involves dividing an original original feature feature map into 2×2=4 map into feature maps 2x2=4 feature mapsofofthe thesame samesize, size,and andthe thethird third scale involvesdividing scale involves dividing thethe original original feature feature map4x4=16 map into into 4×4=16 feature feature maps maps of the sameof the same size. In thissize. In this

way, the features transmitted by the previous-layer network are divided into 21 feature maps. way, the features transmitted by the previous-layer network are divided into 21 feature maps.

[0157]

[0157] AAmulti-layer multi-layerresidual residualblock blockisisadded added between between designated designated layers layers ofnetwork of the the network after after

updating the updating the convolutional convolutionallayers layers and andpooling poolinglayers layerstotoconstruct constructaamulti-scale multi-scalepyramid pyramidpooling pooling residual network. The multi-layer residual block adopts either a two-layer residual structure or a residual network. The multi-layer residual block adopts either a two-layer residual structure or a

three-layer residual structure: three-layer residual structure:

the two-layer residual structure includes a first convolutional layer with a 3×3 kernel and a second the two-layer residual structure includes a first convolutional layer with a 3x3 kernel and a second

convolutional layer convolutional layerwith witha a3x33×3 kernel kernel connected connected in sequence, in sequence, where where the theof input input of the the first first convolutional layer and convolutional layer the output and the output of of the the second second convolutional layer are convolutional layer are concatenated; concatenated;

the three-layer residual structure includes a third convolutional layer with a 1×1 kernel, a fourth the three-layer residual structure includes a third convolutional layer with a 1x1 kernel, a fourth

convolutional layer with a 3×3 kernel, and a fifth convolutional layer with a 1×1 kernel connected convolutional layer with a 3x3 kernel, and a fifth convolutional layer with a 1x1 kernel connected

in sequence,where in sequence, wherethethe input input of third of the the third convolutional convolutional layer layer and and the the of output output of the fifth the fifth

convolutional layer convolutional layer are are concatenated. concatenated.

[0158]

[0158] AAloss loss function functionadopted adoptedduring duringthethetraining trainingofofthe themulti-scale multi-scalepyramid pyramid pooling pooling residual residual

network is as follows: network is as follows:

L 1 where m is the number of training samples in network training, q is the number of target detection where m is the number of training samples in network training, q is the number of target detection

categories fortraining categories for trainingsamples, samples, xkthe X is is the feature feature corresponding corresponding totraining to a kth a kth training sample, sample, yk is a label yk is a label

of the training sample x , W is a weight matrix of the network, b is a bias matrix of the network, T of the training sample X, Wk is a weight matrix of the network, b is a bias matrix of the network, T

denotes matrix denotes matrixtransposition, transposition, ee is is aa natural natural logarithm; logarithm; g g denotes the number denotes the numberofoftarget targetdetection detection categories fora agth categories for gthtraining training sample, sample, and and L is L is loss the the loss function. function.

[0159] Inthe

[0159] In theheading heading face, face, thethe thermal thermal imager imager 27 easily 27 easily captures captures objects objects with human-like with human-like contours, contours,

such as heat-emitting such as heat-emitting cables, cables, motors motorsoperating operatingwith withheat heatgeneration, generation,and andwater waterpumps pumps radiating radiating

heat, whose heat, contourscaptured whose contours capturedbybythe thethermal thermalimager imager 27 27 resemble resemble human human body body contours. contours. DuringDuring

system operation, the system operation, the personnel personnel recognition recognitionalgorithm algorithmmay may misinterpret misinterpret anan infraredimage infrared image of of thethe

human-likecontour human-like contourasasananactual actualhuman human body, body, triggering triggering a falsealarm a false alarm thataffects that affectsthe the production production efficiency efficiency of of the theworking working face face equipment. equipment.

43

[0160]

[0160] InInthe thepresent present disclosure, disclosure, temperature temperature data data of eachofpixel eachinpixel in thecaptured the images images by captured thermal by thermal

imager 27isis reconstructed imager 27 reconstructedtoto form forma atemperature temperature matrix. matrix. Temperature Temperature images images generated generated by the by the

thermal imager thermal imager2727are areinput inputinto intothe the improved improvedYOLOV5 YOLOV5 algorithm algorithm to obtain to obtain human human targets.targets. By By utilizing the temperature matrix, which reflects the different temperatures of objects, the system utilizing the temperature matrix, which reflects the different temperatures of objects, the system

determines whetherthethecontour determines whether contour of of a heat-emitting a heat-emitting object object captured captured bythermal by the the thermal imagerimager 27 27 corresponds to aa human corresponds to body,thereby human body, therebyimproving improving thethe human human target target recognition recognition rate rate of of thethesystem. system. The details are as follows: The details are as follows:

Thetemperature The temperaturedata datacollected collectedbybythe thethermal thermalimager imager 27 27 is is organized organized into into a temperature a temperature matrix matrix

according according toto theposition the position of of each each pixel pixel in the in the infrared infrared image; image; each element each element value invalue in the temperature the temperature

matrix represents matrix represents an an actual actual temperature temperaturevalue valueofofthe thecorresponding correspondingpixel pixelininthe thethermal thermalinfrared infrared image; image;

Byusing By using thermal thermalimaging imagingdata dataofofthe the heading headingface face collected collected by by the the thermal thermal imager imager 27, 27, temperature temperature

matrix data matrix data corresponding correspondingtoto human human targetsand targets andheat-emitting heat-emittingobjects objectssuch suchasasmotors, motors,cables, cables,and and water pumps water pumpsininthe thecaptured capturedinfrared infraredimages imagesareare annotated. annotated. Then, Then, an an SVMSVM algorithm algorithm is to is used used to classify the temperature classify the temperature matrix matrix data, data, obtaining obtaining datasets datasets such such as as temperature temperature range of range of human targets human targets

at at the theunderground headingface underground heading facebased basedononthe thethermal thermalimager imager2727and andthe thetemperature temperaturerange range during during

motoroperation; motor operation;

Continuous single-frameinfrared Continuous single-frame infraredimages imagesobtained obtained from from a thermal a thermal imaging imaging video video collected collected by the by the

thermal imager thermal imager2727are areused usedasasinput inputtotothe theimproved improved YOLOV5 YOLOV5 algorithm, algorithm, ultimately ultimately obtaining obtaining

coordinates of coordinates of detection detection bounding boxesannotated bounding boxes annotatedasashuman human bodies; bodies;

Thetemperature The temperaturedata datawithin withinthe theannotated annotateddetection detectionbounding bounding boxes boxes are are classified classified by by means means of of the SVM algorithm. If the temperature data falls within the temperature range of the human targets, the SVM algorithm. If the temperature data falls within the temperature range of the human targets,

the human the targets are human targets are determined as human determined as bodies;if human bodies; if the the temperature temperature data data matches the temperature matches the temperature

range of range of objects objects such such as as motors motors or or cables, cables,the thehuman targets are human targets aredetermined determined as as non-human bodies. non-human bodies.

[0161]

[0161] According to the According to the wireless wireless positioning positioningmethod method for forthe theworking working face face human-machine human-machine

collaborative operation safety collaborative operation safety monitoring monitoringandand early early warning warning system system provided provided by the by the present present

disclosure, theequipment disclosure, the equipment in heading in the the heading face face is is large, large, and theand the environment environment in the in the working working face is face is complex,with complex, withaalot lot of of metal metal obstructions, obstructions, such such as as anchor anchor rods rods and iron mesh. and iron mesh. AApropagation propagationpath path of of UWB signalsisisaffected UWB signals affectedbybyreflections reflections and anddiffraction diffraction caused by obstacles caused by obstacles like like roadway walls roadway walls

and mining equipment, leading to signal attenuation and non-line-of-sight errors, which results in and mining equipment, leading to signal attenuation and non-line-of-sight errors, which results in

low positioningaccuracy. low positioning accuracy.Since Since thethe real-time real-time accuracy accuracy requirements requirements for afor UWBapositioning UWB positioning system are high, system are high, aacontrol controlunit unit processor processorofofa aUWB UWBbase base station station has has a lowa speed low speed and limited and limited

44 performanceand performance andalso alsohas hasdata dataupload upload and and download download functions. functions. Therefore, Therefore, an acceleration-based an acceleration-based

filtering filteringalgorithm algorithm is is used used to to reduce errors caused reduce errors causedbybythe thewireless wirelesspositioning positioningalgorithm. algorithm. TheThe

algorithm is simple algorithm is simpleand andreliable, reliable,capable capableofofcompleting completing a filtering a filtering task task without without significantly significantly

increasing theworkload increasing the workload of processor. of the the processor.

[0162] Thewireless

[0162] The wireless positioning positioning method methodbased based onon theworking the working face face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety monitoringand andearly earlywarning warning system system includes includes the following the following steps:steps: acquiring acquiring

measurement measurement databetween data between thethe wireless wireless positioning positioning terminal terminal 26 26 andand thethe wireless wireless positioning positioning base base

station at aa time station at timet,t,setting settingthe thewireless wireless positioning positioning base base station station as an original as an original point topoint defineto define the the

position of position of the the wireless wireless positioning positioning terminal 26 within terminal 26 within aa wireless wireless coverage coveragearea areaofofthe thewireless wireless positioning base station as (x , y ), and obtaining distance data s between the wireless positioning t tand obtaining distance data S between the wireless positioning positioning base station as (x, yt),

terminal 26 and the wireless positioning base station; terminal 26 and the wireless positioning base station;

performing differential operations on the distance data s for a preset number of times to obtain a performing differential operations on the distance data S for a preset number of times to obtain a

velocity v and an acceleration a of the wireless positioning terminal 26; velocity v and an acceleration a of the wireless positioning terminal 26;

comparing and determining a velocity v and an acceleration a at the time t with a velocity vt-1 and comparing and determining a velocity V and an t acceleration a at the time t t with a velocity Vt-1 and

an accelerationat-1 an acceleration at-1atatthe thetime timet-1: t-1:

whenthe when thevariations variations in in the the velocity velocity Vvt and andthe the acceleration acceleration a- at- are are within within preset preset threshold threshold ranges, ranges, the position (x , y ) of the current wireless positioning terminal 26 is valid; and the position (x, t yt)t of the current wireless positioning terminal 26 is valid; and

otherwise, based otherwise, based on on the the velocity velocity Vt-1 vt-1 and and the acceleration the acceleration at-1 atat-1 theattime thet-1 time andt-1 and a velocity a velocity Vt-2 and vt-2 and

an accelerationat-2 an acceleration at-2atatthe thetime timet-2, t-2,predicting predicting a predicted a predicted velocity velocity 𝑣𝑡′ a and v't and a predicted predicted acceleration acceleration

′ at the time t, and based on the predicted velocity v't and 𝑎 𝑡 at a't the time t, and based on the predicted velocity 𝑣𝑡′ and the the predicted predicted a't, 𝑎𝑡′ , acceleration acceleration

′ calculating calculating a apredicted predicted position position (x't, , 𝑦𝑡′ )ofofthethewireless (𝑥𝑡y't) wireless positioning positioning terminal terminal 26 at 26 the at thet.time t. time

[0163] Thepresent

[0163] The presentdisclosure disclosurerelates relates to to aa multi-module multi-modulesingle-base-station single-base-stationpositioning positioningmethod method using multi-module-based using single-base-stationpositioning multi-module-based single-base-station positioningtechnology technologytotoachieve achievethe thepositioning positioning of of aa portable positioning portable positioning terminal terminal 31aby 31 by a wireless wireless positioning positioning terminal terminal 26. Abase 26. A single single baserefers station station refers to a base station in which multiple modules and an antenna array form a ranging unit. The multiple to a base station in which multiple modules and an antenna array form a ranging unit. The multiple

modulesrefer modules refer to to multiple multiple identical identical UWB modules. UWB modules. TheThe positioning positioning is achieved is achieved by using by using the the time time

and angle and angle at at which wireless signals which wireless signals arrive arrive at at each each module. Thedisclosure module. The disclosureis is described described with with three three modules:the modules: thewireless wirelesspositioning positioningterminal terminal2626serves serves as as a single a single base base station,which station, which internally internally

includes includes three three ranging ranging modules andaa calculation modules and calculation module, wherethree module, where three antennas antennasform formananequilateral equilateral triangle array, and a base station antenna array composed of three 120° sector antennas is used to triangle array, and a base station antenna array composed of three 120° sector antennas is used to

cover aa 360° cover space; each 360° space; each antenna antennacorresponds correspondstotoa aUWB UWB module, module, and antenna and one one antenna andUWB and one one UWB moduletogether module togetherconstitute constitutea aranging rangingmodule.; module.; thethe portable portable positioning positioning terminal terminal 31 contains 31 contains a a

45

UWB UWB signal signal generation generation module module internally; internally; andand thethe ranging ranging module module can can receive receive UWB UWB signalssignals and and performsignal perform signal analysis analysis and andparsing parsingthrough throughthe thecalculation calculationmodule module within within thethe base base station.The station. The details are as details are as follows: follows:

At the At the time timetb, tb, the the portable portable positioning positioning terminal terminal 3131emits emitsa aUWBUWB wireless wireless signal, signal, denoted denoted as as signalt b; signaltb;

After receiving After receiving the the UWB UWB signal signal signaltthe signaltb, b, the ranging ranging module module withinwithin the wireless the wireless positioning positioning

terminal 26 performs a signal calculation. terminal 26 performs a signal calculation.

[0164] Basedononthe

[0164] Based thetime timeTfTwhen f when thethe signal signal arrivesatatthe arrives themodule, module, thethe distance distance Lf Lbetween f between the the

portable positioning terminal 31 and the base station is calculated as follows： portable positioning terminal 31 and the base station is calculated as follows:

T f = Tfc, Tf= Tf•c,where 3*108m/s. wherec c= =3*10°m/s

[0165] After the

[0165] After thecalculation, calculation, the thedistances distancesfrom from thethe UWBUWB signalsignal signalt signaltb from fromb the the portable portable

positioning terminal positioning terminal 31 to each 31 to each ranging moduleare ranging module areobtained. obtained.The Thedistance distancetotoaa ranging rangingmodule moduleA A

is is la; la; the the distance to aa ranging distance to rangingmodule module B is B is and lb; lb; the anddistance the distance to a ranging to a ranging module Cmodule is lc. C is lc.

[0166] The

[0166] The specific specific angle angle calculation calculation process process is as follows: is as follows:

AnAOA An AOA algorithm algorithm is is used used to to calculateananarrival calculate arrival angle angle of of the the UWB signal,obtaining UWB signal, obtainingthe theangles anglesat at whichthe which theUWB UWB signal signal signalt signaltb of the ofb the portable portable positioning positioning terminal terminal 31 arrives 31 arrives at each at each ranging ranging

module. The angle of arrival at the ranging module A is α; the angle of arrival at the ranging module module. The angle of arrival at the ranging module A is ; the angle of arrival at the ranging module

B is ß; B is β; and andthe theangle angleof of arrival arrival at at the the ranging ranging module module C is . C is γ.

[0167] The

[0167] The difference difference in ainpath a path length length of UWB of the the signal UWB signaltb signal signalt b is related is related to a distance to a distance d betweend between

the ranging the moduleA Aandand ranging module theranging the ranging module module B, the B, the angle angle of α of arrival arrival at the at the ranging ranging module module A A and theangle and the angleß of β of arrival arrival at at thethe ranging ranging module module B. By calculation, B. By calculation, the anglesthe at angles at signal which the which the signal reaches each reaches each module modulecan canbebeinferred. inferred.

[0168] For example:

[0168] For example:totocalculate calculate the the angle angle αof of arrival arrival at atthe theranging rangingmodule module A: since dd is A: since isknown, known,

and laand and la andlblbare arecalculated, calculated, it it isis known known based based on theon theof law law of cosines: cosines:

the angle the angle αcan canthen thenbe becalculated calculated based basedon ongeometric geometricrelated relatedknowledge. knowledge.

[0169] Similarly,

[0169] Similarly, thethe angle angle of arrival of arrival at the at the ranging ranging modulemodule B is ß, B andisthe β, angle and the angle of of arrival at arrival the at the ranging module ranging moduleC Cisis.γ.

46

[0170] Basedononthetheangles

[0170] Based anglesandand distances distances at at which which the the UWB UWB signal signal signalt signaltb from fromb the the portable portable

positioning terminal 31 arrives at each ranging module, the angle of arrival at the ranging module positioning terminal 31 arrives at each ranging module, the angle of arrival at the ranging module

A is αand A is andthethe distance distance to the to the ranging ranging module module A is la;Athe is la; theofangle angle of at arrival arrival at themodule the ranging ranging B module B

is is βß and and the the distance distance to tothe theranging rangingmodule module B is lb; B is lb;and and the thedistance distancetotothe theranging rangingmodule module C is γ C is

and thedistance and the distancetotothetheranging ranging module module C isthelc,position C is lc, the position of the of the portable portable positioning positioning terminal 31 terminal 31

is is determined. Since determined. Since the the relative relative positions positions of theof the modules modules arethefixed, are fixed, theofposition position of the portable the portable

positioning terminal positioning terminal 31 31 can can be be back-calculated back-calculatedbased basedononthethedistance distanceand andangle, angle,and andthethespecific specific position, orientation, and distance of the portable positioning terminal 31 can be obtained through position, orientation, and distance of the portable positioning terminal 31 can be obtained through

the calculation. the calculation.

[0171] Thepresent

[0171] The presentdisclosure disclosure further further provides provides an electronic an electronic device, device, including including at least at least one one processor; and processor; and aamemory memory communicatively communicatively connected connected to the to at the at one least leastprocessor, one processor, where where the the memory memory stores stores instructionsexecutable instructions executable by by the the processor, processor, the the instructions instructions are are executed executed by by the the processor to processor implement the to implement the above-mentioned above-mentionedworking working face face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and early warning warningmethod. method.

[0172] Thepresent

[0172] The presentdisclosure disclosurealso alsoprovides providesa acomputer-readable computer-readable storage storage medium, medium, whichwhich storesstores

computer instructions. The computer instructions. Thecomputer computerinstructions instructionsare areexecutable executablebybya acomputer computerto to implement implement the the

above-mentioned working above-mentioned working faceface human-machine human-machine collaborative collaborative operation operation safety monitoring safety monitoring and and early warning early warning method. method. Those Those skilled skilled in thein thecanartclearly art can clearly understand understand that, forthat, for the convenience the convenience and and brevity of brevity of description, description, the the specific specificworking process of working process of the the above-described above-describedstorage storagedevices devicesandand processing devices, and the relevant explanations, may refer to the corresponding processes in the processing devices, and the relevant explanations, may refer to the corresponding processes in the

foregoing methodembodiments, foregoing method embodiments, which which willwill not not be repeated be repeated here. here.

[0173] Thoseskilled

[0173] Those skilled in in the the art art should should appreciate appreciate that thatthe themodules modules and methodsteps and method stepsdescribed describedinin the embodiments the embodiments disclosed disclosed herein herein can can be implemented be implemented using electronic using electronic hardware, hardware, computer computer software, or aa combination software, or combinationofofboth. both.The The programs programs corresponding corresponding to software to the the software modules modules and and methodsteps method stepscan canbe be stored stored in in random accessmemory random access memory (RAM), (RAM), memory, memory, read-only read-only memorymemory (ROM), (ROM), electrically electricallyprogrammable ROM, programmable ROM, electricallyerasable electrically erasableprogrammable programmable ROM,ROM, registers, registers, hard hard disks, disks,

removabledisks, removable disks,CD-ROMs, CD-ROMs, or other or any any other form form of storage of storage medium medium known known in in the the art. art. To To clearly clearly illustrate illustrate the the interchangeability interchangeability of of electronic electronic hardware hardware and software, and software, the composition the composition and steps of and steps of

each examplehave each example have been been generally generally described described basedbased on functionality on functionality in theinabove the above description. description.

Whetherthese Whether thesefunctions functionsare areexecuted executedininthe theform formofofelectronic electronichardware hardwareororsoftware software depends depends on on the specific application of the technical solution and the design constraints. The skilled person in the specific application of the technical solution and the design constraints. The skilled person in

the field may implement the described functions of each specific application in different ways, but the field may implement the described functions of each specific application in different ways, but

47 such implementation such implementation should should not not be interpreted be interpreted as aas a departure departure from from the of the scope scope the of the present present

disclosure. disclosure.

[0174] Theterms

[0174] The terms"comprise", “comprise”,"include" “include”ororany anyother othersimilar similarlanguage languageareareintended intendedtotocover covernon- non- exclusive inclusion, exclusive inclusion, such such thatthat a process, a process, method, method, article, article, or device/apparatus or device/apparatus that includes that includes a series a series

of elementsnotnot of elements only only includes includes those those elements elements butincludes but also also includes other elements other elements notlisted, not expressly expressly listed, or also includes or also includeselements elements inherent inherent to process, to the the process, method, method, article,article, or device/apparatus. or device/apparatus.

[0175] Heretofore,

[0175] Heretofore, thethe technical technical solutions solutions of theofdisclosure the disclosure have have been been described described withtoreference to with reference

the preferred the preferred implementations shown implementations shown in in thethe accompanying accompanying drawings. drawings. However, However, those skilled those skilled in in the art can readily understand that the scope of protection of the disclosure is apparently not limited the art can readily understand that the scope of protection of the disclosure is apparently not limited

to these to these specific specific implementations. Withoutdeparting implementations. Without departingfrom fromthe theprinciples principlesofofthe thedisclosure, disclosure, those those skilled skilled in inthe theart artmay maymake equivalent changes make equivalent changesororreplacements replacementstotorelated related technical technical features. features. The The

technical solutions obtained after these changes or replacements fall within the scope of protection technical solutions obtained after these changes or replacements fall within the scope of protection

of the disclosure. of the disclosure.

Claims (23)

48 CLAIMS CLAIMS

1. 1. A workingface A working face human-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and earlyand early

warningsystem, warning system,comprising: comprising: a working a working faceface mobile mobile equipment equipment safetysafety management management and and control control system, workingface system, working facecollaborative collaborativeintelligent intelligent wearable wearable devices, devices, an an information sharing platform, information sharing platform, and aa working and faceoutput working face outputcontrol control subsystem, subsystem,

wherein the wherein the working working face face mobile mobile equipment equipment safety safety management managementand andcontrol controlsystem systemisis configured totoperform configured perform real-time real-time perception perception of aofposition a position and and an an orientation orientation of eachofsingle each single equipment unit, equipment unit, perception perception ofoperational of an an operational statusstatus ofsingle of each each single equipment equipment unit, perception unit, perception of an of an environment within aa preset environment within preset range range surrounding surrounding each each single single equipment equipment unit, unit, and and behavior behavior acquisition and analysis for a driver of each single equipment unit; acquisition and analysis for a driver of each single equipment unit;

each ofthe each of theworking workingfaceface collaborative collaborative intelligent intelligent wearable wearable devices devices (13) comprises (13) comprises a portable a portable

positioning terminal positioning terminal (31) (31) and anda amotion motion monitoring monitoring device device (36) (36) that that are attached are attached to a to a worker, worker,

whereinthe wherein theportable portablepositioning positioningterminal terminal(31) (31)isisconfigured configuredto toperceive perceive relativepositions relative positionsandand relative orientations relative orientations between theworker between the workerandand each each single single equipment equipment unita and unit and a position position of a of a workingface, working face, and andthe the motion motionmonitoring monitoring device device (36) (36) is isconfigured configuredtoto acquiremotion acquire motion postures postures of of

the worker in real time; the worker in real time;

the information the informationsharing sharingplatform platform is is configured configured to collect to collect information information from from each single each single

equipment unitand equipment unit andeach eachworking working face face collaborativeintelligent collaborative intelligentwearable wearabledevice device(13) (13)and andachieve achieve information sharing; and information sharing; and

the working the face output working face outputcontrol control subsystem subsystemisisconfigured configuredtotoreceive receiveshared sharedinformation informationfrom from the information the sharing platform, information sharing platform, perform performentry entryand andexit exitidentification identification management management of of workers, workers,

position recognition position recognition and andmonitoring monitoringof of thethe workers workers and and each each single single equipment equipment unit, behavior unit, behavior

recognition, monitoring recognition, andprediction monitoring and predictionof of the the workers workersand andworking working environment environment monitoring, monitoring, and and generate generate corresponding alarm instruction corresponding alarm instruction information, information,wherein wherein the the working environments working environments

comprise gasconcentrations comprise gas concentrationsof of gas, gas, carbon monoxide,and carbon monoxide, andoxygen, oxygen, roof roof displacement, displacement, wind wind speed speed

and wind and windpressure. pressure.

2. The 2. workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and and early early warning system warning system according according to to claim claim 1, 1, wherein wherein the the working workingface facemobile mobileequipment equipment safety safety

management management andand control control system system comprises comprises a wireless a wireless positioning positioning terminal terminal (26), (26), a camera a camera device, device,

49

aa driving driving behavior analysis device behavior analysis (25), and device (25), and a a multi-source multi-source sensor (24) that sensor (24) that are are provided provided on on each each

single single equipment unit; equipment unit;

the wireless the wireless positioning positioning terminal terminal (26) (26)isis configured configuredtotoperform perform real-time real-time perception perception of aof a position and an orientation of each single equipment unit; position and an orientation of each single equipment unit;

the camera the deviceis camera device is configured configured to to capture capture environmental environmentalvideo videoimages imagesin in realtime real timewithin withina a preset range preset range surrounding eachsingle surrounding each single equipment equipmentunit unitand andtotoperform performenvironmental environmental perception; perception;

the driving the driving behavior behavioranalysis analysisdevice device(25) (25)isisconfigured configured to to capture capture driver driver behavior behavior video video

images in real images in real time time and and analyze driver behaviors; analyze driver behaviors; and and

the the multi-source sensor(24) multi-source sensor (24)isis configured configuredtotoacquire acquireananoperational operationalstatus statusofofeach each single single

equipment unit equipment unit in in real real time. time.

3. The 3. workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and and early early warning system warning system according according to to claim claim 2, 2, wherein wherein the the working workingface face output output control control subsystem subsystem comprises comprises a awireless wirelesspositioning positioning basebase station, station, a wireless a wireless network network transmission transmission device, device, an an equipment control equipment control device, device, a voice a voice alarm alarm device,device, and a centralized and a centralized control(15); control platform platform (15);

the wireless positioning base station is configured to cooperate with the wireless positioning the wireless positioning base station is configured to cooperate with the wireless positioning

terminal (26) terminal (26) of of each each single single equipment equipmentunit unitandand thethe portable portable positioning positioning terminal terminal (31) (31) of of each each

workertoto perform worker performperception perceptionofofthe thepositions positions and andorientations orientations of of each each single single equipment equipmentunit unitand and each worker; each worker;

the wireless the wireless network transmission device network transmission device is is configured configured to to enable enable communication between communication between thethe

working face collaborative single equipment units, the wireless positioning terminals (26), and the working face collaborative single equipment units, the wireless positioning terminals (26), and the

working face collaborative intelligent wearable devices (13) and the information sharing platform; working face collaborative intelligent wearable devices (13) and the information sharing platform;

the equipment the equipmentcontrol controldevice deviceisisconfigured configured to to determine determine a hazardous a hazardous statestate of each of each single single

equipment unitandand equipment unit each each worker worker by analyzing by analyzing and processing and processing data, entry data, perform perform and entry exit and exit

identification identification management management ofofworkers, workers, positionrecognition position recognition andand monitoring monitoring of each of each worker worker and and

each single equipment each single equipmentunit, unit,behavior behaviorrecognition, recognition,monitoring monitoringandand prediction prediction of of theworkers the workers andand

workingenvironment working environment monitoring, monitoring, output output corresponding corresponding control control signals signals based based on on control control strategies strategies

to control each device to perform corresponding risk avoidance actions, and transmit the generated to control each device to perform corresponding risk avoidance actions, and transmit the generated

alarm instructioninformation alarm instruction information to centralized to the the centralized control control platform platform (15); (15);

the the voice alarm device voice alarm deviceisis configured configuredto, to,upon uponreceiving receivingthethealarm alarm instructioninformation, instruction information, performalarms perform alarmsofofcorresponding correspondingcategories categoriesbased basedonon differentcategories different categoriesofofthe the alarm alarminstruction instruction information; and information; and

50

the centralized control platform (15) is configured to, based on the shared information of the the centralized control platform (15) is configured to, based on the shared information of the

information sharing platform, information sharing platform,perform performentry entryand andexit exitidentification identification management management of of thethe workers, workers,

position recognition position recognition and andmonitoring monitoringofofeach each worker worker and and each each single single equipment equipment unit, behavior unit, behavior

recognition, monitoring, recognition, monitoring, and prediction of and prediction of the the workers, workers, and and working environmentmonitoring, working environment monitoring, and and

generate correspondingalarm generate corresponding alarminstruction instructioninformation. information.

4. The 4. workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and and early early warningsystem warning systemaccording according to to claim claim 3, 3, wherein wherein modes modes for for the the worker worker entry entry intointo and and exitexit fromfrom the the

workingface working facecomprise: comprise:a anormal normaloperation operationmode, mode, a shifthandover a shift handover mode, mode, a linkage a linkage control control mode, mode,

and and aa temporary authorizationmode. temporary authorization mode.

5. 5. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim4,4,wherein claim whereinthethenormal normal operation operation mode mode is defined is defined as:as: N; Na;

an entry and an entry and exit exit management managementdevice deviceatatthetheworking working face face mobile mobile equipment equipment safety safety

management management andand control control system system performs performs identity identity recognition recognition for a for a current current nath worker, nath worker, and and records an records an entry entry time time T andananexit T inand exittime timeTTof out of thethe worker worker in and in and from from the the working working face; face;

in in aa case case where where nna N, ≤ Nthe a, the current current worker worker is permitted is permitted to enter to enter thethe working working face; face; and and in ain a

case case where where nna> >N,Nthe a, the currentworker current workeris isprohibited prohibited from fromentering entering the the working working face, face, and and overstaffing alarm instruction information is generated, where n is an integer greater than 0; and a overstaffing alarm instruction information is generated, where n is an integer greater than 0; and

in in a a case where case where an an entry entry duration duration Tw ofTw ofcurrent the the current workerworker at the working at the working face isthan face is greater greater than T y, timeout Ty, instruction information timeout instruction information for for the the worker workerisisgenerated, generated,where where Ty aispreset Ty is a preset working working

duration, duration, T Tww= = Tout-Tin.

6. 6. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim4,4,wherein claim whereinthetheshift shifthandover handovermode mode comprises: comprises:

aa current-shift current-shift manager anda anext-shift manager and next-shiftmanager manager perform perform handover handover confirmation confirmation via thevia the

workingface working facecollaborative collaborativeintelligent intelligent wearable devices (13) wearable devices (13) and and the the working workingface faceoutput outputcontrol control subsystem; and subsystem; and

the next-shift the next-shift manager inputsidentities manager inputs identities of of next-shift next-shift workers into aa control workers into control system systemofofthe the workingface working faceoutput outputcontrol controlsubsystem subsystemandand identifies identifies theworkers the workers at at thethe working working face, face, and and in ain a case case where the current where the current worker workerdoes doesnot notbelong belongtotothe the next-shift next-shift workers, identity mismatch workers, identity alarm mismatch alarm

instruction information instruction information is generated. is generated.

7. 7. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim4,4,wherein claim whereinthethelinkage linkagecontrol controlmode mode comprises: comprises:

51

when the when the camera camera device device of of the the working face mobile working face equipment safety mobile equipment safety management and management and

control control system detects that system detects that aa person person not not belonging to the belonging to the current current working faceintends working face intends to to intrude intrude without authorization, without authorization, climb climbover overororbypass bypassthe theentry entryand and exitmanagement exit management device, device, continuous continuous

unauthorizedentry unauthorized entry alarm alarminstruction instruction information information is is generated; generated;

when aaworker when workerpasses passesthrough throughthe theentry entry and andexit exit management management device,identity device, identity cross- cross- verification is performed based on the working face collaborative intelligent wearable device (13) verification is performed based on the working face collaborative intelligent wearable device (13)

of of the the worker and visual worker and visual recognition recognition information information from fromthe theworking workingface facemobile mobile equipment equipment safety safety

management management andand control control system, system, and and in aincase a case where where the identity the identity cross-verification cross-verification fails,alarm fails, alarm instruction instruction information information indicating indicating worker worker identity identity mismatch is generated; mismatch is generated;

whena amanager when manager intends intends to to accessthethesystem, access system,identity identitycross-verification cross-verification is is performed based performed based

on the working on the workingface facecollaborative collaborativeintelligent intelligentwearable wearable device device (13) (13) of the of the manager manager and visual and visual

recognition information recognition information from fromthe theworking workingface facemobile mobile equipment equipment safety safety management management and control and control

system, andininaacase system, and casewhere wherethethe identitycross-verification identity cross-verificationfails, fails, alarm alarminstruction instruction information information indicating indicating manager identity mismatch manager identity mismatchisisgenerated; generated;

whenaaworker when workerfrom fromanyany working working face face passes passes through through the the entry entry andand exit exit management management device, device,

all equipment all at the equipment at the working face is working face is powered off; powered off;

whena aworker when worker from from any any working working face enters face enters the working the working face, face, the the working working face face output output control subsystem control performsworking subsystem performs working environment environment monitoring; monitoring; and and

whenenvironmental when environmental parameters parameters failfail to to meet meet safety safety requirements requirements for for thethe worker worker to perform to perform

production operations, production operations, the the entry entry and and exit exitmanagement deviceopens management device opensananemergency emergency escape escape passage. passage.

8. 8. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim4,4,wherein claim whereinthethetemporary temporary authorization authorization mode mode comprises: comprises:

whena atemporary when temporary worker worker not belonging not belonging to thetocurrent the current working working facetoneeds face needs enter to theenter the workingface, working face,aamanager manager performs performs temporary temporary authorization authorization via working via the the working face output face output control control

subsystem anddefines subsystem and definesaatemporary temporaryentry entryduration durationasastn; tn ;

in in aa case case where an entry where an entry duration 𝑡𝑛′ ofofthe duration t' the current current temporary temporaryworker workeratatthe theworking working faceisis face

greater thant,tn,timeout greater than timeout alarm alarm instruction instruction information information for thefor the temporary temporary worker isworker is generated, generated, where where 𝑡t'𝑛′ represents represents the the entry entry duration duration of of the thecurrent currenttemporary temporary worker at the worker at the working face; and working face; and

a total a totalnumber of temporary number of workersand temporary workers andcurrent currentworkers workersentering enteringthe theworking working face face does does notnot

exceed Na,where exceed N, whereN N isa is a a maximum maximum allowable allowable number number of current-shift of current-shift workers workers at working at the the working face.face.

52

9. The 9. workingface The working facehuman-machine human-machine collaborative collaborative operation operation safetysafety monitoring monitoring and and early early warningsystem warning systemaccording accordingtotoclaim claim3,3,wherein whereinaasafety safety control control strategy strategy of ofthe theworking working face facehuman- human-

machinecollaborative machine collaborativeoperation operationsafety safety monitoring monitoringand andearly earlywarning warning system system comprises comprises a normal a normal

operation control strategy, operation control strategy, aahuman-machine human-machine mutual mutual avoidance avoidance control control strategy, strategy, a wireless a wireless

shutdown controlstrategy, shutdown control strategy, aa post post monitoring monitoring control control strategy, strategy,and anda special personnel a special management personnel management

strategy. strategy.

10. 10. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim9,9,wherein claim whereinthethenormal normal operation operation control control strategycomprises: strategy comprises:

an electronic fence an electronic fence range range is is set, set,comprising comprising aaworking area, an working area, an alarm alarm area, area, aa shutdown area, shutdown area,

and and aasafety safetyarea, area,and andthethestates statesofofthetheworkers workers located located in different in different ranges ranges areasset are set as a working a working state, state,

aa hazardous state,anan hazardous state, off-duty off-duty state, state, and and a safe a safe state, state, respectively; respectively;

whenother when otherworkers workersapproach approach thethe working working areaarea of the of the single single equipment equipment unit,unit, or workers or workers of of the single the single equipment unit approach equipment unit approachthe theworking working area area of of other other single single equipment equipment units, units, imminent imminent

proximityhazard proximity hazardalarm alarminstruction instruction information informationisis generated; generated;

whenother when otherworkers workersenter enterthetheworking working area area of of thethe single single equipment equipment unit, unit, or or workers workers of the of the

single equipment single equipment unit unit enter enter the the working working area area of of single other other single equipment equipment units, hazardous units, hazardous area entry area entry

alarm instruction information is generated; and alarm instruction information is generated; and

the working the faceoutput working face outputcontrol controlsubsystem subsystem performs performs behavior behavior identification identification of of a worker a worker in in the working the area, and working area, andinina acase casewhere wherethethe worker worker extends extends the the headhead and/or and/or limbslimbs beyond beyond a cab a cab protection area, protection area, or or exhibits exhibits fatigue fatigueor oryawn, yawn, abnormal behavioralarm abnormal behavior alarminstruction instructioninformation informationisis generated. generated.

11. 11. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning systemaccording according to claim to claim 9, wherein 9, wherein the human-machine the human-machine mutual avoidance mutual avoidance control control strategy strategy comprises: comprises:

aa worker workernot notbelonging belongingto tothethecurrent currentsingle singleequipment equipment unitunit is defined is defined as aaspedestrian, a pedestrian, a a

workercontrolling worker controllingthe thecurrent currentsingle single equipment equipment unittotomove unit move is defined is defined as aasdriver, a driver, andand other other

workers operating the current single equipment unit are defined as operators; workers operating the current single equipment unit are defined as operators;

whenthe when thepedestrian pedestrianneeds needstotoenter enteraaprotection protectionrange rangeofofthe thecurrent currentsingle single equipment equipmentunit, unit, the working the facecollaborative working face collaborativeintelligent intelligent wearable device(13) wearable device (13)ofofthe the pedestrian pedestriansends sendsananentry entry request to request to the theworking working face face mobile mobile equipment safety management equipment safety management andand control control system system of of thethe driver; driver;

53

upon receiving the request, the driver pauses a personnel proximity protection function of the upon receiving the request, the driver pauses a personnel proximity protection function of the

equipment; and equipment; and

in in a a case where case where an an actual actual protection protection function function pause pause time tr time tr is greater is greater than aprotection than a preset preset protection function pausetime function pause timetr, tR,ororwhere wherethethe working working face face output output control control subsystem subsystem detectsdetects that the that the

pedestrian has safely passed, the single equipment unit automatically resumes a protection function. pedestrian has safely passed, the single equipment unit automatically resumes a protection function.

12. 12. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim9,9,wherein claim whereinthethewireless wirelessshutdown shutdown control control strategy strategy comprises: comprises:

when the when the working workingface faceoutput output control control subsystem subsystem or or the the worker worker detects detects an an emergency emergency condition, an emergency condition, an emergency shutdown shutdown signal signal is sent is sent via via a wireless a wireless terminal, terminal, and the and all all wireless the wireless positioning terminals positioning terminals(26) (26)within within a signal a signal coverage coverage area immediately area immediately control control each each single single equipment unitto equipment unit to shut shut down; down;and and

in in a a case whereanan case where actual actual shutdown shutdown duration duration twq is tgreater wq is greater than a than a preset preset shutdownshutdown duration tws, duration tws,

or or upon inspecting aa working upon inspecting workingcondition conditionand andconfirming confirming thethe safetyofofeach safety eachsingle singleequipment equipment unit, unit,

aa designated managerresumes designated manager resumes a working a working state state ofof eachsingle each singleequipment equipment unit. unit.

13. 13. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warningsystem warning systemaccording accordingtoto claim9,9,wherein claim whereinthethepost postmonitoring monitoring controlstrategy control strategycomprises: comprises:

in in aa case case where where the the number of operators number of operators for for aa single singleequipment unit exceeds equipment unit exceeds a a preset preset number, number,

all equipment at the working face is prohibited from being started; all equipment at the working face is prohibited from being started;

in in aa case case where the current where the current single single equipment unitenters equipment unit entersaa protection protection range rangeofofaa designated designated single equipmentunit single equipment unitfor forcollaborative collaborativeoperations, operations,neither neitherofofthethetwotwo single single equipment equipment unitsunits

generates any alarm generates any alarminstruction instruction information; and information; and

in in a a case where case where oneone of the of the operators operators ofcurrent of the the current single single equipment equipment unit unit leaves theleaves the range of range of

the single the single equipment equipmentunit unitand andthen thenre-approaches re-approaches the the single single equipment equipment unit,unit, alarmalarm instruction instruction

information information is is generated generated and and the alarm the alarm devicedevice is controlled is controlled to issueto a issue a corresponding corresponding alarm. alarm.

14. 14. The workingface The working facehuman-machine human-machine collaborative collaborative operation operation safety safety monitoring monitoring and early and early

warning system warning system according according to to claim claim 9, 9, wherein wherein the the special special personnel personnel management managementstrategy strategy comprises: comprises:

whena asingle when singleequipment equipmentunit unitrequires requiresmaintenance, maintenance,thethe manager manager inputs inputs a maintenance a maintenance task task

into the system into the systemviavia thethe working working face face collaborative collaborative intelligent intelligent wearable wearable device device (13), (13), and the and the single single

equipment unitto equipment unit to be be maintained maintainedstops stopsoutputting outputtingalarm alarmand andcontrol controlsignals; signals;

54

the system the sets aa maintenance system sets duration,and maintenance duration, andwhen when maintenance maintenance personnel personnel enter enter a protection a protection

range of range of the the single single equipment unit to equipment unit to be be maintained, maintained, the the system systemoutputs outputsrelated related voice voiceprompts promptstoto remindthe remind the maintenance maintenancepersonnel personnel toto payattention; pay attention;and and

whenthe when themaintenance maintenance duration duration exceeds exceeds a presetmaintenance a preset maintenance duration, duration, or or thethemanager manager sends sends

aa maintenance completion maintenance completion instructiontotothe instruction thesystem, system,thethesingle singleequipment equipment unit unit to to be be maintained maintained

resumesoutputting resumes outputtingalarm alarmand andcontrol controlsignals. signals.

15. 15. An imagerecognition An image recognitionmethod methodfor fora aworking workingface facehuman-machine human-machine collaborative collaborative operation operation

safety safety monitoring monitoring and early warning and early warning system, system, based based ononthe theworking workingface facehuman-machine human-machine collaborative operation collaborative operation safety safetymonitoring monitoring and and early early warning warning system accordingto system according to any any one one of of claims claims

33 to to 14, 14, comprising comprising thethe following following steps: steps:

acquiring video acquiring video images imagescaptured capturedbybythe thecamera cameradevice deviceatatthe theworking workingface facemobile mobile equipment equipment

safety safety management and management and control control system system as as training training data,and data, andusing using worker worker images images annotated annotated withwith

personnel information as training data labels; personnel information as training data labels;

performingunsupervised performing unsupervisedtraining trainingand anddeep deeplearning learningononimproved improved YOLOV5 YOLOV5 byof by means means the of the training data and the corresponding training data labels to obtain an image recognition model; and training data and the corresponding training data labels to obtain an image recognition model; and

performingvideo performing videoimage image recognition recognition by means by means oftrained of the the trained imageimage recognition recognition model model to to completeidentity complete identityrecognition, recognition,target targettracking, tracking,behavior behaviorprediction predictionforfor workers, workers, and and working working

environmentrecognition. environment recognition.

16. 16. The image recognition The image recognition method method for for the the working workingface face human-machine human-machine collaborative collaborative

operation operation safety safety monitoring and early monitoring and early warning warningsystem systemaccording accordingtotoclaim claim15, 15,wherein wherein thetraining the training data labelscomprise data labels comprise posture posture images images of workers of workers with different with different postures, postures, at different at different shooting shooting angles, angles, and underdifferent and under different shooting shooting categories, categories, and facial and facial imagesimages of the in of the workers workers in different different states; states;

the postures the comprisestanding, postures comprise standing,squatting, squatting, bending bendingdown, down, left-sidestanding, left-side standing,and andright-side right-side standing; standing;

the shooting the shooting angles angles comprise top-downlong-range comprise top-down long-range shooting, shooting, top-down top-down short-range short-range shooting, shooting,

bottom-uplong-range bottom-up long-rangeshooting, shooting,bottom-up bottom-up short-range short-range shooting, shooting, frontallong-range frontal long-range shooting,andand shooting,

frontal short-rangeshooting; frontal short-range shooting;

the shooting the shooting categories categoriescomprise: comprise:visible visiblelight lighthigh-definition high-definitionshooting, shooting,low-illumination low-illumination supplementary lightingshooting, supplementary lighting shooting,and andthermal thermalimager imager(27) (27)shooting; shooting;and and

55

the different the different states states of of the workerscomprise: the workers comprise:an an unoccluded unoccluded state, state, a state a state of wearing of wearing a a protective mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing protective mask, a state of wearing a safety helmet, a state of wearing earplugs, a state of wearing

earmuffs, earmuffs, aa state state of of wearing wearing aa protective protective mask maskand anda safety a safetyhelmet, helmet, a stateofofwearing a state wearing a safety a safety

helmet and earplugs, a state of wearing a safety helmet and earmuffs, a state of wearing a protective helmet and earplugs, a state of wearing a safety helmet and earmuffs, a state of wearing a protective

maskand mask andearplugs, earplugs,a state a stateofofwearing wearing a protective a protective maskmask and earmuffs, and earmuffs, a statea of state of wearing wearing a a protective mask, protective mask, aa safety safety helmet andearplugs, helmet and earplugs, and andaastate state of of wearing wearing aa protective protective mask, mask,aa safety safety helmet and helmet andearmuffs. earmuffs.

17. 17. The image recognition The image recognition method method for for the the working workingface face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and earlywarning warningsystem system according according to to claim claim 15,15, wherein wherein the the image image

recognition model recognition modeladjusts adjusts anchor anchorboxes boxesbybymeans meansof of a a K-means K-means clustering clustering algorithm, algorithm, andand optimal optimal

cluster cluster centers centers of ofthe theK-means clustering algorithm K-means clustering are obtained algorithm are obtained by by partitioning partitioning training training samples samples

and calculating fitness and calculating fitnessby by means of aa swarm means of intelligence optimization swarm intelligence optimization method. method.

18. 18. The image recognition The image recognition method method for for the the working workingface face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and early warning warningsystem system according according to to claim claim 17,17, wherein wherein thethe swarm swarm

intelligence intelligence optimization optimization method comprises: method comprises:

initializing a particle initializing a particle swarm swarmandand randomly randomly generating generating N particles, N particles, where where 1 <N<n, < <1 n<is N < n, n is the the

numberofofanchor number anchor boxes boxes to be to be classified; classified; andand partitioning partitioning thethe n anchor n anchor boxes boxes by taking by taking the Nthe N particles as cluster centers; particles as cluster centers;

calculating and calculating normalizingdistances and normalizing distancesfrom from each each sample sample datadata point point to cluster to the the cluster centers, centers,

calculating and normalizing a distance between any two cluster centers, and calculating fitness of calculating and normalizing a distance between any two cluster centers, and calculating fitness of

the sample the data; and sample data; and

performingoptimization performing optimizationbased basedononthe thefitness fitness of of the the sample sample data data by by means of aa PSO means of algorithm PSO algorithm

to obtain the optimal cluster centers. to obtain the optimal cluster centers.

19. 19. The image recognition The image recognition method method for for the the working workingface face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and early warning warningsystem systemaccording according to to claim claim 18,wherein 18, wherein calculating calculating

and normalizingaadistance and normalizing distance from fromeach eachsample sampledata datapoint pointtotothe thecluster cluster center center comprises: comprises:

K

=

where D represents a distance from uth sample data A in a jth cluster ω of a sample dataset in where D represents a distance from uth sample data Au in a jth ucluster Wj of a sample j dataset

to a cluster center C , and K is the number of cluster centers; to a cluster center C, jand K is the number of cluster centers;

56

′ where D' where represents normalized 𝐷𝑖𝑛represents normalized D, Dinmax(Din) , max(Din)and andmin(Din) min(Din)represent represent aa maximum value maximum value

and and aa minimum value minimum value among among all all D, D in, respectively. respectively.

20. The 20. image recognition The image recognition method methodfor for the the working workingface face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and early warning warningsystem systemaccording according to to claim claim 19,wherein 19, wherein calculating calculating

and normalizingaadistance and normalizing distance between betweenany anytwo two clustercenters cluster centerscomprises: comprises:

D d(w,w) 1i, jN

whereDDrepresents where out represents a distance a distance between between any any two two cluster cluster centers, centers, Ci and C and Cj are C are cluster cluster centers centers

of of an ith cluster an ith cluster Wωand i and the the jthjth cluster cluster w, ω j, respectively, respectively, N is N is number the the number of particles of particles in the in the initialized initialized

particle swarm, and d(ω , ω represents an absolute value of a difference value between the cluster particle swarm, and d(w, wj)i represents j) an absolute value of a difference value between the cluster

centers centers C i and and C; C j; and and Doutis’ isnormalized Dout' normalized Doutand Dout, , and max(Doutand max(Dout) ) and min(Dout min(Dout) ) represent represent a maximum a maximum

value and value and aa minimum value minimum value among among all all Doutrespectively. Dout, , respectively.

21. The 21. image recognition The image recognition method methodfor for the the working workingface facehuman-machine human-machine collaborative collaborative

operation safety monitoring operation safety andearly monitoring and early warning warningsystem system according according to to claim claim 20,20, wherein wherein the the fitness fitness

of the sample of the sampledata data is is calculated calculated as follows: as follows:

F = 1 where F represents the fitness of the sample data. where F represents the fitness of the sample data.

22. The 22. image recognition The image recognition method methodfor for the the working workingface face human-machine human-machine collaborative collaborative

operation safety monitoring operation safety monitoringandand early early warning warning system system according according to 15, to claim claim 15, wherein wherein after after

57

performingimage performing imagerecognition recognitionononeach each frame frame of of a video a video captured captured by by thethe image image recognition recognition model, model,

the method the further comprises: method further comprises:

generating aa video generating trajectory for video trajectory foreach eachworker worker in in the theimage; image; and and

performingworker performing workertrajectory trajectorytracking trackingand andprediction predictionbased basedononthethevideo videotrajectory, trajectory,so soas as to to enable riskdetermination enable risk determination under under partially partially or fully or fully occluded occluded states states of of the worker. the worker.

23. A 23. A wireless wireless positioning positioning method for aa working method for face human-machine working face collaborative human-machine collaborative operation operation

safety safety monitoring monitoring and early warning and early warning system, system, based based ononthe theworking workingface facehuman-machine human-machine collaborative collaborative operation operation safety safetymonitoring monitoring and and early early warning warning system accordingto system according to any any one one of of claims claims

33 to to 14, 14, comprising comprising thethe following following steps: steps:

acquiring measurement acquiring measurement data data between between thethe wireless wireless positioning positioning terminal terminal (26) (26) andand thethe wireless wireless

positioning base station at a time t, and setting the wireless positioning base station as an original positioning base station at a time t, and setting the wireless positioning base station as an original

point to point to define define the the position position of of the the wireless wireless positioning positioning terminal terminal (26) (26) within within aa wireless wireless coverage coverage

area ofthe area of thewireless wireless positioning positioning basebase station station as (x,asyt), (xt,and yt),acquiring and acquiring distancedistance data sthe data S between between the wireless positioning terminal (26) and the wireless positioning base station; wireless positioning terminal (26) and the wireless positioning base station;

performing differential operations on the distance data s for a preset number of times to obtain performing differential operations on the distance data S for a preset number of times to obtain

a velocity v and an acceleration a of the wireless positioning terminal (26); a velocity v and an acceleration a of the wireless positioning terminal (26);

comparing and determining a velocity v and an acceleration a at the time t with a velocity vt- comparing and determining a velocity V and an t acceleration at at the time t t with a velocity V-

11 and anacceleration and an acceleration at-1the a at at time the time t-1: t-1:

when the variations in the velocity v and the acceleration a are within preset threshold ranges, when the variations in the velocity V andt the acceleration at are within t preset threshold ranges,

the position (x , y ) of the current wireless positioning terminal (26) is valid; and the position (x, t y) tof the current wireless positioning terminal (26) is valid; and

otherwise, based otherwise, based on on the the velocity velocity vt-1the V-1 and andacceleration the acceleration a at theat-1 timeat t-1 theand time t-1 and V- a velocity a velocity vt- 2 and 2 anacceleration and an accelerationat-2 at-2 at at the the time time t-2, t-2, predicting predicting aa predicted predictedvelocity 𝑣𝑡′ and velocityv't anda predicted a predicted acceleration 𝑎𝑡′ atat the acceleration a't thetime timet, t,andand based based on predicted on the the predicted velocity velocity ′ 𝑡 and v't and𝑣the the predicted predicted

acceleration a't,′ calculating a predicted position (x't, y't) ′ of′ the wireless positioning terminal (26) acceleration 𝑎𝑡 , calculating a predicted position (𝑥𝑡 , 𝑦𝑡 ) of the wireless positioning terminal (26) at at the the time t. time t.