CN113837059A - Patrol vehicle for advising pedestrians to wear mask in time and control method thereof - Google Patents
Patrol vehicle for advising pedestrians to wear mask in time and control method thereof Download PDFInfo
- Publication number
- CN113837059A CN113837059A CN202111104530.7A CN202111104530A CN113837059A CN 113837059 A CN113837059 A CN 113837059A CN 202111104530 A CN202111104530 A CN 202111104530A CN 113837059 A CN113837059 A CN 113837059A
- Authority
- CN
- China
- Prior art keywords
- mask
- module
- target
- information
- obstacle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 44
- 238000007689 inspection Methods 0.000 claims abstract description 63
- 238000010276 construction Methods 0.000 claims abstract description 19
- 238000000605 extraction Methods 0.000 claims abstract description 11
- 238000001514 detection method Methods 0.000 claims description 65
- 230000033001 locomotion Effects 0.000 claims description 34
- 230000008569 process Effects 0.000 claims description 17
- 230000007246 mechanism Effects 0.000 claims description 13
- 238000012549 training Methods 0.000 claims description 9
- 238000009616 inductively coupled plasma Methods 0.000 claims description 8
- 238000007781 pre-processing Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 claims description 5
- 238000011897 real-time detection Methods 0.000 claims description 4
- 238000012216 screening Methods 0.000 claims description 4
- 230000005540 biological transmission Effects 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012163 sequencing technique Methods 0.000 claims description 3
- 230000001629 suppression Effects 0.000 claims description 3
- 238000009223 counseling Methods 0.000 claims description 2
- 230000002265 prevention Effects 0.000 abstract description 16
- 238000012544 monitoring process Methods 0.000 abstract description 4
- 238000013527 convolutional neural network Methods 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 7
- 230000006399 behavior Effects 0.000 description 6
- 230000004888 barrier function Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 238000013507 mapping Methods 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 241000711573 Coronaviridae Species 0.000 description 1
- 241000700605 Viruses Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000003612 virological effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/165—Management of the audio stream, e.g. setting of volume, audio stream path
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
- G06Q10/047—Optimisation of routes or paths, e.g. travelling salesman problem
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Business, Economics & Management (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- General Engineering & Computer Science (AREA)
- Geometry (AREA)
- Economics (AREA)
- Computing Systems (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Strategic Management (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Game Theory and Decision Science (AREA)
- Quality & Reliability (AREA)
- Computer Graphics (AREA)
- Remote Sensing (AREA)
- Entrepreneurship & Innovation (AREA)
- Marketing (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- General Business, Economics & Management (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Navigation (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of monitoring, early warning and security, and particularly relates to a patrol vehicle for advising pedestrians to wear a mask in time and a control method thereof. Aiming at indoor public occasions, the construction of an indoor grid map is completed through RGB color images and depth images collected by a depth camera carried by a patrol inspection system; calculating the position of the inspection vehicle relative to the world coordinate system through feature matching between adjacent pictures to complete self positioning; by constructing a convolutional neural network model, the extraction and identification of the characteristics of whether the pedestrian wears the mask can be completed; the function of detecting that the inspection vehicle autonomously moves to a target is completed through a path planning navigation module and an obstacle avoidance module; finally, the inspection system completes the dissuading function of pedestrians not wearing the mask and the propaganda work of epidemic prevention knowledge through the target following module and the voice module.
Description
Technical Field
The invention belongs to the technical field of monitoring, early warning and security, and particularly relates to a patrol vehicle for advising pedestrians to wear a mask in time and a control method thereof.
Background
Since the discovery of new coronaviruses, wearing a mask in public places has become an effective means for preventing the spread of viruses, and is also the responsibility that every qualified citizen should comply with. In the intensive places of personnel such as railway station, market, cinema, except calling for everyone through the means of publicity and initiatively wearing the gauze mask, more need combine the development of present stage science and technology, the means through science and technology initiatively advises the personnel who do not wear the gauze mask in time to wear the gauze mask, reduces the possibility of viral propagation. Wear intellectual detection system tour car through indoor gauze mask, wear the real-time detection and the advice of gauze mask to passing personnel in public occasion, can effectual promotion gauze mask wear the efficiency of supervision, have important meaning to epidemic situation prevention and control work.
The design of the intelligent detection inspection vehicle system for indoor mask wearing needs to be supported by corresponding technologies to complete tasks such as autonomous positioning, map construction, autonomous navigation, target identification and the like. However, the intelligent detection patrol vehicle can automatically detect the behavior of the unworn mask and actively advise that the unworn mask cannot be effectively put into practical use, and the intelligent detection patrol vehicle is used in public places to develop and put into application a system which detects the behavior of the unworn mask and advises the behavior of the unworn mask as soon as possible so as to effectively assist the prevention and control work of epidemic situations.
Meanwhile, if a method that related workers supervise in public places is adopted, the number of the workers is large, and unnecessary conflicts with the workers without masks are easily caused. The intelligent detection patrol vehicle can carry out multi-target detection on personnel based on a deep learning method, acquire the position information of the target without wearing the mask, complete self position estimation through a simultaneous positioning and map building system, and autonomously navigate to the position of the target personnel through an optimal path planning algorithm to advise. Therefore, the manpower cost is saved for the development of epidemic prevention work, the propaganda and education work of epidemic prevention knowledge of people can be completed through the intelligent detection tour bus, the consciousness of people on safety and epidemic prevention is effectively improved, and the application prospect is important.
Disclosure of Invention
The invention aims to provide a patrol vehicle for dissuading pedestrians from wearing a mask in time.
The purpose of the invention is realized by the following technical scheme: the system comprises a map construction module, a patrol vehicle positioning module, a mask wearing detection module, a path planning navigation module, an obstacle avoidance module and a target following and voice reminding module;
the map construction module is used for constructing a map according to RGB image information and depth information acquired by shooting through a depth camera carried on the patrol vehicle, establishing a two-dimensional grid map according to acquired data during first cruising, and transmitting the established two-dimensional grid map to the path planning navigation module; the two-dimensional grid map comprises an idle area, an occupied area and an unknown area, wherein the idle area refers to an area through which the patrol vehicle can smoothly pass, the occupied area refers to an area blocked by an obstacle, and the unknown area refers to an area which is not searched by the patrol vehicle;
the patrol vehicle positioning module completes the positioning of the patrol vehicle through the conversion relation among a world coordinate system, a reference coordinate system and a camera coordinate system and transmits positioning information to the path planning navigation module;
the mask wearing detection module inputs image information acquired by shooting of a depth camera carried on the patrol vehicle into a trained mask detection network, performs mask detection, acquires position information of all people not wearing the masks, and transmits the position information of the people closest to the position information to the path planning navigation module;
the road stiffness planning navigation module carries out autonomous path planning based on an initial global map established by the map construction module, self-positioning information of the patrol vehicle positioning module and pedestrian position information which is transmitted by the mask wearing detection module and has the shortest distance, and transmits real-time information of the surrounding environment of the patrol vehicle and real-time data of the self position to the obstacle avoiding module and the voice reminding module;
the obstacle avoiding module patrols surrounding environment information acquired by a depth camera carried on the vehicle in real time, judges whether a dynamic obstacle is encountered in the global path planning process, if the existence of the dynamic obstacle is detected, transmits position information of the obstacle in a world coordinate system to the map building module and the path planning navigation module in real time, plans a local path through the path planning navigation module, and changes the optimal path of the global path planning to acquire the optimal local path planning after bypassing the obstacle;
after the inspection vehicle moves to the position of the target, the target following and voice reminding module carries out voice prompt of wearing the mask, and whether a target following mode is started or not is selected through real-time detection of whether the mask is worn by the target or not; if the target conforms to the prompt and actively wears the mask, the patrol vehicle automatically moves to the next target; if the target person does not comply with the prompt to wear the mask and tries to leave, the target following mode is started, the relative position information of the inspection tour vehicle and the target is obtained in real time through the depth camera, the target is followed in real time according to the relative position, the voice advising mode is switched to the voice commenting mode, and criticizing education is carried out on the target person through the voice module while the target person is followed.
The invention also aims to provide a patrol vehicle control method for advising pedestrians to wear the mask in time.
The purpose of the invention is realized by the following technical scheme: the method comprises the following steps:
step 1: establishing a two-dimensional grid map of a patrol area;
RGB color picture information and depth information acquired by a depth camera carried by a patrol car are transmitted to a front-end visual odometer, the motion of the camera between adjacent images and the appearance of a local map are calculated according to the estimation of the motion between the adjacent images, and the camera poses measured by the visual odometer at different moments and information detected by a loop are transmitted to a rear end for nonlinear optimization to obtain a globally consistent track and map; using loop detection to judge whether the patrol car reaches the previous position, and if loop detection is detected, providing information for back-end processing; the back end establishes a map model consistent with the task requirements according to the estimated track;
the two-dimensional grid map comprises an idle area, an occupied area and an unknown area, wherein the idle area refers to an area through which the patrol vehicle can smoothly pass, the occupied area refers to an area blocked by an obstacle, and the unknown area refers to an area which is not searched by the patrol vehicle;
step 2: positioning the inspection vehicle;
acquiring an initial position of the inspection vehicle on the two-dimensional grid map based on the established two-dimensional grid map; a depth camera carried by a patrol vehicle maps coordinate points in a three-dimensional world to a two-dimensional image plane through a pinhole camera model, ORB feature points of two acquired adjacent frames of images are extracted, a certain threshold value feature point is extracted for each frame of image, the descriptor distance of each feature point is measured and sequenced, and the closest one is taken as a matching point; according to the matched point pairs, the motion estimation problem between the two groups of 3D points is solved by using an ICP (inductively coupled plasma) method, so that the relative pose of the camera is obtained according to the images of the two adjacent frames, and the relative pose relation is mapped to a world coordinate system to obtain the real-time position information of the inspection vehicle;
and step 3: detecting wearing of the mask;
transmitting RGB (red, green and blue) color picture information acquired by a depth camera carried by a patrol vehicle into a trained mask detection network, acquiring the position information of pedestrians not wearing masks, and selecting the pedestrian position information with the shortest relative distance as a counseling target;
and 4, step 4: path planning navigation and obstacle avoidance;
establishing a global optimal path and issuing decision information corresponding to the optimal path to an executing mechanism of the inspection vehicle, driving the inspection vehicle to execute motion operation according to the planned optimal path through linear speed and angular speed instructions corresponding to the decision information, and feeding back linear speed and angular speed data actually executed by the inspection vehicle by an encoder carried by the executing mechanism in real time so as to continuously optimize the position of the inspection vehicle until the position and pose of the inspection vehicle are close to the pose issued by the decision information;
in the process that the inspection vehicle moves to a recommended target through path planning, acquiring position information of an obstacle through a depth camera carried by the outside, and avoiding the obstacle according to the position information of the obstacle; firstly, obtaining a distance value of a surrounding environment through the acquired depth information of the picture; comparing the distance value with an environment map distance value prestored in the first map construction, if the difference value between the distance value and the prestored environment map distance value is smaller than a set threshold value, judging that the distance value is an obstacle, and continuously detecting the state of the obstacle; judging the motion state of the obstacle according to the distance and the angle of the obstacle detected in the two times; when the detection system moves to the obstacle for a certain distance, carrying out obstacle avoidance movement by adopting a corresponding obstacle avoidance method according to the movement state of the obstacle until the position of the target person is finally reached;
and 5: target following and voice reminding;
after the inspection vehicle moves to a target position, voice broadcasting reminding is carried out, a target person is reminded to wear a mask, and the mask is monitored in real time; if the target person finishes wearing the mask within a certain time after the voice prompt, moving to the position of the next target person; prompting again if the target person does not finish wearing the mask after voice prompting; if the target person tries to leave after the voice prompt, target following is started, the characteristics of the identity of the followed target are acquired through the depth camera, the followed target is tracked in real time after being locked, meanwhile, the voice prompt is adjusted from a recommendation mode to a criticizing mode, and broadcasting is carried out once at certain intervals.
The present invention may further comprise:
the method for establishing the two-dimensional grid map in the step 1 specifically comprises the following steps:
step 1.1: setting the size of a map, the actual distance represented by each grid and a grid threshold;
step 1.2: reading in image data of a depth camera, and carrying out preprocessing operation on the data;
step 1.3: initializing, setting the whole grid area to be black and representing an unknown area;
step 1.4: according to the principle that the surrounding environment is detected from near to far, the detected area is set to be gray, and the gray represents the range which can be detected by the depth camera; counting the projection times of each grid by obstacles according to the numerical values of the pixel points, and when the times are greater than a set threshold value, determining that the corresponding grid map is occupied by the obstacles and setting the grid map as an occupied area; setting the area smaller than the threshold value as an idle area, which represents that the area can pass through smoothly;
step 1.5 step 1.4 is repeated until the two-dimensional grid map is established.
The training method of the mask detection network in the step 3 comprises the following steps:
firstly, preprocessing image data in a training process, and extracting labeling information of the used image data; sending the image data into a feature extraction network in batches, and acquiring different prediction feature layers generated by each image through the feature extraction network; transmitting different prediction feature layers into a regional generation network to generate candidate frames, sliding on the prediction feature layers by using a sliding window to generate a series of anchor points, performing classification prediction and boundary frame regression prediction on the different prediction feature layers through convolution layers, applying prediction results to the generated anchor points to obtain all candidate frame information, and screening the generated candidate frames according to a set threshold value through non-maximum suppression processing operation; finally, sequencing each characteristic layer of each picture according to confidence coefficients from low to high; transmitting the candidate frame information generated by the area generation network to the latter half of the network, firstly carrying out flattening operation, and obtaining final output through two full-connection layers, a classification predictor and a boundary frame regression predictor of the network; and then, carrying out reverse transmission on the output, continuously optimizing network parameters through a batch of image data, training to the set iteration times, and obtaining the optimal mask detection network model. .
The invention has the beneficial effects that:
aiming at indoor public occasions, the construction of an indoor grid map is completed through RGB color images and depth images collected by a depth camera carried by a patrol inspection system; calculating the position of the inspection vehicle relative to the world coordinate system through feature matching between adjacent pictures to complete self positioning; by constructing a convolutional neural network model, the extraction and identification of the characteristics of whether the pedestrian wears the mask can be completed; the function of detecting that the inspection vehicle autonomously moves to a target is completed through a path planning navigation module and an obstacle avoidance module; finally, the following module and the voice module complete the disseminating function of the inspection system to pedestrians who do not wear the mask and the propaganda work of epidemic prevention knowledge. The invention can effectively assist related workers to finish epidemic prevention work, not only saves the labor cost of epidemic prevention work investment, but also avoids the problem of unnecessary conflict caused by the fact that the workers advise pedestrians to wear the mask.
Drawings
Fig. 1 is a system architecture diagram of a patrol vehicle for advising pedestrians to wear a mask in time according to the present invention.
FIG. 2 is a flow chart of the map building module implementation of the present invention.
Fig. 3 is a flow chart of the patrol vehicle positioning module in the present invention.
Fig. 4 is a flow chart of the implementation of the mask wearing detection module according to the present invention.
FIG. 5 is a flow chart of the path planning navigation module according to the present invention.
Fig. 6 is a flow chart of an implementation of the obstacle avoidance module of the present invention.
FIG. 7 is a flow chart of an implementation of the object following module in the present invention.
FIG. 8 is a flow chart of an implementation of the voice prompt module of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
The invention provides an automatic patrol vehicle for advising pedestrians to wear a mask in time based on an autonomous navigation technology and a control method thereof based on the requirement of monitoring the behavior that the mask is not worn in an indoor public place in the current epidemic prevention and control work. Establishing an indoor grid map through video streams collected by a depth camera, and simultaneously completing the detection work of people who do not wear the mask by utilizing a depth learning network; performing path planning through a decision system, and issuing a motion instruction to an execution mechanism to finish autonomous navigation of a target position; meanwhile, the voice broadcasting module is used for disseminating pedestrians who do not wear the mask in indoor public places and disseminating epidemic prevention knowledge, so that disseminating of epidemic prevention knowledge is completed, technical support is provided for disseminating pedestrians to wear the mask in indoor public places, and great significance is brought to epidemic prevention work.
The utility model provides a coach pedestrian in time wears tour car of gauze mask, includes: the system comprises a map building module, a patrol vehicle positioning module, a mask wearing detection module, a path planning navigation module, an obstacle avoiding module, a target following module and a voice reminding module;
the map building module: and the system is responsible for constructing the indoor map according to the RGB image information and the depth information acquired by shooting by the depth camera carried on the inspection tour vehicle. When the system firstly navigates, an indoor grid map is established according to the acquired data, the grid map mainly comprises three parts, namely an idle area, an occupied area and an unknown area, wherein the idle area refers to an area which can be smoothly passed by the system, the occupied area refers to an area blocked by an obstacle, and the unknown area refers to an area which is not yet explored by the system. And establishing an indoor global map through the initial patrol of the system, and transmitting the established two-dimensional grid map to a path planning navigation module.
Tour car orientation module: and the self-positioning of the inspection system is completed through the conversion relation among the world coordinate system, the reference coordinate system and the camera coordinate system. The environment picture collected by the carried depth camera is mapped to a two-dimensional plane through a camera pinhole model principle to obtain a pose transformation relation of a camera coordinate system relative to a world coordinate system, and the relative position relation between a reference coordinate system and the camera coordinate system is fixed, so that the position relation between the reference coordinate system and the world coordinate system can be calculated through the position relation between the camera coordinate system and the world coordinate system and the position relation between the reference coordinate system and the camera coordinate system, the real-time positioning function of the inspection tour vehicle is completed, and the positioning information is transmitted to a path planning navigation module.
Detection module is worn to gauze mask: firstly, preprocessing image data in a training process, and extracting labeling information of the used image data; sending the image data into a feature extraction network in batches, and acquiring different prediction feature layers generated by each image through the feature extraction network; transmitting different prediction feature layers into a regional generation network to generate candidate frames, mainly sliding a 3 × 3 sliding window on the prediction feature layers to generate a series of anchor points, performing classification prediction and boundary frame regression prediction on the different prediction feature layers through 1 × 1 convolutional layers, applying prediction results to the generated anchor points to obtain all candidate frame information, screening the generated candidate frames according to a certain threshold value through non-maximum suppression processing operation, and finally only reserving 2000 candidate frames in each feature layer of each picture according to the confidence degree from low to high; transmitting the candidate frame information generated by the area generation network to the latter half of the network, firstly carrying out flattening operation, and obtaining final output through two full-connection layers, a classification predictor and a boundary frame regression predictor of the network; and then, carrying out reverse transmission on the output, continuously optimizing network parameters through a batch of image data, training to the set iteration times, and obtaining the optimal wearing mask detection network model. The method comprises the steps that image information obtained by shooting of a depth camera carried on a detection patrol car is subjected to mask detection on pedestrians in indoor public places through a mask detection network model obtained through training, and whether the pedestrians wear masks or not is judged by classifying the pedestrians wearing the masks and the pedestrians not wearing the masks. If pedestrians do not wear the mask in the shot picture, the position information of all the persons not wearing the mask is uploaded to the inspection tour bus, and the persons not wearing the mask are judged to be closest to the relative position of the inspection tour bus through analysis of the position information, so that the position information of the persons closest to the persons is transmitted to the path planning navigation module, and a target is provided for the next step of autonomous navigation of the inspection tour bus to the specified position for recommendation.
The road strength planning navigation module: and based on an initial global map established by the map construction module, self-positioning information of the patrol vehicle positioning module and pedestrian position information which is transmitted by the mask wearing detection module and is closest to the system, autonomous path planning is carried out and mobile decision data are issued in real time. Firstly, global path planning is started, and the optimal path moving from the current position to the target position is calculated mainly through a path planning algorithm; then, decision data such as angular velocity, linear velocity and the like required by system movement are sent to an execution mechanism through a decision mechanism; the execution structure controls the movement of the inspection vehicle according to the issued instruction, and feeds real movement data detected by the encoder back to the decision mechanism in real time to continuously optimize the position, thereby realizing the autonomous navigation function of the inspection system. And real-time information of the surrounding environment of the inspection vehicle and real-time data of the position of the inspection vehicle are transmitted to the obstacle avoiding module and the voice reminding module.
Obstacle avoidance module: and judging whether the system meets a dynamic barrier in the global path planning process or not based on the surrounding environment information acquired by the depth camera carried by the detection system in real time. And if the existence of the dynamic barrier is detected, transmitting the position information of the barrier in a world coordinate system to a map construction module and a path planning navigation module in real time, planning a local path through the path planning navigation module, and changing the optimal route of the global path planning to obtain the optimal local path planning after bypassing the barrier.
A target following module: after the detection system moves to the position of the target, voice prompt of wearing the mask is carried out, and whether a target following mode is started or not is selected through real-time detection of whether the target wears the mask or not. If the target person follows the prompt to actively wear the mask, the detection system automatically moves to the next target; the target following mode is enabled if the target person is not following the prompt to wear the mask and attempt to exit. And acquiring the relative position information of the inspection vehicle and the target in real time through the depth camera, and following the target in real time according to the relative position. And transmits the behavior of the target person escaping from the advice to the voice prompt module.
The voice reminding module: the detection patrol vehicle moves to the position of the target person through the target navigation module and the obstacle avoidance module, and carries out voice prompt of wearing a mask advise. Carry out pronunciation warning to the personnel who do not wear the gauze mask, if received the instruction that the target was followed the module and is issued, then switch over pronunciation advice mode to pronunciation criticize mode, follow the target personnel and carry out criticizing education through voice module to it simultaneously.
A control method of a patrol vehicle for advising pedestrians to wear a mask in time comprises the following steps:
(1) a map construction step: RGB color picture information and depth information acquired by a depth camera carried by the system are transmitted to a front-end visual odometer, and the motion of the camera between adjacent images and the appearance of a local map are calculated according to the estimation of the motion between the adjacent images; transmitting the camera pose measured by the visual odometer at different moments and the information detected by loop back to back-end nonlinear optimization to obtain a globally consistent track and map; using loop detection to judge whether the system reaches the previous position, if loop is detected, providing information for back-end processing; and the back end establishes a map model consistent with the task requirements according to the estimated track. And uploading the map model to a patrol vehicle positioning module and a path planning navigation module.
(2) And a positioning step of the inspection vehicle: based on a grid map established by a map building module, the initial position of a detection system on the grid map is obtained, a depth camera carried by a patrol vehicle is detected, coordinate points (unit is meter) in a three-dimensional world are mapped to a two-dimensional image plane (unit is pixel) through a pinhole camera model, ORB feature points of two adjacent frames of images are extracted, a certain threshold value feature point is extracted from each frame of image, the description sub-distance of each feature point is measured and sequenced, and the nearest one is taken as a matching point. And solving the motion estimation problem between two groups of 3D points by utilizing an ICP (inductively coupled plasma) method according to the matched point pairs, so as to acquire the relative pose of the camera according to the images of two adjacent frames, and mapping the relative pose relationship to a world coordinate system to obtain the real-time position information of the inspection tour vehicle. And uploading the position information to a path planning navigation module.
(3) A mask wearing detection step: transmitting RGB color picture information acquired by a depth camera carried by the system into a mask detection network; establishing a network structure of a feature extraction network, and setting an optimal network model weight parameter trained by the network; preprocessing operations such as normalization, tensor conversion, size adjustment and the like are carried out on each picture in the input port cover detection network; transmitting the preprocessed picture into a prediction model to obtain prediction results such as a target boundary box, a category label and a confidence score; and mapping the prediction result to the original image and drawing the category information and the confidence information of the prediction result. The method comprises the steps of detecting video streams through a neural network, obtaining position information of pedestrians not wearing masks, mainly comprising translation information and rotation information of the pedestrians not wearing the masks under a three-dimensional space coordinate, storing the information into a list, and selecting the pedestrian position information with the shortest relative distance as a recommendation target through circularly traversing relative position distance information between all the pedestrians not wearing the masks in the list and a detection patrol vehicle, and transmitting the pedestrian position information to a path planning navigation module.
(4) Path planning and navigation: packaging the initial global map data transmitted by the map construction module, the position data of the inspection vehicle in the global map transmitted by the inspection vehicle positioning module and the position data of the target pedestrian transmitted by the mask wearing detection module into a path planning navigation initialization function; establishing a global optimal path and issuing decision information corresponding to the optimal path to an execution mechanism, driving a detection system to execute motion operation according to the planned optimal path through linear velocity and angular velocity instructions corresponding to the decision information, and feeding back linear velocity and angular velocity data actually executed by the system through an encoder carried by the execution mechanism in real time so as to continuously optimize the position of the system until the pose issued by the decision information is approached; and finally moving to the position of the target pedestrian through repeated circulation, and uploading the information to the voice reminding module.
(5) Obstacle avoidance step: in the process that the inspection vehicle moves to the target personnel through path planning, the position information of the obstacle is acquired through a depth camera carried by the outside, and the obstacle is avoided according to the position information of the obstacle. Firstly, obtaining a distance value of a surrounding environment through the acquired depth information of the picture; comparing the distance value with an environment map distance value prestored in the first map construction, if the difference value between the distance value and the prestored environment map distance value is smaller than a set threshold value, judging that the distance value is an obstacle, and continuously detecting the state of the obstacle; judging the motion state of the obstacle according to the distance and the angle of the obstacle detected in the two times; and when the detection system moves to the obstacle for a certain distance, carrying out obstacle avoidance movement by adopting a corresponding obstacle avoidance method according to the movement state of the obstacle. Until finally reaching the location of the target person.
(6) A target following step: the detection tour car moves to the target position through the path planning navigation module to carry out voice broadcast reminding, and if the pedestrian still does not wear the mask and tries to leave, the target following module is started. Firstly, acquiring the characteristics of the identity of a followed target through a depth camera; after the following target is locked, the following target is tracked in real time, and the relative direction between the following target and the inspection tour vehicle is obtained; detecting the locked following target in real time, and calculating to obtain the relative distance between the following target and the inspection tour vehicle; determining a movement route, and decision information such as angular speed and linear speed of movement according to the acquired relative direction and relative distance between the inspection tour vehicle and the pedestrian target; and the following of the detection system to the target personnel is completed by sending the decision information to the executing mechanism.
(7) A voice reminding step: the detection system moves to the position of a target person through the path navigation module, the tour vehicle positioning module determines that the target person finally reaches the designated position, the voice broadcasting module is executed to remind the target person to wear the mask and monitor the mask in real time, and if the target person finishes the action of wearing the mask within a certain time after voice prompt, the detection system moves to the position of the next target person; prompting again if the target person does not finish wearing the mask after voice prompting; and if the target person tries to leave after voice prompt, the information is issued to the target following module, and when the target following module is started, the voice prompt module is adjusted from the recommendation mode to the criticizing mode and broadcasts once at a certain interval.
The invention has the beneficial effects that:
aiming at indoor public occasions, the construction of an indoor grid map is completed through RGB color images and depth images collected by a depth camera carried by a patrol inspection system; calculating the position of the inspection vehicle relative to the world coordinate system through feature matching between adjacent pictures to complete self positioning; by constructing a convolutional neural network model, the extraction and identification of the characteristics of whether the pedestrian wears the mask can be completed; the function of detecting that the inspection vehicle autonomously moves to a target is completed through a path planning navigation module and an obstacle avoidance module; finally, the following module and the voice module complete the disseminating function of the inspection system to pedestrians who do not wear the mask and the propaganda work of epidemic prevention knowledge. The invention can effectively assist related workers to finish epidemic prevention work, not only saves the labor cost of epidemic prevention work investment, but also avoids the problem of unnecessary conflict caused by the fact that the workers advise pedestrians to wear the mask.
Example 1:
as shown in figure 1, the invention mainly comprises: the system comprises a map building module, a patrol vehicle positioning module, a mask wearing detection module, a path planning navigation module, an obstacle avoiding module, a target following module and a voice reminding module.
The specific implementation process of each module is as follows:
1 map construction module
The implementation process of the module is shown in fig. 2:
(1) some parameters needed to be set for initializing the local map mainly refer to setting the size of the map and the size of the grids, namely the actual distance represented by each grid;
(2) reading in image data of a depth camera carried by a system, and carrying out preprocessing operation on the data;
(3) setting the whole grid area to be black, wherein the unknown area in three modes of the grid map is represented;
(4) according to the principle that the surrounding environment is detected from near to far, the detected area is set to be gray, and represents the range which can be detected by the camera;
(5) counting the projection times of each grid by obstacles according to the numerical values of the pixel points, and when the times are greater than a certain threshold value, considering that the corresponding grid map is occupied by the obstacles, setting the grid map as an occupied area, and setting an area smaller than the threshold value as an idle area to represent that the area can pass through smoothly;
(6) and (5) repeating the execution until the two-dimensional grid map is established, and uploading the established real-time two-dimensional grid map to a path planning navigation module.
2 tour car orientation module
The implementation process of the module is shown in FIG. 3:
(1) initializing a grid map established by a map building module;
(2) acquiring an initial position of the inspection vehicle on the grid map;
(3) an acquisition system external sensor depth camera maps coordinate points (in meters) in the three-dimensional world through a pinhole camera model to data of a two-dimensional image plane (in pixels).
(4) Extracting ORB characteristic points of the two adjacent frames of images, and screening the ORB characteristic points according to a certain threshold value;
(5) measuring the descriptor distance of each feature point, sequencing, taking the nearest one as a matching point, and matching the feature points between the two images;
(6) according to the matched point pairs, the motion estimation problem between two groups of 3D points is solved by using an ICP (inductively coupled plasma) method, and the relative pose of the camera is obtained according to the relative motion of two adjacent frames of images;
(7) and mapping the relative pose relation to a world coordinate system to obtain real-time position information of the inspection tour vehicle relative to the world coordinate system, and uploading the position information to a path planning navigation module.
3 mask wearing detection module
The implementation process of the module is shown in fig. 4:
(1) initializing parameters required by a mask detection network;
(2) transmitting RGB color picture information acquired by a depth camera carried by the system into a mask detection network;
(3) establishing a network structure of a feature extraction network, and setting an optimal network model weight parameter trained by the network;
(4) preprocessing each picture in the input port cover detection network such as normalization, tensor conversion, size adjustment and the like;
(5) transmitting the preprocessed picture into a prediction model to obtain prediction results such as a boundary box, a category label and a confidence score;
(6) and mapping the prediction result to the original image and drawing the category information and the confidence information of the prediction result.
(7) Circularly executing the steps (1) to (6), finishing the detection of the input video stream by the neural network, acquiring the position information of the pedestrian without wearing the mask, mainly comprising translation information and rotation information of the pedestrian under a three-dimensional space coordinate, and storing the information into a list;
(8) and selecting the pedestrian position information with the shortest relative distance to transmit to the path planning navigation module through circularly traversing the relative position distance information of all pedestrians not wearing the masks in the list and the inspection tour vehicle.
4 route planning navigation module
The implementation process of the module is shown in FIG. 5:
(1) initializing parameters required by path planning navigation;
(2) acquiring initial position information of a system detected by a patrol vehicle positioning module and determining the position information of a target pedestrian through a mask detection module;
(3) planning a global optimal path according to the initial map established by the map building module;
(4) the detection system moves according to the planned global optimal path, and an encoder corresponding to the execution system uploads the angular speed and linear speed information of the current system in real time;
(5) the information is compared with angular velocity and linear velocity information sent by a decision mechanism, negative feedback operation is carried out, and the system pose is continuously optimized, so that the expected pose is approached;
(6) and (4) and (5) are circulated until the detection system moves to the target pedestrian position.
5 obstacle avoiding module
The implementation process of the module is shown in fig. 6:
(1) initializing parameters required by obstacle avoidance of the obstacle;
(2) obtaining a distance value of the surrounding environment through the acquired depth information of the picture;
(3) comparing the distance value with an environment map distance value prestored in the first map construction;
(4) if the difference value between the distance value and the pre-stored distance value of the environment map is larger than a set threshold value, judging that the obstacle exists, and continuously detecting and refreshing the state of the obstacle;
(5) judging the motion state of the obstacle according to the distance and the angle of the obstacle detected in the two times;
(6) when the detection system moves to the obstacle for a certain distance, a corresponding obstacle avoidance method is adopted to carry out obstacle avoidance movement according to the movement state of the obstacle;
(7) and (5) circulating the steps (2) to (6) until the target person is finally moved to the position of the target person.
6 target following module
The implementation process of the module is shown in FIG. 7:
(1) initializing target following required parameters;
(2) after the voice broadcast reminding is carried out for a certain time, whether the pedestrian wears the mask or not is judged, and if the pedestrian still does not wear the mask and tries to leave, the target following module is started;
(3) acquiring and acquiring the characteristics of the following target personnel through a depth camera;
(4) after the following target is locked, the following target is tracked in real time, and the relative direction between the following target and the inspection tour vehicle is obtained;
(5) detecting the locked following target in real time, and calculating to obtain the relative distance between the following target and the inspection tour vehicle;
(6) determining decision information such as a movement route, a movement angular velocity and linear velocity and the like according to the obtained relative direction and relative distance between the inspection tour vehicle and the pedestrian target;
(7) and the following behavior of the inspection vehicle to the target personnel is detected by sending the decision information to an executing mechanism.
7 voice reminding module
The implementation process of the module is shown in fig. 8:
(1) initializing parameters required by voice reminding;
(2) judging whether the vehicle reaches a specified position or not through a patrol vehicle positioning module, if so, triggering voice broadcast, and if not, not triggering;
(3) monitoring the target in real time, and moving to the position of the next target person if the target person finishes wearing the mask within a certain time after voice prompt;
(4) prompting again if the target person does not finish wearing the mask after voice prompting;
(5) if the target person tries to leave after the voice prompt, the information is sent to a target following module;
the voice prompt module is adjusted from the recommendation mode to the criticizing mode and broadcasts once at intervals of a certain time.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (4)
1. The utility model provides a coach car of gauze mask is in time worn to advise pedestrian, its characterized in that: the system comprises a map construction module, a patrol vehicle positioning module, a mask wearing detection module, a path planning navigation module, an obstacle avoidance module and a target following and voice reminding module;
the map construction module is used for constructing a map according to RGB image information and depth information acquired by shooting through a depth camera carried on the patrol vehicle, establishing a two-dimensional grid map according to acquired data during first cruising, and transmitting the established two-dimensional grid map to the path planning navigation module; the two-dimensional grid map comprises an idle area, an occupied area and an unknown area, wherein the idle area refers to an area through which the patrol vehicle can smoothly pass, the occupied area refers to an area blocked by an obstacle, and the unknown area refers to an area which is not searched by the patrol vehicle;
the patrol vehicle positioning module completes the positioning of the patrol vehicle through the conversion relation among a world coordinate system, a reference coordinate system and a camera coordinate system and transmits positioning information to the path planning navigation module;
the mask wearing detection module inputs image information acquired by shooting of a depth camera carried on the patrol vehicle into a trained mask detection network, performs mask detection, acquires position information of all people not wearing the masks, and transmits the position information of the people closest to the position information to the path planning navigation module;
the road stiffness planning navigation module carries out autonomous path planning based on an initial global map established by the map construction module, self-positioning information of the patrol vehicle positioning module and pedestrian position information which is transmitted by the mask wearing detection module and has the shortest distance, and transmits real-time information of the surrounding environment of the patrol vehicle and real-time data of the self position to the obstacle avoiding module and the voice reminding module;
the obstacle avoiding module patrols surrounding environment information acquired by a depth camera carried on the vehicle in real time, judges whether a dynamic obstacle is encountered in the global path planning process, if the existence of the dynamic obstacle is detected, transmits position information of the obstacle in a world coordinate system to the map building module and the path planning navigation module in real time, plans a local path through the path planning navigation module, and changes the optimal path of the global path planning to acquire the optimal local path planning after bypassing the obstacle;
after the inspection vehicle moves to the position of the target, the target following and voice reminding module carries out voice prompt of wearing the mask, and whether a target following mode is started or not is selected through real-time detection of whether the mask is worn by the target or not; if the target conforms to the prompt and actively wears the mask, the patrol vehicle automatically moves to the next target; if the target person does not comply with the prompt to wear the mask and tries to leave, the target following mode is started, the relative position information of the inspection tour vehicle and the target is obtained in real time through the depth camera, the target is followed in real time according to the relative position, the voice advising mode is switched to the voice commenting mode, and criticizing education is carried out on the target person through the voice module while the target person is followed.
2. A patrol vehicle control method for advising pedestrians to wear a mask in time is characterized by comprising the following steps:
step 1: establishing a two-dimensional grid map of a patrol area;
RGB color picture information and depth information acquired by a depth camera carried by a patrol car are transmitted to a front-end visual odometer, the motion of the camera between adjacent images and the appearance of a local map are calculated according to the estimation of the motion between the adjacent images, and the camera poses measured by the visual odometer at different moments and information detected by a loop are transmitted to a rear end for nonlinear optimization to obtain a globally consistent track and map; using loop detection to judge whether the patrol car reaches the previous position, and if loop detection is detected, providing information for back-end processing; the back end establishes a map model consistent with the task requirements according to the estimated track;
the two-dimensional grid map comprises an idle area, an occupied area and an unknown area, wherein the idle area refers to an area through which the patrol vehicle can smoothly pass, the occupied area refers to an area blocked by an obstacle, and the unknown area refers to an area which is not searched by the patrol vehicle;
step 2: positioning the inspection vehicle;
acquiring an initial position of the inspection vehicle on the two-dimensional grid map based on the established two-dimensional grid map; a depth camera carried by a patrol vehicle maps coordinate points in a three-dimensional world to a two-dimensional image plane through a pinhole camera model, ORB feature points of two acquired adjacent frames of images are extracted, a certain threshold value feature point is extracted for each frame of image, the descriptor distance of each feature point is measured and sequenced, and the closest one is taken as a matching point; according to the matched point pairs, the motion estimation problem between the two groups of 3D points is solved by using an ICP (inductively coupled plasma) method, so that the relative pose of the camera is obtained according to the images of the two adjacent frames, and the relative pose relation is mapped to a world coordinate system to obtain the real-time position information of the inspection vehicle;
and step 3: detecting wearing of the mask;
transmitting RGB (red, green and blue) color picture information acquired by a depth camera carried by a patrol vehicle into a trained mask detection network, acquiring the position information of pedestrians not wearing masks, and selecting the pedestrian position information with the shortest relative distance as a counseling target;
and 4, step 4: path planning navigation and obstacle avoidance;
establishing a global optimal path and issuing decision information corresponding to the optimal path to an executing mechanism of the inspection vehicle, driving the inspection vehicle to execute motion operation according to the planned optimal path through linear speed and angular speed instructions corresponding to the decision information, and feeding back linear speed and angular speed data actually executed by the inspection vehicle by an encoder carried by the executing mechanism in real time so as to continuously optimize the position of the inspection vehicle until the position and pose of the inspection vehicle are close to the pose issued by the decision information;
in the process that the inspection vehicle moves to a recommended target through path planning, acquiring position information of an obstacle through a depth camera carried by the outside, and avoiding the obstacle according to the position information of the obstacle; firstly, obtaining a distance value of a surrounding environment through the acquired depth information of the picture; comparing the distance value with an environment map distance value prestored in the first map construction, if the difference value between the distance value and the prestored environment map distance value is smaller than a set threshold value, judging that the distance value is an obstacle, and continuously detecting the state of the obstacle; judging the motion state of the obstacle according to the distance and the angle of the obstacle detected in the two times; when the detection system moves to the obstacle for a certain distance, carrying out obstacle avoidance movement by adopting a corresponding obstacle avoidance method according to the movement state of the obstacle until the position of the target person is finally reached;
and 5: target following and voice reminding;
after the inspection vehicle moves to a target position, voice broadcasting reminding is carried out, a target person is reminded to wear a mask, and the mask is monitored in real time; if the target person finishes wearing the mask within a certain time after the voice prompt, moving to the position of the next target person; prompting again if the target person does not finish wearing the mask after voice prompting; if the target person tries to leave after the voice prompt, target following is started, the characteristics of the identity of the followed target are acquired through the depth camera, the followed target is tracked in real time after being locked, meanwhile, the voice prompt is adjusted from a recommendation mode to a criticizing mode, and broadcasting is carried out once at certain intervals.
3. The patrol vehicle control method for advising pedestrians of wearing the mask in time according to claim 2, wherein: the method for establishing the two-dimensional grid map in the step 1 specifically comprises the following steps:
step 1.1: setting the size of a map, the actual distance represented by each grid and a grid threshold;
step 1.2: reading in image data of a depth camera, and carrying out preprocessing operation on the data;
step 1.3: initializing, setting the whole grid area to be black and representing an unknown area;
step 1.4: according to the principle that the surrounding environment is detected from near to far, the detected area is set to be gray, and the gray represents the range which can be detected by the depth camera; counting the projection times of each grid by obstacles according to the numerical values of the pixel points, and when the times are greater than a set threshold value, determining that the corresponding grid map is occupied by the obstacles and setting the grid map as an occupied area; setting the area smaller than the threshold value as an idle area, which represents that the area can pass through smoothly;
step 1.5 step 1.4 is repeated until the two-dimensional grid map is established.
4. The patrol vehicle control method for advising pedestrians of wearing the mask in time according to claim 2 or 3, wherein: the training method of the mask detection network in the step 3 comprises the following steps:
firstly, preprocessing image data in a training process, and extracting labeling information of the used image data; sending the image data into a feature extraction network in batches, and acquiring different prediction feature layers generated by each image through the feature extraction network; transmitting different prediction feature layers into a regional generation network to generate candidate frames, sliding on the prediction feature layers by using a sliding window to generate a series of anchor points, performing classification prediction and boundary frame regression prediction on the different prediction feature layers through convolution layers, applying prediction results to the generated anchor points to obtain all candidate frame information, and screening the generated candidate frames according to a set threshold value through non-maximum suppression processing operation; finally, sequencing each characteristic layer of each picture according to confidence coefficients from low to high; transmitting the candidate frame information generated by the area generation network to the latter half of the network, firstly carrying out flattening operation, and obtaining final output through two full-connection layers, a classification predictor and a boundary frame regression predictor of the network; and then, carrying out reverse transmission on the output, continuously optimizing network parameters through a batch of image data, training to the set iteration times, and obtaining the optimal mask detection network model.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111104530.7A CN113837059B (en) | 2021-09-22 | 2021-09-22 | Patrol vehicle for prompting pedestrians to wear mask in time and control method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111104530.7A CN113837059B (en) | 2021-09-22 | 2021-09-22 | Patrol vehicle for prompting pedestrians to wear mask in time and control method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113837059A true CN113837059A (en) | 2021-12-24 |
| CN113837059B CN113837059B (en) | 2023-12-19 |
Family
ID=78960152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111104530.7A Active CN113837059B (en) | 2021-09-22 | 2021-09-22 | Patrol vehicle for prompting pedestrians to wear mask in time and control method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113837059B (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114821388A (en) * | 2022-03-10 | 2022-07-29 | 慧之安信息技术股份有限公司 | Indoor mask detection method and device based on edge calculation |
| CN116164770A (en) * | 2023-04-23 | 2023-05-26 | 禾多科技（北京）有限公司 | Path planning method, path planning device, electronic equipment and computer readable medium |
| CN117011231A (en) * | 2023-06-27 | 2023-11-07 | 盐城工学院 | Strip steel surface defect detection method and system based on improved YOLOv5 |
| CN117475115A (en) * | 2023-11-11 | 2024-01-30 | 华中师范大学 | Path guiding system in virtual-real fusion environment and working method thereof |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107167139A (en) * | 2017-05-24 | 2017-09-15 | 广东工业大学 | A kind of Intelligent Mobile Robot vision positioning air navigation aid and system |
| CN111860230A (en) * | 2020-07-03 | 2020-10-30 | 北京航空航天大学 | Automatic detection system and method based on behavior of video monitoring personnel not wearing mask |
| US20210109537A1 (en) * | 2019-10-09 | 2021-04-15 | Wuhan University | Autonomous exploration framework for indoor mobile robotics using reduced approximated generalized voronoi graph |
| CN113110457A (en) * | 2021-04-19 | 2021-07-13 | 杭州视熵科技有限公司 | Autonomous coverage inspection method for intelligent robot in indoor complex dynamic environment |
| CN113175996A (en) * | 2021-05-19 | 2021-07-27 | 云从科技集团股份有限公司 | Temperature monitoring method and device, computer readable storage medium and computer equipment |
-
2021
- 2021-09-22 CN CN202111104530.7A patent/CN113837059B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107167139A (en) * | 2017-05-24 | 2017-09-15 | 广东工业大学 | A kind of Intelligent Mobile Robot vision positioning air navigation aid and system |
| US20210109537A1 (en) * | 2019-10-09 | 2021-04-15 | Wuhan University | Autonomous exploration framework for indoor mobile robotics using reduced approximated generalized voronoi graph |
| CN111860230A (en) * | 2020-07-03 | 2020-10-30 | 北京航空航天大学 | Automatic detection system and method based on behavior of video monitoring personnel not wearing mask |
| CN113110457A (en) * | 2021-04-19 | 2021-07-13 | 杭州视熵科技有限公司 | Autonomous coverage inspection method for intelligent robot in indoor complex dynamic environment |
| CN113175996A (en) * | 2021-05-19 | 2021-07-27 | 云从科技集团股份有限公司 | Temperature monitoring method and device, computer readable storage medium and computer equipment |
Non-Patent Citations (1)
| Title |
|---|
| 汤义勤;高彦波;邹宏亮;叶建军;曾林;: "基于机器视觉的室内无轨巡检机器人导航系统", 自动化与仪表, no. 08 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114821388A (en) * | 2022-03-10 | 2022-07-29 | 慧之安信息技术股份有限公司 | Indoor mask detection method and device based on edge calculation |
| CN116164770A (en) * | 2023-04-23 | 2023-05-26 | 禾多科技（北京）有限公司 | Path planning method, path planning device, electronic equipment and computer readable medium |
| CN117011231A (en) * | 2023-06-27 | 2023-11-07 | 盐城工学院 | Strip steel surface defect detection method and system based on improved YOLOv5 |
| CN117011231B (en) * | 2023-06-27 | 2024-04-09 | 盐城工学院 | Strip steel surface defect detection method and system based on improved YOLOv5 |
| CN117475115A (en) * | 2023-11-11 | 2024-01-30 | 华中师范大学 | Path guiding system in virtual-real fusion environment and working method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113837059B (en) | 2023-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113837059B (en) | Patrol vehicle for prompting pedestrians to wear mask in time and control method thereof | |
| US20210397185A1 (en) | Object Motion Prediction and Autonomous Vehicle Control | |
| US20190147255A1 (en) | Systems and Methods for Generating Sparse Geographic Data for Autonomous Vehicles | |
| Fernández-Caballero et al. | Optical flow or image subtraction in human detection from infrared camera on mobile robot | |
| US11475671B2 (en) | Multiple robots assisted surveillance system | |
| EP3710783A1 (en) | Lightweight vehicle localization systems and methods | |
| CN103310190B (en) | Based on the facial image sample collection optimization method of isomery active vision network | |
| CN113071518B (en) | Automatic unmanned driving method, minibus, electronic equipment and storage medium | |
| CN112541449A (en) | Pedestrian trajectory prediction method based on unmanned aerial vehicle aerial photography view angle | |
| JP7339485B2 (en) | measurement system | |
| CN105014675A (en) | Intelligent mobile robot visual navigation system and method in narrow space | |
| CA3179005A1 (en) | Artificial intelligence and computer vision powered driving-performance assessment | |
| Cardarelli et al. | Multisensor data fusion for obstacle detection in automated factory logistics | |
| Sasaki et al. | Long-term demonstration experiment of autonomous mobile robot in a science museum | |
| CN114379544A (en) | Automatic parking system, method and device based on multi-sensor pre-fusion | |
| Adachi et al. | Turning at intersections using virtual lidar signals obtained from a segmentation result | |
| Bogoslavskyi et al. | Where to park? minimizing the expected time to find a parking space | |
| CN113084776A (en) | Intelligent epidemic prevention robot and system based on vision and multi-sensor fusion | |
| CN111739332B (en) | Parking lot management system | |
| EP4089649A1 (en) | Neuromorphic cameras for aircraft | |
| CN110473256A (en) | A kind of vehicle positioning method and system | |
| Llorca et al. | Assistive pedestrian crossings by means of stereo localization and rfid anonymous disability identification | |
| WO2022004333A1 (en) | Information processing device, information processing system, information processing method, and program | |
| CN114800615A (en) | Robot real-time scheduling system and method based on multi-source perception | |
| Khruahong et al. | Develop the framework conception for hybrid indoor navigation for monitoring inside building using quadcopter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |