WO2019196313A1 - 机器人行走障碍检测方法、装置、计算机设备和存储介质 - Google Patents

机器人行走障碍检测方法、装置、计算机设备和存储介质 Download PDF

Info

Publication number
WO2019196313A1
WO2019196313A1 PCT/CN2018/102854 CN2018102854W WO2019196313A1 WO 2019196313 A1 WO2019196313 A1 WO 2019196313A1 CN 2018102854 W CN2018102854 W CN 2018102854W WO 2019196313 A1 WO2019196313 A1 WO 2019196313A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
coordinate system
human body
coordinates
camera
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2018/102854
Other languages
English (en)
French (fr)
Inventor
曾伟
周宝
王健宗
肖京
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Publication of WO2019196313A1 publication Critical patent/WO2019196313A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Definitions

  • the present application relates to the field of robot obstacle avoidance technology, and in particular, to a robot walking obstacle detection method and apparatus, and a computer device and a storage medium storing computer readable instructions.
  • the obstacle detection of the robot is the key factor for the successful navigation movement of the robot.
  • the pedestrian in the movement adds many difficulties to the obstacle avoidance of the robot.
  • the inventor realized that in the current robot obstacle detection method, there are many problems, such as limited range of ultrasonic detection, and it is difficult to comprehensively cover the three-dimensional space even if a large number is loaded; although the laser detection has high precision, the same problem exists.
  • the coverage is detected by the depth camera, but due to the huge data processing and visual depth of field limitation, there is also a problem that the obstacle detection accuracy is not high.
  • these methods are mainly for stationary objects, but the pedestrians are equated with other obstacle detection, which is more passive when formulating strategies to avoid pedestrians, reducing the efficiency of effective obstacle avoidance.
  • the purpose of the present application is to solve at least one of the above technical drawbacks, in particular, technical defects that are difficult to avoid obstacles.
  • the present application provides a robot walking obstacle detecting method, the robot having a camera, a laser emitter, and a laser receiver, the method comprising the following steps: Step S10: acquiring an image through a camera, applying an image human body detection algorithm to obtain a human joint point Human body image coordinates (u, v) in the image coordinate system; step S20: converting human body image coordinates (u, v) in the image coordinate system into image physical coordinates (x, y) in the image physical coordinate system; step S30 : determining, according to x, y, the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter irradiating the key joint points of the human body, starting the laser emitter to emit laser light to measure the distance Z′ of the human body to the robot; step S40: according to x, y, Z 'calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system; step S50: according to the coordinates (X 1 , Y
  • the present application also provides a robot walking obstacle detecting device, the robot having a camera, a laser emitter, and a laser receiver, the device comprising: an image coordinate acquiring module, configured to acquire an image through a camera, and apply an image human body detecting algorithm to obtain a human body The human body image coordinates (u, v) of the off node in the image coordinate system; the physical coordinate acquisition module is used to convert the human body image coordinates (u, v) in the image coordinate system into the image physical coordinates in the image physical coordinate system ( x, y); human body distance acquisition module for determining the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key joint points of the human body according to x, y, and starting the laser emitter to emit laser light to measure the distance from the human body to the robot Z a camera coordinate acquisition module for calculating coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'; a crowd spacing acquisition module
  • the application also provides a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the processor to perform a robot walking
  • the obstacle detecting method includes the following steps: Step S10: acquiring an image by a camera, and applying an image human body detecting algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system; Step S20: Converting the human body image coordinates (u, v) in the image coordinate system to the image physical coordinates (x, y) in the image physical coordinate system; Step S30: determining the vertical deflection of the laser emitter illuminating the key human joint points according to x, y The angle ⁇ and the horizontal deflection angle ⁇ start the laser emitter to emit laser light to measure the distance Z′ of the human body to the robot; step S40: calculate the coordinates of the human joint point in the camera coordinate system according to x, y, Z′ (X, Y, Z); Step
  • the present application also provides a non-volatile storage medium storing computer readable instructions that, when executed by one or more processors, cause one or more processors to perform a robotic walking obstacle detection
  • the method for detecting a walking obstacle of a robot includes the following steps: Step S10: acquiring an image by a camera, applying an image human body detection algorithm to obtain body image coordinates (u, v) of a human joint point in an image coordinate system; and step S20: placing the image The human body image coordinates (u, v) in the coordinate system are converted into image physical coordinates (x, y) in the image physical coordinate system; step S30: determining the vertical deflection angle ⁇ of the laser emitter irradiating the key human joint points according to x, y And the horizontal deflection angle ⁇ , the laser emitter is activated to emit laser light to measure the distance Z' of the human body to the robot; step S40: calculating the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to
  • the above-mentioned robot walking obstacle detecting method, device, computer equipment and storage medium by combining image recognition and laser ranging to determine the actual coordinates of the human body (in the coordinate of the camera coordinate system), the real-time position of the human body can be accurately obtained, thereby determining any two people.
  • the distance between the two is used to judge the walking path in real time, and accurate pedestrian obstacle detection is realized.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment
  • FIG. 2 is a schematic flow chart of a method for detecting a walking obstacle of a robot according to an embodiment
  • FIG. 3 is a schematic diagram of a human body detection process of an embodiment
  • FIG. 4 is a diagram showing an example of human body detection of an embodiment
  • FIG. 5 is a schematic diagram of a module of a robot walking obstacle detecting device according to an embodiment.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment.
  • the computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus.
  • the non-volatile storage medium of the computer device stores an operating system, a database, and computer readable instructions.
  • the database may store a sequence of control information.
  • the processor may implement a processor.
  • a robot walking obstacle detection method The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • Computer readable instructions may be stored in the memory of the computer device, the computer readable instructions being executable by the processor to cause the processor to perform a robotic walking obstacle detection method.
  • the network interface of the computer device is used to communicate with the terminal connection. It will be understood by those skilled in the art that the structure shown in FIG. 1 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • the robot walking obstacle detecting method described below can be applied to an intelligent robot such as a customer service robot, a sweeping robot, and the like.
  • FIG. 2 is a schematic flow chart of a method for detecting a walking obstacle of a robot according to an embodiment.
  • a robot walking obstacle detecting method the robot having a camera, a laser emitter, and a laser receiver, the method comprising the following steps:
  • Step S10 acquiring an image by using a camera, and applying an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system.
  • the image human body detection algorithm refers to a technique for recognizing a human body through image recognition technology.
  • This type of method is currently the more mainstream human detection method. It mainly uses various static features of images such as edge features, shape features, statistical features or transform features to describe the human body. Representative features include Haar wavelet features, HOG features, and Edgelet. Features, Shapelet features, and outline template features.
  • Papageorgiou and Poggio first proposed the concept of Harr wavelet, Viola et al. introduced the concept of integral graph, accelerated the extraction speed of Harr features, and applied the method to human detection, combined with human motion and appearance mode to construct human detection system. The better detection effect lays the foundation for the development of human detection technology.
  • HOG Histogram of Oriented Gradients
  • the "Edgelet” feature which is a short line or curve segment, is applied to human detection of a single image of a complex scene, yielding a detection rate of approximately 92% on the CAVIAR database.
  • Shapelet features can be automatically obtained using machine learning methods, namely Shapelet features.
  • the algorithm first extracts the gradient information of different directions from the training samples, and then uses the AdaBoost algorithm to train, thus obtaining the Shapelet feature.
  • the method refers to constructing a template by using information such as edge contour, texture and gray scale of the target object in the image, and detecting the target by template matching.
  • the basic idea of this kind of method is to divide the human body into several components, and then separately test each part of the image, and finally integrate the detection results according to a certain constraint relationship, and finally determine whether there is a human body.
  • This type of method refers to image acquisition by two or more cameras, and then analyzing the three-dimensional information of the target in the image to identify the human body.
  • the image human body detection algorithm is not specifically limited as long as the human body and its joint points can be identified.
  • Human joint points refer to key nodes of the human body such as the head, neck, shoulders, upper arms, elbows, wrists, waist, chest, thighs, knees, calves, ankles, etc. There are multiple.
  • human body detection can be performed using a method based on a human body part.
  • OpenPose is an open source library written in C++ using OpenCV and Caffe for real-time detection of multi-threaded multi-person key points.
  • the main detection of human body detection consists of two parts: human joint point detection and joint point correlation detection.
  • the joint point detection uses a Convolutional Pose Machine.
  • the attitude machine can infer the position of the key points of the human body by adding the convolutional neural network, using the texture information of the image, the spatial structure information and the center map. Simultaneously, a convolutional neural network is used in parallel to detect the correlation between joint points, ie which joint points should be connected together.
  • Fig. a is a complete input image
  • Fig. b is a confidence feature map of joint points
  • Fig. c is a joint point correlation feature map
  • Fig. d is a series of candidate joint point matching after sparse method
  • figure, figure e is the figure after finally completing the combined connection of the joint points of the human body.
  • an image is acquired by the camera, and an image human body detection algorithm is applied to obtain a series of human body image coordinates ⁇ (u k1 , v k1 ), ..., (u ki , v ki ),...,(u kn ,v kn ) ⁇ , where k represents the kth person and i represents the i-th body joint point.
  • k represents the kth person
  • i represents the i-th body joint point.
  • the image coordinate system is a two-dimensional coordinate system in the imaging plane of the camera.
  • the origin is usually located at the upper left corner of the screen.
  • the width (transverse) of the screen is the x-axis
  • the height (longitudinal) of the image is the y-axis, in units of pixels.
  • Step S20 Convert the human body image coordinates (u, v) in the image coordinate system to the image physical coordinates (x, y) in the image physical coordinate system. Since the image coordinate system is based on the number of pixels, which is not conducive to calculation, it is necessary to convert the human body image coordinates (u, v) in the image coordinate system into the physical coordinates (x, y) of the image in the image physical coordinate system. Easy to calculate.
  • the image physical coordinate system is an image coordinate system expressed in physical units (for example, meters), which can be understood as a translation coordinate system of an image coordinate system, and is also a two-dimensional coordinate system.
  • step S20 includes:
  • the human body image coordinates in the image coordinate system are converted into image physical coordinates in the image physical coordinate system, and the conversion relationship is:
  • the above formula is a matrix operation formula
  • (u, v) is the human body image coordinate in the image coordinate system
  • (x, y) is the human body image coordinate in the image physical coordinate system
  • u 0 and v 0 are the camera optical axis.
  • the coordinate values in the image coordinate system, dx and dy are the physical size values of the x-axis and y-axis of each pixel.
  • Step S30 Determine, according to x, y, the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key joint points of the human body, and activate the laser emitter to emit laser light to measure the distance Z' of the human body to the robot.
  • the key human joint point here is at least one of the above-mentioned human joint points, for example, all joint points in the above-mentioned human joint points, or several important joint points, such as a head, a shoulder, an elbow, a wrist, Knees and ankles. In some embodiments, for convenience, only one of the important joint points may be selected as a key human joint point, such as a head.
  • the distance from the key human joint point to the robot is obtained, which is equivalent to the distance from the human body to the robot.
  • step S30 includes:
  • d u is the offset of the camera coordinate system relative to the image physical coordinate system
  • d v is the offset of the laser coordinate system relative to the image physical coordinate system
  • d is the horizontal distance of the laser emitter to the imaging plane of the camera.
  • the camera coordinate system is a three-dimensional coordinate system whose origin is the optical center of the camera.
  • the X and Y axes are parallel to the x and y axes of the image coordinate system.
  • the z axis is the optical axis of the camera, and the z axis is perpendicular to the imaging plane of the camera.
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ are relative to the camera coordinate system.
  • the laser emitter After calculating the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ , the laser emitter adjusts to the corresponding position according to the current posture and ⁇ , ⁇ to align the key human body.
  • the node is turned off and then a laser is emitted to measure the distance Z' of the human body to the robot.
  • each human joint point can be obtained, so that the specific position (X, Y, Z) of the joint point of the human body can be determined through subsequent steps, and the spacing between any two persons can be determined.
  • Step S40 Calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, and Z'.
  • the position of the imaging plane of the camera coordinate system and the coordinate origin of the laser coordinate system differ only by one d in the z-axis direction, ignoring the slight deviation of the position between the laser emitter and the camera (usually the laser emitter and the camera are not far apart in space)
  • the laser measurement value is the true value of the corresponding world coordinate system in the camera coordinate system. We can find the distance between two people in the real world coordinate system according to the proportional relationship between the camera coordinate value and the real world coordinate value.
  • step S40 includes:
  • the coordinates (X, Y, Z) of each human joint point in the camera coordinate system are calculated, and the corresponding human joint points between any two people are calculated according to (X, Y, Z).
  • the distance L where the coordinates of the camera coordinate system (X, Y, Z) and the coordinates (x, y) of the image physical coordinate system are:
  • Step S50 Calculate between the corresponding human joint points of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance L The distance L between two people in the real world coordinate system can be obtained from the proportional relationship between the camera coordinate value and the real world coordinate value.
  • Step S60 determining the walking direction according to the distance L between the respective human joint points of any two persons.
  • the robot can determine the walking strategy at this time.
  • at least one factor analysis such as the distance between the human body and the robot, the walking speed of the human body, and the walking speed of the robot may be combined.
  • the body image coordinates of the human joint point are ⁇ (x k1 , y k1 ),...,(x ki ,y ki ),...,(x kn ,y kn ) ⁇ , and calculate the spacing of any two of them. Finally, select the best separation distance to pass.
  • FIG. 5 is a schematic diagram of a module of a robot walking obstacle detecting device according to an embodiment.
  • the present application further provides a robot walking obstacle detecting device, the robot having a camera, a laser emitter, and a laser receiver, the device comprising: an image coordinate acquiring module 100 and a physical coordinate acquiring module 200.
  • the image coordinate acquiring module 100 is configured to acquire an image by using a camera, and apply an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system; the physical coordinate acquiring module is configured to use the human body image in the image coordinate system.
  • the coordinates (u, v) are converted into image physical coordinates (x, y) in the physical coordinate system of the image; the human body distance acquisition module is configured to determine the vertical deflection angle ⁇ and the horizontal deflection of the laser emitter to illuminate the key joint points of the human body according to x and y.
  • the camera coordinate acquisition module is used to calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'
  • the crowd spacing acquisition module is configured to calculate the corresponding human body of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance L between the nodes is closed; the walking direction determining module is configured to determine the walking direction according to the distance L between the respective human joint points of any two persons.
  • the image coordinate acquisition module 100 acquires an image through a camera, and applies an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system.
  • the image human body detection algorithm refers to a technique for recognizing a human body through image recognition technology.
  • This type of method is currently the more mainstream human detection method. It mainly uses various static features of images such as edge features, shape features, statistical features or transform features to describe the human body. Representative features include Haar wavelet features, HOG features, and Edgelet. Features, Shapelet features, and outline template features.
  • Papageorgiou and Poggio first proposed the concept of Harr wavelet, Viola et al. introduced the concept of integral graph, accelerated the extraction speed of Harr features, and applied the method to human detection, combined with human motion and appearance mode to construct human detection system. The better detection effect lays the foundation for the development of human detection technology.
  • HOG Histogram of Oriented Gradients
  • the "Edgelet” feature which is a short line or curve segment, is applied to human detection of a single image of a complex scene, yielding a detection rate of approximately 92% on the CAVIAR database.
  • Shapelet features can be automatically obtained using machine learning methods, namely Shapelet features.
  • the algorithm first extracts the gradient information of different directions from the training samples, and then uses the AdaBoost algorithm to train, thus obtaining the Shapelet feature.
  • the method refers to constructing a template by using information such as edge contour, texture and gray level of the target object in the image, and detecting the target by template matching method.
  • the basic idea of this kind of method is to divide the human body into several components, and then separately test each part of the image, and finally integrate the detection results according to a certain constraint relationship, and finally determine whether there is a human body.
  • This type of method refers to image acquisition by two or more cameras, and then analyzing the three-dimensional information of the target in the image to identify the human body.
  • the image human body detection algorithm is not specifically limited as long as the human body and its joint points can be identified.
  • Human joint points refer to key nodes of the human body such as the head, neck, shoulders, upper arms, elbows, wrists, waist, chest, thighs, knees, calves, ankles, etc. There are multiple.
  • human body detection can be performed using a method based on a human body part.
  • OpenPose is an open source library written in C++ using OpenCV and Caffe for real-time detection of multi-threaded multi-person key points.
  • the main detection of human body detection consists of two parts: human joint point detection and joint point correlation detection.
  • the joint point detection uses a Convolutional Pose Machine.
  • the attitude machine can infer the position of the key points of the human body by adding the convolutional neural network, using the texture information of the image, the spatial structure information and the center map. Simultaneously, a convolutional neural network is used in parallel to detect the correlation between joint points, ie which joint points should be connected together.
  • Fig. a is a complete input image
  • Fig. b is a confidence feature map of joint points
  • Fig. c is a joint point correlation feature map
  • Fig. d is a series of candidate joint point matching after sparse method
  • figure, figure e is the figure after finally completing the combined connection of the joint points of the human body.
  • an image is acquired by the camera, and an image human body detection algorithm is applied to obtain a series of human body image coordinates ⁇ (u k1 , v k1 ), ..., (u ki , v ki ),...,(u kn ,v kn ) ⁇ , where k represents the kth person and i represents the i-th body joint point.
  • k represents the kth person
  • i represents the i-th body joint point.
  • the image coordinate system is a two-dimensional coordinate system in the imaging plane of the camera.
  • the origin is usually located at the upper left corner of the screen.
  • the width (transverse) of the screen is the x-axis
  • the height (longitudinal) of the image is the y-axis, in units of pixels.
  • the physical coordinate acquisition module 200 converts the human body image coordinates (u, v) in the image coordinate system into image physical coordinates (x, y) in the image physical coordinate system. Since the image coordinate system is based on the number of pixels, which is not conducive to calculation, it is necessary to convert the human body image coordinates (u, v) in the image coordinate system into the physical coordinates (x, y) of the image in the image physical coordinate system. Easy to calculate.
  • the image physical coordinate system is an image coordinate system expressed in physical units (for example, meters), which can be understood as a translation coordinate system of an image coordinate system, and is also a two-dimensional coordinate system.
  • the physical coordinate acquisition module 200 is specifically configured to:
  • the human body image coordinates in the image coordinate system are converted into image physical coordinates in the image physical coordinate system, and the conversion relationship is:
  • the above formula is a matrix operation formula
  • (u, v) is the human body image coordinate in the image coordinate system
  • (x, y) is the human body image coordinate in the image physical coordinate system
  • u 0 and v 0 are the camera optical axis.
  • the coordinate values in the image coordinate system, dx and dy are the physical size values of the x-axis and y-axis of each pixel.
  • the human body distance acquisition module 300 determines the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key human joint points according to x, y, and activates the laser emitter to emit laser light to measure the distance Z' of the human body to the robot.
  • the key human joint point here is at least one of the above-mentioned human joint points, for example, all joint points in the above-mentioned human joint points, or several important joint points, such as a head, a shoulder, an elbow, a wrist, Knees and ankles. In some embodiments, for convenience, only one of the important joint points may be selected as a key human joint point, such as a head.
  • the distance from the key human joint point to the robot is obtained, which is equivalent to the distance from the human body to the robot.
  • the human body distance acquisition module 300 is specifically configured to:
  • d u is the offset of the camera coordinate system relative to the image physical coordinate system
  • d v is the offset of the laser coordinate system relative to the image physical coordinate system
  • d is the horizontal distance of the laser emitter to the imaging plane of the camera.
  • the camera coordinate system is a three-dimensional coordinate system whose origin is the optical center of the camera.
  • the X and Y axes are parallel to the x and y axes of the image coordinate system.
  • the z axis is the optical axis of the camera, and the z axis is perpendicular to the imaging plane of the camera.
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ are relative to the camera coordinate system.
  • the laser emitter After calculating the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ , the laser emitter adjusts to the corresponding position according to the current posture and ⁇ , ⁇ to align the key human body.
  • the node is turned off and then a laser is emitted to measure the distance Z' of the human body to the robot.
  • the camera coordinate acquisition module 400 calculates the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, and Z'.
  • the position of the imaging plane of the camera coordinate system and the coordinate origin of the laser coordinate system differ only by one d in the z-axis direction, ignoring the slight deviation of the position between the laser emitter and the camera (usually the laser emitter and the camera are not far apart in space)
  • the laser measurement value is the true value of the corresponding world coordinate system in the camera coordinate system. We can find the distance between two people in the real world coordinate system according to the proportional relationship between the camera coordinate value and the real world coordinate value.
  • the camera coordinate acquisition module 400 is specifically configured to:
  • the coordinates (X, Y, Z) of each human joint point in the camera coordinate system are calculated, and the corresponding human joint points between any two people are calculated according to (X, Y, Z).
  • the distance L where the coordinates of the camera coordinate system (X, Y, Z) and the coordinates (x, y) of the image physical coordinate system are:
  • the crowd spacing acquisition module 500 calculates the corresponding human joint points of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance between L The distance L between two people in the real world coordinate system can be obtained from the proportional relationship between the camera coordinate value and the real world coordinate value.
  • the walking direction determining module 600 determines the walking direction based on the distance L between the respective human joint points of any two persons. When the position of each person is determined, thereby determining the distance L between any two people, the robot can determine the walking strategy at this time. When determining the walking strategy, at least one factor analysis such as the distance between the human body and the robot, the walking speed of the human body, and the walking speed of the robot may be combined.
  • the body image coordinates of the human joint point are ⁇ (x k1 , y k1 ),...,(x ki ,y ki ),...,(x kn ,y kn ) ⁇ , and calculate the spacing of any two of them. Finally, select the best separation distance to pass.
  • the walking direction determining module 600 uses the quick sorting algorithm to quickly find the nearest or intersecting points between the two discrete point groups, so that the distance between the camera coordinates and the real world coordinate values can be calculated between the points.
  • the distance and the distance between each two are constantly detected, and the best passing distance can be found in real time.
  • the application also provides a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the processor to perform any of the above The steps of the robot walking obstacle detecting method according to the embodiment.
  • the present application also provides a storage medium storing computer readable instructions that, when executed by one or more processors, cause one or more processors to perform the robotic walking described in any of the above embodiments The steps of the obstacle detection method.
  • the above-mentioned robot walking obstacle detection method, device, computer equipment and storage medium acquire images through a camera, and apply image human body detection algorithm to obtain human body image coordinates (u, v) of human joint points in an image coordinate system;
  • the human body image coordinates (u, v) are converted into image physical coordinates (x, y) in the image physical coordinate system;
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter irradiating the key human joint points are determined according to x, y, Starting the laser emitter to emit laser light to measure the distance Z' of the human body to the robot; calculating the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'; according to the corresponding human joint of any two people Point the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system to calculate the distance L between the corresponding human joint points of
  • the real-time position of the human body can be accurately obtained, thereby determining the spacing L between any two people, and then determining the walking path in real time through L, realizing Accurate pedestrian obstacle detection.
  • the storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Human Computer Interaction (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Manipulator (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种机器人行走障碍检测方法和装置，该方法通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标（u，v）（S10）；将图像坐标系下的人体图像坐标（u，v）转换为图像物理坐标系下的图像物理坐标（x，y）（S20）；根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z'（S30）；根据x、y、Z'计算得到人体关节点在摄像机坐标系的坐标（X，Y，Z）（S40）；根据任意两人的相应人体关节点在摄像机坐标系的坐标（X 1，Y 1，Z 1）和（X 2，Y 2，Z 2）计算任意两人的相应人体关节点之间的距离L（S50）；根据任意两人的相应人体关节点之间的距离L确定行走方向（S60），实现精确行人避障。还提供了一种计算机设备和存储介质。

Description

机器人行走障碍检测方法、装置、计算机设备和存储介质
本申请要求于2018年4月10日提交中国专利局、申请号为201810314149.5，发明名称为“机器人行走障碍检测方法、装置、计算机设备和存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器人避障技术领域，具体而言，本申请涉及一种机器人行走障碍检测方法和装置，以及一种计算机设备和存储有计算机可读指令的存储介质。
背景技术
在机器人的导航过程中，机器人的障碍检测是机器人成功导航运动的关键因素，相对于静止物体，移动中的行人更是给机器人的避障增加了诸多难点。发明人意识到在目前的机器人障碍检测方法中，存在诸多难题，如超声波检测范围有限，即使装载较多的数量，也难以全面地覆盖三维空间；激光检测虽然精度较高，也存在同样的问题；通过深度摄像机检测覆盖范围较大，但是由于庞大的数据处理和视觉景深限制，也存在障碍物检测精度不高的问题。并且这些方法主要是针对静止的物体，但是把行人等同于其他障碍物检测，在制定策略避让行人时显得比较被动，降低了有效避障的效率。
发明内容
本申请的目的旨在至少能解决上述的技术缺陷之一，特别是难以避障行人的技术缺陷。
本申请提供一种机器人行走障碍检测方法，所述机器人具有摄像机、激光发射器、激光接收器，所述方法包括如下步骤：步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
本申请还提供一种机器人行走障碍检测装置，所述机器人具有摄像机、激光发射器、激光接收器，所述装置包括：图像坐标获取模块，用于通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；物理坐标获取模块，用于将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；人体距离获取模块，用于根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测 量人体到机器人的距离Z′；摄像机坐标获取模块，用于根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；人群间距获取模块，用于根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；行走方向确定模块，用于根据任意两人的相应人体关节点之间的距离L确定行走方向。
本申请还提供一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行一种机器人行走障碍检测方法，所述机器人行走障碍检测方法包括以下步骤：步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
本申请还提供一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行一种机器人行走障碍检测方法，所述机器人行走障碍检测方法包括以下步骤：步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
上述的机器人行走障碍检测方法、装置、计算机设备和存储介质，通过结合图像识别和激光测距确定人体实际坐标(在摄像机坐标系的坐标)，可以精确得到人体的实时位置，从而确定任意两人之间的间距，再实时判断行走路径，实现了精确的行人避障检测。
附图说明
图1为一个实施例中计算机设备的内部结构示意图；
图2为一个实施例的机器人行走障碍检测方法流程示意图；
图3为一个实施例的人体检测流程示意图；
图4为一个实施例的人体检测示例图；
图5为一个实施例的机器人行走障碍检测装置模块示意图。
具体实施方式
图1为一个实施例中计算机设备的内部结构示意图。如图1所示，该计算机设备包括通过系统总线连接的处理器、非易失性存储介质、存储器和网络接口。其中， 该计算机设备的非易失性存储介质存储有操作系统、数据库和计算机可读指令，数据库中可存储有控件信息序列，该计算机可读指令被处理器执行时，可使得处理器实现一种机器人行走障碍检测方法。该计算机设备的处理器用于提供计算和控制能力，支撑整个计算机设备的运行。该计算机设备的存储器中可存储有计算机可读指令，该计算机可读指令被处理器执行时，可使得处理器执行一种机器人行走障碍检测方法。该计算机设备的网络接口用于与终端连接通信。本领域技术人员可以理解，图1中示出的结构，仅仅是与本申请方案相关的部分结构的框图，并不构成对本申请方案所应用于其上的计算机设备的限定，具体的计算机设备可以包括比图中所示更多或更少的部件，或者组合某些部件，或者具有不同的部件布置。
以下描述的机器人行走障碍检测方法，可以应用于智能机器人，例如客服机器人、扫地机器人等等。
图2为一个实施例的机器人行走障碍检测方法流程示意图。
一种机器人行走障碍检测方法，该机器人具有摄像机、激光发射器、激光接收器，该方法包括如下步骤：
步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)。
图像人体检测算法，是指通过图像识别技术识别出人体的技术。图像人体检测方法有三种，分别为基于全局特征的方法、基于人体部位的方法、基于立体视觉的方法。
基于全局特征的方法
该类方法是目前较为主流的人体检测方法，主要采用边缘特征、形状特征、统计特征或者变换特征等图像的各类静态特征来描述人体，其中代表性的特征包括Haar小波特征、HOG特征、Edgelet特征、Shapelet特征和轮廓模板特征等。
(1)基于Haar小波特征的方法
Papageorgiou和Poggio最早提出Harr小波的概念，Viola等引进了积分图的概念，加快了Harr特征的提取速度，并将该方法应用于人体检测，结合人体的运动和外观模式构建人体检测系统，取得了较好的检测效果，为人体检测技术的发展奠定了基础。
(2)基于HOG特征的方法
将梯度方向直方图(Histogram of Oriented Gradients，HOG)的概念应用于用于人体检测，在MIT人体数据库上获得近乎100％的检测成功率；在包含视角、光照和背景等变化的INRIA人体数据库上，也取得了大约90％的检测成功率。
(3)基于edgelet特征的方法
“小边”(Edgelet)特征，即一些短的直线或者曲线片段，并将其应用于复杂场景的单幅图像的人体检测，在CAVIAR数据库上取得了大约92％的检测率。
(4)基于Shapelet特征的方法
可以利用机器学习的方法自动得到特征，即Shapelet特征。该算法首先从训练样本提取图片不同方向的梯度信息，然后利用AdaBoost算法进行训练，从而得到Shapelet特征。
(5)基于轮廓模板的方法
该方法是指利用图像中目标物体的边缘轮廓、纹理和灰度等信息构建模板，通 过模板匹配的方法检测目标。
基于人体部位的方法
该类方法的基本思想是把人体分成几个组成部分，然后对图像中每部分分别检测，最后将检测结果按照一定的约束关系进行整合，最终判断是否存在人体。
基于立体视觉的方法
该类方法是指通过2个或2个以上的摄像机进行图像采集，然后分析图像中目标的三维信息以识别出人体。
在本实施例中，并不对图像人体检测算法进行具体限定，只要能够识别出人体及其关节点即可。人体关节点，是指例如头部、颈部、肩部、上臂、手肘、手腕、腰部、胸口、大腿、膝盖、小腿、脚踝等等人体关键部位的关键节点，每一个人的人体关节点具有多个。
在本实施例中，可以采用基于人体部位的方法来进行人体检测。OpenPose是一个利用OpenCV和Caffe并以C++写成的开源库，用来实现多线程的多人关键点实时检测。人体检测主要检测包含两个部分：人体关节点检测和关节点的相关性检测。其中关节点检测使用了卷积的姿态机(Convolutional Pose Machine)。姿态机通过加入卷积神经网络(convolution neural network)，利用图像的纹理信息，空间结构信息和中心特征图(center map)，通过有监督地学习，可以推断出人体关键点的位置。同时并行地使用一个卷积神经网络检测关节点之间的相关性，即哪些关节点应该连接到一起。
如图3所示，当采集到包含人体的图像时，我们使用整张图像作为检测网络的输入，并且同时将图像输入到两个并行的二分网络，同时检测关节点的置信度和关节点的相关度。同时将关节点置信度特征图和关节点相关度特征图和输入图像组合到一起，作为下一阶段的输入，实际的检测过程中，级联了6个这样的检测网络。
如图4所示，图a为完整的输入图像，图b为关节点的置信特征图，图c为关节点相关性特征图，图d为通过稀疏的方法实现一系列的候选关节点匹配后的图，图e为最终完成人体各个关节点的组合连接后的图。
图像中可能会识别到多个人，因此在本实施例中，通过摄像机获取图像，应用图像人体检测算法得到一系列的人体图像坐标{(u k1,v k1),...,(u ki,v ki),...,(u kn,v kn)}，其中k代表第k个人，i代表第i个人体关节点。为了便于理解，以下的某些计算过程只以一个关节点为例。
图像坐标系是摄像机成像平面中的二维坐标系，原点通常位于画面的左上角，画面的宽(横向)为x轴，画面的高(纵向)为y轴，以像素点个数为单位。通过识别人体在画面中的位置，得到人体各节点在图像坐标系下的人体图像坐标(u,v)，u表示x轴第u个像素点，v表示y轴第v个像素点。
步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)。由于图像坐标系以像素点个数为单位，不利于计算，因此需要将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)，以方便计算。图像物理坐标系为以物理单位(例如米)表示的图像坐标系，可以理解是图像坐标系的平移坐标系，也是二维坐标系。
在其中一个实施例中，步骤S20具体过程包括：
将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转 换关系为：
Figure PCTCN2018102854-appb-000001
其中上述公式为矩阵运算公式，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′。
此处的关键人体关节点为上述人体关节点中的至少一个，例如可以是上述人体关节点中的全部关节点，或者几个重要的关节点，例如头部、肩部、手肘、手腕、膝盖、脚踝。在一些实施例中，为了方便，可以只选取其中一个重要的关节点作为关键人体关节点，例如头部。获取了关键人体关节点到机器人的距离，即相当于获取了人体到机器人的距离。
在其中一个实施例中，步骤S30具体过程包括：
确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
Figure PCTCN2018102854-appb-000002
Figure PCTCN2018102854-appb-000003
d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。摄像机坐标系是三维坐标系，其原点为摄像机的光心，其X轴、Y轴与图像坐标系的x、y轴平行，z轴为摄像机的光轴，z轴与摄像机成像平面垂直。
垂直偏转角度α和水平偏转角度β是相对于摄像机坐标系而言的，计算垂直偏转角度α和水平偏转角度β后，激光发射器根据当前姿势以及α、β调整到相应位置以对准关键人体关节点，然后发射激光以测量人体到机器人的距离Z′。
通过上述步骤，可以获取到各个人体关节点的x、y、Z′，从而可以通过后续步骤确定人体关节点的具体位置(X,Y,Z)，并确定任意两人之间的间距。
步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)。
摄像机坐标系的成像平面的位置和激光坐标系的坐标原点在z轴方向的距离只相差一个d，忽略激光发射器和摄像机在位置微小偏差(通常激光发射器和摄像机在空间上距离不远)，激光测量值则为摄像机坐标系中对应的世界坐标系的真实值，我们可以根据摄像机坐标值和真实的世界坐标值的比例关系求出真实的世界坐标系中两人的间距。
在其中一个实施例中，步骤S40具体过程包括：
根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
Figure PCTCN2018102854-appb-000004
f为摄像机焦距，Z＝Z′+d。
步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L。可以根据摄像机坐标值和真实的世界坐标值的比例关系求出真实的世界坐标系中两人的间距L。
步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。当确定了各人位置后，从而确定了任意两人之间的距离L，机器人此时便可以确定行走策略。在确定行走策略时，可以结合人体与机器人之间的距离、人体行走速度、机器人行走速度等至少一个因素分析。
假设人体关节点人体图像坐标为{(x k1,y k1),...,(x ki,y ki),...,(x kn,y kn)}，分别计算其中任意两人的间距，最后选取其中最佳的间隔距离通过。
假设计算P 1P 2两人之间的间隔，其中：
P 1＝{(x 11,y 11),...,(x 1i,y 1i),...,(x 1n,y 1n)}
P 2＝{(x 21,y 21),…,(x 2i,y 2i),…,(x 2n,y 2n)}
由于两人之间的距离和X轴方向的距离成正比，因此可以只关注X轴坐标，所以可以将两人的所有检测到的点组成一个队列：
P＝{x 11,x 21,…,x 1i,x 2i,…,x 1n,x 2n}
运用快速排序算法，可以快速找到两个离散点群之间最近或者交叉的点，这样就可以根据前面所述摄像机坐标值和真实的世界坐标值的比例关系算出这些点之间的距离，并且不停的检测每两人之间的距离，就能实时的找出最佳的通过距离。
图5为一个实施例的机器人行走障碍检测装置模块示意图。对应上述的机器人行走障碍检测方法，本申请还提供一种机器人行走障碍检测装置，所述机器人具有摄像机、激光发射器、激光接收器，所述装置包括：图像坐标获取模块100、物理坐标获取模块200、人体距离获取模块300、摄像机坐标获取模块400、人群间距获取模块500、行走方向确定模块600。
图像坐标获取模块100用于通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；物理坐标获取模块用于将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；人体距离获取模块用于根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；摄像机坐标获取模块用于根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；人群间距获取模块用于根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；行走方向确定模块用于根据任意两人的相应人体关节点之间的距离L确定行走方向。
图像坐标获取模块100通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)。
图像人体检测算法，是指通过图像识别技术识别出人体的技术。图像人体检测方法有三种，分别为基于全局特征的方法、基于人体部位的方法、基于立体视觉的方法。
基于全局特征的方法
该类方法是目前较为主流的人体检测方法，主要采用边缘特征、形状特征、统计特征或者变换特征等图像的各类静态特征来描述人体，其中代表性的特征包括Haar小波特征、HOG特征、Edgelet特征、Shapelet特征和轮廓模板特征等。
(1)基于Haar小波特征的方法
Papageorgiou和Poggio最早提出Harr小波的概念，Viola等引进了积分图的概念，加快了Harr特征的提取速度，并将该方法应用于人体检测，结合人体的运动和外观模式构建人体检测系统，取得了较好的检测效果，为人体检测技术的发展奠定了基础。
(2)基于HOG特征的方法
将梯度方向直方图(Histogram of Oriented Gradients，HOG)的概念应用于用于人体检测，在MIT人体数据库上获得近乎100％的检测成功率；在包含视角、光照和背景等变化的INRIA人体数据库上，也取得了大约90％的检测成功率。
(3)基于edgelet特征的方法
“小边”(Edgelet)特征，即一些短的直线或者曲线片段，并将其应用于复杂场景的单幅图像的人体检测，在CAVIAR数据库上取得了大约92％的检测率。
(4)基于Shapelet特征的方法
可以利用机器学习的方法自动得到特征，即Shapelet特征。该算法首先从训练样本提取图片不同方向的梯度信息，然后利用AdaBoost算法进行训练，从而得到Shapelet特征。
(5)基于轮廓模板的方法
该方法是指利用图像中目标物体的边缘轮廓、纹理和灰度等信息构建模板，通过模板匹配的方法检测目标。
基于人体部位的方法
该类方法的基本思想是把人体分成几个组成部分，然后对图像中每部分分别检测，最后将检测结果按照一定的约束关系进行整合，最终判断是否存在人体。
基于立体视觉的方法
该类方法是指通过2个或2个以上的摄像机进行图像采集，然后分析图像中目标的三维信息以识别出人体。
在本实施例中，并不对图像人体检测算法进行具体限定，只要能够识别出人体及其关节点即可。人体关节点，是指例如头部、颈部、肩部、上臂、手肘、手腕、腰部、胸口、大腿、膝盖、小腿、脚踝等等人体关键部位的关键节点，每一个人的人体关节点具有多个。
在本实施例中，可以采用基于人体部位的方法来进行人体检测。OpenPose是一个利用OpenCV和Caffe并以C++写成的开源库，用来实现多线程的多人关键点实时检测。人体检测主要检测包含两个部分：人体关节点检测和关节点的相关性检测。其中关节点检测使用了卷积的姿态机(Convolutional Pose Machine)。姿态机通过加入卷积神经网络(convolution neural network)，利用图像的纹理信息，空间结构信息和中心特征图(center map)，通过有监督地学习，可以推断出人体关键点的位置。同时并行地使用一个卷积神经网络检测关节点之间的相关性，即哪些关节点应该连接到一起。
如图3所示，当采集到包含人体的图像时，我们使用整张图像作为检测网络的输入，并且同时将图像输入到两个并行的二分网络，同时检测关节点的置信度和关节点的相关度。同时将关节点置信度特征图和关节点相关度特征图和输入图像组合到一起，作为下一阶段的输入，实际的检测过程中，级联了6个这样的检测网络。
如图4所示，图a为完整的输入图像，图b为关节点的置信特征图，图c为关节点相关性特征图，图d为通过稀疏的方法实现一系列的候选关节点匹配后的图，图e为最终完成人体各个关节点的组合连接后的图。
图像中可能会识别到多个人，因此在本实施例中，通过摄像机获取图像，应用图像人体检测算法得到一系列的人体图像坐标{(u k1,v k1),...,(u ki,v ki),...,(u kn,v kn)}，其中k代表第k个人，i代表第i个人体关节点。为了便于理解，以下的某些计算过程只以一个关节点为例。
图像坐标系是摄像机成像平面中的二维坐标系，原点通常位于画面的左上角，画面的宽(横向)为x轴，画面的高(纵向)为y轴，以像素点个数为单位。通过识别人体在画面中的位置，得到人体各节点在图像坐标系下的人体图像坐标(u,v)，u表示x轴第u个像素点，v表示y轴第v个像素点。
物理坐标获取模块200将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)。由于图像坐标系以像素点个数为单位，不利于计算，因此需要将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)，以方便计算。图像物理坐标系为以物理单位(例如米)表示的图像坐标系，可以理解是图像坐标系的平移坐标系，也是二维坐标系。
在其中一个实施例中，物理坐标获取模块200具体用于：
将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转换关系为：
Figure PCTCN2018102854-appb-000005
其中上述公式为矩阵运算公式，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
人体距离获取模300块根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′。
此处的关键人体关节点为上述人体关节点中的至少一个，例如可以是上述人体关节点中的全部关节点，或者几个重要的关节点，例如头部、肩部、手肘、手腕、膝盖、脚踝。在一些实施例中，为了方便，可以只选取其中一个重要的关节点作为关键人体关节点，例如头部。获取了关键人体关节点到机器人的距离，即相当于获取了人体到机器人的距离。
在其中一个实施例中，人体距离获取模块300具体用于：
确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
Figure PCTCN2018102854-appb-000006
Figure PCTCN2018102854-appb-000007
d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。摄像机坐标系是三维坐标系，其原点为摄像机的光心，其X轴、Y轴与图像坐标系的x、y轴平行，z轴为摄像机的光轴，z轴与摄像机成像平面垂直。
垂直偏转角度α和水平偏转角度β是相对于摄像机坐标系而言的，计算垂直偏转角度α和水平偏转角度β后，激光发射器根据当前姿势以及α、β调整到相应位置以对准关键人体关节点，然后发射激光以测量人体到机器人的距离Z′。
摄像机坐标获取模块400根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)。
摄像机坐标系的成像平面的位置和激光坐标系的坐标原点在z轴方向的距离只相差一个d，忽略激光发射器和摄像机在位置微小偏差(通常激光发射器和摄像机在空间上距离不远)，激光测量值则为摄像机坐标系中对应的世界坐标系的真实值，我们可以根据摄像机坐标值和真实的世界坐标值的比例关系求出真实的世界坐标系中两人的间距。
在其中一个实施例中，摄像机坐标获取模块400具体用于：
根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
Figure PCTCN2018102854-appb-000008
f为摄像机焦距，Z＝Z′+d。
人群间距获取模块500根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L。可以根据摄像机坐标值和真实的世界坐标值的比例关系求出真实的世界坐标系中两人的间距L。
行走方向确定模块600根据任意两人的相应人体关节点之间的距离L确定行走方向。当确定了各人位置后，从而确定了任意两人之间的距离L，机器人此时便可以确定行走策略。在确定行走策略时，可以结合人体与机器人之间的距离、人体行走速度、机器人行走速度等至少一个因素分析。
假设人体关节点人体图像坐标为{(x k1,y k1),...,(x ki,y ki),...,(x kn,y kn)}，分别计算其中任意两人的间距，最后选取其中最佳的间隔距离通过。
假设计算P 1P 2两人之间的间隔，其中：
P 1＝{(x 11,y 11),...,(x 1i,y 1i),...,(x 1n,y 1n)}
P 2＝{(x 21,y 21),…,(x 2i,y 2i),…,(x 2n,y 2n)}
由于两人之间的距离和X轴方向的距离成正比，因此可以只关注X轴坐标，所以可以将两人的所有检测到的点组成一个队列：
P＝{x 11,x 21,…,x 1i,x 2i,…,x 1n,x 2n}
行走方向确定模块600运用快速排序算法，可以快速找到两个离散点群之间最 近或者交叉的点，这样就可以根据前面所述摄像机坐标值和真实的世界坐标值的比例关系算出这些点之间的距离，并且不停的检测每两人之间的距离，就能实时的找出最佳的通过距离。
本申请还提供一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行上述任一项实施例所述机器人行走障碍检测方法的步骤。
本申请还提供一种存储有计算机可读指令的存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行上述任一项实施例所述机器人行走障碍检测方法的步骤。
上述的机器人行走障碍检测方法、装置、计算机设备和存储介质，通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；根据任意两人的相应人体关节点之间的距离L确定行走方向。通过结合图像识别和激光测距确定人体实际坐标(在摄像机坐标系的坐标)，可以精确得到人体的实时位置，从而确定任意两人之间的间距L，再通过L来实时判断行走路径，实现了精确的行人避障检测。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，该计算机程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，前述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory，ROM)等非易失性存储介质，或随机存储记忆体(Random Access Memory，RAM)等。

Claims (16)

  1. 一种机器人行走障碍检测方法，所述机器人具有摄像机、激光发射器、激光接收器，所述方法包括如下步骤：
    步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；
    步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；
    步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；
    步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；
    步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；
    步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
  2. 根据权利要求1所述的机器人行走障碍检测方法，所述步骤S20包括：
    将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转换关系为：
    Figure PCTCN2018102854-appb-100001
    其中，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
  3. 根据权利要求2所述的机器人行走障碍检测方法，所述步骤S30包括：
    确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
    Figure PCTCN2018102854-appb-100002
    Figure PCTCN2018102854-appb-100003
    d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。
  4. 根据权利要求3所述的机器人行走障碍检测方法，所述步骤S40包括：
    根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
    Figure PCTCN2018102854-appb-100004
    f为摄像机焦距，Z＝Z′+d。
  5. 一种机器人行走障碍检测装置，所述机器人具有摄像机、激光发射器、激光接收器，所述装置包括：
    图像坐标获取模块，用于通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；
    物理坐标获取模块，用于将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；
    人体距离获取模块，用于根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；
    摄像机坐标获取模块，用于根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；
    人群间距获取模块，用于根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；
    行走方向确定模块，用于根据任意两人的相应人体关节点之间的距离L确定行走方向。
  6. 根据权利要求5所述的机器人行走障碍检测装置，所述物理坐标获取模块用于：
    将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转换关系为：
    Figure PCTCN2018102854-appb-100005
    其中，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
  7. 根据权利要求6所述的机器人行走障碍检测装置，所述人体距离获取模块用于：
    确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
    Figure PCTCN2018102854-appb-100006
    Figure PCTCN2018102854-appb-100007
    d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。
  8. 根据权利要求7所述的机器人行走障碍检测装置，所述摄像机坐标获取模块用于：
    根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
    Figure PCTCN2018102854-appb-100008
    f为摄像机焦距，Z＝Z′+d。
  9. 一种计算机设备，包括存储器和处理器，所述存储器中存储有计算机可读指令，所述计算机可读指令被所述处理器执行时，使得所述处理器执行一种机器人行走障碍检测方法，所述机器人行走障碍检测方法包括以下步骤：
    步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；
    步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；
    步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；
    步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；
    步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；
    步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
  10. 根据权利要求9所述的计算机设备，所述步骤S20包括：
    将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转换关系为：
    Figure PCTCN2018102854-appb-100009
    其中，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
  11. 根据权利要求10所述的计算机设备，所述步骤S30包括：
    确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
    Figure PCTCN2018102854-appb-100010
    Figure PCTCN2018102854-appb-100011
    d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。
  12. 根据权利要求11所述的计算机设备，所述步骤S40包括：
    根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
    Figure PCTCN2018102854-appb-100012
    f为摄像机焦距，Z＝Z′+d。
  13. 一种存储有计算机可读指令的非易失性存储介质，所述计算机可读指令被一个或多个处理器执行时，使得一个或多个处理器执行一种机器人行走障碍检测方法，所述机器人行走障碍检测方法包括以下步骤：
    步骤S10：通过摄像机获取图像，应用图像人体检测算法得到人体关节点在图像坐标系下的人体图像坐标(u,v)；
    步骤S20：将图像坐标系下的人体图像坐标(u,v)转换为图像物理坐标系下的图像物理坐标(x,y)；
    步骤S30：根据x、y确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′；
    步骤S40：根据x、y、Z′计算得到人体关节点在摄像机坐标系的坐标(X,Y,Z)；
    步骤S50：根据任意两人的相应人体关节点在摄像机坐标系的坐标(X 1,Y 1,Z 1)和(X 2,Y 2,Z 2)计算任意两人的相应人体关节点之间的距离L；
    步骤S60：根据任意两人的相应人体关节点之间的距离L确定行走方向。
  14. 根据权利要求13所述的非易失性存储介质，所述步骤S20包括：
    将图像坐标系下的人体图像坐标转换为图像物理坐标系下的图像物理坐标，转换关系为：
    Figure PCTCN2018102854-appb-100013
    其中，(u,v)为图像坐标系下的人体图像坐标，(x,y)为在图像物理坐标系下的人体图像坐标，u 0和v 0为摄像机光轴在图像坐标系下的坐标值，dx和dy为每个像素在x轴和y轴的物理尺寸值。
  15. 根据权利要求14所述的非易失性存储介质，所述步骤S30包括：
    确定激光发射器照射关键人体关节点的垂直偏转角度α和水平偏转角度β，启动激光发射器发射激光以测量人体到机器人的距离Z′，其中：
    Figure PCTCN2018102854-appb-100014
    Figure PCTCN2018102854-appb-100015
    d u是摄像机坐标系相对于图像物理坐标系的偏移量，d v是激光坐标系相对于图像物理坐标系的偏移量，d为激光发射器到摄像机成像平面的水平距离。
  16. 根据权利要求15所述的非易失性存储介质，所述步骤S40包括：
    根据x、y、d、Z′计算得到各个人体关节点在摄像机坐标系的坐标(X,Y,Z)，根据(X,Y,Z)计算得到任意两人的相应人体关节点之间的距离L，其中摄像机坐标系的坐标(X,Y,Z)与图像物理坐标系的坐标(x,y)之间的关系为：
    Figure PCTCN2018102854-appb-100016
    f为摄像机焦距，Z＝Z′+d。
PCT/CN2018/102854 2018-04-10 2018-08-29 机器人行走障碍检测方法、装置、计算机设备和存储介质 Ceased WO2019196313A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810314149.5 2018-04-10
CN201810314149.5A CN108628306B (zh) 2018-04-10 2018-04-10 机器人行走障碍检测方法、装置、计算机设备和存储介质

Publications (1)

Publication Number Publication Date
WO2019196313A1 true WO2019196313A1 (zh) 2019-10-17

Family

ID=63704999

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/102854 Ceased WO2019196313A1 (zh) 2018-04-10 2018-08-29 机器人行走障碍检测方法、装置、计算机设备和存储介质

Country Status (2)

Country Link
CN (1) CN108628306B (zh)
WO (1) WO2019196313A1 (zh)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111324114A (zh) * 2020-01-22 2020-06-23 南宁职业技术学院 扫地机器人及其路径规划方法
CN112116529A (zh) * 2020-09-23 2020-12-22 浙江浩腾电子科技股份有限公司 一种基于ptz相机的gps坐标与像素坐标的转换方法
CN112230652A (zh) * 2020-09-04 2021-01-15 安克创新科技股份有限公司 行走机器人及其移动控制的方法、计算机存储介质
CN113506340A (zh) * 2021-06-15 2021-10-15 浙江大华技术股份有限公司 一种云台位姿预测的方法、设备和计算机可读存储介质

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109508688B (zh) * 2018-11-26 2023-10-13 平安科技（深圳）有限公司 基于骨架的行为检测方法、终端设备及计算机存储介质
CN110119682A (zh) * 2019-04-04 2019-08-13 北京理工雷科电子信息技术有限公司 一种红外遥感图像火点识别方法
WO2021217311A1 (zh) * 2020-04-26 2021-11-04 深圳市大疆创新科技有限公司 可移动平台及其控制方法、存储介质
CN113268063A (zh) * 2021-06-03 2021-08-17 北京京东乾石科技有限公司 机器人的控制方法、装置和非易失性计算机可读存储介质
CN114968218B (zh) * 2022-04-18 2025-02-07 福建天晴在线互动科技有限公司 一种实现界面控件跨层级水平等距移动的方法及其系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6250911A (ja) * 1985-08-30 1987-03-05 Kajima Corp 自走式ロボツト
CN104375505A (zh) * 2014-10-08 2015-02-25 北京联合大学 一种基于激光测距的机器人自主寻路方法
CN105550667A (zh) * 2016-01-25 2016-05-04 同济大学 一种基于立体相机的骨架信息动作特征提取方法
CN105676845A (zh) * 2016-01-19 2016-06-15 中国人民解放军国防科学技术大学 一种智能安保服务机器人复杂环境避障方法及安保服务机器人
CN106020207A (zh) * 2016-07-26 2016-10-12 广东宝乐机器人股份有限公司 自移动机器人行走方法与装置
CN106291535A (zh) * 2016-07-21 2017-01-04 触景无限科技（北京）有限公司 一种障碍物检测装置、机器人及避障系统
CN106313046A (zh) * 2016-09-27 2017-01-11 成都普诺思博科技有限公司 移动机器人多级避障系统
CN207182092U (zh) * 2017-05-09 2018-04-03 叶仕通 一种用于移动机器人的驱动装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102499692B (zh) * 2011-11-30 2014-04-30 沈阳工业大学 一种超声波步态检测方法
KR101913332B1 (ko) * 2011-12-23 2018-10-31 삼성전자주식회사 이동 장치 및 이동 장치의 위치 인식 방법
JP6027703B1 (ja) * 2016-09-12 2016-11-16 シャープ株式会社 情報処理装置および制御プログラム
CN206224246U (zh) * 2016-10-19 2017-06-06 九阳股份有限公司 一种实现目标定位与跟踪的机器人
CN107423729B (zh) * 2017-09-20 2023-12-19 湖南师范大学 一种面向复杂视觉场景下的远距离类脑三维步态识别系统及实现方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6250911A (ja) * 1985-08-30 1987-03-05 Kajima Corp 自走式ロボツト
CN104375505A (zh) * 2014-10-08 2015-02-25 北京联合大学 一种基于激光测距的机器人自主寻路方法
CN105676845A (zh) * 2016-01-19 2016-06-15 中国人民解放军国防科学技术大学 一种智能安保服务机器人复杂环境避障方法及安保服务机器人
CN105550667A (zh) * 2016-01-25 2016-05-04 同济大学 一种基于立体相机的骨架信息动作特征提取方法
CN106291535A (zh) * 2016-07-21 2017-01-04 触景无限科技（北京）有限公司 一种障碍物检测装置、机器人及避障系统
CN106020207A (zh) * 2016-07-26 2016-10-12 广东宝乐机器人股份有限公司 自移动机器人行走方法与装置
CN106313046A (zh) * 2016-09-27 2017-01-11 成都普诺思博科技有限公司 移动机器人多级避障系统
CN207182092U (zh) * 2017-05-09 2018-04-03 叶仕通 一种用于移动机器人的驱动装置

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111324114A (zh) * 2020-01-22 2020-06-23 南宁职业技术学院 扫地机器人及其路径规划方法
CN112230652A (zh) * 2020-09-04 2021-01-15 安克创新科技股份有限公司 行走机器人及其移动控制的方法、计算机存储介质
CN112116529A (zh) * 2020-09-23 2020-12-22 浙江浩腾电子科技股份有限公司 一种基于ptz相机的gps坐标与像素坐标的转换方法
CN113506340A (zh) * 2021-06-15 2021-10-15 浙江大华技术股份有限公司 一种云台位姿预测的方法、设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN108628306A (zh) 2018-10-09
CN108628306B (zh) 2021-06-25

Similar Documents

Publication Publication Date Title
WO2019196313A1 (zh) 机器人行走障碍检测方法、装置、计算机设备和存储介质
Jörgensen et al. Monocular 3d object detection and box fitting trained end-to-end using intersection-over-union loss
Zou et al. Coslam: Collaborative visual slam in dynamic environments
JP7292492B2 (ja) オブジェクト追跡方法及び装置、記憶媒体並びにコンピュータプログラム
WO2017133009A1 (zh) 一种基于卷积神经网络的深度图像人体关节定位方法
CN111652047B (zh) 基于彩色图和深度图的人体姿势识别方法及存储介质
Marcu et al. A multi-stage multi-task neural network for aerial scene interpretation and geolocalization
Lu et al. 2-D/3-D fusion-based robust pose normalisation of 3-D livestock from multiple RGB-D cameras
CN104392223A (zh) 二维视频图像中的人体姿态识别方法
CN116740539A (zh) 基于轻量化目标检测网络的视觉slam方法及系统
Hu et al. Mobile panoramic vision for assisting the blind via indexing and localization
Gouidis et al. Accurate hand keypoint localization on mobile devices
Nguyen et al. Combined YOLOv5 and HRNet for high accuracy 2D keypoint and human pose estimation
Chiavazza et al. Low-latency monocular depth estimation using event timing on neuromorphic hardware
Lu et al. Automatic teat detection for rotary milking system based on deep learning algorithms
US12586231B2 (en) Posture estimation apparatus, posture estimation method, and computer-readable recording medium
Velesaca et al. Deep learning-based human height estimation from a stereo vision system
CN116152861A (zh) 一种基于单目深度估计的多人行为检测方法
CN114663917A (zh) 基于多视角的多人三维人体位姿估计方法及装置
Mao et al. Basic theories and methods of target’s height and distance measurement based on monocular vision
Wang et al. Im2fit: Fast 3d model fitting and anthropometrics using single consumer depth camera and synthetic data
CN115546829B (zh) 基于zed立体相机的行人空间信息感知方法及装置
Chen et al. A real-time photogrammetric system for acquisition and monitoring of three-dimensional human body kinematics
Korovin et al. Human pose estimation applying ANN while RGB-D cameras video handling
Sarsenov et al. Detection of Objects and Trajectories in Real-time using Deep Learning by a Controlled Robot.

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18914729

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18914729

Country of ref document: EP

Kind code of ref document: A1