WO2019196313A1 - Procédé et appareil de détection d'obstacle à la marche d'un robot, dispositif informatique, et support d'informations - Google Patents

Procédé et appareil de détection d'obstacle à la marche d'un robot, dispositif informatique, et support d'informations Download PDF

Info

Publication number
WO2019196313A1
WO2019196313A1 PCT/CN2018/102854 CN2018102854W WO2019196313A1 WO 2019196313 A1 WO2019196313 A1 WO 2019196313A1 CN 2018102854 W CN2018102854 W CN 2018102854W WO 2019196313 A1 WO2019196313 A1 WO 2019196313A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
coordinate system
human body
coordinates
camera
Prior art date
Application number
PCT/CN2018/102854
Other languages
English (en)
Chinese (zh)
Inventor
曾伟
周宝
王健宗
肖京
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2019196313A1 publication Critical patent/WO2019196313A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Definitions

  • the present application relates to the field of robot obstacle avoidance technology, and in particular, to a robot walking obstacle detection method and apparatus, and a computer device and a storage medium storing computer readable instructions.
  • the obstacle detection of the robot is the key factor for the successful navigation movement of the robot.
  • the pedestrian in the movement adds many difficulties to the obstacle avoidance of the robot.
  • the inventor realized that in the current robot obstacle detection method, there are many problems, such as limited range of ultrasonic detection, and it is difficult to comprehensively cover the three-dimensional space even if a large number is loaded; although the laser detection has high precision, the same problem exists.
  • the coverage is detected by the depth camera, but due to the huge data processing and visual depth of field limitation, there is also a problem that the obstacle detection accuracy is not high.
  • these methods are mainly for stationary objects, but the pedestrians are equated with other obstacle detection, which is more passive when formulating strategies to avoid pedestrians, reducing the efficiency of effective obstacle avoidance.
  • the purpose of the present application is to solve at least one of the above technical drawbacks, in particular, technical defects that are difficult to avoid obstacles.
  • the present application provides a robot walking obstacle detecting method, the robot having a camera, a laser emitter, and a laser receiver, the method comprising the following steps: Step S10: acquiring an image through a camera, applying an image human body detection algorithm to obtain a human joint point Human body image coordinates (u, v) in the image coordinate system; step S20: converting human body image coordinates (u, v) in the image coordinate system into image physical coordinates (x, y) in the image physical coordinate system; step S30 : determining, according to x, y, the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter irradiating the key joint points of the human body, starting the laser emitter to emit laser light to measure the distance Z′ of the human body to the robot; step S40: according to x, y, Z 'calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system; step S50: according to the coordinates (X 1 , Y
  • the present application also provides a robot walking obstacle detecting device, the robot having a camera, a laser emitter, and a laser receiver, the device comprising: an image coordinate acquiring module, configured to acquire an image through a camera, and apply an image human body detecting algorithm to obtain a human body The human body image coordinates (u, v) of the off node in the image coordinate system; the physical coordinate acquisition module is used to convert the human body image coordinates (u, v) in the image coordinate system into the image physical coordinates in the image physical coordinate system ( x, y); human body distance acquisition module for determining the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key joint points of the human body according to x, y, and starting the laser emitter to emit laser light to measure the distance from the human body to the robot Z a camera coordinate acquisition module for calculating coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'; a crowd spacing acquisition module
  • the application also provides a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the processor to perform a robot walking
  • the obstacle detecting method includes the following steps: Step S10: acquiring an image by a camera, and applying an image human body detecting algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system; Step S20: Converting the human body image coordinates (u, v) in the image coordinate system to the image physical coordinates (x, y) in the image physical coordinate system; Step S30: determining the vertical deflection of the laser emitter illuminating the key human joint points according to x, y The angle ⁇ and the horizontal deflection angle ⁇ start the laser emitter to emit laser light to measure the distance Z′ of the human body to the robot; step S40: calculate the coordinates of the human joint point in the camera coordinate system according to x, y, Z′ (X, Y, Z); Step
  • the present application also provides a non-volatile storage medium storing computer readable instructions that, when executed by one or more processors, cause one or more processors to perform a robotic walking obstacle detection
  • the method for detecting a walking obstacle of a robot includes the following steps: Step S10: acquiring an image by a camera, applying an image human body detection algorithm to obtain body image coordinates (u, v) of a human joint point in an image coordinate system; and step S20: placing the image The human body image coordinates (u, v) in the coordinate system are converted into image physical coordinates (x, y) in the image physical coordinate system; step S30: determining the vertical deflection angle ⁇ of the laser emitter irradiating the key human joint points according to x, y And the horizontal deflection angle ⁇ , the laser emitter is activated to emit laser light to measure the distance Z' of the human body to the robot; step S40: calculating the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to
  • the above-mentioned robot walking obstacle detecting method, device, computer equipment and storage medium by combining image recognition and laser ranging to determine the actual coordinates of the human body (in the coordinate of the camera coordinate system), the real-time position of the human body can be accurately obtained, thereby determining any two people.
  • the distance between the two is used to judge the walking path in real time, and accurate pedestrian obstacle detection is realized.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment
  • FIG. 2 is a schematic flow chart of a method for detecting a walking obstacle of a robot according to an embodiment
  • FIG. 3 is a schematic diagram of a human body detection process of an embodiment
  • FIG. 4 is a diagram showing an example of human body detection of an embodiment
  • FIG. 5 is a schematic diagram of a module of a robot walking obstacle detecting device according to an embodiment.
  • FIG. 1 is a schematic diagram showing the internal structure of a computer device in an embodiment.
  • the computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus.
  • the non-volatile storage medium of the computer device stores an operating system, a database, and computer readable instructions.
  • the database may store a sequence of control information.
  • the processor may implement a processor.
  • a robot walking obstacle detection method The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device.
  • Computer readable instructions may be stored in the memory of the computer device, the computer readable instructions being executable by the processor to cause the processor to perform a robotic walking obstacle detection method.
  • the network interface of the computer device is used to communicate with the terminal connection. It will be understood by those skilled in the art that the structure shown in FIG. 1 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation of the computer device to which the solution of the present application is applied.
  • the specific computer device may It includes more or fewer components than those shown in the figures, or some components are combined, or have different component arrangements.
  • the robot walking obstacle detecting method described below can be applied to an intelligent robot such as a customer service robot, a sweeping robot, and the like.
  • FIG. 2 is a schematic flow chart of a method for detecting a walking obstacle of a robot according to an embodiment.
  • a robot walking obstacle detecting method the robot having a camera, a laser emitter, and a laser receiver, the method comprising the following steps:
  • Step S10 acquiring an image by using a camera, and applying an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system.
  • the image human body detection algorithm refers to a technique for recognizing a human body through image recognition technology.
  • This type of method is currently the more mainstream human detection method. It mainly uses various static features of images such as edge features, shape features, statistical features or transform features to describe the human body. Representative features include Haar wavelet features, HOG features, and Edgelet. Features, Shapelet features, and outline template features.
  • Papageorgiou and Poggio first proposed the concept of Harr wavelet, Viola et al. introduced the concept of integral graph, accelerated the extraction speed of Harr features, and applied the method to human detection, combined with human motion and appearance mode to construct human detection system. The better detection effect lays the foundation for the development of human detection technology.
  • HOG Histogram of Oriented Gradients
  • the "Edgelet” feature which is a short line or curve segment, is applied to human detection of a single image of a complex scene, yielding a detection rate of approximately 92% on the CAVIAR database.
  • Shapelet features can be automatically obtained using machine learning methods, namely Shapelet features.
  • the algorithm first extracts the gradient information of different directions from the training samples, and then uses the AdaBoost algorithm to train, thus obtaining the Shapelet feature.
  • the method refers to constructing a template by using information such as edge contour, texture and gray scale of the target object in the image, and detecting the target by template matching.
  • the basic idea of this kind of method is to divide the human body into several components, and then separately test each part of the image, and finally integrate the detection results according to a certain constraint relationship, and finally determine whether there is a human body.
  • This type of method refers to image acquisition by two or more cameras, and then analyzing the three-dimensional information of the target in the image to identify the human body.
  • the image human body detection algorithm is not specifically limited as long as the human body and its joint points can be identified.
  • Human joint points refer to key nodes of the human body such as the head, neck, shoulders, upper arms, elbows, wrists, waist, chest, thighs, knees, calves, ankles, etc. There are multiple.
  • human body detection can be performed using a method based on a human body part.
  • OpenPose is an open source library written in C++ using OpenCV and Caffe for real-time detection of multi-threaded multi-person key points.
  • the main detection of human body detection consists of two parts: human joint point detection and joint point correlation detection.
  • the joint point detection uses a Convolutional Pose Machine.
  • the attitude machine can infer the position of the key points of the human body by adding the convolutional neural network, using the texture information of the image, the spatial structure information and the center map. Simultaneously, a convolutional neural network is used in parallel to detect the correlation between joint points, ie which joint points should be connected together.
  • Fig. a is a complete input image
  • Fig. b is a confidence feature map of joint points
  • Fig. c is a joint point correlation feature map
  • Fig. d is a series of candidate joint point matching after sparse method
  • figure, figure e is the figure after finally completing the combined connection of the joint points of the human body.
  • an image is acquired by the camera, and an image human body detection algorithm is applied to obtain a series of human body image coordinates ⁇ (u k1 , v k1 ), ..., (u ki , v ki ),...,(u kn ,v kn ) ⁇ , where k represents the kth person and i represents the i-th body joint point.
  • k represents the kth person
  • i represents the i-th body joint point.
  • the image coordinate system is a two-dimensional coordinate system in the imaging plane of the camera.
  • the origin is usually located at the upper left corner of the screen.
  • the width (transverse) of the screen is the x-axis
  • the height (longitudinal) of the image is the y-axis, in units of pixels.
  • Step S20 Convert the human body image coordinates (u, v) in the image coordinate system to the image physical coordinates (x, y) in the image physical coordinate system. Since the image coordinate system is based on the number of pixels, which is not conducive to calculation, it is necessary to convert the human body image coordinates (u, v) in the image coordinate system into the physical coordinates (x, y) of the image in the image physical coordinate system. Easy to calculate.
  • the image physical coordinate system is an image coordinate system expressed in physical units (for example, meters), which can be understood as a translation coordinate system of an image coordinate system, and is also a two-dimensional coordinate system.
  • step S20 includes:
  • the human body image coordinates in the image coordinate system are converted into image physical coordinates in the image physical coordinate system, and the conversion relationship is:
  • the above formula is a matrix operation formula
  • (u, v) is the human body image coordinate in the image coordinate system
  • (x, y) is the human body image coordinate in the image physical coordinate system
  • u 0 and v 0 are the camera optical axis.
  • the coordinate values in the image coordinate system, dx and dy are the physical size values of the x-axis and y-axis of each pixel.
  • Step S30 Determine, according to x, y, the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key joint points of the human body, and activate the laser emitter to emit laser light to measure the distance Z' of the human body to the robot.
  • the key human joint point here is at least one of the above-mentioned human joint points, for example, all joint points in the above-mentioned human joint points, or several important joint points, such as a head, a shoulder, an elbow, a wrist, Knees and ankles. In some embodiments, for convenience, only one of the important joint points may be selected as a key human joint point, such as a head.
  • the distance from the key human joint point to the robot is obtained, which is equivalent to the distance from the human body to the robot.
  • step S30 includes:
  • d u is the offset of the camera coordinate system relative to the image physical coordinate system
  • d v is the offset of the laser coordinate system relative to the image physical coordinate system
  • d is the horizontal distance of the laser emitter to the imaging plane of the camera.
  • the camera coordinate system is a three-dimensional coordinate system whose origin is the optical center of the camera.
  • the X and Y axes are parallel to the x and y axes of the image coordinate system.
  • the z axis is the optical axis of the camera, and the z axis is perpendicular to the imaging plane of the camera.
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ are relative to the camera coordinate system.
  • the laser emitter After calculating the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ , the laser emitter adjusts to the corresponding position according to the current posture and ⁇ , ⁇ to align the key human body.
  • the node is turned off and then a laser is emitted to measure the distance Z' of the human body to the robot.
  • each human joint point can be obtained, so that the specific position (X, Y, Z) of the joint point of the human body can be determined through subsequent steps, and the spacing between any two persons can be determined.
  • Step S40 Calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, and Z'.
  • the position of the imaging plane of the camera coordinate system and the coordinate origin of the laser coordinate system differ only by one d in the z-axis direction, ignoring the slight deviation of the position between the laser emitter and the camera (usually the laser emitter and the camera are not far apart in space)
  • the laser measurement value is the true value of the corresponding world coordinate system in the camera coordinate system. We can find the distance between two people in the real world coordinate system according to the proportional relationship between the camera coordinate value and the real world coordinate value.
  • step S40 includes:
  • the coordinates (X, Y, Z) of each human joint point in the camera coordinate system are calculated, and the corresponding human joint points between any two people are calculated according to (X, Y, Z).
  • the distance L where the coordinates of the camera coordinate system (X, Y, Z) and the coordinates (x, y) of the image physical coordinate system are:
  • Step S50 Calculate between the corresponding human joint points of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance L The distance L between two people in the real world coordinate system can be obtained from the proportional relationship between the camera coordinate value and the real world coordinate value.
  • Step S60 determining the walking direction according to the distance L between the respective human joint points of any two persons.
  • the robot can determine the walking strategy at this time.
  • at least one factor analysis such as the distance between the human body and the robot, the walking speed of the human body, and the walking speed of the robot may be combined.
  • the body image coordinates of the human joint point are ⁇ (x k1 , y k1 ),...,(x ki ,y ki ),...,(x kn ,y kn ) ⁇ , and calculate the spacing of any two of them. Finally, select the best separation distance to pass.
  • FIG. 5 is a schematic diagram of a module of a robot walking obstacle detecting device according to an embodiment.
  • the present application further provides a robot walking obstacle detecting device, the robot having a camera, a laser emitter, and a laser receiver, the device comprising: an image coordinate acquiring module 100 and a physical coordinate acquiring module 200.
  • the image coordinate acquiring module 100 is configured to acquire an image by using a camera, and apply an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system; the physical coordinate acquiring module is configured to use the human body image in the image coordinate system.
  • the coordinates (u, v) are converted into image physical coordinates (x, y) in the physical coordinate system of the image; the human body distance acquisition module is configured to determine the vertical deflection angle ⁇ and the horizontal deflection of the laser emitter to illuminate the key joint points of the human body according to x and y.
  • the camera coordinate acquisition module is used to calculate the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'
  • the crowd spacing acquisition module is configured to calculate the corresponding human body of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance L between the nodes is closed; the walking direction determining module is configured to determine the walking direction according to the distance L between the respective human joint points of any two persons.
  • the image coordinate acquisition module 100 acquires an image through a camera, and applies an image human body detection algorithm to obtain human body image coordinates (u, v) of the human joint point in the image coordinate system.
  • the image human body detection algorithm refers to a technique for recognizing a human body through image recognition technology.
  • This type of method is currently the more mainstream human detection method. It mainly uses various static features of images such as edge features, shape features, statistical features or transform features to describe the human body. Representative features include Haar wavelet features, HOG features, and Edgelet. Features, Shapelet features, and outline template features.
  • Papageorgiou and Poggio first proposed the concept of Harr wavelet, Viola et al. introduced the concept of integral graph, accelerated the extraction speed of Harr features, and applied the method to human detection, combined with human motion and appearance mode to construct human detection system. The better detection effect lays the foundation for the development of human detection technology.
  • HOG Histogram of Oriented Gradients
  • the "Edgelet” feature which is a short line or curve segment, is applied to human detection of a single image of a complex scene, yielding a detection rate of approximately 92% on the CAVIAR database.
  • Shapelet features can be automatically obtained using machine learning methods, namely Shapelet features.
  • the algorithm first extracts the gradient information of different directions from the training samples, and then uses the AdaBoost algorithm to train, thus obtaining the Shapelet feature.
  • the method refers to constructing a template by using information such as edge contour, texture and gray level of the target object in the image, and detecting the target by template matching method.
  • the basic idea of this kind of method is to divide the human body into several components, and then separately test each part of the image, and finally integrate the detection results according to a certain constraint relationship, and finally determine whether there is a human body.
  • This type of method refers to image acquisition by two or more cameras, and then analyzing the three-dimensional information of the target in the image to identify the human body.
  • the image human body detection algorithm is not specifically limited as long as the human body and its joint points can be identified.
  • Human joint points refer to key nodes of the human body such as the head, neck, shoulders, upper arms, elbows, wrists, waist, chest, thighs, knees, calves, ankles, etc. There are multiple.
  • human body detection can be performed using a method based on a human body part.
  • OpenPose is an open source library written in C++ using OpenCV and Caffe for real-time detection of multi-threaded multi-person key points.
  • the main detection of human body detection consists of two parts: human joint point detection and joint point correlation detection.
  • the joint point detection uses a Convolutional Pose Machine.
  • the attitude machine can infer the position of the key points of the human body by adding the convolutional neural network, using the texture information of the image, the spatial structure information and the center map. Simultaneously, a convolutional neural network is used in parallel to detect the correlation between joint points, ie which joint points should be connected together.
  • Fig. a is a complete input image
  • Fig. b is a confidence feature map of joint points
  • Fig. c is a joint point correlation feature map
  • Fig. d is a series of candidate joint point matching after sparse method
  • figure, figure e is the figure after finally completing the combined connection of the joint points of the human body.
  • an image is acquired by the camera, and an image human body detection algorithm is applied to obtain a series of human body image coordinates ⁇ (u k1 , v k1 ), ..., (u ki , v ki ),...,(u kn ,v kn ) ⁇ , where k represents the kth person and i represents the i-th body joint point.
  • k represents the kth person
  • i represents the i-th body joint point.
  • the image coordinate system is a two-dimensional coordinate system in the imaging plane of the camera.
  • the origin is usually located at the upper left corner of the screen.
  • the width (transverse) of the screen is the x-axis
  • the height (longitudinal) of the image is the y-axis, in units of pixels.
  • the physical coordinate acquisition module 200 converts the human body image coordinates (u, v) in the image coordinate system into image physical coordinates (x, y) in the image physical coordinate system. Since the image coordinate system is based on the number of pixels, which is not conducive to calculation, it is necessary to convert the human body image coordinates (u, v) in the image coordinate system into the physical coordinates (x, y) of the image in the image physical coordinate system. Easy to calculate.
  • the image physical coordinate system is an image coordinate system expressed in physical units (for example, meters), which can be understood as a translation coordinate system of an image coordinate system, and is also a two-dimensional coordinate system.
  • the physical coordinate acquisition module 200 is specifically configured to:
  • the human body image coordinates in the image coordinate system are converted into image physical coordinates in the image physical coordinate system, and the conversion relationship is:
  • the above formula is a matrix operation formula
  • (u, v) is the human body image coordinate in the image coordinate system
  • (x, y) is the human body image coordinate in the image physical coordinate system
  • u 0 and v 0 are the camera optical axis.
  • the coordinate values in the image coordinate system, dx and dy are the physical size values of the x-axis and y-axis of each pixel.
  • the human body distance acquisition module 300 determines the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter to illuminate the key human joint points according to x, y, and activates the laser emitter to emit laser light to measure the distance Z' of the human body to the robot.
  • the key human joint point here is at least one of the above-mentioned human joint points, for example, all joint points in the above-mentioned human joint points, or several important joint points, such as a head, a shoulder, an elbow, a wrist, Knees and ankles. In some embodiments, for convenience, only one of the important joint points may be selected as a key human joint point, such as a head.
  • the distance from the key human joint point to the robot is obtained, which is equivalent to the distance from the human body to the robot.
  • the human body distance acquisition module 300 is specifically configured to:
  • d u is the offset of the camera coordinate system relative to the image physical coordinate system
  • d v is the offset of the laser coordinate system relative to the image physical coordinate system
  • d is the horizontal distance of the laser emitter to the imaging plane of the camera.
  • the camera coordinate system is a three-dimensional coordinate system whose origin is the optical center of the camera.
  • the X and Y axes are parallel to the x and y axes of the image coordinate system.
  • the z axis is the optical axis of the camera, and the z axis is perpendicular to the imaging plane of the camera.
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ are relative to the camera coordinate system.
  • the laser emitter After calculating the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ , the laser emitter adjusts to the corresponding position according to the current posture and ⁇ , ⁇ to align the key human body.
  • the node is turned off and then a laser is emitted to measure the distance Z' of the human body to the robot.
  • the camera coordinate acquisition module 400 calculates the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, and Z'.
  • the position of the imaging plane of the camera coordinate system and the coordinate origin of the laser coordinate system differ only by one d in the z-axis direction, ignoring the slight deviation of the position between the laser emitter and the camera (usually the laser emitter and the camera are not far apart in space)
  • the laser measurement value is the true value of the corresponding world coordinate system in the camera coordinate system. We can find the distance between two people in the real world coordinate system according to the proportional relationship between the camera coordinate value and the real world coordinate value.
  • the camera coordinate acquisition module 400 is specifically configured to:
  • the coordinates (X, Y, Z) of each human joint point in the camera coordinate system are calculated, and the corresponding human joint points between any two people are calculated according to (X, Y, Z).
  • the distance L where the coordinates of the camera coordinate system (X, Y, Z) and the coordinates (x, y) of the image physical coordinate system are:
  • the crowd spacing acquisition module 500 calculates the corresponding human joint points of any two persons according to the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system of the corresponding human joint points of any two persons.
  • the distance between L The distance L between two people in the real world coordinate system can be obtained from the proportional relationship between the camera coordinate value and the real world coordinate value.
  • the walking direction determining module 600 determines the walking direction based on the distance L between the respective human joint points of any two persons. When the position of each person is determined, thereby determining the distance L between any two people, the robot can determine the walking strategy at this time. When determining the walking strategy, at least one factor analysis such as the distance between the human body and the robot, the walking speed of the human body, and the walking speed of the robot may be combined.
  • the body image coordinates of the human joint point are ⁇ (x k1 , y k1 ),...,(x ki ,y ki ),...,(x kn ,y kn ) ⁇ , and calculate the spacing of any two of them. Finally, select the best separation distance to pass.
  • the walking direction determining module 600 uses the quick sorting algorithm to quickly find the nearest or intersecting points between the two discrete point groups, so that the distance between the camera coordinates and the real world coordinate values can be calculated between the points.
  • the distance and the distance between each two are constantly detected, and the best passing distance can be found in real time.
  • the application also provides a computer device comprising a memory and a processor, the memory storing computer readable instructions, the computer readable instructions being executed by the processor, causing the processor to perform any of the above The steps of the robot walking obstacle detecting method according to the embodiment.
  • the present application also provides a storage medium storing computer readable instructions that, when executed by one or more processors, cause one or more processors to perform the robotic walking described in any of the above embodiments The steps of the obstacle detection method.
  • the above-mentioned robot walking obstacle detection method, device, computer equipment and storage medium acquire images through a camera, and apply image human body detection algorithm to obtain human body image coordinates (u, v) of human joint points in an image coordinate system;
  • the human body image coordinates (u, v) are converted into image physical coordinates (x, y) in the image physical coordinate system;
  • the vertical deflection angle ⁇ and the horizontal deflection angle ⁇ of the laser emitter irradiating the key human joint points are determined according to x, y, Starting the laser emitter to emit laser light to measure the distance Z' of the human body to the robot; calculating the coordinates (X, Y, Z) of the human joint point in the camera coordinate system according to x, y, Z'; according to the corresponding human joint of any two people Point the coordinates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) of the camera coordinate system to calculate the distance L between the corresponding human joint points of
  • the real-time position of the human body can be accurately obtained, thereby determining the spacing L between any two people, and then determining the walking path in real time through L, realizing Accurate pedestrian obstacle detection.
  • the storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Manipulator (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne un procédé et un appareil de détection d'obstacle à la marche d'un robot. Le procédé consiste : à acquérir une image au moyen d'une caméra, et à utiliser un algorithme de détection d'image de corps humain pour obtenir des coordonnées d'image de corps humain (u, v) d'un point d'articulation de corps humain dans un système de coordonnées d'image (S10); à convertir les coordonnées d'image de corps humain (u, v) sous le système de coordonnées d'image en coordonnées physiques d'image (x, y) sous un système de coordonnées physiques d'image (S20); à déterminer un angle de déviation vertical α et un angle de déviation horizontal β d'un émetteur laser irradiant le point d'articulation de corps humain clé selon x et y, et à démarrer l'émetteur laser pour émettre un laser destiné à mesurer une distance Z' du corps humain au robot (S30); à calculer des coordonnées (X, Y, Z) du point d'articulation du corps humain dans le système de coordonnées de la caméra en fonction des coordonnées X, y et Z' (S40); à calculer, en fonction des coordonnées (X1, Y1, Z1) et (X2, Y2, Z2) des points d'articulation de corps humain correspondants de deux personnes quelconques dans le système de coordonnées de la caméra, une distance L entre les points d'articulation de corps humain correspondants des deux personnes quelconques (S50); et à déterminer un sens de marche selon la distance L entre les points d'articulation de corps humain correspondants des deux personnes quelconques (S60). Un évitement précis d'obstacle piéton peut être obtenu. La présente invention porte également sur un dispositif informatique et un support d'informations.
PCT/CN2018/102854 2018-04-10 2018-08-29 Procédé et appareil de détection d'obstacle à la marche d'un robot, dispositif informatique, et support d'informations WO2019196313A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810314149.5A CN108628306B (zh) 2018-04-10 2018-04-10 机器人行走障碍检测方法、装置、计算机设备和存储介质
CN201810314149.5 2018-04-10

Publications (1)

Publication Number Publication Date
WO2019196313A1 true WO2019196313A1 (fr) 2019-10-17

Family

ID=63704999

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/102854 WO2019196313A1 (fr) 2018-04-10 2018-08-29 Procédé et appareil de détection d'obstacle à la marche d'un robot, dispositif informatique, et support d'informations

Country Status (2)

Country Link
CN (1) CN108628306B (fr)
WO (1) WO2019196313A1 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112116529A (zh) * 2020-09-23 2020-12-22 浙江浩腾电子科技股份有限公司 一种基于ptz相机的gps坐标与像素坐标的转换方法
CN112230652A (zh) * 2020-09-04 2021-01-15 安克创新科技股份有限公司 行走机器人及其移动控制的方法、计算机存储介质
CN113506340A (zh) * 2021-06-15 2021-10-15 浙江大华技术股份有限公司 一种云台位姿预测的方法、设备和计算机可读存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109508688B (zh) * 2018-11-26 2023-10-13 平安科技（深圳）有限公司 基于骨架的行为检测方法、终端设备及计算机存储介质
CN110119682A (zh) * 2019-04-04 2019-08-13 北京理工雷科电子信息技术有限公司 一种红外遥感图像火点识别方法
CN112753210A (zh) * 2020-04-26 2021-05-04 深圳市大疆创新科技有限公司 可移动平台及其控制方法、存储介质
CN113268063A (zh) * 2021-06-03 2021-08-17 北京京东乾石科技有限公司 机器人的控制方法、装置和非易失性计算机可读存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6250911A (ja) * 1985-08-30 1987-03-05 Kajima Corp 自走式ロボツト
CN104375505A (zh) * 2014-10-08 2015-02-25 北京联合大学 一种基于激光测距的机器人自主寻路方法
CN105550667A (zh) * 2016-01-25 2016-05-04 同济大学 一种基于立体相机的骨架信息动作特征提取方法
CN105676845A (zh) * 2016-01-19 2016-06-15 中国人民解放军国防科学技术大学 一种智能安保服务机器人复杂环境避障方法及安保服务机器人
CN106020207A (zh) * 2016-07-26 2016-10-12 广东宝乐机器人股份有限公司 自移动机器人行走方法与装置
CN106291535A (zh) * 2016-07-21 2017-01-04 触景无限科技（北京）有限公司 一种障碍物检测装置、机器人及避障系统
CN106313046A (zh) * 2016-09-27 2017-01-11 成都普诺思博科技有限公司 移动机器人多级避障系统
CN207182092U (zh) * 2017-05-09 2018-04-03 叶仕通 一种用于移动机器人的驱动装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102499692B (zh) * 2011-11-30 2014-04-30 沈阳工业大学 一种超声波步态检测方法
KR101913332B1 (ko) * 2011-12-23 2018-10-31 삼성전자주식회사 이동 장치 및 이동 장치의 위치 인식 방법
JP6027703B1 (ja) * 2016-09-12 2016-11-16 シャープ株式会社 情報処理装置および制御プログラム
CN206224246U (zh) * 2016-10-19 2017-06-06 九阳股份有限公司 一种实现目标定位与跟踪的机器人
CN107423729B (zh) * 2017-09-20 2023-12-19 湖南师范大学 一种面向复杂视觉场景下的远距离类脑三维步态识别系统及实现方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS6250911A (ja) * 1985-08-30 1987-03-05 Kajima Corp 自走式ロボツト
CN104375505A (zh) * 2014-10-08 2015-02-25 北京联合大学 一种基于激光测距的机器人自主寻路方法
CN105676845A (zh) * 2016-01-19 2016-06-15 中国人民解放军国防科学技术大学 一种智能安保服务机器人复杂环境避障方法及安保服务机器人
CN105550667A (zh) * 2016-01-25 2016-05-04 同济大学 一种基于立体相机的骨架信息动作特征提取方法
CN106291535A (zh) * 2016-07-21 2017-01-04 触景无限科技（北京）有限公司 一种障碍物检测装置、机器人及避障系统
CN106020207A (zh) * 2016-07-26 2016-10-12 广东宝乐机器人股份有限公司 自移动机器人行走方法与装置
CN106313046A (zh) * 2016-09-27 2017-01-11 成都普诺思博科技有限公司 移动机器人多级避障系统
CN207182092U (zh) * 2017-05-09 2018-04-03 叶仕通 一种用于移动机器人的驱动装置

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112230652A (zh) * 2020-09-04 2021-01-15 安克创新科技股份有限公司 行走机器人及其移动控制的方法、计算机存储介质
CN112116529A (zh) * 2020-09-23 2020-12-22 浙江浩腾电子科技股份有限公司 一种基于ptz相机的gps坐标与像素坐标的转换方法
CN113506340A (zh) * 2021-06-15 2021-10-15 浙江大华技术股份有限公司 一种云台位姿预测的方法、设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN108628306A (zh) 2018-10-09
CN108628306B (zh) 2021-06-25

Similar Documents

Publication Publication Date Title
WO2019196313A1 (fr) Procédé et appareil de détection d'obstacle à la marche d'un robot, dispositif informatique, et support d'informations
Jörgensen et al. Monocular 3d object detection and box fitting trained end-to-end using intersection-over-union loss
Zou et al. Coslam: Collaborative visual slam in dynamic environments
WO2017133009A1 (fr) Procédé permettant de positionner une articulation humaine à l'aide d'une image de profondeur d'un réseau de neurones convolutionnel
JP7292492B2 (ja) オブジェクト追跡方法及び装置、記憶媒体並びにコンピュータプログラム
Marcu et al. A multi-stage multi-task neural network for aerial scene interpretation and geolocalization
Hu et al. Mobile panoramic vision for assisting the blind via indexing and localization
Gouidis et al. Accurate hand keypoint localization on mobile devices
CN116740539A (zh) 基于轻量化目标检测网络的视觉slam方法及系统
Lu et al. 2-D/3-D fusion-based robust pose normalisation of 3-D livestock from multiple RGB-D cameras
Nguyen et al. Combined YOLOv5 and HRNet for high accuracy 2D keypoint and human pose estimation
Lu et al. Automatic teat detection for rotary milking system based on deep learning algorithms
CN114663917A (zh) 基于多视角的多人三维人体位姿估计方法及装置
Wang et al. Im2fit: Fast 3d model fitting and anthropometrics using single consumer depth camera and synthetic data
Deng et al. Biological Eagle-eye Inspired Target Detection for Unmanned Aerial Vehicles Equipped with a Manipulator
Alhwarin Fast and robust image feature matching methods for computer vision applications
Le 3-D human pose estimation in traditional martial art videos
Velesaca et al. Deep Learning-based Human Height Estimation from a Stereo Vision System
Chen et al. A real-time photogrammetric system for acquisition and monitoring of three-dimensional human body kinematics
Sarsenov et al. Detection of Objects and Trajectories in Real-time using Deep Learning by a Controlled Robot.
Korovin et al. Human pose estimation applying ANN while RGB-D cameras video handling
Wang et al. Semantic Mapping Based on Visual SLAM with Object Model Replacement Visualization for Cleaning Robot
CN109934045A (zh) 行人检测方法和装置
Ding et al. Weakly structured information aggregation for upper-body posture assessment using ConvNets
US20240119620A1 (en) Posture estimation apparatus, posture estimation method, and computer-readable recording medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18914729

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18914729

Country of ref document: EP

Kind code of ref document: A1