CN112008727A - Elevator-taking robot key control method based on bionic vision and elevator-taking robot - Google Patents
Elevator-taking robot key control method based on bionic vision and elevator-taking robot Download PDFInfo
- Publication number
- CN112008727A CN112008727A CN202010901742.7A CN202010901742A CN112008727A CN 112008727 A CN112008727 A CN 112008727A CN 202010901742 A CN202010901742 A CN 202010901742A CN 112008727 A CN112008727 A CN 112008727A
- Authority
- CN
- China
- Prior art keywords
- target key
- key
- target
- elevator
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
- B25J9/1697—Vision controlled systems
Abstract
The application discloses a method and a device for controlling keys of a ladder-taking robot based on bionic vision and the ladder-taking robot, wherein the method comprises the steps of receiving click instructions of target keys, and determining plane position information of the target keys by combining mark information of the target keys and based on the bionic vision identification; acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera; and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key. The method and the device for controlling the elevator are used for solving the problem that the efficiency of a robot for executing tasks is influenced due to the fact that the existing elevator control button scheme is complex in flow.
Description
Technical Field
The application relates to the technical field of mobile robot application, in particular to a method and a device for controlling keys of a ladder-taking robot based on bionic vision and the ladder-taking robot.
Background
With the deep development of digitization and intelligent technologies in various fields, the service robot market keeps a rapid demand growth mode, and is applied to wide application fields such as transportation, repair, security, rescue, monitoring, interaction and the like. There may be a need for a robot to ride on a ladder independently in different application fields. For example, a transport robot may involve riding an elevator alone in the process of delivering food and drink to staff in an office building. For another example, a rescue robot may involve riding an elevator alone when performing a rescue task in a multi-story building. The robot can control the elevator button to realize the function of reaching the target floor when the robot takes the elevator alone.
The inventor finds that the existing scheme that the robot can control the elevator button mostly needs to modify the elevator and install an elevator module, and the robot controls the elevator through wireless communication with the elevator module. If the elevator cannot be refitted, the method cannot be realized, and is very inconvenient. Aiming at the problems of the existing elevator control button scheme, some solutions also appear, and mainly the robot is added with a mechanical arm and a camera device, the realization principle is that the robot performs blind scanning after entering an elevator, performs S-shaped scanning on a control panel after scanning an elevator control panel, performs multiple times of identification and detection after scanning a target button, and controls the mechanical arm to execute key operation after confirming that the target button is in the visual field center of the camera. The inventor finds that although the mode does not need to modify the elevator, the process is complex, and the efficiency of the robot for executing tasks is affected.
Disclosure of Invention
The application mainly aims to provide a key control method and device for a robot taking a ladder based on bionic vision and the robot taking the ladder, and solves the problems that the existing scheme for controlling the elevator button is complex in flow and affects the efficiency of the robot in executing tasks.
In order to achieve the above object, according to a first aspect of the present application, a method for controlling keys of a ladder-riding robot based on bionic vision is provided. The method comprises the following steps:
after receiving a click indication of a target key, determining plane position information of the target key based on bionic visual recognition by combining marking information of the target key;
acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera;
and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
Optionally, determining the plane position information of the target key based on the bionic visual recognition by combining the identification information of the target key includes:
and determining the plane position information of the target key according to the RGB camera, the target detection model and the preset mark recognition model.
Optionally, determining the plane position information of the target key according to the RGB camera, the target detection model, and the preset mark recognition model includes:
the method comprises the steps that RGB images containing an elevator control panel are obtained through an RGB camera;
positioning a boundary frame of all keys in a robot visual line range from an RGB image containing an elevator control panel according to a target detection model, and determining central points of all keys according to the boundary frame of all keys;
identifying corresponding mark information in the boundary frames of all keys according to a preset mark identification model;
and taking the central point corresponding to the key with the marking information as the marking information of the target key as the plane position information of the target key.
Optionally, after receiving the click indication of the target key, before determining the plane position information of the target key based on the biomimetic visual recognition in combination with the indication information of the target key, the method further includes:
detecting whether the target key is in the current sight range of the RGB camera;
if the target key is not in the current sight range of the RGB camera, judging the position relation between the key in the current sight range and the target key;
controlling the mechanical arm to adjust the position according to the position relation;
and if the target key is in the current sight range of the RGB camera, performing identification information combined with the target key and determining the plane position information of the target key based on bionic visual recognition.
Optionally, the method further includes:
before each adjustment, judging whether the adjusted position is in a safety range corresponding to the mechanical arm;
if the current time is not within the safe range, stopping adjustment and returning to click failure.
Optionally, the RGB camera and the depth camera are an integrated camera; the integrated camera is positioned at the tail end of the mechanical arm.
In order to achieve the above object, according to a second aspect of the present application, a key control device for a robot for taking a ladder based on bionic vision is provided.
The determining unit is used for determining the plane position information of the target key based on bionic visual recognition by combining the marking information of the target key after receiving the click indication of the target key;
the acquisition unit is used for acquiring the space coordinate corresponding to the target key according to the plane position information of the target key and by combining the depth camera;
and the control unit is used for controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
Optionally, the determining unit is further configured to:
and determining the plane position information of the target key according to the RGB camera, the target detection model and the preset mark recognition model.
Optionally, the determining unit includes:
the acquisition module is used for acquiring an RGB image containing an elevator control panel through an RGB camera;
the positioning module is used for positioning the boundary frames of all keys in the visual line range of the robot from the RGB image containing the elevator control panel according to the target detection model and determining the central point peripheries of all keys according to the boundary frames of all keys;
the identification module is used for identifying corresponding mark information in the boundary frames of all the keys according to a preset mark identification model;
and the determining module is used for taking the central point corresponding to the key with the mark information as the mark information of the target key as the plane position information of the target key.
Optionally, the apparatus further comprises:
the detection unit is used for detecting whether the target key is in the current sight range of the RGB camera or not after receiving the click indication of the target key and before determining the plane position information of the target key based on bionic visual recognition by combining the marking information of the target key;
the judging unit is used for judging the position relation between the keys in the current sight range and the target keys if the target keys are not in the current sight range of the RGB camera;
the adjusting unit is used for controlling the mechanical arm to adjust the position according to the position relation;
and the execution unit is used for executing and combining the marking information of the target key and determining the plane position information of the target key based on the bionic vision recognition if the target key is in the current sight range of the RGB camera.
Optionally, the adjusting unit further includes:
the judging module is used for judging whether the adjusted position is within the corresponding safety range of the mechanical arm before each adjustment;
and the returning module is used for stopping adjustment and returning click failure if the current time is not within the safety range.
Optionally, the RGB camera and the depth camera are an integrated camera; the integrated camera is positioned at the tail end of the mechanical arm.
In order to achieve the above object, according to a third aspect of the present application, there is provided a ladder robot based on bionic vision, comprising an RGB camera, a depth camera, a mechanical arm, and a processor:
the RGB camera is used for collecting RGB images containing an elevator control panel;
the depth camera is used for acquiring distance information between the target key and the depth camera;
a robotic arm for executing processor-generated move and click instructions;
the processor is used for executing the bionic vision-based elevator-taking robot key control method in any one of the first aspects.
In order to achieve the above object, according to a fourth aspect of the present application, there is provided a computer-readable storage medium storing computer instructions for causing a computer to execute the biomimetic vision based elevator-taking robot key control method according to any one of the first aspect
In the embodiment of the application, the elevator taking robot key control method and device based on the bionic vision and the elevator taking robot, after receiving the click indication of the target key, the plane position information of the target key is determined by combining the marking information of the target key and based on the bionic vision identification; acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera; and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key. It can be seen that, when the elevator-taking robot key control is realized, after a key click instruction is received, the plane position information and the mark information of all keys are determined according to the bionic visual recognition, then the space coordinate of the target key is determined by combining the depth camera, and finally the mechanical movement is controlled according to the space coordinate and the target key is clicked. Compared with the existing key control method, S-shaped scanning is not needed, positioning is faster, and in addition, after the target key is scanned, a plurality of identification and detection processes are not needed, so that the process is simplified. In conclusion, the key control mode of the application has higher efficiency.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, serve to provide a further understanding of the application and to enable other features, objects, and advantages of the application to be more apparent. The drawings and their description illustrate the embodiments of the invention and do not limit it. In the drawings:
fig. 1 is a flowchart of a bionic vision-based elevator-taking robot key control method according to an embodiment of the application;
fig. 2 is a block diagram of a bionic vision-based elevator-taking robot key control device according to an embodiment of the present application;
fig. 3 is a block diagram of another bionic vision-based elevator-taking robot key control device according to an embodiment of the application.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
According to an embodiment of the application, a method for controlling keys of a robot for taking a ladder based on bionic vision is also provided, as shown in fig. 1, the method includes:
s101, after receiving a click indication of a target key, determining plane position information of the target key based on bionic visual recognition by combining marking information of the target key.
The click instruction of the target key may be sent by the robot management platform, or the target key information may be extracted from information corresponding to a task after the robot receives the task sent by the robot management platform. Such as tasks involving the delivery of goods or meals to a certain floor. In addition, the click instruction for the elevator switch button can be generated by the robot itself according to the current environment or position. For example, when the elevator needs to be taken, a click instruction for clicking a button corresponding to going upstairs or downstairs is generated when the elevator is located outside the elevator.
And after receiving the click instruction, determining the plane position information of the target key based on the bionic vision recognition by combining the marking information of the target key. The method specifically comprises the following steps based on bionic visual recognition:
1) the method comprises the steps that RGB images containing an elevator control panel are obtained through an RGB camera;
compared with a common camera, the RGB camera used in the embodiment can improve the quality of images and provide better data support for subsequent positioning and identification.
2) Positioning the boundary frames of all keys from the RGB image containing the elevator control panel according to the target detection model, and determining the central points of all keys according to the boundary frames of all keys
In practical application, the target detection model may be any one of the existing target detection models, for example, a mobilenet-ssd network, an rcnn series, a yolo series, and an ssd, and a front-end network, for example, a vgg, AlexNet, and squeezet target detection algorithm model. After the RGB image is acquired by the RGB camera, the RGB image is transmitted to a processor inside the robot to process the image, preferably, in this embodiment, the RGB image is subjected to target detection through a mobilenet-ssd network, and a bounding box of all keys on the elevator control panel is located.
And after determining the boundary frame of the keys, determining the central point of the boundary frame, and taking the central point as the plane position information of each key. The central point is used as the plane position information of each key, so that a better fault-tolerant rate can be realized for clicking, namely, the clicking success can be ensured after clicking, and the condition that the feedback is not successful after clicking due to the fact that the clicking position is not accurate is not easy to occur (after clicking, the keys are not provided with lamps and are not bright).
3) And identifying the corresponding mark information in the boundary frames of all the keys according to a preset mark identification model.
After the boundary frames of all the keys on the elevator control panel are located, the content shown in all the keys, namely the mark information corresponding to the keys, is identified through a preset mark identification model. The mark information includes the number of floors, up and down marks, switch marks, etc.
4) And taking the central point corresponding to the key with the marking information as the marking information of the target key as the plane position information of the target key.
After the marking information of all keys is obtained, the plane position information of the target key can be determined according to the marking information of the target key. The plane position information of the target key is the pixel position of the target key in the RGB image.
And S102, acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera.
The time plane position information determined by the bionic vision is not a three-dimensional space position, so that the depth information of the target key relative to the RGB camera is required to be determined. Specifically, the present embodiment is determined by a depth camera, and the depth camera and the RGB camera are integrated into an integrated camera, that is, the two positions are the same. The distance between the target key and the depth camera, namely the distance from the central point of the target key to the depth camera, namely the depth of the target key relative to the RGB camera can be obtained through the depth camera. After the depth information is determined, the spatial coordinates of the target key under the coordinate system of the integrated camera can be obtained by determining the plane position before combination.
And S103, controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
Specifically, after obtaining the spatial coordinates of the target key in the integrated camera coordinate system, the coordinates of the target key in the coordinate system using the mechanical arm as the mechanical arm are obtained through calculation (coordinate conversion) according to the spatial coordinates, then the mechanical arm is controlled to move according to the coordinates of the target key in the coordinate system using the mechanical arm as the mechanical arm, and after the click end of the mechanical arm for clicking moves to the spatial coordinate position, a click command is generated to control the click end to complete the click action of the target key.
From the above description, it can be seen that, in the elevator-taking robot key control method based on bionic vision according to the embodiment of the present application, after receiving a click indication of a target key, the plane position information of the target key is determined based on the bionic vision identification in combination with the identification information of the target key; acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera; and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key. It can be seen that, when the elevator-taking robot key control is realized, after a key click instruction is received, the plane position information and the mark information of all keys are determined according to the bionic visual recognition, then the space coordinate of the target key is determined by combining the depth camera, and finally the mechanical movement is controlled according to the space coordinate and the target key is clicked. Compared with the existing key control method, S-shaped scanning is not needed, positioning is faster, and in addition, after the target key is scanned, a plurality of identification and detection processes are not needed, so that the process is simplified. In conclusion, the key control mode of the application has higher efficiency.
Further, if the target key is not within the current sight range of the RGB camera, after receiving the click indication of the target key, before determining the plane position information of the target key based on the biomimetic vision recognition in combination with the identification information of the target key, the method for controlling the elevator-taking robot key based on the biomimetic vision in the embodiment further includes: detecting whether the target key is in the current sight range of the RGB camera; if the target key is not in the current sight range of the RGB camera, judging the position relation between the key in the current sight range and the target key; controlling the mechanical arm to adjust the position according to the position relation; and if the target key is in the current sight range of the RGB camera, performing identification information combined with the target key and determining the plane position information of the target key based on bionic visual recognition.
When the target key is not in the current sight range of the RGB camera, the positional relationship between the key and the target key in the current sight range, such as above, below, left, and right of the target key, needs to be determined in time. The position relationship can be determined according to the actual position arrangement among the buttons, and then the corresponding adjustment of the mechanical arm is carried out according to the position relationship, so that the target key is in the current sight range. Moving downwards a preset distance above the target key; and if the key is positioned below the target key, the key is moved upwards by a preset distance. The preset distance can be freely adjusted according to actual conditions, for example, the preset distance can be adjusted and set according to the spacing distance between the keys and the like.
In practical application, the mechanical arm is provided with a movable safety range, so that before adjustment (before movement) each time, whether the adjusted position is in the safety range corresponding to the mechanical arm is judged; if the current time is not within the safe range, stopping adjustment and returning to click failure. If the mobile terminal is in a safe range, the mobile terminal can move.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
According to an embodiment of the present application, there is provided a ladder-taking robot key control device based on bionic vision corresponding to the ladder-taking robot key control method based on bionic vision in fig. 1, as shown in fig. 2, including:
the determining unit 201 is configured to determine, after receiving a click instruction of a target key, planar position information of the target key based on bionic visual recognition by combining the identification information of the target key;
the acquiring unit 202 is configured to acquire a spatial coordinate corresponding to the target key according to the plane position information of the target key and by combining the depth camera;
and the control unit 203 is used for controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
Specifically, the implementation of each module in this embodiment may refer to the related implementation in the method embodiment, and is not described again.
From the above description, it can be seen that, in the elevator-taking robot key control device based on the bionic vision in the embodiment, after receiving the click indication of the target key, the planar position information of the target key is determined based on the bionic vision identification in combination with the identification information of the target key; acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera; and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key. It can be seen that, when the elevator-taking robot key control is realized, after a key click instruction is received, the plane position information and the mark information of all keys are determined according to the bionic visual recognition, then the space coordinate of the target key is determined by combining the depth camera, and finally the mechanical movement is controlled according to the space coordinate and the target key is clicked. Compared with the existing key control method, S-shaped scanning is not needed, positioning is faster, and in addition, after the target key is scanned, a plurality of identification and detection processes are not needed, so that the process is simplified. In conclusion, the key control mode of the application has higher efficiency.
Further, the determining unit 201 is further configured to:
and determining the plane position information of the target key according to the RGB camera, the target detection model and the preset mark recognition model.
Further, as shown in fig. 3, the determining unit 201 includes:
the acquiring module 2011 is configured to acquire an RGB image including the elevator control panel through the RGB camera;
the positioning module 2012 is used for positioning a boundary frame of all keys in the robot view line range from the RGB image including the elevator control panel according to the target detection model, and determining center points of all keys according to the boundary frame of all keys;
the identification module 2013 is used for identifying corresponding mark information in the boundary frames of all the keys according to the preset mark identification model;
a determining module 2014, configured to use a central point corresponding to the key with the identification information as the identification information of the target key as the plane position information of the target key.
Further, as shown in fig. 3, the apparatus further includes:
the detection unit 204 is configured to detect whether the target key is within a current sight range of the RGB camera after receiving a click instruction of the target key and before determining planar position information of the target key based on biomimetic visual recognition in combination with the identification information of the target key;
the judging unit 205 is configured to, if the target key is not within the current sight range of the RGB camera, judge a position relationship between the key and the target key within the current sight range;
an adjusting unit 206, configured to control the mechanical arm to adjust the position according to the position relationship;
and the execution unit 207 is configured to execute, if the target key is within the current sight range of the RGB camera, combining the marking information of the target key and determining the plane position information of the target key based on the bionic visual recognition.
Further, as shown in fig. 3, the adjusting unit 206 further includes:
a determining module 2061, configured to determine, before each adjustment, whether the adjusted position is within a safety range corresponding to the manipulator;
and a returning module 2062, configured to stop the adjustment if the current click is not within the safe range, and return a click failure.
Furthermore, the RGB camera and the depth camera are integrated cameras; the integrated camera is positioned at the tail end of the mechanical arm.
Specifically, the implementation of each module in this embodiment may refer to the related implementation in the method embodiment, and is not described again.
According to the embodiment of the application, still provide a robot of taking advantage of ladder based on bionical vision, its characterized in that includes RGB camera, degree of depth camera, arm, treater:
the RGB camera is used for collecting RGB images containing the elevator control panel;
the depth camera is used for acquiring distance information between the target key and the depth camera;
a robotic arm for executing processor-generated move and click instructions;
and the processor is used for executing the bionic vision-based elevator-taking robot key control method in the figure 1.
In the embodiment of the application, when the elevator taking robot realizes the key control of the elevator taking robot, after a key click instruction is received, the plane position information and the mark information of all keys are determined according to the bionic vision identification, then the space coordinate of a target key is determined by combining a depth camera, and finally the mechanical movement is controlled according to the space coordinate and the target key is clicked. Compared with the existing key control method, S-shaped scanning is not needed, positioning is faster, and in addition, after the target key is scanned, a plurality of identification and detection processes are not needed, so that the process is simplified. In conclusion, the key control mode of the application has higher efficiency.
According to an embodiment of the present application, there is also provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the biomimetic vision-based elevator-riding robot key control method described in fig. 1.
It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present application is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. A key control method of a ladder-riding robot based on bionic vision is characterized by comprising the following steps:
after receiving a click indication of a target key, determining plane position information of the target key based on bionic visual recognition by combining marking information of the target key;
acquiring a space coordinate corresponding to the target key according to the plane position information of the target key and by combining a depth camera;
and controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
2. The biomimetic vision based elevator-taking robot key control method according to claim 1, wherein determining the planar position information of the target key based on biomimetic vision recognition in combination with the identification information of the target key comprises:
and determining the plane position information of the target key according to the RGB camera, the target detection model and the preset mark recognition model.
3. The method as claimed in claim 2, wherein determining the planar position information of the target key according to the RGB camera, the target detection model, and the preset mark recognition model comprises:
the method comprises the steps that RGB images containing an elevator control panel are obtained through an RGB camera;
positioning a boundary frame of all keys in a robot visual line range from an RGB image containing an elevator control panel according to a target detection model, and determining central points of all keys according to the boundary frame of all keys;
identifying corresponding mark information in the boundary frames of all keys according to a preset mark identification model;
and taking the central point corresponding to the key with the marking information as the marking information of the target key as the plane position information of the target key.
4. The biomimetic vision based elevator-taking robot key control method according to claim 1, wherein after receiving the click indication of the target key, before determining the planar position information of the target key based on the biomimetic vision recognition in combination with the designation information of the target key, the method further comprises:
detecting whether the target key is in the current sight range of the RGB camera;
if the target key is not in the current sight range of the RGB camera, judging the position relation between the key in the current sight range and the target key;
controlling the mechanical arm to adjust the position according to the position relation;
and if the target key is in the current sight range of the RGB camera, performing identification information combined with the target key and determining the plane position information of the target key based on bionic visual recognition.
5. The biomimetic vision based elevator-taking robot key control method according to claim 4, wherein the method further comprises:
before each adjustment, judging whether the adjusted position is in a safety range corresponding to the mechanical arm;
if the current time is not within the safe range, stopping adjustment and returning to click failure.
6. The stair taking robot key control method based on bionic vision as claimed in claim 5, wherein the RGB camera and the depth camera are an integrated camera; the integrated camera is positioned at the tail end of the mechanical arm.
7. A key control device of a robot for taking a ladder based on bionic vision is characterized by comprising:
the determining unit is used for determining the plane position information of the target key based on bionic visual recognition by combining the marking information of the target key after receiving the click indication of the target key;
the acquisition unit is used for acquiring the space coordinate corresponding to the target key according to the plane position information of the target key and by combining the depth camera;
and the control unit is used for controlling the mechanical arm to move according to the space coordinate corresponding to the target key so as to click the target key.
8. The biomimetic vision-based elevator-taking robot key control device according to claim 7, wherein the determining unit is further configured to:
and determining the plane position information of the target key according to the RGB camera, the target detection model and the preset mark recognition model.
9. The utility model provides a take advantage of terraced robot based on bionical vision which characterized in that, includes RGB camera, degree of depth camera, arm, treater:
the RGB camera is used for collecting RGB images containing an elevator control panel;
the depth camera is used for acquiring distance information between the target key and the depth camera;
a robotic arm for executing processor-generated move and click instructions;
a processor for executing the bionic vision-based elevator-taking robot key control method of any one of the preceding claims 1 to 6.
10. A computer-readable storage medium storing computer instructions for causing a computer to execute the biomimetic vision-based elevator robot key control method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010901742.7A CN112008727A (en) | 2020-08-31 | 2020-08-31 | Elevator-taking robot key control method based on bionic vision and elevator-taking robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010901742.7A CN112008727A (en) | 2020-08-31 | 2020-08-31 | Elevator-taking robot key control method based on bionic vision and elevator-taking robot |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112008727A true CN112008727A (en) | 2020-12-01 |
Family
ID=73515555
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010901742.7A Pending CN112008727A (en) | 2020-08-31 | 2020-08-31 | Elevator-taking robot key control method based on bionic vision and elevator-taking robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112008727A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112529953A (en) * | 2020-12-17 | 2021-03-19 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| CN114348811A (en) * | 2021-12-06 | 2022-04-15 | 深圳市普渡科技有限公司 | Robot, robot elevator taking method, device and storage medium |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1481772A2 (en) * | 2003-05-29 | 2004-12-01 | Fanuc Ltd | Robot system for controlling the movements of a robot utilizing a visual sensor |
| CN102701033A (en) * | 2012-05-08 | 2012-10-03 | 华南理工大学 | Elevator key and method based on image recognition technology |
| CN105867390A (en) * | 2016-06-16 | 2016-08-17 | 中南大学 | Method for controlling transport robot to autonomously enter elevator |
| CN106625697A (en) * | 2016-11-15 | 2017-05-10 | 同济大学 | Intelligent desktop type mechanical arm system based on machine vision technology |
| CN107639638A (en) * | 2017-09-27 | 2018-01-30 | 广东工业大学 | A kind of keyboard mechanical arm based on machine vision |
| CN108177150A (en) * | 2016-12-08 | 2018-06-19 | 广州映博智能科技有限公司 | Door of elevator positioning and grabbing device and the method for view-based access control model |
| CN109760058A (en) * | 2019-02-25 | 2019-05-17 | 文浩 | A kind of boarding robot and its boarding robot key control method |
| CN109760070A (en) * | 2019-03-15 | 2019-05-17 | 深圳市锐曼智能装备有限公司 | Robot elevator push button position control method and system |
| CN110696004A (en) * | 2019-09-18 | 2020-01-17 | 五邑大学 | Intelligent navigation robot and control method thereof |
| US20200086493A1 (en) * | 2018-09-18 | 2020-03-19 | Kinova Inc. | Vision guided robot arm and method for operating the same |
| CN111360851A (en) * | 2020-02-19 | 2020-07-03 | 哈尔滨工业大学 | Hybrid servo control device and method for robot integrating touch and vision |
| CN111531531A (en) * | 2020-03-27 | 2020-08-14 | 向仲宇 | Robot key control method and system |
| CN111598172A (en) * | 2020-05-18 | 2020-08-28 | 东北大学 | Dynamic target grabbing posture rapid detection method based on heterogeneous deep network fusion |
-
2020
- 2020-08-31 CN CN202010901742.7A patent/CN112008727A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1481772A2 (en) * | 2003-05-29 | 2004-12-01 | Fanuc Ltd | Robot system for controlling the movements of a robot utilizing a visual sensor |
| CN102701033A (en) * | 2012-05-08 | 2012-10-03 | 华南理工大学 | Elevator key and method based on image recognition technology |
| CN105867390A (en) * | 2016-06-16 | 2016-08-17 | 中南大学 | Method for controlling transport robot to autonomously enter elevator |
| CN106625697A (en) * | 2016-11-15 | 2017-05-10 | 同济大学 | Intelligent desktop type mechanical arm system based on machine vision technology |
| CN108177150A (en) * | 2016-12-08 | 2018-06-19 | 广州映博智能科技有限公司 | Door of elevator positioning and grabbing device and the method for view-based access control model |
| CN107639638A (en) * | 2017-09-27 | 2018-01-30 | 广东工业大学 | A kind of keyboard mechanical arm based on machine vision |
| US20200086493A1 (en) * | 2018-09-18 | 2020-03-19 | Kinova Inc. | Vision guided robot arm and method for operating the same |
| CN109760058A (en) * | 2019-02-25 | 2019-05-17 | 文浩 | A kind of boarding robot and its boarding robot key control method |
| CN109760070A (en) * | 2019-03-15 | 2019-05-17 | 深圳市锐曼智能装备有限公司 | Robot elevator push button position control method and system |
| CN110696004A (en) * | 2019-09-18 | 2020-01-17 | 五邑大学 | Intelligent navigation robot and control method thereof |
| CN111360851A (en) * | 2020-02-19 | 2020-07-03 | 哈尔滨工业大学 | Hybrid servo control device and method for robot integrating touch and vision |
| CN111531531A (en) * | 2020-03-27 | 2020-08-14 | 向仲宇 | Robot key control method and system |
| CN111598172A (en) * | 2020-05-18 | 2020-08-28 | 东北大学 | Dynamic target grabbing posture rapid detection method based on heterogeneous deep network fusion |
Non-Patent Citations (1)
| Title |
|---|
| 李瀚辉: "基于视觉引导的机器人乘坐电梯算法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112529953A (en) * | 2020-12-17 | 2021-03-19 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| CN112529953B (en) * | 2020-12-17 | 2022-05-03 | 深圳市普渡科技有限公司 | Elevator space state judgment method and device and storage medium |
| CN114348811A (en) * | 2021-12-06 | 2022-04-15 | 深圳市普渡科技有限公司 | Robot, robot elevator taking method, device and storage medium |
| CN114348811B (en) * | 2021-12-06 | 2024-04-09 | 深圳市普渡科技有限公司 | Robot, robot boarding method, robot boarding device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110861094B (en) | Robot control method, robot, and readable storage medium | |
| EP3686703B1 (en) | Control method, apparatus and system for robot, and applicable robot | |
| CN112537702B (en) | Robot elevator taking guiding method and device, terminal equipment and readable storage medium | |
| US10083418B2 (en) | Distributed autonomous robot systems and mehtods | |
| CN106347919A (en) | Automatic warehousing system | |
| CN112008727A (en) | Elevator-taking robot key control method based on bionic vision and elevator-taking robot | |
| CN107272710A (en) | A kind of medical merchandising machine people system and its control method of view-based access control model positioning | |
| CN109108973A (en) | One-way fashion path dispatching method and system | |
| CN106247943A (en) | Article 3-D positioning method, device and system | |
| CN109052077A (en) | Robot call method and device based on Internet of Things | |
| CN109656252A (en) | A kind of middle control degree system and positioning navigation method based on AGV | |
| CN106256512A (en) | Robot device including machine vision | |
| US20190084008A1 (en) | Instruction projecting device, package sorting system and instruction projecting method | |
| CN113420682A (en) | Target detection method and device in vehicle-road cooperation and road side equipment | |
| US20230280759A1 (en) | Autonomous Robotic Navigation In Storage Site | |
| CN114089735A (en) | Method and device for adjusting shelf pose of movable robot | |
| CN109571408A (en) | The angle calibration system method and storage medium of a kind of robot, stock container | |
| CN115609587A (en) | Target following method, device, equipment and storage medium | |
| CN110069057A (en) | A kind of obstacle sensing method based on robot | |
| CN113946728A (en) | Intelligent building integration management system | |
| CN115248039A (en) | Multi-robot-multi-person cooperation control method, device and system | |
| CN112123328B (en) | Man-machine cooperation control method and system | |
| CN110989607A (en) | Robot path planning and walking control method and system based on serial number | |
| US20220291696A1 (en) | Transport system, control apparatus, transport method, and program | |
| US10755100B2 (en) | Information processing device, system, information processing method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 702, 7th floor, NO.67, Beisihuan West Road, Haidian District, Beijing 100089 Applicant after: Beijing Yunji Technology Co.,Ltd. Address before: Room 702, 7th floor, NO.67, Beisihuan West Road, Haidian District, Beijing 100089 Applicant before: BEIJING YUNJI TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20201201 |