CN111242994A - Semantic map construction method and device, robot and storage medium - Google Patents
Semantic map construction method and device, robot and storage medium Download PDFInfo
- Publication number
- CN111242994A CN111242994A CN201911424096.3A CN201911424096A CN111242994A CN 111242994 A CN111242994 A CN 111242994A CN 201911424096 A CN201911424096 A CN 201911424096A CN 111242994 A CN111242994 A CN 111242994A
- Authority
- CN
- China
- Prior art keywords
- semantic
- information
- map
- spatial
- robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000010276 construction Methods 0.000 title claims abstract description 31
- 238000000034 method Methods 0.000 claims abstract description 35
- 238000002372 labelling Methods 0.000 claims abstract description 19
- 238000004590 computer program Methods 0.000 claims description 21
- 238000013507 mapping Methods 0.000 claims description 8
- 238000013528 artificial neural network Methods 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007726 management method Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241001417527 Pempheridae Species 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
Abstract
The application is applicable to the technical field of service robot map construction, and relates to a semantic map construction method, a semantic map construction device, a robot and a storage medium, wherein the semantic map construction method comprises the following steps: receiving traveling instruction information sent by a user, and controlling the robot to travel according to the traveling instruction information; acquiring an image of a target area to construct a space map; collecting, identifying and analyzing the spatial semantic information of the semantic object in the target area; acquiring time information when the spatial semantic information is acquired, and determining a key frame with the most adjacent time according to the time information; according to the key frame, marking the spatial semantic information to the spatial map; and detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map. The method and the device can solve the problems of low efficiency and poor accuracy of the traditional manual labeling navigation map.
Description
Technical Field
The application belongs to the technical field of service robot map construction, and particularly relates to a semantic map construction method and device, a robot and a storage medium.
Background
With the development of the robot technology and the continuous deepening of artificial intelligence research, the service robot gradually plays an indispensable role in human life, has more and more intelligent functions, and is widely applied to the fields of catering, cargo transportation and the like. In some practical application scenarios, the robot can reach a specified position on a map to provide a service based on the autonomous positioning navigation function of the robot, but the service of the robot cannot leave the navigation map. Therefore, the navigation map plays an important role in the global positioning and navigation process of the robot. In order to ensure that the robot navigation is smoothly carried out (arbitrarily appointing a starting point and an end point), a relatively complete map needs to be constructed. Meanwhile, map semantic information is indispensable to navigation planning and robot service management.
However, the existing robot navigation map construction needs more manual labeling and semantic addition, for example, service points such as guest room positions and guest welcome positions need to be labeled on the map, but hotels generally have many guest rooms, and if manual labeling is used, the labeling speed is low and the deviation of position information labeling occurs due to large workload.
Disclosure of Invention
The embodiment of the application provides a semantic map construction method and device, a robot and a storage medium, and can solve the problems of low efficiency and poor accuracy of the traditional manual labeling of a navigation map.
In a first aspect, an embodiment of the present application provides a semantic map construction method, which is applied to a robot, and the method includes:
receiving traveling instruction information sent by a user, and controlling the robot to travel according to the traveling instruction information;
collecting spatial information of a target area to construct a spatial map;
collecting, identifying and analyzing the spatial semantic information of the semantic object in the target area;
acquiring time information when the spatial semantic information is acquired, and determining a key frame with the most adjacent time according to the time information;
according to the key frame, marking the spatial semantic information to the spatial map;
and detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
In a second aspect, an embodiment of the present application provides a semantic map building apparatus, where the apparatus includes:
the traveling control module is used for receiving traveling instruction information sent by a user and controlling the robot to travel according to the traveling instruction information;
the map building module is used for collecting the spatial information of the target area to build a spatial map;
the semantic analysis module is used for acquiring, identifying and analyzing the spatial semantic information of the semantic object in the target area;
the determining module is used for acquiring time information when the spatial semantic information is acquired and determining a key frame with the most adjacent time according to the time information;
the labeling module is used for labeling the space semantic information to the space map according to the key frame;
and the map forming module is used for detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
In a third aspect, an embodiment of the present application provides a robot, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the method when executing the computer program.
In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the method.
In a fifth aspect, the present application provides a computer program product, which when run on a terminal device, causes the terminal device to execute the method of any one of the above first aspects.
It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.
Compared with the prior art, the embodiment of the application has the advantages that: the problem that traditional manual marking navigation map, inefficiency, accuracy are poor can be solved to this application to improve the on-the-spot engineering efficiency of deploying of service robot, improve engineering efficiency, alleviate engineering personnel work load, reduce the service operation cost.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic flow chart diagram illustrating a semantic map construction method according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of a mobile phone of a semantic map building apparatus according to an embodiment of the present application;
FIG. 3 is a schematic structural diagram of a robot provided in an embodiment of the present application;
fig. 4 is a schematic structural diagram of a robot and a surveying module used in cooperation according to an embodiment of the present disclosure.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.
Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.
The semantic map construction method provided by the embodiment of the application can be suitable for the service robot. In practical applications, the navigation map cannot be separated when autonomous navigation of the robot is performed. The navigation map provides a basis for path planning and walking obstacle avoidance of the service robot, and the walking problem of the service robot such as a sweeper is basically solved. However, for hotel service robots and hospital service robots, the map is required to have necessary semantic information for convenient service management. Therefore, quickly constructing a semantic map is very important for a service robot. However, the existing service robot navigation map construction needs more manual labeling, for example, service points such as guest room positions and guest welcome positions need to be labeled on the map, but hotels generally have many guest rooms, and if manual labeling is used, the labeling speed is low and the position information labeling deviation occurs due to large workload.
Therefore, the semantic map construction method provided by the application can solve the problems of low efficiency and poor accuracy of the traditional manual labeling of the navigation map.
The following describes an exemplary semantic map construction method provided by the present application with reference to specific embodiments.
Referring to fig. 1, a schematic flow chart of a semantic map construction method provided in an embodiment of the present application is shown. In the embodiment, the execution main body of the semantic map construction method is a robot, the robot comprises a work survey auxiliary module and a map construction module, the work survey auxiliary module can acquire a visual image and a depth information image by using an acquisition unit (such as a depth camera), and the map construction system acquires data by using a distance sensor (such as a laser radar) to construct a spatial map. The method comprises the following steps:
s101: and receiving traveling instruction information sent by a user, and controlling the robot to travel according to the traveling instruction information.
In this embodiment, when the robot scans and constructs a spatial map of a target area, the robot receives advance instruction information sent by a user from a control terminal in advance, so as to control the robot to acquire spatial information of the target area according to the advance instruction information. The travel instruction information may include a voice travel instruction and a wireless remote control travel instruction.
S102: and collecting the spatial information of the target area to construct a spatial map.
In this embodiment, the robot may accurately measure distance information to objects around the target area through a laser radar, and construct a spatial map using data collected by the laser radar. The spatial map may be a 2D planar map or a 3D stereoscopic map. The spatial information may be a distribution image of objects within the target area.
S103: and collecting, identifying and analyzing the spatial semantic information of the semantic object in the target area.
In this embodiment, the robot acquires the visual image and the depth information image in the target region through the acquisition unit of the work survey auxiliary module, and simultaneously identifies the spatial semantic information of the semantic object in the target region. The spatial semantic information comprises one or more of character information and spatial distance information, wherein the spatial distance information comprises a horizontal distance and a vertical distance.
S104: and acquiring time information when the spatial semantic information is acquired, and determining a key frame with the most adjacent time according to the time information.
In this embodiment, the mapping module of the robot receives the spatial semantic information sent by the work and survey assistance module, determines time information when the spatial semantic information is collected, determines a plurality of key frames before the time information, and performs a difference operation on the time information and the time information of each collected key frame to determine a nearest key frame, so as to further associate the image information collected by the mapping module with the spatial semantic information, thereby facilitating subsequent navigation using the map.
S105: and labeling the spatial semantic information to the spatial map according to the key frame.
In this embodiment, according to the key frame, the coordinate position of the key frame in the space map is determined, and then the space distance information of the identified space semantic information is utilized to label the space semantic information on the map.
S106: and detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
In this embodiment, the information for ending mapping may be instruction information for stopping mapping sent by the user through a terminal device, or trigger information when the robot returns to a departure point through a set planned path, or the robot confirms that the robot has traveled to an edge of the target area through an acquisition unit in the survey assistance module, and indicates that mapping of the target area is completed.
In an embodiment, in order to facilitate the collection of valid spatial semantic information, the work survey assistance module may set to start scene recognition, for example, when the collection unit recognizes a hotel front desk or a guest room door, it is determined that a target scene is recognized, and then recognition of semantic information in the target scene is started, and the valid semantic information in the target scene is analyzed and labeled to the spatial map, so as to avoid an error or unnecessary navigation instruction during navigation.
For example, another embodiment of the present application provides a semantic map construction method, which mainly relates to a process of performing scene recognition on the target area. The method comprises the following steps:
and carrying out target identification on the target area through a deep learning neural network so as to confirm whether a target scene exists in the target area.
And if the target scene exists, responding, identifying, collecting and analyzing the spatial semantic information of the semantic object in the target scene.
In this embodiment, the scene recognition may be implemented by training the deep learning neural network. The target scene can be a hotel foreground, a room door, a passenger elevator and the like. When scene recognition is carried out on the target area, if semantic extraction is appointed to be carried out on a certain semantic object, when the target area is recognized to have the target scene of the semantic object through the deep learning neural network, the semantic extraction is started, for example, only the semantic extraction is appointed to be carried out on a guest room doorplate, the guest room doorway scene recognition can be started, when the guest room doorway is not found, the semantic extraction is not carried out, the complexity of semantic mapping can be further simplified, and the engineering efficiency is improved.
In one embodiment, when a semantic object is identified, the semantic object is tracked along with the movement of the robot, and when the distance between the central point of the semantic object and the acquisition unit is the minimum, the horizontal distance and the vertical distance of the semantic object are acquired and marked in the space map, so that the robot can quickly confirm the semantic object during subsequent navigation, and further service is provided.
The embodiment of the application provides a semantic map construction method, which mainly relates to a process of calculating the closest distance between a semantic object and an acquisition unit of a robot. The method comprises the following steps:
when the spatial semantic information of the semantic object in the target area is collected, identified and analyzed, the method further comprises the following steps:
and when the semantic object is identified, sequentially detecting and acquiring N distances between the semantic object and the robot, wherein N is not less than 0 and is an integer.
In this embodiment, the distance between the robot and the semantic object is calculated by the acquisition unit on the robot, so as to judge and calculate the closest distance between the acquisition unit and the semantic object.
And determining whether the distance of the (N-1) th object is greater than the distance of the Nth object according to the N distances so as to determine the time point when the distance between the acquisition unit and the semantic object gradually decreases to just increase.
And if the distance of the (N-1) th object is smaller than the distance of the (N-1) th object, determining that the time point for acquiring the distance of the (N-1) th object is the closest moment of the semantic object and the robot, and calculating to obtain the horizontal distance and the vertical distance between the semantic object and the robot.
Optionally, after the horizontal distance and the vertical distance between the semantic object and the robot are obtained through calculation, the method includes: identifying and analyzing the semantic object to obtain character information; forming space semantic information by the character information, the horizontal distance and the vertical distance; the work and survey auxiliary module can send the space semantic information to a map building module and mark the space semantic information on the space map.
The space position relation between each acquisition unit of the work survey auxiliary module and the ground projection center point of the robot is measured, the space position relation is used for calculating three-dimensional space transformation from a local coordinate system of the acquisition unit to a robot coordinate system, and the transformation is used for transforming target coordinates (such as house numbers) measured and calculated by the acquisition unit to the robot coordinate system in the process of drawing construction.
In an embodiment, when the space map is a plane map, the text information and the horizontal distance information of the semantic object are marked to the plane map according to the key frame.
In an embodiment, when the space map is a three-dimensional map, the text information, the horizontal distance information and the vertical distance information of the semantic object are marked to the three-dimensional map according to the key frame.
In one embodiment, as shown in fig. 4, the work survey auxiliary module is used in on-site survey and map building, and the work survey auxiliary module can be fixed outside the robot when in use, and the work survey auxiliary module and the map building module of the robot have data communication connection, and together complete the construction of the spatial semantic map.
The work survey auxiliary module has the characteristic of being detachable, and can be detached and installed on other robots after the construction is completed.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Corresponding to the semantic map constructing method described in the foregoing embodiment, fig. 2 shows a structural block diagram of the semantic map constructing apparatus provided in the embodiment of the present application, and for convenience of explanation, only the relevant parts to the embodiment of the present application are shown.
Referring to fig. 2, the apparatus includes: the system comprises a traveling control module 100, a map building module 200, a semantic parsing module 300, a determining module 400, a labeling module 500 and a map forming module 600.
And the traveling control module is used for receiving traveling instruction information sent by a user and controlling the robot to travel according to the traveling instruction information.
The map building module is used for collecting the spatial information of the target area to build a spatial map.
And the semantic analysis module is used for acquiring, identifying and analyzing the spatial semantic information of the semantic object in the target area.
The determining module is used for acquiring time information when the spatial semantic information is acquired, and determining a key frame with the nearest time according to the time information.
And the labeling module is used for labeling the space semantic information to the space map according to the key frame.
The map forming module is used for detecting whether information for finishing map building is received or not, if so, forming a navigation map with semantics, and if not, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
Fig. 3 is a schematic structural diagram of a robot according to an embodiment of the present application. As shown in fig. 3, the robot 3 of this embodiment includes: at least one processor 30 (only one processor is shown in fig. 3), a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, the steps of any of the various method embodiments described above being implemented when the computer program 32 is executed by the processor 30.
The robot 3 may include, but is not limited to, a processor 30, a memory 31. Those skilled in the art will appreciate that fig. 3 is merely an example of the robot 3, and does not constitute a limitation of the robot 3, and may include more or less components than those shown, or combine some components, or different components, such as input and output devices, network access devices, etc.
The Processor 30 may be a Central Processing Unit (CPU), and the Processor 30 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 31 may in some embodiments be an internal storage unit of the robot 3, such as a hard disk or a memory of the robot 3. The memory 31 may also be an external storage device of the robot 3 in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the robot 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the robot 3. The memory 31 is used for storing an operating system, an application program, a BootLoader (BootLoader), data, and other programs, such as program codes of the computer program. The memory 31 may also be used to temporarily store data that has been output or is to be output.
It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the above-mentioned method embodiments.
The embodiments of the present application provide a computer program product, which when running on a robot, enables the robot to implement the steps in the above method embodiments when executed.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), random-access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.
Claims (10)
1. A semantic map construction method is applied to a robot, and is characterized by comprising the following steps:
receiving traveling instruction information sent by a user, and controlling the robot to travel according to the traveling instruction information;
collecting spatial information of a target area to construct a spatial map;
collecting, identifying and analyzing the spatial semantic information of the semantic object in the target area;
acquiring time information when the spatial semantic information is acquired, and determining a key frame with the most adjacent time according to the time information;
according to the key frame, marking the spatial semantic information to the spatial map;
and detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
2. The semantic map construction method of claim 1, wherein the collecting and identifying spatial semantic information that resolves semantic objects within the target region further comprises:
performing target identification on the target area through a deep learning neural network to confirm whether a target scene exists in the target area;
and if the target scene exists, responding, identifying, collecting and analyzing the spatial semantic information of the semantic object in the target scene.
3. The semantic map construction method according to claim 1, wherein the collecting and identifying for resolving the spatial semantic information of the semantic objects in the target area further comprises:
when the semantic object is identified, sequentially detecting and acquiring N distances between the semantic object and the robot, wherein N is more than or equal to 0 and is an integer;
determining whether the distance of the (N-1) th is greater than the distance of the Nth according to the N distances;
and if the distance of the (N-1) th object is smaller than the distance of the (N-1) th object, determining that the time point for acquiring the distance of the (N-1) th object is the closest moment of the semantic object and the robot, and calculating to obtain the horizontal distance and the vertical distance between the semantic object and the robot.
4. The semantic map construction method of claim 3, wherein the calculating a horizontal distance and a vertical distance between the semantic object and the robot comprises:
identifying and analyzing the semantic object to obtain character information;
forming space semantic information by the character information, the horizontal distance and the vertical distance;
and carrying out space transformation on the space semantic information, and marking the space semantic information on the space map.
5. The semantic mapping method according to claim 1, wherein the spatial semantic information comprises one or more of text information, spatial distance information, wherein the spatial distance information comprises horizontal distance and vertical distance.
6. The semantic map construction method according to claim 5, wherein when the space map is a plane map, the text information and horizontal distance information of the semantic objects are labeled to the plane map according to the key frames.
7. The semantic map construction method according to claim 5, wherein when the spatial map is a three-dimensional map, the text information, the horizontal distance information and the vertical distance information of the semantic object are labeled to the three-dimensional map according to the key frame.
8. A semantic mapping apparatus, the apparatus comprising:
the traveling control module is used for receiving traveling instruction information sent by a user and controlling the robot to travel according to the traveling instruction information;
the map building module is used for collecting the spatial information of the target area to build a spatial map;
the semantic analysis module is used for acquiring, identifying and analyzing the spatial semantic information of the semantic object in the target area;
the determining module is used for acquiring time information when the spatial semantic information is acquired and determining a key frame with the most adjacent time according to the time information;
the labeling module is used for labeling the space semantic information to the space map according to the key frame;
and the map forming module is used for detecting whether the information for finishing map building is received, if so, forming a navigation map with semantics, otherwise, continuously controlling the robot to move, and acquiring the image and the space semantic information of the target area to continuously construct the space map.
9. A robot comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424096.3A CN111242994B (en) | 2019-12-31 | 2019-12-31 | Semantic map construction method, semantic map construction device, robot and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911424096.3A CN111242994B (en) | 2019-12-31 | 2019-12-31 | Semantic map construction method, semantic map construction device, robot and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111242994A true CN111242994A (en) | 2020-06-05 |
| CN111242994B CN111242994B (en) | 2024-01-09 |
Family
ID=70879597
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911424096.3A Active CN111242994B (en) | 2019-12-31 | 2019-12-31 | Semantic map construction method, semantic map construction device, robot and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111242994B (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111679688A (en) * | 2020-06-18 | 2020-09-18 | 小狗电器互联网科技（北京）股份有限公司 | Charging method and device for self-walking robot, readable medium and electronic equipment |
| CN111743463A (en) * | 2020-06-18 | 2020-10-09 | 小狗电器互联网科技（北京）股份有限公司 | Cleaning method and device for target object, readable medium and electronic equipment |
| CN112632211A (en) * | 2020-12-30 | 2021-04-09 | 上海思岚科技有限公司 | Semantic information processing method and equipment for mobile robot |
| CN112883132A (en) * | 2021-01-15 | 2021-06-01 | 北京小米移动软件有限公司 | Semantic map generation method, semantic map generation device and electronic equipment |
| CN113552879A (en) * | 2021-06-30 | 2021-10-26 | 北京百度网讯科技有限公司 | Control method and device of self-moving equipment, electronic equipment and storage medium |
| CN113806455A (en) * | 2020-06-12 | 2021-12-17 | 纳恩博（北京）科技有限公司 | Map construction method, map construction equipment and storage medium |
| WO2022267795A1 (en) * | 2021-06-23 | 2022-12-29 | 追觅创新科技(苏州)有限公司 | Regional map processing method and apparatus, storage medium, and electronic device |
| CN115600156A (en) * | 2022-11-14 | 2023-01-13 | 苏州魔视智能科技有限公司（Cn） | Semantic map fusion method, device, equipment and medium based on minimum tree |
| CN112883132B (en) * | 2021-01-15 | 2024-04-30 | 北京小米移动软件有限公司 | Semantic map generation method, semantic map generation device and electronic equipment |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780735A (en) * | 2016-12-29 | 2017-05-31 | 深圳先进技术研究院 | A kind of semantic map constructing method, device and a kind of robot |
| CN109272554A (en) * | 2018-09-18 | 2019-01-25 | 北京云迹科技有限公司 | A kind of method and system of the coordinate system positioning for identifying target and semantic map structuring |
| CN110243370A (en) * | 2019-05-16 | 2019-09-17 | 西安理工大学 | A kind of three-dimensional semantic map constructing method of the indoor environment based on deep learning |
| CN110298873A (en) * | 2019-07-05 | 2019-10-01 | 青岛中科智保科技有限公司 | Construction method, construction device, robot and the readable storage medium storing program for executing of three-dimensional map |
-
2019
- 2019-12-31 CN CN201911424096.3A patent/CN111242994B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780735A (en) * | 2016-12-29 | 2017-05-31 | 深圳先进技术研究院 | A kind of semantic map constructing method, device and a kind of robot |
| CN109272554A (en) * | 2018-09-18 | 2019-01-25 | 北京云迹科技有限公司 | A kind of method and system of the coordinate system positioning for identifying target and semantic map structuring |
| CN110243370A (en) * | 2019-05-16 | 2019-09-17 | 西安理工大学 | A kind of three-dimensional semantic map constructing method of the indoor environment based on deep learning |
| CN110298873A (en) * | 2019-07-05 | 2019-10-01 | 青岛中科智保科技有限公司 | Construction method, construction device, robot and the readable storage medium storing program for executing of three-dimensional map |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113806455A (en) * | 2020-06-12 | 2021-12-17 | 纳恩博（北京）科技有限公司 | Map construction method, map construction equipment and storage medium |
| CN113806455B (en) * | 2020-06-12 | 2024-03-29 | 未岚大陆(北京)科技有限公司 | Map construction method, device and storage medium |
| CN111679688A (en) * | 2020-06-18 | 2020-09-18 | 小狗电器互联网科技（北京）股份有限公司 | Charging method and device for self-walking robot, readable medium and electronic equipment |
| CN111743463A (en) * | 2020-06-18 | 2020-10-09 | 小狗电器互联网科技（北京）股份有限公司 | Cleaning method and device for target object, readable medium and electronic equipment |
| CN112632211A (en) * | 2020-12-30 | 2021-04-09 | 上海思岚科技有限公司 | Semantic information processing method and equipment for mobile robot |
| CN112883132A (en) * | 2021-01-15 | 2021-06-01 | 北京小米移动软件有限公司 | Semantic map generation method, semantic map generation device and electronic equipment |
| CN112883132B (en) * | 2021-01-15 | 2024-04-30 | 北京小米移动软件有限公司 | Semantic map generation method, semantic map generation device and electronic equipment |
| WO2022267795A1 (en) * | 2021-06-23 | 2022-12-29 | 追觅创新科技(苏州)有限公司 | Regional map processing method and apparatus, storage medium, and electronic device |
| CN113552879A (en) * | 2021-06-30 | 2021-10-26 | 北京百度网讯科技有限公司 | Control method and device of self-moving equipment, electronic equipment and storage medium |
| CN115600156A (en) * | 2022-11-14 | 2023-01-13 | 苏州魔视智能科技有限公司（Cn） | Semantic map fusion method, device, equipment and medium based on minimum tree |
| CN115600156B (en) * | 2022-11-14 | 2023-03-28 | 苏州魔视智能科技有限公司 | Semantic map fusion method, device, equipment and medium based on minimum tree |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111242994B (en) | 2024-01-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111242994A (en) | Semantic map construction method and device, robot and storage medium | |
| US11204247B2 (en) | Method for updating a map and mobile robot | |
| Xu et al. | An occupancy grid mapping enhanced visual SLAM for real-time locating applications in indoor GPS-denied environments | |
| EP3505869B1 (en) | Method, apparatus, and computer readable storage medium for updating electronic map | |
| US11360216B2 (en) | Method and system for positioning of autonomously operating entities | |
| CN106780735B (en) | Semantic map construction method and device and robot | |
| US11340610B2 (en) | Autonomous target following method and device | |
| US20200088858A1 (en) | Multi-sensor calibration method, multi-sensor calibration device, computer device, medium and vehicle | |
| US6690451B1 (en) | Locating object using stereo vision | |
| US20210172741A1 (en) | Accompanying service method and device for intelligent robot | |
| CN112161618B (en) | Storage robot positioning and map construction method, robot and storage medium | |
| CN102087530A (en) | Vision navigation method of mobile robot based on hand-drawing map and path | |
| EP4050449A1 (en) | Method and device for robot positioning, smart robot, and storage medium | |
| Prieto et al. | As-is building-structure reconstruction from a probabilistic next best scan approach | |
| CN113116224B (en) | Robot and control method thereof | |
| KR20210029586A (en) | Method of slam based on salient object in image and robot and cloud server implementing thereof | |
| CN107063242A (en) | Have the positioning navigation device and robot of virtual wall function | |
| Zhang et al. | Seeing Eye Phone: a smart phone-based indoor localization and guidance system for the visually impaired | |
| US11785430B2 (en) | System and method for real-time indoor navigation | |
| Feng et al. | Visual map construction using RGB-D sensors for image-based localization in indoor environments | |
| Shoukat et al. | Cognitive robotics: Deep learning approaches for trajectory and motion control in complex environment | |
| WO2022193193A1 (en) | Data processing method and device | |
| CN110779533B (en) | Navigation method, system and electronic equipment | |
| CN109389677A (en) | Real-time construction method, system, device and the storage medium of house three-dimensional live map | |
| CN112987707A (en) | Automatic driving control method and device for vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |