WO2023224391A1

WO2023224391A1 - Method and system for creating virtual meta map using spatial data acquired by intelligent indoor autonomous mobile robot

Info

Publication number: WO2023224391A1
Application number: PCT/KR2023/006706
Authority: WO
Inventors: 이대윤
Original assignee: 주식회사 인티그리트
Priority date: 2022-05-18
Filing date: 2023-05-17
Publication date: 2023-11-23

Abstract

A system for creating a virtual meta map using spatial data acquired by an intelligent indoor autonomous mobile robot according to one embodiment of the present invention is characterized by comprising: an autonomous mobile robot or mobility which transmits location information about a target object located in an indoor space, the location information being acquired by a visualization sensor while the robot or mobility travels autonomously; and a cloud server which generates, in advance, a virtual map containing object information about the autonomous mobile robot in which the overall terrain of the indoor space and the object information are mapped, displays the autonomous mobile robot or mobility and the location information of the target object within the virtual space, generates a meta map reflecting driving commands for guiding the autonomous mobile robot or mobility to the target object within the virtual space, and then converts a virtual space coordinate system in the meta map to a real-life coordinate system and provides the real-life coordinate system to the autonomous mobile robot or mobility, wherein the autonomous mobile robot or mobility moves to the target object in the indoor space on the basis of information obtained as a result of converting the virtual space coordinate system in the meta map to the real-life coordinate system.

Description

Method and system for creating a virtualized meta map using spatial data acquired by an intelligent indoor autonomous robot

The present invention relates to a method and system for creating a virtualized meta map using spatial data acquired by an intelligent indoor autonomous robot.

3D spatial information technology, the basic technology of 3D GIS (Geographic Information System), refers to information technology that expresses it similarly to the real world by adding height (depth), image, and attributes to 2D location information.

The core of this 3D spatial information technology is to create a 3D electronic map and include all related information within it, which can be used for u-City, telematics, LBS (Location Based Service), urban planning and development, disaster prevention, and transportation. It can be said to lay the foundation for the public sector and related industries such as control and environment.

Additionally, these days, it is very common to use map services through car navigation systems or mobile devices when moving to a specific place. These map services were commonly two-dimensional (2D) maps that provided buildings, roads, etc. in a two-dimensional plan, but recently, three-dimensional (3D) map services have also emerged. As these 3D map services equally project the real world, their usability is increasing. In particular, web-based 3D map services that can be accessed from anywhere are currently already being provided in many countries and companies.

Accordingly, the present invention seeks to disclose a method and system for creating a virtualized meta map using spatial data acquired by an intelligent indoor self-driving robot.

[Prior art literature]

Registered Patent Publication No. 10-2242834

The purpose of the present invention is to provide a method and system for creating a virtualized meta map using spatial data acquired by an intelligent indoor self-driving robot that can solve conventional problems.

In order to solve the above problem, a virtualized meta map production system using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention provides location information of objects located in the indoor space acquired by a visualization sensor during the autonomous driving process. Autonomous robots or mobility that transmit; And generating in advance a virtualized map containing object information of an autonomous robot in which the overall terrain and object information of the indoor space is mapped, and displaying the autonomous robot or mobility and object location information in the virtual space; , After generating a meta map reflecting driving commands for the autonomous robot or mobility to move to the target within the virtual space, the virtual space coordinate system in the meta map is converted to a real coordinate system to enable the autonomous robot or It includes a cloud server that provides mobility, and the autonomous robot or mobility is characterized in that it moves to a target in the indoor space based on information that converts the virtual space coordinate system in the meta map into a real coordinate system.

In one embodiment, the cloud server sends the location information and characteristic information of people and/or objects measured during the autonomous driving process of the autonomous robot or mobility, and the real coordinate information of the indoor space in which the autonomous robot or mobility drives to the outside. Information collection unit that collects information from servers; A virtual space modeling unit that models the indoor space into a 3D virtual space using a digital twin technique based on the indoor coordinates of the indoor space; a driving information agent unit that provides driving commands for an autonomous robot or mobility avatar, which is an agent located in a virtual space, to move to a destination; It is characterized by comprising a meta-map generator that generates a meta-map including driving commands for the autonomous robot or mobility avatar to move to the target in the virtual space.

In one embodiment, the meta map generator includes a mapping unit that generates a mapping map that maps spatial coordinates of a virtual space, which is a digital twin space generated by the virtual space modeling unit, to spatial coordinates of the real world; a synchronization unit that synchronizes the location information of the self-driving robot or mobility in the real world within the mapping map and the location of the avatar of the self-driving robot or mobility in the virtual space; a target designator that specifies or creates a target within the mapping map at a location designated by an external terminal; a movement path generator that calculates and provides a movement route from the location of the autonomous robot or mobility within the mapping map to the target; and a meta map providing and updating unit that generates a meta map reflecting the movement path generated by the movement path creation unit and the driving command provided by the driving information agent within the movement path and transmits it to an autonomous robot or mobility in the real world. The meta map providing and updating unit reflects in real time the detection information of obstacles (moving objects) detected while moving along the movement path displayed on the meta map and recalculates the movement path to the target using the existing meta map. It is characterized by providing an updated map.

Using a virtualized meta map production system and method using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention, autonomous driving is possible by projecting the movement according to the manipulation command of a digital object in the virtual space in the real world. There is an advantage in being able to remotely control robots or mobility.

In addition, there is an advantage that events acquired from autonomous robots or mobility can be projected into virtual space and a meta map reflecting this can be created.

Figure 1a is the architecture of the digital twin, and Figure 1b is the architecture of the metaverse.

Figure 2 is a network configuration diagram of a virtualized meta map production system using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention.

FIG. 3 is a detailed configuration diagram of the autonomous robot or mobility shown in FIG. 2.

FIG. 4 is a detailed configuration diagram of the cloud server shown in FIG. 2.

Figure 5 is a detailed configuration diagram of the meta map generator shown in Figure 4.

FIG. 6 is an example diagram showing a metamap converted to real coordinates supported by the cloud server shown in FIG. 2 and a process of converting real coordinates into virtual space coordinates.

Figure 7 is a diagram illustrating an example computing environment in which one or more embodiments disclosed herein may be implemented.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention.

However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts that are not related to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected," but also the case where it is "electrically connected" with another element in between. . In addition, when a part is said to "include" a certain component, this does not mean excluding other components unless specifically stated to the contrary, but may further include other components, and one or more other features. It should be understood that it does not exclude in advance the possibility of the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

The terms "about", "substantially", etc. used throughout the specification are used to mean at or close to that value when manufacturing and material tolerances inherent in the stated meaning are presented, and are intended to enhance the understanding of the present invention. Precise or absolute figures are used to assist in preventing unscrupulous infringers from taking unfair advantage of stated disclosures. The term “step of” or “step of” as used throughout the specification of the present invention does not mean “step for.”

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware. Meanwhile, '~ part' is not limited to software or hardware, and '~ part' may be configured to reside in an addressable storage medium or may be configured to reproduce one or more processors.

Therefore, as an example, '~part' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. Contains fields, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or may be further separated into additional components and 'parts'. Additionally, components and 'parts' may be implemented to regenerate one or more CPUs within a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, apparatus, or device may instead be performed on a server connected to the terminal, apparatus, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed in a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is identifying data of the terminal. It can be interpreted as

Hereinafter, based on the attached drawings, a virtualized meta map production system and method using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention will be described in more detail.

First, before explaining the present invention, the differences between the digital twin and metaverse mentioned herein will be briefly explained.

Digital Twin and Metaverse are similar in their overall framework of being implemented through a virtual world, but they differ in how they view the real world. In the case of digital twin, it is a technology that implements objects from the real world, such as cities, machines, and factories, in the virtual world and operates in the same way as the real world based on this.

It is called a digital twin because it embodies reality in the virtual world like a twin.

As a simple example, in screen golf, the act of swinging the club and hitting the ball takes place in reality, but the act of making the ball fly and scoring points all take place in the virtual world.

In the early stages of digital twins, they simply imitated reality in the virtual world, but recently, with the development of Internet of Things, computing, and artificial intelligence technologies, various scenarios that cannot be simulated due to realistic constraints can be implemented virtually instead. .

Rather than simply predicting results, it has become possible to make predictions closer to reality based on various sensors and actual data and respond to problems in real time.

Predix, which is actually used by GE (General Electronics), is a cloud platform for integrating GE's facilities and software into the industrial Internet to support smart factories. Using this, we safely collect and analyze data from various machines and facilities in the real world through Predix. It is a digital twin platform that operates facilities more efficiently in reality based on the results.

Figure 1a shows the structure of a digital twin that creates a digital model of a real object based on model data and data collected by sensors in reality and implements data analysis, prediction, and situation awareness in reality again through visualization.

In the case of metaverse, it is a term that combines meta, which means transcendence, and universe, which means the real world. Metaverse allows you to experience various virtual worlds with an avatar that is a projection of yourself, but unlike digital twins, it does not mean that the virtual world is a realization of the real world. Metaverse is about producing and sharing various experiences and values through interaction between the virtual and real worlds by escaping from the initial 3D virtual world and crossing with reality.

Zepeto, a metaverse platform operated by Naver Jet, is a platform that implements avatars and virtual worlds based on facial recognition and augmented reality.

It is operated based on the virtual world map, BuildIt, which makes up the map, and ZEPETO Studio, which creates avatar costumes. Through avatars, users can communicate directly with other avatars in various virtual spaces. In addition, in the case of BTS, a new song was released like a concert within the game Fortnite.

Mesh, Microsoft's remote collaboration platform, is a metaverse platform based on mixed reality (MR) that allows users from different regions to feel as if they are working in the same space through avatars.

In this way, the metaverse is expanding to areas of daily life such as games, industry, meetings, and shopping. In the future, the fusion of more and more areas and activities of reality with virtuality will accelerate. Figure 1b shows four elements represented by the augmented reality of the metaverse, a mirror world implemented by adding additional information to the physical world, life logging that reproduces information such as emotions, movements, and bodies experienced by humans in a virtual world, and the virtual world. This diagram shows the concept of the metaverse that connects reality and virtuality with metaverse general-purpose technologies such as XR (Extended Reality) and data technology.

Figure 2 is a network configuration diagram of a virtualized meta map production system using spatial data acquired by an intelligent indoor autonomous robot according to an embodiment of the present invention, and Figure 3 is a detailed configuration of the autonomous robot or mobility shown in Figure 2. 4 is a detailed configuration diagram of the cloud server shown in FIG. 2, FIG. 5 is a detailed configuration diagram of the meta map generator shown in FIG. 4, and FIG. 6 is a reality supported by the cloud server shown in FIG. 2. This is an example diagram showing the metamap converted into coordinates and the process of converting real coordinates into virtual space coordinates.

First, as shown in FIG. 2, the virtualized meta map production system 100 using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention is an autonomous robot or mobility 200 and a cloud server. Includes 300.

Here, each component communicates through a network, and the network refers to a connection structure that allows information exchange between nodes such as a plurality of terminals and servers. An example of such a network includes the network 170, PAN ( One or more of the following networks: personal area network (LAN), local area network (LAN), campus area network (CAN), metropolitan area network (MAN), wide area network (WAN), broadband network (BBN), and the Internet. It can be included. Additionally, the network 170 may include any one or more of network topologies including a bus network, star network, ring network, mesh network, star-bus network, tree or hierarchical network, etc. It is not limited to this.

In the following, the term at least one is defined as a term including singular and plural, and even if the term at least one does not exist, each component may exist in singular or plural, and may mean singular or plural. This should be self-explanatory. Additionally, whether each component is provided in singular or plural form may vary depending on the embodiment.

The plurality of self- driving robots or mobility 200 may be configured to collect characteristic information of people and/or objects while driving in different designated indoor spaces.

In addition, the plurality of autonomous robots or mobility 200 detects and avoids obstacles on a designated movement path, recognizes environmental information such as temperature, humidity, or air condition, tracks, detects changes, moves at high speed, monitors images, or determines blind spots. You can perform various actions such as:

Depending on the embodiment, an autonomous robot or mobility may be called various terms such as a mobile robot, a mobile robot for surveillance, or a robot for surveillance.

The self-driving robot or mobility 200 includes a processor 210, a memory 220, a robot camera 230, a LiDAR sensor 240, an autonomous driving unit 250, and a communication unit 260.

The processor 210 executes commands to control driving operations of the autonomous robot or mobility 200 according to implementation commands (operation information) in the meta map provided by the cloud server 300, which will be described later.

Additionally, the processor 210 may be configured to extract or calculate characteristic information and spatial coordinate information of objects and/or people detected by the robot camera 230 and lidar scanner 240, which will be described later.

In addition, the processor 210 detects people and/or objects located in the indoor space through the shooting information of the robot camera and the LiDAR signal detected by the LiDAR sensor, and uses the external characteristics of the detected objects and people as objects. It may be a configuration extracted through a recognition model.

The object recognition model can be implemented using various algorithms, such as the Scale Invariant Feature Transform (SIFT) algorithm, Histogram of Oriented Grandients (HOG) algorithm, Haar algorithm, and Ferm algorithm.

In addition, the processor 210 may be configured to calculate the location and density of the objects and people in the indoor space based on the location at which the object and/or person is detected and the detection distance from the person and/or object. there is.

Here, the density is determined based on the number of LiDAR data distributed or located in a specific area, and the correlation between the LiDAR data is determined based on the Euclidean distance, reflectance, normal vector, etc. between the LiDAR data.

That is, the processor 210 can determine LiDAR data distributed in similar areas and have similar properties based on the degree to which the LiDAR data is distributed and the similarity between the LiDAR data, and cluster them into individual groups.

In addition, the processor 210 may be configured to process sensor data and the characteristic data required to extract characteristic data of the object and/or person into an encrypted message and then store it in a memory (buffer) to be described later. there is.

Additionally, it may be a configuration in which messages stored in memory (buffer) are processed and provided as one record in a worker thread.

Additionally, the memory 220 may be configured to store records processed by the processor 210.

Additionally, the memory 220 may be a space that stores various environmental information (sound, etc.) collected during autonomous driving.

Additionally, the memory 220 can store data and commands (application programs) for the operation of an autonomous robot or mobility.

At least some of these applications may be downloaded from an external server via wireless communication. The memory 220 is a flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory, etc.), Magnetic memory, magnetic disk, optical disk, RAM (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read-Only Memory: ROM), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read) -Only Memory) may include at least one storage medium.

The robot camera 230 may be configured to photograph the indoor space along the movement path. The captured video is used by the above-described processor to recognize or determine whether it is an object and/or a person and to extract characteristic information (face, body, gender, etc.) of the object and/or person.

The LIDAR sensor unit 240 may be configured to measure the position of an object and/or person. For reference, the LiDAR sensor 240 is capable of measuring the position of objects in space with a wide viewing angle at a distance, and is used to track the position of objects in a limited area of space even if the positions of some objects change.

For reference, the lidar sensor unit 240 emits light of a radar wavelength to the outside and measures the time it takes for this emitted light to reflect off an external object and return to provide information about the location of objects and/or people. Generate (LIDAR data).

Meanwhile, the LiDAR sensor unit 240 of the present invention uses LiDAR data to classify not only the presence or absence of objects (objects and people), but also the type of object, using a 2D LiDAR sensor consisting of a plurality of layers. It could be a dimensional LiDAR sensor.

However, when the 2D LiDAR sensor is tilted or rotated, LiDAR data containing 3D information can be secured, so the LiDAR sensor unit 240 used in the present invention is a 2D LiDAR configured to be tilted or rotated. Of course, it can be implemented with a sensor.

The autonomous driving unit 250 is configured to enable autonomous driving along a set route and is equipped with a lidar sensor that detects obstacles, a satellite navigation autonomous driving algorithm, etc., and the technology for autonomous driving is only a known technology. Therefore, a more detailed description will be omitted.

The communication unit 260 is configured to communicate bidirectionally with a digital twin map production server, which will be described later, and communicates with any internal component or at least one external terminal through a wired/wireless communication network.

At this time, the external arbitrary terminal may include a server (not shown), another terminal (not shown), etc. Here, wireless Internet technologies include Wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), and HSDPA (High Speed Downlink Packet Access). ), HSUPA (High Speed Uplink Packet Access), IEEE 802.16, Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced), Wireless Mobile Broadband Service (WMBS), etc. In this case, the communication unit transmits and receives data according to at least one wireless Internet technology, including Internet technologies not listed above. In addition, short-range communication technologies include Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, and Near Field Communication (NFC). , Ultrasound Communication (USC), Visible Light Communication (VLC), Wi-Fi, Wi-Fi Direct, etc. may be included. Additionally, wired communication technologies may include Power Line Communication (PLC), USB communication, Ethernet, serial communication, optical/coaxial cables, etc.

Additionally, the communication unit 260 can transmit information to and from any terminal through a universal serial bus (USB).

In addition, the communication unit 260 supports technical standards or communication methods for mobile communication (e.g., Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), EV -DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed Downlink Packet Access), HSUPA (High Speed Uplink Packet Access), LTE (Long Term Evolution), LTE-A (Long Term Evolution-Advanced, etc.), wireless signals are transmitted and received with a base station, the server, and the other terminals on a mobile communication network constructed according to (Long Term Evolution-Advanced), etc.).

Next, the cloud server 300 creates a virtual space that maps the coordinate information of the indoor space, displays the target coordinate information in the virtual space, and provides control information that records driving command information of the autonomous robot or mobility. It may be configured to generate a recorded meta map and provide it to the autonomous robot or mobility.

In addition, the cloud server 300 updates the meta map by reflecting the location information of obstacles or objects detected in real time by an autonomous robot or mobility running in the indoor space in the coordinates of the virtual space, and updates the meta map It may be configured to update the driving path and driving information to the target according to the location information of the object.

In addition, the cloud server 300 generates in advance a virtualized map containing the object information of the autonomous robot to which the overall terrain and object information of the indoor space is mapped, and the autonomous robot or mobility within the virtual space. After displaying target location information and generating a meta map reflecting driving commands for the autonomous robot or mobility to move to the target within the virtual space, the virtual space coordinate system within the meta map is converted to a real coordinate system. It may be converted and provided as the autonomous robot or mobility.

More specifically, the cloud server 300 includes an information collection unit 310, a virtual space modeling unit 320, a driving information agent unit 330, and a meta map creation unit 340.

The information collection unit 310 may be configured to collect measurement information measured by a pre-registered autonomous robot or mobility device 200. Here, the measurement information may be information including location information and characteristic information of people and/or objects measured during the autonomous driving process.

Additionally, the information collection unit 310 may be configured to receive real coordinate information of the indoor space in which the autonomous robot or mobility vehicle runs from an external server.

Next, the virtual space modeling unit 320 may be configured to model the indoor space as a three-dimensional virtual space based on the indoor coordinates of the indoor space.

Here, the three-dimensional virtual space, which is the space of the virtual world, may be configured in the form of VS (Virtual Space), MS (Mixed Space), and/or DTS (Digital Twin Space) depending on the characteristics of the space.

VS (Virtual Space) may be a purely digital space, DTS (Digital Twin Space) may be a virtual space based on the real world, and MS (Mixed Space) may be a mixture of VS and DTS.

In the case of MS, it can be provided in the form of self-driving robots or mobility adapting to the real world environment, or in the form of rendering the real world environment in the virtual world.

The space of the virtual world may be the concept of a basic virtual space in which autonomous robots or mobility can operate.

These virtual world spaces can be created and utilized in various ways depending on the user.

Meanwhile, a number of agents exist in the virtual world space, and an agent may refer to a user or program in the virtual world space.

Here, the agent as a program is in the form of an artificial intelligence agent and may be a virtual avatar or persona that exists on behalf of a third-party business operator. The physical characteristics of the space of the virtual world to which the agent belongs may be applied to these agents, and the service profile set in the space of the virtual world may be applied.

Additionally, the agent may have characteristics based on information about the physical device used by the user. For example, the agent may have a viewing angle according to the display characteristics of the physical device used by the user, or may have control characteristics according to the controller of the physical device.

In addition, various digital objects are located in the space of the virtual world, and as a key element of world information in the space of the virtual world, it can be a general term for objects that provide mutual interaction functions with agents.

At this time, the digital object may be formed in the object area (OA), which is an area included in the space of the virtual world.

Additionally, digital objects may include display objects (DO), interaction objects (IO), web objects (WO), and streaming objects (SO).

At this time, each of the display object, interaction object, web object, and streaming object may optionally include a display area for displaying content.

The virtual space created by the virtual space modeling unit 320 may be a space in which the real world environment, including roads, buildings, and landmarks existing in the real world, is equally projected into the virtual space.

The virtual space modeling unit 320 creates a virtual space by replicating the space using a digital twin technique based on three-dimensional information about the specific space.

For example, three-dimensional information about a specific space may be any one of a blueprint for the space, 3D scanned data, a floor plan, or data generated by actually measuring the specific space.

In addition, the virtual space modeling unit 130 calculates the difference with the object in the object detection data using a 3D model (3D model reconstruction) placed in the digital twin based on the initial position, and minimizes the difference. Through optimization, geographic data including the location and angle of an object can be estimated and the estimated geographic data can be implemented as a digital twin space.

The digital twin referred to in the present invention is used in various systems and industries such as manufacturing, medicine, transportation, aerospace, and construction, and uses standardized data to generate IoT data, 3D objects, and 3D maps. It may be formed based on (3DGeographic Map)

For reference, Digital Twin technology uses a digital twin model that maintains the identity and consistency of an actual physical model or an actual sensor information model that can be dynamically analyzed in real time, so that future phenomena such as an actual physical model or an actual sensor information model can be used. It is known as a technology that can predict and control the actual physical model or actual sensor information model.

In addition, the virtual space created by the virtual space modeling unit 320 is divided into a grid of a predetermined standard, and each grid is given address information and spatial information including unique coordinate values.

Next, the driving information agent unit 330 may be configured to provide driving commands for an autonomous robot or mobility avatar, which is an agent located in a virtual space, to move to the destination.

Here, the driving command may be a command for the movement path, movement speed, movement direction, etc. from the current location in virtual space to the target.

In addition, the driving information agent unit 330 detects people and/or objects located on the driving path in an indoor space in real time, and determines the movement path and movement speed to the target according to the location of the detected person and/or object. , It may be configured to provide a driving command that corrects or modifies the direction of movement.

Next, the meta map generator 340 creates a virtual world space in the form of VS (Virtual Space), MS (Mixed Space), and/or DTS (Digital Twin Space) generated by the virtual space modeling unit 320. It may be a configuration that generates a meta map containing driving commands for an autonomous robot or mobility in the real world to move to a target within the map.

In addition, the meta map generated by the meta map generator 340 is a meta map that converts 3D coordinates of virtual space into real coordinates so that autonomous robots or mobility in the real world can recognize it. It may be a map that includes spatial data acquired from mobility, object and/object characteristic information, location information, etc.

More specifically, the meta map generator 340 includes a mapping unit 341, a synchronization unit 342, a target designation unit 343, a movement path creation unit 344, and a meta map provision and update unit 345. Includes.

The mapping unit 341 may be configured to generate a mapping map that maps the spatial coordinates of the virtual space, which is a digital twin space created by the virtual space modeling unit 320, with the spatial coordinates of the real world.

The synchronization unit 342 may be configured to synchronize the location information of the self-driving robot or mobility in the real world within the mapping map and the location of the avatar of the self-driving robot or mobility in the virtual space.

Additionally, the target designator 343 may be configured to designate or create a target within the mapping map at a location designated by an external terminal.

The movement path creation unit 344 may be configured to calculate and provide a movement route from the location of the autonomous robot or mobility within the mapping map to the target.

For reference, the movement path generator 344 is an additional device that avoids the detected obstacle when an autonomous robot or mobility in the real world moving along the calculated movement path detects an obstacle (moving object, etc.) on the movement path. The movement route is recalculated and provided.

The meta map providing and updating unit 345 generates a meta map reflecting the movement route generated by the movement route creation unit 344 and the driving command provided by the driving information agent within the movement route, thereby providing the real world as described above. It may be transmitted to an autonomous robot or mobility.

In addition, the meta map providing and updating unit 345 reflects in real time the detection information of obstacles (moving objects) detected while moving along the movement path displayed on the meta map and recalculates the movement path to the target. It may be a configuration that updates and provides an existing meta map.

Figure 6 is a flowchart explaining a method of creating a virtualized meta map using spatial data acquired by an intelligent indoor self-driving robot according to an embodiment of the present invention.

Referring to FIG. 6, the method (S700) of creating a virtualized meta map using spatial data acquired by an intelligent indoor autonomous robot according to an embodiment of the present invention first uses indoor spatial information acquired by a visualization sensor from an autonomous robot or mobility. And after acquiring the location data of the target located in the indoor spatial information (S710), analyzing the characteristics of the target, calculating the location coordinates, and providing them to the cloud server (S720), the cloud server 300 A meta map of the virtual space mapped to the indoor space is created (S730), the location and characteristic information of the self-driving robot and the target are displayed in the generated meta map (S740), and the self-driving robot or mobility is displayed in the virtual map (space). When a driving command to drive to the destination is provided (S750), the coordinates in the meta map where the driving command is recorded are converted into a real coordinate system that can be recognized by the autonomous robot or mobility and transmitted (S760), and the autonomous robot or mobility At (200), a process of implementing the driving command in the meta map may be included.

In addition, the S700 process further includes the step of generating and updating a new driving command to bypass the person and/or object when object information about a person and/or object in the autonomous robot or mobility is located within the driving line. It can be included.

In addition, the user (administrator) can view the virtual map to provide and monitor commands to manipulate and move the robot's position, and the location data of the virtualized map can be converted into a coordinate system recognized by the actual robot and provided. It provides the advantage of having

7 is a diagram illustrating an example computing environment in which one or more embodiments disclosed herein may be implemented, and is an illustration of a system 1000 that includes a computing device 1100 configured to implement one or more embodiments described above. shows. For example, computing device 1100 may include a personal computer, server computer, handheld or laptop device, mobile device (mobile phone, PDA, media player, etc.), multiprocessor system, consumer electronics, minicomputer, mainframe computer, Distributed computing environments including any of the above-described systems or devices, etc. are included, but are not limited thereto.

Computing device 1100 may include at least one processing unit 1110 and memory 1120. Here, the processing unit 1110 may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. and can have multiple cores. Memory 1120 may be volatile memory (eg, RAM, etc.), non-volatile memory (eg, ROM, flash memory, etc.), or a combination thereof. Additionally, computing device 1100 may include additional storage 1130. Storage 1130 includes, but is not limited to, magnetic storage, optical storage, etc. The storage 1130 may store computer-readable instructions for implementing one or more embodiments disclosed in this specification, and other computer-readable instructions for implementing an operating system, application program, etc. may also be stored. Computer-readable instructions stored in storage 1130 may be loaded into memory 1120 for execution by processing unit 1110. Computing device 1100 may also include input device(s) 1140 and output device(s) 1150.

Here, the input device(s) 1140 may include, for example, a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, or any other input device, etc. Additionally, output device(s) 1150 may include, for example, one or more displays, speakers, printers, or any other output devices. Additionally, the computing device 1100 may use an input device or output device provided in another computing device as the input device(s) 1140 or the output device(s) 1150.

Additionally, computing device 1100 may include communication connection(s) 1160 that allows computing device 1100 to communicate with another device (e.g., computing device 1300).

Here, communication connection(s) 1160 may include a modem, network interface card (NIC), integrated network interface, radio frequency transmitter/receiver, infrared port, USB connection, or other device for connecting computing device 1100 to another computing device. May contain interfaces. Additionally, communication connection(s) 1160 may include a wired connection or a wireless connection. Each component of the computing device 1100 described above may be connected by various interconnections such as buses (e.g., peripheral component interconnect (PCI), USB, firmware (IEEE 1394), optical bus structure, etc.) and may be interconnected by a network 1200. As used herein, terms such as "component", "system", etc. generally refer to computer-related entities that are hardware, a combination of hardware and software, software, or software in execution.

For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both the application running on the controller and the controller can be components. One or more components may reside within a process and/or thread of execution, and a component may be localized on one computer or distributed between two or more computers.

What has been described above is only one embodiment for implementing a user-customized information provision system using an autonomous driving device according to the present invention, and the present invention is not limited to the above embodiment, but is as claimed in the following patent claims. Likewise, it will be said that the technical spirit of the present invention exists to the extent that anyone with ordinary knowledge in the field to which the present invention pertains can make various modifications without departing from the gist of the present invention.

*Explanation of symbols*

100: Virtualized meta map production system using spatial data acquired by an intelligent indoor autonomous robot

200: Self-driving robot or mobility

210: processor

220: memory

230: Robot camera

240: LiDAR sensor

250: Autonomous driving department

260: Department of Communications

270: Environmental information sensor

300: Cloud server

310: Information Collection Department

320: Virtual space modeling department

330: Driving information agent unit

340: Meta map generation unit

341: mapping unit

342: synchronization unit

343: Target designation unit

344: Movement path creation unit

345: Metamap provision and update department

Claims

Autonomous driving robot or mobility that transmits location information of objects located in indoor space acquired through visualization sensors during the autonomous driving process; and

Creating a virtualized map in advance containing object information of a self-driving robot with the overall topography and object information of the indoor space mapped, and displaying the self-driving robot or mobility and target location information in the virtual space, After generating a meta map reflecting driving commands for the autonomous robot or mobility to move to the destination within the virtual space, the virtual space coordinate system in the meta map is converted to a real coordinate system to enable the autonomous robot or mobility Includes a cloud server provided by

The autonomous robot or mobility

A virtualized meta map production system using spatial data acquired by an intelligent indoor self-driving robot that moves to a target in an indoor space based on information converted from the virtual space coordinate system in the meta map to a real coordinate system.
According to paragraph 1,

The cloud server is

An information collection unit that collects from an external server location information and characteristic information of people and/or objects measured during the autonomous driving process of the autonomous robot or mobility, and real coordinate information of the indoor space in which the autonomous robot or mobility drives;

A virtual space modeling unit that models the indoor space into a 3D virtual space using a digital twin technique based on the indoor coordinates of the indoor space;

a driving information agent unit that provides driving commands for an autonomous robot or mobility avatar, which is an agent located in a virtual space, to move to a destination;

Characterized by comprising a meta map generator that generates a meta map containing driving commands for the autonomous robot or mobility avatar to move to the target in the virtual space.

A virtualized meta-map production system using spatial data acquired by an intelligent indoor self-driving robot.
According to paragraph 2,

The meta map generator

A mapping unit that generates a mapping map that maps spatial coordinates of the virtual space, which is the digital twin space generated by the virtual space modeling unit, to spatial coordinates of the real world;

a synchronization unit that synchronizes the location information of the self-driving robot or mobility in the real world within the mapping map and the location of the avatar of the self-driving robot or mobility in the virtual space;

a target designator that specifies or creates a target within the mapping map at a location designated by an external terminal;

a movement path generator that calculates and provides a movement route from the location of the autonomous robot or mobility within the mapping map to the target; and

It includes a meta map providing and updating unit that generates a meta map reflecting the movement path generated by the movement path creation unit and the driving command provided by the driving information agent within the movement path and transmits it to an autonomous robot or mobility in the real world. do,

The meta map provision and update unit updates the existing meta map with a meta map that recalculates the movement path to the target by reflecting in real time the detection information of obstacles (moving objects) detected while moving along the movement path displayed on the meta map. A virtualized meta map production system using spatial data acquired by an intelligent indoor self-driving robot.