KR20210003065A - Method and system for collecting data - Google Patents

Abstract

According to various embodiments of the present invention, provided is a data collecting method, which comprises: a time stamping step of collecting, by a data collecting device including first and second sensors, first and second sensor data through the first and second sensors, respectively, while moving in a target region, and tagging first and second time stamping values to the first and second sensor data; a data generating step of generating temporary map data of the target region and temporary location data of a time point corresponding to the first time stamp value on the basis of the first sensor data collected while the data collecting device moves; a detecting step of detecting a loop closure re-visiting a location through which the data collecting device passes; a loop closing step of generating corrected location data by correcting the temporary location data in response to the detection of the loop closure, and generating moving path information and map information on the target region on the basis of the corrected location data; a tagging step of estimating a sensing location of a time point corresponding to the second time stamp value on the basis of the moving path information, and tagging the sensing location to the second sensor data; and a storage step of storing the second sensor data tagged with the sensing location. According to the present invention, costs and time for collecting sensor data can be reduced.

Description

Data collection method and system {Method and system for collecting data}

The present disclosure relates to a method and system for collecting data, and more particularly, to a method and system for collecting sensor data tagged with an indoor location.

In order to provide a location-based service to a user, it is necessary to obtain accurate location information of a user or a target object. Such location information is generally obtained based on a Global Positioning System (GPS) signal. However, there is a lot of difficulty in measuring the location of a user or a target object in a shaded area of a GPS signal such as between tall buildings or a space where the GPS signal is difficult to reach, such as inside a building.

As a method for providing location-based services even in spaces where GPS signals are difficult to reach, such as inside buildings, Korean Patent Laid-Open Publication No. 10-2012-0029976 (published on March 27, 2012) "Standard for indoor wireless positioning In the information collection method and apparatus, indoor wireless positioning service method using a plurality of access points installed indoors, scan information for an indoor access point is collected, and monitoring access is performed using preset location information of at least one monitoring access point. Map the scan information of the point to an indoor map, and generate location information on the indoor map of the virtual access point by using the scan information of at least one virtual access point obtained through the scan information, and the indoor Disclosed is a method of collecting reference point information for indoor wireless positioning by generating identification information for each cell that is preset on a map and includes the monitoring access point and the virtual access point. However, this method has a problem that indoor map information must exist in advance.

The problem to be solved by various embodiments of the present disclosure is that while moving an indoor space using a data collection device, while searching the surroundings, measuring one's own location and creating a map, and at the same time collecting sensor data tagged with the sensing location. To provide a way.

As a technical means for achieving the above-described technical problems, a data collection method according to the first aspect of the present disclosure is provided. In the data collection method, a data collection device including first and second sensors collects first and second sensor data respectively through the first and second sensors while moving within a target area, and the first and second sensors 2 a time stamping step of tagging the first and second timestamp values on the sensor data, respectively; A data generating step of generating temporary map data of the target area and temporary location data of a time point corresponding to the first timestamp value based on the first sensor data collected while the data collection device moves; A detecting step of detecting a loop closure that revisits the location where the data collection device has passed; A loop closing step of generating corrected position data by correcting the temporary position data in response to detection of the loop closure, and generating moving route information and map information of the target area based on the corrected position data; A tagging step of estimating a sensing position at a point in time corresponding to the second timestamp value based on the movement path information and tagging the sensing position to the second sensor data; And a storage step of storing the second sensor data tagged with the sensing position.

A data collection system according to a second aspect of the present disclosure includes a first sensor; A second sensor; A time stamping module for collecting first and second sensor data through the first and second sensors, respectively, and tagging first and second time stamp values on the first and second sensor data, respectively; A loop closure for generating temporary map data of a target area based on the first sensor data, generating temporary location data at a time point corresponding to the first timestamp value, and revisiting a previously past location And generating corrected position data by correcting the temporary position data in response to detection of the loop closure, and generating movement route information and map information of a target area based on the corrected position data.Simultaneous Localization and Mapping ) module; A tagging module for estimating a sensing position at a point in time corresponding to the second timestamp value based on the movement path information and tagging the sensing position to the second sensor data; And a storage module for storing the second sensor data tagged with the sensing position.

According to a third aspect of the present disclosure, a computer program stored in a medium is provided for executing the above-described data collection method using a computer.

According to various embodiments of the present disclosure, while moving an indoor space of a target building using a data collection device, while searching the surroundings, measuring its own location and creating an indoor map of the target building, the sensing location is tagged. It is to provide a way to collect sensor data. Time stamping is performed while collecting sensor data, and since the sensing position of the sensor data is estimated using the time stamp value, an accurate sensing position can be obtained. In addition, according to various embodiments of the present disclosure, since an indoor map of the target building is created and sensor data in the target building is collected at the same time, cost and time required for collecting sensor data may be reduced.

1 shows an example of an overall system configuration according to an embodiment.
2 is an exemplary block diagram illustrating an internal configuration of a mapping robot according to an exemplary embodiment.
3 is an exemplary block diagram illustrating a processor of a mapping robot according to an embodiment.
4 and 5 are reference diagrams for explaining an operation of a processor of a mapping robot according to an exemplary embodiment.
6 is an exemplary block diagram illustrating an internal configuration of a map cloud server according to an embodiment.
7 is an exemplary block diagram illustrating an internal configuration of a service robot according to an exemplary embodiment.

Hereinafter, various exemplary embodiments will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present disclosure. However, the technical idea of the present disclosure may be modified and implemented in various forms, and thus is not limited to the embodiments described herein. In describing various embodiments of the present disclosure, when it is determined that a detailed description of a related known technology may obscure the spirit of the present disclosure, a detailed description of the known technology will be omitted. The same or similar components are given the same reference numerals, and duplicate descriptions thereof will be omitted.

Throughout the specification, when an element is said to be “connected” to another element, this includes not only the case that it is “directly connected”, but also the case that it is “electrically connected” with another element in between. . When a certain element "includes" another element, it means that another element may be further included, rather than excluding another element other than the other element unless specifically stated to the contrary.

Some embodiments may be described with functional block configurations and various processing steps. Some or all of these functional blocks may be implemented with various numbers of hardware and/or software components that perform a specific function. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors, or may be implemented by circuit configurations for a predetermined function. Functional blocks of the present disclosure may be implemented in various programming languages or scripting languages. The functional blocks of the present disclosure may be implemented as an algorithm executed on one or more processors. Functions performed by a function block of the present disclosure may be performed by a plurality of function blocks, or functions performed by a plurality of function blocks in the present disclosure may be performed by one function block.

Connection lines or connecting members between elements shown in the drawings are merely illustrative of functional connections and/or physical or circuit connections. In an actual device, connections between elements may be implemented by various functional connections, physical connections, or circuit connections that can be replaced or added.

1 shows an example of an overall system configuration according to an embodiment.

Referring to FIG. 1, the mapping robot 100, the map cloud server 200, and the service robots 300a, 300b, and 300c are communicatively connected to each other through a network.

The mapping robot 100 and the service robots 300a, 300b, and 300c may be devices that operate indoors of a specific building that cannot receive GPS signals. In this specification, a specific building in which the mapping robot 100 and the service robots 300a, 300b, and 300c are active is referred to as a target building. The interior map of the target building does not exist in advance, and indoor map information of the target building may be generated by the mapping robot 100.

Hereinafter, for convenience of description, it is assumed that the mapping robot 100 and the service robot 300 are robot devices equipped with a moving platform, and embodiments of the present invention will be described, but the present invention is not limited thereto. The mapping robot 100 and the service robot 300 may include service platforms including sensors, for example, devices such as portable terminals, notebook computers, and computers.

In this specification, the mapping robot 100 is a device that generates indoor map information of a target building, and may be referred to as a data collection device. Although the mapping robot 100 is presented as a representative example of a data collection device in the present specification, the present invention is not limited thereto.

The service robots 300a, 300b, and 300c may identify their own location using indoor map information of a target building generated by the mapping robot 100 and perform a service function assigned to them. The service robots 300a, 300b, and 300c not only can identify their own location by using the indoor map information of the target building, but can also perform autonomous driving along a path for moving to a desired location. The service robots 300a, 300b, and 300c may perform logistics service, customer guide service, security service, and the like according to a task assigned to them. The number of service robots 300a, 300b, and 300c disclosed in FIG. 1 is not limited to the present invention, and may be referred to as a service robot 300 when the service robots 300a, 300b, and 300c are generally described. have.

The map cloud server 200 may be a computing device that stores indoor map information of a target building generated by the mapping robot 100 and provides indoor map information to the service robot 300. The map cloud server 200 may calculate and provide a path for the service robot 300 to move. Although the map cloud server 200 is shown to be connected to the mapping robot 100 via a network in FIG. 1, this is exemplary. It may be understood that the map cloud server 200 is disposed in the mapping robot 100, and in this case, the function of the map cloud server 200 may be performed by the mapping robot 100. In the present specification, the data collection device may be a concept encompassing the mapping robot 100 and the map cloud server 200.

The mapping robot 100, the map cloud server 200, and the service robot 300 may transmit and receive data using a network. The network may include a wireless communication network connected to the mapping robot 100 and the service robot 300 and a wired communication network connected to the map cloud server 200. The wireless communication network may include a mobile communication network, a wireless LAN, a local area wireless communication network, and the like. Wireless communication is, for example, LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). ), and the like may include cellular communication. According to an example, the wireless communication network may include at least one of wireless fidelity (WiFi), Bluetooth, Bluetooth low power (BLE), Zigbee, near field communication (NFC), or radio frequency (RF). The wired communication network may include at least one of, for example, universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard-232 (RS-232), power line communication, or plain old telephone service (POTS).

The network may include at least one of a telecommunication network, for example, a computer network, the Internet, or a telephone network. The network is one or more of networks such as personal area network (PAN), local area network (LAN), campus area network (CAN), metropolitan area network (MAN), wide area network (WAN), broadband network (BBN), and the Internet. It can include any network. The network may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, but is not limited thereto.

The service robot 300 may check its own location using indoor map information of a target building provided from the map cloud server 200. The indoor map information may include not only indoor spatial information of the target building, but also sensor data that enables the service robot 300 to check its own location. The sensor data may include different sensor values depending on the location within the target building. For example, the sensor data may include image data photographed at different locations within the target building. The sensor data may include data of wireless signals (eg, WiFi signal, Bluetooth signal, ZigBee signal, etc.) measured at different locations within the target building. Data of wireless signals may include types and strengths of wireless signals. The sensor data may include geomagnetic values measured at different locations within the target building.

The sensor data provided from the map cloud server 200 has different values for different locations in the target building, and the service robot 300 can accurately check its own location using the sensor data only if the location corresponding to the value is correct. have. In the related art, sensor data was collected by repeating the process of measuring the sensor output after placing the sensor at a specific location of the target building at all preset locations in the target building. In the case of collecting sensor data through such a process, there is a difficulty in accurately positioning the sensor at a specific position set in advance, and since sensor values must be measured at all preset positions, it takes a lot of time.

The mapping robot 100 according to the present invention may collect not only the first sensor data but also the second sensor data while moving within the target building. The mapping robot 100 may tag first and second timestamp values to each of the first and second sensor data. The mapping robot 100 generates temporary map data and temporary route information using the collected first sensor data, and performs loop closing when a loop closure is detected to correct the temporary map data to generate indoor map information of the target building. And it is possible to generate moving route information by correcting the temporary route information. The first sensor data is data for generating indoor map information and moving route information, and may be referred to as spatial data. The movement path information is information on a path that the mapping robot 100 has moved.

The mapping robot 100 accurately estimates the sensing position coordinates where the mapping robot 100 was located at the time when the second sensor data was acquired by using the first and second timestamp values tagged to the first and second sensor data. can do. The mapping robot 100 may accurately map the second sensor data and its sensing position coordinates by tagging the estimated sensing position coordinates to the second sensor data. Therefore, it is possible to tag and store the accurate sensing position coordinates of the second sensor data collected at each of a number of locations in the target building, and this process is performed during the time when the mapping robot 100 generates indoor map information of the target building. As it is done, there is little additional time required.

1 is an exemplary illustration of a system capable of implementing a data collection method according to the present invention. Although the data collection method according to the present invention is described as being exemplarily performed by the mapping robot 100 of FIG. 1, the data collection method according to the present invention may be performed by a portable computing device such as a smartphone or a notebook computer.

The data collection method according to the present invention may be performed in the mapping robot 100 and the map cloud server 200, and in this case, the mapping robot 100 and the map cloud server 200 configure one data collection device. I can.

For example, as the mapping robot 100 moves, first and second sensor data are collected through first and second sensors, respectively, and first and second timestamp values are respectively added to the first and second sensor data. Can be time stamped. The mapping robot 100 may provide the map cloud server 200 with first and second sensor data time-stamped with first and second timestamp values, respectively.

The map cloud server 200 may generate temporary map data based on the first sensor data, and may generate temporary location data at a time point corresponding to the first timestamp value. The map cloud server 200 detects a loop closure, and when a loop closure is detected, corrects temporary location data to generate corrected location data, and based on the corrected location data, moving route information and an indoor map of the target building Can generate information. The map cloud server 200 may estimate a sensing position at a time point corresponding to the second timestamp value based on the moving route information, and tag the sensing position to the second sensor data. The map cloud server 200 may store second sensor data to which the sensing location is tagged.

Meanwhile, the correction position and the sensing position may include direction information. For example, the correction position and the sensing position may be a concept including both position information capable of specifying a position in space and direction information capable of specifying a direction at the corresponding position.

2 is an exemplary block diagram illustrating an internal configuration of a mapping robot according to an exemplary embodiment.

Referring to FIG. 2, the mapping robot 100 includes a processor 110, a memory 120, a first sensor unit 130, a second sensor unit 140, a communication unit 150, and a driving unit 160. .

The processor 110 operates according to the code stored in the memory 120 and controls the overall operation of the first sensor unit 130, the second sensor unit 140, the communication unit 150, and the driving unit 160. The memory 120 may store codes for operating the processor 110 and may store data generated by the operation of the processor 110. The processor 110 uses the first and second sensor data collected from the first sensor unit 130 and the second sensor unit 140 to generate indoor map information of the target building and information on its own movement path, and senses Second sensor data to which the location is tagged may be generated. The processor 110 will be described in more detail below.

The first sensor unit 130 may generate spatial data (ie, first sensor data) used for the processor 110 to generate indoor map information and moving route information of a target building. The first sensor unit 130 may include at least one selected from a group including a laser scanner, a camera, and an RGBD sensor. A laser scanner is a device that accurately draws the surroundings by emitting a laser pulse and measuring the distance to the object by receiving the reflected laser pulse from the surrounding target object and returning, and includes Lidar. can do. The laser scanner may be a 3D laser scanner, and may generate 3D spatial data. The laser scanner may generate 3D spatial data at a preset period. When the mapping robot 100 moves, the 3D spatial data generated by the laser scanner changes every frame. The processor 110 extracts feature points of 3D spatial data that change for each frame and matches the feature points, thereby identifying the location and movement path of the mapping robot 100 and generating indoor map information of the target building.

The camera may be, for example, a 360 degree camera, and may be a device that photographs all directions of the mapping robot 100 in the front, rear, left, right, up and down directions. The camera can generate image data photographed in all directions. The processor 110 extracts feature points of image data that change for each frame and matches the feature points, thereby identifying the location and movement path of the mapping robot 100 and generating indoor map information of the target building.

According to an example, the processor 110 may analyze image data acquired from a camera to identify a plane, and may generate 3D map data by analyzing changes in the planes. In addition, the processor 110 may determine its own location in the 3D map data.

The RGBD sensor is a sensor that combines the function of a sensor that captures RGB images like a camera and the function of a depth sensor that detects the distance to surrounding targets. The RGBD sensor can measure not only the image of the surrounding object but also the distance to the surrounding object. The processor 110 may generate indoor map information of the target building based on the output data of the RGBD sensor and determine its own location.

The second sensor unit 140 is a device for generating second sensor data tagged with a sensing position, and may include at least one selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor. The sensing position means a position at which the second sensor data is acquired, and the second sensor data tagged with the sensing position means that the sensing position is stored together with the second sensor data. The sensing position may be stored as metadata of the second sensor data. The sensing location may be stored as coordinates on an indoor map of the target building. The second sensor data tagged with the sensing location may be stored in the map cloud server 200 and provided to the service robot 300 as part of indoor map information of the target building through a network. The second sensor data tagged with the sensing position may be stored in the memory 120 of the mapping robot 100.

The second sensor unit 140 may include a plurality of sensors selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor. For example, the second sensor unit 140 may include a camera and a WiFi module. The second sensor unit 140 may also include a 2D laser scanner. The second sensor unit 140 may include a 2D camera. The sensor devices of the second sensor unit 140 may be cheaper and have lower performance than the sensors of the first sensor unit 130.

The Wi-Fi module may detect identification information of access points detected at the sensing location and strength of a Wi-Fi signal received from each of the access points. The Bluetooth module may detect identification information of Bluetooth devices detected at the sensing location and strength of a Wi-Fi signal received from each of the Bluetooth devices. The geomagnetic sensor can detect the Earth's magnetic field at the sensing position.

The communication unit 150 may be connected to a network to communicate with the map cloud server 200 and/or the service robot 300. The communication unit 150 may be referred to as a communication interface, and may establish wireless communication between the mapping robot 100 and an external device, for example, the map cloud server 200. Wireless communication is, for example, LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). ), and the like may include cellular communication. According to an example, wireless communication may include at least one of wireless fidelity (WiFi), Bluetooth, Bluetooth low power (BLE), Zigbee, near field communication (NFC), or radio frequency (RF).

The driving unit 160 is controlled by the processor 110 and provides a means for the mapping robot 100 to move within the target building. The driving unit 160 may include a motor and a plurality of wheels. For example, the driving unit 160 may include a motor driver, a wheel motor, and a rotation motor. The motor driver may play a role of driving a wheel motor for driving the mapping robot 100. The wheel motor may drive a plurality of wheels for driving the mapping robot 100. The rotation motor may be driven to change or rotate the wheel of the mapping robot 100.

The mapping robot 100 may further include an obstacle recognition unit. The obstacle recognition unit may include at least one of an infrared sensor, an ultrasonic sensor, a cliff sensor, an attitude sensor, a collision sensor, and a light flow sensor. The infrared sensor may include a sensor that receives a signal from an infrared remote control for remotely controlling the mapping robot 100. The ultrasonic sensor may include a sensor for determining a distance between an obstacle and a robot using an ultrasonic signal. The cliff sensor may include a sensor for detecting a cliff or a cliff in a range of 360 degrees. The attitude reference system (ARS) may include a sensor capable of detecting the attitude of the mapping robot 100. The posture sensor may include a sensor composed of 3 axes of acceleration and 3 axes of gyro for detecting the amount of rotation of the mapping robot 100. The collision sensor may include a sensor that detects a collision between the mapping robot 100 and an obstacle. The collision sensor may detect a collision between the mapping robot 100 and an obstacle in a range of 360 degrees. The optical flow sensor (OFS, Optical Flow Sensor) may include a sensor capable of measuring the traveling distance of the robot on various floor surfaces and the phenomenon that the mapping robot 100 rotates while driving.

3 is an exemplary block diagram illustrating a processor of a mapping robot according to an embodiment.

Referring to FIG. 3, the processor 110 may include a time stamping module 111, a Simultaneous Localization and Mapping (SLAM) module 112, a tagging module 113, and a storage module 114.

The time stamping module 111 may collect first sensor data through the first sensor unit 130 and tag first timestamp values on the first sensor data. The time stamping module 111 may collect second sensor data through the second sensor unit 140 and tag second timestamp values on the second sensor data. It is assumed that the first sensor unit 130 is a 3D laser scanner, and the second sensor unit 140 is a WiFi module. The first sensor data may be referred to as spatial data received from a 3D laser scanner exemplified by the first sensor unit 130, and the second sensor data is received from a WiFi module exemplified by the second sensor unit 140. It may be referred to as sensor data.

Timestamping may mean tagging the data by storing a timestamp value corresponding to the time at which the data is received in association with the data or storing it as metadata of the data. A timestamp is information expressed in a consistent format to indicate a specific time, and can be conveniently used when comparing two or more times or calculating a period. The format of the timestamp does not limit the present invention.

The first sensor unit 130 and the second sensor unit 140 may operate independently of each other. Therefore, the collection period of the first sensor data of the first sensor unit 130 (eg, 3D laser scanner) and the collection period of the second sensor data of the second sensor unit 140 (eg, Wi-Fi module) are independent of each other. to be. The reception time of the first sensor data and the reception time of the second sensor data may not be synchronized with each other. For example, the first sensor data may be received at a time interval selected from approximately 1 ms to 100 ms, and the second sensor data may be received at a time interval selected from approximately 1 ms to 1 sec. Each of the first and second sensor data may not be received at a certain period.

The time stamping module 111 may store the first sensor data tagged with the first timestamp value and second sensor data tagged with the second timestamp value in the memory 120. Each of the first and second sensor data may be understood as a set of data sequentially received while the mapping robot 100 moves.

The SLAM module 112 may perform simultaneous position estimation and map creation functions based on the first sensor data, and may estimate the path the mapping robot 100 has moved. Since the first timestamp values are tagged in the first sensor data, the SLAM module 112 can grasp information on which location passes at what time. Since the simultaneous location estimation and mapping function of the SLAM module 112 is known, a detailed description of how the simultaneous location estimation and mapping function can be performed is not described.

According to an example, the SLAM module 112 may generate temporary location data of a time point corresponding to the first timestamp value while generating temporary map data based on the first sensor data. The temporary location data refers to data obtained by temporarily estimating a location through which the mapping robot 100 has passed at a time point corresponding to the first timestamp value. When the first sensor data is 3D image data received from a 3D laser scanner, the SLAM module 112 may perform simultaneous position estimation and map creation functions based on sequentially received 3D image data. In this step, the map data temporarily created by the SLAM module 112 and its own location data may include a sensing error, and the sensing error may accumulate over time and increase. Temporary map data and temporary location data may mean data in which such sensing errors are not removed. For example, the temporary map data and the temporary location data may include a sensing error.

The SLAM module 112 may perform a loop closing function as a method for removing a sensing error. The SLAM module 112 may detect a loop closure. In the present specification, the loop closure means revisiting the location where the mapping robot 100 has passed. The SLAM module 112 compares the current data, which is the currently received first sensor data, with the previous data, which is the previously received first sensor data, and determines that a loop closure has occurred if the similarity between them is greater than a preset threshold. I can. When the first sensor unit 130 is a 3D laser scanner, 3D image data acquired at the same location may be very similar. The 3D image data received from the 3D laser scanner may be converted into a preset coordinate system according to the posture of the mapping robot 100 so as to be easily compared with each other. Revisiting the location where the mapping robot 100 has passed may include not only revisiting the same coordinates, but also revisiting a location adjacent to the previously visited location.

The current data is not compared with all of the previous data, but may be compared with some of the previous data to reduce the amount of computation. Some previous data to be compared may be previous data of a starting point or previous data of an intersection location. Some of the previous data to be compared may be previous data at a point in time when a loop closure may occur based on a movement path predicted based on temporary location data.

According to another embodiment, the SLAM module 112 may also use second sensor data when detecting a loop closure. The SLAM module 112 compares the current data, which is the currently received second sensor data, with the previous data, which is the previously received second sensor data, and determines that a loop closure has occurred when the similarity between them is greater than a preset threshold. I can. When the second sensor unit 140 is a Wi-Fi module, wireless signal data acquired at the same location may be very similar. Even for the second sensor data, the current data is not compared with all of the previous data, but may be compared with some of the previous data to reduce the amount of computation.

When the SLAM module 112 detects the loop closure based on the first sensor data, the SLAM module 112 may determine a previous passing point through which the current location has previously passed using the first timestamp value tagged to the first sensor data. The previous passing point determined using the first sensor data is referred to as the first previous passing point. Even when a loop closure is detected based on the second sensor data, the SLAM module 112 may determine a previous passing point through which the current position has previously passed by using the second timestamp value tagged to the second sensor data. . The previous passing time identified by using the second sensor data is referred to as the second previous passing time. The SLMA module 112 may determine that the loop closure has finally occurred when the difference between the first previous passing time and the second previous passing time is less than a preset threshold.

The SLAM module 112 may generate indoor map information of a target building and information on a moving route in which the mapping robot 100 has moved by removing a sensing error from temporary map data and temporary location data in response to detection of the roof closure. The SLAM module 112 may generate corrected position data by correcting temporary position data in response to detection of the loop closure, and may generate movement path information based on the corrected position data. In addition, when a roof closure is detected, the SLAM module 112 may generate indoor map information of a target building by removing a sensing error from temporary map data.

According to an example, since the difference between the temporary position data at the current time point at which the loop closure is detected and the temporary position data at the first previous pass time corresponds to a sensing error accumulated between the first previous pass time and the current time point, the SLAM module ( 112) Matches the temporary position data at the current point in time when the loop closure is detected and the temporary position data at the first previous passage point, and the temporary position data at the current point in time when the loop closure is detected and the temporary position data at the first previous passage point. Information on the sensing error can be identified based on the difference. The SLAM module 112 may generate corrected position data by correcting a sensing error in the temporary position data. It can be understood that the corrected position data has a sensing error removed from the temporary position data. Corrected position data regarding the position of the mapping robot 100 also includes an estimation error, but includes a much smaller error than the temporary position data.

The SLAM module 112 may generate movement path information based on the corrected position data. Since the first sensor data is time stamped with a first timestamp value, the movement path information includes corrected position data at a time point corresponding to the first timestamp value. That is, the movement path information is information about a path that the mapping robot 100 has moved, and may include information about a time and a location actually passed through due to the first sensor data tagged with the first timestamp value.

The tagging module 113 may estimate the sensing position of a viewpoint corresponding to the second timestamp value based on the movement path information. Since the movement path information includes information on the time and location at which the mapping robot 100 moved, the tagging module 113 can identify the location where the mapping robot 100 was located at a time corresponding to the second timestamp value. have. This position may be referred to as a sensing position. According to an example, the tagging module 113 may estimate the sensing position of the viewpoint corresponding to the second timestamp value by interpolating by using the corrected position data of the viewpoint corresponding to the first timestamp value. The time point corresponding to the first timestamp value may not exactly match the time point corresponding to the second timestamp value. Two first timestamp values adjacent to the second timestamp value may be selected. Corrected position data corresponding to the two first timestamp values may be determined. Based on the difference between the second timestamp value and the two adjacent first timestamp values, assuming that the mapping robot 100 moves at a linear constant velocity between the points of time corresponding to the two first timestamp values. By performing the operation, it is possible to estimate the sensing position at the time point corresponding to the second timestamp value.

It may not be assumed that the mapping robot 100 performs a linear constant velocity movement between positions corresponding to two adjacent first timestamp values. For example, the mapping robot 100 may move irregularly. In this case, the position of the viewpoint corresponding to the second timestamp value may be determined based on the positions of the viewpoint corresponding to two or more first timestamp values adjacent to the second timestamp value. The tagging module 133 predicts the type of movement of the mapping robot 100 based on the positions of the viewpoints corresponding to two or more first timestamp values, and the type of the movement and the time point corresponding to the second timestamp values. Based on the positions, the position of a viewpoint corresponding to the second timestamp value may be estimated.

The tagging module 113 may estimate the sensing position based on the position at which the mapping robot 100 was located at a time point corresponding to the second timestamp value. Strictly, the position of the viewpoint corresponding to the second timestamp value means the position of the first sensor unit 130. On the other hand, the sensing position may mean the position of the second sensor unit 140.

The first sensor unit 130 and the second sensor unit 140 may exist at different locations within the mapping robot 100. The processor 110 may pre-store information on a positional relationship between the first sensor unit 130 and the second sensor unit 140. The processor 110 calculates a relationship between the position of the second sensor unit 140 with the position of the first sensor unit 130 according to the posture of the mapping robot 100, and uses the calculated position relationship to determine the first sensor. The position of the second sensor unit 140 may be accurately calculated from the position of the unit 130. Through this, the tagging module 133 may estimate the sensing position based on the position where the first sensor unit 130 was located at a time corresponding to the second timestamp value.

The tagging module 113 may tag the estimated sensing position to the second sensor data time-stamped with the second timestamp value. As the tagging module 113 performs the above operation on all of the second sensor data, all of the second sensor data may be tagged with a sensing position. When the second sensor unit 140 is a camera, the second sensor data tagged with the sensing position may include an image captured by the camera and the position of the camera. The location of the camera included in the second sensor data may include a photographing direction of the camera. In the case of a fixed camera, the photographing direction of the camera may be determined through the posture of the mapping robot 100. In the case of a gimbal camera, the photographing direction of the camera may be determined based on the posture and gimbal angle of the mapping robot 100.

The storage module 114 may store second sensor data to which the sensing position is tagged. The storage module 114 may store second sensor data tagged with a sensing position in the memory 120. The storage module 114 may store the second sensor data tagged with the sensing location in the map cloud server 200. The storage module 114 may combine the second sensor data tagged with the sensing location with indoor map information of the target building.

When the second sensor unit 140 includes a plurality of types of sensors, a plurality of second sensor data may exist, and a plurality of second sensor data each tagged with a sensing position may be generated by the tagging module 113. I can. The indoor map information of the target building may include data from a plurality of second sensors. For example, when the second sensor unit 140 includes a camera and a Wi-Fi module, the second sensor data may include image data captured by the camera at the sensing position and data of Wi-Fi signals received at the sensing position.

The processor 110 may further include an autonomous driving module for autonomous driving of the mapping robot 100, and the driving unit 160 may be driven by the control of the autonomous driving module. The autonomous driving module may directly determine a path to which the mapping robot 100 will move. The mapping robot 100 may search for a target building before the indoor map information of the target building is created. The autonomous driving module may have an algorithm for searching for a target building. For example, the autonomous driving module has an algorithm that rotates in a specific direction at all intersections in order to search for a target building, or an algorithm that determines a route so that a loop closure occurs when a preset travel distance or time comes. Can have.

According to another embodiment, the function of determining the path to which the mapping robot 100 will move may be performed in the map cloud server 200, and the processor 110 may receive the path to which it will move from the map cloud server 200. I can.

4 and 5 are reference diagrams for explaining an operation of a processor of a mapping robot according to an exemplary embodiment.

* Referring to FIGS. 4 and 5, a temporary movement path P1 before the loop closing operation of the SLAM module 112 and an estimated movement path P2 estimated after the loop closing operation are illustrated. It is assumed that the mapping robot 100 moves the rectangular path counterclockwise starting from the lower left. 5 is an enlarged view of portion A in the reference diagram of FIG. 4.

The temporary movement path P1 may be determined based on the temporary location data ei. As can be seen from the temporary movement path P1, the temporary location data ei may gradually deviate from the actual path as sensing errors accumulate as the mapping robot 100 moves.

The loop closing operation may be an operation of matching the temporary position data ei at two points when the loop closure is detected. The correction position data ri generated by the loop closing operation may be understood as removing a sensing error from the temporary position data ei.

When the processor 110 determines a sensing position each time the second sensor data is received from the second sensor unit 140 and tags the determined sensing position to the second sensor data, the sensing position of the second sensor data (mj ) May be determined as a position on the temporary movement path P1 as shown in FIG. 5 or may be determined as a position of adjacent temporary position data ei on the temporary movement path P1. The processor 110 may determine the current position based on the first sensor data as temporary position data ei, and the processor 110 determines the sensing position of the second sensor data based on the temporary position data ei. I can.

According to the present embodiments, the second sensor data is timestamped with a second timestamp value. The processor 110 may determine the sensing position of the second sensor data as the position on the estimated movement path P2 by using the second timestamp value of the second sensor data. As a result, the sensing position of the second sensor data can be more accurate, and there is little time to be additionally administered to determine the sensing position of the second sensor data. In addition, a very large number of second sensor data may be acquired along the path of the mapping robot 100 in the target building. Second sensor data measured at very many sensing locations within the target building may be collected. Conventionally, since the operator had to move with the sensor device to a preset location and collect sensor data, sensor data could be collected only for a limited location, but according to these embodiments, the mapping robot 100 moves within the target building. The second sensor data may be collected at intervals of about several centimeters on the path.

6 is an exemplary block diagram illustrating an internal configuration of a map cloud server according to an embodiment.

Referring to FIG. 6, the map cloud server 200 includes a processor 210, a memory 220, and a communication unit 230.

The processor 110 operates according to the code stored in the memory 120 and includes a memory 220 and a communication unit 230. The memory 220 may store a code for operating the processor 210 and may store data generated by an operation of the processor 210. The memory 220 may include a storage device that stores indoor map information of a target building. According to another example, the memory 220 may include a database of a separate storage server connected through a network, and indoor map information of a target building may be stored in the database.

The processor 210 may receive indoor map information of the target building from the mapping robot 100. The indoor map information may include a 3D map of the target building. The indoor map information may include second sensor data to which the sensing location is tagged. The sensing location may be specified as a location on the indoor map information of the target building.

The processor 210 may provide the service robot 300 with indoor map information of the target building. The service robot 300 may determine its own location by using second sensor data included in the indoor map information of the target building. In the second sensor data, the accurately estimated sensing position is tagged. The service robot 300 may estimate its own location by comparing values measured using sensors in the target building with second sensor data.

According to another embodiment, the processor 210 may receive information on a departure location and information on a destination from the mapping robot 100 and/or the service robot 300. The processor 210 may determine an optimal path to move from the departure point to the destination using indoor map information of the target building, and provide information on the optimal path to the mapping robot 100 and/or the service robot 300. have. The mapping robot 100 and/or the service robot 300 receiving information on the optimal path may move according to the optimal path. According to another example, the mapping robot 100 may include an autonomous driving module, and may move according to a path determined by the autonomous driving module.

According to another embodiment, the processor 210 may perform functions of the SLAM module 112, the tagging module 113, and the storage module 114 of FIG. 3. For example, the processor 210 may receive the first and second sensor data from the mapping robot 100 to which first and second timestamp values are timestamped, respectively.

The processor 210 may generate temporary map data based on first sensor data time-stamped with the first timestamp value, and generate temporary location data of a time point corresponding to the first timestamp value.

The processor 210 may detect a loop closure that revisits a location where the mapping robot 100 has passed. When the roof closure is detected, the processor 210 may correct temporary location data to generate corrected location data, and generate movement route information and indoor map information of a target building based on the corrected location data.

The processor 210 may estimate a sensing position at a point in time corresponding to the second timestamp value based on the moving path information, and tag the sensing position to the second sensor data. The processor 210 may store second sensor data to which the sensing position is tagged.

The communication unit 230 may be connected to a network to communicate with the mapping robot 100 and/or the service robot 300. The communication unit 230 may be referred to as a communication interface, and may establish communication with the mapping robot 100 and/or the service robot 300. The communication unit 230 may access a network through wired communication. Wired communication may include at least one of a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), and an integrated service digital network (ISDN).

7 is an exemplary block diagram illustrating an internal configuration of a service robot according to an exemplary embodiment.

Referring to FIG. 7, the service robot 300 includes a processor 310, a memory 320, a second sensor unit 330, a communication unit 340, and a driving unit 350.

The processor 310 operates according to the code stored in the memory 320 and controls the overall operation of the memory 320, the second sensor unit 330, the communication unit 340, and the driving unit 350. The memory 320 may store a code for operating the processor 310 and may store data generated by the operation of the processor 310. The processor 310 may estimate its own position in the target building using sensor data received from the second sensor unit 330 and indoor map information of the target building stored in the map cloud server 200. The processor 310 will be described in more detail below.

The second sensor unit 330 may include a sensor that generates sensor data of the same type as the second sensor data included in the indoor map information of the target building. The second sensor unit 330 may correspond to the second sensor unit 140 of the mapping robot 100. For example, when the second sensor unit 140 of the mapping robot 100 includes at least one selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor, the second sensor unit 330 is also At least one sensor of the same type as the sensor unit 140 may be included. When the second sensor unit 140 of the mapping robot 100 includes a camera and a Wi-Fi module, and the indoor map information of the target building includes image data of the camera and wireless signal data of the Wi-Fi module, the second sensor unit ( 330 may include only one camera or a Wi-Fi module, or may include both a camera and a Wi-Fi module.

The sensor of the first sensor unit 130 of the mapping robot 100 is expensive and may have higher performance than the sensor of the second sensor unit 140. Since the service robot 300 does not include an expensive sensor such as the first sensor unit 130 of the mapping robot 100, it can be manufactured inexpensively compared to the mapping robot 100, and various Can be manufactured in different types.

The communication unit 340 may be connected to a network to communicate with the mapping robot 100, the map cloud server 200, and/or another service robot 300. The communication unit 340 may be referred to as a communication interface supporting wireless communication. The wireless communication may include at least one of cellular communication, wireless fidelity (WiFi), Bluetooth, Bluetooth low power (BLE), Zigbee, near field communication (NFC), or radio frequency (RF).

The driving unit 350 is controlled by the processor 310 and provides a means for the service robot 300 to move. The driving unit 350 may include a motor and a plurality of wheels. For example, the driving unit 350 may include a motor driver, a wheel motor, and a rotation motor. The motor driver may play a role of driving a wheel motor for driving the service robot 300. The wheel motor may drive a plurality of wheels for driving the service robot 300. The rotation motor may be driven to change or rotate the wheel of the service robot 300.

According to an embodiment, the processor 310 may receive sensor data from the second sensor unit 330. The processor 310 may receive indoor map information of the target building from the map cloud server 200. The processor 310 may compare the second sensor data tagged with the sensing location included in the indoor map information of the target building with the sensor data received from the second sensor unit 330 to determine a similarity between them. The processor 310 may extract second sensor data having high similarity and estimate the current position of the service robot 300 based on the sensing position tagged in the second sensor data.

According to another embodiment, when receiving sensor data from the second sensor unit 330, the processor 310 may transmit the sensor data to the map cloud server 200. The map cloud server 200 may estimate the current location of the service robot 300 based on sensor data received from the service robot 300 and may transmit information about the current location to the service robot 300.

When a path in which the mapping robot 100 and the service robot 300 moves is designated in the target building, the mapping robot 100 may receive the second sensor data at all positions on the path. The indoor map information of the target building may include second sensor data acquired at all locations on the route, and the second sensor data may be tagged with an accurate sensing location. When the service robot 300 is located on a designated path in the target building, there is a high possibility that second sensor data having a very high similarity to the sensor data received through the second sensor unit 330 exists. Therefore, the current position of the service robot 300 can be estimated very accurately.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded in a computer-readable medium. In this case, the medium may be one that continuously stores a program executable by a computer, or for execution or download. In addition, the medium may be a variety of recording means or storage means in a form in which a single piece of hardware or several pieces of hardware are combined. The medium is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magnetic-optical media such as floptical disks, and And ROM, RAM, flash memory, and the like may be configured to store program instructions. In addition, examples of other media include an app store that distributes applications, a site that supplies or distributes various software, and a recording medium or storage medium managed by a server.

In the above, the present invention has been described by specific matters such as specific elements and limited embodiments and drawings, but this is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Anyone with ordinary knowledge in the technical field to which the invention belongs can make various modifications and changes from these descriptions.

Accordingly, the spirit of the present invention is limited to the above-described embodiments and should not be defined, and all ranges equivalent to or equivalently changed from the claims to be described later as well as the claims to be described later are the scope of the spirit of the present invention. It will be said to belong to.

Claims (12)

Collecting spatial data for generating map information and moving route information using a lidar sensor provided in the data collection device;
Collecting image data photographed from a plurality of directions using a camera sensor provided in the data collection device and direction information specifying a photographing direction of the camera sensor;
Stamping a first time stamp value on the spatial data based on a time point at which the spatial data is collected;
Stamping a second time stamp value on the image data based on a time point at which the image data is collected;
Generating temporary map data by using the spatial data and the first time stamp value, and generating temporary location data corresponding to the location of the data collection device at the time the spatial data is collected;
Detecting, by the data collection device, a loop closure revisiting a location corresponding to the temporary location data;
Based on new spatial data collected from the lidar sensor when the data collection device revisits the location, moving path information and map information of the data collection device are removed by removing a sensing error from the temporary location data. Generating;
Estimating a sensing position at a time point corresponding to the second time stamp value based on moving path information of the data collection device, and tagging the estimated sensing position to the image data; And
And combining the image data tagged with the estimated sensing position and direction information specifying a photographing direction of the camera sensor with the map information. The method of claim 1,
The location of the camera sensor for which the photographing direction is specified is determined as the estimated sensing location. The method of claim 2,
So that the location of the other device in the building can be measured based on the image taken by the device different from the data collection device,
The map information, the image data tagged with the estimated sensing position, and direction information of the camera sensor at the estimated sensing position are stored as map data for the building. The method of claim 1,
The movement path information includes corrected position data,
The corrected position data is generated based on a time point corresponding to the first timestamp value by removing a sensing error from the temporary position data,
The sensing position of the viewpoint corresponding to the second timestamp value is estimated through interpolation using corrected position data of the viewpoint corresponding to the first timestamp value. The method of claim 4,
A map, characterized in that the sensing position of the viewpoint corresponding to the second timestamp value is estimated using the corrected position data of the viewpoint corresponding to the first timestamp value for each of the image data captured in the plurality of directions How to write information. The method of claim 1,
Direction information specifying the photographing direction of the camera sensor is set using the attitude information of the data collection device. The method of claim 6,
The camera sensor includes at least one of a fixed camera or a gimbal camera,
The photographing direction of the fixed camera is set through the posture of the data collection device,
The photographing direction of the gimbal camera is set using a posture of the data collection device and an angle of the gimbal camera. The method of claim 7,
And the data collection device includes a posture sensor capable of detecting a posture of the data collection device. The method of claim 1,
Detecting the loop closure comprises:
Calculating a similarity between the spatial data;
Calculating a degree of similarity between the image data; And
And detecting the loop closure based on the similarity between the spatial data and the similarity between the image data. The method of claim 1,
The data collection device includes a mapping robot,
The lidar sensor and the camera sensor are located in the mapping robot,
The mapping robot is connected to a network and moves in a building while performing communication with an external device. A lidar sensor that collects spatial data for generating map information and moving route information;
A camera sensor for obtaining image data photographed in a plurality of directions;
A time stamping module for stamping a first time stamp value on the spatial data based on a time point at which the spatial data is collected, and stamping a second time stamp value on the image data based on a time point at which the image data is collected;
A loop for generating temporary map data by using the spatial data and the first time stamp value, generating temporary location data at the time when the spatial data is collected, and revisiting a location corresponding to the temporary location data A SLAM (Simultaneous Localization and Mapping) module that detects a closure and corrects the temporary location data in response to the detection of the loop closure to generate movement route information and map information;
A tagging module for estimating a sensing position at a time point corresponding to the second time stamp value based on the movement path information, and tagging the estimated sensing position to the image data; And
Data comprising a processor that collects direction information specifying a photographing direction of the camera sensor, and combines the image data tagged with the estimated sensing position and direction information specifying the photographing direction of the camera sensor with the map information Collection system. A program executed by one or more processes in an electronic device and stored in a computer-readable medium,
The above program,
Collecting spatial data for generating map information and moving route information using a lidar sensor provided in the data collection device;
Collecting image data photographed from a plurality of directions using a camera sensor provided in the data collection device and direction information specifying a photographing direction of the camera sensor;
Stamping a first time stamp value on the spatial data based on a time point at which the spatial data is collected;
Stamping a second time stamp value on the image data based on a time point at which the image data is collected;
Generating temporary map data by using the spatial data and the first time stamp value, and generating temporary location data corresponding to the location of the data collection device at the time the spatial data is collected;
Detecting, by the data collection device, a loop closure revisiting a location corresponding to the temporary location data;
Based on new spatial data collected from the lidar sensor when the data collection device revisits the location, moving path information and map information of the data collection device are removed by removing a sensing error from the temporary location data. Generating;
Estimating a sensing position at a time point corresponding to the second time stamp value based on moving path information of the data collection device, and tagging the estimated sensing position to the image data; And
A computer-readable medium comprising instructions for performing the step of combining the image data tagged with the estimated sensing position and direction information specifying a photographing direction of the camera sensor with the map information. Stored in the program.

Images