KR20200043329A - Method and system for collecting data - Google Patents

Abstract

Provided is a data collection method according to various embodiments. The data collection method comprises: a time stamping step in which a data collection apparatus comprising first and second sensors, while traveling within a target area, collects first and second sensor data respectively through the first and second sensors, and tags first and second time stamp values respectively to the first and second sensor data; a data generation step in which on the basis of the first sensor data that the data collection apparatus collects while traveling, temporary map data of the target area and temporary location data corresponding to the first time stamp value are generated; a detection step in which a loop closure revisiting locations by which the data collection apparatus has passed is detected; a loop closing step in which in response to detection of the loop closure, corrected location data is generated by correcting the temporary location data, and on the basis of the corrected location data, travelling route information and map information of the target area are generated; a tagging step in which on the basis of the travelling route information, a sensing location of a time point corresponding to the second time stamp value is estimated, and the sensing location is tagged to the second sensor data; and a storage step in which the second sensor data tagged with the sensing location is stored.

Description

Method and system for collecting data

The present disclosure relates to a method and system for collecting data, and more particularly, to a method and system for collecting sensor data tagged with an indoor location.

In order to provide a location-based service to a user, it is necessary to obtain accurate location information of a user or a target object, and this location information is generally obtained based on a Global Positioning System (GPS) signal. However, there are many difficulties in measuring the location of a user or a target object in a space where the GPS signal is difficult to reach, such as a shaded area of the GPS signal, such as between high buildings or the interior of a building.

As a method for providing location-based services even in spaces where GPS signals are difficult to reach, such as inside buildings, Korean Patent Publication No. 10-2012-0029976 (published on March 27, 2012) "Standards for indoor wireless positioning Information collection method and apparatus, indoor wireless positioning service method using a plurality of access points installed indoors "collects scan information for an indoor access point, and monitors access using preset location information of at least one monitoring access point Map scan information for a point to an indoor map, generate location information on the indoor map for the virtual access point by using scan information for at least one virtual access point obtained through the scan information, and generate the indoor information. Preset on the map and the monitoring access point and the Generating identification information for each cell including an access point, a method of collecting the reference point information for the indoor wireless positioning is disclosed. However, this method has a problem that indoor map information must exist in advance.

The problem to be solved by the various embodiments of the present disclosure is to collect the sensor data tagged with the sensing location while measuring its own location while creating a map while navigating the surroundings while moving the indoor space using a data collection device. Is to provide a way.

As a technical means for achieving the above technical problems, a data collection method according to a first aspect of the present disclosure is provided. The data collection method collects first and second sensor data through the first and second sensors, respectively, while the data collection device including the first and second sensors moves within a target area, and the first and second A timestamp step of tagging the first and second timestamp values respectively in the two sensor data; A data generation step of generating temporary map data of the target area and temporary location data at a time corresponding to the first timestamp value based on the first sensor data collected while the data collection device is moving; A detection step of detecting a loop closure re-visiting a location where the data collection device has passed; A loop closing step in response to the detection of the loop closure, correcting the temporary position data to generate corrected position data, and generating movement route information and map information of the target area based on the corrected position data; A tagging step of estimating a sensing position at a time point corresponding to the second timestamp value based on the movement path information, and tagging the sensing position in the second sensor data; And a storage step of storing the second sensor data tagged with the sensing location.

A data collection system according to a second aspect of the present disclosure includes a first sensor; A second sensor; A time stamping module for collecting first and second sensor data through the first and second sensors, respectively, and tagging the first and second sensor data with first and second timestamp values, respectively; Based on the first sensor data, temporary map data of a target area is generated, temporary location data at a time point corresponding to the first timestamp value, and a loop closure revisiting a previously passed location. Detecting, and correcting the temporary position data in response to the detection of the loop closure to generate corrected position data and generating motion path information and map information of a target area based on the corrected position data SLAM (Simultaneous Localization and Mapping) ) module; A tagging module for estimating a sensing position at a time point corresponding to the second timestamp value based on the movement path information, and tagging the sensing position in the second sensor data; And a storage module for storing the second sensor data tagged with the sensing location.

According to a third aspect of the present disclosure, a computer program stored in a medium is provided to execute the above-described data collection method using a computer.

According to various embodiments of the present disclosure, while moving the indoor space of the target building by using a data collection device, while searching for the surroundings, measuring their location and creating an indoor map of the target building, the sensing location is tagged It provides a method for collecting sensor data. Time stamping is performed while collecting sensor data, and the sensing position of the sensor data is estimated using the time stamp value, so that an accurate sensing position can be obtained. In addition, according to various embodiments of the present disclosure, since the indoor map of the target building is generated and the sensor data in the target building is collected, the cost and time required to collect the sensor data can be reduced.

1 shows an example of an overall system configuration according to an embodiment.
2 is an exemplary block diagram illustrating an internal configuration of a mapping robot according to an embodiment.
3 is an exemplary block diagram illustrating a processor of a mapping robot according to an embodiment.
4 and 5 are reference diagrams for explaining the operation of the processor of the mapping robot according to an embodiment.
6 is an exemplary block diagram illustrating an internal configuration of a map cloud server according to an embodiment.
7 is an exemplary block diagram illustrating an internal configuration of a service robot according to an embodiment.

Hereinafter, various embodiments will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present disclosure pertains can easily implement them. However, the technical spirit of the present disclosure may be implemented by being modified in various forms and is not limited to the embodiments described herein. In describing various embodiments of the present disclosure, when it is determined that the detailed description of related publicly known technologies may obscure the gist of the technical spirit of the present disclosure, detailed description of the publicly known technologies will be omitted. Identical or similar elements will be given the same reference numbers and redundant description will be omitted.

Throughout the specification, when an element is said to be "connected" to another element, this includes not only "directly connected" but also "electrically connected" with other elements in between. . When an element is said to "include" another element, this means that other elements may be further included, rather than excluding other elements, unless specifically stated otherwise.

Some embodiments can be described with functional block configurations and various processing steps. Some or all of these functional blocks may be implemented with various numbers of hardware and / or software configurations that perform particular functions. For example, the functional blocks of the present disclosure can be implemented by one or more microprocessors, or by circuit configurations for a given function. The functional blocks of the present disclosure can be implemented in various programming languages or scripting languages. The functional blocks of the present disclosure can be implemented with an algorithm running on one or more processors. Functions performed by the function blocks of the present disclosure may be performed by a plurality of function blocks, or functions performed by the plurality of function blocks in the present disclosure may be performed by a single function block.

The connecting lines or connecting members between the elements shown in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In an actual device, connections between elements may be implemented by various functional connections, physical connections, or circuit connections that are replaceable or added.

1 shows an example of an overall system configuration according to an embodiment.

Referring to FIG. 1, the mapping robot 100, the map cloud server 200, and the service robots 300a, 300b, and 300c are communicatively connected to each other through a network.

The mapping robot 100 and the service robots 300a, 300b, and 300c may be devices that operate indoors in a specific building that cannot receive GPS signals. In this specification, a specific building in which the mapping robot 100 and service robots 300a, 300b, and 300c are active is referred to as a target building. The internal map of the target building does not exist in advance, and indoor map information of the target building may be generated by the mapping robot 100.

Hereinafter, for convenience of description, it is assumed that the mapping robot 100 and the service robot 300 are robot devices equipped with a mobile platform, but embodiments of the present invention will be described, but the present invention is not limited thereto. The mapping robot 100 and the service robot 300 may include service platforms including sensors, for example, devices such as portable terminals, notebooks, and computers.

In the present specification, the mapping robot 100 is a device that generates indoor map information of a target building, and may be referred to as a data collection device. Although a mapping robot 100 is presented as a representative example of a data collection device in the present specification, the present invention is not limited thereto.

The service robots 300a, 300b, and 300c may use the indoor map information of the target building generated by the mapping robot 100 to determine their location and perform a service function assigned to them. The service robots 300a, 300b, and 300c can not only identify their location using indoor map information of the target building, but also autonomously drive along a path to move to a desired place. The service robots 300a, 300b, and 300c may perform a logistics service, a guide service for a customer, a security service, etc. according to a task assigned to them. The number of service robots 300a, 300b, and 300c disclosed in FIG. 1 does not limit the present invention, and may be referred to as a service robot 300 when the service robots 300a, 300b, and 300c are generally described. have.

The map cloud server 200 may be a computing device that stores indoor map information of the target building generated by the mapping robot 100 and provides indoor map information to the service robot 300. The map cloud server 200 may calculate and provide a path to which the service robot 300 moves. Although the map cloud server 200 is illustrated in FIG. 1 as being connected to the mapping robot 100 through a network, this is exemplary. It may be understood that the map cloud server 200 is disposed in the mapping robot 100, and in this case, the function of the map cloud server 200 may be performed by the mapping robot 100. In this specification, the data collection device may be a concept encompassing the mapping robot 100 and the map cloud server 200.

The mapping robot 100, the map cloud server 200, and the service robot 300 may transmit and receive data using a network. The network may include a wireless communication network connected to the mapping robot 100 and the service robot 300 and a wired communication network connected to the map cloud server 200. The wireless communication network may include a mobile communication network, a wireless LAN, and a short-range wireless communication network. Wireless communication includes, for example, LTE, LTE-A (LTE Advance), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM). ), And the like. According to an example, the wireless communication network may include at least one of WiFi (wireless fidelity), Bluetooth, Bluetooth low power (BLE), Zigbee (Zigbee), near field communication (NFC), or radio frequency (RF). The wired communication network may include at least one of a universal serial bus (USB), a high definition multimedia interface (HDMI), a recommended standard-232 (RS-232), power line communication, or plain old telephone service (POTS).

The network may include at least one of a telecommunications network, for example, a computer network, the Internet, or a telephone network. The network may include one or more of a network such as a personal area network (PAN), local area network (LAN), campus area network (CAN), metropolitan area network (MAN), wide area network (WAN), broadband network (BBN), and the Internet. It can include any network. The network may include, but is not limited to, any one or more of network topologies, including bus networks, star networks, ring networks, mesh networks, star-bus networks, tree or hierarchical networks, and the like.

The service robot 300 may confirm its location using indoor map information of the target building provided from the map cloud server 200. The indoor map information may include indoor space information of the target building, as well as sensor data allowing the service robot 300 to check its location. The sensor data may include different sensor values depending on the location in the target building. For example, the sensor data may include image data captured at different locations within the target building. The sensor data may include data of wireless signals (eg, WiFi signals, Bluetooth signals, Zigbee signals, etc.) measured at different locations in the target building. The data of the radio signals may include the type and strength of the radio signals. The sensor data may include geomagnetic values measured at different locations within the target building.

The sensor data provided from the map cloud server 200 has different values for different locations in the target building, and only when the location corresponding to the value is correct, the service robot 300 can accurately determine its location using the sensor data. have. In the related art, sensor data was collected by repeating the process of measuring the sensor output at all predetermined positions in the target building after placing the sensor at a specific position in the target building. When collecting sensor data through this process, there is a difficulty in accurately positioning the sensor at a specific preset position, and it takes a lot of time because sensor values must be measured at all preset positions.

The mapping robot 100 according to the present invention may collect the second sensor data as well as the first sensor data while moving within the target building. The mapping robot 100 may tag the first and second timestamp values to the first and second sensor data, respectively. The mapping robot 100 generates temporary map data and temporary route information using the collected first sensor data, and when loop closure is detected, performs loop closing to correct the temporary map data to generate indoor map information of the target building. And correct the temporary route information to generate moving route information. The first sensor data may be referred to as spatial data as data for generating indoor map information and moving route information. The movement path information is information about a path that the mapping robot 100 has moved.

The mapping robot 100 accurately estimates the sensing position coordinates at which the mapping robot 100 was located at the time when the second sensor data was acquired by using the first and second timestamp values tagged to the first and second sensor data. can do. The mapping robot 100 can accurately map the second sensor data and its sensing position coordinates by tagging the estimated sensing position coordinates to the second sensor data. Accordingly, the exact sensing position coordinates of the second sensor data collected at each of a number of locations in the target building can be tagged and stored, and this process is performed while the mapping robot 100 generates indoor map information of the target building. Since it is done, there is little additional time.

1 exemplarily shows a system capable of implementing a data collection method according to the present invention. Although the data collection method according to the present invention is exemplarily described as being performed by the mapping robot 100 of FIG. 1, the data collection method according to the present invention may be performed by a portable computing device such as a smart phone or a laptop.

The data collection method according to the present invention may be performed by the mapping robot 100 and the map cloud server 200, and in this case, the mapping robot 100 and the map cloud server 200 configure one data collection device. You can.

For example, as the mapping robot 100 moves, the first and second sensor data are collected through the first and second sensors, respectively, and the first and second timestamp values are respectively assigned to the first and second sensor data. Can be time stamped. The mapping robot 100 may provide the first and second sensor data, wherein the first and second timestamp values are time stamped, respectively, to the map cloud server 200.

The map cloud server 200 may generate temporary map data based on the first sensor data and may generate temporary location data at a time point corresponding to the first timestamp value. The map cloud server 200 detects a loop closure, and if the loop closure is detected, corrects temporary location data to generate correction location data, and based on the correction location data, moves route information and indoor map of the target building Information can be generated. The map cloud server 200 may estimate a sensing position at a time point corresponding to a second timestamp value based on the movement route information, and tag the sensing position in the second sensor data. The map cloud server 200 may store the second sensor data tagged with the sensing location.

Meanwhile, the correction position and the sensing position may include direction information. For example, the correction position and the sensing position may be a concept including both position information that can specify a position in space and direction information that can specify a direction at the corresponding position.

2 is an exemplary block diagram illustrating an internal configuration of a mapping robot according to an embodiment.

Referring to FIG. 2, the mapping robot 100 includes a processor 110, a memory 120, a first sensor unit 130, a second sensor unit 140, a communication unit 150, and a driving unit 160. .

The processor 110 operates according to codes stored in the memory 120 and controls overall operations of the first sensor unit 130, the second sensor unit 140, the communication unit 150, and the driving unit 160. The memory 120 may store code for operating the processor 110 and may store data generated by the operation of the processor 110. The processor 110 uses the first and second sensor data collected from the first sensor unit 130 and the second sensor unit 140 to generate indoor map information of the target building and its own movement path information, and sensing The second sensor data tagged with the location may be generated. The processor 110 will be described in more detail below.

The first sensor unit 130 may generate spatial data (ie, first sensor data) used by the processor 110 to generate indoor map information and moving path information of the target building. The first sensor unit 130 may include at least one selected from a group including a laser scanner, a camera, and an RGBD sensor. The laser scanner is a device that precisely draws the surroundings by firing a laser pulse and measuring the distance to the object after receiving the returned laser pulse reflected from the target object, and includes a lidar. can do. The laser scanner may be a 3D laser scanner and may generate 3D spatial data. The laser scanner can generate 3D spatial data at a preset period. When the mapping robot 100 moves, the 3D spatial data generated by the laser scanner changes every frame. The processor 110 may extract the feature points of the 3D spatial data that change for each frame and match the feature points, thereby grasping the location and movement path of the mapping robot 100 and generating indoor map information of the target building.

The camera may be, for example, a 360-degree camera, and may be a device that photographs all directions of up, down, left, right, up and down at the position of the mapping robot 100. The camera can generate image data captured in all directions. The processor 110 may extract the feature points of the image data that change for each frame and match the feature points, thereby grasping the location and the moving path of the mapping robot 100 and generating indoor map information of the target building.

According to an example, the processor 110 may analyze the image data acquired from the camera to identify a plane, and generate 3D map data by analyzing changes in planes. Also, the processor 110 may grasp its own position in the 3D map data.

The RGBD sensor is a sensor that combines the function of a sensor that shoots an RGB image, such as a camera, and the depth sensor that detects the distance to a target object around it. The RGBD sensor can measure not only the image of the surrounding object, but also the distance to the surrounding object. The processor 110 may generate indoor map information of the target building based on the output data of the RGBD sensor and identify its own location.

The second sensor unit 140 is a device for generating second sensor data tagged with a sensing location, and may include at least one selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor. The sensing position means a position where the second sensor data is acquired, and the second sensor data tagged with the sensing position means that the sensing position is stored together with the second sensor data. The sensing location may be stored as meta data of the second sensor data. The sensing location may be stored as coordinates on the indoor map of the target building. The second sensor data tagged with the sensing location may be stored in the map cloud server 200 and provided to the service robot 300 as part of the indoor map information of the target building through the network. The second sensor data tagged with the sensing location may be stored in the memory 120 of the mapping robot 100.

The second sensor unit 140 may include a plurality of sensors selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor. For example, the second sensor unit 140 may include a camera and a Wi-Fi module. The second sensor unit 140 may also include a two-dimensional laser scanner. The second sensor unit 140 may include a two-dimensional camera. The sensor devices of the second sensor unit 140 may be cheaper and have lower performance than the sensors of the first sensor unit 130.

The Wi-Fi module can detect the identification information of the access points sensed at the sensing location and the strength of the Wi-Fi signal received from each of the access points. The Bluetooth module can detect the identification information of Bluetooth devices sensed at the sensing location and the strength of the Wi-Fi signal received from each of the Bluetooth devices. The geomagnetic sensor can detect the Earth's magnetic field at a sensing position.

The communication unit 150 may be connected to a network to communicate with the map cloud server 200 and / or the service robot 300. The communication unit 150 may be referred to as a communication interface, and may establish wireless communication between the mapping robot 100 and an external device, such as a map cloud server 200. Wireless communication includes, for example, LTE, LTE-A (LTE Advance), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM). ), And the like. According to an example, the wireless communication may include at least one of WiFi (wireless fidelity), Bluetooth, Bluetooth low power (BLE), Zigbee (Zigbee), near field communication (NFC), or radio frequency (RF).

The driving unit 160 is controlled by the processor 110 and provides a means for the mapping robot 100 to move within the target building. The driving unit 160 may include a motor and a plurality of wheels. For example, the driving unit 160 may include a motor driver, a wheel motor, and a rotating motor. The motor driver may serve to drive a wheel motor for driving of the mapping robot 100. The wheel motor may drive a plurality of wheels for driving the mapping robot 100. The rotating motor may be driven to change or rotate the wheel of the mapping robot 100.

The mapping robot 100 may further include an obstacle recognition unit. The obstacle recognition unit may include at least one of an infrared sensor, an ultrasonic sensor, a cliff sensor, a posture sensor, a collision sensor, and a light flow sensor. The infrared sensor may include a sensor that receives a signal from an infrared remote controller for remotely controlling the mapping robot 100. The ultrasonic sensor may include a sensor for determining the distance between the obstacle and the robot using an ultrasonic signal. The cliff sensor may include a sensor for detecting cliffs or cliffs in the 360 degree range. The attitude sensor (ARS, Attitude Reference System) may include a sensor capable of detecting the attitude of the mapping robot 100. The posture sensor may include a sensor composed of three axes and three axes of acceleration for detecting the amount of rotation of the mapping robot 100. The collision sensor may include a sensor that detects a collision between the mapping robot 100 and an obstacle. The collision sensor may detect a collision between the mapping robot 100 and an obstacle in a 360 degree range. The optical flow sensor (OFS, Optical Flow Sensor) may include a sensor that can measure the phenomenon of a wheel when driving the mapping robot 100 and the driving distance of the robot on various floor surfaces.

3 is an exemplary block diagram illustrating a processor of a mapping robot according to an embodiment.

Referring to FIG. 3, the processor 110 may include a time stamping module 111, a Simultaneous Localization and Mapping (SLAM) module 112, a tagging module 113, and a storage module 114.

The time stamping module 111 may collect the first sensor data through the first sensor unit 130 and tag the first time stamp values in the first sensor data. The time stamping module 111 may collect second sensor data through the second sensor unit 140 and tag the second time stamp values in the second sensor data. It is assumed that the first sensor unit 130 is a 3D laser scanner, and the second sensor unit 140 is a Wi-Fi module. The first sensor data may be referred to as spatial data received from a 3D laser scanner illustrated by the first sensor unit 130, and the second sensor data may be received from a Wi-Fi module illustrated by the second sensor unit 140. It may be referred to as sensor data.

Timestamping may mean tagging the data by storing a timestamp value corresponding to the time at which the data was received in association with the data or by storing it as metadata of the data. Timestamp is information expressed in a consistent format to represent a specific time, and can be conveniently used when comparing two or more times or calculating a period. The format of the timestamp does not limit the invention.

The first sensor unit 130 and the second sensor unit 140 may operate independently of each other. Therefore, the collection period of the first sensor data of the first sensor unit 130 (eg, a 3D laser scanner) and the collection period of the second sensor data of the second sensor unit 140 (eg, a Wi-Fi module) are independent of each other. to be. The reception time of the first sensor data and the reception time of the second sensor data may not be synchronized with each other. For example, the first sensor data may be received at a time interval selected between approximately 1 ms and 100 ms, and the second sensor data may be received at a time interval selected between approximately 1 ms and 1 sec. Each of the first and second sensor data may not be received at a certain period.

The time stamping module 111 may store the first sensor data tagged with the first time stamp value and the second sensor data tagged with the second time stamp value in the memory 120. Each of the first and second sensor data may be understood as a set of data sequentially received while the mapping robot 100 moves.

The SLAM module 112 may perform a simultaneous location estimation and map creation function based on the first sensor data, and estimate a path that the mapping robot 100 has moved. Since the first timestamp values are tagged in the first sensor data, the SLAM module 112 can grasp information about at what time and at what location. Since the simultaneous location estimation and mapping function of the SLAM module 112 is known, it is not described in detail how the simultaneous location estimation and mapping function can be performed.

According to an example, the SLAM module 112 may generate temporary location data at a time point corresponding to the first timestamp value while generating temporary map data based on the first sensor data. The temporary location data refers to data that temporarily estimates a location where the mapping robot 100 passes at a time point corresponding to the first timestamp value. When the first sensor data is 3D image data received from a 3D laser scanner, the SLAM module 112 may perform simultaneous location estimation and map creation functions based on sequentially received 3D image data. In this step, the map data temporarily created by the SLAM module 112 and its location data may include a sensing error, and the sensing error may accumulate and increase over time. The temporary map data and the temporary location data may refer to data in which the sensing error is not removed. For example, temporary map data and temporary location data may include a sensing error.

The SLAM module 112 may perform a loop closing function as a method for removing sensing errors. SLAM module 112 may detect a loop closure. In this specification, the loop closure means re-visiting the location where the mapping robot 100 has passed. The SLAM module 112 compares the current data, which is the first sensor data, which is currently received, with the previous data, which is the first sensor data, which was previously received, and determines that a loop closure has occurred when the similarity between them is greater than a preset threshold. You can. When the first sensor unit 130 is a 3D laser scanner, 3D image data acquired at the same location may be very similar. The 3D image data received from the 3D laser scanner may be converted into a preset coordinate system according to the posture of the mapping robot 100 so as to be easily compared with each other. Re-visiting the location where the mapping robot 100 has passed may include not only revisiting the same coordinates, but also re-visiting a location adjacent to the previously visited location.

The current data is not compared with all of the previous data, but can be compared with some of the previous data to reduce the amount of computation. Some of the previous data to be compared may be previous data at a starting point or previous data at an intersection location. Some of the previous data to be compared may be previous data at a time when a loop closure may occur based on a predicted movement path based on the temporary location data.

According to another embodiment, the SLAM module 112 may also use second sensor data when detecting a loop closure. The SLAM module 112 compares current data, which is currently received second sensor data, with previous data, which is previously received second sensor data, and determines that a loop closure has occurred when the similarity between them is greater than a preset threshold. You can. When the second sensor unit 140 is a Wi-Fi module, wireless signal data acquired at the same location may be very similar. Also for the second sensor data, the current data is not compared with all of the previous data, but can be compared with some of the previous data to reduce the computation amount.

When the loop closure is detected based on the first sensor data, the SLAM module 112 may use the first timestamp value tagged to the first sensor data to determine a previous passing time that has passed the current position. The previous passing time point identified using the first sensor data is referred to as the first passing time point. The SLAM module 112 may grasp the time point of the previous passage that has passed the current position using the second timestamp value tagged to the second sensor data even when the loop closure is detected based on the second sensor data. . The previous passing time point identified using the second sensor data is referred to as the second previous passing time point. The SLMA module 112 may determine that a loop closure has finally occurred when a difference between a first previous passing time and a second previous passing time is less than a preset threshold.

The SLAM module 112 may generate the indoor map information of the target building and the movement path information that the mapping robot 100 has moved by removing the sensing error from the temporary map data and the temporary location data in response to the detection of the loop closure. The SLAM module 112 may correct the temporary position data in response to the detection of the loop closure to generate the corrected position data, and generate movement path information based on the corrected position data. In addition, when the loop closure is detected, the SLAM module 112 may generate indoor map information of the target building by removing the sensing error from the temporary map data.

According to an example, the difference between the temporary position data at the current time point at which the loop closure was detected and the temporary position data at the first previous passing time point corresponds to the accumulated sensing error between the first previous passing time point and the current time point, and thus the SLAM module ( 112) coincides with the temporary position data of the current time point at which the loop closure was detected, and the temporary position data of the current time point at which the loop closure was detected, and the temporary position data of the first time point before the passage closure, Based on the difference, information about the sensing error can be grasped. The SLAM module 112 may generate the corrected position data by correcting the sensing error in the temporary position data. It may be understood that the correction position data has the sensing error removed from the temporary position data. The corrected position data about the position of the mapping robot 100 also includes an estimation error, but a much smaller error than the temporary position data.

The SLAM module 112 may generate movement path information based on the correction position data. Since the first sensor data is time-stamped with the first timestamp value, the movement path information includes correction position data at a time point corresponding to the first timestamp value. That is, the movement path information is information about the path that the mapping robot 100 has moved, and may include information about the time and location that actually passed due to the first sensor data tagged with the first timestamp value.

The tagging module 113 may estimate a sensing position at a time point corresponding to the second timestamp value based on the movement path information. Since the movement path information includes information on the time and location that the mapping robot 100 has moved, the tagging module 113 can grasp the location where the mapping robot 100 was located at the time corresponding to the second timestamp value. have. This location may be referred to as a sensing location. According to an example, the tagging module 113 may estimate a sensing position at a time point corresponding to the second timestamp value by interpolating using correction position data at a time point corresponding to the first timestamp value. The time point corresponding to the first timestamp value may not exactly match the time point corresponding to the second timestamp value. Two first timestamp values adjacent to the second timestamp value may be selected. Correction position data corresponding to the two first timestamp values may be determined. Interpolation based on the difference between the second timestamp value and the adjacent two first timestamp values, assuming that the mapping robot 100 performs linear constant velocity motion between time points corresponding to the two first timestamp values. By performing the calculation, it is possible to estimate the sensing position at the time point corresponding to the second timestamp value.

It may not be assumed that the mapping robot 100 performs linear constant velocity motion between positions corresponding to two adjacent first timestamp values. For example, the mapping robot 100 may move irregularly. In this case, a position of a time point corresponding to the second timestamp value may be determined based on positions of a time point corresponding to two or more first timestamp values adjacent to the second timestamp value. The tagging module 133 predicts the type of motion of the mapping robot 100 based on the positions of the time points corresponding to the two or more first time stamp values, and the type of the motion and the time points corresponding to the second time stamp values. Based on the positions, a position of a time point corresponding to the second timestamp value can be estimated.

The tagging module 113 may estimate the sensing position based on the position where the mapping robot 100 was located at a time point corresponding to the second timestamp value. The position of the time point that corresponds to the second timestamp value strictly refers to the position of the first sensor unit 130. On the other hand, the sensing position may mean the position of the second sensor unit 140.

The first sensor unit 130 and the second sensor unit 140 may exist at different locations within the mapping robot 100. The processor 110 may store information regarding a positional relationship between the first sensor unit 130 and the second sensor unit 140 in advance. The processor 110 calculates the relationship of the position of the second sensor unit 140 with respect to the position of the first sensor unit 130 according to the posture of the mapping robot 100, and uses the calculated positional relationship to the first sensor The position of the second sensor unit 140 may be accurately calculated from the position of the unit 130. Through this, the tagging module 133 may estimate the sensing position based on the position where the first sensor unit 130 was located at a time corresponding to the second timestamp value.

The tagging module 113 may tag the estimated sensing position in the second sensor data whose timestamp value is timestamped. Since the tagging module 113 performs the above operation on all of the second sensor data, all of the second sensor data may be tagged with a sensing position. When the second sensor unit 140 is a camera, the second sensor data tagged with the sensing location may include an image captured by the camera and a location of the camera. The position of the camera included in the second sensor data may include a photographing direction of the camera. In the case of a fixed camera, the photographing direction of the camera may be determined through the posture of the mapping robot 100. In the case of a gimbal camera, the photographing direction of the camera may be determined based on the posture and the gimbal angle of the mapping robot 100.

The storage module 114 may store second sensor data tagged with a sensing location. The storage module 114 may store the second sensor data tagged with the sensing location in the memory 120. The storage module 114 may store the second sensor data tagged with the sensing location in the map cloud server 200. The storage module 114 may combine the second sensor data tagged with the sensing location into the indoor map information of the target building.

When the second sensor unit 140 includes a plurality of types of sensors, a plurality of second sensor data may also exist, and a plurality of second sensor data, each of which is sensed, may be generated by the tagging module 113. You can. The indoor map information of the target building may include a plurality of second sensor data. For example, when the second sensor unit 140 includes a camera and a Wi-Fi module, the second sensor data may include image data captured by the camera at the sensing location and data of Wi-Fi signals received at the sensing location.

The processor 110 may further include an autonomous driving module for autonomous driving of the mapping robot 100, and the driving unit 160 may be driven by the control of the autonomous driving module. The autonomous driving module may directly determine a path to which the mapping robot 100 moves. The mapping robot 100 may search for the target building before the indoor map information of the target building is created. The autonomous driving module may have an algorithm for searching a target building. For example, the autonomous driving module has an algorithm that rotates in a specific direction at all intersections to search for a target building, or an algorithm that determines a path so that a loop closure occurs when a preset travel distance or time is reached. Can have

According to another embodiment, the function of determining the path to which the mapping robot 100 moves may be performed by the map cloud server 200, and the processor 110 may receive a path to which the map cloud server 200 moves. You can.

4 and 5 are reference diagrams for explaining the operation of the processor of the mapping robot according to an embodiment.

4 and 5, the temporary movement path P1 before the loop closing operation of the SLAM module 112 and the estimated movement path P2 estimated after the loop closing operation are illustrated. It is assumed that the mapping robot 100 moves the rectangular path counterclockwise starting from the lower left. 5 is an enlarged view of part A in the reference diagram of FIG. 4.

The temporary movement path P1 may be determined based on the temporary location data ei. As can be seen from the temporary movement path P1, the temporary location data ei may gradually deviate from the actual path as the sensing error accumulates as the mapping robot 100 moves.

The loop closing operation may be an operation of matching the temporary position data ei at two time points when the loop closure is detected. It may be understood that the correction position data ri generated by the loop closing operation removes the sensing error from the temporary position data ei.

When the processor 110 determines the sensing position each time the second sensor data is received from the second sensor unit 140 and tags the determined sensing position to the second sensor data, the sensing position of the second sensor data (mj ) May be determined as a location on the temporary movement path P1 as illustrated in FIG. 5, or may be determined as a location of adjacent temporary location data ei on the temporary movement path P1. The processor 110 may determine the current location as the temporary location data ei based on the first sensor data, and the processor 110 may determine the sensing location of the second sensor data based on the temporary location data ei. You can.

According to the present embodiments, the second sensor data is time-stamped with a second timestamp value. The processor 110 may determine the sensing position of the second sensor data as a position on the estimated movement path P2 using the second timestamp value of the second sensor data. As a result, the sensing position of the second sensor data can be more accurate, and there is little time to be additionally administered to determine the sensing position of the second sensor data. In addition, a very large number of second sensor data may be acquired along the path that the mapping robot 100 has moved in the target building. Second sensor data that is measured at a number of sensing locations within the target building can be collected. In the related art, since the operator had to move the sensor device to a preset position and collect sensor data, the sensor data could be collected only for a limited position, but according to the present embodiments, the mapping robot 100 moves within the target building. The second sensor data may be collected at intervals of several cm on the path.

6 is an exemplary block diagram illustrating an internal configuration of a map cloud server according to an embodiment.

Referring to FIG. 6, the map cloud server 200 includes a processor 210, a memory 220, and a communication unit 230.

The processor 110 operates according to codes stored in the memory 120 and includes a memory 220 and a communication unit 230. The memory 220 may store code for operating the processor 210 and may store data generated by the operation of the processor 210. The memory 220 may include a storage device that stores indoor map information of the target building. According to another example, the memory 220 may include a database of a separate storage server connected through a network, and indoor map information of the target building may be stored in the database.

The processor 210 may receive indoor map information of the target building from the mapping robot 100. The indoor map information may include a 3D map of the target building. The indoor map information may include second sensor data tagged with a sensing location. The sensing location may be specified as a location on the indoor map information of the target building.

The processor 210 may provide indoor map information of the target building to the service robot 300. The service robot 300 may grasp its own position using the second sensor data included in the indoor map information of the target building. The second sensor data is tagged with an accurately estimated sensing position. The service robot 300 may estimate its own position by comparing values measured using a sensor in the target building with the second sensor data.

According to another embodiment, the processor 210 may receive departure information and destination information from the mapping robot 100 and / or the service robot 300. The processor 210 may use the indoor map information of the target building to determine the optimal route to move from the origin to the destination, and provide information regarding the optimal route to the mapping robot 100 and / or the service robot 300 have. The mapping robot 100 and / or the service robot 300 receiving information regarding the optimal path may move according to the optimal path. According to another example, the mapping robot 100 may include an autonomous driving module and may move according to a path determined by the autonomous driving module.

According to another embodiment, the processor 210 may perform the functions of the SLAM module 112, the tagging module 113, and the storage module 114 of FIG. 3. For example, the processor 210 may receive the first and second sensor data from which the first and second timestamp values are time stamped, respectively, from the mapping robot 100.

The processor 210 may generate temporary map data based on the first sensor data whose timestamp value is time stamped, and may generate temporary location data at a time point corresponding to the first timestamp value.

The processor 210 may detect a loop closure that visits the location where the mapping robot 100 has passed again. When the loop closure is detected, the processor 210 may correct the temporary location data to generate correction location data, and may generate movement path information and indoor map information of the target building based on the correction location data.

The processor 210 may estimate a sensing position at a time point corresponding to a second timestamp value based on the movement path information, and tag the sensing position in the second sensor data. The processor 210 may store the second sensor data tagged with the sensing location.

The communication unit 230 may be connected to a network to communicate with the mapping robot 100 and / or the service robot 300. The communication unit 230 may be referred to as a communication interface, and may establish communication with the mapping robot 100 and / or the service robot 300. The communication unit 230 may access the network through wired communication. The wired communication may include at least one of a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), and an integrated service digital network (ISDN).

7 is an exemplary block diagram illustrating an internal configuration of a service robot according to an embodiment.

Referring to FIG. 7, the service robot 300 includes a processor 310, a memory 320, a second sensor unit 330, a communication unit 340, and a driving unit 350.

The processor 310 operates according to codes stored in the memory 320 and controls overall operations of the memory 320, the second sensor unit 330, the communication unit 340, and the driving unit 350. The memory 320 may store code for operating the processor 310 and may store data generated by the operation of the processor 310. The processor 310 may estimate its own position in the target building using sensor data received from the second sensor unit 330 and indoor map information of the target building stored in the map cloud server 200. The processor 310 will be described in more detail below.

The second sensor unit 330 may include a sensor that generates sensor data of the same type as the second sensor data included in the indoor map information of the target building. The second sensor unit 330 may correspond to the second sensor unit 140 of the mapping robot 100. For example, when the second sensor unit 140 of the mapping robot 100 includes at least one selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor, the second sensor unit 330 also includes a second At least one sensor of the same type as the sensor unit 140 may be included. When the second sensor unit 140 of the mapping robot 100 includes the camera and the Wi-Fi module, the indoor map information of the target building includes the image data of the camera and the wireless signal data of the Wi-Fi module, the second sensor unit ( 330) may include only a camera or a Wi-Fi module, or both a camera and a Wi-Fi module.

The sensor of the first sensor unit 130 of the mapping robot 100 may be more expensive and have higher performance than the sensor of the second sensor unit 140. Since the service robot 300 does not include an expensive sensor, such as the first sensor unit 130 of the mapping robot 100, it can be manufactured inexpensively compared to the mapping robot 100, and various according to each function. Can be produced in kind.

The communication unit 340 may be connected to a network to communicate with the mapping robot 100, the map cloud server 200, and / or other service robots 300. The communication unit 340 may be referred to as a communication interface supporting wireless communication. The wireless communication may include at least one of cellular communication, wireless fidelity (WiFi), Bluetooth, Bluetooth low power (BLE), Zigbee, near field communication (NFC), or radio frequency (RF).

The driving unit 350 is controlled by the processor 310 and provides a means for the service robot 300 to move. The driving unit 350 may include a motor and a plurality of wheels. For example, the driving unit 350 may include a motor driver, a wheel motor, and a rotating motor. The motor driver may serve to drive a wheel motor for driving the service robot 300. The wheel motor may drive a plurality of wheels for driving the service robot 300. The rotating motor may be driven to change or rotate the wheel of the service robot 300.

According to an embodiment, the processor 310 may receive sensor data from the second sensor unit 330. The processor 310 may receive indoor map information of the target building from the map cloud server 200. The processor 310 may compare the sensor data received from the second sensor unit 330 and the second sensor data tagged with the sensing location included in the indoor map information of the target building to determine similarity between them. The processor 310 may extract the second sensor data having a high degree of similarity and estimate the current location of the service robot 300 based on the sensing location tagged to the second sensor data.

According to another embodiment, when receiving the sensor data from the second sensor unit 330, the processor 310 may transmit the sensor data to the map cloud server 200. The map cloud server 200 may estimate the current location of the service robot 300 based on sensor data received from the service robot 300 and may transmit information about the current location to the service robot 300.

When a path to which the mapping robot 100 and the service robot 300 move is designated in the target building, the mapping robot 100 may receive second sensor data at all positions on the path. The indoor map information of the target building may include second sensor data acquired at all locations on the route, and the second sensor data may be tagged with an accurate sensing location. When the service robot 300 is located on a designated path in the target building, there is a high possibility that the second sensor data having a very similar degree to the sensor data received through the second sensor unit 330 exists. Therefore, the current position of the service robot 300 can be estimated very accurately.

The embodiment according to the present invention described above may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program can be recorded on a computer-readable medium. In this case, the medium may be continuously stored in a program executable by a computer or may be stored for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or several hardware combinations, and is not limited to a medium directly connected to a computer system, but may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks, And program instructions including ROM, RAM, flash memory, and the like. In addition, examples of other media include an application store for distributing applications, a site for distributing or distributing various software, and a recording medium or storage medium managed by a server.

In the above, the present invention has been described by specific matters such as specific components, etc. and limited examples and drawings, which are provided to help the overall understanding of the present invention, but the present invention is not limited to the above embodiments, and Those skilled in the art to which the invention pertains may seek various modifications and changes from these descriptions.

Accordingly, the spirit of the present invention is not limited to the above-described embodiments, and should not be determined, and the scope of the spirit of the present invention as well as the claims to be described later, as well as all ranges that are equivalent to or equivalently changed from the claims Would belong to

Claims (13)

A data collection device including first and second sensors collects first and second sensor data through the first and second sensors, respectively, while moving within a target area, and provides first and second sensor data to the first and second sensor data. A timestamp step of tagging the first and second timestamp values, respectively;
A data generation step of generating temporary map data of the target area and temporary location data at a time corresponding to the first timestamp value based on the first sensor data collected while the data collection device is moving;
A detection step of detecting a loop closure re-visiting a location where the data collection device has passed;
A loop closing step in response to the detection of the loop closure, correcting the temporary position data to generate corrected position data, and generating movement route information and map information of the target area based on the corrected position data;
A tagging step of estimating a sensing position at a time point corresponding to the second timestamp value based on the movement path information, and tagging the sensing position in the second sensor data; And
And a storage step of storing the second sensor data tagged with the sensing location. According to claim 1,
The movement path information includes correction position data at a time point corresponding to the first timestamp value,
A data collection method characterized in that a sensing position at a time point corresponding to the second timestamp value is estimated through interpolation using correction position data at a time point corresponding to the first timestamp value. According to claim 1,
The step of detecting the loop closure is
Calculating the similarity between the first sensor data;
Calculating similarity between the second sensor data; And
And detecting the loop closure based on the similarity between the first sensor data and the similarity between the second sensor data. According to claim 1,
The first sensor is a laser scanner, a camera, a data collection method comprising at least one selected from the group comprising an RGBD sensor. According to claim 1,
The second sensor includes a camera, a Wi-Fi module, a Bluetooth module, a geomagnetic sensor, a laser scanner, an RGBD sensor, and a data collection method comprising at least one selected from the group. The method of claim 5,
The first sensor is a 3D laser scanner data collection method. According to claim 1,
The correction position and the sensing position is a data collection method characterized in that it comprises direction information. According to claim 1,
The time stamping step, the data generation step, the detection step, the loop closing step, the tagging step, and the storage step is a data collection method characterized in that performed by a processor of the data collection device. According to claim 1,
The time stamping step is performed by a processor of the data collection device,
The data generation step, the detection step, the loop closing step, the tagging step, and the storage step is a data collection method characterized in that performed by a server computing device to access the data collection device via a network. A computer program stored in a medium for executing the method of any one of claims 1 to 9 using a computer. A first sensor;
A second sensor;
A time stamping module for collecting first and second sensor data through the first and second sensors, respectively, and tagging the first and second sensor data with first and second timestamp values, respectively;
Based on the first sensor data, temporary map data of a target area is generated, temporary location data at a time point corresponding to the first timestamp value, and a loop closure revisiting a previously passed location. Detecting, and correcting the temporary position data in response to the detection of the loop closure to generate corrected position data and generating motion path information and map information of a target area based on the corrected position data SLAM (Simultaneous Localization and Mapping) ) module;
A tagging module for estimating a sensing position at a time point corresponding to the second timestamp value based on the movement path information, and tagging the sensing position in the second sensor data; And
And a storage module for storing the second sensor data tagged with the sensing location. The method of claim 11,
The first sensor is a laser scanner, a camera, a data collection system comprising at least one selected from the group comprising an RGBD sensor. The method of claim 11,
The first sensor is a three-dimensional laser scanner,
The second sensor is a data collection system including at least one selected from a group including a camera, a Wi-Fi module, a Bluetooth module, and a geomagnetic sensor.

Images