WO2023043092A1

WO2023043092A1 - Computing device for estimating global position by using external positioning device, and method therefor

Info

Publication number: WO2023043092A1
Application number: PCT/KR2022/012968
Authority: WO
Inventors: 연수용; 유수현; 원지혜; 이동환
Original assignee: 네이버랩스 주식회사
Priority date: 2021-09-17
Filing date: 2022-08-30
Publication date: 2023-03-23
Also published as: KR20230041461A

Abstract

Various embodiments may provide a computing device for estimating a global position by using an external positioning device, and a method therefor. The computing device according to various embodiments can detect global time information and global position information, identify an artificial marker that indicates the global position information, while traveling adjacently to a positioning device including the artificial marker, confirm global position information and global time information about the positioning device on the basis of the artificial marker, determine relative position information between the positioning device and the computing device on the basis of the artificial marker, and determine global position information and global time information about the computing device on the basis of the relative position information and the global position information.

Description

Computing device and method for estimating global position using an external positioning device

Various embodiments relate to a computing device and method for estimating a global position using an external positioning device.

The present invention is based on the research results of the ICT Convergence Industry Innovation Technology Development Project supported by the Institute of Information and Communication Planning and Evaluation (IITP) with financial resources from the government (Ministry of Science and ICT) in 2021 (Task ID: 1711125948 / Task No. 2019- 0-01309-003).

In general, there is a technology in which a computing device such as a mobile robot or an autonomous vehicle moves around an unknown environment and creates a map of the environment using only a sensor attached to the computing device. A computing device creates a map only in a local coordinate system based on its own location, not in a global coordinate system. In order to create a map in the global coordinate system using such a computing device, an expensive positioning module such as a global navigation satellite system (GNSS) module must be mounted on the computing device. However, due to large volume, heavy weight, complex structure, high price, etc., there are limitations in mounting an expensive positioning module such as a GNSS module in a computing device.

Various embodiments provide a computing device and method for estimating a global position using an external positioning device.

Various embodiments provide a computing device and method capable of generating a map in a global coordinate system without including an expensive positioning module such as a GNSS module.

A method of a computing device according to various embodiments includes detecting global time information and global location information, and identifying an artificial landmark while driving adjacent to a positioning device including an artificial landmark representing the global position information; determining the global location information and the global time information of the positioning device based on the artificial landmark; determining relative location information between the positioning device and the computing device based on the artificial landmark; The method may include determining global location information and global time information of the computing device based on the relative location information and the global location information.

A computing device according to various embodiments includes a memory and a processor connected to the memory and configured to execute at least one instruction stored in the memory, wherein the processor detects global time information and global location information; , driving adjacent to a positioning device including an artificial landmark representing the global location information, identifying the artificial landmark, and checking the global location information and global time information of the positioning device based on the artificial landmark; , determine relative location information of the positioning device and the computing device based on the artificial landmark, and determine global location information and global time information of the computing device based on the relative location information and the global location information. can be configured.

According to various embodiments, a computing device may estimate its global location using an external location device. Through this, the computing device may generate a map in the global coordinate system based on its global location. That is, the computing device can create a map in the global coordinate system without including an expensive positioning module such as a GNSS module. In this case, if the computing device is implemented to generate a map in the local coordinate system, the computing device may generate an optimized map by combining the local coordinate system and the global coordinate system through optimization. Accordingly, the cost required to manufacture the computing device can be reduced, and furthermore, the cost required to implement a computer system for generating a map in the global coordinate system can be reduced.

1 is a diagram illustrating a configuration of a computing system according to various embodiments.

2 is a diagram illustrating signal flow in a computing system according to various embodiments.

3 and 4 are diagrams for describing operational characteristics of a computing device according to various embodiments.

5 is a diagram illustrating an internal configuration of a positioning device according to various embodiments.

6 is a diagram illustrating an operation procedure of a positioning device according to an embodiment.

7 is a diagram illustrating an operation procedure of a positioning device according to another embodiment.

8 is a diagram illustrating an internal configuration of a computing device according to various embodiments.

FIG. 9 is a diagram showing a detailed configuration of the processor of FIG. 8 .

10 is a diagram illustrating an operating procedure of a computing device according to an exemplary embodiment.

11 is a diagram illustrating an operating procedure of a computing device according to another embodiment.

Hereinafter, specific details for the implementation of the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, if there is a risk of unnecessarily obscuring the gist of the present disclosure, detailed descriptions of well-known functions or configurations will be omitted.

In the accompanying drawings, identical or corresponding elements are given the same reference numerals. In addition, in the description of the following embodiments, overlapping descriptions of the same or corresponding components may be omitted. However, omission of a description of a component does not intend that such a component is not included in an embodiment.

Advantages and features of the disclosed embodiments, and methods of achieving them, will become clear with reference to the following embodiments in conjunction with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below and may be implemented in various different forms, but only the present embodiments make the present disclosure complete, and the present disclosure does not extend the scope of the invention to those skilled in the art. It is provided only for complete information.

Terms used in this disclosure will be briefly described, and the disclosed embodiments will be described in detail. The terminology used in the present disclosure has been selected from general terms that are currently widely used as much as possible while considering functions in the present disclosure, but they may vary depending on the intention or precedent of a person skilled in the related field, the emergence of new technologies, and the like. In addition, in a specific case, there is also a term arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present disclosure should be defined based on the meaning of the terms and the general content of the present disclosure, not simply the names of the terms.

Expressions in the singular number in this disclosure include plural expressions unless the context clearly dictates that they are singular. Also, plural expressions include singular expressions unless the context clearly specifies that they are plural. When it is said that a certain part includes a certain component in the entire specification, this means that it may further include other components without excluding other components unless otherwise stated.

Also, the terms 'module' or 'unit' used in the present disclosure mean a software or hardware component, and the 'module' or 'unit' performs certain roles. However, 'module' or 'unit' is not meant to be limited to software or hardware. A 'module' or 'unit' may be configured to reside in an addressable storage medium and may be configured to reproduce one or more processors. Thus, as an example, a 'module' or 'unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, and attributes. , procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, or variables. Functions provided within components and 'modules' or 'units' may be combined into a smaller number of components and 'modules' or 'units', or further components and 'modules' or 'units'. can be further separated.

According to the present disclosure, a 'module' or 'unit' may be implemented with a processor and a memory. 'Processor' should be interpreted broadly to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, and the like. In some circumstances, 'processor' may refer to an application specific integrated circuit (ASIC), programmable logic device (PLD), field programmable gate array (FPGA), or the like. 'Processor' refers to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of a plurality of microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or a combination of any other such configurations. You may. Also, 'memory' should be interpreted broadly to include any electronic component capable of storing electronic information. 'Memory' includes random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), erasable-programmable read-only memory (EPROM), It may also refer to various types of processor-readable media, such as electrically erasable PROM (EEPROM), flash memory, magnetic or optical data storage, registers, and the like. A memory is said to be in electronic communication with the processor if the processor can read information from and/or write information to the memory. Memory integrated with the processor is in electronic communication with the processor.

Hereinafter, the term 'position' may also be referred to as the term 'pose', and rotation (yaw) and tilt (pitch, roll) representing coordinates or orientation in a 3D space ) may include at least one of For example, a position or posture may be expressed in a maximum of 6 degrees of freedom, or may be expressed in some of the 6 degrees of freedom.

1 is a diagram illustrating a configuration of a computing system 100 according to various embodiments.

Referring to FIG. 1 , the computer system 100 includes a positioning device 110 and a computing device 120, and the computing device 120 determines the global location of the computing device 120 using the positioning device 110. It can be configured to estimate.

The positioning device 110 may have a function of detecting its global location information and global time information. For example, the positioning device 110 may receive GNS information from a satellite and detect global location information and global time information from the GNS information. The positioning device 110 may be fixedly installed at a predetermined location or movably implemented. Also, the positioning device 110 may include an artificial mark (eg, the artificial mark 310 of FIG. 3 ). The artificial indicia may represent global location information of the location device 110 . For example, the artificial marker may include a specific pattern for expressing identification information about the positioning device 110 . The artificial marker may include, for example, at least one of a fiducial marker, a visual marker, or a signage.

The computing device 120 may have a function of determining its own global location information and global time information using the positioning device 110 . Specifically, the computing device 120 may identify an artificial landmark of the positioning device 110 while driving in an outdoor environment or an indoor environment. Also, the computing device 120 may check global position information of the positioning device 110 and determine relative position information with the positioning device 110 based on the artificial mark. Through this, the computing device 120 may determine its own global location information based on the global location information of the positioning device 110 and relative location information with the positioning device 110 . Also, the computing device 120 may determine its own global time information corresponding to the global time information of the positioning device 110 through temporal pairing with the positioning device 110 . Accordingly, the computing device 120 may generate a map in a global coordinate system (which may also be referred to as a world coordinate system) using its global location information and global time information. For example, the computing device 120 may be an object that is a target of a specific function or action in a 3D space, such as augmented reality (AR), robotics, or autonomous driving, but is not limited thereto. The computing device 120 may include, for example, at least one of a mobile robot or vehicle capable of autonomous driving, or a portable computer or a wearable computer providing augmented reality.

2 is a diagram illustrating signal flow in a computing system 100 according to various embodiments. 3 and 4 are diagrams for describing operational characteristics of the computing device 120 according to various embodiments.

Referring to FIG. 2 , the positioning device 110 may include an artificial mark 310 in step 210 . For example, the artificial marker 310 may include a specific pattern for expressing identification information about the positioning device 110 . For example, as shown in FIG. 3 , a pattern may be defined by the shape, size, and arrangement of at least one figure. As another example, although not shown, the pattern may be defined as text consisting of at least one character. Also, the positioning device 110 may detect its own global time information and global location information in step 220 . For example, the positioning device 110 may receive GNS information from a satellite and detect global location information and global time information from the GNS information. Here, the positioning device 110 may periodically receive GNS information, thereby periodically detecting global location information and global time information.

The computing device 120 may identify the artificial mark 310 of the positioning device 110 while driving in step 230 . Specifically, the computing device 120 may photograph the artificial landmark 310 , obtain an image having the artificial landmark 310 , analyze the image, and extract identification information about the positioning device 110 . At this time, the computing device 120 may perform temporal pairing with the positioning device 110 . According to an embodiment, the computing device 120 may receive global time information of the positioning device 110 from the positioning device 110 in step 240 . According to another embodiment, the computing device 120 may check the time at which the image was obtained based on a predetermined global standard time corresponding to the global time information of the positioning device 110 . According to another embodiment, the computing device 120 may acquire an image in a stationary state exceeding a predetermined time interval and check the time at which the image was acquired.

According to one embodiment, the computing device 120 may extract identification information from the image of the artificial landmark 310 in a manner as shown in FIG. 3 . At this time, the computing device 120 may extract feature points of the artificial landmark 310 using a predetermined algorithm. Specifically, the computing device 120 may map the image of the artificial landmark 310 to the virtual plane 320 and then extract feature points of the artificial landmark 310 . Here, the virtual plane 320 is defined for the computing device 120 and may represent a rectangular plane determined by a predetermined format/standard for the artificial landmark 310 . For example, the feature points may include corner points, and the corner points may represent vertices of a figure included in a pattern of the artificial landmark 310 on the virtual plane 320 . Also, the computing device 120 may extract coordinates of feature points on the virtual plane 320 . Through this, the computing device 120 may recognize a pattern from feature points of the artificial landmark 310 and may extract identification information corresponding to the pattern.

Next, the computing device 120 may check global time information and global location information of the positioning device 110 based on the artificial mark 310 in step 250 . At this time, the computing device 120 uses the identification information of the positioning device 110 to obtain global location information of the positioning device 110 from a database previously stored therein or a database stored in an external server (not shown). can be checked. Also, the computing device 120 may check global time information of the positioning device 110 in correspondence with the global location information of the positioning device 110 . According to an embodiment, the computing device 120 may check global time information received in step 240 as global time information of the positioning device 110 . According to another embodiment, the computing device 120 may check global time information of the positioning device 110 from a time when an image is obtained based on a predetermined global standard time.

Next, the computing device 120 may determine relative position information with the positioning device 110 in step 260 . The relative position information may include at least one of a relative distance between the positioning device 110 and the computing device 120 or a relative posture of the positioning device 110 with respect to the computing device 120 . At this time, the computing device 120 may determine relative position information using at least one of a camera module and a LiDAR sensor. For example, the computing device 120 may determine relative position information using the camera module 810 . As another example, the computing device 120 may determine the relative location information using the camera module 810 and the LIDAR sensor.

According to one embodiment, computing device 120 may determine the relative pose of artificial landmark 310 to computing device 120 in a manner as shown in FIG. 4 . Specifically, the computing device 120, as shown in (a) of FIG. 4, the artificial landmark 310 for the virtual plane 320 before the image of the artificial landmark 310 is mapped to the virtual plane 320. ) can extract the posture of the image. At this time, the computing device 120 may extract the orientation of each of the X-axis, Y-axis, and Z-axis of the image of the artificial landmark 310 with respect to the virtual plane 320 . Then, the computing device 120 may determine a posture change of the image of the artificial landmark 310 for mapping the image of the artificial landmark 310 to the virtual plane 320, as shown in (b) of FIG. there is. At this time, the computing device 120 may determine the respective movement distances, angles, and the like of the X-axis, Y-axis, and Z-axis of the image of the artificial landmark 310 . Through this, the computing device 120 may determine the relative posture of the artificial landmark 310 from the change in posture of the image of the artificial landmark 310 for mapping the image of the artificial landmark 310 to the virtual plane 320 .

Next, the computing device 120 may determine its own global time information and global location information based on the global time information, global location information, and relative location information of the positioning device 110 in step 270 . At this time, the computing device 120 may determine its own global location information by combining global location information and relative location information of the positioning device 110 . Also, the computing device 120 may determine its own global time information using the global time information of the positioning device 110 . According to an embodiment, the computing device 120 may determine its own global time information to match the global time information of the positioning device 110 . According to another embodiment, the computing device 120 may determine its own global time information by applying interpolation to the global time information of the positioning device 110 .

5 is a diagram illustrating an internal configuration of a positioning device 110 according to various embodiments.

Referring to FIG. 5 , the positioning device 110 may include at least one of a positioning module 510, an artificial landmark 520, a communication module 530, a memory 540, and a processor 550. In some embodiments, at least one of the components of the positioning device 110 (eg, the communication module 530) may be omitted, and at least one other component may be added. In some embodiments, at least any two of the components of the positioning device 110 (eg, the positioning module 510 and the communication module 530) may be implemented as a single integrated circuit.

The positioning module 510 may receive positioning information from an external device in the positioning device 110 . For example, the external device may include a satellite, and positioning information may include GPS information. That is, the positioning module 510 may receive GNS information from GSS satellites. Here, the positioning module 510 may periodically receive GNS information.

The artificial landmark (eg, the artificial landmark 310 of FIGS. 3 and 4 ) 520 may represent global location information of the positioning device 110 . For example, the artificial landmark 520 may include a specific pattern for expressing identification information about the positioning device 110 . For example, a pattern may be defined by the shape, size, arrangement, etc. of at least one figure. As another example, the pattern may be defined as text consisting of at least one character. The artificial marker 520 may include, for example, at least one of a fiducial marker, a visual marker, and signage.

The communication module 530 may perform communication with an external device in the positioning device 110 . For example, the external device may include at least one of the computing device 120 or a server (not shown). The communication module 530 may include at least one of a wired communication module and a wireless communication module. The wired communication module may be connected to an external device through a wired connection terminal (not shown) and communicate with the external device through a wired connection. The wireless communication module may include at least one of a short-distance communication module and a long-distance communication module. The short-distance communication module may communicate with an external device in a short-range communication method. For example, the short-range communication method may include at least one of Bluetooth, WiFi direct, and infrared data association (IrDA). The remote communication module may communicate with an external device through a remote communication method. Here, the remote communication module may communicate with an external device through a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a local area network (LAN) or a wide area network (WAN).

The memory 540 may store various data used by at least one component of the positioning device 110 . For example, the memory 540 may include at least one of volatile memory and non-volatile memory. The data may include at least one program and related input data or output data. The program may be stored as software including at least one instruction in the memory 540 .

The processor 550 may execute a program of the memory 540 to control at least one element of the positioning device 110 . Through this, the processor 550 may perform data processing or calculation. At this time, the processor 550 may execute instructions stored in the memory 540 .

According to various embodiments, the processor 550 may detect global time information and global location information of the positioning device 110 . For example, the processor 550 may detect global location information and global time information from GNS information. Here, as GNS information is periodically received, the processor 550 may periodically detect global location information and global time information.

6 is a diagram illustrating an operation procedure of the positioning device 110 according to an embodiment.

Referring to FIG. 6 , the positioning device 110 may include an artificial mark 520 in step 610 . Artificial landmark 520 may represent global location information of location device 110 . For example, the artificial landmark 520 may include a specific pattern for expressing identification information about the positioning device 110 . For example, a pattern may be defined by the shape, size, arrangement, etc. of at least one figure. As another example, the pattern may be defined as text consisting of at least one character. The artificial marker 520 may include, for example, at least one of a fiducial marker, a visual marker, and signage.

Next, the positioning device 110 may detect its own global time information and global location information in step 620 . The processor 550 may detect global time information and global location information of the positioning device 110 . For example, the processor 550 may detect global location information and global time information from GNS information. Here, as GNS information is periodically received, the processor 550 may periodically detect global location information and global time information.

According to an embodiment, when the measurement device 110 is fixedly installed in a predetermined location, global location information corresponding to the predetermined location will be detected. In this case, when the measuring device 110 is installed, global location information of a predetermined place may be pre-registered in an external server. At this time, corresponding to the identification information of the measurement device 110, global location information of a predetermined place may be registered.

According to another embodiment, if the measurement device 110 is implemented to be movable, global location information of the current location of the measurement device 110 will be detected. In this case, the processor 550 may register global location information in the current location to an external server whenever the location is changed. In this case, the global location of the measurement device 110 may be updated in response to the identification information of the measurement device 110 .

7 is a diagram illustrating an operation procedure of the positioning device 110 according to another embodiment.

Referring to FIG. 7 ,

steps

710 and 720 of this embodiment are similar to

steps

610 and 620 of the above-described embodiment, so detailed descriptions thereof are omitted. However, in this embodiment, the positioning device 110 may transmit global time information in step 740 . For example, the positioning device 110 may periodically emit global time information at the current location according to a short-range communication method. At this time, the processor 550 may emit global time information through the communication module 530 . As another example, the positioning device 110 may receive a time synchronization request from the computing device 120 in step 730 and transmit global time information in step 740 in response thereto. Here, the processor 550 may receive a time synchronization request and emit global time information according to a short-range communication method. At this time, the processor 550 may receive a time synchronization request from the computing device 120 through the communication module 530 and emit global time information at the current location.

8 is a diagram illustrating an internal configuration of a computing device 120 according to various embodiments. FIG. 9 is a diagram showing a detailed configuration of the processor 870 of FIG. 8 .

Referring to FIG. 8 , the computing device 120 includes a camera module 810, a sensor module 820, a communication module 830, an input module 840, an output module 850, a memory 860, or a processor ( 870) may include at least one. In some embodiments, at least one of the components of the computing device 120 may be omitted, and at least one other component may be added. In some embodiments, at least any two of the components of computing device 120 may be implemented as a single integrated circuit.

The camera module 810 may capture an image of the surrounding environment of the computing device 120 . For example, images may include moving images and still images. According to an embodiment, the camera module 810 may include at least one of at least one lens, at least one image sensor, an image signal processor, or a flash.

The sensor module 820 may sense the state of the environment surrounding the computing device 120 and generate an electrical signal or data value corresponding thereto. The sensor module 820 may include, for example, at least one of a lidar sensor, a radar sensor, an infrared (IR) sensor, or a distance sensor. Additionally, the sensor module 820 may detect an operating state of the computing device 120 and generate an electrical signal or data value therefor. In this case, the sensor module 820 may include at least one of a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor. may contain one more.

The communication module 830 may perform communication with an external device in the computing device 120 . For example, the external device may include at least one of a server (not shown) and the positioning device 110 . The communication module 830 may include at least one of a wired communication module and a wireless communication module. The wired communication module may be connected to an external device through a wired connection terminal (not shown) and communicate with the external device through a wired connection. The wireless communication module may include at least one of a short-distance communication module and a long-distance communication module. The short-distance communication module may communicate with an external device in a short-range communication method. For example, the short-distance communication method may include at least one of Bluetooth, Wi-Fi Direct, and infrared communication. The remote communication module may communicate with an external device through a remote communication method. Here, the remote communication module may communicate with an external device through a network. For example, the network may include at least one of a cellular network, the Internet, or a computer network such as a LAN or WAN.

The input module 840 may input a signal to be used in at least one component of the computing device 120 . For example, the input module 840 may include at least one of a microphone, a mouse, and a keyboard. In some embodiments, the input module 840 may include at least one of a touch circuitry set to sense a touch or a sensor circuit set to measure the strength of a force generated by a touch.

The output module 850 may output information from the computing device 120 . In this case, the output module 850 may include at least one of a display module for visually displaying information and an audio module for aurally reproducing information. For example, the display module may include at least one of a display, a hologram device, and a projector. For example, the display module may be implemented as a touch screen by being assembled with at least one of a touch circuit and a sensor circuit of the input module 840 . For example, the audio module may include at least one of a speaker, receiver, earphone, or headphone.

The memory 860 may store various data used by at least one component of the computing device 120 . For example, the memory 860 may include at least one of volatile memory and non-volatile memory. The data may include at least one program and related input data or output data. The program may be stored as software including at least one instruction in memory 860 .

The processor 870 may execute a program of the memory 860 to control at least one component of the computing device 120 . Through this, the processor 870 may perform data processing or calculation. At this time, the processor 870 may execute instructions stored in the memory 860.

According to various embodiments, the processor 870 may have a function of determining its own global location information and global time information using the positioning device 110 . In some embodiments, processor 870 may include artificial landmark identification 971 , relative position determination 973 , absolute position determination 975 , and position optimization 977 as shown in FIG. 9 . can include This is only for explaining the functions of the processor 870 by dividing them, and does not mean that the processors 870 are necessarily physically separated. In other words, at least any two of the components of the processor 870 may be implemented as a single integrated circuit.

The artificial landmark identification unit 971 may identify the artificial landmark 520 of the positioning device 110 while the computing device 120 is traveling in an outdoor environment or an indoor environment. Specifically, the artificial landmark identification unit 971 photographs the artificial landmark 520 through the camera module 810, obtains an image having the artificial landmark 520, analyzes the image, and locates the positioning device 110. Identification information about can be extracted. At this time, the computing device 120 may perform temporal pairing with the positioning device 110 .

Also, the artificial landmark identification unit 971 may check global time information and global location information of the positioning device 110 based on the artificial landmark 520 . At this time, the computing device 120 may check global location information of the positioning device 110 from a database previously stored therein or a database stored in an external server by using the identification information of the positioning device 110 . Also, the computing device 120 may check global time information of the positioning device 110 in correspondence with global location information of the positioning device 110 .

The relative position determiner 973 may determine relative position information with the positioning device 110 based on the artificial mark 520 . The relative position information may include at least one of a relative distance between the positioning device 110 and the computing device 120 or a relative posture of the positioning device 110 with respect to the computing device 120 . At this time, the relative position determiner 973 may determine the relative position information using at least one of the lidar sensor of the camera module 810 or the sensor module 820 . For example, the relative position determiner 973 may determine relative position information using the camera module 810 . As another example, the relative position determiner 973 may determine relative position information using the camera module 810 and the LIDAR sensor.

The absolute location determining unit 975 may determine global location information of the computing device 120 based on global location information of the positioning device 110 and relative location information with the positioning device 110 . At this time, the absolute location determination unit 975 may determine its own global location information by combining global location information and relative location information of the positioning device 110 . Also, the absolute position determiner 975 may determine global time information of the computing device 120 in response to the global time information of the positioning device 110 through temporal pairing with the positioning device 110 .

The location optimizer 977 may optimize the global location of the computing device 120 in the global coordinate system. In this case, the location optimizer 977 may generate a map in the global coordinate system using global location information and global time information of the computing device 120 . When the computing device 120 is implemented to generate a map in the local coordinate system, the location optimizer 977 may generate an optimized map by combining the local coordinate system and the global coordinate system through optimization. Here, when the difference between the local coordinate system and the global coordinate system is large, the position optimization unit 977 may generate an optimized map after performing an initialization step of matching the local coordinate system and the global coordinate system. For example, the location optimizer 977 may generate a map using a simultaneous localization and mapping (SLAM) algorithm.

10 is a diagram illustrating an operating procedure of the computing device 120 according to an exemplary embodiment.

Referring to FIG. 10 , the computing device 120 may identify the artificial mark 310 of the positioning device 110 while driving in step 1010 . Specifically, while the computing device 120 is driving in an outdoor environment or an indoor environment, the processor 870 captures the artificial landmark 520 through the camera module 810 to create an image having the artificial landmark 520. Acquiring and analyzing the image, identification information about the positioning device 110 may be extracted. At this time, the computing device 120 may perform temporal pairing with the positioning device 110 . For example, the processor 870 may determine the time at which the image was obtained based on a predetermined global standard time that corresponds to the global time information of the positioning device 110 . Here, the global standard time may be coordinated universal time (UTC). For example, in a state in which the computing device 120 is stopped to exceed a predetermined time interval, the processor 870 may acquire an image and check the time at which the image was obtained.

According to an embodiment, the processor 870 may extract identification information from images of the artificial landmarks 310 and 520 as described above with reference to FIG. 3 . At this time, the processor 870 may extract feature points of the artificial landmarks 310 and 520 using a predetermined algorithm. Specifically, the processor 870 may map images of the artificial landmarks 310 and 520 onto the virtual plane 320 and then extract feature points of the artificial landmarks 310 . Here, the virtual plane 320 is defined for the computing device 120 and may represent a rectangular plane determined by a predetermined format/standard for the artificial landmarks 310 and 520 . For example, the feature points may include corner points, and the corner points may represent vertices of figures included in patterns of the artificial markers 310 and 520 on the virtual plane 320 . Also, the processor 870 may extract coordinates of feature points on the virtual plane 320 . Through this, the processor 870 may recognize a pattern from feature points of the artificial landmark 310 and may extract identification information corresponding to the pattern.

Next, the computing device 120 may check global time information and global location information of the positioning device 110 based on the artificial mark 520 in step 1020 . At this time, the processor 870 may check global location information of the positioning device 110 from a database previously stored inside or a database stored in an external server by using the identification information of the positioning device 110 . For example, the processor 870 may search a pre-stored database and check global location information of the positioning device 110 stored in correspondence with the identification information on the positioning device 110 . As another example, the processor 870 transmits identification information on the positioning device 110 to an external server through the communication module 830 and, in response, receives global location information of the positioning device 110 from the server. can Also, the processor 870 may check global time information of the positioning device 110 in response to global location information of the positioning device 110 . For example, the processor 870 may check global time information of the positioning device 110 from a time when an image is obtained based on a predetermined global standard time. Here, the global standard time may be Coordinated Universal Time (UTC).

Next, the computing device 120 may determine relative position information with the positioning device 110 in step 1030 . The relative position information may include at least one of a relative distance between the positioning device 110 and the computing device 120 or a relative posture of the positioning device 110 with respect to the computing device 120 . At this time, the processor 870 may determine the relative location information through at least one of the camera module 810 and the sensor module 820 . For example, the processor 870 may determine relative location information using the camera module 810 . As another example, the processor 870 may determine the relative location information using the camera module 810 and the LIDAR sensor. In this case, the processor 870 may additionally use intensity information detected using the LIDAR sensor while determining the relative position information using the camera module 810 .

According to one embodiment, processor 870 may determine a relative pose of artificial landmarks 310 and 520 to computing device 120 as described above with reference to FIG. 4 . Specifically, the processor 870, as shown in (a) of FIG. 4 , before the images of the artificial landmarks 310 and 520 are mapped to the virtual plane 320, the artificial landmark for the virtual plane 320 ( 310 and 520) can extract the posture of the image. At this time, the processor 870 may extract the X-axis, Y-axis, and Z-axis orientations of the images of the artificial landmarks 310 and 520 with respect to the virtual plane 320 . And, as shown in (b) of FIG. 4, the processor 870 changes the posture of the images of the artificial landmarks 310 and 520 so that the images of the artificial landmarks 310 and 520 are mapped to the virtual plane 320. can decide At this time, the processor 870 may determine the respective movement distances, angles, and the like of the X-axis, Y-axis, and Z-axis of the image of the artificial landmark 310 . Through this, the processor 870 determines the relative posture of the artificial landmark 310 or 520 from the change in posture of the image of the artificial landmark 310 or 520 for mapping the image of the artificial landmark 310 or 520 to the virtual plane 320. can decide

Next, the computing device 120 may determine its own global time information and global location information based on the global time information, global location information, and relative location information of the positioning device 110 in step 1040 . At this time, the processor 870 may determine global location information of the computing device 120 by combining global location information and relative location information of the positioning device 110 . Also, the processor 870 may determine global time information of the computing device 120 as global time information of the positioning device 110 . According to an embodiment, the processor 870 may determine global time information of the computing device 120 to match global time information of the positioning device 110 . According to another embodiment, the processor 870 may determine the global time information of the computing device 120 by applying interpolation to the global time information of the positioning device 110 . For example, the processor 870 may determine global location information from global time information of the computing device 120 based on an error model (err _GPS ) defined as in Equation 1 below. .

Here, τ represents global temporal information of the computing device 120, T is represented by a 4×4 transformation matrix,

represents the global location information of the computing device 120,

Represents the relative position information of the positioning device 110, that is, the artificial landmark 520, with respect to the computing device 120, t is expressed as a 3 × 1 translation vector,

represents the distance between the artificial marker 520 and the positioning module 510 in the positioning device 110,

may indicate global location information of the positioning device 110.

Through this, the computing device 120 may optimize the global position of the computing device 120 in the global coordinate system. In this case, the processor 870 may generate a map in the global coordinate system using global location information and global time information of the computing device 120 . When the computing device 120 is implemented to generate a map in the local coordinate system, the processor 870 may generate an optimized map by combining the local coordinate system and the global coordinate system through optimization. For example, the processor 870 adds a cost function based on an error model (err _GPS ) as shown in [Equation 1] to a cost function for generating a map in the local coordinate system, and obtains the following [Equation 2] The same optimized cost function (

), and based on this, an optimized map can be created. Here, when the difference between the local coordinate system and the global coordinate system is large, the processor 870 may generate an optimized map after performing an initialization step of matching the local coordinate system and the global coordinate system. For example, the processor 870 may perform the initialization step by using a closed-form solution.

Here, the first term and the second term are cost functions for generating a map in the local coordinate system. represents a cost function based on an image obtained by , and the third term may represent a cost function based on an error model (err _GPS ) as in [Equation 1] above. In detail,

represents the global location information of the computing device 120,

Represents a measurement value from a lidar sensor or distance sensor, or a camera module 810, etc.

Represents a relative posture between the reference position of the camera module 810 and the computing device 120 in the computing device 120,

represents the covariance,

may represent a projection model for the image. In some embodiments, if another sensor such as an inertial measurement unit (IMU) is further used in addition to a lidar sensor or a distance sensor and the camera module 810 to generate a map in a local coordinate system, the A cost function by another sensor may be added to [Equation 2].

11 is a diagram illustrating an operating procedure of a computing device 120 according to another embodiment.

Referring to FIG. 11 , step 1110 of this embodiment is similar to step 1010 of the above-described embodiment, so detailed description thereof is omitted. However, in this embodiment, the computing device 120 may receive the global time information of the positioning device 110 in step 1117 after identifying the artificial mark 520 of the positioning device 110 in step 1110 . At this time, the processor 870 may receive global time information of the positioning device 110 from the positioning device 110 so that the computing device 120 can perform temporal pairing with the positioning device 110 . For example, as the positioning device 110 periodically emits global time information at the current location according to a short-distance communication method, the processor 870 transmits the global time information of the positioning device 110 through the communication module 830. information can be received. As another example, if the artificial display 520 of the positioning device 110 is identified in step 1110 , the processor 870 may transmit a time synchronization request in step 1113 . Here, the processor 870 may emit a time synchronization request through the communication module 830 according to a short-range communication method. Thereafter, the processor 870 may receive global time information of the positioning device 110 in response to the time synchronization request in step 1117 . Here, the processor 870 may receive global time information of the positioning device 110 through the communication module 830 according to a short-range communication method.

Next, the computing device 120 may check global time information and global location information of the positioning device 110 based on the artificial mark 520 in step 1120 . At this time, the processor 870 may check global location information of the positioning device 110 from a database previously stored inside or a database stored in an external server by using the identification information of the positioning device 110 . For example, the processor 870 may search a pre-stored database and check global location information of the positioning device 110 stored in correspondence with the identification information on the positioning device 110 . As another example, the processor 870 transmits identification information on the positioning device 110 to an external server through the communication module 830 and, in response, receives global location information of the positioning device 110 from the server. can Also, the processor 870 may check global time information of the positioning device 110 in response to global location information of the positioning device 110 . For example, the processor 870 may check global time information received in step 1117 as global time information of the positioning device 110 . Next, the computing device 120 proceeds to step 1130, and

steps

1130 and 1140 of this embodiment are similar to

steps

1030 and 1040 of the above-described embodiment, so detailed descriptions thereof are omitted.

According to various embodiments, the computing device 120 may estimate its own global location using the external positioning device 110 . Through this, the computing device 120 may generate a map in the global coordinate system based on its global location. That is, the computing device 120 may generate a map in the global coordinate system without including an expensive positioning module such as a GNSS module. In this case, if the computing device 120 is implemented to generate a map in the local coordinate system, the computing device 120 may generate an optimized map by combining the local coordinate system and the global coordinate system through optimization. The computing device 120 does not include an expensive positioning module such as a GNSS module, but devices including an inexpensive GPS receiver may correspond to the computing device 120 .

According to various embodiments, a method of computing device 120 detects global time information and global location information, and proximate location device 110 including artificial indicia 310, 520 representing global location information. While driving, identifying the artificial landmarks 310 and 520 (

steps

230, 1010 and 1110), based on the artificial landmarks 310 and 520, global location information and global time information of the positioning device 110 Determining relative position information of the positioning device 110 and the computing device 120 based on the artificial landmarks 310 and 520 (

steps

250, 1020 and 1120) (steps 260 and 1030) , 1130 ), and determining global location information and global time information of the computing device 120 based on the relative location information and the global location information (

operations

270 , 1040 , and 1140 ).

According to various embodiments, the positioning device 110 may be configured to detect global time information and global location information based on GPS information received from GPS satellites.

According to various embodiments, the step of identifying the artificial landmarks 310 and 520 (

steps

230, 1010 and 1110) is to take a picture of the artificial landmarks 310 and 520 to have the artificial landmarks 310 and 520. It may include obtaining an image and extracting identification information about the positioning device 110 by analyzing the image.

According to various embodiments, the

steps

250, 1020, and 1120 of checking the global location information and global time information of the positioning device 110 include the global location of the positioning device 110 using the identification information. It may include checking information, and checking global time information of the positioning device 110 corresponding to the global location information of the positioning device 110 .

According to various embodiments, the checking of the global time information of the positioning device 110 may include checking the global time information received from the positioning device 110 (steps 240 and 1117) or a predetermined global standard time. Based on , checking global time information from a time when an image is acquired may be included.

According to various embodiments, the relative position information may include at least one of a relative distance between the positioning device 110 and the computing device 120 or a relative posture of the positioning device 110 with respect to the computing device 120. can

According to various embodiments, the steps of determining the relative location information (

steps

260, 1030, and 1130) include determining the relative location information using at least one of the camera module 810 or a lidar sensor. can include

According to various embodiments, the

steps

270, 1040, and 1140 of determining global location information and global time information of the computing device 120 may include relative location information and global location information of the positioning device 110. In combination, determining the global location information of the computing device 120, and, in relation to the global location information of the computing device 120, the global time information of the computing device 120 as the global time information of the location device 110. It may include the step of determining.

According to various embodiments, global location information and global time information of the computing device 120 may be used to generate map data in a global coordinate system.

According to various embodiments, the computing device 120 includes a memory 860 and a processor 870 coupled to the memory 860 and configured to execute at least one instruction stored in the memory 860, The processor 870 detects the global time information and the global location information, and the artificial landmarks 310 and 520 while driving adjacent to the positioning device 110 including the artificial landmarks 310 and 520 representing the global position information. and, based on the artificial landmarks 310 and 520, determine global location information and global time information of the positioning device 110, and based on the artificial landmarks 310 and 520, the positioning device 110 and computing Determine relative location information of device 120 and, based on the relative location information and global location information, determine global location information and global time information of computing device 120 .

According to various embodiments, the positioning device 110 includes a positioning module 510 configured to receive GPS information from a satellite, and a processor 550 configured to detect global time information and global location information from the GPS information. ) may further include.

According to various embodiments, the computing device 120 further includes a camera module 810 connected to the processor 870, and the processor 870, via the camera module 810, the artificial landmark 310, 520), obtain images with artificial landmarks 310 and 520, analyze the images, and extract identification information about the positioning device 110.

According to various embodiments, the processor 870 checks global location information of the positioning device 110 using the identification information, and in response to the global location information of the positioning device 110, the positioning device 110 It can be configured to check the global time information of.

According to various embodiments, the computing device 120 further includes a communication module 830 connected to the processor 870, and the processor 870, via the communication module 830, the positioning device 110 Global time information may be received from, or based on a predetermined global standard time, global time information may be checked from a time when an image is acquired.

According to various embodiments, the computing device 120 further includes at least one of a camera module 810 and a lidar sensor connected to the processor 870, and the processor 870 includes the camera module 810 or It may be configured to determine relative location information based on point cloud data detected using at least one of the LIDAR sensors.

According to various embodiments, the processor 870 determines the global location information of the computing device 120 by combining the relative location information and the global location information of the positioning device 110, and determines the global location information of the computing device 120. Regarding location information, the global time information of the positioning device 110 may be configured to determine global time information of the computing device 120 .

The above method may be provided as a computer program stored in a computer readable recording medium to be executed on a computer. The medium may continuously store a computer-executable program or temporarily store it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or combined hardware, but is not limited to a medium directly connected to a certain computer system, and may be distributed on a network. Examples of the medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROM and DVD, magneto-optical media such as floptical disks, and ROM, RAM, flash memory, etc. configured to store program instructions. In addition, examples of other media include recording media or storage media managed by an app store that distributes applications, a site that supplies or distributes various other software, and a server.

The methods, acts or techniques of this disclosure may be implemented by various means. For example, these techniques may be implemented in hardware, firmware, software, or combinations thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchange of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and the design requirements imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementations should not be interpreted as causing a departure from the scope of the present disclosure.

In a hardware implementation, the processing units used to perform the techniques may include one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs) ), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, and other electronic units designed to perform the functions described in this disclosure. , a computer, or a combination thereof.

Accordingly, the various illustrative logical blocks, modules, and circuits described in connection with this disclosure may be incorporated into a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may be implemented or performed in any combination of those designed to perform the functions described in A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

In firmware and/or software implementation, the techniques include random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), PROM ( on a computer readable medium, such as programmable read-only memory (EPROM), erasable programmable read-only memory (EPROM), electrically erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, or the like. It can also be implemented as stored instructions. Instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functionality described in this disclosure.

Although the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter in one or more stand-alone computer systems, the disclosure is not limited thereto and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. there is. Further, aspects of the subject matter in this disclosure may be implemented in a plurality of processing chips or devices, and storage may be similarly affected across multiple devices. These devices may include PCs, network servers, and portable devices.

Although the present disclosure has been described with respect to some embodiments, various modifications and changes may be made without departing from the scope of the present disclosure that can be understood by those skilled in the art. Moreover, such modifications and variations are intended to fall within the scope of the claims appended hereto.

Claims

In the method of computing device,

detecting global time information and global location information and identifying the artificial landmark while driving adjacent to a positioning device including an artificial landmark representing the global position information;

checking the global location information and global time information of the positioning device based on the artificial landmark;

determining relative position information between the positioning device and the computing device based on the artificial landmark; and

determining global location information and global time information of the computing device based on the relative location information and the global location information;

including,

method.
According to claim 1,

The positioning device,

Based on GNS information received from a satellite, configured to detect the global time information and global location information

method.
According to claim 1,

Identifying the artificial marker comprises:

acquiring an image having the artificial landmark by photographing the artificial landmark; and

Analyzing the image to extract identification information about the positioning device

including,

method.
According to claim 3,

Checking the global location information and the global time information of the positioning device includes:

checking the global location information of the positioning device using the identification information; and

checking the global time information of the positioning device in correspondence with the global location information of the positioning device;

including,

method.
According to claim 4,

The step of checking the global time information of the positioning device,

checking the global time information received from the positioning device; or

Checking global time information from a time when the image is acquired based on a predetermined global standard time

including,

method.
According to claim 1,

The relative location information is

the relative distance between the positioning device and the computing device; or

Relative attitude of the positioning device to the computing device

including at least one of

method.
According to claim 1,

The step of determining the relative position information,

Determining the relative position information using at least one of a camera module and a LiDAR sensor

including,

method.
According to claim 1,

Determining the global location information and the global time information of the computing device comprises:

combining the relative position information and the global position information of the positioning device to determine the global position information of the computing device; and

determining the global time information of the computing device with the global time information of the positioning device, in relation to the global location information of the computing device;

including,

method.
According to claim 1,

The global location information and the global time information of the computing device,

used to generate map data in the global coordinate system,

method.
A non-transitory computer readable recording medium having recorded thereon a program for executing the method of claim 1 in the computing device.
In a computing device,

Memory; and

a processor coupled with the memory and configured to execute at least one instruction stored in the memory;

the processor,

detecting global time information and global positioning information and identifying the artificial landmark while driving adjacent to a positioning device comprising an artificial landmark representing the global positioning information;

Based on the artificial landmark, determine the global location information and global time information of the positioning device;

Based on the artificial landmark, determine relative position information between the positioning device and the computing device;

Determine global location information and global time information of the computing device based on the relative location information and the global location information.

made up,

computing device.
According to claim 11,

The positioning device,

a positioning module configured to receive GPS information from a satellite; and

A processor configured to detect the global time information and global location information from the GNS information

Including more,

computing device.
According to claim 11,

Camera module connected to the processor

Including more,

the processor,

Obtaining an image having the artificial landmark by photographing the artificial landmark through the camera module;

Analyze the image to extract identification information about the positioning device.

made up,

computing device.
According to claim 13,

the processor,

Checking the global location information of the positioning device using the identification information;

Corresponding to the global location information of the positioning device, check the global time information of the positioning device.

made up,

computing device.
15. The method of claim 14,

Communication module connected to the processor

Including more,

the processor,

Receiving the global time information from the positioning device through the communication module; or

Based on a predetermined global standard time, to check the global time information from the time when the image was acquired.

made up,

computing device.
According to claim 11,

The relative location information is

the relative distance between the positioning device and the computing device; or

Relative attitude of the positioning device to the computing device

including at least one of

computing device.
According to claim 11,

At least one of a camera module or lidar sensor connected to the processor

Including more,

the processor,

To determine the relative position information using at least one of the camera module and the LIDAR sensor

made up,

computing device.
According to claim 11,

the processor,

combine the relative position information and the global position information of the positioning device to determine the global position information of the computing device;

With respect to the global location information of the computing device, determine the global time information of the computing device with the global time information of the positioning device.

made up,

computing device.
According to claim 11,

The global location information and the global time information of the computing device,

used to generate map data in the global coordinate system,

computing device.