KR20130002834A - Method for autonomous movement and apparatus thereof - Google Patents

Abstract

PURPOSE: An autonomous moving method and a device thereof are provided to increase data processing speed by converting 3D distance data into a Voxel shape. CONSTITUTION: 3D distance data and driving information are obtained(S201). The 3D distance data is converted into Voxel shaped data(S202). A driving map is generated based on the Voxel shaped data and the driving information(S203). A driving path is determined based on the driving map(S204). [Reference numerals] (S201) Obtaining 3D distance data and driving information; (S202) Converting the 3D distance data into Voxel shaped data; (S203) Forming a driving map based on the Voxel shaped data and the driving information; (S204) Determining a driving path based on the driving map; (S205) Displaying the driving path

Description

Autonomous Movement Method and Apparatus {Method for Autonomous Movement and Apparatus Thereof}

This specification relates to an autonomous movement method.

Recently, techniques for allowing vehicles such as two-wheelers, cars, and buses to autonomously travel without artificial manipulation have been studied. In such an autonomous mobile device, when the device moves while tracking an obstacle on a road, it is necessary to recognize an image of an obstacle placed in a driving direction by using a camera and determine a driving path by processing the recognized 3D distance data in real time. It is important.

As a method of recognizing obstacles and processing three-dimensional distance data, there is an occupancy grid method that represents a set of cells having a value of 1 (occupied) and 0 (unoccupied) according to the existence of an obstacle.

If you use the occupied grid method, you can do this offline because of the large amount of data information, or exclude the data of obstacles in unnecessary space by recursively dividing one node into eight nodes in three-dimensional space. The octree technique is mainly used to compress only the remaining data.

However, these methods are not only able to process data slowly, but also cannot determine the driving path efficiently, such as when the front tunnel exists, the entire tunnel is recognized as an obstacle and it is impossible to pass through the tunnel.

In the present specification, in 3D location recognition and map generation to determine the current location and the topography of the autonomous mobile device, the 3D distance data obtained through the sensor is converted into a voxel shape to process the data processing speed. The object of the present invention is to be able to recognize the three-dimensional position of the autonomous mobile device within a short time by raising.

In addition, the present specification is to reconstruct the three-dimensional distance data of the obstacle located on the driving path of the autonomous mobile device into a three-dimensional world (3D world) so that it is possible to clearly identify the obstacle and determine the driving path correctly. have.

The present specification comprises the steps of obtaining three-dimensional distance data and driving information; Converting the three-dimensional distance data into voxel shape data; Generating a driving map based on the converted voxel shape data and the driving information; And determining a driving route based on the driving map.

The method may further include displaying the driving route.

The 3D distance data may be obtained through at least one of a stereo camera, a depth camera, a mobile stereo camera, and lidar equipment.

The three-dimensional distance data may be formed based on at least one of three-dimensional image data of obstacles, distance information, and movement information.

The driving information may include one or more of steering, wheel speed, position information, and image based position estimation information.

The converting into voxel-shaped data may include: dividing the three-dimensional distance data into a cube-shaped voxel; Generating an arbitrary image space in which the voxel-shaped data is to be recorded; Performing an occupancy test on each of the divided voxels; And recording the presence or absence of an obstacle for each voxel on the basis of the occupancy test result in the arbitrary image space to generate the voxel-shaped data.

The occupancy test calculates the number of point data included in the divided voxels, and defines the occupied voxel if the number of the point data is equal to or greater than a preset threshold value, and the number of the point data is the threshold. If it is less than the value, it is characterized by being defined as an unoccupied voxel.

The driving map may be generated by concatenating only the newly acquired region except for the overlapped region of the 3D distance data which is continuously obtained based on the driving information.

The driving map may include a stop obstacle.

The displaying of the driving route may further include displaying information including a moving speed, a traveling time, a destination, and an azimuth.

In addition, the driving route is characterized by being displayed as a map of a fixed orientation or a three-dimensional solid shape in the form of a paper map.

In addition, the present specification, the sensor unit for obtaining three-dimensional distance data and driving information; And a controller configured to convert the 3D distance data into voxel-shaped data, generate a driving map based on the converted voxel-shaped data and the driving information, and determine a driving route based on the driving map. It is characterized by.

The apparatus may further include an output unit that displays the driving route.

The 3D distance data may be obtained through at least one of a stereo camera, a depth camera, a mobile stereo camera, and lidar equipment.

The three-dimensional distance data may be formed based on at least one of three-dimensional image data of obstacles, distance information, and movement information.

The driving information may include one or more of steering, wheel speed, location information, and image-based location estimation information.

The control unit may further divide the three-dimensional distance data into a cube-shaped voxel, generate an arbitrary image space in which the voxel-shaped data is to be recorded, and perform an occupancy test on each of the divided voxels. And recording the presence or absence of an obstacle for each voxel in the arbitrary image space based on the occupancy test result to generate the voxel-shaped data.

The occupancy test calculates the number of point data included in the divided voxels, and defines the occupied voxel if the number of the point data is equal to or greater than a preset threshold value, and the number of the point data is the threshold. If it is less than the value, it is characterized by being defined as an unoccupied voxel.

The driving map may be generated by concatenating only the newly acquired region except for the overlapped region of the 3D distance data which is continuously obtained based on the driving information.

The driving map may include a stop obstacle.

The output unit may further display information including a moving speed, a moving time, a destination, and an azimuth.

In addition, the driving route is characterized by being displayed as a map of a fixed orientation or a three-dimensional solid shape in the form of a paper map.

According to the present specification, by processing a large amount of three-dimensional distance data in real time using Voxel-based data compression technology, it is possible to generate a driving map of the mobile device at an improved speed and to reconstruct the three-dimensional world based on this have.

In addition, according to the present specification, not only obstacles having heights but also complex structures such as tunnels and overpasses can be detected, so that a more accurate driving route can be determined.

1 is a diagram illustrating an autonomous mobile device according to an embodiment of the present disclosure.
2 is a flowchart illustrating a driving map generation and driving route determination step using a voxel technique according to an exemplary embodiment of the present specification.
FIG. 3A is a diagram illustrating 3D distance data and transformed voxel shape data, and FIG. 3B is a flowchart illustrating a process of converting 3D distance data into voxel shape data.
4 is a view showing in detail the driving map generation step according to an embodiment of the present disclosure.
5 is a view showing in detail the driving path determination step according to an embodiment of the present disclosure.

The embodiments disclosed herein may be embodied in the form of program instructions that may be executed by various computer means and recorded on a computer readable medium. Computer-readable media may include, alone or in combination with the program instructions, file data, data structures, and the like. Program instructions recorded on the media may be those specially designed and constructed for the present invention, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Hardware devices specially configured to store and execute program instructions such as magneto-optical media and ROM, RAM, flash memory and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and examples thereof are the same.

It is to be noted that the technical terms used herein are merely used to describe particular embodiments, and are not intended to limit the present invention. In addition, the technical terms used herein should be interpreted as meanings generally understood by those of ordinary skill in the art, unless defined otherwise in this specification, and excessively inclusive It should not be interpreted in the sense of or in the sense of being excessively reduced. In addition, when a technical term used in this specification is an erroneous technical term that does not accurately express the concept of the technology disclosed in this specification, it should be understood that technical terms which can be understood by a person skilled in the art are replaced.

Also, the singular forms "as used herein include plural referents unless the context clearly dictates otherwise. In this specification, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or some steps It should be construed that it may not be included or may further include additional components or steps.

Further, the suffix "part" for a component used in the present specification is given or mixed in consideration of ease of specification, and does not have a meaning or role that is different from itself.

Further, in the description of the technology disclosed in this specification, a detailed description of related arts will be omitted if it is determined that the gist of the technology disclosed in this specification may be obscured. It is to be noted that the attached drawings are only for the purpose of easily understanding the concept of the technology disclosed in the present specification, and should not be construed as limiting the spirit of the technology by the attached drawings.

Hereinafter, exemplary embodiments disclosed herein will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an autonomous mobile device according to an embodiment of the present disclosure.

Referring to FIG. 1, the autonomous mobile apparatus 100 may include a 3D sensor unit 110, a sensor unit 120, a GPS transceiver 130, a controller 140, and an output unit 150. .

The 3D sensor unit 110 is a camera system for capturing the omnidirectional, rearward and / or lateral orientations at once using a rotating reflector, a condenser lens, and an image pickup device, and is applied to a security facility, a surveillance camera, and a robot vision. The shape of the rotor reflector is various, such as a hyperbolic surface, a spherical surface, a cone shape, and a compound type. In addition, a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) is used as the imaging device. The image projected onto the image pickup surface of the image pickup device (i.e., the omnidirectional image) is a distorted image that is reflected by the rotating reflector and is not suitable for human observation. Therefore, the 3D sensor unit 110 generates a new panoramic image by converting the coordinates of the output of the image pickup device through a microprocessor or the like to accurately observe the image.

According to an embodiment of the present disclosure, the 3D sensor unit 110 is a stereo camera, a depth camera, a moving stereo camera in order to obtain three-dimensional distance data by capturing the omnidirectional three-dimensional distance. stereo camera) and light detection and Ranging (LIDAR) equipment may be configured to include one or more.

A stereo camera is an imaging device composed of a plurality of cameras. The omnidirectional image obtained through the 3D sensor unit 110 provides two-dimensional information about the periphery of the 3D sensor unit 110. If a plurality of images taken from different directions through a plurality of cameras is used, three-dimensional information about the 3D sensor unit 110 may be obtained. Such a stereo camera may be used for location recognition and map generation of an autonomous mobile device or a mobile robot.

Depth cameras are cameras that capture or measure obstacles to extract images and distance data. That is, the depth camera generates image or image data by capturing obstacles like a general camera, and generates distance data by measuring a distance from the camera at an actual position corresponding to a pixel of each image.

For example, a 3D-TOF camera or the like may be used as the depth camera. The 3D-TOF camera, as the name suggests, is a camera that measures depth by time of flight theory. That is, the camera emits light with a very short pulse (about 20 MHz) using an infrared LED, and calculates a return time difference to find a depth value. It's like a radar camera.

Depth cameras can filter out distance data when capturing objects to pick out only obstacles within a certain distance. That is, by using the distance data of the depth camera, only objects located at a target depth field, except for obstacles in a near field and a far field, can be selected. Can be.

A mobile stereo camera refers to a camera that changes the position of a stereo camera actively according to an obstacle's distance to fix a viewing angle on an observation obstacle. Stereo cameras generally arrange two cameras in parallel and acquire an image, and calculate a distance to an obstacle according to stereo parallax of the acquired image. Such a stereo camera is a passive camera in which the optical axes are always parallel and fixed. Mobile stereo cameras, on the other hand, actively change the geometric position of the optical axis to fix the viewing angle. This control of the viewing angle of the stereo camera according to the distance of the obstacle is called gaze control. The stereoscopic control stereo camera maintains a constant stereo parallax for moving obstacles to provide stereoscopic observers with more natural stereoscopic images and useful information in distance measurement or stereoscopic image processing of obstacles.

Lidar (LIDAR) equipment is provided to detect the presence and distance of the obstacle located in front of the autonomous mobile device (100). Lidar equipment is a type of active remote sensing that uses the same principles as radar to obtain the desired information without direct contact with objects. Lidar equipment shoots a laser on a target to acquire information and detects the parallax and energy change of electromagnetic waves reflected from the target and acquires the desired distance information.

Lidar equipment is divided into three types according to the purpose or object to be measured: DIAL (Differentail Absorption LIDAR), Doppler LIDAR, and Range finder LIDAR. DIAL is used to measure the concentration of water vapor, ozone, and pollutants in the atmosphere by using two lasers with different absorption to the object to be measured.Doppler LIDAR uses the Doppler principle to move objects. Used for speed measurement. In general, however, Lidar refers to Range Finder LIDAR, which is known as the Global Positioning System (GPS), Inertial Navigation System (INS), and Laser Scanner (LASER SCANNER). 3D terrain information is acquired by combining distance information with an object using a system.

In the case of utilizing the lidar system in the autonomous mobile device 100, various sensors implemented by GPS, INS, 3D laser surveying technology, and photogrammetry are integrated and mounted on the autonomous mobile device 100, and the autonomous mobile device ( In addition to the operation of 100), it is implemented to acquire the position measurement and visual information of the feature on the driving route.

According to an embodiment of the present disclosure, the lidar equipment obtains three-dimensional distance data by detecting the presence of an obstacle located in front of the moving path of the autonomous mobile device 100, the distance to the obstacle, and the movement of the obstacle, and obtained the data. Is transmitted to the controller 140 to allow the autonomous mobile device 100 to move to a space free of obstacles.

In addition, according to an embodiment of the present disclosure, the 3D sensor unit 110 obtains image-based position estimation information (visual odometry) through the above-listed equipment. The position estimation information refers to information obtained by estimating a current position of the autonomous mobile device 100 based on known obstacles through 3D distance data acquisition. The autonomous mobile apparatus 100 may determine the driving route of the autonomous mobile apparatus 100 using the position estimation information as the driving information.

The sensor unit 120 detects various state changes of the autonomous mobile device 100 such as movement, temperature, and pressure of the autonomous mobile device 100, and obtains driving information of the autonomous mobile device 100. The obtained driving information is processed as information for driving operation of the autonomous mobile device 100 through the control unit 140.

The sensor unit 120 may include a steering detector 121, a speed sensor 122, an acceleration sensor 123, and a position sensor 124.

The steering detector 121 is interlocked with a steering wheel of the autonomous vehicle 100, a rotor that rotates, a gear that rotates integrally with the rotor, and a detector that detects a phase change due to rotation of a magnetic material generating magnetic force. It may include, but is not limited to, a PCB substrate and a housing for mounting a calculator and a calculator for calculating and outputting a negative input. Therefore, the steering detection unit 121 may be various steering wheel angle detection devices that further include additional components or omit some of the components.

The speed sensor 122 or the acceleration sensor 123 is provided inside or on the wheel of the autonomous vehicle 100 to acquire wheel speed of the autonomous vehicle 100, movement speed information of the autonomous vehicle 100, and the like. .

The position sensor 124 or the GPS transceiver 130 is used to obtain information about the position of the autonomous mobile device 100. Basically, the autonomous mobile device 100 includes a sensor to obtain information about its own movement. An example of such a sensor is the position sensor 124 or the GPS transceiver 130, and when used alone in the open air, it usually has an error of 10 m or more and can be used only as auxiliary information for obtaining an absolute position. Accordingly, the autonomous mobile device 100 attaches an additional survey sensor for identifying an absolute position without error, and processes the information obtained from the survey sensor to predict the location information of the autonomous mobile device 100.

According to an embodiment of the present disclosure, the autonomous mobile device 100 may include lidar equipment as an additional measurement sensor.

The position sensor 124 or the GPS transmitting / receiving unit 130 instructs the control unit 140 realized by a microcomputer or the like to read the map information of the current location via a communication base station (not shown) according to the operation result of the setting input unit. can do. The controller 140 may display the map image by driving the output unit 150 realized by a liquid crystal display. The position sensor 124 or the GPS transceiver 130 may be connected to a communication base station to transmit and receive the location of the autonomous mobile device 100 to another mobile device or data management computer existing in the wired / wireless network.

According to an embodiment of the present disclosure, the autonomous mobile device 100 recognizes the current position of the autonomous mobile device 100 through the position sensor 124 or the GPS transceiver 130, and moves to a driving path determined as a result of obstacle detection. It is controlled to, and can display the current position and the determined driving route through the output unit 150 as an image.

The controller 140 is electrically connected to the 3D sensor unit 110 so as to control various configurations of the autonomous mobile device 100 and process the 3D distance data. The controller 140 converts the electrical signal output from the 3D sensor unit 110 into an image signal, and generates a 3D driving map around the 3D sensor unit 110 from an image implemented through the converted image signal. The generated 3D driving map is updated whenever the position of the 3D sensor unit 110 is changed by the control of the controller 140. In addition, the controller 140 grasps the current location of the autonomous mobile device 100 and the topography of the surroundings from the 3D map to perform location control.

According to an embodiment of the present disclosure, the controller 140 processes the 3D distance data obtained through the 3D sensor unit 110 into a voxel shape, and generates a 3D driving map by interpolating the processed image. . Thereafter, the autonomous mobile apparatus 100 determines the driving route based on the generated driving map.

Image processing, driving map generation, and driving route determination based on voxels will be described in detail below.

The controller 140 may control a driving unit (not shown) of the autonomous mobile device 100 to move the autonomous mobile device 100 along the determined driving path. The controller 140 may control the forward or backward movement by controlling the rotational movement of the autonomous movement apparatus 100 so that the autonomous movement apparatus 100 moves in a desired direction, or by adjusting the engine or wheel shaft included in the driving unit. In addition, the controller 140 may perform a control for correcting a path error when the autonomous movement device 100 deviates from the determination path due to an error in the actual driving direction with respect to the determined driving path.

The output unit 150 may be fixed to the autonomous mobile device 100 in an embedded form or may be provided in a detachable form. In addition, the output unit 150 may be used to predict the moving direction of the autonomous mobile device 100 by displaying the driving route determined by the control of the controller 140.

The output unit 150 includes a display unit 151 and displays the driving route determined by the controller 150 using a user interface (UI) or a graphical user interface (GUI). The display unit 151 includes a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), and a flexible display (flexible). and at least one of a 3D display.

In addition, the output unit 150 may include an audio output unit 152, and may include an audio jack 153 that performs a function of connecting an earphone. Here, the audio output unit 152 may be a speaker.

The output unit 150 may display information such as a moving speed, a moving time, a destination, and an azimuth of the autonomous mobile device 100 in addition to the display of the driving route according to the exemplary embodiment of the present specification. The display of the driving route or the like may use a two-dimensional orientation fixed map in the form of a paper map or a three-dimensional map.

In addition, the autonomous mobile apparatus 100 may further include a storage unit (not shown) as necessary. The storage unit (not shown) stores various menu screens, various user interfaces (UIs), and / or graphical user interfaces (GUIs).

In addition, the storage unit stores the equation required to calculate the number of voxels after the conversion in converting the three-dimensional distance data into voxel-shaped data.

In addition, the storage unit stores data and programs required for the autonomous mobile device 100 to operate.

The storage unit may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD memory). Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EPM), Programmable Read-Only Memory (PROM), Random Access Memory (RAM), Static Random Access Memory (SRAM), Magnetic Memory It may include at least one storage medium of a magnetic disk, an optical disk.

Although the configuration of the general autonomous mobile device 100 has been described above, the autonomous mobile device 100 to which an embodiment of the present specification can be applied is not limited to the above-listed configuration, and may further include additional functional units. In carrying out the functions described above, other functional units not listed may be used.

2 is a flowchart illustrating a driving map generation and driving route determination step using a voxel technique according to an exemplary embodiment of the present specification.

Referring to FIG. 2, first, the autonomous mobile apparatus 100 obtains forward three-dimensional distance data and driving information (S201).

According to an embodiment of the present disclosure, the 3D distance data may be obtained through the 3D sensor unit 110 provided in the autonomous movement apparatus 100.

In addition, according to an embodiment of the present disclosure, the 3D distance data is formed based on one or more of the 3D image data, the distance information, and the movement information of the obstacle.

The autonomous mobile apparatus 100 detects the presence of the obstacle located in front of the moving path, the distance to the obstacle and the movement of the obstacle by using the 3D sensor unit 110, and transmits the detected three-dimensional distance data to the controller 140. To allow the autonomous mobile device 100 to move to a space free of obstacles.

In addition, according to an embodiment of the present disclosure, the driving information may include at least one of steering, wheel speed, location information, and image-based location estimation information, and the driving information is provided in the autonomous mobile device 100. It can be obtained through the interface (not shown), various sensors 121 to 124 or the GPS transceiver 130.

In the above, only a part of the driving information is enumerated, but the present invention is not limited thereto and may be used to determine a driving route using various driving information. The autonomous mobile apparatus 100 may use a radar or a video camera to obtain driving information. Additional structures, such as can be configured to include parts.

Afterwards, the autonomous mobile apparatus 100 converts the 3D distance data into a voxel shape (S202).

The data obtained through the 3D sensor unit 110 is composed of irregular point data, and in order to interpret them, it may be necessary to define an adjacent relationship between the points. The method of defining the adjacency relationship between points includes interpolating point data to define a grid or directly processing point data rather than a regular grid.

According to the exemplary embodiment of the present specification, the controller 140 of the autonomous mobile apparatus 100 converts the 3D distance data obtained through the 3D sensor unit 110 into voxel shape data in the form of a lattice. The voxel conversion step includes a three-dimensional space division step, an image space generation step, an occupancy test step, and a three-dimensional image generation step, which will be described in detail with reference to FIG. 3.

Next, the autonomous mobile device generates a driving map based on the data and the driving information converted into the voxel shape (S203).

The generation of the travel map in the autonomous mobile device 100 is an operation of storing the position of the obstacle around the autonomous mobile device 100 in an appropriate manner. As a method of expressing and storing a driving map, a free-space map, which is a method of expressing an empty space mainly of a driving area of the autonomous mobile device 100, a geometric figure, that is, a line segment, of an obstacle in the driving area. Or an object-oriented map, which is a method of displaying a set of polygons, and a composite-space map, in which empty space and objects can be displayed together.

The control unit 104 of the autonomous mobile device 100 continuously acquires data and driving information converted into a voxel shape, detects an obstacle located in front of the autonomous mobile device 100, and moves the autonomous mobile device 100 forward. Create a driving map for the direction. Hereinafter, the driving map generation will be described in detail with reference to FIG. 4.

Thereafter, the autonomous mobile device determines the driving route based on the driving map (S204).

The autonomous mobile device 100 determines the driving path of the autonomous mobile device 100 by finding a driving possible area excluding the obstacle area on the generated driving map so that the autonomous mobile device 100 can travel to an area free of obstacles. .

In addition, the autonomous mobile device may display a driving route (S205).

The display of the driving route is made in a visual or audio form through the output unit provided in the autonomous vehicle 100, and not only the driving route but also information such as a moving speed, a moving time, a destination, an azimuth angle, etc. of the autonomous vehicle 100. It can be displayed in various UI or GUI forms. The display of the driving route or the like may use a two-dimensional orientation fixed map in the form of a paper map or a three-dimensional map.

Hereinafter, the voxel conversion step, the driving map generation step, and the driving route determination step among the steps shown in FIG. 2 will be described in more detail.

FIG. 3A is a diagram illustrating 3D distance data and transformed voxel shape data, and FIG. 3B is a flowchart illustrating a process of converting 3D distance data into voxel shape data.

Referring to FIG. 3A, three-dimensional distance data 311 is converted into voxel shape data 313 by voxel shape conversion.

The voxel-based technology uses three-dimensional volumetric pixels, that is, a voxel structure, based on a silhouette image for reconstruction of a real-time three-dimensional image. This is a method of restoring the 3D structure by back-projecting each voxel into a 2D silhouette image in the 3D voxel space, and cutting off the non-silhouette area leaving the object area existing inside the silhouette. At this time, eight vertices of the voxel cube are projected onto the image plane to determine whether the object region is included.

Referring to FIG. 3B, a process of converting 3D distance data 311 into voxel-shaped data 313 is illustrated.

First, in converting a voxel shape, the controller 140 of the autonomous mobile apparatus 100 uses the 3D distance data 311 obtained through the 3D sensor unit 110 to form a plurality of voxels of a cube shape having a predetermined size. The operation is divided into 312 (s301). The controller 140 first calculates the number of voxels after voxel shape conversion for division. This may be calculated by a pre-stored equation considering the length of one side of the v-axis, the y-axis and the z-axis of the voxel within the effective field of view. The controller 140 divides the entire space into a cube form from any voxel center point in the 3D space based on the calculated number of voxels and the voxel length of each axis.

Next, the control unit 140 generates an arbitrary image space (s302). This image space is a space for recording voxel-shaped data indicating whether each voxel of a three-dimensional original image is occupied by an obstacle. Therefore, the controller 140 divides the image space into the same number of voxel shapes as the three-dimensional original image, so that each voxel is occupied by an object and can be recorded.

Thereafter, the controller 140 performs occupancy testing on each of the divided voxels (s303). This is done by counting the number of point data included in each voxel. Whether or not the voxel space is occupied as an object is defined as occupied voxel if the number of point data included in each voxel is greater than or equal to a preset threshold value (for example, a value occupying 50% of the space). If less, it is defined as a non-occupied voxel (Non-Occupied Voxel). The controller 140 determines that an object exists in the 3D space in the case of the occupied voxel, and determines that the object does not exist in the case of the unoccupied voxel. Occupancy tests are performed on all voxels.

Finally, the controller 140 records the presence or absence of an obstacle for each voxel in an arbitrary image space based on the result of the occupancy test to generate voxel-shaped data 313 (S304). If it is determined that the space is occupied by the object, this is performed by giving a constant pixel value to the voxel in the image space and recording the occupancy. The control unit 140 performs this operation on all voxels, thereby generating voxel shape data in which the three-dimensional distance data is changed into a cube.

In the above, an example of a technique for converting three-dimensional distance data into voxel-shaped data has been described. However, it will be apparent to those skilled in the art, and even if there are some or no major changes, or some omissions or additions, The technical idea is not limited.

The voxel-based 3D image processing compresses a large amount of 3D distance data in real time, thereby generating a driving map of the mobile device at an improved speed and reconstructing the 3D world based on the same.

4 is a view showing in detail the driving map generation step according to an embodiment of the present disclosure.

The control unit 140 of the autonomous mobile device 100 can obtain histogram matching information over time by continuously histogram matching the three-dimensional distance data obtained continuously in front of the driving route. Repeating in the entire driving process of 100 may simultaneously obtain the driving route and the driving map of the autonomous mobile apparatus 100.

Specifically, as shown in FIG. 4, the 3D distance data is continuously obtained while the autonomous mobile apparatus 100 is traveling, and the range of distances that can be obtained at each time point is limited. Therefore, in order to create a 3D map, only newly acquired images should be concatenated, excluding overlapping regions of the 3D data acquired at each viewpoint. At this time, the map must be accurately pasted based on the driving information including the driving direction, the traveling speed, and the like of the autonomous mobile apparatus 100. In FIG. 4, stop obstacles such as buildings, traffic signs, traffic lights, etc. are not displayed, but may include them in map generation.

In addition, according to an exemplary embodiment of the present disclosure, the driving map generation may not be performed alone, but may process the information of the sensor in real time and simultaneously process the obstacle recognition through the conversion of the 3D distance data.

In addition, according to an embodiment of the present disclosure, the generation of the driving map may be performed by the controller 140 of the autonomous mobile apparatus 100.

5 is a view showing in detail the driving path determination step according to an embodiment of the present disclosure.

The generated driving map is divided into a free space in which the autonomous mobile device 100 can travel, and an obstacle area that interferes with driving. Therefore, the controller 140 may determine the driving route of the autonomous mobile apparatus 100 by finding a driving possible region excluding the obstacle region in the driving map.

Referring to FIG. 5, the controller 140 detects a space free of obstacles based on the map generated in the driving map generation step, and controls a driving unit (not shown) so that the autonomous mobile device 100 can travel. In addition, the controller 140 may control the output unit 150 to display the determined driving route, and in addition, the output unit to display information such as a moving speed, a moving time, a destination, and an azimuth angle of the autonomous mobile device 100. 150 may be controlled. The display of the driving route or the like may use a two-dimensional orientation fixed map in the form of a paper map or a three-dimensional map.

According to an embodiment of the present disclosure, the driving map generated based on the voxel detects not only an obstacle having a height but also a complex structure such as a tunnel and an overpass, so that a more accurate driving path may be determined.

100: autonomous shifter
110: 3D sensor unit
120:
130: GPS transceiver
140:
150: output unit

Claims (22)

Obtaining three-dimensional distance data and driving information;
Converting the three-dimensional distance data into voxel shape data;
Generating a driving map based on the converted voxel shape data and the driving information; And
And determining a driving route based on the driving map. The method of claim 1,
And displaying the driving route. The method of claim 1 or 2, wherein the three-dimensional distance data,
And at least one of a stereo camera, a depth camera, a mobile stereo camera, and lidar equipment. The method of claim 1 or 2, wherein the three-dimensional distance data,
The autonomous movement method, characterized in that formed on the basis of one or more of the three-dimensional image data, the distance information, the movement information of the obstacle. The method of claim 1 or 2, wherein the driving information,
And at least one of steering, wheel speed, position information, and image based position estimation information. The method of claim 1 or 2, wherein converting the voxel-shaped data into
Dividing the three-dimensional distance data into a cube-shaped voxel;
Generating an arbitrary image space in which the voxel-shaped data is to be recorded;
Performing an occupancy test on each of the divided voxels; And
And recording the presence or absence of an obstacle for each voxel in an arbitrary image space based on the occupancy test result to generate the voxel-shaped data. The method of claim 6, wherein the occupancy test,
The number of point data included in the divided voxels is calculated, and if the number of the point data is greater than or equal to a preset threshold, the occupied voxel is defined. If the number of the point data is less than the threshold, the unoccupied voxel is determined. Autonomous movement method, characterized in that defined as. The driving map of claim 1, wherein the driving map comprises:
Based on the driving information, the autonomous movement method is generated by concatenating only the newly acquired area except for the overlapped area of the three-dimensional distance data that is continuously obtained. The method of claim 8, wherein the driving map,
Autonomous movement method comprising a stop obstacle. The method of claim 2, wherein displaying the driving route comprises:
And further displaying information including movement speed, movement time, destination, and azimuth. The method of claim 2, wherein the driving route,
An autonomous movement method characterized by being displayed as a two-dimensional orientation fixed map or a three-dimensional three-dimensional map in the form of a paper map. A sensor unit for acquiring three-dimensional distance data and driving information; And
And converting the three-dimensional distance data into voxel-shaped data, generating a driving map based on the converted voxel-shaped data and the driving information, and determining a driving route based on the driving map. Autonomous shifting device characterized in that. The method of claim 12,
And an output unit for displaying the driving route. The method of claim 12 or 13, wherein the three-dimensional distance data,
An autonomous mobile device obtained through at least one of a stereo camera, a depth camera, a mobile stereo camera, and lidar equipment. The method of claim 12 or 13, wherein the three-dimensional distance data,
The autonomous mobile device, characterized in that formed on the basis of at least one of the three-dimensional image data, the distance information, the movement information of the obstacle. The method of claim 12 or 13, wherein the driving information,
And at least one of steering, wheel speed, position information, and image based position estimation information. The method of claim 12 or 13, wherein the control unit,
Divide the three-dimensional distance data into a cube-shaped voxel, generate an arbitrary image space in which the voxel-shaped data is to be recorded, perform an occupancy test on each of the divided voxels, and the occupancy test And the voxel-shaped data is generated by recording the presence or absence of an obstacle for each voxel in an arbitrary image space based on a result. The method of claim 17, wherein the occupancy test,
The number of point data included in the divided voxels is calculated, and if the number of the point data is greater than or equal to a preset threshold, the occupied voxel is defined. If the number of the point data is less than the threshold, the unoccupied voxel Autonomous mobile device, characterized in that defined as. The method of claim 12 or 13, wherein the driving map,
Based on the driving information, the autonomous mobile device is generated by concatenating only the newly acquired area except for the overlapped area of the three-dimensional distance data that is continuously obtained. The method of claim 19, wherein the driving map,
Autonomous moving device comprising a stationary obstacle. The method of claim 13, wherein the output unit,
An autonomous mobile device further comprising information including a moving speed, a moving time, a destination, and an azimuth. The method of claim 13, wherein the driving route,
An autonomous mobile device characterized by being displayed as a two-dimensional orientation fixed map or a three-dimensional three-dimensional map in the form of a paper map.

Images