KR102106889B1 - Mini Integrated-control device - Google Patents

Abstract

In a preferred embodiment of the present invention, the small integrated control device includes a main processor that processes large-capacity sensor data using a multi-core CPU, and an auxiliary processor that processes the large-capacity sensor data in parallel using the same clock as the main processor. It uses the same clock as the main processor, it characterized in that it comprises a graphics processor for processing in parallel the operation that occurs when processing the large-capacity sensor data.

Description

Mini Integrated-control device

The present invention relates to a small integrated controller that receives a large amount of sensor data and processes it in parallel to obtain rapid results.

1 shows an example of a conventional controller. In the existing controller system, as shown in FIG. 1, a large number of CPU cores have been required to process large-capacity sensor data, and a plurality of PCs 110, 111, 112, 113, 114, and 115 are gigabit switches to secure them. It was used by connecting through (120).

In addition to the multiple PCs, the external sensor and additional parts for controlling the system are connected to individual products, so the controller system has a considerable size and volume.

"Towards Fully Autonomous Driving: Systems and Algorithms", 2011 IEEE Intelligent Vehicles Symposium (IV) Baden-Baden, Germany, June 5-9, 2011

When the large-capacity sensor data 100 shown in FIG. 1 is input to PC1 110, PC1 110 is another PC such as PC2, PC3, PC4, PC5 and PC6 (111, 112, 113, 114 and 115) The large-capacity sensor data received was transmitted and shared. However, in the case of data transmission and sharing, the Gigabit Ethernet switch is theoretically 1 Gb / s, but in reality, it has a transmission speed of 50 MB / s, so there is a problem that it cannot efficiently share large amounts of data between PCs.

To solve this problem, PC 1 (110) selects and shares only a few important frames among the received large-capacity sensor data, so PCs other than PC 1 (110) (111, 112, 113, 114, and 115) There is a problem in that a precise calculation result value cannot be obtained because the calculation is performed based on the sensor data having a reduced resolution.

In a preferred embodiment of the present invention, the small integrated control device includes a main processor that processes large-capacity sensor data using a multi-core CPU, and an auxiliary processor that processes the large-capacity sensor data in parallel using the same clock as the main processor. It uses the same clock as the main processor, it characterized in that it comprises a graphics processor for processing in parallel the operation that occurs when processing the large-capacity sensor data.

Preferably, the graphic processing unit receives 3D distance data among the large-capacity sensor data, converts the 3D distance data into voxel data, and 3D point cloud data included in each voxel. A calculation unit for calculating the mean and covariance of the; Passing to calculate the traversability, which is a probability value for passing through each voxel by calculating the surface direction angle and height of each voxel based on the Eisen value and Eisen vector value calculated based on the average and covariance values. Counting operation unit; And a frequency detection unit for numerically quantifying the number of times the voxel is scanned (Occupancy).

Preferably, the graphic processing unit transmits the pass coefficient for each voxel and the number of scans of each voxel to the main processor, and the main processor accumulates the pass coefficient for each voxel and the number of scans of each voxel. It is characterized by generating a map by using.

Preferably, the moving object is characterized by returning the traveling path using the driving path image photographed by the image photographing sensor mounted on the moving object and the map generated by the main processor.

In a preferred embodiment of the present invention, the main processor is characterized by including a layered stack including a system layer (System Layer), an interface layer (Interface Layer), a core layer (Core Layer) and an application layer (Application Layer). Is done.

As another preferred embodiment of the present invention, a method for processing large-capacity sensor data in a small integrated control device includes processing large-capacity sensor data using at least one multi-core CPU in a main processor; Processing a sensor data related to environmental recognition among the large-capacity sensor data using the same clock as the main processor in a coprocessor; and the large-capacity sensor data using the same clock as the main processor using a multi-core in a graphics processor Including; performing the operation processing, characterized in that the main processor, the auxiliary processor and the graphics processing unit processes the large-capacity sensor data in parallel.

As another preferred embodiment of the present invention, a method for processing large-capacity sensor data in a small integrated control device includes processing large-capacity sensor data using at least one multi-core CPU in a main processor; Processing laser-based sensor data among the large-capacity sensor data in an auxiliary processor using the same clock as the main processor; and in a graphic processing unit using the same clock as the main processor, multi-core image-based sensor data among the large-capacity sensor data Including; processing in parallel, including, the main processor, the auxiliary processor and the graphics processing unit to process the large-capacity sensor data in parallel, the graphics processing unit is a three-dimensional distance data of the large-capacity sensor data rectangular It is divided into voxel units of, and calculates a probability value that can pass through each voxel based on average and covariance values of 3D point cloud data in each voxel, and uses the number of times each voxel is scanned. To the accuracy of the probability of passing through each voxel. It characterized in that for calculating the weights.

As a preferred embodiment of the present invention, the small integrated control device has an effect that can be utilized in the fields of artificial intelligence, military equipment, factory automation, mobile server equipment, and autonomous driving robots.

1 shows an example of a general controller.
2 is a preferred embodiment of the present invention, showing an example of a small integrated device 200.
3 to 4 is a preferred embodiment of the present invention, showing the internal configuration of the small integrated control device 300.
5 to 7 are preferred embodiments of the present invention, and show an embodiment of performing arithmetic processing in a graphic processing unit.
8 shows a layered stack 800 supported by the main processor 310 as one preferred embodiment of the present invention.
9 is a preferred embodiment of the present invention, showing an embodiment of a mobile body equipped with a small integrated control device or using a small integrated control device.
10 is a flowchart of processing large-capacity sensor data in parallel in a graphic processing unit of a small integrated control device in a preferred embodiment of the present invention.
11 is a flowchart of performing arithmetic processing in a graphic processing unit of a small integrated control device in a preferred embodiment of the present invention.

The terminology used herein is for describing the embodiments and is not intended to limit the present invention. In this specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein, "comprises" and / or "comprising" refers to the components, steps, operations and / or elements mentioned above, the presence of one or more other components, steps, operations and / or elements. Or do not exclude additions.

Unless otherwise defined, all terms (including technical and scientific terms) used in the present specification may be used as meanings commonly understood by those skilled in the art to which the present invention pertains. In addition, terms defined in the commonly used dictionary are not ideally or excessively interpreted unless specifically defined.

Embodiments of the invention are not limited to server computer systems, desktop computer systems, laptops, handheld devices, smartphones, tablets, other thin notebooks, systems on a chip (SOC). It can also be used in devices, and other devices such as embedded applications. Handheld devices include cellular phones, Internet protocol devices, digital cameras, personal digital assistants (PDAs), and handheld PCs. Further, the apparatus, methods, and systems described herein are not limited to physical computing devices, but may also be used for software optimization for energy savings and efficiency.

2 is a preferred embodiment of the present invention, showing an example of a small integrated device 200.

The small integrated device 200 uses the at least one multi-core CPU to process the large-capacity sensor data, the main processor 210, and uses the same clock as the main processor 210, and uses the same processor to process the large-capacity sensor data 220 , It uses the same clock as the main processor, and includes a graphic processing unit 230 that performs arithmetic processing of large-capacity sensor data. As a preferred embodiment of the present invention, the main processor 210, the auxiliary processor 220, and the graphic processing unit 230 can process large-capacity sensor data in parallel.

The small integrated device 200 may be implemented to expand the main processor 210, the auxiliary processor 220, and the graphic processing unit 230 in series or in parallel using the input / output interface 240.

In one preferred embodiment of the present invention, the main processor 210 may be any type of data processor, including a general purpose or special purpose central processing unit (CPU), an on-demand semiconductor (ASIC) or a digital signal processor (DSP). .

For example, the main processor 210 may be a general-purpose processor such as a Core ™ i3, i5, i7, 2 Duo and Quad, Xeon ™, or Itanium ™ processor. The main processor 210 may be a processor manufactured for a special purpose, for example, a network or communication processor, a compression engine, a graphics processor, a co-processor, or an embedded processor. The main processor 210 may be implemented with one or more chips included in one or more packages. The main processor 210 can use any of a number of process technologies, such as, for example, BiCMOS, CMOS, or NMOS.

In one preferred embodiment of the present invention, the main processor 210 may include at least one multi-core CPU (212, 214). Communication between the at least one or more multi-core CPUs 212 and 214 is possible using a Quick Path Interconnect (QPI) protocol. The main processor 210 may interconnect the at least one multicore CPU 210 and 212 using a packet-based point-to-point interconnect bus.

In one preferred embodiment of the present invention, the graphics processing unit 230 includes logic for executing graphic commands, such as 3D or 2D graphic commands. The graphics processing unit 230 may execute industry standard graphic commands such as commands designated by Open GL and / or Direct X application programming interfaces (APIs) (eg, OpenGL 4.1 and Direct X 11).

The input / output interface 240 may support a network communication such as a local area network, a wide area network or the Internet, and communication with an internal unit or an external device. The input / output interface 240 may further include an adapter, a hub, and the like to provide access to the internal unit or external devices and network communication. The input / output interface 240 may be implemented on the same chip as the main processor 210 or on the same board, or may be implemented on a separate chip and / or package connected to the main processor 210.

3 to 4 is a preferred embodiment of the present invention, showing the internal configuration of the small integrated control device 300.

The small integrated control device 300 includes a main processor 310, an auxiliary processor 320, and a graphic processing unit 330.

Preferably, the small integrated control device 300 may further include an input / output interface 340 or an Ethernet switch 350.

Preferably, the small integrated control device 300 may further include a power supply unit (not shown) that supplies power to the main processor 310, the auxiliary processor 320, and the graphic processing unit 330.

Preferably, the small integrated control device 300 may further include a microcontrol unit (MCU) (not shown) that controls the main processor 310, the auxiliary processor 320, and the graphics processor 330. In this case, the MCU can perform communication such as Ethernet or RS232 with the main processor 310, and is connected by CAN communication with the power supply unit, so that it can be implemented to control subsequent measures of peripheral devices for the power status and fault conditions. have.

As a preferred embodiment of the present invention, the small integrated control device 300 processes the large-capacity sensor data in parallel using the main processor 310, the auxiliary processor 320, and the graphics processor 330 using the same system clock. You can.

As a preferred embodiment of the present invention, it is possible to assume an example where the small integrated control device 300 is implemented in an autonomous driving robot, a mobile robot, a moving object, or the like.

In this case, the main processor 310 automatically selects a driving path from an autonomous driving robot, a mobile robot, a moving object, and the like, and moves to a destination avoiding obstacles. And an autonomous driving robot, a mobile robot, a moving object, or the like, and generating a driving map or a map displaying a movement path for autonomous driving based on data on obstacles located in close proximity.

In this case, the auxiliary processor 320 processes the calculation related to environmental recognition among the large-capacity sensor data received from the main processor 310, and the graphic processor 330 performs parallel processing to process the image-related calculation among the large-capacity sensor data. Can be done.

Operations related to environmental recognition performed by the coprocessor 320 include operations for processing laser-based sensor data. The laser-based sensor data includes sensor data detected by a laser scanner or the like.

Operations related to an image performed by the graphic processing unit 330 include a camera image, arithmetic processing, and the like. Also, the calculation processing derived from the laser-based sensor data may be processed by the graphic processing unit 330.

In one preferred embodiment of the present invention, the main processor 310 is LADAR sensor information, 2D, 3D LADAR distance indicating position data, distance data, data on obstacles in an adjacent position, etc. Information, CAMERA image information, etc. can be received. In this case, the main processor 310 may be implemented to receive a large amount of sensor data using an Ethernet switch 350 or the like.

The main processor 310 transmits data so that the graphic data processing unit 330 performs the calculation processing of the distance data and the obstacles in the adjacent position among the large-capacity sensor data, and when the graphic processing unit 330 ends the calculation processing, the result It may be implemented to receive, and generate a driving map or a map based thereon. 5 to 6 for an embodiment of processing an operation in the graphic processing unit 330.

8 shows a layered stack 800 supported by the main processor 310 as one preferred embodiment of the present invention.

The layered stack 800 includes a system layer (810), an interface layer (Interface Layer) 820, a core layer (Core Layer) 830 and an application layer (Application Layer) 840.

The interface layer 820 supports MCU, sensor, and communication interfaces.

The core layer 830 is a dynamic environment real-time grasp, visual mileage grasp, location estimation such as laser scan matching, dynamic obstacle detection and tracking, laser-based environmental perception such as environmental perception, RRT sampling based global path planning, It supports planning such as costom planning and dynamic obstacle avoidance, driving control such as Basic Waypoint following and Fast Waypoint following, and other math libraries and utility libraries.

The coprocessor 320 may support and process environmental obstacles such as dynamic obstacle detection and tracking, and laser-based environmental recognition among the core layers 830. In addition, it can be implemented to process the part that can optimize parallel processing. Planning SW can also be processed by dividing it into each grid so that it can be processed by the coprocessor 320 if parallelization is possible.

5 to 6 is a preferred embodiment of the present invention, showing an embodiment of performing arithmetic processing in the graphics processing unit.

The graphic processing unit 500 uses the same clock as the main processor, and uses multi-cores to process computational processing of large-capacity sensor data in parallel.

The graphic processing unit 500 includes a calculation unit 532, a pass coefficient calculation unit 534, and a frequency detection unit 536.

The calculation unit 532 may receive 3D distance data, camera images, driving data, etc. from the large-scale sensor data from the main processor 510. As a preferred embodiment of the present invention, the 3D distance data is formed based on one or more of 3D image data, distance information, and movement information of an obstacle.

The calculation unit 532 converts 3D distance data into voxel data as in the embodiment of FIG. 6. Then, the average and covariance of 3D point cloud data included in each voxel is calculated.

Referring to FIG. 6, in a preferred embodiment of the present invention, the operation unit 532 divides the 3D distance data into a rectangular-sized voxel having a constant size (S610, S620, and S630). In this case, each voxel (S610, S620, S630) is independent of each other, and the calculation of each voxel can be quickly processed by a graphics processor implemented in multi-core. As a preferred embodiment of the present invention, the height of the voxel may be set to the maximum height that can be detected by a 3D distance sensor, such as a velodyne sensor.

In a preferred embodiment of the present invention, the calculation unit 532 calculates the mean and covariance of the 3D point cloud data included in each voxel (S610, S620, S630). .

The pass coefficient calculation unit 534 calculates an Eisen value and an Eisen vector value using the mean and covariance of each voxel calculated by the operation unit 532. In a preferred embodiment of the present invention, the height of the 3D point cloud data (FIG. 6, S611) in each voxel is obtained using the Eigen value, and 3 in each voxel using the Eisen vector value. The surface direction angle (FIG. 6, S613) of the dimensional point cloud data is obtained.

The pass coefficient calculation unit 534 calculates the height and surface direction angle of the three-dimensional point cloud data in each voxel using the Eisen value and the Eisen vector value, and determines whether it can pass through each voxel based on this. Calculate probability values. The surface orientation angle can be calculated through surface Normal. In the present invention, this probability value is referred to as a traversability. The pass coefficient can be obtained as in Equation 1.

In Equation 1, N (x) is a function for calculating the surface normal, and x represents the index of each voxel.

The frequency detection unit 536 quantifies the number of times each voxel is scanned (Occupancy). It should be noted that grasping the number of times each voxel is scanned is not limited to being performed only in the graphic processing unit 530, but can also be performed in the main processor 510.

As a preferred embodiment of the present invention, a laser sensor mounted on a mobile robot, a moving object, an autonomous robot, etc. has a higher number of times of scanning an object at a close distance than a number of times of scanning an object at a long distance. In a preferred embodiment of the present invention, the frequency detection unit 536 is located nearer as the number of times each voxel is scanned (Occupancy) is higher than a predetermined number of times by using these characteristics such as a laser scanner, a laser sensor, etc. It can be understood that the possibility recognized by the back is correct. In addition, it is possible to reduce errors in sensor data and errors generated when recognizing positions in a mobile robot, a moving object, or an autonomous robot.

As a preferred embodiment of the present invention, the graphic processing unit 530 transmits the number of times each voxel was scanned (Occupancy) information and the pass coefficient of each pixel to the main processor 510.

When the main processor 510 receives information on the number of scans of each voxel (Occupancy) and the pass coefficient of each pixel, the main processor 510 accumulates information on the number of scans of each voxel (Occupancy) and the pass coefficient of each pixel. The 3D driving map or map is generated through the driving map generating unit 511 by using the enemy. The main processor 510 may obtain movement information and a driving map over time by performing histogram matching on 3D distance data continuously obtained with respect to the front of the driving route. The generated 3D map can be stored on an internal chip, internal board, or remote device.

The driving map may be generated by the main processor 510 as shown in Equation 2.

In this case, Weight _t = Weight _{t -1} + Occupancy _t

In a preferred embodiment of the present invention, the moving object, the mobile robot, the autonomous robot, and the like can automatically return to the traveled path by referring to the driving map generated by the main processor 510.

In one preferred embodiment of the present invention, the driving map generation is not performed by the main processor 510 alone, but is processed in real time in parallel with the calculation processing by the graphic processing unit 530 and environmental recognition by the auxiliary processor. have.

7 is a preferred embodiment of the present invention, showing an example of calculating the pass coefficient by recognizing the three-dimensional distance data in the graphics processing unit.

The graphic processing unit recognizes the 3D distance data (710), converts the recognized 3D distance data into voxel data (720), and then averages and covariates the 3D point cloud data in each voxel. Based on the Eisen value and Eisen vector value, the pass coefficient is detected. As the value of the pass coefficient increases, it is displayed in dark, indicating that an obstacle is present. The portion where the value of the pass coefficient is large is darkly displayed (711), and the lower the value of the pass coefficient is, the lighter the shadow is displayed. For example, when the pass coefficient is 0.1, the shadow is lightly displayed, the leaf portion of the tree can be represented, and when the pass coefficient is 0.7, the shadow is darkly displayed and the pillar of the tree can be represented.

7C is a preferred embodiment of the present invention, and shows an example of setting a path by calculating a pass coefficient value in a mobile object, a mobile robot, and an autonomous robot 700 using a small integrated control device.

The higher the pass coefficient value, the lower the probability of passing, the darker the shade is displayed, and the lower the pass coefficient value, the higher the probability of passing through, and the lighter the shade.

9 is a preferred embodiment of the present invention, showing an embodiment of a mobile body equipped with a small integrated control device or using a small integrated control device.

The mobile body 900 may be implemented to include a 3D sensor unit, a sensor unit, a GPS transceiver unit, a control unit, and an output unit (not shown in the internal configuration). The sensor unit may include a steering sensing unit, a speed sensor, an acceleration sensor, and a position sensor.

The 3D sensor unit is a camera system that photographs the omnidirectional, posterior and / or lateral directions at once using a rotating reflector, a condenser lens, and an imaging device, and is applied to security facilities, surveillance cameras, and robot vision. The shape of the rotating reflector is various, such as a hyperbolic surface, a spherical surface, a cone, or a complex type. In addition, a CCD (Charge Coupled Device) or a Complementary Metal Oxide Semiconductor (CMOS) is used. The image projected on the imaging surface of the imaging element (that is, the omnidirectional image) is a distorted image that is not suitable for human observation as it is reflected by a rotating reflector. Therefore, the 3D sensor unit generates a new panoramic image by converting the coordinates of the output of the imaging element through a microprocessor or the like for accurate observation of the image.

The 3D sensor unit photographs omnidirectionally in three dimensions to obtain three-dimensional distance data, such as a stereo camera, a depth camera, a moving stereo camera, and a lidar (Light Detection and Ranging; LIDAR) may include one or more of the equipment.

A stereo camera is an imaging device composed of a plurality of cameras. The omnidirectional image obtained through the 3D sensor unit 110 provides two-dimensional information about the surroundings of the 3D sensor unit. If a plurality of images photographed in different directions through a plurality of cameras are used, 3D information about the surroundings of the 3D sensor unit may be obtained. Such a stereo camera is also used for location recognition and map generation of a moving object or mobile robot.

The depth camera is a camera that captures or measures obstacles and extracts image and distance data. That is, the depth camera generates image or image data by imaging an obstacle like a general camera, and measures distance from the camera at an actual location corresponding to a pixel of each image to generate distance data.

The mobile stereo camera refers to a camera in which the position of the stereo camera is actively changed according to the distance of the obstacle to fix the viewing angle for the observed obstacle. In general, a stereo camera can place two cameras in parallel and acquire an image, and calculate a distance to an obstacle according to the stereo parallax of the acquired image.

Such a stereo camera is a passive camera in which the optical axes are always arranged in parallel and fixed. On the other hand, the mobile stereo camera actively changes the geometrical position of the optical axis to fix the viewing angle. Controlling the viewing angle of the stereo camera according to the distance of the obstacle is referred to as the viewing angle control.

The stereoscopic viewing angle control stereo camera maintains a constant stereo parallax for a moving obstacle to provide a more natural stereoscopic image to a stereoscopic image observer, and provides useful information in measuring a distance to an obstacle or processing stereo images.

LIDAR equipment is provided to detect the presence and distance of obstacles located in front of the mobile body 900. LIDAR equipment is a type of active remote sensing that acquires desired information without direct contact with objects using the same principle as radar. The lidar equipment shoots a laser at a target to acquire information, and detects a parallax and energy change of electromagnetic waves reflected back from the target to obtain desired distance information.

Lidar equipment is divided into three types: DIAL (DIfferentail Absorption LIDAR), Doppler LIDAR, and Range finder LIDAR, depending on the purpose or object to be measured. DIAL is used to measure the concentration of water vapor, ozone, and pollutants in the atmosphere using two lasers with different absorption levels for the object to be measured, and Doppler LIDAR uses the Doppler principle to move the object. Used for speed measurement. In general, however, a lidar refers to a range finder lidar, which is a Global Positioning System (GPS), an Inertial Navigation System (INS), and a laser scanner (LASER SCANNER). ) 3D terrain information is acquired by combining distance information with an object using a system.

The lidar equipment detects the presence of an obstacle located in front of the moving path of the moving object 900, the distance to the obstacle, and the movement of the obstacle to obtain 3D distance data, and transmits the obtained data to the control unit to a space free of obstacles Allow the moving body 900 to move.

The output unit includes a display unit, and displays a driving route determined through the driving map generated by the driving map generation unit (FIGS. 5 and 511) of the control unit or the main processor using a user interface (UI) or a graphical user interface (GUI). Can be implemented.

10 is a flowchart of processing large-capacity sensor data in parallel in a graphic processing unit of a small integrated control device in a preferred embodiment of the present invention.

 A method for processing large-capacity sensor data in a small integrated control device includes processing large-capacity sensor data using at least one multi-core CPU in a main processor (S1010), and using the same clock as the main processor in the sub-processor. Process of sensor data related to environmental recognition among sensor data (S1020) and

It is characterized in that the graphic processing unit performs a step (S1030) of performing the calculation processing among the large-capacity sensor data in parallel using the same clock as the main processor using multi-core.

The graphic processing unit also performs parallel processing on large-capacity sensor data received from the main processor. Referring to FIG. 11, the graphic processing unit of the small integrated control device receives 3D distance data among large-capacity sensor data from the main processor (S1110). Thereafter, the 3D distance data is converted into voxel data (S1120), and the average and covariance of 3D point cloud data included in each voxel is calculated (S1130).

Thereafter, the graphic processing unit calculates an Eisen value and an Eisen vector value based on the average and covariance values (S1140), and calculates the surface direction angle and height of each voxel based on the Eisen value and the Eisen vector value to obtain traversability. To calculate.

Thereafter, the number of scans of each voxel (Occupancy) can be numerically assigned to weight the accuracy of the calculated pass coefficient (S1160).

The method of the present invention can also be embodied as computer readable codes on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, optical data storage devices, etc., and are also implemented in the form of carrier waves (for example, transmission over the Internet). Includes. The computer-readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

In the above, the present invention has been described with reference to specific embodiments shown in the drawings, but this is only exemplary, and a person having ordinary skill in the art to which the present invention pertains can make various modifications and variations therefrom. Therefore, the protection scope of the present invention should be interpreted by the claims, which will be described later, and all technical ideas within the equivalent and equivalent ranges should be interpreted as being included in the protection scope of the present invention.

Claims (40)

delete delete delete delete delete delete A main processor which processes sensor data using at least one multi-core CPU, and generates a map using cumulatively the number of scans of each voxel (Occupancy) and the pass coefficient calculated for each voxel;
An auxiliary processor using the same clock as the main processor and processing laser-based sensor data among the sensor data; and
Using the same clock as the main processor, the distance data among the sensor data is divided into voxel units, and each voxel is based on the average and covariance values of 3D point cloud data included in each voxel. A graphic processing unit for calculating a pass coefficient, which is a probability value for passing through each voxel by calculating the surface direction angle and height, and transmitting information on the number of times each voxel has been scanned and the pass coefficient of each pixel to the main processor; Including,
The graphic processing unit further processes sensor data related to image processing among the laser-based sensor data, and the main processor receives the number of times each voxel was scanned and the pass coefficient of each pixel from the graphic processing unit, and Integrated control device, characterized in that for generating a three-dimensional map as a basis. delete delete delete A main processor for processing sensor data using at least one multi-core CPU;
An auxiliary processor using the same clock as the main processor, processing laser-based sensor data among the sensor data, and the laser-based sensor data includes sensor data obtained through a laser scanner; and
It uses the same clock as the main processor, processes image-based sensor data in real time among the sensor data, and the image-based sensor data includes a graphic processing unit including a camera image; and the graphic processing unit includes the laser-based sensor Among the data, sensor data related to image processing is further processed,
The graphic processing unit also
Receives 3D distance data among the sensor data, converts the 3D distance data into voxel data, and calculates average and covariance of 3D point cloud data included in each voxel Operation unit;
A pass to calculate a traversability, which is a probability value for passing through each voxel by calculating the surface direction angle and height of each voxel based on the Eisen value and the Eisen vector value calculated based on the average and covariance values. Counting calculator; and
Integrated control device comprising a; frequency detection unit for quantifying the number of times each voxel is scanned (Occupancy). delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete As a method of processing sensor data in an integrated control device,
Processing sensor data using at least one multi-core CPU in a main processor supporting a layered stack including a system layer, an interface layer, a core layer supporting environment recognition and driving control, and an application layer;
In a coprocessor, a process related to environmental recognition supported by the core layer is processed using the same clock as the main processor, and the operation related to the environmental recognition includes an operation of processing laser-based sensor data among the sensor data. To do;
The graphic processing unit uses a multi-core to perform calculation processing among the sensor data using the same clock as the main processor, and the calculation processing includes processing image-based sensor data among the sensor data. ,
The graphic processing unit
The calculation unit receives 3D distance data among the sensor data, converts the 3D distance data into voxel data, and calculates average and covariance of 3D point cloud data included in each voxel. To do;
The pass coefficient calculation unit calculates the surface direction angle and height of each voxel based on the Eisen value and the Eisen vector value calculated based on the average and covariance values, and is the probability of passing through each voxel. Calculating;
The method comprising the steps of: quantifying the number of times the voxel is scanned by the frequency detection unit (Occupancy) and weighting the accuracy of the pass coefficient using the number of times the number of each voxel is scanned. . delete delete ◈ Claim 33 was abandoned when payment of the set registration fee was made.◈ The method of claim 30, wherein the voxel is
The method is characterized in that it has a rectangular shape and the height of the rectangle is set to the maximum height that can be detected by the 3D distance sensor. delete As a method of processing sensor data in an integrated control device, the method
Processing the sensor data using at least one multi-core CPU in the main processor
Processing laser-based sensor data among the sensor data in an auxiliary processor using the same clock as the main processor; and
In the graphic processor using the same clock as the main processor, processing the image-based sensor data among the sensor data using a multi-core; includes,
The graphic processing unit divides the 3D distance data among the sensor data into rectangular voxel units, and calculates a probability value that can pass through each voxel based on average and covariance values of 3D point cloud data in each voxel. A method of calculating and calculating a weight for the accuracy of a probability value that can pass through each voxel using the number of times each voxel is scanned. ◈ Claim 36 was abandoned when payment of the set registration fee was made.◈ 36. The method of claim 35, wherein the main processor
Method characterized by supporting the system layer (System Layer), the interface layer (Interface Layer), the core layer (Core Layer) and the application layer (Application Layer). ◈ Claim 37 was abandoned when payment of the set registration fee was made.◈ The method of claim 35, using an input / output interface
And implementing the main processor, the coprocessor, and the graphic processor to expand serially or in parallel. delete delete delete

Images