KR102186830B1 - A lidar system for artificial intelligent - Google Patents

Abstract

The present invention relates to a lidar system responding to an artificial intelligence, capable of being suitable for the requirements of artificial intelligence by precisely detecting only a specific area or accurately detecting only necessary data after detecting data of the entire detecting area of a lidar system. According to the present invention, the lidar system responding to the artificial intelligence comprises a control unit for storing a received signal as scanned data, detecting the presence or absence of an object and a physical quantity based on the scanned data, and controlling the scanned data to be transmitted to an upper control end. The control unit is operated as a system responding to artificial intelligence, such as a mode of scanning a specific scan area set in accordance with setting of an area to be scanned or specifying of a reference of an object to be tracked, by a user, a mode of scanning a specified scanning area when the upper control end specifies (specifying of the scan area) an object to be reported and tracked from the scan data, and a mode of selecting a detailed detection object requiring independent attention in accordance with an object detecting reference after scanning a basic area, and tracking the selected detailed detection object to obtain physical data so as to report the same to the upper control end.

Description

A lidar system for artificial intelligent}

The present invention relates to a LiDAR system corresponding to artificial intelligence, and in particular, after detecting the data of the entire detection area of Lidar, only a specific area is precisely detected or only the necessary data is accurately detected. It relates to a lidar system corresponding to artificial intelligence adapted to

In general, a lidar system is a detector that sequentially scans a laser for a short time and measures a time when the scanned laser pulse is reflected from a subject and returned to the device to detect the distance between the subject and the lidar body. The device that measures the delay time of this light scans multiple laser pulses at various angles and measures the distance to all objects in front of the lidar. This measuring process is called "scan". At this time, the subject is detected using the round trip time of the light reflected within a certain time within a specified distance, and the detection uses high-speed shooting to create a sprite image like a film of a movie. Acquire and detect the object in front.

As a result, most of the data acquired by the lidar within the observation range are duplicated or repeated data, and since it is a method of comparing and removing duplicate data of each scanned data, most of the computer resources are wasted in processing the duplicated data. have.

It scans the entire area periodically and detects the devices in front of it. It scans the entire area with a single TX/RX, mirror, and rotating body, or detects the entire area with multiple TX/RXs, rotating bodies, mirrors, and lenses. The device has been put into practical use.

Although this technology is in the early stages of application to the operation of autonomous vehicles, various auxiliary sensors have to be used due to lack of functions necessary for application to the field. In particular, since the lidar looks and detects the entire area, there is a disadvantage of wasting resources such as lidar and attached computers by looking at unnecessary redundant objects, such as removing redundant data by comparing each measured almost similar screen. .

Further, a prior art for detecting a subject using a lidar system is disclosed in <Patent Document 1> below.

In the prior art disclosed in <Patent Document 1>, two front lidar sensors are used to stop an emergency situation, and a rotating lidar sensor is mounted in the center of the vehicle to obtain 360-degree environmental information through the 360-degree rotating lidar sensor. It is used to detect obstacles, and is used for lane recognition and tracking through a web cam camera installed in front of the vehicle, and it is used for emergency stop situations, obstacle detection, lane recognition and tracking, and to safely autonomously drive the vehicle through this It provides a vehicle autonomous driving method using a camera and a lidar sensor.

However, this prior art can also be used to acquire environmental information using a rotating lidar sensor and use it to detect obstacles by analyzing it, but since the lidar looks at the entire area and detects, it is possible to compare almost similar screens measured every time. It wastes resources by continuously removing redundant data and comparing unnecessary redundant objects such as removing redundant data.

Republic of Korea Patent Registration 10-1998298 (Registration on July 3, 2019) (Autonomous vehicle driving method using camera and lidar sensor)

Accordingly, the present invention has been proposed to solve problems occurring in the general Lidar system and the prior art, and after detecting data of the entire detection area of Lidar, only a specific area is precisely detected or Its purpose is to provide a lidar system corresponding to artificial intelligence and a control method thereof that is adapted to the requirements of artificial intelligence by accurately detecting only data.

Another object of the present invention is to cope with an artificial intelligence capable of monitoring a more precise distance and speed, moving direction, and overall size of the object by paying attention to the moving objects in the sensor when an object moving differently from the background screen is detected in the entire detection area. It is to provide a lidar system and a control method thereof.

In order to achieve the above object, the "LIDAR system corresponding to artificial intelligence" according to the present invention includes: a light generating device for generating laser pulsed light for object detection; A MEMS mirror unit for reflecting the laser pulse light generated by the light generating device at an azimuth and latitude set by a control unit within a predetermined azimuth and latitude range of the measurement area; A lens that focuses the laser pulse light reflected through the MEMS mirror unit and scans the subject, and focuses and outputs the received light reflected from the subject; An optical receiver configured to convert the received light focused through the lens into an electrical signal to generate a detection signal, amplify and convert the generated detection signal into a digital signal, and output a measurement signal; A control unit that stores the measurement signal output from the optical receiver as scan data, detects the presence or absence of an object and a physical quantity based on the scan data, and controls to transmit the scan data to an upper control unit; And a communication unit interworking with the control unit to transmit scan data to an upper control unit.

In the above, the control unit controls the transmission of laser pulsed light for object detection, measures the presence or absence of an object and the distance to the object by using the detection time of the measurement signal reflected and returned from the object, and measures a plurality of laser pulsed light at the same angle. The speed is detected as a physical quantity by detecting the amount of change per unit time of the measurement signal and detecting the movement direction and size of the object as a physical quantity by changing the scanning angle of the laser pulsed light, and tracking a specific object by changing the scanning angle. It is characterized.

In addition, the "LIDAR system corresponding to artificial intelligence" according to the present invention includes an area setting unit that transmits a specific area setting value set by a user to the controller,

The control unit is characterized in that the control unit sets a specific scan area in the entire scan area according to the area setting value set by the area setting unit, and acquires and stores only detection data of the set specific scan area, or transmits it to an upper control unit.

When the control unit receives the scan area setting value from the upper control unit through the communication unit, the control unit sets a specific scan area from the entire scan area based on the received scan area setting value, and acquires only the detection data of the set specific scan area. It is characterized in that the storage and transmission to the upper control stage.

The control unit scans the set basic scan area to acquire and store scan data, analyzes the stored scan data, and creates a detection list when an object is detected, and if there is a detailed detection object requiring precise tracking from the detected object, MEMS By controlling the scanning area of laser pulsed light through mirror control, only the detailed detection object is tracked through only a specific scan area to store the detailed detection object list and scan data, and the physical quantity of the scan data of the detailed detection object is analyzed to determine an abnormal object If so, it is characterized in that the abnormal object is reported to the upper control terminal.

According to the present invention, after detecting the data of the entire detection area of Lidar, a Lidar system suitable for the requirements of artificial intelligence is provided while minimizing the consumption of resources by precisely detecting only a specific area or only necessary data. There is an effect it can provide.

In addition, according to the present invention, when a specific object that moves differently from the background screen is detected in the entire detection area, it is possible to monitor more precise distance and speed, moving direction, and overall size of the object by paying attention to the moving objects on the sensor. There is an effect that the LiDAR system can independently track and monitor objects without using.

In addition, according to the present invention, it is possible to intensively monitor objects to be careful among objects in the entire scan area, so that it can be used as a sensor for driving a vehicle, and provides time to cope with predicted situations that may occur in advance. There is also an advantage that can be used as an artificial intelligence system that promotes.

In particular, in the case of reporting only the data of a specific scan area out of the entire scan area to the upper control level, the amount of transmitted/received data can be reduced, thereby preventing wasted resources, and the processing speed of transmission/reception data by reducing the amount of data. There is also an effect that can be improved.

1 is an exemplary diagram of a structure of a lidar system corresponding to artificial intelligence according to the present invention,
2 is a configuration diagram of a lidar system corresponding to artificial intelligence according to the present invention,
3 is a conceptual diagram of setting a scan area in the present invention;
4 is a flow chart of a scan operation of the lidar system according to the present invention;
5 is a flowchart of controlling a scan area through communication with an upper control terminal in the present invention;
6 is a flowchart of an operation of independently setting a scan area and tracking an object in the lidar system in the present invention;
7 is an exemplary diagram of a data packet during communication between a lidar system and an upper control terminal in the present invention.

Hereinafter, a lidar system corresponding to artificial intelligence according to a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The terms or words used in the present invention described below should not be construed as being limited to a conventional or dictionary meaning, and the inventor should appropriately define the concept of terms in order to describe his own invention in the best way. It should be interpreted as a meaning and concept consistent with the technical idea of the present invention based on the principle that it can be.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are only preferred embodiments of the present invention, and do not represent all the technical spirit of the present invention, and various equivalents that can replace them at the time of the present application It should be understood that there may be variations.

1 is a structural diagram of a lidar system corresponding to artificial intelligence according to a preferred embodiment of the present invention, and FIG. 2 is a configuration diagram of a lidar system corresponding to artificial intelligence according to the present invention, a light generating device 101, A MEMS (micro-electromechanical system) mirror unit 105, a lens 102, an optical receiving device 106, a control unit 108, a region setting unit 110, and a communication unit 111 may be included.

In FIG. 2, reference numeral 10 denotes a subject to be detected, reference numeral 103 denotes laser pulsed light scanned onto the subject, and reference numeral 104 denotes reflected light reflected from the subject.

The light generating device 101 plays a role of generating laser pulsed light for object detection under the control of the control unit 108, and is an output unit that drives a laser diode and a laser that emits laser pulsed light in conjunction with the output unit. It may be composed of a diode and a collimator that converts laser light emitted from the laser diode into parallel light.

The MEMS mirror unit 105 serves to reflect the laser pulsed light by determining the azimuth and latitude of the laser pulsed light generated by the light generating device 101 according to the angle control of the control unit 108. The MEMS mirror unit 105 includes a MEMS mirror that reflects laser pulses and a mirror controller that controls the MEMS mirror, and precise angle control is possible by the mirror controller.

The lens 102 focuses the laser pulsed light transmitted through the MEMS mirror unit 105 and scans it on the subject 10, and the received light reflected from the subject 10 is focused and transmitted to the light receiving device 106 Plays a role. The laser pulsed light can be expanded and focused through the lens 102.

The optical receiver 106 converts the received light focused through the lens 102 into an electrical signal to generate a detection signal, amplifies and converts the generated detection signal into a digital signal, and converts the generated detection signal into a measurement signal. ). The optical receiving device 106 may include a receiving diode that converts the received optical signal into an electrical signal.

The control unit 108 stores the measurement signal output from the optical receiver 106 as scan data, detects the presence or absence of an object and a physical quantity based on the scan data, and controls to transmit the scan data to the upper control unit. Do it. To this end, the control unit 108 includes a memory for storing scan data inside, a data analyzer for detecting the presence or absence of an object and physical quantity from the detected data, controlling the storage of scan data and controlling the communication of the detected scan data, and a MEMS mirror unit according to the scan area ( 105) may include a control module for controlling the laser pulse scanning angle.

The communication unit 111 interworks with the control unit 108 to transmit scan data to the upper control unit, and serves to interface the scan area setting command transmitted from the upper control unit to the control unit 108.

In addition, the "LIDAR system corresponding to artificial intelligence" according to the present invention transmits a specific region setting value set by a user to the control unit 108 or transmits reference information of a detection object to be detected by the user. ) May further include a region setting unit 110 for transmission.

Here, the area setting unit 110 is an optional specification. That is, it may not be used when the control unit 108 sets a scan area according to a scan area setting command through the communication unit 111 or when the control unit 108 sets a scan area and automatically tracks an object. .

The operation of the LIDAR system corresponding to the artificial intelligence according to the present invention configured as described above will be described in detail with reference to FIGS. 1 to 7 attached thereto.

First, the laser pulsed light for object detection generated from the light generating device 101 is reflected at a desired angle by the MEMS mirror unit 105, and the reflected laser pulsed light is transmitted through the lens 102 at a desired angle. It is transmitted as a focused laser pulsed light 103 so as not to spread.

The laser pulsed light 103 is scanned on the subject 10, and the scanned light is reflected and partially returned. The returned received light 104 is again focused through the lens 102 and then input to the light receiving device 106, and the weak optical signal detected by the light receiving device 106, which is a detector, is converted into an electrical received signal. The converted and converted received signal is amplified and converted into a digital measurement signal through an appropriate amplifier and digital circuit and transmitted to the control unit 108.

In this case, when the MEMS mirror unit 105 is configured as a mirror having a high-speed response characteristic, it may transmit or receive a laser signal at an arbitrary angle. Therefore, if the prepared software in the control unit 108 gives a command to the sensor to pay attention to the area in which the selected object is detected, the given area can be more closely tracked and monitored according to this command.

The laser scans this limited area more precisely, so that the movement of the target can be observed. The entire system is operated by an embedded system with a plurality of small threads, and each thread tracks and monitors the subject by organically performing communication with the higher-level device, MEMS mirror control and laser scan of the device.

The control unit 108 controls the transmission of laser pulsed light for object detection, and measures the presence or absence of an object and the distance to the object using the detection time of a measurement signal (scan data) reflected from the object 10 and returned. . In addition, a plurality of laser pulses are transmitted at the same angle, and the amount of change per unit time of the measurement signal is detected, and the speed is detected as a physical amount. Also, by changing the scanning angle of the laser pulsed light, the moving direction and size of the object can be detected as a physical quantity. When a specific object that moves differently from the background screen is detected in the entire detection area, it is possible to track the specific object by changing the scanning angle of the laser pulsed light.

The operation of the lidar system of FIG. 2 can be understood through the diagram of FIG. 3.

Lida scans the entire field of view (FOV) area and sends this data to the upper level controller/signal processor. This signal processor can request the rider's control unit 108 to detect a specific area based on the scanned data. For example, in FIG. 3, it is possible to request more detailed detection of the object 201 moving in proximity or at a close distance, and thus scan data necessary for extracting necessary physical data such as the moving speed and direction of the subject or measured physical data may be requested.

In this operation, through the communication unit 111 of FIG. 2, the upper concept controller outside the rider exchanges packet data in the form of data of FIG. 7 with the rider, and controls the rider's operation or exchanges measurement data.

3, the object 204, 205, etc. are far from the rider device, but in the case of movement, or the object 202 that is dull or stationary in motion, etc. If desired, it is possible to request a partial scan of the area from a controller of a higher concept, and to meet this request, a partial scan operation function can be performed.

Here, the lidar system of the present invention can operate in three operation modes.

For example, as the first operation mode, the user sets the scan target area through the area setting unit 110, or scans a specific scan area set according to the reference of the target to be tracked, and scans as the second operation mode. When data is transmitted to the upper control terminal through the communication unit 111, and when the upper control terminal views the scan data and designates the target to be traced (scan area designation), the designated specific scan area is scanned. The third operation mode is the default area. After scanning the object, it selects a specific object that needs attention according to the object detection criteria, tracks the selected detailed detection object, that is, a specific object, acquires physical data, and reports it to the upper control. .

Each mode will be described in more detail as follows.

First, as shown in FIG. 4, in the first mode, the control unit 108 sets a specific scan area in the entire scan area according to the area setting value set by the area setting unit 110, and the set specific scan area It acquires and stores only the detection data of, or transmits it to the upper control.

For example, the user sets a specific scan area in which an object to be tracked is located in the basic scan area or a specific scan area to be scanned, or inputs reference information for scanning a specific scan area through the area setting unit 110.

The control unit 108 checks whether the user has set the specific area as the scan area in step S11, and if the user has not set the specific area as the scan area, proceeds to step S12 to scan the default scan area set by default, and the step The scan data of the basic scan area acquired in S13 is stored. The scanned data stored in this way may be transmitted as it is to the upper control terminal.

On the contrary, when the user sets a specific scan area as in step S14, the process moves to step S15 to scan only the specific area set by the user. Scanning for a specific area can be implemented simply by controlling the azimuth and latitude of the MEMS mirror unit 105 by the control unit 108, and since the azimuth and latitude control of the MEMS mirror unit 105 is a general method, a detailed description thereof Is omitted.

Subsequently, the process moves to step S16 to store the scan data of a specific area in the internal memory, and moves to the basic scanning device setting step in step S17.

Here, the scan data may include object presence information, distance information, speed information, size information, movement direction information, and the like.

In this way, if you scan only a specific area set by the user, you only need to scan a specific target to be monitored. As in the past, it is possible to prevent resource consumption by excluding specific scan data and deleting the rest of the entire scan data. Since only the object is intuitively scanned, the time required to observe a specific object can be shortened.

In the second mode, as shown in FIG. 5, when a scan area setting value is received from an upper control terminal through the communication unit 111, a specific scan area is set from the entire scan area based on the received scan area setting value, This is a method of acquiring and storing only detection data of a set specific scan area, and transmitting the detection data to the upper control terminal.

That is, in step S21, it is checked through the communication unit 111 whether the scan area is restricted from the upper control level, and if a specific scan area has not been set from the upper control level, the process moves to step S22 and scans the default area set by default. Then, the scanned data is stored in the internal memory, and the process moves to step S23 to transmit the stored scan data to the upper control unit through the communication unit 111.

On the contrary, when a specific scan area is set by the upper control level as in step S24, only the specific area set by the upper control level is scanned. Scanning for a specific area can be implemented simply by controlling the azimuth and latitude of the MEMS mirror unit 105 in the control unit 108, and the azimuth and latitude control of the MEMS mirror unit 105 is a general method for controlling MEMS. Therefore, a detailed description thereof will be omitted.

Subsequently, the process moves to step S25 to store the scan data of the specific area in the internal memory, and transmits the stored scan data of the specific area to the upper control terminal through the communication unit 111. Therefore, the upper control level can precisely scan only objects requiring unusual movement or attention in the entire scan area, thereby improving accuracy in object detection. Here, the scan data may include object presence information, distance information, speed information, size information, movement direction information, and the like.

7 is an example of a communication data packet when communicating with an upper control terminal.

When the data type is full scan data, the entire basic data area is determined as scan data and transmitted. When the data type is partial data (specific area scan), the extracted data field (2,23,32,100) of the set specific data area ( It includes physical quantity data, velocity data, size data, movement direction data, etc.), and includes a data verification field and a data end field.

If you scan only a specific area set by the upper control level, you only need to scan a specific target to be monitored, so you can prevent resource consumption of deleting the rest of the scan data except for the specific scan data, as in the past. Since only the object in the area is intuitively scanned, the time required to observe a specific object can be shortened.

In the third mode, as shown in FIG. 6, the lidar system sets its own scan area and tracks an object.

To this end, the control unit 108 scans the basic scan area set in step S31 to obtain and store scan data, analyzes the scan data stored in step S32, and creates a detection list when object detection occurs.

Next, in step S33, it is checked whether there is an object requiring attention from the detected object by using the reference set by the user or the object tracking reference information set by the upper control unit. Here, the object requiring attention may be a moving object as shown in FIG. 3 in the entire scan data. That is, the entire scan data may determine that a moving subject except a fixed subject is an object requiring attention.

If there is a detailed detection object that requires precise tracking as a result of the confirmation in step S33, the process moves to step S34 and controls the scanning area of the laser beam as the scan area of the detailed detection object through MEMS mirror control to detect detailed detection through only a specific scan area. It tracks only the object and stores the detailed detection object list and scan data. When the scan of the detailed detection object from the entire scan data is completed through this process (S35), the process moves to step S36 to analyze the physical quantity of the scan data of the detailed detection object to determine the presence or absence of an abnormal object. Here, the criterion for determining the presence or absence of an abnormal object may be set in advance, and may be, for example, a moving object that approaches, an object that is too close to the lidar system, or an object that approaches quickly. In addition, the presence or absence of an abnormal object can be determined through various setting criteria.

If there is an abnormal object among the detailed objects as a result of the determination in step S36, the process moves to step S38 and reports the abnormal object to the upper control terminal. Here, the report form for the abnormal object may be implemented in a packet form as shown in FIG. 7.

The upper control level analyzes the contents of the abnormal data reported by the LIDAR system and can request the necessary data to the LIDAR system, and for such additional or additionally requested data, Fig. 7 within the range of the prescribed communication protocol. It can be implemented by changing the packet type such as, or adding a new field.

In this way, by intensively managing specific points independently in the lidar system, the device using this can be used as a device that focuses on managing objects that need attention among various objects. In particular, when using this lidar system as a sensor device for driving a vehicle, it is necessary to have a margin of time to cope with the predicted situation that may occur in advance. As a result, a safe driving environment can be promoted, and even when applied to an aircraft, it can be utilized as an artificial intelligence system that can detect safety accidents that may occur in advance.

Although the invention made by the present inventor has been described in detail according to the above embodiment, the present invention is not limited to the above embodiment, and it is common knowledge in the art that various changes can be made without departing from the gist of the invention. It is self-evident to those who have

101: light generator
102: lens
103: laser pulsed light
104: reflected light
105: MEMS mirror unit
106: optical receiver
108: control unit
110: area setting unit
111: communication department

Claims (5)

A light generator for generating laser pulsed light for object detection;
A MEMS mirror unit for reflecting the laser pulse light generated by the light generating device at an azimuth and latitude set by a control unit within a predetermined azimuth and latitude range of the measurement area;
A lens that focuses the laser pulse light reflected through the MEMS mirror unit and scans the subject, and focuses and outputs the received light reflected from the subject;
An optical receiver for converting the received light focused through the lens into an electrical signal to generate a detection signal, amplifying the generated detection signal and converting it into a digital signal, and outputting a measurement signal;
A control unit that stores the measurement signal output from the optical receiver as scan data, detects the presence or absence of an object and a physical quantity based on the scan data, and controls to transmit the scan data to an upper control unit; And
And a communication unit for transmitting the scan data to the upper control unit in conjunction with the control unit,
The control unit controls the transmission of laser pulsed light for object detection, measures the presence or absence of an object and the distance to the object by using the detection time of the measurement signal reflected and returned from the object, and measures a plurality of laser pulsed light at the same angle. It transmits and detects the amount of change per unit time of the measurement signal to detect the speed as a physical quantity, and by changing the scanning angle of the laser pulsed light, the moving direction and size of the object are detected as a physical quantity, and a specific object that moves differently from the background screen in the entire detection area When detected, it tracks a specific object through the change of the scanning angle of the laser pulsed light,
An area setting unit for transferring a specific area setting value set by a user to the control unit or for transferring reference information of a detection target to be detected by a user to the control unit; Including more,
The control unit sets a specific scan area in the entire scan area according to the area setting value set by the area setting unit, and acquires and stores only the detection data of the set specific scan area, or transmits it to an upper control unit. Lidar system in response to intelligence
delete delete In claim 1, the control unit, when receiving a scan area setting value from an upper control unit through the communication unit, sets a specific scan area from the entire scan area based on the received scan area setting value, and detects the set specific scan area. A lidar system corresponding to artificial intelligence, characterized in that only data is acquired, stored, and transmitted to the upper control terminal.
In claim 1, the control unit scans the set basic scan area to obtain and store scan data, analyzes the stored scan data to create a detection list when an object is detected, and moves the detected object differently from the background screen. When a detection object exists, the scanning area of the laser beam is controlled through the MEMS mirror unit control, tracking only the detailed detection object through a specific scan area, saving a list of detailed detection objects and scan data, and the physical quantity of the scan data of the detailed detection object A lidar system corresponding to artificial intelligence, characterized in that, when it is determined that the object is an abnormal object by analyzing the object, the abnormal object is reported to the upper control unit.

Images