WO2023113191A1

WO2023113191A1 - Device for fixing plurality of sensors and system comprising same

Info

Publication number: WO2023113191A1
Application number: PCT/KR2022/015935
Authority: WO
Inventors: 박근호; 정성환; 김선형
Original assignee: 한국전자기술연구원
Priority date: 2021-12-13
Filing date: 2022-10-19
Publication date: 2023-06-22
Also published as: KR102642742B1; KR20230089043A

Abstract

The present invention provides a complex sensor fixing device and a system comprising same, the complex sensor fixing device comprising: a first fixing portion having a predetermined length in the z-axis direction; a first camera holding portion disposed at an upper end of the first fixing portion while having an opening area disposed toward a first direction; a first camera sensor inserted through the opening area in the first direction of the first camera holding portion to be fixed thereto; a second fixing portion disposed on the first camera holding portion while being disposed in parallel to the first fixing portion in the z-axis direction; a second camera holding portion having the same structure as the first camera holding portion and disposed at an upper end of the second fixing portion while having an opening area disposed toward a second direction; a second camera sensor having the same structure as the first camera sensor and inserted through the opening area in the second direction of the second camera holding portion to be fixed thereto; a third fixing portion disposed on the second camera holding portion while being disposed in parallel to the second fixing portion in the z-axis direction; and a lidar sensor disposed on the third fixing portion.

Description

A plurality of sensor fixtures and a system including the same

The present invention is a device for acquiring fusion data of a lidar sensor and a camera sensor, and a lidar sensor can be mounted on the top, and n camera sensors capable of observing all directions in the z-axis direction of the lidar sensor can be mounted. It relates to a plurality of sensor fixing devices and a system including the same.

For road traffic safety, various sensors need to be applied. Accordingly, various sensors have recently been researched and developed for vehicle safety or road traffic safety. For example, lidar sensors that enable precise measurement with a small error range and low distortion probability, and camera sensors that can take various images using optics are being researched and developed. As a result, a technology for judging the surrounding environment is being developed. By the way, the feature of recognizing the color of the camera sensor and the feature of recognizing the distance of the LIDAR sensor require calibration work in order to fuse with each other. However, since the characteristic difference between the color recognition feature of the camera sensor and the distance recognition feature of the LIDAR sensor is very large, calibration work is very difficult.

An object of the present invention is to provide a plurality of sensor fixing devices capable of facilitating information fusion between a lidar sensor and a camera sensor without separate calibration, and a system including the same.

In addition, the present invention is to provide a plurality of sensor fixing device and a system including the same that can acquire the data of the lidar sensor and the camera sensor without location limitation by determining the phase of the plurality of sensors without separate calibration.

According to an embodiment of the present invention, a complex sensor fixing device according to an embodiment includes a first fixing part having a certain length in the z-axis direction, and a first fixing part disposed on top of the first fixing part so that the opening area faces the first direction. 1 camera holder, a first camera sensor inserted and fixed through the opening area of the first camera holder in the first direction, disposed on the first camera holder and in the z-axis direction with the first fixing unit A second fixing part arranged side by side, a second camera holding part having the same structure as the first camera holding part and being disposed on top of the second fixing part so that an opening area faces a second direction, the first camera holding part A second camera sensor having the same structure as the sensor and being inserted and fixed through an opening area in the second direction of the second camera holder, disposed on the second camera holder and in the direction of the second fixing unit and the z-axis direction It is characterized in that it comprises a third fixing portion disposed side by side, and a LiDAR sensor disposed on the third fixing portion.

The complex sensor fixing device is disposed inside the first fixing part and is disposed inside the lidar sensor and wiring used to drive the first and second camera sensors, and is disposed inside the second fixing part and includes the lidar sensor and the second fixing device. It is characterized in that it further comprises a wire used for driving the camera sensor and a wire disposed inside the third fixing unit and used for driving the lidar sensor.

In addition, the complex sensor fixing device includes a third camera holding part disposed between an upper end of the third fixing part and the lidar sensor and having an opening area facing in a third direction, and the third camera holding part in the third direction. A third camera sensor inserted and fixed through an opening area of the , a fourth camera sensor disposed on the third camera holder, disposed parallel to the third fixing unit in the z-axis direction, and disposed below the lidar sensor. Characterized in that it further includes the government.

In addition, the composite sensor fixing device includes a fourth camera holder disposed between an upper end of the fourth fixing part and the lidar sensor and having an opening area facing a fourth direction, and a fourth camera holder disposed in the fourth direction. A fourth camera sensor inserted and fixed through an opening area of the , a fifth camera sensor disposed on the fourth camera holder, disposed parallel to the fourth fixing unit in the z-axis direction, and disposed below the lidar sensor. Characterized in that it further includes the government.

In addition, the complex sensor fixing device includes a first angle display structure disposed around the fifth fixing part to display a photographing angle of the fourth camera sensor, and disposed around the fourth fixing part to display a photographing angle of the third camera sensor. A second angle display structure for displaying a photographing angle, a third angle display structure disposed around the third fixing part and displaying a photographing angle of the second camera sensor, and a third angle display structure disposed around the second fixing part to display the photographing angle of the first camera It is characterized in that it further comprises at least one of the fourth angle display structures for displaying the photographing angle of the sensor.

On the other hand, the complex sensor fixing device is disposed inside the first fixing unit and is disposed inside the lidar sensor and wirings used to drive the first to fourth camera sensors, and is disposed inside the second fixing unit and includes the lidar sensor and Wiring used to drive the second to fourth camera sensors, disposed inside the third fixing unit, and disposed inside the lidar sensor and the third and fourth camera sensors, disposed inside the fourth fixing unit And characterized in that it further comprises a wire used for driving the lidar sensor and a fourth camera sensor, a wire disposed inside the fifth fixing unit and used for driving the lidar sensor.

A complex sensor fixing system according to an embodiment of the present invention includes a complex sensor fixing device in which a lidar sensor and a plurality of camera sensors are fixed, and a control device supporting the operation of the complex sensor fixing device. It is characterized in that it comprises the described first fixing part, first camera holding part, first camera sensor, second fixing part, second camera holding part, second camera sensor, third fixing part and lidar sensor.

Here, the control device is a power supply unit for supplying power required to drive the lidar sensor, the first camera sensor, and the second camera sensor, and a power supply necessary for operating the lidar sensor, the first camera sensor, and the second camera sensor. An input unit for generating an input signal, a communication circuit for receiving sensor signals from the lidar sensor, the first camera sensor, and the second camera sensor, driving the lidar sensor, the first camera sensor, and the second camera sensor and a display that outputs a screen according to the screen, the power supply unit, the input unit, the communication circuit, and a processor that controls the display.

In addition, the processor divides the screen of the display according to driving of the lidar sensor, the first camera sensor, and the second camera sensor, and provides a sensor signal corresponding to the sensor signal provided by the lidar sensor to the divided screen areas. A first screen, a second screen corresponding to a sensor signal provided by the first camera sensor, and a third screen corresponding to a sensor signal provided by the second camera sensor may be controlled to be output.

Additionally, the processor checks whether the second screen and the third screen overlap, and controls to output information corresponding to the overlap if they overlap by a specified size or more.

According to the plurality of sensor fixing device and the system including the same according to the present invention, the present invention can facilitate accurate phase detection of the lidar sensor and the camera sensor, and convergence of data obtained without calibration based on the phase of the sensor makes it possible

In addition, the present invention supports to adaptively install at least one camera sensor according to a corresponding installation environment without restrictions on location, direction, and number.

Other, various effects will be described along with the description of the specific content of the invention.

1 is a view showing an example of a complex sensor fixing device according to an embodiment of the present invention.

2 is a view showing another example of a complex sensor fixing device according to an embodiment of the present invention.

3 is a diagram showing an example of a complex sensor fixing system according to an embodiment of the present invention.

4 is a diagram showing an example of a configuration of a control device of a complex sensor fixing system according to an embodiment of the present invention.

It should be noted that in the following description, only parts necessary for understanding the embodiments of the present invention are described, and descriptions of other parts will be omitted without disturbing the gist of the present invention.

The terms or words used in this specification and claims described below should not be construed as being limited to ordinary or dictionary meanings, and the inventors have appropriately used the concept of terms to describe their inventions in the best way. It should be interpreted as a meaning and concept consistent with the technical spirit of the present invention based on the principle that it can be defined in the following way. Therefore, the embodiments described in this specification and the configurations shown in the drawings are only preferred embodiments of the present invention, and do not represent all of the technical spirit of the present invention, so various equivalents that can replace them at the time of the present application It should be understood that there may be variations and variations.

Hereinafter, the present invention normalizes the lidar sensor reference, the position and shooting direction of the camera sensor using a fixed phase device to support the configuration of data acquired using the sensors as absolute coordinates without a separate calibration operation. Due to the advantage of not requiring calibration, the present invention supports use in various types of terrain without restrictions such as on roads, traffic lights, telephone poles, intersections, etc. where traffic data should be collected.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

Referring to FIG. 1 , a complex sensor fixing device 100 according to an embodiment of the present invention includes a first fixing part 131, a first camera holding part 140, a first camera sensor 121, and a second fixing part. 132 , the second camera holder 150 , the second camera sensor 122 , the third fixing unit 133 and the lidar sensor 110 may be included.

The first fixing part 131 may have a certain length in the z-axis direction and a certain area in the xy plane so as to fix the complex sensor fixing device 100 to a specific structure or specific environment. For example, the first fixing part 131 may be provided in a cylindrical shape or a cylindrical shape. In particular, the first fixing unit 131 may include an empty space therein to transmit/receive signals related to driving the first camera sensor 121, the second camera sensor 122, and the lidar sensor 110. Alternatively, a part of a wire connecting the first camera sensor 121 , the second camera sensor 122 , and the lidar sensor 110 may be disposed on at least a part of the inside and outside of the first fixing part 131 . The first fixing part 131 is a sensor (eg, at least one of the first camera sensor 121, the second camera sensor 122, and the lidar sensor 110) under the environment in which the complex sensor fixing device 100 is installed. ), while having a certain strength and hardness so as to support it, it may be provided with a material having strong characteristics against corrosion due to moisture. For example, at least a part of the first fixing part 131 may be formed of a metal material having moisture resistance, a plastic structure, synthetic rubber, or a polymer structure.

The first camera holder 140 may be coupled to an upper end of the first fixing part 131 . Even though the first camera holder 140 is coupled to the top of the first fixing unit 131, it can be rotated on the xy plane according to a user's manipulation. The first camera holder 140 may be provided in a “C” shape in which the first camera sensor 121 can be seated as a whole. For example, the first camera holder 140 may include a first support part 141 , a first pillar part 142 and a first cover part 143 . Here, one side of the first support part 141 is coupled to the upper end of the first fixing part 131 and is disposed to support the bottom surface of the first camera sensor 121, and the other side is the bottom of the first pillar part 142. Can be combined with cotton. At least a part of the lower part of the first pillar part 142 may be coupled to the other side of the first support part 141 and at least a part of the upper part may be coupled to one side of the first cover part 143 . The first cover portion 143 may have one side coupled to the upper portion of the first pillar portion 142 and the other side coupled to the lower portion of the second fixing portion 132 . The first support part 141 and the first cover part 143 are arranged in a structure that is arranged vertically with the first pillar part 142 interposed therebetween, and are arranged vertically in the same direction (eg, y-axis direction). can be placed appropriately. The height of the first pillar portion 142 may be formed equal to a partial thickness of the first camera sensor 121 or may be formed larger than a partial thickness of the first camera sensor 121 by a designated height. The first camera holder 140 has at least one of the structures (eg, at least one of the first support part 141, the first pillar part 142, and the first cover part 143) inside and outside the first A wiring (eg, wiring pattern or cable) connected to the camera sensor 121 may be further included.

The first camera sensor 121 may be inserted and fixed inside the first camera holder 140 . The first camera sensor 121 may be disposed to be in contact with at least one of, for example, an upper surface of the first support part 141 and a lower surface of the first pillar part 142 . Alternatively, the first camera sensor 121 may contact and be fixed to the inside of the first pillar part 142 . Alternatively, various structures (eg, adhesive tape, sponge or rubber, etc.) , or between the first camera sensor 121 and the first cover part 143. Alternatively, the complex sensor fixing device 100 penetrates through a peripheral structure (eg, at least one of the first support part 141, the first pillar part 142, and the first cover part 143), and the first camera sensor ( 121) may further include bolts or screws screwed together. The shooting center direction of the first camera sensor 121 may be the direction opposite to the direction in which the first pillar part 142 is arranged among the directions in which the first support part 141 and the first cover part 143 are arranged side by side. there is.

The second fixing part 132 may be disposed on the same axis as the first fixing part 131 based on the z-axis. That is, the z-axis center line of the first fixing part 131 and the z-axis center line of the second fixing part 132 may be extended. The second fixing part 132 may be formed with the same material and the same structure as the first fixing part 131 . For example, the second fixing part 132 may be formed in a cylindrical tubular shape in which an internal through hole penetrating in the z-axis direction is disposed. The lower part of the second fixing part 132 may be fixed to the first camera holder 140 , and the upper part of the second fixing part 132 may be fixed to the second camera holder 150 . Wires for driving the lidar sensor 110 and the second camera sensor 122 may be disposed inside the second fixing unit 132 .

The second camera holder 150 may have the same structure as the first camera holder 140 . For example, the second camera holder 150 may include a second support part 151 , a second pillar part 152 and a second cover part 153 . The second support part 151, the second pillar part 152, and the second cover part 153 are the same as the first support part 141, the first pillar part 142, and the first cover part 143 described above. can have a structure. The second camera holder 150 is provided in a “C” shape, and may have an opening direction different from that of the first camera holder 140 .

The second camera sensor 122 may have the same structure as the first camera sensor 121 . The second camera sensor 122 may be seated and fixed to the second camera holder 150 . The photographing central point of the second camera sensor 122 may coincide with the opening direction of the second camera holder 150 . Accordingly, on the xy plane, a photographing angle captured by the second camera sensor 122 may have a different angle from an angle photographed by the first camera sensor 121 . The second camera sensor 122 includes at least one structure included in the second camera holder 150 (eg, among the second support part 151, the second pillar part 152, and the second cover part 153). at least one) and wires formed on the second fixing part 132, the first camera holding part 140, and the first fixing part 131 (eg, a power supply line for supplying power required to drive the second camera sensor 122). and wires for transmitting and receiving signals related to driving the second camera sensor 122).

The lower part of the third fixing part 133 may be fastened to one side (eg, the second cover part 153) of the second camera holder 150, and the other side may be fastened to the lidar sensor 110. The third fixing part 133 may be disposed on the same axis as the second fixing part 132 and the first fixing part 131 along the z-axis. Alternatively, the z-axis center line of the third fixing part 133 may be disposed at a position extending from the z-axis center line of the second fixing part 132 and the first fixing part 131 . The third fixing part 133 may have the same or similar material and structure as the second fixing part 132 . For example, a through hole penetrating in the z-axis direction may be formed in the center of the third fixing part 133 . A wire (or cable) necessary for driving the lidar sensor 110 may be disposed in the inner through hole of the third fixing part 133 .

The lidar sensor 110 may be fixed to an upper portion of the third fixing part 133 . The lidar sensor 110 may determine a distance to a surrounding object by irradiating a signal in a direction of 360 degrees on the xy plane and then receiving a corresponding signal. The lidar sensor 110 may be electrically connected through a wire disposed on at least one side of the inside and outside of the third fixing part 133 .

Referring to FIG. 2 , a complex sensor fixing device 101 according to another example of the present invention includes a first fixing part 131, a first camera holding part 140, a first camera sensor 121, and a second fixing part. (132), second camera holder 150, second camera sensor 122, third fixing part 133, third camera holding part 160, third camera sensor 123, fourth fixing part 134 , a fourth camera holder 170 , a fourth camera sensor 124 , a fifth fixing unit 135 and a lidar sensor 110 . Prior to the description, the first fixing part 131, the first camera holding part 140, the first camera sensor 121, the second fixing part 132, the second camera holding part 150, and the second camera The sensor 122, the third fixing unit 133, and the lidar sensor 110 may have substantially the same configuration as the configurations described in FIG. 2 above. Accordingly, detailed descriptions of related components will be omitted.

The third camera holder 160 may be disposed above the third fixing unit 133 . The third camera holder 160 also includes the same structure as the first camera holder 140 or the second camera holder 150 described above, for example, a support part, a pillar part, and a cover part, and has a C-shaped structure, The third camera sensor 123 may be inserted and fixed into the opening. For example, the third camera holder 160 may have a support portion coupled to an upper end of the third fixing portion 133 . At least a portion of wires related to driving the third camera sensor 123 (eg, power lines and signal wires for camera operation) may be disposed on at least a portion of the inside and outside of the third camera holder 160 . The third camera holder 160 is disposed between the third fixing part 133 and the fourth fixing part 134 and can rotate at a predetermined angle in response to a user's manipulation.

The third camera sensor 123 may be inserted into and fixed to the third camera holder 160 . At least a portion of the third camera sensor 123 may be fixed in contact with at least a portion of the third camera holder 160 or may be fixed using an additional structure (eg, sponge, rubber, bolt, etc.). The third camera sensor 123 may have a direction (or a photographing center line or a photographing angle) disposed differently from those of the first camera sensor 121 , the second camera sensor 122 , and the fourth camera sensor 124 .

The fourth fixing part 134 may be disposed between the third camera holder 160 and the fourth camera holder 170 . The fourth fixing part 134 may be disposed parallel to the first fixing part 131 , the second fixing part 132 , and the third fixing part 133 based on the z-axis. Alternatively, the first fixing part 131, the second fixing part 132, the third fixing part 133, and the fourth fixing part 134 may be positioned on a straight line based on the z-axis. The fourth fixing part 134 may have the same height as at least some of the other fixing parts. One side of the fourth fixing part 134 is coupled to one side of the cover part of the third camera holder 160, and the other side of the fourth fixing part 134 is coupled to one side of the support part of the fourth camera holder 170. can The fourth fixing part 134 may be provided in a cylindrical shape having a certain length in the z-axis direction.

The fourth camera holder 170 may have a structure substantially identical to or similar to the previously described third camera holder 160 . However, the opening direction of the fourth camera holder 170 is different from other camera holders (eg, the first camera holder 140, the second camera holder 150, and the third camera holder 160). can be placed.

The fourth camera sensor 124 may have the same structure as other camera sensors (eg, the first camera sensor 121 , the second camera sensor 122 , and the third camera sensor 123 ). However, a photographing angle of the fourth camera sensor 124 may be disposed differently from photographing angles of other camera sensors. At least a portion of the wiring required to drive the fourth camera sensor 124 may be disposed on at least one side of the inside and outside of the fourth camera holder 170 .

The fifth fixing part 135 may be disposed between the lidar sensor 110 and the fourth camera holder 170 . The fifth fixing part 135 may be disposed on the same z-axis as the fourth fixing part 134 . Wires necessary for driving the lidar sensor 110 may be disposed on at least a portion of the inside and outside of the fifth fixing unit 135 .

Meanwhile, an angle display structure 135a (eg, a protractor) may be disposed at the bottom of the above-described second to fifth fixing

parts

132, 133, 134, and 135 to recognize the direction in which each camera sensor is directed. can In the angle display structure 135a, printed information representing 0 to 360 degrees may be disposed. The angle display structure 135a may be fixed to at least one of the second to fifth fixing

parts

132 , 133 , 134 , and 135 , and may be aligned along the z-axis.

An indicator 174 (eg, an arrow) may be printed and disposed on each cover of the first to

fourth camera holders

140 , 150 , 160 , and 170 . The indicator 174 is disposed on the upper surface of the cover of each of the

camera holders

140, 150, 160, and 170 and can be observed in the z-axis to -z-axis direction. As the first to

fourth camera holders

140, 150, 160, and 170 rotate, the complex sensor fixing device 101 having the above structure is seated on each of the

camera holders

140, 150, 160, and 170. And it supports the user to recognize the directions in which the fixed

camera sensors

121, 122, 123, and 124 are directed. Accordingly, the user can use the angle display structure 135a and the indicator 174 disposed on each of the fixing

parts

132, 133, 134, and 135 to determine the direction the

camera sensors

121, 122, 123, and 124 are pointing at. Rotation angles of the

camera sensors

121 , 122 , 123 , and 124 (or angles in the opening directions of the

camera holders

140 , 150 , 160 , and 170 ) may be determined so that the directions do not overlap.

Inside the first fixing part 131, wires (or cables) necessary for driving the lidar sensor 110 and the first to

fourth camera sensors

121, 122, 123, and 124 are disposed, and the second fixing part Inside the 132, wires (or cables) necessary for driving the lidar sensor 110 and the second to

fourth camera sensors

122, 123, and 124 are disposed, and inside the third fixing part 133, the lidar Wires (or cables) necessary for driving the sensor 110 and the third and

fourth camera sensors

123 and 124 are disposed, and the lidar sensor 110 and the fourth camera sensor are inside the fourth fixing part 134. (124) Wires (or cables) necessary for driving may be disposed, and only wires necessary for driving the lidar sensor 110 may be disposed inside the fifth fixing part 135. The first to

fourth camera sensors

121 , 122 , 123 , and 124 may be disposed with the same protrusion amount protruding from the

respective camera holders

140 , 150 , 160 , and 170 .

The complex

sensor fixing device

100 or 101 of the present invention described in FIGS. 1 and 2 described above adjusts the number of cameras to be used for image capture by adding or removing a camera sensor, a camera holder, and a fixing part according to user needs. can In addition, the user may add or remove necessary camera sensors according to the calculation of the range required for observation.

3 is a view showing an example of a complex sensor fixation system according to an embodiment of the present invention, and FIG. 4 is a diagram showing an example of the configuration of a control device of the complex sensor fixation system according to an embodiment of the present invention.

Referring to FIGS. 3 and 4 , the complex sensor fixing system 10 of the present invention may include a complex

sensor fixing device

100 or 101 and a control device 200 . Here, the composite

sensor fixing device

100 or 101 may correspond to the composite sensor fixing device 100 described above with reference to FIG. 1 . As another example, in the complex sensor fixing system 10 of the present invention, the complex sensor fixing device may correspond to the complex sensor fixing device 101 described above with reference to FIG. 2 .

The control device 200 includes a housing that can be fastened to the first fixing part 131 among the complex

sensor fixing devices

100 or 101, or the complex sensor through a specific structure in which the first fixing part 131 is mounted. It may be connected to the

fixing device

100 or 101. The control device 200 may include an input unit 210, a communication circuit 220, a display 230, a memory 240, a power supply unit 260, and a processor 250.

The input unit 210 may generate an input signal for controlling the lidar sensor 110 and a plurality of camera sensors disposed in the complex

sensor fixing device

100 or 101 . In this regard, the input unit 210 may include a switch for turning on or off the power of each sensor. Alternatively, the input unit 210 may generate an input signal for adjusting the signal collection period of the lidar sensor 110 in response to user manipulation.

The communication circuit 220 may support formation of a communication channel of the control device 200 . In the previous description, it has been described that the wiring for driving the lidar sensor 110 and the camera sensors is disposed inside the fixing parts, but in the complex

sensor fixing device

100 or 101 of the present invention, the lidar sensor 110 and the plurality The camera sensors of may include a wireless communication circuit that can be connected to the communication circuit 220 of the control device 200 while each including a battery so as to be independently driven. Alternatively, the communication circuit 220 may provide a cable connection interface that can be connected to the aforementioned lidar sensor 110 and a plurality of camera sensors by wire. The communication circuit 220 transmits a control signal for controlling the lidar sensor 110 and the plurality of camera sensors to each sensor, and receives sensor signals collected by the lidar sensor 110 and the plurality of camera sensors. and can be transmitted to the processor 250.

The display 230 may output information according to the operation of the complex sensor fixing system 10 . For example, the display 230 may display operating states of the lidar sensor 110 and a plurality of camera sensors. In addition, the display 230 may output a screen corresponding to signals collected by the lidar sensor 110 and a plurality of camera sensors. In this regard, the display 230 may divide the screen and output a screen corresponding to sensor signals transmitted by each sensor on the divided screen. At this time, the display 230 performs screen division according to the number of sensors in response to the control of the processor 250 (eg, 3 screen divisions when there are 2 camera sensors, 5 screen divisions when there are 4 camera sensors), , It is possible to output a screen corresponding to the signal provided by the sensors to each of the divided areas. In addition, the display 230 classifies and outputs the identification number of each sensor to screen areas, so that it is possible to intuitively check which screen area corresponds to which camera sensor collects images. By comparing the screens output on the display 230 with each other, the user can check whether there is an overlapping area among the camera sensors and adjust the photographing angles of the camera sensors so that they do not overlap.

The memory 240 may adjust data and programs necessary for operating the control device 200 . For example, the memory 240 may store programs necessary for operating the lidar sensor 110 and programs necessary for operating each camera sensor. According to settings, the memory 240 may store sensor signals collected by the lidar sensor 110 and sensor signals (or images) collected by each camera sensor. The sensor signals stored in the memory 240 may be transmitted to an external electronic device (eg, a management office device that manages the complex sensor fixing system 10 ) in response to control of the processor 250 .

The power supply unit 260 may supply power necessary for operating the control device 200 . Additionally, the power supply unit 260 may support power supply to the lidar sensor 110 and a plurality of camera sensors.

The processor 250 recognizes when a camera sensor is physically added or removed from the complex

sensor fixing device

100 or 101, and accordingly controls the screen output of the display 230, controls power supply, and acquires the sensor. It can perform storage and transmission control of signals. As an example, the processor 250 compares images collected by a plurality of camera sensors, and when there are images overlapping within a certain angle range, a message informing that the shooting angles of the corresponding camera sensors are overlapping is sent. can be printed out. For example, the processor 250 may highlight image objects indicating camera sensors with overlapping photographing areas or output audio signals corresponding to identification numbers of corresponding camera sensors.

As described above, the composite sensor fixing system 10 of the present invention provides a composite sensor fixing device in which the distance between the lidar sensor and the camera sensors can be regularly arranged, so that the y-axis direction of the lidar sensor is 0 degrees. It supports to measure the photographing direction of the camera sensors based on the standard. Here, in consideration of the characteristics of the lidar sensor that can observe all directions, the lidar sensor is placed at the top of the complex sensor fixing device (100 or 101), and the camera sensors that can observe only one direction are By providing an indicator capable of measuring the observation direction (or image capturing direction) of the camera sensors while arranging them in a row based on the z-axis of the ida sensor 110, the obtained without separate calibration between the lidar sensor and the camera sensors It is possible to support convergence of sensor data and to acquire data in various environments by changing the number of camera sensors.

On the other hand, the embodiments disclosed in this specification and drawings are only presented as specific examples to aid understanding, and are not intended to limit the scope of the present invention. In addition to the embodiments disclosed herein, it is obvious to those skilled in the art that other modifications based on the technical idea of the present invention can be implemented.

[Description of code]

10: Complex sensor fixation system

100, 101: composite sensor fixture

110: lidar sensor

121, 122, 123, 124: camera sensor

131, 132, 133, 134, 135: fixed part

135a: angle display structure

140, 150, 160, 170: camera holder

200: control device

210: input unit

220: communication circuit

230: display

240: memory

250: processor

260: power supply

Claims

A first fixing part having a certain length in the z-axis direction;

a first camera holder disposed on an upper end of the first fixing unit and having an opening area facing a first direction;

a first camera sensor inserted and fixed through an opening of the first camera holder in the first direction;

a second fixing part disposed on the first camera holder and parallel to the first fixing part in the z-axis direction;

a second camera holder having the same structure as the first camera holder and disposed above the second fixing part with an opening area facing a second direction;

a second camera sensor that has the same structure as the first camera sensor and is inserted and fixed through an opening area of the second camera holder in the second direction;

a third fixing part disposed on the second camera holder and parallel to the second fixing part in the z-axis direction;

A complex sensor fixing device comprising a; LiDAR sensor disposed on the third fixing part.
According to claim 1,

a wire disposed inside the first fixing unit and used to drive the lidar sensor and the first and second camera sensors;

a wire disposed inside the second fixing unit and used to drive the lidar sensor and the second camera sensor;

A complex sensor fixing device further comprising a wiring disposed inside the third fixing part and used for driving the lidar sensor.
According to claim 1,

a third camera holder disposed between an upper end of the third fixing unit and the lidar sensor and having an opening area facing a third direction;

a third camera sensor inserted and fixed through an opening of the third camera holder in the third direction;

A complex sensor fixing device further comprising a fourth fixing part disposed on the third camera holder, disposed parallel to the third fixing part in the z-axis direction, and disposed under the lidar sensor. .
According to claim 3,

a fourth camera holder disposed between an upper end of the fourth fixing unit and the lidar sensor and having an opening area facing a fourth direction;

a fourth camera sensor inserted and fixed through an opening of the fourth camera holder in the fourth direction;

A complex sensor fixing device further comprising a fifth fixing part disposed on the fourth camera holder, disposed parallel to the fourth fixing part in the z-axis direction, and disposed under the LiDAR sensor. .
According to claim 4,

a first angle display structure disposed around the fifth fixing unit and displaying a photographing angle of the fourth camera sensor;

a second angle display structure disposed around the fourth fixing unit and displaying a photographing angle of the third camera sensor;

a third angle display structure disposed around the third fixing unit and displaying a photographing angle of the second camera sensor;

a fourth angle display structure disposed around the second fixing unit and displaying a photographing angle of the first camera sensor; A composite sensor fixture further comprising at least one of the following.
According to claim 4,

a wire disposed inside the first fixing unit and used to drive the lidar sensor and first to fourth camera sensors;

a wire disposed inside the second fixing unit and used to drive the lidar sensor and the second to fourth camera sensors;

a wire disposed inside the third fixing unit and used to drive the lidar sensor and the third and fourth camera sensors;

a wire disposed inside the fourth fixing unit and used to drive the lidar sensor and the fourth camera sensor;

The composite sensor fixing device further comprising a wiring disposed inside the fifth fixing part and used for driving the lidar sensor.
A complex sensor fixing device in which a lidar sensor and a plurality of camera sensors are fixed;

A control device supporting the operation of the complex sensor fixing device; includes,

The complex sensor fixture is

A first fixing part having a certain length in the z-axis direction;

a first camera holder disposed on an upper end of the first fixing unit and having an opening area facing a first direction;

a first camera sensor inserted and fixed through an opening of the first camera holder in the first direction;

a second fixing part disposed on the first camera holder and parallel to the first fixing part in the z-axis direction;

a second camera holder having the same structure as the first camera holder and disposed above the second fixing part with an opening area facing a second direction;

a second camera sensor that has the same structure as the first camera sensor and is inserted and fixed through an opening area of the second camera holder in the second direction;

a third fixing part disposed on the second camera holder and parallel to the second fixing part in the z-axis direction;

A composite sensor fixing system comprising the lidar sensor disposed on the third fixing part.
According to claim 7,

The control device

a power supply unit supplying power required to drive the lidar sensor, the first camera sensor, and the second camera sensor;

an input unit generating input signals necessary for operating the lidar sensor, the first camera sensor, and the second camera sensor;

a communication circuit receiving sensor signals from the lidar sensor, the first camera sensor, and the second camera sensor;

a display outputting a screen according to driving of the lidar sensor, the first camera sensor, and the second camera sensor;

A complex sensor fixing system comprising the power supply unit, the input unit, the communication circuit, and a processor that controls the display.
According to claim 8,

The processor

Dividing the screen of the display according to driving of the lidar sensor, the first camera sensor, and the second camera sensor, and a first screen corresponding to the sensor signal provided by the lidar sensor to the divided screen areas; A complex sensor fixing system characterized in that control is performed to output a second screen corresponding to a sensor signal provided by a first camera sensor and a third screen corresponding to a sensor signal provided by the second camera sensor.
According to claim 9,

The processor

The complex sensor fixing system, characterized in that for checking whether the second screen and the third screen overlap, and controlling to output information corresponding to the overlap if they overlap by a specified size or more.