KR102315703B1 - A method and apparatus for sharing object sensing information of infra sensor - Google Patents

Abstract

The present invention relates to a method and apparatus for sharing object sensing information of an infrastructure sensor. A method of operating an infrastructure sensor device according to an embodiment of the present invention includes the steps of: (a) acquiring an image of a region of interest; (b) extracting at least one object included in the region of interest image; (c) obtaining positioning information for the at least one object by using the absolute coordinates of each of the at least one object; and (d) transmitting the location information to the at least one object, wherein the transmitted location information may be used for each of the at least one object to recognize a location of each other.

Description

A method and apparatus for sharing object sensing information of infra sensor

The present invention relates to a method and apparatus for sharing object sensing information of an infrastructure sensor, and more particularly, transmits object sensing information of an infrastructure sensor to surrounding objects, calculates absolute coordinates of an object using map information, and , to a method and apparatus for performing positioning error correction by comparing the position of an object measured by an infrastructure sensor with a position measured by each object.

These days, along with the development of science and technology, autonomous vehicle technology that operates automatically without a driver is being developed, and some tangible results are showing in road driving using such an autonomous vehicle. Such autonomous vehicle technology can implement autonomous driving by installing various sensors in the vehicle to collect all sensing information related to the vehicle's driving, and then using it.

In this case, the autonomous vehicle uses sensors such as a camera, lidar, and radar installed in the vehicle to recognize and determine surrounding conditions and control the vehicle to move to a desired path. However, there is a problem in that the camera, lidar, and radar mounted on the vehicle cannot recognize the situation of the blind area, which is an invisible area, because they use light, laser, and radio waves. Therefore, technology to solve these problems is required, but research on this is insufficient.

[Patent Document 1] Korean Patent Publication No. 10-2015-0075774

The present invention was created to solve the above problems, and an object of the present invention is to provide a method and an apparatus for sharing object sensing information of an infrastructure sensor.

In addition, the present invention is an identifier for each object (identification, ID), absolute coordinates, movement direction, movement speed, acceleration, and cooperation for transmitting positioning information including at least one of the shooting time of the image of the region of interest to surrounding objects It aims to implement sensing technology.

Another object of the present invention is to calculate absolute coordinates of objects included in an image of a region of interest by using absolute coordinates of major road marking lines of a high-precision map.

Another object of the present invention is to perform positioning error correction by comparing the position of the object measured by the infrastructure sensor device with the position of the object measured by the object device.

Objects of the present invention are not limited to the objects mentioned above, and other objects not mentioned will be clearly understood from the description below.

In order to achieve the above objects, a method of operating an infrastructure sensor device according to an embodiment of the present invention comprises the steps of: (a) acquiring an image of a region of interest; (b) extracting at least one object included in the region of interest image; (c) obtaining positioning information for the at least one object by using the absolute coordinates of each of the at least one object; and (d) transmitting the location information to the at least one object, wherein the transmitted location information may be used for each of the at least one object to recognize a location of each other.

In an embodiment, the method of operating the infrastructure sensor device may further include, after step (b), allocating an identifier (ID) to each of the at least one extracted object.

In an embodiment, the positioning information may include at least one of an identifier (ID) for each of the at least one object, an absolute coordinate, a moving direction, a moving speed, an acceleration, and a capturing time of the image of the region of interest. .

In an embodiment, the step (c) may include calculating the absolute coordinates of each of the at least one object; and calculating at least one of a moving direction, a moving speed, and an acceleration of each of the at least one object by using the absolute coordinates.

In an embodiment, the calculating of the absolute coordinates may include: calculating absolute coordinates of the road marking lines included in the image of the region of interest by using the absolute coordinates of the road marking lines of the map information; calculating absolute coordinates of all pixels of the region of interest image by using absolute coordinates of road marking lines included in the region of interest image; and calculating the absolute coordinates of each of the at least one object matching the absolute coordinates of all pixels of the region of interest image.

In an embodiment, the calculating of the absolute coordinates for all pixels of the region of interest image includes interpolating or extrapolating the absolute coordinates of road marking lines included in the region of interest image, and the region of interest It may include; calculating absolute coordinates for all pixels of the image.

In an embodiment, a method of operating an object device includes: (a) receiving positioning information for at least one object from an infrastructure sensor device; and (b) calculating a first location of the at least one object by using the location information, wherein the location information may be obtained using the absolute coordinates of each of the at least one object.

In an embodiment, the positioning information includes an identifier (ID) for each of the at least one object, absolute coordinates, a moving direction, a moving speed, an acceleration, and a recording time of an ROI image including the at least one object. It may include at least one.

In an embodiment, the step (b) comprises: calculating the first position; comparing the second position of the at least one object calculated by the object device with the first position of the at least one object included in the positioning information; and calculating the first location as the location of the at least one object when a difference between the first location and the second location is greater than a threshold value.

In an embodiment, the infrastructure sensor device includes: a sensor unit configured to acquire an image of a region of interest; a control unit extracting at least one object included in the region of interest image and obtaining positioning information on the at least one object by using the absolute coordinates of each of the at least one object; and a communication unit configured to transmit the positioning information to the at least one object, wherein the transmitted positioning information may be used for each of the at least one object to recognize a location of each other.

In an embodiment, the controller may allocate an identification (ID) to each of the at least one extracted object.

In an embodiment, the positioning information may include at least one of an identifier (ID) for each of the at least one object, an absolute coordinate, a moving direction, a moving speed, an acceleration, and a capturing time of the image of the region of interest. .

In an embodiment, the controller may calculate absolute coordinates of each of the at least one object, and calculate at least one of a movement direction, a movement speed, and an acceleration of each of the at least one object by using the absolute coordinates.

In an embodiment, the controller calculates the absolute coordinates of the road marking line included in the image of the region of interest by using the absolute coordinates of the road marking line of the map information, and uses the absolute coordinates of the road marking line included in the image of the region of interest. Thus, absolute coordinates of all pixels of the region of interest image may be calculated, and absolute coordinates of each of the at least one object matching the absolute coordinates of all pixels of the region of interest image may be calculated.

In an embodiment, the controller may calculate absolute coordinates for all pixels of the region of interest image by interpolating or extrapolating the absolute coordinates of the road marking lines included in the region of interest image. .

In an embodiment, the object device may include: a communication unit configured to receive positioning information on at least one object from an infrastructure sensor device; and a controller configured to calculate a first position of the at least one object by using the positioning information, wherein the positioning information may be obtained using the absolute coordinates of each of the at least one object.

In an embodiment, the positioning information includes an identifier (ID) for each of the at least one object, absolute coordinates, a moving direction, a moving speed, an acceleration, and a recording time of an ROI image including the at least one object. It may include at least one.

In an embodiment, the object device may further include a sensor unit configured to calculate a second position of the at least one object, wherein the controller includes: the second position of the at least one object calculated by the object device and the The first position of the at least one object included in the positioning information is compared, and when the difference between the first position and the second position is greater than a threshold value, the first position is calculated as the position of the at least one object can do.

Specific details for achieving the above objects will become clear with reference to the embodiments to be described in detail below in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, but may be configured in various different forms, and those of ordinary skill in the art to which the present invention belongs ( Hereinafter, "a person skilled in the art") is provided to fully inform the scope of the invention.

According to an embodiment of the present invention, objects located in a blind spot may recognize each other by implementing a cooperative sensing technology for transmitting positioning information to surrounding objects.

In addition, according to an embodiment of the present invention, the position of each object is more precisely determined by comparing the position of the object measured by the infrastructure sensor device and the position of the object measured by the object device and performing positioning error correction. can be calculated.

The effects of the present invention are not limited to the above-described effects, and potential effects expected by the technical features of the present invention will be clearly understood from the following description.

1 is a diagram illustrating an operation method of an infrastructure sensor device for sharing object sensing information according to an embodiment of the present invention.
2 is a diagram illustrating a road traffic system for sharing object sensing information according to an embodiment of the present invention.
3 is a diagram illustrating an operation method of an infrastructure sensor device for calculating absolute coordinates of an object according to an embodiment of the present invention.
4 is a diagram illustrating a conceptual diagram for calculating absolute coordinates of an object according to an embodiment of the present invention.
5 is a diagram illustrating a method of operating an object device for correcting a position of an object according to an embodiment of the present invention.
6 is a diagram illustrating a conceptual diagram for correcting the position of an object according to an embodiment of the present invention.
7 is a diagram illustrating a functional configuration of an infrastructure sensor device according to an embodiment of the present invention.
8 is a diagram illustrating a functional configuration of an object device according to an embodiment of the present invention.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail.

Various features of the invention disclosed in the claims may be better understood upon consideration of the drawings and detailed description. The apparatus, methods, preparations, and various embodiments disclosed herein are provided for purposes of illustration. The disclosed structural and functional features are intended to enable those skilled in the art to specifically practice the various embodiments, and are not intended to limit the scope of the invention. The terms and sentences disclosed are for the purpose of easy-to-understand descriptions of various features of the disclosed invention, and are not intended to limit the scope of the invention.

In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a method and apparatus for sharing object sensing information of an infrastructure sensor according to an embodiment of the present invention will be described.

1 is a diagram illustrating an operation method of an infrastructure sensor device 110 for sharing object sensing information according to an embodiment of the present invention. 2 is a diagram illustrating a road traffic system for sharing object sensing information according to an embodiment of the present invention.

1 and 2 , step S101 is a step of acquiring an ROI image.

Step S103 is a step of extracting at least one object 120 - 1 to 120 - 6 included in the ROI image. In an embodiment, using artificial intelligence technology, at least one object 120 - 1 to 120 - 6 and road marking lines such as lanes and crosswalks may be extracted from the image of the region of interest.

In an embodiment, an identifier (ID) may be assigned to each of the at least one extracted object 120 - 1 to 120 - 6 . Here, the at least one object 120 - 1 to 120 - 6 may include an object such as a car, a pedestrian, a bicycle, a falling object, etc. within the ROI.

Step S105 is a step of acquiring positioning information for at least one object 120-1 to 120-6 by using the absolute coordinates of each of the at least one object 120-1 to 120-6. For example, the positioning information may include at least one of an identifier for each of the at least one object 120-1 to 120-6, an absolute coordinate, a moving direction, a moving speed, an acceleration, and a capturing time of an image of the region of interest. can

In an embodiment, the absolute coordinates of each of the at least one object 120-1 to 120-6 may be calculated. In an embodiment, with respect to the object to which the identifier is assigned, the movement trajectory of the object may be tracked. In this case, the movement direction and movement speed of the object may be estimated by calculating the coordinate movement per reference time of the object based on the identifier of the object.

In an embodiment, the absolute coordinates of the road marking line included in the ROI image may be calculated by using the absolute coordinates of the road marking line of the map information.

In this case, absolute coordinates of all pixels of the region of interest image may be calculated by using the absolute coordinates of the road marking lines included in the region of interest image.

Thereafter, the absolute coordinates of each of the at least one object 120 - 1 to 120 - 6 matching the absolute coordinates of all pixels of the ROI image may be calculated. For example, absolute coordinates of all pixels of the region of interest image may be calculated by interpolating or extrapolating the absolute coordinates of road marking lines included in the region of interest image.

Also, at least one of a movement direction, a movement speed, and an acceleration of each of the at least one object 120-1 to 120-6 may be calculated using the absolute coordinates.

Step S107 is a step of transmitting positioning information to at least one object 120-1 to 120-6. For example, positioning information may be transmitted to at least one object 120-1 to 120-6 using a WSMP protocol of IEEE 802.11p/Wave or a sidelink of 3GPP V2X communication.

In an embodiment, the positioning information may be transmitted through a cooperative sensing message. In this case, each field of the cooperative sensing message may be defined as shown in Table 1 below, and the time at which the infrastructure sensor device 110 captures the ROI image may be input in the Time Stamp field.

Message ID Time Stamp object ID coordinate
(x,y) direction of movement
(dx,dy) speed
(vx,vy) acceleration
(ax,ay) One 2 . . . N

3 is a diagram illustrating an operation method of the infrastructure sensor device 110 for calculating the absolute coordinates of an object according to an embodiment of the present invention. 4 is a diagram illustrating a conceptual diagram for calculating absolute coordinates of an object according to an embodiment of the present invention.

Referring to FIGS. 3 and 4 , step S301 is a step of calculating the absolute coordinates of the road marking lines included in the ROI image by using the absolute coordinates of the road marking lines of the map information. In an embodiment, the map information may mean a high-precision map acquired offline. For example, referring to FIG. 4 , the absolute coordinates of the road marking line of the ROI image may be mapped.

Step S303 is a step of calculating absolute coordinates for all pixels of the region of interest image by using the absolute coordinates of the road marking lines included in the region of interest image. In an embodiment, the absolute coordinates of all pixels of the region of interest image may be calculated by interpolating or extrapolating the absolute coordinates of the road marking lines included in the region of interest image. For example, referring to FIG. 4 , absolute coordinates may be calculated by dividing latitude and longitude for each pixel of an ROI image.

Step S305 is a step of calculating the absolute coordinates of each of the at least one object 120-1 to 120-6 matching the absolute coordinates of all pixels of the region of interest image. That is, in all pixels of the ROI image, absolute coordinates corresponding to pixels of at least one object 120 - 1 to 120 - 6 may be matched 1:1 to calculate the absolute coordinates of each object.

5 is a diagram illustrating an operation method of the object device 120-1 for correcting the position of an object according to an embodiment of the present invention. 6 is a diagram illustrating a conceptual diagram for correcting the position of an object according to an embodiment of the present invention.

5 and 6 , step S501 is a step of receiving positioning information on at least one object 120 - 1 to 120 - 4 from the infrastructure sensor device. In this case, the at least one object 120 - 1 to 120 - 4 may include the object device 120 - 1 and surrounding objects 120 - 2 to 120 - 4 . In an embodiment, the positioning information may be obtained using the absolute coordinates of each of the at least one object 120-1 to 120-4.

In step S503, the second location 620 of the at least one object 120-1 to 120-4 calculated by the object device 120-1 and the at least one object 120-1 to 120-1 included in the positioning information 120-4) of the first position 610 is compared.

In an embodiment, the object device 120-1 is configured to provide a second position ( 620) can be calculated.

In step S505, when the difference between the first location 610 and the second location 620 is greater than a threshold value, the first location 610 is recognized as the location of the at least one object 120-1 to 120-4. is a step to For example, in the case of the object 120-2 and the object 120-4, since the difference between the first location 610 and the second location 620 is greater than a threshold value, the object 120-2 and the object 120-4 Positioning error correction for recognizing the position of (120-4) may be performed.

Accordingly, location and movement information of itself and surrounding objects calculated by the object device 120-1 through its own sensor and the location of at least one object 120-1 to 120-6 received from the infrastructure sensor device 110 And by comparing the movement information, the object device 120 - 1 may correct a positioning error with respect to its own position and the position of another object.

In addition, it is possible to recognize an object located in a blind spot through such object position recognition. For example, since the object 120-1 and the object 120-2 are located in the blind area of the intersection, it is impossible to recognize each object with its own sensor, but the infrastructure sensor device 110 detects the object 120- 1) and the object 120-2 are recognized, and location information is transmitted to the object 120-1 and the object 120-2, so the object 120-1 and the object 120-2 are cooperatively sensed. ) can recognize each other.

In an embodiment, when the difference between the first position 610 and the second position 620 is less than a threshold value, positioning error correction may not be performed. In this case, the first location 610 or the second location 620 may be recognized as the location of the at least one object 120-1 to 120-6. For example, in the case of the object 120-1 and the object 120-3, since the difference between the first location 610 and the second location 620 is smaller than a threshold value, the object 120-1 and the object 120-3 Positioning error correction for recognizing the position of (120-3) may not be performed.

7 is a diagram illustrating a functional configuration of the infrastructure sensor device 110 according to an embodiment of the present invention.

Referring to FIG. 7 , the infrastructure sensor device may include a sensor unit 710 , a storage unit 720 , a control unit 730 , and a communication unit 740 .

The sensor unit 710 may acquire an ROI image including at least one object 120 - 1 to 120 - 6 . In an embodiment, the sensor unit 710 may include at least one of a camera, a radar, and a lidar.

The storage unit 720 may store map information. In one embodiment, the map information may be delivered and stored offline or via a wireless communication network.

In an embodiment, the storage unit 720 may be configured as a volatile memory, a non-volatile memory, or a combination of a volatile memory and a non-volatile memory. In addition, the storage unit 720 may provide the stored data according to the request of the control unit 730 .

The controller 730 may process the ROI image to extract at least one object 120 - 1 to 120 - 6 . In one embodiment, the controller 730 may include an image processing embedded. The controller 730 may acquire positioning information on the at least one extracted object 120-1 to 120-6.

In an embodiment, the controller 730 may include at least one processor or microprocessor, or may be a part of the processor. Also, the controller 730 may be referred to as a communication processor (CP). The controller 730 may control each operation of the infrastructure sensor device 110 according to various embodiments of the present disclosure.

The communication unit 740 may transmit positioning information to the at least one object 120-1 to 120-6. In an embodiment, the communication unit 740 may include at least one of a wired communication module and a wireless communication module. All or part of the communication unit 740 may be referred to as a 'transmitter', 'receiver', or 'transceiver'.

8 is a diagram illustrating a functional configuration of an object device 120-1 according to an embodiment of the present invention.

Referring to FIG. 8 , the object device 120 - 1 may include a sensor unit 810 , a communication unit 820 , and a control unit 830 .

The sensor unit 810 may detect at least one object 120-1 to 120-6. That is, the object device 120 - 1 may detect not only itself but also surrounding objects through its sensor unit 810 . In an embodiment, the sensor unit 810 may include at least one of a camera, a radar, and a lidar.

The communication unit 820 may receive positioning information on at least one object 120 - 1 to 120 - 6 from the infrastructure sensor device 110 .

In an embodiment, the communication unit 820 may include at least one of a wired communication module and a wireless communication module. All or part of the communication unit 820 may be referred to as a 'transmitter', 'receiver', or 'transceiver'.

The controller 830 may calculate the positions of the at least one object 120 - 1 to 120 - 6 by using the positioning information. In an embodiment, the controller 830 controls the at least one object 120-1 included in the location and positioning information of the at least one object 120-1 to 120-6 calculated by the object device 120-1. to 120-6) may be compared to perform positioning error correction.

In an embodiment, the controller 830 may include at least one processor or microprocessor, or may be a part of the processor. Also, the controller 830 may be referred to as a communication processor (CP). The controller 830 may control each operation of the object device 120 - 1 according to various embodiments of the present disclosure.

The functional configuration of the object device 120-1 described in FIG. 8 may be equally applied to the functional configuration of the at least one neighboring objects 120-2 to 120-6.

The above description is merely illustrative of the technical spirit of the present invention, and various changes and modifications may be made by those skilled in the art without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present specification are not intended to limit the technical spirit of the present invention, but to illustrate, and the scope of the present invention is not limited by these embodiments.

The protection scope of the present invention should be interpreted by the claims, and all technical ideas within the scope equivalent thereto should be understood to be included in the scope of the present invention.

110: infrastructure sensor device
120-1 to 120-6: object
610: first position
620: second position

Claims (18)

acquiring an image of a region of interest;
extracting at least one object included in the region of interest image;
calculating the absolute coordinates of the road marking lines included in the ROI image by using the absolute coordinates of the road marking lines of the map information;
calculating absolute coordinates of all pixels of the region of interest image by using absolute coordinates of road marking lines included in the region of interest image;
calculating absolute coordinates of each of the at least one object matching the absolute coordinates of all pixels of the region of interest image;
calculating positioning information including at least one of a moving direction, a moving speed, and an acceleration of each of the at least one object by using the absolute coordinates; and
transmitting the positioning information to the at least one object;
including,
The transmitted positioning information is used for each of the at least one object to recognize the location of each other,
How infrastructure sensor devices work.
According to claim 1,
After extracting the at least one object,
allocating an identifier (ID) to each of the at least one extracted object;
further comprising,
How infrastructure sensor devices work.
According to claim 1,
The positioning information includes at least one of an identification (ID), absolute coordinates, moving direction, moving speed, acceleration, and capturing time of the region of interest image for each of the at least one object,
How infrastructure sensor devices work.
delete delete According to claim 1,
Calculating absolute coordinates for all pixels of the region of interest image includes:
calculating absolute coordinates for all pixels of the region of interest image by interpolating or extrapolating the absolute coordinates of road marking lines included in the region of interest image;
containing,
How infrastructure sensor devices work.
Receiving positioning information for at least one object from the infrastructure sensor device;
comparing the second position of the at least one object calculated by the object device with the first position of the at least one object included in the positioning information; and
calculating the first location as a location of the at least one object when a difference between the first location and the second location is greater than a threshold value;
including,
The positioning information is obtained using the absolute coordinates of each of the at least one object,
The first position includes the absolute coordinates of each of the at least one object,
How the object device works.
8. The method of claim 7,
The positioning information includes at least one of an identification (ID), absolute coordinates, moving direction, moving speed, acceleration, and capturing time of an ROI image including the at least one object for each of the at least one object doing,
How the object device works.
delete a sensor unit for acquiring an image of a region of interest;
extracting at least one object included in the region of interest image,
calculating the absolute coordinates of the road marking lines included in the image of the region of interest by using the absolute coordinates of the road marking lines of the map information,
calculating absolute coordinates for all pixels of the region of interest image by using the absolute coordinates of road marking lines included in the region of interest image,
calculating the absolute coordinates of each of the at least one object matching the absolute coordinates of all pixels of the region of interest image,
a control unit for calculating positioning information including at least one of a moving direction, a moving speed, and an acceleration of each of the at least one object by using the absolute coordinates; and
a communication unit for transmitting the positioning information to the at least one object;
including,
The transmitted positioning information is used for each of the at least one object to recognize the location of each other,
infrastructure sensor devices.
11. The method of claim 10,
The control unit is
Allocating an identifier (identification, ID) to each of the extracted at least one object,
infrastructure sensor devices.
11. The method of claim 10,
The positioning information includes at least one of an identification (ID), absolute coordinates, moving direction, moving speed, acceleration, and capturing time of the region of interest image for each of the at least one object,
infrastructure sensor devices.
delete delete 11. The method of claim 10,
The control unit is
calculating absolute coordinates for all pixels of the region of interest image by interpolating or extrapolating the absolute coordinates of road marking lines included in the region of interest image;
infrastructure sensor devices.
a communication unit for receiving positioning information on at least one object from the infrastructure sensor device;
a sensor unit for calculating a second position of the at least one object; and
comparing the second position of the at least one object with the first position of the at least one object included in the positioning information,
a control unit configured to calculate the first position as a position of the at least one object when a difference between the first position and the second position is greater than a threshold value;
including,
The positioning information is obtained using the absolute coordinates of each of the at least one object,
The first position includes the absolute coordinates of each of the at least one object,
object device.
17. The method of claim 16,
The positioning information includes at least one of an identification (ID), absolute coordinates, moving direction, moving speed, acceleration, and capturing time of an ROI image including the at least one object for each of the at least one object doing,
object device.
delete

Images