JP6996368B2 - Server, obstacle recognition device - Google Patents

Description

The present disclosure relates to a technique for recognizing an obstacle in the traveling direction of a mobile device.

In Patent Document 1, the position information of the vehicle and the characteristic amount of the driving operation of the driver are acquired by using the switching of the driving of the vehicle from the automatic driving mode to the manual driving mode as a trigger, and the automatic driving mode is used by using these. It is described that the correction amount of the driving operation in the above is determined and the driving operation in the automatic operation mode is corrected by using the correction amount.

International Publication No. 2015/159341

When the situation around the vehicle is detected by a sensor and the detection result is used to recognize an obstacle that should avoid a collision with the vehicle for automatic driving, it is preferable that the recognition accuracy of the obstacle is high. In order to improve the recognition accuracy of obstacles, for example, using the relationship between the driver's driving operation in the manual operation mode and the sensor detection result, whether or not the sensor detection result in the automatic operation mode indicates an obstacle. It is also possible to judge whether or not. However, if the above relationship cannot be obtained, for example, when the driver is not operated, it cannot be determined whether or not the detection result of the sensor indicates an obstacle. Therefore, a new technique capable of improving the recognition accuracy of obstacles has been required.

The present disclosure has been made to solve the above-mentioned problems, and can be realized as the following forms.

According to one embodiment of the present disclosure, the likelihood that an obstacle is present in the traveling direction, which is calculated based on the detection result of the target in the traveling direction of the moving device, is the first threshold indicating that there is no obstacle. Even below, the above is not greater than or equal to the second threshold value larger than the first threshold value indicating the presence of the obstacle, and is larger than the first threshold value indicating the presence of the provisional obstacle, which is to provisionally determine that the presence of the obstacle is present. When the value becomes smaller than the second threshold value, the detection result and the calculation result of the likelihood are received from the obstacle recognition device, the correspondence relationship between the target and the obstacle, and the object in the detection result. The provisional obstacle received from the correction unit (220) that corrects the presence of the provisional obstacle to the absence of the obstacle or the presence of the obstacle by using the similarity with the mark, and the obstacle recognition device. A storage unit (240) that stores the detection result and the calculation result corresponding to the presence of an object, and the correction result of the provisional obstacle with the correction unit as data for supervised learning, and the storage unit. The learning unit (230) that performs supervised learning using the data for supervised learning stored in the unit and reconstructs the likelihood calculation logic, and the likelihood calculation logic are applied to the obstacle recognition device. It is provided with an update unit (245) for updating. At least one of the correction result by the correction unit and the likelihood calculation logic is transmitted to the obstacle recognition device.
According to the second aspect of the present disclosure, an obstacle recognition device (60) mounted on a mobile mobile device (100) and capable of communicating with a server (250) is provided. This obstacle recognition device acquires the detection result of the target in the traveling direction of the moving device, and determines in advance the likelihood that an obstacle that needs to avoid collision with the moving device exists in the traveling direction. The likelihood calculation unit (40) that calculates using the likelihood calculation logic, and if the likelihood is equal to or less than the first threshold value (X), it is determined that there is no obstacle, and the likelihood is the first threshold value. If it is greater than or equal to the second threshold value (Y), it is determined that there is an obstacle, and if the likelihood is larger than the first threshold value and smaller than the second threshold value, it is provisionally determined that there is an obstacle. It is provided with a judgment unit (50), which determines that there is a provisional obstacle. When the determination unit determines that there is a provisional obstacle, the detection result and the calculation result of the likelihood calculation unit are transmitted to the server, and the determination that there is a provisional obstacle is that there is no obstacle. Alternatively, the correction result corrected with the obstacle is received from the server, the likelihood calculation logic reconstructed using the detection result and the calculation result is received from the server, and the likelihood calculation unit receives the correction result. The likelihood calculation logic to be used is updated with the likelihood calculation logic received from the server .

According to this form, when the likelihood is larger than the first threshold value and smaller than the second threshold value, it is determined that there is a provisional obstacle, so that the safety in recognizing the obstacle can be enhanced, and the provisional obstacle can be improved. Since the imaging result and the likelihood calculation result when it is determined to be present are acquired and corrected, the communication load between the obstacle recognition device and the server is reduced as compared with the case where all the imaging results and the likelihood calculation results are acquired. Can be mitigated. Further, since the detection result and the likelihood calculation result when it is determined that there is a temporary obstacle and the correction result with the provisional obstacle are stored in the storage unit, the recognition accuracy of the obstacle is improved. It is possible to efficiently collect the learning data for making the result. Furthermore, since supervised learning is performed using the data when it is determined that there is a provisional obstacle, suspicious data, for example, because the frequency of occurrence is low, is considered in the initial likelihood calculation logic in the likelihood calculation unit. It is possible to efficiently learn the data of the scenes that did not exist. In addition, the reconstructed likelihood calculation logic updates the likelihood calculation logic of the obstacle recognition device, so that the frequency of determining that there is a provisional obstacle in the obstacle recognition device is reduced, and the obstacle is determined. The recognition accuracy can be improved.

The figure for demonstrating the structure of an obstacle recognition system. The figure which shows mainly the process on the vehicle side in the obstacle recognition process. A diagram showing mainly server-side processing among obstacle recognition processing. The image figure which shows the detection result of the 1st sensor. An image diagram of the process in which the first likelihood calculation unit calculates the first likelihood. An image diagram of the process in which the second likelihood calculation unit calculates the second likelihood. An image diagram of the process in which the third likelihood calculation unit calculates the third likelihood. The image figure which shows the detection range of a sensor part. Image diagram showing the likelihood synthesis result. The figure which shows the correction result of the correction part. The figure which shows the learning process. Image diagram showing the effect of learning. Image diagram showing the effect of learning.

-Embodiment <Structure of obstacle recognition system>
The configuration of the obstacle recognition system 10 will be described with reference to FIG. The obstacle recognition system 10 includes an obstacle recognition device 60 and a server 250. The obstacle recognition device 60 is mounted on a mobile device having a mobile function. In this embodiment, the mobile device is a vehicle 100. In the present embodiment, the server 250 is provided in the driving support center 200 that manages a plurality of vehicles 100, and is configured to be able to communicate with the plurality of vehicles 100.

The vehicle 100 includes a sensor unit 30 and a driving support control unit 20 in addition to the obstacle recognition device 60.

The sensor unit 30 detects a target in the traveling direction of the vehicle 100. The traveling direction includes the moving direction of the vehicle 100 and its surroundings. The traveling direction includes the front direction and its surroundings when the vehicle 100 moves forward, and includes the rear direction and its surroundings when the vehicle 100 moves backward. For example, the sensor unit 30 can detect a vehicle existing in a lane adjacent to the traveling lane of the vehicle 100. In the present embodiment, the sensor unit 30 includes a first sensor 31, a second sensor 32, and a third sensor 33. The first sensor 31 is an image pickup device that captures an image of the surroundings of the vehicle 100, and is a monocular camera in the present embodiment. In other embodiments, the image pickup device may be a stereo camera. The second sensor 32 is LIDAR (Light Detection And Ranging or Laser Imaging Detection And Ranging), and detects the presence or absence of a target in the traveling direction of the vehicle 100 by using a laser. The third sensor 33 is a millimeter wave sensor, and uses radio waves in the millimeter wave band to determine the presence or absence of a target in the traveling direction of the vehicle 100, the distance between the target and the vehicle 100, the position of the target, and the size of the target. Now, the shape of the target and the relative speed of the target with respect to the vehicle 100 are detected. In another embodiment, the sensor unit 30 may be composed of only the first sensor 31, or may include a sensor other than the first sensor 31, the second sensor 32, and the third sensor 33.

The obstacle recognition device 60 is composed of an ECU (Electronic Control Unit) including a CPU, a memory, and an input / output interface for transmitting and receiving signals. The CPU functions as the likelihood calculation unit 40 by expanding and executing the likelihood calculation program stored in the memory. Further, the CPU functions as the determination unit 50 by expanding and executing the determination program stored in the memory.

The likelihood calculation unit 40 acquires the detection result of the sensor unit 30, and calculates the likelihood that an obstacle exists in the traveling direction of the vehicle 100 by the likelihood calculation logic. The "obstacle" refers to a target that needs to avoid a collision with the vehicle 100. In the present embodiment, the likelihood calculation unit 40 includes a first likelihood calculation unit 41, a second likelihood calculation unit 42, and a third likelihood calculation unit 43. The first likelihood calculation unit 41 calculates the first likelihood by using the likelihood calculation logic in which the relationship between the detection result of the first sensor 31 and the likelihood is defined. Similarly, the second likelihood calculation unit 42 calculates the second likelihood using the likelihood calculation logic in which the relationship between the detection result of the second sensor 32 and the likelihood is defined. The third likelihood calculation unit 43 calculates the third likelihood by using the likelihood calculation logic in which the relationship between the detection result of the third sensor 33 and the likelihood is defined. The likelihood calculation unit 40 calculates the likelihood of the first likelihood, the second likelihood, and the third likelihood by using a predetermined combined likelihood calculation logic. The likelihood synthesized by the likelihood calculation unit 40 is also referred to as "synthetic likelihood".

The determination unit 50 acquires the likelihood calculated by the likelihood calculation unit 40. The determination unit 50 determines that there is no obstacle when the likelihood is equal to or less than the predetermined first threshold value X, and when the likelihood is greater than or equal to the first threshold value X, the determination unit 50 determines that the likelihood is greater than or equal to the second threshold value Y. When it is judged that there is an obstacle and the likelihood is larger than the first threshold value X and smaller than the second threshold value Y, it is judged that there is a temporary obstacle. .. A value larger than the first threshold value X and smaller than the second threshold value Y is also referred to as an "intermediate value". In the present embodiment, the likelihood that the determination unit 50 determines is the synthetic likelihood.

The obstacle recognition device 60 transmits the recognition result by the obstacle recognition device 60 to the driving support control unit 20. The obstacle recognition device 60 is configured to be able to acquire the position of the obstacle from the detection result of the sensor unit 30, and when it is determined that there is an "obstacle", the signal indicating "there is an obstacle" and the corresponding object. The position of the obstacle is transmitted to the driving support control unit 20. In the present embodiment, when the obstacle recognition device 60 determines that "there is a temporary obstacle", the signal indicating "there is an obstacle" and the position of the obstacle are transmitted to the driving support control unit 20. Send. When the obstacle recognition device 60 determines that there is no obstacle, it does not have to transmit a signal to the driving support control unit 20, and there is no obstacle and a signal indicating "no obstacle". And may be sent.

The driving support center 200 includes a server 250 and a display unit 210 connected to the server 250. The server 250 includes a CPU, a memory, an input / output interface for transmitting and receiving signals, and a storage unit 240. The CPU functions as a correction unit 220, a learning unit 230, and an update unit 245 by expanding and executing a program stored in the memory.

The display unit 210 displays various images, texts, and the like output from the server 250. The display unit 210 also functions as a reception unit that receives a touch operation from the administrator of the driving support center 200 with a finger or a pen. The acceptance result of the display unit 210 is transmitted to the server 250.

The correction unit 220 acquires the image pickup result of the image pickup apparatus and the calculation result of the likelihood calculation unit 40 corresponding to "there is a provisional obstacle". The correction unit 220 corrects "with provisional obstacles" to "without obstacles" or "with obstacles". The correction is performed using the degree of similarity between the correspondence between the target stored in the memory and the obstacle and the target appearing in the imaging result. The correction unit 220 is configured to be able to determine the degree of similarity between the target appearing in the image pickup result and the target indicating an obstacle in the above-mentioned correspondence relationship, and when the degree of similarity is equal to or higher than a predetermined threshold value. , The target appearing in the imaging result is judged to be an obstacle. When the correction unit 220 corrects the judgment of "there is a provisional obstacle" by the judgment unit 50, the correction unit 220 transmits the correction result to the obstacle recognition device 60.

The storage unit 240 includes the detection result of the sensor unit 30 and the calculation result of the likelihood calculation unit 40 corresponding to the "provisional obstacle", and the correction result of the correction unit 220 correcting the "provisional obstacle". , Is stored as data for supervised learning. The storage unit 240 is also a database for accumulating data for supervised learning.

The learning unit 230 reconstructs the synthetic likelihood calculation logic by executing supervised learning using the data of the storage unit 240. The update unit 245 updates the composite likelihood calculation logic of the likelihood calculation unit 40 using the learning result.

<Obstacle recognition processing>
The obstacle recognition process shown in FIGS. 2 and 3 is repeatedly executed while the obstacle recognition system 10 is on.

First, the processing on the vehicle 100 side will be described. As shown in FIG. 2, the likelihood calculation unit 40 calculates the likelihood that an obstacle exists in the traveling direction of the vehicle 100 by using the detection result of the sensor unit 30 (step S10).

As shown in FIG. 4, the first sensor 31 detects the targets S1, S2, S3, S4 and the position m1 of the lane marker such as the white line, the white broken line, and the yellow line. As shown in FIG. 5, the first likelihood calculation unit 41 divides the detection result of the first sensor 31 into a plurality of regions and calculates the first likelihood for each region. Specifically, the first likelihood calculation unit 41 executes semantic segmentation in which each pixel in the detection result is assigned to the divided grid and the first likelihood is calculated for each grid. The number of grids may be determined according to the detection sensitivity of the sensor unit 30. Here, the target S1 is paper, the target S2 and S4 are vehicles, and the target S3 is a person. However, in the likelihood calculation logic of the first likelihood calculation unit 41, the target S2 and S4 are described. It is recognized as a vehicle from the shape, the first likelihood is calculated as the first threshold value X or more, and the first likelihood is set as an intermediate value for the grid in which the targets S1 and S3 are present.

Similarly, the second likelihood calculation unit 42 divides the detection result of the second sensor 32 into grids as shown in FIG. 6, and calculates the second likelihood for each grid. The third likelihood calculation unit 43 divides the detection result of the third sensor 33 into grids as shown in FIG. 7, and calculates the third likelihood for each grid.

As shown in FIG. 8, the sensor unit 30 can detect the targets S1, S2, S3, and S4 in a predetermined range in the traveling direction of the vehicle 100, and the first likelihood calculation unit 41 and the second likelihood calculation unit. 42, the first likelihood, the second likelihood, and the third likelihood calculated by the third likelihood calculation unit 43 are the likelihoods in a predetermined range in the traveling direction of the vehicle 100. In the present embodiment, the likelihood calculation unit 40 calculates the composite likelihood in one of the vertical cross sections in the traveling direction indicated by the arrow in FIG. 8 in which the target is detected. In the present embodiment, the likelihood calculation unit 40 calculates the combined likelihood in the vertical cross section at the distance Dst_1 where the targets S1, S2, and S3 exist. The vertical cross section at the distance Dst_1 is the vertical cross section closest to the vehicle 100 among the vertical cross sections in which the target is detected. The likelihood calculation unit 40 may calculate the composite likelihood of the vertical cross section at the distance Dst_1 and then calculate the composite likelihood of the vertical cross section at the distance Dst_1 where the target S4 exists.

The determination unit 50 acquires the composite likelihood calculated by the likelihood calculation unit 40, determines that there is no obstacle when the composite likelihood is equal to or less than the first threshold value X, and determines that the composite likelihood is the second threshold value. If it is Y or more, it is determined that there is an obstacle, and if the composite likelihood is an intermediate value, it is determined that there is a provisional obstacle (FIG. 2, step S15).

As shown in FIG. 9, the likelihood calculation unit 40 sets each likelihood calculated by the first likelihood calculation unit 41, the second likelihood calculation unit 42, and the third likelihood calculation unit 43 vertically at the distance Dst_1. Synthesize in cross section. The likelihood calculation unit 40 defines a predetermined synthetic likelihood calculation logic that defines the relationship between the situation of the vehicle 100 at the time of detection of each sensor and the coefficient multiplied by each likelihood of each likelihood calculation unit. Is used to calculate the composite likelihood. The situation of the vehicle 100 at the time of detection by each sensor is, for example, time and weather. In the combined likelihood calculation logic, for example, for a sensor whose detection sensitivity decreases in a certain time zone or weather, the likelihood corresponding to the detection result of the sensor is weighted more than the likelihood corresponding to the detection result of another sensor. It may be specified to be smaller.

The grid S2b shown in FIG. 9 corresponds to the target S2 at the distance Dst_1, and is determined to have “obstacles”. The grids S1b and S3b correspond to the targets S1 and S3, respectively, and are determined to have "provisional obstacles". The other grids are judged to be "no obstacles". Since the target S4 shown in FIGS. 4 to 8 is located in the vertical cross section at the distance Dst_2, it does not appear in FIG.

If there is a grid of intermediate likelihood values, that is, if there is a grid that is determined by the determination unit 50 to have "provisional obstacles" (FIG. 2, step S20), the obstacle recognition device 60 is the server 250. A signal indicating a connection request is transmitted to (step S40).

When the obstacle recognition device 60 determines that there is a "temporary obstacle" and receives a signal indicating "there is an obstacle" and the position of the obstacle, the driving support control unit 20 provides driving support. It is determined whether or not it can be continued (step S50). The case where driving support can be continued is, for example, a case where the behavior of a target determined as an obstacle until after a predetermined period has elapsed can be predicted from the direction of the target and the relative speed with respect to the vehicle 100. .. The case where it is impossible to continue driving support is, for example, a case where the behavior of a target determined as an obstacle cannot be predicted until after a predetermined period of time has elapsed. When the obstacle recognition device 60 determines that "there is a temporary obstacle" and the driving support control unit 20 receives the signal indicating "there is an obstacle" and the position of the obstacle, step S50 is performed. There is a possibility that it will be judged negative. When the step S50 is NO, the driving support control unit 20 transmits a signal indicating that the driving support cannot be continued to the obstacle recognition device 60.

Upon receiving the signal that the driving support cannot be continued, the obstacle recognition device 60 waits until the correction result by the correction unit 220 of the server 250 is received (step S60, NO). The driving support control unit 20 may stop the vehicle 100 while waiting. Alternatively, the driving support control unit 20 may temporarily turn off the automatic driving function and notify the occupants of the vehicle 100 that the automatic driving function is turned off.

When the obstacle recognition device 60 receives the correction result from the server 250, the obstacle recognition device 60 controls the operation support control of the signal indicating "with obstacle" or "without obstacle" and the position of the obstacle according to the correction result. It is transmitted to the unit 20. When there is no grid of intermediate likelihood values (step S20, NO), when driving support can be continued even if there is a grid of intermediate likelihood values (step S50, YES), or when the correction result is received. (Step S60, YES), the driving support control unit 20 uses, for example, the information indicating “there is an obstacle” and the position information of the obstacle transmitted from the obstacle recognition device 60 to support driving. Control is executed (step S80).

Next, the processing on the server 250 side will be described. As shown in FIG. 3, when there is a connection request from the obstacle recognition device 60 (step S100, YES), the server 250 executes correction preparation (step S110). The correction preparation is, for example, to acquire the image pickup result of the image pickup device and the calculation result of the likelihood calculation unit 40 corresponding to "there is a provisional obstacle" from the obstacle recognition device 60. In step S110, the server 250 may further call the observer of the driving support center 200.

The correction unit 220 corrects "there is a provisional obstacle" by the judgment unit 50 by using the correspondence relationship between the target and the obstacle (step S120). The correction unit 220 superimposes the image pickup result and the composite likelihood so that the image pickup result and the position m1 shown in FIG. 9 match. The correction unit 220 has a "provisional obstacle" when the similarity between the grid target determined to be "provisional obstacle" and the target in the above correspondence is equal to or higher than a predetermined threshold value. Correct the grid determined to be "with obstacles". When the similarity is less than the threshold value, the correction unit 220 corrects the grid determined to have "provisional obstacles" to "no obstacles".

The correction unit 220 may display an image in which the imaging result and the composite likelihood shown in FIG. 9 are superimposed on the display unit 210. The observer judges the similarity between the predetermined correspondence between the target and the obstacle and the target appearing in the imaging result, and displays the grid that is judged to have "provisional obstacle". It may be modified using 210. The correction unit 220 may acquire the correction result by the observer and position the correction result by the observer as the correction result by the correction unit 220.

Comparing the likelihood synthesis result shown in FIG. 9 with the correction result by the correction unit 220 shown in FIG. 10, the lattice S1b determined to have "provisional obstacles" is "disordered" by the correction unit 220. The grid S3b, which has been corrected to "no object" and determined to have "provisional obstacles", has been corrected to "with obstacles". The lattice S3b is further provided with an attribute indicating "first behavior" by the correction unit 220. The first behavior is a behavior in which it is more difficult to predict the behavior of an obstacle after a predetermined period of time has elapsed, as compared with other obstacles. For example, when the obstacle is a vehicle, it is possible to predict the position after a predetermined period has elapsed from the direction of the vehicle, the speed of the vehicle, and the like. However, when the obstacle is a person, an animal, or a running bicycle, it is more difficult to predict the position after a predetermined period from the direction and speed than the vehicle. The correction unit 220 refers to the correspondence between the obstacle and the attribute stored in the memory, and assigns the attribute to such an obstacle as an obstacle that needs special interpretation.

When the correction is completed, the correction unit 220 transmits the correction result to the obstacle recognition device 60 (step S130). When the obstacle recognition device 60 receives the correction result, the above-mentioned processes of steps S60 to S80 (FIG. 2) are executed.

Further, when the correction is completed, the storage unit 240 supervisedly learns the detection result of the sensor unit 30, the calculation result of the likelihood calculation unit 40, and the correction result of "provisional obstacle" by the correction unit 220. It is stored as data for (step S140).

<Learning process>
Next, the learning process executed in the obstacle recognition system 10 will be described. The learning process is repeatedly executed in parallel with the above-mentioned obstacle recognition process. As shown in FIG. 11, the learning unit 230 determines whether or not it is the learning timing (step S200). The learning timing is, for example, a timing when the number of data newly stored in the storage unit 240 becomes a predetermined number or more, or a timing when a predetermined time has elapsed from the previous learning. The learning timing may be the timing at which the data to which the attribute is attached is stored.

When it is the learning timing (step S200, YES), the learning unit 230 executes supervised learning using the data of the storage unit 240 (step S210). Specifically, in the learning unit 230, when "with provisional obstacle" is corrected to "without obstacle", the calculation result of the likelihood calculation unit 40 corresponding to "with provisional obstacle" is the first. The calculation result of the likelihood calculation unit 40 corresponding to "provisional obstacle" is obtained so that the threshold value is X or less, and when "provisional obstacle" is corrected to "provisional obstacle". The composite likelihood calculation logic is reconstructed so as to be equal to or higher than the second threshold value Y.

When learning is performed by the learning unit 230, the update unit 245 determines whether or not it is the update timing (step S220). The update timing is, for example, the timing at which the server 250 can communicate with the obstacle recognition device 60. The update timing may be the timing when the driving support function by the driving support control unit 20 is turned off. Alternatively, the driving support function by the driving support control unit 20 may be turned on and the vehicle 100 may be stopped at the timing.

When it is the update timing (step S220, YES), the update unit 245 distributes the synthetic likelihood calculation logic reconstructed by the learning unit 230 to the obstacle recognition device 60, and the likelihood calculation unit 40 The composite likelihood calculation logic is updated (step S230). When there are a plurality of vehicles 100 capable of communicating with the server 250, the update unit 245 may deliver the update result to the plurality of vehicles 100.

If it is not the learning timing (step S200, NO), or if it is not the update timing (step S220, NO), the server 250 does not perform learning (step S210) or update (step S230), and processes in step S200. Return to.

The effect of learning will be explained. The vertical axis of FIG. 12 is the composite likelihood, and the horizontal axis is the grid position. As shown in FIG. 12, for the target detected at a certain position, even if the synthetic likelihood that is larger than the first threshold value X and smaller than the second threshold value Y is calculated before learning, the first is as the learning progresses. The synthetic likelihood of the threshold value X or less or the synthetic likelihood of the second threshold value Y or more is calculated.

The vertical axis of FIG. 13 is the composite likelihood, and the horizontal axis is the composite likelihood appearance rate. As shown in FIG. 13, as the learning progresses, the appearance rate of the synthetic likelihood larger than the first threshold value X and smaller than the second threshold value Y decreases, and the synthetic likelihood of the first threshold value X or more or the second threshold value Y or more decreases. The appearance rate of degrees increases.

According to this embodiment, when the likelihood is larger than the first threshold value X and smaller than the second threshold value Y, it is determined that there is a provisional obstacle, so that the safety in recognizing the obstacle can be enhanced. Further, since the imaging result and the likelihood calculation result when it is determined that there is a temporary obstacle are acquired and corrected, the obstacle recognition device 60 is compared with the case where all the imaging results and the likelihood calculation results are acquired. And the communication load between the server 250 and the server 250 can be reduced. Further, since the storage unit 240 stores the detection result and the likelihood calculation result when it is determined that there is a provisional obstacle, and the correction result with the provisional obstacle, the recognition accuracy of the obstacle can be determined. It is possible to efficiently collect learning data for improvement. Furthermore, since supervised learning is performed using the data when it is determined that there is a provisional obstacle, suspicious data, for example, because the frequency of occurrence is low, is considered in the initial likelihood calculation logic in the likelihood calculation unit 40. It is possible to efficiently learn the data of the scenes that did not exist. Further, since the likelihood calculation logic of the obstacle recognition device 60 is updated by the reconstructed likelihood calculation logic, the frequency of determining that there is a provisional obstacle in the obstacle recognition device 60 is reduced, and the failure is reduced. It is possible to improve the recognition accuracy of an object.

Further, according to the above embodiment, since the composite likelihood is calculated from the likelihood of each sensor and the judgment is made using the composite likelihood, the obstacle recognition accuracy in the obstacle recognition device 60 can be improved.

Further, according to the above embodiment, the correction unit 220 corrects the divided area in the area determined to have "provisional obstacle", so that the load on the correction can be reduced.

Further, according to the above embodiment, it is possible to efficiently construct a database for storing supervised data for constructing the likelihood calculation logic.

-Other embodiment 1
In the above embodiment, the learning unit 230 may perform supervised learning by weighting the data to which the attribute is given among the data stored in the storage unit 240.

According to this form, for example, since the obstacle which is the first behavior is weighted and supervised learning is performed, it is possible to improve the learning efficiency for the obstacle whose behavior is difficult to predict, and it is possible to recognize the obstacle. It is possible to improve the recognition accuracy of obstacles in the device 60.

-Other embodiment 2
In the above embodiment, the sensor unit 30 is composed of a plurality of sensors, the likelihood calculation unit 40 synthesizes the likelihood of each sensor, and the determination unit 50 makes the above determination using the combined likelihood. On the other hand, when the sensor unit 30 is composed of one sensor, the determination unit 50 may make the above determination using one likelihood. The learning unit 230 may reconstruct the likelihood calculation logic instead of the synthetic likelihood calculation logic.

-Other embodiment 3
In the above embodiment, the learning unit 230 may reconstruct the combined likelihood calculation logic and the likelihood calculation logic of each sensor.

-Other embodiment 4
In the above embodiment, in the driving support center 200, the obstacle recognition is performed by the reconstructed likelihood calculation logic before the likelihood calculation logic reconstructed by the learning unit 230 is transmitted to the obstacle recognition device 60. Tests may be performed to confirm safety. The test may be performed using a simulation device or a test vehicle equipped with an obstacle recognition device 60 having an updated likelihood calculation logic.

-Other embodiment 5
In the above embodiment, the vehicle 100 is mentioned as the moving device, but other vehicles (for example, two-wheeled vehicles), drones, and robots may be used as long as they have a moving function. In the case of a robot, for example, it may be grounded on the road surface with wheels such as an automobile, or it may be a type that walks on two legs.

-Other embodiment 6
In the above embodiment, the likelihood calculation unit 40 calculates the likelihood in one vertical cross section in which a target is detected among the vertical cross sections of the vehicle 100 with respect to the traveling direction. On the other hand, the likelihood calculation unit 40 calculates the likelihood in a predetermined distance range in the traveling direction, and the correction unit 220 selects a vertical cross section using the image pickup result of the imaging device, and the likelihood calculation unit 40 Of the calculation results, the likelihood corresponding to the vertical cross section may be acquired and "there is a provisional obstacle" may be corrected.

The present disclosure can also be realized in various forms other than the obstacle recognition system 10. For example, an obstacle recognition device 60, an obstacle recognition method, a database construction method for storing data for supervised learning, a computer program for realizing such a method, a storage medium for storing the computer program, and obstacle recognition. It can be realized in the form of a mobile device or the like equipped with the device 60. Further, in the above embodiment, some or all of the functions and processes realized by the software may be realized by the hardware. In addition, some or all of the functions and processes realized by the hardware may be realized by the software. As the hardware, various circuits such as an integrated circuit, a discrete circuit, or a circuit module combining these circuits may be used.

10 Obstacle recognition system, 30 Sensor unit, 40 Likelihood calculation unit, 50 Judgment unit, 60 Obstacle recognition device, 100 Vehicle, 220 Correction unit, 230 Learning unit, 240 Storage unit, 245 Update unit, 250 Server

Claims (8)

A server (250) capable of communicating with an obstacle recognition device (60) mounted on a mobile mobile device (100).
Even if the likelihood of the presence of an obstacle in the traveling direction calculated based on the detection result of the target in the traveling direction of the moving device is equal to or less than the first threshold value indicating that there is no obstacle, the presence of the obstacle is detected. It is not greater than or equal to the second threshold value larger than the first threshold value, and is larger than the first threshold value and smaller than the second threshold value, indicating that there is a provisional obstacle, which is to provisionally determine that there is an obstacle. In that case, the detection result and the calculation result of the likelihood are received from the obstacle recognition device, and the degree of similarity between the correspondence between the target and the obstacle and the target in the detection result is used. The correction unit (220) that corrects the provisional obstacle to the absence of the obstacle or the presence of the obstacle,
The detection result and the calculation result corresponding to the provisional obstacle presence received from the obstacle recognition device, and the correction result with the provisional obstacle by the correction unit, are supervised learning. A storage unit (240) for storing data and
A learning unit (230) that performs supervised learning using the data for supervised learning stored in the storage unit and reconstructs the likelihood calculation logic.
An update unit (245) for updating the likelihood calculation logic to the obstacle recognition device is provided.
A server that transmits at least one of the correction result by the correction unit and the likelihood calculation logic to the obstacle recognition device. The server according to claim 1 .
In the learning unit, when the provisional obstacle is corrected without the obstacle, the calculation result corresponding to the provisional obstacle is equal to or less than the first threshold value, and the provisional obstacle is present. A server that reconstructs the likelihood calculation logic so that the calculation result corresponding to the provisional obstacle is equal to or higher than the second threshold value when is corrected to have the obstacle. The server according to claim 1 or 2 .
The correspondence further includes whether or not the behavior of the obstacle after the lapse of a predetermined period is the predetermined first behavior.
The correction unit further corrects the provisional obstacle to the obstacle, and the behavior of the obstacle corresponding to the provisional obstacle is the first behavior. The first behavior is given as an attribute of the obstacle.
The storage unit further stores the data for the supervised learning to which the attribute is given.
The learning unit is a server that weights the data for the supervised learning to which the attribute is given and performs the supervised learning. An obstacle recognition device (60) mounted on a mobile mobile device (100) and capable of communicating with a server (250).
The likelihood calculation unit (40) that acquires the detection result of the target in the traveling direction of the moving device and calculates the likelihood that an obstacle exists in the traveling direction by using a predetermined likelihood calculation logic. )When,
When the likelihood is equal to or less than the first threshold value (X), it is determined that there is no obstacle, and when the likelihood is greater than or equal to the second threshold value (Y), the obstacle is present. Judgment unit (50) for determining that there is a provisional obstacle, which is to provisionally determine that there is an obstacle when the likelihood is larger than the first threshold value and smaller than the second threshold value. ) And,
When the determination unit determines that there is a provisional obstacle, the detection result and the calculation result of the likelihood calculation unit are transmitted to the server.
The server receives the correction result in which the determination that the provisional obstacle exists is corrected to the absence of the obstacle or the presence of the obstacle.
The likelihood calculation logic reconstructed using the detection result and the calculation result is received from the server, and the likelihood calculation logic used by the likelihood calculation unit is updated to the likelihood calculation logic received from the server. , Obstacle recognition device. The obstacle recognition device according to claim 4 .
The likelihood calculation logic after the update is an obstacle recognition device in which the likelihood is less likely to be a value larger than the first threshold value and smaller than the second threshold value than the likelihood calculation logic before the update. The obstacle recognition device according to claim 4 or 5 .
A control unit (20) for determining whether or not to continue operation is provided based on the determination result by the determination unit.
When the determination unit determines that there is a provisional obstacle and the control unit determines that the operation cannot be continued, the mobile device is stopped or the movement is stopped until the correction result is received. An obstacle recognition device that turns off the automatic operation function of the device. The obstacle recognition device according to any one of claims 4 to 6 .
The detection result is a detection result of a plurality of sensors (31, 32, 33) that detect a target in the traveling direction.
The likelihood calculation unit synthesizes the likelihood calculated for each of the plurality of sensors.
The determination unit is an obstacle recognition device that makes the determination using the synthesized likelihood. The obstacle recognition device according to any one of claims 4 to 7 .
The likelihood calculation unit divides the detection result into a plurality of regions in the vertical cross section in which the target is detected among the vertical cross sections with respect to the traveling direction, and calculates the likelihood for each of the divided regions. ,
The judgment unit makes the judgment for each area and makes the judgment.
An obstacle recognition device that transmits the likelihood of a region determined to have a provisional obstacle among the plurality of regions to the server as a calculation result.

Images