JP7156107B2 - In-vehicle device, learning system and data transmission method - Google Patents

Description

The present invention relates to an in-vehicle device, a learning system, and a data transmission method.

Artificial intelligence has made it possible to process various types of information. Patent Literature 1 discloses a system in which image data captured by a vehicle is transmitted to a server, and the server learns using the image data.

In the system disclosed in Patent Literature 1, a vehicle performs image recognition and transmits recognition result data. The server receives the recognition result data and determines whether the recognition result data is valid. When it is determined that the recognition result data is not valid, the vehicle is requested to send the image data. If the server determines that the recognition result data is valid, the image data is not transmitted from the vehicle to the server. Therefore, the amount of communication sent from the car to the server is reduced.

Japanese Patent Application Laid-Open No. 2018-206108

The system disclosed in Patent Literature 1 can reduce communication traffic. However, in order to improve processing accuracy by learning, it is preferable to supply a large amount of highly accurate data as learning data.

If all the data acquired by the vehicle were to be sent to the server, the amount of communication would be too large. In addition, the use of data with poor accuracy may reduce the learning accuracy.

The present disclosure has been made based on this situation, and its object is to provide an in-vehicle device, a learning system, and a data transmission method that can reduce the communication load and suppress the deterioration of the learning accuracy. That's what it is.

The above objects are achieved by the combination of features stated in the independent claims, and the sub-claims define further advantageous embodiments. The symbols in parentheses described in the claims indicate the corresponding relationship with specific means described in the embodiments described later as one aspect, and do not limit the disclosed technical scope.

One disclosure relating to an in-vehicle device for achieving the above object is
An in-vehicle device (3) mounted in a vehicle (2) for transmitting learning data to a server (6),
a data acquisition unit (S1) that acquires driving situation data while the vehicle is running;
an object recognition unit (S3) that executes object recognition processing based on the traveling situation data;
a recognition accuracy determination unit (S6) that determines the accuracy of object recognition processing by the object recognition unit;
a data transmission determination unit (S7) that determines to transmit learning data determined based on driving situation data to a server when the accuracy determined by the recognition accuracy determination unit is equal to or greater than a threshold ; ,
a transmission processing unit (S9) for transmitting the learning data determined to be transmitted by the data transmission determination unit from the transmission unit (24) to the server ;
A buffer (34) for temporarily storing learning data ,
The transmission processing unit causes the transmitting unit to transmit learning data for a continuous fixed period of time, but if it is determined that the continuous fixed time period of learning data is not stored in the buffer, the continuous fixed time period of data is not obtained. Remove the training data from the buffer .

This in-vehicle device does not transmit all of the learning data determined based on the acquired driving situation data to the server. The in-vehicle device performs object recognition processing and determines the accuracy of the object recognition processing. Then, based on the accuracy, it is determined whether or not to transmit the learning data. Therefore, the amount of data transmitted by the in-vehicle device can be reduced compared to transmitting all the learning data from the in-vehicle device. Therefore, the communication load for transmitting learning data is reduced.

Moreover, this in-vehicle device does not transmit the learning data when the accuracy of the object recognition processing is low. Learning data that has not been transmitted is not used for learning. Therefore, it is possible to prevent the accuracy of learning from deteriorating.

One disclosure related to a learning system for achieving the above object is
A learning system comprising an in-vehicle device (3) mounted on a vehicle (2) and transmitting learning data to a server (6), and a server,
In-vehicle equipment
a data acquisition unit (S1) that acquires driving situation data while the vehicle is running;
an object recognition unit (S3) that executes object recognition processing based on the traveling situation data;
a recognition accuracy determination unit (S6) that determines the accuracy of object recognition processing by the object recognition unit;
a data transmission determination unit (S7) that determines whether or not to transmit learning data determined based on driving situation data to a server based on the accuracy determined by the recognition accuracy determination unit when the accuracy is equal to or greater than a threshold; ,
a transmission processing unit (S9) for transmitting the learning data determined to be transmitted by the data transmission determination unit from the transmission unit (24) to the server;
The server
a receiving unit (65) for receiving learning data;
a learning unit (71) that performs learning based on the learning data received by the receiving unit;
an accuracy re-determining unit (70) that determines the accuracy of the object recognition processing by the object recognition unit by executing processing with higher accuracy than the object recognition processing executed by the object recognition unit ;
The learning unit determines to use the learning data for learning when the accuracy determined by the accuracy redetermining unit satisfies the accuracy to be used for learning .

A data transmission method for achieving the above object is a data transmission method executed by the vehicle-mounted device.

That is, an in-vehicle device (3) mounted in a vehicle (2) is a data transmission method for transmitting learning data to a server (6),
Acquire driving situation data while the vehicle is running,
Execute object recognition processing based on driving situation data,
Determines the accuracy of object recognition processing,
Based on the accuracy of object recognition processing, when the accuracy is equal to or higher than a threshold, determining to transmit learning data determined based on driving situation data to a server,
causing the determined learning data to be transmitted from the transmission unit (24) to the server ;
The transmitting unit transmits learning data for a continuous fixed time period. Delete learning data that does not become data for a certain amount of time from the buffer .

It is a figure which shows the structure of the learning system 1 of embodiment. 3 is a diagram showing an in-vehicle device 3 and various devices that exchange data with the in-vehicle device 3. FIG. 4 is a flowchart showing processing executed by the in-vehicle device 3; FIG. 4 is a flowchart showing detailed processing of a data transmission process executed in S9 of FIG. 3; FIG. 3 is a diagram showing the configuration of a server 6; FIG. 3 is a block diagram showing functions executed by a control unit 66; FIG.

Hereinafter, embodiments will be described based on the drawings. FIG. 1 is a diagram showing the configuration of a learning system 1 of this embodiment. The learning system 1 includes an in-vehicle device 3 mounted in a vehicle 2 and a server 6 .

The vehicle 2 is not particularly limited as long as it is a vehicle that travels on roads. Vehicles 2 include not only passenger cars but also trucks and buses. The base station 4 performs wireless communication with the in-vehicle device 3 . LTE (Long Term Evolution) can be used for communication between the base station 4 and the in-vehicle device 3 . LTE is sometimes called 4G. The base station 4 transmits the data received from the in-vehicle device 3 to the server 6 outside the vehicle via the communication network 5 . Further, when data for the in-vehicle device 3 is provided from the server 6 , the data is transmitted to the in-vehicle device 3 .

The in-vehicle device 3 collects traveling situation data while the vehicle 2 is traveling, and generates learning data based on the traveling situation data. In-vehicle device 3 transmits the learning data to server 6 via base station 4 and communication network 5 . The server 6 performs machine learning using learning data to generate and update behavior prediction algorithms.

[Configuration of in-vehicle device 3]
FIG. 2 shows the in-vehicle device 3 and various devices mounted on the vehicle 2 for exchanging data with the in-vehicle device 3 . As shown in FIG. 2 , the vehicle 2 is equipped with an object detection sensor 21 , a camera 22 , and a wireless device 23 in addition to the vehicle-mounted device 3 .

The object detection sensor 21 is a sensor that detects objects around the vehicle 2 . A specific example of the object detection sensor 21 is a millimeter wave radar or Lidar. Objects detected by the object detection sensor 21 include three-dimensional objects such as vehicles and people existing on the road and its surroundings. Of course, the object detection sensor 21 detects not only moving objects such as vehicles and people, but also stationary objects such as road signs and guardrails. Also, road markings written on the road surface may be detectable by the object detection sensor 21 .

The camera 22 sequentially captures images of the surroundings of the vehicle 2 . The wireless device 23 is capable of wireless communication with the base station 4 . The radio 23 has a transmitter 24 and a receiver 25 . The transmission unit 24 modulates and amplifies the data provided from the in-vehicle device 3 and transmits the data as radio waves. Learning data is supplied from the in-vehicle device 3 to the transmission unit 24 . The receiving unit 25 receives radio waves, demodulates and amplifies the radio waves, and extracts data carried by the radio waves. The receiving unit 25 supplies the extracted data to the in-vehicle device 3 .

Object detection sensor 21 , camera 22 and in-vehicle device 3 are connected to LAN bus 26 . The in-vehicle device 3 acquires data indicating peripheral objects detected by the object detection sensor 21 (hereinafter referred to as peripheral object data) via the LAN bus 26 . The in-vehicle device 3 also acquires image data captured by the camera 22 via the LAN bus 26 .

Various sensors for detecting vehicle behavior data are also directly or indirectly connected to the LAN bus 26 , and the in-vehicle device 3 can also acquire vehicle behavior data via the LAN bus 26 . The vehicle behavior data is data that indicates the behavior of the vehicle 2 or data that changes depending on the behavior of the vehicle 2 . Examples of vehicle behavior data include vehicle speed, acceleration of the vehicle 2, steering angle, yaw rate, direction of travel of the vehicle 2, position of the vehicle 2, and the like. The peripheral object data is data indicating the traveling conditions outside the vehicle 2, and the vehicle behavior data is data indicating the traveling conditions of the vehicle 2 itself. These peripheral object data and vehicle behavior data are driving situation data.

The in-vehicle device 3 can be implemented by a computer including a processor 31, a ROM 32, a RAM 33, a buffer 34, and the like. The ROM 32 stores a data transmission program for causing a general-purpose computer to function as the in-vehicle device 3 . The processor 31 executes the program stored in the ROM 32 while using the temporary storage function of the RAM 33, so that the in-vehicle device 3 executes the processing shown in FIG. Executing the process shown in FIG. 3 means executing the data transmission method corresponding to the data transmission program.

The buffer 34 temporarily stores driving condition data and learning data determined from the driving condition data. Note that the RAM 33 can also be used as the buffer 34 .

[Processing executed by in-vehicle device 3]
Next, processing executed by the in-vehicle device 3 will be described with reference to FIGS. 3 and 4. FIG. The in-vehicle device 3 executes the processing shown in FIG. 3 for each data acquisition cycle. The data acquisition cycle is determined based on the cycle of updating the command value for the behavior control of the vehicle 2, and is 100 ms, for example.

Step (hereinafter, step is omitted) S1 corresponds to a data acquisition section, and acquires driving situation data. As described above, the driving situation data includes peripheral object data. Surrounding object data is acquired from the object detection sensor 21 and the camera 22 via the LAN bus 26 . The driving situation data also includes vehicle behavior data. Vehicle behavior data is also obtained via the LAN bus 26 .

In S2, the driving condition data acquired in S1 is stored in the buffer 34. FIG. Note that in the process shown in FIG. 3, all of the multiple types of driving situation data are acquired at the same data acquisition cycle. However, the present invention is not limited to this, and some driving situation data may be acquired in a shorter cycle. For example, vehicle behavior data may be obtained at shorter intervals.

S3 corresponds to an object recognition unit. In S3, an object recognition process is performed. The object recognition processing is performed using the traveling situation data stored in the buffer 34 . The traveling situation data used for object recognition processing is one or more of data detected by the millimeter wave radar, data detected by Lidar, and image data captured by the camera 22 .

In the object recognition processing, it is recognized whether the object is a moving object or a stationary object. It also knows the distance to the object and the relative orientation in which the object lies. From the distance and relative orientation to these objects and the coordinates of the vehicle 2, the coordinates at which the objects are located can also be determined. Also, for some objects, the type and size of the object are determined. Vehicles and people can be recognized as types of objects. Various methods such as HOG (Histogram of Oriented Gradients) and R-CNN (convolutional neural network) can be used as methods for recognizing the type of object. The size of the object is determined based on the size of the object in the image (hereinafter referred to as the size of the object image) and the distance to the object. Even if the size of the object image is the same, the larger the distance to the object, the larger the determined size of the object. The distance to the object recognized by executing the object recognition process is stored in the buffer 34 .

S4 corresponds to the behavior control unit. In S4, the target value of behavior control is updated. In behavior control, the object recognition result in S3 is used. Vehicle behavior data acquired in S1 is also used. The specific contents of the behavior control can be set variously. For example, the control may be such that the vehicle 2 is automatically driven to the destination. Also, lane keeping control for keeping the lane in which the vehicle is traveling, and control for following the vehicle in front while maintaining the inter-vehicle distance are also possible. In this embodiment, this behavior control is not particularly limited.

In S5, it is determined whether or not there is a certain period of time or more before the start of the next process. The fixed time is the time required to perform the data selection process, and the specific time is set in advance. The data selection process means S6 to S8 and S91 and S92 shown in FIG. If the judgment result of S5 is NO, the process shown in FIG. 3 is terminated. When the processing shown in FIG. 3 is completed, the processing shown in FIG. 3 is started again when the data acquisition cycle has elapsed since the processing shown in FIG. 3 was started last time. If the determination result of S3 is YES, the process proceeds to S6.

S6 is processing corresponding to the recognition accuracy determination unit. In S6, recognition accuracy is determined. The recognition accuracy means the accuracy of the object recognition processing executed in S3. Recognition accuracy can be determined based on a comparison of the position of the object to a predicted position that can be predicted from changes in the position of the object previously determined.

Previously determined object positions are stored in buffer 34 . The moving speed and moving direction of the object can be calculated from the change in the position of the object determined before the previous time. Based on the position of the object determined in the previous process of S3 and the calculated moving speed and moving direction, it is possible to calculate the predicted position where the object is expected to be detected this time. The greater the difference between the predicted position calculated this time and the position of the object determined in S3 this time, the lower the recognition accuracy is set.

Further, the recognition accuracy can be determined by using the difference between the detected object position and the predicted object position in time series. For example, the recognition accuracy can be determined based on the integral value of the difference within a predetermined criterion period. The integral value increases as the object recognition accuracy decreases. Therefore, the larger the integrated value, the worse the object recognition accuracy can be determined. Note that the object recognition accuracy may be indicated by a continuous value, but it can also be indicated by a multi-step discontinuous value such as three steps. Also, the recognition accuracy may be determined using a plurality of terms such as "good", "average", and "bad".

In addition to the position of the object, or instead of the position of the object, the recognition accuracy may be determined by comparing the size of the detected object with a standard size range that can be determined from the type of object. good. For example, suppose an object is recognized as a person when it is actually a telephone pole. In this case, the size of the recognized object is likely to be out of the standard size range for humans. The more out of the standard size range, the worse value or level the recognition accuracy is determined.

S7 is processing corresponding to the data transmission determination unit. In S7, it is determined whether or not to transmit learning data. The data for learning is data determined from the driving condition data acquired in S1 or the driving condition data. As an example of the data for learning, data of a preset type among the driving situation data acquired in S1 can be used as the data for learning. For example, one or both of the coordinates of the vehicle 2, peripheral object data, and image data can be included in the learning data. If it can be determined that the recognition accuracy determined in S6 is high, it is determined to transmit the learning data. If the recognition accuracy is equal to or higher than the threshold, it is assumed that the recognition accuracy is high.

In S8, the learning data determined to be transmitted in S7 is stored in the buffer 34. FIG. S9 corresponds to a transmission processing unit, which executes data transmission processing. In the data transmission process, the process shown in FIG. 4 is executed.

In FIG. 4, in S91, it is determined whether or not the learning data stored in the buffer 34 includes data for which a group is not established. A group is continuous learning data for a certain period of time. The fixed time is the time required for the server 6 to predict the position to which the moving object will move. A certain period of time is set in advance. The fixed period of time can be a different period of time depending on the type of object. For example, the fixed time for people may be set to a shorter time than the fixed time for cars. Thus, groups can be by object type. A specific time is set based on experiments and the like. As a specific example of time, the fixed time for a car can be 30 seconds, and the fixed time for a person can be 10 seconds.

When the fixed time for the car is set to 30 seconds, in S91, it is determined whether or not there is learning data for the continuous time of less than 30 seconds in the learning data for the car stored in the buffer 34. will do. If there is learning data with a continuous time of less than 30 seconds, it is determined that there is group failure data. If there is group failure data, that is, if the determination result of S91 is YES, the process proceeds to S92. On the other hand, if the determination result of S91 is NO, the process of FIG. 4 is terminated without executing S92.

At S<b>92 , the learning data for which it is determined that the group is not established is deleted from the buffer 34 . In S93, it is determined whether or not there is a time equal to or longer than the minimum time required to perform the data transmission process before the start of the next process. This minimum time can be preset. Also, the minimum time may be determined each time according to the current communication environment. In the case of best-effort communication, the communication speed changes according to the communication environment. Therefore, the time required to establish communication and transmit at least the minimum unit of learning data, which is a payload, fluctuates.

If the judgment result of S93 is NO, the process shown in FIG. 4 is terminated. On the other hand, if the judgment result of S93 is YES, it will progress to S94. In S94, the execution time until the start of the next process is determined, and the data transmission process is executed during the execution time. The data transmission process is a process of establishing communication with the base station 4, adding a header and footer, generating transmission data with the payload as learning data, and transmitting the transmission data. Also, the transmitted data is deleted from the buffer 34 .

It is not always possible to transmit all of the learning data that has not been transmitted by executing S94 once. FIFO (First-In First-Out) can be adopted as the order of transmitting the plurality of learning data stored in the buffer 34 . In addition, in S94, it is not necessary to transmit all the data of one group. The learning data that could not be transmitted will be transmitted when S94 is executed from the next time onward. However, a retention upper limit time may be set, and learning data retained in the buffer 34 exceeding the retention upper limit time may be deleted in units of groups. Also, when the storage capacity of the buffer 34 is insufficient, old learning data is deleted.

[Configuration of Server 6]
FIG. 5 shows the configuration of the server 6. As shown in FIG. The server 6 includes a learning data storage unit 61 , a high-precision map storage unit 62 , a communication unit 63 and a control unit 66 . The learning data storage unit 61 is a writable storage unit, and sequentially stores learning data transmitted from the in-vehicle device 3 . The high-precision map storage unit 62 stores high-precision map data. A high-definition map is a road map represented in three dimensions. Specifically, the high-definition map includes information on the position and shape of objects around the road, such as road signs and guardrails around the road. In addition, the high-precision map expresses roads with high precision in the horizontal direction as well. The high-definition map also shows the locations and types of road segment destinations, as well as road markings.

The communication unit 63 is connected to the communication network 5 . The communication section 63 has a transmission section 64 and a reception section 65 . The transmission unit 64 outputs data to be transmitted to the communication network 5 . The receiving unit 65 receives data transmitted from the in-vehicle device 3 via the base station 4 and the communication network 5 . An example of the received data is learning data. The receiving section 65 supplies the received data to the control section 66 .

The control unit 66 can be implemented by a computer including a processor 67, ROM 68, RAM 69, and the like. The ROM 68 stores a program for causing a general-purpose computer to function as the controller 66 . The processor 67 executes the program stored in the ROM 68 while using the temporary storage function of the RAM 69, so that the control unit 66 functions as an accuracy redetermining unit 70 and a learning unit 71 as shown in FIG. Realize Also, realizing these functions means that the method corresponding to the program stored in the ROM 68 is executed.

The accuracy re-determining unit 70 executes processing with higher accuracy than the object recognition processing executed by the in-vehicle device 3 in S3, and determines the accuracy of the object recognition processing executed in S2. An example of high-precision processing is processing using the high-precision map stored in the high-precision map storage unit 62 . A high definition map can be used to determine where an object is located.

A specific description will be given. A high-definition map contains information such as the location of lane markings, the location of sidewalks, and the width of road shoulders. Therefore, with the high-definition map, it is also possible to determine whether the coordinates at which the vehicle is located are where the vehicle can be located. Vehicles can be located on roads, parking lots, and shoulders, but typically not on sidewalks. Moreover, even on a road, it cannot exist in a place where there are three-dimensional objects such as signs and guardrails. If the coordinates of the vehicle are at a location where the vehicle cannot be located in advance, it can be determined that the determination accuracy of the vehicle's location is low.

Also, if a high-definition map is used, it can be determined whether the coordinates at which a person is located are places where a person can be located. People are not normally located in the roadway, except when crossing the road. In other words, people usually do not continuously move on the roadway in the extension direction of the roadway. Therefore, when the coordinates indicating the position of the person are continuously on the roadway and moving in the extension direction of the road, it can be determined that the determination accuracy of the position of the person is low. Moreover, even if it is a road or sidewalk, a person cannot be positioned where there are three-dimensional objects such as signs and guardrails. Even when the coordinates of a person are in a preset place where the person cannot be located, it can be determined that the determination accuracy of the person's position is low.

Also, using the high-definition map and the current position and direction of travel of the vehicle, it is sometimes possible to determine the features that should be included in the image captured by the camera 22 . Features include, for example, three-dimensional objects such as road signs and specific buildings. In addition, planar objects such as road markings are also included in the features. If it can be determined that the actually captured image does not include a feature that should be included in the image, it can be determined that the accuracy of the object recognition processing is low. The accuracy redetermination unit 70 provides the learning unit 71 with only learning data whose accuracy of the re-determined object recognition processing satisfies the accuracy to be used for learning.

Further, high-precision processing is not limited to this. For example, it is assumed that both the object recognition processing and the accuracy redetermining unit 70 in the in-vehicle device 3 use pattern matching processing for the image cut out by the window while moving the window for cutting out the image. In this case, the size of the window used in the accuracy redetermining unit 70 may be set to be smaller than the window used in the object recognition processing in the in-vehicle device 3 for object recognition. By doing so, the object can be recognized with higher accuracy than the object recognition processing executed by the in-vehicle device 3 . Then, when the object can be recognized with high accuracy, the recognition result can be compared with the recognition result in the in-vehicle device 3 to re-determine the object recognition accuracy.

The learning unit 71 stores the learning data provided from the accuracy redetermining unit 70 in the learning data storage unit 61 . Then, the learning data accumulated in the learning data storage unit 61 is used to perform machine learning of the behavior prediction algorithm of the moving object. The behavior prediction algorithm is an algorithm that predicts the future movement behavior of the mobile object from the past movement trajectory of the mobile object. Input data input to the behavior prediction algorithm includes data related to the moving object such as past positions and velocities of the moving object. In addition, the input data may be data indicating the surrounding conditions of the moving object, such as data relating to three-dimensional objects existing around the moving object, data indicating road surface conditions, and the like.

Input data for the behavior prediction algorithm is included in the learning data. However, data acquired by the server 6 based on the learning data may be included in the input data. For example, if the learning data includes the position of the vehicle 2, the server 6 accesses the weather database using the communication network 5 to know the weather at that position. Weather is an example of input data that the server 6 acquires based on learning data.

The behavior prediction algorithm is not particularly limited, and various algorithms can be adopted. For example, algorithms using regression methods such as support vector machines, algorithms with tree structures such as decision trees, neural networks, time-series algorithms such as state space models, and clustering algorithms such as the k-nearest neighbor method. can do. Also, both supervised and unsupervised learning can be employed. In the case of supervised learning, learning can be performed using, as a correct answer, the position of the moving pair at the final time out of the chronologically continuous learning data used for learning.

In the present embodiment described above, the in-vehicle device 3 does not transmit all of the learning data determined based on the acquired driving situation data to the server 6 . The in-vehicle device 3 performs object recognition processing (S3) and determines the accuracy of the object recognition processing (S6). Then, it is determined whether or not to transmit the learning data based on the object recognition accuracy (S7). Therefore, the amount of data transmitted by the in-vehicle device 3 can be reduced compared to transmitting all the learning data determined from the driving situation data from the in-vehicle device 3 . Therefore, the communication load for transmitting learning data is reduced.

In addition, if data with low object recognition accuracy is used for learning, the accuracy of learning is lowered. However, in this embodiment, if the recognition accuracy determined in S6 is low, the learning data is not transmitted. Learning data that has not been transmitted is not used in the learning of the action prediction algorithm in the server 6 . Therefore, it is possible to prevent the accuracy of learning from deteriorating.

Further, in this embodiment, even if it is decided to transmit the learning data in S7, the learning data for which the group is not established is not transmitted (S92). The learning data for which the group is not established does not contribute much to the improvement of learning accuracy. Therefore, by not transmitting the learning data for which the group is not established, it is possible to reduce the communication load while suppressing the deterioration of the learning accuracy.

Furthermore, in the present embodiment, the server 6 executes object recognition processing with higher accuracy than the object recognition processing executed by the in-vehicle device 3, and redetermines the accuracy of the object recognition processing. As a result, the in-vehicle device 3 can determine that the recognition accuracy of the learning data, which could not be determined to be low, is low. Learning data with low recognition accuracy determined by the accuracy redetermining unit 70 is not used for learning of the action prediction algorithm. Thereby, it is possible to further suppress the deterioration of the learning accuracy.

In addition, since the result of the object recognition processing in the in-vehicle device 3 is used to update the target value of the behavior control, if the object recognition processing in the in-vehicle device 3 takes time, the behavior control performance of the vehicle 2 deteriorates. . However, in the present embodiment, the in-vehicle device 3 does not perform high-precision processing corresponding to the precision redetermination unit 70 . As a result, deterioration in the behavior control performance of the vehicle 2 can also be suppressed.

Further, in the present embodiment, the in-vehicle device 3, after updating the target value of the behavior control, only when there is a certain time or more before the start of the next process, S7 corresponding to the data transmission determination unit and the data transmission S9 corresponding to the processing unit is executed. In other words, the in-vehicle device 3 sets the processing priority of the data transmission determination unit (S7) and the transmission processing unit (S9) lower than the priority of the behavior control of the vehicle 2. Also by this, the deterioration of the behavior control performance of the vehicle 2 can be suppressed.

Although the embodiments have been described above, the disclosed technology is not limited to the above-described embodiments, and the following modifications are also included in the disclosed scope. Various modifications can be made. In the following description, the elements having the same reference numerals as the reference numerals used so far are the same as the elements having the same reference numerals in the previous embodiments unless otherwise specified. Moreover, when only part of the configuration is described, the previously described embodiments can be applied to the other portions of the configuration.

<Modification 1>
For example, if the accuracy redetermining unit 70 of the server 6 decides not to use the object recognition accuracy for learning due to low object recognition accuracy, the fact is fed back to the developer who develops the recognition algorithm for the object recognition processing in the in-vehicle device 3. can be In this way, the developer can save man-hours for extracting scenes for which there is room for improvement in the recognition algorithm from a large amount of data.

<Modification 2>
The accuracy redetermination unit 70 may perform object recognition using a machine-learned object recognition algorithm.

<Modification 3>
The controller and techniques described in this disclosure may be implemented by a special purpose computer comprising a processor programmed to perform one or more functions embodied by a computer program. Alternatively, the controller and techniques described in this disclosure may be implemented by dedicated hardware logic circuitry. Alternatively, the controller and techniques described in this disclosure may be implemented by one or more dedicated computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. Hardware logic circuits are, for example, ASICs and FPGAs.

Moreover, the storage medium for storing the computer program is not limited to the ROM, and may be stored in a computer-readable non-transition tangible storage medium as instructions executed by the computer. For example, the program may be stored in a flash memory.

1: learning system 2: vehicle 3: onboard device 4: base station 5: communication line 6: server 21: object detection sensor 22: camera 23: wireless device 24: transmitter 25: receiver 26: LAN bus 31: processor 32 : ROM 33: RAM 34: Buffer 61: Learning data storage unit 62: High precision map storage unit 63: Communication unit 64: Transmission unit 65: Reception unit 66: Control unit 67: Processor 68: ROM 69: RAM 70: Accuracy Redetermination unit 71: Learning unit S1: Data acquisition unit S3: Object recognition unit S4: Behavior control unit S6: Recognition accuracy determination unit S7: Data transmission determination unit S9: Transmission processing unit

Claims (9)

An in-vehicle device (3) mounted in a vehicle (2) for transmitting learning data to a server (6),
a data acquisition unit (S1) that acquires driving situation data while the vehicle is running;
an object recognition unit (S3) that executes an object recognition process based on the traveling situation data;
a recognition accuracy determination unit (S6) that determines the accuracy of the object recognition processing by the object recognition unit;
Based on the accuracy determined by the recognition accuracy determining unit, when the accuracy is equal to or greater than a threshold value, the data transmission determining unit determines to transmit the learning data determined based on the driving situation data to the server. (S7);
a transmission processing unit (S9) for transmitting the learning data determined to be transmitted by the data transmission determination unit from the transmission unit (24) to the server ;
A buffer (34) that temporarily stores the learning data ,
The transmission processing unit causes the transmission unit to transmit the learning data for a continuous fixed time period, but if it is determined that the learning data for the continuous fixed time period is not stored in the buffer, an in-vehicle device that deletes from the buffer the learning data that does not become minute data . The object recognition unit sequentially determines the position of the object,
The recognition accuracy determination unit performs the object recognition processing based on a comparison between the position of the object sequentially determined by the object recognition unit and a predicted position determined based on a change in the position of the object determined before the previous time. 2. The in-vehicle device of claim 1, which determines accuracy. The object recognition unit determines the type and size of the object,
The recognition accuracy determination unit compares the size of the object recognized by the object recognition unit with a standard size of the object determined based on the type of the object recognized by the object recognition unit, and performs the object recognition processing. 3. An in-vehicle device according to claim 1 or 2, for determining accuracy. A behavior control unit (S4) that controls the behavior of the vehicle,
The in-vehicle device according to any one of claims 1 to 3 , wherein processing priority of said recognition accuracy determination unit, said data transmission determination unit, and said transmission processing unit is lower than that of said behavior control unit. A learning system comprising the in-vehicle device according to any one of claims 1 to 4 and the server,
The server is
a receiving unit (65) that receives the learning data;
A learning system comprising a learning unit (71) that performs learning based on the learning data received by the receiving unit. The server includes an accuracy re-determining unit (70) that executes processing with higher accuracy than the object recognition processing executed by the object recognition unit and determines the accuracy of the object recognition processing by the object recognition unit,
6. The learning system according to claim 5 , wherein said learning unit decides to use said learning data for said learning when the accuracy determined by said accuracy redetermining unit satisfies the accuracy to be used for learning. . A learning system comprising an in-vehicle device (3) mounted on a vehicle (2) for transmitting learning data to a server (6), and the server,
The in-vehicle device
a data acquisition unit (S1) that acquires driving situation data while the vehicle is running;
an object recognition unit (S3) that executes an object recognition process based on the traveling situation data;
a recognition accuracy determination unit (S6) that determines the accuracy of the object recognition processing by the object recognition unit;
A data transmission determination unit (S7 )When,
a transmission processing unit (S9) for transmitting the learning data determined to be transmitted by the data transmission determination unit from the transmission unit (24) to the server;
The server is
a receiving unit (65) that receives the learning data;
a learning unit (71) that performs learning based on the learning data received by the receiving unit;
an accuracy redetermining unit (70) that executes a process with higher accuracy than the object recognition processing executed by the object recognition unit and determines the accuracy of the object recognition processing by the object recognition unit ;
The learning system , wherein the learning unit decides to use the learning data for the learning when the accuracy determined by the accuracy redetermining unit satisfies the accuracy to be used for learning. 8. The learning system according to claim 6 , wherein the processing executed by said accuracy redetermination unit is processing using a high-precision map representing map data in three dimensions. A data transmission method in which an in-vehicle device (3) mounted on a vehicle (2) transmits learning data to a server (6),
Acquiring driving situation data while the vehicle is running;
executing object recognition processing based on the traveling situation data;
determining the accuracy of the object recognition process;
Based on the accuracy of the object recognition process, when the accuracy is equal to or higher than a threshold, determining to transmit the learning data determined based on the driving situation data to the server;
causing the transmission unit (24) to transmit the determined learning data to the server ;
The learning data for a continuous predetermined time is transmitted from the transmission unit, but it is determined that the learning data for the continuous predetermined time is not stored in a buffer (34) for temporarily storing the learning data. In this case, deleting from the buffer the learning data that does not become continuous data for a certain period of time ;
A data transmission method in which the in-vehicle device executes a process including :

Images