WO2021171398A1 - 推論装置、運転支援装置、推論方法、および、サーバ - Google Patents

推論装置、運転支援装置、推論方法、および、サーバ Download PDF

Info

Publication number
WO2021171398A1
WO2021171398A1 PCT/JP2020/007607 JP2020007607W WO2021171398A1 WO 2021171398 A1 WO2021171398 A1 WO 2021171398A1 JP 2020007607 W JP2020007607 W JP 2020007607W WO 2021171398 A1 WO2021171398 A1 WO 2021171398A1
Authority
WO
WIPO (PCT)
Prior art keywords
inference
machine learning
unit
data
learning model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2020/007607
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
太郎 奥田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to CN202080097069.4A priority Critical patent/CN115136161A/zh
Priority to DE112020006796.3T priority patent/DE112020006796T5/de
Priority to JP2022502639A priority patent/JP7254240B2/ja
Priority to PCT/JP2020/007607 priority patent/WO2021171398A1/ja
Priority to US17/790,297 priority patent/US20230035526A1/en
Publication of WO2021171398A1 publication Critical patent/WO2021171398A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/096—Transfer learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/082—Selecting or switching between different modes of propelling
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0051—Handover processes from occupants to vehicle
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W60/00—Drive control systems specially adapted for autonomous road vehicles
    • B60W60/005—Handover processes
    • B60W60/0053—Handover processes from vehicle to occupant
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0108—Measuring and analyzing of parameters relative to traffic conditions based on the source of data
    • G08G1/0112—Measuring and analyzing of parameters relative to traffic conditions based on the source of data from the vehicle, e.g. floating car data [FCD]
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0125—Traffic data processing
    • G08G1/0129—Traffic data processing for creating historical data or processing based on historical data
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/01—Detecting movement of traffic to be counted or controlled
    • G08G1/0104—Measuring and analyzing of parameters relative to traffic conditions
    • G08G1/0137—Measuring and analyzing of parameters relative to traffic conditions for specific applications
    • G08G1/0145—Measuring and analyzing of parameters relative to traffic conditions for specific applications for active traffic flow control
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/09623—Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
    • G08G1/096716—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control where the received information does not generate an automatic action on the vehicle control
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096708—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control
    • G08G1/096725—Systems involving transmission of highway information, e.g. weather, speed limits where the received information might be used to generate an automatic action on the vehicle control where the received information generates an automatic action on the vehicle control
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/09—Arrangements for giving variable traffic instructions
    • G08G1/0962—Arrangements for giving variable traffic instructions having an indicator mounted inside the vehicle, e.g. giving voice messages
    • G08G1/0967—Systems involving transmission of highway information, e.g. weather, speed limits
    • G08G1/096766—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission
    • G08G1/096775—Systems involving transmission of highway information, e.g. weather, speed limits where the system is characterised by the origin of the information transmission where the origin of the information is a central station
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/215—Selection or confirmation of options

Definitions

  • This disclosure relates to an inference device, a driving support device, an inference method, and a server that make inferences using a trained model in machine learning (hereinafter referred to as "machine learning model").
  • machine learning model a trained model in machine learning
  • Patent Document 1 when the estimation result using the unsupervised data is similar to the estimation result using the supervised data, the teacher information of the unsupervised data is provided based on the teacher information possessed by the supervised data.
  • the information processing device to be created is disclosed.
  • Patent Document 1 When inference is performed using a machine learning model, data having a degree of similarity to the data input when the machine learning model is trained may be input to the machine learning model. As a result, there is a problem that an invalid inference result may be output.
  • the technique disclosed in Patent Document 1 is a technique for determining the similarity between supervised data and unsupervised data among training data, and the similarity between the data input at the time of inference and the data input at the time of learning. It is not a technique to judge. Therefore, in order to solve the above problems, the technique disclosed in Patent Document 1 cannot be used.
  • the present disclosure has been made to solve the above problems, and an object of the present disclosure is to provide an inference device capable of preventing the output of invalid inference results.
  • the inference device inputs the data acquired by the data acquisition unit into the data acquisition unit that acquires the data and the first machine learning model that outputs the first inference result by inputting the data, and the first inference result.
  • the second inference result inferred by inputting the data acquired by the data acquisition unit into the inference unit that infers the data and the second machine learning model that outputs the second inference result by inputting the data, and the data acquisition unit acquired the second inference result.
  • FIG. 1 shows the structural example of the inference apparatus which concerns on Embodiment 1.
  • FIG. It is a figure which showed the relationship between the 1st machine learning model and the 2nd machine learning model in Embodiment 1.
  • FIG. It represents the degree of similarity [S] between the vehicle peripheral data and the output vehicle peripheral data in the first embodiment (Equation 1).
  • Equation 1 an example of an image of an inference result determination threshold value statistically set based on the distribution of “correct answer data similarity” and the distribution of “incorrect answer data similarity” is shown.
  • FIG. 1 shows the structural example of the inference apparatus which concerns on Embodiment 1.
  • FIG. 1 shows the relationship between the 1st machine learning model and the 2nd machine learning model in Embodiment 1.
  • FIG. represents the degree of similarity [S] between the vehicle peripheral data and the output vehicle peripheral data in the first embodiment (Equation 1).
  • FIG. 1 the method in which the second model selection unit selects the selected second machine learning model by comparing the similarity for each second machine learning model calculated by the similarity calculation unit with the threshold value for determining the inference result. It is a figure which shows an example of an image. In the second embodiment, the image of the method in which the provisional second model selection unit selects the provisional second machine learning model based on the representative similarity calculated by the representative similarity calculation unit and the learning similarity information.
  • FIG. 14A and 14B are diagrams showing an example of the hardware configuration of the inference device according to the first embodiment or the inference device according to the second embodiment.
  • FIG. 5 is a diagram showing a configuration example of an inference system in which a server is provided with an inference device according to the first embodiment or an inference device according to the second embodiment, and the server and a vehicle are connected via a network.
  • FIG. 1 is a diagram showing a configuration example of the inference device 1 according to the first embodiment.
  • the inference device 1 is provided in the driving support device 100 that outputs information for supporting the driving by the driver of the vehicle (hereinafter referred to as "driving support information"). It is assumed that the vehicle for which the driving support device 100 supports driving has an automatic driving function. Even when the vehicle has an automatic driving function, the driver can drive the vehicle by himself / herself without executing the automatic driving function.
  • the driving support device 100 outputs driving support information when the driver is driving himself / herself in a vehicle capable of automatic driving.
  • the driving support information is information indicating that an object such as another vehicle, a signal, or a sign around the vehicle has been recognized.
  • the driving support device 100 is mounted on the vehicle.
  • the inference device 1 infers driving support information based on data related to the vehicle surroundings (hereinafter referred to as “vehicle peripheral data”) and the first machine learning model 16, and outputs the inferred driving support information.
  • vehicle peripheral data data related to the vehicle surroundings
  • the first machine learning model 16 has been trained to infer an inference result (hereinafter referred to as "first inference result") by supervised learning in which correct answer data corresponding to the input data is prepared.
  • the first machine learning model 16 has been learned to infer driving support information as the first inference result by supervised learning in which correct answer data corresponding to vehicle peripheral data is prepared.
  • the inference device 1 uses the second machine learning model 17 to determine whether or not the driving support information inferred by the first machine learning model 16 is valid.
  • the second machine learning model 17 has been trained using the same input data as the first machine learning model 16 so that the input data and the output data which is the inference result (hereinafter referred to as "the second inference result") become equal. be.
  • the second machine learning model 17 inputs the vehicle peripheral data, and outputs the output data having the same contents as the vehicle peripheral data (hereinafter referred to as "output vehicle peripheral data") as the second inference result. I have already learned.
  • the first machine learning model 16 and the second machine learning model 17 are machine learning models learned by deep learning in a neural network.
  • FIG. 2 is a diagram showing the relationship between the first machine learning model 16 and the second machine learning model 17 in the first embodiment.
  • the output of the neural network at the time of inference depends on the bias of the data input at the time of learning (hereinafter referred to as "data at the time of learning"). Therefore, the first machine learning model 16 can derive a correct solution when the vehicle peripheral data having the same content as the learning data of the first machine learning model 16 is input at the time of inference, in other words, correct driving. It is assumed that support information can be inferred.
  • the second machine learning model 17 learns to output the output data having the same contents as the input vehicle peripheral data, that is, the output vehicle peripheral data.
  • the vehicle peripheral area used as the input of the second machine learning model 17 is used.
  • the difference between the data and the output vehicle peripheral data is reduced.
  • the vehicle peripheral data having characteristics different from the learning data of the second machine learning model 17 is input to the second machine learning model 17 at the time of inference
  • the vehicle peripheral area used as the input of the second machine learning model 17 is used.
  • the vehicle peripheral data input to the second machine learning model 17 at the time of inference is the vehicle peripheral data input to the first machine learning model 16 at the time of inference.
  • the degree of whether or not the characteristics of the vehicle peripheral data input by the second machine learning model 17 at the time of inference and the characteristics of the output vehicle peripheral data are close to each other is determined by the vehicle peripheral data and the output vehicle peripheral data. It is called "similarity" of. The smaller the difference between the vehicle peripheral data input by the second machine learning model 17 at the time of inference and the output vehicle peripheral data output by the second machine learning model 17, the smaller the difference between the vehicle peripheral data and the output vehicle peripheral data. The degree of similarity is large. It can be said that the greater the similarity between the vehicle peripheral data and the output vehicle peripheral data, the closer the vehicle peripheral data at the time of inference to the data at the time of learning.
  • the inferred driving support information is assumed to be correct.
  • the vehicle peripheral data input by the second machine learning model 17 at the time of inference and the output vehicle peripheral data output by the second machine learning model 17 are large.
  • the similarity with the data is small. It can be said that the smaller the similarity between the vehicle peripheral data and the output vehicle peripheral data, the different the characteristics of the vehicle peripheral data at the time of inference from the data at the time of learning. If the characteristics of the vehicle surrounding data at the time of inference are different from the characteristics of the data at the time of learning, it is assumed that the inferred driving support information is incorrect.
  • the inference device 1 calculates the similarity between the vehicle peripheral data and the output vehicle peripheral data, and determines whether or not the driving support information is appropriate based on the calculated similarity.
  • the method of calculating the similarity in the inference device 1 will be described later. Further, the details of the method of determining whether or not the driving support information is appropriate performed by the inference device 1 will be described later.
  • the reasoning device 1 determines that the driving support information inferred based on the first machine learning model 16 is appropriate, the reasoning device 1 outputs the driving support information to the driving support device 100.
  • the first machine learning model 16 is a machine learning model used for acquiring information indicating that an object has been recognized, as described above, but this is only an example.
  • the first machine learning model 16 is a machine learning model used for any purpose to obtain an arbitrary first inference result.
  • the inference device 1 includes a data acquisition unit 11, an inference unit 12, a similarity calculation unit 13, a judgment unit 14, an output unit 15, a first machine learning model 16, and a second machine learning model 17.
  • the first machine learning model 16 and the second machine learning model 17 are provided in the inference device 1, but this is only an example.
  • the first machine learning model 16 and the second machine learning model 17 may be provided outside the inference device 1 at a place where the inference device 1 can be referred to.
  • the inference unit 12 includes a first inference unit 121 and a second inference unit 122.
  • the data acquisition unit 11 acquires information around the vehicle.
  • the information around the vehicle includes, for example, information about the position of the own vehicle, a captured image of the periphery of the vehicle, or map information.
  • the data acquisition unit 11 is, for example, a map stored in a GPS (Global Positioning System) mounted on the vehicle, an imaging device mounted on the vehicle (not shown), or a server outside the vehicle (not shown).
  • GPS Global Positioning System
  • the data acquisition unit 11 acquires vehicle peripheral data to be input to the first machine learning model 16 and the second machine learning model 17 based on the acquired vehicle peripheral information.
  • the vehicle peripheral data is data indicating one or more feature quantities extracted from the vehicle peripheral information.
  • the feature amount is the position of the own vehicle, the position of another vehicle, the position of a pedestrian, or the like.
  • the data acquisition unit 11 acquires vehicle peripheral data indicating one or more feature quantities based on the vehicle peripheral information.
  • the data acquisition unit 11 outputs vehicle peripheral data indicating one or more feature quantities to the inference unit 12 and the similarity calculation unit 13.
  • vehicle peripheral data showing one or more feature quantities is also simply referred to as "vehicle peripheral data”.
  • the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 into the first machine learning model 16 to infer driving support information. Further, the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 into the second machine learning model 17 and infers the output vehicle peripheral data. Specifically, the first inference unit 121 of the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 into the first machine learning model 16 to infer driving support information. The second inference unit 122 of the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 into the second machine learning model 17, and infers the output vehicle peripheral data. The first inference unit 121 outputs the inferred driving support information to the determination unit 14. The second inference unit 122 outputs the inferred output vehicle peripheral data to the similarity calculation unit 13.
  • the similarity calculation unit 13 inputs the output vehicle peripheral data inferred by the second inference unit 122 by inputting the vehicle peripheral data acquired by the data acquisition unit 11 into the second machine learning model 17, and the vehicle acquired by the data acquisition unit 11. Based on the peripheral data, the degree of similarity between the vehicle peripheral data acquired by the data acquisition unit 11 and the inferred output vehicle peripheral data is calculated.
  • FIG. 3 shows the degree of similarity [S] between the vehicle peripheral data and the output vehicle peripheral data in the first embodiment (Equation 1).
  • "x" represents the vehicle peripheral data input by the second inference unit 122 as the input of the second machine learning model
  • "x'" represents the vehicle peripheral data that the second inference unit 122 uses as the second machine learning model.
  • Output that is inferred by inputting vehicle peripheral data Represents vehicle peripheral data.
  • the similarity between the vehicle peripheral data and the output vehicle peripheral data is represented by the cosine similarity using the vehicle peripheral data and the output vehicle peripheral data.
  • the cosine similarity is determined by (Equation 1) as one value regardless of the number of features included in the vehicle peripheral data and the output vehicle peripheral data. The closer the vehicle peripheral data and the output vehicle peripheral data have, the greater the similarity.
  • the similarity calculation unit 13 calculates the similarity between the vehicle peripheral data and the output vehicle peripheral data based on the (Equation 1) represented by FIG.
  • the similarity calculation unit 13 outputs the calculated similarity to the determination unit 14.
  • the determination unit 14 determines whether or not to output the driving support information inferred by the first inference unit 121 by comparing the similarity calculated by the similarity calculation unit 13 with the inference result determination threshold value. Specifically, the determination unit 14 first sets a threshold value for determining the inference result. The determination unit 14 calculates the "correct answer data similarity” based on the "correct answer test data” and the “correct answer inference result” at the time of verification of the first machine learning model 16, and at the time of verification of the first machine learning model 16. The inference result determination threshold is set based on the "incorrect answer data similarity" calculated based on the "incorrect answer test data” and the "incorrect answer inference result”.
  • the "correct answer test data” is vehicle peripheral data as test data in which the output driving support information was the correct answer as a result of inputting to the first machine learning model 16 at the time of verification of the first machine learning model 16. ..
  • the vehicle peripheral data as test data includes one or more feature quantities.
  • the "correct answer inference result” is the output vehicle peripheral data output when the above “correct answer test data” is input to the second machine learning model 17.
  • the “correct answer data similarity” is the similarity between the "correct answer test data” and the “correct answer inference result” calculated based on the above "correct answer test data” and the above "correct answer inference result”.
  • “Incorrect answer test data” is vehicle peripheral data as test data in which the output driving support information was incorrect as a result of inputting to the first machine learning model 16 at the time of verification of the first machine learning model 16. Is.
  • the “incorrect answer inference result” is output vehicle peripheral data output when the above “incorrect answer test data” is input to the second machine learning model 17.
  • the “incorrect answer data similarity” is the similarity between the above “incorrect answer test data” and the above “incorrect answer inference result” calculated based on the above “incorrect answer test data” and the above “incorrect answer inference result”. Degree.
  • the determination unit 14 calculates the "correct answer data similarity” and the "incorrect answer data similarity” in the same manner as the similarity calculation unit 13 calculates the similarity between the vehicle peripheral data and the output vehicle peripheral data. (See (Equation 1) in FIG. 3).
  • the "correct answer test data”, “correct answer inference result”, “incorrect answer test data”, and “incorrect answer inference result” are obtained by the inference device 1 at the time of verification of the first machine learning model 16 and the second machine learning model 17. It is assumed that it is stored in a referenceable storage unit (not shown).
  • the determination unit 14 statistically sets the inference result determination threshold value based on the distribution of the “correct answer data similarity” and the distribution of the “incorrect answer data similarity”.
  • FIG. 4 shows an example of an image of an inference result determination threshold value statistically set based on the distribution of “correct answer data similarity” and the distribution of “incorrect answer data similarity” in the first embodiment. Shown. When vehicle peripheral data including a plurality of feature quantities can be input to the first machine learning model 16 and the second machine learning model 17 at the time of inference, the first machine learning model using the test data including the plurality of feature quantities. 16 and the second machine learning model 17 are verified.
  • the horizontal axis indicates “correct answer data similarity” or “incorrect answer data similarity”. The vertical axis shows the number of "correct answer data similarity” or the number of "incorrect answer data similarity”.
  • the determination unit 14 compares the similarity calculated by the similarity calculation unit 13 with the inference result determination threshold value to provide driving support inferred by the first inference unit 121. Determine whether to output information. Specifically, in the first embodiment, when the similarity calculated by the similarity calculation unit 13 is larger than the inference result determination threshold value, the determination unit 14 outputs the driving support information inferred by the first inference unit 121. to decide. The determination unit 14 outputs the determination result of whether or not to output the driving support information inferred by the first inference unit 121 to the output unit 15. When the determination unit 14 determines that the driving support information inferred by the first inference unit 121 is to be output, the determination unit 14 associates the determination result with the driving support information and outputs the driving support information to the output unit 15.
  • the similarity is represented by the cosine similarity using the vehicle peripheral data and the output vehicle peripheral data. Therefore, it can be said that the larger the similarity, the closer the characteristics of the vehicle peripheral data and the characteristics of the output vehicle peripheral data are. Therefore, when the similarity calculated by the similarity calculation unit 13 is larger than the inference result determination threshold value, the determination unit 14 determines that the driving support information inferred by the first inference unit 121 is output. It's just an example. Depending on the method of setting the similarity, it may be said that the larger the similarity, the closer the characteristics of the vehicle peripheral data and the characteristics of the output vehicle peripheral data are.
  • the similarity is represented by the difference between the vehicle peripheral data and the output vehicle peripheral data.
  • the larger the similarity in other words, the difference, the closer the characteristics of the vehicle peripheral data and the characteristics of the vehicle peripheral data are.
  • the determination unit 14 determines that the driving support information is output when the similarity is smaller than the inference result determination threshold value. In this way, the determination unit 14 determines that the driving support information is output is not limited to the case where the similarity is larger than the inference result determination threshold value.
  • the determination unit 14 may determine whether or not to output the driving support information by comparing the similarity with the threshold value for determining the inference result.
  • the determination unit 14 calculates the "correct answer data similarity” and the "incorrect answer data similarity” and sets the inference result determination threshold value. It's just an example.
  • the similarity calculation unit 13 calculates the similarity between the vehicle peripheral data acquired by the data acquisition unit 11 and the output vehicle peripheral data inferred by inputting the vehicle peripheral data into the second machine learning model 17. At that time, the "correct answer data similarity” and the “incorrect answer data similarity” may be calculated and output to the determination unit 14.
  • the determination unit 14 sets the inference result determination threshold value based on the “correct answer data similarity” and the “incorrect answer data similarity” output from the similarity calculation unit 13.
  • the threshold value for determining the inference result is set based on “similarity”. It is assumed that the inference result determination threshold value set at the time of verification is stored in the storage unit. The inference result determination threshold value is set before the determination unit 14 determines whether or not to output the driving support information inferred by the first inference unit 121 using the inference result determination threshold value. Just do it.
  • the output unit 15 When the determination unit 14 determines that the driving support information inferred by the first inference unit 121 is output, the output unit 15 outputs the driving support information to the driving support device 100. When the determination unit 14 determines that the driving support information inferred by the first inference unit 121 is not output, the output unit 15 outputs notification information notifying the driving support device 100 that automatic driving is not possible. .. When the notification information is output from the output unit 15, the driving support device 100 asks the driver of the vehicle whether or not to switch from the automatic driving to the manual driving. Specifically, for example, a message as to whether or not the inquiry unit (not shown) of the driving support device 100 switches from automatic driving to manual driving on the touch panel display (not shown) provided in the vehicle. Is displayed.
  • the driver confirms the touch panel display and inputs an instruction as to whether or not to switch to manual operation by touching the touch panel display or the like.
  • the driving control unit (not shown) of the driving support device 100 switches the vehicle to manual driving control.
  • the operation control unit of the operation support device 100 controls to automatically stop the vehicle.
  • FIG. 5 is a flowchart for explaining the operation of the inference device 1 according to the first embodiment.
  • the data acquisition unit 11 acquires vehicle peripheral data to be input to the first machine learning model 16 and the second machine learning model 17 based on the acquired vehicle peripheral information (step ST501).
  • the data acquisition unit 11 outputs the acquired vehicle peripheral data to the inference unit 12 and the similarity calculation unit 13.
  • the first inference unit 121 of the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 in step ST501 into the first machine learning model 16 to infer driving support information (step ST502).
  • the first inference unit 121 outputs the inferred driving support information to the determination unit 14.
  • the second inference unit 122 of the inference unit 12 inputs the vehicle peripheral data acquired by the data acquisition unit 11 in step ST501 into the second machine learning model 17 and infers the output vehicle peripheral data (step ST503).
  • the second inference unit 122 outputs the inferred output vehicle peripheral data to the similarity calculation unit 13.
  • the similarity calculation unit 13 is acquired by the data acquisition unit 11 based on the output vehicle peripheral data inferred by the second inference unit 122 in step ST503 and the vehicle peripheral data acquired by the data acquisition unit 11 in step ST501.
  • the degree of similarity between the generated vehicle peripheral data and the output vehicle peripheral data is calculated (step ST504).
  • the similarity calculation unit 13 outputs the calculated similarity to the determination unit 14.
  • the determination unit 14 outputs the driving support information inferred by the first inference unit 121 in step ST502 by comparing the similarity calculated by the similarity calculation unit 13 in step ST504 with the inference result determination threshold value. It is determined whether or not to do so (step ST505). The determination unit 14 outputs the determination result of whether or not to output the driving support information inferred by the first inference unit 121 to the output unit 15.
  • the output unit 15 When the determination unit 14 determines in step ST505 that the driving support information inferred by the first inference unit 121 is to be output, the output unit 15 outputs the driving support information to the driving support device 100 (step ST506). .. When the determination unit 14 determines that the driving support information inferred by the first inference unit 121 is not output, the output unit 15 outputs notification information notifying the driving support device 100 that automatic driving is not possible. ..
  • the first inference unit 121 of the inference unit 12 determines whether or not the determination unit 14 outputs the driving support information (step ST505).
  • the driving support information is inferred (step ST502), and when the determination unit 14 determines that the first inference result is to be output, the first inference unit 121 is supposed to output the inferred driving support information.
  • the first inference unit 121 of the inference unit 12 may infer the driving support information when the determination unit 14 determines that the driving support information is output.
  • FIG. 6 illustrates the operation of the inference device 1 when the first inference unit 121 determines that the determination unit 14 outputs the driving support information and then infers the driving support information in the first embodiment. It is a flowchart for doing.
  • the specific operations of steps ST601 to ST606 are the same as the specific operations of step ST501, step ST503 to step ST505, step ST502, and step ST506 of FIG. 5, respectively.
  • the operation of inferring the driving support information by the first inference unit 121 determines whether or not the determination unit 14 outputs the driving support information (step ST605).
  • Step ST604 The point that is later is different from the operation of the inference device 1 shown in the flowchart of FIG. As shown in FIG. 6, unnecessary driving support information by the first inference unit 121 is obtained by causing the first inference unit 121 to infer the driving support information after the determination unit 14 determines that the driving support information is output. Inference operation can be omitted.
  • the accuracy of machine learning models in supervised learning is enhanced by validation based on supervised test data.
  • the learning data at the time of learning the machine learning model is not biased with respect to the data input in the actual operation. be. If the data that was not considered at the time of learning is input to the machine learning model at the time of inference in actual operation, an erroneous inference result may be output.
  • the inference device 1 according to the first embodiment is inferred by inputting the vehicle peripheral data which is the input of the first machine learning model 16 acquired at the time of inference in the actual operation into the second machine learning model 17.
  • the similarity between the output vehicle peripheral data and the acquired vehicle peripheral data is calculated, and the similarity is compared with the inference result determination threshold.
  • the inference device 1 determines whether or not the acquired vehicle peripheral data has characteristics similar to the learning data of the first machine learning model 16. Then, the inference device 1 determines whether or not to input the acquired vehicle peripheral data into the first machine learning model 16 and output the inferred driving support information.
  • driving support information Is output. That is, the inference device 1 outputs the driving support information while ensuring the reliability of the driving support information. In other words, the inference device 1 can prevent the output of an invalid inference result.
  • the inference device 1 determines that the driving support information is not output as a result of comparing the similarity with the threshold value for determining the inference result, the inference device 1 does not output the driving support information and notifies that automatic driving cannot be performed. Is output.
  • the driving support device 100 prevents the automatic driving control based on the driving support information.
  • the reasoning device 1 notifies that automatic driving is not possible, and causes the driving support device 100, which performs automatic driving control based on the driving support information output by the reasoning device 1, to prevent automatic driving control. The reliability of the automatic operation control in the support device 100 can be improved.
  • the inference device 1 outputs the first inference result (driving support information) by inputting the data to the data acquisition unit 11 that acquires the data (vehicle peripheral data).
  • the inference unit 12 that inputs the data acquired by the data acquisition unit 11 and infers the first inference result
  • the second machine learning model 17 that outputs the second inference result by inputting the data.
  • the degree of similarity between the data acquired by the data acquisition unit 11 and the second inference result based on the second inference result inferred by inputting the data acquired by the data acquisition unit 11 and the data acquired by the data acquisition unit 11.
  • the determination unit 14 that determines whether or not to output the first inference result. And, when it is determined that the determination unit 14 outputs the first inference result, the output unit 15 for outputting the first inference result is provided. Therefore, the inference device 1 can prevent the output of an invalid inference result.
  • Embodiment 2 In the first embodiment, the first machine learning model 16 and the second machine learning model 17 are each one. In the second embodiment, when a plurality of the first machine learning model 16 and the second machine learning model 17 exist, the inference device 1a determines whether or not to output the driving support information. Will be described.
  • FIG. 7 is a diagram showing a configuration example of the inference device 1a according to the second embodiment.
  • the same reference numerals are given to the same configurations as those of the inference device 1 described with reference to FIG. 1 in the first embodiment, and duplicate description will be omitted.
  • the second embodiment there are a plurality of first machine learning models 16-1 to 16-n.
  • the correspondence between the first machine learning models 16-1 to 16-n and the second machine learning models 17-1 to 17-n means that the first machine learning models 16-1 to 16-1 to learned based on the same learning data. It means that it is 16-n and the second machine learning model 17-1 to 17-n. That is, in the second embodiment, there are a plurality of paired first machine learning models 16-1 to 16-n and second machine learning models 17-1 to 17-n learned based on the same learning data. There are pairs.
  • the plurality of first machine learning models 16-1 to 16-n are machine learning models used for the same purpose.
  • the plurality of first machine learning models 16-1 to 16-n have different learning data input during learning.
  • the first machine learning models 16-1 to 16-n are used for the purpose of acquiring information indicating that an object has been recognized, like the first machine learning model 16 of the first embodiment.
  • Use a machine learning model as an example, the plurality of first machine learning models 16-1 to 16-n are collectively referred to as the first machine learning model 16.
  • the plurality of second machine learning models 17-1 to 17-n are collectively referred to as a second machine learning model 17.
  • the inference device 1a performs parallel calculation or sequential calculation on the second machine learning model 17, and one of the plurality of second machine learning models 17 is the second machine learning model 17 (hereinafter, “selected second machine”). "Learning model”) is selected.
  • the inference device 1a selects the first machine learning model 16 (hereinafter, referred to as “selection first machine learning model”) corresponding to the selection second machine learning model.
  • selection first machine learning model the first machine learning model 16
  • the inference device 1a determines whether or not to output the driving support information inferred by the selected first machine learning model. Details of parallel calculation or sequential calculation performed by the inference device 1a on the second machine learning model 17 and a method of selecting the selected second machine learning model and the selected first machine learning model in the inference device 1a will be described later. do.
  • the inference device 1a according to the second embodiment is different from the inference device 1 according to the first embodiment in that it includes a representative similarity calculation unit 18 and a provisional second model selection unit 19. Further, the inference device 1a according to the second embodiment is different from the inference device 1 according to the first embodiment in that the determination unit 14a includes the first model selection unit 141 and the second model selection unit 142. Further, the inference device 1a according to the second embodiment is different from the inference device 1 according to the first embodiment in the operations of the inference unit 12 and the similarity calculation unit 13.
  • the first inference unit 121 of the inference unit 12 infers driving support information for each first machine learning model 16. Since the specific method in which the first reasoning unit 121 infers the driving support information using the first machine learning model 16 has already been described in the first embodiment, detailed description thereof will be omitted.
  • the first inference unit 121 outputs the inferred driving support information for each first machine learning model 16 to the determination unit 14a.
  • the first inference unit 121 associates the driving support information with the identifiable information of the first machine learning model 16 inferring the driving support information, and outputs the driving support information to the determination unit 14a.
  • the second inference unit 122 of the inference unit 12 infers the output vehicle peripheral data for each second machine learning model 17. Since the specific method in which the second reasoning unit 122 infers the output vehicle peripheral data using the second machine learning model 17 has already been described in the first embodiment, detailed description thereof will be omitted.
  • the second inference unit 122 outputs the inferred output vehicle peripheral data for each second machine learning model 17 to the similarity calculation unit 13.
  • the second inference unit 122 outputs the output vehicle peripheral data to the similarity calculation unit 13 in association with the second machine learning model 17 inferring the output vehicle peripheral data with identifiable information.
  • the similarity calculation unit 13 calculates the similarity for all the second machine learning models 17. Specifically, the similarity calculation unit 13 is based on the output vehicle peripheral data inferred by the second inference unit 122 and the vehicle peripheral data acquired by the data acquisition unit 11 for each second machine learning model 17. The degree of similarity between the vehicle peripheral data acquired by the data acquisition unit 11 and the inferred output vehicle peripheral data is calculated. Since the specific method for calculating the similarity by the similarity calculation unit 13 has already been described in the first embodiment, detailed description thereof will be omitted.
  • the similarity calculation unit 13 outputs the calculated similarity for each second machine learning model 17 to the determination unit 14a.
  • the similarity calculation unit 13 outputs the similarity for each second machine learning model 17 to the determination unit 14a in association with the information that can identify the second machine learning model 17.
  • the configuration of the inference device 1a is divided into a case where the inference device 1a performs parallel calculation and selects the selected second machine learning model and a case where the inference device 1a performs sequential calculation and selects the selected second machine learning model.
  • I will explain. First, each configuration of the inference device 1a when the inference device 1a performs parallel calculation and selects the selected second machine learning model will be described. Of the configurations of the inference device 1a, the same configuration as that of the inference device 1 that has already been explained will not be duplicated.
  • the determination unit 14a sets an inference result determination threshold value for each pair of the corresponding first machine learning model 16 and the second machine learning model 17. Since the method in which the determination unit 14a sets the inference result determination threshold value is the same as the inference result determination threshold value by the determination unit 14 described in the first embodiment, detailed description thereof will be omitted.
  • the inference result determination threshold value may be set in advance and stored in a storage unit (not shown).
  • the second model selection unit 142 of the determination unit 14a selects from the second machine learning model 17 by comparing the similarity calculated by the similarity calculation unit 13 with the inference result determination threshold value.
  • FIG. 8 shows, in the second embodiment, the second model selection unit 142 selects the second machine by comparing the similarity of each second machine learning model 17 calculated by the similarity calculation unit 13 with the threshold value for determining the inference result. It is a figure which shows an example of the image of the method of selecting a learning model. For example, the second model selection unit 142 selects one of the plurality of second machine learning models 17 whose similarity is larger than the threshold for determining the inference result as the selected second machine learning model. do.
  • the second model selection unit 142 uses, for example, the second machine learning model 17 for which the largest similarity is calculated. Selection Select as the second machine learning model.
  • the second model selection unit 142 outputs information about the selected selected second machine learning model to the first model selection unit 141.
  • the second model selection unit 142 does not select the selected second machine learning model when there is no second machine learning model 17 whose similarity is larger than the threshold for determining the inference result.
  • the second model selection unit 142 outputs information to the effect that the selected second machine learning model has not been selected to the first model selection unit 141.
  • the first model selection unit 141 selects the selected first machine learning model corresponding to the selected second machine learning model selected by the second model selection unit 142.
  • the first model selection unit 141 outputs information regarding the selected selected first machine learning model to the determination unit 14a.
  • the first model selection unit 141 does not select the selected first machine learning model when the information indicating that the selected second machine learning model has not been selected is output from the second model selection unit 142.
  • the first model selection unit 141 outputs information to the effect that the selected first machine learning model has not been selected to the determination unit 14a.
  • the determination unit 14a When the determination unit 14a outputs the information about the selected first machine learning model from the first model selection unit 141, the determination unit 14a outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model. to decide. When the determination unit 14a outputs the information that the selected first machine learning model has not been selected from the first model selection unit 141, the determination unit 14a outputs all the driving support information inferred by the first inference unit 121. Judge not to. The determination unit 14a outputs to the output unit 15 the determination result of whether or not to output the driving support information inferred by the first inference unit 121.
  • the determination unit 14a determines that the driving support information inferred by the first inference unit 121 is to be output, the determination unit 14a associates the driving support information determined to be output and outputs the driving support information to the output unit 15. Specifically, the determination unit 14a outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model.
  • each configuration of the inference device 1a when the inference device 1a performs sequential calculation to select the selected second machine learning model will be described.
  • the same configuration as that of the inference device 1 described in the first embodiment will not be duplicated.
  • the vehicle peripheral data as the learning data of the second machine learning model 17 and the learning are performed in advance for each second machine learning model 17.
  • the similarity with the output vehicle peripheral data, which is the output data inferred by inputting the time data into the second machine learning model 17, (hereinafter referred to as “learning data similarity”) is calculated, and is used as the learning similarity information. , It is stored in the storage part.
  • the learning similarity information includes the learning data similarity calculated based on the learning data of all the second machine learning models 17 for a certain second machine learning model 17. Specifically, for example, in the learning similarity information, regarding the second machine learning model 17-1, the learning data of the second machine learning model 17-1 and the learning time of the second machine learning model 17-1 are included. Data similarity with the output data inferred by inputting the data into the second machine learning model 17-1, the learning data of the second machine learning model 17-2, and the second machine learning model 17-2.
  • the training data similarity with the output data inferred by inputting the training data of the machine learning model 17-n into the second machine learning model 17-1 is included.
  • the learning similarity information includes the learning data of the second machine learning model 17-1 and the learning time of the second machine learning model 17-1. Data similarity with the output data inferred by inputting the data into the second machine learning model 17-1, the learning data of the second machine learning model 17-2, and the second machine learning model 17-2.
  • the training data similarity with the output data inferred by inputting the training data into the second machine learning model 17-1, ..., the training data of the second machine learning model 17-n, and the second The training data similarity with the output data inferred by inputting the training data of the machine learning model 17-n into the second machine learning model 17-1 is included.
  • the learning similarity information is generated, for example, during learning of the second machine learning model 17.
  • the representative similarity calculation unit 18 uses the vehicle peripheral data acquired by the data acquisition unit 11 and the vehicle peripheral data as a second machine learning model 17 among a plurality of second machine learning models 17 (hereinafter, “representative second”).
  • the degree of similarity with the output vehicle peripheral data (hereinafter referred to as “representative similarity") inferred by inputting into the "machine learning model”) is calculated.
  • the representative similarity calculation unit 18 uses any second machine learning model 17 among the plurality of second machine learning models 17 as the representative second machine learning model.
  • the representative similarity calculation unit 18 may calculate the representative similarity in the same manner as the similarity calculation unit 13 calculates the similarity.
  • the representative similarity calculation unit 18 outputs the calculated representative similarity to the provisional second model selection unit 19.
  • the provisional second model selection unit 19 is one of a plurality of second machine learning models 17 based on the representative similarity calculated by the representative similarity calculation unit 18 and the learning similarity information, and the second machine.
  • a learning model 17 (hereinafter referred to as a “provisional second machine learning model”) is selected.
  • the provisional second model selection unit 19 specifies the learning data similarity closest to the representative similarity among the learning data similarity included in the learning similarity information.
  • the provisional second model selection unit 19 selects the second machine learning model 17 for which the specified learning data similarity is calculated as the provisional second machine learning model.
  • FIG. 9 shows, in the second embodiment, the provisional second model selection unit 19 uses the provisional second machine learning model based on the representative similarity calculated by the representative similarity calculation unit 18 and the learning similarity information. It is a figure which shows an example of the image of the selection method. As an example, FIG. 9 shows that the learning data similarity closest to the representative similarity calculated by the representative similarity calculation unit 18 is the learning data of the second machine learning model (2) 17-2 and the learning data. Was input to the second machine learning model (2) 17-2 to show that the similarity with the inferred output data was the closest. In this case, the provisional second model selection unit 19 selects the second machine learning model (2) 17-2 as the provisional second machine learning model. The provisional second model selection unit 19 outputs information regarding the selected provisional second machine learning model to the determination unit 14a.
  • the second model selection unit 142 of the determination unit 14a selects the selected second machine learning model by comparing the similarity calculated based on the provisional second machine learning model with the inference result determination threshold value.
  • the similarity calculated based on the provisional second machine learning model is obtained by inputting the vehicle peripheral data acquired by the data acquisition unit 11 calculated by the similarity calculation unit 13 and the vehicle peripheral data into the provisional second machine learning model. It is the degree of similarity with the output vehicle peripheral data inferred.
  • the second model selection unit 142 selects the provisional second machine learning model as the selection second machine learning model.
  • the second model selection unit 142 outputs information about the selected selected second machine learning model to the first model selection unit 141.
  • the second model selection unit 142 does not select the selected second machine learning model when the similarity calculated based on the provisional second machine learning model is equal to or less than the inference result determination threshold value.
  • the second model selection unit 142 outputs information to the effect that the selected second machine learning model has not been selected to the first model selection unit 141.
  • the first model selection unit 141 selects the selected first machine learning model corresponding to the selected second machine learning model selected by the second model selection unit 142.
  • the first model selection unit 141 outputs information regarding the selected selected first machine learning model to the determination unit 14a.
  • the first model selection unit 141 does not select the selected first machine learning model when the information indicating that the selected second machine learning model has not been selected is output from the second model selection unit 142.
  • the first model selection unit 141 outputs information to the effect that the selected first machine learning model has not been selected to the determination unit 14a.
  • the determination unit 14a When the determination unit 14a outputs the information about the selected first machine learning model from the first model selection unit 141, the determination unit 14a outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model. to decide. When the determination unit 14a outputs the information that the selected first machine learning model has not been selected from the first model selection unit 141, the determination unit 14a outputs all the driving support information inferred by the first inference unit 121. Judge not to. The determination unit 14a outputs to the output unit 15 the determination result of whether or not to output the driving support information inferred by the first inference unit 121.
  • the determination unit 14a determines that the driving support information inferred by the first inference unit 121 is to be output, the determination unit 14a associates the driving support information determined to be output and outputs the driving support information to the output unit 15. Specifically, the determination unit 14a outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model.
  • the representative similarity calculation unit 18 and the provisional second model selection unit 19 function only when the inference device 1a performs sequential calculation to select the selected second machine learning model.
  • the inference device 1a may be configured not to include the representative similarity calculation unit 18 and the provisional second model selection unit 19. good. In this case, it is not essential that the similarity information at the time of learning is stored.
  • the inference device 1a performs sequential calculation to select the selected second machine learning model
  • the inference device 1a selects the selected second machine learning model without calculating the similarity based on the acquired vehicle peripheral data and all the second machine learning models. Since the second machine learning model can be selected, the processing load can be reduced as compared with the case where the parallel calculation is performed and the selected second machine learning model is selected.
  • FIG. 10 is a flowchart for explaining the operation of the inference device 1a according to the second embodiment.
  • the specific operations of steps ST1001 and ST10008 are the same as the specific operations of steps ST501 and ST506 of FIG. 5 described in the first embodiment, and thus duplicated description will be omitted. ..
  • the first inference unit 121 of the inference unit 12 infers the driving support information for each first machine learning model 16 (step ST1002).
  • the first inference unit 121 outputs the inferred driving support information for each first machine learning model 16 to the determination unit 14a.
  • the first inference unit 121 associates the first machine learning model 16 with the identifiable information and outputs the driving support information to the determination unit 14a.
  • the second inference unit 122 of the inference unit 12 infers the output vehicle peripheral data for each second machine learning model 17 (step ST1003).
  • the second inference unit 122 outputs the inferred output vehicle peripheral data to the similarity calculation unit 13.
  • the second inference unit 122 outputs the output vehicle peripheral data to the similarity calculation unit 13 in association with the information that can identify the second machine learning model 17.
  • the similarity calculation unit 13 calculates the similarity for all the second machine learning models 17 (step ST1004).
  • the similarity calculation unit 13 outputs the calculated similarity for each second machine learning model 17 to the determination unit 14a.
  • the similarity calculation unit 13 outputs the similarity for each second machine learning model 17 to the determination unit 14a in association with the information that can identify the second machine learning model 17.
  • the second model selection unit 142 of the determination unit 14a compares the similarity calculated by the similarity calculation unit 13 in step ST1004 with the inference result determination threshold value, and is among the plurality of second machine learning models 17. Select from Select the second machine learning model (step ST1005).
  • FIG. 11 is a flowchart for explaining the specific operation of step ST1005 of FIG. 10 when the inference device 1a performs parallel calculation and selects the selected second machine learning model.
  • the second model selection unit 142 of the determination unit 14a compares the similarity calculated by the similarity calculation unit 13 in step ST1004 of FIG. 10 with the inference result determination threshold value, and thus is among the second machine learning models 17. Select from Select the second machine learning model.
  • the second model selection unit 142 determines whether or not there is a second machine learning model 17 whose similarity is larger than the inference result determination threshold value among the plurality of second machine learning models 17 (step). ST1101).
  • the second model selection unit 142 has a similarity degree for inference result determination.
  • the second machine learning model 17 larger than the threshold value is selected as the selected second machine learning model (step ST1102).
  • the second model selection unit 142 outputs information about the selected selected second machine learning model to the first model selection unit 141.
  • step ST1101 when there is no second machine learning model 17 whose similarity is larger than the inference result determination threshold value (when “NO” in step ST1101), the second model selection unit 142 selects the second machine learning model. Do not select. The second model selection unit 142 outputs information to the effect that the selected second machine learning model has not been selected to the first model selection unit 141. The operation of the inference device 1a proceeds to step ST1006 in FIG.
  • FIG. 12 is a flowchart for explaining the specific operation of step ST1005 of FIG. 10 when the inference device 1a selects the selected second machine learning model for which sequential calculation has been performed.
  • the representative similarity calculation unit 18 combines the vehicle peripheral data acquired by the data acquisition unit 11 in step ST1001 of FIG. 10 with the output vehicle peripheral data inferred by inputting the vehicle peripheral data into the representative second machine learning model.
  • the representative similarity is calculated (step ST1201).
  • the representative similarity calculation unit 18 outputs the calculated representative similarity to the provisional second model selection unit 19.
  • the provisional second model selection unit 19 is one of a plurality of second machine learning models 17 based on the representative similarity calculated by the representative similarity calculation unit 18 in step ST1201 and the learning similarity information. , Select a provisional second machine learning model (step ST1202). Specifically, the provisional second model selection unit 19 specifies the learning data similarity closest to the representative similarity among the learning data similarity included in the learning similarity information. The provisional second model selection unit 19 selects the second machine learning model 17 for which the specified learning data similarity is calculated as the provisional second machine learning model. The provisional second model selection unit 19 outputs information regarding the selected provisional second machine learning model to the determination unit 14a.
  • the second model selection unit 142 of the determination unit 14a selects the selected second machine learning model by comparing the similarity calculated based on the provisional second machine learning model with the inference result determination threshold value. Specifically, for example, the second model selection unit 142 determines whether or not the similarity calculated based on the provisional second machine learning model is larger than the inference result determination threshold value (step ST1203). In step ST1203, when it is determined that the similarity calculated based on the provisional second machine learning model is larger than the inference result determination threshold (when “YES” in step ST1203), the second model selection unit 142 determines. Select the provisional second machine learning model Select as the second machine learning model. The second model selection unit 142 outputs information about the selected selected second machine learning model to the first model selection unit 141.
  • step ST1203 when it is determined that the similarity calculated based on the provisional second machine learning model is equal to or less than the inference result determination threshold value (when “NO” in step ST1203), the second model selection unit 142 determines. Selection The second machine learning model is not selected. The second model selection unit 142 outputs information to the effect that the selected second machine learning model has not been selected to the first model selection unit 141. The operation of the inference device 1a proceeds to step ST1006 in FIG.
  • the first model selection unit 141 selects the selected first machine learning model corresponding to the selected second machine learning model selected by the second model selection unit 142 (step ST1006).
  • the first model selection unit 141 outputs information regarding the selected selected first machine learning model to the determination unit 14a.
  • the first model selection unit 141 does not select the selected first machine learning model when the information indicating that the selected second machine learning model has not been selected is output from the second model selection unit 142.
  • the first model selection unit 141 outputs information to the effect that the selected first machine learning model has not been selected to the determination unit 14a.
  • the determination unit 14a determines whether or not to output the driving support information (step ST1007). Specifically, when the determination unit 14a outputs information about the selected first machine learning model from the first model selection unit 141 in step ST1006, the first inference unit 121 becomes the selected first machine learning model. It is judged that the driving support information inferred based on the output is output. When the determination unit 14a outputs the information that the selected first machine learning model has not been selected from the first model selection unit 141 in step ST1006, the driving support information inferred by the first inference unit 121 Judges that neither is output. The determination unit 14a outputs to the output unit 15 the determination result of whether or not to output the driving support information inferred by the first inference unit 121.
  • the determination unit 14a determines that the driving support information inferred by the first inference unit 121 is to be output, the determination unit 14a associates the driving support information determined to be output and outputs the driving support information to the output unit 15. Specifically, the determination unit 14a outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model.
  • the first inference unit 121 of the inference unit 12 determines whether or not the determination unit 14a outputs the driving support information (step ST1007).
  • the first reasoning unit 121 is supposed to output the inferred driving support information. It's just an example.
  • the first inference unit 121 of the inference unit 12 may infer the driving support information when it is determined that the determination unit 14a outputs the driving support information.
  • FIG. 13 illustrates the operation of the inference device 1a when the first inference unit 121 determines that the determination unit 14a outputs the driving support information and then infers the driving support information in the second embodiment. It is a flowchart for doing.
  • the specific operations of steps ST1301 to ST1308 are the same as the specific operations of step ST1001, step ST1003 to step ST1007, step ST1002, and step ST1008 in FIG. 10, respectively.
  • the operation of inferring the driving support information by the first inference unit 121 determines whether or not the determination unit 14a outputs the driving support information (step ST1307).
  • Step ST1306) The point that is later is different from the operation of the inference device 1a shown in the flowchart of FIG. As shown in FIG. 13, unnecessary driving support information by the first inference unit 121 is obtained by causing the first inference unit 121 to infer the driving support information after determining that the determination unit 14a outputs the driving support information. Inference operation can be omitted.
  • the second machine learning model 17 Each time, the similarity between the output vehicle peripheral data inferred by inputting the vehicle peripheral data acquired during actual operation into the second machine learning model 17 and the acquired vehicle peripheral data is calculated, and the similarity and the inference result are determined.
  • the second machine learning model selected from the plurality of second machine learning models 17 is selected by comparing with the threshold value. Then, the inference device 1a selects the selected first machine learning model corresponding to the selected second machine learning model, and outputs the driving support information inferred by the first inference unit 121 based on the selected first machine learning model.
  • the inference device 1a has a feature that the acquired vehicle peripheral data is close to the learning data of the first machine learning model 16 even when a plurality of the first machine learning model 16 and the second machine learning model 17 exist.
  • the driving support information is output. That is, the inference device 1a outputs the driving support information while ensuring the reliability of the driving support information. In other words, the inference device 1a can prevent the output of an invalid inference result.
  • the inference device 1a does not output the driving support information and outputs the notification information notifying that the automatic operation cannot be performed when it is determined that the driving support information is not output. ..
  • the driving support device 100 prevents the automatic driving control based on the driving support information.
  • the inference device 1a notifies that automatic operation is not possible, and causes the operation support device 100, which performs automatic operation control based on the operation support information output by the inference device 1a, to prevent automatic operation control.
  • the reliability of the automatic operation control in the support device 100 can be improved.
  • the inference unit 12 can infer the first inference result (driving support information) for each first machine learning model 16, and the similarity calculation unit 13 calculates the similarity for each second machine learning model 17. It is possible, and the determination unit 14a selects from a plurality of second machine learning models 17 by comparing the similarity calculated by the similarity calculation unit 13 with the inference result determination threshold. Selects the selected first machine learning model corresponding to the selected second machine learning model selected by the second model selection unit 142 from the second model selection unit 142 for selecting and the plurality of first machine learning models 16.
  • the first model selection unit 141 is provided, and the inference unit 12 is configured to determine whether or not to output the first inference result inferred based on the selected first machine learning model selected by the first model selection unit 141. bottom. Therefore, the inference device 1a can prevent the output of an invalid inference result.
  • FIG. 14A and 14B are diagrams showing an example of the hardware configuration of the inference device 1 according to the first embodiment or the inference device 1a according to the second embodiment.
  • the data acquisition unit 11, the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, the output unit 15, the representative similarity calculation unit 18, and the provisional second model selection unit 19 The function is realized by the processing circuit 1401. That is, the inference devices 1 and 1a output the driving support information inferred based on the first machine learning model 16 by comparing the similarity calculated based on the second machine learning model 17 with the threshold value for determining the inference result.
  • It is provided with a processing circuit 1401 for performing control for determining whether or not.
  • the processing circuit 1401 may be dedicated hardware as shown in FIG. 14A, or may be a CPU (Central Processing Unit) 1405 that executes a program stored in the memory 1406 as shown in FIG. 14B.
  • CPU Central Processing Unit
  • the processing circuit 1401 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable). Gate Array) or a combination of these is applicable.
  • the processing circuit 1401 is the CPU 1405, the data acquisition unit 11, the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, the output unit 15, the representative similarity calculation unit 18, and the provisional second model selection unit 19
  • the function is realized by software, firmware, or a combination of software and firmware. That is, the data acquisition unit 11, the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, the output unit 15, the representative similarity calculation unit 18, and the provisional second model selection unit 19 are HDDs (Hard Disk Drive). ) 1402, a CPU 1405 that executes a program stored in a memory 1406 or the like, and a processing circuit 1401 such as a system LSI (Large-Scale Integration).
  • LSI Large-Scale Integration
  • the programs stored in the HDD 1402, the memory 1406, etc. are the data acquisition unit 11, the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, the output unit 15, the representative similarity calculation unit 18, and the provisional first. 2 It can also be said that the procedure or method of the model selection unit 19 is executed by a computer.
  • the memory 1406 is, for example, a RAM, a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Emergency Memory), an EEPROM (Electrically Emergency Memory), a volatile Memory, etc.
  • a semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versaille Disc), or the like is applicable.
  • the data acquisition unit 11, the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, the output unit 15, the representative similarity calculation unit 18, and the provisional second model selection unit 19 are partially described. It may be realized by dedicated hardware, and partly by software or firmware.
  • the data acquisition unit 11 and the output unit 15 are realized by the processing circuit 1401 as dedicated hardware, and the inference unit 12, the similarity calculation unit 13, the judgment units 14, 14a, and the representative similarity calculation unit 18 are realized.
  • the function of the provisional second model selection unit 19 can be realized by the processing circuit 1401 reading and executing the program stored in the memory 1406.
  • the inference devices 1 and 1a include a device such as a driving support device 100, an input interface device 1403 and an output interface device 1404 that perform wired communication or wireless communication.
  • the second machine learning model 17 uses the same input data as the first machine learning model 16, and the input data and the output data which is the second inference result become equal.
  • the second machine learning model 17 may be a binary classification machine learning model in which the input data has been learned as a correct answer label.
  • the binary classification machine learning model when all the input data is learned as the correct answer and the input data containing the feature amount different from the input data input at the time of learning is input at the time of inference based on the machine learning model, the input data is input. Outputs information indicating an abnormality.
  • a typical method of a binary classification machine learning model is a method called OC-SVM (One Class Support Vector Machine).
  • the inference devices 1 and 1a are provided in the driving support device 100 mounted on the vehicle, but this is only an example.
  • the inference devices 1, 1a are provided in the server, and the driving support information is output to the driving support device 100 mounted on the vehicle, which is an external device of the inference devices 1, 1a, via the network. May be good.
  • FIG. 15 shows an inference system in which the inference device 1 according to the first embodiment or the inference device 1a according to the second embodiment is provided in the server 300, and the server 300 and the vehicle (not shown) are connected via a network. It is a figure which shows the configuration example.
  • the inference devices 1 and 1a are provided in the driving support device 100 mounted on the vehicle, and the driving support information inferred based on the first machine learning model 16 is provided. It was decided whether or not to output, but this is just an example.
  • the inference devices 1 and 1a according to the first embodiment and the second embodiment can be applied to various devices that need to control the output of information inferred based on the trained model.
  • the inference device Since the inference device according to the present disclosure is configured to prevent the output of invalid inference results, it can be applied to an inference device that makes inferences using a machine learning model.
  • 1,1a inference device 11 data acquisition unit, 12 inference unit, 121 first inference unit, 122 second inference unit, 13 similarity calculation unit, 14,14a judgment unit, 141 first model selection unit, 142 second model Selection unit, 15 output unit, 16 first machine learning model, 17 second machine learning model, 18 representative similarity calculation unit, 19 provisional second model selection unit, 100 operation support device, 300 server, 1401 processing circuit, 1402 HDD 1,403 input interface device, 1404 output interface device, 1405 CPU, 1406 memory.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Automation & Control Theory (AREA)
  • Analytical Chemistry (AREA)
  • Computational Linguistics (AREA)
  • Mechanical Engineering (AREA)
  • Atmospheric Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Chemical & Material Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Transportation (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Traffic Control Systems (AREA)
PCT/JP2020/007607 2020-02-26 2020-02-26 推論装置、運転支援装置、推論方法、および、サーバ Ceased WO2021171398A1 (ja)

Priority Applications (5)

Application Number Priority Date Filing Date Title
CN202080097069.4A CN115136161A (zh) 2020-02-26 2020-02-26 推理装置、驾驶辅助装置、推理方法和服务器
DE112020006796.3T DE112020006796T5 (de) 2020-02-26 2020-02-26 Inferenzvorrichtung, fahrassistenzvorrichtung, inferenzverfahren und server
JP2022502639A JP7254240B2 (ja) 2020-02-26 2020-02-26 推論装置、運転支援装置、推論方法、および、サーバ
PCT/JP2020/007607 WO2021171398A1 (ja) 2020-02-26 2020-02-26 推論装置、運転支援装置、推論方法、および、サーバ
US17/790,297 US20230035526A1 (en) 2020-02-26 2020-02-26 Inference device, driving assistance device, inference method, and server

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2020/007607 WO2021171398A1 (ja) 2020-02-26 2020-02-26 推論装置、運転支援装置、推論方法、および、サーバ

Publications (1)

Publication Number Publication Date
WO2021171398A1 true WO2021171398A1 (ja) 2021-09-02

Family

ID=77490839

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2020/007607 Ceased WO2021171398A1 (ja) 2020-02-26 2020-02-26 推論装置、運転支援装置、推論方法、および、サーバ

Country Status (5)

Country Link
US (1) US20230035526A1 (https=)
JP (1) JP7254240B2 (https=)
CN (1) CN115136161A (https=)
DE (1) DE112020006796T5 (https=)
WO (1) WO2021171398A1 (https=)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023051582A (ja) * 2021-09-30 2023-04-11 太陽誘電株式会社 判定装置、判定システム、判定方法およびプログラム
WO2024190519A1 (ja) * 2023-03-15 2024-09-19 オムロン株式会社 移動経路決定システム、自律型移動装置、移動経路決定方法、プログラム

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7435793B2 (ja) * 2020-08-05 2024-02-21 日本電信電話株式会社 推論処理装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005309765A (ja) * 2004-04-21 2005-11-04 Fuji Xerox Co Ltd 画像認識装置、画像抽出装置、画像抽出方法及びプログラム
JP2015103188A (ja) * 2013-11-27 2015-06-04 株式会社リコー 画像解析装置、画像解析方法及び画像解析プログラム
JP2019091236A (ja) * 2017-11-14 2019-06-13 富士通株式会社 特徴抽出装置、特徴抽出プログラム、および特徴抽出方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6661398B2 (ja) * 2016-02-03 2020-03-11 キヤノン株式会社 情報処理装置および情報処理方法
US20170330109A1 (en) 2016-05-16 2017-11-16 Purepredictive, Inc. Predictive drift detection and correction
JP6224811B1 (ja) * 2016-12-07 2017-11-01 たけおかラボ株式会社 ルールセットを選択可能な推論エンジンを有するプログラム、装置及び方法
JP6760200B2 (ja) * 2017-05-16 2020-09-23 富士通株式会社 推定プログラム、推定方法および推定装置
JP6853148B2 (ja) 2017-09-07 2021-03-31 日本電信電話株式会社 検知装置、検知方法及び検知プログラム
JP2019087012A (ja) 2017-11-07 2019-06-06 キヤノン株式会社 情報処理装置、情報処理方法、コンピュータプログラム、及び記憶媒体
US12293260B2 (en) * 2017-11-21 2025-05-06 Amazon Technologies, Inc. Generating and deploying packages for machine learning at edge devices
US11537931B2 (en) * 2017-11-29 2022-12-27 Google Llc On-device machine learning platform to enable sharing of machine-learned models between applications
US11410063B2 (en) * 2018-09-28 2022-08-09 B.yond, Inc. Self-intelligent improvement in predictive data models
US20200026289A1 (en) * 2019-09-28 2020-01-23 Ignacio J. Alvarez Distributed traffic safety consensus

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005309765A (ja) * 2004-04-21 2005-11-04 Fuji Xerox Co Ltd 画像認識装置、画像抽出装置、画像抽出方法及びプログラム
JP2015103188A (ja) * 2013-11-27 2015-06-04 株式会社リコー 画像解析装置、画像解析方法及び画像解析プログラム
JP2019091236A (ja) * 2017-11-14 2019-06-13 富士通株式会社 特徴抽出装置、特徴抽出プログラム、および特徴抽出方法

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023051582A (ja) * 2021-09-30 2023-04-11 太陽誘電株式会社 判定装置、判定システム、判定方法およびプログラム
JP7744197B2 (ja) 2021-09-30 2025-09-25 太陽誘電株式会社 判定装置、判定システム、判定方法およびプログラム
WO2024190519A1 (ja) * 2023-03-15 2024-09-19 オムロン株式会社 移動経路決定システム、自律型移動装置、移動経路決定方法、プログラム

Also Published As

Publication number Publication date
CN115136161A (zh) 2022-09-30
JP7254240B2 (ja) 2023-04-07
JPWO2021171398A1 (https=) 2021-09-02
DE112020006796T5 (de) 2022-12-29
US20230035526A1 (en) 2023-02-02

Similar Documents

Publication Publication Date Title
JP7254240B2 (ja) 推論装置、運転支援装置、推論方法、および、サーバ
US20220114627A1 (en) Methods and systems for automatic processing of images of a damaged vehicle and estimating a repair cost
US10997717B2 (en) Method and system for generating a confidence score using deep learning model
CN114186090A (zh) 一种图像标注数据的智能质检方法及系统
US20110025483A1 (en) Driving support system, driving support apparatus, method, and computer-readable medium
WO2020166278A1 (ja) システム、訓練装置、訓練方法及び予測装置
US10788828B2 (en) Method and apparatus for operating autonomous driving controller of vehicle
US20250077778A1 (en) Producing calibrated confidence estimates for open-ended answers by generative artificial intelligence models
EP3869422A1 (en) Model update support method, model update support apparatus, and model update support program
JP2019086475A (ja) 学習プログラム、検出プログラム、学習方法、検出方法、学習装置および検出装置
US20240037388A1 (en) Method of learning neural network, feature selection apparatus, feature selection method, and recording medium
KR102384797B1 (ko) 세그멘테이션 레이블링 데이터 기초한 학습 방법 및 이를 이용한 자율 주행 장치
CN119414432B (zh) 静默式行泊车辆定位切换方法、装置、设备及存储介质
US20230230166A1 (en) Methods and systems for automatic classification of a level of vehicle damage
US20240037389A1 (en) Method of learning neural network, feature selection apparatus, feature selection method, and recording medium
JP7214018B2 (ja) 物体認識装置、運転支援装置、サーバ、および、物体認識方法
CN118876953A (zh) 自动泊车方法、装置、设备及存储介质
JP2023129312A (ja) 分類装置、分類方法、及びプログラム
CN113128541B (zh) 一种标志物的匹配方法及装置
CN115782472A (zh) 胎压调节方法、存储介质、控制器及车辆
JP7233606B2 (ja) 教師データ生成装置、および、教師データ生成方法
CN119580330B (zh) 稳健适应年龄性别差异的亲属识别方法及装置
JP7676977B2 (ja) 情報処理装置及び情報処理方法
CN114001939B (zh) 一种线路连接装置监测方法、装置、设备及存储介质
US20260011116A1 (en) Information processing apparatus, information processing method, and non-transitory computer-readable storage medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20921193

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022502639

Country of ref document: JP

Kind code of ref document: A

122 Ep: pct application non-entry in european phase

Ref document number: 20921193

Country of ref document: EP

Kind code of ref document: A1