JP7204818B2 - Vehicle control system and vehicle control method - Google Patents

Description

TECHNICAL FIELD The present disclosure relates to a vehicle control system and a vehicle control method that implement cruise control of a vehicle that appropriately responds to the behavior of surrounding objects existing around the vehicle.

In recent years, it is related to driving control that recognizes surrounding objects around the own vehicle and appropriately controls the behavior of the own vehicle by using information acquired by external information acquisition sensors such as cameras and radars installed in the vehicle as input. Technology development is progressing. Here, driving control includes, for example, ADAS (Advanced Driver Assistance System) or AD (Autonomous Driving).

Recognition of surrounding objects in the vicinity of the vehicle is based on the output results of a rule-based model that recognizes surrounding objects based on information acquired by sensors that acquire information outside the vehicle, and in recent years, neural networks have been used. It uses the output results of machine learning models. If a device (hereinafter referred to as a "recognition device") that performs calculations in a model that performs such object recognition (hereinafter referred to as an "object recognition model") fails, recognition will not be consistent with the behavior of the actual surrounding objects. There is concern that the results will be output from the object recognition model. Hereinafter, a recognition result that does not match the behavior of the actual surrounding object will be referred to as an "abnormal" recognition result. If an abnormal recognition result is output from the object recognition model, the behavior of surrounding objects cannot be accurately recognized. In this case, there is a possibility that the own vehicle will not respond appropriately to the behavior of the surrounding objects, such as the own vehicle getting too close to the surrounding objects or the own vehicle decelerating in a scene where there are no surrounding objects. be.

Conventionally, there has been disclosed a technology related to handling when an abnormal recognition result is output from a recognition device (see Patent Document 1, for example). Patent Document 1 discloses a technology in which a plurality of recognition devices are provided, and when one recognition device outputs an abnormal recognition result, processing in the recognition device is stopped, and the remaining other recognition devices recognize surrounding objects. disclosed.

JP 2017-182771 A

Even if there is no failure in the recognition device and the calculation of the object recognition model can be executed normally, the recognition device may output an abnormal recognition result. For example, in an object recognition model that recognizes surrounding objects using an image captured by a camera as an input, when light emitted from the surrounding object or natural light suddenly enters the camera, the camera temporarily corrects information about the surrounding environment. Therefore, the surrounding objects that could be recognized by the object recognition model until then may no longer be recognized. In other words, there is a concern that the object recognition model may output a recognition result indicating that the peripheral object has disappeared even though the peripheral object actually continues to exist. In addition, it may not be possible to accurately recognize surrounding objects in a specific scene due to insufficient parameter adjustment in the rule-based model used in the object recognition model or insufficient teacher data used in learning the machine learning model. In other words, the reason why an abnormal recognition result is output is that the object recognition model is not in a scene in which the surrounding objects are accurately recognized, and that the recognition device is out of order.

In Patent Literature 1, when an abnormal recognition result is output from the recognition device, the processing of the recognition device is stopped regardless of the cause of the output of the abnormal recognition result. In other words, in Document 1, it is not possible to determine that the reason why the abnormal recognition result is output is that the object recognition model does not accurately recognize the surrounding objects. If the cause of the output of an abnormal recognition result is that the object recognition model is not in a scene where the surrounding objects can be accurately recognized, it is possible to quickly output a recognition result that is normal simply by leaving the scene. can. However, in Cited Document 1, since the process of the recognition device is stopped, there is a problem that it takes a long time until the recognition result without abnormality is output again.

On the other hand, when an abnormal recognition result is output, if only leaving the scene in which the surrounding objects cannot be accurately recognized, an abnormal recognition result is output due to the failure of the recognition device. It is not possible to determine when If an abnormal recognition result is output due to a failure of the recognition device, restart the recognition device after stopping all calculations performed by the recognition device in order to remove the failure that occurred in the recognition device. It needs to be in a state where it can perform operations again. Hereinafter, the process of making the recognition device ready to perform calculations again is referred to as "reset process". If the recognition device is still malfunctioning, there is a problem that abnormal recognition results continue to be output even if the user leaves a scene in which peripheral objects cannot be accurately recognized.

As described above, depending on whether or not there is an abnormality in the recognition result or whether or not there is a failure in the recognition device, the processing required to quickly acquire a recognition result with no abnormality differs. In order to accurately recognize surrounding objects over a longer period of time and perform optimal driving control of the own vehicle, it is necessary to appropriately determine the recognition result and the state of the recognition device, and perform driving control according to the determination result. There is

The present disclosure is made to solve such problems, and an object thereof is to provide a vehicle control system and a vehicle control method capable of performing optimum running control of own vehicle over a longer period of time. do.

In order to solve the above problems, a vehicle control system according to the present disclosure includes a main processing device that controls running of a vehicle, and a sub-processing device that monitors the state of the main processing device. A main recognition processing unit that recognizes surrounding objects around the vehicle based on information outside the vehicle that indicates the state of the surrounding environment, and a recognition result of the main recognition processing unit, the information outside the vehicle, and vehicle information related to the behavior of the vehicle. and a main operation amount calculation unit for calculating an operation amount for running control. , a control switching unit for switching commands relating to running control to the main processing device or the sub processing device, a sub recognition processing unit for recognizing surrounding objects based on information outside the vehicle according to commands from the control switching unit, and a command from the control switching unit, a sub-operation amount calculation unit that calculates an operation amount for performing travel control based on the information outside the vehicle and the vehicle information; an abnormality determination unit that determines whether there is an abnormality in the recognition result by the main recognition processing unit based on the recognition result by the main recognition processing unit; Main processing based on a scene determination unit that determines whether or not the scene is a recognizable scene that indicates that the recognition result by the unit is a reliable environment, a determination result by the abnormality determination unit, and a determination result by the scene determination unit. It has a failure determination unit that determines whether or not there is a failure in the device, and an instruction switching unit that commands the main processing unit or the sub-processing unit to perform running control based on the determination result of the failure determination unit .

Advantageous Effects of Invention According to the present disclosure, it is possible to perform optimal travel control of the own vehicle over a longer period of time.

1 is a block diagram showing the overall configuration of a vehicle control system according to an embodiment; FIG. 4 is a flow chart showing the operation of the vehicle control system according to the embodiment; 4 is a flow chart showing the operation of an abnormality determination unit according to the embodiment; FIG. 5 is a diagram for explaining scene determination in a scene determination unit according to the embodiment; It is a figure which shows the content of the determination in the failure determination part by embodiment. FIG. 10 is a diagram showing contents of commands in the command switching unit according to the embodiment; It is a figure which shows an example of the hardware constitutions of the vehicle control system by embodiment. It is a figure which shows an example of the hardware constitutions of the vehicle control system by embodiment.

<Embodiment>
<Configuration>
FIG. 1 is a block diagram showing the overall configuration of a vehicle control system according to an embodiment.

The vehicle control system includes a vehicle information acquisition sensor 1 , a vehicle exterior information acquisition sensor 2 , a main processing device 3 and a sub processing device 4 . The main processing device 3 and the sub-processing device 4 recognize peripheral objects in the same manner as the recognition device described above. Also, the main processing device 3 and the sub processing device 4 have a function of performing running control of the own vehicle. and run control as needed.

The vehicle information acquisition sensor 1 acquires vehicle information, which is information relating to the operation of the own vehicle, and includes a vehicle speed sensor 7 , a steering angle sensor 8 and an acceleration sensor 9 . The sensor information shown in FIG. 1 includes vehicle information.

A vehicle speed sensor 7 detects the running speed of the host vehicle. A steering angle sensor 8 detects the steering angle of the host vehicle. The acceleration sensor 9 detects acceleration of the own vehicle. The vehicle information acquisition sensor 1 is not limited to these, and may include other sensors as long as they can detect the operating state of the own vehicle. Other sensors include, for example, sensors that detect operating conditions such as brakes or gears.

The vehicle exterior information acquisition sensor 2 acquires vehicle exterior information indicating the state of the surrounding environment of the vehicle, and includes a camera 10, a millimeter wave radar 11, a Lidar (Light Detection And Ranging) 12, and a GPS (Global Positioning System) device 13. , and an in-vehicle communication device 14 . The sensor information shown in FIG. 1 includes information outside the vehicle.

A camera 10, a millimeter wave radar 11, and a lidar 12 detect the surrounding environment of the own vehicle. The GPS device 13 receives GPS signals from a plurality of GPS satellites and measures the position of the own vehicle. The in-vehicle communication device 14 performs communication between the vehicle and other vehicles (vehicle-to-vehicle communication) or communication between the vehicle and the roadside device (road-vehicle communication). The vehicle exterior information acquisition sensor 2 is not limited to these, and may include other sensors as long as they can acquire the state of the surrounding environment of the own vehicle. Other sensors include, for example, ultrasonic sonar. Also, the number of sensors mounted on the vehicle may be singular (for example, camera only) or plural (for example, camera and radar).

The main processing device 3 is connected to the vehicle information acquisition sensor 1, the vehicle exterior information acquisition sensor 2, the main processing device 3, the sub processing device 4, and the output switching device 5, respectively. The main processing device 3 has a main recognition processing section 15 and a main operation amount calculation section 18 .

The main recognition processing unit 15 performs processing for recognizing the positions, moving speeds, attribute information, and the like of surrounding objects existing around the own vehicle. Specifically, the main recognition processing unit 15 receives the vehicle exterior information acquired by the vehicle exterior information acquisition sensor 2, performs calculations for the object recognition models 16 and 17, and outputs recognition results. Note that the main recognition processing unit 15 only needs to have at least one object recognition model.

The main operation amount calculation unit 18 performs calculations for executing travel control of the host vehicle. Specifically, the main operation amount calculation unit 18 inputs the vehicle information acquired by the vehicle information acquisition sensor 1, the vehicle outside information acquired by the vehicle outside information acquisition sensor 2, and the recognition result by the main recognition processing unit 15, , and the operation amount of the actuator 6 is calculated.

In the present disclosure, a case where the main processing device 3 has the main recognition processing unit 15 and the main operation amount calculation unit 18 will be described. , 17 may be distributed to separate devices. A separate device may be provided outside the vehicle.

Also, the object recognition models 16 and 17 may be included in the vehicle exterior information acquisition sensor 2 . For example, the camera 10 or the lidar 12 has an object recognition model, and the information obtained by each is used as an input to execute processing with the object recognition model. Then, the recognition result by each object recognition model may be input to the main manipulated variable calculator 18 provided in another device.

Here, the object recognition model is input with information acquired by one or more sensors of the vehicle exterior information acquisition sensor 2, and outputs recognition results including the presence of surrounding objects and the relative positions of the surrounding objects and the own vehicle. It refers to a model that As the object recognition model, a rule-based model in which a feature extraction method of input information or a standard for detecting surrounding objects is manually designed may be used. In addition, a machine learning model that uses a neural network, which has been developed in recent years, to perform object recognition by learning feature extraction or object detection criteria based on a large amount of teacher data may be used. In general, machine learning models can be expected to recognize objects with higher accuracy than rule-based models.

The sub-processing device 4 is connected to the vehicle information acquisition sensor 1, the vehicle exterior information acquisition sensor 2, the main processing device 3, and the output switching device 5, respectively. The sub-processing device 4 has control switching units 19 and 20 , a sub-recognition processing unit 25 , and a sub-operation amount computing unit 26 .

The control switching section 19 has an abnormality determining section 21 , a scene determining section 22 , a failure determining section 23 and an instruction switching section 24 . The configuration of the control switching section 20 is the same as that of the control switching section 19 . A plurality of control switching units 20 may be provided according to the number of object recognition models that the main recognition processing unit 15 has.

The sub-recognition processing unit 25 has one or more object recognition models. When the main recognition processing unit 15 or the main operation amount computing unit 18 cannot execute processing, the sub-recognition processing unit 25 and the sub-operation amount computing unit 26 perform peripheral object recognition processing or actuator 6 processing as alternatives. Calculates the manipulated variable of

The main recognition processing section 15 and the sub recognition processing section 25 may perform calculations by different methods in consideration of the respective calculation performances of the main processing device and the sub processing device. The same applies to the main manipulated variable calculator 18 and the sub manipulated variable calculator 26 .

The output switching device 5 is connected to each of the main processing device 3, the sub-processing device 4, and the actuator 6. FIG. Actuator 6 includes a steering motor that adjusts the steering angle, a throttle motor that adjusts the intake air amount of the engine, and the like.

In the present disclosure, a case where the sub processing device 4 includes the control switching unit 19, the sub recognition processing unit 25, and the sub operation amount calculation unit 26 will be described. Each function of the calculation unit 26 may be distributed to separate devices. The same applies to the abnormality determination section 21, the scene determination section 22, the failure determination section 23, and the command switching section 24 of the control switching section 19. FIG. A separate device may be provided outside the vehicle.

Each device described above (including the vehicle information acquisition sensor 1 and the vehicle exterior information acquisition sensor 2) is connected using a line such as CAN (Control Area Network), Ethernet (registered trademark), or SPI (Serial Peripheral Interface). be done.

<Action>
FIG. 2 is a flow chart showing the operation of the vehicle control system.

In step S<b>101 , the main processing device 3 and the sub-processing device 4 acquire vehicle information from the vehicle information acquisition sensor 1 and acquire vehicle exterior information from the vehicle exterior information acquisition sensor 2 .

In step S102, the main recognition processing unit 15 recognizes peripheral objects existing around the host vehicle. Specifically, the main recognition processing unit 15 inputs the vehicle exterior information acquired from the vehicle exterior information acquisition sensor 2 to the object recognition models 16 and 17, and integrates the recognition results of the object recognition models 16 and 17 into one. The results are output as recognition results by the main recognition processing unit 15 . The recognition result includes the types of surrounding objects, the relative positions with respect to the host vehicle, and the like. The main recognition processing unit 15 outputs the recognition result to each of the main operation amount calculation unit 18 and the abnormality determination unit 21 .

In step S103, the abnormality determination unit 21 determines whether there is an abnormality in the recognition result of the main recognition processing unit 15 based on at least one of the vehicle information and the outside information and the recognition result of the main recognition processing unit 15 ( details will be described later). The abnormality determination section 21 outputs the determination result to the failure determination section 23 .

Simultaneously with step S103, in step S104, the scene determination unit 22 determines a scene in which an object can be recognized with high accuracy by the object recognition model (hereinafter referred to as a "recognizable scene") based on the vehicle information and the vehicle exterior information. ) (details will be described later). The scene determination unit 22 outputs determination results to the failure determination unit 23 .

In step S105, the failure determination unit 23 determines whether or not there is a failure in the main processing device 3 based on the determination result of the abnormality determination unit 21 and the determination result of the scene determination unit 22 (details will be described later).

In step S<b>106 , the command switching unit 24 determines whether the recognition result by the main recognition processing unit 15 is abnormal based on the determination result by the failure determination unit 23 . If there is an abnormality in the recognition result, the process proceeds to step S108. On the other hand, if there is no abnormality in the recognition result, the process proceeds to step S107.

In step S<b>107 , the instruction switching unit 24 determines whether or not the main processing device 3 has a failure based on the determination result of the failure determination unit 23 . If there is no failure in the main processing unit 3, the process proceeds to step S109. On the other hand, if there is a failure in the main processing device 3, the process proceeds to step S110.

In step S108, the main operation amount calculation section 18 calculates the operation amount of the actuator 6 using the recognition result by the main recognition processing section.

In step S109, the main operation amount calculation unit 18 calculates the operation amount of the actuator 6 without using the recognition result by the main recognition processing unit.

In step S110, the command switching unit 24 commands the main processing unit 3 to perform reset processing.

In step S111, the sub-recognition processing unit 25 recognizes surrounding objects.

In step S<b>112 , the sub operation amount calculator 26 calculates the operation amount of the actuator 6 .

In step S<b>113 , when the operation amount is input from the main processing device 3 , the output switching device 5 outputs the operation amount input from the main processing device 3 to the actuator 6 . Further, the output switching device 5 outputs the operation amount input from the sub-processing device 4 to the actuator 6 when the operation amount is not input from the main processing device 3 .

In the above, the main recognition processing unit 15 integrates the recognition results of the respective object recognition models 16 and 17 into one, and uses the main operation amount calculation unit 18 and the control switching units 19 and 20 as the recognition result by the main recognition processing unit 15. Although the case of outputting to is described, it is not limited to this. The main recognition processing section 15 may output the recognition results of the object recognition models 16 and 17 to the control switching sections 19 and 20, respectively. By doing so, the command switching unit 24 can issue commands according to the recognition results of the object recognition models 16 and 17 .

In the above, an overview of the operation of controlling the running of the own vehicle according to the presence or absence of an abnormality in the recognition result of the main recognition processing unit 15 and the presence or absence of a failure in the main processing device 3 has been described. Below, a method for determining whether or not there is an abnormality in the recognition result by the main recognition processing unit 15 (step S103 in FIG. 2), a method for determining a recognizable scene (step S104 in FIG. The details of the determination method (step S105 in FIG. 2), the command switching method, and the command output method (steps S106 to S112 in FIG. 2) will be described in order.

<Method for Determining Abnormality in Recognition Results by Main Recognition Processing Unit 15>
As described above, the recognition result of the peripheral object is obtained by the main recognition processing unit 15 calculating the object recognition models 16 and 17 that receive the information outside the vehicle as input. If the vehicle exterior information acquisition sensor 2 can accurately acquire vehicle exterior information and the main recognition processing unit 15 can calculate the object recognition models 16 and 17 using the vehicle exterior information as an input, surrounding objects can be accurately recognized. . However, if the main processing unit 3 fails, the main recognition processing unit 15 cannot compute the object recognition models 16 and 17 as in the normal state, and there is concern that an abnormal recognition result may be output. If the main manipulated variable calculation section 18 calculates the manipulated variable using an abnormal recognition result, there is a possibility that appropriate travel control cannot be executed.

In order to prevent the main manipulated variable calculation unit 18 from calculating the manipulated variable using an abnormal recognition result in order to execute appropriate travel control, whether an abnormal recognition result is output from the object recognition models 16 and 17 is determined. It is necessary to determine whether

Here, the following (1) to (4) are given as examples of cases where the recognition result is considered to be abnormal.
(1) A case where the host vehicle is slowing down and the surrounding objects that were recognized in the past are suddenly not recognized even though the relative positions of the peripheral objects with respect to the host vehicle have not changed significantly.
(2) Although the vehicle is traveling in a similar area and the number of peripheral objects to be recognized is not expected to change suddenly, the number of peripheral objects to be recognized suddenly increases compared to the past. case.
(3) When the position of the peripheral object to be recognized does not change from the past point in time even though the own vehicle is running.
(4) A case where the number of peripheral objects to be recognized continuously increases or decreases even though the host vehicle and the peripheral vehicles are stopped and the number of peripheral objects to be recognized is not expected to increase or decrease.

In other words, regarding the behavior or position of the own vehicle, there is a recognition result that suddenly shows a large change compared to the past recognition result, that there is no change from the past recognition result, or that the recognition result continues to change from the past. When it appears, it can be determined that the recognition result is abnormal.

Therefore, based on at least one of the vehicle information and the outside information and the recognition result by the main recognition processing unit 15, the past recognition result by the main recognition processing unit 15 and the new recognition result are compared, and the comparison is performed. The presence or absence of abnormality in the new recognition result is determined depending on whether or not the result matches the conditions set according to the behavior or position of the host vehicle.

FIG. 3 is a flowchart showing the operation of the abnormality determination section 21. As shown in FIG. In FIG. 3, it is determined whether or not the amount of change between the number of peripheral objects newly recognized by the main recognition processing unit 15 and the number of peripheral objects previously recognized by the main recognition processing unit 15 falls within a preset range. A method of determining whether or not there is an abnormality in a new recognition result by the main recognition processing unit 15 will be described.

In step S<b>201 , the abnormality determination unit 21 acquires vehicle information from the vehicle information acquisition sensor 1 , acquires vehicle exterior information from the vehicle exterior information acquisition sensor 2 , and acquires recognition results from the main recognition processing unit 15 .

In step S202, the abnormality determination unit 21 sets a range of variation (hereinafter referred to as "normal range") for determining that the new recognition result by the main recognition processing unit 15 is normal. The maximum and minimum values indicating the normal range are information regarding the behavior of the own vehicle such as vehicle speed and acceleration obtained from the vehicle information acquisition sensor 1, information regarding the position where the own vehicle travels obtained from the outside information acquisition sensor 2, and the above. It is set by a map that inputs an object recognition model that outputs a recognition result whose presence or absence is to be confirmed.

In step S203, the abnormality determination unit 21 calculates the average value of the number of surrounding objects in the past several recognition results as a comparison target when determining whether or not there is an abnormality in the new recognition result.

In step S<b>204 , the abnormality determination unit 21 calculates the amount of change between the number of surrounding objects in the new recognition result and the average value of the number of surrounding objects in the past several recognition results.

In step S205, the abnormality determination unit 21 determines whether or not the amount of change calculated in step S204 is within the normal range set in step S202. If the amount of change is within the normal range, the process proceeds to step S206. On the other hand, if the amount of change is not within the normal range, the process proceeds to step S207.

In step S206, the abnormality determination unit 21 determines that the new recognition result by the main recognition processing unit 15 is normal.

In step S207, the abnormality determination unit 21 determines that the new recognition result by the main recognition processing unit 15 is abnormal.

In the above, the average value of the number of peripheral objects in the past few recognition results was used as a comparison target when judging whether there is an abnormality in a new recognition result. Other statistics such as values or weighted moving averages may be used.

A range of possible values of the number of peripheral objects in the new recognition result, which is set according to the number of peripheral objects recognized in the past, may be used for comparison between the new recognition result and the past recognition result. For example, consider a case where the number of peripheral objects recognized in the past was two, but the object recognition model cannot recognize the peripheral objects, and the number of peripheral objects in the new recognition result becomes zero. In this case, if the possible range of the number of surrounding objects in the new recognition result is set to 1 to 5, the number of surrounding objects (0) in the new recognition result is the range of the possible number of surrounding objects in the new recognition result. (1 to 5), the new recognition result is determined to be abnormal.

Position information of surrounding objects may be used instead of the amount of change for comparison between the new recognition result and the past recognition result. In this case, the condition for comparison of recognition results is set to the case where there is a change in the position information of the surrounding object. By using the position information of the surrounding object, it is possible to determine that there is an abnormality even when the recognition result by the main recognition processing unit 15 does not change.

Information other than the number of surrounding objects and positional information of the surrounding objects may be used for comparison between the new recognition result and the past recognition result.

In the above description, the case of judging whether or not the result of comparison between the new recognition result and the statistical amount of the past recognition result matches the set condition has been described. It may be determined whether or not the fluctuation matches the set condition. For example, the set value is the sum of the amount of change in the number of recognition results (number of surrounding objects, etc.) between a certain recognition result and the recognition result obtained one time before that recognition result, over several past recognition results. It may be determined whether or not it falls within the range of In this case, even if the number of surrounding objects continues to fluctuate within the set range of the amount of change described above, the recognition result when the total value of the amount of change exceeds the set value is judged as an abnormal recognition result. can do.

In the above description, the case where it is determined that the recognition result is abnormal when the result of comparison between the new recognition result and the statistic of the past recognition result does not match the set condition even once was explained. It may be determined that the recognition result is abnormal when the condition is not met a plurality of times. For example, it is actually possible that the number of surrounding objects suddenly decreases due to many moving objects moving to the back side of the shield at the same time. In this case, there is a possibility that an abnormal recognition result may be erroneously determined by comparing the recognition results only once. Therefore, it is also possible to count the number of times that the comparison result does not match the set condition, and determine that the recognition result is abnormal when the count reaches a certain value within a certain period of time.

Note that when different control switching units 19 and 20 (different abnormality determination units) are used for each of the object recognition models 16 and 17 of the main recognition processing unit 15, the results of recognition by the object recognition models 16 and 17 are input, and the above-described The presence or absence of abnormality in each recognition result is determined by the method of .

By the method described above, the abnormality determination unit 21 can determine whether or not there is an abnormality in the recognition result by the main recognition processing unit 15 .

<Method for Determining Recognizable Scenes>
As described above, the main recognition processing unit 15 receives information acquired from the vehicle exterior information acquisition sensor 2 and recognizes surrounding objects existing around the own vehicle. Therefore, if information about the surroundings of the own vehicle can be accurately obtained from various sensors, the main recognition processing section 15 can accurately recognize surrounding objects.

However, depending on the scene, the main recognition processing unit 15 may not be able to accurately recognize surrounding objects because information around the vehicle cannot be accurately acquired from the vehicle exterior information acquisition sensor 2 . For example, when light emitted from a surrounding object or natural light suddenly enters the camera 10, the camera 10 is temporarily unable to accurately acquire information about the surroundings of the vehicle. may not be able to recognize surrounding objects with high accuracy.

In addition, parameter adjustment of rule-based models used as object recognition models and machine learning model learning can be achieved by using a large amount of teacher data in scenes that cover all regions, weather, and time of day where vehicles actually travel. , it becomes possible to recognize surrounding objects with high accuracy. On the other hand, in situations where it is difficult to collect training data, such as around an accident or disaster site, or in areas where the appearance of training data is significantly different from when training data was collected due to new urban development, etc., the main recognition is performed. There is a possibility that the processing unit 15 cannot recognize surrounding objects with high accuracy.

As described above, even if there is no failure in the main processing device 3, depending on the scene, the main recognition processing section 15 is likely to output an abnormal recognition result. If it is determined that the main processing device 3 has failed based only on the fact that the main recognition processing unit 15 outputs an abnormal recognition result, the reset processing that is originally unnecessary is performed. At this time, if it can be determined that the scene in which the host vehicle is present is not a recognizable scene, it is possible to consider that the abnormal recognition result output from the main recognition processing section 15 is not caused by a failure of the main processing device 3. can be done.

In order to determine that the scene is recognizable, it is necessary to determine whether the scene is one in which information around the vehicle can be accurately obtained from the vehicle exterior information acquisition sensor 2, or whether the scene is within the training data used for learning the machine learning model. It suffices if it is possible to determine whether or not the scene exists in . Such scene determination is based on the conditions assumed to be satisfied if the scene is recognizable by taking into account various information such as weather, ambient brightness, or electrical noise from the sensor, and the correlation of these information. It can be realized by determining whether or not there is a match.

Therefore, the object recognition models 16 and 17 of the main recognition processing unit 15 are configured so that it is possible to determine whether or not there is a failure in the main processing unit 3 in an environment in which surrounding objects can be accurately recognized. When a value derived using a plurality of feature amounts obtained by inputting at least one of the vehicle exterior information and the vehicle information to the is within the threshold value, it is determined that the scene is recognizable.

A method for determining a recognizable scene will be described with reference to FIG. In FIG. 4, an input value In1, a predetermined reference value P1, and a relationship between the input value In1 and the reference value P1 are shown in a two-dimensional space with the feature amount A and the feature amount B used for determining a recognizable scene as axes. It shows a Euclidean distance Dc (details will be described later), a circle whose center is a reference value P1 and whose radius is a predetermined threshold value Th1.

The feature amount A and the feature amount B are statistics of information outside the vehicle, such as an average luminance value, a maximum luminance value, a variance, or a luminance difference between adjacent pixels in the image acquired by the camera 10 . In addition to the statistical quantity, the feature quantity A and the feature quantity B may be obtained by repeating filtering or pooling processing. In addition, since there is a concern that the vibration of the vehicle, etc., may affect the sensing of the outside information acquisition sensor 2, vehicle information including the behavior of the vehicle may be used as the feature amount A and the feature amount B. FIG. Furthermore, positional information acquired by the GPS device 13 or non-numerical information such as weather may be quantified by comparing with a preset map, and the numerical values may be used as the feature amount A and the feature amount B. FIG.

In FIG. 4, the area inside the circle is the recognizable scene. The reference value P1 and the threshold value Th1 are set based on the performance of the vehicle exterior information acquisition sensor 2 . Specifically, a reference value P1 and a threshold value Th1 indicate a range of values of the feature amount A and the feature amount B that satisfy a scene where it is assumed that the outside information acquisition sensor 2 can acquire the outside information accurately. A reference value P1 and a threshold Th1 are set so as to include an area where the

Also, in the evaluation of the recognition performance of the object recognition models 16 and 17, the scenes in which the object recognition models 16 and 17 were able to accurately recognize the surrounding objects were collected, and the feature amount A and the feature amount B were obtained in those scenes. The reference value P1 and the threshold value Th1 may be set from the range of possible values.

Let the coordinates of the input value In1 be (A1, B1) and the coordinates of the reference value P1 be (Ap, Bp) in a two-dimensional space with the feature amount A and the feature amount B as axes. A Euclidean distance D between the input value In1 and the reference value P1 is represented by the following formula (1).

By the way, the feature amount A and the feature amount B may have different degrees of contribution to the recognition result. For example, when the feature amount A has a greater influence on the vehicle exterior information than the feature amount B, it is desired that the Euclidean distance D is greatly influenced by the input value A1. In other words, the Euclidean distance D is desired to be more sensitive to changes in the feature amount A.

Therefore, (Ap-A1) and (Bp-B1), which are components parallel to each coordinate in Equation (1), are adjusted using parameters c1 and c2. The Euclidean distance D adjusted in this way is hereinafter referred to as "Euclidean distance Dc". The Euclidean distance Dc is represented by the following formula (2).

When the Euclidean distance Dc is less than or equal to the threshold value Th1, the scene determination unit 22 determines that the host vehicle exists in a recognizable scene.

Note that the reference value P1 and the threshold value Th1 may be changed according to the feature amount A, the feature amount B, or other feature amounts.

In the above description, the case of calculating the Euclidean distance Dc in a two-dimensional space with two parameters of the feature amount A and the feature amount B as axes is described. The Euclidean distance Dc may be calculated in space in the same manner as described above.

In the above description, the case where the Euclidean distance Dc is used as the distance between the input value In1 and the reference value P1 has been described. Other methods of determining may be used.

For simplification, determination of recognizable scenes according to combinations of feature amounts may be set in advance in a map. Another method may be used as long as it is a method of calculating a feature amount from at least one of the vehicle information and the outside information and determining whether or not the feature amount satisfies a predetermined condition.

When different control switching units 19 and 20 (different scene determination units) are used for each of the object recognition models 16 and 17 of the main recognition processing unit 15, different vehicle information and outside information are input to each scene determination unit. good too. Since the object recognition models 16 and 17 differ in the vehicle exterior information acquisition sensor 2 that acquires information used as input, the object recognition models 16 and 17 also differ in recognizable scenes. Therefore, by using different scene determination units for each of the object recognition models 16 and 17, it is possible to determine a recognizable scene suitable for each of the object recognition models 16 and 17. FIG.

By the method described above, the scene determination unit 22 can determine whether or not the scene is a recognizable scene.

<Method for Determining Presence or Absence of Failure of Main Processing Unit 3>
As described above, the reason why the abnormal recognition result is output is that the object recognition models 16 and 17 are not in a scene where the surrounding objects can be accurately recognized, and that the main processing unit 3 is out of order. If the cause of the output of an abnormal recognition result is that the object recognition model is not in a scene where the surrounding objects can be accurately recognized, it is possible to quickly output a recognition result that is normal simply by leaving the scene. can. On the other hand, if an abnormal recognition result is output due to a failure of the main processing unit 3, all operations performed by the main processing unit 3 are stopped in order to eliminate the failure that occurred in the main processing unit 3. Unless a process (reset process) is performed to restart the main processing unit 3 so that the operation can be performed again (reset process), no error-free recognition results are output.

In addition, there are cases where the main recognition processing unit 15 outputs an abnormal recognition result because the main processing unit 3 cannot be cleared even if the reset processing is performed, such as when the main processing unit 3 is damaged. In this case, since the object recognition models 16 and 17 cannot be calculated normally by the main recognition processing unit 15, it is necessary to perform processing for recognizing peripheral objects by another device.

As described above, the necessary processing differs depending on the cause of the output of the abnormal recognition result from the main recognition processing unit 15 and the degree of failure of the main processing device 3 . In order to appropriately select the necessary processing, it is necessary to clarify the cause of abnormal recognition result output from the main recognition processing unit 15 and the degree of failure of the main processing device 3, and determine the state of the main processing device 3. I wish I could.

Therefore, the failure determination unit 23 receives the determination result of the abnormality determination unit 21 and the determination result of the scene determination unit 22 as input, and determines whether or not there is a failure in the main processing unit 3 and the extent of the failure.

A method for determining the presence or absence of a failure in the main processing unit 3 and the extent of the failure will be described with reference to FIG. FIG. 5 is a diagram showing the content of determination by the failure determination unit 23. As shown in FIG.

First, the failure determination unit 23 confirms the determination result of the abnormality determination unit 21 (whether or not there is an abnormality in the recognition result of the main recognition processing unit 15).

When the determination result of the abnormality determination unit 21 is "no abnormality", the failure determination unit 23 determines "no abnormality in the recognition result and no failure in the main processing unit". Here, although the abnormality determination unit 21 determines that there is no abnormality in the recognition result, the scene determination unit 22 may determine that the scene is not a recognizable scene. In this case, since the abnormality determination unit 21 determines that there is continuity and no abnormality compared with the past recognition results, the failure determination unit 23 determines that there is no abnormality in the recognition results.

When the determination result of the abnormality determination unit 21 is "abnormal", the failure determination unit 23 confirms whether or not the determination result of the scene determination unit 22 is a recognizable scene.

If the determination result of the scene determination unit 22 is "not a recognizable scene", the reason why the recognition result is abnormal is that the main recognition processing unit 15 cannot accurately recognize the surrounding objects because the scene is not recognizable. Assuming that there is, the failure determination unit 23 determines that "there is an abnormality in the recognition result and there is no failure in the main processing unit".

If the judgment result of the scene judging section 22 is "a recognizable scene," the cause of the abnormality in the recognition result is that the main recognition processing section 15 has failed and the surrounding objects are normally recognized even though the scene is recognizable. Assuming that this is because it is in a state where it cannot be performed, the failure determination unit 23 determines that "there is an abnormality in the recognition result and there is a failure in the main processing unit".

After the failure determination unit 23 determines that "there is an abnormality in the recognition result and that the main processing unit , there is a failure in the main processing unit” is repeated more than a preset number of times, it is assumed that a failure that cannot be removed by the reset processing has occurred in the main processing unit 3, and the failure determination unit 23 determines that “ There is a major failure in the main processing unit."

In the above description, the case where the failure determination unit 23 makes determinations other than “a serious failure of the main processing unit” is described, with the determination results of the abnormality determination unit 21 and the scene determination unit 22 being input once. In order to deal with sudden changes in results and continuation of the same determination result, the determination results of the abnormality determination unit 21 and the scene determination unit 22 for the past several times may also be input for determination.

For example, the failure determination unit 23 determines that the determination result of the abnormality determination unit 21 is "abnormal" and the determination result of the scene determination unit 22 is "not a recognizable scene" for a predetermined number of times or less in a certain period of time. , it may be determined that "there is an abnormality in the recognition result and there is no failure in the main processing unit", and when the set number of times is reached, it is determined that "there is an abnormality in the recognition result and there is a failure in the main processing unit". . By making such determination, when the main processing unit 3 has no failure and the scene is recognizable, the main recognition processing unit 15 momentarily changes the recognition result, and the abnormality determination unit 21 determines that there is an abnormality. Even if it is determined, the failure determination unit 23 can accurately determine that "there is no failure in the main processing unit".

Further, the failure determination unit 23 determines that the determination result of the abnormality determination unit 21 is "no abnormality" and the determination result of the scene determination unit 22 is "not a recognizable scene" for a predetermined number of times or less in a certain period of time. , it may be determined that "there is no abnormality in the recognition result and no failure in the main processor", and when the set number of times is reached, it may be determined that "there is an abnormality in the recognition result and there is no failure in the main processor". . If the state in which the scene is not recognizable continues, the abnormality determination unit 21 determines whether or not there is an abnormality in the recognition result based only on the recognition result outside the recognizable scene. The recognition result outside the recognizable scene is considered to be less accurate than the recognition result in the recognizable scene, and the reliability of the determination result of the abnormality determination unit 21 is also low. Therefore, by making such a determination, the failure determination unit 23 determines to perform the same processing as when there is an abnormality in the recognition result when the reliability of the determination result of the abnormality determination unit 21 is low. Travel control can be performed appropriately.

Note that when different control switching units 19 and 20 (different failure determination units) are used for each of the object recognition models 16 and 17 of the main recognition processing unit 15, the determination results of each abnormality determination unit 21 and each scene determination unit 22 are As an input, the presence or absence of a failure in the main processing unit 3 and the degree of failure are determined.

By the method described above, the failure determination unit 23 can determine the presence or absence of a failure in the main processing unit 3 and the extent of the failure.

<Instruction switching method and instruction output method>
As described above, in order to appropriately perform travel control, the recognition result and processing used when calculating the operation amount of the actuator 6 are determined according to the abnormality of the recognition result, the presence or absence of a failure in the main processing device 3, and the degree of failure. different. By switching commands to the respective processing units and functional blocks according to the abnormality of the recognition result and the presence or absence of a failure in the main processing unit 3, optimum running control can be realized. The instruction switching unit 24 receives the judgment result of the failure judging unit 23 and switches the instruction to the main processing unit 3 or the sub processing unit 4 .

Instruction switching method and details of instructions will be described with reference to FIG. FIG. 6 is a diagram showing the contents of commands in the command switching unit.

When the failure determination unit 23 determines that there is no abnormality in the recognition result and no failure in the main processing unit, the instruction switching unit 24 instructs the main processing unit 3 to use the recognition result of the main recognition processing unit 15. It instructs the main manipulated variable calculation section 18 to perform the calculation.

When the failure determination unit 23 determines that "there is an abnormality in the recognition result and there is no failure in the main processing unit", the command switching unit 24 notifies the main processing unit 3 of object recognition for which it is determined that there is an abnormality in the recognition result. It instructs the main manipulated variable calculator 18 to perform the calculation without using the recognition result by the model. The main operation amount calculation unit 18 calculates the operation amount of the actuator 6 by inputting the past recognition result by the main recognition processing unit 15 or the vehicle information acquired by the vehicle information acquisition sensor 1 .

When the failure determination unit 23 determines that "there is an abnormality in the recognition result and that the main processing unit has a failure", the instruction switching unit 24 instructs the main processing unit 3 to stop all operations in the main processing unit 3. , to execute a process (reset process) to turn the power back on (reboot) to enable the operation again. Further, the command switching unit 24 instructs the sub-recognition processing unit 25 and the sub-operation amount calculation unit 26 to perform object recognition processing so that the vehicle can be controlled while the main processing unit 3 is performing reset processing. It commands execution of processing and calculation of the manipulated variable of the actuator 6, respectively.

When the failure determination unit 23 determines that "there is a serious failure in the main processing unit 3", the instruction switching unit 24 stops all operations in the main processing unit 3 and turns off the power. command to The command switching unit 24 also commands the sub-recognition processing unit 25 and the sub-operation amount calculation unit 26 to perform object recognition processing and calculation of the operation amount of the actuator 6, respectively.

In the above description, the case where the abnormality determination unit 21 determines whether or not there is an abnormality in the recognition result by the main recognition processing unit 15 has been described. While the calculation by the main recognition processing unit 15 is stopped, the abnormality determination unit 21 determines whether there is an abnormality in the recognition result by the sub recognition processing unit 25, and if the determination result is an abnormal recognition result, the command switching unit 24 may command the sub-manipulated amount calculator 26 not to use the recognition result based on the sub-recognition result, and may command the main manipulated variable calculator 18 to calculate the manipulated variable. Even when the sub-recognition processing unit 25 performs recognition processing according to the instruction, the recognition result used by the sub-manipulated amount calculation unit 26 for calculation is appropriately determined according to the presence or absence of abnormality in the recognition result by the sub-recognition processing unit 25. can be selected.

In the above description, the sub-processing device 4 performs processing related to travel control while the main processing device 3 is in the process of resetting. can be By doing so, it is possible to reduce the calculation load on the sub-processing device 4 and simplify the configuration of the sub-processing device 4 .

When different control switching units 19 and 20 (different command switching units) are used for each of the object recognition models 16 and 17 of the main recognition processing unit 15, the determination result of each failure determination unit 23 is input to the main processing unit. 3, or instruct the sub-recognition processing unit 25 and the sub-operation amount calculation unit 26. In this case, unlike the case where the recognition results by the respective object recognition models 16 and 17 are integrated into one and used as the recognition result by the main recognition processing unit 15, the failure judgment unit 23 determines that "there is an abnormality in the recognition result, the main processing unit When it is determined that there is no failure, the instruction switching unit 24 performs the calculation of the main operation amount calculation unit 18 without using only the recognition result by the object recognition model that is determined to be abnormal in the main processing device 3. order to do so. As a result, the main operation amount calculation unit 18 can calculate the operation amount of the actuator 6 by inputting not only past recognition results but also recognition results from other object recognition models that have been determined to be normal.

On the other hand, if there is one or more failure determination units 23 that determine that "there is an abnormality in the recognition result and there is a failure in the main processing unit", the instruction switching unit 24 instructs the main processing unit 3 to perform reset processing. , and commands the sub-recognition processing unit 25 and the sub-operation amount calculation unit 26 to perform object recognition processing and calculation of the operation amount of the actuator 6, respectively. As a result, reset processing can be performed before a fault that occurs in the calculation of one object recognition model affects other calculations in the main processing unit 3 .

<effect>
According to the present disclosure, commands relating to running control of the host vehicle to the main processing device 3 or the sub processing device 4 are issued according to the presence or absence of an abnormality in the recognition result by the main recognition processing unit 15 and the presence or absence of a failure in the main processing device 3. switch. As a result, it is possible to carry out optimal running control of the own vehicle with respect to the behavior of surrounding objects over a longer period of time.

<Hardware configuration>
Each function of the main recognition processing unit 15 and the main operation amount calculation unit 18 described above is realized by a processing circuit. That is, the main processing unit 3 has a processing circuit for recognizing surrounding objects and calculating the operation amount of the actuator. The processing circuit may be dedicated hardware, and a processor (CPU, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, DSP (Digital Signal Processor)) that executes a program stored in memory may be called).

If the processing circuit is dedicated hardware, as shown in FIG. 7, the processing circuit 27 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, or an ASIC (Application Specific Integrated Circuit). , FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the main recognition processing unit 15 and the main operation amount calculation unit 18 may be implemented by the processing circuit 27 , or the functions may be collectively implemented by one processing circuit 27 .

When the processing circuit 27 is the processor 28 shown in FIG. 8, each function of the main recognition processing section 15 and the main manipulated variable calculation section 18 is realized by software, firmware, or a combination of software and firmware. Software or firmware is written as a program and stored in memory 29 . The processor 28 implements each function by reading and executing the program recorded in the memory 29 . That is, the main processing device 3 has a memory 29 for storing a program that results in the execution of the step of recognizing a peripheral object and the step of calculating the manipulated variable of the actuator. It can also be said that these programs cause a computer to execute the procedures or methods of the main recognition processing section 15 and the main manipulated variable calculation section 18 . Here, memory means non-volatile or volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory). It may be a flexible semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a DVD (Digital Versatile Disc), or any storage medium that will be used in the future.

As for the functions of the main recognition processing unit 15 and the main manipulated variable calculation unit 18, some functions may be implemented by dedicated hardware, and other functions may be implemented by software or firmware.

Thus, the processing circuitry may implement each of the functions described above through hardware, software, firmware, or a combination thereof.

Although the hardware configuration of the main processing device 3 has been described above, the hardware configuration of the sub processing device 4 is the same.

It should be noted that the embodiments can be appropriately modified or omitted within the scope of the present disclosure.

1 vehicle information acquisition sensor 2 vehicle exterior information acquisition sensor 3 main processing device 4 sub processing device 5 output switching device 6 actuator 7 vehicle speed sensor 8 steering angle sensor 9 acceleration sensor 10 camera 11 millimeter wave radar , 12 Lidar, 13 GPS device, 14 in-vehicle communication device, 15 main recognition processing unit, 16, 17 object recognition model, 18 main operation amount calculation unit, 19, 20 control switching unit, 21 abnormality determination unit, 22 scene determination unit, 23 failure determination unit, 24 instruction switching unit, 25 sub recognition processing unit, 26 sub operation amount calculation unit, 27 processing circuit, 28 processor, 29 memory.

Claims (16)

a main processing device that controls running of the vehicle;
a sub-processing device that monitors the state of the main processing device;
with
The main processing device is
a main recognition processing unit that recognizes peripheral objects existing around the vehicle based on outside information indicating the state of the surrounding environment of the vehicle;
a main operation amount calculation unit that calculates an operation amount for performing the travel control based on the recognition result by the main recognition processing unit, the vehicle exterior information, and the vehicle information regarding the operation of the vehicle;
has
The sub-processing device is
a control switching unit that switches commands relating to the running control to the main processing device or the sub processing device according to the presence or absence of an abnormality in the recognition result of the main recognition processing section and the presence or absence of a failure in the main processing device;
a sub-recognition processing unit that recognizes the peripheral object based on the information outside the vehicle according to the command from the control switching unit;
a sub-operation amount calculation unit that calculates the operation amount for performing the travel control based on the vehicle exterior information and the vehicle information according to the command of the control switching unit;
has
The control switching unit is
an abnormality determination unit that determines whether there is an abnormality in the recognition result of the main recognition processing unit based on at least one of the outside information and the vehicle information and the recognition result of the main recognition processing unit;
Based on at least one of the vehicle exterior information and the vehicle information, it is determined whether or not the surrounding environment of the vehicle is a recognizable scene indicating that the recognition result by the main recognition processing unit is a reliable environment. a scene determination unit to
a failure determination unit that determines whether or not there is a failure in the main processing unit based on the determination result of the abnormality determination unit and the determination result of the scene determination unit;
a command switching unit that commands the main processing device or the sub processing device to perform the running control based on the determination result of the failure determination unit;
A vehicle control system comprising: a vehicle exterior information acquisition sensor that acquires the vehicle exterior information;
a vehicle information acquisition sensor that acquires the vehicle information;
The vehicle control system of claim 1 , further comprising: 3. The vehicle control system according to claim 1, wherein said travel control includes AD (Autonomous Driving) control and ADAS (Advanced Driver Assistance System) control. 4. The vehicle control according to any one of claims 1 to 3 , wherein the main recognition processing unit recognizes the surrounding object using a plurality of rule-based models or a plurality of machine learning models that receive the information outside the vehicle as input. system. 2. The abnormality determination unit determines that the recognition result by the main recognition processing unit is abnormal when the recognition result by the main recognition processing unit in a predetermined period does not satisfy a predetermined condition. The vehicle control system described in . wherein the scene determination unit determines that the scene is not the recognizable scene when a value obtained by calculating based on at least one of the vehicle exterior information and the vehicle information does not satisfy a predetermined condition. Item 1. The vehicle control system according to item 1. The scene determination unit determines the feature amount based on at least one feature amount derived based on at least one of the vehicle exterior information and the vehicle information and a predetermined reference value. 7. The vehicle control system according to claim 6 , wherein when the distance between said feature amount and said reference value in a multidimensional space of .times..times..times..times..times..times..times..times..times..times..times..times. When the abnormality determination unit determines that the recognition result by the main recognition processing unit is abnormal, and the scene determination unit determines one or more times that the scene is recognizable, the failure determination unit performs the main processing. 2. The vehicle control system according to claim 1 , wherein the device is determined to be faulty. When the failure determination unit determines that there is a failure in the main processing unit, the instruction switching unit suspends all operations on the main processing unit and then restarts the main processing unit so that the operation can be performed again. 2. The vehicle control system according to claim 1 , which commands to perform a reset process, which is a process for resetting. 10. The command switching unit according to claim 9 , wherein the command switching unit commands the sub-recognition processing unit and the sub-operation amount calculation unit to perform the running control until the main processing unit completes the reset processing. A vehicle control system as described. The failure determination unit determines that the recognition result by the main recognition processing unit is abnormal after the main processing unit has performed the reset process a predetermined number of times or more, and the scene determination is performed. if it is determined that the part is the recognizable scene, it is determined that a failure that cannot be dealt with by the reset process has occurred in the main processing device;
10. The command switching unit commands the main processing unit to stop power supply, and commands the sub -recognition processing unit and the sub-operation amount calculation unit to perform the running control. 11. The vehicle control system according to 10 . The command switching unit calculates the main manipulated variable when the abnormality determination unit determines that the recognition result by the main recognition processing unit is abnormal and the failure determination unit determines that there is no failure in the main processing device. 2. The vehicle control system according to claim 1 , wherein the vehicle control system instructs a section to perform the travel control without using the recognition result by the main recognition processing section. The command switching unit determines that the abnormality determination unit determines that the recognition result by the main recognition processing unit is not abnormal, and the number of times the scene determination unit determines that the scene is not recognizable is equal to or greater than a predetermined number of times. 2. The vehicle control system according to claim 1 , wherein, in the case, the main operation amount calculation unit is instructed to perform the travel control without using the recognition result by the main recognition processing unit. a main processing device that controls running of the vehicle;
a sub-processing device that monitors the state of the main processing device;
with
The main processing device is
a main recognition processing unit that recognizes peripheral objects existing around the vehicle based on outside information indicating the state of the surrounding environment of the vehicle;
a main operation amount calculation unit that calculates an operation amount for performing the travel control based on the recognition result by the main recognition processing unit, the vehicle exterior information, and the vehicle information regarding the operation of the vehicle;
has
The sub-processing device is
a control switching unit that switches commands relating to the running control to the main processing device or the sub processing device according to the presence or absence of an abnormality in the recognition result of the main recognition processing section and the presence or absence of a failure in the main processing device;
a sub-recognition processing unit that recognizes the peripheral object based on the information outside the vehicle according to the command from the control switching unit;
a sub-operation amount calculation unit that calculates the operation amount for performing the travel control based on the vehicle exterior information and the vehicle information according to the command of the control switching unit;
has
The sub-processing device has a plurality of control switching units corresponding to each of the recognition results when there are a plurality of recognition results by the main recognition processing unit,
A vehicle control system in which each of the control switching units switches a command relating to the running control to the main processing device or the sub processing device according to the presence or absence of an abnormality in the recognition result and the presence or absence of a failure in the main processing device. a main processing device that controls running of the vehicle;
a sub-processing device that monitors the state of the main processing device;
A vehicle control method in a vehicle control system comprising
In the main processing device,
a main recognition processing unit recognizing peripheral objects present around the vehicle based on vehicle exterior information indicating the state of the surrounding environment of the vehicle;
A main operation amount calculation unit calculates an operation amount for performing the travel control based on the recognition result by the main recognition processing unit, the vehicle exterior information, and the vehicle information regarding the operation of the vehicle,
In the sub-processing device,
a control switching unit that switches commands relating to the running control to the main processing unit or the sub processing unit according to whether there is an abnormality in the recognition result of the main recognition processing unit and whether the main processing unit is malfunctioning;
a sub-recognition processing unit recognizing the peripheral object based on the information outside the vehicle in accordance with the command from the control switching unit;
a sub-operation amount calculation unit, in accordance with the command from the control switching unit, calculates the operation amount for performing the travel control based on the vehicle-external information and the vehicle information;
The switching of the command related to the running control by the control switching unit includes:
an abnormality determination unit determining whether or not there is an abnormality in the recognition result of the main recognition processing unit based on at least one of the outside information and the vehicle information and the recognition result of the main recognition processing unit;
The scene determination unit determines, based on at least one of the vehicle exterior information and the vehicle information, that the surrounding environment of the vehicle is a recognizable scene indicating that the recognition result by the main recognition processing unit is an environment that can be trusted. determine whether or not
a failure determination unit determining whether or not there is a failure in the main processing device based on the determination result of the abnormality determination unit and the determination result of the scene determination unit;
A vehicle control method , comprising: an instruction switching unit instructing the main processing unit or the sub processing unit to perform the running control based on a determination result by the failure determination unit . a main processing device that controls running of the vehicle;
a sub-processing device that monitors the state of the main processing device;
A vehicle control method in a vehicle control system comprising
In the main processing device,
a main recognition processing unit recognizing peripheral objects present around the vehicle based on vehicle exterior information indicating the state of the surrounding environment of the vehicle;
A main operation amount calculation unit calculates an operation amount for performing the travel control based on the recognition result by the main recognition processing unit, the vehicle exterior information, and the vehicle information regarding the operation of the vehicle,
In the sub-processing device,
a control switching unit that switches commands relating to the running control to the main processing unit or the sub processing unit according to whether there is an abnormality in the recognition result of the main recognition processing unit and whether the main processing unit is malfunctioning;
a sub-recognition processing unit recognizing the peripheral object based on the information outside the vehicle in accordance with the command from the control switching unit;
a sub-operation amount calculation unit, in accordance with the command from the control switching unit, calculates the operation amount for performing the travel control based on the vehicle-external information and the vehicle information;
The sub-processing device has a plurality of control switching units corresponding to each of the recognition results when there are a plurality of recognition results by the main recognition processing unit,
A vehicle control method, wherein each of the control switching units switches the command relating to the running control to the main processing device or the sub processing device according to the presence or absence of abnormality in the recognition result and the presence or absence of failure of the main processing device.

Images