JP2017182771A - Device, method and program for detecting object - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device, a method and a program for detecting an object, each enabling fail-safe object detection to be performed while minimizing an additional constituent even when one of a plurality of object detectors gets into an abnormal condition.SOLUTION: An object detection device 101 comprises: an information acquisition section 102 for acquiring object information on an object; an object detection section 103 including a plurality of object detectors 113, 123, 133 each using the object information to perform predetermined object detection processing; a detector abnormality sensing section 104 for sensing the abnormal condition of each of the object detectors 113, 123, 133; and a detector switching section 105 for changing the object detection processing of the object detector in which the abnormal condition is not detected when the detector abnormality sensing section 104 senses the abnormal condition of one of the object detectors 113, 123, 133.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an object detection apparatus, an object detection method, and an object detection program that detect an object, and relates to an object detection apparatus, an object detection method, and an object detection program that detect an object from information such as an image in a fail-safe manner. .

  In recent years, many researches and commercializations of automatic driving vehicles (robot cars) driven by machines instead of drivers have been conducted. An object detection device (general object detection device) that detects a general object is used as one of elemental functions in this autonomous driving vehicle. The general object detection device is a camera device or a distance measurement sensor device such as a radar or a stereo camera. A general object is detected from information such as an image transferred from the network. The general objects include pedestrians, vehicles, road signs, buildings, road areas, etc., the general object detection device for autonomous vehicles detects these objects, and the travel control device for autonomous vehicles Based on the detection information from the object detection device, it is an essential requirement to control the autonomous vehicle body and move it safely to the target location.

  Therefore, the general object detection device must be designed so that it can safely cope with an abnormality. Such a design configuration is called fail-safe, and fail-safe design is a design method that always controls to the safe side when an abnormality occurs in the device, or is a design method that makes such design reliable. It is one of. In other words, it is based on the premise that devices and systems will always fail.

  As an object detection apparatus as described above, Patent Document 1 discloses an object position that can estimate the position of an object in the front-rear direction even when it is difficult to determine the position of the object in the front-rear direction using a radar apparatus. A sensing device is disclosed. In this object position detection device, when the radar failure determination unit determines that the radar device has failed, the camera device estimates the moving direction of the object, and based on the estimated moving direction, the front-rear direction position of the object is estimated. It is characterized by estimation. That is, this is a fail-safe technique whose main purpose is to switch to another device (camera device) and continue operation when a radar sensor device that inputs a beat signal to the object detection unit fails.

  Patent Document 2 discloses an image processing apparatus that can improve the performance of object detection using an object detection method suitable for the environment even when the environment of the monitoring target region changes. Yes. In this image processing apparatus, an object detection unit suitable for the environment is used from among a plurality of object detection units provided based on information about the environment detected by a sensor different from the imaging device.

Japanese Patent Laid-Open No. 2014-21709 JP 2008-47991 A

  However, in the above conventional technology, only fail-safe technology for a failure in a device that acquires information on an object such as a radar device is disclosed, and any fail-safe technology for an object detector that performs object detection processing for detecting an object is examined. There has been a need for further improvements in fail-safe technology for object detectors.

  The present disclosure has been made in order to solve the above-described problem. Even when one object detector of a plurality of object detectors is in an abnormal state, a failure is performed while minimizing additional components. An object of the present invention is to provide an object detection apparatus, an object detection method, and an object detection program that can perform safe object detection.

  An object detection device according to an aspect of the present disclosure is an object detection device that detects an object, and an acquisition unit that acquires object information about the object, and each of which performs predetermined object detection processing using the object information. An object detection unit including a plurality of object detectors to be executed, an abnormality detection unit that detects an abnormal state of each of the plurality of object detectors, and one object of the plurality of object detectors by the abnormality detection unit A change unit that changes object detection processing of an object detector in which an abnormal state is not detected when an abnormal state of the detector is detected.

  According to the present disclosure, even if one of the plurality of object detectors is in an abnormal state, fail-safe object detection can be performed while minimizing additional components.

It is a block diagram which shows an example of the main structures of the vehicle carrying the object detection apparatus in Embodiment 1 of this indication. It is a block diagram which shows an example of a structure of the object detection apparatus shown in FIG. 3 is a flowchart illustrating an example of an object detection process at startup of the object detection apparatus illustrated in FIG. 2. It is a figure which shows an example of the detection process allocation vote which the detector switch part shown in FIG. 2 uses. It is a figure which shows an example of the image which the image information which the object detector shown in FIG. 2 uses for an object detection process represents. It is a figure for demonstrating an example of the object which the object detector detected from the image shown in FIG. It is a figure which shows an example of the meter image which shows the automatic driving | running state displayed on the display apparatus shown in FIG. FIG. 3 is a flowchart illustrating an example of an abnormal object detection process when an abnormality occurs in one of the object detectors of the object detection device illustrated in FIG. 2. FIG. It is a figure which shows an example of the detection process allocation vote which a detector switch part uses when abnormality generate | occur | produces in one of the object detectors shown in FIG. It is a block diagram for demonstrating an example of the fail safe operation | movement when abnormality arises in one of the object detectors of the object detection apparatus shown in FIG. It is a figure which shows an example of the instrument image displayed on a display apparatus when abnormality generate | occur | produces in one of the object detectors shown in FIG. It is a figure which shows another example of the detection process allocation tag which a detector switch part uses when abnormality generate | occur | produces in one of the object detectors shown in FIG. It is a block diagram which shows an example of a structure of the object detection apparatus in Embodiment 2 of this indication. It is a flowchart which shows an example of the object detection process at the time of starting of the object detection apparatus shown in FIG. It is a figure which shows an example of the NN storage part which the object detector of the object detection apparatus shown in FIG. 13 refers. It is a flowchart which shows an example of the object detection process at the time of abnormality when abnormality generate | occur | produces in one of the object detectors of the object detection apparatus shown in FIG. It is a figure which shows an example of the NN storage part which an object detector refers when abnormality occurs in one of the object detectors of the object detection apparatus shown in FIG.

(Knowledge that became the basis of the present invention)
As described above, Patent Document 1 discloses a fail-safe technique for a malfunction in an apparatus that acquires information on an object such as a radar apparatus, but fails regarding an object detector that executes an object detection process for detecting an object. No safe technology is disclosed.

  Further, in Patent Document 2, since a plurality of object detection units are provided, it is possible to assume an operation in which the object detection unit for fail-safe corresponds when the object detection unit breaks down. There is a problem that a separate object detection unit is required.

  In the field of autonomous vehicles, various proposals have been made regarding fail-safe technology when a camera device or a ranging sensor device fails. However, although an object detection device that receives information from a camera device or the like and detects a general object also fails, no consideration has been given to a design or a technical solution in consideration of fail-safe in the object detection device.

  In order to solve such a problem, an object detection device according to an aspect of the present disclosure is an object detection device that detects an object, and uses an acquisition unit that acquires object information about the object, and the object information. An object detection unit including a plurality of object detectors each executing a predetermined object detection process, an abnormality detection unit detecting an abnormal state of each of the plurality of object detectors, and the abnormality detection unit When an abnormal state of one of the object detectors is detected, the object detection processing of the abnormal object detector in which the abnormal state is detected among the plurality of object detectors is stopped, and the abnormal state is And a changing unit that changes object detection processing of a normal object detector that is not detected.

  With such a configuration, when an abnormal state of one of the object detectors is detected, the object detection process of the abnormal object detector in which the abnormal state is detected is stopped and the abnormal state is detected. Since the object detection process of the normal object detector that has not been changed has been changed, the object detection process of the abnormal object detector can be substituted for the normal object detector without newly providing a fail-safe object detector. Can do. As a result, even if one of the plurality of object detectors is in an abnormal state, fail-safe object detection can be performed while minimizing additional components.

  The object detection processing has processing priority, and the change unit detects the normal object when an abnormality state of one object detector of the plurality of object detectors is detected by the abnormality detection unit. The object detection process of the normal object detector that executes the object detection process with the lowest priority among the detectors may be changed to the object detection process of the abnormal object detector.

  With such a configuration, when an abnormal state of one of the plurality of object detectors is detected, the normal object detector that executes the object detection process with the lowest priority among the normal object detectors. Since the object detection process is changed to the object detection process of the abnormal object detector, the object detection process of the abnormal object detector can be substituted for the normal object detector that executes the object detection process with the lowest priority. The object detection process having a high priority can be continuously executed while minimizing the performance degradation of the object detection unit.

  The plurality of object detectors include at least three or more object detectors, and the change unit detects an abnormal state of one of the plurality of object detectors by the abnormality detection unit. In this case, one of the normal object detectors is caused to execute object detection processing with the highest priority, and the remaining normal object detectors are allowed to execute object detection processing with the next highest priority. It may be.

  With such a configuration, when an abnormal state of one object detector of a plurality of object detectors is detected, object detection processing having the highest priority over one normal object detector among normal object detectors And the remaining normal object detectors are caused to execute the object detection process with the next highest priority, so that the object detection process with the higher priority can be reliably and continuously executed.

  The plurality of object detectors include at least three or more object detectors, and the change unit detects an abnormal state of one of the plurality of object detectors by the abnormality detection unit. In this case, one of the normal object detectors is caused to execute object detection processing with the highest priority, and in addition to the object detection processing already executed by the remaining normal object detectors, the abnormality You may make it perform the object detection process of an object detector.

  With such a configuration, when an abnormal state of one object detector of a plurality of object detectors is detected, object detection processing having the highest priority over one normal object detector among normal object detectors In addition to the object detection process already executed by the remaining normal object detectors, the object detection process of the abnormal object detector is executed, so that all the object detection processes are executed continuously. Can do.

  The object detection unit may execute a recovery process for returning the abnormal object detector to a normal state when the abnormal object detector stops the object detection process.

  With this configuration, when the abnormal object detector stops the object detection process, recovery processing is performed to return the abnormal object detector to the normal state. Can be restored.

  Each of the plurality of object detectors includes an object detector using a neural network, and the object detection unit uses the data group for generating a neural network for detecting the object, Generating a neural network for use by each of the object detectors and assigning each generated neural network to each of the plurality of object detectors, each of the plurality of object detectors using the assigned neural network; The object is detected, and the change unit detects a neural network used by the normal object detector when the abnormality detection unit detects an abnormal state of one of the plurality of object detectors. It may be changed.

  With such a configuration, a neural network used by each of a plurality of object detectors is generated using a data group for generating a neural network for detecting an object, and each generated neural network is converted to a plurality of objects. Assigned to each of the detectors, each of the plurality of object detectors detects the object using the assigned neural network. In this state, when an abnormal state of one of the plurality of object detectors is detected, the neural network used by the normal object detector that has not detected the abnormal state is changed. When an abnormal state of one of the object detectors is detected, it is possible to eliminate the time spent for creating a new neural network and to prevent a blank period of object detection processing.

  The object detection unit further includes a storage unit that stores each of the plurality of generated neural networks in an individual area, and the change unit is one object detector of the plurality of object detectors by the abnormality detection unit. When the abnormal state is detected, the neural network used by the normal object detector may be changed by changing the individual area of the storage unit referred to by the normal object detector.

  With such a configuration, when an abnormal state of one of the plurality of object detectors is detected, the normal object detector is changed by changing the individual area of the storage unit referred to by the normal object detector. Therefore, the neural network used by the normal object detector can be instantaneously changed, and the object detection process using the neural network can be reliably and continuously executed.

  Each of the plurality of object detectors may be an object detector having the same performance.

  With such a configuration, the object detection processing of the abnormal object detector can be easily substituted for the normal object detector, and the cost of the object detection device can be reduced.

  The object information may include image information obtained by photographing the object with a predetermined camera device.

  With such a configuration, it is possible to detect a general object including a pedestrian, a vehicle, a road sign, a building, a road area, and the like using image information obtained by photographing the object with the camera device.

  The object detection device is mounted on a vehicle including a travel control device that controls a travel state, and the object detection unit adds the detection results of the plurality of object detectors to the travel control device based on an integrated detection result. You may make it further provide the operation control signal transmission part which outputs a control signal.

  With such a configuration, the object detection device is mounted on the vehicle, and the object detection unit controls the vehicle control state based on the integrated detection result obtained by integrating the detection results of the plurality of object detectors. Therefore, it is possible to detect general objects including pedestrians, other vehicles, road signs, buildings, road areas, etc. necessary for vehicle travel control even when an abnormality occurs in the object detector. In addition, using the integrated detection result obtained by integrating the detection results, fail-safe automatic driving control of the autonomous driving vehicle can be realized.

  The object is a first object located in a first distance range closest to the vehicle; a second object located in a second distance range farther than the first distance range from the vehicle; A third object located in a third distance range farther than the second distance range from the vehicle, the plurality of object detectors being a first object detector for detecting the first object; , A second object detector for detecting the second object, and a third object detector for detecting the third object, wherein the changing unit is configured to detect the first object by the abnormality detection unit. When an abnormal state of the detector is detected, the object detection process of the first object detector is stopped, and the third object detection is performed so that the third object detector detects the first object. The object detection processing of the vessel may be changed.

  With such a configuration, when an abnormal state of the first object detector is detected, the object detection processing of the first object detector is stopped, and the third object detector detects the first object. Since the object detection processing of the third object detector is changed to the first object detector, the first object located in the first distance range closest to the vehicle, for example, even when the abnormality of the first object detector occurs, It is possible to detect pedestrians, other vehicles, road signs, buildings, road areas, and the like, and to continue automatic driving of autonomous vehicles.

  In addition, the present disclosure can be realized not only as an object detection apparatus having the above-described characteristic configuration but also an object detection method for executing characteristic processing corresponding to the characteristic configuration included in the object detection apparatus. It can also be realized as such. It can also be realized as an object detection program that causes a computer to execute characteristic processing included in such an object detection method. Therefore, also in the following other aspects, the same effects as those of the object detection device can be obtained.

  An object detection method according to another aspect of the present disclosure is an object detection method of an object detection device that detects an object, acquires object information related to the object, and executes a plurality of object detection processes using the object information. And detecting an abnormal state of each of the plurality of object detection processes, and detecting an abnormal state of one of the plurality of object detection processes, and detecting an abnormal state of the plurality of object detection processes. The abnormal object detection process in which is detected is stopped, and the processing content of the normal object detection process in which no abnormal state is detected is changed.

  An object detection program according to another aspect of the present disclosure is an object detection program for causing a computer to function as an object detection device that detects an object, the object information relating to the object being acquired by the computer, and the object When a plurality of object detection processes are executed using information, an abnormal state of each of the plurality of object detection processes is detected, and an abnormal state of one of the plurality of object detection processes is detected The abnormal object detection process in which an abnormal state is detected among the plurality of object detection processes is stopped, and the processing content of the normal object detection process in which no abnormal state is detected is changed.

  Needless to say, the object detection program as described above can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

  In addition, a part of the constituent elements of the object detection device according to the embodiment of the present disclosure and other constituent elements may be configured as a system dispersed in a plurality of computers.

  Note that each of the embodiments described below is for showing a specific example of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements. In all the embodiments, the contents can be combined.

  Hereinafter, the object detection device system according to each embodiment of the present disclosure will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram illustrating an example of a main configuration of a vehicle on which the object detection device according to the first embodiment of the present disclosure is mounted. As shown in FIG. 1, the vehicle 1 includes a camera device 11, a distance measuring sensor device 12, a travel control device 13, a display device 14, and an object detection device 101. The vehicle 1 is an automatic driving vehicle that performs automatic driving using a traveling control device 13 that controls a traveling state and an object detection device 101 that detects an object. The vehicle 1 is not particularly limited to an autonomous driving vehicle, and may be a normal vehicle that is driven by a driver.

  The camera device 11 is composed of a CCD (Charge Coupled Devices) image sensor or the like. The camera device 11 captures an image including various objects (general objects) positioned in front of the vehicle 1, and the image information obtained by capturing the object is detected by the object detection device 101. Output to. Note that the configuration of the camera device 11 is not particularly limited to this example, and a plurality of camera devices may capture an object such as the rear, right side, or left side of the vehicle 1.

  The distance measuring sensor device 12 is composed of a radar device or the like, measures distances to various objects (general objects) located in front of the vehicle 1, and outputs distance information indicating the measured distances to the object detection device 101. . The configuration of the distance measuring sensor device 12 is not particularly limited to this example, and a plurality of distance measuring sensor devices are used to measure the distance to an object located at the rear, right side, left side, etc. of the vehicle 1. Alternatively, a stereo camera or the like may be used to output image information obtained by imaging distance information indicating the distance to the object.

  In the case of an autonomous driving vehicle, the object detection device 101 detects objects such as pedestrians, vehicles, road signs, buildings, and road areas as general objects, and generates a control signal (driving control signal) based on the object detection result. Output to the travel control device 13.

  The travel control device 13 includes an engine control unit and the like, and controls the travel state of the vehicle 1 based on various information including the object detection result of the object detection device 101, and performs accelerator operation, brake operation, steering wheel operation, and the like. Control automatically.

  The display device 14 is composed of a liquid crystal display or the like installed on the instrument panel of the vehicle 1, and displays various information output from the travel control device 13, such as the state of automatic driving and the operation state of the object detection device 101. indicate.

  FIG. 2 is a block diagram showing an example of the configuration of the object detection device shown in FIG. As shown in FIG. 2, the object detection apparatus 101 includes an information acquisition unit 102, an object detection unit 103, a detector abnormality detection unit 104, a detector switching unit 105, and an operation control signal transmission unit 106. The object detection unit 103 includes three object detectors 113, 123, and 133.

  The information acquisition unit 102 acquires image information obtained by capturing an object from the camera device 11 as object information related to the object, acquires distance information indicating a distance to the object from the distance measurement sensor device 12, and acquires the acquired image information and The distance information is output to the object detection unit 103. The object information is not particularly limited to the above example, and various other information regarding the object to be detected may be used.

  The object detection unit 103 manages the operations of the object detectors 113, 123, and 133, gives the image information and distance information output from the information acquisition unit 102 to the object detectors 113, 123, and 133, and the object detector 113, 123 and 133 cause the object detection process to be executed. Each of the object detectors 113, 123, and 133 performs a predetermined object detection process using the image information and the distance information. Each of the object detectors 113, 123, and 133 is an object detector having the same performance, and is preferably an object detector having the same configuration. The number and configuration of the object detectors are not particularly limited to the above example, and two or four or more object detectors can be used, or object detectors having different performance can be used. is there. The information processed by the object detector is not particularly limited to the above-mentioned image information, etc., and general objects are detected from information corresponding to detection signals of various sensor devices corresponding to radio waves, heat, sound, infrared rays, etc. It doesn't matter.

  Further, the object detectors 123 and 133 output the detection results of the objects detected by the object detection process to the object detector 113, and the object detector 113 and the detection results of the object detectors 123 and 133 are detected. And the integrated detection result is output to the operation control signal transmitter 106.

  The detector abnormality detection unit 104 detects the abnormal state of each of the object detectors 113, 123, and 133, and transmits detector abnormality information that notifies the object detector that has detected the abnormal state to the detector switching unit 105 and the operation control signal transmission. Notification to the unit 106. For example, the detector abnormality detection unit 104 monitors changes in the temperature and power consumption of the object detectors 113, 123, and 133 in hardware, and detects the object detectors 113, 123, and 133 in software. The abnormal state of the object detectors 113, 123, and 133 is detected by monitoring the output frequency of the detection results. When the object detectors 113, 123, 133 are operating normally, the detector abnormality detection unit 104 controls the detector normal information indicating that all the object detectors 113, 123, 133 are normal. You may transmit to the signal transmission part 106. FIG.

  The detector switching unit 105 controls the object detection unit 103 to switch the processing contents of the object detectors 113, 123, and 133, so that one of the object detectors 113, 123, and 133 is detected by the detector abnormality detection unit 104. When an abnormal state of one object detector is detected, the object detection process of the abnormal object detector in which the abnormal state is detected is stopped, and the object detection process of the normal object detector in which the abnormal state is not detected is changed. The object detection unit 103 performs a recovery process for returning the abnormal object detector to a normal state when the abnormal object detector stops the object detection process.

  In addition, each object detection process of the object detectors 113, 123, 133 has a priority of processing, and the detector switching unit 105 is detected by the detector abnormality detection unit 104 using the object detectors 113, 123, 133. When an abnormal state of one of the object detectors is detected, the object detection unit 103 is controlled to execute object detection processing with the lowest priority among normal object detectors in which no abnormal state is detected. The object detection process of the normal object detector is changed to the object detection process of the abnormal object detector in which the abnormal state is detected.

  In addition, the detector switching unit 105 controls the object detection unit 103 when the abnormal state of one of the object detectors 113, 123, and 133 is detected by the detector abnormality detection unit 104. One of the normal object detectors in which no abnormal state has been detected is caused to execute the highest priority object detection process, and the remaining normal object detectors have the next highest priority object detection process. Is executed.

  Further, the object detected by the object detection device 101 is positioned in the first distance range that is closest to the vehicle 1 and in the second distance range that is farther from the vehicle 1 than the first distance range. And a third object located in a third distance range farther from the vehicle 1 than the second distance range. The object detector 113 detects a first object located in the first distance range, the object detector 123 detects a second object located in the second distance range, and the object detector 133 A third object located in the third distance range is detected.

  Further, when the abnormal state of the object detector 113 is detected by the detector abnormality detection unit 104, the detector switching unit 105 controls the object detection unit 103 to stop the object detection process of the object detector 113, The object detection process of the object detector 133 is changed so that the object detector 133 detects the first object located in the first distance range.

  In addition, the detector switching unit 105 controls the object detection unit 103 when the abnormal state of one of the object detectors 113, 123, and 133 is detected by the detector abnormality detection unit 104. One of the normal object detectors in which no abnormal state has been detected is caused to execute object detection processing with the highest priority, and the remaining normal object detectors are already executing object detection processing. In addition, the object detection process of the abnormal object detector in which the abnormal state is detected may be executed.

  The driving control signal transmission unit 106 outputs a driving control signal to the traveling control device 13 based on the integrated detection result obtained by integrating the detection results of the object detectors 113, 123, and 133.

  Next, the operation of the object detection apparatus 101 shown in FIG. 2 will be described. FIG. 3 is a flowchart illustrating an example of an object detection process at startup of the object detection apparatus 101 illustrated in FIG. The startup-time object detection process shown in FIG. 3 is a process executed when the object detection device 101 is mounted on the vehicle 1 (automatic driving vehicle) and the automatic driving function is started.

  First, the detector switching unit 105 determines the object detection processing performed by the object detectors 113, 123, and 133 based on the detection processing assignment vote stored in an internal memory (not shown) as object detection. The object detection unit 103 that manages the devices 113, 123, and 133 is notified (step S11).

  Here, the detector switching unit 105 stores a detection processing assignment form in which a detector ID (identification number) for identifying an object detector, processing content, and priority is associated with each other in a table format in an internal memory. ing. FIG. 4 is a diagram showing an example of a detection process assignment slip used by the detector switching unit 105 shown in FIG. As shown in FIG. 4, the detection process assignment vote T1 is composed of a detector IDT2 that is an identifier associated with each object detector 113, 123, 133, a process content T3, and a priority T4. The

  In the example shown in FIG. 4, “object detector 113”, “object detector 123”, and “object detector 133” are used as the detector IDT2 for ease of explanation, and “ “Distance”, “Medium distance”, “Long distance”, and “Detection result integration” are used, and “1”, “2”, and “3” are used as the priority T4. “Short distance” indicates processing contents for detecting a general object at a short distance (first distance range) to the vehicle 1, and “medium distance” and “far distance” are also intermediate to the vehicle 1. The processing contents for detecting a general object at a distance (second distance range) and a far distance (third distance range) are shown. “Detection result integration” indicates the detection results of the object detectors 113, 123, and 133. Indicates the process to be integrated. The priority T4 indicates processing with higher priority as the numerical value is smaller.

  For example, the detector switching unit 105 sets the processing content of the object detector 113 to “near distance” and “detection result integration” with the priority “1” based on the detection processing assignment form T1 shown in FIG. The object detection unit 103 is notified to set the processing content of the device 123 to “medium distance” with the priority “2” and the processing content of the object detector 133 to “far” with the priority “3”.

  In addition, although the example which the detector switching part 105 preserve | saves as a storage method of a detection process allocation tag was demonstrated, it is not specifically limited to this example, Even if apparatuses other than the object detection apparatus 101 store a detection process allocation tag Good. Further, the object detection unit 103 stores the pair information of the processing content T3 and the priority T4, and the detector switching unit 105 notifies the object detection unit 103 of the pair information of the detector IDT2 and the priority T4. Also good. The priority T4 is determined based on the distance from the vehicle 1 to the general object. However, the priority T4 is not particularly limited to this example, and the type of the general object to be detected in the processing content T3 (person, road sign, obstacle, The priority value may be changed in accordance with the animal or the like (for example, the priority is lowered in the order of person, road sign, obstacle, and animal).

  Here, the object detectors 113, 123, and 133 are object detectors having the same performance, and in the image in which the general object to be detected is captured by the camera device 11 as the general object to be detected becomes a farther distance. Therefore, the calculation amount of the object detector 123 increases from the calculation amount of the object detector 113, and the calculation amount of the object detector 133 increases from the calculation amount of the object detector 123.

  Therefore, in the present embodiment, the object detection processing time for one image (one frame) of the object detector 133 is longer than that of the object detectors 113 and 123. Therefore, in the present embodiment, as an example, the information acquisition unit 102 transfers an image to the object detection unit 103 by 60 FPS (Frames Per Second), and the detection frequency is 60 FPS, and the object detection unit 123 is the detection frequency. Detects an ordinary object at 30 FPS and the object detector 133 at 15 FPS. The object detector 113 integrates the detection results of the object detectors 113, 123, and 133 with a detection frequency of 60 FPS.

  That is, the detection result of the object detector 113 is updated every frame, but the detection result of the object detector 123 is updated every two frames, and the detection result of the object detector 133 is updated every four frames. The distance to the general object is determined by distance information from the distance measuring sensor device 12. When the distance measuring sensor device 12 is not used, the distance to the general object is determined by the size of the image area for one frame of the general object to be detected by each object detector 113, 123, 133. Also good.

  Next, the object detection unit 103 inputs the image information and the like from the information acquisition unit 102 to the object detectors 113, 123, and 133, and the object detector 113 according to the notification from the detector switching unit 105, that is, the detection processing allocation vote. , 123, 133 are caused to execute object detection processing with the notified processing contents and priority, and the operations of the object detectors 113, 123, 133 are started (step S12). At this time, the information acquisition unit 102 may acquire the image information of the camera device 11 and may acquire and acquire the distance information with respect to the general object by the distance measuring sensor device 12. In addition, the object detection unit 103 transfers the detection results of the general objects from the object detectors 113, 123, and 133 to the operation control signal transmission unit 106.

  FIG. 5 is a diagram illustrating an example of an image represented by image information used by the object detectors 113, 123, and 133 illustrated in FIG. 2 for object detection processing, and FIG. 6 illustrates the object detector 113 from the image illustrated in FIG. , 123, 133 are diagrams for explaining an example of the object detected.

  When image information representing one frame image P1 shown in FIG. 5 is input to the information acquisition unit 102, and the object detectors 113, 123, and 133 execute object detection processing on the image P1 transferred from the information acquisition unit 102. 6, image information representing an image equivalent to the detection result image P <b> 2 illustrated in FIG. 6 is transferred to the operation control signal transmission unit 106.

  In the example shown in FIG. 6, based on the detection processing assignment slip T1, the object detector 113 detects a road accessible area N1 and another vehicle N2 as an object at a short distance, and the object detector 123 The road passable area M1 and the person M2 are detected as distance objects, and the object detector 133 detects the road passable area F1 and the road sign F2 as long distance objects. The object detector 113 integrates these detection results and transfers them to the operation control signal transmitter 106. As described above, the object detectors 113, 123, and 133 to which each processing content is assigned based on the distance from the vehicle 1 detect a general object that may exist in each distance range.

  Next, when the object detection unit 103 starts operating, the detector abnormality detection unit 104 starts monitoring the object detectors 113, 123, and 133 (step S13). At this time, the detector abnormality detection unit 104 detects the abnormal state of each of the object detectors 113, 123, and 133, and sends the object detector that has detected the abnormal state to the detector switching unit 105 and the operation control signal transmission unit 106. Notice.

  Next, the driving control signal transmission unit 106 outputs a driving control signal to the traveling control device 13 based on the detection result from the object detecting unit 103, and the traveling control device 13 automatically outputs the vehicle 1 based on the driving control signal. Operation is started (step S14).

  FIG. 7 is a diagram showing an example of an instrument image showing the automatic operation state displayed on the display device 14 shown in FIG. When the object detectors 113, 123, and 133 are operating in a normal state, for example, the instrument image shown in FIG. 7 is displayed on the display device 14 provided on the instrument panel of the vehicle 1. This example is an example in the case where the vehicle 1 is an electric vehicle, and for example, a battery remaining amount meter BM, a speed meter SM, and a power meter PM are displayed as instrument images.

  In addition, “Cruise Mode” M1 indicating that the automatic operation is normally performed is displayed as an operation mode at the lower center of the speedometer SM. Further, “100% Short” D1 indicating the operation state of the object detector 113 (a state in which an object detection process at a short distance is being executed with a processing capacity of 100%) is displayed thereon, and the object detector is displayed thereon. “100% Middle” D2 indicating the operation state of 123 (the state in which the medium distance object detection process is executed with 100% processing capacity) is displayed, and the operation state of the object detector 133 (100% “100% Long” D3 is displayed, which indicates a state in which a long-distance object detection process is being executed with a processing capability.

  By displaying the instrument image as described above on the display device 14, it is possible to show the passenger that the vehicle is in automatic operation and the states of the object detectors 113, 123, and 133.

  The above is the processing content when the automatic driving vehicle of the object detection apparatus 101 according to the present embodiment is activated (when the automatic driving function is activated).

  Next, as an example of the fail-safe operation of the object detection apparatus 101, a case where an abnormality has occurred in one of the object detectors 113, 123, and 133 after the start of automatic operation will be described. FIG. 8 is a flowchart showing an example of an abnormal object detection process when an abnormality occurs in one of the object detectors 113, 123, 133 of the object detection apparatus 101 shown in FIG.

  First, when one of the object detectors 113, 123, and 133 detects an operation abnormality, the detector abnormality detection unit 104 detects which object detector is in an abnormal state to the detector switching unit 105 and the operation control signal transmission unit 106. Detector abnormality information indicating whether or not there is is notified (step S21). For example, when an abnormality occurs in the object detector 113, the detector abnormality detection unit 104 indicates to the detector switching unit 105 and the operation control signal transmission unit 106 that the detector abnormality indicating that the object detector 113 is in an abnormal state. Notify information.

  Next, the detector switching unit 105 that has received the detector abnormality information uses a detection process assignment form to be used when one of the object detectors 113, 123, and 133 is abnormal from the detection process assignment form T1 shown in FIG. The object detection unit 103 is notified that the processing contents of the object detectors 113, 123, and 133 are switched so as to perform a fail-safe operation in accordance with the detection process assignment form (step S22). . Here, the detection processing assignment slip when one of the object detectors 113, 123, 133 is abnormal is the detector switching unit for each object detector in the abnormal state, as in the detection processing assignment slip T1 shown in FIG. The detector switching unit 105 reads out and uses the detection processing assignment sheet for the object detector in the abnormal state from the internal memory. Note that the detection processing assignment vote for the object detector in the abnormal state is not particularly limited to the example stored in the internal memory, and for example, the detector switching unit 105 or the like can process the object detectors 113, 123, and 133. Based on the above, a detection processing assignment slip that can perform an appropriate fail-safe operation may be created.

  FIG. 9 is a diagram illustrating an example of a detection processing assignment slip used by the detector switching unit 105 when an abnormality occurs in the object detector 113 illustrated in FIG. In the detection processing assignment form T1a shown in FIG. 9, the processing content T3a and the priority T4a are changed, and the processing content and priority of the object detector 113 are changed to “restoration processing” and “−” (no priority). The processing content and priority of the object detector 133 are changed to “short distance, detection result integration” and “1”, and the processing content and priority of the object detector 123 are maintained. Here, the “recovery process” indicates a process of stopping the operation of the abnormal object detector in which the abnormal state is detected and returning the abnormal object detector to the normal state.

  For example, when an abnormality occurs in the object detector 113, the detector switching unit 105 sets the processing content of the object detector 113 to “restoration processing” with the priority “−” based on the detection processing assignment form T1a illustrated in FIG. ”, The processing content of the object detector 123 is set to“ medium distance ”with the priority“ 2 ”, and the processing content of the object detector 133 is set to“ short distance ”and“ integration of detection results ”with the priority“ 1 ”. Thus, the object detection unit 103 is notified.

  Next, the object detection unit 103 changes the processing of the object detectors 113, 123, and 133 based on the detection processing assignment vote when one of the object detectors 113, 123, and 133 is abnormal, and the integrated detection after the change is made. The result is output to the operation control signal transmitter 106 (step S23). For example, when an abnormality occurs in the object detector 113, the object detection unit 103 performs “restoration processing” for returning the object detector 113 to a normal state based on the processing content T3a, and the processing content T3a and the priority. Based on the degree T4a, the object detection process of the object detector 133 is changed to “short distance, detection result integration” and “priority 1”.

  FIG. 10 is a block diagram for explaining an example of a fail-safe operation when an abnormality occurs in the object detector 113 of the object detection apparatus 101 shown in FIG. As shown in FIG. 10, when an abnormality occurs in the object detector 113, the object detector 103 executes a recovery process for returning the object detector 113 to a normal state, and the object detector 123 The middle-range object detection process is executed with priority 2, and the detection result is output to the object detector 133. The object detector 133 performs the near-distance object detection process and the detection result integration process with priority 1, The integrated detection result is output to the operation control signal transmitter 106.

  Then, the operation control signal transmission unit 106 causes the travel control device 13 of the vehicle 1 to reduce the travel speed of the vehicle 1 or stop it at a safe place based on the detector abnormality information and the integrated detection result of the object detection unit 103. An operation control signal for performing a fail-safe operation such as a fault is transmitted (step S24).

  FIG. 11 is a diagram illustrating an example of an instrument image displayed on the display device 14 when an abnormality occurs in one of the object detectors 113, 123, and 133 illustrated in FIG. For example, when an abnormality occurs in the object detector 113 and the abnormal object detection process shown in FIG. 8 is performed, the instrument image shown in FIG. 11 is displayed on the display device 14 provided on the instrument panel of the vehicle 1. The Also in this example, the battery remaining amount meter BM and the power meter PM are displayed as the instrument image as in the instrument image shown in FIG. 7, but the display content of the speedometer SMa is changed.

  That is, “Fail Safe Mode” M2 indicating that a fail-safe operation is being performed is displayed as an operation mode in the lower center of the speedometer SMa. Further, “0% Recovery” D1a indicating that the object detector 113 is in an abnormal state, the object detection process is stopped, and the recovery process is being performed is displayed on the object detector 123. “100% Middle” D2 indicating an operation state (a state in which a medium-distance object detection process is being executed with a processing capacity of 100%) is displayed, and an operation state of the object detector 133 (a processing capacity of 100%) “100% Short” D3a is displayed, indicating that the object detection process at a short distance is being executed.

  By displaying the instrument image as described above on the display device 14, it is possible to indicate to the passenger that the object detector 113 is in a restored state and is in a fail-safe operation state for avoiding danger.

  With the above processing, in the present embodiment, when an abnormal state of one of the plurality of object detectors 113, 123, 133 is detected, the object of the abnormal object detector in which the abnormal state is detected Since the detection process is stopped and the object detection process of the normal object detector where the abnormal state is not detected has been changed, the object detection process of the abnormal object detector is not provided without newly providing a fail-safe object detector. Can be substituted for a normal object detector. As a result, even if one of the plurality of object detectors 113, 123, 133 is in an abnormal state, fail-safe object detection can be performed while minimizing additional components. .

  In addition, when an abnormal state of one object detector among the plurality of object detectors 113, 123, 133 is detected, the change contents of the object detection process of the normal object detector in which the abnormal state is not detected are: It is not specifically limited to the example shown in FIG. 9, and various changes are possible. For example, depending on the state of the vehicle 1 and the external environment, the object detection process of the normal object detector may be changed based on a detection process assignment form as shown in FIG.

  FIG. 12 is a diagram showing another example of the detection processing assignment slip used by the detector switching unit 105 when an abnormality occurs in the object detector 113 shown in FIG. In the detection process assignment form T1b shown in FIG. 12, the processing contents and priorities of the object detectors 113 and 133 and the priority of the object detector 123 are the same as those of the detection process assignment form T1a shown in FIG. The processing content T3b of 123 is changed to “long distance and medium distance, detection frequency 50%”.

  In the present embodiment, the detection frequency in the normal state is 60 FPS for short-distance object detection processing, 30 FPS for medium-distance object detection processing, and 15 FPS for long-distance object detection processing, but in the example shown in FIG. When an abnormality occurs in the object detector 113, based on the processing content T3b of the detection processing assignment vote T1b, the object detector 123 performs half of the calculation resources for medium-range object detection processing and the other half for long distances. The general object is detected by allocating to the object detection process. That is, in this case, the detection frequency is changed to 60 FPS for the short-distance object detection process, 15 FPS for the medium-distance object detection process, and 7 FPS for the long-distance object detection process.

  Therefore, when an abnormal state of the object detector 113 is detected, the object detector 133 is caused to execute the short-range object detection processing and detection result integration processing having the highest priority, and the object detector 123 has already executed Since the long-distance object detection process is executed in addition to the medium-distance object detection process, all the object detection processes can be continuously executed.

(Embodiment 2)
FIG. 13 is a block diagram illustrating an exemplary configuration of the object detection device according to the second embodiment of the present disclosure. In the present embodiment, an object detection apparatus 101a using a neural network (hereinafter referred to as “NN”) in an autonomous driving vehicle will be described with reference to FIG. The object detection apparatus 101a shown in FIG. 13 is basically the same as the object detection apparatus 101 shown in FIG. 2 except that the object detection process is performed using a neural network described below. Are denoted by the same reference numerals, and description thereof is omitted.

  As shown in FIG. 13, the object detection apparatus 101a includes an information acquisition unit 102, an object detection unit 103a, a detector abnormality detection unit 104, a detector switching unit 105, an operation control signal transmission unit 106, and an NN (neural network) storage unit. 201. The object detection unit 103a includes three object detectors 113a, 123a, 133a, and an NN (neural network) storage unit 202. The NN storage unit 202 includes three NN (neural network) storage units 213, 223, and 233 as individual storage areas.

  In recent years, as a general object detection using NN, it is possible to obtain high detection performance by using NN that has been learned by a technique called “Deep Learning” (deep learning). Instead of the conventional object detector using In the present embodiment, object detection processing is executed using a neural network that has been learned using deep learning as follows.

  The NN storage unit 201 stores NN (neural network) generation data as a data group for generating a neural network for detecting an object. The NN generation data is data in which parameters relating to a node necessary for generating an NN learned for each purpose by the Deep Learning method, a network configuration between nodes, and the like are defined. The method for acquiring the NN generation data is not particularly limited to this example. For example, the NN storage unit 201 may be omitted and the NN generation data may be acquired from a predetermined server or the like via a predetermined network.

  The object detection unit 103a reads the NN generation data from the NN storage unit 201, generates a neural network used by each of the object detectors 113a, 123a, and 133a using the NN generation data, and outputs the generated neural networks to the NN. The data are stored in the storage units 213, 223, and 233 of the storage unit 202, respectively. The configuration of the NN storage unit 202 is not particularly limited to the above example. For example, each neural network may be stored in an individual storage unit. In addition, the memory areas (storage units 213, 223, 233) in which the generated neural networks are stored are multiplexed in order to improve fault tolerance, and are replicated to a place where the object detectors 113a, 123a, 133a can refer to. It may be.

  The object detection unit 103a manages the operation of the object detectors 113a, 123a, and 133a, gives the image information and distance information output from the information acquisition unit 102 to the object detectors 113a, 123a, and 133a, and detects them. The neural networks of the NN storage units 213, 223, and 233 are allocated to the object detectors 113a, 123a, and 133a in accordance with the processing contents of the object detectors 113a, 123a, and 133a based on the detection processing assignment vote from the detector switching unit 105.

  Each of the object detectors 113a, 123a, and 133a executes object detection processing using the assigned neural network using the image information and distance information from the information acquisition unit 102. Each of the object detectors 113a, 123a, and 133a is an object detector having the same performance, and is preferably an object detector having the same configuration.

  The object detectors 123a and 133a output the detection result of the object detected by the object detection process using the assigned neural network to the object detector 113a. The object detector 113a The detection results of the detectors 123 a and 133 a are integrated, and the integrated detection result is output to the operation control signal transmitter 106.

  The detector abnormality detection unit 104 detects the abnormal state of each of the object detectors 113a, 123a, and 133a, and transmits the detector abnormality information notifying the object detector that has detected the abnormal state to the detector switching unit 105 and the operation control signal transmission. Notification to the unit 106.

  The detector switching unit 105 controls the object detection unit 103a to switch the processing contents of the object detectors 113a, 123a, and 133a, so that one of the object detectors 113a, 123a, and 133a is detected by the detector abnormality detection unit 104. When an abnormal state of one object detector is detected, the object detection process of the abnormal object detector in which the abnormal state is detected is stopped, and the object detection process of the normal object detector in which the abnormal state is not detected is changed. The object detection unit 103a executes a restoration process for returning the abnormal object detector to a normal state when the abnormal object detector stops the object detection process.

  The detector switching unit 105 also stores the NN memory that the normal object detector refers to when the abnormal state of one of the object detectors 113a, 123a, and 133a is detected by the detector abnormality detection unit 104. The neural network used by the normal object detector is changed by changing the NN storage units 213, 223, and 233 of the unit 202.

  Next, the operation of the object detection apparatus 101a shown in FIG. 13 will be described. FIG. 14 is a flowchart illustrating an example of a startup object detection process of the object detection apparatus 101a illustrated in FIG. The object detection process shown in FIG. 14 is a process executed when the object detection device 101a is mounted on the vehicle 1 (automatic driving vehicle) (see FIG. 1) and the automatic driving function is started.

  First, the detector switching unit 105 performs object detection performed by the object detectors 113a, 123a, and 133a based on the detection process assignment vote (for example, the detection process assignment vote shown in FIG. 4) stored in the internal memory. The processing contents of the processing are notified to the object detection unit 103a that manages the object detectors 113a, 123a, and 133a (step S31).

  Next, the object detection unit 103a reads NN generation data from the NN storage unit 201 based on the information notified from the detector switching unit 105, and generates a neural network for each of the object detectors 113a, 123a, and 133a. The data are stored in the NN storage units 213, 223, and 233, respectively, and the neural networks of the NN storage units 213, 223, and 233 are assigned to the object detectors 113a, 123a, and 133a (step S32).

  FIG. 15 is a diagram illustrating an example of the NN storage units 213, 223, and 233 referred to by the object detectors 113a, 123a, and 133a of the object detection apparatus 101a illustrated in FIG. In the example shown in FIG. 15, a neural network for short-range object detection processing is stored in the NN storage unit 213 of the NN storage unit 202, and a neural network for medium-range object detection processing is stored in the NN storage unit 223. A neural network for long-distance object detection processing is stored in the NN storage unit 233.

  At this time, the object detection unit 103a assigns the NN storage unit 213 to the object detector 113a, the NN storage unit 223 to the object detector 123a, and the NN storage unit 233 to the object detector 133a. Therefore, the object detector 113a refers to the NN storage unit 213 and uses a neural network for object detection processing at a short distance, and the object detector 123a refers to the NN storage unit 223 to detect an intermediate distance object. A neural network for detection processing is used, and the object detector 133a refers to the NN storage unit 233 and uses a neural network for long-distance object detection processing.

  Next, the object detection unit 103a inputs the image information and the like from the information acquisition unit 102 to the object detectors 113a, 123a, and 133a, and the object detector 113a according to the notification from the detector switching unit 105, that is, the detection processing allocation vote. , 123a, and 133a, the object detection process using the neural network of the NN storage units 213, 223, and 233 is executed with the notified processing contents and priority, and the operations of the object detectors 113a, 123a, and 133a are performed. Start (step S33).

  Next, when the object detection unit 103a starts operating, the detector abnormality detection unit 104 starts monitoring the object detectors 113a, 123a, and 133a (step S34).

  Next, the driving control signal transmitting unit 106 outputs a driving control signal to the traveling control device 13 based on the detection result from the object detecting unit 103a, and the traveling control device 13 automatically outputs the vehicle 1 based on the driving control signal. Operation is started (step S35).

  The above is the processing content when the automatic driving vehicle of the object detection apparatus 101a in the present embodiment is started (when the automatic driving function is started).

  Next, as an example of the fail-safe operation of the object detection apparatus 101a, a case where an abnormality has occurred in one of the object detectors 113a, 123a, and 133a after the start of automatic operation will be described. FIG. 16 is a flowchart illustrating an example of an abnormal object detection process when an abnormality occurs in one of the object detectors 113a, 123a, and 133a of the object detection apparatus 101a illustrated in FIG.

  First, when the detector abnormality detection unit 104 detects one operation abnormality of the object detectors 113a, 123a, and 133a, the detector switching unit 105 and the operation control signal transmission unit 106 indicate which object detector is in an abnormal state. Detector abnormality information indicating whether or not there is is notified (step S41). For example, when an abnormality occurs in the object detector 113a, the detector abnormality detection unit 104 notifies the detector switching unit 105 and the operation control signal transmission unit 106 that the object detector 113a is in an abnormal state. Notify information.

  Next, the detector switching unit 105 that has received the detector abnormality information uses a detection process assignment form to be used when one of the object detectors 113a, 123a, and 133a is abnormal from the detection process assignment form T1 shown in FIG. The processing contents of the object detectors 113a, 123a, and 133a are switched so as to perform a fail-safe operation in accordance with the detection process assignment form (for example, the detection process assignment form T1a shown in FIG. 9). This is notified to the object detection unit 103a (step S42).

  Next, the object detection unit 103a uses the neural network object detectors 113a, 123a, 123a of the NN storage units 213, 223, 233 based on the detection processing assignment vote when one of the object detectors 113a, 123a, 133a is abnormal. The processing of the object detectors 113a, 123a, and 133a is changed by changing the assignment to the 133a and switching the NN storage units 213, 223, and 233 referenced by the object detectors 113a, 123a, and 133a, and the integration after the change The detection result is output to the operation control signal transmitter 106 (step S43). For example, when an abnormality occurs in the object detector 113a, the object detection unit 103a performs a “recovery process” for returning the object detector 113a to a normal state based on the processing content T3a, and the processing content T3a and the priority. Based on the degree T4a, the object detection process of the object detector 133a is changed to “short distance, detection result integration” and “priority 1”.

  FIG. 17 illustrates the NN storage units 213, 223, and 233 to which the object detectors 113a, 123a, and 133a refer when an abnormality occurs in one of the object detectors 113a, 123a, and 133a of the object detection apparatus 101a illustrated in FIG. It is a figure which shows an example. The example shown in FIG. 17 is an example when an abnormality occurs in the object detector 113a. The object detector 103a stops the object detector 113a, the NN storage unit 223 is inserted in the object detector 123a, and the object detector. The NN storage unit 213 is allocated to 133a.

  Accordingly, the object detector 133a refers to the NN storage unit 213 and uses a short-distance object detection processing neural network, and the object detector 123a refers to the NN storage unit 223 and refers to the medium-range object. A neural network for detection processing is used.

  Then, the operation control signal transmission unit 106 causes the travel control device 13 of the vehicle 1 to reduce the travel speed of the vehicle 1 or stop it at a safe place based on the detector abnormality information and the integrated detection result of the object detection unit 103a. An operation control signal for performing a fail-safe operation such as a fault is transmitted (step S44).

  With the above processing, in the present embodiment, in addition to the effects of the first embodiment, when an abnormal state of one of the plurality of object detectors 113a, 123a, 133a is detected, a normal object By changing the NN storage units 213, 223, and 233 referred to by the detector, the neural network used by the normal object detector in which the abnormal state is not detected is changed. Therefore, the plurality of object detectors 113a and 123a are used. When an abnormal state of one of the object detectors 133a is detected, it is possible to eliminate the time spent for creating a new neural network and to prevent a blank period of object detection processing. As a result, even when an abnormal state of one of the plurality of object detectors 113a, 123a, and 133a is detected, the neural network used by the normal object detector can be changed instantaneously, Since the object detection process using the network can be reliably and continuously executed, it is possible to prevent the automatic operation from being blanked.

  In the above description, an example in which the object detection device of the present disclosure is applied to an autonomous driving vehicle has been described. However, the present invention is not particularly limited to this example, and is applicable to different product fields such as robots and drones (unmanned aerial vehicles). it can.

  In addition, each component described in each embodiment may be realized as software. In this case, the software is recorded on a non-temporary recording medium such as one or a plurality of ROM (Read only memory), an optical disk, a hard disk drive, and the software is executed when the software is executed by a processor. The identified function is executed by the processing device and the peripheral device.

  Each component described in each embodiment may be typically realized as an LSI (Large Scale Integration) which is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them. Although referred to as LSI here, it may be called IC (semiconductor integrated circuit), system LSI, super LSI, or ultra LSI depending on the degree of integration. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out component integration using this technology.

  The object detection device according to the present disclosure is useful as an object detection device that detects an object by minimizing additional components and further improving the fail-safe object detection device.

DESCRIPTION OF SYMBOLS 1 Vehicle 11 Camera apparatus 12 Distance sensor apparatus 13 Traveling control apparatus 14 Display apparatus 101, 101a Object detection apparatus 102 Information acquisition part 103, 103a Object detection part 104 Detector abnormality detection part 105 Detector switching part 106 Operation control signal transmission part 113, 113a, 123, 123a, 133, 133a Object detector 201 NN storage unit 202 NN storage unit 213, 223, 233 NN storage unit

Claims (13)

An object detection device for detecting an object,
An acquisition unit for acquiring object information relating to the object;
An object detection unit including a plurality of object detectors each performing a predetermined object detection process using the object information;
An abnormality detector for detecting an abnormal state of each of the plurality of object detectors;
When the abnormality detection unit detects an abnormal state of one of the plurality of object detectors, the object detection processing of the abnormal object detector in which the abnormal state is detected among the plurality of object detectors A change unit that changes the object detection processing of a normal object detector in which an abnormal state is not detected,
An object detection apparatus comprising: The object detection processing has processing priority,
The changing unit executes an object detection process having the lowest priority among the normal object detectors when an abnormality state of one of the plurality of object detectors is detected by the abnormality detection unit. Changing the object detection process of the normal object detector to the object detection process of the abnormal object detector,
The object detection apparatus according to claim 1. The plurality of object detectors includes at least three or more object detectors;
The change unit has the highest priority over one normal object detector of the normal object detectors when the abnormality detection unit detects an abnormal state of one of the plurality of object detectors. The object detection process with a high degree is executed, and the object detection process with the next highest priority is executed with the remaining normal object detectors.
The object detection apparatus according to claim 2. The plurality of object detectors includes at least three or more object detectors;
The change unit has the highest priority over one normal object detector of the normal object detectors when the abnormality detection unit detects an abnormal state of one of the plurality of object detectors. The object detection process having a high degree is executed, and the object detection process of the abnormal object detector is executed in addition to the object detection process already executed by the remaining normal object detectors.
The object detection apparatus according to claim 2. The object detection unit executes a restoration process for returning the abnormal object detector to a normal state when the abnormal object detector stops the object detection process.
The object detection apparatus in any one of Claims 1-4. Each of the plurality of object detectors includes an object detector using a neural network;
The object detection unit generates a neural network used by each of the plurality of object detectors using a data group for generating a neural network for detecting the object, and generates each of the generated neural networks. Assigned to each of a plurality of object detectors,
Each of the plurality of object detectors detects the object using an assigned neural network;
The change unit changes a neural network used by the normal object detector when an abnormal state of one of the plurality of object detectors is detected by the abnormality detection unit.
The object detection apparatus according to claim 1. The object detection unit further includes a storage unit that stores each of the generated neural networks in an individual area,
The changing unit changes an individual area of the storage unit referred to by the normal object detector when an abnormality state of one of the plurality of object detectors is detected by the abnormality detecting unit. By changing the neural network used by the normal object detector,
The object detection apparatus according to claim 6. Each of the plurality of object detectors is an object detector having the same performance.
The object detection apparatus according to claim 1. The object information includes image information obtained by photographing the object with a predetermined camera device.
The object detection apparatus according to claim 1. The object detection device is mounted on a vehicle including a travel control device that controls a travel state,
The object detection unit further includes an operation control signal transmission unit that outputs a control signal to the travel control device based on an integrated detection result obtained by integrating detection results of the plurality of object detectors.
The object detection apparatus according to claim 1. The object is
A first object located in a first distance range closest to the vehicle;
A second object located in a second distance range farther than the first distance range from the vehicle;
A third object located in a third distance range farther than the second distance range from the vehicle,
The plurality of object detectors are:
A first object detector for detecting the first object;
A second object detector for detecting the second object;
A third object detector for detecting the third object;
The change unit stops the object detection process of the first object detector when the abnormality detection unit detects an abnormal state of the first object detector, and the third object detector Changing the object detection process of the third object detector to detect the first object;
The object detection apparatus according to claim 9. An object detection method of an object detection device for detecting an object,
Obtaining object information about the object;
A plurality of object detection processes are executed using the object information,
Detecting an abnormal state of each of the plurality of object detection processes;
When an abnormal state of one of the plurality of object detection processes is detected, the abnormal object detection process in which the abnormal state is detected among the plurality of object detection processes is stopped, and the abnormal state is detected. Change the content of normal object detection processing that is not
Object detection method. An object detection program for causing a computer to function as an object detection device for detecting an object,
In the computer,
Obtaining object information about the object;
A plurality of object detection processes are executed using the object information,
Detecting an abnormal state of each of the plurality of object detection processes;
When an abnormal state of one of the plurality of object detection processes is detected, the abnormal object detection process in which the abnormal state is detected among the plurality of object detection processes is stopped, and the abnormal state is detected. Change the content of normal object detection processing that is not
An object detection program for executing processing.

Images