JP2020050047A - Object detection device - Google Patents

Abstract

To provide an object detection device which can improve the detection accuracy of an object when a prescribed condition is satisfied while suppressing the lowering of the detection accuracy of the object when the prescribed condition is not satisfied.SOLUTION: An object detection device detects an object existing around a vehicle 10 by inputting a sensor signal from a sensor 2 into a neural network (NN), a first partial network out of a plurality of partial networks possessed by at least one layer of the NN is made to perform learning in advance so as to detect the object when a prescribed condition related to the vehicle 10 is satisfied, and the other partial network has a detection part 33 which is made to perform learning in advance so as to detect the object even if the prescribed condition is not satisfied, and a network control part 32 for controlling the NN so as to use the first partial network for the detection of the object when the prescribed condition is satisfied, and on the other hand, when the prescribed condition is not satisfied, not to use the first partial network for the detection of the object.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an object detection device that detects an object.

  Techniques for detecting an object represented in an image have been studied. In recent years, a technique for improving detection accuracy by using a so-called deep neural network (hereinafter simply referred to as DNN) to detect an object has been proposed (for example, see Non-Patent Documents 1 and 2). .

  In such a technique, by learning DNN using a large number of images representing a known object as teacher data, the known object represented on the input image of the DNN can be detected with high accuracy. Become like However, an object having a small number of images usable as teacher data may be a detection target. Therefore, by using the learned parameter set of the previously learned DNN as an initial value, the DNN is re-learned with respect to the teacher data representing a relatively small number of specific objects, so that the DNN of the specific object is A technique for improving detection accuracy has been proposed (for example, see Non-Patent Document 3). Such a technique is also called fine-tuning.

Wei Liu et al., "SSD: Single Shot MultiBox Detector", ECCV2016, 2016 Shaoqing Ren et al., `` Faster R-CNN: Towards Real-Time ObjectDetection with Region Proposal Networks '', NIPS, 2015 Filip Radenovic et al., `` CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples '', ECCV2016, 2016

  In fine-tuning, the entire specific layer of the plurality of layers of the DNN may be re-learned. However, when the entire specific layer is re-learned, the DNN over-fits the teacher data used in re-learning the DNN, and the object represented in the previously learned teacher data is On the other hand, the detection accuracy may be reduced.

  Therefore, the present invention provides an object detection device that can improve the detection accuracy of an object when the predetermined condition is satisfied, while suppressing a decrease in the detection accuracy of the object when the predetermined condition is not satisfied. With the goal.

  According to one embodiment, an object detection device is provided. This object detection device detects an object existing around the vehicle by inputting a sensor signal obtained from a sensor mounted on the vehicle to a neural network, and the neural network has an input layer to which the sensor signal is input, An output layer that outputs a detection result of the object, and a plurality of layers connected between the input layer and the output layer, at least one of the plurality of layers has the same structure, and Has a plurality of partial networks that operate on a signal input to the layer in parallel with each other, and a first partial network of the plurality of partial networks is configured to detect an object when a predetermined condition regarding a vehicle is satisfied. The sub-networks that have been learned in advance to detect the sub-networks other than the first sub-network among the plurality of sub-networks do not satisfy a predetermined condition. Also, a detection unit that is learned in advance to detect an object and a first partial network used to detect an object when a predetermined condition regarding a vehicle is satisfied. On the other hand, when a predetermined condition regarding a vehicle is not satisfied, And a network control unit for controlling the neural network so that the first partial network is not used for detection of.

  ADVANTAGE OF THE INVENTION The object detection apparatus which concerns on this invention has the effect that the detection accuracy of an object can be improved when the predetermined condition is satisfy | filled, suppressing the fall of the object detection accuracy when a predetermined condition is not satisfied. .

1 is a schematic configuration diagram of a vehicle control system in which an object detection device is mounted. FIG. 2 is a hardware configuration diagram of an electronic control device that is one embodiment of the object detection device. FIG. 4 is a functional block diagram of a processor of the electronic control device regarding a vehicle control process including an object detection process. It is a schematic structure figure of a variable layer which DNN used as a discriminator has. 6 is an operation flowchart of a discriminator control process executed by a discriminator control unit. 5 is an operation flowchart of a vehicle control process including an object detection process.

  Hereinafter, the object detection device will be described with reference to the drawings. This object detection device uses a DNN as a discriminator for detecting an object existing around a vehicle from an image obtained by a camera mounted on the vehicle. The DNN used by the object detection device has an input layer to which an image is input, an output layer to output a detection result of the object, and a plurality of hidden layers connected between the input layer and the output layer, At least one of the plurality of hidden layers has the same structure and includes a plurality of partial networks that operate on signals input to the hidden layer in parallel with each other. Then, a first partial network of the plurality of partial networks is previously learned to detect an object when a predetermined condition regarding the vehicle is satisfied, and a first partial network of the plurality of partial networks other than the first partial network is included. The partial network is learned in advance to detect an object even when a predetermined condition is not satisfied. When the predetermined condition is satisfied, the object detection device uses the first partial network together with another partial network to improve the detection accuracy of the object when the predetermined condition is satisfied, On the other hand, when the predetermined condition is not satisfied, the first partial network is not used, thereby suppressing a decrease in object detection accuracy.

  Hereinafter, an example in which the object detection device is applied to a vehicle control system will be described. In this example, the object detection device performs an object detection process on an image obtained by a camera mounted on the vehicle, so that various objects existing around the vehicle, for example, other vehicles, people, A road sign or a road sign is detected, and the vehicle is automatically driven based on the detection result. In this example, the predetermined condition is that the current position of the vehicle is included in an area where the number of times the vehicle itself travels is equal to or more than the predetermined number.

  FIG. 1 is a schematic configuration diagram of a vehicle control system in which the object detection device is mounted. FIG. 2 is a hardware configuration diagram of an electronic control device which is one embodiment of the object detection device. In the present embodiment, the vehicle control system 1 mounted on the vehicle 10 and controlling the vehicle 10 is an example of a camera 2 for photographing the periphery of the vehicle 10, a positioning information receiver 3, and an object detection device. A certain electronic control unit (ECU) 4. The camera 2, the positioning information receiver 3, and the ECU 4 are communicably connected via an in-vehicle network 5 conforming to a standard such as a controller area network.

  The camera 2 is an example of an imaging unit that is a sensor for detecting an object present within a predetermined detection range, and is configured by an array of photoelectric conversion elements having sensitivity to visible light, such as a CCD or a C-MOS. It has a two-dimensional detector and an imaging optical system that forms an image of a region to be photographed on the two-dimensional detector. The camera 2 is mounted, for example, in the cabin of the vehicle 10 so as to face the front of the vehicle 10. Then, the camera 2 captures an image of the front area of the vehicle 10 at a predetermined shooting cycle (for example, 1/30 to 1/10 seconds), and generates an image including the front area. The image obtained by the camera 2 may be a color image or a gray image. The image generated by the camera 2 is an example of a sensor signal.

  Each time the camera 2 generates an image, the camera 2 outputs the generated image to the ECU 4 via the in-vehicle network 5.

  The positioning information receiver 3 is an example of a positioning unit, and acquires positioning information indicating the current position of the vehicle 10 at predetermined intervals. For example, the positioning information receiver 3 can be a GPS receiver. Each time the positioning information receiver 3 acquires the positioning information, the positioning information receiver 3 outputs the acquired positioning information to the ECU 4 via the in-vehicle network 5.

  The ECU 4 controls the vehicle 10. In the present embodiment, the ECU 4 controls the vehicle 10 to automatically drive the vehicle 10 based on an object detected from a series of time-series images obtained by the camera 2. To this end, the ECU 4 includes a communication interface 21, a memory 22, and a processor 23.

  The communication interface 21 is an example of a communication unit, and has an interface circuit for connecting the ECU 4 to the in-vehicle network 5. That is, the communication interface 21 is connected to the camera 2 via the in-vehicle network 5. Then, each time the communication interface 21 receives an image from the camera 2, the communication interface 21 passes the received image to the processor 23.

  The memory 22 is an example of a storage unit, and includes, for example, a volatile semiconductor memory and a nonvolatile semiconductor memory. The memory 22 stores various data used in the object detection processing executed by the processor 23 of the ECU 4, for example, an image received from the camera 2, various parameters for specifying a classifier used in the object detection processing, Also, the number of times of travel for each area is stored. Further, the memory 22 stores map information. The map information includes information for specifying each range of a plurality of regions. In the present embodiment, the area serving as a unit for checking the number of times of travel is, for example, one that is divided by administrative divisions, such as the Hokkaido region, Tokyo or Osaka, or that is geographically divided, such as a plain or mountainous area. Or may be classified according to the type of road, such as a highway or a highway. Furthermore, the area serving as a unit for checking the number of times of travel may be classified according to a combination thereof.

  The processor 23 is an example of a control unit, and includes one or a plurality of CPUs (Central Processing Units) and peripheral circuits thereof. The processor 23 may further include another arithmetic circuit such as a logical operation unit, a numerical operation unit, or a graphic processing unit. Then, while the vehicle 10 is traveling, each time an image is received from the camera 2, the processor 23 executes a vehicle control process including an object detection process on the received image. Then, the processor 23 controls the vehicle 10 to automatically drive the vehicle 10 based on the detected object around the vehicle 10.

  FIG. 3 is a functional block diagram of the processor 23 of the ECU 4 regarding the vehicle control processing including the object detection processing. The processor 23 includes a traveling number update unit 31, a discriminator control unit 32, an object detection unit 33, a driving plan unit 34, and a vehicle control unit 35. These units included in the processor 23 are, for example, functional modules realized by a computer program operating on the processor 23. Alternatively, these units included in the processor 23 may be dedicated arithmetic circuits provided in the processor 23. In addition, among these units included in the processor 23, the traveling number update unit 31, the discriminator control unit 32, and the object detection unit 33 execute the object detection process.

  The traveling frequency updating unit 31 updates the traveling frequency of the vehicle 10 for each area represented by the map information based on the current position of the vehicle 10. For example, the traveling frequency update unit 31 includes the current position of the vehicle 10 represented by the latest positioning information received from the positioning information receiver 3 among a plurality of regions represented in the map information stored in the memory 22. Identify areas where Then, the traveling frequency updating unit 31 increments the traveling frequency of the area including the current position of the vehicle 10 by one.

  In addition, in order to prevent the number of times of travel in one area from increasing a plurality of times in one travel of the vehicle 10, the travel number update unit 31 determines a predetermined number of times for the area in which the number of travels is once incremented. Until the period (for example, 6 hours, 1 day, or 1 week) has elapsed, even if the vehicle 10 is included in the area, the number of times of travel may not be incremented.

  The discriminator control unit 32 is an example of a network control unit, and determines whether a predetermined condition regarding the vehicle 10 is satisfied. If the predetermined condition is satisfied, the classifier control unit 32 is used by the object detection unit 33 to detect an object. Of the plurality of sub-networks of the DNN used as a discriminator used for the object detection. The DNN used as the discriminator is implemented as software by a computer program for object detection processing that operates on the processor 23. However, the DNN used as the discriminator may be implemented as a dedicated arithmetic circuit included in the processor 23.

  In the present embodiment, the DNN used as the discriminator is, for example, a DNN having a convolutional neural network (CNN) type architecture, such as Single Shot MultiBox Detector, Faster R-CNN, VGG-19, or AlexNet. it can. Alternatively, the DNN used as a discriminator has a CNN architecture for segmentation for identifying, for each pixel of the image, an object that may be represented by the pixel, from an input image, such as a Fully Convolutional Network. It may be DNN.

  That is, the DNN used as a discriminator in the present embodiment includes an input layer to which an image is input, an output layer to output a result of object detection, and a plurality of hidden layers connected between the input layer and the output layer. And a layer. The plurality of hidden layers include a convolution layer and a pooling layer. Further, the plurality of hidden layers may include a fully connected layer.

  Furthermore, in the present embodiment, at least one of the input layer or the plurality of hidden layers of the DNN has the same structure, and signals input to the layer are parallel to each other. A plurality of partial networks for operation are provided. Note that a layer having a plurality of partial networks is hereinafter referred to as a variable layer for convenience of description. Further, that a plurality of sub-networks have the same structure means that data input to each of the plurality of sub-networks is the same, and the same type of operation (for example, convolution, Pooling).

FIG. 4 is a schematic configuration diagram of a variable layer among a plurality of layers of the DNN used as a discriminator. In this example, convolution is performed on the i-th (i is an integer of 1 or more) variable layer 400 in order from the input side of the DNN, in parallel with each other, on the feature map x _i obtained by the upstream layer of the variable layer 400. A plurality of convolution modules Tf1 (x) to Tfn (x) and Tc1 (x) to TcN (x) (where n and N are each an integer of 1 or more) for performing the operation are provided. These convolution modules are examples of a plurality of partial networks. Among them, the convolution modules Tc1 (x) to TcN (x) are always connected regardless of the determination result of the discriminator control unit 32. That is, the convolution modules Tc1 (x) to TcN (x) include teacher data including a large number of images when a predetermined condition regarding the vehicle 10 is not satisfied (in the present embodiment, a large number of images obtained for various areas are The learning is performed in advance according to a learning method such as an error back propagation method based on the teaching data (including the teacher data), and is used for object detection regardless of the area where the vehicle 10 is traveling.

  On the other hand, each of the convolution modules Tf1 (x) to Tfn (x) is an example of a first partial network, has the same structure as the convolution modules Tc1 (x) to TcN (x), and differs from each other. This corresponds to the condition relating to the vehicle 10 (in the present embodiment, different regions among a plurality of regions represented in the map information). Each of the convolution modules Tf1 (x) to Tfn (x) is connected between an upstream layer and a downstream layer of the variable layer 400 via switches S1 to Sn. Each of the convolution modules Tf1 (x) to Tfn (x) includes teacher data including a plurality of images obtained under corresponding predetermined conditions (in the present embodiment, includes a plurality of images obtained for a corresponding area. Based on the teacher data), learning is performed in advance according to a learning method such as an error back propagation method. For example, when the DNN is learned using the teacher data of the region of interest, only the switch corresponding to the region of interest among the switches S1 to Sn is turned on, and the convolution modules Tf1 (x) to Tfn (x) Only the convolution module corresponding to the area of interest is connected to the DNN. Then, the DNN is learned using the teacher data of the region of interest. Thereby, the convolution modules Tf1 (x) to Tfn (x) are specialized for the corresponding area. Therefore, by connecting the convolution module of the convolution modules Tf1 (x) to Tfn (x) corresponding to the area where the vehicle 10 travels to the DNN, the detection accuracy of the object when the vehicle 10 travels in that area Is improved.

In the variable layer 400, the sum of the output of each of the convolution modules Tc1 (x) to TcN (x) and the output of the convolution module whose corresponding switch is turned on among the convolution modules Tf1 (x) to Tfn (x) Is calculated. Then, for the sum, the ratio of the total number of convolution modules included in the variable layer 400 to the number of convolution modules that performed the convolution operation ((N + n) / (N + ΣSbk), where Sbk (k = 1,. .., n) is next 1 when the switch Sk is on, the switch Sk is bool variable becomes zero when off) is the sum of the obtained values and xi by is multiplied, feature map x _{i +} Output as _i . Then, feature map x _{i + i} is input to the next layer.

  It should be noted that the DNN used as the discriminator may have two or more variable layers as shown in FIG. In this case, two or more variable layers may be provided so as to be continuous with each other, or another layer such as a pooling layer may be provided between the two or more variable layers. Further, two or more convolution modules may correspond to one predetermined condition. In this case, when a predetermined condition is satisfied, the discriminator control unit 32 may turn on a switch connected to each of the two or more convolution modules corresponding to the predetermined condition. Further, when the DNN used as the discriminator has one or more total coupling layers, one of the total coupling layers may be formed as a variable layer. In this case, each of the plurality of partial networks may be configured to calculate one output value using all output values of the plurality of neurons of the upstream layer as inputs. Further, individual sub-networks may include multiple layers. In this case, each partial network may include, for example, two or more of a convolutional layer, a pooling layer, and a fully connected layer, or any one of the convolutional layer, the pooling layer, and the fully connected layer. More than one layer may be included.

  When a predetermined condition regarding the vehicle 10 is satisfied among the switches S1 to Sn, the discriminator control unit 32 turns on a switch corresponding to the predetermined condition and turns off other switches. As described above, in the present embodiment, the predetermined condition is that the current position of the vehicle 10 is included in an area where the number of times of travel of the vehicle 10 itself is equal to or more than the predetermined number. Therefore, the discriminator control unit 32 turns on a switch corresponding to the area including the current position of the vehicle, when the number of times of travel of the vehicle 10 itself is equal to or more than the predetermined number in the area including the current position of the vehicle. For example, when the convolution module Tf1 (x) corresponds to the Hokkaido region and the convolution module Tf2 (x) corresponds to the Kyushu region, the current position of the vehicle 10 is located in the Hokkaido region, and the vehicle 10 itself is in the Hokkaido region. When the number of times of traveling is equal to or more than the predetermined number, the discriminator control unit 32 turns on the switch S1 corresponding to the convolution module Tf1 (x) and turns off the switch S2 corresponding to the convolution module Tf2 (x). With this, among the convolution modules Tf1 (x) to Tfn (x), for the area where the vehicle 10 is currently traveling, when the number of times the vehicle 10 travels is equal to or more than the predetermined number threshold, the convolution module corresponding to the area Will be used for object detection.

  FIG. 5 is an operation flowchart of a discriminator control process executed by the discriminator control unit 32. The discriminator control unit 32 determines whether the vehicle 10 travels a predetermined period (for example, 1 minute, 10 minutes, 1 hour, etc.) or at a predetermined timing (for example, every time the vehicle 10 travels a predetermined distance (for example, 10 km to 50 km), or Each time the vehicle 10 moves to a different area), the discriminator control process is executed according to the operation flowchart shown in FIG.

  The discriminator control unit 32 of the processor 23 compares the current position of the vehicle 10 represented by the latest positioning information received by the processor 23 from the positioning information receiver 3 with the map information stored in the memory 22. From among the plurality of areas represented by the map information, the area including the current position of the vehicle 10 (hereinafter, referred to as the current area) is specified (step S101). The discriminator control unit 32 reads the number of travels of the vehicle 10 in the specified current area from the memory 22, and determines whether the number of travels is equal to or greater than a predetermined number threshold Th (Step S102). The predetermined number threshold Th is set to, for example, 10 to 100.

When the number of times the vehicle 10 travels in the current area is equal to or greater than the number threshold Th (Step S102-Yes), the discriminator control unit 32 determines the plurality of DNNs of the discriminator used for object detection by the object detection unit 33. The classifier is controlled to use the partial network corresponding to the current area among the partial networks (step S103). On the other hand, when the number of travels of the vehicle 10 in the current area is less than the number-of-times threshold Th (Step S102-No), the discriminator control unit 32 controls the discriminator so as not to use the partial network corresponding to the current area (Step S102). S104).
After step S103 or S104, the discriminator control unit 32 ends the discriminator control processing.

The object detection unit 33 is an example of a detection unit. Each time an image is obtained from the camera 2, the image is input to a DNN used as a discriminator, and the surroundings of the vehicle 10 represented in the image are displayed. The object to be detected existing in the object is detected. At this time, of the plurality of partial networks of the DNN, the partial network controlled to be used by the discriminator control unit 32 is used for object detection. The object to be detected includes, for example, a moving object such as a car or a person. In addition, the detection target object may further include a road sign or a road sign such as a lane marking and a stationary object such as a traffic light.
The object detection unit 33 outputs information indicating the type of the detected object and the position on the image to the driving planning unit 34.

The driving plan unit 34 generates one or more scheduled routes of the vehicle 10 based on the objects detected from the images so that the objects existing around the vehicle 10 do not collide with the vehicle 10. The scheduled travel route is represented, for example, as a set of target positions of the vehicle 10 at each time from the current time to a predetermined time ahead. For example, each time the driving plan unit 34 receives an image from the camera 2, the driving planning unit 34 performs a viewpoint conversion process using information such as a mounting position of the camera 2 on the vehicle 10, thereby converting the received image into a bird's-eye view image. Convert. Then, the driving planning unit 34 performs tracking processing using a Kalman Filter or the like on a series of bird's-eye images to track the detected object for each image, and from the trajectory obtained from the tracking result. , A predicted trajectory of each of the objects up to a predetermined time ahead. The driving plan unit 34 sets the predicted value of the distance between each of the tracked objects and the vehicle 10 up to a predetermined time ahead based on the predicted trajectory of each of the tracked objects to be equal to or more than a predetermined distance. Next, a scheduled travel route of the vehicle 10 is generated. At that time, the driving planning unit 34 refers to, for example, the current position of the vehicle 10 represented by the positioning information obtained from the positioning information receiver 3 and the map information stored in the memory 22, and You may check the number of lanes that can travel. Then, when there are a plurality of lanes on which the vehicle 10 can travel, the driving plan unit 34 may generate the planned traveling route so as to change the lane on which the vehicle 10 travels. At that time, the driving plan unit 34 may determine the positional relationship between the vehicle 10 and the lane in which the vehicle 10 is traveling or the lane to be changed with reference to the position of the lane marking detected from the image. . Furthermore, when the traffic light detected from the image indicates a temporary stop, the driving plan unit 34 may set the planned traveling route so that the vehicle 10 stops at the stop line corresponding to the traffic light. .
Note that the driving planning unit 34 may generate a plurality of planned traveling routes. In this case, the driving plan unit 34 may select a route that minimizes the sum of the absolute values of the acceleration of the vehicle 10 from the plurality of planned traveling routes.

  The operation planning unit 34 notifies the vehicle control unit 35 of the generated planned traveling route.

  The vehicle control unit 35 controls each unit of the vehicle 10 so that the vehicle 10 travels along the notified scheduled travel route. For example, the vehicle control unit 35 obtains the acceleration of the vehicle 10 according to the notified travel schedule route and the current vehicle speed of the vehicle 10 measured by the vehicle speed sensor (not shown), and sets the accelerator so that the acceleration is obtained. Set the opening or brake amount. Then, vehicle control unit 35 obtains a fuel injection amount according to the set accelerator opening, and outputs a control signal corresponding to the fuel injection amount to the fuel injection device of the engine of vehicle 10. Alternatively, the vehicle control unit 35 outputs a control signal corresponding to the set brake amount to the brake of the vehicle 10.

  Further, when changing the course of the vehicle 10 so that the vehicle 10 travels along the planned traveling route, the vehicle control unit 35 determines the steering angle of the vehicle 10 according to the planned traveling route, and determines the steering angle according to the steering angle. The control signal is output to an actuator (not shown) that controls the steered wheels of the vehicle 10.

  FIG. 6 is an operation flowchart of a vehicle control process including an object detection process executed by the processor 23. Each time an image is received from the camera 2, the processor 23 executes a vehicle control process according to the operation flowchart shown in FIG. In the operation flowchart described below, the processing of steps S201 to S202 corresponds to the object detection processing.

  When a predetermined condition relating to the vehicle 10 is satisfied, the discriminator control unit 32 of the processor 23 outputs a partial network corresponding to the predetermined condition among a plurality of partial networks included in the variable layer of the DNN used as the discriminator. Is turned on (step S201). That is, as shown in FIG. 4, the discriminator control unit 32 turns on a switch corresponding to a partial network corresponding to a predetermined condition so that the partial network is used for object detection. Then, the object detection unit 33 of the processor 23 inputs the image obtained from the camera 2 to the discriminator, and detects an object around the vehicle 10 represented in the image (Step S202).

  The driving plan unit 34 of the processor 23 tracks the detected object, and generates a scheduled travel route of the vehicle 10 so as to be at least a predetermined distance from a predicted trajectory of the object estimated based on the tracking result. (Step S203). Then, the vehicle control unit 35 of the processor 23 controls the vehicle 10 so that the vehicle 10 travels along the planned traveling route (Step S204). Then, the processor 23 ends the vehicle control processing.

  As described above, this object detection device is configured such that when a predetermined condition regarding a vehicle is satisfied, the object detection device is configured to satisfy the predetermined condition among a plurality of partial networks included in a variable layer of a neural network used as a discriminator. The classifier is controlled to use the sub-network that has been learned in advance to correspond. Thus, the object detection device can improve the detection accuracy of the object when the predetermined condition is satisfied. On the other hand, when the predetermined condition is not satisfied, the object detection device controls the discriminator so as not to use the partial network previously learned so as to correspond to the predetermined condition. Thus, the object detection device can suppress a decrease in object detection accuracy when the predetermined condition is not satisfied.

  According to the modification, the discriminator control unit 32 may control the discriminator based on the traveling frequency instead of the number of times the vehicle 10 has traveled. In this case, the traveling frequency updating unit 31 associates the area where the vehicle 10 is traveling with the identification information for identifying the area, and associates the date and time when the vehicle 10 traveled (hereinafter, simply referred to as traveling date and time). Is stored in the memory 22. The discriminator control unit 32 stores, for a region in which the vehicle 10 is traveling, a traveling stored in association with identification information of the region included in a latest predetermined period (for example, one week, one month, or the like). The number of dates and times may be obtained as the traveling frequency. Then, when the traveling frequency of the area where the vehicle 10 is currently traveling is equal to or more than the predetermined number threshold, the discriminator control unit 32 causes the discriminator to use the partial network corresponding to the area for object detection. What is necessary is to control.

  According to another modification, an area serving as a unit for calculating the number of times of travel or the frequency of travel may be set without using map information. In this case, the number of travels or the travel date and time is stored for each section of the real space having a predetermined size (for example, 10 km square to 50 km square). For example, the traveling count updating unit 31 sets the traveling count to 1 for each of a plurality of sections included in an area of a predetermined size (larger than the section, for example, 100 km square to 500 km square) around the current position of the vehicle 10. Increment or the date and time of travel may be stored. Further, the plurality of partial networks of the variable layer of the classifier are respectively associated with different representative points, and are learned in advance using teacher data obtained from an area centered on the corresponding representative point. . Then, the discriminator control unit 32 determines that the sum of the number of times of travel of each section included in the area of the predetermined size centered on the current position of the vehicle 10 or the number of travel dates and times within the latest predetermined period is equal to or greater than the predetermined number of times threshold In the case of, the classifier is controlled to use the partial network associated with the representative point closest to the current position of the vehicle 10 among the representative points.

  According to another modification, the discriminator control unit 32 may control the discriminator based on the current position of the vehicle 10 instead of the number of times of travel or the frequency of travel. In this case, the discriminator control unit 32 refers to, for example, the map information, specifies the area including the current position of the vehicle 10 represented by the positioning information received from the positioning information receiver 3, and specifies the area. What is necessary is just to control the classifier so that the partial network corresponding to the specified area is used for object detection. In this case, the running frequency updating unit 31 may be omitted.

  According to still another modification, the discriminator control unit 32 may control the discriminator used for detecting an object based on a condition other than the position of the vehicle 10. For example, the discriminator control unit 32 may determine that the predetermined condition is satisfied when the current time is included in the night time zone. In this case, one of the plurality of partial networks included in the variable layer of the discriminator used for detecting the object is previously learned using an image obtained at night as teacher data. Then, the discriminator control unit 32 transmits a clock (not shown) mounted on the vehicle 10 or a wireless communication module (not shown) mounted on the vehicle 10 that receives a wireless signal for notifying time to the processor 23. It is determined whether or not the received current time is included in the night time zone. Then, when the current time is included in the night time zone, the discriminator control unit 32 may control the discriminator to use the partial network previously learned for the night time zone for object detection. Accordingly, the object detection device can improve the detection accuracy of the object at night while preventing a reduction in the detection accuracy of the object during the day. Note that, also in this case, the traveling count updating unit 31 may be omitted.

  When the result of the object detection by the object detection unit 33 is used for driving assistance, each of the plurality of partial networks in the variable layer of the discriminator may be associated with a different driver. In this case, each partial network is learned in advance using the image obtained by the camera 2 when the corresponding driver drives the vehicle 10 as teacher data. Then, the discriminator control unit 32 drives the vehicle 10 based on an in-vehicle image obtained by an in-vehicle camera (not shown) mounted in the cabin of the vehicle 10 so as to photograph a driver's face of the vehicle 10. Identify the driver you are using. The discriminator control unit 32 identifies the driver according to, for example, template matching using a template for each driver stored in the memory 22 in advance, or another face recognition method for recognizing a face represented in an image. do it. Then, the classifier control unit 32 may control the classifier to use the partial network corresponding to the identified driver for detecting an object. Accordingly, even if a driver causes a difference in the way an object is displayed on an image due to, for example, a difference in a route or time zone in which the vehicle 10 travels depending on the driver, the object detection device can detect the object. Detection accuracy can be improved.

A parameter set (such as a weight coefficient) for defining a partial network associated with a predetermined condition included in the variable layer of the discriminator can communicate with the vehicle 10 via a wireless base station and a communication network. It may be stored in the server. In this case, if the discriminator control unit 32 determines that the predetermined condition is satisfied, the discriminator control unit 32 transmits the identification information of the vehicle 10 to the server via a wireless communication module (not shown) mounted on the vehicle 10. And a request signal including identification information indicating the predetermined condition and a signal indicating requesting a parameter set of the partial network. Upon receiving the request signal via the wireless base station and the communication network, the server returns a response signal including a parameter set corresponding to a predetermined condition to the vehicle 10 via the communication network and the wireless base station. The discriminator control unit 32 extracts a parameter set from the response signal received via the wireless communication module and applies the parameter set to a partial network, whereby the partial network corresponds to a predetermined condition.
According to this modification, the object detection device can reduce the number of partial networks included in the classifier, so that the memory capacity required for the memory 22 can be reduced. Further, it becomes easy to update the parameter set of the partial network corresponding to the predetermined condition.

  The object detection device according to the above embodiment or the modification may be applied to a sensor signal acquired by a sensor for detecting an object existing around the vehicle 10 other than the camera 2. As such a sensor for detecting an object existing within a predetermined detection range, for example, a LIDAR sensor or a laser sensor mounted on the vehicle 10 may be used. In this case, the DNN used by the object detection unit 33 as a discriminator may be learned in advance so as to detect an object from a sensor signal acquired by a sensor mounted on the vehicle 10. Also in this case, the discriminator has one or more variable layers including a plurality of partial networks as in the above-described embodiment or the modified example, and the plurality of partial networks use teacher data under corresponding conditions. What is necessary is just to learn beforehand.

  In addition, the computer program for realizing the function of each unit of the processor 23 of the object detection device according to the above embodiment or the modification is stored in a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium, or an optical recording medium. It may be provided in recorded form.

  As described above, those skilled in the art can make various changes according to the embodiments within the scope of the present invention.

Reference Signs List 1 vehicle control system 2 camera 3 positioning information receiver 4 electronic control device (object detection device)
5 In-vehicle network 21 Communication interface 22 Memory 23 Processor 31 Travel count update unit 32 Discriminator control unit 33 Object detection unit 34 Driving planning unit 35 Vehicle control unit

Claims (1)

An object existing around the vehicle is detected by inputting a sensor signal obtained from a sensor mounted on a vehicle to a neural network, and the neural network has an input layer to which the sensor signal is input, and an input layer of the object. An output layer that outputs a detection result, including a plurality of layers connected between the input layer and the output layer, at least one of the plurality of layers has the same structure, And a plurality of partial networks that operate on signals input to the layer in parallel with each other, wherein a first partial network of the plurality of partial networks satisfies a predetermined condition regarding the vehicle. In the learning, the partial networks other than the first partial network among the plurality of partial networks are learned in advance to detect the object. A detection unit in advance are learned to detect the object even if the predetermined condition is not satisfied,
When the predetermined condition regarding the vehicle is satisfied, the first partial network is used for detecting the object. On the other hand, when the predetermined condition regarding the vehicle is not satisfied, the first partial network is used for detecting the object. A network control unit that controls the neural network so as not to use a network;
An object detection device having:

Images