JP2021157699A - System, program, machine learning method, machine learning model, and others - Google Patents

Abstract

To provide a technique which allows for detecting a target object without relying solely on electromagnetic waves within a given wavelength band, such as those of lasers, radars, etc.SOLUTION: A system 100 provided herein comprises a camera 21 mounted in a vehicle and configured to generate a captured image of the outside of the vehicle, detection units 22, 33 configured to detect speed violation control devices from the captured image using a machine learning model, and control units 24, 32 configured to provide control based on detection results of the detection units 22, 33.SELECTED DRAWING: Figure 1

Description

The present invention relates to systems, programs, machine learning methods, machine learning models, and the like.

Speed control devices for cracking down on vehicle overspeed violations may be installed at speed control points on the road on which the vehicle travels. The speed control device irradiates a moving vehicle with a laser or radar in a predetermined wavelength band (hereinafter, these may be collectively referred to as "electromagnetic waves"), and receives or receives the reflected radio waves or reflected light. By doing so, the speed of the vehicle is estimated. Then, when the vehicle is traveling at a speed exceeding the legal speed set on the road, the speed control device detects the vehicle as a vehicle in violation of overspeed and takes a picture with a camera.

In general, speed cameras may be installed in places where the driver tends to unknowingly increase the speed of the vehicle, such as highways and general roads with good visibility. Therefore, if the speed control device can be detected in the vehicle and the driver can be alerted, it is possible to support the safe driving of the driver.

Conventionally, a detector mounted on a vehicle and detecting a laser or radar emitted from a speed control device to notify the existence of the speed path control device has been provided. According to this detector, it is possible to notify the driver of the existence of the speed control device and promote safe driving.

JP-A-2018-196066

However, simply detecting an electromagnetic wave may receive an electromagnetic wave emitted from a vending machine or an automatic door and mistakenly recognize it as a speed control device.

In view of the above background, one of an object of the present invention is to provide a technique for detecting a target indicating a speed control point without relying only on electromagnetic waves in a predetermined wavelength band such as a laser or radar. ..

Another object of the present invention is to collect captured images of a predetermined target image captured from a vehicle on a server.

Furthermore, one of the objects of the present invention is to provide a technique for generating a learning model for detecting a target indicating that it is a speed control point.

Further, the object of the invention of the present application is not limited to these, and the intention to acquire the right by divisional application, amendment, etc. for the configuration for the purpose of obtaining the effect produced from the portion of the configuration disclosed in the present specification, drawings, etc. Have. For example, the present specification discloses a problem in which the part described as "can be" is replaced with "is a problem" in the present specification. The issues are described as independent, and the intention is to acquire the rights to solve each issue independently by divisional application, amendment, etc. Even if the subject matter is implicitly grasped from the description in the specification, the applicant intends to make a part of the structure described in the specification amended or to be claimed in the divisional application. It also discloses a structure that solves problems that combine these independent problems, and has the intention to acquire rights.

(1) The system of one aspect is based on a detection unit that detects a target indicating that it is a speed control point of the vehicle by using a machine learning model from an image taken outside the vehicle, and a detection result in the detection unit. It has a configuration including a control unit that performs control based on the control. In this way, it is possible to detect a target indicating that it is a speed control point from the captured image. For example, it is possible to reduce the possibility of misidentifying an electromagnetic wave source other than a speed control device such as a vending machine or an automatic door as a speed control device.

(2) In the system of one aspect, in the system of (1), the detection unit detects the target object by the primary detection unit that detects the target object by using the first machine learning model, and the primary detection unit. When this is done, it has a secondary detection unit that detects the target object using a second machine learning model, and the control unit is based on the detection results of the primary detection unit and the secondary detection unit. It is good to control. In this way, since the target object is detected by a two-step method using the first machine learning model and the second machine learning model, the target object can be detected more accurately.

(3) In the system of (2), the primary detection unit is provided in an in-vehicle device mounted on a vehicle, and the secondary detection unit communicates with the in-vehicle device via a communication network. It is recommended that it be installed on the server. In this way, the in-vehicle device does not need to include the second machine learning model, and for example, the second machine learning model can be shared by a plurality of in-vehicle devices. Further, if the captured image in which the target is detected by the primary detection of the in-vehicle device is transmitted to the server, the processing load and the communication load of the server can be reduced as compared with the case where all the captured images are transmitted.

(4) The system of one aspect is the system of (2) or (3), in which the primary detection unit performs pattern matching with binary data as the first machine learning model, thereby performing the target from the image. It is preferable that the secondary detection unit detects an object and detects the target object from the image by using a deep learning model as the second machine learning model. In this way, it can be expected that the primary detection will be performed at a higher speed than the secondary detection.

(5) In the system of (2) to (4), the system of one aspect further includes a detector for detecting electromagnetic waves in a predetermined wavelength band in the vehicle, and the primary detection unit is the first machine learning model. It is preferable to detect the target based on the detection result of the detector and the detection result of the detector. In this way, the primary detection can be performed in combination with the detection method of the conventional speed control device.

(6) In the system of (5), the control unit is the target object from the image based on the first machine learning model during the period when the electromagnetic wave is received by the detector. When is detected, the above control may be performed. In this way, when an electromagnetic wave in a predetermined wavelength band used by the speed control device is received, control is performed when the speed control device is detected. Therefore, when the speed control device really exists, the control is performed. The possibility of being able to do it can be increased.

(7) In one aspect of the system, in any of the systems (1) to (6), the front control unit issues an alarm from an alarm unit provided on the vehicle based on the detection result in the primary detection unit. It is preferable to have a first control unit that controls output. In this way, it is possible to give an alarm to a person (for example, a driver) in the vehicle based on the detection result of the primary detection.

(8) In one aspect of the system, in any of the systems (2) to (6), the control unit outputs an alarm from an alarm unit provided on the vehicle based on the detection result in the secondary detection unit. It is preferable to have a first control unit that controls the operation. In this way, it is possible to give an alarm to a person (for example, a driver) in the vehicle based on the detection result of the secondary detection.

(9) In the system of one aspect, in the systems of (2) to (6), the control unit outputs a first alarm from an alarm unit provided in the vehicle based on the detection result of the primary detection unit. It is preferable to have a first control unit that controls and outputs a second alarm different from the first alarm based on the detection result in the secondary detection unit. In this way, a person (for example, a driver) in the vehicle can recognize the alarm by distinguishing between the result of the primary detection and the result of the secondary detection.

(10) In one aspect of the system, in any of the systems (7) to (9), the first control unit controls to output different alarms depending on the certainty of the detection result of the target object. It is good. In this way, different alarms are issued depending on the certainty of the detection result of the speed control device, so that a person (for example, a driver) in the vehicle can grasp the possibility that the speed control device actually exists.

(11) The system of one aspect includes a storage unit for storing the position information of the target object in the systems (1) to (10), and the control unit is a plurality of the detection units mounted on different vehicles. It is preferable to have a second control unit that controls updating the position information of the target object in the storage unit based on the detection result in the unit. In this way, the position information of the position where the target object is detected can be stored and updated according to the detection result of the target object.

(12) In the system of (7) to (10), the system of one aspect includes a storage unit for storing the position information of the target object, and the control unit is a plurality of the detection units mounted on different vehicles. It has a second control unit that controls updating the position information of the target object in the storage unit based on the detection result in the unit, and the first control unit has the target object stored in the storage unit. It is preferable to control to output the alarm based on the position information and the position of the vehicle. In this way, the position information of the position where the target is detected is stored, updated according to the detection result of the target, and the person in the vehicle (for example, the driver) according to the position of the vehicle with respect to the target. ) Can be alerted.

(13) In the system of (11) or (12), the first control unit is provided in an in-vehicle device mounted on a vehicle, and the second control unit communicates with the in-vehicle device. It may be installed in a server that communicates via a network. In this way, since the server acquires the result of detecting the target object by the in-vehicle device, the detection result of the target object can be shared by a plurality of in-vehicle devices.

(14) One aspect of the system is a system including an in-vehicle device mounted on a vehicle and a server provided on a communication network, and the in-vehicle device photographs the outside of the vehicle and captures an image. A server including a generated camera and a vehicle-side transmission unit that transmits the captured image to the server via the communication network, the server receiving the captured image from the in-vehicle device via the communication network. It is preferable to include a side receiving unit, a detection unit that detects a target object from the captured image using a machine learning model, and a storage unit that stores the detection result in the detection unit. In this way, the server can acquire the captured image from the in-vehicle device, detect the target object, and store the detection result. In particular, the server can obtain detection results of a large number of targets by acquiring captured images from a plurality of vehicles.

(15) In the system of (14), the in-vehicle device has a primary detection unit that detects the target object from the captured image by using a machine learning model having a lower detection standard than the machine learning model. The vehicle-side transmission unit transmits the captured image in which the target object is detected by the primary detection unit to the server, and the detection unit serves as a secondary detection unit at the server-side reception unit. It is preferable to detect the target object from the received captured image. By doing so, since the captured image in which the target object is detected by the primary detection of the in-vehicle device is transmitted to the server, the communication load and the secondary detection load can be reduced.

(16) In the system of (14), the in-vehicle device includes a position information acquisition unit for acquiring the position information of the vehicle, and the vehicle-side transmission unit includes the photographed image together with the photographed image. The position information of the vehicle when the image is taken is transmitted to the server, and the storage unit stores the detection result in association with the position information. In this way, the position information of the position where the target is detected can be stored in the server.

(17) In the system of (16), the in-vehicle device has a vehicle-side receiving unit that receives a notification based on the information stored in the storage unit and an output unit that outputs information based on the notification. And, it is good to have. In this way, the vehicle can be notified of the presence of the target.

(18) One aspect of the system is a system including an in-vehicle device, which includes a detection unit that detects a target object by using a first machine learning model from an image taken by the in-vehicle device outside the vehicle. When the target object is detected by the detection unit, the transmission unit that transmits the image to the server and the server receives the result of the process of detecting the target object from the image using the second machine learning model. It is a system including a receiving unit that performs control, and a control unit that controls based on the detection result in the detection unit and the result of the received processing. In this way, even if the in-vehicle device does not have the second machine learning model, the detection result of the target object using the second machine learning model can be obtained.

(19) The system of one aspect acquires each image of a plurality of images of a target object indicating that it is a speed control point of the vehicle, and the acquired object is included in each of the acquired images. Learning is performed using the associated teacher data. In this way, it is possible to generate a learning model for detecting a target indicating that the vehicle is a speed control point from the image.

(20) One aspect of the system is an image taken of the outside of the vehicle by a camera provided in the vehicle, and the electromagnetic wave is detected by a detector provided in the vehicle to detect an electromagnetic wave in a predetermined wavelength band. Supervised learning is performed using the image taken at the time as a regular example. In this way, since the captured image when the electromagnetic wave is received is used as a correct example, it is possible to accurately obtain a correct example used for supervised learning for detecting the target object.

(21) In the system of (20), the system of one aspect further includes a detection unit that detects a target indicating that it is a speed control point of the vehicle from the image, detects the target, and said. Supervised learning may be performed using the image when the detector detects an electromagnetic wave in a predetermined wavelength band as a positive example. In this way, the image taken when the target is not detected but the electromagnetic wave in the predetermined wavelength band is detected by the detector, that is, the detector is determined by another electromagnetic wave source such as a vending machine. It is possible to reduce the possibility of learning the captured image as a positive example when the electromagnetic wave in the wavelength band is detected.

(22) In the system of (20) or (21), the system of one aspect includes a detection unit that detects the target from the image, and the detector does not detect electromagnetic waves in the predetermined wavelength band. However, it is advisable to perform supervised learning using the image when the target is detected as a negative example. In this way, if the electromagnetic wave of a predetermined wavelength is not detected but the speed control device is detected, it is considered that an object that is confusing with the target is reflected in the captured image. For example, a good negative example can be obtained.

(23) In the system of (19) to (22), the system of one aspect accepts and posts the image taken when the person in the vehicle visually confirms the target. It is advisable to perform the supervised learning using the image as a positive example. By doing so, it is possible to generate a higher quality machine learning model by performing supervised learning using a photographed image in which the user visually confirms that the target object is included.

(24) In the system of (23), in addition to the image taken when a person in the vehicle visually confirms the target, the system of one aspect includes the predetermined wavelength band in the detector. It is advisable to accept the posting of the detection result of the electromagnetic wave of the above, and perform the supervised learning as a positive example or a negative example of the posted image according to the detection result for which the posting is accepted. In this way, a better machine learning model can be generated by using the detection result of the electromagnetic wave of a predetermined wavelength body in addition to the photographed image in which the user visually confirms that the target object is included. Can be done.

(25) In the system of (19) to (24), the system of one aspect accepts an input of a response from a person in the vehicle when the target is detected from the image and gives an alarm, and accepts the input. Depending on the result of the response, the supervised learning may be performed using the image as a positive example or a negative example. In this way, when an alarm is given, a person in the vehicle inputs a response as to whether this is a correct alarm or a false alarm, and the result of this response is reflected in the machine learning model, so that a better machine learning model is generated. be able to.

(26) In the system of one aspect (1) to (25), the target may be a vehicle control device. In this way, it is possible to provide a technique for detecting a vehicle control device as a target.

(27) The program of one aspect is a program for realizing at least a part of the functions of the system according to any one of (1) to (26) in an in-vehicle device and realizing the functions realized in the in-vehicle device in a computer. Is. In this way, it is possible to provide a program for detecting a target object from an image.

(28) In one aspect of the machine learning method, each image of a plurality of images obtained by photographing a target object indicating that it is a speed control point of a vehicle is acquired, and the acquired object is included in each of the acquired images. Learning is performed using the teacher data associated with the above. In this way, it is possible to generate a learning model for detecting a target indicating that the vehicle is a speed control point from the image.

(29) One aspect of the machine learning model is an input layer into which an image is input and an output layer that outputs data according to the certainty that the image includes a target indicating that it is a speed control point of a vehicle. And at least one intermediate layer in which parameters are learned using teacher data that inputs each image of a plurality of images and outputs data according to the probability that the target is included in the image. Have. In this way, it is possible to provide a machine learning model for detecting a target indicating that the vehicle is a speed control point from the image.

The inventions shown in (1) to (29) above can be arbitrarily combined. For example, it is preferable to add at least a part of the structure of at least one invention after (2) to all or part of the structure of the invention shown in (1). In particular, the invention shown in (1) may be an invention in which at least a part of at least one of the inventions after (2) is added. Further, an arbitrary configuration may be extracted from the inventions shown in (1) to (29), and the extracted configurations may be combined. The applicant of the present application intends to acquire rights to an invention including these configurations. Further, even if there is a description of "in the case of" or "in the case of", it is not described as a configuration limited to that case or at that time. These show examples of better configurations and are willing to acquire rights in these cases and in occasional configurations. In addition, the places where the description is accompanied by an order are not limited to this order. It also discloses a configuration in which some parts have been deleted or the order has been changed, and has the intention of acquiring rights.

According to one aspect, it is possible to provide a technique for detecting a speed control device without relying only on electromagnetic waves in a predetermined wavelength band. Further, according to another aspect of the present invention, the captured images of a predetermined target image captured from the vehicle can be aggregated in the server. Furthermore, according to still another aspect of the present invention, it is possible to provide a technique for generating a learning model for detecting a target indicating that it is a speed control point.

Further, the effect of the invention of the present application is not limited to this, and the effect produced from the portion of the configuration disclosed in the present specification, drawings, etc. is also disclosed, and the configuration exhibiting the effect is also disclosed by divisional application, amendment, etc. Have the intention to acquire the right. For example, in the present specification, the part described as "can be" is a description that clearly indicates the effect to be exerted, and there is a part that shows the effect even if there is no description of "can be". Moreover, even if there is no such description, there is an effect that can be grasped by the configuration.

It is a figure which shows the structure of the detection system of embodiment of this invention. It is a figure which shows the example of the object detection processing of embodiment of this invention. It is a figure which shows the example of the object detection processing of embodiment of this invention. It is a figure which shows the example of the object detection processing of embodiment of this invention. It is a figure which shows the example of the object detection processing of embodiment of this invention. It is a flowchart of the detection method of embodiment of this invention. It is a flowchart of the machine learning method of embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. It should be noted that the embodiments described below show an example of the case where the present invention is carried out, and the present invention is not limited to the specific configuration described below. In carrying out the present invention, a specific configuration according to the embodiment may be appropriately adopted.

FIG. 1 is a diagram showing a configuration of a system 100 according to an embodiment of the present invention. The system 100 includes a detector 10, a drive recorder 20 as an in-vehicle device (also referred to as a vehicle-side device), and a server 30. The detector 10 and the drive recorder 20 are mounted on the vehicle. Vehicles include, for example, private cars and commercial cars, and the types of vehicles are special cars such as passenger cars, buses, trucks, and forklifts. The server 30 is installed at a place different from the vehicle. The drive recorder 20 and the server 30 can send and receive information to and from each other by communication via a communication network. The communication network is, for example, LTE, 4G, 5G or other type of public communication line. The drive recorder 20 can be directly or indirectly connected to a communication network such as a mobile phone network by wireless communication. In the example of FIG. 1, the detector 10 and the drive recorder 20 mounted on one vehicle are shown with respect to the server 30, but the drive recorders 20 of many vehicles are connected to one server 30. .. In this case, a plurality of drive recorders 20 are mounted on different vehicles.

The detector 10 detects electromagnetic waves in a predetermined wavelength band transmitted from the speed control device. The speed control device is a device installed at a speed control point of a vehicle. The vehicle speed control point is the point where the vehicle speed and control are performed. At the speed control point, the speed control device may be fixedly installed in advance and the point may be fixed, or a relatively lightweight and small speed control device may be installed and the point may not be fixed. In some cases. The speed control device corresponds to a target indicating that it is a speed control point of the vehicle. The electromagnetic wave is, for example, a radio wave in a predetermined wavelength band exemplified by microwaves (radio wave of a radar method) or light in a predetermined wavelength band exemplified by infrared light (for example, 905 nm, 850 nm, 950 nm, 1900 nm). Laser-type light, for example, pulsed light). There are two types of speed control devices, one is a radar type and the other is a laser type. The detector 10 of the present embodiment corresponds to both of them. For this purpose, the detector 10 includes a radar detector 11 and a laser detector 12. The radar detection unit 11 includes a reception unit that receives radio waves in the radar wavelength band emitted from a radar-type speed control device. For example, when the intensity of the radio wave in the radar wavelength band exceeds the threshold value, the radar detection unit 11 may determine that the radio wave has been detected. The laser detection unit 12 includes a light receiving unit that receives light in the laser wavelength band emitted from a laser-type speed control device. For example, when the intensity of light in the laser wavelength band exceeds the threshold value, the laser detection unit 12 may determine that the light has been detected. Further, the detector 10 may have an alarm unit 13. The alarm unit 13 issues an alarm when a predetermined condition is satisfied. The alarm unit 13 may issue an alarm in response to a signal from the drive recorder 20, for example. The alarm may be given (for example, by using a method that can be perceived by a person such as sound, display, or light emission. In addition to these, the detector 10 may include an operation unit or the like that receives an operation input. ..

The detector 10 is installed at a position where electromagnetic waves from the speed control device can be received, for example, on the dashboard of the vehicle or near the upper part of the windshield. When the radar detection unit 11 detects the radar, the detector 10 issues an alarm indicating that from the alarm unit 13. Further, when the detector 10 detects the radar, it outputs a signal to that effect to the control unit 24 of the drive recorder 20. When the laser detection unit 12 detects the laser, the detector 10 issues an alarm indicating that from the alarm unit 13. When the detector 10 detects the laser, it outputs a signal to that effect to the control unit 24 of the drive recorder 20. That is, the drive recorder 20 warns a person in the vehicle when it detects a radar-type speed control device and when it detects a laser-type speed control device. The person in the vehicle is typically the driver. Hereinafter, a case where a person in the vehicle is used as a driver will be described. Further, the drive recorder 20 reports (notifies) to the drive recorder 20 separately from the case where the radar type speed control device is detected and the case where the laser type speed control device is detected.

The drive recorder 20 includes a camera 21, a primary detection unit 22, a position information acquisition unit 23, a control unit 24, an alarm unit 25 as an output unit, a communication unit 26, and an operation input unit 27. .. The drive recorder 20 corresponds to a photographing device. The camera 21 is installed toward the front of the vehicle. The camera 21 continuously photographs the outside of the front of the vehicle and generates time-series captured images. The captured image is a still image unless otherwise specified. A moving image format file may be generated by arranging a plurality of captured images in chronological order. The speed control device installed above or to the side of the road (shoulder, side of the road) is photographed by the camera 21. The camera 21 may be called, for example, a spherical camera or a hemispherical camera.

The primary detection unit 22 detects the speed control device from the captured image by performing object detection processing on the captured image generated by the camera 21. In the present embodiment, the detection standard of the speed control device in the primary detection unit 22 is lower than the detection standard of the speed control device in the secondary detection unit 33, which will be described later. In other words, the permissible degree of erroneous detection in the primary detection unit 22 (that is, the degree to which an object other than the target speed control device is allowed to be erroneously detected as the speed control device) is determined by the secondary detection unit 33. Greater than that. As a result, the speed control device can be detected by the primary detection unit 22 as little as possible for the captured image in which the speed control device is detected by the secondary detection.

Specifically, the primary detection unit 22 stores patterns of images (particularly, images obtained by taking a picture from an in-vehicle camera) prepared in advance for various speed control devices as binary data. The binary data showing the image pattern of this speed control device corresponds to the first machine learning model. Then, the primary detection unit 22 obtains the degree of coincidence between the rectangular partial image in the captured image and the binary data. This degree of coincidence is expressed as the probability that the object in the partial image is a speed control device, that is, the certainty. When this probability exceeds a predetermined threshold value, the primary detection unit 22 determines that the captured image includes (that is, is reflected) the speed control device. The primary detection unit 22 detects the speed control device in the captured image by such pattern matching. When the speed control device is detected in the captured image, the primary detection unit 22 outputs the captured image and the detection result to the control unit 24. Note that the primary detection unit 22 may perform object detection processing using a machine learning model by a method other than pattern matching. The primary detection unit 22 may extract an image feature amount from the captured image and perform object detection processing based on the image feature amount and a machine learning model defined in advance. As the feature amount of the image, for example, SIFT feature amount, SURF feature amount or other feature amount can be used. As described above, in the present embodiment, the primary detection unit 22 performs the object detection process for detecting the target object based on the content of the captured image.

The position information acquisition unit 23 acquires the position information of the vehicle. The position information acquisition unit 23 of the present embodiment acquires the position information by calculation. Specifically, the position information acquisition unit 23 includes GPS receivers that receive GPS signals from a plurality of GPS satellites, and calculates the position information from the received plurality of GPS signals by a predetermined calculation formula. obtain. The position information acquisition unit 23 outputs the acquired position information to the control unit 24. In addition, this position information may include altitude information.

The control unit 24 controls each unit of the drive recorder 20 (corresponding to the first control unit). The control unit 24 is, for example, a microcomputer including an arithmetic processing circuit and a memory. The arithmetic processing circuit includes, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or other arithmetic processing circuits. The memory includes, for example, a RAM (Random Access Memory) or other volatile memory. The arithmetic processing circuit performs various controls by temporarily reading data into a memory and performing arithmetic processing. The control unit 24 mainly controls the alarm unit 25 and the communication unit 26. The control unit 24 obtains a captured image in which the speed control device is detected from the primary detection unit 22, acquires the position information of the vehicle from the position information acquisition unit 23, and further detects the radar or laser from the detector 10. To get. When the control unit 24 obtains the result that the speed control device is detected from the primary detection unit 22, the control unit 24 combines the corresponding captured image, the position information at the time of the capture, and the detection result of the radar or the laser. As a set of data, the communication unit 26 is controlled and transmitted from the communication unit 26 to the server 30. When the control unit 24 obtains the result that the speed control device is detected from the primary detection unit 22, and the detection result that the radar or the laser is detected from the detector 10 is not received, the captured image. And the position information, the information that the radar and the laser were not detected is added to make a set of data.

When the primary detection unit 22 detects the speed control device, the alarm unit 25 as an output unit outputs an alarm according to the control of the control unit 24. The alarm unit 25 may be a buzzer that outputs a buzzer sound. Alternatively, the alarm unit 25 may be a voice output device that outputs a predetermined alarm sound for notifying the driver of the existence of the speed control device. In this case, the alarm voice may be an electronic melody or a linguistic message. Further, the alarm unit 25 may include a display panel such as a liquid crystal panel, and display an alarm image on the display panel. Further, the alarm unit 25 may be a lamp that informs the driver of the existence of the speed control device depending on whether or not it is lit or the lighting color. Further, the alarm unit 25 is not limited to the one that appeals to the auditory sense or the visual sense as described above, but may notify the driver of the existence of the speed control device by appealing to other perceptions (for example, tactile sense) of the driver. ..

The alarm output unit 25 may output an alarm via the detector 10 instead of or in addition to the configuration for outputting the alarm in the drive recorder 20. In this case, the alarm output unit 25 includes an interface for connecting to the detector 10 via a wired or wireless communication path. The control unit 24 outputs a signal for issuing an alarm via the detector 10 from the alarm unit 25 to the detector 10. In the detector 10, the alarm unit 13 issues an alarm in response to receiving this signal. The alarm of the alarm unit 13 may be performed by the method described above. For example, if the detector 10 is a monitor-type device installed on a dashboard, if the detector 10 gives an alarm by display, the driver may easily see the display related to the alarm.

The communication unit 26 is a wireless LAN module in the present embodiment. The communication unit 26 is connected to a mobile phone communication network, which is a wide area communication network, via a mobile router or a smartphone with a tethering function by connecting to a mobile router or a smartphone with a tethering function via a wireless LAN. Alternatively, the communication unit 26 may be a communication module directly connected to the mobile phone communication network. In this case, a SIM card is installed in this communication module, and the SIM card is connected to a wide area communication network via a mobile phone communication network corresponding to the SIM card.

In the present embodiment, the communication unit 26 communicates with the server 30 in particular according to the control of the control unit 24. Specifically, as described above, the communication unit 26 sends the captured image, the position information, and the radar or laser detection result to the server 30 as a set of data when the speed control device is used by the primary detection unit 22. Send. Further, when the speed control device is detected in the secondary detection by the secondary detection unit 33 described later, the communication unit 26 receives a report to that effect from the server 30.

When the control unit 24 receives a report that the speed control device has been detected from the captured image by the primary detection unit 22, the control unit 24 receives a report that the radar or laser has been detected from the detector 10, and the communication unit 26. When the server 30 receives a report that the speed control device has been detected in the secondary detection, the alarm unit 25 is controlled to output an alarm. At this time, the cause of the alarm output (that is, whether it is a report from the primary detection unit 22, a report from the radar detection unit 11, a report from the laser detection unit 12, or a report from the detector 10). Alarms with different contents may be output depending on whether there are any of them, whether they correspond to a plurality of them at the same time, whether the report is from the server 30, etc.).

Further, for example, the speed control device is detected only by the primary detection unit 22, the speed control device is detected only by the detector 10, the speed control device is detected by both the primary detection unit 22 and the detector 10, and the speed control device is detected by the server 30. The content of the alarm may be set so that the alert level increases in the order of detection. For example, when the alarm is output by voice, the higher the alert level, the louder the volume may be. Further, for example, when displaying an alarm as an image, the image may be displayed in a color corresponding to the alert level.

The control unit 24 does not output an alarm only when the primary detection unit 22 receives a report that the speed control device has been detected from the captured image, but reports that the detector 10 has detected a radar or a laser. When the primary detection unit 22 does not output an alarm even if it receives it, and receives a report that the speed control device has been detected from the captured image, and also receives a report that the radar or laser has been detected from the detector 10. An alarm may be output to. Also in this case, when the communication unit 26 receives a report from the server 30 that the speed control device has been detected in the secondary detection, an alarm is issued regardless of the report from the primary detection unit 22 and the detector 10. You may output it.

The operation input unit 27 inputs the operation performed by the driver. The operation input unit 27 may include, for example, a push-type or other type of physical button, touch sensor or proximity sensor operation, or other operation device. The operation input unit 27 may accept input of voice operation or gesture operation. The operation input unit 27 may have, for example, a microphone when accepting a voice operation input, and may have, for example, a camera or an optical sensor when accepting a gesture operation. The operation input unit 27 may receive an input of an operation signal indicating the operation of the driver from a device other than the drive recorder 20 (for example, a detector 10 or a one-touch button to which a predetermined function is assigned). In addition to this, the drive recorder 20 may have a terminal portion for connecting to an OBD2 connector provided on the vehicle in order to acquire information on the state of the vehicle (that is, vehicle information). The drive recorder 20 may be provided with various sensors exemplified by the acceleration sensor and the illuminance sensor, if necessary.

The server 30 includes a communication unit 31, a control unit 32, a secondary detection unit 33, a storage unit 34, and a learning unit 35. The communication unit 31 is a communication module, receives the information transmitted from the drive recorder 20, and transmits the information to the drive recorder 20. In particular, the communication unit 31 receives a set of data including a captured image, position information, and information on the presence or absence of radar and laser detection transmitted from the drive recorder 20. Further, the communication unit 31 transmits the detection result of the secondary detection unit 33 to the drive recorder 20.

The control unit 32 controls the communication unit 31, the secondary detection unit 33, the storage unit 34, and the learning unit 35 (corresponding to the second control unit). The control unit 32 is, for example, a microcomputer including an arithmetic processing circuit and a memory. When the communication unit 31 receives a set of data from the drive recorder 20, the control unit 32 causes the secondary detection unit 33 to perform secondary detection on the captured image. Further, when the speed control device is detected as a result of the secondary detection by the secondary detection unit 33, the control unit 32 associates the result of the secondary detection with the position information received together with the captured image. In addition to storing the captured image in the storage unit 34, the captured image is stored in the learning unit 35 as a positive example.

The secondary detection unit 33 detects the speed control device from the captured image by performing object detection processing on the captured image. The secondary detection unit 33 detects an object by an algorithm different from that of the primary detection unit 22. In the present embodiment, the secondary detection unit 33 detects an object using a machine learning model (corresponding to a second machine learning model) such as a CNN (Convolutional Neural Network). Specifically, the secondary detection unit 33 uses a deep learning model as a machine learning model. Since the captured image transmitted from the drive recorder 20 is an image in which the speed control device has already been detected by the primary detection unit 22 of the drive recorder 20, the secondary detection unit 33 has higher accuracy than the primary detection unit 22. Detects speed control equipment. That is, the secondary detection unit 33 determines that the object misidentified as the speed control device by the primary detection unit 22 is not the speed control device.

The machine learning model used by the secondary detection unit 33 can also determine the type of speed control device. There are a plurality of types of speed control devices called H system, LH system, speed warning safety system, portable type, semi-portable type, etc., and the secondary detection unit 33 detects these speed control devices from the captured image. At the same time, the type is also determined. The secondary detection unit 33 acquires the probability, that is, the certainty, that the object included in the partial image of the captured image is a speed control device of each type by the deep learning model. When this probability is the highest and exceeds a predetermined threshold value, it is determined that most of the speed control devices having the highest probability are present. If the probability of any type of speed camera does not exceed the threshold, it is determined that there is no speed camera in the partial image.

2A to 2D are diagrams showing an example of an object detection process in the secondary detection unit 33. In the examples of FIGS. 2A to 2D, different types (types 1 to 4) of speed control devices are detected. The secondary detection unit 33 identifies a region in which the speed control device is detected in the captured image, and attaches a label of the discriminated type to the region. The secondary detection unit 33 detects not only the speed control device installed on the side of the road as shown in FIGS. 2A to 2C, but also the speed control device installed on the road as shown in FIG. 2D.

When the speed control device is detected from the captured image by the secondary detection unit 33, the storage unit 34 detects information that the speed control device exists, information on the type of the speed control device, and the speed control device. This is a database that stores data associated with the position information of the vehicle when the captured image is obtained as control shared data. Map information is also stored in the storage unit 34, and information indicating the existence of the speed control device and information of its type are arranged at points on the map corresponding to the position information associated therewith. It is stored in a format (in association with the location information on the map). That is, the storage unit 34 stores the distribution map of the speed control device.

The learning unit 35 performs machine learning for generating the machine learning model used in the secondary detection unit 33. The learning in the learning unit 35 may be performed asynchronously with the other configurations described above. Therefore, the server 30 stores or outputs only the learning data to be used for this machine learning, and the learning unit 35 may be provided in a device other than the server 30 as a learning device. The system 100 that performs processing for machine learning centered on the learning unit 35 can be regarded as a machine learning system, and the server 30 can also be regarded as a learning device.

The learning unit 35 learns by using each image of a plurality of images taken by the speed control device as teacher data associated with the inclusion of the speed control device in the image. The learning unit 35 attaches a label indicating that it is a positive example to each captured image of a plurality of captured images captured by various types of speed control devices, and generates teacher data as learning data. In addition, the captured images used as teacher data include, for example, a person such as a police officer who controls at a speed control point, a vehicle such as a police vehicle that may be parked at a speed control point, and a speed control. It may include at least one of the landscapes (eg, cityscapes) of a point that is likely to be set as a point. Since these are elements that are relatively likely to be reflected together with the speed control device, the effect of generating a higher quality machine learning model can be expected by using this captured image as learning data. The positive example teacher data is data in which a captured image is labeled with a rectangular area of the speed control device and the type of the speed control device.

Further, it is preferable that a photographed image including an object which is similar in appearance to the speed control device but is not the speed control device is used as the teacher data of a negative example. Such an object is a gate-shaped structure (for example, a gate-shaped structure; see FIG. 2D). This is because such structures may appear to be similar to the H system, and the negative example teacher data includes the rectangular area of the object that was mistaken for a speed camera in the captured image and the label of the non-speed camera. It is the data with. The label may be, for example, manually or automatically attached by the administrator who manages the server 30. In this way, the effect of generating a higher quality machine learning model from the captured image can be expected.

For example, the following means may be adopted as means for automating at least a part of the work of labeling the captured image. The learning unit 35 attaches a label to the captured image based on the result of the secondary detection in the secondary detection unit 33 and the detection result by the detector 10 sent from the drive recorder 20, and learns in the learning unit 35. Generate teacher data as data for use.

Specifically, when the speed control device is detected by the secondary detection unit 33 and the electromagnetic wave in the predetermined wavelength band is detected even in the detection result by the detector 10, the learning unit 35 corrects the captured image. Let's take the example teacher data. The positive example teacher data is data in which a captured image is labeled with a rectangular area of the speed control device and the type of the speed control device. In the learning unit 35, the speed control device is detected by the secondary detection unit 33, but when the detection result by the detector 10 does not detect an electromagnetic wave in a predetermined wavelength band, the appearance is similar to the speed control device. An object that is not actually a speed control device is erroneously detected as a speed control device by the secondary detection unit 33, and the captured image is used as a negative example of teacher data. The teacher data of the negative example is data in which the captured image is labeled with a rectangular area of an object misidentified as a speed control device and a non-speed control device.

If the speed control device is not detected by the secondary detection unit 33, the learning unit 35 does not use the captured image as the learning data. This is because the captured image in which the speed control device was not detected by the secondary detection unit 33 contains an object that is mistaken for the speed control device in the primary detection, but it is judged that the speed control device is not already detected in the secondary detection. This is because the photographed image is made, so the quality is not so good as a negative example. Even if the speed control device is not detected by the secondary detection unit 33, if the detection result of the detector 10 detects an electromagnetic wave in a predetermined wavelength band, it is said that the speed control device is actually present. On the other hand, there is a possibility that the detector 10 has detected an electromagnetic wave from a source other than a speed control device such as a vending machine, so the captured image is not used as learning data.

It is desirable that the plurality of captured images used as the learning data include images captured by the speed control device from each of a plurality of different directions. The plurality of different directions may include a front side surface and a direction in which at least one side surface can be photographed, and in particular, a direction in which the speed control device is viewed from slightly obliquely forward with respect to the front surface. It is also desirable to include images taken from each of a plurality of different directions for a mobile (eg, portable and semi-portable) speed camera (see FIG. 2B). This is because it is considered that such a mobile speed camera is placed on the shoulder of the road and has more chances of being photographed from diagonally forward than the fixed speed camera. In this case, when the vehicle is relatively far from the speed camera, the front surface of the speed camera is easily photographed, but when the vehicle is relatively close to the speed camera, the front surface of the speed camera is photographed. It becomes difficult to photograph the side surface, and it becomes easy to photograph the side surface (for example, a part of the front surface and the right side surface with respect to the traveling direction of the vehicle). Therefore, by including images taken from each direction of the speed control device, a high-quality machine learning model that suppresses the change in the detection accuracy of the speed control device depending on the distance between the vehicle and the speed control device is generated. You can expect the effect that can be done. In this sense, the plurality of captured images used as learning data also include images captured by the speed control device at each distance of a plurality of different distances, so that the speed control based on the distance between the vehicle and the speed control device can also be included. The effect of generating a high-quality machine learning model that suppresses changes in the detection accuracy of the device can be expected. The captured image used as the regular teacher data may be as described above.

The learning unit 35 performs deep learning using the teacher data of the positive example and the negative example as described above as learning data, and generates a machine learning model which is a deep learning model. The secondary detection unit 33 can improve the accuracy of the secondary detection by performing the secondary detection using the machine learning model generated by the learning unit 35.

FIG. 3 is a flowchart of a detection method according to an embodiment of the present invention. In FIG. 1, an example in which the system includes the detector 10 has been described, but in the description of the detection method in FIG. 3, an example in which the detector 10 is not used will be described. In FIG. 3, the left column is the processing in the drive recorder 20, and the right column is the processing in the server 30.

First, the drive recorder 20 photographs the front of the vehicle with the camera 21 (step S21). The primary detection unit 22 detects the speed control device from the captured image obtained by the camera 21 using a machine learning model (for example, for pattern matching) (step S22). The control unit 24 determines whether the speed control device is detected by the primary detection in the primary detection unit 22 (step S23), and if the speed control device is not detected (NO in step S23), the camera 21 continues to take a picture. (Step S21).

When the speed control device is detected by the primary detection in the primary detection unit 22 (YES in step S23), the control unit 24 causes the alarm unit 25 to output an alarm (first alarm) (step S24), and position information. Position information is acquired from the acquisition unit 23 (step S25). Then, the control unit 24 controls the communication unit 26 to transmit data including the captured image in which the speed control device is detected and the position information at that time to the server 30 (step S26).

The communication unit 31 of the server 30 receives the data transmitted from the drive recorder 20 (step S31). The control unit 32 controls the secondary detection unit 33, and causes the secondary detection unit 33 to perform an object detection process for detecting the speed control device from the received captured image (step S32). The secondary detection unit 33 detects the speed control device from the captured image using the machine learning model which is a deep learning model. The control unit 32 determines whether or not the speed control device is detected by the secondary detection unit 33 (step S33), and if the speed control device is not detected (NO in step S33), the process ends. As a result, the drive recorder 20 only outputs an alarm based on the primary detection, and the process ends.

When the speed control device is detected by the secondary detection unit 33 (YES in step S33), the storage unit 34 is sent together with the type of the speed control device and the captured image in which the speed control device is detected. It is stored in association with the position information (step S34). Further, in this case, the control unit 32 controls the communication unit 31 and records the detection result data including the fact that the speed control device is detected by the secondary detection and the type of the speed control device in the drive recorder. It is transmitted to 20 (step S35).

The communication unit 26 of the drive recorder 20 receives the data from the server 30 (step S27). The control unit 24 controls the alarm unit 25 in response to the reception of this data, outputs an alarm (second alarm) (step S28), and ends the process. The content of this second warning is different from that of the first warning, and the driver of the vehicle can confirm that it is the second warning.

FIG. 4 is a flowchart of the machine learning method according to the embodiment of the present invention. The operation when the process of labeling the captured image is automated will be described below. When the server 30 as the learning device receives the captured image from the drive recorder 20 and the detection result by the detector 10 at the communication unit 31, the secondary detection unit 33 performs the secondary detection (step S41). The learning unit 35 stores the result of the secondary detection as learning data based on the result of the secondary detection and the detection result. Specifically, in the learning unit 35, when the speed control device is detected by the secondary detection unit 33 (YES in step S42), and the detection result by the detector 10 also detects an electromagnetic wave in a predetermined wavelength band. (YES in step S43) stores the captured image as regular teacher data (step S44).

In the learning unit 35, when the speed control device is detected by the secondary detection unit 33 (YES in step S42), but the electromagnetic wave in the predetermined wavelength band is not detected in the detection result by the detector 10 (step S43). NO), the captured image is stored as negative example teacher data (step S45). If the speed control device is not detected by the secondary detection unit 33, the learning unit 35 does not use the captured image as learning data (step S46).

The learning unit 35 determines whether or not the conditions for executing learning are satisfied (step S47). The learning unit 35 makes it a condition for learning execution that a predetermined number or more of learning data has been accumulated. When the female right is satisfied (YES in step S47), the learning unit 35 executes learning to generate a machine learning model using the learning data including the accumulated positive and negative teacher data. (Step S48).

The following method may be further adopted as the learning method of the learning unit 35.
The system 100 may have a function of posting information about the speed control device when the driver visually confirms the speed control device. For example, when the driver visually confirms the speed control device, the driver performs a predetermined operation (posting operation) using the operation input unit 27. In response to the acceptance of the posting operation, the control unit 24 acquires a captured image (for example, the latest captured image) captured when the posting operation is accepted from the camera 21. The control unit 24 transmits the acquired captured image from the communication unit 26 to the server 30 together with data indicating that the posting operation has been accepted. The communication unit 31 in the server 30 accepts posts by receiving these data. The control unit 32 supplies the received data to the learning unit 35. The learning unit 35 uses the photographed image as regular teacher data associated with the inclusion of the speed control device to perform supervised learning. If the driver posts information about a speed camera, the captured image is likely to include a speed camera. Therefore, if supervised learning is performed using this captured image, a better quality deep learning model can be generated, and for example, when a new type of speed control device appears, an effect corresponding to this can be expected. Further, the drive recorder 20 may be configured so that the type of the speed control device and the position information of the speed control device can be input by the posting operation. In this case as well, the drive recorder 20 transmits the input data to the server 30. The server 30 stores this data in the storage unit 34.

Regarding the function of posting information about the speed control device, the control unit 24 receives a posting operation and receives a captured image (for example, the latest captured image) taken when the posting operation is accepted. The communication unit 26 transmits to the server 30 the detection result of the radar or the laser when the operation is accepted (for example, at least one predetermined period before and after the timing of the posting operation) as a set of data. May be good. Then, the control unit 32 supplies this set of data to the learning unit 35. When the learning unit 35 determines that the captured image was captured when the radar or laser was received based on the detection result, it is a positive example associated with the inclusion of the speed control device in the captured image. Supervised learning is performed using it as teacher data. Further, when the learning unit 35 determines based on the detection result that the captured image was captured when the radar or laser is not received (for example, the intensity is below the threshold value), this imaging is performed. Supervised learning may be performed by using it as negative example teacher data associated with the fact that the image does not include a speed camera. When the driver posts information about the speed camera, if the radar or laser is detected, it is highly possible that the speed camera is actually captured in the captured image. On the other hand, if the radar or laser is not detected, there is a relatively high possibility that the speed control device is not actually captured in the captured image due to a driver's misunderstanding or the like. In this way, if learning is performed using the captured image and the detection result of the radar or the laser, a higher quality deep learning model can be generated.

The function of posting information about this speed control device may be made available only to a specific driver. For example, in the system 100, depending on the number of posts so far (for example, the number of posts exceeds the threshold value) or according to the reliability of the information posted in the past (for example, the reliability exceeds the threshold value). It is good to accept posts from specific drivers and use them for learning. The reliability is, for example, the number of drivers whose posted information is judged to be correct by another driver and has performed an operation to that effect. In this way, a better quality deep learning model can be generated.

Further, the system 100 may have a function of acquiring an input of a response from the driver when an alarm is issued and learning according to the result of the response. When the alarm unit 25 outputs an alarm, the driver operates the operation input unit 27 to input a response indicating that the alarm is correct when the speed control device can be visually confirmed. .. On the other hand, if the driver cannot visually confirm the speed control device, this alarm may be a false alarm. When the driver determines that the alarm is a false alarm, the driver operates the operation input unit 27 to input a response indicating that the alarm is a false alarm. In response to receiving any of these operations, the control unit 24 acquires a captured image (for example, the latest captured image) captured when the operation is accepted from the camera 21. The control unit 24 transmits data in which the acquired captured image is associated with the result of the response from the driver from the communication unit 26 to the server 30. In the server 30, the communication unit 31 receives this data. The control unit 32 supplies the received data to the learning unit 35. The learning unit 35 uses this captured image as positive teacher data when the alarm is correct and as negative teacher data when the alarm is false, depending on the result of the response, to perform supervised learning. In this way, the learning unit 35 can generate a higher quality deep learning model by making it a deep learning model that reflects the response from the driver. In particular, if the captured image taken at the time of false alarm is used as negative example teacher data, the false alarm should be reduced when there is a structure similar in appearance to the speed control device and an erroneous alarm is given. The effect of generating a deep learning model can be expected.

The alarm may be as follows.
In the drive recorder 20, when the speed of the vehicle is below the threshold value (for example, 30 km / h or less), the control unit 24 does not output an alarm from the alarm unit 25 even when the speed control device is detected. It may be. For example, the control unit 24 prevents the primary detection unit 22 from detecting the speed control device when the speed of the vehicle is equal to or less than the threshold value. Instead of not outputting the alarm, the alarm unit 25 may lower the alarm level as compared with the case where the speed of the vehicle is higher than the threshold value. For example, when the speed of the vehicle is a threshold value, it is unlikely that the vehicle is in violation of speed, and it is presumed that safe driving is being performed. Therefore, even if the drive recorder 20 does not output an alarm or lowers the alarm level, it is considered that there is little inconvenience for the driver. The threshold value may be, for example, the legal speed of the road on which the vehicle travels. The drive recorder 20 may acquire the speed of the vehicle via the OBD2 connector provided on the vehicle, or may acquire the speed using the vehicle speed sensor.

The machine learning model generated in this way includes an input layer into which an image is input, an output layer in which the image includes a speed control device and outputs data according to the certainty, and each image of a plurality of images. It is preferable to have at least one intermediate layer in which the parameters are learned using the teacher data, which outputs the data according to the certainty that the speed control device is included in the image. The parameter learning is performed by the learning of the learning unit 35 described above. In this way, it is possible to provide a machine learning model for detecting the speed control device from the image. The data according to the accuracy is data that changes depending on the accuracy, and may be a value indicating the probability that the image includes a speed control device. In this case, if the probability exceeds the threshold value, it is determined that the image includes a speed control device. Not limited to this, the data indicating the certainty that the image includes the speed control device may be information indicating whether or not the image includes the speed control device. The secondary detection unit 33, which performs secondary detection using such a deep learning model, inputs an image taken by the drive recorder 20 into the deep learning model into the input layer, calculates in the intermediate layer, and performs calculation from the output layer. Based on the output data, it is determined whether or not the speed control device has been detected.

As described above, according to the system 100 of the embodiment of the present invention, the primary detection unit is not detected by the detector 10 alone and outputs an alarm, but is obtained from the captured image obtained by the camera 21. Since the existence of the speed control device is recognized by detecting the speed control device using the machine learning model in 22 and the secondary detection unit 33, an electromagnetic wave source other than the speed control device such as an automatic door is detected as the speed control device. It is possible to reduce the possibility of issuing an alarm (false report).

Further, the system 100 according to the embodiment of the present invention includes a drive recorder 20 and a server 30, and a captured image in which the speed control device is detected by the primary detection after the primary detection by the drive recorder 20 which is an in-vehicle device. Is secondary detected on the server 30. Therefore, after reducing the load of the detection process on the drive recorder 20, the communication load on transmitting the captured image from the drive recorder 20 to the server 30 can be reduced, and the detection process having a relatively large processing load on the server 30 can be performed. It can be carried out. Even if the drive recorder 20 does not have the ability to execute processing using the deep learning model or is difficult to execute (for example, the processing time is long) because the microcomputer of the drive recorder 20 is small or the like. The drive recorder 20 can obtain the processing result of the detection process using the deep learning model. In the server 30, when the speed control device is detected from the captured image by the primary detection, the speed control device is detected by using the deep learning model, so that the detection process is performed for all the captured images as compared with the case where the detection process is performed. , The processing load can be reduced. Further, the drive recorder 20 does not need to include a deep learning model, and for example, the deep learning model can be shared by a plurality of drive recorders. In this way, the drive recorder 20 can output an alarm even when the vehicle travels for the first time in a place where a mobile speed control measure is installed. Therefore, it is possible to support the safe driving of the driver.

Further, in the system 100 of the embodiment of the present invention, an alarm is output even when the speed control device is detected by the primary detection in which the detection accuracy of the speed control device is relatively low, and the detection accuracy of the speed control device is improved. When the speed control device is detected by a relatively high secondary detection, an alarm different from that in the case of the primary detection is output, so that an early warning and a highly accurate warning can be provided to the driver of the vehicle.

Further, in the system 100 of the embodiment of the present invention, if the learning data is acquired using the detection result of the captured image using the machine learning model and the detection result of the detector, a person is a positive example. It is possible to automatically prepare learning data for positive and negative examples without determining whether it is a negative example or a negative example. However, if the learning data is generated by discriminating whether the person is a positive example or a negative example, the system 100 can generate a high-quality deep learning model.

The drive recorder 20 may not give at least one of the alarm based on the primary detection and the alarm based on the secondary detection. If the drive recorder 20 does not give an alarm based on the primary detection, it is less likely that an alarm will be generated even though the speed control device does not actually exist. When the drive recorder 20 is prevented from giving an alarm based on the secondary detection, for example, the communication time between the drive recorder 20 and the server 30 may be long depending on the capacity of the drive recorder 20 or the quality of the communication network, or other cases. When the drive recorder 20 takes a long time from shooting to obtaining the result of the secondary detection due to the above cause, it is not necessary to give the alarm at an erroneous timing.

The subject that realizes each function described in the above embodiment (the subject that executes each process) can be modified in various ways. For example, the process performed by the drive recorder 20 in the above embodiment may be performed by the detector 10 which is another in-vehicle device. For example, the control unit of the detector 10 may acquire a captured image from the drive recorder 20 to perform primary detection, and when the speed control device is detected, transmit the captured image to the server 30. When the control unit of the detector 10 receives the data from the server 30, the control unit may control the alarm unit 13 in response to the reception of the data to give an alarm (second alarm). In-vehicle devices are not limited to drive recorders and detectors, but are car navigation devices mounted on vehicles, information processing devices exemplified by smartphones and tablet computers (for example, information processing devices that function as car navigation devices). It may be a device installed in a vehicle) or other equipment mounted on a vehicle. Further, the camera that photographs the outside of the vehicle may be a camera included in a digital video camera or other imaging device.

The in-vehicle device may perform the primary detection and the secondary detection. In this case, when the speed control device detects from the captured image in the primary detection, the in-vehicle device performs the detection process of the speed control device using the deep learning model, so that the detection process is performed on all the captured images as compared with the case where the detection process is performed. Therefore, the processing load on the in-vehicle device can be reduced. In addition, since it is not necessary to consider the time required for communication between the in-vehicle device and the server, it is also desirable to issue an alarm based on the secondary detection at an accurate timing. Further, when the in-vehicle device determines that the speed control device has been detected by performing the primary detection and the secondary detection, the in-vehicle device may transmit the position information indicating the position of the vehicle at the time of the detection to the server 30. .. Further, it is preferable that the object detection process of one step or three or more steps is performed instead of performing the object detection process of two steps of primary detection and secondary detection. In the case of one step, for example, it is preferable that the object detection process using the deep learning model is performed. For the object detection processing of three or more stages, it is preferable that the object detection processing is performed by different algorithms (for example, using different machine learning models).

In the above embodiment, the system 100 includes the drive recorder 20 and the server 30, but the system 100 may be configured by the in-vehicle device alone. In this case, there may be only one detection unit, and this detection unit may perform highly accurate object detection using, for example, a deep learning model. In this case as well, the in-vehicle device may perform primary detection and secondary detection, or store the position information of the speed control device and update it. Further, the in-vehicle device may transmit this position information to another in-vehicle device, for example, by communication between vehicles, and share this with a plurality of vehicles.

In this way, the system controls based on a detection unit that detects a target indicating that it is a speed control point using a machine learning model from an image taken outside the vehicle, and a detection result in the detection unit. It suffices to have a configuration including a control unit for performing the operation.

Further, in the primary detection unit 22 and the secondary detection unit 33 of the above-described embodiment, as described above, whether or not the speed control device exists in the captured image can be obtained with a probability. That is, the probability that the object in the rectangular frame in the captured image is a speed control device can be obtained. Therefore, the alarm unit 25 may prepare an alarm level according to this certainty and output an alarm according to the corresponding alarm level. For example, when the alarm is voice, the volume of the voice may be adjusted according to the alarm level.

Further, in the above embodiment, the system 100 detects the speed control device as a target, and the server 30 generates a distribution map of the speed control device, but the system 100 describes the target other than the speed control device. , The same process may be performed to generate a distribution map. For example, by targeting various landmarks, the system 100 can generate a distribution map of POI (Point of Interest).

In the drive recorder 20, whether or not the alarm can be output may be controlled according to the result of the object detection process on the captured image according to the environment in which the own device (vehicle) is placed. The control unit 24 of the drive recorder 20 uses a sensor to indicate that the vehicle is traveling in a relatively dark environment (for example, when the illuminance measured by the illuminance sensor is a threshold value), or at night, etc. When the surrounding environment is dark in a predetermined time zone, the output of the alarm may be limited according to the result of the object detection process (for example, the output may be stopped). In other words, the control unit 24 detects, for example, that the vehicle is traveling in a relatively bright environment with a sensor (for example, when the illuminance measured by the illuminance sensor exceeds the threshold value), or When the surrounding environment is bright, such as in the daytime, the output of the alarm may be limited according to the result of the object detection process. Further, the system 100 may output an alarm when a radar or a laser is detected by the detector 10 during the period in which the output of the alarm according to the result of the object detection process is restricted. This alarm may be different from the alarm according to the result of the object detection process. In this way, the drive recorder 20 outputs an alarm according to the result of the object detection process when the vehicle is traveling in an environment where the speed control device can be easily detected from the captured image, which is unnecessary. It can be expected to have the effect of reducing processing or reducing the possibility that an erroneous alarm output will occur. However, when the drive recorder 20 is traveling in a relatively dark environment by a method using an image taken by a night-vision camera or another method, even in such an environment, depending on the result of the object detection process. It is desirable to output an alarm.

As the above embodiment, the case where the target indicating that the speed control point is the speed control device has been described. Targets indicating that they are speed control points should be replaced or combined. Scenery of a person such as a police officer who controls at a speed control point, a vehicle such as a police vehicle that may be parked at a speed control point, and a point that is likely to be set as a speed control point (for example, It should include at least one of the cityscapes). In particular, police officers may wear uniforms, and police vehicles have a unique appearance. Therefore, it is considered that the police vehicle can be detected by the above-mentioned object detection process using this as a clue.

The scope of the present invention is not limited to the configuration explicitly described in the specification, but also includes a combination of various aspects of the present invention disclosed in the present specification. Of the present invention, the configuration for which a patent is sought is specified in the appended claims, but even if the configuration is not currently specified in the claims, it is disclosed in the present specification. We intend to make this configuration the scope of claims in the future.

The invention of the present application is not limited to the configuration described in the above-described embodiment. The components of each of the above-described embodiments and modifications may be arbitrarily selected and combined. Further, any component of each embodiment or modification, and any component described in the means for solving the invention or a component described in the means for solving the invention are embodied. And may be configured in any combination. We also intend to acquire the rights to these in the amendment or divisional application of the present application. Further, even if there is a description of "in the case of" or "in the case of", it is not described as a configuration limited to that case or at that time. It also discloses these cases and unusual configurations, and has the intention to acquire the rights. In addition, the places where the description is accompanied by an order are not limited to this order. It also discloses a configuration in which some parts have been deleted or the order has been changed, and it has the intention of acquiring rights.

In addition, by changing to a design registration application, we have the intention to acquire the rights to the entire design or partial design. Although the entire drawing of the device is drawn with solid lines, it is a drawing that includes not only the whole design but also the partial design requested for a part of the device. For example, it is a drawing which includes a part of the device as a partial design regardless of the member as well as a part of the member of the device as a partial design. As a part of the device, it may be a part of the device or a part of the member. Not only the whole design, but also the partial design in which any part of the solid line part of the drawing is a broken line part is intended to be granted the right. In addition, regarding the modules, members, parts, etc. inside the housing of the device, those displayed in the drawings are all independently subject to transactions, and similarly, the application for design registration is filed. They are willing to make changes and acquire rights.

10 Detector 11 Radar detector 12 Laser detector 13 Alarm unit 20 Drive recorder 21 Camera 22 Primary detection unit 23 Position information acquisition unit 24 Control unit 25 Alarm unit 26 Communication unit 27 Operation input unit 30 Server 31 Communication unit 32 Control unit 33 Secondary detection unit 34 Storage unit 35 Learning unit 100 System

Claims (29)

A detection unit that detects a target indicating that it is a speed control point of the vehicle using a machine learning model from an image taken outside the vehicle.
A control unit that controls based on the detection result in the detection unit,
System with. The detection unit
A primary detector that detects the target using the first machine learning model,
When the target is detected by the primary detection unit, the secondary detection unit that detects the target using the second machine learning model, and the secondary detection unit.
Have,
The control unit
Control is performed based on the detection results of the primary detection unit and the secondary detection unit.
The system according to claim 1. The primary detection unit is provided on an in-vehicle device mounted on a vehicle and is provided.
The secondary detection unit is provided in a server that communicates with the in-vehicle device via a communication network.
The system according to claim 2. The primary detection unit detects the target object from the image by performing pattern matching with the binary data as the first machine learning model.
The secondary detection unit detects the target from the image using a deep learning model as the second machine learning model.
The system according to claim 2 or 3. The vehicle is further equipped with a detector for detecting electromagnetic waves in a predetermined wavelength band.
The primary detection unit detects the target based on the first machine learning model and the detection result of the detector.
The system according to any one of claims 2 to 4. The control unit performs the control when the target is detected from the image based on the first machine learning model during the period when the electromagnetic wave is received by the detector.
The system according to claim 5. The control unit has a first control unit that controls to output an alarm from an alarm unit provided in the vehicle based on the detection result in the primary detection unit.
The system according to any one of claims 1 to 6. The control unit has a first control unit that controls to output an alarm from an alarm unit provided in the vehicle based on the detection result in the secondary detection unit.
The system according to any one of claims 2 to 6. The control unit controls to output a first alarm from an alarm unit provided in the vehicle based on the detection result in the primary detection unit, and the first alarm is output based on the detection result in the secondary detection unit. Has a first control unit that controls to output a second alarm different from
The system according to any one of claims 2 to 6. The first control unit controls to output different alarms depending on the certainty of the detection result of the target object.
The system according to any one of claims 7 to 9. A storage unit for storing the position information of the target object is provided.
The control unit
It has a second control unit that controls to update the position information of the target object in the storage unit based on the detection results of the plurality of detection units mounted on different vehicles.
The system according to any one of claims 1 to 10. A storage unit for storing the position information of the target object is provided.
The control unit
It has a second control unit that controls to update the position information of the target object in the storage unit based on the detection results of the plurality of detection units mounted on different vehicles.
The first control unit
Control to output the alarm is performed based on the position information of the target object stored in the storage unit and the position of the vehicle.
The system according to any one of claims 7 to 10. The first control unit is provided on an in-vehicle device mounted on a vehicle.
The system according to claim 11 or 12, wherein the second control unit is provided in a server that communicates with the in-vehicle device via a communication network. It is a system equipped with an in-vehicle device mounted on a vehicle and a server provided on a communication network.
The in-vehicle device is
A camera that captures the outside of the vehicle and generates a captured image,
A vehicle-side transmitter that transmits the captured image to the server via the communication network.
With
The server
A server-side receiver that receives the captured image from the in-vehicle device via the communication network, and
A detector that detects a target from the captured image using a machine learning model,
A storage unit that stores the detection result in the detection unit,
System with. The in-vehicle device includes a primary detection unit that detects the target object from the captured image using a machine learning model having a lower detection standard than the machine learning model.
The vehicle-side transmission unit transmits the captured image in which the target is detected by the primary detection unit to the server.
The system according to claim 14, wherein the detection unit detects the target object from the captured image received by the server-side reception unit as a secondary detection unit. The in-vehicle device includes a position information acquisition unit that acquires the position information of the vehicle.
The vehicle-side transmission unit transmits the captured image and the position information of the vehicle when the captured image is captured to the server.
The system according to claim 14, wherein the storage unit stores the detection result in association with the position information. The in-vehicle device is
A vehicle-side receiver that receives a notification based on the information stored in the storage unit,
An output unit that outputs information based on the notification,
16. The system of claim 16. A system equipped with in-vehicle equipment
The in-vehicle device
A detector that detects a target using the first machine learning model from an image taken outside the vehicle,
When the target is detected by the detection unit, the transmission unit that transmits the image to the server and
A receiving unit that receives the result of a process of detecting a target object from the image using the second machine learning model from the server.
A control unit that performs control based on the detection result in the detection unit and the received result of the processing.
System with. Acquisition of each image of a plurality of images of a target indicating that it is a vehicle speed control point, and learning using teacher data associated with the inclusion of the target in each of the acquired images. Do, system. An image taken of the outside of the vehicle by a camera provided in the vehicle, and the image taken when the electromagnetic wave is detected by a detector provided in the vehicle for detecting an electromagnetic wave in a predetermined wavelength band is positive. As an example, a system for supervised learning. The said The system according to claim 20, wherein supervised learning is performed using an image as a positive example. A detection unit for detecting the target object from the image is provided.
The system according to claim 20 or 21, wherein supervised learning is performed using the image when the target is detected, although the detector does not detect electromagnetic waves in the predetermined wavelength band. Accepting the posting of the image taken when the person in the vehicle visually confirmed the target object,
The system according to any one of claims 19 to 22, wherein the supervised learning is performed using the posted image as a positive example. In addition to the image taken when the person in the vehicle visually confirms the target object, the detector accepts posting of the detection result of the electromagnetic wave in the predetermined wavelength band.
The system according to claim 23, wherein the posted image is supervised-learned as a positive example or a negative example according to the detection result of receiving the posting. When the target is detected from the image and an alarm is given, the input of the response from the person in the vehicle is accepted.
The system according to any one of claims 19 to 24, wherein supervised learning is performed using the image as a positive example or a negative example according to the result of the response that receives the input. The target is a vehicle control device,
The system according to any one of claims 1 to 25. At least a part of the function of the system according to any one of claims 1 to 26 is realized by an in-vehicle device.
A program for making a computer realize the functions realized by the in-vehicle device. Acquisition of each image of a plurality of images of a target indicating that it is a vehicle speed control point, and learning using teacher data associated with the inclusion of the target in each of the acquired images. Machine learning method to do. The input layer where the image is input and
An output layer that outputs data according to the accuracy, and an output layer that includes a target indicating that the image is a speed control point of the vehicle.
At least one intermediate layer in which parameters are learned using teacher data in which each image of a plurality of images is input and data corresponding to the probability that the target is included in the image is output, and
Machine learning model with.

Images