JP2021037904A - Object detection device and system and method for the same - Google Patents

Abstract

To provide a method by which an accuracy to determine existence of a foreign object at a door catching part of a sliding side door of a vehicle is improved.SOLUTION: An object detection device comprises a picture analysis part 20 to detect an object in a door catching part periphery area, on the basis of an imaged picture of the door catching part periphery area of a vehicle sliding side door, acquired by a picture acquisition part 10. The picture analysis part 20 has a learning server 28 and during a learning time, the server inputs imaged pictures of the door catching part periphery area and learns a learning model in which the imaged picture and information of existence of a foreign object included in the imaged picture are used as teacher data and at the detection time, the server inputs an actually imaged picture of the door catching part periphery area and outputs information of existence of the foreign object included in the imaged picture.SELECTED DRAWING: Figure 1

Description

The present invention relates to an object detection device, an object detection system, and an object detection method.

In railway operations, running a vehicle with a foreign object caught in the door of the vehicle leads to a serious accident. However, since the foreign matter pinching detection function of the vehicle door itself cannot detect the pinching of foreign matter having a size of 10 to 15 mm or less, measures for detecting the door pinching state continue to be required. For example, the image before opening the door captured by the camera provided at the upper part of the meeting part of the passenger door of the railroad vehicle is acquired as a reference image, and the image after getting on and off the passenger is acquired as a judgment image. When comparing the reference image and the judgment image, the vertical components of both images are emphasized, the difference between the reference image with the emphasized vertical component and the judgment image is obtained, and the entrance door for passengers gets on and off based on this difference. There is a technique for detecting a foreign substance sandwiched between the two (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2005-349997

However, as in Patent Document 1, since the appearance of the object in the captured image differs depending on the direction in which the object is imaged and the imaging conditions, the accuracy of the image analysis is lowered and the presence or absence of the object is present. There is a problem that the judgment accuracy is lowered.

An object of the present invention is to provide an object detection system and an object detection method capable of improving the determination accuracy of the presence or absence of an object based on the difference in the appearance of the object in a captured image.

In order to solve the above problem, one aspect of the object detection device of the present invention is a region around the door clamp portion based on an image captured by the image acquisition unit of the region around the door clamp portion of the side sliding door of the vehicle. It is an object detection device having an image analysis unit that detects an object, and the image analysis unit inputs an image captured in the area around the door sandwiching portion during learning, and the captured image and the presence or absence of foreign matter contained in the captured image are present. It is characterized by having a machine learning unit that learns a learning model using information as teacher data, inputs an image captured in the area around the door sandwiched portion at the time of detection, and outputs information on the presence or absence of foreign matter contained in the captured image. And.

Another aspect of the object detection device of the present invention is that the image analysis unit has an image generation unit that performs image processing so as to cut out a region of the door end rubber portion in the captured image acquired by the image acquisition unit. It is characterized by.

In another aspect of the object detection device of the present invention, a region of the door end rubber portion in the captured image acquired by the image acquisition unit is cut out, and a region other than the cut out region of the door end rubber portion is used as a predetermined background. It is characterized by having an image generation unit that performs image processing so as to mask with a color or a background pattern.

In another aspect of the object detection device of the present invention, the image analysis unit cuts out a region of the door end rubber portion in the captured image acquired by the image acquisition unit, and the region other than the cut out region of the door end rubber portion is used. It is characterized by having an image generation unit that masks an area with a predetermined background color or background pattern and performs image processing so as to shape the masked door end rubber portion into a rectangular shape.

One aspect of the object detection system of the present invention is provided with the object detection device according to each of the above-described inventions, and from the image pickup device arranged so as to be able to image the area around the door sandwich portion of the side sliding door of the vehicle, the periphery of the door sandwich portion. It is characterized by having an image acquisition unit that acquires a captured image of a region.

One aspect of the object detection method of the present invention is a step of acquiring an image of the area around the door pinch portion from an image pickup device arranged so that the area around the door pinch portion of the side sliding door of the vehicle can be imaged, and the area around the door pinch portion. It is an object detection method having a step of detecting an object in the area around the door sandwiching portion based on the captured image of the region, and at the time of learning, the captured image of the area around the door sandwiching portion is input, and the captured image and the said The step of learning the learning model using the information on the presence or absence of foreign matter contained in the captured image as teacher data, and the presence or absence of foreign matter contained in the captured image by inputting the captured image of the area around the door sandwiched portion at the time of detection. It is characterized by having a step of outputting information.

According to the present invention, it is possible to improve the accuracy of determining the presence or absence of an object.

It is a functional block diagram for demonstrating the function of an object detection system. It is a figure which showed the imaging range of the imaging apparatus, normal and abnormal captured images. It is a figure which shows the structure of the control part which comprises the image analysis part. It is a functional block diagram for explaining the learning process of machine learning of a learning model. It is a figure for demonstrating the process from the acquisition of the captured image to the determination of presence / absence of abnormality in an actual detection scene. It is a flowchart for demonstrating the machine learning operation at the time of learning. It is a flowchart for demonstrating the foreign matter determination process in a detection scene. It is a figure for demonstrating the processing process of a captured image. It is a figure for demonstrating the processing process of a captured image. It is a figure which showed the OK image and the NG image in the evaluation test of a learning model. It is a figure for demonstrating the outline of the learning model used in the evaluation test of a learning model. It is a figure which showed the evaluation result of the evaluation test of a learning model.

Hereinafter, the information transmission / reception system according to the embodiment of the present invention will be described with reference to FIGS. 1 to 12.

FIG. 1 is a functional block diagram for explaining a function of the object detection device 1 which is an embodiment of the present invention.

Hereinafter, preferred embodiments of the present disclosure will be described.
[System configuration]
FIG. 1 shows the configuration of the object detection system according to the embodiment. As shown in the figure, the object detection system 1 includes an image pickup device 10 and an image analysis device 20 as an object detection device.

[Imaging device]
The image pickup apparatus 10 includes a camera 11, a control unit 12, and a communication unit 13, and is a side sliding door (door) of a railroad vehicle such as a train or a train, for example, a double sliding door (door unit) consisting of two doors. The image around the sandwiched portion is imaged, and the captured image is output. Here, the periphery of the door sandwiching portion refers to the periphery of the door end rubber extending in the longitudinal direction in a state where the door end rubbers of the sliding doors of both sliding doors are in contact with each other. As shown in FIG. 1, the imaging range is a predetermined distance from the door end rubber region including each door end rubber of the door portion and the end portion of the platform separated from the door portion by a predetermined distance in the direction opposite to the door side. Areas up to (hatched areas in FIGS. 1 and 2). In addition to both sliding doors, a single sliding door may be used. The image pickup device 10 including the camera 11, the control unit 12, and the communication unit 13 corresponds to the image acquisition unit according to claim 1.

The camera 11 is movably arranged above the side sliding door and captures an image of the vicinity of the door sandwiching portion. The camera 11 is moved by an actuator (not shown) or the like. The movement of the camera 11 by the actuator and the imaging by the camera 11 are controlled by the control unit 12. If the camera position is fixed, it will not be possible to try various learning data including captured images, such as a picture seen from directly above and a picture in which the rubber part is more visible. The position is controlled so as to move not only in the left-right direction but also in the front-back direction with respect to the door.

As described above, the captured image captured by the camera 11 is the door from the door end rubber region (first region) including each door end rubber of the door portion and the end portion of the platform separated from the door portion by a predetermined interval. An image is taken of a region (third region (first region + second region)) that is a combination of regions (second regions) up to a predetermined distance in the direction opposite to the side, but the rubber part of the door tip, which will be described later, is cut out. The captured image is an image obtained by cutting out only the region of the door end rubber portion from the captured image of the third region described above by a predetermined image process. Further, when the captured image of the second region is acquired, it is an image obtained by cutting out the second region from the captured image of the third region by a predetermined image processing. Here, since the techniques of image processing for cutting out a specific image from the above-mentioned arbitrary image and image processing for performing rectangular shaping described later are known, the description thereof will be omitted here.

The control unit 12 controls the movement of the camera 11 and the imaging by the camera 11. When the control unit 12 acquires the captured image from the camera 11, the control unit 12 controls the communication unit 13 to supply (transmit) the captured image to the storage device 25 via the network 5. The captured image may be supplied to the learning image storage unit 38 that stores the learning image in the learning server 28, which will be described later, instead of the storage device 25.

The communication between the image pickup device 10 and the image analysis device 20 is established via the Internet as the network 5, for example, using Wi-Fi. Here, the network 5 is a concept including a base station and a relay station (not shown), and the communication method may satisfy that there is wide area connectivity and bidirectional communication is possible. However, the communication between the image pickup device 10 and the image analysis device 20 may be not only wireless communication but also wired communication, but in this case, the camera 11 is movably arranged above the side sliding door inside the vehicle. ..

[Image analyzer]
The image analysis device 20 determines whether or not there is a foreign substance in the vicinity of the door sandwiching portion by the correct answer rate (%) according to the learning model learned by the learning server 28 described later based on the captured image around the door sandwiching portion. .. The image analysis device 20 includes a control unit 22, a communication unit 24, a storage device 25, an input unit 26, a learning server 28, and a display unit 30. The image analysis device 20 corresponds to the image analysis unit, and the learning server 28 corresponds to the machine learning unit.

The storage device 25 is a non-volatile storage device composed of a hard disk, a flash memory, or the like, receives an captured image transmitted from the camera 11, and stores the received captured image. Further, when the storage device 25 receives the acquisition request of the captured image around the door sandwiching portion from the control unit 22, the storage device 25 transmits the captured image specified by the acquisition request to the control unit 22.

The communication unit 24 receives the captured image around the door sandwiching unit from the storage device 25 under the control of the control unit 22. The input unit 26 is a user interface that receives various operations from the user, and examples thereof include a keyboard, a mouse, a wheel device capable of rotating operations, a touch panel, a voice input device, and the like, which are used in a learning model construction process described later. The captured image around the door sandwiching portion, the processing of the captured image, the input signal for selecting the image, and the like are generated and supplied to the control unit 22. The display unit 30 is a user interface for displaying a screen for presenting various information to the inspector, and examples thereof include a liquid crystal display.

(Learning server)
The learning server 28 is configured to include a learning model storage unit 36 in which a learning model constructed by machine learning is stored, a deep learning engine 37, and a learning image storage unit 38 in which a learning image is stored, and is controlled. It is communicably connected to the unit 22, the communication unit 24, the input unit 26, and the display unit 30 via a bus line.

Here, the outline of the function of the deep learning engine 37 will be described. The deep learning engine 37 performs machine learning according to a learning model constructed by a neural network including, for example, a multi-layer neural network. A learning model constructed by a neural network including an input layer, an output layer, and an intermediate layer can use an appropriate method. For example, CNN (Convolutional Neural Net work) can also be applied.

In the present embodiment, the processed image to be described later, the determination result (teacher data) associated with the image, and the newly detected image, which are temporarily stored in the learning image storage unit 38, are stored. A learning model is trained based on the training data included. In other words, the deep learning engine 37 uses the newly detected imaging information as input data, and learns the learning model based on the input data and the above-mentioned teacher data.

Specifically, the learning image storage unit 38 sequentially reads out the processed image to be described later, and the read processed image is stored in the learning model storage unit 36. Fill in the model. Then, as the output, the parameters of the learning model are updated so that the information reflected in the processed captured image input to the learning model matches the correct answer information (information that there is no foreign matter in the door sandwiching portion). The parameters to be learned here are various coefficients that define a neural network that is a learning model. The correct answer information is input by the inspector via the input unit 26, is associated with the input processed image, and is stored in the learning image storage unit 38.

Then, in this way, the learning model is optimized while applying the teacher data one after another. Since various methods of supervised learning of neural networks are known, detailed description thereof will be omitted here. As a result, the learning process is completed and the learning model is derived. The derived learning model is stored in the learning model storage unit 36.

The deep learning engine 37 uses the captured image stored in the learning model storage unit 36 and loads it into the learning model (learning model optimized at that time) stored in the storage unit 36 to perform the input imaging. The normality / abnormality of the image (abnormality means a state in which a foreign object is caught in the door sandwiched portion) is calculated as the accuracy (correct answer rate: 0 to 100 [%]), and the calculated accuracy is output. The foreign matter determination unit 33 of the control unit 22 that receives the output outputs a normal result indicating that the inspection result is OK if the accuracy is equal to or higher than the predetermined threshold value, and the inspection result if the accuracy is less than the predetermined threshold value. An abnormal result indicating NG is output.

Further, the deep learning engine 37 associates the captured image with the above-mentioned accuracy and stores it in the learning image storage unit 38. It should be noted that the scene of the learning process (learning construction) of the learning model and the scene of using the learning model after learning at the time of actual foreign matter detection (scene of detection (foreign matter determination processing)) are independent. That is, in the learning process, the learning model is only constructed and updated, and the learning model after learning is used in the actual foreign matter determination processing scene.

(Control unit 22)
The control unit 22 includes a screen generation unit 31, a display control unit 32, a foreign matter determination unit 33, a ROM 34, a RAM 35, and the like. A program 40 for causing the screen generation unit 31, the display control unit 32, and the foreign matter determination unit 33 to execute various operations is stored in the ROM 34. The screen generation unit 31, the display control unit 32, and the foreign matter determination unit 33 virtually operate as the screen generation unit 31, the display control unit 32, and the foreign matter determination unit 33 by reading and executing the program 40 in the ROM 34. Details of the operations of the screen generation unit 31, the display control unit 32, and the foreign matter determination unit 33 will be described later. The program 40 may be stored in the storage device 25 instead of the ROM 34, or may be distributed and stored in both the storage device 25 and the ROM 34.

The screen generation unit 31 trims a portion unnecessary for determining foreign matter from the captured image around the door sandwiching portion, masks an area other than the door end rubber (background of the door end rubber), and has a trapezoidal door end rubber area. It has a function of converting (image processing) the image of the above into an image of a rectangular rubber area at the door end. By performing such image processing and training the learning model, it becomes easier to recognize the foreign matter caught in the door sandwiching portion, and the foreign matter detection accuracy and the foreign matter determination accuracy are improved. The display control unit 32 controls the display of the information supplied from the control unit 22 on the display unit 30.

The foreign matter determination unit 33 receives an image newly captured in the vehicle to which the camera 11 is attached in the actual foreign matter determination processing scene, and the foreign matter is generated according to the learning model previously constructed in the learning process by the deep learning engine 37. A side sliding door opening / closing control unit (not shown) that outputs a normal result indicating that the inspection result is OK or an abnormal result indicating an inspection result NG indicating that there is a foreign substance to the display unit 30 and executes opening / closing control of the side sliding door of the vehicle. ).

The learning process of machine learning the learning model and the foreign matter judgment process of outputting the judgment result according to the learning model after the learning model is constructed will be described below.

[Learning process of machine learning a learning model]
Hereinafter, the learning process of machine learning the learning model will be described with reference to FIGS. 4 and 6. The learning of the learning model is performed by the deep learning engine 37. First, the control unit 12 of the image pickup device 10 acquires a captured image of the area around the door sandwiching portion captured by the camera 11 and stores the captured image in the storage device 25 via the network 5 and the communication unit 24. The screen generation unit 31 processes the captured image stored in the storage device 25 by image processing described later.

(Processing of captured image)
In the processing of the captured image, the following four types of processing of (1) to (4) are applied to the captured image acquired from the storage device 25.
(1) Image processing is performed so as to cut out a region of the rubber part of the door end in the acquired captured image (see FIG. 8 (A-1)) (see FIG. 8 (A-2)).
(2) A region other than the region of the door end rubber portion cut out in (1) is masked with a predetermined background color and background pattern (see the central view of FIG. 8 (A-3)).
(3) The rubber portion of the door end whose background is masked in (2) is shaped into a rectangular shape (see the right figure of FIG. 8 (A-3)).
(4) Cut out a part of the entrance / exit area (hereinafter, referred to as "center of the entrance / exit") located on the side sliding door side of the platform in the acquired captured image (see FIG. 9 (A-1)). Is subjected to image processing (see FIG. 9 (A-2)). Next, at the end of the processing of (1) to (4), the processed captured image is temporarily stored in the learning image storage unit 38.

(Machine learning operation)
The machine learning operation executed by the image analysis device 20, that is, the learning operation of the image analysis device 20 at the time of learning will be described with reference to the flowchart of FIG.

In step S101, the deep learning engine 37 acquires the processed captured image from the captured image of the region around the door sandwiching portion. Here, the processed image is (1) an image in which the region of the door end rubber portion is cut out, and (2) a region other than the region of the door end rubber portion is masked with a predetermined background color and background pattern. The captured image, (3) the captured image in which the rubber part of the door end with the masked background is shaped into a rectangular shape, and (4) a part of the entrance / exit area located on the side sliding door side of the platform in the captured image is cut out. This is a captured image, which is temporarily stored in the storage device 25.

In step S102, the input unit 26 inputs information (foreign matter presence / absence information) as to whether or not a foreign matter is caught in the area around the door sandwiching portion based on the acquired processed image. Here, the foreign matter presence / absence information refers to information indicating that there is no foreign matter and the inspection result is OK (information indicating a normal result) and information indicating that there is a foreign matter and the inspection result is NG (information indicating an abnormal result).

In step S103, the control unit 22 stores the foreign matter presence / absence information in the learning image storage unit 38 as teacher data in association with the acquired captured image. The deep learning engine 37 may be instructed to store the teacher data in the learning image storage unit 38.

In step S104, the deep learning engine 37 takes the captured image of the processed door sandwiching portion peripheral region as an input, learns based on the teacher data, builds a learning model, and updates the above-mentioned parameters of the learning model.

[Foreign matter judgment processing when an object is detected]
After the above-mentioned machine learning is completed, the foreign matter determination process for determining whether or not a foreign matter is caught in the door sandwiching portion of the side sliding door of the vehicle during the time when the vehicle is actually operating will be described with reference to the flowchart of FIG. To do.

In step S201, the control unit 22 acquires the processed captured image from the captured image of the region around the door sandwiching portion. The processed captured image is an image captured by the camera 11 while the vehicle is in operation, processed by the above-mentioned processing, and is temporarily stored in the storage device 25.

In step S202, the control unit 22 reads out the learning model stored in the learning model storage unit 36, inputs the processed captured image around the door sandwiching portion to the deep learning engine 37, and the foreign matter determination unit 33 is deep. Based on the correct answer rate information output from the learning engine 37, a determination regarding the presence or absence of an abnormality is performed, and the determination result is output to the display unit 30. The determination result output to the display unit 30 refers to correct answer information, which may be indicated by the correct answer rate (%), or may be indicated by OK (normal) or NG (abnormal). Further, not only the above determination result but also the captured image which is the target of the determination result is displayed on the display unit 30.

[Learning model evaluation test]
Below, FIG. 13 shows three types of learning models using each of the captured images subjected to the above-mentioned three types of processing. As shown in FIG. 13, there are three types of learning models used in this evaluation test: a door tip rubber cutout model, a door tip rubber mask model, and a door tip rubber rectangular model. The door end rubber portion cut-out model is obtained by cutting out the circumscribed rectangle of the door tip rubber portion and then reducing the size to a predetermined model size for learning. The circumscribing rectangle is the smallest rectangle that can include a contour.

In the door tip rubber part mask model, after cutting out with the circumscribing rectangle of the door tip rubber part, the area other than the door tip rubber part (background of the door tip rubber part) is masked in gray, and it is made into a predetermined model size. It was reduced and learned. In the door tip rubber rectangular model, the area of the door tip rubber portion is corrected to a rectangle by adjusting the width to a predetermined model size, and further reduced to a predetermined model size for learning. The model size after the reduction in this evaluation is 80 pixels, 96 pixels, 128 pixels, and 224 pixels.

The abnormality determination process is executed by applying the above-mentioned three types of learning models, and the evaluation results of each are shown in FIGS. 14 and 15. In the evaluation test, the learning images to be used in the main evaluation test are read from the learning image storage unit 38, the total number of the read learning images, the number of captured images with no foreign matter and the inspection result is normal (OK). , The number of captured images for which there is a foreign substance and the inspection result is abnormal (NG), the number of images for which "normal" is determined to be normal (first normal number), and the number of images for which "abnormal" is determined to be normal (first) (Abnormal number of images), "normal" is determined to be abnormal (second abnormal number of sheets), "abnormal" is determined to be abnormal (second normal number of sheets), and the above first with respect to the total number of images. The ratio of the normal number of sheets and the added number of the second normal number of sheets was calculated as the correct answer rate (%), and the target learning model was evaluated. 15 to 17 show the evaluation results of the learning model having a correct answer rate of 90% or more.

<Evaluation result of rubber cut-out model at door end>
(Example 1)
Model size 80 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" judged normal 79, "abnormal" judged normal 3 The number of sheets for which "normal" was determined to be abnormal was 29, the number of sheets for which "abnormal" was determined to be abnormal was 1779, and the correct answer rate was 98.3%.

(Example 2)
Model size 96 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" determined to be normal 108, "abnormal" determined to be normal 1 The number of sheets determined to be "normal" as abnormal was 0, the number of sheets determined to be "abnormal" was 1781, and the correct answer rate was 99.9%.

(Example 3)
Model size 128 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" determined to be normal 108, "abnormal" determined to be normal 3 The number of sheets determined to be "normal" as abnormal was 0, the number of sheets determined to be "abnormal" was 1779, and the correct answer rate was 99.8%.

(Example 4)
Model size 224 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" judged normal 107, "abnormal" judged normal 1 The number of sheets determined to be "normal" as abnormal was 1, the number of sheets determined to be "abnormal" was 1781, and the correct answer rate was 99.9%.

As is clear from the above evaluation results, the door-end rubber part cut-out model is a model obtained by cutting out the circumscribing rectangle of the door-end rubber part and then reducing it to a predetermined model size for learning. Since it is an image that is specialized in the area around the rubber part of the door compared to the captured image, it is easy to discriminate foreign matter and the time required for the discriminating process can be shortened.

<Evaluation result of door-end rubber mask model>
(Example 1)
Model size 96 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" determined to be normal 0, "abnormal" determined to be normal 0 The number of sheets for which "normal" was determined to be abnormal was 108, the number of sheets for which "abnormal" was determined to be abnormal was 1782, and the correct answer rate was 94.3%.

Here, in the determination of the presence or absence of foreign matter using an image of the entire side sliding door, the color and texture of the vehicle may change according to the vehicle type, which may adversely affect the determination, resulting in a decrease in determination accuracy. On the other hand, in the door tip rubber mask model, after cutting out with the circumscribing rectangle of the door tip rubber part, the area other than the door tip rubber part is masked with gray or the like, and it is reduced to a predetermined model size for learning. Therefore, since it is not affected by the above-mentioned adverse effects other than the region of the rubber portion of the door end, it becomes easier to recognize the presence or absence of foreign matter and erroneous determination can be suppressed.

<Evaluation result of door-end rubber rectangular model>
(Example 1)
Model size 96 pixels, total number of images 3842, OK number 1879, NG number 1963, "normal" judged normal 1876, "abnormal" judged normal 0 , The number of sheets determined to be "normal" as abnormal was 3, and the number of sheets determined to be "abnormal" was 1963, and the correct answer rate was 99.9%.

(Example 2)
Model size 128 pixels, total number of images 1890, OK number 108, NG number 1782, "normal" determined to be normal 108, "abnormal" determined to be normal 3 The number of sheets determined to be "normal" as abnormal was 0, the number of sheets determined to be "abnormal" was 1779, and the correct answer rate was 99.8%.

As is clear from the above evaluation results, in the door-end rubber rectangular model, the area of the door-end rubber portion is corrected to a rectangle by adjusting the width to a predetermined model size, and further reduced to a predetermined model size for learning. Therefore, since the image as it actually looks is used as compared with the image obtained by capturing the entire side sliding door, it is easy to discriminate the foreign matter and the time required for the discriminating process can be shortened.

As described above, according to the above-described embodiment, as the learning model used for determining the foreign matter, the door end rubber is cut out from the circumscribing rectangle of the door end rubber portion and then reduced to a predetermined model size for learning. Cut-out model, door-end rubber mask model, door-end rubber part that was learned by masking the area other than the door-end rubber part and reducing it to a predetermined model size after cutting out with the circumscribing rectangle of the door-end rubber part. Since we are using a door-end rubber rectangular model that has been trained by adjusting the width of the area to a predetermined model size and reducing it to a predetermined model size, the direction and conditions for imaging the object Because it is not easily affected by the difference in the appearance of the object in the captured image due to the above, it is possible to suppress the decrease in the accuracy of the image analysis and improve the accuracy of determining the presence or absence of the object.

The series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software can execute various functions by installing a computer embedded in dedicated hardware or various programs. It is installed from a program recording medium on a possible, eg, general purpose personal computer, tablet device, or smartphone.

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be a program that is processed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

Further, the embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

1 ... Object detection system, 5 ... Network, 10 ... Imaging device, 11 ... Camera, 12 ... Control unit, 13 ... Communication unit, 20 ... Image analysis device, 22 ... Control unit, 24 ... Communication unit, 25 ... Storage device , 26 ... Input unit, 28 ... Learning server, 30 ... Display unit, 31 ... Image generation unit, 32 ... Display control unit, 33 ... Foreign matter determination unit, 34 ... ROM, 35 ... RAM, 36 ... Learning model storage unit, 37 ... Deep learning engine, 38 ... Learning image storage, 40 ... Program

Claims (6)

An object detection device having an image analysis unit that detects an object in the area around the door sandwiching portion based on an image captured in the area around the door sandwiching portion of the side sliding door of the vehicle acquired by the image acquisition unit.
The image analysis unit
At the time of learning, the learning model is learned by inputting the captured image of the area around the door sandwiching portion and using the captured image and the information on the presence or absence of foreign matter contained in the captured image as teacher data.
At the time of detection, it has a machine learning unit that inputs an image of the area around the door sandwiched portion and outputs information on the presence or absence of foreign matter contained in the image.
An object detection device characterized by this. The image analysis unit has an image generation unit that performs image processing so as to cut out a region of the door end rubber portion in the captured image acquired by the image acquisition unit.
The object detection device according to claim 1. The image analysis unit cuts out a region of the door end rubber portion in the captured image acquired by the image acquisition unit, and masks an region other than the cut out region of the door end rubber portion with a predetermined background color or background pattern. It has an image generation unit that performs image processing so as to do so.
The object detection device according to claim 1. The image analysis unit cuts out a region of the door end rubber portion in the captured image acquired by the image acquisition unit, and masks an region other than the cut out region of the door end rubber portion with a predetermined background color or background pattern. It has an image generation unit that performs image processing so as to shape the masked rubber portion of the door end into a rectangular shape.
The object detection device according to claim 1. The object detection device according to any one of claims 1 to 4 is provided, and the object detection device is provided.
It has the image acquisition unit that acquires an image of the area around the door sandwiching portion from an imaging device arranged so that the area around the door sandwiching portion of the side sliding door of the vehicle can be imaged.
An object detection system characterized by this. A step of acquiring an image of the area around the door sandwiching part from an imaging device arranged so that the area around the door sandwiching part of the side sliding door of the vehicle can be imaged.
A step of detecting an object in the area around the door sandwiching portion based on an image captured in the area surrounding the door sandwiching portion, and
It is an object detection method having
At the time of learning, the step of learning the learning model by inputting the captured image of the area around the door sandwiching portion and using the captured image and the information on the presence or absence of foreign matter contained in the captured image as teacher data.
At the time of detection, it has a step of inputting an image captured in the area around the door sandwiching portion and outputting information on the presence or absence of foreign matter contained in the captured image.
An object detection method characterized by this.