JP7097215B2 - Vehicle estimation device, learning device, and vehicle estimation method - Google Patents

Description

The present invention relates to a vehicle estimation device, a learning device, and a vehicle estimation method for managing vehicles entering and exiting at a work site.

Conventionally, various vehicles such as vehicle-type construction equipment, vehicles for carrying in construction materials, and vehicles for carrying out industrial waste frequently come and go at work sites such as construction sites. In particular, a concrete mixer truck (hereinafter referred to as a ready-mixed concrete truck) is used to place concrete on the work site, and a dump truck is used to carry out excavated soil from the work site cyclically at intervals of several minutes. The number of vehicles entering or exiting (hereinafter referred to as "entrance / exit") reaches several hundreds per gate during peak hours.
For ready-mixed trucks and dump trucks, payment will be made based on the number of vehicles that have entered the work site. Therefore, for example, a guard man at the work site records the number of vehicles such as a ready-mixed concrete truck that has entered the work site, and is used to confirm the settlement cost.
In addition, vehicles may stop near the work site and stand by in order to enter the work site, which is one of the causes of traffic congestion in the surrounding roads.
For this reason, there were cases where the government requested a report on the number of vehicles entering and leaving the work site because it could be a factor. For example, as a method of managing such vehicles entering and exiting a parking lot, a technique of recognizing a vehicle number plate using an image of a vehicle entering and exiting a parking lot by a camera is disclosed (for example, Patent Document). 1).

Japanese Unexamined Patent Publication No. 2015-197780

However, in order to recognize the number plate and the vehicle number and manage the vehicle as in the above-mentioned conventional technique, it is necessary to register the vehicle number in advance, and the work of registering is troublesome. In addition, among the vehicles entering and leaving the work site, the repeat rate of the same vehicle is low for vehicles other than dump trucks, ready-mixed trucks, refueling vehicles, and industrial waste collection vehicles, and the vehicle with the vehicle number once registered is the same work site again. It is rare to enter. For this reason, while it takes time and effort to register the vehicle number in advance, unregistered vehicles enter one after another, and it is difficult to continue the operation of vehicle management.
In addition, in a relatively small work site, the vehicle often enters the work site while moving backward, and if it is a prerequisite to recognize the number plate provided on the front part of the vehicle, the rear part of the vehicle. It was difficult to recognize the number plate provided in.

In view of the above problems, the present invention provides a vehicle estimation device, a learning device, and a vehicle estimation method that can manage vehicles entering and exiting at a work site without increasing labor.

In order to solve the above-mentioned problems, the vehicle estimation device according to one aspect of the present invention is image data of an image of at least one of a vehicle entering the work site or a vehicle leaving the work site. The image data acquisition unit for acquiring the image and the estimation unit for estimating the attribute information about the vehicle for the vehicle in the image by inputting the image data of the image into the trained model for estimating the attribute information about the vehicle. The trained model corresponds to the training data in which the vehicle type of the vehicle is associated with the image captured by the vehicle extracted from the image data, and the image obtained by capturing the load extracted from the image data. It is a vehicle estimation device created by executing deep learning using learning data associated with the names of loads of vehicles .

Further, one aspect of the present invention is the vehicle estimation device described above, wherein the estimation unit estimates at least one of the vehicle type of the vehicle in the image and the load of the vehicle in the image.

Further, one aspect of the present invention is the vehicle estimation device described above, wherein the image is an image of at least one of a vehicle entering the work site or a vehicle leaving the work site. An image of the front part of the vehicle including the front part which is the front side in the traveling direction of the vehicle, and an image of the front part of the vehicle including the end portion in the direction of the front part and all or a part of the load of the vehicle. A vehicle rear image including all or a part of the load of the vehicle and an end in the direction of the rear which is the rear side in the traveling direction of the vehicle, and a package image including the entire load of the vehicle. , At least one of the vehicle rear images including the rear.

Further, in one aspect of the present invention, a region where the vehicle enters and exits at the work site is set as an imaging region by performing deep learning using learning data in which the vehicle and the attribute information related to the vehicle are associated with each other. It is a learning device provided with a model creation unit for creating an estimation model for estimating the attribute information in a vehicle in the image data captured by the user , and the estimation model corresponds to an image captured by the vehicle extracted from the image data. Deep learning is executed using learning data in which the vehicle type of the vehicle is associated with each other and learning data in which the name of the load of the vehicle is associated with the image obtained by capturing the load extracted from the image data. It is a learning device that is created by the above.

Further, one aspect of the present invention is the above-mentioned learning device, which evaluates a distribution unit that distributes each of the training data into training data and test data, and an estimation model created by the model creation unit. An evaluation unit is further provided, and the model creation unit creates an estimation model by executing deep learning using the training data distributed by the distribution unit, and the evaluation unit creates the test data in the estimation model. By inputting the image corresponding to, the estimation result of estimating the attribute information of the vehicle in the image corresponding to the test data is acquired, and the acquired estimation result and the attribute information of the image corresponding to the test data are obtained. Evaluate the estimation model based on.

Further, one aspect of the present invention is the vehicle estimation method described above, in which the image data acquisition unit acquires image data of an image in which the area where the vehicle enters and exits at the work site is the image pickup area, and the estimation unit obtains the image data. By inputting the image data of the image into the trained model for estimating the attribute information about the vehicle, the attribute information about the vehicle is estimated for the vehicle in the image, and the trained model gives the vehicle the attribute information. It is created by executing deep learning using the training data corresponding to the above, and the trained model corresponds to the vehicle type of the vehicle to the image captured by the vehicle extracted from the image data. It is created by executing deep learning using the attached learning data and the learning data in which the name of the load of the vehicle is associated with the image captured by the load extracted from the image data. It is a vehicle estimation method characterized by being.

According to the present invention, since the attribute information such as the vehicle type and the load of the vehicle entering and leaving the work site is estimated, it is possible to manage the vehicles entering and leaving the work site without increasing the time and effort.

It is a figure which shows the example which applied the vehicle detection system 30 of an embodiment to a work site 1. It is a block diagram which shows the structural example of the vehicle estimation apparatus 40 of an embodiment. It is a block diagram which shows the structural example of the trained model storage device 50 of embodiment. It is a block diagram which shows the structural example of the learning apparatus 60 of an Embodiment. It is a figure which shows the example of the image used in the vehicle detection system 30 of an embodiment. It is a flowchart which shows the operation example of the learning apparatus 60 of an embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a diagram showing an example in which the vehicle detection system 30 of the embodiment is applied to the work site 1. As shown in FIG. 1, the vehicle detection system 30 is applied to the work site 1. The work site 1 is provided with, for example, an entrance / exit gate 10 and an image pickup device 20.

The entrance / exit gate 10 is provided at the entrance / exit of the work site 1. Panel gates 11 provided so as to surround the work site 1 are provided at both ends of the entrance / exit gate 10. In the entrance / exit gate 10, for example, a gate frame is formed by columns 12 (supports 12a and 12b) provided on the left and right sides of the entrance / exit and a horizontal member 13 erected on the upper side between the columns 12. For example, a caster gate 14 having an upper end suspended from a horizontal member 13 and which can be opened and closed by running a wheel (caster) provided at the lower end is provided along the framework of the gate.

The image pickup apparatus 20 is installed near the center of the horizontal member 13 in the length direction, for example. The image pickup device 20 is installed so that the image pickup direction faces downward, for example, and captures an image of a bird's-eye view of the vehicle 15 (vehicles 15a, 15b) entering and exiting the entrance / exit gate 10.
The image pickup device 20 is, for example, an omnidirectional camera capable of shooting at an angle of view of 360 degrees. For example, when the vehicle 15 passes through the entrance / exit gate 10, the image pickup apparatus 20 captures an image of a bird's-eye view of the passing vehicle 15. The image pickup device 20 may capture a still image or may capture a moving image. Further, the image pickup apparatus 20 may capture a black-and-white image or may capture a color image.
The image pickup apparatus 20 transmits the image data of the captured image to the vehicle detection system 30.

The vehicle detection system 30 detects the attribute information of the vehicle 15 in the image based on the image data of the image captured by the image pickup device 20. Here, the attribute information of the vehicle 15 is information indicating attributes related to the vehicle, and is, for example, information indicating a vehicle type, a load, an operation, and the like. The operation here is information indicating whether the vehicle 15 has entered the work site 1 or has left the work site 1.

The vehicle detection system 30 includes, for example, a vehicle estimation device 40, a trained model storage device 50, and a learning device 60.
The vehicle estimation device 40 estimates the attribute information of the vehicle in the image based on the image data of the image captured by the image pickup device 20. The vehicle estimation device 40 uses the trained model stored in the trained model storage device 50 when estimating the attribute information of the vehicle.
The trained model here is a model created by executing deep learning using training data showing the correspondence between the vehicle captured in the image as an input and the attribute information of the vehicle as an output. ..

Here, deep learning will be described.
First, machine learning, which is the basis of deep learning, finds the regularity and relevance common to the data group from a large amount of data, and makes judgments and predictions for the data based on the regularity and relevance. It is a method. In machine learning, humans specify features (features) that should be noted for data groups. Therefore, it is necessary to appropriately specify the feature amount after fully considering the relationship between the content of the data (learning data) used by humans for learning and the judgment and prediction used as the learning result.
On the other hand, in the case of deep learning, it is not necessary for humans to specify the features. This is because in deep learning, artificial intelligence thinks and decides the setting and combination of features by using a multi-layered structure algorithm (also called deep neural network, artificial intelligence, etc.) modeled on the human brain neural circuit. .. For example, when trying to estimate the vehicle type using an image of a vehicle as learning data, in the case of machine learning, various feature quantities that humans can think of, such as color and shape, are used as feature quantities. It must be specified, and which feature quantity is effective for estimating the vehicle type cannot be known without performing machine learning. On the other hand, in deep learning, the feature amount that is effective for estimating the vehicle type is considered and determined by oneself, and the learning is automatically performed by oneself. However, in deep learning, it is necessary to read (learn) a large amount of learning data in order for artificial intelligence to determine effective features for estimation. In addition, since the direction of learning changes depending on the content of the learning data to be read, it is necessary to carefully select the learning data.
However, in order to increase the number of images for training data in order to improve the quality of training data, it is not a good idea to blindly increase the number of images for training data because labor costs and other costs are incurred as a practical problem. ..
Therefore, in the present embodiment, a certain amount of images for learning data (for example, 100 images) are prepared, and deep learning is executed using the prepared learning data. Then, after determining the direction of the estimation model by adjusting the variables used when performing deep learning, by increasing the training data along that direction, an efficient and highly accurate estimation model is created. do.

For example, when the same input as the trained input is made, the trained model outputs the output corresponding to the trained input as the estimated vehicle attribute information.
The trained model storage device 50 stores the trained model. The learning device 60 creates a trained model to be stored in the trained model storage device 50.

The vehicle detection system 30 uses the attribute information of the vehicle estimated by the vehicle estimation device 40 as the detection result. The vehicle detection system 30 outputs the detection result to, for example, the display device 70.
The display device 70 outputs the detection result by the vehicle detection system 30. In the example of FIG. 1, the display device 70 indicates that, for the vehicle 15a, the vehicle type (vehicle) is a dump vehicle, the load (load) is excavated soil, and the operation of the vehicle is exit. Further, the display device 70 indicates that the vehicle type of the vehicle 15b is a ready-mixed concrete truck, the load is ready-mixed concrete (ready-mixed concrete), and the operation of the vehicle is admission.

FIG. 2 is a block diagram showing a configuration example of the vehicle estimation device 40 of the embodiment. The vehicle estimation device 40 includes, for example, an image data acquisition unit 41, a preprocessing unit 42, a vehicle type estimation unit 43, a load estimation unit 44, an estimation result output unit 45, and an estimation result storage unit 46. .. Here, the vehicle type estimation unit 43 and the load estimation unit 44 are examples of the “estimation unit”.

The image data acquisition unit 41 acquires image data of an image captured by the image pickup device 20. The image data acquisition unit 41 outputs the acquired image data to the preprocessing unit 42.
The pre-processing unit 42 performs pre-processing on the image corresponding to the image data acquired by the image data acquisition unit 41. The pre-processing here is a process of processing an image so that the processing by the vehicle type estimation unit 43 and the load estimation unit 44, which will be described later, can be performed correctly.
The preprocessing unit 42, for example, extracts various objects captured in the image from the image. The preprocessing unit 42 extracts various objects captured in the image by, for example, performing edge detection. The preprocessing unit 42 detects an edge surrounding the object based on the difference in luminance at the outer circumference of the edge. Specifically, the preprocessing unit 42 fits various rectangular frames (cells) into the image, compares the brightness between the inside and the outside of the frame, and compares the brightness distribution inside the frame with the brightness distribution on the outside of the frame. When the difference in brightness is small, it is determined that there is an object in the frame, and the object is extracted from the image by extracting the frame.

For example, the preprocessing unit 42 compares the images in time series, and selects an object that changes in time series from the objects extracted from each of the images. As already described, the image here is an image of the vehicle 15 entering and exiting the work site 1. Therefore, when an object that changes in time series is captured in the image, it is considered that the changing object is the vehicle 15 that enters and exits. That is, the preprocessing unit 42 selects the vehicle 15 that is the estimation target in the image and the load of the vehicle 15 by selecting the object that changes in time series.
Further, the preprocessing unit 42 scales the image of the vehicle 15 extracted from the image and the image of the load of the vehicle 15 to have the same size, for example.
The preprocessing unit 42 outputs the image data of the vehicle 15 extracted from the image and the image of the load of the vehicle 15 to the vehicle type estimation unit 43 and the load estimation unit 44.

The vehicle type estimation unit 43 estimates the vehicle type of the vehicle in the image based on the image data of the image in which the vehicle is extracted by the preprocessing unit 42. The vehicle type estimation unit 43 outputs the output obtained by inputting image data to the trained model stored in the vehicle type trained model storage unit 51 (see FIG. 3) of the trained model storage device 50 as a vehicle in the image. It is estimated to be the vehicle type of. The vehicle type estimation unit 43 outputs the estimation result to the estimation result output unit 45 and stores the estimation result in the estimation result storage unit 46.

The load estimation unit 44 estimates the load of the vehicle in the image based on the image data of the image from which the load is extracted by the preprocessing unit 42. The load estimation unit 44 outputs the output obtained by inputting image data to the trained model stored in the load trained model storage unit 52 (see FIG. 3) of the trained model storage device 50 as a vehicle in the image. Estimated to be the load of. The load estimation unit 44 outputs the estimation result to the estimation result output unit 45 and stores the estimation result in the estimation result storage unit 46.

The estimation result output unit 45 outputs information indicating the vehicle type and load of the vehicle in the image estimated by the vehicle type estimation unit 43 and the load estimation unit 44 to the display device 70 in association with the image.
The estimation result storage unit 46 stores information in which the vehicle type and load of the vehicle in the image estimated by the vehicle type estimation unit 43 and the load estimation unit 44 are associated with the image.

FIG. 3 is a block diagram showing a configuration example of the trained model storage device 50 of the embodiment. The trained model storage device 50 includes, for example, a vehicle type trained model storage unit 51 and a load-learned model storage unit 52.
The vehicle type trained model storage unit 51 stores a trained model that estimates the vehicle type based on the vehicle captured in the image. This trained model is a model created by executing deep learning on a vehicle captured in an image using learning data associated with the vehicle type.

The load-learned model storage unit 52 stores a trained model that estimates the load based on the load of the vehicle captured in the image. This trained model is a model created by performing deep learning using the learning data in which the name of the load is associated with the load of the vehicle captured in the image.
The trained model stored in each of the vehicle type trained model storage unit 51 and the load trained model storage unit 52 described above is created by, for example, the learning device 60.

FIG. 4 is a block diagram showing a configuration example of the learning device 60 of the embodiment. The learning device 60 includes, for example, an image data acquisition unit 61, an area image extraction unit 62, a vehicle type learning data creation unit 63, a load learning data creation unit 64, a learning data distribution unit 65, and a vehicle type learning model. It includes a creation unit 66, a load learning model creation unit 67, and a learning model evaluation unit 68. Here, the vehicle type learning model creation unit 66 and the load learning model creation unit 67 are examples of the “model creation unit”. The learning data distribution unit 65 is an example of the “distribution unit”. The learning model evaluation unit 68 is an example of the “evaluation unit”.

The image data acquisition unit 61 acquires image data of an image captured by the vehicle 15. The image data acquisition unit 61 may acquire not only the image of the vehicle 15 entering and exiting the work site 1 captured by the image pickup device 20, but also the image captured at another work site. When the image data acquisition unit 61 acquires an image at another work site, it is preferable that the image pickup position and angle of view in the image are the same as the image pickup position and angle of view of the image pickup device 20 at the work site 1. If the imaging conditions such as the imaging position and the angle of view are the same as the imaging conditions of the imaging device 20, a trained model for accurately estimating the attribute information of the vehicle 15 is created based on the image captured by the imaging device 20. Because it can be done. The image data acquisition unit 61 outputs the acquired image data to the area image extraction unit 62.

The area image extraction unit 62 uses an image corresponding to the image data acquired by the image data acquisition unit 41 to detect an object having a size equivalent to that of a vehicle by detecting an object using a technique such as edge detection performed by the preprocessing unit 42. By assuming that the data is extracted, the vehicle area in which the vehicle to be detected is imaged and the load area in which the load is imaged are extracted. The area image extraction unit 62 selects a vehicle or a load to be learned from the objects extracted from the image.
The area image extraction unit 62 outputs the image data of the image extracted from the vehicle area to the vehicle type learning data creation unit 63. Further, the area image extraction unit 62 outputs the image data of the image obtained by extracting the load area to the load learning data creation unit 64.

The vehicle type learning data creation unit 63 associates the vehicle type with the image extracted by the area image extraction unit 62 to learn the correspondence between the vehicle captured in the image and the vehicle type. create. The vehicle type learning data creation unit 63 associates a vehicle captured in an image with a vehicle type of the vehicle based on, for example, an operation of an input device such as a keyboard or a mouse by an operator. In this case, the vehicle type learning data creation unit 63 acquires information indicating an operation of a mouse or the like by an operator input to an input unit (not shown), and associates the vehicle captured in the image with the vehicle type. The vehicle type learning data creation unit 63 outputs the created learning data to the learning data distribution unit 65.

By associating the name of the load with the image from which the load area is extracted by the area image extraction unit 62, the load learning data creation unit 64 assigns the vehicle load captured in the image and the name of the load. Create learning data to learn the correspondence of. The load learning data creation unit 64 associates a vehicle captured in an image with a vehicle type of the vehicle based on, for example, an operation of an input device such as a keyboard or a mouse by an operator. In this case, the load learning data creation unit 64 acquires information indicating an operation of a mouse or the like by an operator input to an input unit (not shown), and associates the vehicle captured in the image with the vehicle type of the vehicle. .. The load learning data creation unit 64 outputs the created learning data to the learning data distribution unit 65.

The learning data distribution unit 65 distributes the learning data created by the vehicle type learning data creation unit 63 into training data and test data. Further, the learning data distribution unit 65 distributes the learning data created by the load learning data creation unit 64 into training data and test data. Here, the training data is data used when a trained model is created by the vehicle type learning model creation unit 66 and the load learning model creation unit 67, which will be described later. The test data is data used for evaluating the trained model created by the vehicle type learning model creation unit 66 and the load learning model creation unit 67.

The learning data distribution unit 65 may divide the learning data created by the vehicle type learning data creation unit 63 into two equal parts, training data and test data, or may distribute them at a predetermined ratio. For example, the learning data distribution unit 65 distributes the attribute information associated with the image of the learning data created by the vehicle type learning data creation unit 63 so that both the training data and the test data include the same amount. Specifically, when the learning data includes two learning data in which the vehicle 15 captured in the image is associated with the vehicle type "raw computer" as its attribute information, one of the learning data is training. Allocate the remaining one to the data and the remaining one to the test data.

The learning data distribution unit 65 outputs the training data distributed from the learning data created by the vehicle type learning data creation unit 63 to the vehicle type learning model creation unit 66. Further, the learning data distribution unit 65 outputs distribution training data from the learning data created by the load learning data creation unit 64 to the load learning model creation unit 67.
The learning data distribution unit 65 does not show each of the test data distributed from the learning data created by the vehicle type learning data creation unit 63 and the test data distributed from the learning data created by the load learning data creation unit 64. Store in the storage unit.

The vehicle type learning model creation unit 66 creates a trained model by executing deep learning using the training data for learning the vehicle type output by the training data distribution unit 65.
The load learning model creation unit 67 creates a trained model by executing deep learning using the training data for learning the load output by the training data distribution unit 65.

The learning model evaluation unit 68 evaluates the trained model created by each of the vehicle type learning model creation unit 66 and the load learning model creation unit 67. For example, the learning model evaluation unit 68 inputs the output obtained by inputting the image of the test data distributed by the learning data distribution unit 65 into the trained model, and the attribute information associated with the image of the test data. Evaluate the trained model by comparing.

For example, if the 15 of the vehicles captured in the image have test data to which the vehicle type "raw computer" is associated as the attribute information, the learning model evaluation unit 68 may perform an image of the test data (captured by the vehicle 15). (Image) is input to the trained model that estimates the vehicle type. In this case, if the trained model has learned about an image of a vehicle that matches or is similar to the vehicle 15 captured in the input image, the vehicle type corresponding to the trained image. Is output.

When the learning model evaluation unit 68 outputs the vehicle type "ready-mixed concrete vehicle" corresponding to the image of the test data (the image in which the vehicle 15 is captured) from the trained model as the estimated value of the vehicle type. Evaluate that the estimation result is correct. On the other hand, the learning model evaluation unit 68 has a vehicle type (for example, a "dump vehicle") different from the vehicle type "ready-conducted vehicle" corresponding to the image of the test data (the image in which the vehicle 15 is captured) from the trained model. ”) Is output as an estimated value for each vehicle type, it is evaluated that the estimated result is not correct.

Here, the attribute information of the vehicle will be described.
Vehicle types include, for example, dump vehicles, ready-mixed vehicles, paraceal vehicles, trucks, unic, trailers, pump vehicles, rough terrain cranes, sludge carriers, industrial waste / garbage collection vehicles, refueling vehicles, vacuum vehicles, freight vehicles, and passenger vehicles. And so on.
Loads include, for example, PCa columns, PCa beams, PCa outer walls, steel columns, steel beams, steel stairs, reinforcing bars, formwork, decks, ALC, boards, glass / aluminum fittings, air conditioning equipment, sanitary equipment, electrical equipment, etc. There are temporary construction, excavated soil, etc.
The above-mentioned attribute information (vehicle type and load) is only an example, and the attribute information detected by the vehicle detection system 30 may be arbitrarily set according to the scale and situation of the work site 1.

Here, an image in which a vehicle or a load is captured will be described with reference to FIG.
FIG. 5 is a diagram showing an example of an image used in the vehicle detection system 30 of the embodiment. FIG. 5A is a plan view of the vehicle 15 from a bird's-eye view. 5 (b) to 5 (f) show an example when a vehicle or a load is present in a predetermined imaging region P. In FIGS. 5 (b) to 5 (f), the upper side of the imaging region P shows the inner side of the work site 1, and the lower side of the imaging region P shows the outer side of the work site 1. That is, FIGS. 5 (b) to 5 (f) show time-series changes in the imaging region P when the vehicle 15 enters the work site 1.

As described above, in the vehicle detection system 30 of the embodiment, the attribute information of the vehicle is detected based on the image of the vehicle or the load. For this reason, it is preferable that the image captures the vehicle or the load so that it can be detected. Further, since the detection (estimation) of the property information of the vehicle is performed by inputting an image to the trained model, the image used for training and the image to be estimated are the same under the same imaging conditions. It is preferable that the vehicle or load to be estimated is imaged in a certain area. Therefore, in the present embodiment, an image in which a part of the vehicle 15 is captured in a predetermined region in the imaging region P as shown in FIGS. 5 (b) to 5 (f) is acquired.

Here, it is premised that the image pickup apparatus 20 captures an image from a position overlooking the vehicle 15. Further, the image pickup apparatus 20 takes an image so that the entrance / exit gate 10 is located in the left-right direction in the center of the image.

Further, when the vehicle 15 passes through the entrance / exit gate 10, the image pickup apparatus 20 determines whether or not a part of the passing vehicle 15 has entered a predetermined area, and a part of the vehicle 15 has entered a predetermined area. The image is taken so that it is imaged in. Specifically, the image pickup apparatus 20 includes, for example, an image pickup determination unit (not shown). The image pickup determination unit is, for example, from the boundary line S in the left-right direction in the center of the image pickup area P, the predetermined area E1 above the boundary line S, and the boundary line S as shown in FIGS. 5 (b) to 5 (f). A predetermined area E2 is set on the lower side. The image pickup determination unit monitors the image pickup area P in time series and detects whether or not there is a change in the area E1 or the area E2. When there is a predetermined change in the area E1 or the area E2, the image pickup determination unit determines that the vehicle 15 has entered or exited, and causes the vehicle 15 to take an image of the entry or exit of the vehicle 15 in chronological order.

As shown in FIG. 5A, when the vehicle 15 is viewed from a bird's-eye view, the vehicle 15 is classified into a front end 150, a front portion 151, a loading portion 152, a rear portion 153, and a rear end 154. be able to.
FIG. 5B shows how the front portion 151, which is the front side in the traveling direction of the vehicle 15, enters the entrance / exit gate 10 in the imaging region P. The image pickup determination unit determines that the vehicle 15 has entered, for example, when there is a change in a predetermined ratio (for example, 30%) in the region E1. When the image pickup determination unit determines that the vehicle 15 has entered the entrance / exit gate 10, the image pickup determination unit 20 captures an image captured at a predetermined time (for example, 5 seconds) before the determination time from the image pickup device 20 to the vehicle detection system 30. To send to. In this case, the image transmitted by the image pickup apparatus 20 is a "vehicle front image" in which the front 151 of the vehicle 15 is captured.
FIG. 5C shows a state in which the image pickup region P includes the front end 150 of the vehicle 15. The image pickup determination unit is, for example, when there is a predetermined change in the area including the upper end of the image pickup area P within a predetermined time (for example, 10 seconds) from the time when the vehicle 15 is determined to have entered the entrance / exit gate 10. , The image pickup device 20 is made to capture the image captured at that time. In this case, the image captured by the image pickup apparatus 20 is a "vehicle front image" in which the entire front portion 151 of the vehicle 15 and a part of the loading portion 152 are captured.
FIG. 5D shows a state in which the imaging region P includes the rear end 154, which is the end of the rear portion 153, which is the rear side when viewed in the traveling direction of the vehicle 15. The image pickup determination unit is, for example, after a predetermined change occurs in the area including the lower end of the image pickup area P within a predetermined time (for example, 10 seconds) from the time when the vehicle 15 determines that the vehicle has entered the entrance / exit gate 10. When the change does not occur, the image captured at that time is transmitted from the image pickup device 20 to the vehicle detection system 30. In this case, the image captured by the image pickup apparatus 20 is a "vehicle rear image" in which the entire rear portion 153 of the vehicle 15 and a part of the loading portion 152 are captured.
FIG. 5 (e) shows a state in which the entire loading portion 152 of the vehicle 15 has come to the imaging region P. For example, when the image pickup determination unit reaches the upper end of the rectangular shape of the loading unit 152 at the upper end of the image pickup region P within a predetermined time (for example, 10 seconds) from the time when the vehicle 15 determines that the vehicle has entered the entrance / exit gate 10. To transmit the image captured at that time from the image pickup device 20 to the vehicle detection system 30. In this case, the image captured by the image pickup apparatus 20 is a "vehicle packaging image" in which all or part of the loading portion 152 of the vehicle 15 is captured.
FIG. 5 (f) shows how the rear portion 153 of the vehicle 15 leaves the imaging region P. For example, the image pickup determination unit changes to a predetermined ratio (for example, 30%) of the area E2 within a predetermined time (for example, 10 seconds) from the time when the vehicle 15 determines that the vehicle has entered the entrance / exit gate 10. When it does not occur, the image captured at that time is transmitted from the image pickup device 20 to the vehicle detection system 30. In this case, the image captured by the image pickup apparatus 20 is a "vehicle rear image" in which the rear 153 of the vehicle 15 is captured.

In the above description, the case where the image pickup apparatus 20 includes an image pickup determination unit has been described as an example, but the present invention is not limited to this. A functional unit equivalent to the image pickup determination unit may be provided in the vehicle estimation device 40 and / or the learning device 60.

FIG. 6 is a flowchart showing an operation example of the learning device 60 of the embodiment.
In the learning device 60 of the present embodiment, first, deep learning is performed using an image for learning data in which a vehicle or a load is captured, and an estimation model for estimating a vehicle or a load from the image is created. Then, by adjusting the hyperparameters used in the deep learning algorithm, the probability that the estimation model estimates correctly (hereinafter referred to as the recognition rate) is improved. The hyperparameters here are various parameters that are determined in advance when performing deep learning, for example, the number of layers of the neural network, the units of the intermediate layer (also referred to as nodes, cohesion points, etc.). Variables such as the number of and the coefficient of regularization.
Then, the direction of learning in deep learning is determined based on the recognition rate of the estimation model created by setting predetermined hyperparameters. The direction of learning here is to understand how the recognition rate changes by adjusting (tuning) which hyperparameters and how much to change which hyperparameters. It is to decide a concrete policy such as what kind of learning data (image) should be increased and how much in order to improve the recognition rate. As a result, it is possible to grasp what kind of image the learning data required to improve the recognition rate is, and it is possible to efficiently prepare the learning data (image).

First, the learning device 60 acquires image data (step S10). The image data acquisition unit 61 acquires image data of an image captured from a bird's-eye view of the vehicle 15 and an image captured by a bird's-eye view of the load of the vehicle 15.
Next, the learning device 60 extracts a region in which each of the vehicle 15 in the image and the load of the vehicle 15 is imaged based on the acquired image data (step S11). The area image extraction unit 62 extracts the region in which the vehicle 15 is captured from the image captured so as to take a bird's-eye view of the vehicle 15. Further, the area image extraction unit 62 extracts a region in which the load of the vehicle 15 is captured from the image captured so as to take a bird's-eye view of the load of the vehicle 15.
Next, the learning device 60 creates learning data based on the extracted image (step S12). The vehicle type learning data creation unit 63 creates learning data in which the vehicle type of the vehicle 15 is associated with the image captured by the vehicle 15. The load learning data creation unit 64 creates learning data in which the image of the load of the vehicle 15 is associated with the name of the load of the vehicle 15.
Next, the learning device 60 distributes the created learning data into training data and test data (step S13). The learning data distribution unit 65 distributes the learning data created by each of the vehicle type learning data creation unit 63 and the load learning data creation unit 64 into training data and test data.

Next, the learning device 60 performs deep learning using the training data and the test data (step S14). The vehicle type learning model creation unit 66 is estimated by, for example, being created by the vehicle type learning data creation unit 63 and executing deep learning using the training data and the data distributed by the training data distribution unit 65. Create a model. Then, the learning device 60 calculates the recognition rate of the created estimation model using the test data. Specifically, the learning model evaluation unit 68 is created by the vehicle type learning data creation unit 63 in the estimation model created by the vehicle type learning model creation unit 66, and is distributed to the test data by the training data distribution unit 65. The estimation result output by inputting the image of the obtained data is acquired. The learning model evaluation unit 68 determines that correct estimation has been made when the acquired estimation result and the vehicle type associated with the image of the test data match. On the other hand, the learning model evaluation unit 68 determines that an incorrect estimation has been made when the acquired estimation result and the vehicle type associated with the image of the test data do not match. Further, the learning model evaluation unit 68 is created by the load learning data creation unit 64 in the estimation model created by the load learning model creation unit 67, and is distributed to the test data by the learning data distribution unit 65. Get the estimation result output by inputting the image of. The learning model evaluation unit 68 determines that the correct estimation has been made when the acquired estimation result and the name of the load associated with the image of the test data match. On the other hand, the learning model evaluation unit 68 determines that an incorrect estimation has been made when the acquired estimation result and the name of the load associated with the image of the test data do not match. The learning model evaluation unit 68 calculates the ratio of the test data for which the correct estimation is made to the total number of test data as the recognition rate.

Next, the learning device 60 determines whether or not the recognition rate of the created estimation model has reached a predetermined threshold value or more (step S15).
Then, when the recognition rate of the created estimated model is equal to or higher than a predetermined threshold value, the learning device 60 stores the estimated model in the trained model storage device 50 (step S16).

On the other hand, when the recognition rate of the created estimation model is less than a predetermined threshold value, the learning device 60 determines whether or not the learning data is insufficient (step S17). In the learning device 60, for example, the recognition rate calculated using a part of the test data (for example, half of the total number of test data) is lower than the recognition rate using the total number of test data. If so, it is determined that the training data is insufficient. On the other hand, when the learning device 60 has a recognition rate equivalent to, for example, a recognition rate calculated using a part of the test data and a recognition rate using the total number of test data, the learning data is not insufficient. Is determined.
When the learning device 60 determines that the learning data is insufficient, the learning device 60 returns to the process shown in step S10 and recreates the estimation model. When recreating the estimation model, increase the number of images to make up for the lack of training data.

Further, when it is determined that the learning data is not insufficient, the learning device 60 adjusts the hyperparameters when executing deep learning (step S18). After adjusting the hyperparameters, the learning device 60 returns to the process shown in step S14 and performs deep learning again.

As described above, the vehicle estimation device 40 of the present embodiment acquires image data of an image of at least one of the vehicle 15 entering the work site 1 and the vehicle 15 leaving the work site 1. The image data acquisition unit 41 and the trained model for estimating the attribute information about the vehicle (for example, the trained model stored in the vehicle type trained model storage unit 51 or the load-learned model storage unit 52) A vehicle type estimation unit 43 and a load estimation unit 44 that estimate attribute information about the vehicle 15 in the image by inputting to the stored trained model) are provided, and the trained model has attribute information in the vehicle. It is a model created by executing deep learning using the learning data corresponding to. Thereby, the vehicle estimation device 40 of the present embodiment can estimate the attribute information of the vehicle 15 captured in the image. Therefore, it is not necessary to specify the vehicle 15 by registering or inquiring the vehicle number, and it is possible to manage the vehicle 15 entering and exiting at the work site 1 without increasing the time and effort.

All or part of the vehicle estimation device 40 and the learning device 60 in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, a "computer-readable recording medium" is a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line, and dynamically holds the program for a short period of time. It may also include a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or a client in that case. Further, the above program may be for realizing a part of the above-mentioned functions, and may be further realized for realizing the above-mentioned functions in combination with a program already recorded in the computer system. It may be realized by using a programmable logic device such as FPGA.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs and the like within a range that does not deviate from the gist of the present invention.

1 ... Work site, 10 ... Entrance / exit gate, 20 ... Imaging device, 30 ... Vehicle detection system, 40 ... Vehicle estimation device, 50 ... Learned model storage device, 60 ... Learning device, 41 ... Image data acquisition unit, 43 ... Vehicle type estimation unit, 44 ... Load estimation unit, 63 ... Vehicle type learning data creation unit, 64 ... Load learning data creation unit, 65 learning data distribution unit, 66 ... Vehicle type learning model creation unit, 67 ... Load learning unit Model creation department, 68 ... Learning model evaluation department

Claims (6)

An image data acquisition unit that acquires image data of an image captured by at least one of a vehicle entering the work site or a vehicle leaving the work site.
By inputting the image data of the image into a trained model for estimating the attribute information about the vehicle, the estimation unit for estimating the attribute information about the vehicle for the vehicle in the image is provided.
In the trained model, the learning data in which the vehicle type of the vehicle is associated with the image of the vehicle extracted from the image data and the image of the load extracted from the image data are captured by the vehicle. It is created by executing deep learning using the learning data associated with the name of the load.
Vehicle estimation device. The vehicle estimation device according to claim 1, wherein the estimation unit estimates at least one of the vehicle type of the vehicle in the image and the load of the vehicle in the image. The image is an image of a bird's-eye view of at least one of a vehicle entering the work site or a vehicle leaving the work site, and includes a front portion on the front side in the traveling direction of the vehicle. Part image, vehicle front image including the end in the front direction and all or part of the vehicle load, rear view of all or part of the vehicle load and the vehicle traveling direction 1 Or the vehicle estimation device according to claim 2. The above-mentioned in the vehicle in the image data imaged with the area where the vehicle enters and exits at the work site as the image pickup area by executing the deep learning using the learning data in which the vehicle and the attribute information related to the vehicle are associated with each other. It is a learning device equipped with a model creation unit that creates an estimation model that estimates attribute information.
In the estimation model, the learning data in which the vehicle type of the vehicle is associated with the image captured by the vehicle extracted from the image data and the image of the load extracted from the image data are captured by the vehicle. It is created by executing deep learning using the learning data associated with the name of the load.
Learning device. A distribution unit that distributes each of the training data into training data and test data,
It also has an evaluation unit that evaluates the estimation model created by the model creation unit.
The model creation unit creates an estimation model by executing deep learning using the training data distributed by the distribution unit.
By inputting an image corresponding to the test data into the estimation model, the evaluation unit acquires an estimation result of estimating the attribute information of the vehicle for the vehicle in the image corresponding to the test data, and the acquired estimation result. The learning device according to claim 4, wherein the estimation model is evaluated based on the attribute information of the image corresponding to the test data. The image data acquisition unit acquires image data of an image whose imaging area is the area where the vehicle enters and exits at the work site.
The estimation unit estimates the attribute information about the vehicle for the vehicle in the image by inputting the image data of the image into the trained model for estimating the attribute information about the vehicle.
The trained model is created by executing deep learning using the training data in which the attribute information is associated with the vehicle .
In the trained model, the learning data in which the vehicle type of the vehicle is associated with the image of the vehicle extracted from the image data and the image of the load extracted from the image data are captured by the vehicle. It is created by executing deep learning using the learning data associated with the name of the load.
A vehicle estimation method characterized by that.

Images