KR102131893B1 - Apparatus and method for determining the vehicle cover fault automatically - Google Patents

Abstract

Provided are a device and a method for automatically recognizing a fault of a freight car cover. The device for automatically recognizing a fault of a freight car cover comprises: a photographing unit mounted on a vehicle and outputting an image photographed in front of the vehicle (hereinafter referred to as front photographed image); a location sensing unit providing location information of the vehicle; a device processor recognizing a freight car with a poor cargo cover among driving freight cars based on the output front photographed image and generating cover fault information including a vehicle number plate of the freight car which has been recognized that a state of a cargo cover is faulty and information on a location in which the front photographed image is photographed; and a storage unit storing generated cover fault information.

Description

Apparatus and method for determining the vehicle cover fault automatically}

The present invention relates to an apparatus and method for automatically recognizing a defective cover of a lorry, and more particularly, to an apparatus and method for automatically recognizing a defective cover of a lorry that can determine whether the cover is defective from an image of a freight car taken from a front image. will be.

When a violation occurs when loading a load on a freight vehicle, this often causes a secondary accident such as dropping the load, and frequently acts as a factor that causes a danger in a large traffic accident.

The Road Traffic Act has safety standards for loading cargo, which must be marked with a red cloth of 30cm in width and 50cm in length. In addition, when loading the cargo, be sure to put on a cover and securely operate it.

Accordingly, the unloaded freight vehicles represent various types of vehicles, such as uncovered or unbundled vehicles, which have poor binding conditions, and which have a fear of falling or falling cargo due to uneven load.

In order to eradicate the loading failure of the truck, a safety cover of the truck needs to be used, but there are many cases where this is not observed, and there is no way to properly crack down the driver, so there is a difficulty in crackdown.

In particular, the cause of traffic accidents due to the defective vehicle cover still exists due to the lack of police personnel who conduct cargo control and lack of punishment standards and the absence of automatic control system technology.

Freight cars loaded with various loads are ignoring safety measures and often do not comply with traffic regulations such as speeding, threatening the safety of other drivers.

In addition, although the loading box fixing device such as a container is loose or a loose vehicle and a bad vehicle for fixing a spare tire are subject to crackdown, there is a situation in which a technology capable of automatically cracking down has not been presented so far.

Domestic registered patent 10-1282864 (2013.07.01. registered)

In order to solve the above-mentioned problems, a technical problem to be achieved by the present invention is to provide an apparatus and method for automatically detecting a cover defect of a freight vehicle capable of automatically detecting a cover defect of a freight vehicle.

The problems of the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

As a means for solving the above-described technical problem, according to an embodiment of the present invention, an auto-recognition device for a defective vehicle cover is mounted on a vehicle and captures an image of the front of the vehicle (hereinafter referred to as a “front image”). A photographing unit to output; A location sensing unit providing location information of the vehicle; On the basis of the output front image, a freight vehicle that is in a poor cargo cover state is recognized among the moving vehicles, and a cover including a vehicle license plate of the freight vehicle recognized as having a poor freight cover condition and location information on which the front image is captured. A device processor that generates bad information; And a device storage unit configured to store the generated cover defect information.

The apparatus processor may include a freight vehicle determination unit to determine whether a freight vehicle exists by applying a vehicle extraction model previously learned based on artificial intelligence to the front image; It includes; and, if it is determined that a freight vehicle is present in the front shot image, a cover defect determination unit for determining whether the cargo cover is defective by applying a previously extracted vehicle cover defect extraction model based on artificial intelligence.

The freight vehicle determination unit may include a candidate vehicle extraction unit that extracts a candidate freight vehicle by applying the vehicle extraction model to the front shot image; A calculating unit for calculating the height of the extracted candidate cargo vehicle; And a detection target determining unit determining whether the candidate cargo vehicle is an actual cargo vehicle based on the calculated height of the vehicle.

When the candidate defective vehicle is determined to be a freight vehicle that is a detection target vehicle, the defective cover determination unit sets an area of interest including the freight vehicle in the front photographed image to determine whether to load a freight vehicle traveling in front of the vehicle. A region of interest setting unit; And applying the extraction model for the defective cover of the freight car to the region of interest including the set freight car to determine whether or not the freight is loaded. Includes; load and cover determination unit for extracting the vehicle license plate of the freight car.

The cargo loading and lid determination unit extracts an area (hereinafter referred to as an'object area') where a freight vehicle exists within the region of interest, and an area where cargo is loaded in the object area (hereinafter referred to as a'cargo area'). ), after determining whether a cargo cover is installed in the cargo area and the type of the cover, determine whether the cover state is defective and the type of the cover state according to the determined type of cover, and determine that the cover state is poor One or more regions are extracted from the object region and added to the cover defect information.

On the other hand, according to another embodiment of the present invention, a method for automatically recognizing a defective cover of a lorry is installed on the periphery of the road to photograph a road including at least one lane, but to photograph the tail of a vehicle driving on the road, At least one information collecting device that transmits road photographing information including a video (hereinafter referred to as a'road photographing image') and current location information; And a freight vehicle having a poor cargo cover state among the freight cars driving on the road based on a road captured image among the road shooting information received from the one or more information collecting devices, and the vehicle of the freight vehicle recognized as having a poor cargo cover state. It includes a license plate and the cargo cover failure recognition server that generates and stores the cover failure information including the location information where the road photographed image was taken.

The van cover automatic recognition server includes a database storing road photographing information received from the one or more information collecting devices; And applying a previously extracted vehicle extraction model and a freight vehicle cover failure extraction model based on artificial intelligence to the stored road photographed image to determine whether the cargo cover condition is defective in the cargo loading space, and to determine whether the cargo cover condition is poor. It includes a server processor for extracting the license plate.

The server processor may include a freight vehicle determination unit for determining whether a freight vehicle exists by applying a vehicle extraction model previously learned based on artificial intelligence to the road photographed image; It includes; and, if it is determined that a freight vehicle is present in the road photographed image, a cover defect determination unit for determining whether the condition of the freight cover is defective by applying a previously learned freight car cover defect extraction model based on artificial intelligence.

The vehicle determination unit includes: a candidate vehicle extraction unit for extracting a candidate cargo vehicle by applying a vehicle extraction model to the road photographed image; A calculating unit for calculating the height of the extracted candidate cargo vehicle; And a detection target determining unit determining whether the candidate cargo vehicle is an actual cargo vehicle based on the calculated height of the vehicle.

When the candidate defective vehicle determines that the candidate freight vehicle is a freight vehicle that is a detection target vehicle, the region of interest setting the region of interest including the freight vehicle in the road photographing image to determine whether the candidate freight vehicle is loaded or not. part; And applying the extraction model of the truck cover defect to the region of interest including the set cargo vehicle, to determine whether the cargo is loaded, if it is determined that the cargo is loaded, determine whether the cargo cover is defective, and if the cargo cover is determined to be defective, the Includes; load and cover determination unit for extracting the vehicle license plate of the freight car.

The cargo loading and lid determination unit extracts an area (hereinafter referred to as an'object area') where a freight vehicle exists within the region of interest, and an area where cargo is loaded in the object area (hereinafter referred to as a'cargo area'). ), after determining whether a cargo cover is installed in the cargo area and the type of the cover, determine whether the cover state is defective and the type of the cover state according to the determined type of cover, and determine that the cover state is poor One or more regions are extracted from the object region and added to the cover defect information.

According to the present invention, by using the image recognition technology to automatically detect and manage the defective load of the cargo vehicle by real-time detection and management of the unloaded cargo vehicle, thereby reducing road traffic accidents, reducing the cost of cargo vehicles It can be used as a management system.

In addition, according to the present invention, even if the loading of the cargo is legal, the lid state is poor, or when the loading of the cargo is inappropriate, the lid state may also be automatically recognized in real time.

In addition, according to the present invention, as a state-of-the-art road management technology that can reduce physical and human damage from falling accidents while driving, it can be used as a service platform for smart city services or autonomous vehicles in the future.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is a block diagram showing a truck cover defective extraction model generation server according to an embodiment of the present invention,
FIG. 2 is a view showing an image taken in front of one frame to describe an operation of extracting a region of interest from the image taken in front,
3 is a block diagram showing an automatic recognition device for a defective cover of a truck according to an embodiment of the present invention;
4 is a block diagram showing in detail the device processor shown in FIG. 3;
5 is a view for explaining the operation of the vehicle determination unit determines the vehicle to be detected;
FIG. 6 is a view for explaining whether the cover defectiveness determination unit extracts a defective cover and an operation of extracting a license plate;
7 is a diagram schematically illustrating the concept of refinement of a region of interest through an object extraction process;
8 is an exemplary view showing an object extraction image of a vehicle to be detected that can be obtained through the object extraction process described above;
9 is a view showing an automatic recognition system for a defective cover of a truck according to another embodiment of the present invention;
10 and 11 are conceptual diagrams showing an operation in which the information collecting device provides a road photographed image,
12 is a flow chart for explaining a method for generating a vehicle extraction model of a vehicle cover defective extraction model generation server according to an embodiment of the present invention;
13 is a flow chart for explaining a method for generating a defective model of a freight car cover defective extraction server generation server according to an embodiment of the present invention;
14 is a flowchart for explaining a method for automatically recognizing a defective cover of a freight car of the automatic recognition of a cover defective product according to an embodiment of the present invention.

The above objects, other objects, features and advantages of the present invention will be readily understood through the following preferred embodiments related to the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed contents are thorough and complete and that the spirit of the present invention is sufficiently conveyed to those skilled in the art.

In this specification, when an element, component, device, or system is mentioned to include a component consisting of a program or software, the element, component, device, or system is the program or software, even if there is no explicit mention. It should be understood that includes hardware (eg, memory, CPU, etc.) or other programs or software (eg, drivers required to run the operating system or hardware) necessary to execute or operate.

Also, it should be understood that an element (or component) may be implemented in software, hardware, or in any form of software and hardware, unless otherwise specified in the implementation of any element (or component).

In addition, the terminology used herein is for describing the embodiments and is not intended to limit the present invention. In the present specification, the singular form also includes the plural form unless otherwise specified in the phrase. As used herein,'comprises' and/or'comprising' does not exclude the presence or addition of one or more other components.

Hereinafter, with reference to the accompanying drawings, a specific technical content to be carried out in the present invention will be described in detail. It should be noted that, in some cases, parts that are commonly known in describing the invention and that are not significantly related to the invention are not described in order to prevent chaos from coming into account in explaining the invention for no apparent reason.

Each configuration of the servers 100, 500, or devices 200, 400 shown in FIGS. 1, 3, and 9 indicates that they may be functionally and/or logically separated, and each configuration must be separately The average expert in the technical field of the present invention can easily infer that it does not mean that it is classified as a physical device or written in a separate code, for example, a desktop PC (Personal Computer), a server, a laptop PC (Laptop PC), a netbook computer (Netbook Computer), may be one of all the electronic devices that can install and run the program, such as a smartphone.

1 is a block diagram illustrating a server 100 for extracting a defective cover of a truck according to an embodiment of the present invention.

The truck cover defective extraction model generation server 100 shown in FIG. 1 applies a plurality of front vehicle images to a learning (for example, deep learning) algorithm, and among the trucks loaded with cargo on a real road, the truck has poor cover installation. It is possible to train and generate a vehicle extraction model that can select and a truck vehicle defect extraction model.

To this end, the truck cover defective extraction model generation server 100 includes a user interface unit 110, an input unit 120, a database 130, a pre-processing unit 140, a server memory 150, and a server processor 160. can do.

The user interface 110 is a device for interfacing between the user and the server 100, and receives a user command and transmits it to the server processor 160, or displays a result processed by the server processor 160 on the screen. have.

The input unit 120 may include a communication circuit that receives the front shooting videos from a plurality of vehicle devices (not shown) in a wired or wireless communication method, or may receive a plurality of front shooting videos stored in a portable memory (not shown).

The front shooting video may include a plurality of front shooting images (ie, frames) photographing the front of the vehicle driving on the road, and may be an image photographed to include the tail of the front vehicle.

In addition, the front photographed image may be an image including an image of a front left road, a front right road, or a front road based on a vehicle equipped with a camera and photographing the front. Accordingly, the rear aft of the front vehicle includes the right side and the rear side of the vehicle driving the front left road, the left side and the rear side of the vehicle driving the front right road, or the rear side of the vehicle driving the front road. can do.

As shown in FIG. 2, each front-viewing image may be a horizontal line of the road surface, that is, a frame photographed such that the vanishing point is located in the 2/3 area.

The database 130 may collect and store a plurality of front shot videos or frame front shot images obtained from vehicle devices or a portable memory through the input unit 120. In the latter case, the user can extract and store a frame necessary for model creation from the front-facing video.

In addition, the database 130 stores a region of interest (ROI) labeled for each type or type of lorry in the pre-processing unit 140, and the labeled region of interest may include a tail shape of the vehicle. The type or shape of the freight car may be classified by tonnage or by manufacturer, for example, a freight car of less than 1 to 2.5 tons, and a freight car of less than 2.5 to 8.5 tons.

In addition, the database 130 may store a region of interest labeled for each type of cover used when loading the cargo in the truck in the pre-processing unit 140.

The pre-processing unit 140 image-processes each region of interest extracted from the collected anterior images. The pre-processing unit 140 may generate a labeled truth image by allowing a user to display the tail of the vehicle differently according to the type of the vehicle when the tail of the vehicle is displayed in the region of interest among the front image.

FIG. 2 is a diagram illustrating an image taken in front of one frame to describe an operation of extracting a region of interest from an image taken in front.

Referring to FIG. 2, the user operates the user interface unit 110 to select a front shot image to be used to generate a vehicle extraction model or a truck cover defective extraction model among the front shot videos displayed on the screen.

First, in order to generate a vehicle extraction model, the user may select a front shot image (hereinafter referred to as a'first front shot image') in which the rear of the vehicle is displayed. The pre-processing unit 140 equally divides the selected first front-view image into a horizontal line into three equal parts and analyzes the area corresponding to the lower third (that is, the area less than one-third of the vanishing point) into the analysis area (hereinafter, referred to as the'first'. Analysis region'), and the extracted first analysis region may be displayed on the screen.

If the shape, clarity, etc. of the vehicle tail included in the extracted first analysis region is inadequate for learning to generate a vehicle extraction model, or if the vehicle tail is not displayed, the user may control the size or position of the first analysis region. 1 Can be reset from the front shot.

The user selects and crops a region including the tail of the vehicle from among the displayed first analysis regions using the user interface 110 and extracts the tail region of the vehicle existing in the cropped region along the outline. Then, the pre-processing unit 140 may extract the rectangle as the first region of interest by connecting the longest X-axis and Y-axis of the extracted outline. In addition, when the user designates the vehicle rear region with lines (or faces) of different colors or different thicknesses according to the type and size of the vehicle existing in the vehicle rear region in the first region of interest using the user interface 110. , The pre-processing unit 140 may label the first region of interest in different colors or thicknesses to display the type or size of the vehicle, whereby the first truth image labeled for the vehicle type (ie, the labeled first region of interest) ).

Next, in order to create a model for extracting a defective cover of a truck, the user may select a front shot image (hereinafter referred to as a “second front shot image”) in which a tail including a license plate of the vehicle is displayed among the front shot videos. The pre-processing unit 140 may extract the second analysis area from the second front-side image captured in the same manner as described above and display it on the screen.

When the user crops the area including the license plate of the vehicle among the second analysis areas, and extracts the license plate area of the vehicle existing in the cropped area along the outline, the pre-processing unit 140 is interested in the second from the extracted outline. Regions can be extracted. When a user designates a vehicle license plate area with lines (or faces) of different colors or different thicknesses according to the type of vehicle license plate in the second area of interest (classified by various criteria such as shape, size, or color), the preprocessor 140 ) May label the second region of interest with different colors or thicknesses to display the license plate of the vehicle, thereby generating a labeled second truth image (ie, the labeled second region of interest) for the type of vehicle license plate. .

In addition, in order to create a model for extracting a defective cover of a truck, the user manipulates the user interface 110 to display a front view image including the rear of the truck equipped with the cover while the cargo is loaded in the front view video (hereinafter, ' You can choose more). The cover of the van has various types, such as a net, a tent-type tarpaulin, and a hard case-type storage box, and the third front shot images may be a poorly mounted image or a normal image or a non-covered image for each type of cover. .

The pre-processing unit 140 may extract the third analysis area from the third front-viewing image and display it on the screen in the same manner as described above.

When the user crops the area including the cover of the freight car among the third analysis areas, and extracts the area of the freight car cover existing in the cropped area along the outline, the preprocessor 140 selects the third area of interest from the extracted outline. Can be extracted. If a user designates a defective area or a normal area of the cover with lines (or faces) of different colors or different thicknesses depending on the type of the cover in the third area of interest and whether the cover is properly mounted, or a poorly mounted type, the preprocessing unit ( 140) may label the third area of interest in different colors or thicknesses to display the cover of the lorry, whereby the labeled third truth image (ie, the labeled third point of interest) for the type of cover and the state in which the cover is mounted Region).

The pre-processing unit 140 may generate regions of interest labeled with different colors and/or shapes by user selection, that is, labeling data sets and store them in the database 130.

In addition, the preprocessor 140 learns by adjusting at least one of the labeled region of interest, that is, brightness (B, brightness), contrast (C, contrast), and orientation (O, orientation) of the first to third truth images. The number of input data can be increased. Accordingly, a number of different label data may be generated from one region of interest, which is reflected in learning or deep learning of various situations of an image taken in the front that may change depending on the environment such as weather and time when photographing an actual road. can do.

The server memory 150 may include volatile memory and/or non-volatile memory. In the server memory 150, for example, in order to implement and/or provide an operation, a function, and the like provided by the vehicle cover defective extraction model generation server 100, commands or data related to the components 110 to 160, One or more programs and/or software, operating systems, and the like may be stored.

The program stored in the server memory 150 may include a model generation program for implementing a vehicle extraction model and a truck vehicle defect extraction model. The vehicle extraction model is a program for extracting the rear shape of a vehicle by applying the collected front shot images to a learning algorithm. The truck cover defective extraction model is a program for extracting whether the vehicle extracted from the front image is a truck, and when the cargo is loaded, the cover is properly installed.

The learning algorithm may be an artificial intelligence algorithm, specifically a deep learning algorithm (eg, DNN, RNN, CNN, SVM, etc.). In the case of deep learning, it is most important to secure learning data, and for this purpose, as described in the pre-processing unit 140, a region of interest is set and cropped in one entire frame, and a vehicle rear region and a freight vehicle are set in the cropped dataset. A labeling operation is performed to indicate the position, size, or color of the cover, and a learning data set can be configured by labeling with the same criteria as possible.

The server processor 160 executes one or more programs stored in the server memory 150 to control the overall operation of the server 100. For example, the server processor 160 may execute the predictive model generation program stored in the server memory 150 to learn and generate a vehicle extraction model or a truck cover defect extraction model. To this end, the server processor 160 may include a model generator 162.

The model generation unit 162 may generate a vehicle extraction model for extracting the rear end of the vehicle by learning a plurality of regions of interest labeled according to the type or shape of the vehicle from the front shot image based on deep learning. The model generation unit 162 generates a vehicle extraction model by performing a learning algorithm using the first region of interest selected by the user as input data and the labeled first region of interest corresponding to the first region of interest as output data. Can.

In addition, the model generation unit 162 learns a plurality of second areas of interest labeled according to the type or shape of the vehicle license plate among the front shot images, extracts the vehicle license plate, and labels the vehicle according to whether the cargo is loaded and whether the cover is defective. By learning a large number of third areas of interest, a truck cover defect extraction model capable of extracting whether the truck is defective can be generated.

The generated vehicle extraction model and the van cover defective extraction model may be installed and executed in the van cover automatic recognition device 200 or the van cover automatic recognition server 500.

3 is a block diagram illustrating an automatic recognition device 200 for a defective cover of a truck according to an embodiment of the present invention.

Referring to FIG. 3, an automatic recognition device 200 for a defective cover of a truck according to an embodiment of the present invention includes a photographing unit 210, a location sensing unit 220, an output unit 230, a device memory 240, and a device storage It may include a unit 250, the device communication unit 260 and the device processor 270.

The imaging unit 210, the location sensing unit 220, the output unit 230, the device memory 240, the device communication unit 260, and the device processor 270 may be implemented as one device or one module, or the imaging unit Only 210 can be implemented as a separate independent device.

The photographing unit 210 may be mounted on a vehicle and photograph the front of a road that is driving, and output the photographed image of the road to the device processor 270 along with the photographing time. The front shooting image may be a video.

The photographing unit 210 is installed to photograph along the driving direction from the road surface of the road closest to the front of the vehicle, but is installed so that the vanishing point of the road surface horizontal line in the captured image is located within 2/3 of the front shooting image. Can shoot.

The location sensing unit 220 may sense location information of a driving vehicle and output it to the device processor 270 along with a sensing time.

When it is determined that the cover of the freight vehicle is defective by the device processor 270, the output unit 230 provides a visual or audible form with the attribute information to determine whether the cover of the freight vehicle is defective so that a user in the vehicle can recognize it. . The attribute information includes the time, location, and license plate number of the vehicle that is determined to be defective.

The device memory 240 may include volatile memory and/or non-volatile memory. The device memory 240 includes, for example, commands or data related to the components 210 to 270 in order to implement and/or provide operations and functions provided by the road surface damaged information providing device 200. The above programs and/or software, operating systems, and the like may be stored.

The program stored in the device memory 240 is a matching program for visually matching various sensing data, and a vehicle extraction model and a vehicle extraction cover extraction model that are learned and generated based on a learning algorithm in the vehicle vehicle cover defect extraction model generation server 100. It may include.

The device processor 270 executes one or more programs stored in the device memory 240 to control the overall operation of the device 200.

For example, the device processor 270 executes a matching program to synchronize the front shot image input from the photographing unit 210 and the location information input from the location sensing unit 220 at the same time, and the vehicle extraction model. Based on the time-synchronized anterior shooting image, the van number of the truck and the front plate image of the van identified as being in poor cargo cover state are captured. It is possible to generate cover defect information including the location information.

FIG. 4 is a block diagram showing the device processor 270 illustrated in FIG. 3 in detail, FIG. 5 is a view for explaining an operation of the vehicle determination unit 272 to determine a target vehicle, and FIG. 6 is a cover failure determination unit 274 is a view for explaining whether the cover is defective and the operation of extracting the license plate.

Referring to FIG. 4, the device processor 270 may include a freight vehicle determination unit 272 and a cover failure determination unit 274.

The truck determination unit 272 may determine whether a truck exists by applying a vehicle extraction model previously learned based on artificial intelligence to the front image. To this end, the freight vehicle determination unit 272 includes a candidate vehicle extraction unit 272a, a calculation unit 272b, and a detection target determination unit 272c.

Referring to FIG. 5, the candidate vehicle extracting unit 272a may extract a candidate cargo vehicle by applying a vehicle extraction model to an image taken in front. In detail, the candidate vehicle extracting unit 272a may input a front photographed image into the vehicle extraction model to recognize a trailing contour of a preceding vehicle among the front photographed images and extract it as a candidate dispatch vehicle.

The calculation unit 272b calculates the height of the vehicle body among the rear contours of the preceding vehicle extracted by the candidate vehicle extraction unit 272a as the height of the candidate cargo vehicle.

The detection target determination unit 272c may determine whether the candidate cargo vehicle is an actual cargo vehicle based on the height of the vehicle calculated by the calculation unit 272b. Since the aft height for each type of truck is predefined, the detection target determining unit 272c may determine that the truck is a truck if the calculated height falls within a standard error range of a predefined height.

In addition, the detection target determining unit 272c transmits the front photographing image including the candidate cargo vehicle determined as the freight vehicle and the location information of the freight vehicle among the front photographed images to the region of interest setting unit 274a.

In the case of FIG. 5, three candidate freight vehicles 2 to 4 have been extracted, and an actual freight vehicle, that is, one vehicle 2 is extracted as a detection target vehicle, and the remaining candidate freight vehicles 3 and 4 are It was judged as a normal vehicle.

On the other hand, if it is determined that the cover failure determining unit 274 exists in the front image, it is possible to determine whether the condition of the cargo cover is defective by applying a model for extracting a vehicle cover failure previously learned based on artificial intelligence. To this end, the cover defect determining unit 274 includes a region of interest setting unit 274a and a cargo loading and lid determining unit 274b.

Referring to FIG. 6, when the candidate cargo vehicle is determined to be the target vehicle, the region of interest setting unit 274a includes the region of interest (ROI_1) including the lorry to determine whether or not the lorry traveling in front of the vehicle is loaded with cargo. ) Can be set for the front shot image.

For example, the region of interest setting unit 274a divides the time-synchronized 1080p front image into three equal parts, sets a region of 1/3 below the vanishing point as the region of interest (ROI_1), and the other regions of interest (ie, a freight vehicle). When the license plate at the distance becomes unrecognizable, the license plate cannot be recognized, so object recognition primary filtering can be performed by excluding it from the region of interest. The 1/3 area below the vanishing point may be set as a region of interest (ROI_1), for example, a region larger than this, up to a size of 1 frame.

The cargo loading and cover determination unit 274b determines whether to load the cargo by applying a truck cover defective extraction model to the region of interest ROI_1 including the freight vehicle set in the region of interest setting unit 274a, and load is loaded. When it is determined, the condition of the cargo cover is defective, and when it is determined that the condition of the cargo cover is poor, the vehicle license plate of the freight vehicle may be extracted.

In addition, the cargo loading and cover determination unit 274b may generate cover failure information including the vehicle license plate and the front image taken location information and the front shot image of the freight vehicle judged to be in a bad cargo cover state.

In detail, the cargo loading and cover determination unit 274b may extract an area (hereinafter referred to as an'object area', ROI_2) in which a truck is located in the region of interest ROI_1. The cargo loading and lid determination unit 274b may set the region of interest ROI_2 including the freight vehicle using the location information input from the detection target determination unit 272c.

Then, the cargo loading and cover determination unit 274b detects an area (hereinafter referred to as'cargo area', ROI_3) in which the cargo is loaded in the object area ROI_2, and when cargo is loaded, the cargo area ( It is possible to determine whether a cargo cover is installed in ROI_3) and the type of the cover. The types of cargo covers vary depending on the purpose, performance, etc., such as the shape of a tent, the shape of a mesh, and the shape of a hard case for opening and closing a door.

If a cargo cover is installed and the cover type is determined, the cargo loading and cover determination unit 274b determines whether the cover state is defective or not and the type of the cover state is defective according to the determined type of cover, and it is determined that the cover state is poor. The above-described region ROI_3 may be extracted in the object region ROI_2 and added to the cover defect information.

In addition, even if it is determined that the cargo cover is not installed, the cargo loading and cover determination unit 274b may generate a vehicle defect plate information by extracting the vehicle license plate of the cargo vehicle using the truck vehicle defect extraction model.

7 is a diagram schematically illustrating the concept of refinement of a region of interest through an object extraction process.

Referring to FIG. 7, the cover defect determination unit 274 extracts a corresponding object (carrier, license plate, cover, etc.) while gradually reducing the area of interest ROI_1 from a wide area to the area of interest for primary filtering of object recognition. can do.

8 is an exemplary view showing an object extraction image of a vehicle to be detected that can be obtained through the object extraction process described above.

Referring to FIG. 8, in the case of a lorry located at the top, an object area (ROI_2), a defective cover area (ROI_3), and a vehicle license plate (ROI_3) loaded with cargo were extracted. In the case of the object region ROI_2, the cover is not fixed and is floating, and the cargo may be dropped. In the case of a truck located in the middle, the cover of the defective cover area ROI_3 has a mesh shape, and the falling of the box-shaped freight is concerned. In the case of the truck located at the bottom, an additionally poorly loaded cargo was extracted on the lid of the defective cover area ROI_3. As a result, even if the loading of the cargo is legal, the lid state is poor, or the loading of the cargo is inappropriate, so that it is possible to extract all of them according to an embodiment of the present invention.

On the other hand, the device processor 270 recognizes the tail of the freight car in the above-described manner, recognizes the cover and the license plate of the freight car at the same time, and the location information of the front and front shooting images of the original where the cover defect of the freight car was extracted. And cover defect information including the vehicle license plate and the shooting time is generated and stored in the device storage unit 250, and may be visually serviced using a Geographic Information System (GIS).

In addition, in the above-described embodiment, the region of interest ROI_1 is set by the cover failure determining unit 274, but the vehicle of interest may be recognized after setting the region of interest ROI_1 by the freight vehicle determination unit 272.

The device communication unit 260 may transmit cover failure information to the service providing server 300.

The service providing server 300 may be the server 100 of FIG. 1 described above, and as a separate server, refers to the location information among cover failure information provided from the device communication unit 260 to provide a GIS for a truck with a poor cover condition. Can be displayed on the map.

9 is a view illustrating an automatic recognition system for a defective cover of a truck according to another embodiment of the present invention, and FIGS. 10 and 11 are conceptual views illustrating an operation of the information collection device 400 to provide a road photographed image.

Referring to FIG. 9, the autonomous vehicle cover failure automatic recognition system includes one or more information collecting devices 400 and a autonomous vehicle cover failure automatic recognition server 500.

The information collecting device 400 is installed on the periphery of the road to photograph a road including at least one lane, but to photograph the rear of a vehicle driving on the road, and an image of the road (hereinafter referred to as a “road shooting image”). ) And road photographing information including current location information may be transmitted to the server 500. The information collecting device 400 may be a CCTV (closed-type TV) installed on the road. The road photographed video may be a frame-based video or a still video.

To this end, the information collection device 400 may include a camera device 410, a GPS module 420 and a communication module 430. The operation of the camera device 410 and the GPS module 420 is the same as the photographing unit 210 and the location sensing unit 220 described with reference to FIG. 3.

The communication module 430 transmits the road image information including the road image and the location information and the shooting time to the vehicle cover automatic recognition server 500.

The vehicle cover automatic failure recognition server 500 recognizes a vehicle having a poor cargo cover state among trucks driving on the road based on a road photographing image among road photographing information received from one or more information collecting devices 400, and the cargo cover It is possible to generate and store defective cover information including the license plate number of the lorry recognized as bad and the location information on which the road image is captured.

The automatic recognizing server 500 of the truck cover failure includes a server communication unit 510, a database 520, a server image processing unit 530, a server memory 540, a server processor 550, a server storage unit 560, and a display unit 570. ). The operation of the automatic recognition of the faulty cover of the freight car 500 is similar to the automatic recognition of the faulty cover of the freight car 200 described with reference to FIGS. 3 to 8, so a detailed description thereof will be omitted.

The server communication unit 510 communicates with a plurality of information collection devices 400.

The database 520 stores road image information received from a plurality of information collecting devices 400.

The server image processing unit 530 may perform image processing to display the road photographed image on which the truck is photographed by matching the GIS space on the display unit 570 when a truck having a poor cover state is included as a result of learning the road photographed image.

The server memory 540 includes a matching program for visually synchronizing road photographing images and location information, and a vehicle extraction model and a vehicle extraction cover extraction model, which are trained and generated based on a learning algorithm, in the vehicle production module 100. It can contain. The vehicle extraction model and the freight car cover defect extraction model may be generated in the manner described in FIG. 1, however, the images used for model generation may be images actually captured in CCTV as shown in FIG. 10.

The server processor 550 synchronizes the frame and location information of the road photographing images received from the information collecting device 400, and then extracts the vehicle extraction model and the truck cover defect extraction model previously learned based on artificial intelligence on the road photographed image. By applying, it is possible to determine whether the cargo cover space is defective or not, and to extract the vehicle license plate of the cargo vehicle determined to have a poor cargo cover.

To this end, the server processor 550 includes a freight vehicle determination unit 552 and a cover failure determination unit 554.

Referring to FIG. 10, the freight vehicle determination unit 552 determines whether a vehicle to be detected (a freight vehicle) exists by applying a vehicle extraction model to a road photographed image.

When it is determined that a truck is present in the road photographed image, the cover defective determination unit 554 applies a truck cover defective extraction model to the road photographed image to determine whether the cargo cover is defective, the cover area and the vehicle license plate (ROI_3). Can be extracted.

The display unit 570 may display images of a vehicle having a poor cover state among road photographing images received from the plurality of information collecting devices 400 for each of the information collecting devices 400 as illustrated in FIG. 11. The image displayed on the display unit 570 may be at least one of ROI_1 to ROI_3.

12 is a flow chart for explaining a method for generating a vehicle extraction model of the vehicle cover defective extraction model generation server 100 according to an embodiment of the present invention.

Since the server 100 for the method of generating the defective model for the cover of the van of FIG. 12 has been described with reference to FIG. 1, a detailed description thereof will be omitted.

Referring to FIG. 12, the truck cover defect extraction model generation server 100 collects and stores front images captured by a plurality of cameras (S1210 ).

The freight vehicle cover defect extraction model generation server 100 preprocesses the collected front shot images to set a first ROI including the vehicle (S1220 ).

The vehicle cover defect extraction model generation server 100 labels the first ROI set in step S1220 based on the type or shape of the vehicle and stores it (S1230).

The freight vehicle cover defect extraction model generation server 100 generates a vehicle extraction model by performing a learning algorithm on a plurality of first ROIs labeled by the user (S1240).

13 is a flow chart for explaining a method for generating a vehicle cover defective extraction model of the vehicle cover defective extraction model generation server 100 according to an embodiment of the present invention.

Referring to FIG. 13, the truck cover defect extraction model generation server 100 collects and stores front images captured by a plurality of cameras (S1310 ).

The truck cover defect extraction model generation server 100 pre-processes the collected front shot images to set a second ROI including the vehicle license plate (S1320).

The truck cover defect extraction model generation server 100 labels the second ROI set in step S1320 based on the type of the vehicle license plate and stores it (S1330).

The truck cover defective extraction model generation server 100 pre-processes the front image and sets a third ROI including the truck cover, labels the stored third ROI based on the defective state of the cover, and stores it (S1340, S1350).

The truck cover defective extraction model generation server 100 generates a truck cover extraction model capable of extracting the truck cover and license plate by performing a learning algorithm on a plurality of second and third ROIs labeled by the user (S1360). .

14 is a flowchart for explaining a method for automatically recognizing a defective vehicle cover in the automatic recognition of the defective vehicle cover 200 according to an embodiment of the present invention.

Since the apparatus 200 for executing the method for automatically recognizing the defective cover of the lorry of FIG. 14 has been described with reference to FIG. 3, detailed description thereof will be omitted.

Referring to FIG. 14, the automatic recognition device for the defective cover of the truck 200 stores the photographed front photographed image and location information (S1410 ).

The autonomous vehicle defect recognition apparatus 200 may extract a candidate cargo vehicle by applying a vehicle extraction model to the stored front-view image (S1420 ).

The autonomous vehicle defect recognition device 200 calculates the calculated height of the candidate cargo vehicle (S1430), and if it is determined to be a cargo vehicle (S1440-Yes), the region of interest (ROI_1) including the cargo vehicle can be set in the front image. Yes (S1450).

The automatic recognition of the defective cover of the freight car 200 determines whether to load the freight by applying the freight cover defective extraction model to the set region of interest ROI_1 (S1460 ).

When it is determined that the cargo vehicle cover failure automatic recognition device 200 is loaded (S1470-Yes), the object region ROI_2 where the freight vehicle exists in the region of interest ROI_1 is extracted, and the cargo in the object region ROI_2 It is possible to detect whether the cover is defective by detecting the loaded area (S1480).

If the lid of the freight car automatic recognition apparatus 200 is judged to have a bad cover state (S1490-Yes), one or more areas ROI_3 determined to have a bad cover state are extracted in the object area ROI_2, and the vehicle of the freight car is The license plate may be extracted to generate and store defective cover information (S1495).

On the other hand, although the above has been described and illustrated in connection with a preferred embodiment for illustrating the technical idea of the present invention, the present invention is not limited to the configuration and operation as shown and described as described above, and deviates from the scope of the technical idea. It will be understood by those skilled in the art that numerous changes and modifications are possible to the present invention. Accordingly, all such suitable changes and modifications and equivalents should also be considered within the scope of the present invention. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

100: truck cover defective extraction model generation server
200: automatic recognition device for defective cover of the truck
300: service providing server
400: information collecting device
500: automatic recognition server of the truck cover failure

Claims (11)

A photographing unit mounted on a vehicle and outputting an image photographed in front of the vehicle (hereinafter referred to as a “front photographing image”);
A location sensing unit providing location information of the vehicle;
On the basis of the output front image, a freight vehicle that is in a poor cargo cover state is recognized among the moving vehicles, and a cover including a vehicle license plate of the freight vehicle recognized as having a poor freight cover condition and location information on which the front image is captured. A device processor that generates bad information; And
Includes; a device storage unit for storing the generated cover defect information,
The device processor,
A vehicle extraction model previously learned based on artificial intelligence is applied to the front image.
A freight car determination unit to determine whether a freight car exists by applying; And
If it is determined that a freight vehicle exists in the front image, the front image is taken
Applied to the image based on artificial intelligence to apply the pre-trained truck cover defective extraction model to determine the type of cargo cover, determine whether the cargo cover is defective according to the determined type of cargo cover, and apply the truck cover defective extraction model Result Even if it is determined that the cargo cover is not installed, the cover defective determination unit for generating a cover failure information; Automatic vehicle cover failure recognition device comprising a. delete According to claim 1,
The vanity determination unit,
A candidate vehicle extraction unit that extracts a candidate cargo vehicle by applying the vehicle extraction model to the front shot image;
A calculating unit for calculating the height of the extracted candidate cargo vehicle; And
And a detection object determining unit determining whether the candidate freight vehicle is an actual freight vehicle based on the calculated height of the vehicle. According to claim 3,
The cover failure determining unit,
When the candidate cargo vehicle is determined to be a vehicle to be detected, a region of interest setting unit configured to set a region of interest including the vehicle in the front photographed image in order to determine whether to load or not a cargo vehicle traveling in front of the vehicle; And
Applying the model for extracting the defective cover of the lorry to the region of interest including the set freight car to determine whether or not the cargo is loaded, and if it is determined that the load is loaded, determine whether the condition of the cargo cover is defective, and determining that the condition of the cargo cover is poor, Cargo vehicle cover defective automatic recognition device comprising a; load and cover determination unit for extracting the vehicle license plate of the freight car. According to claim 4,
The cargo loading and cover determination unit,
After extracting the area (hereinafter referred to as'object area') where a freight vehicle exists within the region of interest, detecting the area where the cargo is loaded (hereinafter referred to as'cargo area') in the object area, and then After determining whether the cargo cover is installed in the area and the type of the cover, the cover state is determined to be defective and the cover state is defective according to the determined type of the cover, and one or more areas determined to be in poor cover state are located in the object area. Automated vehicle cover defective recognition device characterized in that it is extracted from and added to the cover failure information. It is installed on the periphery of the road to photograph a road including at least one lane, but to photograph the rear of a vehicle driving the road, and an image of the road (hereinafter referred to as a “road shooting image”) and current location information. At least one information collecting device for transmitting the road photographing information including; And
Based on the road photographing image among the road photographing information received from the one or more information collecting devices, a vehicle license plate of a truck that recognizes a cargo vehicle having a poor cargo cover state among the trucks driving the road, and recognizes that the cargo cover condition is poor. And a van cover recognition server for generating and storing cover failure information including location information on which the road photographed image is captured;
The vanity cover automatic recognition server,
A database storing road photographing information received from the one or more information collecting devices; And
Based on artificial intelligence, the vehicle extraction model and the freight vehicle cover failure extraction model previously applied to the stored road photographed image are applied to determine whether the cargo cover condition is poor in the cargo loading space, and the vehicle of the freight vehicle determined to have a poor cargo cover condition. Server processor for extracting the license plate; includes,
The server processor,
A freight vehicle determination unit to determine whether a freight vehicle exists by applying a vehicle extraction model previously learned based on artificial intelligence to the road photographed image; And
When it is determined that a freight vehicle is present in the road photographed image, the type of cargo cover is judged by applying a preliminarily extracted model of the truck cover to the road photographed image based on artificial intelligence, and the cargo cover is determined according to the determined type of cargo cover. Determining whether the condition is defective, and applying the above-described vehicle cover defect extraction model, even if it is determined that the cargo cover is not installed, a cover failure determination unit that generates cover failure information; system. delete delete The method of claim 6,
The vanity determination unit,
A candidate vehicle extraction unit that extracts a candidate cargo vehicle by applying a vehicle extraction model to the road photographed image;
A calculating unit for calculating the height of the extracted candidate cargo vehicle; And
And a detection object determining unit determining whether the candidate cargo vehicle is an actual cargo vehicle based on the calculated height of the vehicle. The method of claim 9,
The cover failure determining unit,
If the candidate cargo vehicle is determined to be a vehicle to be detected, a region of interest setting unit configured to set a region of interest including the cargo vehicle in the road photographed image to determine whether or not the candidate cargo vehicle is loaded; And
By applying the freight car cover defective extraction model to the region of interest including the set freight car, it is determined whether to load the cargo, if it is determined that the cargo is loaded, it is determined whether the cargo cover is defective, and if it is determined that the cargo cover is poor, the freight vehicle Cargo loading and cover determination unit for extracting the vehicle license plate; Automated vehicle cover failure recognition system comprising a. The method of claim 10,
The cargo loading and cover determination unit,
After extracting the area (hereinafter referred to as'object area') where a freight vehicle exists within the region of interest, detecting the area where the cargo is loaded (hereinafter referred to as'cargo area') in the object area, and then After determining whether the cargo cover is installed in the area and the type of the cover, the cover state is determined to be defective and the cover state is defective according to the determined type of the cover, and one or more areas determined to be in poor cover state are located in the object area. Automated system for detecting defective cover of trucks, characterized in that it is extracted from and added to the cover defective information.

Images