KR20230105836A - Tires management apparatus and method for vehicle - Google Patents

Abstract

The present invention relates to a tire management device and method for a vehicle, which monitors the condition of a tire using a camera, analyzes a filmed tire image based on deep learning to detect tire cord tearing, and provides information on tire cord tearing by outputting along with a warning, it is possible to detect a tire cord tear that cannot be detected by the tire pressure monitoring system, and the driver can easily check the tire cord tear, prevent accidents and improve driving safety.

Description

Vehicle tire management device and its method {TIRES MANAGEMENT APPARATUS AND METHOD FOR VEHICLE}

The present invention relates to a tire management device for a vehicle and a method for monitoring a tire condition using an image to detect a tire cord break.

A vehicle runs by being driven by a power source and transmitting generated power to wheels through various parts.

A tire is a part through which power of a vehicle is transmitted to the road surface, and as power is transmitted to the road surface by friction between the surface of the tire and the road surface, it is an important part that may lead to an accident when a tire is defective or defective.

If there is a problem with the tire, the probability of an accident increases, so a vehicle is equipped with a sensor to check the condition of the tire.

A tire pressure monitoring system (TPMS) is mounted on a tire, measures the air pressure of the tire, and transmits the measured pressure value to the controller of the vehicle through wireless communication. The controller generates an alarm when the tire pressure is lower than the reference value so that the driver can check the tire condition.

Among tire abnormalities, cord rounding is a phenomenon in which the tire is deformed as the pressure is concentrated on the cut portion by the internal air pressure when the carcass cord serving as the skeleton of the tire is cut by an instantaneous impact.

A cord cut occurs as a result of a momentary external impact caused by an obstacle (speed bump, pothole, curb) while driving, and if the tire is left unattended while the cord is broken, a serious accident such as tearing the tire may occur.

When a tire has a puncture, the pressure inside the tire decreases and can be detected using a TPMS. However, when a tire cord break occurs, since the tire pressure does not change, there is a problem in that it is not possible to detect whether a cord break has occurred through the TPMS.

Accordingly, measures to prevent cord wounding or to prevent tire tearing have been proposed, such as Korean Patent Registration No. 10-1350646 'Structure for preventing cord wounding of pneumatic tires'.

However, since it cannot cope with code rounding that has already occurred, it is necessary to detect the case of code rounding and notify the driver so that the driver can recognize it.

Republic of Korea Patent No. 10-1350646

An object of the present invention is to provide a vehicle tire management device and method for monitoring the condition of a tire using a camera, analyzing an image based on deep learning, detecting a tire cord cut, and outputting a warning.

Vehicle tire management apparatus according to the present invention in order to achieve the above object, a plurality of cameras; an image recognizing unit recognizing a tire from the images of the plurality of cameras and generating a tire image; a learning unit analyzing the image of the tire and detecting an abnormal region estimated by code rounding of the tire; The tire image captured while the vehicle speed is at the set speed is applied to the learning unit to request an analysis, and in response to the analysis result of the learning unit, it is determined whether or not the abnormal region is misrecognized so as to determine an error in the code rounding of the tire. an abnormality judgment unit that determines; and an input/output unit outputting a warning about an abnormality of the tire. includes

The abnormal determination unit may request the tire image from the image recognizing unit when the vehicle speed is less than the set speed, and determine the state of the tire based on the tire image taken at less than the set speed.

The abnormality determination unit may determine an abnormality of the tire based on an analysis result of a plurality of tire images while the tire rotates at least n times.

The abnormality determining unit calculates the location and size of the abnormal region, and if the size of the abnormal region is less than or equal to a set size, it is determined as an erroneous recognition, and if the size of the abnormal region exceeds the set size, it is determined that the tire cord is rounded. It is characterized by doing.

The input/output unit outputs the warning using a combination of at least one of a warning message, a warning sound, and a warning light regarding the tire abnormality, and outputs the tire image including the abnormal area.

The learning unit may analyze the tire image based on an anchor box set to detect an object, and detect the abnormal region adjacent to the anchor box.

The learning unit may output the analysis result including the coordinates of the center point of the abnormal region, the horizontal length of the abnormal region, the vertical length of the abnormal region, and the anchor box.

The learning unit is characterized in that, among a plurality of anchor boxes, an anchor box whose object detection accuracy exceeds a predetermined value is output to the abnormal region.

An operation method of a vehicle tire management apparatus according to the present invention includes the steps of taking a video by a plurality of cameras; If the speed of the vehicle is less than the set speed, generating a tire image by recognizing the tire from the image recognition unit; Based on deep learning, a learning unit analyzes the image of the tire and detects an abnormal region estimated by code rounding of the tire; Determining whether or not the abnormal region is erroneously recognized by an abnormality determination unit to determine an abnormality in cord rounding of the tire; and outputting a warning about an abnormality of the tire by an input/output unit. includes

The determining of the abnormality may include determining the abnormality of the tire based on an analysis result of a plurality of tire images while the tire rotates at least n times.

In the step of determining the abnormality, the location and size of the abnormal region are calculated, and if the size of the abnormal region is less than a set size, it is determined as an erroneous recognition, and if the size of the abnormal region exceeds the set size, the tire It is characterized by judging by code rounding.

The detecting of the abnormal region may include analyzing the tire image based on an anchor box configured to detect an object, and detecting the abnormal region adjacent to the anchor box from the tire image.

The detecting of the abnormal region may include outputting an anchor box having an object detection accuracy of more than 70% among the plurality of anchor boxes as the abnormal region.

The detecting of the abnormal region may include outputting the analysis result including the coordinates of the center point of the abnormal region, the horizontal length of the abnormal region, the vertical length of the abnormal region, and the anchor box.

The outputting of the warning may include outputting the warning as a combination of at least one of a warning message, a warning sound, and a warning light for the abnormality of the tire, and outputting the tire image including the abnormal area.

According to one aspect, the vehicle tire management device and method of the present invention photograph a tire using a camera and analyze the image to monitor the condition of the tire, thereby detecting a tire cord cut that cannot be detected by a tire pressure monitoring system. can detect

According to one aspect of the present invention, by detecting a tire cord break using an image and outputting information on the cord break together with a warning, the driver can easily check the location of the abnormality, prevent accidents, and improve driving safety. can

1 is a block diagram showing the configuration of a vehicle tire management device according to an embodiment of the present invention.
2 is a diagram referenced to explain an image analysis method for detecting a code break in a vehicle tire management apparatus according to an embodiment of the present invention.
3 is a diagram referenced to explain detection of code break through image analysis of a vehicle tire management apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating an operating method of a vehicle tire management device according to an embodiment of the present invention.
5 is an exemplary diagram for outputting information on code rounding of a vehicle tire management apparatus according to an embodiment of the present invention.

Hereinafter, the present invention will be described with reference to the accompanying drawings.

In this process, the thickness of lines or the size of components shown in the drawings may be exaggerated for clarity and convenience of description. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to the intention or custom of a user or operator. Therefore, definitions of these terms will have to be made based on the content throughout this specification.

1 is a block diagram showing the configuration of a vehicle tire management device according to an embodiment of the present invention.

As shown in FIG. 1, a camera unit 120, a communication unit 170, an image recognition unit 130, a learning unit 140, an abnormality determination unit 150, an input/output unit 190, a data unit 180 and a control unit 110.

The camera unit 120 includes a plurality of cameras 121 to 129 to capture images of the interior and exterior of the vehicle. The camera unit 120 captures front and rear images of the vehicle and combines the plurality of images to generate a 360-degree surround view image centered on the vehicle.

The image recognition unit 130 generates a surround view image by combining a plurality of images input from the camera unit 120 and outputs the image through the input/output unit 190 .

The image recognizing unit 130 converts images input from the plurality of cameras 121 to 129 of the camera unit 120 into a certain format, and recognizes a designated area of the vehicle from the image to determine the learning unit 140 and abnormalities. Applies to section 150.

The image recognizing unit 130 recognizes a tire from a plurality of images photographed through the camera unit 120 and detects an area including the tire. The image recognizing unit 130 generates a tire image of a region including a tire in the image.

The image recognition unit 130 may recognize a tire from an image based on the shape and color of the tire included in the image. In addition, the image recognition unit 130 may recognize a tire based on the learning data generated through the learning unit 140 .

The learning unit 140 accumulates images of tires with abnormalities and information on locations where the abnormalities occur, and repeats a process of comparing and analyzing tire images to generate learning data for the state of the tires.

In addition, the learning unit 140 accumulates input images to update learning data for the state of the tire.

The learning unit 140 analyzes the tire image, detects an abnormal region as an abnormal region, and outputs an analysis result.

Corresponding to the analysis result of the learning unit 140, the abnormality determining unit 150 determines whether the tire is abnormal or not by determining whether the abnormal region or the abnormal region is erroneously recognized.

The abnormality determination unit 150 measures the position and size of the code rounding. The anomaly determination unit 150 compares the size of the area with an anomaly with a set size to determine whether it is misrecognized.

The input/output unit 190 includes at least one input means among buttons, switches, and touch input means to receive data from the driver and apply it to the control unit 110 .

The input/output unit 190 includes at least one output means among a display, a speaker, and a lamp, and transmits vehicle status information and driving status information according to a control command of the control unit 110 at least among messages, sound effects, and status lights. It outputs in one form, and outputs a warning in a combination of at least one of a corresponding warning message, warning sound, and warning light when an abnormality or failure occurs or is in a dangerous situation.

Also, the input/output unit 190 outputs a front image and a rear image of the vehicle, and outputs a surround view image.

The communication unit 170 includes a plurality of communication modules and performs wired or wireless communication according to a control command of the control unit 110 . The communication unit 170 transmits information inside the vehicle through CAN communication, which is a communication method inside the vehicle. The communication unit 170 communicates with other external terminals or external servers through a network to transmit and receive vehicle data.

The data unit 180 stores the learning data 181 of the learning unit 140, data for object recognition, and reference data 182 for determining abnormality.

The input/output unit 190 includes at least one input means among buttons, switches, and touch pads, and includes at least one output means among displays, lamps, and speakers.

In addition, the input/output unit 190 outputs a guide or warning in response to a control command from the control unit 110 . The input/output unit 190 may output a warning by combining at least one of a guide message, a warning message, a warning sound, a warning light, and a voice guide.

The input/output unit 190 outputs tire abnormalities and information about tire abnormalities based on the results of monitoring the tire conditions.

The control unit 110 processes data input and output through the input/output unit 190 and the communication unit 170 and manages data of the data unit 180 . The control unit 110 controls an operation by determining a state of the vehicle based on data input from a plurality of sensors (not shown).

When the learning data is generated through the learning unit 140, the control unit 110 stores it in the learning data 181 of the data unit 180, and when an abnormality of the tire is detected through the abnormality determination unit 150, the abnormality occurs. The tire image is stored in the data unit 180.

In addition, the controller 110 generates a warning when a tire abnormality is detected and applies the warning to the input/output unit 190 .

The control unit 110 outputs an image of a tire in which an abnormality is detected in relation to tire cord rounding through the input/output unit 190 so that the driver can check the image of the tire where the cord rounding has occurred. In addition, the controller 110 stores an image of a tire in which an error occurs in the data 180 .

2 is a diagram referenced to explain an image analysis method for detecting a code break in a vehicle tire management apparatus according to an embodiment of the present invention.

As shown in FIG. 2 , when a cord break occurs in a tire of an automobile, the surface of the tire protrudes like the first region 11 and the shape of the tire is deformed.

The learning unit 140 receives a plurality of images where a tire break occurs, receives a label, which is information about a region where a code break occurs, in each image, and repeatedly analyzes the image based on deep learning, thereby generating learning data for the corresponding image. do.

The learning unit 140 receives the coordinates of the region and labels for the width (w) and height (h) of the region in the first region 11 where code rounding has occurred, and recognizes the corresponding region as code rounding. .

The learning unit 140 detects whether an anomaly has occurred and a location where an anomaly occurred from an image based on the anchor box 12 set for object detection based on learning data for code rounding, and sets it as an anomaly area.

The learning unit 140 detects an anomaly using an anchor box adjacent to a location where an anomaly occurs and sets it as an anomaly area.

Anchor Boxes are bounding boxes with a predefined shape for detecting and recognizing objects from images, and are generated from data by a K-means algorithm and obtain prior information on the size and shape of objects in the data set. do. Each anchor box is designed to detect objects of different sizes and shapes, and is used to detect objects of similar size to the anchor box.

The learning unit 140 learns to detect a code inversion region close to the anchor box in the tire image based on the anchor box.

The learning unit 140 outputs values of 5 channels as shown in Equation 1 below, based on the object recognition network.

x and y are the center points of the cut prediction region, w is the horizontal length, h is the vertical length, and a is the anchor box.

The learning unit 140 uses an object recognition network to determine an anchor box adjacent to an incision of a label among 16384 (128 × 128) anchor boxes defined for each output position, that is, IoU (Intersection For anchor boxes with over union) exceeding 70%, learning is performed so that the inferred values of x, y, w, and h are the same as the labels.

In Equation 1, the position and size items are obtained by using mean square error (MSE) to obtain a loss and adding the cross entropy loss of c (0 is background, 1 is code rounding) Learning is performed using Stochastic Gradient Descent (SGD).

The learning unit 140 generates learning data for tire code rounding upon completion of learning.

3 is a diagram referenced to explain detection of code break through image analysis of a vehicle tire management apparatus according to an embodiment of the present invention.

As shown in FIG. 3 , the image recognizing unit 130 recognizes a tire from a surround view image or an image of a side camera and extracts a tire area.

The abnormality determination unit 150 applies the tire image captured while the vehicle speed is less than the set speed to the learning unit 140 and receives an analysis result based on the object recognition deep learning network to determine whether the tire code has been raised. judge

The abnormal determination unit 150 determines whether the area of the tire in the image is larger than a certain size, such as the 11th area 21 and the 12th area 23 in one tire image, according to the angle of view according to the position of the camera and the rotational position of the wheel. On the other hand, since there is a small area of the tire, such as the twelfth area 22, a plurality of tire images are applied to the learning unit 140 to detect an anomaly.

The abnormality determining unit 150 detects code rounding based on the tire image input from the image recognizing unit 130 when the vehicle speed is less than the set speed or the vehicle temporarily stops or stops as the tire rotates while driving.

The anomaly determination unit 150 detects code rounding based on a plurality of images when the wheel rotates at least once in the tire image.

In some cases, the anomaly determination unit 150 may detect an anomaly of the tire, that is, a code rounding, by analyzing the tire image based on the learning data and detecting a portion where the shape is changed.

When an area similar to the anchor box is detected by the learning unit 140, the abnormality determination unit 150 determines that code rounding has occurred in the area of the anchor box. At this time, the anomaly determination unit 150 determines the size of the detected anomaly area, and determines that it is a false detection when the size is less than or equal to a set size.

The abnormality determination unit 150 applies data according to the abnormality detection to the controller 110, and the controller 110 generates a warning and controls the input/output unit 190.

4 is a flowchart illustrating an operating method of a vehicle tire management device according to an embodiment of the present invention.

As shown in FIG. 4 , when the car is turned on (ON) (S310) and driving starts, the camera unit 120 captures an image while driving based on a plurality of cameras.

When the speed of the vehicle is less than the set speed while driving (S320), the abnormality determination unit 150 starts detecting an abnormality in the tire and requests an image of the tire from the image recognition unit 130. The abnormality determination unit 150 detects a tire abnormality based on an image when the vehicle travels at a speed of 3 km/h or less or is temporarily stopped.

When the vehicle speed is higher than the set speed, it is difficult to recognize the shape of the tire in the image, so the abnormal determination unit 150 analyzes the tire image taken while the vehicle speed is less than the set speed to detect code rounding.

The image recognition unit 130 receives images from a plurality of cameras of the camera unit 120 (S330), and recognizes the tire from the side image of the vehicle.

The image recognizing unit 130 extracts the tire area (wheel area) recognized from the side image to generate a tire image and applies it to the abnormality determination unit 150 (S340).

The abnormality determining unit 150 applies the tire image input from the image recognizing unit 130 to the learning unit 140 to request an analysis, and detects tire abnormalities, particularly code rounding, according to the result.

The abnormality determination unit 150 determines whether or not the tire has a cord round based on the analysis result through the learning unit 140 (S360).

Based on the object recognition network, the learning unit 140 detects an anomaly region estimated by code rounding close to an anchor box according to pre-generated learning data, and applies the analysis result to the anomaly determination unit 150 .

When code rounding is not detected, the abnormality determination unit 150 repeats tire abnormality detection for a set number of times (n) (S380) (S320 to S360).

While driving, the tire rotates, and depending on the installation position of the camera or the shooting angle, the front of the tire is not included in the image, but part of the surface is displayed. In order to confirm, at least n times, code rounding detection for a plurality of tire images is attempted.

When code rounding is detected, the anomaly determination unit 150 determines whether the size of the detected anomaly region is larger than a set size (S370).

When the size of the abnormal region according to the detected code rounding is less than the set size, the abnormality determination unit 150 reattempts the detection of the code rounding through the learning unit 140 (S320 to S370) since there is a possibility of erroneous detection due to foreign matter. ).

The abnormality determination unit 150 repeats detecting code rounding for tire images photographed over time in a state where the vehicle speed is less than the set speed. When the vehicle speed increases, the abnormality determining unit 150 stops detecting a tire cord break.

The abnormality determination unit 150 detects code rounding of at least n tire images captured at different times. The abnormality determination unit 150 detects code rounding for the tire image up to the set number of times even if code rounding is detected before reaching the set number n (S380).

When the set number of times n is reached, the abnormality determination unit 150 ends code rounding detection. When the set period is reached, the abnormality determination unit 150 detects code rounding again for a set number of times.

The abnormality determining unit 150 determines that the tire is code-rounded if the size of the abnormal region estimated by code rounding exceeds a set size.

When it is determined that the tire is code-rounded, the abnormality determination unit 150 transmits code-rounded data to the control unit 110 .

The control unit 110 stores data on the code rounding area in the data unit 180 (S390), generates a warning about the tire's code rounding, and applies it to the input/output unit 190.

In response to the control command of the control unit 110, the input/output unit 190 outputs a warning about tire cord rounding with a combination of at least one of a warning message, a warning sound, and a warning light (S400).

The input/output unit 190 outputs a cord cut position and a tire image through a display provided thereon. For example, the input/output unit 190 may output a warning message and a warning sound indicating that a code break has occurred on the left rear wheel of the vehicle, and display an image of a tire where a code break has occurred.

5 is an exemplary diagram for outputting information on code rounding of a vehicle tire management apparatus according to an embodiment of the present invention.

As shown in (a) of FIG. 5 , the camera unit 120 captures an image, and the image recognizing unit 130 recognizes a tire in the input image.

As shown in (b) of FIG. 5 , the image recognizing unit 130 detects a tire area and generates a tire image.

The abnormality determination unit 150 applies the tire image to the learning unit 140 and requests analysis. The learning unit 140 analyzes an input tire image based on learning data for code rounding to detect an abnormal region estimated by code rounding. The learning unit 140 outputs the analysis result including the abnormal area to the abnormality determining unit 150 .

In response to the analysis result, the abnormality determination unit 150 requests the learning unit 140 to analyze a new tire image when there is no abnormal region estimated by code rounding. The abnormality determination unit 150 applies a plurality of tire images to the learning unit 140 in consideration of the area of the tire included in the image, and requests analysis.

Based on the analysis result of the learning unit 140, the anomaly determination unit 150 compares the size of the detected anomaly area with a set size, and determines that an area smaller than the set size is misrecognized when an abnormal area is detected.

As shown in (c) of FIG. 5 , the abnormality determining unit 150 determines an abnormality in cord rounding of the tire.

The control unit 110 generates a warning according to the determination result of the abnormality determination unit 150 and outputs a warning about the cord rounding of the tire through the input/output unit 190 .

The input/output unit 190 outputs the tire image displaying the code-rounded region 19 through a display, for example, AVN (Audio Video Navigation). In addition, the input/output unit 190 outputs at least one of a warning message, a warning sound, and a warning light to notify the driver that tire replacement is required.

Accordingly, the present invention can monitor the condition of the tire based on the image of the camera to detect the cord break of the tire, and output a warning about the cord break so that the tire can be inspected or replaced.

Although the present invention has been described with reference to the embodiments shown in the drawings, it should be noted that this is only exemplary and various modifications and equivalent other embodiments are possible from those skilled in the art to which the technology pertains. will understand Therefore, the true technical protection scope of the present invention should be defined by the claims below.

110: control unit 120: camera unit
130: image recognition unit 140: learning unit
150: abnormality determination unit 170: communication unit
180: data unit 190: input/output unit

Claims (15)

multiple cameras;
an image recognizing unit recognizing a tire from the images of the plurality of cameras and generating a tire image;
a learning unit analyzing the image of the tire and detecting an abnormal region estimated by code rounding of the tire;
The tire image captured while the vehicle speed is at the set speed is applied to the learning unit to request an analysis, and in response to the analysis result of the learning unit, it is determined whether or not the abnormal region is misrecognized to determine an error in the code rounding of the tire. an abnormality determination unit that determines; and
an input/output unit outputting a warning about an abnormality of the tire; Vehicle tire management device comprising a. According to claim 1,
Wherein the abnormal determination unit requests the image recognition unit for the image of the tire when the vehicle speed is less than the set speed and determines the state of the tire based on the image of the tire photographed at less than the set speed. According to claim 1,
The vehicle tire management apparatus according to claim 1 , wherein the abnormality determination unit determines an abnormality of the tire based on an analysis result of a plurality of tire images while the tire rotates at least n times. According to claim 1,
The abnormality determination unit calculates the location and size of the abnormal region;
If the size of the abnormal area is less than the set size, it is determined as an erroneous recognition,
and if the size of the abnormal area exceeds the set size, it is determined that the tire cord is rounded. According to claim 1,
The input/output unit outputs the warning as a combination of at least one of a warning message, a warning sound, and a warning light for an abnormality of the tire,
The tire management device for a vehicle, characterized in that for outputting the tire image including the abnormal area. According to claim 1,
Wherein the learning unit analyzes the tire image based on an anchor box set to detect an object, and detects the abnormal area at a position adjacent to the anchor box. According to claim 6,
wherein the learning unit outputs the analysis result including the coordinates of the center point of the abnormal region, the horizontal length of the abnormal region, the vertical length of the abnormal region, and the anchor box. According to claim 6,
The vehicle tire management device, characterized in that the learning unit outputs an anchor box whose object detection accuracy exceeds a predetermined value among a plurality of anchor boxes to the abnormal area. Taking a video by a plurality of cameras;
If the speed of the vehicle is less than the set speed, generating a tire image by recognizing the tire from the image recognition unit;
Based on deep learning, a learning unit analyzes the image of the tire and detects an abnormal region estimated by code rounding of the tire;
Determining whether or not the abnormal region is erroneously recognized by an abnormality determination unit to determine an abnormality in cord rounding of the tire; and
outputting a warning about an abnormality of the tire by an input/output unit; Method of operating a vehicle tire management device comprising a. According to claim 9,
The step of determining the above is,
An operation method of a vehicle tire management apparatus, characterized in that determining an abnormality of the tire based on an analysis result of a plurality of tire images while the tire rotates at least n times. According to claim 9,
The step of determining the above is,
The position and size of the abnormal area are calculated, and if the size of the abnormal area is less than a set size, it is determined as an erroneous recognition, and if the size of the abnormal area exceeds the set size, it is determined that the tire cord is rounded. Operation method of a vehicle tire management device to be. According to claim 9,
The step of detecting the abnormal area is,
An operation method of a vehicle tire management apparatus, characterized in that by analyzing the tire image based on an anchor box set to detect an object, and detecting the abnormal area at a position adjacent to the anchor box from the tire image. According to claim 12,
The step of detecting the abnormal area is,
An operation method of a vehicle tire management apparatus, characterized in that outputting an anchor box having an object detection accuracy of more than 70% among the plurality of anchor boxes as the abnormal area. According to claim 12,
The step of detecting the abnormal area is,
and outputting the analysis result including the coordinates of the center point of the abnormal region, the horizontal length of the abnormal region, the vertical length of the abnormal region, and the anchor box. According to claim 12,
The step of outputting the warning is,
outputting the warning with a combination of at least one of a warning message, a warning sound, and a warning light for the abnormality of the tire;
A method of operating a vehicle tire management apparatus, characterized in that outputting the tire image including the abnormal area.

Images