JP2023183769A - Tire tread surface abrasion determination system and method using deep artificial neural network - Google Patents

Abstract

To provide a system and a method for automatically determining the abrasion state of a tire tread surface using a deep artificial neural network with only a single image.SOLUTION: A tire tread surface abrasion determination system comprises: an image reception unit which receives an image of a tire tread surface; an image division unit which generates an image in which a tire portion and a background portion are divided from the image received by the image reception unit; and an output unit which outputs the abrasion level of the tire tread surface on the image generated by the image division unit as any of normal, replacement and dangerous by using a learned deep artificial neural network.SELECTED DRAWING: Figure 2

Description

The present invention relates to a system and method for determining tire tread surface wear, and more particularly to a system and method for automatically determining the wear state of a tire tread surface using only a single image and using a deep artificial neural network.

The tread surfaces of tires installed on passenger cars and freight vehicles in the city wear and erode in proportion to the distance traveled. Therefore, for safe driving, tires that have worn beyond a certain level must be replaced.

Figure 1 shows a structure for measuring the degree of wear called the wear limit line. Referring to Figure 1, in order to objectively judge the state of wear on the tire tread surface, tire manufacturers for each vehicle form a structure for measuring the degree of wear called the wear limit line 2 on the tire tread surface. There is. However, general consumers without specialized knowledge do not make use of such wear limit line 2, and experience difficulty in determining the wear state of automobile tires by themselves with the naked eye. As a result, a situation arises in which a person unnecessarily visits an automobile repair shop or a tire changing shop to find out whether or not a tire needs to be replaced in a situation where the time for replacing the tire is delayed or the tire does not need to be replaced.

In this regard, Patent Document 1 discloses a method and device for measuring tire wear so that it is not necessary to measure the depth of tire tread grooves one by one. Patent Document 1 discloses that when a tire wear measurement device receives a moving image of a tire, it generates a three-dimensional image based on this and determines the shape of the tire tread based on the depth of the tread area in the three-dimensionally formed image. Make it possible to measure the degree of wear.

Moreover, Patent Document 2 discloses a tire wear judgment device and method so that a warning can be automatically given that a tire is excessively worn. Patent Document 2 allows the degree of tire wear to be determined based on the braking distance of the vehicle calculated by a separate sensor unit.

In this way, many prior technologies have been proposed to automatically determine the degree of tire wear, but these methods require the complicated work of shooting and analyzing videos to measure the wear state of the tire tread surface. However, there are limitations as a separate sensor is required. Therefore, the present invention was derived through the Ministry of Science, Technology and Information Communication's 2020 Artificial Intelligence Online Competition Outstanding Achievement Business Commercialization Support (Problem Number: A0712-20-1015).

Korean registered patent No. 10-1534259 Korean registered patent No. 10-1469563

An object of the present invention is to determine the degree of tire wear by utilizing only a single image, and to reduce the amount of calculation required for algorithm calculation.

Another object of the present invention is to enable the wear state of a tire tread surface to be easily measured using a user's mobile device and a general imaging device without a separate sensor.

In order to achieve the above object, the present invention includes an image receiving unit that receives an image of a tire tread surface, and an image dividing unit that generates an image in which a tire portion and a background portion are divided from the image received by the image receiving unit. and an output unit that uses a learned deep artificial neural network to output the degree of wear of the tire tread surface on the image generated by the image segmentation unit as normal, replacement, or dangerous. is one of its characteristics.

Preferably, the image dividing unit may include a probability extraction module that extracts a probability that each pixel of the image received by the image receiving unit corresponds to a tire.

Preferably, the image dividing unit further includes a probability multiplication module that multiplies the probability extracted by the probability extraction module for each pixel corresponding to the image received by the image receiving unit.

Preferably, the method further includes a learning unit that trains the deep artificial neural network with images labeled as normal, replacement, or dangerous depending on the degree of tire wear based on the condition of the tread surface and the shading information between the treads. Can be done.

Preferably, the learning unit learns the deep artificial neural network using a single image, and the output unit outputs the degree of wear of a tire tread surface using a single image, thereby minimizing the amount of calculation.

Preferably, the vehicle may further include an image capturing unit that captures an image of the tire tread surface and transmits the image to the image receiving unit.

Preferably, the device may further include a display unit that displays the output value of the output unit.

The present invention also includes an image receiving step of receiving an image of a tire tread surface, an image dividing step of dividing the image received in the image receiving step into a tire portion and a background portion, and a step of determining the state of the tread surface and shadows between the treads. a training phase in which a deep artificial neural network is trained with images labeled as either normal, replacement, or critical depending on the degree of tire wear based on the information; and the trained deep artificial neural network is used to Another feature is that it includes an output step for outputting the degree of wear of the tire tread surface on the image generated in the dividing step as normal, replacement, or dangerous.

Preferably, the image segmentation step may include a probability extraction step of extracting a probability that each pixel of the image received in the image reception step corresponds to a tire.

Preferably, the image segmentation step may include a probability multiplication step of multiplying the probability extracted in the probability extraction step for each pixel corresponding to the image received by the image reception step.

According to the present invention, there is an advantage that the image dividing unit and the output unit that measure the degree of wear on the tire tread surface using an artificial neural network model can judge the degree of tire wear using only a single image. .

Further, the present invention has the advantage that the wear state of the tread surface can be measured using a user's camera without a separate sensor.

This shows a structure for measuring the degree of wear called the wear limit line. 1 shows a configuration diagram of a tire tread surface wear determination system according to an embodiment of the present invention. 1 shows a configuration diagram of a tire tread surface wear determination system in which algorithm calculations are performed at a user terminal according to an embodiment of the present invention. 1 shows a configuration diagram of a tire tread surface wear determination system in which algorithm calculations are performed on a server according to an embodiment of the present invention. 1 illustrates a deep artificial neural network structure according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited or limited by the exemplary embodiments. Identical reference numerals presented in each drawing indicate elements that perform substantially the same function.

The objects and effects of the present invention can be naturally understood or made clearer by the following description, but the objects and effects of the present invention are not limited to the following description. In addition, in explaining the present invention, if it is determined that a specific explanation of a known technique according to the present invention may unnecessarily obscure the gist of the present invention, the detailed explanation shall be provided. Omitted.

FIG. 2 shows a configuration diagram of a tire tread surface wear determination system 1 according to an embodiment of the present invention. Referring to FIG. 2, the tire tread wear determination system 1 may include a user terminal 10 and a server 30. The tire tread wear determination system 1 may include an image capturing section 100, an image receiving section 200, an image dividing section 300, an output section 400, a display section 500, and an information storage section 700.

The tire tread surface wear determination system 1 can determine the wear state of the tire tread surface using only a single image 4 taken by a user terminal 10 carried by an ordinary driver. The tire tread wear determination system 1 can inject a large amount of tire tread surface images and whether or not replacement is necessary determined from the images into a deep artificial neural network model to learn. The tire tread surface wear determination system 1 can determine the degree of wear based on the tread surface condition of the single image 4 and the shading information between the treads.

The tire tread wear determination system 1 can also be used by staff at a tire replacement store to easily determine the degree of wear on the tire tread surface of a customer vehicle visiting the tire replacement store and to provide guidance to the customer.

The tire tread surface wear determination system 1 can be provided in the form of an API because wear analysis can be performed remotely.

The user terminal 10 is a terminal that is portable and capable of transmitting and receiving data via a network, and includes a smartphone, a notebook, and the like. At this time, the user terminal 10 may be a terminal installed with software for outputting the degree of tire wear according to the embodiment of the present invention. The user terminal 10 can be connected to the server 30 via a wireless or wired network.

The server 30 can be configured to be capable of performing a large amount of calculations for training the deep artificial neural network. The server 30 can be connected to the user terminal 10 via a wired or wireless network.

The image capturing unit 100 may capture an image of the tire tread surface and transmit the image to the image receiving unit 200. The image capturing unit 100 may be included in the user terminal 10 or may be a third device. The image capturing unit 100 can transmit the captured image to the image receiving unit 200. The image capturing unit 100 can be connected to the image receiving unit 200 via a wireless or wired network. When the image capturing unit 100 is wirelessly connected to the image receiving unit 200, the image capturing unit 100 uses CDMA (Code Division Multiple Access), WCDMA (registered trademark) (Wideband CDMA), LTE (Long Term Evolution), or WiFi in a broad sense. Communication networks can be used.

The display unit 500 can display the output value of the output unit 400. The display unit 500 may be included in the user terminal 10 or may be a third device. The display unit 500 can display the degree of wear of the tire tread surface together with an image taken by a user or a driver.

The information storage unit 700 can store labeled images used by the learning unit 600. The information storage unit 700 can store images taken by a user or a driver to measure the degree of wear on the tire tread surface. The information storage unit 700 can allow images taken by a user or a driver to be used for training of a deep artificial neural network. The information storage unit 700 can store previously measured tire tread surface wear degree output values.

The image receiving unit 200 can receive an image of a tire tread surface. The image receiving unit 200 can receive images captured by the image capturing unit 100 or a general digital camera. The image receiving unit 200 is connected to a third device via a wired or wireless network, and can receive images taken or stored by the third device. The image receiving unit 200 may receive an image of a tire tread surface and transmit it to the image dividing unit 300.

The image receiving unit 200 can preprocess the received image. The image receiving unit 200 may perform preprocessing to normalize pixel values of the received image. If the pixel value of the received image is between 0 and 255, the image receiving unit 200 may preprocess (scale) the pixel value between −1 and 1. The image receiving unit 200 may perform preprocessing to adjust the size and resolution of the received image to be input to the deep artificial neural network.

The image dividing unit 300 can generate an image in which the tire portion and the background portion are divided from the image received by the image receiving unit 200. The image segmentation unit 300 may reduce image information for the background region so that the deep artificial neural network in the output unit 400 can concentrate calculations on the tire tread surface. The image dividing unit 300 can generate an image in which the tire part is divided, so that the image is less influenced by the background part. Therefore, the image dividing section 300 can output the degree of wear of the tire tread surface from the output section 400 with high accuracy.

FIG. 5 shows a deep artificial neural network structure according to an embodiment of the present invention. Referring to FIG. 5, the image segmentation unit 300 may include a probability extraction module 310 and a probability multiplication module 330.

The probability extraction module 310 may extract the probability that each pixel of the image received by the image receiving unit 200 corresponds to a tire. The probability extraction module 310 may extract a probability (Tread Probability Mask) of whether each pixel corresponds to a tire or a background at a pixel level using a segmentation model. In the probability extraction module 310, criteria for determining probability are not set by the user. That is, in the probability extraction module 310, the standard for determining probability is set by learning via a deep artificial neural network.

The probability extraction module 310 may extract the probability that a particular pixel corresponds to a tire to a value between 0 and 1. If the probability extraction module 310 extracts a pixel with a high probability, it can be determined that the pixel has a high probability of being part of a tire.

If the probability extracted by the probability extraction module 310 is greater than a certain reference value, the corresponding pixel can be considered as part of a tire. The reference value of the probability extraction module 310 can be set by a user. Further, the probability extraction module 310 may set the reference value through learning of a deep artificial neural network. In one embodiment, when the probability extracted by the probability extraction module 310 is greater than or equal to 0.5, the pixel may be considered as part of a tire.

The probability multiplication module 330 may multiply the probability extracted by the probability extraction module for each pixel corresponding to the image received by the image reception unit 200. The probability multiplication module 330 multiplies the pixel values of the existing image by the probability that each pixel corresponds to a tire, thereby diluting information of a background portion unrelated to the tire. The probability multiplication module 330 may generate an image in which information regarding the background region is diluted and information regarding the tire tread surface is enhanced by multiplying each pixel corresponding to the input image by the extracted probability.

The probability multiplication module 330 can generate a Mutiplied Activation Map 305 by multiplying the Activation Map 301, which is a model generated by a CNN-based deep artificial neural network, by the Tread Probability Mask 303. The probability multiplication module 330 can input the Mutiplied Activation Map 305 to the output unit 400.

The Mutiplied Activation Map 305 is an image in which the tire portion and the background portion are divided. The Mutiplied Activation Map 305 can be configured such that when the output unit 400 outputs the wear degree of the tire tread surface, it is less influenced by the background region.

The output unit 400 can output the degree of wear of the tire tread surface on the image generated by the image segmentation unit 300 as normal, replacement, or dangerous using the learned deep artificial neural network. The output unit 400 can measure the wear state of the tire tread surface by inputting the image generated by the image dividing unit 300 to a learned deep artificial neural network and analyzing pixels in the image.

The output unit 400 can use an artificial neural network-based neural network algorithm (DNN, CNN) as a deep artificial neural network model. When using a CNN (Convolutional Neural Network) as a deep artificial neural network model, the artificial neural network can be used with only minimal preprocessing. A CNN (Convolutional Neural Network) consists of one or more synthetic stacks and a general artificial neural network layer placed on top, further exploiting weights and integration layers. CNNs (Convolutional Neural Networks) have the advantage of being easier to train and using fewer parameters than traditional artificial neural network techniques.

From the value output by the output unit 400, the driver can determine whether tire replacement is necessary. If the value output by the output unit 400 is "normal", it means that there is little need for replacement, and if it is "replaced", it means that there is a need for replacement, and if it is "dangerous", it means that there is a need for replacement. This means that there is a need for immediate replacement.

The learning unit 600 can train the deep artificial neural network with images labeled as normal, replacement, or dangerous depending on the degree of tire wear based on the condition of the tread surface and the shading information between the treads. The images used in the learning section 600 are selected by an annotation group of three or more tire experts who have been evaluating tire wear levels for more than 10 years, and are selected as normal, replacement, or dangerous using a majority voting method. It can be classified and labeled.

The learning unit 600 can use an image in which the tire portion and the background portion are separated as input values for the deep artificial neural network to be trained. The learning unit 600 can use the image passed through the image dividing unit 300 as an input value. The learning unit 600 can improve the accuracy and precision of learning using images in which the tire portion is emphasized.

The learning unit 600 learns the deep artificial neural network using a single image, and the output unit 400 minimizes the amount of computation of the tire tread wear judgment system 1 by outputting the wear degree of the tire tread surface using a single image. can be converted into The single image 4 has the advantage that it requires less calculation in algorithm calculations than a moving image or multiple images.

According to this embodiment, the user or driver can easily measure the degree of wear of the tire tread surface using only the single image 4. The user or driver can check whether the tire needs to be replaced in three stages: normal, replacement, or dangerous, depending on the wear condition of the tire tread surface, using just a single image 4 taken with a smartphone. There is no need to visit a car center to check if it is available.

FIG. 3 shows a configuration diagram of a tire tread surface wear determination system 1 in which algorithm calculations are performed on the user terminal 10 according to an embodiment of the present invention. Referring to FIG. 3, the user terminal 10 may include an image capturing unit 100, an image receiving unit 200, an image dividing unit 300, an output unit 400, and a display unit 500. The server 30 can include a learning section 600 and an information storage section 700. According to this embodiment, the tire tread surface wear determination system 1 causes the learning unit 600 included in the server 30 to learn a deep artificial neural network. The tire tread wear determination system 1 transmits the learned deep artificial neural network from the learning unit 600 included in the server 30 to the output unit 400. The tire tread surface wear determination system 1 outputs the wear degree of the tire tread surface from the image dividing section 300 and the output section 400 included in the user terminal 10 .

FIG. 4 shows a configuration diagram of a tire tread surface wear determination system in which algorithm calculations are performed by the server 30 according to an embodiment of the present invention. Referring to FIG. 4, the user terminal 10 may include an image capturing unit 100, an image receiving unit 200, and a display unit 500. The server 30 can include an image segmentation section 300, an output section 400, a learning section 600, and an information storage section 700. According to this embodiment, the tire tread surface wear determination system 1 causes the learning unit 600 included in the server 30 to learn a deep artificial neural network. The tire tread surface wear determination system 1 transmits an image from an image receiving section 200 included in the user terminal 10 to an image dividing section 300 included in the server 30 . The tire tread surface wear determination system 1 outputs the degree of tire tread surface wear from an image dividing section 300 and an output section 400 included in the server 30.

In another embodiment of the present invention, a method for determining tire tread wear may include an image receiving step, an image segmentation step, a learning step, and an outputting step.

The image receiving step may receive an image of the tire tread surface. The image receiving step refers to the operation performed by the image receiving unit 200 described above.

The image dividing step can divide the image received in the image receiving step into a tire portion and a background portion. The image segmentation step refers to the operation performed by the image segmentation unit 300 described above.

The image segmentation step may include a probability extraction step and a probability multiplication step.

The probability extraction step may extract the probability that each pixel of the image received in the image reception step corresponds to a tire. The probability extraction step refers to the operation performed by the probability extraction module 310 described above.

In the probability multiplication step, the probability extracted in the probability extraction step may be multiplied for each pixel corresponding to the image received by the image reception step. The probability multiplication step refers to the operation performed by the probability multiplication module 330 described above.

The training phase can train the deep artificial neural network with images labeled as either normal, replacement, or critical depending on the degree of tire wear based on the tread surface condition and shading information between the treads. The learning stage refers to the operation performed by the learning unit 600 described above.

The output step can output the degree of wear of the tire tread surface on the image generated in the image segmentation step as either normal, replacement, or dangerous, using the trained deep artificial neural network. The output stage refers to the operation performed by the output unit 400 described above.

The tire tread wear determination system 1 can be stored and executed in the user terminal 10. The tire tread wear determination system 1 can be stored in the user terminal 10 in the form of an application.

A user or a driver can execute the vehicle management application to execute the tire tread wear determination system 1 included in the application. The driver can take a picture of the tire by touching the camera on the application. In the application, example photos can be presented to increase the accuracy of wear condition determination.

The image receiving section 200 receives a single image 4 taken by the user or the driver, and the output section 400 outputs the wear state. The application can display the need for tire replacement on the user terminal 10 as either normal, replacement, or critical depending on the final measurements.

Using a vehicle management application according to an embodiment of the present invention, through wear measurement from a single image 4, camera equipment of a mobile device owned by an ordinary driver can be used without a separate sensor attached to the vehicle. This allows drivers to check the degree of tire wear in real time without having to directly measure tread thickness.

Although the present invention has been described in detail through typical embodiments, those with ordinary knowledge in the technical field to which the present invention pertains will understand that the above-described embodiments deviate from the scope of the present invention. You will understand that many variations are possible. Therefore, the scope of rights in the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below, but also by all modifications and variations derived from the claims and equivalent concepts. It must be done.

1 Tire tread surface wear judgment system 2 Wear limit line 4 Single image 10 User terminal 30 Server 100 Image capturing section 200 Image receiving section 300 Image dividing section 301 Activation Map
303 Tread Probability Mask
305 Mutiplied Activation Map
310 probability extraction module 330 probability multiplication module 400 output section 500 display section 600 learning section 700 information storage section

The probability multiplication module 330 can generate a Mul tiplied Activation Map 305 by multiplying the Activation Map 301, which is a model generated by a CNN-based deep artificial neural network, by the Tread Probability Mask 303. The probability multiplication module 330 can input the Mul tiplied Activation Map 305 to the output unit 400.

The Mul tiplied Activation Map 305 is an image in which the tire part and the background part are divided. The Mul tiplied Activation Map 305 can be configured to be less influenced by background parts when the output unit 400 outputs the degree of wear of the tire tread surface.

Claims (10)

A tire tread wear determination system using a deep neural network,
an image receiving unit that receives an image of the tire tread surface;
an image dividing unit that generates an image in which a tire portion and a background portion are divided from the image received by the image receiving unit;
an output unit that outputs the degree of wear of the tire tread surface on the image generated by the image segmentation unit as normal, replacement, or dangerous using a trained deep artificial neural network; Wear judgment system. The image dividing section is
The tire tread wear determination system according to claim 1, further comprising a probability extraction module that extracts a probability that each pixel of the image received by the image receiving unit corresponds to a tire. The image dividing section is
The tire tread wear determination system according to claim 2, further comprising a probability multiplication module that multiplies the probability extracted by the probability extraction module for each pixel corresponding to the image received by the image reception unit. 2. The tire according to claim 1, further comprising a learning unit for training the deep artificial neural network with images labeled as normal, replacement, or dangerous depending on the degree of tire wear based on the condition of the tread surface and the shading information between the treads. tire tread wear determination system. The learning department is
Train a deep artificial neural network with a single image,
The output section is
Outputs the degree of wear on the tire tread surface in a single image,
The tire tread surface wear determination system according to claim 4, characterized in that the amount of calculation is minimized. The tire tread surface wear determination system according to claim 1, further comprising an image capturing section that captures an image of the tire tread surface and transmits the image to the image receiving section. The tire tread surface wear determination system according to claim 1, further comprising a display unit that displays the output value of the output unit. A method for determining tire tread surface wear using a deep neural network, the method comprising:
an image receiving step of receiving an image of the tire tread surface;
an image dividing step of dividing the image received in the image receiving step into a tire portion and a background portion;
a training step in which a deep artificial neural network is trained with images labeled as normal, replacement, or dangerous depending on the degree of wear of the tire based on the condition of the tread surface and the shading information between the treads;
an output step of outputting the degree of wear of the tire tread surface on the image generated in the image segmentation step as one of normal, replacement, or dangerous, using a trained deep artificial neural network; How to judge wear. The image segmentation step includes:
The method of claim 8, further comprising a probability extracting step of extracting a probability that each pixel of the image received in the image receiving step corresponds to a tire. The image segmentation step includes:
The method of claim 9, further comprising a probability multiplication step of multiplying the probability extracted in the probability extraction step for each pixel corresponding to the image received by the image reception step.

Images