KR102367702B1 - Tire size automatic recognition system and method using deep artificial neural network - Google Patents

Abstract

The present invention provides a system, which includes: an image receiving unit for receiving an image of a tire sidewall; a learning unit for training a deep artificial neural network using an image labeled with the position and type of a character indicating a tire standard value engraved on a tire sidewall; and an output unit for outputting a standard value of a tire from the image of the tire sidewall received by the image receiving unit using the learned deep artificial neural network.

Description

TIRE SIZE AUTOMATIC RECOGNITION SYSTEM AND METHOD USING DEEP ARTIFICIAL NEURAL NETWORK

The present invention relates to a system and method for automatic tire size recognition, and more particularly, to a system and method for automatically recognizing a tire size using a deep artificial neural network with only a single image captured without a separate sensor.

Cross-sectional width, aspect ratio, or wheel size information is engraved numerically on the sidewall part of tires installed on commercially available passenger cars and trucks during the production process to indicate the shape specifications of the tires.

1 shows characters indicating tire standard values engraved on a sidewall portion of a tire. Referring to FIG. 1 , the standard number of 225/45R17 91W is marked on the sidewall of the tire. From the front, 225 is the cross-sectional width (mm), 45 is the aspect ratio (%), R is the radial structure, 17 is the outer diameter of the rim, 91 is the load index, and W is the speed symbol.

This tire specification information is used to find a tire of the same size during maintenance or replacement of a tire, and is the most important information to inquire about price information of a tire. In the current tire price inquiry website and app service, it is difficult for consumers to extract the exact tire size from the sidewall by themselves.

2 shows examples of characters indicating tire standard values for each tire manufacturer. Referring to Figure 2, manufacturer A is marked with a standard value of 195/65R15, and manufacturer B is marked with a standard value of 215/45R18 and 89V. The positions of the characters marked for Manufacturer A and Manufacturer B are also different.

Therefore, since the standard number of the tire engraved on the sidewall of the tire differs depending on the manufacturer, the location, method, and number of digits are different. there is.

In this regard, No. 10-0511004 discloses a system and method for recognizing a tire standard so that the tire standard can be accurately recognized without causing a tire standard recognition error, and operating and investment costs can be greatly reduced. do. The above-mentioned prior patent document allows the tire standard to be recognized by combining the mold number recognized by the three-dimensional laser scanner and the tread color line recognized by the color recognition image system.

As such, in the conventional case, a number of prior literatures for automatically recognizing characters indicating the standard value of a tire have been proposed, but general consumers have to use a 3D laser scanner equipment that is difficult to operate, and the fact that it is engraved on the tire sidewall Since it is a method that recognizes color lines marked on the tread surface rather than numerical values, there is a limitation that it cannot be applied to tires without color lines marked. Therefore, this specification was derived through the support of the Ministry of Science and Technology Information and Communication's 2020 artificial intelligence online contest excellent performance and commercialization support. (task number: A0712-20-1015)

Korean Patent Registration No. 10-0511004

An object of the present invention is to easily recognize characters representing standard values of tires by using a user's mobile device and a general photographing apparatus without a separate sensor.

Another object of the present invention is to make it possible to accurately recognize characters representing standard values of tires even in the shooting patterns of general consumers who take pictures in various environments and angles.

In order to achieve the above object, the present invention provides an image receiving unit for receiving an image of a tire sidewall; a learning unit that trains a deep artificial neural network by using an image labeled with the position and type of the character indicating the tire standard value engraved on the tire sidewall; and an output unit for outputting a standard value of a tire from the image of the tire sidewall received by the image receiving unit using the learned deep artificial neural network.

Preferably, the learning unit can train the deep artificial neural network with an image in which the text indicating the standard value of the tire is divided into boxes, and coordinates on the sidewall of the tire where the box is located are labeled on the box.

Preferably, the learning unit can train the deep artificial neural network with an image labeled by classifying characters indicating the standard value of a tire into boxes, classifying the types of characters in the box, and labeling.

Preferably, the output unit may include a character recognition module that recognizes all characters on the image of the tire sidewall and classifies each recognized character into a box.

Preferably, the output unit may further include a character classification module for classifying the types of characters in the box divided by the character recognition module.

Preferably, the output unit calculates a probability that the character classified by the character classification module is a character indicating a standard value of the tire based on the coordinates on the tire sidewall where the box is located, and the probability exceeds a specific threshold value It may further include a character extraction module for extracting characters.

Preferably, the output unit may further include a character reconstruction module that reconstructs the characters extracted by the character extraction module in such a way as to indicate the standard value of the tire.

Preferably, the output unit may further include a character reconstruction module that reconstructs the characters extracted by the character extraction module in such a way as to indicate the standard value of the tire.

The present invention also provides an image receiving step of receiving an image of a tire sidewall; A learning step of learning a deep artificial neural network with an image labeled based on the position and type of the character indicating the standard value of the tire engraved on the tire sidewall; and an output step of outputting a tire standard value from the tire sidewall image received in the image receiving step using the learned deep artificial neural network.

Preferably, the outputting step may include a character recognition step of recognizing all characters on the image of the tire sidewall and classifying each recognized character with a box.

Preferably, the outputting step may further include a character classification step of classifying the types of characters in the box divided by the character recognition step.

Preferably, the outputting step calculates a probability that the character classified in the character classification step is a character indicating a standard value of the tire based on the coordinates on the tire sidewall where the box is located, and the probability sets a specific threshold value It may further include a character extraction step of extracting more than the character.

Preferably, the outputting step may further include a character reconfiguration step of reconstructing the characters extracted by the character extraction step in such a way as to indicate a standard value of a tire.

According to the present invention, there is an advantage that the output unit can output the standard value of the tire from the image of the tire sidewall by using the artificial neural network model as a single image photographed by the user without a separate sensor.

In addition, the present invention has the advantage that the learning unit can accurately recognize the characters representing the standard value of the tire even in the shooting pattern of general consumers who take pictures in various environments and angles through learning using various images.

1 shows the letters indicating the tire standard value engraved on the sidewall portion of the tire.
2 shows examples of characters indicating tire standard values for each tire manufacturer.
3 is a block diagram of a tire standard automatic recognition system according to an embodiment of the present invention.
4 is a block diagram of an output unit according to an embodiment of the present invention.
5 shows an example in which the character extraction module extracts characters according to an embodiment of the present invention.
6 is a block diagram of a character reconstruction module according to an embodiment of the present invention.
7 is an embodiment of the connection algorithm of the neighboring character connection module. FIG. 7 (a) shows the step of calculating the center coordinates of the box, and FIG. 7 (b) shows the step of calculating the distance between the letter centers. and FIG. 7( c ) shows the step of connecting the nearest character centers.
Fig. 8 is an embodiment of the concatenation algorithm of the neighboring character filtering module. Fig. 8 (a) shows a state in which more than two nearest neighboring characters are connected, and Fig. 8 (b) shows an erroneously connected nearest neighboring character. indicates the removed state.
9 is an embodiment related to the connection algorithm of the character sequence determination module. FIG. 9 (a) shows that the order and pattern of tire specification values match, and FIG. 9 (b) shows the order and pattern of tire specification values. This indicates a mismatch.
10 is an embodiment of an algorithm for outputting a tire standard value of an output unit according to an embodiment of the present invention. FIG. 10 (a) shows a step of recognizing a character, and FIG. Fig. 10 (c) shows the step of extracting the character, and Fig. 10 (d) shows the step of reconstructing the character.
11 shows the structure of a deep artificial neural network according to an embodiment of the present invention.
12 is a block diagram of a tire standard automatic recognition system in which an algorithm operation is performed in a user terminal according to an embodiment of the present invention.
13 is a block diagram of a tire standard automatic recognition system in which an algorithm operation is performed in a server according to an embodiment of the present invention.
14 illustrates a screen for performing automatic tire standard recognition of a vehicle management application according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the contents described in the accompanying drawings. However, the present invention is not limited or limited by the exemplary embodiments. The same reference numerals provided in the respective drawings indicate members that perform substantially the same functions.

Objects and effects of the present invention can be naturally understood or made clearer by the following description, and the objects and effects of the present invention are not limited only by the following description. In addition, in the description of the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted.

3 is a block diagram of a tire standard automatic recognition system 1 according to an embodiment of the present invention. Referring to FIG. 3 , the tire standard automatic recognition system 1 may include a user terminal 10 and a server 30 . The tire standard automatic recognition system 1 may include an image capturing unit 100 , an image receiving unit 200 , an output unit 300 , a display unit 400 , a learning unit 500 , and an information storage unit 600 . can

The tire standard automatic recognition system 1 may automatically recognize the tire standard only with a single image of the tire sidewall photographed by the user terminal 10 possessed by ordinary drivers. The tire standard automatic recognition system 1 can learn by injecting the tire sidewall images taken from various environments and angles and the learning data labeling the positions and types of characters in the sidewall into the deep artificial neural network model. The tire standard automatic recognition system 1 may recognize a character indicating a tire standard value from a sidewall image taken from a mobile device without a separate light and scanner.

The user terminal 10 is a portable terminal capable of transmitting and receiving data through a network, and includes a smart phone, a notebook computer, and the like. In this case, the user terminal 10 is a terminal in which software for automatically recognizing a tire standard according to an embodiment of the present invention is installed. The user terminal 10 may be connected to the server 30 through a wireless or wired network.

The server 30 may be configured to enable a large amount of computation for learning of the deep artificial neural network. The server 30 may be connected to the user terminal 10 through a wired or wireless network.

The image capturing unit 100 may capture an image of the tire sidewall and transmit it to the image receiving unit 200 . The image capture unit 100 may be provided in the user terminal 10 or may be a third device. The image capturing unit 100 may transmit the captured image to the image receiving unit 200 . The image capturing unit 100 may be connected to the image receiving unit 200 through a wireless or wired network. The image capturing unit 100 may use a broad mobile communication network such as CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), LTE (Long Term Evolution) or WiFi when wirelessly connected to the image receiving unit 200 .

The display unit 400 may display an output value of the output unit 300 . The display unit 400 may be provided in the user terminal 10 or may be a third device. The display unit 400 may display the model name of the tire and the standard value of the tire.

The information storage unit 600 may store a labeled image used in the learning unit 500 . The information storage unit 600 may store an image captured by a user or a driver to recognize a tire standard value engraved on a tire sidewall. The information storage unit 600 may store images captured by a user or a driver to be used for learning a deep artificial neural network.

The image receiving unit 200 may receive an image of the tire sidewall. The image receiving unit 200 may receive an image captured by the image capturing unit 100 or a general digital camera. The image receiver 200 may be connected to a third device through a wired or wireless network to receive an image photographed or stored by the third device. The image receiving unit 200 may receive the image of the tire sidewall and transmit it to the output unit 300 .

The image receiver 200 may pre-process the received image. The image receiving unit 200 may perform pre-processing in a manner of normalizing pixel values of the received image. In this embodiment, when the pixel value of the received image is between 0 and 255, the image receiving unit 200 may preprocess (scale) the pixel value to be between -1 and 1. The image receiving unit 200 may perform pre-processing of adjusting the size and resolution of the received image to be input to the deep artificial neural network.

The learning unit 500 may learn the deep artificial neural network by using the image labeled with the position and type of the character indicating the tire standard value engraved on the tire sidewall. The image used in the learning unit 500 may be labeled with a box including letters indicating numerical values engraved on a tire sidewall by constituting a plurality of human annotation groups.

The learning unit 500 may learn the deep artificial neural network from a single image, and the output unit 300 may automatically recognize the tire standard only with a single image. A single image has an advantage in that the amount of computation required for algorithm operation is smaller than that of a 3D image or a plurality of images.

The learning unit 500 may classify the text indicating the standard value of the tire into boxes, and train the deep artificial neural network with an image in which the coordinates on the sidewall of the tire where the box is located are labeled on the box. The learning unit 500 may learn a position of a character indicating a tire standard value among various characters existing on the tire interwall through box labeling.

The learning unit 500 may classify characters representing standard values of tires into boxes, classify the types of characters in the boxes, and train the deep artificial neural network with labeled images. In order to accurately classify the same type of characters that can be expressed differently depending on the font or size, the learning unit 500 may classify the type of character and learn from the labeled image.

The learning unit 500 divides the text indicating the standard value of the tire into boxes, labels the box with coordinates on the sidewall of the tire where the box is located, and classifies and labels the type of letter in the box. You can learn the coordinates and types of standard numerical values on the month.

The output unit 300 may output the standard value of the tire from the image of the tire sidewall received by the image receiving unit 200 using the learned deep artificial neural network. The output unit 300 may use an artificial neural network-based neural network algorithm (DNN, CNN) as a deep artificial neural network model. When CNN (Convolutional Neural Network) is used as a deep artificial neural network model, artificial neural networks can be used with minimal preprocessing. CNN (Convolutional Neural Network) consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and integration layers. Convolutional Neural Networks (CNNs) have the advantage of being easier to train than conventional artificial neural network techniques and using a small number of parameters.

With the value output by the output unit 300 , the driver can check the model name of the tire and the standard value of the tire. In this embodiment, the output unit 300 may output a tire standard value such as 215/60R/16 in the image of the tire sidewall. Here, 215 is the cross-sectional width (mm), 60 is the aspect ratio (%), R is the radial racer, and 16 is the outer diameter of the rim.

4 is a block diagram of an output unit 300 according to an embodiment of the present invention. Referring to FIG. 4 , the output unit 300 may include a character recognition module 310 , a character classification module 330 , a character extraction module 350 , and a character reconstruction module 370 .

The character recognition module 310 may recognize all characters on the image of the tire sidewall and classify each recognized character into a box. The character recognition module 310 may recognize all the characters displayed in the image of the tire sidewall without discriminating whether they are characters representing the tire standard value. The character extraction module 350 may determine whether the character recognized by the character recognition module 310 is a character representing a tire standard value.

The text classification module 330 may classify the type of text in the box divided by the text recognition module 310 . The character classification module 330 may classify the types of all characters including numbers, letters, or symbols. The character classification module 330 can accurately classify all characters on the tire sidewall image using a deep artificial neural network learned from images taken with various patterns such as various brightness, various angles, or presence or absence of lighting.

The character extraction module 350 calculates the probability that the character classified by the character classification module 330 is a character indicating the standard value of the tire based on the coordinates on the sidewall of the tire where the box is located, and the probability sets a specific threshold value Exceeding characters can be extracted. In the character extraction module 350, the criterion for calculating the probability is not set by the user. That is, in the character extraction module 350, the criterion for calculating the probability is set to learning through the deep artificial neural network.

The character extraction module 350 may calculate a probability that a specific character corresponds to a character indicating a tire standard value as a value between 0 and 1. If the value calculated by the character extraction module 350 is large, it can be seen that the probability that the corresponding character corresponds to the character indicating the tire standard value is high.

When the probability calculated by the character extraction module 350 is equal to or greater than a certain reference value, the corresponding character may be viewed as a character indicating a tire standard value. The text extraction module 350 may set the reference value by the user. In addition, the text extraction module 350 may set the reference value through learning of the deep artificial neural network. In this embodiment, when the probability calculated by the character extraction module 350 is 0.5 or more, the corresponding character may be viewed as a character indicating the tire standard value.

The character extraction module 350 may extract only the characters whose calculated probability is equal to or greater than the reference value as characters representing the tire standard value. The characters extracted by the character extraction module 350 may be reconstructed according to a method in which the character reconstruction module 370 displays the tire standard value.

5 shows an example of extracting characters by the character extraction module according to an embodiment of the present invention. Referring to FIG. 5 , the character extraction module 350 may extract the standard value of the tire regardless of the arrangement (direction) of the standard value of the tire photographed by the user. The character extraction module 350 does not require an image obtained by photographing the standard value of the tire at a specific angle and direction. Accordingly, the character extraction module 350 may provide convenience to the user in photographing the standard value of the tire.

The character reconstruction module 370 may reconstruct the characters extracted by the character extraction module 350 in such a way as to indicate the standard value of the tire. The character reconstruction module 370 may reconstruct the characters in the order of cross-sectional width/aspect ratio/radial lacer/external diameter of the rim.

6 is a block diagram of a character reconstruction module 370 according to an embodiment of the present invention. Referring to FIG. 6 , the character reconstruction module 370 may include a neighboring character connection module 371 , a neighboring character filtering module 373 , and a character order determining module 375 . The character reconstruction module 370 correlates with the angle (direction) of the tire standard value displayed in the image of the tire side surface through the neighboring character connection module 371 , the neighboring character filtering module 373 , and the character order determining module 375 . It is possible to reconstruct tire specification figures without Accordingly, the character reconstruction module 370 may be recognized normally even if the direction in which the tire standard values are photographed is not separately aligned and photographed. Through this, the character reconstruction module 370 may provide strong recognition performance for a photographing environment having various angles and arrangements.

7 is an embodiment related to the connection algorithm of the neighboring character connection module 371. FIG. 7 (a) shows the step of calculating the center coordinates of the box, and FIG. 7 (b) shows the calculation of the distance between the character centers. 7(c) shows a step of connecting the nearest character centers.

Referring to FIG. 7A , the neighbor text connection module 371 may calculate the center coordinates of each box including the text extracted by the text extraction module 350 . The character extraction module 350 may calculate the intersection of the diagonals of each box as the central coordinate. The center coordinates of the box calculated by the neighboring character connection module 371 may be used to measure the distance to the neighboring character. Referring to FIG. 7B , the neighboring character connection module 371 may connect any one character to the neighboring character. The neighbor character connection module 371 may connect the 1st, 2nd, and 3rd characters from the left and 5th, 6th, 7th, 8th, and 9th characters from the right and the 4th character based on the 4th character. Referring to FIG. 7C , the neighbor character connection module 371 may release the connection except for the left and right neighboring characters that are closest to each other among the neighboring characters connected to any one character.

According to the present embodiment, the character connection module 371 may connect the characters extracted by the character extraction module 350 in order through this process. That is, the character connection module 371 is the first character 2 times, the 2nd character 3 times, the 3rd character 4 times, the 4th character 5 times, the 5th character 6 times, and the 6th character 7 times , by connecting letter 8 with number 9, it is possible to reconstruct the standard value of the tire composed of the order of letters 1 to 9 or the order of letter 9 to letter 1.

When the number of characters connected by the neighboring character connection module 371 to any one character exceeds two, the neighboring character filtering module 373 may perform a filtering process such that the number of connected characters is two or less. In particular, in the case of a character at the end, since there is no neighboring character on one side, two nearest neighboring characters can be connected in the same direction, so a filtering process may be required.

8 is an embodiment of the connection algorithm of the neighbor character filtering module 373. FIG. 8 (a) shows a state in which more than two nearest neighbor characters are connected, and FIG. 8 (b) shows the erroneously connected latest Indicates the state in which the adjacent character has been removed.

Referring to (a) of FIG. 8 , the 3rd character is simultaneously connected to the 1st and 2nd characters to the left and connected to the 4th character to the right. Thus, character 3 is associated with more than 2 characters. Here, since the first character is not the closest neighboring character to the third character, it is erroneously connected. It can be seen that the 7th character is also incorrectly connected with the 9th character.

Referring to (b) of FIG. 8 , the neighbor character filtering module 373 may confirm that the connection between the first character erroneously connected to the third character and the ninth character erroneously connected to the 7th character is removed. The filtering method of the neighbor character filtering module 373 is as follows. When the neighboring character filtering module 373 detects that more than two characters are connected to any one character, the neighboring character filtering module 373 may check the angle of a line connected to each character. The neighboring character filtering module 373 may not remove the connection because the characters are connected differently to the left and right when the angle of the connected line segments is greater than or equal to 90 degrees. If the angle of the connected line segment is less than or equal to 90 degrees, the neighboring character filtering module 373 may delete a longer segment from among the two connecting lines because two characters are connected in either the left or right direction.

The character order determining module 375 may determine the order of the characters filtered and connected by the neighboring character filtering module 373 . The character order determining module 375 may reconstruct the character by determining either a forward direction or a reverse direction according to a connection between characters. The character order determination module 375 may define the first and last characters of one character array connected by the closest characters to each other as Head or Tail.

9 is an embodiment related to the connection algorithm of the character order determination module 375. FIG. 9 (a) shows that the order and pattern of tire standard values match, and FIG. 9 (b) shows the tire standard values. Indicates that the order and pattern do not match.

Referring to FIG. 9A , the character order determination module 375 may define the first character as the head and the ninth character as the tail. The character order determining module 375 may arrange the characters from the Head to the Tail in order and check whether they match the standard numerical arrangement pattern of the tire. According to this embodiment, the letters from the head to the tail are 195/65R15, which is the standard numerical arrangement pattern of the tire consisting of (cross-sectional width) + (/) + (flat ratio) + (radial racer) + (outer diameter of rim) and match Accordingly, the character order determining module 375 may determine to reconstruct the first character in the order of the characters from the Head to the Tail by viewing the first character as the head and the ninth character as the tail.

Referring to FIG. 9B , the character order determining module 375 may define the ninth character as the head and the first character as the tail. The character order determining module 375 may arrange the characters from the Head to the Tail in order and check whether they match the standard numerical arrangement pattern of the tire. According to this embodiment, the letters from the head to the tail are 195/65R15, which is the standard numerical arrangement pattern of the tire consisting of (cross-sectional width) + (/) + (flat ratio) + (radial racer) + (outer diameter of rim) and does not match Accordingly, the character order determining module 375 may determine that the 9th character is not reconstructed in the order of the characters from the Head to the Tail by viewing the 9th character as the Head and the 1st character as the Tail.

10 is an embodiment of an algorithm for outputting a tire standard value of an output unit according to an embodiment of the present invention. FIG. 10 (a) shows a step of recognizing a character, and FIG. Fig. 10 (c) shows the step of extracting the character, and Fig. 10 (d) shows the step of reconstructing the character.

Referring to (a) of FIG. 10 , the character recognition module 310 may recognize all characters displayed in the image of the tire side surface in addition to the tire standard values. The character recognition module 310 may recognize characters engraved on the lower end in addition to the tire standard values. Referring to FIG. 10B , the character classification module 330 may classify the character recognized by the character recognition module 310 . The character classification module 330 may classify the recognized characters such as 195/65R15, 002511, and MADE IN KOREA.

Referring to (c) of FIG. 10 , the character extraction module 350 may extract only 195/65R15, which is a standard value of a tire, from among 195/65R15, 002511, and MADE IN KOREA. Referring to (d) of FIG. 10 , the character reconstruction module 370 converts the standard value of the tire to 195/65R15 that matches the standard numerical pattern of the tire among 195/65R15 or 51R56/591 according to the connection order of the extracted characters. can be reconstructed.

11 shows the structure of a deep artificial neural network according to an embodiment of the present invention. Referring to FIG. 11 , when the image of the tire sidewall is input to the CNN-based deep artificial neural network, the character recognition module 310 recognizes both the character 3 indicating the tire specification value and the character 5 not indicating the tire specification value. can recognize The character recognition module 310 recognizes all the characters of 195/65R15(3) and made in korea (5). The character classification module 330 may receive each character recognized by the character recognition module 310 and classify the type of each character. The character recognition module 310 may classify the types of characters such as 1, 9, 5, and / irrespective of their order. The character extraction module 350 may extract 195/65R15(3) as characters representing tire standard values, and may not extract made in korea, etc. (5) as characters that do not represent tire standard values. The character reconstruction module 370 converts the characters 1, 9, 5, /, 6, 5, R, 1, and 5, which are extracted by the character extraction module 350 regardless of the order, to 195/65R15, which is a form representing the tire standard value. can be reconstructed as

12 is a block diagram of a tire standard automatic recognition system 1 in which an algorithm operation is performed in the user terminal 10 according to an embodiment of the present invention. Referring to FIG. 12 , the user terminal 10 may include an image capturing unit 100 , an image receiving unit 200 , an output unit 300 , and a display unit 400 . The server 30 may include a learning unit 500 and an information storage unit 600 . According to this embodiment, the tire standard automatic recognition system 1 trains the deep artificial neural network in the learning unit 500 included in the server 30 . The tire standard automatic recognition system 1 transmits the learned deep artificial neural network from the learning unit 500 included in the server 30 to the output unit 300 . The tire standard automatic recognition system 1 outputs the tire standard value from the output unit 300 included in the user terminal 10 .

13 is a block diagram of a tire standard automatic recognition system 1 in which algorithm operation is performed in the server 30 according to an embodiment of the present invention. Referring to FIG. 13 , the user terminal 10 may include an image capturing unit 100 , an image receiving unit 200 , and a display unit 400 . The server 30 may include an output unit 300 , a learning unit 500 , and an information storage unit 600 . According to this embodiment, the tire standard automatic recognition system 1 trains the deep artificial neural network in the learning unit 500 included in the server 30 . The tire standard automatic recognition system 1 transmits an image from the image receiving unit 200 included in the user terminal 10 to the output unit 300 included in the server 30 . The tire standard automatic recognition system 1 outputs the tire standard value from the output unit 300 included in the server 30 .

In another embodiment of the present invention, the method for automatically recognizing a tire standard may include an image receiving step, a learning step, and an outputting step.

The image receiving step may receive an image of the tire sidewall. The image receiving step refers to an operation performed by the above-described image receiving unit 200 .

In the learning stage, a deep artificial neural network can be trained with an image labeled based on the position and type of the character indicating the standard value of the tire engraved on the tire sidewall. The learning step refers to an operation performed by the above-described learning unit 500 .

The output step may output the standard value of the tire from the image of the tire sidewall received by the image receiving step using the learned deep artificial neural network. The output step means an operation performed by the above-described output unit 300 .

The outputting step may include a text recognition step, a text classification step, a text extraction step, and a text reconstruction step.

In the character recognition step, all characters on the image of the tire sidewall may be recognized, and each recognized character may be divided into a box. The character recognition step refers to an operation performed by the aforementioned character recognition module 310 .

The text classification step may classify the type of text in the box divided by the text recognition step. The text classification step refers to an operation performed by the aforementioned text classification module 330 .

The character extraction step calculates the probability that the character classified by the character classification step is a character indicating the standard value of the tire based on the coordinates on the tire sidewall where the box is located, and extracts characters whose probability exceeds a specific threshold. can The character extraction step refers to an operation performed by the above-described character extraction module 350 .

In the character reconstruction step, the characters extracted by the character extraction step can be reconstructed in such a way that the standard value of the tire is indicated. The text reconstruction step refers to an operation performed by the aforementioned text reconstruction module 370 .

14 shows a screen for performing automatic tire standard recognition of a vehicle management application according to an embodiment of the present invention. Referring to FIG. 14 , the tire standard automatic recognition system 1 may be stored and executed in the user terminal 10 . The tire standard automatic recognition system 1 may be stored in the user terminal 10 in the form of an application.

The user or driver may execute the vehicle management application to execute the tire standard automatic recognition system 1 included in the application. After selecting whether the tire to be photographed in the application is the right front tire, right rear tire, left front tire, or left rear tire, the driver may touch the photographing/uploading portion to photograph the tire sidewall portion. The application may present an example photo to increase the recognition of the tire specification value. The application may output the model name of the tire and the tire standard value as the photographed image and display it on the screen. The application may calculate the tire replacement cost based on the printed tire model and tire specification value.

Although the present invention has been described in detail through representative embodiments above, those of ordinary skill in the art to which the present invention pertains will understand that various modifications are possible within the limits without departing from the scope of the present invention with respect to the above-described embodiments. will be. Therefore, the scope of the present invention should not be limited to the described embodiments and should be defined by all changes or modifications derived from the claims and equivalent concepts as well as the claims to be described later.

1: Tire size automatic recognition system
3: Characters indicating the tire size
5: Characters that do not indicate tire specification values
10: user terminal
30 : Server
100: image recording unit
200: image receiving unit
300: output unit
310: character recognition module
330: character classification module
350: character extraction module
370: character reconstruction module
371: Neighbor character connection module
373: neighbor character filtering module
375: character sequencing module
400: display unit
500: study department
600: information storage unit

Claims (14)

In the tire standard automatic recognition system using a deep artificial neural network,
an image receiving unit for receiving an image of a tire sidewall;
a learning unit that trains a deep artificial neural network by using an image labeled with the position and type of the character indicating the tire standard value engraved on the tire sidewall; and
An output unit for outputting a tire standard value from the image of the tire sidewall received by the image receiving unit using the learned deep artificial neural network;
the output unit,
a character reconstruction module that reconfigures the characters constituting the standard value of the tire in a way that indicates the standard value of the tire;
The character reconstruction module,
Each letter constituting the standard value of the tire is connected with the nearest neighboring letter, and the letters are reconstructed in the order of connection that matches the marking pattern of the standard value of the tire,
A tire standard automatic recognition system, characterized in that it automatically recognizes the tire standard value regardless of the angle (direction) of the tire standard value displayed in the image of the tire sidewall.
The method of claim 1,
The learning unit,
A tire standard automatic recognition system, characterized in that the text indicating the standard value of the tire is divided into boxes, and the deep artificial neural network is trained with the image in which the coordinates on the sidewall of the tire where the box is located are labeled on the box.
The method of claim 1,
The learning unit,
A tire standard automatic recognition system, characterized in that the text indicating the standard value of the tire is divided into boxes, and the deep artificial neural network is trained using the labeled images by classifying the letter types in the box.
The method of claim 1,
the output unit,
A tire standard automatic recognition system comprising a character recognition module that recognizes all characters on the image of a tire sidewall and classifies each recognized character into a box.
5. The method of claim 4,
the output unit,
Tire standard automatic recognition system, characterized in that it further comprises a character classification module for classifying the type of the character in the box divided by the character recognition module.
6. The method of claim 5,
the output unit,
A character extraction module that calculates the probability that the character classified by the character classification module is a character indicating the standard value of the tire based on the coordinates on the tire sidewall where the box is located, and extracts characters whose probability exceeds a specific threshold Tire standard automatic recognition system, characterized in that it further comprises.
The method of claim 1,
The character reconstruction module,
a neighbor character connection module that calculates the center coordinates of the box containing the characters, measures the distance to the neighboring characters through the central coordinates, and connects them with the left and right nearest neighboring characters;
a neighboring character filtering module that removes non-nearest neighboring characters when the number of neighboring characters connected to one character exceeds two; and
A tire standard automatic recognition system comprising: a character sequence determining module that determines a direction that matches a marking pattern of a standard value of a tire among the forward and reverse directions of the connected characters.
The method of claim 1,
The tire standard automatic recognition system further comprising: an image photographing unit that captures an image of the tire sidewall and transmits the image to the image receiving unit.
The method of claim 1,
The tire standard automatic recognition system further comprising a; a display unit for displaying the output value of the output unit.
In the automatic tire standard recognition method using a deep artificial neural network,
an image receiving step of receiving an image of a tire sidewall;
A learning step of learning a deep artificial neural network with an image labeled based on the position and type of the character indicating the standard value of the tire engraved on the tire sidewall; and
An output step of outputting a tire standard value from the tire sidewall image received in the image receiving step using the learned deep artificial neural network;
The output step is
a character reconstruction step of reconstructing the characters constituting the standard value of the tire in a way that indicates the standard value of the tire;
The character reconstruction step is,
Each letter constituting the tire standard value is connected with the nearest neighboring letter, and the letters are reconstructed in the order of connection that matches the marking pattern of the tire standard value,
A method for automatically recognizing the tire standard value regardless of the angle (direction) of the tire standard value displayed in the image of the tire sidewall. How to automatically recognize tire size.
11. The method of claim 10,
The output step is
A method for automatically recognizing a tire standard, comprising: a character recognition step of recognizing all characters on the image of a tire sidewall and classifying each recognized character into a box.
12. The method of claim 11,
The output step is
The automatic tire standard recognition method according to claim 1, further comprising a character classification step of classifying the types of characters in the box divided by the character recognition step.
13. The method of claim 12,
The output step is
A character extraction step of calculating the probability that the character classified by the character classification step is a character indicating the standard value of the tire based on the coordinates on the tire sidewall where the box is located, and extracting characters whose probability exceeds a specific threshold Tire standard automatic recognition method, characterized in that it further comprises.
11. The method of claim 10,
The character reconstruction step is,
a neighboring character connection step of calculating the central coordinates of the box containing the characters, measuring the distance to the neighboring characters through the central coordinates, and connecting the left and right nearest neighboring characters;
a neighboring character filtering step of removing non-nearest neighboring characters when the number of neighboring characters connected to one character exceeds two; and
A method for automatically recognizing a tire standard, comprising: a character order determining step of determining a direction that matches a marking pattern of a standard value of a tire among the forward and reverse directions of the connected characters.

Images