KR102162641B1 - System and method for recognizing meter information based on deep learning - Google Patents

Abstract

Disclosed are a meter information recognition system based on deep learning and a method thereof. In the present invention, remote meter reading of the accumulated amount using a portable device is possible, remote meter reading is possible without restrictions on the sort and type of meter, the effect for a photographing environment is minimized so as to ensure accurate meter reading in various environments, and automatically recognizes the accumulated amount and a unique number of the meter in a meter reading system of the meter using deep learning, so that a separate measuring means is not installed on a mechanical meter, and remote meter reading can be performed through portable devices of a meter reader or a user who has installed an app linked with the meter information recognition system.

Description

Deep learning based meter information recognition system and method {SYSTEM AND METHOD FOR RECOGNIZING METER INFORMATION BASED ON DEEP LEARNING}

The present invention relates to a system and method for recognizing meter information based on deep learning, and more specifically, remote meter reading of the accumulated amount using a portable device is possible, and remote meter reading is possible without restrictions on the type and type of meter, The present invention relates to a deep learning-based meter information recognition system and method that automatically recognizes the accumulated amount and unique number of a meter in a meter meter reading system using deep learning so that the effect on the shooting environment is minimized so that accurate meter reading can be performed in various environments.

In general, in order to check the integrated amount of an integrated meter that displays the amount of electricity, gas, water, etc., the meter reader visually checks the integrated amount on a predetermined day, and the rate for the integrated amount subtracted from the value of the previous month. After applying, issue an invoice.

In the conventional meter reading system, a meter reading method is used in which the meter reader manually records the meter reading amount identified through the meter in the meter reading amount field corresponding to each address and customer number.

However, the conventional meter reading system has a problem that an error may occur when a meter reader incorrectly records or inputs handwritten data into a database of a server through an input means such as a keyboard.

In addition, even when a handheld terminal such as a PDA is used instead of handwriting, there is a problem that the meter reader may input incorrectly.

In addition, there were various problems such as refusal for the meter reader to visit each home and verbal violence against the meter reader.

In addition, an incorrect usage fee is added, which is the subject of civil complaints, causing distrust in the meter reading administration, and accordingly, the cost used for re-checking becomes a factor of national waste.

In recent years, in order to solve this problem, a digital meter with a remote data transmission device added to each customer is installed as a remote meter reading method, so that the meter reader does not visit the meter individually, and the meter reading data read through the digital meter is transmitted from a distance. A method of calculating the usage fee using the number transmitted to the server and adding it to each customer is proposed.

Korean Patent Application Publication No. 10-2004-0000261 (name of the invention: a remote meter reading device for a meter, a remote meter reading system and a remote meter reading method using the same) discloses a configuration capable of remote meter reading through wireless.

However, the remote meter reading using wireless has a problem in that the accumulated amount is transmitted using a communication line, so that power supply and separate wireless communication lines must be installed respectively.

In addition, the method of remote meter reading of a meter according to the prior art has a problem that economical cost increases due to the need to replace all of the existing mechanical meters with digital meters.

In addition, the conventional meter remote meter reading method has a problem in that an error occurs in the transmission process of the metered data value, and an incorrect meter reading value is transmitted or the meter reading is impossible due to a line failure.

In addition, in the case of remote meter reading of a mechanical meter, a sensor such as a separate reed switch must be installed. At this time, a separate installation for remote meter reading is required due to different standards depending on the manufacturer of the meter, and replacement of the meter or If correction is required, there is a problem that the reference value of the sensor must be newly corrected.

Korean Patent Application Publication No. 10-2004-0000261 (Name of invention: remote meter reading device of meter, remote meter reading system and remote meter reading method using the same)

In order to solve this problem, the present invention enables remote meter reading of the accumulated amount using a portable device, enables remote meter reading without restrictions on the type and type of meter, and minimizes the effect on the shooting environment, so that it is accurate even in various environments. An object of the present invention is to provide a deep learning-based meter information recognition system and method that automatically recognizes the accumulated amount and unique number of a meter in a meter meter reading system using deep learning so that meter reading is performed.

In order to achieve the above object, an embodiment of the present invention is a deep learning-based meter information recognition system, in which an image of a meter to be read is input from a mobile terminal using an object recognition artificial intelligence model to detect and detect a meter object. The display object 410 is detected from the measured meter object, and coordinate information is calculated by sequentially detecting the accumulated amount display unit object and the barcode display unit object from the detected display object, and the calculated using the numerical recognition artificial intelligence model. It includes a modeling server that recognizes the number of the integrated amount display unit object and the barcode display unit object based on coordinate information.

In addition, the object recognition artificial intelligence model according to the embodiment is characterized in that it learns the classification of a meter object from training data including a meter image using a deep learning model based on a convolutional neural network (CNN).

In addition, the object recognition artificial intelligence model according to the above embodiment includes an original image, an image in which the meter is enlarged or reduced to an arbitrary size, an image in which the left/right position of the meter is inverted, and the brightness is adjusted to have an arbitrary illuminance. It is characterized by learning based on learning data classified as an image, an image in which the position of the meter is skewed in an arbitrary direction, and an image in which the position of the meter has an angle with an arbitrary inclination.

In addition, the number recognition artificial intelligence model according to the above embodiment is characterized in that it learns number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN).

In addition, it characterized in that it further comprises a management server that provides the image of the meter to be read transmitted from the mobile terminal according to the embodiment to the modeling server, and receives and stores the number recognized from the modeling server.

In addition, the modeling server according to the embodiment includes: a first modeling unit configured to detect a position of a meter object; A second modeling unit configured to detect a display object in the sensed meter object and calculate coordinate information by detecting the positions of the accumulated amount display unit object and the barcode display unit object in the display object; And a third modeling unit for recognizing the number of the integrated amount display unit object and the barcode display unit object based on the calculated coordinate information using a number recognition artificial intelligence model.

In addition, a method for recognizing meter information based on deep learning according to an embodiment of the present invention includes the steps of: a) a management server receiving an image of a meter to be read from a mobile terminal and inputting it to a modeling server; b) detecting, by the modeling server, a meter object using an object recognition artificial intelligence model of an image of a meter to be read; c) The modeling server detects a display object from the meter object detected in step b) using the object recognition artificial intelligence model, and sequentially detects the accumulated amount display unit and the barcode display unit object from the detected display object to coordinate Calculating information; d) recognizing, by the modeling server, the numbers of the integrated amount display unit object and the barcode display unit object based on the coordinate information calculated in step c) using a number recognition artificial intelligence model; And e) receiving and storing, by the management server, the number recognized from the modeling server.

In addition, the object recognition artificial intelligence model according to the embodiment is characterized in that it learns the classification of a meter object from training data including a meter image using a deep learning model based on a convolutional neural network (CNN).

In addition, the object recognition artificial intelligence model according to the above embodiment includes an original image, an image in which the meter is enlarged or reduced to an arbitrary size, an image in which the left/right position of the meter is inverted, and the brightness is adjusted to have an arbitrary illuminance. It is characterized by learning based on the learning data classified as an image, an image in which the position of the meter is skewed in an arbitrary direction, and an image in which the position of the meter is adjusted to have an arbitrary inclination.

In addition, the number recognition artificial intelligence model according to the above embodiment is characterized in that it learns number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN).

The present invention enables remote meter reading of the accumulated amount using a portable device, remote meter reading is possible without restrictions on the type and type of meter, and deep learning is performed so that accurate meter reading can be performed in various environments by minimizing the effect on the shooting environment. In the meter reading system used, there is an advantage of being able to automatically recognize the accumulated quantity and unique number of the meter.

In addition, the present invention has the advantage of performing remote meter reading through a meter reader or a user's portable device who has installed an app linked with the meter information recognition system without installing a separate measuring means on the mechanical meter.

In addition, the present invention has the advantage of recognizing a usage amount and a unique number using only an image captured by a meter reader or a user's portable device.

In addition, the present invention has the advantage of being able to block the possibility of an input error when the amount of usage or the unique number is incorrectly recorded by the meter reader or the handwritten data is entered into the database of the server through an input means such as a keyboard.

In addition, the present invention has the advantage that it can be used without installing a separate communication line for providing measured information for each meter.

In addition, the present invention has the advantage of being able to easily convert the red dose of a mechanical meter into digital data.

1 is a block diagram schematically showing the configuration of a deep learning-based meter information recognition system according to an embodiment of the present invention.
FIG. 2 is a block diagram showing the configuration of a modeling server of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
FIG. 3 is an exemplary diagram illustrating a process of recognizing a meter in the deep learning-based meter information recognition system according to the embodiment of FIG. 1;
FIG. 4 is another exemplary diagram for explaining a process of recognizing a meter in the deep learning-based meter information recognition system according to the embodiment of FIG. 1;
FIG. 5 is another exemplary view for explaining a process of recognizing a meter in the deep learning-based meter information recognition system according to the embodiment of FIG. 1;
6 is an exemplary view illustrating a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
7 is another exemplary view for explaining a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
FIG. 8 is another exemplary view for explaining a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
9 is another exemplary diagram for explaining a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
FIG. 10 is another exemplary view for explaining a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
FIG. 11 is another exemplary view for explaining a learning process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
12 is another exemplary view for explaining a meter recognition process of the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
13 is another exemplary view for explaining a process of recognizing a meter in the deep learning-based meter information recognition system according to the embodiment of FIG. 1.
14 is a flowchart illustrating a process of recognizing a deep learning-based meter information recognition system according to an embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to a preferred embodiment of the present invention and the accompanying drawings, but it will be described on the premise that the same reference numerals refer to the same elements.

Prior to describing specific details for the implementation of the present invention, it should be noted that configurations that are not directly related to the technical gist of the present invention have been omitted within the scope not disturbing the technical gist of the present invention.

In addition, the terms or words used in the specification and claims are based on the principle that the inventor can define the concept of an appropriate term to describe his or her invention in the best way. Should be interpreted as.

In this specification, the expression that a certain part "includes" a certain component does not exclude other components, but means that other components may be further included.

In addition, terms such as "... unit", "... group", and "... module" mean units that process at least one function or operation, which can be classified into hardware, software, or a combination of the two.

In addition, the term “at least one” is defined as a term including the singular and plural, and even if the term “at least one” does not exist, each component may exist in the singular or plural, and may mean the singular or plural. Will say self-evident.

In addition, it will be said that each component may be provided in singular or plural, and may be changed according to embodiments.

Hereinafter, a preferred embodiment of a deep learning-based meter information recognition system and method according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of a deep learning-based meter information recognition system according to an embodiment of the present invention, and FIG. 2 is a modeling server of a deep learning-based meter information recognition system according to the embodiment of FIG. It is a block diagram showing the configuration.

As shown in FIGS. 1 and 2, the deep learning-based meter information recognition system according to an embodiment of the present invention detects and detects a meter object using an object recognition artificial intelligence model for an input meter reading object image. The display object is detected from the measured meter object, and coordinate information is calculated by sequentially detecting the accumulated amount display unit object and the barcode display unit object) from the detected display object, and integration based on coordinate information calculated using a number recognition artificial intelligence model. As a configuration for recognizing, outputting, and storing numbers of a quantity display unit object and a barcode display unit object, the mobile terminal 100, the management server 200, and the modeling server 300 may be included.

The mobile terminal 100 may be a terminal of a meter reader who wants to transmit an image of a meter to be read using a web page, an app page, a program, or an application, or a terminal of a building user in which a meter to be read is installed.

In addition, the mobile terminal 100 is a wireless communication device, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad , Tablet PC, and the like, may include all kinds of handheld-based portable communication devices.

Further, the mobile terminal 100 is a photo/electric conversion device capable of photographing a meter to be read, and may include a camera device including a CCD sensor, a CMOS sensor, or an image sensor that converts an optical signal into an electric signal.

In addition, the mobile terminal 100 may be installed with an application program for transmitting an image of a meter to be read through a network connection and a camera device to the management server 200 or the modeling server 300.

The management server 200 receives the image of the meter to be read from the mobile terminal 100 connected through the network and inputs it to the modeling server 300.

In addition, the management server 200 receives the integrated amount information of the meter to be read and output as information by the modeling server 300, and the unique information of the meter extracted from the barcode, and the received integrated amount information and the unique information The matched digital information is stored in a database (not shown).

In addition, the management server 200 may be a server system installed in a company that manages the meter.

In addition, in the present embodiment, the management server 200 and the modeling server 300 are separately described for convenience of explanation, but the present invention is not limited thereto, and the management server 200 and the modeling server 300 are provided as one server system. It will be apparent to those skilled in the art that it can be configured in.

The modeling server 300 detects an image of a meter to be read using an object recognition artificial intelligence model, detects a display object from the detected meter object, and detects an accumulated amount display object and a barcode display object from the detected display object. It may be configured to include a first modeling unit 310 and a second modeling unit 320 so as to sequentially detect and calculate coordinate information.

As shown in FIG. 3, the first modeling unit 310 is a component that detects the position of the meter object 400 from the meter image 500 to be read, and uses the object recognition artificial intelligence model to identify the meter object 400. To detect.

That is, the first modeling unit 310 detects the position of the meter object 400 in the meter image 500 to be read, and the meter object image area including the meter object 400 detected in the meter image 500 to be metered Extract.

The object recognition artificial intelligence model learns the classification of the meter object 400 from training data including the meter image using a deep learning model based on a convolutional neural network (CNN).

In addition, in the present invention, the gas meter is described as an exemplary embodiment for convenience of description, but the present invention is not limited thereto, and all meters such as an electric meter and a water meter operated mechanically or digitally may be included.

As shown in FIG. 4, the second modeling unit 320 detects the display object 410 using the object recognition artificial intelligence model in the meter object image area 510 including the detected meter object 400.

That is, the second modeling unit 320 detects the position of the display object 410 through the first bounding box 411 using the object recognition artificial intelligence model in the meter object image area 510, and A display object image area including the display object 410 detected in the object image area 510 is extracted.

In addition, the second modeling unit 320 is displayed as a second bounding box 421 using an object recognition artificial intelligence model in the display object image area 520 including the detected display object 410 as shown in FIG. 5. The accumulated amount display unit object 420 is detected.

In addition, the second modeling unit 320 uses the object recognition artificial intelligence model in the display object image area 520 including the detected display object 410 to display a barcode display unit object 430 displayed as the third bounding box 431. ).

That is, the second modeling unit 320 detects the positions of the accumulated amount display object 420 and the barcode display object 430 displayed by the second bounding box 421 and the second bounding box 431.

In addition, the second modeling unit 320 calculates position coordinate information of the detected integrated amount display object 420 and the barcode display object 430.

Here, the object recognition artificial intelligence model can be regarded as a kind of analysis models created through a method called deep learning in machine learning.

Therefore, the object recognition artificial intelligence model may be used as an expression of a deep learning model or a deep learning analysis model.

In addition, machine learning is an application of artificial intelligence that allows complex systems to automatically learn and improve from experience without being explicitly programmed.

Also, the accuracy and validity of machine learning models may depend in part on the data used to train those models.

In addition, the object recognition artificial intelligence model may repeatedly learn a plurality of meter images selected based on a result of comparing a plurality of images photographed by a meter with each other as training data.

That is, as shown in Figs. 6 to 11, the image 531 in which the meter is enlarged or reduced to an arbitrary size based on the original image 530 of the meter and the size of the meter is adjusted, and the left/right position of the meter is An inverted image 532, an image 533 whose brightness is adjusted to have an arbitrary illuminance, an image 534 including a dummy area 534a due to the position of the meter being biased in an arbitrary direction, the position of the meter at an arbitrary slope The learning is performed based on the learning data classified as an image 535 formed with an angle of.

In this way, by taking into account images that can be taken in various conditions such as lighting, shooting angle, shaking during shooting, shooting composition, photo resolution, etc., by learning in advance images that have made various changes to the original image. , It is possible to improve the recognition rate for images captured in an actual environment.

In the present embodiment, for convenience of explanation, an image including a display object image area is described, but the present invention is not limited thereto, and it is understood by those skilled in the art that an image including a meter object image area and an image of a meter to be read may be learned as learning data. It will be self-evident.

In addition, the modeling server 300 includes a third modeling unit 330 for recognizing the number of the integrated amount display unit 420 and the barcode display unit 430 based on the calculated coordinate information using a number recognition artificial intelligence model. can do.

The third modeling unit 330 recognizes and outputs the number of the accumulated amount display unit object and the barcode display unit object based on the coordinate information calculated using the number recognition artificial intelligence model.

Here, the number recognition artificial intelligence model learns number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN).

In addition, the CRNN model used in the third modeling unit 330 is a deep learning structure capable of recognizing and digitizing numbers in an image, and the accumulated amount display unit object 420 and the barcode display unit detected by the second modeling unit 320 Through a model created by utilizing the object 430 area for learning, the accumulated amount (or amount of use) of the meter and the number of the unique number are recognized.

In addition, the CRNN model has a structure as shown in Table 1, and a plurality of blocks may be composed of a combination of a single convolution block and deep learning layers.

Block number Block Setting hyperparameter Number of repetitions
Block 1 3x3 Single Convolution Block Number of filters: 64
One 2x2 Max Pooling stride: 2
Block 2 3x3 Single Convolution Block Number of filters: 128
One 2x2 Max Pooling stride: 2
Block 3 1x1 Single Convolution Block Number of filters: 64

3 3x3 Single Convolution Block Number of filters: 128 Add 2x2 Max pooling stride: 2 One
Block 4 3x3 Single Convolution Block Number of filters: 256 One 1x1 Single Convolution Block Number of filters: 128
8 3x3 Single Convolution Block Number of filters: 256 Add
Block 5 Reshape Output: (50, 2048) vector

One Dense Layer Output: (50, 512) vector ReLU activation Dropout Probability: 0.6
Block 6 Bidirectional GRU Hidden layer: 512 merge: sum
One Bidirectional GRU Hidden layer: 512 merge: concatenate Dropout Probability: 0.6 Block 7 Dense Layer Softmax activation

In addition, an n×n single convolution block may have a structure as shown in Table 2.

Layer name Hyper parameter Conv2D Number of filters
Kernel size: nxn
Padding: "same"
Kernel initialization method: "he_normal" Batch Normalization LeakyReLU alpha: 0.1

In addition, the CRNN model can be trained using a data amplification technique in order to show the same performance in any environment in which the accumulated amount and the unique number are the same.

In addition, the data amplification technique may include adjusting the size of an image to an arbitrary size, inverting left/right, color conversion, arbitrary space adjustment, and arbitrary angle adjustment, as shown in Table 3.

Data amplification technology The details Random image size adjustment (Random Resizing) Aspect ratio: Randomly select between 0.7 and 1.3 times.
Width and Height: Converts the shorter of the width and height to a value between 0.25 and 2 times. Random left and right reverse
(Random Left-right Flip) 50% chance to reverse left and right

Random color adjustment
(Random Distortion) Convert RGB image to HSV to convert color.
Hue: Adds a value between -0.1 and 0.1 to the existing image and converts between 0 and 1.
Saturation: Randomly selects a value between 0.66 and 1.5 and multiplies the saturation value of the existing image.
Value: A value between 0.66 and 1.5 is randomly selected and multiplied by the value of the existing image. Random space adjustment
(Random Padding) Adjusts the space by moving the image by up to 10 pixels horizontally and 5 pixels vertically, and giving an arbitrary margin to black. Arbitrary angle adjustment
(Random Rotation) The image is tilted by setting a random value between -0.05 and 0.05 degrees.
In order to create the same shape as the original image, the space is adjusted to black in the margins.

In addition, the third modeling unit 330 uses the number recognition artificial intelligence model based on deep learning, as shown in FIG. 12, in the accumulated amount display unit object 420 area, the accumulated amount information 422 which is the accumulated amount (or amount of use) of the meter. ) Is recognized and extracted.

In addition, the third modeling unit 330 recognizes and extracts the barcode information 432, which is the unique information of the meter, in the area of the bar code display unit object 430, as shown in FIG. 13, using a number recognition artificial intelligence model based on deep learning. .

In addition, the third modeling unit 330 matches the extracted accumulated amount information 422 with the barcode information 432 and transmits the final result to the management server 200.

The following describes a deep learning-based meter information recognition method according to an embodiment of the present invention.

14 is a flowchart illustrating a process of recognizing a deep learning-based meter information recognition system according to an embodiment of the present invention, and will be described with reference to FIGS. 1 to 5 and 14.

A meter reading target meter is photographed in either the mobile terminal 100 of the meter reader's terminal or the building user's terminal where the meter reading target meter is installed, and the captured meter reading target meter image 500 is accessed from the mobile terminal 100 through a network. It is transmitted to the managed server 200.

The management server 200 receives the meter image 500 to be read from the mobile terminal 100 and inputs it to the modeling server 300 (S100).

The modeling server 300 detects the position of the meter object 400 using an object recognition artificial intelligence model from the input meter image 500, and the meter object 400 detected by the meter image 500 to be metered The meter object image area 510 including the is extracted (S200).

In addition, the modeling server 300 detects the display object 410 in the area of the meter object 400 extracted in step S200 using the object recognition artificial intelligence model, and the accumulated amount display unit object from the detected display object 410 The coordinate information is calculated by sequentially detecting 420 and the barcode display unit 430 (S300).

That is, in step S300, the modeling server 300 detects the position of the display object 410 through the first bounding box 411 using the object recognition artificial intelligence model in the meter object image area 510, The display object image area 520 including the display object 410 detected in the meter object image area 510 is extracted.

In addition, the modeling server 300 uses the object recognition artificial intelligence model in the display object image area 520 including the detected display object 410 to display the accumulated amount display unit object 420 displayed as the second bounding box 421 Wow, the barcode display object 430 displayed by the third bounding box 431 is detected.

In addition, in step S300, the modeling server 300 detects and detects the positions of the accumulated amount display object 420 and the barcode display object 430 displayed by the second bounding box 421 and the second bounding box 431 Position coordinate information of the accumulated amount display unit 420 and the barcode display unit 430 is calculated.

In addition, the modeling server 300 may perform a learning process of the object recognition artificial intelligence model to learn the classification of the meter object 400 from the training data including the meter image using a deep learning model based on a convolutional neural network (CNN). have.

Here, the object recognition artificial intelligence model is an original image, an image adjusted by expanding or reducing the meter to an arbitrary size, an image in which the left/right position of the meter is inverted, an image whose brightness is adjusted to have an arbitrary illuminance, and the position of the meter. A can be learned based on the training data classified as an image biased in an arbitrary direction and an angle-adjusted image in which the position of the meter has an arbitrary inclination.

Subsequently, the modeling server 300 determines the number of the accumulated amount display unit 420 and the barcode display unit 430 based on the coordinate information of the accumulated amount display unit object 420 and the barcode display unit 430 calculated in step S300. ) The number of the area is recognized (S400) using a number recognition artificial intelligence model.

That is, in step S400, the modeling server 300 uses a deep learning-based numeric recognition program model created by learning the detected integration amount display unit object 420 area and barcode display unit 430 area. Or usage) and the number of the unique number.

In addition, the modeling server 300 may perform a learning process of a number recognition artificial intelligence model to learn number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN).

Subsequently, the modeling server 300 matches the accumulated amount (or usage) of the meter recognized in step S400 and the number of the unique number, respectively, with the accumulated amount information 422 and the barcode information 432 to match the final result to the management server ( 200), and the management server 200 receives the final result from the modeling server 300 and stores it in a database (S500).

Therefore, remote meter reading of the accumulated amount is possible using a portable device, remote meter reading is possible without restrictions on the type and type of meter, and deep learning is used to ensure accurate meter reading even in various environments by minimizing the effect on the shooting environment. The meter reading system can automatically recognize the accumulated quantity and unique number of the meter and store it as digital information.

In addition, it is possible to perform remote meter reading through a meter reader or a user's portable device who has installed an app linked with the meter information recognition system without installing a separate measuring means on the mechanical meter.

As described above, the present invention has been described with reference to preferred embodiments of the present invention, but those skilled in the art can variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. You will understand that you can do it.

In addition, reference numerals in the claims of the present invention are provided for clarity and convenience of description, and are not limited thereto. In the process of describing the embodiments, the thickness of the lines shown in the drawings, the size of components, etc. May be exaggerated for clarity and convenience of description.

In addition, the above-described terms are terms defined in consideration of functions in the present invention and may vary according to the intention or custom of users and operators, so interpretation of these terms should be made based on the contents throughout the present specification. .

In addition, even if not explicitly shown or described, a person having ordinary knowledge in the technical field to which the present invention pertains can make various modifications including the technical idea according to the present invention from the description of the present invention. It is obvious, and this still belongs to the scope of the present invention.

In addition, the above embodiments described with reference to the accompanying drawings are described for the purpose of describing the present invention, and the scope of the present invention is not limited to these embodiments.

100: mobile terminal 200: management server
300: modeling server 310: first modeling unit
320: second modeling unit 330: third modeling unit
400: meter object 410: display object
411: first bounding box 420: integrated amount display object
421: second bounding box 422: first recognition information
430: bar code display unit 431: third bounding box
432: second recognition information 500: image of the meter to be read
510: meter object image area 520: display object image area
530: Original image 531: First training image
532: second learning image 533: third learning image
534: fourth training image 534a: dummy area
535: 5th learning image

Claims (10)

The image of the meter to be read from the mobile terminal 100 is detected using an object recognition artificial intelligence model, and the display object 410 is displayed through the bounding box from the detected meter object 400. The display object image area 520 is extracted by detecting the position, and the accumulated amount display unit object 420 and the barcode display unit object 430 through a bounding box in the display object image area 520 using an object recognition artificial intelligence model. ) Is sequentially detected to calculate coordinate information, and the calculated amount display unit object 420 based on coordinate information of the calculated total amount display unit object 420 and the barcode display unit object 430 using a number recognition artificial intelligence model By recognizing the number of the barcode display unit 430, extracting the accumulated amount information 422 and the barcode information 432 from the recognized numerical information, and matching the extracted accumulated amount information 422 with the barcode information 432 Deep learning-based meter information recognition system including a modeling server 300 that transmits to the management server 200. The method of claim 1,
The object recognition artificial intelligence model learns the classification of the meter object 400 from training data including the meter image using a deep learning model based on a Convolutional Neural Network (CNN). The method of claim 2,
The object recognition artificial intelligence model includes an original image, an image adjusted by expanding or reducing the meter to an arbitrary size, an image in which the left/right position of the meter is inverted, an image whose brightness is adjusted to have an arbitrary illuminance, and the position of the meter. A deep learning-based meter information recognition system, characterized in that learning based on learning data classified as an image biased in an arbitrary direction and an image in which the position of the meter has an angle with an arbitrary inclination. The method of claim 1,
The number recognition artificial intelligence model is a deep learning-based meter information recognition system, characterized in that for learning number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN). The method according to any one of claims 1 to 4,
Provides the image of the meter to be read from the mobile terminal 100 to the modeling server 300, and the integration amount information 422 and barcode information 432 transmitted from the modeling server 300, and the integration Deep learning-based meter information recognition system, characterized in that it further comprises a management server (200) for receiving and storing the matching result of the amount information (422) and the barcode information (432). The method of claim 5,
The modeling server 300 includes: a first modeling unit 310 for detecting the position of the meter object 400 using a deep learning-based object recognition artificial intelligence model;
The detected meter object 400 detects the position of the display object 410 through the first bounding box 411 to extract the display object image area 520, and the second display object image area 520 The accumulated amount display object 420 is sensed using the bounding box 421, and the barcode display object 430 is sequentially detected using the third bounding box 431 in the display object image area 520 and positioned. A second modeling unit 320 for calculating coordinate information; And
A number recognition system that recognizes the numbers of the accumulated amount display object 420 and the barcode display object 430 based on the calculated coordinate information of the calculated accumulated amount display unit object 420 and barcode display unit 430 using a number recognition artificial intelligence model. 3 modeling unit 330; Deep learning-based meter information recognition system, comprising: a. a) the management server 200 receiving the image of the meter to be read from the mobile terminal 100 and inputting the image to the modeling server 300;
b) detecting the meter object 400 using an object recognition artificial intelligence model for the image of the meter to be read by the modeling server 300;
c) The modeling server 300 detects the position of the display object 410 through a bounding box in the meter object 400 detected in step b) using the object recognition artificial intelligence model, and the display object image area ( 520) is extracted, and coordinate information is sequentially detected through a bounding box in the display object image area 520 using an object recognition artificial intelligence model to detect the integration amount display unit object 420 and the barcode display unit object 430. Calculating;
d) The modeling server 300 uses a number recognition artificial intelligence model to calculate the accumulated amount display unit 420 calculated in step c) and the accumulated amount display unit object 420 based on coordinate information of the barcode display unit 430 And barcode display unit Recognizes the number of the object 430, extracts the accumulated amount information 422 and the barcode information 432 from the recognized numerical information, and matches the extracted accumulated amount information 422 and the barcode information 432 Sending it to the management server 200; And
e) receiving and storing, by the management server 200, the number recognized from the modeling server 300; deep learning-based meter information recognition method comprising a. The method of claim 7,
The object recognition artificial intelligence model learns the classification of the meter object 400 from training data including the meter image using a deep learning model based on a convolutional neural network (CNN). The method of claim 8,
The object recognition artificial intelligence model includes an original image, an image adjusted by expanding or reducing the meter to an arbitrary size, an image in which the left/right position of the meter is inverted, an image whose brightness is adjusted to have an arbitrary illuminance, and the position of the meter. Deep learning-based meter information recognition method, characterized in that learning based on learning data classified as an image biased in an arbitrary direction and an angle-adjusted image having a position of a meter having an arbitrary inclination. The method of claim 7,
The number recognition artificial intelligence model learns number recognition from training data including numbers using a deep learning model based on a convolutional recurrent neural network (CRNN).

Images