JP6780129B1 - Equipment information collection system - Google Patents
Equipment information collection system Download PDFInfo
- Publication number
- JP6780129B1 JP6780129B1 JP2019547327A JP2019547327A JP6780129B1 JP 6780129 B1 JP6780129 B1 JP 6780129B1 JP 2019547327 A JP2019547327 A JP 2019547327A JP 2019547327 A JP2019547327 A JP 2019547327A JP 6780129 B1 JP6780129 B1 JP 6780129B1
- Authority
- JP
- Japan
- Prior art keywords
- character recognition
- recognition engine
- character
- common
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/98—Detection or correction of errors, e.g. by rescanning the pattern or by human intervention; Evaluation of the quality of the acquired patterns
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Character Discrimination (AREA)
Abstract
サーバ（１０）と、携帯端末（２０）と、受信した画像データから認識文字列を抽出して送信する文字認識エンジン（３０，３５〜３８）と、を備える設備機器情報収集システム（１００）であって、サーバ（１０）は、評価用画像データを各文字認識エンジン（３０，３５〜３８）に送信し、各文字認識エンジン（３０，３５〜３８）から認識文字列を受信する。そして、受信した認識文字列とユーザ（６０）が認識文字列を正しく修正した登録文字列とから各文字認識エンジン（３０，３５〜３８）の誤認識文字数をそれぞれ算出し、誤認識文字数の少ない文字認識エンジンを新常用文字認識エンジンに選択する。そして、選択した新常用文字認識エンジンを常用文字認識エンジン（３０）に置き換える。An equipment information collection system (100) including a server (10), a mobile terminal (20), and a character recognition engine (30, 35-38) that extracts and transmits a recognition character string from received image data. Therefore, the server (10) transmits the evaluation image data to each character recognition engine (30, 35 to 38), and receives the recognition character string from each character recognition engine (30, 35 to 38). Then, the number of misrecognized characters of each character recognition engine (30, 35 to 38) is calculated from the received recognition character string and the registered character string in which the user (60) correctly corrects the recognition character string, and the number of misrecognized characters is small. Select the character recognition engine as the new common character recognition engine. Then, the selected new common character recognition engine is replaced with the common character recognition engine (30).
Description
本発明は、携帯端末で取得した画像データ又は音声データに基づいてビルなどに設置される空調機器、昇降機等の設備機器の型名、製造番号等の機器情報を収集するシステムに関する。 The present invention relates to a system for collecting device information such as model names and serial numbers of air-conditioning devices, elevators and other equipment installed in buildings and the like based on image data or audio data acquired by a mobile terminal.
ビルに設置される空調機器、昇降機等の設備機器は、定期点検が必要なものがある。一方、定期点検の際に必要となる型名、製造番号等の機器情報がデータベースに格納されていない場合もある。そこで、設備機器のユーザに撮影機能と位置情報取得機能とを有するスマートフォン等の携帯端末を用いて上記の機器情報をデータベースに登録してもらい、その登録情報に基づいて定期点検を行う方法が提案されている（例えば、特許文献１参照）。 Some equipment such as air conditioners and elevators installed in buildings require regular inspections. On the other hand, device information such as model name and serial number required for periodic inspection may not be stored in the database. Therefore, we propose a method in which users of equipment equipment are asked to register the above equipment information in a database using a mobile terminal such as a smartphone that has a shooting function and a location information acquisition function, and periodic inspections are performed based on the registered information. (See, for example, Patent Document 1).
特許文献１に記載された従来技術では、ユーザが撮影した銘板の画像データをＯＣＲ処理して機器番号を抽出し、抽出した機器番号を修正してデータベースに登録するようにしている。このため、ＯＣＲの文字認識精度が低いと修正が多くなって登録に手間が掛かったり、誤った機器番号を登録してしまい、機器情報の収集効率が低下する場合があった。 In the prior art described in Patent Document 1, the image data of the nameplate taken by the user is subjected to OCR processing to extract the device number, and the extracted device number is corrected and registered in the database. For this reason, if the character recognition accuracy of the OCR is low, many corrections are required, which may take time and effort for registration, or the wrong device number may be registered, resulting in a decrease in device information collection efficiency.
そこで本開示は、設備機器情報収集システムの文字認識精度の向上を図り、設備機器の情報収集効率を向上させることを目的とする。 Therefore, it is an object of the present disclosure to improve the character recognition accuracy of the equipment information collection system and improve the information collection efficiency of the equipment.
本開示の設備機器情報収集システムは、ネットワークに接続されたサーバと、ネットワークに接続され、設備機器情報を含むデータを取得し、取得したデータを送信する携帯端末と、ネットワークに接続されてデータを受信し、受信したデータを解析してデータに含まれる文字列を認識し、認識した文字列を認識文字列として送信する常用文字認識エンジンと、を備える設備機器情報収集システムであって、携帯端末は、取得したデータを常用文字認識エンジンとサーバとに送信し、常用文字認識エンジンから認識文字列を受信し、受信した認識文字列をユーザが正しい文字列に修正した登録文字列をサーバに送信し、サーバは、携帯端末からデータと登録文字列とを受信し、常用文字認識エンジンから認識文字列を受信し、受信したデータと認識文字列と登録文字列とを対応付けて格納する文字認識データベースを含み、文字認識データベースに格納したデータと認識文字列と登録文字列とに基づいて、ネットワークに接続されてデータを受信し、受信したデータを解析してデータに含まれる文字列を認識し、認識した文字列を認識文字列として送信する候補文字認識エンジン及び常用文字認識エンジンの文字認識精度の指標をそれぞれ算出し、算出した各指標に基づいて、常用文字認識エンジンと候補文字認識エンジンとの中から新常用文字認識エンジンを選択し、選択した新常用文字認識エンジンを常用文字認識エンジンに置き換えること、を特徴とする。 The equipment information collection system of the present disclosure includes a server connected to a network, a mobile terminal connected to the network to acquire data including equipment information, and transmit the acquired data, and data connected to the network. It is an equipment information collection system equipped with a common character recognition engine that receives, analyzes the received data, recognizes the character string contained in the data, and transmits the recognized character string as a recognition character string, and is a mobile terminal. Sends the acquired data to the common character recognition engine and the server, receives the recognition character string from the common character recognition engine, and sends the registered character string in which the user corrects the received recognition character string to the correct character string to the server. Then, the server receives the data and the registered character string from the mobile terminal, receives the recognition character string from the common character recognition engine, and stores the received data, the recognition character string, and the registered character string in association with each other. Including the database, based on the data stored in the character recognition database, the recognition character string, and the registered character string, it is connected to the network to receive the data, analyze the received data, and recognize the character string contained in the data. , The index of character recognition accuracy of the candidate character recognition engine and the common character recognition engine that sends the recognized character string as the recognition character string is calculated respectively, and based on each calculated index, the common character recognition engine and the candidate character recognition engine It is characterized by selecting a new common character recognition engine from among the above and replacing the selected new common character recognition engine with a common character recognition engine.
このように、サーバが常用文字認識エンジンから認識文字列を受信し、携帯端末からデータとユーザが認識文字列を正しい文字列に修正した登録文字列とを受信し、受信したデータと認識文字列と登録文字列とに基づいて常用文字認識エンジンと候補文字認識エンジンとの文字認識精度の指標をそれぞれ算出し、算出した各指標に基づいて、常用文字認識エンジンと候補文字認識エンジンとの中から新常用文字認識エンジンを選択し、選択した新常用文字認識エンジンを常用文字認識エンジンに置き換えるので、常に文字認識精度の高い文字認識エンジンを用いて文字認識を行うことができる。これにより、設備機器情報収集システムの文字認識精度を向上させて設備機器の情報収集効率を向上させることができる。 In this way, the server receives the recognition character string from the common character recognition engine, receives the data from the mobile terminal and the registered character string in which the user corrects the recognition character string to the correct character string, and the received data and the recognition character string. And the index of the character recognition accuracy of the common character recognition engine and the candidate character recognition engine is calculated based on the registered character string, and based on each calculated index, the common character recognition engine and the candidate character recognition engine are selected. Since the new common character recognition engine is selected and the selected new common character recognition engine is replaced with the common character recognition engine, character recognition can always be performed using the character recognition engine having high character recognition accuracy. As a result, the character recognition accuracy of the equipment information collection system can be improved and the information collection efficiency of the equipment can be improved.
本開示において、文字認識エンジンとは、画像データを受信し、受信した画像データを解析して画像データに含まれる文字列を認識し、認識した文字列を認識文字列として送信するものと、音声データを受信し、受信した音声データを解析して音声データに含まれる文字列を認識し、認識した文字列を認識文字列として送信するものと含む。 In the present disclosure, the character recognition engine is an engine that receives image data, analyzes the received image data, recognizes a character string included in the image data, and transmits the recognized character string as a recognition character string, and voice. It includes those that receive data, analyze the received voice data, recognize the character string included in the voice data, and transmit the recognized character string as a recognition character string.
本開示の設備機器情報収集システムにおいて、サーバは、文字認識データベースに格納したデータから評価用データを選択し、選択した評価用データを常用文字認識エンジンと候補文字認識エンジンとに送信し、常用文字認識エンジンと候補文字認識エンジンとから認識文字列をそれぞれ受信し、受信した各認識文字列と評価用データに対応する登録文字列とを対比して常用文字認識エンジンと候補文字認識エンジンとの指標をそれぞれ算出してもよい。 In the equipment information collection system of the present disclosure, the server selects evaluation data from the data stored in the character recognition database, transmits the selected evaluation data to the common character recognition engine and the candidate character recognition engine, and uses the common character. Recognition character strings are received from the recognition engine and the candidate character recognition engine, and the received recognition character strings are compared with the registered character strings corresponding to the evaluation data, and the indicators of the common character recognition engine and the candidate character recognition engine are compared. May be calculated respectively.
このように、サーバは常用文字認識エンジンと候補文字認識エンジンが認識した認識文字列と認識文字列をユーザが正しい文字列に修正した登録文字列とを対比して常用文字認識エンジンと候補文字認識エンジンとの指標を算出するので、常用文字認識エンジンと候補文字認識エンジンとの文字認識精度を正確に評価することができる。 In this way, the server compares the recognition character string recognized by the common character recognition engine and the candidate character recognition engine with the registered character string obtained by modifying the recognition character string to the correct character string, and compares the common character recognition engine and the candidate character recognition. Since the index with the engine is calculated, the character recognition accuracy between the common character recognition engine and the candidate character recognition engine can be accurately evaluated.
本開示の設備機器情報収集システムにおいて、指標は、常用文字認識エンジン又は候補文字認識エンジンから受信した認識文字列と評価用データに対応する登録文字列とを対比して算出した常用文字認識エンジン又は候補文字認識エンジンの誤認識文字数でもよい。 In the equipment information collection system of the present disclosure, the index is a common character recognition engine or a common character recognition engine calculated by comparing a recognition character string received from a common character recognition engine or a candidate character recognition engine with a registered character string corresponding to evaluation data. It may be the number of misrecognized characters of the candidate character recognition engine.
これにより、簡便な構成で常用文字認識エンジンと候補文字認識エンジンとの文字認識精度を正確に評価することができる。 As a result, the character recognition accuracy of the common character recognition engine and the candidate character recognition engine can be accurately evaluated with a simple configuration.
本開示の設備機器情報収集システムにおいて、評価用データは、文字認識データベースに格納したデータから選択した複数の類似パターンデータであり、サーバは、選択した複数の類似パターンデータを常用文字認識エンジンと候補文字認識エンジンとにそれぞれ送信し、常用文字認識エンジンと候補文字認識エンジンとから複数の類似パターンデータに対応する複数の認識文字列をそれぞれ受信し、指標は、受信した複数の認識文字列と複数の類似パターンデータに対応する複数の登録文字列とを対比して算出した常用文字認識エンジン又は候補文字認識エンジンの誤認識回数としてもよい。 In the equipment information collection system of the present disclosure, the evaluation data is a plurality of similar pattern data selected from the data stored in the character recognition database, and the server selects the selected multiple similar pattern data as a common character recognition engine and a candidate. It sends to the character recognition engine, and receives multiple recognition character strings corresponding to multiple similar pattern data from the common character recognition engine and the candidate character recognition engine, and the index is the received multiple recognition character strings and multiple. It may be the number of erroneous recognitions of the common character recognition engine or the candidate character recognition engine calculated by comparing with a plurality of registered character strings corresponding to the similar pattern data of.
これにより、常用文字認識エンジンを誤認識回数の少ないものに置き換えることができる。 As a result, the common character recognition engine can be replaced with one having a small number of false recognitions.
本開示の設備機器情報収集システムにおいて、サーバは、常用文字認識エンジンと候補文字認識エンジンとからそれぞれ受信した認識文字列と評価用データに対応する登録文字列とを対比して常用文字認識エンジンと候補文字認識エンジンの誤認識文字数をそれぞれ算出すると共に、文字認識データベースに格納したデータから複数の類似パターンデータを選択し、選択した複数の類似パターンデータを常用文字認識エンジンと候補文字認識エンジンとにそれぞれ送信し、常用文字認識エンジンと候補文字認識エンジンとから複数の類似パターンデータに対応する複数の認識文字列をそれぞれ受信し、受信した複数の認識文字列と複数の類似パターンデータに対応する複数の登録文字列とを対比して常用文字認識エンジンと候補文字認識エンジンとの誤認識回数をそれぞれ算出し、常用文字認識エンジンと候補文字認識エンジンとの各指標は、算出した各誤認識文字数と算出した各誤認識回数とにそれぞれ重み付けをして加算した数値でもよい。 In the equipment information collection system of the present disclosure, the server compares the recognition character string received from the common character recognition engine and the candidate character recognition engine with the registered character string corresponding to the evaluation data, and sets the common character recognition engine. In addition to calculating the number of misrecognized characters of the candidate character recognition engine, multiple similar pattern data are selected from the data stored in the character recognition database, and the selected multiple similar pattern data are used as the common character recognition engine and the candidate character recognition engine. Each is transmitted, and a plurality of recognition character strings corresponding to a plurality of similar pattern data are received from the common character recognition engine and the candidate character recognition engine, respectively, and a plurality of received recognition character strings and a plurality of similar pattern data are supported. The number of false recognitions between the common character recognition engine and the candidate character recognition engine is calculated by comparing with the registered character string of, and each index of the common character recognition engine and the candidate character recognition engine is the calculated number of false recognition characters. It may be a numerical value obtained by weighting and adding each calculated number of false recognitions.
これにより、常用文字認識エンジンと候補文字認識エンジンの文字認識精度を総合的に評価し、常用文字認識エンジンを最適化することができる。 As a result, the character recognition accuracy of the common character recognition engine and the candidate character recognition engine can be comprehensively evaluated, and the common character recognition engine can be optimized.
本開示の設備機器情報収集システムにおいて、サーバは、常用文字認識エンジンと候補文字認識エンジンとの文字認識精度の指標を所定の時間間隔でそれぞれ算出し、算出した各指標に基づいて新常用文字認識エンジンの選択を行ってもよい。 In the equipment information collection system of the present disclosure, the server calculates the character recognition accuracy indexes of the common character recognition engine and the candidate character recognition engine at predetermined time intervals, and recognizes new common characters based on the calculated indexes. You may make an engine selection.
これにより、常用文字認識エンジンを文字認識精度の高いものに維持することができる。 As a result, the common character recognition engine can be maintained with high character recognition accuracy.
本開示の設備機器情報収集システムにおいて、サーバは、常用文字認識エンジンから受信した認識文字列と携帯端末から受信した登録文字列とを対比して常用文字認識エンジンの誤認識文字数を算出し、常用文字認識エンジンの誤認識文字数の時系列的な変化が所定の閾値に達しない場合に、常用文字認識エンジンと候補文字認識エンジンとの文字認識精度の指標をそれぞれ算出し、算出した各指標に基づいて新常用文字認識エンジンの選択を行ってもよい。 In the equipment information collection system of the present disclosure, the server calculates the number of erroneously recognized characters of the common character recognition engine by comparing the recognition character string received from the common character recognition engine with the registered character string received from the mobile terminal, and uses it regularly. When the time-series change in the number of erroneously recognized characters in the character recognition engine does not reach a predetermined threshold, the character recognition accuracy indexes of the common character recognition engine and the candidate character recognition engine are calculated, respectively, and based on the calculated indexes. You may select a new common character recognition engine.
通常、常用文字認識エンジンの文字認識精度は時間と共に向上していくので、常用文字認識エンジンの文字認識精度の向上が停滞した際に、常用文字認識エンジンをより文字認識精度の高い文字認識エンジンに置き換えることができる。 Normally, the character recognition accuracy of the common character recognition engine improves over time, so when the improvement of the character recognition accuracy of the common character recognition engine stagnates, the common character recognition engine becomes a character recognition engine with higher character recognition accuracy. Can be replaced.
本開示の設備機器情報収集システムにおいて、サーバが第１国に配置され、携帯端末が第１国と異なる第２国に配置されてもよい。 In the equipment information collection system of the present disclosure, the server may be arranged in the first country, and the mobile terminal may be arranged in a second country different from the first country.
これにより、外国に設置されている設備機器の情報を効果的に収集することができる。 As a result, it is possible to effectively collect information on equipment installed in foreign countries.
本開示は、設備機器情報収集システムの文字認識精度の向上を図り、設備機器の情報収集効率を向上させることができる。 The present disclosure can improve the character recognition accuracy of the equipment information collection system and improve the information collection efficiency of the equipment.
以下、図面を参照しながら実施形態の設備機器情報収集システム１００について説明する。図１に示すように、設備機器情報収集システム１００は、サーバ１０と、携帯端末２０と、常用文字認識エンジン３０とを備えている。 Hereinafter, the equipment information collecting system 100 of the embodiment will be described with reference to the drawings. As shown in FIG. 1, the equipment information collecting system 100 includes a server 10, a mobile terminal 20, and a common character recognition engine 30.
サーバ１０は、ＣＰＵを含み情報処理や演算を行う演算部１１と、動作プログラム、データ等を格納する記憶部１２と、ネットワーク１５との通信を行う通信部１３とを含むコンピュータである。サーバ１０の記憶部１２には後で図４を参照して説明する文字認識データベース５０が格納されている。 The server 10 is a computer including a calculation unit 11 including a CPU for performing information processing and calculation, a storage unit 12 for storing an operation program, data, and the like, and a communication unit 13 for communicating with the network 15. The storage unit 12 of the server 10 stores a character recognition database 50, which will be described later with reference to FIG.
携帯端末２０は、画像を撮像するカメラ２１と、音声データを取得するマイク２２と、画像或いは文字列を表示するディスプレイ２３とを備え、ネットワーク１５に接続されてネットワーク１５に接続された他の機器との間で通信を行うものである。携帯端末２０は、例えば、スマートホンやカメラ付き携帯電話で構成してもよい。 The mobile terminal 20 includes a camera 21 for capturing an image, a microphone 22 for acquiring audio data, and a display 23 for displaying an image or a character string, and is connected to the network 15 and another device connected to the network 15. It communicates with. The mobile terminal 20 may be composed of, for example, a smart phone or a mobile phone with a camera.
常用文字認識エンジン３０は、ネットワーク１５に接続され、ネットワーク１５から画像データを受信し、受信した画像データを解析して画像データに含まれる文字列を認識し、認識した文字列を認識文字列としてネットワーク１５に送信する。常用文字認識エンジン３０は、クラウド上に配置されている。また、クラウド上には、第１〜第４候補文字認識エンジン３５〜３８が配置されている。第１〜第４候補文字認識エンジン３５〜３８は、常用文字認識エンジン３０と同様、ネットワーク１５に接続され、ネットワーク１５から画像データを受信し、受信した画像データを解析して画像データに含まれる文字列を認識し、認識した文字列を認識文字列としてネットワーク１５に送信するものである。なお、候補文字認識エンジンの数は、４つに限らず、１つでも４つ以外の複数でもよい。 The common character recognition engine 30 is connected to the network 15, receives image data from the network 15, analyzes the received image data, recognizes a character string included in the image data, and uses the recognized character string as a recognition character string. Send to network 15. The common character recognition engine 30 is arranged on the cloud. Further, the first to fourth candidate character recognition engines 35 to 38 are arranged on the cloud. Like the common character recognition engine 30, the first to fourth candidate character recognition engines 35 to 38 are connected to the network 15, receive image data from the network 15, analyze the received image data, and include the received image data in the image data. It recognizes a character string and transmits the recognized character string as a recognition character string to the network 15. The number of candidate character recognition engines is not limited to four, and may be one or a plurality of candidates other than four.
次に、図２〜５を参照しながら、実施形態の設備機器情報収集システム１００の動作について説明する。以下の説明では、図１に示すユーザ６０が携帯端末２０で空調機器４０の銘板４１の画像を撮像することにより、空調機器４０の機器情報を収集する場合について説明する。 Next, the operation of the equipment information collecting system 100 of the embodiment will be described with reference to FIGS. 2 to 5. In the following description, a case where the user 60 shown in FIG. 1 collects device information of the air-conditioning device 40 by capturing an image of the nameplate 41 of the air-conditioning device 40 with the mobile terminal 20 will be described.
図２、図３のステップＳ１０１に示すように、ユーザ６０は、携帯端末２０のカメラ２１によって空調機器４０の銘板４１の画像を撮像する。図２、図３のステップＳ１０２に示すように、携帯端末２０は、カメラ２１によって取得した画像データをネットワーク１５を介して常用文字認識エンジン３０とサーバ１０に送信する。 As shown in step S101 of FIGS. 2 and 3, the user 60 captures an image of the nameplate 41 of the air conditioner 40 by the camera 21 of the mobile terminal 20. As shown in step S102 of FIGS. 2 and 3, the mobile terminal 20 transmits the image data acquired by the camera 21 to the common character recognition engine 30 and the server 10 via the network 15.
図２、図３のステップＳ１０３に示すように、常用文字認識エンジン３０は、携帯端末２０から受信した画像データを解析して画像データ中に含まれる文字列を認識する。そして、認識した文字列を認識文字列としてネットワーク１５を介して携帯端末２０とサーバ１０とに送信する。 As shown in step S103 of FIGS. 2 and 3, the common character recognition engine 30 analyzes the image data received from the mobile terminal 20 and recognizes the character string included in the image data. Then, the recognized character string is transmitted as a recognition character string to the mobile terminal 20 and the server 10 via the network 15.
携帯端末２０は、常用文字認識エンジン３０から受信した認識文字列をディスプレイ２３に表示する。図３のステップＳ１０５に示すように、ユーザ６０はディスプレイ２３に表示された認識文字列と銘板４１に記載されている文字列とを対比し、認識文字列を銘板４１に記載されている文字列と一致する正しい文字列に修正して登録文字列を生成する。図３のステップＳ１０６に示すように、ユーザ６０は、生成した登録文字列を携帯端末２０からサーバ１０に送信する。 The mobile terminal 20 displays the recognition character string received from the common character recognition engine 30 on the display 23. As shown in step S105 of FIG. 3, the user 60 compares the recognition character string displayed on the display 23 with the character string described on the nameplate 41, and sets the recognition character string as the character string described on the nameplate 41. Generate a registered string by modifying it to the correct string that matches. As shown in step S106 of FIG. 3, the user 60 transmits the generated registration character string from the mobile terminal 20 to the server 10.
図２、図３のステップＳ１０７に示すように、サーバ１０の通信部１３は、携帯端末２０から銘板４１の画像データと登録文字列を受信し、常用文字認識エンジン３０から認識文字列を受信する。そして、サーバ１０の演算部１１は、通信部１３で受信した銘板４１の画像データと登録文字列と認識文字列とを図４に示す文字認識データベース５０に格納する。 As shown in step S107 of FIGS. 2 and 3, the communication unit 13 of the server 10 receives the image data of the nameplate 41 and the registered character string from the mobile terminal 20, and receives the recognition character string from the common character recognition engine 30. .. Then, the calculation unit 11 of the server 10 stores the image data of the nameplate 41 received by the communication unit 13, the registered character string, and the recognition character string in the character recognition database 50 shown in FIG.
図４に示すように、文字認識データベース５０は、携帯端末２０から受信した画像データと登録文字列、及び、常用文字認識エンジン３０から受信した認識文字列とを対応付けて格納するものである。ここで、画像データＡＡＡＡは、ユーザ６０が携帯端末２０のカメラ２１で撮像した銘板４１の画像データである。認識文字列「ＰｕＨｙＤＰ２８０００ＳＣＭＥＢＳＧＧＧＧ」は、常用文字認識エンジン３０が携帯端末２０から受信した画像データＡＡＡＡを解析し、認識した文字列である。登録文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」は、ユーザ６０が携帯端末２０に送信された認識文字列「ＰｕＨｙＤＰ２８０００ＳＣＭＥＢＳＧＧＧＧ」を銘板４１に記載されている文字列と一致するように正しく修正した文字列である。従って、登録文字列は、画像データを正しく認識した場合の文字列となる。図４の認識文字列中の下線は、常用文字認識エンジン３０が誤認識した文字を示している。この様に、文字認識データベース５０は、携帯端末２０で撮像した銘板４１の画像データと、常用文字認識エンジン３０が認識した認識文字列と、画像データを正しく認識した場合の文字列である登録文字列とを対応付けて格納したものである。As shown in FIG. 4, the character recognition database 50 stores the image data received from the mobile terminal 20, the registered character string, and the recognition character string received from the common character recognition engine 30 in association with each other. Here, the image data AAAA is image data of the nameplate 41 imaged by the user 60 with the camera 21 of the mobile terminal 20. The recognition character string "PuHyDP28000SCMEBSGGGG" is a character string recognized by the common character recognition engine 30 by analyzing the image data AAAA received from the mobile terminal 20. Registered character string "PUHY-DP280SCM-E-BSG" is a character string which the user 60 is described in the recognized character string "P u H y DP280 00 SC ME BSG GGG " nameplate 41 which is transmitted to the portable terminal 20 A string that has been correctly modified to match. Therefore, the registered character string is a character string when the image data is correctly recognized. The underline in the recognition character string of FIG. 4 indicates a character erroneously recognized by the common character recognition engine 30. In this way, the character recognition database 50 includes the image data of the name plate 41 captured by the mobile terminal 20, the recognition character string recognized by the common character recognition engine 30, and the registered character which is a character string when the image data is correctly recognized. It is stored in association with columns.
サーバ１０は、図３のステップＳ１０８で所定の時間が経過したかどうかを判断し、図３のステップＳ１０８でＮＯと判断した場合には、図３のステップＳ１０１に戻り、携帯端末２０からの画像データと登録文字列を受信し、常用文字認識エンジン３０から認識文字列を受信し、受信した画像データと登録文字列と認識文字列とを文字認識データベース５０に格納していく。ここで、所定の時間は、例えば、一日でもよいし、一週間でもよいし、一ヶ月でもよい。 The server 10 determines whether or not the predetermined time has elapsed in step S108 of FIG. 3, and if it determines NO in step S108 of FIG. 3, returns to step S101 of FIG. 3 and the image from the mobile terminal 20. The data and the registered character string are received, the recognition character string is received from the common character recognition engine 30, and the received image data, the registered character string, and the recognition character string are stored in the character recognition database 50. Here, the predetermined time may be, for example, one day, one week, or one month.
サーバ１０は、図３のステップＳ１０８でＹＥＳと判断した場合には、図３のステップＳ１０９〜ステップＳ１１２において常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８との文字認識精度を示す指標をそれぞれ算出し、算出した各指標に基づいて常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の中の１つを新常用文字認識エンジンに選択する。そして、選択した新常用文字認識エンジンを常用文字認識エンジン３０に置き換える。 If the server 10 determines YES in step S108 of FIG. 3, the character recognition accuracy of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 in steps S109 to S112 of FIG. Each of the indexes indicating the above is calculated, and one of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 is selected as the new common character recognition engine based on the calculated indexes. Then, the selected new common character recognition engine is replaced with the common character recognition engine 30.
以下、図３、図５を参照しながら、指標として認識文字列と登録文字列とを対比して算出した誤認識文字数を用いる場合について説明する。 Hereinafter, a case where the number of misrecognized characters calculated by comparing the recognition character string and the registered character string is used as an index will be described with reference to FIGS. 3 and 5.
図３のステップＳ１０９、図５に示すように、サーバ１０の演算部１１は、文字認識データベース５０に格納した画像データの中から、画像データＡＡＡＡを常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８との評価に用いる評価用画像データＡＡＡＡとして選択する。そして、演算部１１は、評価用画像データＡＡＡＡを通信部１３から常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８とに送信する。 As shown in steps S109 and 5 of FIG. 3, the calculation unit 11 of the server 10 uses the image data AAAA from the image data stored in the character recognition database 50 as the common character recognition engine 30 and the first to fourth candidates. It is selected as the evaluation image data AAAAA used for evaluation with the character recognition engine 35 to 38. Then, the calculation unit 11 transmits the evaluation image data AAAA from the communication unit 13 to the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38.
図３のステップＳ１１０に示すように、常用文字認識エンジン３０と、第１〜第４候補文字認識エンジン３５〜３８は、受信した評価用画像データＡＡＡＡを解析して評価用画像データＡＡＡＡに含まれる文字列を認識し、認識した文字列を認識文字列としてサーバ１０に送信する。図５に示すように、第１候補文字認識エンジン３５は、認識文字列「ＰｕＨｙＤＰ２８０００ＳＣＭＥＢＳＧＧＧＧ」をサーバ１０に送信し、第２候補文字認識エンジン３６は、認識文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ――ＢＳＧ−」をサーバ１０に送信する。常用文字認識エンジン３０、第３、第４候補文字認識エンジン３７，３８についても同様にそれぞれ認識文字列をサーバ１０に送信する。 As shown in step S110 of FIG. 3, the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 analyze the received evaluation image data AAAA and include the evaluation image data AAAA in the evaluation image data AAAA. The character string is recognized, and the recognized character string is transmitted to the server 10 as a recognition character string. As shown in FIG. 5, the first candidate character recognition engine 35 transmits the recognition character string “PuHyDP28000SCMEBSGGGG” to the server 10, and the second candidate character recognition engine 36 transmits the recognition character string “PUHY-DP280SCM-E—BSG”. -”Is transmitted to the server 10. Similarly, the common character recognition engine 30, the third and fourth candidate character recognition engines 37 and 38 also transmit the recognition character strings to the server 10.
図３のステップＳ１１１、図５に示すように、サーバ１０は、常用文字認識エンジン３０、第１〜第４候補文字認識エンジン３５〜３８の文字認識精度を示す指標である誤認識文字数を算出する。図５に示すように、サーバ１０の演算部１１は、文字認識データベース５０から評価用画像データＡＡＡＡに対応する登録文字列を対応登録文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」として読み出す。そして、第１候補文字認識エンジン３５から受信した認識文字列「ＰｕＨｙＤＰ２８０００ＳＣＭＥＢＳＧＧＧＧ」と対応登録文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」とを対比する。対応登録文字列は、評価用画像データＡＡＡＡを正しく認識した場合の文字列であるから、両文字列を対比することにより、第１候補文字認識エンジン３５の認識文字列中の誤認識文字数を算出することができる。本例では、図５に示すように、サーバ１０の演算部１１は第１候補文字認識エンジン３５の誤認識文字数を８と算出する。 As shown in steps S111 and 5 of FIG. 3, the server 10 calculates the number of erroneously recognized characters, which is an index indicating the character recognition accuracy of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38. .. As shown in FIG. 5, the calculation unit 11 of the server 10 reads the registered character string corresponding to the evaluation image data AAAA from the character recognition database 50 as the corresponding registered character string “PUHY-DP280SCM-E-BSG”. Then, the recognition character string "PuHyDP28000SCMEBSGGGG" received from the first candidate character recognition engine 35 and the corresponding registration character string "PUHY-DP280SCM-E-BSG" are compared. Since the corresponding registration character string is a character string when the evaluation image data AAAA is correctly recognized, the number of erroneously recognized characters in the recognition character string of the first candidate character recognition engine 35 is calculated by comparing both character strings. can do. In this example, as shown in FIG. 5, the calculation unit 11 of the server 10 calculates the number of erroneously recognized characters of the first candidate character recognition engine 35 as 8.
同様に、サーバ１０の演算部１１は、第２候補文字認識エンジン３６から受信した認識文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ――ＢＳＧ−」と対応登録文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」とを対比して第２候補文字認識エンジン３６の誤認識文字数を２と算出する。常用文字認識エンジン３０、第３、第４候補文字認識エンジン３７，３８についても同様にそれぞれの誤認識文字数を算出する。 Similarly, the arithmetic unit 11 of the server 10 has the recognition character string "PUHY-DP280SCM-E-BSG-" received from the second candidate character recognition engine 36 and the corresponding registration character string "PUHY-DP280SCM-E-BSG". Is compared and the number of erroneously recognized characters of the second candidate character recognition engine 36 is calculated as 2. Similarly, the number of erroneously recognized characters is calculated for the common character recognition engine 30 and the third and fourth candidate character recognition engines 37 and 38.
図３のステップＳ１１２、図５に示すように、サーバ１０の演算部１１は、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の中で誤認識文字数の最も少ない第２候補文字認識エンジン３６を新常用文字認識エンジンとして選択する。そして、サーバ１０は、新常用文字認識エンジンとして選択した第２候補文字認識エンジン３６を常用文字認識エンジン３０に置き換える。これ以後、携帯端末２０は、新たな常用文字認識エンジン３０に画像データを送信し、新たな常用文字認識エンジン３０から誤認識文字数の少ない認識文字列を受信するので、ユーザ６０の修正の手間が少なくなり、効果的に設備機器の情報を収集することができる。 As shown in steps S112 and 5 of FIG. 3, the calculation unit 11 of the server 10 has the second smallest number of erroneously recognized characters among the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38. The candidate character recognition engine 36 is selected as the new common character recognition engine. Then, the server 10 replaces the second candidate character recognition engine 36 selected as the new common character recognition engine with the common character recognition engine 30. After that, the mobile terminal 20 transmits the image data to the new common character recognition engine 30, and receives the recognition character string with a small number of erroneously recognized characters from the new common character recognition engine 30, so that the user 60 has to take the trouble of correcting it. It is possible to reduce the number and effectively collect information on equipment.
このように、実施形態の設備機器情報収集システム１００では、サーバ１０は常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８とが評価用画像データから認識した認識文字列と評価用画像データを正しく認識した場合の文字列である対応登録文字列とを対比して、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の誤認識文字数を算出し、常用文字認識エンジン３０と第１〜第４候補文字認識エンジンの内で誤認識文字数の少ないものを新常用文字認識エンジンとして選択する。そして、選択した新常用文字認識エンジンを常用文字認識エンジン３０に置き換える。これにより、設備機器情報収集システム１００の文字認識精度の向上を図り、設備機器の情報収集効率を向上させることができる。また、所定の時間間隔で新常用文字認識エンジンの選択を行い、新常用文字認識エンジンを常用文字認識エンジン３０に置き換えるので、常用文字認識エンジン３０を文字認識精度の高いものに維持することができ、設備機器の情報収集効率を向上させることができる。 As described above, in the equipment information collecting system 100 of the embodiment, the server 10 evaluates the recognition character string recognized by the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 from the evaluation image data. The number of erroneously recognized characters of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 is calculated by comparing with the corresponding registered character string which is a character string when the image data for use is correctly recognized. Among the character recognition engine 30 and the first to fourth candidate character recognition engines, the one with a small number of erroneously recognized characters is selected as the new common character recognition engine. Then, the selected new common character recognition engine is replaced with the common character recognition engine 30. As a result, the character recognition accuracy of the equipment information collection system 100 can be improved, and the information collection efficiency of the equipment can be improved. Further, since the new common character recognition engine is selected at a predetermined time interval and the new common character recognition engine is replaced with the normal character recognition engine 30, the common character recognition engine 30 can be maintained with high character recognition accuracy. , Information collection efficiency of equipment can be improved.
次に、図３、図６を参照しながら、指標として常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の誤認識回数を用いて新常用文字認識エンジンの選択を行う動作について説明する。 Next, with reference to FIGS. 3 and 6, an operation of selecting a new common character recognition engine using the number of erroneous recognitions of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 as indicators. Will be described.
図３のステップＳ１０９、図６に示すように、サーバ１０の演算部１１は、文字認識データベース５０から複数の類似パターンデータを選択する。複数の類似パターンデータは、同一の銘板４１の異なる画像のセットで、例えば、クリアな画像、少しぼけた部分のある画像、正面からの画像、斜めからの画像等を組み合わせたものである。類似パターンデータは、同一の銘板４１の画像データであるから、対応登録文字列は一つとなる。図６に示す例では、サーバ１０の演算部１１は、複数の類似パターンデータＣＣＣＣ，ＣＣＣｃ，ｃＣＣＣを選択して、常用文字認識エンジン３０と、第１〜第４候補文字認識エンジン３５〜３８とに送信する。 As shown in steps S109 and 6 of FIG. 3, the calculation unit 11 of the server 10 selects a plurality of similar pattern data from the character recognition database 50. The plurality of similar pattern data is a set of different images of the same nameplate 41, for example, a combination of a clear image, an image having a slightly blurred portion, an image from the front, an image from an angle, and the like. Since the similar pattern data is the image data of the same nameplate 41, the corresponding registration character string is one. In the example shown in FIG. 6, the calculation unit 11 of the server 10 selects a plurality of similar pattern data CCCC, CCCc, and cCCC, and sets the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38. Send to.
図３のステップＳ１１０に示すように、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８は、受信した複数の類似パターンデータを解析して複数の認識文字列をサーバ１０に送信する。図６に示すように、第１候補文字認識エンジン３５は、３つの類似パターンデータＣＣＣＣ，ＣＣＣｃ，ｃＣＣＣを解析して３つの認識文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」、「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」、「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−ＥＢＳＧ」をサーバ１０に送信する。 As shown in step S110 of FIG. 3, the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 analyze a plurality of received similar pattern data and transmit a plurality of recognition character strings to the server 10. Send. As shown in FIG. 6, the first candidate character recognition engine 35 analyzes three similar pattern data CCCC, CCCc, and cCCC, and three recognition character strings "PUHY-DP280SCM-E-BSG" and "PUHY-DP280SCM-". "E-BSG" and "PUHY-DP280SCM-EBSG" are transmitted to the server 10.
図３のステップＳ１１１、図６に示すように、サーバ１０の演算部１１は、第１候補文字認識エンジン３５から受信した各認識文字列と対応登録文字列「ＰＵＨＹ−ＤＰ２８０ＳＣＭ−Ｅ−ＢＳＧ」とを対比して、両者が一致しているかどうかを判断する。図６に示すように、第１候補文字認識エンジン３５が送信した３の認識文字列の内、一番目と二番目の認識文字列は対応登録文字列と一致しており、三番目の認識文字列は対応登録文字列と一致していない。従って、サーバ１０の演算部１１は、第１候補文字認識エンジン３５の誤認識回数を１と算出する。 As shown in steps S111 and 6 of FIG. 3, the calculation unit 11 of the server 10 refers to each recognition character string received from the first candidate character recognition engine 35 and the corresponding registration character string “PUHY-DP280SCM-E-BSG”. To determine if they match. As shown in FIG. 6, among the three recognition character strings transmitted by the first candidate character recognition engine 35, the first and second recognition character strings match the corresponding registration character strings, and the third recognition character. The column does not match the corresponding registration string. Therefore, the calculation unit 11 of the server 10 calculates the number of erroneous recognitions of the first candidate character recognition engine 35 as 1.
同様に、サーバ１０の演算部１１は、第２候補文字認識エンジン３６から受信した３つの認識文字列と対応登録文字列とを対比して誤認識回数を２と算出する。また、常用文字認識エンジン３０、第３、第４候補文字認識エンジン３７，３８についても同様に誤認識回数を算出する。 Similarly, the calculation unit 11 of the server 10 compares the three recognition character strings received from the second candidate character recognition engine 36 with the corresponding registered character strings, and calculates the number of erroneous recognitions as 2. Further, the number of erroneous recognitions is similarly calculated for the common character recognition engine 30, the third and fourth candidate character recognition engines 37 and 38.
図３のステップＳ１１２、図６に示すように、サーバ１０の演算部１１は、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の中で誤認識回数の最も少ない第１候補文字認識エンジン３５を新常用文字認識エンジンとして選択する。そして、サーバ１０は、新常用文字認識エンジンとして選択した第１候補文字認識エンジン３５を常用文字認識エンジン３０に置き換える。 As shown in steps S112 and 6 of FIG. 3, the calculation unit 11 of the server 10 is the first among the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38, which has the smallest number of false recognitions. The candidate character recognition engine 35 is selected as the new common character recognition engine. Then, the server 10 replaces the first candidate character recognition engine 35 selected as the new common character recognition engine with the common character recognition engine 30.
これにより、誤認識回数が少ない文字認識エンジンを常用文字認識エンジン３０とすることができる。 As a result, the character recognition engine with a small number of false recognitions can be used as the common character recognition engine 30.
次に、図７を参照しながら、誤認識文字数と誤認識回数とを組み合わせて総合指標Ｓを算出し、算出した総合指標Ｓに基づいて新常用文字認識エンジンを選択する動作について説明する。 Next, with reference to FIG. 7, an operation of calculating the total index S by combining the number of misrecognized characters and the number of misrecognitions and selecting the new common character recognition engine based on the calculated total index S will be described.
先に図５を参照して説明したように、サーバ１０の演算部１１は、第１、第２候補文字認識エンジン３５，３６に評価用画像データを送信し、認識文字列を受信し、受信した認識文字列と対応登録文字列と対比して誤認識文字数を算出する。また、先に図６を参照して説明したように、サーバ１０の演算部１１は、第１、第２候補文字認識エンジン３５，３６に複数の類似パターンデータを送信し、認識文字列を受信し、受信した認識文字列と対応登録文字列と対比して誤認識回数を算出する。そして、図７に示すように、サーバ１０の演算部１１は、誤認識文字数と誤認識回数にそれぞれ重みａ，ｂを掛けて重み付けしたものを加算して総合指標Ｓの数値を算出する。 As described above with reference to FIG. 5, the arithmetic unit 11 of the server 10 transmits the evaluation image data to the first and second candidate character recognition engines 35 and 36, receives the recognition character string, and receives the recognition character string. The number of erroneously recognized characters is calculated by comparing the recognized character string with the corresponding registered character string. Further, as described above with reference to FIG. 6, the arithmetic unit 11 of the server 10 transmits a plurality of similar pattern data to the first and second candidate character recognition engines 35 and 36, and receives the recognition character string. Then, the number of false recognitions is calculated by comparing the received recognition character string with the corresponding registration character string. Then, as shown in FIG. 7, the calculation unit 11 of the server 10 calculates the numerical value of the comprehensive index S by multiplying the number of misrecognized characters and the number of misrecognitions by the weights a and b, respectively, and adding the weights.
図７に示す例のように、第２候補文字認識エンジン３６の総合指標Ｓ２が第１候補文字認識エンジン３５の総合指標Ｓ１よりも小さい場合には、サーバ１０の演算部１１は総合指標Ｓの小さい第２候補文字認識エンジン３６を常用文字認識エンジン３０に選択する。そして、サーバ１０は、新常用文字認識エンジンとして選択した第２候補文字認識エンジン３６を常用文字認識エンジン３０に置き換える。 As in the example shown in FIG. 7, when the comprehensive index S2 of the second candidate character recognition engine 36 is smaller than the comprehensive index S1 of the first candidate character recognition engine 35, the calculation unit 11 of the server 10 determines the comprehensive index S. The small second candidate character recognition engine 36 is selected as the common character recognition engine 30. Then, the server 10 replaces the second candidate character recognition engine 36 selected as the new common character recognition engine with the common character recognition engine 30.
これにより、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の文字認識精度を総合的に評価し、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の内で最適な文字認識エンジンを新常用文字認識エンジンに選択することができ、常用文字認識エンジン３０を最適化することができる。 As a result, the character recognition accuracy of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 is comprehensively evaluated, and the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 are evaluated. The most suitable character recognition engine can be selected as the new common character recognition engine, and the common character recognition engine 30 can be optimized.
次に、図８，図９を参照しながら、設備機器情報収集システム１００の他の動作について説明する。先に図２、図３を参照して説明した動作と同様の動作には、同様のステップ番号を付して説明は省略する。 Next, other operations of the equipment information collecting system 100 will be described with reference to FIGS. 8 and 9. The same step numbers as those described above with reference to FIGS. 2 and 3 will be assigned the same step numbers, and the description thereof will be omitted.
常用文字認識エンジン３０は、ＡＩを適用したものであり、時間経過と共に、文字認識精度が向上してくることが一般的で、時間経過と共に、誤認識文字数は低下傾向を示す。従って、誤認識文字数が減少せず、停滞する場合には、常用文字認識エンジン３０の文字認識精度の向上が他の文字認識エンジンよりも低くなっている可能性がある。 The common character recognition engine 30 applies AI, and generally the character recognition accuracy improves with the passage of time, and the number of erroneously recognized characters tends to decrease with the passage of time. Therefore, if the number of erroneously recognized characters does not decrease and is stagnant, there is a possibility that the improvement of the character recognition accuracy of the common character recognition engine 30 is lower than that of other character recognition engines.
そこで、本動作は、図８、図９に示すように、サーバ１０の演算部１１が常用文字認識エンジン３０の誤認識文字数と誤認識文字数の時系列変化とを算出し、図８のステップＳ２０２で誤認識文字数の時系列的変化が所定の閾値に達しない場合に常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８との文字認識精度の指標を算出して新常用文字認識エンジンを選択するものである。以下、説明する。 Therefore, in this operation, as shown in FIGS. 8 and 9, the calculation unit 11 of the server 10 calculates the number of erroneously recognized characters of the common character recognition engine 30 and the time-series change in the number of erroneously recognized characters, and in step S202 of FIG. When the time-series change in the number of erroneously recognized characters does not reach a predetermined threshold, the index of character recognition accuracy between the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 is calculated and the new common character is used. It selects the recognition engine. This will be described below.
サーバ１０の演算部１１は、図８、図９のステップＳ１０４に示すように常用文字認識エンジン３０から認識文字列を受信し、図８、図９のステップＳ１０６に示すように携帯端末２０から登録文字列を受信したら、図８、図９のステップＳ２０１に示すように、常用文字認識エンジン３０の誤認識文字数を算出する。誤認識文字数の算出は、先に図５を参照して説明したと同様、常用文字認識エンジン３０から受信した認識文字列と携帯端末２０から受信した登録文字列とを対比して誤認識文字数を算出する。そして、サーバ１０の演算部１１は、算出した誤認識文字数を記憶部１２に格納する。 The calculation unit 11 of the server 10 receives the recognition character string from the common character recognition engine 30 as shown in step S104 of FIGS. 8 and 9, and registers it from the mobile terminal 20 as shown in step S106 of FIGS. 8 and 9. When the character string is received, the number of erroneously recognized characters of the common character recognition engine 30 is calculated as shown in step S201 of FIGS. 8 and 9. The number of misrecognized characters is calculated by comparing the recognition character string received from the common character recognition engine 30 with the registered character string received from the mobile terminal 20 and calculating the number of misrecognized characters in the same manner as described above with reference to FIG. calculate. Then, the calculation unit 11 of the server 10 stores the calculated number of misrecognized characters in the storage unit 12.
図８に示すように、サーバ１０の演算部１１は、図９のステップＳ２０２で所定時間、或いは所定回数だけ常用文字認識エンジン３０の誤認識文字数を算出したかを判断する。サーバ１０の演算部１１は、図９のステップＳ２０２でＮＯと判断した場合には、図９のステップＳ１０１に戻って携帯端末２０からの画像データと登録文字列を受信し、常用文字認識エンジン３０から認識文字列を受信し、誤認識文字数を算出して記憶部１２に格納していく。 As shown in FIG. 8, the calculation unit 11 of the server 10 determines in step S202 of FIG. 9 whether the number of erroneously recognized characters of the common character recognition engine 30 has been calculated for a predetermined time or a predetermined number of times. If the calculation unit 11 of the server 10 determines NO in step S202 of FIG. 9, returns to step S101 of FIG. 9 and receives the image data and the registered character string from the mobile terminal 20, and the common character recognition engine 30 The recognition character string is received from, the number of erroneously recognized characters is calculated, and the number is stored in the storage unit 12.
サーバ１０の演算部１１は、図９のステップＳ２０２でＹＥＳと判断したら図９のステップＳ２０３に進み、記憶部１２から誤認識文字数データを読み出して誤認識文字数の時系列的変化を算出する。時系列的変化は、例えば、所定期間における誤認識文字数の低減数、或いは低減割合である。 If the calculation unit 11 of the server 10 determines YES in step S202 of FIG. 9, the process proceeds to step S203 of FIG. 9, reads the misrecognized character number data from the storage unit 12, and calculates the time-series change in the number of misrecognized characters. The time-series change is, for example, the number of reductions in the number of misrecognized characters in a predetermined period, or the reduction rate.
そして、サーバ１０の演算部１１は、図９のステップＳ２０４で算出した時系列的変化が所定の閾値に達しているかどうかを判断する。例えば、時系列的変化が所定期間における誤認識文字数の低減数が所定の閾値に達していない場合、或いは、低減割合が所定の閾値に達していない場合、サーバ１０の演算部１１は、図９のステップＳ２０４でＹＥＳと判断して図３のステップＳ１０９に進む。 Then, the calculation unit 11 of the server 10 determines whether or not the time-series change calculated in step S204 of FIG. 9 has reached a predetermined threshold value. For example, when the time-series change does not reach the predetermined threshold value for the reduction number of erroneously recognized characters in the predetermined period, or when the reduction rate does not reach the predetermined threshold value, the calculation unit 11 of the server 10 determines FIG. In step S204 of the above, YES is determined, and the process proceeds to step S109 of FIG.
そして、サーバ１０は、図８、図３のステップＳ１０９〜ステップＳ１１２に示すように、評価用画像データを常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８とに送信し、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８とから認識文字列を受信する。そして、図５を参照して説明したように、受信した認識文字列と対応登録文字列とから常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８の誤認識文字数をそれぞれ算出し、誤認識文字数の少ない第２候補文字認識エンジン３６を新常用文字認識エンジンに選択する。そして、サーバ１０は、新常用文字認識エンジンとして選択した第２候補文字認識エンジン３６を常用文字認識エンジン３０に置き換える。 Then, as shown in steps S109 to S112 of FIGS. 8 and 3, the server 10 transmits the evaluation image data to the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38. The recognition character string is received from the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38. Then, as described with reference to FIG. 5, the number of erroneously recognized characters of the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 is calculated from the received recognition character string and the corresponding registered character string, respectively. Then, the second candidate character recognition engine 36 having a small number of erroneously recognized characters is selected as the new common character recognition engine. Then, the server 10 replaces the second candidate character recognition engine 36 selected as the new common character recognition engine with the common character recognition engine 30.
これにより、常用文字認識エンジン３０の文字認識精度の向上が停滞しているような場合に、文字認識精度の文字認識エンジンに置き換えることができる。 As a result, when the improvement of the character recognition accuracy of the common character recognition engine 30 is stagnant, it can be replaced with the character recognition engine of the character recognition accuracy.
以上説明した実施形態の設備機器情報収集システム１００において、サーバ１０と携帯端末２０とは同一の第１国に配置されてもよいし、携帯端末２０は第１国と異なる第２国に配置されていてもよい。これにより、外国に設置されている設備機器の情報を収集することができる。 In the equipment information collecting system 100 of the embodiment described above, the server 10 and the mobile terminal 20 may be arranged in the same first country, or the mobile terminal 20 may be arranged in a second country different from the first country. You may be. This makes it possible to collect information on equipment installed in foreign countries.
また、実施形態の設備機器情報収集システム１００では、常用文字認識エンジン３０、第１〜第４候補文字認識エンジン３５〜３８は、受信した画像データを解析して認識文字列を送信するものとして説明したがこれに限らず、受信した音声データを解析して認識文字列を送信するもでもよい。 Further, in the equipment information collecting system 100 of the embodiment, the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38 are described as analyzing the received image data and transmitting the recognition character string. However, the present invention is not limited to this, and the received voice data may be analyzed and the recognition character string may be transmitted.
この場合、例えば、ユーザ６０が携帯端末２０のマイク２２から入力した音声データを常用文字認識エンジン３０により解析して認識文字列を抽出し、抽出した認識文字列を携帯端末２０に送信し、ユーザ６０が認識文字列を修正して登録文字列としてもよい。 In this case, for example, the voice data input by the user 60 from the microphone 22 of the mobile terminal 20 is analyzed by the common character recognition engine 30 to extract a recognition character string, and the extracted recognition character string is transmitted to the mobile terminal 20 to be transmitted to the user. 60 may modify the recognition character string to make it a registration character string.
この場合も、実施形態の設備機器情報収集システム１００と同様、サーバ１０は、常用文字認識エンジン３０と第１〜第４候補文字認識エンジン３５〜３８に音声データを送信し、受信した認識文字列と登録文字列とに基づいて、各文字認識エンジン３０，３５〜３８の文字認識精度を算出し、文字認識精度の高い文字認識エンジンを新常用文字認識エンジンとして選択する。そして、選択した新常用文字認識エンジンを常用文字認識エンジン３０に置き換える。 In this case as well, as in the equipment information collecting system 100 of the embodiment, the server 10 transmits voice data to the common character recognition engine 30 and the first to fourth candidate character recognition engines 35 to 38, and receives the recognition character string. The character recognition accuracy of each character recognition engine 30, 35 to 38 is calculated based on the registered character string and the character recognition engine having high character recognition accuracy is selected as the new common character recognition engine. Then, the selected new common character recognition engine is replaced with the common character recognition engine 30.
１０ サーバ、１１ 演算部、１２ 記憶部、１３ 通信部、１５ ネットワーク、２０ 携帯端末、２１ カメラ、２２ マイク、２３ ディスプレイ、３０ 常用文字認識エンジン、３５〜３８ 候補文字認識エンジン、４０ 空調機器、４１ 銘板、５０ 文字認識データベース、６０ ユーザ、１００ 設備機器情報収集システム。
10 server, 11 arithmetic unit, 12 storage unit, 13 communication unit, 15 network, 20 mobile terminal, 21 camera, 22 microphone, 23 display, 30 common character recognition engine, 35-38 candidate character recognition engine, 40 air conditioning equipment, 41 Nameplate, 50-character recognition database, 60 users, 100 equipment information collection system.
Claims (8)
前記ネットワークに接続され、設備機器情報を含むデータを取得し、取得したデータを送信する携帯端末と、
前記ネットワークに接続されてデータを受信し、受信したデータを解析してデータに含まれる文字列を認識し、認識した文字列を認識文字列として送信する常用文字認識エンジンと、を備える設備機器情報収集システムであって、
前記携帯端末は、取得したデータを前記常用文字認識エンジンと前記サーバとに送信し、前記常用文字認識エンジンから認識文字列を受信し、受信した認識文字列をユーザが正しい文字列に修正した登録文字列を前記サーバに送信し、
前記サーバは、
前記携帯端末からデータと登録文字列とを受信し、前記常用文字認識エンジンから認識文字列を受信し、受信したデータと認識文字列と登録文字列とを対応付けて格納する文字認識データベースを含み、前記文字認識データベースに格納したデータと認識文字列と登録文字列とに基づいて、前記ネットワークに接続されてデータを受信し、受信したデータを解析してデータに含まれる文字列を認識し、認識した文字列を認識文字列として送信する候補文字認識エンジン及び前記常用文字認識エンジンの文字認識精度の指標をそれぞれ算出し、
算出した各前記指標に基づいて、前記常用文字認識エンジンと前記候補文字認識エンジンとの中から新常用文字認識エンジンを選択し、
選択した前記新常用文字認識エンジンを前記常用文字認識エンジンに置き換えること、
を特徴とする設備機器情報収集システム。Servers connected to the network and
A mobile terminal that is connected to the network, acquires data including equipment information, and transmits the acquired data.
Equipment information including a common character recognition engine that is connected to the network, receives data, analyzes the received data, recognizes a character string contained in the data, and transmits the recognized character string as a recognition character string. It ’s a collection system,
The mobile terminal transmits the acquired data to the common character recognition engine and the server, receives a recognition character string from the common character recognition engine, and registers the received recognition character string modified by the user into a correct character string. Send the character string to the server and
The server
Includes a character recognition database that receives data and registered character strings from the mobile terminal, receives recognition character strings from the common character recognition engine, and stores the received data, recognition character strings, and registered character strings in association with each other. Based on the data stored in the character recognition database, the recognition character string, and the registered character string, the data is connected to the network, the data is received, the received data is analyzed, and the character string contained in the data is recognized. The index of the character recognition accuracy of the candidate character recognition engine and the common character recognition engine that transmits the recognized character string as the recognition character string is calculated, respectively.
Based on each of the calculated indexes, a new common character recognition engine is selected from the common character recognition engine and the candidate character recognition engine.
Replacing the selected new common character recognition engine with the common character recognition engine,
Equipment equipment information collection system featuring.
前記サーバは、
前記文字認識データベースに格納したデータから評価用データを選択し、
選択した前記評価用データを前記常用文字認識エンジンと前記候補文字認識エンジンとに送信し、
前記常用文字認識エンジンと前記候補文字認識エンジンとから認識文字列をそれぞれ受信し、
受信した各認識文字列と前記評価用データに対応する登録文字列とを対比して前記常用文字認識エンジンと前記候補文字認識エンジンとの前記指標をそれぞれ算出すること、
を特徴とする設備機器情報収集システム。The equipment information collection system according to claim 1.
The server
Select the evaluation data from the data stored in the character recognition database, and select
The selected evaluation data is transmitted to the common character recognition engine and the candidate character recognition engine.
Recognized character strings are received from the common character recognition engine and the candidate character recognition engine, respectively.
To calculate the index of the common character recognition engine and the candidate character recognition engine by comparing each received recognition character string with the registered character string corresponding to the evaluation data.
Equipment equipment information collection system featuring.
前記指標は、前記常用文字認識エンジン又は前記候補文字認識エンジンから受信した認識文字列と前記評価用データに対応する登録文字列とを対比して算出した前記常用文字認識エンジン又は前記候補文字認識エンジンの誤認識文字数であること、
を特徴とする設備機器情報収集システム。The equipment information collection system according to claim 2.
The index is the common character recognition engine or the candidate character recognition engine calculated by comparing the recognition character string received from the common character recognition engine or the candidate character recognition engine with the registered character string corresponding to the evaluation data. The number of misrecognized characters in
Equipment equipment information collection system featuring.
前記評価用データは、前記文字認識データベースに格納したデータから選択した複数の類似パターンデータであり、
前記サーバは、
選択した複数の前記類似パターンデータを前記常用文字認識エンジンと前記候補文字認識エンジンとにそれぞれ送信し、
前記常用文字認識エンジンと前記候補文字認識エンジンとから複数の前記類似パターンデータに対応する複数の認識文字列をそれぞれ受信し、
前記指標は、受信した複数の認識文字列と複数の前記類似パターンデータに対応する複数の登録文字列とを対比して算出した前記常用文字認識エンジン又は前記候補文字認識エンジンの誤認識回数であること、
を特徴とする設備機器情報収集システム。The equipment information collection system according to claim 2.
The evaluation data is a plurality of similar pattern data selected from the data stored in the character recognition database.
The server
A plurality of selected similar pattern data are transmitted to the common character recognition engine and the candidate character recognition engine, respectively.
A plurality of recognition character strings corresponding to the plurality of similar pattern data are received from the common character recognition engine and the candidate character recognition engine, respectively.
The index is the number of erroneous recognitions of the common character recognition engine or the candidate character recognition engine calculated by comparing the received plurality of recognition character strings with the plurality of registered character strings corresponding to the plurality of similar pattern data. thing,
Equipment equipment information collection system featuring.
前記サーバは、前記常用文字認識エンジンと前記候補文字認識エンジンとからそれぞれ受信した認識文字列と前記評価用データに対応する登録文字列とを対比して前記常用文字認識エンジンと前記候補文字認識エンジンの誤認識文字数をそれぞれ算出すると共に、
前記文字認識データベースに格納したデータから複数の類似パターンデータを選択し、選択した複数の前記類似パターンデータを前記常用文字認識エンジンと前記候補文字認識エンジンとにそれぞれ送信し、前記常用文字認識エンジンと前記候補文字認識エンジンとから複数の前記類似パターンデータに対応する複数の認識文字列をそれぞれ受信し、受信した複数の認識文字列と複数の前記類似パターンデータに対応する複数の登録文字列とを対比して前記常用文字認識エンジンと前記候補文字認識エンジンとの誤認識回数をそれぞれ算出し、
前記常用文字認識エンジンと前記候補文字認識エンジンとの各前記指標は、算出した各誤認識文字数と算出した各誤認識回数とにそれぞれ重み付けをして加算した数値であること、
を特徴とする設備機器情報収集システム。The equipment information collection system according to claim 2.
The server compares the recognition character strings received from the common character recognition engine and the candidate character recognition engine with the registered character strings corresponding to the evaluation data, and compares the common character recognition engine and the candidate character recognition engine. In addition to calculating the number of misrecognized characters in
A plurality of similar pattern data are selected from the data stored in the character recognition database, and the selected plurality of similar pattern data are transmitted to the common character recognition engine and the candidate character recognition engine, respectively, and the common character recognition engine A plurality of recognition character strings corresponding to the plurality of similar pattern data are received from the candidate character recognition engine, and the received plurality of recognition character strings and a plurality of registered character strings corresponding to the plurality of similar pattern data are obtained. By comparison, the number of erroneous recognitions between the common character recognition engine and the candidate character recognition engine is calculated.
Each of the indexes of the common character recognition engine and the candidate character recognition engine is a numerical value obtained by weighting and adding the calculated number of false recognition characters and the calculated number of false recognitions.
Equipment equipment information collection system featuring.
前記サーバは、
前記常用文字認識エンジンと前記候補文字認識エンジンとの文字認識精度の前記指標を所定の時間間隔でそれぞれ算出し、算出した各前記指標に基づいて前記新常用文字認識エンジンの選択を行うこと、
を特徴とする設備機器情報収集システム。The equipment information collection system according to any one of claims 2 to 5.
The server
The index of the character recognition accuracy of the common character recognition engine and the candidate character recognition engine is calculated at predetermined time intervals, and the new common character recognition engine is selected based on each of the calculated indexes.
Equipment equipment information collection system featuring.
前記サーバは、前記常用文字認識エンジンから受信した認識文字列と前記携帯端末から受信した登録文字列とを対比して前記常用文字認識エンジンの誤認識文字数を算出し、
前記常用文字認識エンジンの誤認識文字数の時系列的な変化が所定の閾値に達しない場合に、前記常用文字認識エンジンと前記候補文字認識エンジンとの文字認識精度の前記指標をそれぞれ算出し、算出した各前記指標に基づいて前記新常用文字認識エンジンの選択を行うこと、
を特徴とする設備機器情報収集システム。The equipment information collection system according to any one of claims 2 to 6.
The server calculates the number of erroneously recognized characters of the common character recognition engine by comparing the recognition character string received from the common character recognition engine with the registered character string received from the mobile terminal.
When the time-series change in the number of erroneously recognized characters of the common character recognition engine does not reach a predetermined threshold value, the index of the character recognition accuracy of the common character recognition engine and the candidate character recognition engine is calculated and calculated, respectively. To select the new common character recognition engine based on each of the indicators.
Equipment equipment information collection system featuring.
前記サーバが第１国に配置され、
前記携帯端末が第１国と異なる第２国に配置されていること、
を特徴とする設備機器情報収集システム。
The equipment information collection system according to any one of claims 2 to 7.
The server is located in the first country
The mobile terminal is located in a second country different from the first country,
Equipment equipment information collection system featuring.
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| PCT/JP2019/013179 WO2020194576A1 (en) | 2019-03-27 | 2019-03-27 | Facility device information collection system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| JP6780129B1 true JP6780129B1 (en) | 2020-11-04 |
| JPWO2020194576A1 JPWO2020194576A1 (en) | 2021-04-08 |
Family
ID=72609744
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| JP2019547327A Active JP6780129B1 (en) | 2019-03-27 | 2019-03-27 | Equipment information collection system |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP6780129B1 (en) |
| CN (1) | CN113614737B (en) |
| WO (1) | WO2020194576A1 (en) |
Family Cites Families (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2903779B2 (en) * | 1991-06-27 | 1999-06-14 | 日本電気株式会社 | Character string recognition method and apparatus |
| JPH10222612A (en) * | 1997-02-07 | 1998-08-21 | Hitachi Ltd | Document recognizing device |
| JP4055976B2 (en) * | 1999-07-29 | 2008-03-05 | 株式会社リコー | Document image processing method, document image processing apparatus, and recording medium |
| JP3803219B2 (en) * | 1999-12-14 | 2006-08-02 | 三菱電機株式会社 | Full-text search device and full-text search method |
| JP2003178067A (en) * | 2001-12-10 | 2003-06-27 | Mitsubishi Electric Corp | Portable terminal-type image processing system, portable terminal, and server |
| JP5434586B2 (en) * | 2009-12-29 | 2014-03-05 | オムロン株式会社 | Word recognition method, word recognition program, and information processing apparatus |
| EP2541441A4 (en) * | 2010-02-26 | 2014-10-15 | Rakuten Inc | Information processing device, information processing method, and recording medium that has recorded information processing program |
| EP2639705B1 (en) * | 2010-11-10 | 2016-04-13 | Rakuten, Inc. | Related-word registration device, information processing device, related-word registration method, program for related-word registration device, and recording medium |
| JP6102374B2 (en) * | 2013-03-15 | 2017-03-29 | オムロン株式会社 | Reading character correction program and character reading device |
| JP2016177383A (en) * | 2015-03-18 | 2016-10-06 | グローリー株式会社 | Identification number retrieval system and identification number retrieval method |
| CN107533740B (en) * | 2015-05-20 | 2023-07-25 | 三菱电机株式会社 | Device management apparatus, device management system, and recording medium |
| JP6740602B2 (en) * | 2015-12-11 | 2020-08-19 | ダイキン工業株式会社 | Information processing equipment |
| JP6591120B2 (en) * | 2017-06-28 | 2019-10-16 | 三菱電機ビルテクノサービス株式会社 | Device maintenance support system, device maintenance support device, and device maintenance support program |
-
2019
- 2019-03-27 JP JP2019547327A patent/JP6780129B1/en active Active
- 2019-03-27 WO PCT/JP2019/013179 patent/WO2020194576A1/en active Application Filing
- 2019-03-27 CN CN201980094258.3A patent/CN113614737B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| WO2020194576A1 (en) | 2020-10-01 |
| JPWO2020194576A1 (en) | 2021-04-08 |
| CN113614737B (en) | 2023-08-29 |
| CN113614737A (en) | 2021-11-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| RU2613734C1 (en) | Video capture in data input scenario | |
| CN109800320B (en) | Image processing method, device and computer readable storage medium | |
| CN110336838B (en) | Account abnormity detection method, device, terminal and storage medium | |
| CN111861731A (en) | Post-credit check system and method based on OCR | |
| CN110363112B (en) | Method and system for checking wiring position of distribution cable | |
| JP2014067171A (en) | Dictionary updating system for person authentication, dictionary updating device for person authentication, dictionary updating program for person authentication and system for person authentication | |
| CN111444075B (en) | Method for automatically discovering key influence indexes | |
| US20200210459A1 (en) | Method and apparatus for classifying samples | |
| CN111026738A (en) | Regional population monitoring method and system, electronic equipment and storage medium | |
| JP2021114700A (en) | Work support system and work support method | |
| KR20120066397A (en) | Method and apparatus for extracting text area, and automatic recognition system of number plate using the same | |
| JP5390440B2 (en) | Centralized information processing apparatus and centralized information processing system | |
| CN111325082A (en) | Personnel concentration degree analysis method and device | |
| CN111768170A (en) | Method and device for displaying operation correction result | |
| CN110555488A (en) | Image sequence auditing method and system, electronic equipment and storage medium | |
| CN110472499B (en) | Pedestrian re-identification method and device | |
| JP6780129B1 (en) | Equipment information collection system | |
| CN113807342A (en) | Method and related device for acquiring equipment information based on image | |
| CN111062448A (en) | Equipment type recognition model training method, equipment type recognition method and device | |
| CN111078983A (en) | Method for determining page to be identified and learning device | |
| CN110543812A (en) | information extraction method and device, electronic equipment and storage medium | |
| CN111077997A (en) | Point reading control method in point reading mode and electronic equipment | |
| CN111090989B (en) | Prompting method based on character recognition and electronic equipment | |
| CN113888600A (en) | Trajectory determination method and apparatus, electronic device and computer-readable storage medium | |
| CN110674809A (en) | Merchant information input method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20190829 |
|
| TRDD | Decision of grant or rejection written | ||
| A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20200915 |
|
| A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20201014 |
|
| R150 | Certificate of patent or registration of utility model |
Ref document number: 6780129 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
| S533 | Written request for registration of change of name |
Free format text: JAPANESE INTERMEDIATE CODE: R313533 |
|
| R350 | Written notification of registration of transfer |
Free format text: JAPANESE INTERMEDIATE CODE: R350 |
|
| R250 | Receipt of annual fees |
Free format text: JAPANESE INTERMEDIATE CODE: R250 |