JP2024028353A - Information processor and computer program - Google Patents

Abstract

To provide an information processor and a computer program capable of reducing workload on a meter reader in a meter-reading operation of reading out a measurement value of a meter with a device.SOLUTION: An information processor comprises a storage unit and a control unit. The storage unit stores entry data including identification information indicating types of an analysis engine and an identification engine associated with a master code. The control unit detects a secondary code of a measuring instrument and detects the master code included in the secondary code from photographed image data, detects the entry data associated with the master code from the storage unit, detects an image of a portion where a measurement value is displayed from the image data by using the identification engine corresponding to the identification information, and in a case where the image is detected, calculates measurement value data from the image by using the analysis engine corresponding to the identification information.SELECTED DRAWING: Figure 3

Description

Embodiments of the present invention relate to an information processing device and a computer program.

BACKGROUND OF THE INVENTION Conventionally, as part of meter reading work for equipment installed in office buildings, factories, public facilities, etc., inspectors have carried out maintenance of the equipment by reading meters that indicate the operating status of the equipment. In addition, in recent years, in order to eliminate the need for manual meter reading work, high-performance equipment has been introduced that is compatible with systems that transmit measurement data indicating operating status to a server via a network, where it can be stored and managed. It's coming.

However, equipment compatible with such systems is expensive and has not become widespread, and there are still many equipment that require meter reading by inspectors, and meter reading by inspectors is still not performed visually. ing. In addition, when it comes to reading the gas, electricity, and water meters of ordinary houses, it is common for meter readers to go out and read the meters individually.

Recently, services that support meter reading work have been provided. This service utilizes AI (Artificial Intelligence) to read meter values with equipment, but at the meter reading work site, inspectors and meter readers must: 1) Photograph the meter, 2) Image recognition using AI, 3) Image It is necessary to confirm the recognition result and 4) input it into the form.

These work routines may take more time than traditional paper-based meter readings. If it takes a long time, for example, if the work is done outdoors, depending on the season and the weather, the physical load on the meter reader may increase.

Patent No. 6514390

The problem to be solved by the present invention is to provide an information processing device and a computer program that can reduce the workload of a meter reader during meter reading work in which the measured value of a meter is read by a device.

The information processing device of the embodiment includes a storage unit and a control unit. The storage unit stores entry data including identification information indicating the type of recognition engine and analysis engine associated with the master code. The control unit detects the two-dimensional code of the measuring instrument from the captured image data, detects the master code included in the two-dimensional code, detects entry data associated with the master code from the storage unit, and stores the recognition information. A recognition engine corresponding to the identification information is used to detect the image of the part where the measurement value is displayed from the image data, and when an image is detected, an analysis engine corresponding to the identification information is used to extract the measurement value data from the image. demand.

1 is a diagram schematically showing an information processing system including an information processing device according to the present invention. 2 is a circuit block diagram showing the configuration of the portable terminal device shown in FIG. 1. FIG. 3 is a flowchart for explaining the operation of the portable terminal device shown in FIG. 2. 3 is a diagram showing an example of a display when reading a two-dimensional barcode in the operation shown in FIG. 2. FIG. 3 is a diagram showing an example of a display when reading measured values of the operation shown in FIG. 2. FIG. 3 is a diagram showing an example of a warning display when reading a two-dimensional barcode in the operation shown in FIG. 2. FIG. 3 is a diagram showing an example of a display when reading measured values of the operation shown in FIG. 2. FIG. FIG. 3 is a diagram showing an example of a display when correcting a meter reading value in the operation shown in FIG. 2; FIG. 3 is a diagram showing an example of a display when inputting a form for the operation shown in FIG. 2;

Hereinafter, one embodiment will be described with reference to the drawings.
FIG. 1 shows the configuration of an information processing system including an information processing device and a computer program according to an embodiment. This information processing system performs meter reading to read the values measured by meters. The meter readings obtained through meter reading are used for creating forms, etc. The form may be related to, for example, billing for usage fees.

In the following explanation, an example will be given in which the above information processing device and computer program are applied to reading meters for gas, electricity, water, etc. installed in office buildings, factories, public facilities, general houses, etc. I will explain.

In this example, a case in which a meter reader (or inspector) visits individual houses, buildings, facilities, etc., and reads electricity meters installed at or near each of them will be explained. Hereinafter, the installed location will be referred to as the meter reading site.

As shown in FIG. 1, this information processing system includes a portable terminal device 100, a wireless base station 200, a client terminal 300, a server 400, and a network 500. The server 400 communicates with the portable terminal device 100 or the client terminal 300 and performs information processing on their behalf to realize cloud computing related to meter reading. This system processes information obtained from meters MT1 to MTn installed at meter reading sites, and creates forms related to, for example, usage charges.

The meters MT1 to MTn are devices that measure and display numerical values such as electricity consumption, water supply consumption, drainage to sewerage, gas consumption, etc., as well as numerical values indicating the operating status of equipment that uses them. be. The meters MT1 to MTn are assumed to measure values related to infrastructure and lifelines, but may also measure values other than these.

In the following description, the numerical values measured by each of the meters MT1 to MTn will be referred to as measured values. Further, the value read, recorded or reported by a meter reader by operating the portable terminal device 100 is referred to as a meter reading value. That is, the measured value and the meter reading value are substantially the same value.

Furthermore, each meter has a sticker on which a two-dimensional barcode is printed on or near the surface on which the measured value is displayed. This two-dimensional barcode includes a master code for distinguishing the attached meter from other meters. In the following explanation, a case where a QR code (registered trademark) is applied as a two-dimensional barcode will be explained as an example.

This master code may be a plurality of alphanumeric characters, for example, and may be unique to the meter to which it is assigned, or may be unique at a level that distinguishes it from other meters within the same facility, or may be unique at a level that distinguishes it from other meters within the same facility. It may be unique at a level that distinguishes it from other meters within a region (for example, an administrative division).

In addition, 2D barcodes contain attribute information that indicates attributes (electricity, gas, water, supply entity, installation location, date and time of manufacture, date and time of installation, specifications that indicate performance, etc.), and for aggregating meter readings. It may also include input designation information that designates the application software to be used and the input fields of the summary table created by the application software.

Further, the two-dimensional barcode is decoded using a predetermined algorithm to obtain the above-mentioned master code. Note that a character string indicating this master code may be printed on the above-mentioned sticker or the like together with the two-dimensional barcode.

Further, each meter indicates a measured value in either a rotary type or a digital type on its display portion. In the case of a digital type, for example, multiple 7-segment displays that display numbers from 0 to 9 are used, and numerical values (measured values) are shown according to the amount of electricity used and the operating status of equipment as mentioned above. There are several possible configurations. In the case of a rotary type, for example, it is a mechanical type that uses multiple number wheels with numbers from 0 to 9 written on them, and the number wheels rotate according to the amount used, resulting in a multi-digit number (measured value). A direct reading configuration is possible.

The portable terminal device 100 is an information processing device that is carried by a meter reader when reading a meter, and is used to optically read and automatically input the measured values of the meters MT1 to MTn installed at the meter reading site, and send the data to the server 400. . Details will be described later.

The wireless base station 200 wirelessly communicates with the mobile terminal device 100 and connects the mobile terminal device 100 to the network 500. For example, if the network 500 is a LAN built within a business office, it is an access point that uses wireless LAN (IEEE802.11 series) as the communication method. If the network 500 is a mobile communication network of a communication carrier, it is a base station device that performs wireless communication compatible with standards such as 3G, 3.9G (LTE (registered trademark), etc.), 4G, 5G, etc.

In other words, wireless LAN and mobile phone networks are just examples, and as long as the mobile terminal device 100 can be connected to the network 500 by wireless communication, it is not limited to these methods, and Bluetooth (registered trademark) can be used. Any communication method is applicable. Furthermore, the network is not limited to one network, but may be a plurality of networks (for example, a local LAN and a mobile communication network) connected to each other.

The client terminal 300 is a terminal installed in the management department of the user of the meter reading service of the system and used by the operator of that department, and is, for example, an information processing device such as a desktop or laptop personal computer. .

The client terminal 300 has a function of downloading the data file uploaded to the server 400 by the portable terminal device 100, and performing reference/viewing, editing/aggregation, data conversion/file creation, and form issuing.

Further, the client terminal 300 may include all or part of the data processing functions of the portable terminal device 100 and execute the same processing as the portable terminal device 100. That is, the client terminal 300 may have similar functions so as to perform data processing in the portable terminal device 100 instead, except for photographing at the meter reading site.

The server 400 is the core of the cloud computing provided by the system, and stores and manages information uploaded from the mobile terminal device 100 and the client terminal 300, and performs predetermined data processing on the information. Alternatively, this stored information or data-processed information is provided (downloaded) to the portable terminal device 100 or the client terminal 300, thereby realizing data sharing among the users of the system.

Furthermore, the server 400 handles some or all of the data processing by the portable terminal device 100 and the client terminal 300 on behalf of each device, thereby making it possible to save the computer resources of each device. The server 400 also provides the OS and application software used by the portable terminal device 100 and the client terminal 300 to each device, and updates them to the latest information to realize services.

Therefore, the database of the server 400 stores the meter reading data uploaded from the mobile terminal device 100, and stores various programs of the mobile terminal device 100 and the client terminal 300.

The services provided by the server 400 may be provided on a subscription basis to the portable terminal device 100 or client terminal 300 owned (or lent) by the client receiving the service.

Next, with reference to FIG. 2, the portable terminal device 100 will be described.
The portable terminal device 100 is a terminal that a meter reader carries to a meter reading site and uses as an operator, and is a portable information processing device such as a smartphone, a tablet computer, or a laptop personal computer.

Further, as shown in FIG. 2, the portable terminal device 100 has a hardware configuration similar to that of a general computer, and includes at least a communication section 101, an input section 102, a display section 103, a camera 104, and It includes a GNSS (Global Navigation Satellite System) receiving section 105, a storage section 106, and a control section 110.

The communication unit 101 is a wireless communication interface that establishes a wireless communication link with the wireless base station 200 and performs communication, and communicates with the server 400 via the wireless base station 200 and the network 500. The wireless communication corresponds to a communication method conforming to the specifications of the wireless base station 200. Examples of this communication method include various standards such as wireless LAN (IEEE802.11 series), 3G, 3.9G (LTE (registered trademark), etc.), 4G, 5G, Bluetooth (registered trademark), etc. Either one may be used, and it is not limited to these.

The input unit 102 includes input devices such as a touch panel placed on a display unit 103, which will be described later, and a key switch provided on the casing of the portable terminal device, and receives input of various information and instructions from a meter reader. Detect. The touch panel allows input using a stylus or finger, and various methods such as a capacitance method, a resistive film method, and a projection infrared method can be applied.

The display unit 103 visually provides information to the meter reader, and displays, for example, information input fields, software keys, and various images (photos, CG (computer graphics) images).

The information input fields and software keys are displayed under the control of the control unit 110, which will be described later, in a manner that corresponds to the operation on the input unit 102 described above. Various display devices such as a liquid crystal panel, an organic EL (Electro Luminescence) panel, and electronic paper can be used as the device used for display.

The camera 104 includes an optical system including a plurality of lenses and a zoom mechanism that changes the distance between these lenses, an imaging unit including an image sensor such as a CMOS (Complementary MOS), and an image sensor. The digital camera is equipped with a signal processing unit that generates image data in a predetermined format (for example, JPEG (Joint Photographic Experts Group)) from captured image signals.

Note that the image data obtained by the camera 104 includes additional information (for example, Exif (Exchangeable image (file format) data) is added and recorded by the control unit 110.

The GNSS receiving unit 105 performs position measurement (positioning) using signals emitted from a plurality of GNSS satellites, ground reference stations, etc., and receives the current time.

The storage unit 106 stores the OS (Operating System) and application software of the control unit 110, which will be described later, data generated during the operation of the application software, various parameters, data input from meter readers, and images captured by the camera 104. RAM (Random Access Memory ), ROM (Read Only Memory), flash memory such as SSD (Solid State Drive), and storage devices such as HDD (Hard Disk Drive) are installed in combination according to the characteristics of the data.
The storage unit 106 also stores various data such as form data 106a, setting information data 106b, and meter image data 106c.

The form data 106a is data created in order to total the meter reading values of each meter every predetermined meter reading period, and is uploaded to the server 400 when new data is created. Specifically, the form data 106a is data created for each meter reading site. The form data 106a is data in which information such as meter information, meter image, current meter reading value, current usage amount, previous meter reading value, and previous usage amount is associated with each other. If installed, the above information is included for each meter.

Here, the meter information includes a master code for identifying the meter, information indicating what the meter measures (tenant, parking lot, shared space, etc.), and type (gas, electricity, water, etc.). The meter image is an image that serves as the basis for meter reading values, that is, an image photographed at the time of meter reading.

The current meter reading value is the value read during the current meter reading. The current usage amount is the difference between the current meter reading value and the previous meter reading value. The previous meter reading value is the value read in the previous meter reading. The previous usage amount is the difference between the previous meter reading value and the meter reading value the day before last.

The setting information data 106b is setting information necessary for totaling the meter reading values of each meter. Specifically, like the form data 106a, it is created for each meter at each meter reading site. Here, the setting information data for each meter will be referred to as entry data.

Entry data includes the master code of the meter, meter type, number of meter digits, number of display digits, presence or absence of rounding, position of decimal point, shooting conditions (timer setting time, etc.), and type of recognition engine and analysis engine to be used. Including.

The meter image data 106c is image data taken by the camera 104 of the meter reading value for meter reading, and when it is actually used to obtain the meter reading value, the part displaying the measured value is cut out from the image. and recorded in the form data 106a as a meter image.

The control unit 110 includes a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), a chipset, a RAM, a ROM, etc., and centrally controls each unit of the portable terminal device 100.

ROM stores firmware and setting values (various parameters). The CPU loads the OS and application software from the storage unit 106 into the RAM according to the firmware, executes the OS and application software, and uses the RAM as a work area to achieve various control functions. do.

The control unit 110 implements at least the following control functions by executing the OS and application software. That is, by executing software (programs), the control unit 110 functions as a communication control unit 111, an input control unit 112, a display control unit 113, an image processing unit 114, a position information processing unit 115, and a meter reading data aggregation processing unit 116. functions, and these functions can function in conjunction with each other.

Note that it is also possible to integrate some or all of these functions with other functions, or to divide each of the functions 111 to 116 into multiple functional blocks from a different perspective and explain them using different expressions. .

The communication control unit 111 controls the communication unit 101 to establish a wireless communication link with the wireless base station 200 according to a predetermined communication protocol, and further communicates with the server 400 through the network 500. Data download and data upload to the server 400 are performed.

The input control unit 112 controls the input unit 102 and interprets the meter reader's request from the meter reader's operation on the input unit 102 based on the results detected by the input unit 102, inputs information (character strings, etc.), and inputs data. It accepts import of images and changes in association, and imports image data captured by the camera 104.

The display control unit 113 controls the display unit 103 to display various information including characters and images, and a graphical user interface (hereinafter referred to as GUI (Graphical User Interface)). Examples of the GUI include the aforementioned information input field and software keys, and these displays correspond to input through the touch panel of the input unit 102. In addition, a guide display such as indicating the image capture position is performed.

The image processing unit 114 performs image processing on image data captured by the camera 104, and has, for example, a function as a recognition engine and a function as an analysis engine.

The recognition engine detects a QR code or displays the measured values of various meters from an image based on the image data, using pattern recognition using AI (Artificial Intelligence), for example, on the image data obtained by the camera 104. The meter section (hereinafter referred to as the meter section) is equipped with a function to detect the position.

On the other hand, the analysis engine decodes the QR code and performs character recognition processing (OCR (Optical Character Recognition)) such as optical character recognition on the portion of the image detected by the recognition engine. Multiple types of analysis engines are prepared for character recognition processing. Each analysis engine corresponds to a specific type of meter and converts the displayed measurement value into text using an analysis algorithm that is appropriate for the respective measurement value display.

Specifically, since the typeface of the Arabic numerals differs depending on the meter, and the combination of numbers and background colors differs, an analysis engine is prepared that has an analysis algorithm suitable for the display of each meter.

The position information processing unit 115 includes an acceleration sensor, a gyro sensor, etc., and uses the detection results thereof, the positioning result by the GNSS receiving unit 105, and/or the positioning information obtained from the wireless base station 200 by the communication control unit 111 to , the position of the portable terminal device 100 is determined.

The meter reading data aggregation processing section 116 centrally controls various processes related to the aggregation of meter reading data, and includes a form creation section 116a, a QR code processing section 116b, a meter reading section 116c, and a correction processing section 116d. .

The form creation unit 116a performs processing to create form data 106a based on data obtained by meter reading.
The QR code processing unit 116b detects a QR code included in an image obtained by the camera 104, and performs a process of analyzing the detected QR code.

The meter reading unit 116c performs a process of photographing the measured values of the meters MT1 to MTn based on the analysis results of the QR code processing unit 116b, converting the measured values into text by image analysis, and obtaining a meter reading value.
The correction processing section 116d performs a process of correcting the meter reading value obtained by the meter reading section 116c in response to a correction request from the operator.

Note that the detection section, meter reading section, and timer starting section in the claims correspond to the meter reading section 116c. Further, the display control section in the claims corresponds to the display control section 113 and the meter reading section 116c.

Next, the operation of the information processing system having the above configuration will be explained. In the following description, among the processes in the portable terminal device 100, meter reading data aggregation processing (hereinafter referred to as meter reading data aggregation processing) will be described in particular.

This meter reading data aggregation processing is carried out by the meter reading data aggregation processing unit 116, which supports the meter reader in reading the meters MT1 to MTn, reads the measured values displayed on the meters MT1 to MTn, and tallies the resulting form data. 106a.

FIG. 3 is a flowchart for explaining the meter reading data aggregation process. This meter reading data aggregation process is realized by the control unit 110 executing application software (program) functioning as the meter reading data aggregation processing unit 116. Note that although the following description will be made using one application software, it may be realized by a plurality of application software operating in cooperation or in succession.

When the application software is executed, in step 301, the form creation unit 116a displays a template of the form data 106a, and the process proceeds to step 302. Specifically, the form creation unit 116a reads the template data of the form data 106a stored in the storage unit 106, and outputs the read template data to the display control unit 113. On the other hand, the display control unit 113 displays the form on the display unit 103 based on the template data.

In step 302, the form creation unit 116a determines whether the operator has given a request for meter reading or any other request. Here, if a request for meter reading is given, the process moves to step 304; on the other hand, if any other request is given, the process moves to step 303, and other processing according to the request is executed. do.

Specifically, when the operator touches an input field of a form displayed on the display unit 103, the input unit 102 detects this operation and notifies the input control unit 112. Since the touched position is an input field, the input control unit 112 determines that the operator is requesting input of a meter reading value. In response to this determination, the form creation unit 116a notifies the QR code processing unit 116b of the occurrence of the meter reading request, and proceeds to step 304.

In step 304, the QR code processing unit 116b activates the camera 104, and proceeds to step 305. The activated camera 104 repeatedly takes pictures at a predetermined cycle and continues to output image data obtained by taking pictures to the image processing unit 114 until the process moves to step 320, which will be described later.

In step 305, the QR code processing unit 116b displays a guide to read the QR code on the display unit 103, and proceeds to step 306.

Specifically, the QR code processing unit 116b instructs the display control unit 113 to display the above-mentioned guidance, and the display control unit 113 that receives the instruction instructs the display unit 103 to display the information as shown in FIG. 4, for example. Display a GUI (Graphical User Interface) that prompts reading.

In the example of FIG. 4, a shooting guide frame Fq is displayed, and inside the shooting guide frame Fq, the image shot by the camera 104 is displayed as is, and outside the shooting guide frame Fq, for example, a gray mask with lower transmittance is displayed. The display prompts the operating meter reader to place the QR code within the photographing guide frame Fq. Upon seeing this, the meter reader moves the camera 104 of the portable terminal device 100 over the QR codes pasted on the meters MT1 to MTn forward and backward, left and right, and up and down to capture the composition within the shooting guide frame Fq. Adjust so that the above QR code fits within the QR code.

In the example of FIG. 4, a zoom key Z and software keys S1 to S4 are provided as the GUI. The zoom key Z is used to receive an instruction for the image processing unit 114 to perform enlargement processing on image data photographed by the camera 104.

For example, "1X" is no enlargement processing, "2X" is a process that doubles the angle of view at the "1X" ratio, "3X" is a process that triples the angle of view at the "1X" ratio, and "Maximum" is a process that triples the angle of view at the "1X" ratio. Corresponding to the process of maximizing the angle of view in terms of performance with the "1X" ratio, the image processing unit 114 performs the process according to the operated zoom key Z, and displays the image photographed by the camera 104.

In the above description, the digital zoom is explained as an example in which the angle of view is changed by image processing by the image processing unit 114. However, the optical zooming mechanism provided in the camera 104 is driven and controlled to change the angle of view according to the operated software key. Alternatively, it may be combined with digital zoom by the image processing unit 114.

Further, the "form" of the software key S1 shown in FIG. 4 is used to move to step 301, for example, to proceed to processing for editing a form. "QR" of the software key S2 is for moving to step 303, for example, to move to a process for reading a QR code. At the time of step 305, the process for reading the QR code is in progress, so in the example of FIG. 4, S2 is displayed darker than the other software keys S1, S3, and S4 to indicate that it is currently active. It shows.

"Re-photographing" of the software key S3 is to re-photograph the meter's measured value by, for example, moving to step 312 in a step after reading the meter's measured value once (for example, from step 320 described later). This is for transitioning to processing for "Registration" of the software key S4 is for inputting the measurement value read from the meter into the form as a meter reading value.

In step 306, the QR code processing unit 116b performs tracking processing to search for a QR code image included in the image data photographed by the camera 104, and proceeds to step 307.

Specifically, the QR code processing unit 116b instructs the image processing unit 114 to perform a process of searching for a QR code on the image data corresponding to the shooting guide frame Fq. On the other hand, the image processing unit 114 uses a recognition engine to search for a QR code on the image data corresponding to the shooting guide frame Fq.

In step 307, the QR code processing unit 116b determines whether a QR code has been detected from the image data captured by the camera 104. Here, if the QR code can be detected, the process moves to step 308, whereas if it cannot be detected, the process moves to step 306 to try to detect the QR code again.

Specifically, the QR code processing unit 116b waits for the QR code search result by the image processing unit 114. On the other hand, when the image processing unit 114 detects the QR code from the image data corresponding to the shooting guide frame Fq using the recognition engine, it notifies the QR code processing unit 116b of this fact, and In response, the QR code processing unit 116b moves to step 308.

On the other hand, if the image processing unit 114 cannot detect the QR code from the image data corresponding to the shooting guide frame Fq using the recognition engine, it notifies the QR code processing unit 116b of this fact. The QR code processing unit 116b moves to step 306.

In step 308, the QR code processing unit 116b detects a master code from the QR code detected in step 307, and proceeds to step 309.
Specifically, the QR code processing unit 116b causes the image processing unit 114 to analyze the QR code, and waits for the analysis result. In response, the image processing unit 114 uses the analysis engine to decode the QR code image detected by the recognition engine in step 307, and detects the master code included in the QR code. do.

The detected master code is notified from the image processing unit 114 to the QR code processing unit 116b, and the QR code processing unit 116b, having been notified of the master code, proceeds to step 309. Further, the QR code reading guidance display started in step 305 is ended, and the display is switched to, for example, the one shown in FIG. That is, the image taken by the camera 104 is displayed as is.

In step 309, the QR code processing unit 116b detects entry data including the master code detected in step 308 from the setting information data 106b, reads the detected entry data into the work area of the control unit 110, and proceeds to step 310. .

In step 310, the QR code processing unit 116b refers to meter reading history data stored separately in the work area based on the master code detected in step 308, and determines whether or not the same meter has been read recently. judge. Here, if the meter has been read (there is a meter reading history), the process moves to step 311; on the other hand, if the meter has not been read (there is no meter reading history), the meter reading unit 116c is notified, and then the process moves to step 312. do.

In step 311, the QR code processing section 116b displays a warning to explain that the QR code of a meter that has already been read has been read, and notifies the meter reading section 116c, after which the process proceeds to step 312.
Specifically, the QR code processing unit 116b causes the display control unit 113 to display, for example, a warning display Al as shown in FIG. 6 on the display unit 103.

In step 312, the meter reading unit 116c, which was notified in step 310 or step 311, searches for an image of the meter part included in the image data taken by the camera 104, based on the entry data detected in step 309. is started, and the process moves to step 313.

Specifically, the meter reading unit 116c refers to the entry data detected in step 309, and transmits the identification information of the recognition engine and analysis engine used in the image processing unit 114, which is included in this entry data, to the image processing unit 114. Notice.

In response, the image processing unit 114 uses a recognition engine corresponding to the notified identification information to analyze an image based on the image data taken by the camera 104, and converts an image of the meter portion included in this image. Start the meter part search process.

The meter partial search process started here is repeatedly executed at a predetermined cycle until proceeding to step 320, which will be described later. That is, based on the image data photographed by the camera 104, coordinates indicating the range of the image of the meter portion are determined at each predetermined period.

Note that the shape, notation, color scheme, etc. of the meter part vary widely depending on the type of meter and model, and for recognition and analysis, a recognition engine and an analysis engine corresponding to the meter type are used. There is a need to. Therefore, as described above, the entry data includes identification information indicating the type of recognition engine and analysis engine in association with the master code.

In step 313, the meter reading unit 116c determines whether the meter portion can be detected from the image data captured by the camera 104. Here, if the meter portion can be detected, the process moves to step 314, whereas if it cannot be detected, the process moves to step 312 to continue detecting the meter part.

Specifically, the meter reading section 116c waits for the search results of the meter section by the image processing section 114. On the other hand, when the image processing unit 114 detects the position of the meter portion (coordinates on the image) from the image data taken by the camera 104 using the recognition engine, it notifies the meter reading unit 116c of this fact. In response to this, the meter reading section 116c moves to step 314.

On the other hand, if the image processing unit 114 cannot detect the meter portion from the image data using the recognition engine, it notifies the meter reading unit 116c of this fact, and the meter reading unit 116c then proceeds to step 312. do.

In step 314, the meter reading unit 116c displays the position of the meter portion detected in step 313 on the meter image displayed on the display unit 103, starts a timer, and proceeds to step 315.

Specifically, the meter reading unit 116c notifies the display control unit 113 of the coordinates of the meter portion notified from the image processing unit 114, and starts a timer. On the other hand, the display control unit 113 displays a meter frame F indicating the reading position of the meter portion, as shown in FIG. 7, for example, based on the coordinate information notified from the meter reading unit 116c. Thereby, the meter reader can recognize which part on the image is to be read by the portable terminal device 100 as the meter part.

In step 315, the meter reading unit 116c starts automatic adjustment of the focal length of the camera 104 (autofocus adjustment), and proceeds to step 316.
Specifically, the meter reading unit 116c controls the camera 104 to adjust the focal length, exposure, etc. regarding the coordinates of the meter portion notified from the image processing unit 114 among the image data obtained by the camera 104. Start adjusting.

In step 316, the meter reading unit 116c determines whether the timer started in step 314 has counted a predetermined time T (time T has elapsed). Here, if the predetermined time T has elapsed, the process moves to step 320, whereas if the predetermined time T has not elapsed, the process moves to step 317.

Note that this predetermined time T can be arbitrarily set in advance by the meter reader, and can be selected from a plurality of times, such as 0.1 seconds, 0.5 seconds, 1 second, and 3 seconds, for example. Alternatively, the setting of time T may be included in the entry data, and the value may be followed in step 316.

In step 317, the meter reading unit 116c determines whether or not the meter portion within the meter frame F is in focus (in focus) in the image data obtained by the camera 104. Here, if the image is in focus, the process moves to step 320, whereas if it is not in focus, the process moves to step 318.

In step 318, the meter reading section 116c determines whether or not the meter reader has operated the shutter. Here, if the shutter operation has been performed, the process moves to step 320, whereas if the shutter operation has not been performed, the process moves to step 319.

Specifically, the meter reading section 116c monitors notifications from the input control section 112. When a meter reader performs a shutter operation on the input unit 102, the input control unit 112 detects this operation and notifies the meter reading unit 116c.

In step 319, the meter reading section 116c determines whether or not the meter has been lost. Here, if the meter part is lost, the process moves to step 312 and the search for the meter part is started again. On the other hand, if the meter has not been lost, the process moves to step 316, and as described above, the timeout in step 316, the focusing in step 317, and the shutter operation in step 318 are monitored in this order.

Specifically, the meter reading unit 116c determines whether or not the recognition engine of the image processing unit 114 has lost sight of the meter (the state of not being able to detect it has continued for a predetermined period of time). If it is lost, the image processing unit 114 stops notifying the coordinates of the meter portion, and therefore the display of the meter frame F by the display control unit 113 also stops.

In step 320, the meter reading unit 116c saves the latest image data obtained by the camera 104 at the time of transition to step 320 in the storage unit 106 as data to be read from the measured value (meter image data 106c). , the process moves to step 321.

Specifically, the meter reading unit 116c calculates an image of a portion of the latest image data corresponding to the meter frame F based on the coordinates notified from the recognition engine of the image processing unit 114. The meter image data 106c is cut out and stored in the storage unit 106 as meter image data 106c.

In step 321, the meter reading unit 116c performs character recognition processing on the meter image data 106c saved in step 320, reads the measured value displayed on the meter, converts it into text, and proceeds to step 322.

Specifically, the meter reading unit 116c instructs the image processing unit 114 to perform character recognition processing using the analysis engine. On the other hand, the image processing unit 114 uses the analysis engine determined in step 312 to perform character recognition processing on the meter image data 106c, and obtains a numeric string converted into text. This number string is sent to the meter reading section 116c.

In step 322, the meter reading unit 116c refers to the entry data specified in step 309, determines the position of the decimal point for the number string obtained in step 321, and generates a temporary meter reading value. This provisional meter reading value is notified to the correction processing unit 116d, and the process moves to step 323.

Specifically, the meter reading unit 116c refers to the entry data specified in step 309, and based on the information included in this data about the number of digits of the meter, the number of displayed digits, the presence or absence of rounding, and the position of the decimal point, The numerical string obtained in step 321 is rounded off as necessary, and the position of the decimal point is determined to generate a temporary meter reading value.

In step 323, the correction processing unit 116d displays a GUI for adjusting the provisional meter reading value on the display unit 103, and proceeds to step 324.
Specifically, the correction processing unit 116d outputs the temporary meter reading value generated in step 322 to the display control unit 113, and causes the GUI to be displayed. In this GUI, for example, as shown in FIG. 8, the meter image data 106c saved in step 320 is displayed as a meter part display P, and the digits are arranged to match this display P, and a temporary meter reading value is displayed and corrected. A modification window M is displayed to perform the modification.

In the example shown in FIG. 8, the correction window M virtually displays each digit on a rotating reel to resemble the appearance of the actual meter part, and displays the temporary meter reading value as the initial value. . Then, by tracing the display of each reel up or down with a finger or the like, a virtually rotated display is performed.

The input control unit 112 detects such a tracing operation through the input unit 102, and the display control unit 113 rotates and displays the reel according to the detection result, allowing the meter reader to make intuitive corrections. can.

Note that data for a plurality of types of display formats for the modified window M are stored in the storage unit 106 in advance. Furthermore, a format code specifying the display format is included in the entry data in association with the master code. Then, the modification processing unit 116d reads out the display format data of the format code included in the entry data from the storage unit 106 and displays it, thereby making the display format of the modification window M match the meter image data 106c. be able to.

In step 324, the correction processing unit 116d shifts to processing according to the operation from the meter reader. Here, if there is an operation to request re-photography (operation to the software key S3 mentioned above), the process moves to step 312, and an operation to correct the provisional meter reading value (operation to the correction window M mentioned above) is performed. ) has been performed, the process moves to step 325, and if there is an operation requesting to confirm the provisional meter reading value as the meter reading value (operation to the software key S4 described above), the process moves to step 326. do. Specifically, the correction processing unit 116d waits for the detection result of the operation on the input unit 102 from the input control unit 112, and shifts to processing according to the notified detection result.

In step 325, the correction processing unit 116d corrects the provisional meter reading value in response to a request from the meter reader, and proceeds to step 324. At this time, the meter reader can input correction instructions while comparing the meter portion display P and the correction window M on the display unit 103.

Specifically, in the example of FIG. 8, when the meter reader performs an operation to rotate the reel virtually displayed in the correction window M with a finger or the like, this operation is transmitted to the input control unit 112 through the input unit 102. The display control unit 113 virtually rotates and displays the operated reel according to the detection result.

In step 326, the correction processing unit 116d detects the numerical value displayed in the correction window M as a meter reading value, creates the form data 106a by inputting this value into the input field of the form operated in step 302, and The process moves to step 304.

Specifically, in the example of FIG. 8, the correction processing unit 116d detects the value displayed in the center column among the reels virtually displayed in the correction window M as the meter reading value. For example, as shown in FIG. 9, the meter reading value is input into the input field of form X (the field of "meter reading value" for "this month"). It also updates the meter reading history in the work area and saves it as meter reading complete.

In addition, as shown in FIG. 9, the correction processing unit 116d inputs the "meter reading value" of "previous month" from the value in the "meter reading value" column of "this month" in the "increase amount" input field of "this month". The value obtained by subtracting the value in the field is calculated and inputted, and an image based on the meter image data 106c is displayed in the "meter image" field.

Thereafter, when the meter reader performs an operation to request the end of the inspection work through the input unit 102, the form data 106a in the work area is saved in the storage unit 106 by an interrupt process not shown in FIG. 3, and the process is executed. finish. This saved form data 106a is uploaded to the server 400 at a predetermined timing via the communication unit 101.

As described above, in the information processing system having the above configuration, when the meter portion of the meters MT1 to MTn is detected in the portable terminal device 100, the timer started at that time times out or the camera 104 is activated. When the camera 104 focuses, the meter portion is detected from the image taken by the camera 104 and the measured value is read without the need for shutter operation.

Therefore, according to the information processing system configured as described above, after the meter portion is detected, the meter reader can read the meter portion from the image captured by the camera 104 without operating the shutter. Therefore, the meter reader can reduce the burden of meter reading work of reading meter values with a device.

It should be noted that the present invention is not limited to the above-described embodiments as they are, and in the implementation stage, the constituent elements can be modified and embodied without departing from the spirit of the invention. Moreover, various inventions can be formed by appropriately combining the plurality of components disclosed in the above embodiments. For example, a configuration in which some components are deleted from all the components shown in the embodiments is also conceivable. Furthermore, the components described in different embodiments may be combined as appropriate.

For example, in the above embodiment, after detecting the meter portion in step 313, that portion is indicated by the meter frame F, and when the started timer times out or when the camera 104 focuses, the measured value is read. However, instead of this, for example, when the meter portion is detected in step 313, the measured value may be read immediately. In other words, the process may immediately proceed from step 313 to step 320.
It goes without saying that various other modifications can be made without departing from the spirit of the invention.
[C1]
computer,
a detection unit that detects an image of a portion displaying a measurement value based on the image data;
A program characterized in that, when the detection unit detects an image of a portion displaying the measurement value, the program functions as a meter reading unit that obtains the measurement value from the image.
[C2]
further causing the computer to function as a timer starting unit that starts a timer when the detection unit detects an image of a portion displaying the measured value;
The program according to claim 1, wherein the meter reading unit calculates the measured value from an image when the timer measures a predetermined time.
[C3]
1 . The meter-reading unit detects an image of a portion where the detection unit displays the measured value, and when the image is in focus, the meter-reading unit obtains the measured value from the image. The program described in.
[C4]
The computer is further caused to function as a display control unit that displays an image based on the image data and also displays a display for identifying a portion of the image in which the measurement value detected by the detection unit is displayed on the displayed image. The program according to claim 1, characterized in that:
[C5]
a detection unit that detects an image of a portion displaying a measurement value based on the image data;
An information processing device comprising: a meter reading section that calculates the measured value from the image when the detection section detects an image of a portion displaying the measured value.

100... Portable terminal device, 101... Communication unit, 102... Input unit, 103... Display unit, 104... Camera, 105... GNSS receiving unit, 106... Storage unit, 106a... Form data, 106b... Setting information data, 106c... Meter image data, 110... Control section, 111... Communication control section, 112... Input control section, 113... Display control section, 114... Image processing section, 115... Position information processing section, 116... Meter reading data aggregation processing section, 116a... Form creation unit, 116b...QR code processing unit, 116c...meter reading unit, 116d...correction processing unit, 200...wireless base station, 300...client terminal, 400...server, 500...network, MT1 to MTn...meter.

Claims (5)

computer,
Detecting a two-dimensional code of a measuring instrument from the photographed image data and detecting a master code included in the two-dimensional code,
detecting entry data including identification information indicating the type of recognition engine and analysis engine associated with the master code from the storage unit;
detecting an image of a portion displaying a measurement value from the image data using the recognition engine corresponding to the identification information;
when the image is detected, using the analysis engine corresponding to the identification information to obtain measurement value data from the image;
A computer program that makes it work. It is determined whether the detected two-dimensional code has already been read for meter reading, and when it is determined that the two-dimensional code has been read, a message indicating that the two-dimensional code has been read is displayed. The computer program according to claim 1, further configured to function as:. 2. The computer program according to claim 1, further configured to detect the image of a portion where the measured value is displayed, and to obtain the measured value from the image when the image is in focus. 2. The device according to claim 1, further comprising: displaying an image based on the image data; and displaying, on the displayed image, a display for identifying a portion of the image in which the detected measurement value is displayed. computer program. a storage unit that stores entry data including identification information indicating the type of recognition engine and analysis engine associated with the master code;
From the photographed image data, a two-dimensional code of the measuring instrument is detected, the master code included in the two-dimensional code is detected, the entry data associated with the master code is detected from the storage unit, and the identification is performed. Using the recognition engine corresponding to the information, detect an image of a portion displaying the measurement value from the image data, and when detecting the image of the portion displaying the measurement value, the image corresponds to the identification information. An information processing device comprising: a control unit that uses the analysis engine to obtain measurement value data from the image.