JP2022132395A - Computer apparatus, control method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing system configured to input a result of inspection more efficiently, an information processing method, and a computer program.

SOLUTION: An information processing system includes an image acquisition unit of an information processing apparatus, a learning unit of a host server, a learning result storage unit, a meter reading value generation unit, a form data storage unit, and an input unit. The image acquisition unit acquires image data related to a meter and transmits the image data to the host server. The learning unit generates learning results for acquiring a meter reading value from image data related to a meter reading value area, for each meter type, on the basis of training data. The learning result storage unit stores the learning results. The meter reading value generation unit generates a meter reading value from image data related to the meter reading value area, on the basis of the stored learning results. The form data storage unit stores form data. The input unit inputs the generated meter reading value to an input item of form data which is determined by a two-dimensional code related to a meter acquired as image data.

SELECTED DRAWING: Figure 15

COPYRIGHT: (C)2022,JPO&INPIT

Description

TECHNICAL FIELD Embodiments of the present invention relate to an information processing system, an information processing method, and a computer program.

2. Description of the Related Art In meter reading work for equipment installed in office buildings or factories, inspectors read meters that indicate the operating states of the equipment to maintain the equipment. In recent years, equipment that can connect to a network and store and manage data on a server has been introduced. However, there are only a few such facilities, and there are still many facilities that do not support network connection. Therefore, inspection staff still perform visual inspection of these equipment.

Meter reading work by inspectors is unavoidable due to human error. Therefore, a method has been devised in which an image including the meter reading value is photographed and an image recognition technology is applied to the photographed image to automatically read the meter reading value. However, the inspector may input the read meter value into the wrong input item.

JP-A-8-7078

The problem to be solved by the present invention is to provide an information processing system, an information processing method, and a computer program capable of inputting inspection work results into a form more efficiently.

An information processing system according to an embodiment is an information processing system that includes a host server and an information processing device that can communicate with the host server, and that inputs meter readings that indicate the operating status of equipment to form data. The information processing system includes an image acquisition unit of the information processing device, a learning unit of the host server, a learning result storage unit of the host server, a meter reading value generation unit of the host server, a form data storage unit, and an input unit. have The image acquisition unit acquires image data related to the meter and transmits the acquired image data to the host server. Based on the teacher data, the learning unit generates, for each type of meter, a learning result used to acquire the meter reading value from the image data related to the meter reading value region of the meter. The learning result storage unit stores learning results. The meter reading value generation unit generates a meter reading value from the image data related to the meter reading value area acquired by the image acquisition unit, based on the learning result stored in the learning result storage unit. The form data storage unit stores the form data. The input unit inputs the meter reading value generated by the meter reading value generation unit to the input item of the form data determined by the two-dimensional code related to the meter acquired as image data by the image acquisition unit.

1 is a system configuration diagram showing a configuration example of an inspection support system 1 according to a first embodiment; FIG. 2 is a functional block diagram showing the functional configuration of the information processing apparatus 100 according to the first embodiment; FIG. FIG. 2 is a functional block diagram showing the functional configuration of an upper server 200 according to the first embodiment; FIG. 4 is a diagram showing a specific example of a form data table according to the first embodiment; FIG. A diagram showing a specific example of a form image according to the first embodiment. A diagram showing a specific example of a form input image according to the first embodiment. FIG. 4 is a diagram showing a specific example of reading a two-dimensional barcode according to the first embodiment; The figure which shows one specific example of reading of the meter-reading value of 1st Embodiment. A diagram showing a specific example of a form input image in which meter reading values are input according to the first embodiment. A diagram showing a specific example of a form input image in which meter reading values are input according to the first embodiment. FIG. 5 is a diagram showing a specific example of a form input image to which a meter reading value and an evidence image are input according to the first embodiment; 4 is a flowchart showing the flow of processing for generating a meter reading value input according to the first embodiment; FIG. 5 is a functional block diagram showing the functional configuration of an information processing apparatus 100a according to the second embodiment; FIG. 4 is a functional block diagram showing the functional configuration of a host server 200a according to the second embodiment; FIG. 11 is a functional block diagram showing the functional configuration of an information processing apparatus 100b according to the third embodiment; FIG. 10 is a diagram showing a specific example of a form image of a modified example of the embodiment; FIG. 11 is a diagram showing a specific example of a form image corresponding to a multimeter according to a modified example of the embodiment;

Hereinafter, an information processing system, an information processing method, and a computer program according to embodiments will be described with reference to the drawings.

(First embodiment)
FIG. 1 is a system configuration diagram showing a configuration example of an inspection support system 1 of the first embodiment. The inspection support system 1 includes an information processing device 100 , a host server 200 and a plurality of meters 300 . The information processing apparatus 100 and the upper server 200 are communicably connected to each other via a network 400 .

The information processing device 100 is configured using devices such as smart phones, tablet computers, smart glasses, smart heads, or personal computers. The information processing device 100 includes a processor, a memory, an auxiliary storage device, and the like connected via a bus. The information processing device 100 is operated by an inspector. The inspector may be, for example, a person who inspects the meter 300 or a person who collects the inspection results. The inspector may be any person who operates the information processing device 100 . The information processing apparatus 100 inputs the numerical value indicated by the meter 300 into the form data as the meter reading value. The form data is data into which numerical values (meter reading values) indicated by a plurality of meters 300 can be input.

The host server 200 is configured using a device such as a personal computer, server, workstation, or industrial computer. The host server 200 includes processors, memories, auxiliary storage devices, and the like connected via a bus. Host server 200 generates a recognition engine for enabling input of numerical value indicated by meter 300 in information processing apparatus 100 . The generated recognition engine may be stored in the information processing device 100 . The upper server 200 may be realized by a cloud computing system.

The meter 300 is, for example, a digital type having a 7-segment display that displays numbers from 0 to 9, and is configured to indicate at least one digit or more numerical value (meter reading value) according to the operating state of the equipment. Meter 300 is not limited to being digital with a seven segment display. For example, the meter 300 is, for example, a mechanical type using a number wheel on which numbers from 0 to 9 are described, and is configured to indicate a multi-digit numerical value (meter reading value) by rotating according to the usage amount. Alternatively, it may be configured as a needle type in which a scale and characters indicating units are displayed on a dial.

The network 400 may be, for example, a wired communication network such as the Internet, or a wireless communication network such as a wireless LAN (Local Area Network) or a mobile phone communication network. The network 400 may be any communication network as long as communication is possible between the information processing device 100 and the upper server 200 .

FIG. 2 is a functional block diagram showing the functional configuration of the information processing apparatus 100 of the first embodiment. By executing a form data generation program, the information processing apparatus 100 includes a communication unit 101, an input unit 102, a display unit 103, an imaging unit 104, a form definition storage unit 105, a learning result storage unit 106, a form data storage unit 107, and a control unit. It functions as a device having the unit 108 .

A communication unit 101 is a network interface. The communication unit 101 communicates with the host server 200 via the network 400 . The communication unit 101 may communicate using a communication method such as wireless LAN, wired LAN, Bluetooth (registered trademark), or LTE (Long Term Evolution) (registered trademark).

The input unit 102 is configured using an input device such as a touch panel, mouse, and keyboard. The input unit 102 may be an interface for connecting an input device to the information processing device 100 . In this case, the input unit 102 generates input data (for example, instruction information indicating an instruction to the information processing apparatus 100 ) from an input signal input from the input device, and inputs the input data to the information processing apparatus 100 .

A display unit 103 is an output device such as a CRT (Cathode Ray Tube) display, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like. The display unit 103 may be an interface for connecting an output device to the information processing device 100 . In this case, the display unit 103 generates a video signal from video data and outputs the video signal to a video output device connected thereto.

Although not shown in FIG. 2, the image capturing unit 104 includes, for example, an optical system, an image sensor, and an image processing board that processes an image signal output from the image sensor. is generated and output. The imaging unit 104 captures an identification image in which the identification information of the meter 300 itself or attached to the periphery of the meter 300 is embedded. The imaging unit 104 generates identification image data including an identification image. The identification image is, for example, a two-dimensional barcode. The image capturing unit 104 captures a numerical value (meter reading value) indicated by the meter 300 . The image pickup unit 104 photographs the appearance of the meter 300 as an evidence image serving as proof of meter reading. The imaging unit 104 generates meter image data including the appearance of the meter 300 . The imaging unit 104 displays a live view on the display unit 103 when activated. The live view is a function that enables an inspector to photograph an object captured by the imaging unit 104 while confirming the subject. The inspector can shoot while confirming the live view displayed on the display unit 103 . For example, when photographing the appearance of the meter 300 as an evidence image, the inspector checks the live view displayed on the display unit 103 while adjusting the meter 300 so that the meter reading value of the meter 300 can be determined. Shooting is performed by adjusting the distance from the information processing apparatus 100 .

The form definition storage unit 105 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form definition storage unit 105 stores form definition information. The form definition information is data that defines the input items for the form and the form of the form. Different form definition information may be defined according to the location or meter 300 to be inspected. Therefore, the form definition information may have different data.

The learning result storage unit 106 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 106 stores learning results. The learning result is used to obtain numerical values that can be input to the form data from the photographed image data indicating the numerical values of the meter 300 . The learning result is recorded when the form data generation program is installed in the information processing apparatus 100 . The learning result may be updated when the form data generation program is updated. The learning result may be recorded according to the type of meter 300. The type of meter 300 may be distinguished, for example, by digital type, mechanical type, needle type, etc., and the manufacturer, model, model year, etc. may be distinguished by

The form data storage unit 107 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form data storage unit 107 stores a form data table. The form data table holds the form data. Each value held in the form data is held based on the input items defined in the form definition information. A form image is generated based on the data held by the form data and the form definition information associated with the form data.

FIG. 4 is a diagram showing a specific example of the form data table of the first embodiment. The form data table has form data. The form data has values such as inspection object, date and time, form definition information, first meter, first meter image, second meter, second meter image, third meter and third meter image. An inspection target represents an object to be inspected by an inspector. The date and time represents the date and time when the form data was generated. The form definition information represents the form definition information associated with the form data. The first meter represents the numerical value of the first meter when the associated form definition information has the first meter as an input item. The first meter image represents, when the first meter is photographed, the storage location of the image data of the photographed first meter or the name of the image data. The second meter represents the numerical value of the second meter when the associated form definition information has the second meter as an input item. The second meter image represents, when the second meter is photographed, the storage location of the image data of the second meter photographed or the name of the image data. The 3rd meter indicates the numerical value of the 3rd meter when the associated form definition information has the 3rd meter as an input item. The 3rd meter image represents the storage location of the image data of the 3rd meter or the name of the image data when the 3rd meter is photographed.

In the example shown in FIG. 4, the form data at the top of the form data table has an inspection target value of "8F air conditioning", a date and time value of "2017.12.4 14:23:10", and a form definition information value of "def_1", first meter value is "120", first meter image value is "8f_air_meter1.jpg", second meter value is "110", second meter image value is "8f_air_meter2.jpg", The value of the third meter is "-" and the value of the third meter image is "-". Therefore, according to the form data at the top of the form data table, the inspected location is "8F air conditioning", the form data was generated at "2017.12.4 14:23:10", and the form definition information is "def_1". is used. It can be seen that the form definition information has the first meter, the first meter image, the second meter and the second meter image as input items, but does not have the third meter and the third meter image. Note that the form data table shown in FIG. 4 is merely a specific example. Therefore, the form data table may be configured in a manner different from that in FIG. For example, the form data table may be configured to hold the name of the inspector, may have an overall image holding a single photographed image of a plurality of meters, It may have information indicating the time when the image was captured.

Returning to FIG. 2, the description of the information processing apparatus 100 is continued. The control unit 108 controls operations of each unit of the information processing apparatus 100 . The controller 108 is executed by, for example, a device having a processor and memory. By executing the form generation program, the control unit 108 controls the form data generation unit 109, the form image generation unit 110, the image acquisition unit 111, the input item determination unit 112, the meter reading value generation unit 113, the form data input unit 114, the abnormality It functions as a determination unit 115 and an error processing unit 116 .

The form data generation unit 109 acquires the form definition information from the form definition storage unit 105 based on the instruction to select the form definition information. The form data generation unit 109 generates form data based on the acquired form definition information. The selection instruction may be received via the input unit 102, or may be embedded in the two-dimensional barcode.

The form image generation unit 110 generates a form image based on the form of the form defined by the acquired form definition information and the generated form data. The form image is displayed on the display unit 103 . The form image includes each value held by the form data and an instruction input image associated with the input unit 102 . The input unit 102 receives a predetermined instruction when the inspector touches the instruction input image displayed on the display unit 103 . The predetermined instruction may be activation of the imaging unit 104, generation of a form input image, or recording of form data. The predetermined instruction may be any instruction as long as it is an instruction specified in advance.

The image acquisition unit 111 activates the imaging unit 104 . The image acquisition unit 111 acquires the live view displayed on the display unit 103 from the imaging unit 104 . The image acquisition unit 111 outputs the live view to the input item determination unit 112 or the meter reading value generation unit 113 .

The input item determination unit 112 identifies the two-dimensional barcode from the live view. The input item determination unit 112 determines input items into which meter reading values are input, based on the identification information embedded in the identified two-dimensional barcode image. The identification information includes information necessary for the input item determination unit 112 to identify the input items included in the form data. The identification information includes, for example, information such as "first meter". The identification information may include, for example, information specifying the form definition information, or may include a meter reading value threshold. Note that the identification information is not limited to the two-dimensional barcode. For example, the identification information may be embedded in a bar code, embedded in an IC tag such as an RFID (Radio Frequency Identifier), or recorded in a storage device included in the meter.

The meter reading value generation unit 113 identifies an image showing the meter reading value from the live view. The meter reading value generation unit 113 generates a meter reading value based on the image indicating the specified meter reading value. The meter reading value generation unit 113 generates the meter reading value based on, for example, an image showing the meter reading value and the learning result recorded in the learning result storage unit 106 . The meter reading value generation unit 113 may generate the meter reading value using an existing image recognition technique such as OCR (Optical Character Recognition). The meter reading value generation unit 113 acquires the time when the meter reading value is generated as the meter reading time. The meter reading value generation unit 113 may be configured to read out the generated meter reading value from the speaker of the information processing device 100 by voice.

The form data input unit 114 inputs the generated meter reading value to the determined input item among the input items of the form data. When the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 is photographed, the form data input unit 114 inputs the file name of the photographed image data and the photographing time into the form data. The form data input unit 114 records the input form data in the form data storage unit 107 . The form data input unit 114 may transmit the form data to the host server 200 . The form data input unit 114 may record a PDF file generated based on the form image in the form data storage unit 107 or may be configured to transmit to the host server 200 .

The abnormality determination unit 115 determines whether or not the inputted input value satisfies a predetermined condition. The abnormality determination unit 115 instructs the error processing unit 116 to perform error processing when a predetermined condition is satisfied. Abnormality determination unit 115 does not instruct error processing to error processing unit 116 when a predetermined condition is not satisfied. The predetermined condition is, for example, the difference between the meter reading time, which is the time at which the image indicating the meter reading value was acquired, and the photographing time at which the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 was photographed. If the threshold of is exceeded, it may be determined that a predetermined condition is satisfied. The predetermined condition may be any condition as long as it is a condition specified in advance. The predetermined threshold may be a threshold embedded in the two-dimensional barcode, or may be a threshold included in the form definition information. The predetermined threshold value may be any value as long as it is a predetermined threshold value.

Upon receiving an error processing instruction from abnormality determination unit 115, error processing unit 116 executes error processing. The error process may be, for example, a process of prompting the inspector to re-enter the meter reading value in the input item. The error processing may be any processing as long as it is predetermined processing.

FIG. 3 is a functional block diagram showing the functional configuration of the host server 200 of the first embodiment. By executing the learning program, the host server 200 includes a communication unit 201, an input unit 202, a display unit 203, a user information storage unit 204, a teacher data storage unit 205, a learning result storage unit 206, a form data storage unit 207, and a control unit. 208.

A communication unit 201 is a network interface. The communication unit 201 communicates with the information processing device 100 via the network 400 . The communication unit 201 may communicate using a communication method such as wireless LAN, wired LAN, Bluetooth, or LTE, for example.

The input unit 202 is configured using an input device such as a touch panel, mouse, and keyboard. The input unit 202 may be an interface for connecting an input device to the host server 200 . In this case, the input unit 202 generates input data (for example, instruction information indicating an instruction to the host server 200 ) from an input signal input from the input device, and inputs the input data to the host server 200 .

A display unit 203 is an output device such as a CRT display, liquid crystal display, or organic EL display. The display unit 203 may be an interface for connecting an output device to the host server 200 . In this case, the display unit 203 generates a video signal from the video data and outputs the video signal to the video output device connected thereto.

The user information storage unit 204 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The user information storage unit 204 stores user information. User information is information about a person who uses meter 300 . User information includes, for example, a user identifier that identifies a user.

The teacher data storage unit 205 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The teacher data storage unit 205 stores teacher data. The training data is image data obtained by photographing the meter 300 used by the user. The training data may include image data captured from the front, image data captured from an oblique angle, and the like. The teacher data may be stored in association with the user identifier.

The learning result storage unit 206 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The learning result storage unit 206 stores learning results. A learning result is generated by the learning unit 209 based on the image data stored in the teacher data storage unit 205 . A learning result may be associated with a user identifier and generated for each user.

The form data storage unit 207 is configured using a storage device such as a magnetic hard disk device or a semiconductor storage device. The form data storage unit 207 stores a form data table. The form data table holds the form data. Form data transmitted from the information processing apparatus 100 is recorded in the form data storage unit 207 .

The control unit 208 controls the operation of each unit of the upper server 200 . The controller 208 is executed by a device including, for example, a processor and memory. Control unit 208 functions as learning unit 209 by executing a learning program.

A learning unit 209 generates a learning result based on the teacher data. The learning unit 209 generates a learning result from teacher data by machine learning such as deep learning. A learning result may be generated for each type of meter 300 , or a single learning result may be generated regardless of the type of meter 300 .

FIG. 5 is a diagram showing a specific example of a form image according to the first embodiment. The form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. The form image displayed on the display unit 103 includes an input item image 131 , an input item image 132 , an input item image 133 and an instruction input image 134 .

The input item image 131, the input item image 132, and the input item image 133 are images that display meter reading values held in the form data. The input item image 131, the input item image 132, and the input item image 133 are associated with the input items of the form data. According to FIG. 5, the input item image 131 is associated with the input item "first meter", the input item image 132 is associated with the input item "second meter", and the input item image 133 is associated with the input item "third meter". . When the meter reading value is input to the input item of the form data, the input meter value is displayed in the input item image associated with the input item. For example, when “100” is input to the input item “first meter” of the form data, “100” is displayed in the input item image 131 .

The instruction input image 134 is an image that receives an instruction to generate a form input image. The instruction input image 134 is touch-operated so that the input unit 102 receives an instruction to generate a form input image. A touch operation is an act of the user touching or pressing an instruction input image 134 displayed on the display unit 103 . When the input unit 102 receives a form input image generation instruction, the form image generation unit 110 generates a form input image. The display unit 103 displays the generated form input image. The display unit 103 may display the form input image so as to be superimposed on the form image, or the display unit 103 may discard the form image and display the form input image.

FIG. 6 is a diagram showing a specific example of a form input image according to the first embodiment. The form image generation unit 110 generates a form input image based on the form definition information and the form data. The display unit 103 displays the generated form input image. The form input images displayed on the display unit 103 include an input area image 141 , an input area image 142 , an input area image 143 , an instruction input image 144 and an instruction input image 145 .

An input area image 141, an input area image 142, and an input area image 143 are areas that hold values to be input to form data. The input area image 141, the input area image 142, and the input area image 143 are associated with the input items of the form data. According to FIG. 6, the input area image 141 is associated with the input item "first meter", the input area image 142 is associated with the input item "second meter", and the input area image 143 is associated with the input item "third meter". .

The instruction input image 144 is an image for accepting an instruction to activate the imaging unit 104 . When the instruction input image 144 is touch-operated, the input unit 102 receives an instruction to activate the imaging unit 104 . When the input unit 102 receives a generation instruction for the imaging unit 104 , the image acquisition unit 111 activates the imaging unit 104 . The display unit 103 displays a live view when the imaging unit 104 is activated.

The instruction input image 145 is an image for receiving an input instruction to form data. When the instruction input image 145 is touch-operated, the input unit 102 accepts an input instruction to the form data. When the input unit 102 receives an input instruction to the form data, the form data input unit 114 inputs a value to the form data. For example, when "100" is input as the meter reading value in the input area image 141, "100" is input to the input item "first meter" of the form data.

FIG. 7 is a diagram showing a specific example of reading a two-dimensional barcode according to the first embodiment. FIG. 7 includes meter 300 and display unit 103 of information processing apparatus 100 . Meter 300 includes area 301 and area 302 . Area 301 contains a two-dimensional barcode applied to meter 300 . Area 302 contains the meter reading represented by meter 300 . Display unit 103 includes area 137 . A region 137 includes a two-dimensional barcode image of the region 301 of the meter 300 captured by the imaging unit 104 .

The input item determination unit 112 identifies the two-dimensional barcode displayed in the live view. The input item determination unit 112 determines input items into which meter reading values are input, based on the identification information embedded in the identified two-dimensional barcode image. A balloon 137 a represents identification information embedded in the two-dimensional barcode contained in the area 137 . According to the balloon 137a, it can be seen that "first meter" is embedded as identification information. The input item determination unit 112 determines the input item to be the "first meter".

FIG. 8 is a diagram showing a specific example of reading meter values according to the first embodiment. FIG. 8 includes meter 300 and display unit 103 of information processing apparatus 100 . Meter 300 includes area 301 and area 302 . Regions 301 and 302 are the same as in FIG. Display unit 103 includes area 138 . Area 138 includes an image showing the meter reading of area 302 of meter 300 captured by imaging unit 104 .

When the input item determination unit 112 determines the input items, reading of meter values is started. The meter reading value generation unit 113 identifies an image showing the meter reading value displayed in the live view. The meter reading value generation unit 113 generates a meter reading value based on the image indicating the specified meter reading value. The meter reading value generation unit 113 acquires the generated time when generating the meter reading value.

FIG. 9 is a diagram showing a specific example of a form input image in which meter reading values are input according to the first embodiment. When the meter reading value is generated, the form image generating unit 110 displays the meter reading value on the input area image. The meter reading value is displayed in the input area image associated with the input item. The form input images displayed on the display unit 103 include an input area image 141 a , an input area image 142 a , an input area image 143 a , an instruction input image 144 and an instruction input image 145 . The instruction input image 144 and the instruction input image 145 are the same as in FIG.

The input area image 141a, the input area image 142a, and the input area image 143a are areas that hold values to be input to the form data. According to FIG. 9, the input area image 141a holds "12332", the input area image 142a holds "678", and the input area image 143a holds "90000". According to FIG. 9, the input area image 141 is associated with the input item "first meter", the input area image 142 is associated with the input item "second meter", and the input area image 143 is associated with the input item "third meter". . Therefore, when the instruction input image 145 is touch-operated, the form data input unit 114 inputs the values held in the input area image 141a, the input area image 142a, and the input area image 143a into the form data.

FIG. 10 is a diagram showing a specific example of a form input image in which meter reading values are input according to the first embodiment. When the instruction input image 145 is touch-operated, the form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. In FIG. 10, since the meter reading value is input as the form data, the meter reading value is displayed in the input item image. The form image displayed on the display unit 103 includes an input item image 131 a , an input item image 132 a , an input item image 133 a and an instruction input image 134 . The instruction input image 134 is the same as in FIG.

The input item image 131a, the input item image 132a, and the input item image 133a are images that display meter reading values input to the form data. The input item image 131a, the input item image 132a, and the input item image 133a are associated with the input items of the form data. According to FIG. 10, the input item image 131a is associated with "1st meter" as an input item, "12332" is input as a meter reading value for "1st meter", and the input item image 132a is associated with "12332" as an input item. The "second meter" is associated with "678" as the meter reading value, and the input item image 133a is associated with the "third meter" as the input item. ” indicates that “90000” is input as the meter reading value.

FIG. 11 is a diagram showing a specific example of a form input image to which meter reading values and evidence images are input according to the first embodiment. As shown in FIG. 11, the form input image may be configured to display the evidence image together. Evidence images are captured after meter readings are generated. Specifically, when the input item of the form definition information includes the image of the meter, the form data input unit 114 displays a screen for confirming whether or not to photograph the appearance of the meter 300 after the meter reading value is generated. 103 to display. Imaging unit 104 captures an image of meter 300 upon receiving an instruction to capture an image of meter 300 via input unit 102 . The form data input unit 114 inputs the photographed image data and the photographing time into the form data.

According to FIG. 11, the form images displayed on the display unit 103 are an input item image 131a, an input item image 132a, an input item image 133a, an input item image 131b, an input item image 132b, an input item image 133b, and an instruction input image 134. including. The input item image 131a, the input item image 132a, the input item image 133a, and the instruction input image 134 are the same as in FIG.

The input item image 131b, the input item image 132b, and the input item image 133b are images representing evidence images held in the form data. The input item image 131b, the input item image 132b, and the input item image 133b are associated with the input items of the form data. According to FIG. 11, the input item image 131b is associated with the "first meter image" as the input item, the input item image 132b is associated with the "second meter image" as the input item, and the input item image 133b is associated with the , is associated with "3rd meter image" as an input item.

FIG. 12 is a flowchart showing the flow of processing for generating a meter reading value input according to the first embodiment. The form data generation unit 109 generates form data based on the acquired form definition information (step S101). The form image generating unit 110 generates a form image based on the form of the form defined by the acquired form definition information and the form data (step S102). The input unit 102 receives an instruction to generate a form input image by touching the instruction input image 134 included in the form image (step S103). The form image generation unit 110 generates a form input image (step S104). The input unit 102 accepts an instruction to activate the imaging unit 104 by touching the instruction input image 144 included in the form input image (step S105). The image acquisition unit 111 activates the imaging unit 104 (step S106). A live view is displayed on the display unit 103 .

The input item determination unit 112 identifies the two-dimensional barcode displayed in the live view (step S107). The input item determination unit 112 determines input items into which meter reading values are input, based on the identification information embedded in the identified two-dimensional barcode image (step S108). The meter reading value generation unit 113 identifies an image showing the meter reading value displayed in the live view (step S109). The meter reading value generation unit 113 generates a meter reading value based on the image indicating the specified meter reading value (step S110). The meter-reading value generation part 113 will acquire meter-reading time, if a meter-reading value is produced|generated. The form image generation unit 110 displays the generated meter reading value in the input area image (step S111).

Form data input unit 114 receives an instruction as to whether or not to photograph the appearance of meter 300 (step S112). Form data input unit 114 determines whether or not an instruction to image meter 300 has been received (step S113). If an instruction to capture an image of meter 300 has been received (step S113: YES), imaging unit 104 captures the appearance of meter 300 and acquires image data according to the user's operation (step S114). The imaging unit 104 acquires the shooting time. If the instruction to photograph the meter 300 has not been received (step S113: NO), the process proceeds to step S115.

The form image generation unit 110 determines whether values have been input to all input area images (step S115). If there is an input area image for which no value has been input (step S115: NO), the process proceeds to step S107. If values have been input to all input area images (step S115: YES), the form data input unit 114 determines whether or not an input instruction to the form data has been received (step S116). Specifically, it is determined whether or not the instruction input image 145 has been touch-operated in the form input image. If the input instruction to the form data has not been received (step S116: NO), the process proceeds to step S116. When the input instruction to the form data is accepted (step S116: YES), the form data input unit 114 inputs a value to the form data. The value may be, for example, a meter reading value or a meter image (step S117). The form image generation unit 110 generates a form image based on the form definition information and the form data (step S118). Since the meter reading value is entered in the form data, the meter reading value is displayed in the input item image.

The abnormality determination unit 115 determines whether or not the difference between the meter reading time and the photographing time is smaller than a predetermined threshold (step S119). The predetermined threshold value may be included in the two-dimensional barcode or given in advance. If it is smaller than the predetermined threshold (step S119: YES), the form data input unit 114 records the form data in the form data storage unit 107 (step S120). If it is not smaller than the predetermined threshold (step S119: NO), the error processing unit 116 performs error processing (step S121). In this flowchart, error processing is processing without recording form data in the form data storage unit 107 .

In the information processing apparatus 100 configured as described above, the input item determining unit 112 reads the two-dimensional barcode to acquire the identification information embedded in the two-dimensional barcode. The identification information is associated with input items into which meter reading values are input. The meter reading value generation unit 113 generates the meter reading value of the meter 300 . The form data input unit 114 inputs the meter reading value into the input item associated with the identification information, so that the meter reading value is not entered into the wrong input item, and the result of the inspection work can be input more efficiently. can do

(Second embodiment)
Next, the inspection support system 1 of the second embodiment will be described. The inspection support system 1 in the second embodiment differs from the first embodiment in that it includes an information processing device 100a instead of the information processing device 100 and a higher server 200a instead of the higher server 200. Other configurations are the same. Differences from the first embodiment will be described below.

FIG. 13 is a functional block diagram showing the functional configuration of the information processing device 100a of the second embodiment. The information processing apparatus 100a differs from the information processing apparatus 100 in that it includes a control unit 108a instead of the control unit 108 and does not include the learning result storage unit 106, but otherwise has the same configuration.

The control unit 108a controls the operation of each unit of the information processing device 100a. The controller 108a is executed by, for example, a device having a processor and memory. By executing the form generation program, the control unit 108a controls the form data generation unit 109, the form image generation unit 110a, the image acquisition unit 111a, the input item determination unit 112, the form data input unit 114, the abnormality determination unit 115, and error processing. It functions as part 116 .

The form image generation unit 110a generates a form image based on the form of the form defined by the form definition information and the form data. The form image is displayed on the display unit 103 . The form image includes each value held by the form data and an instruction input image associated with the input unit 102 . The input unit 102 receives a predetermined instruction when the inspector touches the instruction input image displayed on the display unit 103 . The predetermined instruction may be activation of the imaging unit 104, generation of a form input image, or recording of form data. The predetermined instruction may be any instruction as long as it is an instruction specified in advance. When the meter reading value is received from the host server 200, the form image generation unit 110a displays the meter reading value in the input area image.

The image acquisition unit 111 a activates the imaging unit 104 . The image acquisition unit 111 a acquires the live view displayed on the display unit 103 from the imaging unit 104 . Image acquisition section 111 a outputs the live view to input item determination section 112 . The image acquisition unit 111a transmits the live view to the upper server 200a.

FIG. 14 is a functional block diagram showing the functional configuration of the host server 200a of the second embodiment. The host server 200a is different from the host server 200 in that it has a control unit 208a instead of the control unit 208, but otherwise has the same configuration.

The control unit 208a controls the operation of each unit of the upper server 200a. The controller 208a is executed by, for example, a device having a processor and memory. Control part 208a functions as learning part 209 and meter-reading value generation part 210 by running a learning program.

The meter reading value generation unit 210 identifies an image showing the meter reading value from the live view. The meter reading value generation unit 210 generates a meter reading value based on the image indicating the specified meter reading value. The meter reading value generation unit 210 generates the meter reading value based on, for example, an image showing the meter reading value and the learning result recorded in the learning result storage unit 206 . The meter reading value generation unit 210 acquires the time when the meter reading value is generated as the meter reading time. The meter reading value generation unit 210 transmits the meter reading value and the meter reading time to the information processing device 100a.

In the information processing apparatus 100a configured as described above, the input item determination unit 112 reads the two-dimensional barcode to acquire the identification information embedded in the two-dimensional barcode. The identification information is associated with input items into which meter reading values are input. The image acquisition unit 111a transmits the live view to the host server 200a. The meter reading value generation unit 210 of the host server 200a identifies an image showing the meter reading value from the live view and generates the meter reading value. The meter reading value generation unit 210 transmits the meter reading value to the information processing device 100a. The form data input unit 114 inputs the meter reading value into the input item associated with the identification information, so that the meter reading value is not entered into the wrong input item, and the result of the inspection work can be input more efficiently. can do. Further, by generating the meter reading value in the host server 200a, it is not necessary to generate the meter reading value in the information processing device 100a. Therefore, even if the information processing apparatus 100a does not have a function of generating a meter reading value from an image showing the meter reading value, the meter reading value can be input to the form data. Moreover, the learning result used for generating the meter reading value is generated by the host server 200a. Therefore, the information processing apparatus 100a can always acquire the meter reading value generated using the latest learning result, and the recognition accuracy can be improved.

(Third Embodiment)
Next, the inspection support system 1 of the third embodiment will be described. The inspection support system 1 according to the third embodiment differs from the first embodiment in that it includes an information processing device 100b in place of the information processing device 100 and does not include the host server 200, but the rest of the configuration is the same. is. Differences from the first embodiment will be described below.

FIG. 15 is a functional block diagram showing the functional configuration of the information processing device 100b of the third embodiment. The information processing device 100b includes a form input application. The form input application is a program for the information processing apparatus 100b to input the numerical value indicated by the meter 300 into the form data as the meter reading value. The information processing apparatus 100b differs from the information processing apparatus 100 in that it includes a control unit 108b instead of the control unit 108, but otherwise has the same configuration.

The control unit 108b controls the operation of each unit of the information processing device 100b. The control unit 108b is executed by a device including a processor and memory, for example. By executing the form input application, the control unit 108b controls the form data generation unit 109, the form image generation unit 110a, the image acquisition unit 111a, the input item determination unit 112, the form data input unit 114b, the abnormality determination unit 115, and the error processing. It functions as a unit 116 and an activation unit 117 .

The activation unit 117 activates the form input application. When the form input application is started, the activation unit 117 receives an instruction to select form definition information. The activation unit 117 receives an instruction to select form definition information via the input unit 102 . The activation unit 117 outputs an instruction to select the received form definition information to the form data generation unit 109 .

The form data input unit 114b inputs the generated meter reading value to the determined input item among the input items of the form data. When the appearance of the meter 300 or the numerical value (meter reading value) indicated by the meter 300 is photographed, the form data input unit 114b inputs the file name of the photographed image data and the photographing time into the form data. The form data input unit 114 records the input form data in the form data storage unit 107 . The form data input unit 114 b may record the PDF file generated based on the form image in the form data storage unit 107 .

In each of the above embodiments, the form data generation unit 109, the form image generation unit 110, the image acquisition unit 111, the input item determination unit 112, the meter reading value generation unit 113, the form data input unit 114, the abnormality determination unit 115, and the error processing unit 116 is a software functional unit, it may be a hardware functional unit such as an LSI.

According to at least one embodiment described above, by having the form data generation unit 109, the input item determination unit 112, the meter reading value generation unit 113, and the form data input unit 114, the results of inspection work can be input more efficiently. can do.

(Modification of embodiment)
FIG. 16 is a diagram showing a specific example of a form image according to a modification of the embodiment. The form image generation unit 110 generates a form image based on the form definition information and the form data. The display unit 103 displays the generated form image. The form image displayed on the display unit 103 includes an input item image 131, an input item image 132, an input item image 133, an instruction input image 134, a capture instruction image 131c, a capture instruction image 132c, and a capture instruction image 133c. . Here, since the input item image 131, the input item image 132, the input item image 133, and the instruction input image 134 are the same as those in the first embodiment, description thereof will be omitted.

The capture instruction image 131c, the capture instruction image 132c, and the capture instruction image 133c are images for receiving a meter reading value acquisition instruction. The input unit 102 receives an instruction to acquire a meter reading value by touching any one of the take-in instruction image 131c, the take-in instruction image 132c, and the take-in instruction image 133c. When the input unit 102 receives an instruction to acquire a meter reading value, the image acquisition unit 111 activates the imaging unit 104 . A live view is displayed on the display unit 103 .

The meter reading value generation unit 113 identifies an image showing the meter reading value displayed in the live view. The meter reading value generation unit 113 generates the meter reading value based on the image indicating the specified meter reading value. When the meter reading value generation unit 113 generates the meter reading value, it acquires the generated time. The form image generation unit 110 displays the generated meter reading value in the input item image associated with the touch-operated capture instruction image. In the case of FIG. 15, the input item image 131 is associated with the import instruction image 131c, the input item image 132 is associated with the import instruction image 132c, and the input item image 133 is associated with the import instruction image 133c.

In the information processing apparatus 100 configured in this manner, the meter reading value is input based on the captured image indicating the meter reading value. Therefore, it is possible to reduce input errors of meter reading values for the form data.

The form image generator 110 may be configured not to generate the form image and the form input image. In this case, when the form definition information is selected, the instruction input image 144 is displayed on the display unit 103 . When the instruction input image 144 is touch-operated, the imaging unit 104 is activated. A live view is displayed on the display unit 103 . The user images the two-dimensional bar code of the meter 300 and the meter reading value of the meter 300 with the imaging unit 104 in this order. When the two-dimensional barcode is imaged, the input item determination unit 112 acquires the identification information embedded in the two-dimensional barcode and determines input items. The meter reading value generation unit 113 generates the meter reading value when the meter reading value of the meter 300 is captured. When the meter reading value is generated, the form data input unit 114 inputs the meter reading value to the determined input item. In the information processing apparatus 100 configured in this way, the user can continuously input meter reading values without checking the form image. Therefore, the meter reading value can be input to the form data more quickly.

The abnormality determination unit 115 may be configured to issue an alarm when there is an input item for which no value has been input in the form data. By configuring in this way, it is possible to prevent omission of input of meter reading values.

The error processing unit 116 may perform predetermined error processing when the input item determination unit 112 cannot determine the input items of the form data based on the identification information. Predetermined error processing may display, for example, a warning message such as "This is a meter that cannot be entered in a form", or display identification information embedded in a two-dimensional bar code such as "This is a second meter". You may

The form data input unit 114 may be configured to record in the form data storage unit 107 when all values are input to the input items of the form data. In this case, the form data can be recorded in the form data storage unit 107 earlier than the inspector inputs an instruction to record the form data. Therefore, the inspector can perform the inspection work more efficiently.

FIG. 17 is a diagram showing a specific example of a form image compatible with a multimeter according to the modified example of the embodiment. FIG. 17 includes the multimeter 300a and the display unit 103 of the information processing apparatus 100. FIG. The multimeter 300a is a meter that displays meter reading values of different items when a button or switch attached to the meter is pressed. Note that the multimeter 300a may be configured to display meter reading values of different items when an instruction is received from the outside through communication means such as infrared communication.

Multimeter 300 a includes area 301 , area 302 a and button 303 . Area 301 contains a two-dimensional barcode applied to meter 300 . Field 302a contains the meter reading represented by multimeter 300a. The button 303 is a switching button for changing the meter reading value indicated by the multimeter 300a to the meter reading value of another item. When the button 303 is pressed, the meter reading value included in the area 302a is changed to a meter reading value representing another item. For example, an example in which the multimeter 300a can display meter reading values of three items of current, voltage, and power will be described. If the button 303 is pressed when the meter reading value of the current is displayed in the area 302a, the meter reading value of the voltage is displayed. Next, when the button 303 is pressed, the power meter value is displayed. Next, when the button 303 is pressed, the current meter value is displayed. In this way, when the button 303 is pressed, meter values are displayed in the order of current→voltage→power. Note that items that can be displayed by the multimeter 300a are not limited to meter readings of current, voltage, and power. For example, the usage amount of water or gas may be displayed.

The display unit 103 includes an area 137 and an area 142b. A region 137 includes a two-dimensional barcode image of the region 301 of the meter 300 captured by the imaging unit 104 . The input item determination unit 112 identifies the two-dimensional barcode displayed in the live view. The input item determination unit 112 determines input items into which meter reading values are input, based on the identification information embedded in the identified two-dimensional barcode image. A balloon 137b represents the identification information embedded in the two-dimensional barcode contained in the area 137. FIG. According to the balloon 137b, it can be seen that "second meter_+3" is embedded as identification information. The input item determination unit 112 determines the input item to be "second meter". Also, "+3" is embedded as identification information. Therefore, the input item determining unit 112 determines that three meter values are to be input for the input item "second meter".

Area 142b contains a plurality of input area images for the second meter. The three input area images included in the area 142b are associated with "second data" as input items. Therefore, the inspector presses the button 303 to switch the meter reading values displayed in the area 302a, and images the area 302a. The meter reading value generation unit 113 generates a meter reading value from the captured image indicating the meter reading value. The form data input unit 114 sequentially inputs the meter reading values to the input area images associated with the second data.

In the information processing apparatus 100 configured in this way, even if the object to be inspected is the multimeter 300a, it is possible to input the result of the inspection work efficiently.

While several embodiments of the invention have been described, these embodiments have been presented by way of example and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, as well as the scope of the invention described in the claims and equivalents thereof.

DESCRIPTION OF SYMBOLS 1... Inspection support system 100... Information processing apparatus 101... Communication part 102... Input part 103... Display part 104... Imaging part 105... Form definition storage part 106... Learning result storage part 107... Form data Storage unit 108 Control unit 109 Form data generation unit 110 Form image generation unit 111 Image acquisition unit 112 Input item determination unit 113 Meter reading value generation unit 114 Form data input unit 115 Abnormality determination unit 116 Error processing unit 200 Host server 201 Communication unit 202 Input unit 203 Display unit 204 User information storage unit 205 Teacher data storage unit 206 Learning result storage Unit 207 Form data storage unit 208 Control unit 209 Learning unit 300 Meter 400 Network 100a Information processing device 108a Control unit 110a Form image generation unit 111a Image acquisition unit , 200a... Host server, 208a... Control unit, 210... Meter reading value generation unit, 100b... Information processing device, 108b... Control unit, 114b... Form data input unit, 117... Start unit

TECHNICAL FIELD Embodiments of the present invention relate to computer devices, control methods, and computer programs .

The problem to be solved by the present invention is to provide a computer device, a control method, and a computer program that can more efficiently enter the results of inspection work into a form.

A computer device according to an embodiment has a meter reading value generation unit and a storage unit. The meter reading value generation unit generates meter reading value data of the meter from the image data of the meter imaged by a meter reader. The storage unit stores the meter reading value data in an item in the storage unit corresponding to the code included in the image data of the meter, and stores the image data of the meter, the file name of the image data, and the image data. At least one of the name and the storage location of the image data, and at least one of the year, month, day, and time when the image data was acquired are stored.
A computer device according to an embodiment has a code identification unit, a meter reading value generation unit, and a storage unit. The code identification unit identifies a code related to the meter from image data of the meter captured by a meter reader. The meter reading value generation unit generates meter reading value data of the meter from the image data of the meter used for identifying the code. The storage unit stores the meter reading value data in items in the storage unit determined by the identification information related to the code, and stores the image data of the meter, the file name of the image data, the name of the image data, and the At least one of the image data storage locations and at least one of the year, month, day, and time when the image data was obtained are stored.

A computer device, a control method, and a computer program according to embodiments will be described below with reference to the drawings.

Claims (3)

An information processing system that includes a host server and an information processing device that can communicate with the host server, and inputs a meter reading value indicating the operating state of equipment to form data,
an image acquisition unit of the information processing device that acquires image data related to the meter and transmits the acquired image data to the host server;
a learning unit of the host server that generates, for each type of the meter, a learning result used to acquire the meter reading value of the meter from the image data related to the meter reading value area of the meter, based on the teacher data;
a learning result storage unit of the host server that stores the learning result;
A meter reading value generation unit of the host server that generates the meter reading value of the meter from the image data related to the meter reading value area of the meter acquired by the image acquisition unit, based on the learning result stored in the learning result storage unit. When,
a form data storage unit that stores the form data;
an input unit for inputting the meter reading value generated by the meter reading value generating unit into an input item of the form data determined by the two-dimensional code related to the meter acquired as image data by the image acquiring unit;
An information processing system comprising An information processing method in which an information processing system having a host server and an information processing device capable of communicating with the host server inputs a meter reading value indicating an operating state of equipment to form data,
an image acquisition step in which the information processing device acquires image data relating to the meter and transmits the acquired image data to the host server;
a learning step in which the host server generates a learning result for each type of the meter, which is used for obtaining the meter reading value of the meter from the image data related to the meter reading value area of the meter, based on the teacher data;
a learning result storage step in which the host server stores the learning result in a learning result storage unit;
The host server generates the meter reading value from the image data relating to the meter reading value area of the meter obtained in the image obtaining step, based on the learning result stored in the learning result storage unit. a step;
an input step of inputting the meter reading value generated in the meter reading value generating step into an input item of the form data determined by the two-dimensional code related to the meter obtained as image data in the image obtaining step;
An information processing method comprising: A computer program for causing a computer to function as the information processing system according to claim 1 .

Images