JP2023054015A - Meter reading work support system and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a meter reading work support system which can perform gradual and effective work support at a meter reading work site with camera photographing.

SOLUTION: A meter reading work support system comprises: a reading device which reads, when a reading object area A2 of a meter device is photographed in a predetermined recognition state by a communication terminal 10 with a camera, an indicated value Vm of the meter device on the basis of an image P12 of the reading object area acquired from the communication terminal; and a support information presentation device which transmits work support information for a reading work of the indicated value to the communication terminal to be output therefrom. The support information presentation device changes presentation conditions of the work support information according to recognition states at a plurality of stages of a photographed image until an image of the reading object area is readably recognized on the basis of the photographed image after starting photographing by the camera, transmits the work support information in an information form of at least one of the image and sound to the communication terminal, and changes output of the work support information according to the recognition states at the plurality of stages.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2023,JPO&INPIT

Description

TECHNICAL FIELD The present invention relates to a meter reading work device and method and a meter reading work support system, and more particularly to a meter reading work device and method and a meter reading work support system for reading indicated values from captured images of various meters with a mobile camera.

In various factories, plants, etc., the work of reading the indicated value (meter value) of the meter device was conventionally done exclusively by workers making rounds and visually confirming, but recently, images taken by the camera of the meter device There have been proposed a reading work device and method for improving the efficiency of equipment inspection and monitoring work by reading the indicated value from the field work, and a support system for such field work.

In such a meter reading work device, method, and work support system, for example, an image display section for displaying a captured image is provided with guide information for acquiring a target area for recognition processing from an input image together with the captured image. A processing target area extracting/determining unit for extracting an image processing target area based on the guide information is provided while displaying, and an operation guide is presented by image display so that an image of the reading target area can be reliably captured. There is known one that does this (see, for example, Patent Literature 1).

In addition, by determining the degree of reading based on whether or not the position detection symbol of the two-dimensional barcode can be detected, and reflecting the degree of reading in the indicator, the user is notified of whether or not information can be read from the two-dimensional barcode. A device is known in which the position of an imaged object recognition device can be adjusted while checking an indicator (see Patent Document 2, for example).

Japanese Patent Application Laid-Open No. 2004-213215 JP-A-2005-122609

In the above-described conventional meter reading work device and method and meter reading work support system, guide display for facilitating identification of the shape of the recognition processing target area when photographing with a camera and automatic recognition processing at a predetermined image reading degree Therefore, depending on the work environment, it may not be easy to visually recognize the guide display image, or it may be difficult to identify the meter device to be read or to understand the work order. In addition, it has not been possible to provide effective work support for checking meter reading work, reporting after work, and other work.

In addition, even if it is possible to identify the meter to be read according to the predetermined work procedure and schedule, it is possible to predict danger when shooting with the camera, prevent injury accidents, prevent work mistakes and work forgetting, and work details. Such as the function to generate deliverables such as checklists and work reports, etc., it accurately presents effective work support information according to the quality of the work environment and the level of work skills of on-site workers, and shoots with a camera. have not been able to effectively support the field work of meter readings involving

On the other hand, at actual work sites, there are an increasing number of cases where workers with low work skills and who are unfamiliar with the language and terminology used at work sites are doing work, causing work mistakes and forgetting work, and There is concern about the occurrence of injury accidents. In addition, the social impact of accidents is significant, and in addition to improving work quality as preventive measures and measures to prevent recurrence, confirmation by two people, work checklists, and more detailed work reports are required at a higher level. is coming. Therefore, it is desirable to provide a meter reading work method and its support system that can sufficiently meet such demands.

The present invention has been made to solve the above-mentioned conventional problems, and provides step-by-step and effective work support at the work site for meter reading accompanied by camera photography, checking of the work, and deliverables after the work. It is an object of the present invention to provide a meter reading work support system and method that can effectively support the generation of

1) In the meter reading work support system according to the present invention, when a communication terminal equipped with a camera captures an image of a reading target area of a meter device in a predetermined recognition state, based on the image of the reading target area obtained from the communication terminal, A meter reading work support system comprising: a reading device for reading the indicated value of the meter device by using the meter device; and a support information presentation device for transmitting work support information for reading the indicated value to the communication terminal and outputting it wherein the support information presentation device is adapted to recognize a photographed image in a plurality of stages from the start of photographing by the camera until the image of the reading target area is recognized to be readable based on the photographed image. , changing the presentation conditions of the work support information, transmitting the work support information to the communication terminal in at least one of an image and a voice information form, and outputting the work support information according to the recognition state of the plurality of stages; It is characterized by changing

With this configuration, it is possible to present work support information according to the recognition status of the image captured by the camera until the indicated value of the meter device is recognized to be readable. The position and posture of the camera can be guided so that an image with a good recognition rate can be acquired, and the practicality of meter reading work and the reading accuracy of meter indicated values can be improved.

2) A meter reading work support method according to the present invention is a meter reading work support method for supporting the work of reading the indicated value of the meter device based on the image of the reading target area of the meter device by presenting work support information. Then, a movable camera for photographing the meter device, a display for displaying the photographed image of the camera, and a speaker for outputting sound are prepared, and the photographed image is reproduced from the start of photographing by the camera. The conditions for presenting the work support information are variably set in accordance with a plurality of stages of recognition states until the indicated value of the meter device is recognized to be readable. The information is output from either one or both of the display and the speaker, and the output of the work support information is changed according to the recognition state.

With this configuration, it is possible to present work support information according to the recognition status of the image captured by the camera until the indicated value of the meter device is recognized to be readable. The position and posture of the camera can be guided so that an image with a good recognition rate can be acquired, and the practicality of meter reading work and the reading accuracy of meter indicated values can be improved.

According to the present invention, it is possible to provide stepwise and effective work support at a work site for meter reading accompanied by camera photography, checking of the work, and generation of post-work deliverables. Systems and methods can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a schematic configuration diagram of a meter reading work device of one embodiment for carrying out a meter reading work method of first to sixth embodiments according to the present invention; FIG. 2 is a front view of a worker-side information terminal when the meter-reading work method of the first embodiment is carried out in the meter-reading work device of one embodiment of the present invention, showing work support information on the display of the worker-side information terminal; It shows a screen display state in which three possible attention areas are set. FIG. 10 is a front view of a worker-side information terminal when the meter-reading work method of the second embodiment is carried out in the meter-reading work device of one embodiment of the present invention, showing work support information on the display of the worker-side information terminal; A screen display state is shown in which a plurality of marker images and one attention area surrounded by them are set. FIG. 11 is a front view of an operator-side information terminal when the meter-reading work method of the third embodiment is carried out in the meter-reading work device of one embodiment of the present invention, and (a) is read on the display of the operator-side information terminal; (b) shows the state in which the image captured by the camera of the target area is displayed, and (b) the state in which the recognition state of the meter indication value in the reading target area is displayed as a binary image of multiple digits. there is FIG. 11 is a front view of a worker-side information terminal when the meter-reading work method of the fourth embodiment is carried out in the meter-reading work device of one embodiment of the present invention, showing work support information on the display of the worker-side information terminal; In addition to displaying a plurality of marker images, a region of interest, which is a photographed image display region surrounded by the centers of gravity of the plurality of marker images, and a display region of recognition stages and results of recognition processing arranged close to the one region of interest. 4 shows a display state of a screen displaying another attention area. FIG. 10 is a front view of a worker-side information terminal when the meter-reading work method of the fifth embodiment is carried out in the meter-reading work device of one embodiment of the present invention, and the displays of the worker-side information terminal are spaced apart from each other in parallel. The display state of the screen displaying the upper and lower recognition state display areas and the captured image display area positioned between the upper and lower recognition state display areas is shown. FIG. 10 is a front view of a worker-side information terminal in a case where the meter-reading work method of the sixth embodiment is carried out in the meter-reading work device of one embodiment of the present invention, showing work support information inclined on the display of the worker-side information terminal; and a tilted display state on the right side in which a plurality of marker images and one attention area surrounded by them are tilted clockwise in the figure, and correction on the left side in the figure in which the inclination of the attention area is corrected and a completed display state. It is a chart showing an example of reading setting information in the meter reading work device of one embodiment of the present invention, (a) is an example of reading setting information for recognition guidance of the color marker for tilt correction, (b) is a low recognition state Examples of read setting information for determination conditions for being are shown. FIG. 2 is an explanatory diagram of a case where the meter reading work device of one embodiment of the present invention is used and the meter reading work method of the first to sixth embodiments of the present invention is sequentially performed on a plurality of meter devices; An example of a meter device to be read in each embodiment of the present invention is shown. FIG. 10 shows an example of a screen display of a meter reading control device that performs preliminary setting processing in a meter reading work method according to a second embodiment of the present invention; FIG. FIG. 10 shows still another screen display example of the meter reading control device that performs the preliminary setting process in the meter reading work method according to the second embodiment of the present invention; FIG. FIG. 10 is an explanatory diagram of setting processing that is performed in advance in the meter reading work method according to the second embodiment of the present invention;

EMBODIMENT OF THE INVENTION Hereinafter, the form for implementing this invention is demonstrated, referring drawings.

1 to 8 show a meter reading work device according to one embodiment capable of implementing the meter reading work methods of the first to sixth embodiments of the present invention.

First, the configuration will be explained.

A meter reading work device 1 of the present embodiment shown in FIG. 1 is configured as a device capable of a field work support service for supporting meter reading work.

This meter reading work device 1 is a communication terminal 10 such as a smart phone or a tablet-type information terminal, and the communication terminal 10 includes a camera 11 for capturing an image of the meter device Mt, a display 12 for displaying an image captured by the camera 11, A speaker 13 for outputting sound and a meter reading control device 15 (control means) are provided.

The configuration of the meter reading work device 1 includes (1) a configuration including a single portable smart device such as a smartphone or a tablet information terminal as described above, and (2) a smart device (smartphone or tablet information terminal). ) and an external camera (wired or wirelessly connected to a smart device), (3) a configuration including a head-mounted display (HMD) alone, (4) an HMD (using only a camera function) and a smart device (smartphone or tablet information terminal), (5) configuration including HMD and external camera (wired or wirelessly connected to HMD), (6) smart device (including camera, display, speaker) and a notebook PC (including a meter reading control device). The external cameras of the above configurations (2) and (5) are convenient for use in cases where display and photography cannot be achieved at the same time, such as at a high position or behind the operator.

Moreover, the communication terminal 10 is not limited to a specific device as long as it has a movable camera 11 and a display 12 and/or a speaker 13 . Furthermore, the camera 11, the display 12, and the speaker 13 may each be configured as a wearable device or the like that can be worn independently. It may be other information transmission means generated.
In the following description, an embodiment will be described using a meter reading work device 1 configured as a wearable device or the like in which a camera 11, a display 12, and a speaker 13 can be individually worn.

The meter reading control device 15 includes a meter reading section 51 (recognition processing means), a guide information determination section 52 (reading target extraction means), a guide information output section 53 (guide display means), and a reading setting information storage section . is equipped with

The meter reading unit 51 has a communication port capable of receiving image data from the camera 11 of the communication terminal 10, and upon receiving the image data from the camera 11, the meter reading unit 51 reads based on the setting information stored in advance in the reading setting information storage unit 54. an image determination program for determining whether or not the reading target area Z of the meter device Mt has been photographed in a predetermined recognition state; and a recognition processing program for executing predetermined recognition processing on received image data.

When the reading target area Z of the meter device Mt is photographed by the communication terminal 10 in a predetermined recognition state, the meter reading unit 51 acquires an image P12 of the reading target area A2 corresponding to the meter device Mt, and reads the recognition processing program. to read the indicated value Vm (meter value) of the meter device Mt.

The guide information determination unit 52 determines a predetermined guide related to the recognition state of the camera photographing and/or the photographed image based on the reading information from the meter reading unit 51 and the setting information pre-stored in the reading setting information accumulation unit 54. It incorporates a judgment processing program for judging the necessity of information and the content of necessary guide information, and outputs the judgment result related to the guide information to the guide information output section 53 . Predetermined guide information referred to here is a plurality of types of guide information useful for photographing for meter reading work by the camera 11, and information related to the display of the recognition accuracy of the image photographed by the camera and other recognition states and voice output. contains. Note that the recognition accuracy referred to here is the degree of certainty of the image recognition result, and can also be considered as likelihood or probability.

The guide information output unit 53 converts the guide information output from the guide information determination unit 52 according to the determination result and the setting information related to the display of the recognition state of the image into an image signal form capable of being displayed on the screen and/or Alternatively, the information is converted into a signal format capable of audio output, and information is output from at least one of the display 12 and the speaker 13 .

The guide information output unit 53 also cooperates with the guide information determination unit 52 and the reading setting information accumulation unit 54 to provide a check sheet for step-by-step work at the work site for meter reading accompanied by camera photography, and post-work check sheets. It is programmed to demonstrate the function of generating and outputting reports, etc., which are deliverables.

The reading setting information accumulation unit 54 stores setting information necessary for the meter reading unit 51 to determine whether or not the reading target area Z of the meter device Mt has been photographed by the communication terminal 10 in a predetermined recognition state, and setting information necessary for the meter reading. Setting information for determining whether or not predetermined guide information related to camera photography and/or the recognition state of a captured image is necessary based on information read from unit 51 and for determining the content of necessary guide information is provided in advance. It consists of a storage device that captures, stores and saves by inputting setting information.

Further, the meter reading control device 15 is included in the communication terminal 10 such as a smart phone or a tablet type information terminal as described above, but is not limited to this configuration, and is a portable type that can perform data communication with the communication terminal 10 and is portable. , and may be composed of a notebook PC capable of short-distance wireless communication with the communication terminal 10, for example. Further, the meter reading control device 15 may be configured by a database server or the like capable of wireless communication within a closed network with the communication terminal 10 such as a smart phone or a tablet information terminal. In this way, the meter reading control device 15, which is separate from the communication terminal 10 such as a smart phone or a tablet type information terminal constituting the meter reading work device 1 and is connected to the communication terminal 10 for communication, can be used for meter reading. Also called working system.

When the image captured by the camera 11 is received, the meter reading control device 15 transmits work support information for reading the indicated value Vm of the meter device Mt to the communication terminal 10, and transmits the information from the display 12 and the speaker 13 thereof. It is a support information presentation device for outputting.

In addition, when the reading target area Z of the meter device Mt is photographed by the camera 11 of the communication terminal 10 and displayed on the display 12, the meter reading work device 1 can display the area on the display area A1 on the display 12 by the meter reading control device 15. Based on a plurality of types of displayed images P11, P12, etc., particularly an image P12 of a reading target area A2 on the display 12 corresponding to the reading target area Z of the meter device Mt, the field worker sets the indicated value Vm of the meter device Mt. Reading work (typically meter reading work) is performed while being supported by the presentation of work support information. The work support information in this case is images P11, P21, etc. within the respective areas displayed within the plurality of attention areas R11, R21, and R22 within the display area A1.

In order to generate this work support information, the reading setting information accumulation unit 54 of the meter reading control device 15 stores a plurality of ( For example, the first to sixth) work support programs and parameter setting files in CSV format for each meter device Mt used for each work support program are stored.

Then, the meter reading control device 15 executes the work support program, and when the image captured by the camera 11 is received, first, the substantially rectangular outline of the attention area R11 is emphasized with a red line or the like on the upper side of the display area A1. The guide information G11 to be displayed is displayed, and a predetermined individual identification code pasted on the meter device Mt in advance, for example, a two-dimensional code such as a QR code (registered trademark) is displayed in the attention area R11 to prompt the user to scan the code. It's like

This code scanning is performed, for example, by detecting the shape pattern of the corner of the two-dimensional code image P11 by pattern matching and executing a known two-dimensional code scanning process. The individual identification code here may be a color code, bar code, or other identification code.

When the individual identification code pasted on the meter device Mt in advance is photographed by the camera 11 and scanned, the meter reading control device 15 reads the setting information in the reading setting information storage unit 54 corresponding to the individual identification code. Based on this, the image P12 of the reading target area A2 of the meter device Mt and the image P21 of the indicator needle to be read are specified. At that time, setting information extraction processing may be executed to specify guide information G12, G13, G14 such as scales corresponding to the images P12, P21.

The guide information G12, G13, and G14 that are given as necessary are, for example, in the case of an analog meter, the minimum value indication position, the maximum value indication position, the median value indication position, the delimiter position, the color and number of needles, etc. In the case of a digital meter, it is the display area, the number of digits of the displayed value, the position of the decimal point, and the like.

The meter reading control device 15 reads an image P11 of the two-dimensional code that has entered the first attention area R11 indicated by the guide information G11, and an image P12 of the reading target area A2 displayed in the second attention area R21. , character information displayed in the third attention area R22, and the like, selectively displayed alone or in combination, so that the instruction value can be read from the image captured by the camera 11. A change in the image recognition state can be presented so that the operator can recognize it by a change in the display content of the display 12, a voice output content of the speaker 13, or the like.

That is, the meter reading control device 15 serves as control means for outputting work support information presentation from one or both of the display 12 and the speaker 13 .

More specifically, the meter reading control device 15 performs steps until the two-dimensional code image P11 enters the first attention area R11 so that the code can be scanned, and the steps until the two-dimensional code image P11 enters the attention area R11. The stage where the image P12 of the reading target area A2 is displayed in the second attention area R21 as the code scan becomes possible, the adequacy of the photographing range of the camera 11, the direction in which the line of sight of the camera 11 should be moved, the recognition accuracy, etc. While displaying work support information corresponding to the recognition state of the plurality of images P11 and P12 in the third attention area R22 in the form of display of the recognition state of the image of the camera 11 in the form of display of the recognition state, etc. It is possible to present work support information that can be distinguished from the stage of waiting for a readable state.

Before the two-dimensional code image P11 enters the first attention area R11 so that the code can be scanned, the position, size, and inclination of the two-dimensional code image P11 match the first attention area R11. , the image P12 (meter image) of the reading target area A2 matches the position in the image assumed at the time of setting the meter reading, so the recognition accuracy of the meter reading is increased. By notifying the operator of the steps until the two-dimensional code image P11 enters the first region of interest R11 so that the code can be scanned, the operator can quickly improve the recognition accuracy of meter reading. At that time, the degree to which the two-dimensional code image P11 matches the first attention area R11 in terms of position, size, and inclination may be presented as work support information.

The communication terminal 10 and the meter reading control device 15 of the meter reading work device 1 not only carry out a stepwise guide display according to the image recognition state in a screen display form as shown in FIG. According to the type of the meter Mt, a stepwise guide display can be executed according to each image recognition state in a plurality of display forms as shown in FIGS. The unit 54 stores second to sixth work support programs.

Specifically, when the communication terminal 10 and the meter reading control device 15 of the meter reading work device 1 execute the second work support program, as shown in FIG. is a guide display means for displaying, in the display area A1 of the display 12, guide information G21-G24 of the marker display area specifying the attention area R21 in which the image P12 of the reading target area A2 should be displayed as work support information. have a function.

In this case, the meter reading control device 15 of the meter reading control device 15 functions as reading object extracting means for determining whether or not the image P12 of the reading object area A2 projected in the attention area R21 is an image with a predetermined recognition accuracy or more. The meter reading unit 51 functions as recognition processing means for readable and recognizing the indicated value Vm from an image having a predetermined recognition accuracy or more within the attention area R21.

When the communication terminal 10 and the meter reading control device 15 of the meter reading work device 1 execute the third work support program, as shown in FIGS. 4A and 4B, the meter reading control device 15 sets a plurality of colored marker images M21 to M24 having a predetermined shape in the captured image displayed in the display area A1 of the display 12 in the reading setting information accumulation unit 54 as part of the reading setting information; It has a reading setting information creating unit 55 (marker setting means).

Here, FIG. 4(a) illustrates a live view display of the image P30 of the reading target area A2 after binarization (black and white) processing, and FIG. 4(b) shows the reading target area An A2 image P30 exemplifies a plurality of indicated value display areas P31 to P34 obtained by dividing the background of the indicated value Vm by a predetermined number of digits.

In the former case shown in FIG. 4A, the guide information output unit 53 of the meter reading control device 15 displays a plurality of marker images M21 to M24 as work support information in the display area A1 of the display 12. It has a function of guide display means for displaying guide information G21-G24 specifying a plurality of marker display areas to be displayed. In addition, the guide information determination unit 52 determines whether the image P30 of the reading target area is a A reading object that specifies an attention area R21 to be displayed and determines whether or not a part of the photographed image that has entered the attention area R21 is an image with a predetermined recognition accuracy or more of the reading object area (A2). It has the function of extraction means. Further, the meter reading unit 51 has a function of recognition processing means for readable and recognizing the indicated value Vm from the image P30 having a predetermined recognition accuracy or more within the attention area R21.

In the latter case shown in FIG. 4B, the meter reading control device 15 has the reading setting information creating section 55 (marker setting means) in the reading setting information storage section 54 . The meter reading control device 15 has a plurality of indicated value display areas P31 each displaying a part of the indicated value in the reading target area A2 in the display area A1 of the display 12, and displaying the indicated value Vm as a whole. -P34 is set in advance, and a photographed image display area R21 for displaying the photographed image of the camera 11 is also set.

Then, the guide information output unit 53 of the meter reading control device 15 displays a portion of the indicated value corresponding to each of the plurality of indicated value display regions P31 to P34, and an image recognition of a portion of the indicated value. The information is displayed in a display mode according to the accuracy.

Specifically, as shown in FIG. 4(b), the third digit of the indicated value display area P33 of the four indicated value display areas P31 to P34 in the reading target area A2 is truncated. If it is not possible to accurately read the indicated value for the position, the binarized image display of this third digit will be partially white or blackened and will not be a normal numeric display, whereas the other digit positions of the two The valued image display becomes a normal numerical value display. In this case, it is conceivable that the effect of deviation of the photographing direction of the camera 11 differs depending on the numerical display shape, and that the third digit may be stained, shadowed, reflected, etc. A message pointing out such points can be displayed as guide information.

On the other hand, as shown in FIG. 4A, the image P30 of the reading target area A2 among the images captured by the camera 11 is subjected to binarization (black and white) processing, and live view display is performed within the attention area R21 of the display 12. In addition, a message can be displayed as guide information in the attention area R22 according to the result of image recognition and/or character recognition processing within each display digit range.

On the other hand, when the communication terminal 10 and the meter reading control device 15 of the meter reading work device 1 execute the fourth work support program in the display mode as shown in FIG. It has a creation unit 55 (marker setting means).

As shown in FIG. 5, the image P30 of the reading target area A2 here is displayed as a live view image that integrally displays the background of the indicated value Vm in the attention area R22, while the attention area is displayed in the display area A1. In the region of interest R41 located above R21, a numerical value for each digit obtained by performing character recognition processing on the image P40 of the reading target region A2, or a rotating arrow indicating that the character recognition processing is being performed is displayed for each digit. It has become so.

In this case, the meter reading control device 15 provides guide display for displaying a plurality of marker images M21 to M24 and guide information G21 to G24 specifying a plurality of marker display regions as work support information in the display region A1 of the display 12. A plurality of indicated value display areas each displaying part of the indicated value in the reading target area A2 and displaying the indicated value Vm as a whole in the attention area R41 on the upper side of the display area A1. In addition to setting P41-P44, a photographed image display area R21 for displaying the photographed image of the camera 11 is also set, and a part of the indicated value Vm corresponding to each area is displayed in a plurality of indicated value display areas P41-P44. is displayed in a display mode according to the recognition accuracy of the part of the image.

In this case as well, the third and fourth digits of the indicated value display areas P43 and P44 of the four digit indicated value display areas P41 to P44 in the reading target area A2 are distorted, and an accurate indication of the digit position is possible. Since the value cannot be read, a rotating arrow indicating that character recognition processing is being performed is displayed for each digit in the indicated value display areas P43 and P44 indicating the result of recognition processing for the third and fourth digits.

Further, when the communication terminal 10 and the meter reading control device 15 of the meter reading work device 1 execute the fifth work support program in the display mode as shown in FIG. It has a creation unit 55 (marker setting means).

As shown in FIG. 6, the image P12 of the reading target area A2 in this case integrates the background of the indicated value Vm in the attention area R21 sandwiched between the attention area R51 on the upper side and the attention area R22 on the lower side. It is displayed as a live view image of the analog meter displayed on the .

The meter reading control device 15 recognizes and reads the contours of two types of long and short meter pointers Pt1 and Pt2 from the image P12 of the reading target area A2 in the attention area R51 located on the upper side in the display area A1, In the indication value display areas P51 and P52 (“Meter 1” and “Meter 2” in the figure) for each needle, the indicated value is displayed as a read value or the word “under recognition”. there is

Here, the two display areas P51 and P52 in the attention area R51 on the upper side respectively display the indicator needles that are part of the indicator value in the meter reading target area A2, and the overall indicator value of the two needles is displayed. Vm is set to display, and by also setting a photographed image display area R21 that displays the photographed image of the camera 11 as a live view, pointers corresponding to the plurality of indicated value display areas P51 and 52 are displayed. A part of the indicated value of Pt1 or Pt2 is displayed in a display mode according to the recognition accuracy of the part of the image.

Also in this case, for example, if the contour shape of one of the pointers Pt1 and Pt2 in the reading target area A2 cannot be grasped, the indicated value of the pointer cannot be accurately read.

When the communication terminal 10 of the meter reading work device 1 and the meter reading control device 15 execute the sixth work support program in the display mode as shown in FIG. 55 (marker setting means).

Examples of read setting information to be stored in the read setting information storage unit 54 are illustrated in FIGS. 8(a) and 8(b).

In the meter reading work device 1 of the present embodiment, as shown in FIG. 8A, for the meter ID "m001", the respective center points of the four color markers for tilt correction, for example, the coordinates of the center of gravity on the screen ( X1, Y1), (X2, Y2), (X3, Y3), (X4, Y4). Also, the marker color is assumed to be "red".

In this embodiment, there are four markers for tilt correction, but the number is not limited to four, and may be two, three, or five or more. Further, although it is assumed that the tilt correction marker in the present embodiment is a square sticker attached to the meter device Mt, it is not limited to this. It may be the four corners of a bar code (QR code (registered trademark)), or any uniquely identifiable location in the meter device Mt.

Further, in the present embodiment, a plurality of markers are used for tilt correction, but the present invention is not limited to this. good. In this case, the contour of the shape of the meter device Mt is extracted and stored at the time of setting, and this contour is compared with the contour of the meter device Mt extracted at the time of reading the meter. may be performed. In the case of this method, the operator is guided based on the degree of deviation when the contours of the specific object match during setting and when reading the meter.

Further, as shown in FIG. 8(b), the coordinates (X1, Y1), (X2, Y2) of the respective center points of the four color markers for tilt correction, for example, the center of gravity on the screen, for the meter ID "m001" , (X3, Y3) and (X4, Y4). Also, the marker color is set to "red" and the number of digits of the meter is set to 4 digits. In addition, determination thresholds (recognition accuracy, etc.) are set for each digit to determine that characters cannot be recognized due to glare, low illuminance, low contrast, blown-out highlights, blocked-up shadows, and the like.

Next, first to sixth embodiments of the meter reading work method according to the present invention performed by executing the above-mentioned first to sixth work support programs will be described.

In the meter reading work methods of the first to sixth embodiments described below, for convenience of explanation, the work of reading the indicated values of a plurality of meter devices Mt1 to Mt12 as shown in FIG. It is assumed that they are executed sequentially when performing according to the work procedure manual stored in the storage unit 54 .

(First embodiment of meter reading work method)
As shown in FIGS. 2 and 9, in the meter reading work method of the first embodiment, the image P12 of the reading target area A2 on the display 12 is displayed within the display area A1 on the display 12 showing the photographed image of the meter device Mt1. The operation of reading the indicated value Vm of the meter device Mt1 based on is supported by the presentation of the operation support information G11 and the like.

Specifically, first, a movable camera 11 for capturing an image of the meter device Mt, a display 12 for displaying an image captured by the camera, and a speaker 13 for outputting sound are prepared.

Next, the two-dimensional barcode image P11 is placed in the uppermost attention area R11 according to the multi-stage recognition state of the image P12 of the reading target area A2 and/or the image P11 of the attention area R11 photographed by the camera 11. After that, the image P12 of the meter reading target area and, if necessary, the guide information G12-G14 are displayed in the attention area R21 in the middle row, and furthermore, the recognition states of the images P11 and P12 are displayed in the attention area R22 in the lower stage. In particular, depending on the recognition accuracy, work support information preset as work support information corresponding to the recognition state is output from either or both of the display 12 and the speaker 13 .

Therefore, in the present embodiment, the multi-stage recognition state is determined whether the captured image displayed on the display 12 does not include the reading target area A2 and/or the image P12 and/or P11 of the attention area R11 or is less than the predetermined recognition accuracy. and a reading target confirmation step in which the image P12 of the reading target area A2 is included in the photographed image displayed on the display 12 with a predetermined recognition accuracy or higher.

Further, the multi-stage recognition state in the present embodiment includes a first recognition stage for specifying the type of meter device Mt, a second recognition stage for specifying reading target area A2, and an image of reading target area A2. and a third recognition stage for recognizing the indicated value Vm, and the work support information includes display of the state of each of the first to third recognition stages or display of guide information to the next stage. become a thing.

More specifically, in the display area A1 on the display 12, first, in the pre-shooting stage until the image P11 of the two-dimensional code is displayed in the first attention area R11 so as to be scannable, the first attention area is displayed. No image is displayed in the region R11, and a photographed image deviated from the reading target region A2 is simply displayed in the second attention region R21.

When the camera 11 starts photographing, the image P11 of the two-dimensional code can be scanned in the attention area R11, and the image P12 of the reading target area A2 is displayed in the second attention area R21. A camera shooting operation is performed so as to be in a state where

At this photographing start stage, the recognition state of the plurality of images P11 and P12 is displayed in the form of display of the adequacy of the photographing range of the camera 11, the direction in which the line-of-sight direction of the camera 11 should be moved, the recognition accuracy, and other recognition states. The work support information obtained is displayed in characters or the like in the third attention area R22.

With such work support information, from the start of photography to the acquisition stage of the recognition image with the predetermined recognition accuracy, the field worker can quickly make the image captured by the camera 11 ready for reading the indicated value Vm. In addition, the position and posture of the camera 11 and other photographing conditions and environmental conditions can be changed based on the guide information, so that the recognition image can be acquired early.

(Second Embodiment of Meter Reading Work Method)
In the meter reading work method of the second embodiment executed in the display form shown in FIG. 3, a normal photographed image of the meter device Mt (for example, Mt3) is obtained by a preset process as shown in FIGS. 11 to 13. is analyzed, and the reading target area A2 and its neighboring area in the captured image of the camera 11, the outline of the indicator needle of the meter device Mt, the difference in the background color, the scale display, the memory value display, the meter device Mt individual identification symbol, the model Display and other characteristic patterns are extracted, and the reading target area A2 and the positions of the plurality of marker images M21 to M24 are specified based on the characteristic patterns in the captured image of the camera 11 (marker setting stage).

For example, in the case of a multi-needle meter device Mt3 as shown in FIG. The colors and the like of the needle images P21 and P22 are registered in advance as setting value information, and stored as a setting information file in a file format such as a CSV file. Further, as shown in FIG. 12, markers M21 to M23 and the like can be set, and the color and number of the markers, the procedure for correcting the display position of the photographed image, and the like can be set.

Next, in the display area A1 of the display 12, guide information G21-G24 specifying a plurality of, for example, four marker display areas in which the plurality of marker images M21-M24 are to be displayed is displayed (guide display step).

Next, according to the photographing state of the camera 11, a plurality of marker images M21-M24 in the photographed image are specified and displayed in the display area A1, and the display of the plurality of marker images M21-M24 and the guide information G21-G24 are displayed. A region of interest R21 in which the image P12 of the reading target region should be displayed is specified according to the degree of geometric correspondence with the corresponding plurality of marker display regions.

The "degree of geometric correspondence" here means, for example, that the geometric matching rate falls within a predetermined range, and that the display of the plurality of marker images M21-M24 and the plurality of guide information G21-G24 substantially correspond to each other. However, they do not have to match each other, and it is possible to present different guide information according to the distance between the centers of gravity of both.

After specifying the attention area R21, reading target extraction is performed to determine whether or not a part of the photographed image that has entered the attention area R21 is the image P12 with a predetermined recognition accuracy or more of the reading target area A2.

The recognition accuracy here is, for example, the degree of coincidence between the centroid positions of the number of marker images M21 to M24 and the centroid positions of the plurality of marker display areas corresponding to the guide information G21 to G24, or the degree of relative distance. It may be evaluated to

Next, when it is determined that the part of the photographed image that has entered the attention area R21 is the image P12 with a predetermined recognition accuracy or more in the reading target area A2, the instruction value Vm can be read from the image P12 with a predetermined recognition accuracy or more. Recognition processing is executed to read the indicated value Vm at the tip of the pointer image P21.

(Third Embodiment of Meter Reading Work Method)
In the meter reading work method of the third embodiment executed in the display form shown in FIG. 2, a plurality of indicated value display areas P31 to P34 are set in advance, each of which displays a part of the indicated value in the reading target area A2 and the indicated value Vm as a whole.

In addition, a photographed image display area R21 for displaying the photographed image of the camera 11 is also set, and a part of the instruction value corresponding to each area is displayed in a plurality of instruction value display areas P31 to P34. are displayed in a display mode according to the recognition accuracy of each of the images P31 to P34.

In this case, the result of reading is displayed for each digit, and even if the digit is read once, it may be overwritten if a result with higher recognition accuracy is obtained.

Also, the difference in recognition accuracy may be displayed and output as a difference in font or background color.

The attention area R21 may be used as a live view image display area, and an image P30 in the reading target area A2 may be displayed as an integrated image in the captured image display area R21 as shown in FIG. 4A. As shown in 4(b), it is also possible to display and output as images P31-P34 of a plurality of indicated value display areas within the reading target area A2.

At this time, a part of the indicated value corresponding to each of the indicated value display areas P31 to P34 can be displayed and output in a display mode according to the recognition accuracy of the image of the part of the indicated value. The display output of the read value at the digit position and the output of the undetermined display at the unrecognized digit position can be displayed together. At this time, the work support information that appears as a difference in the display images themselves of the plurality of indicated value display areas P31 to P34 is the display of the state of each of the first to third recognition stages or guide information to the next stage. including display.

(Fourth Embodiment of Meter Reading Work Method)
In the meter reading work method of the fourth embodiment executed in the display form shown in FIG. Guide information G21-G24 specifying the marker display area is displayed (guide display step).

Next, according to the photographing state of the camera 11, a plurality of marker images M21-M24 in the photographed image are specified and displayed in the display area A1, and the display of the plurality of marker images M21-M24 and the guide information G21-G24 are displayed. The image P30 of the reading target area, the attention area R21 on the real display side, and the attention area R41 on the recognition result side are specified according to the degree of geometric correspondence with the corresponding marker display areas.

For example, the attention area R21 in which the image P30 of the reading target area A2 should be displayed is specified depending on whether or not the plurality of marker images M21-M24 are displayed in the plurality of marker display areas corresponding to the guide information G21-G24. At the same time, it is determined whether or not the image P30 of the reading target area A2 can be displayed in the attention area R41 as a plurality of digit images P41 to P44 having a predetermined recognition accuracy or more or as recognition information of their numerical values (reading target extraction step).

At this time, the image P30 of the reading target area A2 is displayed as a live view image displayed integrally with the background of the indicated value Vm within the attention area R21.

In the attention area R41 on the recognition result side, according to the result of the character recognition processing of the image P30 of the reading target area A2, a numerical value for each read digit or a rotating arrow indicating that character recognition processing is being performed is displayed for each digit. do.

In this case, the plurality of marker images M21-M24 and the plurality of guide information G21-G24 displayed as work support information in the display area A1 of the display 12 are displayed in the attention area R41 on the upper side of the display area A1. A part of the indicated value in the reading target area A2 is displayed in the area A2, and a reference position is specified that enables setting of a plurality of indicated value display areas P41 to P44 that display the indicated value Vm as a whole.

Further, since the attention area R21 on the real view display side and the attention area R41 displaying the plurality of indication value display areas P41 to P44 on the recognition result side are used together, the recognition state of the captured image can be adjusted according to the recognition accuracy. More accurate display is possible.

(Fifth embodiment of meter reading work method)
In the meter reading work method of the fifth embodiment executed in the display form shown in FIG. A region of interest R21 in the middle between the two is set, and when an image is captured by the camera 11, an image P12 of the reading target region A2 is displayed in the region of interest R21 as a live view image.

Next, in the attention area R51 positioned on the upper side in the display area A1, a process of detecting the contours of the two types of meter pointers Pt1 and Pt2 from the image P12 of the reading target area A2 is executed, and the respective indications are detected. The value is displayed in the display areas P51, P52 as a reading or the word "under recognition".

In the display example of FIG. 6, the contour shape of one Pt2 of the pointers Pt1 and Pt2 in the reading target area A2 is in an unrecognized state. is performed, the fact that the recognition result is awaited is displayed in the attention area R22 on the lower side.

In the present embodiment as well, the two display areas P51 and P52 as a whole numerically display the indication value Vm of the two needles, and the captured image of the camera 11 is displayed as a live view in the attention area R21 as the captured image display area. It is possible to more accurately display the recognition accuracy of the image of the indication value Vm of the pointer Pt1 or Pt2 according to the recognition accuracy.

(Sixth Embodiment of Meter Reading Work Method)
In the meter reading work method of the sixth embodiment executed in the display form shown in FIG. is displayed.

Next, according to the photographing state of the camera 11, a plurality of marker images M21-M24 in the photographed image are specified and displayed in the display area A1, and the display of the plurality of marker images M21-M24 and the guide information G21-G24 are displayed. A region of interest R21 in which the image P30 of the reading target region should be displayed is specified according to the degree of geometric correspondence with the plurality of corresponding marker display regions.

Next, a plurality of marker images photographed by the camera 11 and displayed within the display area A1 of the display 12 are displayed with respect to the plurality of marker display areas specified by the guide information G21-G24 displayed within the display area A1 of the display 12. A geometric deviation amount of M21-M24 is calculated.

That is, the coordinates E21-E24 of the marker points (center of gravity) of the guide information G21-G24 set as a plurality of marker display areas within the display area A1 at the time of setting, and the marker points of the marker images M21-M24 recognized at the time of photographing with the camera. The amount of deviation and the direction of the deviation are obtained from the correspondence relationship with the coordinates e21-e24 of (the center of gravity). For example, a geometric transformation matrix is obtained based on the coordinates of the four markers at the time of setting and reading, and the inclination of the image and the amount of perspective deviation are calculated by affine transformation or the like using the matrix.

In this way, the rectangular images (e21-e24) specified by the coordinates e21-e24 of a plurality of, for example, four marker images M21-M24 and the set coordinates E21- Correction processing such as geometric correction for reducing the amount of deviation can be performed on the image displayed on the display 12 by the geometric conversion processing with the square image (E21-E24) specified by E24.

Next, the operation of this embodiment will be described.

(Action of invention device)
In the meter reading work device and the meter reading work support system of the present embodiment configured as described above, the operation of reading the indicated value Vm of the meter device Mt based on the image P12 or P30 of the reading target area A2 of the meter device Mt is performed. is executed while being supported by presenting work support information, a movable camera 11 for photographing the meter device Mt, a display 12 for displaying the photographed image of the camera 11, a speaker 13 for outputting sound, Guide information G11 (also G12-G14 as necessary), or G21-G24 or A recognition state display for each digit of the indicated value Vm, a message in the attention area R22, and the like are configured to be output from one or both of the display 12 and the speaker 13. FIG.

Therefore, in the present embodiment, until the indicated value Vm of the meter device Mt is recognized to be readable, the work support information corresponding to the recognition state of the image captured by the camera is displayed as an image on the display 12 and/or output as an audio signal from the speaker 13. will be presented. Therefore, even in an environment with poor photographing conditions, the photographing conditions such as the position and posture of the camera 11 and the line-of-sight direction can be favorably guided so that an image photographed by the camera 11 can be acquired as an image with a good recognition rate. , it is possible to improve the practicability of meter reading work and improve the reading accuracy of the indicated value of the meter.

In addition, by presenting the work details of experienced workers to inexperienced workers using work support information, it is possible to prevent work mistakes and accidents by field workers. Even one person can perform automatic reading and checking of the reading result, which is equivalent to the meter reading and checking work that has been done. Furthermore, since this embodiment has a check sheet and report output function, it is possible to prevent work mistakes and forgetting work by field workers, and because the creation of deliverables can be automated, work quality can be easily guaranteed. Become.

In addition, in this embodiment, the movable information communication terminal 10 is integrally provided with the camera 11, the display 12, the speaker 13, and the meter reading control device 15, so that a lightweight and compact device configuration can be realized. .

Further, in the present embodiment, guide information G11 to G14 or guide information G11 to G14 for specifying the attention areas R21, R11 in which the image P12 of the reading target area A2 and the image P11 for specifying the meter type should be displayed in the display area A1 of the display 12 A guide information output unit 53 displaying G21-G24, and determining whether or not the image P12 of the reading target area A2 and the image P11 for specifying the meter type are included in the attention areas R11 and R21 as images having a predetermined recognition accuracy or higher. and a meter reading unit 51 for reading and recognizing the indication value Vm from the images P12 and P11 having a predetermined recognition accuracy or more in the attention areas R11 and R21.

Therefore, after focusing the photographing of the camera 11 on the attention areas R21 and R11, the recognition accuracy of the photographed image can be improved, and the image of the reading target area A2 having a predetermined recognition accuracy or more can be quickly and accurately included in the photographed image. Such camera operation becomes possible.

Moreover, in the present embodiment, the reading setting information creation unit 55 for setting a plurality of marker images M21-M24 in the display image of the display 12, and the guide information G21-M for setting a plurality of marker display areas in the display area A1 of the display 12 The attention area where the image P30 of the reading target area A2 should be displayed according to the degree of geometrical correspondence between the guide information output unit 53 that displays the guide by G24 and the marker display area corresponding to the display of the marker images M21 to M24. A guide information determination unit 52 that specifies R21 (or R41) and determines whether or not a part of the photographed image that has entered the attention area R21 is an image with a predetermined recognition accuracy or more in the reading target area A2. and a meter reading unit 51 for reading and recognizing the indication value Vm from the image P30 (or P40) having a predetermined recognition accuracy or more in the attention area R21.

Therefore, depending on whether or not the plurality of marker images M21 to M24 are displayed corresponding to the respective marker display areas, the attention area R21 in which the image P12 of the reading target area A2 should be displayed is specified and the reading target area A2 is specified. It is possible to determine whether or not the image P12 in the target area R21 is recognized as an image with a predetermined recognition accuracy or more in the attention area R21. can be included in the captured image.

In addition, in the meter reading work device of the present embodiment, the plurality of marker images M21-M24 in the captured image are displayed in the plurality of marker display areas set by the guide information G21-G24 within the display area A1 of the display 12. A geometrical shift amount is calculated, and a correction process is executed to reduce the shift amount of the image P30 displayed on the display 12 by geometrically transforming the plurality of marker images M21 to M24 and the image P30 of the attention area R21. .

Therefore, by correcting the deviation of the camera 11 from the reading target area Z of the meter device Mt such as the tilt of the camera 11 at the time of photographing and reducing the deviation of the displayed image on the display 12, the operation burden during photographing can be reduced. .

Further, in the present embodiment, a plurality of indicated value display areas P31- for displaying a part of the indicated value Vm in the reading target area A2 in the display area A1 of the display 12, and displaying the indicated value Vm as a whole. P34 (or (P41-P44) or (P51, 52)) is set in advance, and an attention area R21 as a photographed image display area for displaying the photographed image of the camera 11 is also set, and a plurality of indication value display areas are set. In P31-P34, a part of the indicated value Vm corresponding to each region is displayed in a display mode according to the recognition accuracy of the image P31, P32, P33 or P34 of the part of the indicated value Vm.

Therefore, by setting the indicated value display areas P31 to P34 for each predetermined number of digits of the indicated value Vm of the meter device Mt, the indicated value Vm (part) in each indicated value display area P31, P32, P33 or P34 can be It becomes possible to set the photographing condition according to the recognition accuracy of the image, and it is possible to quickly execute the recognition processing for increasing the recognition accuracy of any one of the indication value images P31 to P34 with low recognition accuracy.

Further, in the present embodiment, the portable meter reading control device 15 has a first recognition stage for identifying the type of the meter device Mt from an individual identification code, a characteristic pattern, etc. a second recognition step of specifying the reading target area A2 based on the setting information of the meter device Mt obtained, and a third recognition step of recognizing the indicated value Vm based on the image P12 (or P30) of the reading target area A2. , and display the state of each of the first to third recognition stages (display of guide display G11, display of image P11 such as individual identification code, display of image P12 of reading target area A2) in work support information display of attention area R21, display of read values, etc.) or guide information to the next stage (guide information G11, guide information G12-G14, guide information G21-G24, and guide information within attention area R22). there is

Therefore, in each of the first to third recognition stages until the indicated value Vm of the meter device Mt is readable, display of the recognition state or display of guide information to the next stage is possible, and the photographing conditions are improved. An image captured by the camera 11 can be obtained as an image with a good recognition rate even in an environment without the camera.

Further, in this embodiment, the meter reading control device 15 sets the meter information display area R11 or R21 within the display area A1 of the display 12, and based on the image P11 or P12 in the meter information display area R11 or R21, Reading conditions G12-G14 for the position of the reading target area A2 or the indicated value Vm are determined, and the position of the reading target area A2 or the indicated value (Vm) is determined as work support information under a predetermined stage of the recognition state among a plurality of stages. It is configured to output guide information (indicated in R22) regarding reading.

With this configuration, the meter information display area for judging the reading conditions of the position information of the reading target area or the indicated value is set within the display area of the display. It becomes possible to easily determine the position of the area or the reading condition of the indicated value.

In the meter reading work device of the present embodiment, the conditions for presenting the work support information include conditions for changing the sound output from the speaker 13 in accordance with the multi-level recognition state.

Therefore, it is possible to clearly and easily grasp changes in the recognition state in multiple stages as changes in the output sound from the speaker 13, such as changes in the frequency of the output sound or changes in the cycle of the intermittent output sound.

(Action of Inventive Method)
In the meter reading work methods of the first to sixth embodiments, when the work of reading the indicated value Vm of the meter device Mt based on the image P12 or P30 of the reading target area A2 is supported by presenting the work support information, , guide information G11 (also G12-G14 if necessary) or G21-G24 is generated according to the multi-stage recognition state of the image P12 of the reading target area A2 photographed by the camera 11, and the work support information is Either one or both of the display 12 and the speaker 13 are made to output.

Therefore, in the method of the present embodiment, until the indicated value Vm of the meter device Mt is readable and recognized, the work support information is displayed on the display 12 and/or on the speaker 13 according to the recognition state of the image captured by the camera. It will be presented by voice output. Therefore, it is possible to guide the photographing conditions such as the position and posture of the camera 11 and the line of sight direction so that the image photographed by the camera 11 can be acquired as an image with a good recognition rate even in an environment where the photographing conditions are not good, and the meter can be read. It is possible to improve the practicality of the work and improve the reading accuracy of the meter indication value.

For example, even though the guide display for the reading target area A2 is performed, the appearance of the reading target area A2 may cause reflection of lighting, etc., erroneous recognition of objects of the same color, brightness of the work environment, appropriateness of distance, etc. Effective work support information can be presented even in the environment.

In addition, for example, by presenting the details of work performed by a skilled worker to an inexperienced worker, it is possible to prevent work mistakes and accidents by field workers. Even one person can carry out automatic reading and checking of the reading result, which is equivalent to the reading and checking work.

In the meter reading work method of the present embodiment, the multi-stage recognition state is determined whether or not the photographed image displayed on the display 12 includes the image P12 and/or P11 of the reading target area A2 and/or R11. It includes a read target undetermined stage where the recognition accuracy is less than the recognition accuracy, and a read target determination stage where the image P12 and/or P11 of the read target area is included in the captured image displayed on the display 12 with a predetermined recognition accuracy or higher.

Therefore, the recognition state of the image P12 of the reading target area A2 can be easily discriminated in a plurality of stages depending on whether or not the image of the reading target area having a predetermined recognition accuracy or more is included in the captured image of the camera 11 or not.

Furthermore, in the meter reading work methods of the second to sixth embodiments, guide information G11 for specifying an attention area R11 in which an image P11 for individual identification is to be displayed as work support information in the display area A1 of the display 12 Alternatively, a guide display stage of displaying guide information G21-G24 for specifying the attention area R21 in which the image P12 of the reading target area A2 should be displayed, and the display of the plurality of marker images M21-M24 and the plurality of marker display areas. A region of interest R21 (or R41) in which the image P12 or P30 of the reading target region should be displayed is specified according to the degree of geometric correspondence, and a part of the photographed image that enters the region of interest R21 is designated as the reading target region. It includes a reading target extraction step of determining whether or not the image of A2 has a predetermined recognition accuracy or more, and a recognition processing step of recognizing the instruction value Vm from the image having a predetermined recognition accuracy or more in the attention area R21 so that it can be read. there is

Therefore, after concentrating the photographing of the camera on the attention area, the recognition accuracy of the photographed image can be improved, and the image of the reading object area having a predetermined recognition accuracy or more can be quickly and accurately included in the photographed image.

Moreover, in the meter reading operation method of the second to sixth embodiments, in the marker setting step of setting a plurality of marker images M21 to M24 in the photographed image displayed on the display 12, and in the display area A1 of the display 12, As work support information, a guide display step of displaying guide information G21-G24 specifying a plurality of marker display areas in which the plurality of marker images M21-M24 are to be displayed respectively; The attention area R21 in which the image P12 of the reading target area A2 should be displayed is specified by whether or not it is displayed in the display area, and it is determined whether or not the image P12 of the reading target area A2 is an image with a predetermined recognition accuracy or more. and a recognition processing step of recognizing the instruction value Vm from an image having a predetermined recognition accuracy or more in the attention area R21 so as to be readable.

Therefore, depending on whether or not the plurality of marker images M21 to M24 are displayed in the corresponding marker display areas, the attention area R21 in which the image P12 of the reading target area A2 should be displayed is specified, and the image of the reading target area A2 is specified. It is possible to determine whether P12 is an image with a predetermined recognition accuracy or more. Therefore, the image P12 of the reading target area A2 having a predetermined recognition accuracy or more can be quickly and accurately included in the captured image of the camera 11 by relatively simple processing.

Further, in the meter reading work method of the sixth embodiment, a plurality of marker display areas set within the display area A1 of the display 12 are photographed by the camera 11 and displayed within the display area of the display 12. Calculate the amount of geometrical deviation of the marker images M21-M24, and reduce the amount of deviation with respect to the image displayed on the display 12 by geometrically transforming the plurality of marker images M21-M24 and the image P12 of the attention area R21. Executes the correction process.

Therefore, by correcting the amount of deviation such as tilt of the camera from the reading target area of the meter device at the time of photographing and reducing the amount of deviation in the display image, it is possible to reduce the operational burden during photographing. In addition, it is possible to display an arrow indicating the correction direction of the photographing posture for correcting the deviation amount, display the operation direction, or display an instruction by voice, for example, "Please turn to the left". In addition, it is conceivable to output the direction of deviation (roll, yaw, pitch, etc.) so that it can be discriminated by tone color, or to express the amount of deviation itself by volume.

In the meter reading work method of the third to sixth embodiments, the display 12 capable of displaying the captured image of the camera 11 and the work support information is prepared, and in the display area A1 of the display 12, in the reading target area A2, A plurality of indicated value display areas P31-P34, etc. (or (P41-P44) or (P51, P52)) for displaying a part of the indicated value Vm and displaying the indicated value Vm as a whole are set in advance, and the camera 11 A photographed image display area R21 for displaying the photographed image is also set. It is displayed in a display mode according to the accuracy.

Therefore, by setting the indicated value display areas P31-P34 and the like for each predetermined number of digits of the indicated value Vm of the meter device Mt, the photographing condition corresponding to the recognition accuracy of the indicated value image in each indicated value display area P31-P34 can be obtained. can be set, and it is possible to quickly perform camera operation guidance and recognition processing to increase the recognition accuracy of some instruction value images with low recognition accuracy.

In the meter reading work method of the first to sixth embodiments, the multi-stage recognition state includes a first recognition stage for specifying the type of the meter device Mt, a second recognition stage for specifying the reading target area A2, and a third recognition stage for recognizing the instruction value Vm based on the image P12 or P30 of the reading target area A2, and the work support information is a display of the state of each of the first to third recognition stages or a prompt to the next stage. Contains a display of guide information.

Therefore, in each of the first to third recognition stages until the indicated value Vm of the meter device Mt is readable, display of the recognition state or guide information G11, G12-G14, G21-G24, etc. to the next stage is performed. can be displayed, and the image captured by the camera can be acquired as an image with a good recognition rate even in an environment where the shooting conditions are not good.

In the meter reading work method of the first embodiment, the display area A2 for the photographed image and the display area R11 for the meter information such as the individual consciousness code are set in the display area A1 of the display 12, and the display area A1 of the display 12 Based on the images P11, P12, etc., the position of the reading target area A2 or the reading condition of the instruction value Vm is determined, and the position of the reading target area A2 is determined as work support information under the recognition state of a predetermined stage among a plurality of stages. Alternatively, it outputs guide information G11 regarding reading of the indicated value Vm, and G12-G14 as necessary.

Therefore, by setting the meter information display area R11 for determining the reading condition of the position information of the reading target area A2 or the indicated value Vm within the display area A1 of the display 12, the image P12 of the reading target area A2 can be displayed. Along with the recognition, it becomes possible to easily determine the position of the reading target area A2 or the reading condition of the indicated value Vm.

In addition, in the meter reading work method of each embodiment, the number of photographing times of the camera at each stage among the plurality of stages is variably set according to the recognition state of each stage, and the camera at a predetermined stage among the plurality of stages is increased as the recognition accuracy of the photographed image at the predetermined stage is lower, the image P12 of the reading target area A2 having a predetermined recognition accuracy or more is captured in the image captured by the camera 11 at the predetermined stage. is included, and the recognition accuracy of the image captured by the camera 11 at that stage can be raised within the accuracy range in which the indicated value Vm can be read.

By doing so, in a predetermined stage where the reading target area A2 of the indicated value Vm of the meter device Mt has low recognition accuracy, the number of times of photographing with the camera 11 is increased to increase the recognition accuracy of the captured image, thereby improving the recognition accuracy. meter reading becomes possible.

Further, in the meter reading work methods of the third, fourth and sixth embodiments, a part of the indicated value Vm corresponding to each is displayed in the reading target area A2, and the indicated value Vm is displayed as a whole. In addition to presetting the indicated value display areas P31-P34 (P41-P44 or P51, P52 may be used), as work support information according to the recognition state of the image P12 in the reading target area A2, a plurality of indicated value display areas A plurality of divided recognition state images P30 obtained by binarizing captured images such as P31 to P34 may be displayed.

In this case, when a part of the images of the plurality of indicated value display areas P30 has a predetermined degree of accuracy or less, at the recognition stage, the normal display areas P31, P32, P33 or P34 may not be normal due to white spots, black crushed areas, or the like. Since a binarized image from which no indication value can be obtained can be displayed, both the change in display according to the accuracy of the captured image and the result of recognition processing can be presented in a straightforward and visually recognizable manner.

In the meter reading work method of each embodiment, the conditions for presenting the work support information include conditions for changing the sound output from the speaker 13 according to a plurality of stages of recognition states.

Therefore, changes in the recognition state in a plurality of steps can be treated as stepwise compared to changes in the output sound from the speaker 13, such as changes in the frequency of the output sound or changes in the cycle of the intermittent output sound, that is, changes in the strength or pattern of the sound. A change in the recognition state can be easily grasped by making such a change a sound change that can be easily grasped aurally.

Furthermore, in each of the above-described embodiments, the communication terminal 10 includes the camera 11, the display 12, the speaker 13, and the meter reading control device 15, and the meter reading control device 15 reads the meter device Mt using the camera 11. A meter reading unit 51 for reading the indicated value Vm of the meter device Mt based on the image of the reading target area A2 acquired from the camera 11 when the target area is photographed in a predetermined recognition state, and a reading operation of the indicated value Vm. and a guide information output unit 53 for transmitting and outputting the work support information to the display 12 . Further, the meter reading control device 15, which is a support information presenting device, recognizes a photographed image in a plurality of stages from the start of photographing by the camera 11 until the image of the reading target area is recognized to be readable based on the photographed image. Depending on the situation, the presentation conditions of the work support information are changed, the work support information is transmitted to the display 12 and the speaker 13 in at least one of image and voice information forms, and the output of the work support information is output in a plurality of stages of recognition states. change according to

Therefore, until the indicated value Vm of the meter device Mt is recognized to be readable, work support information can be presented according to the recognition state of the image captured by the camera 11. Therefore, the camera 11 can be used to capture images even under poor image capturing conditions. The position and posture of the camera 11 can be guided so that the image can be acquired as an image with a good recognition rate, and the practicality of the meter reading operation and the accuracy of reading the indicated value of the meter can be improved.

Similarly, in each embodiment, the operation of reading the indicated value Vm of the meter device Mt based on the image P12 of the reading target area A2 of the meter device Mt is performed by the guide information G11 and, if necessary, G12-G14 (guide information G21). -G24 or P31-P34) will be implemented. In this method, a movable camera 11 for capturing an image of the meter device Mt, a display 12 for displaying an image captured by the camera 11, and a speaker 13 for outputting sound are prepared in advance. , the conditions for presenting guide information G11 and G12-G14 are variably set according to a multi-stage recognition state from the start of the photographed image until the indicated value Vm of the meter device Mt is recognized, and the camera 11 , the guide information G11, G12-G14, etc. is output from either or both of the display 12 and the speaker 13, and the output of the guide information G11, G12-G14, etc. is changed according to the recognition state. Let

Therefore, until the indicated value Vm of the meter device Mt is recognized to be readable, work support information can be presented according to the recognition state of the image captured by the camera 11. Therefore, the camera 11 can be used to capture images even under poor image capturing conditions. The position and posture of the camera can be guided so that the image can be obtained with a good recognition rate, and the practicality of the meter reading operation and the reading accuracy of the indicated value of the meter can be improved.

As described above, according to each embodiment of the present invention, it is possible to effectively support stepwise and effective work at a work site for meter reading involving camera photography, check the work, and generate post-work deliverables. It is possible to provide a meter reading work device and method, and a meter reading work support system.

Although there are four marker images M21-M24 in each of the above-described embodiments, the number of markers may be two or more. The guide display and its message are only examples, and it goes without saying that different guide displays can be adopted according to various forms of the meter device as shown in FIG.

INDUSTRIAL APPLICABILITY As described above, the present invention can effectively support stepwise and effective work support at a meter reading work site accompanied by camera photography, checking of the work, generation of deliverables after the work, and the like. A meter reading work device and method can be provided. INDUSTRIAL APPLICABILITY The present invention as described above is useful for a meter reading work device and method for reading indicated values from photographed images of various meters by a mobile camera, and a meter reading work support system in general.

Further discussion is provided regarding the meter reading work apparatus and method described in the above embodiments.
1) The meter reading work device according to the above-described embodiment is a meter reading work device that assists the work of reading the indicated value of the meter device based on the image of the reading target area of the meter device by presenting work support information. a camera for photographing the meter device, a display for displaying an image photographed by the camera, a speaker for outputting sound, and a plurality of stages of recognition states for the image of the reading target area photographed by the camera. and control means for outputting the corresponding work support information from one or both of the display and the speaker.

With this configuration, the work support information corresponding to the recognition state of the image captured by the camera is presented by image display and/or voice output until the indicated value of the meter device is recognized to be readable. Therefore, it is possible to guide the shooting conditions such as the camera position and posture, the line of sight direction, etc. so that the image captured by the camera can be acquired as an image with a good recognition rate even in an environment where the shooting conditions are not good. It is possible to improve the practicability and the reading accuracy of the indicated value of the meter. In addition, for example, by presenting the details of work performed by a skilled worker to an inexperienced worker, it is possible to prevent work mistakes and accidents by field workers. Even one person can carry out automatic reading and checking of the reading result, which is equivalent to the reading and checking work.

2) The meter reading work device of the above embodiment includes guide display means for displaying, as the work support information, guide information specifying an attention area where the image of the reading target area should be displayed in the display area of the display. a reading target extracting means for determining whether or not the image of the reading target area is included in the attention area as an image having a predetermined recognition accuracy or more; and recognition processing means for readable recognition of the value.

With this configuration, it is possible to increase the accuracy of recognition of the photographed image after concentrating the photographing of the camera on the attention area, and to quickly and accurately include the image of the reading target area having a predetermined recognition accuracy or higher in the photographed image. become.

3) The meter reading work device of the above embodiment includes marker setting means for setting a plurality of marker images in the photographed image displayed on the display, and the work support information in the display area of the display. guide display means for displaying guide information specifying a plurality of marker display areas in which a plurality of marker images are to be displayed; and degree of geometric correspondence between the display of the plurality of marker images and the plurality of marker display areas. to specify an attention area in which the image of the reading target area is to be displayed, and determine whether or not a part of the photographed image that has entered the attention area is an image with the predetermined recognition accuracy or more of the reading target area. The configuration may include reading target extracting means for executing the determination, and recognition processing means for recognizing the instruction value from the image having the predetermined recognition accuracy or more in the attention area so as to be readable.

In this case, depending on whether or not a plurality of marker images are displayed in each marker display area, it is possible to specify an attention area in which the image of the reading target area should be displayed, and an image of which the image of the reading target area has a predetermined recognition accuracy or higher. As a result, it is possible to quickly and accurately include the image of the reading target area with a predetermined recognition accuracy or more in the captured image by a relatively simple process.

4) In the meter reading work device of the above embodiment, a plurality of indicated values are displayed in the display area of the display, each indicating a part of the indicated value in the reading target area, and displaying the indicated value as a whole. A display area is set in advance, and a photographed image display area for displaying an image photographed by the camera is also set. The indication value may be displayed in a display mode according to the recognition accuracy of a part of the image of the indicated value.

With this configuration, by setting the indicated value display area for each predetermined number of digits of the indicated value of the meter device, the photographing conditions are set according to the recognition accuracy of the image of the indicated value (partial) in each indicated value display area. This makes it possible to quickly perform the recognition processing for increasing the recognition accuracy of a portion of the instruction value image with low recognition accuracy.

5) In the meter reading work device of the above embodiment, the control means has a first recognition stage for specifying the type of the meter device and a second recognition stage for specifying the reading target area as the multi-stage recognition states. A recognition stage and a third recognition stage of recognizing the indicated value based on the image of the reading target area are set, and the state of each of the first to third recognition stages is displayed in the work support information. can be configured to include

With this configuration, it is possible to display the recognition state or guide information to the next stage at each of the first to third recognition stages until the reading of the indicated value of the meter device is recognized, thereby improving the photographing conditions. An image captured by a camera can be acquired as an image with a good recognition rate even in an environment without

6) In the meter reading work device of the above embodiment, the control means sets the meter information display area within the display area of the display, and determines the position of the reading target area based on the image within the meter information display area. Alternatively, the reading condition of the instruction value is determined, and guide information regarding the position of the reading target area or the reading of the instruction value is output as the work support information under a predetermined stage of the recognition state among the plurality of stages. can do.

With this configuration, the meter information display area for judging the reading conditions of the position information of the reading target area or the indicated value is set within the display area of the display. It becomes possible to easily determine the position of the area or the reading condition of the indicated value.

7) In the meter reading work device of the above embodiment, the conditions for presenting the work support information may include conditions for changing the sound output from the speaker according to the recognition states of the plurality of stages. The change in sound may include, for example, conditions for changing the frequency of sound or the cycle of intermittently output sound.

In this case, it is possible to easily grasp the change in the recognition state in multiple stages as the change in the frequency of the output sound from the speaker or the change in the cycle of the intermittent output sound.

8) The meter reading work method according to the above embodiment is a meter reading work method in which the work of reading the indicated value of the meter device based on the image of the reading target area of the meter device is supported by presenting work support information. wherein a movable camera for capturing an image of the meter device, a display for displaying an image captured by the camera, and a speaker for outputting sound are provided, and the image of the reading target area captured by the camera is provided. The work support information is generated according to a plurality of stages of the recognition state of , and the work support information is output from either one or both of the display and the speaker.

With this configuration, the work support information is presented by image display and/or voice output according to the recognition state of the image captured by the camera until the indicated value of the meter device is recognized to be readable. Therefore, it is possible to guide the shooting conditions such as the camera position and posture, the line of sight direction, etc. so that the image captured by the camera can be acquired as an image with a good recognition rate even in an environment where the shooting conditions are not good. It is possible to improve the practicability and the reading accuracy of the indicated value of the meter. In addition, for example, by presenting the details of work performed by a skilled worker to an inexperienced worker, it is possible to prevent work mistakes and accidents by field workers. Even one person can carry out automatic reading and checking of the reading result, which is equivalent to the reading and checking work.

9) In the meter reading work method of the above-described embodiment, the multi-stage recognition state is a reading target whose image of the reading target area is not included in the captured image displayed on the display or whose recognition accuracy is less than a predetermined recognition accuracy. The configuration may include an undetermined step and a reading target determination step in which the image of the reading target area is included in the captured image displayed on the display with the predetermined recognition accuracy or more.

With this configuration, the recognition state of the reading target area can be easily determined in a plurality of stages depending on whether or not the image of the reading target area having a predetermined recognition accuracy or more is included in the captured image of the camera.

10) In the meter reading work method of the above embodiment, a plurality of indicated value display areas for displaying a part of the indicated value corresponding to each and displaying the indicated value as a whole are provided in advance in the reading target area. In addition, a plurality of divided recognition state images obtained by binarizing the photographed images of the plurality of instruction value display regions may be displayed as work support information corresponding to the recognition state of the image of the reading target region. good.

With this configuration, when an image in a part of a plurality of indicated value display areas has a predetermined degree of accuracy or less, in the recognition stage, a normal indicated value in the display area cannot be obtained due to white spots or black crushed areas. Since the converted image can be displayed, both the change in the display according to the accuracy of the captured image and the result of the recognition processing can be presented in a straightforward and visually recognizable manner.

1 meter reading work device 10 information communication terminal (communication terminal)
11 camera 12 display 13 speaker 15 meter reading control device (control means)
51 meter reading unit (recognition processing means)
52 guide information determination section (reading target extraction means)
53 guide information output unit (guide display means)
54 reading setting information storage unit 55 reading setting information creation unit A1 display area A2 reading target area E21, E22, E23, E24 coordinates (set coordinates of marker display area, coordinates of marker points of guide information)
e21, e22, e23, e24 coordinates of marker points in marker image G11 guide information (work support information)
G12, G13, G14 Guide information (work support information)
G21, G22, G23, G24 Guide information (work support information)
M21, M22, M23, M24 Marker image Mt Meter device P11 image (meter information image)
P12 Image of reading target area Pt1, Pt2 Pointer P30 Image (indicated value display area, divided recognition state image)
P31, P32, P33, P34 Indicated value display area (image of indicated value display area)
P40, P50 Image P41, P42, P43, P44 Digit image (indicated value display area)
P51, P52 display area (indicated value display area)
R11 attention area (meter information display area, first attention area)
R21 attention area (captured image display area, second attention area)
R22 attention area (third attention area)
R31, R41, R51 Region of interest

Claims (2)

a reading device for reading an indicated value of the meter device based on an image of the reading target region obtained from the communication terminal when the reading target region of the meter device is photographed by a communication terminal with a camera in a predetermined recognition state;
A meter reading work support system comprising: a support information presentation device that transmits work support information for reading the indicated value to the communication terminal and causes the communication terminal to output the work support information,
The support information presentation device is
The condition for presenting the work support information is changed according to the multi-stage recognition state of the photographed image from the start of photographing by the camera until the image of the reading target area is recognized to be readable based on the photographed image. let
A meter reading work support, characterized in that the work support information is transmitted to the communication terminal in at least one of image and voice information forms, and the output of the work support information is changed according to the recognition states of the plurality of stages. system. A meter reading work support method for supporting the work of reading an indicated value of a meter device based on an image of a reading target area of the meter device by presenting work support information, comprising:
preparing a movable camera for photographing the meter device, a display for displaying an image photographed by the camera, and a speaker for outputting sound;
setting the conditions for presenting the work support information variably according to a plurality of stages of recognition states from the start of photographing by the camera until the photographed image is recognized so that the indication value of the meter device can be read;
When photographing with the camera, the work support information is output from one or both of the display and the speaker, and the output of the work support information is changed according to the recognition state. Reading work support method.

Images