JP2022082140A - Instrument reading system and instrument reading program - Google Patents

Abstract

To set information necessary for reading an instrument accurately and in a short time by utilizing a mobile device such as a smartphone and a tablet terminal.SOLUTION: In an instrument reading system 1000 that acquires an image of an instrument and displays a result of reading a numerical value indicated by the instrument, setting means comprises: region candidate detecting means for detecting a candidate for a region including a scale in the image of the instrument; position candidate detecting means for detecting a candidate for a scale position with respect to each candidate in the region including the detected scale; input means for inputting information on a constraint condition for selecting an optimum candidate from the candidate for the region including the scale and the candidate for the scale position; ranking means for ranking the candidate for the region including the scale and the candidate for the scale position based on the candidate for the region including the scale and the candidate for the scale position and the input constraint condition; and display means for displaying a candidate for a region including an optimum scale and the candidate for the scale position based on a ranking result.SELECTED DRAWING: Figure 1

Description

The present invention is an instrument reading system that supports the work of acquiring and recording the indicated value of an instrument in inspection, maintenance, etc., targeting a type of instrument that expresses the indicated value by a guideline installed in a facility such as a plant or factory. , It is about the instrument reading program.

As one of the inspection work and maintenance work in a plant or factory, there is a work of acquiring and recording the indicated value of the instrument installed in the facility. In addition, among the instruments installed in the facility, there is a type of instrument (pointer type instrument) that expresses the indicated value by a pointer. As a technology to support the reading work of the pointer type instrument, the image of the instrument is taken with the camera installed in the smartphone or tablet terminal or the camera connected to the personal computer, and the indicated value of the instrument is automatically calculated from the taken image. There is a technique for reading in (for example, Patent Document 1).

Patent Document 1 presents a technique in which a user sets information necessary for reading an indicated value in advance and reads the indicated value using the set information. The image of the instrument taken by the camera is displayed on the screen, and the user can set the information necessary for reading on the image of the instrument. The information to be set is the fulcrum position of the pointer, the movable area of the needle, and the area including the scale. The figure that supports these settings is superimposed and displayed on the image of the instrument, and the user moves the figure. The setting work is made more efficient by making it possible to set by transforming.

Japanese Unexamined Patent Publication No. 2006-12133

When introducing an instrument reading system, it is usually considered to confirm in advance whether reading is possible at the actual site where inspection work and maintenance work are performed. Therefore, it is realistic to use a technique such as Patent Document 1 for reading by setting information necessary for reading in advance for each instrument.

However, at the site where inspection work and maintenance work are carried out, the equipment that can be used and the range of activities are limited due to the arrangement of equipment and safety, so for example, setting work can be performed efficiently only with smartphones and tablet terminals. desirable.

Patent Document 1 presents a method of setting a center position and a scale position on which a pointer rotates by being specified by a user on a screen on which an instrument is photographed. However, in order to improve the reading accuracy, it is necessary to specify more accurate positions as the center position and the scale position. It is easy to specify the exact position on the image in an environment where a commonly used personal computer and mouse can be used. However, when the means that can be used is a smartphone or a tablet terminal, the screen is small and the touch operation is performed by a finger, which makes it difficult to specify an accurate position on the image and increases the time required for setting. Therefore, it is considered that a means for more accurately setting the information necessary for reading the instrument is required even on the screen of the smartphone or tablet terminal.

From the above, an object of the present invention is the information required for reading the instrument accurately and in a short time even when a mobile device such as a smartphone or a tablet terminal is used to set the information required for reading the instrument. Is to provide the technology that makes it possible to set.

The instrument reading system according to one aspect of the present invention includes an acquisition means for acquiring an image of the instrument, a setting means for setting instrument information necessary for reading a numerical value indicated by the instrument, and a set means of the instrument. An instrument reading system comprising a reading means for reading a numerical value indicated by the instrument from an image of the instrument based on information and a display means for displaying the result of reading the numerical value indicated by the instrument, wherein the setting means is the said. The area candidate detecting means for detecting the candidate of the area including the scale in the image of the instrument, the position candidate detecting means for detecting the candidate of the scale position for each candidate of the area including the detected scale, and the scale are included. An input means for inputting information on a constraint condition for selecting an optimum candidate from a region candidate and a scale position candidate, a region candidate including the scale, a scale position candidate, and the input constraint. A ranking means for ranking the candidate of the region including the scale and the candidate of the scale position based on the conditions, and the candidate and the scale position of the region including the optimum scale based on the result of the ranking. It is configured as an instrument reading system characterized by having a display means for displaying candidates.

According to the present invention, even when a mobile device such as a smartphone or a tablet terminal is used to set the information required for reading the instrument, it is possible to set the information required for reading the instrument accurately and in a short time. It will be possible.

The block diagram of the computer when the first embodiment is executed by a general computer. The figure which shows an example of the usage situation assumed in the instrument reading system. An example of a diagram showing a type of instrument in which the scales are arranged in a circle and the pointer rotates around the center of the circle and the information necessary for reading the indicated value. An example of the format of instrument information data. An example of the format of instrument setting data. An example of the format of instrument reading result data. An example of a flowchart of processing executed by the instrument setting program. The figure which shows an example of the design of a scale line. The figure which shows another example of the design of a scale line. An example of a circle candidate detected for the image of the instrument of FIG. An example of a circle candidate detected for the image of the instrument of FIG. An example of an image of the scale area extracted from the image of the instrument of FIG. An example of an image of the scale area extracted from the image of the instrument of FIG. An example of an instruction screen for the user to acquire the information specified by the user. The figure which shows an example of the state which the user has specified necessary information. An example of a screen that displays ranked circle candidates and scale position candidates. An example of the screen when the selection of the circle candidate is changed in FIG. An example of a screen for fine-tuning the position of a circle candidate. An example of a flow chart of the processing executed by the instrument reading program. An example of a diagram showing a type of instrument in which the scales are arranged in a straight line and the pointer moves linearly in parallel with the straight line, and information necessary for reading the indicated value. An example of the format of the instrument setting data in the second embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are appropriately omitted and simplified for the sake of clarification of the description. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

The positions, sizes, shapes, ranges, etc. of each component shown in the drawings may not represent the actual positions, sizes, shapes, ranges, etc., in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the position, size, shape, range and the like disclosed in the drawings.

In the following description, various information may be described by expressions such as "table" and "list", but various information may be expressed by a data structure other than these. The "XX table", "XX list", etc. may be referred to as "XX information" to show that they do not depend on the data structure. When expressions such as "identification information", "identifier", "name", "ID", and "number" are used in explaining the identification information, these can be replaced with each other.

When there are a plurality of components having the same or similar functions, they may be described by adding different subscripts to the same reference numerals. However, if it is not necessary to distinguish between these plurality of components, the subscripts may be omitted for explanation.

Further, in the following description, a process performed by executing a program may be described, but the program is determined by being executed by a processor (for example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit)). In order to perform the processed processing while appropriately using storage resources (for example, memory) and / or interface devices (for example, communication port), the main body of the processing may be a processor. Similarly, the main body of the process of executing the program may be a controller, an apparatus, a system, a computer, or a node having a processor. The main body of the processing performed by executing the program may be any arithmetic unit, and may include a dedicated circuit (for example, FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)) for performing specific processing. ..

The program may be installed from a program source into a device such as a calculator. The program source may be, for example, a program distribution server or a computer-readable storage medium. When the program source is a program distribution server, the program distribution server includes a processor and a storage resource for storing the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Further, in the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

In the following, in the instrument reading technology that sets the information required for reading the instrument in advance and reads the indicated value of the instrument based on the set information, the information required for reading the instrument is set for smartphones, tablet terminals, etc. Even when using a mobile device, it is possible to set the information required for reading the instrument accurately and in a short time.

The first embodiment will be described with reference to FIGS. 1 to 19.

FIG. 1 shows a configuration diagram of a computer when the instrument reading system 1000 in the first embodiment is executed by a general computer. As shown in FIG. 2, the instrument reading system 1000 in the present embodiment assumes a case where the instrument 203 is photographed by the terminal 202 held by the user 201 and the terminal 202 reads the indicated value of the instrument 203. is doing. Various information required for reading the indicated value is also set by acquiring the image of the instrument 203 on the terminal 202 and operating the image of the instrument 203 on the screen of the terminal 202. The terminal 202 is a well-known mobile terminal such as a smartphone, a tablet terminal, or a notebook PC, and the component included in FIG. 1 can be included in the terminal 202. Alternatively, it is possible to store a part of data or a program in a server and execute necessary processing via a network. Further, this embodiment is also applicable to a system in which a camera for photographing the instrument 203 is fixedly installed.

The program stored in the memory 104 is stored in a portable non-temporary recording medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), or a USB (Universal Serial Bus) memory, and is installed in a computer. May be good. The function of the instrument reading system 1000 is that the program stored in the memory 104 is executed by the CPU 101, so that the defined processing is realized in cooperation with the hardware. A program executed by the instrument reading system 1000, a function thereof, or a means for realizing the function may be referred to as a "function", a "means", a "part", a "unit", a "module", or the like.

The CPU 101 in FIG. 1 is an information processing device for executing each program in the instrument reading system 1000. The input I / F (interface) 102 includes a camera for photographing the instrument and an input device in a general computer such as a keyboard, a button, a mouse or a touch panel for controlling the start and end of the system. .. The output I / F (interface) 103 is a means for displaying the operating status of the system to the user, and includes a screen of a smartphone or a tablet terminal, or a display device for a general computer. Further, the memory 104 is a storage medium for storing each program in the instrument reading system 1000. The memory 104 includes an instrument setting program 105 and an instrument reading program 106.

In the first embodiment, among the types of instruments whose indicated values are represented by the pointer, the scales are arranged in a circle as shown in the instrument 301 of FIG. 3, and the pointer rotates around the center of the circle as a fulcrum. Assume a type of instrument to be read. The basic information required to read the type of instrument shown in the instrument 301 of FIG. 3 is the circle formed by the scale, the position of the minimum value of the scale, and the position of the maximum value. In FIG. 3, the center position 302 of the circle formed by the scale, the radius 303 of the circle formed by the scale, the minimum value position 304 of the scale, and the maximum value position 305 of the scale are basic information necessary for reading. In the following, as an example, the position of the minimum value and the position of the maximum value of the scale are described as basic information necessary for reading. However, it is not limited to the position of the minimum value and the position of the maximum value of the scale. It may be basic information. For example, if it is known from experience or control that the indicated value indicated by the pointer changes within the range of 80% of the minimum and maximum values, the position of the lower limit value and the position of the upper limit value in the range are known. Should be the basic information required for reading. In this way, the user of this system can arbitrarily set the reading position and the range instead of the above-mentioned minimum value position and maximum value position.

As shown in FIG. 3, for example, the center position 302 of the circle can be represented by the position coordinates with the lower left of the image as the origin. The origin position can be set arbitrarily. Further, the minimum value position 304 of the scale and the maximum value position 305 of the scale can be represented by a relative angle with respect to a line drawn directly below the center position 302 of the circle, for example, as shown in FIG. .. The reference line can also be set arbitrarily. In order for the user to efficiently specify this information, for example, only the minimum value position 304 of the scale and the maximum value position 305 of the scale need to be specified. If only two points are specified, two types of circles are usually required as candidates, but in this embodiment, it is possible to use a constraint that the entire instrument is photographed so as to be reflected in the image as large as possible. .. By using this constraint, it is possible to uniquely identify the circle formed by the scale by using only the minimum value position 304 of the scale and the maximum value position 305 of the scale. Further, the designated information may be used as it is for the minimum value position 304 of the scale and the maximum value position 305 of the scale.

Further, in the type of instrument shown in the instrument 301 of FIG. 3, the minimum value position and the maximum value position are axisymmetric with respect to the center line of the instrument (for example, a vertical line or a horizontal line passing through the center position 302 of the circle). When it is in the position, the circle formed by the scale can be uniquely specified by designating either the minimum value position 304 of the scale or the maximum value position 305 of the scale and the center position 302 of the circle. The above-mentioned information is basic information necessary for reading the instrument, but other information can be added. For example, the positions of the scales other than the minimum value and the maximum value may be added. Further, although the instrument shown in FIG. 3 is in a state of being photographed without tilting, when actually photographing the instrument, it is common that the instrument is photographed at an angle. Therefore, in addition to the information shown above, information regarding the inclination may be added.

In the instrument information data 107, the type of information required for reading the instrument and the type of information specified by the user when setting the instrument are registered. FIG. 4 shows an example of the format of the instrument information data 107. The instrument information ID 401 is an identifier representing the type of instrument information, and any numeric string or character string can be used as long as it is the only identifier in the instrument information data 107. The setting information list 402 lists the types of information necessary for reading the indicated value of the above-mentioned instrument by a character string or the like. For the type of instrument shown in FIG. 3, a character string such as "center position of circle, radius of circle, minimum value position, maximum value position" is described. The "center position of the circle" is the center position 302 of the circle shown in FIG. 3, the "radius of the circle" is the radius 303 of the circle shown in FIG. 3, and the "minimum value position" is the minimum value position of the scale shown in FIG. It is assumed that 304 and the "maximum value position" correspond to the maximum value position 305 of the scale shown in FIG. 3, respectively.

The input information list 403 lists the types of information that the user manually inputs when setting the instrument among the information listed in the setting information list 402. For example, when it is assumed that the user inputs the minimum value position 304 and the maximum value position 305 of the scale for the instrument shown in FIG. 3, the input information list 403 for the above-mentioned example of the setting information list 402 is "minimum". Describe a character string such as "value position, maximum value position". Alternatively, when the user is supposed to input the center position 302 of the circle formed by the scale and the minimum value position 304 of the scale, a character string such as "center position, minimum value position" is described. As described above, the types and combinations of information assumed to be input by the user can be arbitrarily set as long as they are within the range of the types of information included in the setting information list 402.

The instrument setting data 108 stores information necessary for reading the indicated value of each instrument set by the instrument setting program 105. FIG. 5 shows an example of a format for storing the setting information of the instrument shown in FIG. In FIG. 5, the instrument name 501 is a name given to each instrument to be read, and any character string can be used as long as it is the only character string in the instrument setting data 108. The instrument information ID 502 is an identifier representing the type of instrument information, and any one of the instrument information ID 401 in the instrument information data 107 is described. The template image 503 is an image taken by the instrument when setting the information necessary for reading the target instrument. The center position 504 of the circle, the radius 505 of the circle, the minimum value position 506, and the maximum value position 507 are the center position 302 of the circle formed by the scale in FIG. 3, the radius 303 of the circle formed by the scale, and the minimum value position 304 of the scale. And the data corresponding to the maximum value position 305 of the scale is described. Further, the minimum value 508 and the maximum value 509 describe the indicated value of the instrument when the pointer indicates the minimum value position 506 and the maximum value position 507.

The above-mentioned character strings indicating the types of information required for reading are registered in the instrument setting program 105 and the instrument reading program 106, and the setting method and reading method can be determined by combining the types of information required for reading. Make a selection and run it. When it becomes necessary to target an instrument of a type other than expected, the definition of a new type of information required for reading, the registration of a combination of new information in the instrument information data 107, and the instrument setting program 105. A new type of instrument can be set and read by adding necessary information, setting method, and reading method to the instrument reading program 106 as needed.

The instrument reading result data 109 stores the indicated value of the instrument read by the instrument reading program 106. FIG. 6 shows an example of a format for storing the indicated value of the read instrument. In FIG. 6, the instrument name 601 is the name of the instrument corresponding to the read indicated value, and the same name as the instrument name 501 in FIG. 5 is described. In the date and time 602, a numerical value or a character string indicating the date and time when the indicated value was read is described. Reference numeral 603 is a numerical value or a character string representing the read indicated value.

The instrument setting program 105 is a program for setting information necessary for reading the instrument and storing the set information in the instrument setting data 108. FIG. 7 shows an example of a flowchart of the process executed by the instrument setting program 105. The flowchart shown in FIG. 7 is a process for the type of instrument shown in FIG.

As will be described below, in Example 1 and Example 2 described later, the types of instruments shown in the image of the instrument (for example, the image of the circular instrument in FIG. 3 and the image of the rectangular instrument in FIG. 20) are used. An acquisition means for acquisition (for example, a camera which is an input device 102), a setting means for setting instrument information necessary for reading a numerical value indicated by the instrument (for example, an instrument setting program 105), and a set instrument. A reading means for reading the numerical value indicated by the instrument from the image of the instrument based on the information of (for example, the instrument reading program 106) and a display means for displaying the result of reading the numerical value indicated by the instrument (for example, a display device which is an output device 103). ), And the setting means is a region candidate detecting means (for example, a circle candidate in FIG. 3 or a rectangular candidate in FIG. 20) for detecting a region candidate including a scale in an image of the instrument. , The instrument reading program 106) that executes S704 to S707 of FIG. 7, and the position candidate detecting means (for example, S708 of FIG. 7) that detects the candidate of the scale position for each candidate of the region including the detected scale. Input the information regarding the constraint condition for selecting the optimum candidate from the instrument reading program 106) to be executed and the candidate of the area including the scale and the candidate of the scale position (for example, the information specified by S709 in FIG. 7). Based on the input means (for example, the instrument reading program 106 that executes S709 of FIG. 7, the candidate of the area including the scale and the candidate of the scale position, and the input constraint condition, the candidate and the scale of the area including the scale). A ranking means for ranking the position candidates (for example, an instrument reading program 106 that executes S710 in FIG. 7), and a region candidate including an optimum scale and a scale position candidate based on the ranking result. It functions as an instrument reading system characterized by having a display means for displaying (for example, an instrument reading program 106 that executes S711 of FIG. 7).

In step 701 in FIG. 7, the instrument setting program 105 acquires information about the instrument to be read from the instrument information data 107. Specifically, the instrument setting program 105 acquires a list of instrument information ID 401 from the instrument information data 107, displays the list on the screen of the terminal 202, and gives the user an instrument information ID corresponding to the target instrument. Instruct to select. The instrument setting program 105 acquires instrument information corresponding to the instrument information ID selected by the user from the list from the instrument information data 107.

In step 702, the instrument setting program 105 takes an image of the instrument to be read by using the camera installed in the terminal 202. This image is stored as the template image 503 of the instrument setting data 108.

In step 703, the instrument setting program 105 extracts contour lines (edges) from the captured image of the instrument. The contour line can be easily extracted by using a generally well-known method, for example, the Canny method. After extracting the contour line in step 703, the process branches to the process of step 704 and the process of step 705. This branch is intended to accommodate different design scales.

When the types of instruments shown in FIG. 3 are classified by the scale design, they are roughly classified into the types of instruments shown in FIG. 8 and the types of instruments shown in FIG. The type of instrument shown in FIG. 8 is a design 801 in which each scale line is in contact with an arc line that is concentric with a circle composed of scale lines. The arc line 811 may be a line drawn in the same manner as the scale line, but the frame of the instrument may play a similar role. In this type of instrument, the arc and each scale line are connected even in the extraction result of the contour line. On the other hand, the type of instrument shown in FIG. 9 has a design 901 in which each scale line is an independent short line, and an independent contour line is extracted for each scale line also in the extraction of the contour line. In order to flexibly correspond to the design of these two types of scales, in the flowchart shown in FIG. 7, the instrument setting program 105 performs the processing corresponding to the type of scale lines shown in FIG. 8 in step 704, and the process corresponding to the scale lines of the type shown in FIG. Performs processing corresponding to the types of scale lines shown in. FIG. 7 illustrates a case where the process of step 704 and the process of step 705 and subsequent steps are executed, but either process may be selectively executed.

In step 704, the instrument setting program 105 detects the circle candidate using all the contour lines extracted in step 703. As a method for detecting a circle candidate, a generally well-known method, for example, a Hough transform or the like, can be used to easily detect the circle candidate. The instrument setting program 105 detects circle candidates for the contour lines extracted from the image of the instrument shown in FIG. 8, for example, circles as shown in circle candidates 1001, circle candidates 1002, and circle candidates 1003 in FIG. Candidates can be extracted. In FIG. 10, the circle candidate 1001 is a candidate detected for the arc line tangent to the scale line, and the circle candidate 1002 and the circle candidate 1003 are the lines represented by the frame of the instrument detected as the circle candidate. Further, instead of detecting the circle candidate using all the extracted contour lines, the instrument setting program 105 forms a group of contour lines in which the extracted contour lines are connected (for example, 1 constituting the circle shown in FIG. 10). Alternatively, it may be divided into a small area including a plurality of broken lines), and the circle candidate may be detected only for the contour line whose group of the divided contour lines is longer than a predetermined length.

On the other hand, in step 705, the instrument setting program 105 divides the extracted contour lines into a group of connected contour lines (for example, a small area corresponding to the scale line constituting the circle shown in FIG. 11) and is divided. The area occupied by a group of contour lines (for example, the area of a small area corresponding to the scale line shown in FIG. 11) is obtained. The instrument setting program 105 leaves only a set of contour lines whose area is smaller than the threshold value assuming the area of the scale line in advance. By this process, the instrument setting program 105 can extract only the contour line related to the scale line from the image of the instrument of the type shown in FIG.

Next, in step 706, the instrument setting program 105 obtains the position of the center of gravity of each contour line for each group of contour lines left by step 705. Further, in step 707, the instrument setting program 105 detects the circle candidate for the obtained center of gravity position by using the same method as that used in step 704. By the processing of steps 705 to 707, as shown in the circle candidate 1101 of FIG. 11, the circle candidate composed of independent scale lines can be detected.

Of the circle candidates detected in steps 704 and 705 to 707, the candidate detected for the circle composed of the scales touches all the scales like the circle candidate 1001 or the circle candidate 1101. It becomes a shape that intersects all the scales like. In this case, the donut-shaped area sandwiched between the circles that are slightly larger than the radius and the circles that are smaller than the radius based on the radius of the circle candidates is the area including the scale, so each circle candidate has a scale. Candidate for the area.

As described above, the region candidate detecting means (for example, the instrument reading program 106 that executes S704 to S707 in FIG. 7) identifies the shape representing the region including the scale from the contour information extracted from the image of the instrument. , A process of detecting a candidate for a region including a scale from the image of the instrument (for example, S704 in FIG. 7), and a small region corresponding to each scale is extracted from the image of the instrument, and the extracted plurality of small regions are At least of the process of detecting a candidate for the region including the scale from the image of the instrument (for example, S705 to S707 in FIG. 7) by determining whether the constituent shape is a shape representing the region including the scale. Perform one of the processes.

In step 708, the instrument setting program 105 detects the scale position candidate for each circle candidate detected by step 704 and steps 705 to 707. In order to detect the scale position candidate, first, based on the center position of the circle candidate, polar coordinate conversion is performed on the instrument image as shown in FIGS. 8 and 9, and the image of the scale area is converted into a rectangular image. Convert to. Specifically, the instrument setting program 105 assumes a straight line extending in the circumferential direction from the center position of the circle candidate, and acquires the pixel value on the straight line. The instrument setting program 105 can convert a circular image into a rectangular image by connecting the acquired pixels while changing the angle of the straight line little by little with the center position as the axis. At this time, when the instrument setting program 105 acquires only the pixel values in a preset range centered on the radius of the circle candidate, and the circle candidate matches the circle formed by the scale, FIG. 12A. And as shown in FIG. 13, it is possible to acquire an image in which only the scale area is deformed into a rectangular shape. FIG. 12 assumes an image of the scale area extracted from the image shown in FIG. 8, and FIG. 13 assumes an image of the scale area extracted from the image shown in FIG. Further, in FIGS. 12 and 13, the images are based on the assumption that only the scale lines are extracted, but in reality, some numbers and other patterns may be included.

Further, in step 708, the instrument setting program 105 performs a process for leaving only the scale line on the image in the scale area. Specifically, in the case of images as shown in FIGS. 12 and 13, the instrument setting program 105 performs a process of leaving only vertical lines. This method can be easily performed by using a generally well-known morphology conversion or the like. By performing the process of leaving only the lines in the vertical direction, the image shown in FIG. 12 is also converted into the same image as in FIG. The instrument setting program 105 acquires the lateral position of each scale line as a scale position candidate in the case of an image as shown in FIG. 13, for example, with respect to an image having only scale lines.

In step 709, the instrument setting program 105 acquires information specified by the user when setting the instrument. Therefore, the instrument setting program 105 displays an instruction screen as shown in FIG. 14 on the screen of the terminal 202. This screen is the screen displayed after taking a picture of the instrument in step 702. When the screen of the terminal 202 has a touch panel, necessary information can be specified by touching the corresponding position on the image 1401 of the instrument with a finger. The type of information that the user needs to set is determined by the content of the input information list 403 in the instrument information data 107 acquired in step 701. The instrument setting program 105 associates the information specified by the user with the information in the setting information list in order according to the contents of the input information list 403.

FIG. 15 shows an example of a state in which the user has specified necessary information. FIG. 15 shows an example in which the information specified by the user is the minimum value position 304 of the scale and the maximum value position 305 of the scale, the minimum value position 1501 is set, and the maximum value position 1502 is set. There is. The position specified by the user at this time does not have to be an accurate position. Further, in FIG. 14, the button 1402 is a button for returning to the immediately preceding screen, and the instrument setting program 105 may change the screen and return to the immediately preceding screen when the user accepts a selection by a mouse, a screen touch, or the like. can. Similarly on the subsequent screens, the instrument setting program 105 transitions to the immediately preceding screen when this button is selected.

In step 710, the instrument setting program 105 ranks the already detected circle candidates and scale position candidates based on the information specified by the user. Both rankings can be performed based on the distance between the position specified by the user and the circle candidate or the scale position candidate. For example, regarding the ranking of circle candidates, the instrument setting program 105 finds the difference between the distance from the center position of the circle candidate to the position specified by the user and the radius of the circle candidate for each circle candidate, and the difference is larger. The smaller candidate may be the higher-ranked candidate. Regarding the ranking of the scale position candidates, the instrument setting program 105 converts the position specified by the user into the position on the image after the polar coordinate conversion described above, and selects a candidate having a smaller distance from each scale candidate. The candidate with the highest ranking may be selected.

As a method for ranking the circle candidates and the scale position candidates, any method other than these can be used as long as the ranking can be performed. Further, in the ranking of the circle candidate and the scale position candidate, a position other than the position specified by the user may be added to the constraint. For example, when it can be assumed that the target instrument is photographed near the center of the image, a circle candidate whose center position is closer to the center of the image may be selected as a candidate with a higher rank. Furthermore, it is possible to combine a plurality of conditions for ranking.

As described above, the setting means (for example, the instrument setting program 105) acquires the approximate position of the scale position from the input means (for example, the instrument reading program 106 that executes S709 in FIG. 7) and ranks the scale positions. The means (for example, the instrument reading program 106 executing S710 in FIG. 7) detects based on the distance between the candidate region including the detected scale and the position of the candidate scale position and the acquired approximate position. The candidate of the area including the scale and the candidate of the scale position are ranked.

In step 711, the instrument setting program 105 displays the circle candidate and the scale position candidate on the screen of the terminal 202 based on the result of the ranking performed in step 710. FIG. 16 shows a display example of the ranked circle candidates and scale position candidates. In FIG. 16, in the upper part 1601 of the screen, the result of superimposing the ranked circle candidates and scale position candidates on the image of the instrument acquired in step 702 is displayed. In particular, in FIG. 16, only the top three candidates are displayed for both the circle candidate and the scale position candidate, the first candidate is displayed with a thick line, and the center position of the first candidate for the circle candidate is also displayed. It is also displayed. In FIG. 16, the circle candidate is displayed as a broken line, the minimum value scale position candidate is displayed as a square, the maximum value scale position candidate is displayed as a triangle, and the center position of the circle candidate is displayed as a small broken line circle 1602 in the center of the instrument. Further, in FIG. 16, the scale position candidates are only candidates for the first circle candidate. Further, the instrument setting program 105 assigns each candidate a name according to the order, such as "yen 1", "yen 2", "yen 3", "minimum 1", "minimum 2", etc. in FIG. It is automatically assigned and displayed in association with each candidate. The display shown in FIG. 16 is an example, and the number of candidates to be displayed, the thickness of the line, the shape of the symbol, the display color, and the like can be changed as necessary. Alternatively, the user may be able to make individual adjustments according to usability and taste.

Further, in the lower part 1603 of the screen of FIG. 16, the names given to the detected circle candidates and the scale position candidates are listed in order from the top candidate. The circles displayed on the left side of the candidates indicate the candidates for which the black circle is selected, and the white circles indicate the candidates for which the black circle is not selected. At the bottom 1603 of the screen, it is shown that the first-ranked candidate is selected. Only the top three candidates are displayed in the lower 1603 of the screen, but the number of candidates to be displayed may be arbitrarily changed. For example, the instrument setting program 105 may display more candidates than the candidates displayed in the upper part 1601 of the screen.

In step 712, the instrument setting program 105 acquires the result of the operation performed by the user with respect to the content displayed in step 711, and determines in step 713 that the determination button 1604 of FIG. 16 is selected (step). 713; Yes), proceed to step 715. In step 713, when the instrument setting program 105 determines that the user's operation is not the selection of the enter button 1604 (step 713; No), it determines that it is another candidate selection or fine adjustment, and is necessary in step 714. Change the screen display, return to step 712 again, and acquire the user's operation. As the user's operation, for example, it is possible to assume that the selection of the circle candidate or the scale candidate is changed, the position of the circle candidate or the position of the scale candidate is finely adjusted, and the like. If there are other necessary operations, they can be added as appropriate.

As described above, the setting means (for example, the instrument setting program 105) displays a predetermined number of candidates from the candidates having a higher rank among the candidates of the area including the scale on the image of the instrument. Among the candidates for the scale position corresponding to each candidate in the area including the scale, a predetermined number of candidates from the candidates having the highest rank are superimposed and displayed on the image of the instrument.

Further, the setting means (for example, the instrument setting program 105) is displayed at a selection means (for example, 1603 at the bottom of the screen of FIG. 16) for selecting a candidate for a region including a scale displayed on an image or a candidate for a scale position. It has an instrument setting program 105) that accepts selection of circle candidates, maximum value candidates, and minimum value candidates, and can select area candidates and scale position candidates that include the selected scale, and region candidates and scales that include other scales. One or more of the processing of displaying in a form different from the position candidate and the processing of displaying only the scale position candidate for the candidate of the area including the selected scale on the image of the instrument is performed to display the scale. Candidates for the included area and candidates for the scale position can be displayed on the screen of the instrument to improve visibility.

FIG. 17 shows an example of a screen when the selection of the circle candidate is changed in FIG. The circle candidate is selected by selecting the name of the corresponding candidate and the circle mark displayed next to it in the list of candidates displayed at the lower part 1603 of the screen in FIG. 16 by using the mouse, touching the screen, or the like. be able to. Alternatively, the candidate displayed on the image of the instrument at the upper part 1601 of the screen in FIG. 16 may be directly selected. FIG. 17 is a display example when the circle candidate is changed from "yen 1" to "yen 3", and it is shown that "yen 3" is selected by displaying "yen 3" with a thick broken line. Further, a scale position candidate for "Circle 3" is displayed. The instrument setting program 105 can change the selection of the scale position candidate only for the scale position candidate related to the selected circle candidate. In the case of FIG. 17, it is possible to select one of the scale position candidates for the “circle 3”, “minimum 1” to “minimum 3” and “maximum 1” to “maximum 3”.

As described above, the setting means (for example, the instrument setting program 105) displays a list of candidates for the area including the scale and candidates for the scale position, and the candidate or the scale position of the area including the scale selected in the list. The candidates corresponding to the candidates of are displayed on the image of the instrument in a form different from the candidates of the area including other scales or the candidates of the scale position, and the contents selected in the lower part 1603 of the screen are reflected in the upper part 1601 of the screen. Can be done.

Further, FIG. 18 is an example of a screen when finely adjusting the position of the circle candidate. The screen of FIG. 18 is a screen displayed by selecting a button described as “fine adjustment” arranged at the lower part 1702 of the screen of FIG. 17 with a mouse, touching the screen, or the like, and is selected in FIG. It is possible to correct the position of only the circle candidate. By sliding the finger on the screen as shown in the slide direction 1801 of the finger on the screen of FIG. 18, the instrument setting program 105 detects this and adjusts to the movement of the finger as shown in the slide direction 1802 of the circle candidate. The position of the target circle candidate can be corrected. You can also use the mouse to perform similar operations. Further, in FIG. 18, by selecting the adjustment radio button 1803 with a mouse, touching the screen, or the like, the instrument setting program 105 can detect this and fine-tune the size of the circle candidate. As a method of changing the size of the circle candidate, a method of changing the size according to the slide of the finger or the movement of the mouse can be used as in the case of the position of the circle candidate. The position of the scale position candidate can be finely adjusted in the same manner. In the case of the scale position candidate, the range in which fine adjustment is possible may be limited to the circumference determined by the circle candidate.

As described above, the setting means (for example, the instrument setting program 105) can adjust the position of the selected candidate instrument among the candidate of the region including the scale and the candidate of the scale position on the image.

In step 715, the instrument setting program 105 acquires information necessary for reading the instrument from the selected circle candidate and scale position candidate, and stores the information in the instrument setting data 108. This makes it possible to set more accurate information even if the position specified by the user is not accurate. Further, the instrument setting program 105 acquires the indicated value of the instrument for the selected scale position candidate, stores it in the instrument setting data 108, and ends the process. It is assumed that the user inputs the indicated value of the instrument for the scale position candidate, and the instrument setting program 105 displays an instruction on the screen of the terminal 202 to input the indicated value for each scale position candidate. , The result input by the user is saved in the instrument setting data 108.

In the above-mentioned method for setting the information necessary for reading the indicated value of the instrument, after acquiring all the information specified by the user in step 709, the circle candidate and the scale position candidate are ranked in step 710. The result was displayed in step 711. Regarding the processing flow from step 709 to step 711, in addition to the method described above, each time the user sets one piece of information, the instrument setting program 105 sets the circle candidate and the scale candidate based on the set information. Ranking may be performed and the result may be displayed.

Further, when only one of the minimum value position 506 and the maximum value position 507 is set as the information set by the user, the instrument setting program 105 is added to these positions from the center position of the circle candidate. The position that is line-symmetrical to the straight line drawn directly below or the position that is line-symmetrical to the straight line drawn horizontally from the center position of the circle candidate is also added to the ranking criteria of the circle candidate and the scale position candidate. , Circle candidates and scale position candidates may be ranked. This is because, in an instrument in which the scale lines are arranged in a circular shape, the minimum value position and the maximum value position are often line-symmetrical with respect to the center line of the instrument. The position that is line-symmetrical to the straight line drawn vertically or horizontally from the center position of the circle candidate with respect to the set position is automatically used for ranking the circle candidate and the scale position candidate, and the user can use it. Each time one piece of information is set, the circle candidate and the scale candidate are ranked based on the set information, and the result is displayed. That is, each time the setting means (for example, the instrument setting program 105) acquires one piece of information as a constraint condition from the input means (for example, the instrument reading program 106 that executes S709 in FIG. 7), the scale is set. Update the ranking of the candidate area and the candidate scale position, and change the display of the candidate area including the scale and the candidate scale position on the image of the instrument. This makes it possible to set all the information necessary for reading the instrument by setting only one piece of information at the shortest.

As described above, when the shape of the region including the scale is circular, the setting means (for example, the instrument setting program 105) has (1) from the input means (for example, the instrument reading program 106 that executes S709 in FIG. 7). ) Approximate position of the scale position indicating the minimum and maximum values, (2) Approximate position of the center position of the circle and the approximate position of the scale position indicating the minimum value, (3) Approximate position of the center position of the circle The approximate position of the scale position indicating the maximum value, (4) the approximate position of the scale position indicating the minimum value and the center position of the instrument image, (5) the approximate position of the scale position indicating the maximum value and the image of the instrument. The ranking means (for example, the instrument reading program 106 that executes S710 in FIG. 7) obtains the condition of any of the center positions, and the candidate of the region including the detected scale and the above (1) to (1) to ( Based on any of the conditions of 5), the candidate of the region including the scale and the candidate of the scale position may be ranked.

Further, when the setting means (for example, the instrument setting program 105) is an instrument whose scale position is axisymmetric with respect to a specific line, the input means (for example, the instrument reading program 106 that executes S709 in FIG. 7). ), Either the approximate position of the scale position indicating the minimum value or the approximate position of the scale position indicating the maximum value is acquired, and the above-mentioned ranking means (for example, S710 in FIG. 7) is acquired. The instrument reading program 106) that executes the program includes the detected graduations based on the distance between the candidate region including the detected graduations and the position of the candidate graduation position and one of the acquired approximate positions. You may rank the area candidate and the scale position candidate.

The instrument reading program 106 in FIG. 1 reads the indicated value of the instrument from the image taken by the instrument based on the instrument setting data 108 saved by the instrument setting program 105, and reads the read indicated value as the instrument reading result data 109. It is a program for storing in. FIG. 19 shows an example of a flowchart of processing executed by the instrument reading program 106.

In step 1901 in FIG. 19, the instrument reading program 106 acquires setting information regarding the instrument to be read from the instrument setting data 108. Specifically, the instrument reading program 106 acquires a list of instrument names 501 from the instrument setting data 108, displays the list on the screen of the terminal 202, and selects the instrument name corresponding to the target instrument for the user. Instruct to do so. The instrument reading program 106 reads the setting information corresponding to the instrument name selected from the list by the user from the instrument setting data 108.

In step 1902, the instrument reading program 106 uses a camera installed in the terminal 202 to capture an image of the instrument to be read. In step 1901 and step 1902 described above, the flow is such that the user specifies the name of the instrument to be read and then the image of the target instrument is taken. When a symbol or character string is installed around the target instrument and the instrument is photographed, the image pattern, symbol, or character string is photographed together with the target instrument to recognize the image pattern, symbol, or character string. A method of obtaining the name of the instrument can be used. For example, as an image pattern in which the name of the instrument is registered, a well-known QR code (registered trademark) or an image pattern called an AR marker can be used. In addition, when using a symbol or character string representing the name of the instrument, it is possible to obtain the name of the instrument from the captured image by using a well-known character recognition technique.

The template image 503 of the instrument setting data 108 and the image of the instrument taken in step 1902 are both images of the instrument to be read, but the position and orientation of the cameras at the time of shooting are not the same. The images have different positions, orientations, and sizes of the instruments on the image. In this state, it is difficult to read the indicated value of the instrument using the information stored in the instrument setting data 108. Therefore, in step 1903, the instrument reading program 106 performs a process of matching the position, orientation, and size of the instrument on the image taken in step 1902 with the instrument on the template image. In this method, the instrument reading program 106, for example, extracts information called local feature points and information called local feature quantities associated with the well-known images from the template image 503 and the image of the instrument taken in step 1902. do.

Next, the instrument reading program 106 obtains the correspondence between the local feature points extracted from the template image 503 based on the similarity of the local feature quantities and the local feature points extracted from the instrument image taken in step 1902. As a method for finding a correspondence, a well-known round-robin method or the like can be used. The instrument reading program 106 obtains the correspondence of the local feature points, and then obtains a transformation matrix for projecting the image of the instrument taken in step 1902 onto the template image 503 using the obtained correspondence. This transformation matrix is a well-known matrix called the homography matrix.

Finally, the instrument reading program 106 performs a process of projecting the image of the instrument taken in step 1902 onto the template image 503 by using the obtained transformation matrix. As a result, it is possible to acquire an image in which the portion of the instrument in the image taken in step 1902 is fitted to the portion of the instrument on the template image 503, that is, an image in which the position, orientation, and size of the instrument are matched. As a method of matching the instrument on the template image 503 with the instrument on the image taken in step 1902, any method may be used as long as the method can obtain the same result.

In step 1904, the instrument reading program 106 reads the indicated value of the instrument using the image of the instrument obtained in step 1903 and the information of the instrument setting data 108. Since the position, orientation, and size of the instrument on the image obtained in step 1903 match the position, orientation, and size of the instrument on the template image, the instrument reading program 106 may, for example,: The indicated value is read by the method. The instrument reading program 106 first extracts the pointer from the image by extracting the same region as the color of the pointer from the image of the instrument obtained in step 1903.

Further, the instrument reading program 106 obtains the contour line (edge) of the extracted pointer and extracts a straight line from the obtained contour line. As a method for obtaining the contour line, for example, the above-mentioned Canny method can be used. As a method for extracting a straight line, for example, the above-mentioned Hough transform can be used. In addition, any method may be used as long as the same result can be obtained. The instrument reading program 106 can acquire the tip position of the pointer by finding the position where the two longest straight lines of the extracted straight lines intersect. Alternatively, the contour line is directly obtained from the image of the instrument obtained in step 1903, and the tip position of the pointer is obtained by finding the position where the two longest straight lines of the straight lines extracted from the contour line intersect. You may do so. In the extraction of pointers and contour lines, unnecessary areas and contours are defined by limiting the processing range to the range of the circle determined by the center position 504 of the circle and the radius 505 of the circle in the instrument setting data 108. It is possible to suppress the extraction of lines.

The instrument reading program 106 obtains the tip position of the pointer in the image, and then converts the tip position of the pointer into a value by the same calculation method as the values of the minimum value position 506 and the maximum value position 507. As the calculation method to be used, for example, a method of obtaining an angle formed by a straight line directly below the center position 504 of the circle and a straight line extending from the center position 504 of the circle to the target position can be used. Using the tip position of the pointer obtained by this calculation method, the minimum value position 506, the maximum value position 507, the minimum value 508, and the maximum value 509 in the instrument setting data 108, the indicated value of the instrument is calculated by (Equation 1). be able to.

M = (Vmax-Vmin) / (θmax-θmin) × P (Equation 1)
In (Equation 1), M is the indicated value of the obtained instrument, P is the tip position of the pointer, θmin is the minimum value position 506, θmax is the maximum value position 507, Vmin is the minimum value 508, and Vmax is the maximum value 509. .. (Equation 1) is a well-known linear equation that can be used to calculate the indicated value of the instrument when the scales between the minimum value 508 and the maximum value 509 are all evenly spaced. It is a calculation formula by interpolation. If the scale spacing is uneven, information about the intermediate scale position and scale value between the minimum value 508 and the maximum value 509 is added to the instrument setting data 108, and the intermediate scale sandwiching the tip position of the pointer is inserted. The indicated value of the instrument may be calculated by performing linear interpolation using the position and the scale value. The method of detecting the tip position of the pointer and the method of calculating the indicated value of the instrument are not limited to the above-mentioned methods, and any method can be used as long as the same result can be obtained. can.

Finally, in step 1905, the instrument reading program 106 displays the indicated value of the instrument read in step 1904 on the screen of the terminal 202, stores it in the instrument reading result data 109, and ends the process.

As described above, according to the first embodiment, in an instrument reading system in which the scale is arranged in a circle and the pointer reads the indicated value of the type of instrument that rotates around the center of the circle from the image. Even when a mobile device such as a smartphone or a tablet terminal is used to set the information required for reading the instrument, it is possible to set the information required for reading the instrument accurately and in a short time.

The second embodiment will be described with reference to FIGS. 20 to 21. In the first embodiment, as shown in FIG. 3, an instrument of a type (for example, a circular shape) in which the scale is arranged in a circle and the pointer rotates around the center of the circle as a fulcrum is assumed as the instrument to be read. Was. On the other hand, in the second embodiment, as shown in the instrument 2001 of FIG. 20, a type of instrument (for example, a rectangular shape) in which the scale is arranged linearly and the pointer moves linearly in parallel with the straight line is used. Assume as an instrument to be read.

The basic information required to read the readings of the types of instruments shown in FIG. 20 is the position 2002 of the fulcrum of the pointer in FIG. 20, the position of the straight line formed by the scale 2003, and the scale. Information that identifies the area surrounded by the broken line drawn in the minimum value position 2004 and the maximum value position 2005 of the scale, and is the position 2002 of the fulcrum of the pointer (in the range where the scale operates), the position of the straight line formed by the scale 2003, The minimum value position 2004 of the scale and the maximum value position 2005 of the scale are such information. All of this information can be represented, for example, by the position coordinates with the lower left of the image as the origin. The origin position can be set arbitrarily. In order for the user to efficiently specify this information, for example, only the fulcrum position 2002, the minimum value position of the scale 2004, and the maximum value position of the scale 2005 may be specified.

In FIG. 20, the minimum value position 2004 and the maximum value position 2005 of the scale are represented only by the vertical position coordinates. However, when specifying the position on the image, it is specified by a point, that is, since both the X coordinate and the Y coordinate can be acquired, the minimum value position 2004 and the maximum value position 2005 of the scale are saved in the XY format. Assuming that, the above three types of information are the basic information necessary for reading the instrument. Further, by using a constraint that the center of the instrument is taken so as to coincide with the center of the image when the image of the instrument is taken, the instrument setting program 105 can perform the minimum value position 2004 and the maximum value position of the scale. With only one of 2005 and the fulcrum position 2002, the area required for reading the instrument in FIG. 20, that is, the fulcrum position 2002 of the pointer, the position 2003 of the straight line formed by the scale, the minimum value position 2004 of the scale, and It is possible to specify the area surrounded by the broken line drawn at the position of the maximum value position 2005 of the scale. In this case, when the instrument setting program 105 uses only the minimum value position 2004 of the scale, for example, the position in line symmetry with respect to the horizontal line passing through the center of the image in FIG. 20 may be regarded as the maximum value position 2005 of the scale. .. The above-mentioned information is basic information necessary for reading the type of instrument shown in FIG. 20, but other information can be added. For example, when the intervals between the scales are uneven, an intermediate scale position may be added between the minimum value position 2004 of the scale and the maximum value position 2005 of the scale. Further, the instrument shown in FIG. 20 is in a state of being photographed without tilt, but when actually photographing the instrument, it is general that the instrument is photographed at an angle. Therefore, in addition to the information shown above, information regarding the inclination may be added.

In the instrument information data 107, the instrument setting program 105 registers data in which the contents of the setting information list 402 and the input information list 403 are changed according to the above-mentioned information. For example, the setting information list 402 can be described as "fulcrum position, straight line position, minimum value position, maximum value position", and the input information list 403 can be described as "fulcrum position, minimum value position, maximum value position". The "fulcrum position" is the fulcrum position 2002 of the pointer, the "straight line position" is the position of the straight line formed by the scale 2003, the "minimum value position" is the minimum value position of the scale 2004, and the "maximum value position" is the maximum value position of the scale 2005. It is assumed that each of them will correspond to. In the first embodiment, since the center position of the circle, which is the fulcrum position of the pointer, is determined by the detected circle candidate, no input by the user is required, but the fulcrum operates in the instrument of the type shown in FIG. Since it is difficult to uniquely determine the range from the position of the straight line formed by the scale 2003 or the like, in the second embodiment, it is assumed that the user also inputs the “fulcrum position”.

FIG. 21 shows an example of the format of the instrument setting data 108 used in the second embodiment. The difference from FIG. 5 is that the center position 504 of the circle is changed to the fulcrum position 2101 and the radius 505 of the circle is changed to the position 2102 of the straight line. The fulcrum position 2101 and the straight line position 2012 correspond to the fulcrum position 2002 of the pointer in FIG. 20 and the straight line position 2003 formed by the scale, respectively.

In the instrument setting program 105 in the second embodiment, the processing performed in steps 704 and 707 is changed from the detection of the circle candidate to the detection of the straight line candidate. Further, in the processing after step 708, the instrument setting program 105 detects the scale candidate for the detected straight line candidate (step 708), ranks the straight line candidate (step 710), and the straight line based on the ranking result. The candidates may be displayed (step 711) or the like. Further, in the detection of the scale candidate (step 708), since the scale is linear, it is not necessary to perform the polar coordinate conversion used in the first embodiment. The instrument setting program 105 can directly acquire an image as shown in FIG. 12 or 13 from the image of the instrument and detect a candidate for the scale position. Regarding the ranking of straight line candidates (step 710), the instrument setting program 105 specifies the processing performed by the user based on the distance between the position specified by the user and the circle candidate in the first embodiment. It is only necessary to change the process to be performed based on the distance between the position and the straight line candidate. Regarding the display of candidates and the user's operation on the candidates, although there is a difference in shape between a circle and a straight line, basically the same method can be used.

In the instrument reading program 106 in the second embodiment, the indicated value of the instrument can be read by performing the same processing as the processing shown in the flowchart shown in FIG. However, in the second embodiment, the following two points are different from the first embodiment in the process performed in step 1904. First, in the first embodiment, the processing range is limited to the range of the circle determined by the center position 504 of the circle and the radius 505 of the circle, but in the second embodiment, the fulcrum position 2002 and the scale of the pointer are set. The range of processing is limited to a rectangular region (a region sandwiched between 2002 and 2003 in FIG. 20) determined by the position 2003 of the constituent straight line. Further, in the first embodiment, the indicated value was calculated after converting the tip position of the pointer to a relative value from the center position 504 of the circle, but in the second embodiment, the tip of the detected pointer is calculated. The position can be used as it is. For example, in the instrument reading program 106, when the pointer moves in the vertical direction as shown in FIG. 20, the value of the Y coordinate of the pointer position can be used as it is. In this case, the same values may be used for the minimum value position 2004 of the scale and the maximum value position 2005 of the scale. When the pointer moves in the lateral direction, the value of the X coordinate may be used.

As described above, according to the second embodiment, in the instrument reading system in which the scale is arranged linearly and the reading of the instrument of the type in which the pointer moves linearly along the straight line is read from the image. Even when a mobile device such as a smartphone or tablet terminal is used to set the information required for reading the instrument, it is possible to set the information required for reading the instrument accurately and in a short time.

Further, for the circular instrument shown in Example 1, the rectangular instrument shown in Example 2, and various types of instruments included in the circular shape and the rectangular shape, at least each type of instrument has a scale. Instrument information storage means for storing information on the shape of the area to be included, the type of information on the constraint condition, and the order in which the constraint condition is input (for example, storage for storing the instrument information data 107, the instrument setting data 108, and the instrument reading result data 109). A type acquisition means for acquiring the type of the instrument to be set (for example, the instrument setting program 105), and a selection means for selecting a candidate for an area including a scale displayed on the image or a candidate for a scale position (for example, for example). It has an instrument setting program 105) that accepts selection of a circle candidate, a maximum value candidate, and a minimum value candidate displayed at the lower part 1603 of the screen of FIG. 16, and the above setting means has information corresponding to the acquired type of instrument. Is acquired from the instrument information storage means, and based on the acquired information, information on the constraint conditions necessary for ranking the candidate of the area including the scale or the candidate of the scale position is acquired, and the above-mentioned ranking means is acquired. Based on the information, the candidate of the area including the scale or the candidate of the scale position is ranked, and the above-mentioned selection means selects the candidate of the area including the scale or the candidate of the scale position based on the result of the ranking. It is also possible. This allows the instrument reading system to be applied to various types of instruments.

1000 Instrument reading system 101 CPU
102 Input I / F
103 Output I / F
104 Memory 105 Instrument setting program 106 Instrument reading program 107 Instrument information data 108 Instrument setting data 109 Instrument reading result data 201 User 202 Mobile terminal 203 Target instrument

Claims (12)

The acquisition method to acquire the image of the instrument and
Setting means for setting the instrument information required to read the numerical value indicated by the instrument, and
A reading means for reading the numerical value indicated by the instrument from the image of the instrument based on the set information of the instrument.
An instrument reading system comprising a display means for displaying the result of reading the numerical value indicated by the instrument.
The setting means is
A region candidate detecting means for detecting a region candidate including a scale in the image of the instrument, and a region candidate detecting means.
For each candidate in the area including the detected scale, a position candidate detecting means for detecting a candidate for the scale position and a position candidate detecting means.
An input means for inputting information regarding a constraint condition for selecting an optimum candidate from the candidate of the area including the scale and the candidate of the scale position.
A ranking means for ranking the candidate of the region including the scale and the candidate of the scale position based on the candidate of the region including the scale, the candidate of the scale position, and the input constraint condition.
A display means for displaying a candidate for an area including an optimum scale and a candidate for a scale position based on the result of the ranking, and a display means.
An instrument reading system characterized by having. The instrument reading system according to claim 1.
The region candidate detecting means is
A process of detecting a candidate for a region including the scale from the image of the instrument by specifying a shape representing the region including the scale from the contour information extracted from the image of the instrument, and a process of detecting the candidate of the region including the scale from the image of the instrument, respectively. By extracting a small area corresponding to the scale of the instrument and determining whether or not the shape composed of the extracted plurality of small areas represents the area including the scale, the area including the scale is included in the image of the instrument. Processing to detect candidates and
Doing at least one of the processes,
An instrument reading system featuring. The instrument reading system according to claim 1.
The setting means is
The approximate position of the scale position is acquired from the input means, and the position is obtained.
The ranking means is
Candidates for the detected region including the scale and candidates for the scale position based on the distance between the detected region candidate including the scale and the acquired position of the scale position candidate and the acquired approximate position. To rank with,
An instrument reading system featuring. The instrument reading system according to claim 3.
The setting means is
When the shape of the region including the scale is circular, (1) the approximate position of the scale position indicating the minimum value and the maximum value, and (2) the approximate position and the minimum value of the center position of the circle are obtained from the input means. The approximate position of the scale position shown, (3) the approximate position of the circular center position and the approximate position of the scale position indicating the maximum value, (4) the approximate position of the scale position indicating the minimum value and the instrument. Obtain the condition of either the center position of the image of (5) the approximate position of the scale position indicating the maximum value or the center position of the image of the instrument.
The ranking means is
Based on the detected candidate for the region including the scale and the condition of any one of (1) to (5), the candidate for the region including the scale and the candidate for the scale position are ranked. ,
An instrument reading system featuring. The instrument reading system according to claim 3.
The setting means is
When the scale position is an instrument that is axisymmetric with respect to a specific line, the information acquired from the input means is the approximate position of the scale position indicating the minimum value or the scale position indicating the maximum value. Get one of the approximate positions,
The ranking means is
Based on the distance between the detected region candidate including the scale and the position of the candidate position of the scale position and any of the acquired approximate positions, the candidate region including the detected scale and the scale are used. Ranking with candidate positions,
An instrument reading system featuring. The instrument reading system according to claim 3.
The setting means displays a predetermined number of candidates from the candidates having a higher rank among the candidates of the area including the scale on the image of the instrument, and displays the candidates in the area including the scale on each candidate. Among the candidates for the corresponding scale positions, a predetermined number of candidates from the candidates with the highest rank are superimposed and displayed on the image of the instrument.
An instrument reading system featuring. The instrument reading system according to claim 6.
The setting means is
It has a selection means for selecting a candidate for an area including the scale displayed on an image or a candidate for the scale position.
A process of displaying the selected region candidate including the scale and the candidate of the scale position in a form different from the other candidates of the region including the scale and the candidate of the scale position, the region including the selected scale. Processing to display only the candidate of the scale position for the candidate of the instrument on the image of the instrument,
Performing any one or more of the above processes and displaying the candidate of the area including the scale and the candidate of the scale position on the screen of the instrument.
An instrument reading system featuring. The instrument reading system according to claim 7.
The setting means is
A list of candidates for the area including the scale and the candidate for the scale position is displayed, and a candidate for the region including the scale or a candidate corresponding to the candidate for the scale position selected in the list is selected as another candidate for the scale. Display on the image of the instrument in a form different from the candidate of the area including the above or the candidate of the scale position.
An instrument reading system featuring. The instrument reading system according to claim 7.
The setting means is
The position of the selected candidate among the candidate of the region including the scale and the candidate of the scale position on the image of the instrument can be adjusted.
An instrument reading system featuring. The instrument reading system according to claim 6.
The setting means is
Every time one piece of information that becomes a constraint condition is acquired from the input means, the ranking of the candidate of the area including the scale and the candidate of the scale position is updated, and the area including the scale on the image of the instrument is updated. Changing the display of candidates and candidates for the scale position,
An instrument reading system featuring. The instrument reading system according to claim 1.
At least, for each type of instrument, the shape of the area including the scale, the type of information regarding the constraint condition, the instrument information storage means for storing the information regarding the order in which the constraint condition is input, and the type of the instrument to be set are acquired. It has a type acquisition means and a selection means for selecting a candidate for a region including the scale or a candidate for the scale position displayed on the image.
The setting means is
The constraint necessary for ranking the candidate of the area including the scale or the candidate of the scale position based on the acquired information obtained by acquiring the information corresponding to the acquired type of the instrument from the instrument information storage means. Get information about the condition,
The ranking means is
Based on the acquired information, the candidate of the area including the scale or the candidate of the scale position is ranked.
The selection means is
To select a candidate for a region including the scale or a candidate for the scale position based on the result of the ranking.
An instrument reading system featuring. Computer,
Acquisition method to acquire the image of the instrument,
Setting means for setting the instrument information required to read the numerical value indicated by the instrument,
A reading means that reads the numerical value indicated by the instrument from the image of the instrument based on the set information of the instrument.
An instrument reading program that functions as a display means for displaying the result of reading the numerical value indicated by the instrument.
The setting means
A region candidate detecting means for detecting a region candidate including a scale in the image of the instrument,
A position candidate detecting means for detecting a candidate for a scale position for each candidate in the area including the detected scale.
An input means for inputting information regarding a constraint condition for selecting an optimum candidate from the candidate of the area including the scale and the candidate of the scale position.
A ranking means for ranking a candidate for a region including the scale and a candidate for the scale position based on the candidate for the region including the scale, the candidate for the scale position, and the input constraint condition.
A display means for displaying a candidate for an area including an optimum scale and a candidate for a scale position based on the result of the ranking.
An instrument reading program characterized by functioning as.

Images