JP2020135324A - Image processing device, image processing program, image processing method and information collection system - Google Patents

Abstract

To enable the reading of an indication value of an indication needle instrument by suppressing a calculation amount and a memory usage amount required for image processing.SOLUTION: According to the present invention, an image processing device for deriving an indication value of an indication needle instrument from a picked-up image obtained by photographing a display surface of the indication needle instrument includes: a projective transformation part for using a projective transformation parameter between a virtual reference image of the indication needle instrument and an image of the indication needle instrument in the inputted picked-up image to projectively transforming an indication needle detection coordinate sequence having a plurality of coordinate values showing positions of an indication needle of the indication needle instrument in the reference image or the picked-up image; an indication needle detection part for detecting the position of the indication needle from a plurality of pixel values extracted from the picked-up image by using the projectively transformed indication needle detection coordinate sequence or a plurality of pixel values extracted from a picked-up image projectively transformed by using the indication needle detection coordinate sequence; and an indication value derivation part for deriving an indication value corresponding to the position of the indication needle detected by the indication needle detection part.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image processing apparatus, an image processing program, an image processing method, and an information collecting system.

For example, an analog meter is an instrument that measures a measured value of a target object by moving an indicator needle along the scale plate and the scale sequence of the scale plate. Such analog meters are used in various situations. For example, they are installed in equipment in facilities such as factories, and an operator monitors the equipment while visually observing the measured values of the analog meter. It is often used in such cases.

In recent years, in facilities such as factories, as one of the reductions in operating costs, it is required to be able to automatically read the measured values of analog meters and collect the data. There are various methods for automatically reading an analog meter, and one example thereof is replacement of an analog meter or automation of visual inspection work of the analog meter.

Replacing the analog meter is a method of replacing the currently used analog meter with a dedicated meter that outputs the indicated value indicated by the indicator needle as digital data. In this case, there is a problem that it is necessary to replace the analog meter currently in use with a dedicated meter, and the equipment must be stopped during the replacement period.

The automation of the visual inspection work of the analog meter is to take a picture of the indicator needle and the dial of the analog meter with a camera, process the captured image, and read the indicated value indicated by the indicator needle. In this case, the reading of the indicated value can be automated by installing an indicated value reading device including a camera that captures an analog meter and an image processing device that processes the captured image in the facility. Introducing such an indicated value reading device has an advantage that it is not necessary to stop equipment or the like, for example, and therefore there is a strong demand.

Conventionally, there is one disclosed in Patent Document 1, for example, as one relating to automation of visual inspection work of an analog meter. In the technique described in Patent Document 1, in order to detect the position of the indicator needle reflected in the captured image captured by the camera, the entire captured image is binarized and a pixel showing black color in the image is searched for. It is determined that the indicator needle is located in the area where the number of black pixels is the largest.

By the way, the above-mentioned reading device for reading needs to be installed in each analog meter. Therefore, when there are a large number of analog meters, for example, in a facility such as a factory, the introduction cost of the meter reading device becomes high, and it is required to keep the introduction cost low.

Japanese Unexamined Patent Publication No. 2017-126187

However, in the above-mentioned description technique of Patent Document 1, it is necessary to binarize the entire captured image or perform pixel search in the entire image, so that the amount of calculation processing becomes enormous and the load related to the image processing is large. There is a problem. In addition, it becomes necessary to use a high-performance and expensive arithmetic processing device, a camera, or the like, and there is also a problem that it is difficult to suppress the introduction cost.

Therefore, in view of the above problems, the present invention has an image processing device, an image processing program, and an image processing capable of reading the value indicated by the indicator needle of the analog meter while suppressing the amount of calculation and the amount of memory used for image processing. It seeks to provide methods and information gathering systems.

In order to solve such a problem, the first image processing apparatus according to the present invention is an image processing apparatus for deriving an indicated value of an indicating needle instrument from an image captured by photographing the display surface of the indicating needle instrument. It has a plurality of coordinate values indicating the position of the indicator needle of the indicator needle instrument in the reference image using the projection conversion parameter of the virtual reference image of the needle instrument and the image of the indicator needle instrument in the input captured image. A projection conversion unit that projects and transforms the indicator needle detection coordinate sequence or the captured image, and (2) a plurality of pixel values extracted from the captured image using the projection-converted indicator needle detection coordinate sequence, or the indicator needle detection coordinates. Corresponds to the indicator needle detection unit that detects the position of the indicator needle from a plurality of pixel values extracted from the captured image that has been projected and converted using columns, and (3) the position of the indicator needle detected by the indicator needle detection unit. It is characterized by including an instruction value derivation unit for deriving an instruction value.

The second image processing program according to the present invention is an image processing program for deriving the indicated value of the indicator needle meter from an image captured by photographing the display surface of the indicator needle instrument, in which the computer is used as (1) a virtual indicator needle instrument. Indicator needle detection coordinate sequence having a plurality of coordinate values indicating the position of the indicator needle of the indicator needle instrument in the reference image using the projection conversion parameter of the reference needle and the image of the indicator needle instrument in the input captured image. Or, a projection conversion unit that projects and transforms the captured image, and (2) a plurality of pixel values extracted from the captured image using the projection-converted indicator needle detection coordinate sequence, or projection using the indicator needle detection coordinate sequence. From the plurality of pixel values extracted from the converted captured image, the indicator needle detection unit that detects the position of the indicator needle and (3) the indicator value corresponding to the position of the indicator needle detected by the indicator needle detection unit are derived. It is characterized in that it functions as an instruction value derivation unit.

The third image processing method according to the present invention is an image processing method for deriving an indicated value of the indicator needle instrument from an image captured by photographing the display surface of the indicator needle instrument. In the image processing method, (1) the projective conversion unit of the indicator needle instrument Using the projective conversion parameters of the virtual reference image and the image of the indicator needle in the input captured image, the indicator needle detection having a plurality of coordinate values indicating the position of the indicator needle of the indicator needle in the reference image The coordinate sequence or the captured image is projected and converted, and (2) a plurality of pixel values extracted from the captured image by the indicator needle detection unit using the projected-converted indicator needle detection coordinate sequence, or the indicator needle detection coordinate sequence. The position of the indicator needle is detected from a plurality of pixel values extracted from the image captured by the homograph using the above method, and (3) the indicator value derivation unit corresponds to the position of the indicator needle detected by the indicator needle detection unit. It is characterized by deriving an indicated value.

The fourth information collecting system according to the present invention is derived by (1) an image processing device that derives an indicated value of the indicated needle meter from an captured image obtained by photographing the display surface of the indicated needle meter, and (2) an image processing device. A first communication device that transmits a signal including an indicated value of the indicated needle meter, (3) a second communication device that receives the signal transmitted from the first communication device, and (4) a second. The image processing device is the first image processing device according to the present invention, which includes a management device for storing an instruction value included in a signal received by the communication device of the above.

According to the present invention, it is possible to read the value indicated by the indicator needle of the analog meter while suppressing the amount of calculation and the amount of memory used for image processing.

It is a block diagram which shows the whole structure of the information gathering system which concerns on 1st Embodiment. It is explanatory drawing explaining the relative positional relationship between the camera which takes an image of an analog meter, and an analog meter, and an example of the captured image (the 1). It is an internal block diagram which shows the internal structure of the image processing apparatus which concerns on 1st Embodiment. It is explanatory drawing explaining the posture reference information which concerns on 1st Embodiment. It is explanatory drawing explaining the indicator needle detection coordinate sequence which concerns on 1st Embodiment. It is a flowchart which shows the operation of the image processing method which concerns on 1st Embodiment. It is a flowchart which shows the detection method of the coordinates of four vertices by the posture information detection part which concerns on 1st Embodiment. It is explanatory drawing explaining the detection method of the four vertex coordinates by the posture information detection part which concerns on 1st Embodiment. It is explanatory drawing explaining the relationship between the indicator needle detection pixel value string and the indicator needle detection coordinate string in the 1st Embodiment. It is explanatory drawing explaining the relative positional relationship between the camera which images an analog meter and an analog meter, and an example of the captured image (the 2). It is an internal block diagram which shows the internal structure of the image processing apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the posture information detection jig which concerns on 2nd Embodiment. It is explanatory drawing explaining the state which attached the posture information detection jig which concerns on 2nd Embodiment to an analog meter. It is a flowchart which shows the operation of the image processing by the image processing apparatus which concerns on 2nd Embodiment. It is explanatory drawing explaining the projecting transformation which made the four reference points of the posture reference information a conversion source, and the four vertices of a posture information detection jig as a conversion destination. It is a flowchart which shows the detection method of the posture information detection jig of the posture information detection part which concerns on 2nd Embodiment. It is explanatory drawing explaining the method of giving a label to four vertices of the 2nd Embodiment. It is a block diagram which shows the structure of the posture information detection jig which concerns on 3rd Embodiment. It is explanatory drawing explaining the state which attached the posture information detection jig which concerns on 3rd Embodiment to an analog meter. It is a flowchart which shows the detection method of the posture information detection jig of the posture information detection part which concerns on 3rd Embodiment. It is an internal block diagram which shows the internal structure of the image processing apparatus which concerns on 4th Embodiment. It is a flowchart which shows the image processing in the image processing apparatus which concerns on 4th Embodiment. In the first embodiment, it is a block diagram which illustrates the structure of the table which shows the correspondence relationship between the detected indicator needle angle and the indicator value. It is explanatory drawing explaining the case which determines the pixel value of a certain region including each coordinate value of the indicator needle detection coordinate string which concerns on a modification embodiment. It is a block diagram which shows the structure of the scale board and the indicator needle of the analog meter which concerns on the modification embodiment, and the structure of the posture reference information.

(A) First Embodiment In the following, the first embodiment of the image processing apparatus, the image processing program, the image processing method, and the information collecting system according to the present invention will be described in detail with reference to the drawings.

(A-1) Configuration of First Embodiment (A-1-1) Overall Configuration of Information Collection System FIG. 1 is a configuration diagram showing an overall configuration of an information collection system according to the first embodiment.

In FIG. 1, the information collection system 10 according to the first embodiment includes an analog meter 1, a camera 2, an image processing device 3, communication devices 4 and 5, and a management device 6.

In the information collecting system 10, the camera 2 photographs the analog meter 1, acquires the indicated value pointed by the indicator needle of the analog meter 1 from the captured image, reads the indicated value, and collects the indicated value. Is. As a result, the indicated value of the analog meter 1 can be read in real time, and the indicated value can be accumulated and held as history information.

The analog meter 1 is an indicator needle type instrument in which an indicator needle moves along a scale plate and scales arranged on the scale plate to measure a measured value. The analog meter 1 can be installed in equipment or the like in a facility such as a factory, and can be an instrument such as an analog voltmeter or an analog ammeter. Some analog meters 1 have an indicator needle that rotates around the rotation axis in the scale area of the scale plate, and one that has a scale plate in the vertical or horizontal direction and the indicator needle moves in the vertical or horizontal direction. However, in this embodiment, a type in which the indicator needle rotates in the scale area of the scale plate will be described as an example.

The camera 2 is an image pickup device that captures an image of the analog meter 1, and gives the captured image to the connected image processing device 3. As the camera 2, for example, a general-purpose camera that does not have a function of correcting a captured image or the like can be applied. As a result, since an inexpensive camera can be used, the introduction cost of the indicated value reading system can be suppressed.

The image processing device 3 acquires a captured image including the analog meter 1 from the camera 2, analyzes the captured image, and derives an indicated value of the indicator needle in the captured image. Further, the image processing device 3 supplies data including the indicated value to the communication device 4. A detailed description of the image processing device 3 will be described later, but according to this embodiment, since the processing load of the instruction value calculation process can be suppressed, a general-purpose arithmetic processing device such as a microcomputer can be used. The introduction cost of the indicated value reading system can be suppressed.

The communication devices 4 and 5 are communication devices that send and receive signals including the indicated value data of the analog meter 1. The communication device 4 transmits a communication signal including the indicated value data of the analog meter 1 acquired from the image processing device 3, and the communication device 5 receives the communication signal transmitted from the communication device 4 and communicates therewith. The instruction value data included in the signal is given to the management device 6. As the communication devices 4 and 5, for example, those that perform wireless communication according to the wireless communication protocol of multi-hop wireless communication can be applied. The wireless communication protocol is not particularly limited. Further, the communication devices 4 and 5 may communicate signals including the indicated value data of the analog meter 1 by wire communication.

The management device 6 records the indicated value data of the analog meter 1 given by the communication device 5, and manages the indicated value data of the analog meter 1. The management device 6 has an arithmetic processing unit such as a CPU, a display means such as a monitor, and the like, displays the indicated value data of the analog meter 1 in real time, and statistically processes using the past indicated value data for which history has been taken. Etc., and the data can be displayed. The management method of the management device 6 using the indicated value data is not particularly limited, and various methods can be applied.

(A-1-2) Example of Image Captured by Camera FIG. 2 is an explanatory diagram illustrating a relative positional relationship between the camera 2 that captures the analog meter 1 and the analog meter 1 and an example of the captured image.

Since the analog meter 1 is often installed in equipment, for example, it is assumed that the position and orientation of the analog meter 1 cannot be changed. When analyzing the captured image of the camera 2 and calculating the indicated value of the indicator needle of the analog meter 1, it is desirable that the camera 2 is arranged in front of the analog meter 1, but the positional relationship is not always such a case. There is also.

In this embodiment, since it is assumed that a general-purpose camera 2 is used, the captured image is not corrected according to the posture of the camera 2 and the relative positional relationship with the analog meter 1.

Therefore, here, first, the relative positional relationship between the analog meter 1 and the camera 2 will be described, and when there is such a positional relationship, the image of the analog meter 1 projected on the captured image will be described.

FIG. 2A illustrates a case where the camera 2 is arranged in front of the analog meter 1. In this case, the camera 2 can take a picture of the panel portion (the portion where the scale board and the indicator needle are located) of the analog meter 1 from the front, and the captured image P1 as shown in FIG. 2B can be obtained. At this time, the image of the analog meter 1 projected on the captured image P1 is in an upright posture.

The image captured by the camera 2 has a certain size in the image coordinate system, and can be represented by a coordinate value (XY coordinate value) on the XY plane of the image coordinate system.

On the other hand, FIG. 2C illustrates a case where the camera 2 is arranged diagonally to the left with respect to the analog meter 1. In this case, since the camera 2 captures the panel portion of the analog meter 1 from an oblique left direction, the captured image P2 as shown in FIG. 2D can be obtained. That is, the image of the analog meter 1 projected on the captured image P2 has an obliquely deformed posture.

In this way, the posture of the analog meter 1 displayed on the captured image can be changed depending on the position of the camera 2 with respect to the analog meter 1 and the posture of the camera 2, so that the indicated value of the indicator needle of the analog meter 1 displayed on the captured image can be changed. When calculating, it is necessary to consider the posture and deformation of the analog meter 1 in the captured image.

(A-1-3) Detailed Configuration of Image Processing Device FIG. 3 is an internal configuration diagram showing an internal configuration of the image processing device 3 according to the first embodiment.

In FIG. 3, the image processing device 3 according to the first embodiment has an image input unit 31, an attitude information detection unit 32, a storage unit 33, a projection conversion parameter calculation unit 34, an indicator needle detection coordinate sequence projection conversion unit 35, and an instruction. It has a needle detection unit 36, an indicated value calculation unit 37, and an output unit 38.

The hardware of the image processing device 3 is configured to include, for example, a CPU, ROM, RAM, EEPROM, an input / output interface, etc., and the CPU executes a processing program (for example, an image processing program) stored in the ROM. As a result, various functions are realized. The image processing program may be installed in a computer such as a CPU to construct the image processing, and even in that case, the image processing program can be represented as each functional unit of FIG.

The image input unit 31 inputs an captured image including the analog meter 1 from the camera 2 that captures the analog meter 1.

The posture information detection unit 32 detects the coordinate values of the vertices according to the shape of the analog meter 1 in the captured image (actual image) in order to detect the posture of the analog meter 1 reflected in the input captured image. Is.

Here, the coordinate values of the vertices detected by the posture information detection unit 32 are also referred to as "posture information". In the first embodiment, the posture information detection unit 32 detects a rectangular surface shape including the scale plate and the indicator needle in the outer frame of the box-shaped analog meter 1, and four vertices of the surface. Detect coordinates.

For example, when the camera 2 is provided in front of the box-shaped analog meter 1, the camera 2 can take an image of the analog meter 1 from the front. In that case, since the outer frame of the substantially rectangular analog meter 1 is reflected in the captured image in the captured image, the vertex coordinate detection unit 32 determines four sides of the outer frame of the analog meter 1 and intersects the respective sides. May be set as a vertex, and the coordinate value of each vertex may be detected. Further, for example, when the camera 2 photographs the analog meter 1 from an oblique direction with respect to the analog meter 1, the image of the analog meter 1 is reflected in a deformed state such as a trapezoid. In that case, the posture information detection unit 32 determines four sides forming a trapezoid, sets the intersection of each side as a vertex, and detects the coordinate value of each vertex. In any case, the posture information detection unit 32 identifies the position and posture of the analog meter 1 by specifying the outer frame of the analog meter 1 reflected in the captured image, and determines the coordinate values of the four vertices. Try to detect.

The storage unit 33 stores preset posture reference information 331, indicator needle detection coordinate sequence 332, and the like.

The posture reference information 331 is information including coordinate values of a plurality of reference points used in the projective transformation process. For example, it is information about coordinate values indicating the positions of four reference points in a reference image. The indicator needle detection coordinate sequence 332 is information for specifying the position of the indicator needle of the analog meter 1 in the reference image. The detailed configuration of the posture reference information 331 and the indicator needle detection coordinate sequence 332 will be described later.

Here, the reference image includes a virtual image in which the analog meter 1 is ideally upright. The posture of the analog meter 1 reflected in the captured image taken by the camera 2 of the analog meter 1 may be distorted or deformed depending on the relative positional relationship with the camera 2, the posture of the camera 2, etc. The image is a virtual image in which the analog meter 1 is ideally upright and is not distorted or rotated. The "reference images" described in the following description are all virtual.

In this way, the posture reference information 331 stored in the storage unit 33 is the image of the analog meter 1 that is upright and is not distorted or rotated, which is reflected in the virtual reference image, and has four reference points. Information about coordinate values indicating the position. In other words, in the first embodiment, the image processing device 3 may store the attitude reference information 331, which is information on the coordinate values, in the storage unit 33, for example, image information (brightness and color information) corresponding to the reference image. It is not necessary to store information that enables the image to be drawn, including the above.

The projective conversion parameter calculation unit 34 uses the four coordinates detected by the attitude information detection unit 32 (that is, the four coordinates in the actual image taken by the camera 2) and the preset attitude reference information 331. Based on this, the projective conversion parameter (also called the projective conversion coefficient) is derived.

For example, the projective conversion parameter is an image area of the analog meter 1 (for example, a two-dimensional plane) in which an ideally upright analog meter 1 is reflected in an image captured by the camera 2. For example, it is a parameter used when projecting onto a two-dimensional plane), and generally has eight unknowns. In order to determine these eight coefficients, the projective conversion parameter calculation unit 34 uses the coordinates of the four vertices detected by the attitude information detection unit 32 and the coordinate values of the four reference values of the attitude reference information 331. Is calculated using.

The indicator needle detection coordinate sequence projection conversion unit 35 projects all the coordinate values of the indicator needle detection coordinate sequence 332 in the reference image by using the projection conversion parameters derived by the projection conversion parameter calculation unit 34. .. As a result, all the coordinate values of the indicator needle detection coordinate sequence in the reference image are projected and converted from the reference image to the actual image.

The indicator needle detection unit 36 acquires pixel values from the image (original image) captured by the camera 2 based on each coordinate value of the indicator needle detection coordinate sequence after the projective conversion, and is composed of all the acquired pixel values. Create an indicator needle detection pixel value sequence. Then, the indicator needle detection unit 36 detects the position of the indicator needle from all the pixel values constituting the indicator needle detection pixel value sequence. A detailed description of the indicator needle detection unit 36 will be described later.

The instruction value calculation unit 37 calculates an instruction needle angle corresponding to the coordinates detected by the instruction needle detection unit 36 with reference to the instruction needle detection coordinate sequence 332, and outputs an instruction value based on the instruction needle angle. Is.

(A-1-4) Posture reference information 331 and indicator needle detection coordinate sequence 332
FIG. 4 is an explanatory diagram for explaining the posture reference information 331 according to the first embodiment.

FIG. 4A exemplifies the position coordinates of the four reference points constituting the posture reference information 331 in the reference image.

As shown in FIG. 4A, indexes are assigned to the four reference points, for example, point Ar, point Br, point Cr, and point Dr. The allocation of this index may be arbitrarily assigned. Then, as illustrated in FIG. 4 (B), the coordinate values (0,0), (0,400), (400,400), ( It is assumed that 400,0) is stored in the storage unit 33 in association with the index.

The coordinate system of the reference image may be the same as the coordinate system of the captured image captured by the camera 2. The four reference points constituting the attitude reference coordinates can be preset and arbitrarily selected points in the coordinate system of the captured image captured by the camera 2. In the example of FIG. 4A, a square is formed in the reference image, and the four vertices of the square are used as reference points. The size of the square in the reference image is not particularly limited, but may be set to be about the same as the size of the analog meter 1 displayed in the captured image (actual image), for example.

FIG. 5 is an explanatory diagram illustrating the indicator needle detection coordinate sequence 332 according to the first embodiment.

As shown in FIG. 5B, the indicator needle detection coordinate sequence 332 is information having, for example, an “index”, a “coordinate”, and an “indicator needle angle” as items.

In the reference image, the indicator needle detection coordinate sequence 332 assigns an index (label) to each position coordinate in which the indicator needle may exist within the existence region of the rotating indicator needle in order to specify the position value of the indicator needle. .. Then, each index, the XY coordinate value of each index, and the angle of the indicator needle in the analog meter 1 are created in association with each other. That is, the indicator needle detection coordinate sequence 332 has a structure in which the data obtained by pairing the XY coordinate value of each indicator needle detection coordinate (index) and the indicator needle angle value at that time is set as one unit, and a plurality of data are connected. It has become.

For example, it is assumed that the dial and the indicator needle of the analog meter 1 are shown in FIG. 5 (C). In the reference image, the index "G1" is assigned to the position corresponding to the position of the scale "0" of the analog meter 1 in FIG. 5 (C). In this case, the XY coordinate value (100, 100) of the index “G1” is used. Further, when the position of the indicator needle is at "G1", the analog meter 1 corresponds to the scale "0", so "indicator needle angle: 0" is set.

After that, indexes G2, G3, ..., Gn are added to each position corresponding to the position of the indicator needle when the indicator needle angle is moved by a unit angle, and each index G2, G3, ..., Gn The XY coordinate values of are given, and the indicator needle angle is associated with each index. Then, as shown in FIG. 5A, each index constituting the indicator needle detection coordinate string 332 is formed on the virtual circle VC2. The magnitude of the above-mentioned unit angle may be arbitrary.

Here, the reason why the virtual circle VC2 constituting the indicator needle detection coordinate sequence 332 is formed in the existing region of the position of the indicator needle will be described. For example, in the image captured by the camera 2, the indicator needle may not intersect the scale of the analog meter 1, and in such a case, the indicator value pointed to by the indicator needle may not be read. However, as in the first embodiment, by forming the virtual circle VC2 constituting the indicator needle detection coordinate sequence in the existing region of the position of the indicator needle, that is, the radius of the virtual circle VC2 is smaller than that of the virtual circle VC1. By reducing the size, the position coordinates of the indicator needle in the image can be detected and the angle of the indicator needle can be further detected even when the indicator needle and the scale do not intersect.

Since the virtual circle VC2 constituting the indicator needle detection coordinate sequence 332 is formed in the existing region of the position of the indicator needle, it is formed in the region of the virtual circle VC1 formed at the position of each scale of the analog meter 1. Will be done. In other words, the radius of the virtual circle VC2 centered on the point S is smaller than the radius of the virtual circle VC1 centered on the point S.

The point S is the center of a square formed by four reference points constituting the attitude reference coordinates. For example, a line segment connecting the points Ar and Cr and a line connecting the points Br and Dr. It is the intersection with the minute. Further, the point S is a point corresponding to the center where the indicator needle in the analog meter 1 rotates. Therefore, the point S has a positional relationship such that the center of the square formed by the four reference points of the posture reference information 331 coincides with the center of the virtual circle VC2 constituting the indicator needle detection coordinate sequence 332.

(A-2) Operation of the First Embodiment Next, the operation of the image processing by the image processing apparatus 3 according to the first embodiment will be described in detail with reference to the drawings.

FIG. 6 is a flowchart showing the operation of the image processing method according to the first embodiment.

[Step S1]
The captured image (original image) taken by the camera 2 is input to the image input unit 31 of the image processing device 3. Here, it is assumed that the analog meter 1 is projected on the image captured by the camera 2.

[Step S2]
The posture information detection unit 32 detects the image area of the analog meter 1 and shows the outer shape of the analog meter 1 in order to acquire the posture information of the analog meter 1 projected on the input captured image (original image). Detect the coordinate values of the vertices.

Here, the XY coordinate values of the four vertices (point Am, point Bm, point Cm, point Dm) detected by the attitude information detection unit 32 are referred to as attitude information.

Hereinafter, an example of a method in which the posture information detection unit 32 detects the coordinate values of the four vertices indicating the outer shape of the analog meter 1 from the captured image will be described.

FIG. 7 is a flowchart showing a method of detecting the coordinates of four vertices by the posture information detection unit 32 according to the first embodiment, and FIG. 8 shows four by the posture information detection unit 32 according to the first embodiment. It is explanatory drawing explaining the detection method of the vertex coordinates of.

Here, the camera 2 photographs the analog meter 1 from an oblique direction with respect to the analog meter 1, and the captured image is projected in a state in which the posture of the analog meter 1 is deformed as shown in FIG. 8A. Illustrate the case.

The posture information detection unit 32 detects four straight lines indicating the sides of the outer frame of the analog meter 1 reflected in the captured image P (step S11).

Here, for example, the attitude information detection unit 32 performs binarization processing on the original image in order to extract a region including the outer frame of the analog meter 1 from the original image from the camera 2, and the white scale board. The area may be distinguished from the area of the outer frame portion around the scale plate, or the area of the outer frame of the analog meter 1 and the area of the background image may be distinguished. Further, the posture information detection unit 32 may perform a Hough transform on the image to extract a straight line including the side of the outer frame of the analog meter 1.

Then, the posture information detection unit 32 detects the intersection of the two straight lines by using the two straight lines as one combination among the four straight lines detected in the captured image P. The intersection is detected for each combination of different straight lines (step S12).

For example, it is assumed that the four straight lines VV', TT', WW', and UU'in FIG. 8A are detected. At this time, the intersection of the straight line VV'and the straight line TT', the intersection of the straight line TT'and the straight line WW', the intersection of the straight line WW'and the straight line UU', and the intersection of the straight line UU'and the straight line VV'are detected.

The posture information detection unit 32 uses four intersections as four vertices, assigns indexes (labels) of points Am, Bm, Cm, and Dm to each vertex (step S13), and assigns coordinate values of each vertex and each label. Is associated with and stored in the storage unit 33 as posture information in the original image (step S14).

For example, the point Am in FIG. 8A is labeled with the index “Am”, and the coordinate values (Xam, Yam) of the index “Am” are associated with the label “Am”. In this way, for the other three vertices as well, the associated label and the coordinate value of each vertex are associated with each other. The posture information of FIG. 8B thus obtained is stored in the storage unit 33.

[Step S3]
The projective conversion parameter calculation unit 34 is the sum of the XY coordinate values of the four vertices detected by the attitude information detection unit 32 and the XY coordinate values of the four reference points constituting the preset attitude reference information. The projective transformation parameters are derived using the eight XY coordinate values.

Here, the projective transformation parameter can be uniquely derived from the coordinate values of the corresponding points when the homogeneous coordinates are used and there are four sets of corresponding points between the known conversion source and the conversion destination.

For example, the projective conversion parameter calculation unit 34 uses the four XY coordinate values of the points Ar, Br, Cr, and Dr constituting the attitude reference information in FIG. 4 (B) as conversion sources, and the source in FIG. 8 (B). A projective conversion parameter that allows each of the four XY coordinate values of the points Am, Bm, Cm, and Dm that make up the attitude information in the image to be the conversion destination, and each point of the conversion source to be projected to each point of the conversion destination. Is calculated.

[Step S4]
The indicator needle detection coordinate sequence projection conversion unit 35 uses the projection conversion parameters derived by the projection conversion parameter calculation unit 34 to perform projection conversion on all the coordinate values of the indicator needle detection coordinate sequence 332.

For example, the indicator needle detection coordinate sequence 332 illustrated in FIG. 5B is configured to have n coordinate values to which indexes are given, and the n coordinate values are projected by the projective transformation parameter. Will be converted.

[Step S5]
The indicator needle detection unit 36 acquires pixel values from the image (original image) captured by the camera 2 by using the indicator needle detection coordinate sequence after the projective conversion, and the indicator needle detection pixel composed of the pixel values of each coordinate value. Create a value sequence 361.

FIG. 9 is an explanatory diagram illustrating the relationship between the indicator needle detection pixel value sequence 361 and the indicator needle detection coordinate sequence 332 according to the first embodiment.

The indicator needle detection pixel value sequence 361 has pixel values of each coordinate value in the original image for all the indexes constituting the indicator needle detection coordinate sequence 332 after the projective conversion. Further, the indicator needle detection pixel value sequence 361 may also be provided with an indicator needle angle associated with each index of the indicator needle detection coordinate sequence 332.

Here, the above-mentioned "acquiring the pixel value from the original image using the indicator needle detection coordinate sequence after the projective conversion" means that the indicator needle detection unit 36 constitutes the indicator needle detection coordinate sequence after the projective conversion. It means that the pixel value of each coordinate value is acquired from the original image.

Further, since each coordinate value constituting the indicator needle detection coordinate sequence 332 is indexed, the indicator needle detection unit 36 uses the pixel value of each coordinate value of the indicator needle detection coordinate sequence 332 after the projective conversion as the original image. The indicator needle detection pixel value sequence 361 is created by associating each pixel value with the corresponding index. That is, the indicator needle detection pixel value sequence 361 has the same number of indexes as the indexes constituting the indicator needle detection coordinate sequence 332, and each index of the indicator needle detection pixel value sequence 361 has the indicator needle detection coordinate sequence 332. It has a corresponding relationship with each index of.

Therefore, the indicator needle detection pixel value sequence 361 associates the index of the indicator needle detection coordinate sequence 332 with the pixel value of each coordinate value of the indicator needle detection coordinate sequence after the projective conversion in the original image and the indicator needle angle. It will be information.

In the indicator needle detection pixel value sequence 361 of FIG. 9, the pixel value of each coordinate value after the projective conversion in the original image is made to correspond to the index, but each index after the projective conversion in the original image is used. Coordinate values may be associated with each other.

[Step S6]
The indicator needle detection unit 36 calculates the angle of the indicator needle with reference to all the pixel values constituting the indicator needle detection pixel value sequence 361.

As a method of calculating the angle of the indicator needle, for example, the brightness value of each pixel is calculated for all the pixel values constituting the indicator needle detection pixel value sequence 361, and the pixel having the minimum value among all the luminance values is calculated. You may want to find the index of. Thereby, by referring to the indicator needle detection coordinate sequence 332 or the indicator needle detection pixel value sequence 361 in which the indicator needle angle is associated with each index, the indicator needle angle corresponding to the index can be derived.

Further, for example, the brightness value of each pixel is calculated for all the pixel values constituting the indicator needle detection pixel value sequence 361, and each brightness value is binarized using a threshold value, and the number of consecutive black pixels is the largest. The index of the pixel in the center of the large section may be obtained. Thereby, by referring to the indicator needle detection coordinate sequence 332 or the indicator needle detection pixel value sequence 361 in which the indicator needle angle is associated with each index, the indicator needle angle corresponding to the index can be derived.

Here, the reason for deriving the index having the minimum luminance value will be described. For example, a pixel in which the indicator needle is located in the original image has a smaller pixel value (for example, a brightness value) than a pixel in which the indicator needle is not located. Therefore, the indicator needle detection unit 36 can specify the position of the indicator needle by selecting the pixel value string having the smallest pixel value from the pixel value strings constituting the indicator needle detection pixel value sequence.

Further, as described above, since the index of the indicator needle detection pixel value sequence 361 and the index of the indicator needle detection coordinate sequence 332 have a corresponding relationship, the position of the indicator needle can be determined without requiring complicated coordinate conversion processing. If the pixel value of the indicated pixel can be specified and the index corresponding to the pixel value can be specified, the indicator needle angle corresponding to the index can be derived. In other words, since the indicator needle detection pixel value sequence 361 is composed of at least data in which the index and the pixel value are paired, for example, the index of the pixel value which is the minimum value among all the pixel values is set. If it can be specified, the indicator needle angle corresponding to the index can be derived.

[Steps S7, S8]
The instruction value calculation unit 37 calculates the instruction value of the instruction needle based on the instruction needle angle detected by the instruction needle detection unit 36 with reference to the instruction needle detection coordinate sequence 332. Then, the output unit 38 outputs the information including the indicated value to the subsequent stage.

For example, the instruction value calculation unit 37 has a table in which the instruction needle angle as illustrated in FIG. 23 and the instruction value are associated with each other, and derives the instruction value corresponding to the detected instruction needle angle. It may be. More specifically, for example, when the analog meter 1 is a voltmeter or the like, it has a table in which the indicator needle angle and the voltage value correspond to each other, and the voltage value (measured value) corresponding to the indicator needle angle can be obtained. It may be derived as an indicated value.

The method of deriving the indicated value of the indicator needle based on the indicator needle angle detected by the indicator needle detection unit 36 is not limited to the above-mentioned method.

(A-3) Effect of First Embodiment As described above, the image processing apparatus of the first embodiment has the coordinate values of the four points based on the shape of the analog meter reflected in the input captured image. , A projective conversion parameter is derived based on the coordinate values of the four reference points constituting the attitude reference information, and all the coordinate values (coordinate information) of the indicator needle detection coordinate sequence are projected using the projective conversion parameter. To do. Then, using all the coordinate values of the indicator needle detection coordinate sequence after the projective conversion, the pixel values can be acquired from the original image and the indicator values of the indicator needle can be read.

Further, since the image processing apparatus of the first embodiment projects and transforms only the indicator needle detection coordinate sequence using the projective conversion parameters, it can produce an captured image including a large amount of information such as brightness information and color information. Compared to projective conversion, the amount of information to be processed is small. Therefore, the coordinate conversion process performed by the image processing apparatus of the first embodiment is a process as compared with the process of correcting the captured image so as to bring the shape of the analog meter reflected in the input captured image closer to the ideal shape. The amount of calculation and memory usage required for this is small.

Further, in the image processing apparatus of the first embodiment, since the index of the indicator needle detection pixel value sequence and the index of the indicator needle detection coordinate sequence are in a corresponding relationship, the instruction does not require complicated coordinate conversion processing. If the pixel value of the pixel indicating the position of the needle can be specified and the index corresponding to the pixel value can be specified, the indicator needle angle corresponding to the index can be derived. As a result, the amount of calculation and the amount of memory used for the analysis are smaller than the method of analyzing the entire captured image.

Further, since the image processing apparatus of the first embodiment specifies the position of the indicator needle based on the pixel value of the coordinate value corresponding to the index of the indicator needle detection pixel value sequence, the shape of the indicator needle is limited by the prior art. Because it is less than, there are more types of analog meters that can be applied.

Further, since the image processing apparatus of the first embodiment requires a small amount of calculation for the coordinate conversion processing, the power consumption required for the processing is also small. As the image processing device, a general-purpose arithmetic processing device such as a microcomputer is used, and it is assumed that power is supplied from an internal power source (not shown) such as a battery, but the image processing device of the first embodiment is applied. By doing so, the life of the internal power supply can be extended.

(B) Second Embodiment Next, a second embodiment of the image processing apparatus, the image processing program, the image processing method, and the information collecting system according to the present invention will be described in detail with reference to the drawings.

(B-1) Basic Concept of the Second Embodiment When the camera shoots an analog meter, for example, the analog meter or the camera may receive vibration from equipment or the like depending on the operating environment or the like. is there. In such a case, the position of the analog meter or the camera may move slightly, and the analog meter may not be able to take an image in an upright posture.

In addition, depending on the installation environment of the camera that shoots the analog meter, the camera viewpoint may be in a rotating posture from the beginning, and the camera may not be able to shoot the analog meter in an upright posture. For example, as shown in FIG. 10 (A), it is assumed that the camera is tilted with respect to the analog meter (in FIG. 10 (A), the camera is tilted 90 degrees to the right), and such a case is assumed. In the captured image, as shown in FIG. 10B, the analog meter may be reflected in a rotated state.

In order to solve the above problems, there are methods such as using a camera having a function of correcting the orientation of the captured image according to the posture of the camera, or using an instrument for maintaining the posture of the camera. The introduction cost will increase.

Therefore, it is required to more efficiently detect the position and orientation of the analog meter reflected in the captured image, perform image processing on the captured image, and efficiently read the indicated needle indicated value indicated by the indicator needle.

(B-2) Configuration of the Second Embodiment FIG. 11 is an internal configuration diagram showing the internal configuration of the image processing apparatus 3 according to the second embodiment.

In FIG. 11, the image processing device 3A according to the second embodiment has an image input unit 31, an attitude information detection unit 32A, a storage unit 33, a projection conversion parameter calculation unit 34, an indicator needle detection coordinate sequence projection conversion unit 35, and an instruction. It has a needle detection unit 36, an indicated value calculation unit 37, and an output unit 38.

In the image processing device 10A of the second embodiment, the method of detecting the posture information by the posture information detection unit 32A is different from that of the first embodiment. Since the other components can be the same as or corresponding to the components described in the first embodiment, the components different from the first embodiment will be mainly described below.

In the second embodiment, a jig attached to the analog meter 1 in order to detect the posture information of the analog meter 1 of the captured image (original image) of the camera 2 (hereinafter, also referred to as “posture information detection jig”). 50 is used. Then, after attaching the posture information detection jig 50 to a position surrounding the scale plate of the analog meter 1, the camera 2 photographs the analog meter 1, and the posture information detection unit 32A shows the posture information reflected in the captured image. Posture information is detected by specifying the position of the feature point of the detection jig 50.

FIG. 12 is a configuration diagram showing the configuration of the posture information detection jig 50 according to the second embodiment.

The posture information detection jig 50 has a substantially square shape as a whole. When the attitude information detection jig 50 is regarded as a substantially square, the region near the apex on the two sides forming one of the four vertices and the region near the apex of the other three vertices are different from each other. It is configured to have a color.

For example, a region near the apex of the point A of the posture information detection jig 50 will be described. The region R11 and the region R12 are regions formed in the same color. In the region R11, the upper left end 501 is curved, and the lower left end 502 is formed in a straight line. In the region R12, the upper left portion 503 is curved and the upper end right portion 504 is formed in a straight line. As described above, the point A is located at the boundary point between the region R11 and the region R12.

Next, a region near the apex of the point B will be described on behalf of the other vertices, the point B, the point C, and the point D. The region R21 and the region R22 are formed in the same color. The region R21 and the region R22 are regions of colors at positions separated from each other on the color wheel when the colors are expressed by the HSI color system with respect to the regions R11 and R12. In the region R21, the upper right end 505 is curved and the lower right end 506 is formed linearly, and in the region R22, the upper right portion 507 is curved and the upper end left portion 508 is formed linearly. There is. The point B is located at the boundary point between the region R21 and the region R22.

Further, for example, on the upper side of the substantially square posture information detection jig 50, there are regions R11 and regions R21 having different colors, which means that the posture information detection jig 50 has a region near the apex of the point A and a region near the apex. This is because the region near the apex of the point B is configured to have a different color. When performing projective transformation, the correspondence between the conversion source point and the conversion destination point is required. Therefore, in this embodiment, among the four vertices of the posture information detection jig 50, the vertices of the point A are used to distinguish the point A from the other points (point B, point C, point D). The colors of the neighborhood area and the area near the vertices of other points are different. The same applies to the left side of the posture information detection jig 50.

FIG. 13 is an explanatory diagram illustrating a state in which the posture information detection jig 50 according to the second embodiment is attached to the analog meter 1.

The posture information detection jig 50 is attached to the outer frame 111 of the analog meter 1 so as to surround the scale board on the display surface of the scale board of the analog meter 1. At this time, the posture information detection jig 50 is arranged so that the center of the posture information detection jig 50 and the rotation center of the indicator needle are on the same axis, and the center of the posture information detection jig 50 and the posture The posture information detection jig 50 is arranged so that the point A of the information detection jig 50 and the position of the scale “0” of the analog meter 1 are aligned on the straight line L1.

(B-3) Operation of the Second Embodiment (B-3-1) Overall Operation Next, the operation of image processing by the image processing apparatus 3A according to the second embodiment will be described in detail with reference to the drawings. To do.

FIG. 14 is a flowchart showing the operation of image processing by the image processing device 3A according to the second embodiment.

[Step S1]
The captured image (original image) taken by the camera 2 is input to the image input unit 31 of the image processing device 3.

Here, the posture information detection jig 50 of FIG. 12 is attached to the analog meter 1, and the analog meter 1 to which the posture information detection jig 50 is attached is photographed in the input captured image. It is assumed that there is. Further, the format of the image captured by the camera 2 is assumed to have color information such as an RGB format.

[Step S20]
The posture information detection unit 32A detects the posture information detection jig 50 from the input captured image (original image), and the four vertices (point Am, point Bm, point) of the posture information detection jig 50 in the image. The XY coordinate values of Cm and point Dm) are acquired.

Here, the XY coordinate values of the four vertices (point Am, point Bm, point Cm, point Dm) detected by the attitude information detection unit 32A are referred to as attitude information. A detailed description of the method of detecting the posture information detection jig 50 from the original image will be described later.

[Step S3]
The projective conversion parameter calculation unit 34 configures XY coordinate values of four vertices (point Am, point Bm, point Cm, point Dm) detected by the attitude information detection unit 32A, and preset attitude reference information. The projective transformation parameters are derived using a total of eight XY coordinate values including the XY coordinate values of the four reference points (point Ar, point Br, point Cr, point Dr).

FIG. 15 is an explanatory diagram illustrating a projective transformation in which the reference points Ar, Br, Cr, and Dr of the posture reference information are used as the conversion source and the points Am, Bm, Cm, and Dm are the conversion destinations.

As shown in FIG. 15, in the second embodiment as well as the first embodiment, for example, the four XY coordinates of the points Ar, Br, Cr, and Dr constituting the posture reference information of FIG. 4 (B). Each value is used as a conversion source, and each of the four XY coordinate values of Am, Bm, Cm, and Dm that constitute the posture information detected by the posture information detection unit 32A from the original image is used as the conversion destination, and each point of the conversion source. Is calculated as a projective conversion parameter that can be projected to each point of the conversion destination.

[Step S4]
The indicator needle detection coordinate sequence projection conversion unit 35 uses the projection conversion parameters derived from the projection conversion parameter calculation unit 34 to project all the coordinate values of the indicator needle detection coordinate sequence before the projection conversion.

Also in the second embodiment, all the coordinate values constituting the indicator needle detection coordinate sequence of FIG. 5 (B) of the first embodiment are projected and transformed.

[Step S5]
In the same manner as in the first embodiment, the indicator needle detection unit 36 acquires pixel values from the captured image (original image) by using all the coordinate values constituting the indicator needle detection coordinate sequence after the projective conversion. The indicator needle detection pixel value sequence 361 is created.

As illustrated in FIG. 9, the indicator needle detection pixel value sequence 361 has a structure in which data in which an index and a pixel value are paired is set as one unit, and a plurality of the data are connected. The number of data in the indicator needle detection pixel value sequence 361 is the same as the number of data in the indicator needle detection coordinate sequence 332. The index of the indicator needle detection pixel value sequence 361 and the index of the indicator needle detection coordinate sequence 332 have a corresponding relationship with each other.

[Step S6]
The indicator needle detection unit 36 calculates the angle of the indicator needle with reference to all the pixel values constituting the indicator needle detection pixel value sequence 361.

In the second embodiment, as in the first embodiment, for example, the luminance value of each pixel is calculated for all the pixel values constituting the indicator needle detection pixel value sequence 361, and each luminance value is used using the threshold value. May be binarized to obtain the index of the pixel having the minimum value from all the luminance values. Thereby, by referring to the indicator needle detection coordinate sequence 332 or the indicator needle detection pixel value sequence 361 in which the indicator needle angle is associated with each index, the indicator needle angle corresponding to the index can be derived. Further, as in the first embodiment, for example, the brightness value of each pixel is calculated for all the pixel values constituting the indicator needle detection pixel value string 361, and each brightness value is binarized using the threshold value to obtain black color. The index of the pixel in the center of the section having the largest number of consecutive pixels may be obtained.

Here, the reason for deriving the index having the minimum luminance value will be described. For example, a pixel in which the indicator needle is located in the original image has a smaller pixel value (for example, a brightness value) than a pixel in which the indicator needle is not located. Therefore, the indicator needle detection unit 36 can specify the position of the indicator needle by selecting the pixel value string having the smallest pixel value from the pixel value strings constituting the indicator needle detection pixel value sequence.

Further, as described above, since the index of the indicator needle detection pixel value sequence 361 and the index of the indicator needle detection coordinate sequence 332 have a corresponding relationship, the position of the indicator needle can be determined without requiring complicated coordinate conversion processing. If the pixel value of the indicated pixel can be specified and the index corresponding to the coordinate value of the pixel can be specified, the indicator needle angle corresponding to the index can be derived.

[Steps S7, S8]
The instruction value calculation unit 37 calculates the instruction value of the instruction needle based on the instruction needle angle detected by the instruction needle detection unit 36 with reference to the instruction needle detection coordinate sequence 332. Then, the output unit 38 outputs the information including the indicated value to the subsequent stage.

(B-3-2) Detection Method of Posture Information Detection Jig 50 FIG. 16 is a flowchart showing a detection method of the posture information detection jig 50 of the posture information detection unit 32A according to the second embodiment.

Here, the attitude information detection jig 50 is substantially square, and of the four vertices, the two side regions R11 and R12 forming the point A are given "color A", and the other three It is assumed that "color B" is applied to the regions R21, region R22, region R23, and region R24 of each side forming the vertices (point B, point C, point D).

[Step S21]
The posture information detection unit 32A creates an image in which the color A applied to the areas R11 and R12 is emphasized (this is also referred to as a “color A emphasized image”) from the original image.

[Step S22]
The posture information detection unit 32A performs corner detection on the created color A emphasized image. Here, as the corner detection method, for example, Harris corner detection can be used.

[Step S23]
The posture information detection unit 32A selects one corner considered to be the most appropriate from the corners detected in step S22. The method of selecting a corner will be described later.

[Step S24]
Subsequently, the posture information detection unit 32A emphasizes the color B forming the three vertices other than the point A among the four vertices of the posture information detection jig 50 (hereinafter, also referred to as “color B emphasized image”). Call.) Is created from the original image.

[Step S25]
The posture information detection unit 32A performs corner detection on the created color B emphasized image. As for this corner detection method, for example, Harris corner detection is used as in step S22.

[Step S26]
The posture information detection unit 32A selects three corners considered to be the most appropriate from the corners detected in step S25. The method of selecting a corner will be described later.

[Step S27]
Subsequently, the posture information detection unit 32A assigns labels to the four points detected in steps S23 and S26.

Here, a label At is assigned to any of the four points, and in the original image, labels are assigned in the order of point Bt, point Ct, and point Dt in the clockwise direction starting from the point At. A detailed description of assigning labels to the four points will be described later.

[Step S28]
The posture information detection unit 32A replaces the label so that the label of the point formed by the region of the color A of the posture information detection jig 50 becomes At.

At this time, the labels are replaced so as to maintain that the arrangement of the labels is clockwise. Details of the method for determining whether each point is formed by the color A or the color B of the posture information detection jig 50 will be described later. After that, At, Bt, Ct, and Dt are replaced with Am, Bm, Cm, and Dm, respectively, and output is completed.

[Explanation of corner selection method]
Next, the corner selection method in steps S22 and S25 will be described. Here, the points A, B, C, and D, which are the four vertices of the posture information detection jig 50 shown in the captured image (original image), are detected.

The point A is the boundary point between the two regions to which the color A is applied, and is located at the corner which is the intersection of the two prominent edges. Therefore, the point A can be selected from the color A emphasized image. A can be detected.

Further, the points B, C, and D are the boundary points between the two regions to which the color B is applied, and are located at the corners that are the intersections of the two distinct edges, so that the color B is emphasized. Point B, point C, and point D can be detected by selecting a corner from the image.

First, the posture information detection unit 32A creates a binary image (corner binary image) in which the pixels determined to be corners are white and the pixels determined not to be corners are black.

After that, the corner binary image is subjected to expansion processing an appropriate number of times. The corner to be selected in this process is the apex of the posture information detection jig 50. Here, the vertices of the posture information detection jig 50 are distributed with a sufficient distance in the image region. Therefore, it is considered that the corners that are very close to each other are derived from one corner, so this expansion process is performed.

Subsequently, the posture information detection unit 32A counts the number of lumps in the white region in the corner binary image that has undergone the expansion process. Subsequent processing changes depending on the number of the lumps.

If the number of lumps is less than the target number, the threshold value for determining the corner in the corner detection process is relaxed, the corner detection process is retried, or the process is restarted from the re-shooting of the original image.

Here, the target number means the number of corners to be detected. For example, in step S22, since one corner is detected from the color A emphasized image, the target number is one in this case. Further, in step S25, since the case is to detect three corners from the color B emphasized image, in this case, the target number is three.

On the other hand, when the number of lumps exceeds the target number, it is necessary to select as many lumps as the target number from the lumps, and for example, the following lump selection method can be used.

For example, consider a region having an appropriate radius centered on the coordinate value of the center of gravity in each mass region (hereinafter referred to as a mass region). The ratio of the color composition of the mass region in the original image is compared with the ratio of the color composition of the region having an appropriate radius centered on the apex of the posture information detection jig 50 (hereinafter, the jig apex region). Then, the upper required number is selected from those having a similar ratio of the color composition of the mass region and the color composition ratio of the jig apex region.

In determining the similarity of ratios, for example, the color ratio of each mass region and the jig vertex region is considered as a vector, the cosine similarity is calculated from the vector of each mass region and the vector of the jig vertex region, and the cosine similarity is determined. A method such as selecting a high upper required number may be used.

Then, when the number of lumps matches the target number, one pixel which is a corner is selected from each lump. As the method for selecting one pixel, for example, any method can be used, and for example, a method for selecting one pixel located at the center of gravity of the mass region as a corner can be used.

[Detailed operation of step S27]
FIG. 17 is an explanatory diagram illustrating a method of assigning labels to the four vertices of the second embodiment.

First, as illustrated in FIG. 17A, the posture information detection unit 32A assigns temporary labels (point At, point Bt, point Ct, point Dt) to four points. The label may be attached by any method at this time.

Subsequently, the posture information detection unit 32A replaces the temporary labels given to the four points as follows.

First, it is determined whether or not the line segment At-Bt and the line segment Ct-Dt intersect. If they intersect, the labels for points At and Dt are swapped. For example, in FIG. 17B, since the line segment At-Bt and the line segment Ct-Dt intersect, the labels of the point At and the point Dt are exchanged.

Subsequently, it is determined whether or not the line segment Dt-At and the line segment Bt-Ct intersect. When they intersect, the labels for points At and Bt are swapped. For example, in FIG. 17C, since the line segment Dt-At and the line segment Bt-Ct do not intersect, the labels are not replaced.

After that, it is determined whether or not the four points are arranged clockwise on the image. When arranging counterclockwise, the labels of point At and point Ct are exchanged. For example, as shown in FIG. 17C, when looking at the clockwise arrangement starting from the point At, the points At → the point Dt → the point Ct → the point Bt are not arranged in this order, so the points At and the point Ct Replace the label. Then, as shown in FIG. 17D, the labels are arranged in order starting from the point At.

The specific means for determining the line segment intersection and the arrangement direction of the points described above is not limited to the method described above, and any method can be used as long as the labels can be arranged in order starting from the point At. Means may be used.

[Determination method of step S28]
Next, the operation of the determination method in step S28 will be described. This operation is similar to the mass selection method in the above-mentioned corner selection method.

First, consider the color ratio in the original image of a region having an appropriate radius centered on the coordinates of the four vertices (hereinafter, apex region) and a region formed by color A (hereinafter, color A region).

The color ratio of each vertex region and the color ratio of the color A region are considered as vectors, and the cosine similarity is calculated from the vector of the vertex region and the vector of the color A region. Then, the point having the highest cosine similarity is selected as the point At.

(B-4) Effect of Second Embodiment As described above, the image processing apparatus of the second embodiment has the same effect as that of the first embodiment.

Further, according to the image processing device of the second embodiment, the attitude information detection jig is attached to the analog meter, the camera takes a picture of the analog meter to which the posture information detection jig is attached, and the image processing device is the posture. Using the positions of the four vertices of the information detection jig, a projection conversion parameter indicating the difference between the ideal position and orientation of the analog meter reflected in the captured image is derived, and the indicator needle detection coordinates are derived using the projection conversion parameter. Projection transforms all coordinate values (coordinate information) of a column. Then, the indicated values of the indicator needle can be read based on the pixel values of all the coordinate values of the indicator needle detection coordinate sequence after the projective conversion.

In the second embodiment, since a substantially square posture information detection jig composed of a combination of colors at distant positions on the color wheel is used, the hue difference between the colors is sufficient, so that the posture information detection jig The positions of the four vertices can be easily derived. As a result, the projective conversion parameters can be easily derived based on the four vertices of the attitude information detection jig and the four reference points of the attitude information.

Further, since the second embodiment is a simple method of attaching the posture information detection jig to the analog meter, the introduction cost of the indicated value reading system can be suppressed. Further, if the surface of the dial of the analog meter and the attitude information detection jig are reflected in the shooting range of the camera, the effect of the present invention is fully exhibited, so that the relative position between the analog meter and the camera is exhibited. It is possible to relax restrictions on relationships and camera orientation. Therefore, the degree of freedom in installing the camera with respect to the analog meter is increased.

(C) Third Embodiment Next, a third embodiment of the image processing apparatus, the image processing program, the image processing method, and the information collecting system according to the present invention will be described in detail with reference to the drawings.

(C-1) Configuration of Third Embodiment In the third embodiment, similarly to the second embodiment, a posture information detection jig is attached to the analog meter, and the posture information detection unit moves the posture from the captured image. The position of the four vertices of the information detection jig is detected, and the projective conversion parameter is derived based on the coordinate values of the four vertices and the four reference points of the posture reference information.

However, in the third embodiment, the configuration of the posture information detection jig attached to the analog meter and the process of detecting the positions of the four vertices of the posture information detection jig from the image captured by the posture information detection unit are the second. Different from the embodiment.

Therefore, the image processing apparatus of the third embodiment will be described focusing on the components different from those of the second embodiment by using FIG. 11 of the second embodiment.

FIG. 18 is a configuration diagram showing the configuration of the posture information detection jig 50A according to the third embodiment.

The posture information detection jig 50A has a substantially square shape as a whole. Each side of the substantially square posture information detection jig 50A is composed of two colors. For the color combination of each side composed of two colors, for example, when the color is expressed by the HSI color system, the color at a distant position on the hue circle is selected.

On each side composed of two colors, the boundary line of each color region is a straight line a, a straight line b, a straight line c, and a straight line d. Further, the vertex formed by the straight line d and the straight line a is defined as the point A, the vertex formed by the straight line a and the straight line b is defined as the point B, the vertex formed by the straight line b and the straight line c is defined as the point C, and the vertex formed by the straight line c and the straight line d is defined as the point D. There is.

Of the four sides, the two sides having the straight line b and the straight line c as the boundary lines have the same color combination. Therefore, each side of the posture information detection jig 50A is formed of three sets of color combinations. As described above, it is desirable to use at least three color combinations.

FIG. 19 is an explanatory diagram illustrating a state in which the posture information detection jig 50A according to the third embodiment is attached to the analog meter 1.

The method of attaching the posture information detection jig 50A to the analog meter 1 is basically the same as that of the second embodiment. That is, the posture information detection jig 50A is provided on the outer frame 111 of the analog meter 1 so as to surround the scale plate of the analog meter 1.

At this time, the posture information detection jig 50A is arranged so that the center of the posture information detection jig 50A and the rotation center of the indicator needle are on the same axis, and the center of the posture information detection jig 50A and the posture. The posture information detection jig 50 is arranged so that the point A of the information detection jig 50A and the position of the scale "0" of the analog meter 1 are aligned on the straight line L1.

(C-2) Operation of Third Embodiment The image processing method of the image processing apparatus 3A according to the third embodiment is basically described in the second embodiment using the flowchart of FIG. Same as the method.

Therefore, in the third embodiment, the process of detecting the positions of the four vertices of the posture information detection jig 50A by the posture information detection unit 30A will be described in detail.

FIG. 20 is a flowchart showing a detection method of the posture information detection jig 50 of the posture information detection unit 32A according to the third embodiment.

[Step S31]
The posture information detection unit 32A converts the captured image (original image) input from the image input unit 11 into HSI. As a result, the captured image can be made into an image composed of three components of hue, saturation, and brightness.

[Steps S32 to S37]
In steps S32 to S37, the H component difference image is created by focusing on the hue (H component) of each pixel among the three components of hue, saturation, and brightness. Here, among all the pixels of the original image, the following processing is performed with a certain pixel as the pixel of interest, and the processing of steps S32 to S37 is repeated for all the pixels.

[Step S33]
In the original image, the posture information detection unit 32A uses a certain pixel as a pixel of interest, and calculates the absolute value of the difference between the value of the H component of the pixel of interest and the value of the H component of the eight pixels around the pixel of interest. As a result, the absolute difference value of eight H components can be obtained for one pixel of interest.

[Step S34]
The posture information detection unit 32A determines the magnitude relationship of the difference absolute values of the eight H components with respect to the pixel of interest, and sets the maximum value as the H component difference value of the pixel of interest.

[Step S35]
Next, the posture information detection unit 32A acquires the value of the saturation (S component) of the pixel of interest and compares the value of the S component with the threshold value. Then, if the value of the S component of the pixel of interest is less than the threshold value, the process proceeds to step S36. If the value of the S component of the pixel of interest is not less than the threshold value, the process proceeds to step S37.

[Step S36]
When the value of the S component of the pixel of interest is less than the threshold value, the difference value of the H component of the pixel of interest is set to 0.

[Step S37]
The posture information detection unit 32A performs the processes of steps S32 to S37 on all the pixels of the captured image (original image) to create an H component difference image. Therefore, when the processing is not completed for all the pixels, the processing of steps S32 to S37 is performed for all the pixels.

[Step S38]
The attitude information detection unit 32A applies the Hough transform to the H component difference image, and detects the four straight lines L11, L12, L13, and L14 constituting the attitude information detection jig 50A. Each of these four straight lines corresponds to the boundary between two colors on each side constituting the posture information detection jig 50A.

[Step S39]
The posture information detection unit 32A detects four intersections using the four detected straight lines.

Specifically, of the four straight lines, one straight line is used as a reference straight line, and the absolute value of the difference between the slope of the reference straight line and the slopes of the other three straight lines is calculated. Then, two straight lines having a large absolute value of the difference in slope are selected. The intersection of the reference straight line and the two straight lines is calculated. As a result, two intersections are obtained.

By performing the above-mentioned processing for each of the four straight lines and removing the duplication of the coordinate values obtained by this, four coordinate values can be obtained.

[Step S40]
The posture information detection unit 32A labels the four straight lines detected in step S38 as straight lines a, b, c, and d, respectively. Of the four straight lines, two of the intersections detected in step S39 are present on a certain straight line. From the value of the H component of the pixel in the region near the line segment connecting the two intersections in the original image, which straight line of the posture information detection jig 50A is identified and labeled.

[Step S41]
The posture information detection unit 32A labels the four intersections detected in step S39 as points A, B, C, and D.

At the time of labeling, the label of the straight line labeled in step S40 is used in order to determine which two straight lines are the intersection.

(C-3) Effect of Third Embodiment As described above, according to the image processing apparatus of the third embodiment, the same effect as that of the first and second embodiments can be obtained.

(D) Fourth Embodiment Next, a fourth embodiment of the image processing apparatus, the image processing program, the image processing method, and the information collecting system according to the present invention will be described in detail with reference to the drawings.

Hereinafter, the case where the fourth embodiment is applied to the second or third embodiment will be illustrated, but the same effect can be obtained when the fourth embodiment is applied to the first embodiment. ..

FIG. 21 is an internal configuration diagram showing an internal configuration of the image processing apparatus according to the fourth embodiment.

In FIG. 21, the image processing device 3B of the fourth embodiment has an image input unit 31, an attitude information detection unit 32A, a storage unit 33, a projection conversion parameter calculation unit 34B, an original image projection conversion unit 39, and an indicator needle detection unit 36B. , The indicated value calculation unit 37, and the output unit 38.

In the fourth embodiment, each process of the projective conversion parameter calculation unit 34B and the indicator needle detection unit 36B, and the indicator needle detection coordinate sequence projection conversion unit 35 of the first to third embodiments are performed by the original image projection conversion unit 39. It is different from the first to third embodiments in that it is replaced with.

The projective conversion parameter calculation unit 34B includes XY coordinate values of the four vertices Am, Bm, Cm, and Dm detected from the captured image (original image), and four reference points Ar, Br, which constitute the attitude reference information 331. The point at which the projective conversion parameter is calculated based on the XY coordinate values of the eight points of the XY coordinate values of Cr and Dr is the same as in the first to third embodiments, but the method of calculating the projective conversion parameter is different. It is different from the first to third embodiments.

That is, the projective conversion parameter calculation unit 34B uses the four vertices Am, Bm, Cm, and Dm extracted from the captured image (original image) detected from the captured image (original image) as conversion sources, and provides the posture reference information 331. Projection conversion is performed using the four constituent reference points Ar, Br, Cr, and Dr as conversion destinations. As described above, the coordinate values of the conversion source and the conversion destination are different from those of the first to third embodiments.

The original image projective conversion unit 39 projects a captured image (original image) using the projective conversion parameters calculated by the projective conversion parameter calculation unit 34B.

The indicator needle detection unit 36B refers to each coordinate value constituting the indicator needle detection coordinate sequence 332, acquires a pixel value from the original image projected by the original image projection conversion unit 39, and obtains a pixel value from the original image projected by the original image projection conversion unit 39. Is to create.

That is, in the first to third embodiments, the projection conversion parameters are used to project-transform each coordinate value constituting the indicator needle detection coordinate sequence to acquire the pixel value of the corresponding coordinate value in the original image. , The indicator needle detection pixel value sequence was created, but the indicator needle detection unit 36B of the fourth embodiment obtains the pixel values of the coordinate values constituting the indicator needle detection coordinate sequence from the original image after the projective conversion. Acquire and create an indicator needle detection pixel value string.

(D-2) Operation of the Fourth Embodiment FIG. 22 is a flowchart showing image processing in the image processing apparatus 3B according to the fourth embodiment.

[Steps S1, S20]
The captured image (original image) captured by the camera 2 is input to the image input unit 31 of the image processing device 3 (step S1), and the posture information detection unit 32A receives the posture information from the input captured image (original image). The detection jig 50 is detected, and the XY coordinate values of the four vertices (point Am, point Bm, point Cm, point Dm) of the attitude information detection jig 50 in the image are acquired (step S20).

[Step S50]
The projective transformation parameter calculation unit 34 uses the XY coordinate values of the four vertices (point Am, point Bm, point Cm, point Dm) detected by the attitude information detection unit 32 as the conversion source, and constitutes the attitude reference information 4. Using the XY coordinate values of the reference points (point Ar, point Br, point Cr, point Dr) as the conversion destination, the projection conversion parameters that can project the conversion source point to the conversion destination point are calculated.

[Step S51]
The original image projection conversion unit 39 performs a projection conversion of the captured image (original image) using the projection conversion parameters calculated by the projection conversion parameter calculation unit 34B.

[Step S52]
Then, the indicator needle detection unit 36B refers to each coordinate value constituting the indicator needle detection coordinate sequence 332, acquires a pixel value from the original image projected by the original image projection conversion unit 39, and obtains a pixel value from the original image projected by the original image projection conversion unit 39, and the indicator needle detection pixel Create a value column.

[Steps S6 to S8]
Since the processes of steps S6 to S8 are the same as those of the first to third embodiments, detailed description thereof will be omitted here.

(D-3) Effect of Fourth Embodiment As described above, the same effect as that of the first to third embodiments can be obtained by the image processing apparatus of the fourth embodiment.

(E) Other Embodiment (E-1) In the first to third embodiments described above, a plurality of pixel values extracted from the captured image are detected with reference to the projective-transformed indicator needle detection coordinate sequence. I illustrated that. Further, in the fourth embodiment, the case where the position of the indicator needle is detected from a plurality of pixel values extracted from the captured image projected and transformed using the indicator needle detection coordinate sequence is illustrated.

At this time, when detecting the position of the indicator needle from the pixel value of each coordinate value of the indicator needle detection coordinate string, the pixel value of a certain area including each coordinate value of the indicator needle detection coordinate string is taken into consideration, and each coordinate value is included. The position of the indicator needle may be detected from the pixel value in a certain region. This is because, for example, as shown in FIG. 24 (A), a number such as "0" is assigned to the dial, and at that time, the pixel value of the coordinate value in the captured image becomes small. There is. In such a case, it can be avoided by setting the position of the virtual circle VC2 in which each coordinate value of the indicator needle detection coordinate sequence is arranged at a position not covered with the number on the dial. However, if there is a virtual circle VC2 at the position where it is covered with the numbers on the scale, as shown in FIG. 24 (B), the area having a certain extent including each coordinate value of the indicator needle detection coordinate sequence The pixel value may be taken into consideration.

(E-2) In the first to fourth embodiments described above, regarding the calculation of the projection conversion parameter, it is expressed as if the projection conversion parameter is calculated every time the image captured from the camera 2 is input. The calculation process of the projective conversion parameter may be performed at the time of initial setting of the image processing system, or may be performed at predetermined time intervals.

That is, it is not limited to calculating the projective conversion parameter each time the image processing is performed, but the projective conversion parameter calculated once is retained, and the projective conversion parameter is used to use the indicator needle detection coordinate sequence or The original image may be projected and converted. As a result, the processing load related to the calculation of the projective conversion parameters can be reduced. Of course, the projective conversion parameter may be calculated each time the image processing is performed.

(E-3) In the second to fourth embodiments described above, when the posture information detection jig is attached to the analog meter, the posture information detection jig is attached so as to surround the range in which the indicator needle of the analog meter moves. Illustrated. However, the position where the posture information detection jig is attached is not limited to this.

The attitude information detection jig uses the XY coordinate values of the four vertices of the attitude information detection jig shown in the captured image and the XY coordinate values of the four reference values of the attitude reference information, and the projective conversion parameter. Contributes to deriving. Therefore, as long as the posture information detection jig is shown together with the indicator needle of the analog meter in the captured image, the mounting position is not particularly limited.

In this case, it should be noted that the XY coordinate values of the four vertices constituting the attitude information detection jig are determined in consideration of the diameter and position of the circle (virtual circle VC2) formed by the indicator needle detection coordinate sequence. In addition, when shooting with a camera, it is necessary that the posture information detection jig and the movable range of the indicator needle of the analog meter are all within the imaging range, so attention must be paid to this point as well.

(E-4) In the second to fourth embodiments described above, the case where the posture information detection jig is substantially square is illustrated, but the shape of the posture information detection jig may be any quadrangle. In this case, it is necessary to determine the coordinates of the four vertices constituting an arbitrary quadrangle in consideration of the diameter and position of the circle formed by the indicator needle detection coordinate sequence.

(E-5) In the first to fourth embodiments described above, the case where the indicator needle is a rotating analog meter is illustrated, and the coordinate group (indicator needle detection coordinate sequence) for detecting the position of the indicator needle is The case where it exists on the virtual circle VC2 is illustrated. However, each coordinate value of the indicator needle detection coordinate sequence may be arranged on a straight line, or may have any other shape. In this case, it should be noted that the coordinates of the four vertices constituting the posture information detection jig are determined in consideration of the shape formed by the coordinate group.

For example, as shown in FIG. 25 (A), it may be a horizontal instrument in which the indicator needle moves in the lateral direction. In this case, as shown in FIG. 25 (B), each coordinate constituting the indicator needle detection coordinate sequence. The values may be arranged on a straight line in the lateral direction (X-axis direction) according to the movement of the indicator needle.

Further, it may be a vertical instrument in which the indicator needle moves in the vertical direction. In this case, each coordinate value constituting the indicator needle detection coordinate sequence is in the vertical direction (Y-axis direction) according to the movement of the indicator needle. It may be arranged on a straight line.

(E-6) In the second to fourth embodiments described above, when the posture information detection jig is attached to the analog meter, the center of the posture information detection jig and the rotation center of the indicator needle are aligned on the same axis. , An example of attaching to the posture information detection jig analog meter so that the center of the posture information detection jig, the point A of the posture information detection jig, and the position of the scale "0" of the analog meter are aligned on a straight line will be described. did. However, the arrangement is not limited to this, and the attitude is such that the center of the attitude information detection jig, the point A of the attitude information detection jig, and the scale "0" of the analog meter are aligned on a straight line. It is not necessary to attach an information detection jig. In this case, it is necessary to give the image processing apparatus an angle formed by the point A of the posture information detection jig, the center of the posture information detection jig, and the scale "0" of the analog meter.

10 ... Information collection system, 1, 1A ... Analog meter, 2 ... Camera, 3, 3A, 3B ... Image processing device, 4 ... Communication device, 5 ... Communication device, 6 ... Management device, 31 ... Image input unit, 32, 32A ... Attitude information detection unit, 33 ... Storage unit, 331 ... Attitude reference information, 332 ... Indicator needle detection coordinate sequence, 34, 34B ... Projection conversion parameter calculation unit, 35 ... Indicator needle detection coordinate sequence projection conversion unit, 36, 36B ... Indicator needle detection unit, 37 ... Instruction value derivation unit, 38 ... Output unit, 39 ... Original image projection conversion unit, 361 ... Indicator needle detection pixel value sequence, 50, 50A ... Attitude information detection jig.

Claims (12)

In an image processing device that derives an indicated value of the indicator needle instrument from an image captured by photographing the display surface of the indicator needle instrument.
The position of the indicator needle of the indicator needle meter is shown in the reference image by using the projective conversion parameter of the virtual reference image of the indicator needle instrument and the image of the indicator needle instrument in the input captured image. An indicator needle detection coordinate sequence having a plurality of coordinate values, or a projective conversion unit for projecting and converting the captured image.
From a plurality of pixel values extracted from the captured image using the projective-transformed indicator needle detection coordinate sequence, or from a plurality of pixel values extracted from the image captured image projected using the indicator needle detection coordinate sequence. , The indicator needle detection unit that detects the position of the indicator needle,
An image processing apparatus including an instruction value derivation unit for deriving the instruction value corresponding to the position of the instruction needle detected by the instruction needle detection unit. The indicator needle detection coordinate sequence
It has the plurality of coordinate values provided in correspondence with the scale plate of the indicator needle instrument within the region where the indicator needle exists in the reference image.
The image processing apparatus according to claim 1, wherein at least the plurality of coordinate values are associated with the position of the indicator needle in the indicator needle instrument. When the projective conversion unit projects a projective conversion of the indicator needle detection coordinate sequence using the projective conversion parameter,
The indicator needle detection unit refers to each of the coordinate values after the projection conversion of the indicator needle detection coordinate sequence, and corresponds to the smallest pixel value from the plurality of pixel values extracted from the captured image. The image processing apparatus according to claim 1 or 2, wherein the position of the indicator needle is detected. The indicator needle detection coordinate sequence is further provided with identification information for identifying the respective coordinate values for each of the plurality of coordinate values.
The indicator needle detection unit associates the plurality of pixel values extracted from the captured image with reference to the coordinate values after the projective conversion, the identification information, and the position of the indicator needle in the indicator needle instrument. The image processing according to claim 1 or 2, wherein the position of the indicator needle corresponding to the minimum pixel value is detected from a plurality of pixel values with reference to the indicator needle detection pixel value sequence. apparatus. When the projective conversion unit projects a captured image using the projective conversion parameters,
The indicator needle detection unit refers to each of the coordinate values in the indicator needle detection coordinate sequence, and corresponds to the smallest pixel value from the plurality of pixel values extracted from the captured image after the projection conversion. The image processing apparatus according to claim 1 or 2, wherein the position of the indicator needle is detected. The indicator needle detection coordinate sequence is further provided with identification information for identifying the respective coordinate values for each of the plurality of coordinate values.
Refer to the indicator needle detection pixel value sequence in which the indicator needle detection unit associates the plurality of pixel values extracted from the captured image after the projective conversion, the identification information, and the position of the indicator needle in the indicator needle instrument. The image processing apparatus according to claim 1 or 2, wherein the position of the indicator needle corresponding to the minimum pixel value is detected from a plurality of pixel values. From the input captured image, a posture information detection unit that detects each coordinate value of a plurality of vertices that specifies the posture of the indicator needle instrument reflected in the captured image, and
A projective conversion parameter derivation unit that derives the projective conversion parameter based on the coordinate values of the plurality of reference points constituting the reference image and the coordinate values of the plurality of vertices detected by the attitude information detection unit. The image processing apparatus according to any one of claims 1 to 6, wherein the image processing apparatus comprises. The posture information detection unit identifies the shape of the indicator needle meter reflected in the input captured image, and detects each coordinate value of the plurality of vertices corresponding to the shape of the indicator needle instrument. The image processing apparatus according to claim 7. The posture information detection unit is characterized in that it detects each coordinate value of the plurality of vertices constituting the posture information detection jig attached to the display surface of the indicator needle instrument reflected in the input captured image. The image processing apparatus according to claim 7. In an image processing program that derives the indicated value of the indicator needle instrument from an image captured by photographing the display surface of the indicator needle instrument.
Computer,
The position of the indicator needle of the indicator needle instrument is shown in the reference image by using the projective conversion parameter of the virtual reference image of the indicator needle instrument and the image of the indicator needle instrument in the input captured image. An indicator needle detection coordinate sequence having a plurality of coordinate values, or a projective conversion unit for projecting and converting the captured image.
From a plurality of pixel values extracted from the captured image using the projective-transformed indicator needle detection coordinate sequence, or from a plurality of pixel values extracted from the image captured image projected using the indicator needle detection coordinate sequence. , The indicator needle detection unit that detects the position of the indicator needle,
An image processing program characterized by functioning as an instruction value derivation unit for deriving the instruction value corresponding to the position of the instruction needle detected by the instruction needle detection unit. In the image processing method for deriving the indicated value of the indicator needle instrument from the captured image obtained by photographing the display surface of the indicator needle instrument.
The projective conversion unit uses the projective conversion parameter of the virtual reference image of the indicator needle instrument and the image of the indicator needle instrument in the input captured image to indicate the indicator needle instrument in the reference image. An indicator needle detection coordinate sequence having a plurality of coordinate values indicating the position of the needle, or the captured image is projected and converted.
The indicator needle detection unit extracts a plurality of pixel values extracted from the captured image using the projective-transformed indicator needle detection coordinate sequence, or extracts from the image captured image projected using the indicator needle detection coordinate sequence. The position of the indicator needle is detected from the plurality of pixel values,
An image processing method characterized in that an instruction value deriving unit derives the instruction value corresponding to the position of the instruction needle detected by the instruction needle detection unit. An image processing device that derives the indicated value of the indicator needle instrument from the captured image obtained by photographing the display surface of the indicator needle instrument, and
A first communication device that transmits a signal including an indicated value of the indicator needle meter derived by the image processing apparatus, and a first communication device.
A second communication device that receives the signal transmitted from the first communication device, and
A management device for storing the indicated value included in the signal received by the second communication device is provided.
An information collecting system, wherein the image processing device is the image processing device according to any one of claims 1 to 9.

Images