EP1960741A1

EP1960741A1 - A method and a device for obtaining a meter reading of a counter mechanism

Info

Publication number: EP1960741A1
Application number: EP06835824A
Authority: EP
Inventors: Martin SJÖLIN; Staffan Solen
Original assignee: Anoto Group AB
Current assignee: Anoto Group AB
Priority date: 2005-12-09
Filing date: 2006-12-08
Publication date: 2008-08-27
Also published as: WO2007067132A1

Abstract

A method is presented for obtaining a meter reading from a captured image (28) of a counter mechanism (3) . The counter mechanism has at least one symbol position (10X) , each being capable of showing a plurality of symbols (10) in sequential order so as to indicate respective values of said symbol position. Symbol image data (29) for the symbol position is derived from the image. The symbol image data is processed by matching with reference image data (34) representing said plurality of symbols to determine a current value of said symbol position. An output signal (39) representing said meter reading is generated from the current value deter- mined for the or each symbol position. The reference image data (34) is provided as a first set of symbol templates (36) and a second set of symbol templates (38) , and the processing of the symbol image data (29) involves a preprocessing stage (110) and a matching stage (120) . The method involves selecting, based on an outcome of the preprocessing stage, at least one of the first and second sets of symbol templates (36, 38) for use during the matching stage.

Description

A METHOD AND A DEVICE FOR OBTAINING A METER READING QF A

COUNTER MECHANISM

Cross-reference to Related Applications

The present application claims the benefit of Swedish patent application no. 0502721-4, filed on December 9, 2005 which is hereby incorporated by reference.

Field of the Invention

The present invention generally relates to automatic meter reading, and more particularly to a method and a device for obtaining a meter reading of a counter

mechanism having at least one symbol position, each symbol position being capable of showing a plurality of symbols in sequential order so as to indicate respective values of the symbol position, by processing a captured image of the counter mechanism.

Background of the Invention

A very common example of a meter apparatus with a counter mechanism according to the above is a consumption meter such as an electricity meter, gas meter, fuel meter or water meter. The remainder of this document will therefore refer to an electricity meter and its counter mechanism, in an exemplifying but non-limiting sense.

Electricity meters are of course commonplace both in private homes and at public sites such as offices, industries or community facilities and are used to measure the consumption of electrical energy for use by electricity providers when billing the users.

A typical electricity meter 1 is shown in Fig 1.

Inside the meter's apparatus housing 2 there is provided a rotary disk 4 which is caused to rotate, as indicated by an arrow 5, around an axis 6 at a speed which is proportional to the momentary electric power consumption. The rotary disk 4 is mechanically coupled to a counter mechanism 3 in the form of a cyclometer register with a series of rotary reels 12, each reel being provided at its peripheral edge with a plurality of markings which represent respective meter symbols 10 at respective symbol positions IO₁-IO₅ and which typically are the decimal digits 0-9 in sequential order. A fraction of the peripheral edges of the reels is visible through openings in a masking plate 11, each opening being sized to dis- close one symbol in full height. The reels are arranged to rotate at different speeds, such that the reel that represents the first symbol position 1Oi rotates ten times faster than the second reel, which in turn rotates then ten times faster than the next reel, and so on.

Thus, the counter mechanism 3 of the electricity meter 1 can represent a five-digit or six-digit decimal integer, often combined with a decimal value (decimal position 1O_d in Fig 1) . In the situation shown in Fig 1, the counter mechanism 3 displays the current meter reading "12345,0". In addition, the counter mechanism 3 has a cent mark position 10_c. An electricity meter of this type is sometimes referred to as a Ferraris meter.

Conventionally, electricity meters are read by manual visual inspection of the current meter reading as provided by the current positions of the rotary reels. The manual inspection is either performed by service personnel from the electricity providers, or by the site owner himself reading the meter reading and reporting it to the electricity providers by phone or over the Inter- net.

For several reasons, it has been desired to provide automatic or remote meter reading of existing electricity meters. EP-A-O 279 759 discloses a method and a device for optical reading of figures of a counter. The device reads figures on the digital display of the counter and consists of a housing which is fixed to the counter and which contains an optical device and a scanning image analyzer. The optical device forms an image of the digital display in the image plane of the analyzer. The video signal emitted by the analyzer is converted into elementary binary signals by a digitizer. These binary signals are sent to a pattern recognition device comprising a microprocessor which is programmed to compare each combination of binary signals corresponding to one decimal figure with ten combinations which are contained in a memory and which correspond to the ten decimal figures.

Thus, the approach according to EP-A-O 279 759 means that 10 symbol templates in the form of images of the respective decimal digits or figures "0" to "9" are used for matching with symbols of a counter mechanism as contained in a captured image thereof.

If this approach is applied to the electricity meter of Fig 1, there will be a severe drawback in that the symbols of the counter mechanism 3 can only be interpreted accurately if all of them are located in more or less perfect front alignment in the openings of the masking plate, or at least so that all symbols are unambiguously visible to image sensor circuitry in a meter reading device, with at least a major part of each symbol being exposed which is sufficient for successful matching with the 10 symbol templates. However, as is well known to anyone ever having inspected an electricity meter, it is far from seldom that at least one of the counter mechanism's symbol positions is in a transitional state between two adjacent symbols, or, expressed

differently, that two adjacent markings on the same rotary reel are partially visible through the opening of the masking plate.

An example of such a situation is shown in Fig 2. Here, the two least significant integer symbol positions 1Oi and 1O₂ are in transitional states, between "9" and

"0", and "4" and "5", respectively. Moreover, the decimal position 1O_d is also transitional between "9" and "0". If at least one symbol position in the captured image is transitional and contains image components from two adjacent symbols or markings on the same reel, this prior art approach will most likely fail to provide an accurate interpretation of the complete current meter reading.

GB-2 278 471-A discloses another prior art approach of optical meter reading. This approach uses considerably more symbol templates and is therefore capable of interpreting also transitional symbols . More specifically, each symbol template is a 10x10 pixel block, and a total of 90 such symbol templates are provided, where two adjacent templates have an overlap of 9 pixel rows. The 90 symbol templates together represent a development or "flattening out" of the markings on each reel, in the form of a total set of template pixel data consisting of 90 pixel rows of 10 pixels each. When interpreting a particular symbol in a captured image, its 10x10 symbol image data is first matched with the first symbol template, consisting of rows 1-10 in the template pixel data. Then, the symbol image data is matched with the second symbol template, consisting of rows 2-11 in the template pixel data, and so on. Thus, to interpret one captured symbol, as many as 90 matchings are required.

In fact, the resolution of the symbol templates is extremely low in the approach suggested in GB-2 278 471- A. A more realistic resolution would be somewhere between 50x50 and 100x100 pixels per symbol template, and this would increase the complexity of the required matchings dramatically.

Another problem which is relevant in optical meter reading originates from the fact that the markings which represent the symbols of the counter mechanism are in fact curved, since they are provided on the peripheral edge of a cyclometer register reel. Therefore, a symbol in a captured image will be somewhat distorted compared to its ideal planar appearance. Moreover, disturbances or defects in the captured image may appear in the form of shadows or reflections, which may lead to truncated or partially non-readable symbols in the captured image.

A similar problem is skew effects in the captured image because of a misalignment in the mounting of the meter reading device with respect to the front face of the electricity meter and, particularly, its counter mechanism.

Other problems, which are due to the mechanical nature of the counter mechanism, are for instance

horizontally misaligned markings on an individual reel, or a mismatch in the timing between the different reels (e.g. such that a particular reel rotates with a certain deviation with respect to the other reels and either is ahead of them, or lags behind.

Summary of the Invention

In view of the above, an objective of the invention is to solve or at least reduce one or more of the problems discussed above. In particular, an embodiment of the invention seeks to provide a method of obtaining a meter reading optically which is efficient in terms of processing power and memory usage, is accurate and robust and deals with the different practical problems set out above, and is adapted for use with different models of electricity meters and handles individual variations between meters of the same model.

Generally, the above is achieved by a method and a device according to the attached independent patent claims .

A first aspect of the invention is a method of obtaining a meter reading from a captured image of a counter mechanism, the counter mechanism having at least one symbol position, each symbol position being capable of showing a plurality of symbols in sequential order so as to indicate respective values of said symbol position, the method involving: β

deriving symbol image data for the symbol position from the image;

processing the symbol image data by matching with reference image data representing said plurality of symbols to determine a current value of said symbol position; and

generating an output signal representing said meter reading from the current value determined for the or each symbol position,

wherein the reference image data is provided as a first set of symbol templates and a second set of symbol templates,

wherein the processing of the symbol image data involves a preprocessing stage and a matching stage, and wherein the method further involves selecting, based on an outcome of the preprocessing stage, at least one of the first and second sets of symbol templates for use during the matching stage .

The symbols of the counter mechanism each have a respective associated value, and they will typically be decimal digits, or alternatively other kinds of figures or characters. Often, only one single symbol will be shown at a time at the symbol position and consequently appear in the symbol image data derived from the captured image for the symbol position. In this case, the current value of the symbol position will be the value associated with the symbol, as determined by the matching of the symbol image data with the reference image data. An area in the captured image where such one single symbol appears will be referred to as a whole symbol Image in the following.

However, due to the nature of the counter mechanism and its showing of the symbols in sequential order at the symbol position, sometimes two adjacent symbols will both be partially visible at the same time at the symbol position. When this happens, the symbol position, and the two symbols, are said to be transitional, and if the cap- tured image of the counter mechanism is produced at this time, it will consequently contain image components from both of these symbols. An area in the captured image where image components from such two symbols appear will be referred to as a transitional symbol image in the following.

The method according to the first aspect enables optical meter reading with high accuracy using a limited number of matchings.

In one embodiment, each template in said first set unambiguously represents a respective symbol and each template in said second set represents two adjacent symbols in said sequential order. Symbol components representing symbols or parts thereof in said symbol image data are identified in the preprocessing stage, and its outcome will be indicative of a number of such symbol components having been identified. If the outcome of the preprocessing stage indicates that only one symbol component has been identified, it is likely that the symbol image data is a whole symbol image, and therefore only said first set of symbol templates is selected for use during the matching stage.

On the other hand, if the outcome of the preprocessing stage indicates that two symbol components have been identified, the second set of symbol templates may be selected for use during the matching stage. More specifically, one embodiment involves:

a) testing if either of said two symbol components fulfills first predetermined criteria, and then

b) if the testing in a) fails, selecting only said second set of symbol templates for use during the

matching stage,

c) if the testing in a) is successful, cl) selecting said first set of symbol templates and performing a first matching using said first set of symbol templates, c2) determining whether a result of the first matching fulfills second predetermined criteria, and in case said second predetermined criteria are not fulfilled, c3) selecting said second set of symbol templates and

performing a second matching using said second set of symbol templates .

After said step c3), an additional step c4) may be performed which involves choosing, as a result of said matching stage, a single best result, in terms of best correlation between symbol image data and template, from said first matching in step cl) and said second matching in step c3) .

The symbol image data may be binary image data, wherein said symbol components are identified as

contiguous pixel segments in said symbol image data where at least a predetermined number of image pixels of the same binary value are positioned adjacently to each other. Such symbol components are also referred to as connected components in the following.

The matching stage may involve correlation matching of each symbol template in the selected at least one of the first and second sets of symbol templates against symbol components as identified in said symbol image data during said preprocessing stage. In one embodiment, the correlation matching involves locating a reference position for an identified symbol component in said symbol image data, locating a reference position in a given symbol template, and determining a correlation between individual pixels in said identified symbol component and individual pixels at corresponding

positions in said given symbol template, starting from the respective reference positions. This further enables optical meter reading with high accuracy using a limited number of matchings, since the correlation matching can be performed as a single matching operation at a well- defined image position given by the reference position.

The first predetermined criteria mentioned in step a) above may stipulate that the connected component has at least a certain size in at least one image dimension. In one embodiment, the first predetermined criteria therefore include that the connected component has a height of at least 2/3 of the nominal height of symbols in the symbol templates.

The second predetermined criteria mentioned in step c2) above may stipulate that the best correlation between the symbol image data and any template in the first set of templates exceeds a first threshold, such as 75%.

Additionally, the second predetermined criteria may also stipulate that the difference between aforesaid best correlation and the second best correlation between the symbol image data and any template in the first set of templates exceeds a second threshold, such as a

difference in 10%.

In one embodiment, when said symbol image data is found to represent an incomplete symbol which is

truncated at its top or at its bottom compared to its expected appearance on the counter mechanism, the method further involves performing one matching for said symbol image data starting from the top of a given symbol template and another matching for said symbol image data starting from the bottom of said given symbol template.

The method is advantageously performed to obtain a meter reading for a counter mechanism having a series of symbol positions, and the same reference image data is advantageously used for at least some of the symbol positions. The counter mechanism may be a cyclometer register having a series of rotary reels, a peripheral edge of each reel being provided with a plurality of markings representing said plurality of symbols for a respective symbol position.

The counter mechanism may belong to a consumption meter such an electricity meter, gas meter, fuel meter or water meter.

A second aspect of the invention is a meter reading device adapted for use with a counter mechanism according to the above. The meter reading device has an image sensor, a memory and a controller coupled to the image sensor and the memory. The controller is adapted to perform a method as described above for the first aspect.

A third aspect of the invention, similar to the first aspect, is a method of obtaining a meter reading from a captured image of a counter mechanism, the counter mechanism having at least one symbol position, each symbol position being capable of showing a plurality of symbols in sequential order so as to indicate respective values of said symbol position, the method involving:

deriving symbol image data for the symbol position from the image;

generating an output signal representing said meter reading from the current value determined ^' for the or each symbol position,

wherein the reference image data is provided as a first set of symbol templates and a second set of symbol templates, each template in said first set unambiguously representing a respective symbol and each template in said second set representing two adjacent symbols in said sequential order.

The processing of the symbol image data may involve a preprocessing stage and a matching stage, wherein the method of the third aspect may further involve selecting, depending on an outcome of the preprocessing stage, at least .one of the first and second set of symbol templates for use during the matching stage.

A fourth aspect of the invention, similar to the second aspect, is a meter reading device where the controller is adapted to perform a method as described above for the third aspect.

Embodiments of the invention provide optical meter reading with improved accuracy and credibility, since the aforementioned practical problems of flash reflections, symbol transitions, cut-off symbols, and symbol misalignments are handled in a controlled and intelligent manner. Still, the number of symbol templates is kept low, which has benefits both in that little storage space is required and in that the number of required matchings with the captured symbol image data is limited. In turn, this allows a miniaturized implementation of the meter reading device, with a low-power processor and a small memory. Such a miniaturized meter reading device is beneficial in that it allows a slim design and non-obscuring mounting.

Other objectives, features and advantages of the present invention, and embodiments thereof, will appear from the following detailed disclosure, from the attached dependent claims as well as from the drawings.

Brief Description of the Drawings

Embodiments of the present invention will now be described in more detail, reference being made to the enclosed drawings.

Fig 1 is a schematic front view of a typical

electricity meter having a counter mechanism with a plurality of visually readable symbol positions

indicating its current meter reading.

Fig 2 illustrates the counter mechanism in a

situation where some of its symbol positions are

transitional between two adjacent symbols.

Fig 3 is a schematic block diagram of a meter reading device according to one embodiment.

Fig 4 is a schematic sectional view of the meter reading device shown in Fig 3.

Fig 5 illustrates a typical operating scenario where a plurality of meter reading devices are coupled to a central unit, which in turn communicates with a remote server over a communications network.

Fig 6 is a flow chart that illustrates a normal or operating mode of the meter reading device. Detailed Description of the Invention

Fig 3 is a schematic block diagram of a meter reading device 20 according to one embodiment. As is seen in the schematic sectional view of Fig 4, the meter reading device 20 is adapted to be mounted onto a transparent front surface portion 2 ' of the apparatus housing 2 of the electricity meter 1 shown in Fig 1. It is to be noticed, however, that other embodiments of the meter reading device may be adapted for use with other kinds of consumption meters, such as electricity meters, gas meters, fuel meters or water meters, or even with other counter mechanisms than those found in consumption meters .

The meter reading device 20 has a housing 21 and a plurality of components contained therein, the most important of which being a controller 22 with associated memory 30, and an image sensor 26. Controlling the image sensor 26, the controller is adapted to cause capture of an image 28 of the counter mechanism 3 of the electricity meter 1. To this end, a plurality of illuminating LEDs 25 are provided for illuminating the counter mechanism 3 when the image is to be captured. Optics 24 are provided for guiding light reflected from the front edge surface 12a of the reels 12 of the counter mechanism 3 onto a light sensitive area of the image sensor 26.

As seen in Fig 4, the optics include first and second mirrors 24b, 24d and a lens arrangement 24c. In the disclosed embodiment, the first mirror 24b is partly transparent (dichroic) and is therefore advantageously transmissive in a first light spectrum within the visible wavelength range but reflective in a second light

spectrum within the infrared wavelength range. The illuminating LEDs 25 and the image sensor 26 will then operate in the second light spectrum, whereas a human user can inspect the counter mechanism 3 through an inspection window 42 in the housing 21 aligned with the masking plate 11 of the electricity meter 1, so as to obtain a manual meter reading by ocular inspection.

Thus, when mounted, the meter reading device 20 is aligned with the masking plate 11 of the counter

mechanism 3, such that the image sensor 26 of the meter reading device 20 is positioned in an optical path 24a through the optics 24 from the counter mechanism 3. To this end, a rear housing portion 21' of the meter reading device 20 is transparent.

In the disclosed embodiment, the image sensor 26 is a CMOS sensor capable of capturing an 8-bit, 640x480 pixel grayscale image 28. However, various other types of image sensors, including CCD sensors, having different resolutions and/or color/grayscale depths, are equally possible for implementing the image sensor 26. The captured image 28 is provided to the controller 22, which stores the image 28 temporarily in a work memory (RAM) 32 which is part of the controller's associated memory 30. The controller 22 may be implemented by any commercially available CPU, DSP or any other electronic programmable logic device such as FPGA, or as an application-specific integrated circuit (ASIC), or as discrete components, or as any combination of the above.

The meter reading device 20 also has a communication interface 23, through which a resulting output signal 39 can be transmitted as a representation of a current meter reading as ultimately determined by the controller 22. Even if not shown in the drawings, the meter reading device 20 of the disclosed embodiment is coupled to receive electric power for its various components from mains power lines connected to or in the electricity meter. Alternatively, the meter reading device 20 may be provided with its own power supply means, for instance in the form of a long-life battery.

In the embodiment shown in Fig 4, the components 22, 23, 26 and 30 referred to above are mounted on one surface side of a printed circuit board (PCB) 40, whereas the illuminating LEDs 25 are mounted on the opposite surface side of PCB 40.

In addition to the work memory 32, the controller's associated memory 30 also includes a non-volatile memory, such as a flash memory or EEPROM, for storing reference image data 34. Thus, the associated memory 30 is made of two physically different memory types in the disclosed embodiment. As will be explained in more detail in later sections of this document, the reference image data 34 will be used when interpreting the value of each symbol position in the captured image 28 of the counter mechanism 3. To this end, the controller 22 will derive symbol image data 29 by extracting a sub area of the captured image 28 in which the symbol position in question is located. Thus, in the example of Fig 3, the symbol position to be determined is symbol position IO₃, it shows a symbol "3", and the symbol image data 29 derived will thus be based on the sub area of the captured image 28 that contains the image of this symbol.

As has been explained above, sometimes a symbol position is transitional between two adjacent symbols when the image 28 is captured. For instance, the symbol position IO₂ in Fig 3 is transitional between the symbol "4" and the symbol "5". Consequently, for this symbol, the symbol image data 29 derived from the captured image 28 will contain image components which originate both from (the lower part of) the symbol "4" and from (the upper part of) the symbol "5" on the reel 12 for that symbol position.

To provide for efficient and reliable interpretation both of a whole symbol image (the symbol image data 29 of which contains only one image component originating from one reel marking or symbol, therefore not creating any ambiguity as regards its symbol value) and of a

transitional symbol image, the reference image data 34 is provided as a first set of symbol templates 36 (referred to as "whole symbol templates" in the following) and a second set of symbol templates 38 (referred to as

"transitional symbol templates" in the following) .

In the present embodiment, there are 10 whole symbol templates 36 and 10 transitional symbol templates 38.

Each whole symbol templates 36 contains a reference image of a respective symbol "0", "1", "2", "3", "4", "5", "6", "7", "8", and "9" that represents the corresponding reel marking when being located fully visible and centered in the opening in the masking plate 11.

In contrast, each transitional symbol templates 38 contains a reference image of a respective transitional state between two adjacent symbols, i.e. when two

adjacent markings or symbols on a reel 12 are both partially visible in the opening in the masking plate 11. Thus, the 10 transitional symbol templates 38 represent the following pair of adjacent symbols: "0/1", "1/2", "2/3", "3/4", "4/5", "5/6", "6/7", "7/8", "8/9", and "9/0".

As will be described in more detail in the follow- ing, depending on an outcome of a preprocessing of the symbol image data 29, it will be matched against the reference image data 34 according to one of these

selections: 1) only against the whole symbol templates 36, 2) only against the transitional symbol templates 38, or 3) against first the whole symbol templates 36 and then, if necessary, against the transitional symbol templates 38. In other embodiments, other principles may apply as regards the selection of which template set(s) to use for the matching, and/or in what order to use it/them.

Generally, the meter reading device 20 of the disclosed embodiment has three main modes:

1. A template creation mode in which the first and second template sets 36 and 38 (whole symbol templates and transitional symbol templates) are generated together with certain related reference data 35 pertaining to a certain type or model of the electricity meter with which the meter reading device is to be used.

2. A calibration mode in which the related reference data is tuned to account for individual variations between different instances of the electricity meter.

3. A normal (or operational) mode during which the individual meter reading device 20 is operative and provides meter readings upon request or with a certain periodicity.

Before giving a detailed description of the normal mode, the other two modes will be briefly described.

The template creation mode typically takes place at a central location, such as at a manufacturer or distributor of electricity meters, at a manufacturer or

distributor of meter reading devices, or at an electricity provider. Here, an operator connects a meter reading device (or another suitable image capturing equipment having similar optical performance as the type of meter reading device for which symbol templates are to be created) to the face plate of an electricity meter of the relevant type or model. Moreover, a computer such as a personal computer or workstation is connected to receive images of the electricity meter's counter mechanism during forced operation. This means that a large current is looped through the electricity meter, forcing its counter mechanism to rotate much faster than during typical ordinary consumption conditions.

In an image of the counter mechanism, the cent marks 10_c (Fig 1) are identified, and their frequency (or mutual distance between adjacent cent marks) is determined through appropriate image processing, involving for instance a Fourier transform. In an alternative embodiment, the operator may assist in identifying the sub area of the captured image in which the cent marks are located by appropriate user interaction, for instance by marking the relevant area in a magnified reproduction of the captured image on the computer display by means of a computer mouse or joystick.

Then, the first symbol position (1Oi in Fig 1) is identified in a similar way (the present embodiment does not involve interpretation of the decimal position 1Od)• Increasing the forced current fed to the electricity meter as explained above, it is noticed when the first reel 12 starts to rotate and its currently visible symbol 10 starts to move vertically within its opening in the masking plate 11. Counting the cent marks 10c as they pass in the opening in the masking plate 11, it is possible to determine accurately when the first symbol position changes alternately between subsequent whole symbols "1", "2", ..., and their interleaved transitions "0/1", "1/2", etc. Images are captured sequentially as the symbols appear, and the thus captured 10 images of whole symbols and 10 images of transitional symbols are stored in temporary memory.

Each of these 20 captured images are processed by thresholding and binarization, and then symbol components are located in each image. In more detail, a symbol component is a connected component in said symbol image data, identified as a contiguous pixel segment where at least a predetermined number of image pixels of the same binary value are positioned adjacent to each other.

In the disclosed embodiment, 8 or more image pixels positioned adjacent to each other in a contiguous pixel segment are regarded to qualify as a connected component. Other embodiments may use other criteria for identifying connected components.

If the image in question represents a whole symbol, only the largest connected component is kept in the image. If the image represents transitional symbols

(containing image components originating from two

adjacent symbols or markings on the reel in question), also the second largest connected component is kept. After this processing, the respective image is stored as one of the 10 whole symbol templates or 10 transitional symbol templates, as the case may be.

Thus, each symbol will be represented in one of the 10 whole symbol templates (the whole symbol) , and in two of the 10 transitional symbol templates (the upper half of the symbol in one, and the lower half in the other) .

Furthermore, related reference data is obtained by further image capture and processing, in the form of: • The number of symbol positions IO₁-IO₅ of the

counter mechanism

• The nominal width (in pixels) of a symbol in a

captured image

• The nominal height (in pixels) of a symbol in a captured image

• The distance between adjacent symbols (originating from adjacent markings on one and the same reel) in a captured image of transitional symbols

• The relative position or distance between symbol positions in a captured image

• Particulars of the decimal position (optional).

After completed template creation, the generated symbol templates and related reference data are stored permanently. If the meter reading device 20 is available already at the time of template creation, the generated data may be stored directly as reference image data 34 including the first and second template sets 36, 38 and the related reference data 35 in its memory 30, or otherwise the generated data will be stored in a databank from which it may be downloaded into the meter reading device 20 at a later occasion.

The calibration mode occurs during installation of the meter reading device 20 by a service person, etc, on the front face of the individual electricity meter 1 for which it is to be used. The purpose of the calibration is to determine precisely the image positions of the

respective symbol positions as they will appear in the captured images when the device 20 is used in normal mode. However, as an initial step in the disclosed embodiment, a skew detection step is performed. The skew detection serves to detect any angular deviation in the mounting of the meter reading device 20 with respect to the counter mechanism 3 (or more particularly the

openings in the masking plate 11), i.e. an unintentional misorientation between the coordinate plane of the captured images and the coordinate plane of the physical appearance of the symbols 10 on the counter mechanism 3, due to a skewed mounting of the meter reading device 20. To this end, the skew detection will apply edge detection followed by Hough transforms of edge positions to detect naturally appearing edges in the captured image,

originating from for instance the rectangular perimeter of the masking plate 11, and then investigate whether such edges appear at a skewed angle. If so, appropriate skew compensation transformation data will be calculated and stored in the related reference data 35 for use during the rest of the calibration mode, and also for later use in the initial processing of all captured images in normal mode, to compensate for such detected skew .

Then, the calibration mode locates the cent marks in a captured image, similar to what is done in the template creation mode. By means of this and the related reference data 35 obtained in the template creation mode, the approximate image positions of the respective symbol positions in captured images are given. Then, for each symbol position, the symbol contents are derived from a captured image in a neighborhood of the approximate position and tested against the image reference data 34 so as to determine the exact image position where the symbol contents best matches the image reference data 34. The testing is performed by correlation matching

essentially in accordance with how it is performed when matching symbol image data 29 with reference image data 34 in the normal mode, as will be described later.

By connecting a computer to the communication interface 23, a human operator performing the calibration can read a resulting meter reading as provided in the output signal 39 and verify its accuracy by comparing with an ocular inspection of the current meter reading through the inspection window 42. After completed calibration, the controller 22 will store the calibrated image

positions of the respective symbol positions, together with the optional skew compensation transformation data, among the related reference data 35 in memory 30.

The normal mode of the meter reading device 20 will now be described. As seen in the flow chart of Fig 6, the operation of the meter reading device 20 can be divided into four main stages: an image capture stage 100, a symbol image data derivation stage 110, a matching stage 120, and a post-processing stage 130.

During the image capture stage 100, the controller 22 controls the illuminating LEDs 25 and the image sensor 26 in a step 102 to produce two (or more) images of the counter mechanism 3 directly after each other (i.e, within a time frame which is short enough so that no parts of the counter mechanism (particularly the cent marks 10_c) will move in position between the two images) . Each image is taken at a different ^'lighting condition, and the reason for this approach is to reduce the afore- described problem of disturbances or defects appearing in the captured images due to shadows and reflections.

In the disclosed embodiment, as viewed from the left in Fig 3, the first and third LEDs are used when capturing the first image, and the second and fourth LEDs are correspondingly used to capture the second image. As is realized by a person skilled in the art of optical meter reading and image recognition in general, the capturing of these two images involves various aspects in terms of illumination times for the LEDs 25, integration times for the image sensor 26, etc. The particulars thereof are left out herein in order not to obscure the invention in unnecessary detail.

Then, the operation proceeds into the symbol image data derivation stage 110. Here, using the related reference data 35 as generated, calibrated and stored in memory 30 in accordance with the description above, a step 112 will extract a symbol image as a sub area of each of the two captured images at a position therein which can be determined from the related reference data

35. A histogram of the intensity of each extracted symbol image is calculated and analyzed, and based on this the best symbol image is selected in step 114 in terms of the least presence of reflections. Exceptionally, it is difficult to determine the best symbol image in step 114, and in such a case the operation may commence with both symbol images in the following steps and leave the ultimate decision to a later stage 128 that involves confidence value determination.

Then, in step 116, the symbol image selected in step 114 is binarized using suitable image processing techniques well known per se, typically including actions like background extraction, threshold matrix determination and binarization . After binarization, the symbol image is a binary image matrix, where pixels having a first binary value (e.g. 1) represent foreground image contents and pixels having the other binary value (e.g. 0) represents absence of foreground image contents.

Then, in a following step 118, symbol components are identified in the resulting symbol image. In more detail, this involves locating all connected components in the symbol image, as has been described above for the

template creation mode.

When all connected components have been identified in the symbol image, the following operations are

performed. First, if there is only one connected component in the image, then the symbol image is regarded as

containing a whole symbol. Therefore, nothing else has to be made to the symbol image in this step 118.

Otherwise, in case there are more than one connected component in the symbol image, the two connected components are determined which have the largest pixel masses. If those two components are true symbol representations (i.e. represent first and second partial symbols in transition) , then they should not overlap each other vertically. Therefore, a test to this effect is made, and if it turns out successful, all other connected

components (if any) are eliminated from the symbol image. If on the other hand the test fails, then the one of the two components is eliminated that has the largest offset in horizontal pixel direction from the expected position as judged from the related reference data 35, and the procedure above is repeated. (As an exception, when such a horizontally offset component is at least four times larger in pixel mass than the other component, then the other component is instead eliminated before the

procedure is repeated) .

The resulting symbol image at completion of step 118 constitutes the symbol image data 29 which will be used in the following matching stage 130.

The matching stage 130 performs correlation matching of the symbol image data 29, as derived in the preceding stages, with the reference image data 34. In more detail, in an initial step 122 a decision is made as to which of the set(s) of whole symbol templates 36 and transitional symbol templates 38 to perform the matching upon.

Here, benefit is drawn from the outcome of the prior step 118 of extracting symbol components, so that the decision is made based upon the number of connected components in the symbol image data 29. Thus, if there is just one connected component, the correlation matching in the remaining steps of the matching stage 120 is performed only upon the whole symbol templates 36.

On the other hand, when there are two connected components, a test is first made as to whether any of the connected components has a height which is at least 2/3 of the nominal symbol height, as obtained from the related reference data 35. If so, the correlation matching is first performed based on the whole symbol templates 36. If necessary, the correlation matching is then also per- formed based on the transitional symbol templates 38. The criteria that determines whether or not to involve also the transitional symbol templates 38 will be explained in more detail later.

If none of the connected components has a height which is at least 2/3 of the nominal symbol height, it is likely that the symbol image data 29 contains components, originating from two adjacent reel markings, of roughly the same size, and the best match will probably be found with any of the transitional symbol templates 38. There- fore, in this case, the correlation matching in the remaining steps of the matching stage 120 is performed only upon the transitional symbol templates 36.

The correlation matching in step 124 is performed against the selected set of templates, as determined above. Each template in the selected set is correlated against the one or two connected components in the symbol image data 29, resulting in a correlation percentage from the matching.

More specifically, the correlation matching in the disclosed embodiment operates on aforesaid connected components, rather than the entire symbol image data 29 which includes also background pixels, in the following manner. When the symbol image data 29 is a whole symbol image, the top of the single connected component (i.e. the first (uppermost) pixel where the single connected component starts) is chosen as reference position for the matching in the symbol image data 29. Correspondingly, the top of the whole symbol in the whole symbol template is located, and the matching between them occurs by performing a bit-wise logical AND operation between pixels in the connected component and pixels of the symbol in the whole symbol template. The result of the logical AND operation will be the number of matching pixel positions between connected component and symbol template. The correlation percentage is determined by dividing this number with the mean value of the number of pixels of the connected component plus the number of pixels of the symbol in the symbol template

When the symbol image data 29 instead is a transitional symbol image having an upper connected component (e.g. the lower part of a symbol "4") and a lower con- nected component (e.g. the upper part of a following symbol "5"), the reference position for the matching will be the top of the aforesaid lower connected component in the transitional symbol image (e.g. the top of the symbol "5" )

Other embodiments may use other suitable reference positions, such as the top of a single connected component, or the bottom of an upper connected component in a transitional symbol image.

The correlation matching in the disclosed embodiment has a sub step 125a which handles the afore-described problem of truncated or partially non-readable symbols in the captured image because of disturbances or defects caused by shadows or reflections. When this happens, the connected component (s) in the symbol image data 29 will be shorter than the nominal symbol height, or differently put, the relevant area of the symbol image data 29 to be used for correlation is shorter in height than the template. Therefore the sub step 125a involves first matching the connected component (s) in the symbol image data 29 with the template starting from the top of the latter. If the resulting correlation percentage is not sufficiently good (i.e. does to reach a percentage threshold), then a second matching is performed, this time starting from the bottom of the template.

The correlation matching in the disclosed embodiment also has a sub step 125b which is performed if the matching of a particular symbol fails to result in a match that meets a predetermined correlation limit. In such a case, the correlation matching is performed at varying offset positions in vertical and horizontal pixel directions in the symbol image data 29.

In a following step 126, the results from the various correlation matchings performed in the preceding step are compiled, so that their respective correlation percentages can be compared. To this end, for the situation referred to above, where the correlation matching has first been performed in step 124 based on the whole symbol templates 36 when there are two connected components, a test is made to see whether the best match exceeds a threshold, such as correlation percentage >= 75 %. If so, another test is made to see whether there is at least a predetermined difference in the correlation percentage of the best match and the correlation percentage of the second best match, such as 10 % difference in correlation percentage (e.g. 85 % vs 75 %) . If both tests are successful, the outcome of the correlation matching already performed on the whole symbol templates is deemed sufficient, and no further correlation matching is required for the current symbol. On the other hand, if either of the tests fails, the correlation matching step 124 is repeated, this time being performed upon the transitional symbol templates 38, and eventually the results thereof are included in the results compilation in step 126.

In step 127, the best match, i.e. the particular template among the whole symbol templates 36 and/or transitional symbol templates 38 that yields the best correlation percentage with the symbol image data 29, is selected, and the symbol represented by that template will be chosen as the interpreted symbol at the current symbol position IO₁-IO₅ of the counter mechanism 3.

Moreover, based upon the reference position for matching as determined in step 124, step 127 also records a decimal value for the interpreted symbol .

In a step 128, a confidence value is calculated, the purpose of which is to indicate the relative accuracy of the best match. In the disclosed embodiment, the confidence value is determined as a function of the cor- relation percentages of the best and second best matches.

When the stages 110 and 120 have been performed for all symbol positions on the counter mechanism 3, the post processing stage 130 commences. Here, a round logic step 132 will compile the resulting interpreted values for the respective symbol positions into a joint meter reading estimation consisting of an integer value having, of course, the same number of symbol positions IO₅-IO₁ as the counter mechanism 3. If an interpreted symbol at a symbol position 1O_n has a decimal value O 0, as determined above, this will be considered when determining an integer value not only for this symbol position but also for at least the following symbol position 10_n+i. The round logic step 132 takes into account that due to the mechanical nature of the counter mechanism, a certain symbol position may be offset by as much as a couple of decimal units compared to other symbol positions, because the corresponding reel either rotates ahead or lags behind other reels. The round logic step 132 also takes into account that a symbol position which has not yet switched completely from one integer value to the next must not be rounded to that next integer value, since this could result in excessive billing (if the unit of the electricity meter reading is kWh, an incorrectly rounded symbol at symbol position IO₄ would mean a difference in 1000 kWh) .

When a meter reading estimation has been determined in step 132, the result is evaluated in a step 134. Here, use is made of the confidence values determined in step 128. If the confidence value is low for any of the determined symbols, this may indicate an ambiguity in the correlation matching caused by two different templates yielding similar correlation percentages. Such a

situation may arise either because of disturbances or deviations as mentioned above, or because some symbol transitions are hard to differentiate for counter mechanisms where the reel markings are printed in a certain typeface. One example is the transitions "1/2" and "7/8".

Therefore, if a low confidence value is observed for any symbol position in step 134, the controller 22 may wait a predetermined time period and then repeat all the operations starting with a new image capture stage 100. Alternatively, the controller may report the low confidence value in the output signal 39 to a central unit 50 (described below with reference to Fig 5) and receive a command from the latter to repeat the meter reading. Hopefully, the resulting meter reading estimation after the next iteration of step 132 will be found in step 134 to have a better and acceptable confidence value.

In a final step 136, the resulting meter reading estimation is communicated in the output signal 39 together with related information in the form of the con- fidence value and an identity of the meter reading device 20 or electricity meter 1 (preferably established by the operator during aforesaid calibration) . If the meter reading device 20 has a real-time clock, temporal

information on current date and/or time may also be included in the output signal 39.

As seen in Fig 5, a meter reading device 2Oi at an electricity meter Ii may be connected to the aforementioned central unit 50 together with a number of other meter reading devices 20₂~20s at corresponding electricity meters I₂-I₆. To this end, the communication interface 23 may be compliant with any available local wired communication standard, such as USB, Firewire or RS232, or short-range data communication standard such as Bluetooth or IrDA, or packet-switched data communication such as TCP/IP over Ethernet or GPRS over GSM/3G, or an industry sector-specific communication standard such as M-bus over an electric powerline network, to form a respective communication channel 52i~52₆ to the central unit 50. The communication channels 52i-52₆ may be used to command reading (i.e., initiate performance of one iteration of the normal mode operations of Fig 6) at the respective electricity meters, and, in response to such a command, the resulting meter reading with its related information may be communicated as output signals 39i-39ε to the central unit.

The central unit 50 has a communication channel 54 for connection to a remote server 60 which may be

operated by the electricity provider for billing and/or monitoring purposes. The communication channel 54 may involve a communications network 56 - such as a global area network (e.g. the Internet), a telecommunications network or an electric powerline network - and may be compliant with any appropriate communication standard, such as TCP/IP over Ethernet, GPRS over GSM/3G, or M-bus.

Thus, meter readings as obtained from the meter reading devices 20i-20₆ may be reported to the remote server 60 and stored in an associated database 62, and, conversely, the remote server may issue commands intended for the meter reading devices 20i-20₆ and relayed by the central unit 50.

The invention has mainly been described above with reference to a disclosed embodiment. However, as is readily appreciated by a person skilled in the art, other embodiments than the one disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims .

For instance, in alternative embodiments it is envisaged to provide three or even more sets of symbol templates and select between them in different matching situations. This may be useful for instance for other types of counter mechanism symbols than decimal digits.

Moreover, it is also envisaged that alternative embodiments may use both (or all) template sets constant- Iy for all matchings, i.e. without the aforesaid

selecting step. Such alternative embodiments may still benefit from a low number of template matchings compared to the prior art. To this end, there is no particular restriction as regards how the two (or more) template sets are stored or otherwise represented. All templates may for instance be stored in the same way and together on a physical level, wherein the type of symbol content (e.g. whole symbol template or transitional symbol template) for a particular template will determine whether it belongs to the first template set or the second template set .

Further, meter reading does not have to be initiated by an external unit (e.g. central unit 50) but may alternatively be initiated by the controller 22 itself. For instance, the controller 22 may use a real-time clock and predefined calendar settings to perform meter reading at a desired periodicity, such as hourly, daily, weekly or monthly.

In alternative embodiments the output signal 39 is not an external output signal but is instead stored locally in the meter reading device 20 (e.g. in the memory 30), for later access e.g. by connecting an external unit to the device 20 through the communication interface 23.

Finally, the invention can be used for other

purposes than reporting of consumption data for billing. One example is monitoring of a meter at a fuel tank or dispenser for the purpose of operation surveillance or remote identification of the need for maintenance or fuel refill.

Claims

1. A method of obtaining a meter reading from a captured image (28) of a counter mechanism (3), the counter mechanism having at least one symbol position (1O_x), each symbol position being capable of showing a plurality of symbols (10) in sequential order so as to indicate respective values of said symbol position, the method involving:

deriving symbol image data (29) for the symbol position from the image;

processing the symbol image data by matching with reference image data (34) representing said plurality of symbols to determine a current value of said symbol position; and

generating an output signal (39) representing said meter reading from the current value determined for the or each symbol position,

wherein the reference image data (34) is provided as a first set of symbol templates (36) and a second set of symbol templates (38),

wherein the processing of the symbol image data (29) involves a preprocessing stage (110) and a matching stage (120), and

wherein the method further involves selecting, based on an outcome of the preprocessing stage, at least one of the first and second sets of symbol templates (36, 38) for use during the matching stage. 2. A method as defined in claim 1, wherein each template in said first set (36) unambiguously represents a respective symbol and each template in said second set

(38) represents two adjacent symbols in said sequential order .

3. A method as defined in claim 1 or 2, wherein the preprocessing stage (110) involves identifying symbol components representing symbols or parts thereof in said symbol image data (29) and wherein said outcome is indicative of a number of such symbol components having been identified.

4. A method as defined in claim 3, wherein, if said outcome of the preprocessing stage (110) indicates that only one symbol component has been identified, only said first set of symbol templates (36) is selected for use during the matching stage (120) .

5. A method as defined in claim 3, wherein, if said outcome of the preprocessing stage (110) indicates that two symbol components have been identified, said second set of symbol templates (38) is selected for use during the matching stage (120) .

6. A method as defined in claim 3, wherein, if said outcome of the preprocessing stage (110) indicates that two symbol components have been identified, the method involves :

b) if the testing in a) fails, selecting only said second set of symbol templates (38) for use during the matching stage (120),

c) if the testing in a) is successful, cl) selecting said first set of symbol templates (36) and performing a first matching using said first set of symbol templates (36), c2) determining whether a result of the first matching fulfills second predetermined criteria, and in case said second predetermined criteria are not

fulfilled, c3) selecting said second set of symbol templates (38) and performing a second matching using said second set of symbol templates (38) .

7. A method as defined in any one of claims 3-6, said symbol image data (29) being binary image data, wherein said symbol components are identified as

contiguous pixel segments in said symbol image data (29) where at least a predetermined number of image pixels of the same binary value are positioned adjacently to each other.

8. A method as defined in claim 6, wherein after said step c3) an additional step c4) is performed which involves choosing, as a result of said matching stage (120), a single best result, in terms of best correlation between symbol image data and template, from said first matching in step cl) and said second matching in step c3) .

9_V A method as defined in any preceding claim, wherein, when said symbol image data (29) is found to represent an incomplete symbol which is truncated at its top or at its bottom compared to its expected appearance on the counter mechanism, the method further involves performing one matching for said symbol image data (29) starting from the top of a given symbol template and another matching for said symbol image data starting from the bottom of said given symbol template.

10. A method as defined in any preceding claim, wherein said counter mechanism (3) has a series of symbol positions (IO₁-IO₅) and wherein the same reference image data (34) is used for at least some of the symbol

positions .

11. A method as defined in any preceding claim, wherein said counter mechanism (3) is a cyclometer register having a series of rotary reels (12), a

peripheral edge of each reel being provided with a plurality of markings (12a) representing said plurality of symbols (10) for a respective symbol position (10i~ 1O₅)•

12. A method as defined in any preceding claim, wherein at least some of said symbols (10) are decimal digits .

13. A method as defined in any preceding claim, performed to obtain a meter reading (39) for a con- sumption meter (1) .

14. A method as defined in claim 1 or 2, wherein the preprocessing stage (110) involves identifying symbol components representing symbols or parts thereof in said symbol image data (29) and wherein the matching stage (120) involves correlation matching of each symbol template in the selected at least one of the first and second sets of symbol templates (36, 38) against

identified symbol components in said symbol image data (29) .

15. A method as defined in claim 14, wherein the correlation matching involves locating a reference position for an identified symbol component in said symbol image data (29), locating a reference position in a given symbol template, and determining a correlation between individual pixels in said identified symbol component and individual pixels at corresponding

positions in said given symbol template, starting from the respective reference positions.

16. A meter reading device (20) adapted for use with a counter mechanism (3) , the counter mechanism having at least one symbol position (1O_x) , each symbol position being capable of showing a plurality of symbols (10) in sequential order so as to indicate respective values of said symbol position, the meter reading device

comprising :

an image sensor (26) capable of capturing an image (28) of the counter mechanism (3);

a memory (30) capable of storing reference image data (34) representing said plurality of symbols; and

a controller (22) coupled to said image sensor and said memory, the controller being adapted to obtain a meter reading of the counter mechanism from the captured image by deriving symbol image data (29) for the symbol position from the image, processing the symbol image data by matching with said reference image data (34) to determine a current value of said symbol position, and generating an output signal (39) representing said meter reading from the current value determined for the or each symbol position,

wherein the controller is adapted to perform the processing of the symbol image data (29) in a preprocessing stage (110) and a matching stage (120), and wherein the controller is further adapted to select, based on an outcome of the preprocessing stage, at least one of the first and second sets of symbol templates (36, 38) for use during the matching stage.

17. A meter reading device (20) according to claim 16, wherein the controller (22) is adapted to perform the method according to any of claims 2-13.

18. A method of obtaining a meter reading from a captured image (28) of a counter mechanism (3), the • counter mechanism having at least one symbol position (1O_x), each symbol position being capable of showing a plurality of symbols (10) in sequential order so as to indicate respective values of said symbol position, the method involving:

deriving symbol image data (29) for the symbol position from the image;

wherein the reference image data (34) is provided as a first set of symbol templates (36) and a second set of symbol templates (38) , each template in said first set unambiguously representing a respective symbol and each template in said second set representing two adjacent symbols in said sequential order.

19. A method as defined in claim 18, the processing of the symbol image data (29) involving a preprocessing stage (110) and a matching stage (120) , wherein the method further involves selecting, depending on an outcome of the preprocessing stage, at least one of the first and second set of symbol templates (36, 38) for use during the matching stage.

20. A meter reading device (20) adapted for use with a counter mechanism (3) , the counter mechanism having at least one symbol position (1O_x) , each symbol position being capable of showing a plurality of symbols (10) in sequential order so as to indicate respective values of said symbol position, the meter reading device

comprising:

a memory (30) capable of storing reference image data (34) representing said plurality of symbols; and a controller (22) coupled to said image sensor and said memory, the controller being adapted to obtain a meter reading of the counter mechanism from the captured image by deriving symbol image data (29) for the symbol position from the image, processing the symbol image data by matching with said reference image data (34) to determine a current value of said symbol position, and generating an output signal (39) representing said meter reading from the current value determined for the or each symbol position,

wherein the reference image data (34) is provided as a first set of symbol templates (36) and a second set of symbol templates (38), each template in said first set unambiguously representing a respective symbol and each template in said second set representing two adjacent symbols in said sequential order.