JP2019145182A - Reading system, reading method, program, and storage medium - Google Patents

Abstract

To provide a reading system, a reading method, a program and a storage medium that are capable of improving the precision in reading a numeric value on a segment display.SOLUTION: A reading system according to an embodiment of the present invention is provided with an extraction unit, a determination unit and a read unit. The extraction unit extracts, from an input image, a candidate image that is a candidate for a portion where a segment display is imaged. The determination unit calculates, using the candidate image and a preset mask, a relevance ratio that indicates the probability of the segment display being included in the candidate image, and determines, when the relevance ratio is greater than or equal to a preset threshold, that the candidate image is an image of the segment display. The read unit reads a numeric value displayed on the segment display from the candidate image determined to be the image of the segment display.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to a reading system, a reading method, a program, and a storage medium.

  There is a system for reading a numerical value displayed on a segment display. In this system, it is desirable that the accuracy of reading numerical values is high.

JP 2008-243103 A

  The problem to be solved by the present invention is to provide a reading system, a reading method, a program, and a storage medium that can improve the reading accuracy of a numerical value of a segment display.

  The reading system according to the embodiment includes an extraction unit, a determination unit, and a reading unit. The extraction unit extracts a candidate image that is a candidate for a portion where a segment display is taken from an input image. The determination unit uses the candidate image and a preset mask to calculate a matching rate indicating a probability that a segment display is included in the candidate image, and the matching rate is equal to or higher than a preset threshold value. Then, the candidate image is determined to be a segment display image. The reading unit reads a numerical value displayed on the segment display from the candidate image determined to be an image on the segment display.

It is a block diagram showing the structure of the reading system which concerns on embodiment. It is a flowchart showing operation | movement of the reading system which concerns on embodiment. It is a figure which illustrates the process in the reading system which concerns on embodiment. It is a block diagram showing the structure of the reading system which concerns on the 1st modification of embodiment. It is a block diagram showing the structure of the reading system which concerns on the 2nd modification of embodiment. It is a block diagram showing the structure of the reading system which concerns on the 3rd modification of embodiment. It is a block diagram showing the hardware constitutions for realizing the reading system concerning an embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
In the present specification and each drawing, the same elements as those already described are denoted by the same reference numerals, and detailed description thereof is omitted as appropriate.

FIG. 1 is a block diagram illustrating a configuration of a reading system according to the embodiment.
The reading system which concerns on embodiment is used in order to read the numerical value displayed on the segment display from the image containing a segment display. In this embodiment, a segment display represents what contains some display information, such as a number and a character comprised by the display of a some segment.

  As illustrated in FIG. 1, the reading system 1 according to the embodiment includes an imaging unit 11, an extraction unit 12, a conversion unit 13, a cutting unit 14, a determination unit 15, a reading unit 16, and a storage unit 18.

  The imaging unit 11 captures a segment display and acquires a still image. The imaging unit 11 outputs the acquired image to the extraction unit 12. When a moving image is shot by the imaging unit 11, a still image is cut out from the moving image and output to the extraction unit 12. The captured image may include something other than the segment display.

  The extraction unit 12 extracts segment display image candidates from the input image. Here, an image shot by the imaging unit 11 and input to the extraction unit 12 is referred to as an input image. An image that is a part of the input image and is a candidate for the segment display is called a candidate image. The candidate image is a part of the input image determined by the extraction unit 12 that the segment display is captured. There may be a plurality of candidate images output from the extraction unit 12.

  As a specific example, the extraction unit 12 includes a processing unit 12a, a binarization unit 12b, a contour extraction unit 12c, a first selection unit 12d, a quadrature extraction unit 12e, and a second selection unit 12f.

  For example, the processing unit 12a generates a processed image obtained by converting an input image into a gray scale. The processing unit 12a outputs the processed image to the binarization unit 12b. At this time, for example, the processing unit 12a may output a processed image obtained by reducing the input image to a preset size to the binarization unit 12b. At that time, the processing unit 12 a stores the original image that has not been reduced in the storage unit 18.

  The binarization unit 12b binarizes the input processed image and generates a binary image expressed in white and black. The contour extraction unit 12c extracts the contour of a region represented in white, for example, in the binary image.

  The first selection unit 12d calculates the area of the region surrounded by the outline. When a plurality of contours are extracted, the area of each region is calculated. The first selection unit 12d compares each calculated area with a predetermined threshold, and selects only a region whose area is equal to or greater than the threshold. As a result, a region whose area is too small is excluded from the candidates. The quadrangle extraction unit 12e approximates the selected area with a straight line and extracts a quadrangle. For example, the quadrilateral extraction unit 12e acquires the coordinates of the extracted quadrangular vertices.

  The second selection unit 12f compares the extracted rectangular area with a preset condition, and selects an area that meets the condition. Conditions include, for example, that the ratio of the vertical and horizontal dimensions of the rectangle is within a predetermined range, the vertex angle is within a predetermined range, and any vertex of the rectangle is near the outer frame of the image (for example, outside For example, it is not within 3 pixels from the frame. The second selection unit 12f outputs the selected quadrangle to the conversion unit 13 as a final candidate image.

The conversion unit 13 performs the following first process and second process.
In the first process, the conversion unit 13 determines whether the candidate image input from the extraction unit 12 is distorted. When it is determined that the candidate image is distorted, the conversion unit 13 corrects the distortion so that the candidate image approaches the image captured from the front.
In the second process, the conversion unit 13 determines whether the size of the candidate image matches a predetermined size set in advance. When the size of the candidate image does not match the specified size, the conversion unit 13 corrects the candidate image so that the size of the candidate image approaches the specified size.
The order in which the first process and the second process are performed is arbitrary. Alternatively, the first process and the second process may be performed simultaneously. For example, the conversion unit 13 performs projective conversion on the candidate image, thereby performing the first process and the second process at the same time, and correcting the distortion and the size. The conversion unit 13 outputs the corrected candidate image to the cutout unit 14. If it is determined that the candidate image is not distorted and the size of the candidate image matches the specified size, the conversion unit 13 outputs the input candidate image to the cutout unit 14 as it is.

  The cutting unit 14 refers to the storage unit 18. The storage unit 18 stores a preset mask and information thereof. In the present embodiment, the mask is an image corresponding to the segment display to be read. The mask serves as reference information for reading the target segment display. For example, a binary image of a segment display in a state where all the segments are lit is used as a mask. The cutout unit 14 acquires the vertical and horizontal dimensions of the mask and the corresponding position of the mask in the actual segment display from the storage unit 18. Based on these pieces of information, the cutout unit 14 cuts out a part of the input candidate image. The cutout unit 14 outputs the cutout candidate image to the determination unit 15.

  The determination unit 15 refers to the storage unit 18 and calculates the relevance ratio using the input candidate image and the mask. The relevance rate is a value indicating the probability that the candidate image includes the segment display. For example, the higher the matching rate, the higher the possibility that a segment display is included in the candidate image.

  The determination unit 15 compares the matching rate with a preset threshold value. When the relevance ratio is equal to or higher than the threshold, the determination unit 15 determines that the candidate image is a segment display image and outputs the image to the reading unit 16. When there are a plurality of candidate images, the determination unit 15 calculates and determines the relevance ratio for each candidate image.

Specifically, the determination unit 15 includes a binarization unit 15a, a logical operation unit 15b, a matching rate calculation unit 15c, and a comparison unit 15d.
The binarization unit 15a binarizes the candidate image. The logical operation unit 15b calculates a logical product of the binary image and the mask. In the processed image obtained from the logical product, only the region represented in white in the mask and in white in the binary image is represented in white.

  The matching rate calculation unit 15c calculates the matching rate of the binary image with respect to the processed image as the matching rate. That is, the ratio (A2 / A1) of the area A2 of the region expressed in white in the binary image to the area A1 (the number of pixels expressed in white) of the region expressed in white in the processed image is the precision. Calculated.

  The comparison unit 15d compares the matching rate with a threshold value. If the relevance ratio is equal to or higher than the threshold value, the candidate image is output to the reading unit 16. When the relevance ratio is less than the threshold, the comparison unit 15d determines that the candidate image is not a segment display image, and ends the process, for example.

  The reading unit 16 reads a numerical value displayed on the segment display from the input candidate images. For example, the reading unit 16 reads out a numerical value by cutting out a number from the input candidate image and detecting a lit segment. For example, the reading unit 16 displays the read numerical value on a monitor or outputs it to a database.

  The storage unit 18 stores information necessary for processing of the reading system 1 and data generated in the course of processing. For example, in addition to the above-described mask, the storage unit 18 stores a threshold value to be compared when the determination is performed.

With reference to FIG.2 and FIG.3, operation | movement of the reading system 1 which concerns on embodiment is demonstrated.
FIG. 2 is a flowchart illustrating the operation of the reading system according to the embodiment.
FIG. 3 is a diagram illustrating processing in the reading system according to the embodiment.

  The imaging unit 11 captures the segment display and acquires an image (step S11 in FIG. 2). The processing unit 12a processes the input image (step S12a). As a result, a processed image obtained by reducing the input image to gray scale and an original image obtained by converting the input image only to gray scale are generated. FIG. 3A is an example of a processed image. The binarization unit 12b binarizes the processed image and generates a binary image as shown in FIG. 3B (step S12b). The contour extracting unit 12c extracts the contour of the binary image (step S12c).

  The first selection unit 12d selects a region surrounded by the outline (step S12d). Thereby, for example, as shown in FIG. 3C, the region surrounded by the frame A is selected. The quadrangle extraction unit 12e extracts a quadrangle from the original image based on the quadrangle extraction result in the selected region (step S12e). The second selection unit 12f selects the extracted quadrangle (step S12f) and outputs it as a candidate image. FIG. 3D shows a quadrangle that is output as a candidate image.

  As illustrated in FIG. 3E, the conversion unit 13 converts the candidate image so as to correct the distortion and the size. The conversion unit 13 corrects distortion and size, for example, by projective conversion of the candidate image (step S13). FIG. 3H shows an example of a mask. The cutout unit 14 cuts out a part of the candidate image in accordance with the size of the mask (step S14). Thereby, the image shown in FIG. 3F is obtained.

  As shown in FIG. 3G, the binarization unit 15a binarizes the clipped candidate image (step S15a). The logical operation unit 15b calculates the logical product of the binary image and the mask (step S15b). As a result, the image shown in FIG. 3I is obtained. The relevance ratio calculation unit 15c calculates the relevance ratio using the images in FIG. 3G and FIG. 3I (step S15c). The comparison unit 15d compares the matching rate with a preset threshold value (step S15d). When the relevance ratio is equal to or higher than the threshold, the reading unit 16 reads the numerical value of the segment display from the candidate image (step S16).

  The determination unit 15 determines whether there are other candidate images for which steps S15a to S15d have not yet been performed (step S17). If there is another candidate image, step S15a is performed for that candidate image. If there are no other candidate images, the process ends.

The effect of the embodiment will be described.
When reading the segment display, as described above, candidate images that are candidates for the portion where the segment display is photographed are extracted from the input image. Conventionally, for example, distortion or the like is appropriately corrected for the extracted candidate image, and the value of the segment display is read from the candidate image.
But for candidate images. For example, a display similar to a segment display may be included. In this case, the numerical value of another display device may be read as the numerical value of the segment display, and erroneous detection may occur.

  The reading system 1 according to the embodiment includes a determination unit 15 in order to suppress such erroneous detection. The determination unit 15 uses the candidate image and a preset mask to calculate a matching rate indicating the probability that the segment display is included in the candidate image. And the determination part 15 determines with a candidate image being an image of a segment display, when this precision is more than the preset threshold value.

  By making such a determination, it is possible to read only candidate images that are more likely to be segment displays. In particular, in the present embodiment, a mask corresponding to the segment display to be read is prepared in advance, and candidate images are narrowed down using this mask. For this reason, even when a display device similar to the segment display or another segment display is included in the input image, only the segment display to be read can be selected with higher accuracy, and the possibility of erroneous detection can be greatly reduced.

  In the example illustrated in FIG. 1, the reading system 1 according to the embodiment includes the imaging unit 11, but the reading system 1 may not include the imaging unit 11. For example, an image captured by another imaging device may be input to the reading system 1 and a numerical value of the segment display may be read by the reading system 1 from the input image.

Further, the reading system 1 may not include the conversion unit 13 and the cutting unit 14, but it is desirable to include these configurations in order to improve the reading accuracy.
For example, the accuracy of the precision can be improved by correcting the distortion of the candidate image by the conversion unit 13. This makes it possible to more accurately select candidate images taken by the segment display.
By cutting out a part of the candidate image using the mask by the cutting unit 14, subjects other than the reading target can be excluded from the candidate image. Thereby, the precision of the precision can be improved.

  In the above description, the conversion unit 13 corrects the distortion of the candidate image when it is determined in the first process that the candidate image is distorted. Instead of this method, the conversion unit 13 may distort the mask according to the distortion of the candidate image when it is determined in the first process that the candidate image is distorted. In this case as well, the accuracy of the precision can be improved in the subsequent processing by the determination unit 15.

  FIG. 3 shows an example in which a numerical value displayed on the 7-segment display is read using a mask corresponding to the 7-segment display. However, the segment display that can be read by the reading system 1 according to the embodiment is not limited to the 7-segment display. The segment display to be read may be a 14-segment display or a 16-segment display.

  As the mask, an image of a segment display in which at least a part of a plurality of segments is lit is used. More preferably, a binary image of a segment display in which all segments are lit is used as a mask. By doing in this way, the candidate image by which the segment display was image | photographed can be selected more accurately irrespective of the display information of the segment display to be read.

  When a binary image in which all segments are lit is used as a mask, the area of the white region in the mask increases as the number of segments increases. For example, if the candidate image includes a lot of noise and there are many white areas due to the noise, the matching rate between the candidate image and the mask may be high even though the candidate image does not include a segment display. For this reason, it is desirable that the threshold value to be compared with the matching rate is adjusted according to the number of segments in the segment display to be read.

For example, when reading a numerical value of a 7-segment display, a candidate image in which the 7-segment display is captured can be selected with higher accuracy by setting the threshold value to 0.6 or more.
Also, if the threshold is high, there is a high possibility that even if a 7-segment display is copied in the candidate image, it is determined that the image is not a 7-segment display. For this reason, the threshold value is desirably 0.8 or less.
That is, when reading a numerical value of a 7-segment display, the reading accuracy of the numerical value can be improved by setting the threshold value to 0.6 or more and 0.8 or less.

  Moreover, the process in the extraction part 12 can be suitably changed if a candidate image can be extracted. For example, when the size of the input image is small or when the input image is binarized in advance, the processing unit 12a or the binarization unit 12b is not necessary. Moreover, the process for extracting a candidate image from an input image can be changed as appropriate. These modifications will be described below.

(First modification)
FIG. 4 is a block diagram illustrating a configuration of a reading system according to a first modification of the embodiment.
In the example of FIGS. 1 to 3, the frame of the display area of the segment display is a rectangle. Therefore, the extraction unit 12 is provided with a quadrilateral extraction unit 12e in accordance with the shape of the display area.

  When the frame of the display area is circular, as in the reading system 2 shown in FIG. 4, the extraction unit 12 is provided with an ellipse extraction unit 12g instead of the square extraction unit 12e. The ellipse extraction unit 12g extracts an ellipse from the region selected by the first selection unit 12d. At this time, the ellipse extraction unit 12g acquires the coordinates of the extracted ellipse. The ellipse extraction unit 12g extracts an ellipse image corresponding to the coordinates from the original image and outputs the extracted image to the second selection unit 12f.

  The second selection unit 12f compares the input ellipse image with preset conditions, and selects an ellipse that meets the conditions. As the conditions, for example, an elliptical flatness ratio, a luminance distribution, and the like are used. The second selection unit 12f outputs the selected ellipse image to the conversion unit 13 as a candidate image. The subsequent processing is the same as the example described with reference to FIGS. That is, the conversion unit 13 corrects the distortion of the candidate image, and the cutout unit 14 cuts out a part of the candidate image. Then, after the determination by the determination unit 15, the reading unit 16 reads the numerical value of the segment display.

  In general, the numerical values of the segment display are arranged side by side in a predetermined direction. Therefore, even when the frame of the display area of the segment display is a circle or an ellipse, the numerical value displayed on the segment display generally falls within a rectangle. That is, even when the frame of the display area is a circle or an ellipse, the outer shape of the mask is set to a rectangle. By making the outer shape of the mask square, it is possible to increase the ratio of the area where numerical values are displayed with respect to the total area of the mask. As a result, the influence of noise or the like in the candidate image on the precision can be reduced, and the precision of the precision can be improved.

(Second modification)
FIG. 5 is a block diagram illustrating a configuration of a reading system according to a second modification of the embodiment.
The reading system 3 illustrated in FIG. 5 includes an extraction unit 21 instead of the extraction unit 12, the conversion unit 13, and the cutout unit 14 in comparison with the reading system 1.

  In the reading system 3, before reading the numerical value of the segment display, the imaging unit 11 captures only the segment display to be read from the front. Here, an image obtained by photographing only the segment display from the front is referred to as a template image.

The extraction unit 21 includes a feature amount extraction unit 21a, a matching unit 21b, a conversion unit 21c, and a search unit 21d.
The feature amount extraction unit 21a extracts the feature amount of the input image. The feature amount extraction unit 21a refers to the template image stored in the storage unit 18 and extracts the feature amount of the template image. Alternatively, the feature amount of the template image may be extracted in advance by the feature amount extraction unit 21 a and stored in the storage unit 18. In this case, the feature amount extraction unit 21 a refers to the feature amount of the template image stored in the storage unit 18. The feature amount extraction unit 21a outputs the feature amount of the input image and the feature amount of the template image to the matching unit 21b.

The matching unit 21b calculates a correction amount of the input image necessary for matching the feature amount of the input image with the feature amount of the template image. For example, the matching unit 21b calculates a correction amount necessary to bring the segment display distorted in the input image closer to the image photographed from the front. The matching unit 21 b outputs the calculated correction amount and input image to the conversion unit 13.
If the feature quantities cannot be matched, there is a high possibility that the segment display is not included in the input image. Accordingly, in this case, for example, the process is terminated without calculating the correction amount.

  The conversion unit 21c converts the input image based on the input correction amount. Thereby, when the input image is distorted with respect to the template image, the distortion is corrected.

The search unit 21 d refers to the template image stored in the storage unit 18. Then, the search unit 21d searches the input image for a portion that matches the template image. The search unit 21d outputs the matched portion to the determination unit 15 as a candidate image.
The subsequent processing in the determination unit 15 and the reading unit 16 is the same as that of the reading system 1 shown in FIG.

  The feature amount extraction by the feature amount extraction unit 21a is performed using, for example, KAZE, AKAZE (Accelerated KAZE), or SIFT (Scale-invariant feature transform). The matching in the matching unit 21b is performed using, for example, KNN (K Nearest Neighbor) or FLANN (Fast Library for Approximate Nearest Neighbors). The search by the search unit 21d is performed using pattern matching based on the similarity between images, for example.

  For example, when a plurality of segment displays are provided on a certain plane and these segment displays are photographed, these segment displays are similarly distorted in the input image. In the reading system 3 according to this modification, conversion of an input image using a feature amount and matching using a template image are performed. That is, the entire area of the input image is converted. Therefore, it is not necessary to correct the distortion for each candidate image, and the processing can be speeded up compared to the reading system 1 when there are a plurality of segment displays on the same plane.

(Third Modification)
FIG. 6 is a block diagram illustrating a configuration of a reading system according to a third modification of the embodiment.
The reading system 4 illustrated in FIG. 6 includes an extraction unit 31 instead of the extraction unit 12, the conversion unit 13, and the cutout unit 14 in comparison with the reading system 1. The extraction unit 31 includes a learning unit 31a, an input unit 31b, a detection unit 31c, and a processing unit 31d.

  In the reading system 4, a candidate image obtained by photographing a segment display is extracted from an input image using a neural network. For example, teacher data is input to the extraction unit 31 in advance, and neural network learning is performed. The teacher data includes, for example, a distorted image including the segment display, data indicating the position of the segment display in the image, and the distortion amount of the image.

  When the image data is input, the learning unit 31a responds (fires) to the data of the portion where the segment display is taken, and the neuron corresponding to the distortion amount of the image responds. Train a neural network. The learning unit 31 a stores the learned neural network in the storage unit 18. Note that the learning by the learning unit 31a may be performed by a processing device in which processing by the determination unit 15 or the reading unit 16 is executed, or may be performed by using a different processing device. For the learning, a processing device having higher calculation performance is preferably used.

  Thereafter, an image is acquired by the imaging unit 11 in order to read the numerical value of the segment display, and the image is input to the extraction unit 31. When the image is input, the input unit 31 b refers to the learned neural network stored in the storage unit 18. The input unit 31b inputs image data to the neural network.

  If there is a neuron reaction while image data is being input to the neural network, the detection unit 31c detects the reaction. Then, the detection unit 31c detects the coordinates and distortion amount of the image to which the neuron has reacted, and outputs the detected coordinates to the processing unit 31d.

The processing unit 31d extracts a candidate image from the input image based on the input coordinates. Further, the processing unit 31d corrects the distortion of the candidate image based on the input distortion amount. The processing unit 31d outputs the corrected candidate image to the determination unit 15.
Note that the order in which candidate image extraction and distortion correction are performed can be changed as appropriate. For example, the candidate image may be extracted after the distortion of the input image is corrected in the processing unit 31d.
The subsequent processing in the determination unit 15 and the reading unit 16 is the same as that of the reading system 1 shown in FIG.

  In the reading system 3 according to this modification, an image is cut out and corrected using a neural network. For this reason, even when the contrast ratio between the display area of the segment display and its outer frame is not large and it is difficult to extract the contour from the binarized image, the candidate image can be extracted with higher accuracy. Therefore, as compared with the reading system 1, it is possible to read the numerical value of the segment display in the input image with higher accuracy.

FIG. 7 is a block diagram illustrating a hardware configuration for realizing the reading system according to the embodiment.
For example, the reading system according to the embodiment includes the reading device 5 and the imaging device 6 illustrated in FIG. The reading device 5 is, for example, a computer, and includes a ROM (Read Only Memory) 51, a RAM (Random Access Memory) 52, a CPU (Central Processing Unit) 53, and an HDD (Hard Disk Drive) 54.

  The ROM 51 stores a program for controlling the operation of the computer. The ROM 51 stores a program necessary for causing the computer to function as the extraction unit, the conversion unit, the extraction unit, the determination unit, the reading unit, the search unit, and the like in the above-described embodiment.

  The RAM 52 functions as a storage area where the program stored in the ROM 51 is expanded. The CPU 53 reads a control program stored in the ROM 51 and controls the operation of the computer according to the control program. The CPU 53 develops various data obtained by the operation of the computer in the RAM 52. The HDD 54 functions as the storage unit 18 in the above-described embodiment, and stores information necessary for reading and information obtained in the reading process.

  When the reader 5 is used to implement the processing system 4 using a neural network, the reader 5 further includes a GPU (Graphics Processing Unit) and a dedicated chip specialized for neural network processing. Also good. Further, the reading device 5 may include an eMMC (embedded Multi Media Card), an SSD (Solid State Drive), an SSHD (Solid State Hybrid Drive), or the like, instead of the HDD 54.

The imaging device 6 captures a subject (segment display) and transmits the acquired image to the reading device 5. The imaging device 6 is a camera, for example.
The output device 7 outputs the data output from the reading device 5 (the numerical value of the read segment display) so that the user can recognize it. The output device 7 is, for example, a monitor, a printer, or a speaker.

  The reading device 5, the imaging device 6, and the output device 7 are connected to each other by, for example, wired or wireless. Alternatively, they may be connected to each other via a network. Alternatively, at least two of the reading device 5, the imaging device 6, and the output device 7 may be incorporated into one device. For example, the reading device 5 may be integrated with the image processing unit of the imaging device 6 or the like.

  By using the reading system and the reading method according to the embodiment described above, the numerical value displayed on the segment display can be read with higher accuracy. Similarly, by using a program for causing a computer to operate as a reading system, it is possible to cause the computer to read numerical values displayed on the segment display with higher accuracy.

  As mentioned above, although several embodiment of this invention was illustrated, these embodiment is shown as an example and is not intending limiting the range of invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, changes, and the like can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and equivalents thereof. Further, the above-described embodiments can be implemented in combination with each other.

  1 to 4 reading system, 5 reading device, 6 imaging device, 7 output device, 11 imaging unit, 12, 21, 31 extraction unit, 13 conversion unit, 14 cutout unit, 15 determination unit, 16 reading unit, 18 storage unit

Claims (15)

An extraction unit that extracts a candidate image that is a candidate for a portion where a segment display is captured from an input image;
The candidate image is calculated using the candidate image and a preset mask to calculate a matching rate indicating the probability that a segment display is included in the candidate image. When the matching rate is equal to or higher than a preset threshold, the candidate image A determination unit that determines that the image is a segment display image;
A reading unit that reads a numerical value displayed on the segment display from the candidate image determined to be an image of a segment display;
Reading system.   The reading system according to claim 1, wherein the mask is an image of a segment display in which all segments are lit. When the candidate image is distorted, a first process for converting the candidate image so as to approach an image photographed from the front;
A second process for correcting the size of the candidate image to approach the specified size when the size of the candidate image is different from a predetermined size set in advance;
A conversion unit for performing
The reading system according to claim 1, wherein the determination unit calculates the relevance ratio using the converted candidate image and the mask. A first process for distorting the mask in accordance with the distortion of the candidate image when the candidate image is distorted;
A second process for correcting the size of the candidate image to approach the specified size when the size of the candidate image is different from a predetermined size set in advance;
A conversion unit for performing
The reading system according to claim 1, wherein the determination unit calculates the precision using the converted candidate image and the distorted mask. Based on the size and shape of the mask, further comprising a cut-out unit that cuts out a part of the candidate image,
The reading system according to claim 1, wherein the determination unit calculates the relevance ratio using the extracted candidate images. The extraction unit includes:
Extracting a contour from the input image;
Calculate the area of the region surrounded by the outline,
If the area is greater than or equal to a preset threshold, based on the contour, extract an image of a predetermined shape from the input image,
Outputting the extracted image satisfying a preset condition as the candidate image;
The reading system according to any one of claims 1 to 5.   The reading unit according to claim 1 or 2, wherein the extraction unit searches the input image for a portion that matches a template image of a segment display to be read prepared in advance and extracts the matching portion as a candidate image. system. The extraction unit includes:
Extracting features from the input image;
Correcting the input image so that the extracted feature quantity matches the feature quantity of the template image;
Extracting the candidate image that matches the template image from the corrected input image;
The reading system according to claim 7. The extraction unit includes:
Input the input image data into a pre-learned neural network,
Based on the output result of the neural network, detect the portion where the segment display was photographed in the input image, and extract as the candidate image,
The reading system according to claim 1 or 2.   The reading system according to claim 9, wherein the extraction unit further corrects distortion of the candidate image based on the output result of the neural network. The determination unit
Binarizing the candidate images;
A logical product is calculated by superimposing the binarized candidate image and the mask,
Calculating the matching rate between the image obtained from the logical product and the candidate image as the matching rate,
The reading system according to claim 1. The mask is a binary image of a 7-segment display;
The reading system according to claim 1, wherein the threshold value is set to 0.6 or more. Extract candidate images that are candidates for the part where the segment display was taken from the input image,
The candidate image is calculated using the candidate image and a preset mask to calculate a matching rate indicating the probability that a segment display is included in the candidate image. When the matching rate is equal to or higher than a preset threshold, the candidate image Is a segment display image,
A reading method of reading a numerical value displayed on a segment display from the candidate image determined to be an image of a segment display. On the computer,
Extract the candidate images that are candidates for the part where the segment display was taken from the input image,
Using the candidate image and a preset mask, the candidate image is calculated with a precision indicating the probability that the candidate image includes a segment display. When the precision is equal to or higher than a preset threshold, the candidate image To be a segment display image,
The numerical value displayed on the segment display is read from the candidate image determined to be the image of the segment display.
program.   A storage medium storing the program according to claim 14.