JP6656453B2 - Reading system, reading device, program, and storage medium - Google Patents

Description

Embodiments of the present invention relate to a reading system, a reading device , a program, and a storage medium.

  There is a system for reading a numerical value displayed on a segment display. In this system, it is desirable that the accuracy of reading numerical values is high.

JP 2008-243103 A

It is an object of the present invention to provide a reading system, a reading device , a program, and a storage medium that can improve the reading accuracy of a numerical value of a segment display.

Reading system according to the embodiment includes a determine constant section. Extracted from the input image, the candidate image segment display are candidates of the photographed portions, the determining unit, by using the mask set in advance and the candidate image, includes segment display on the candidate image The candidate rate is determined to be an image of a segment display if the precision is equal to or greater than a preset threshold .

FIG. 1 is a block diagram illustrating a configuration of a reading system according to an embodiment. 6 is a flowchart illustrating an operation of the reading system according to the embodiment. FIG. 4 is a diagram illustrating a process in the reading system according to the embodiment. FIG. 9 is a block diagram illustrating a configuration of a reading system according to a first modification of the embodiment. FIG. 13 is a block diagram illustrating a configuration of a reading system according to a second modification of the embodiment. FIG. 14 is a block diagram illustrating a configuration of a reading system according to a third modification of the embodiment. FIG. 2 is a block diagram illustrating a hardware configuration for realizing the reading system according to the embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
In the specification and the drawings of the application, the same elements as those already described are denoted by the same reference numerals, and detailed description will be appropriately omitted.

FIG. 1 is a block diagram illustrating a configuration of the reading system according to the embodiment.
The reading system according to the embodiment is used to read a numerical value displayed on a segment display from an image including the segment display. In the present embodiment, the segment display refers to a display including some display information such as numbers and characters formed by displaying a plurality of segments.

  As illustrated in FIG. 1, the reading system 1 according to the embodiment includes an imaging unit 11, an extraction unit 12, a conversion unit 13, a cutout unit 14, a determination unit 15, a reading unit 16, and a storage unit 18.

  The imaging unit 11 captures a segment display and acquires a still image. The imaging unit 11 outputs the obtained image to the extraction unit 12. When a moving image is captured by the imaging unit 11, a still image is cut out from the moving image and output to the extraction unit 12. The captured image may include an image other than the segment display.

  The extracting unit 12 extracts a candidate for an image of the segment display from the input image. Here, an image captured by the imaging unit 11 and input to the extraction unit 12 is referred to as an input image. An image that is a part of the input image and is a candidate for the segment display is called a candidate image. The candidate image is a part of the input image for which the extraction unit 12 has determined that the segment display has been photographed. The number of candidate images output from the extraction unit 12 may be plural.

  As a specific example, the extracting unit 12 includes a processing unit 12a, a binarizing unit 12b, a contour extracting unit 12c, a first selecting unit 12d, a square extracting unit 12e, and a second selecting unit 12f.

  The processing unit 12a generates, for example, a processed image obtained by converting an input image into a gray scale. The processing unit 12a outputs the processed image to the binarization unit 12b. At this time, the processing unit 12a may output, for example, a processed image obtained by reducing the input image to a preset size to the binarization unit 12b. At that time, the processing unit 12a stores the original image that has not been reduced in the storage unit 18.

  The binarization unit 12b binarizes the input processed image and generates a binary image represented by white and black. The contour extraction unit 12c extracts, for example, a contour of a region represented in white in the binary image.

  The first selection unit 12d calculates the area of the region surrounded by the contour. When a plurality of contours are extracted, the area of each region is calculated. The first selection unit 12d compares each of the calculated areas with a predetermined threshold, and selects only a region whose area is equal to or larger than the threshold. As a result, a region having an area that is too small is excluded from the candidates. The rectangle extracting unit 12e approximates the selected area with a straight line and extracts a rectangle. For example, the rectangle extracting unit 12e acquires the coordinates of the vertices of the extracted rectangle.

  The second selection unit 12f compares the extracted rectangular area with a preset condition and selects an area that meets the condition. The conditions include, for example, that the ratio of the vertical dimension to the horizontal dimension of the rectangle is within a predetermined range, the angle of the vertex is within a predetermined range, and any of the vertices of the rectangle is also near the outer frame of the image (for example, (Within 3 pixels from the frame). The second selection unit 12f outputs the selected rectangle to the conversion unit 13 as a final candidate image.

The conversion unit 13 performs the following first processing and second processing.
In the first process, the conversion unit 13 determines whether the candidate image input from the extraction unit 12 is distorted. When it is determined that the candidate image is distorted, the conversion unit 13 corrects the distortion so that the candidate image approaches an image photographed from the front.
In the second process, the conversion unit 13 determines whether or not the size of the candidate image matches a preset specified size. If the size of the candidate image does not match the specified size, the conversion unit 13 corrects the candidate image so that the size of the candidate image approaches the specified size.
The order in which the first processing and the second processing are performed is arbitrary. Alternatively, the first processing and the second processing may be performed simultaneously. The conversion unit 13 performs the first processing and the second processing at the same time by, for example, projectively converting the candidate image, and corrects the distortion and the size. The conversion unit 13 outputs the corrected candidate image to the extraction unit 14. When it is determined that the candidate image is not distorted and the size of the candidate image matches the specified size, the conversion unit 13 outputs the input candidate image to the extraction unit 14 as it is.

  The extracting unit 14 refers to the storage unit 18. The storage unit 18 stores a preset mask and its information. In the present embodiment, the mask is an image corresponding to the segment display to be read. The mask serves as reference information when reading the target segment display. For example, a binary image of a segment display in which all the segments are turned on is used as a mask. The cutout unit 14 acquires from the storage unit 18 the vertical and horizontal dimensions of the mask and the corresponding position of the mask in the actual segment display. The cutout unit 14 cuts out a part of the input candidate image based on the information. The extraction unit 14 outputs the extracted candidate image to the determination unit 15.

  The determining unit 15 refers to the storage unit 18 and calculates a matching rate using the input candidate image and the mask. The precision is a value indicating the likelihood that a candidate image includes a segment display. For example, the higher the precision, the higher the possibility that the candidate image includes the segment display.

  The determination unit 15 compares the matching rate with a preset threshold. When the matching rate is equal to or larger than the threshold, the determination unit 15 determines that the candidate image is an image of the segment display, and outputs the candidate image to the reading unit 16. When there are a plurality of candidate images, the determination unit 15 calculates and determines a matching rate for each candidate image.

Specifically, the determination unit 15 includes a binarization unit 15a, a logical operation unit 15b, a matching ratio calculation unit 15c, and a comparison unit 15d.
The binarizing unit 15a binarizes the candidate image. The logical operation unit 15b calculates a logical product of the binary image and the mask. In the processed image obtained from the logical product, only the region represented by white in the mask and white in the binary image is represented by white.

  The matching rate calculation unit 15c calculates the matching rate of the binary image with respect to the processed image as the matching rate. That is, the ratio (A2 / A1) of the area A2 of the region represented in white in the binary image to the area A1 of the region represented in white in the processed image (the number of pixels represented in white) is defined as the matching rate. Is calculated.

  The comparing unit 15d compares the matching rate with a threshold. If the matching rate is equal to or larger than the threshold, the candidate image is output to the reading unit 16. When the matching rate is less than the threshold, the comparing unit 15d determines that the candidate image is not an image of the segment display, and ends, for example, the processing.

  The reading unit 16 reads a numerical value displayed on the segment display from the input candidate image. For example, the reading unit 16 cuts out a number from the input candidate image and detects a lit segment to read the numerical value. The reading unit 16 displays the read numerical value on a monitor or outputs the numerical value to a database, for example.

  The storage unit 18 stores information necessary for processing of the reading system 1 and data generated during the processing. For example, the storage unit 18 stores a threshold value and the like to be compared when the determination is performed, in addition to the mask described above.

The operation of the reading system 1 according to the embodiment will be described with reference to FIGS.
FIG. 2 is a flowchart illustrating the operation of the reading system according to the embodiment.
FIG. 3 is a diagram illustrating a process in the reading system according to the embodiment.

  The imaging unit 11 captures an image of the segment display and acquires an image (Step S11 in FIG. 2). The processing unit 12a processes the input image (Step S12a). As a result, a processed image in which the input image is reduced and grayscaled and an original image in which the input image is grayscaled only are generated. FIG. 3A is an example of the processed image. The binarizing unit 12b binarizes the processed image and generates a binary image as shown in FIG. 3B (Step S12b). The contour extraction unit 12c extracts the contour of the binary image (Step S12c).

  The first selection unit 12d selects a region surrounded by the contour (Step S12d). As a result, for example, as shown in FIG. 3C, an area surrounded by the frame A is selected. The quadrangle extraction unit 12e extracts a quadrangle from the original image based on the extraction result of the quadrangle in the selected area (Step S12e). The second selection unit 12f selects the extracted rectangle (Step S12f) and outputs it as a candidate image. FIG. 3D illustrates a rectangle output as a candidate image.

  The conversion unit 13 converts the candidate image so as to correct the distortion and the size, as illustrated in FIG. The conversion unit 13 corrects the distortion and the size by, for example, projectively converting the candidate image (Step S13). FIG. 3H shows an example of a mask. The extraction unit 14 extracts a part of the candidate image according to the size of the mask (Step S14). As a result, an image shown in FIG.

  The binarization unit 15a binarizes the extracted candidate image as illustrated in FIG. 3G (Step S15a). The logical operation unit 15b calculates the logical product of the binary image and the mask (Step S15b). Thereby, the image shown in FIG. 3 (i) is obtained. The matching rate calculation unit 15c calculates the matching rate using the images of FIGS. 3G and 3I (step S15c). The comparing unit 15d compares the matching rate with a preset threshold value (Step S15d). When the matching rate is equal to or larger than the threshold, the reading unit 16 reads the numerical value of the segment display from the candidate image (Step S16).

  The determination unit 15 determines whether there is any other candidate image for which steps S15a to S15d have not been performed yet (step S17). If there is another candidate image, step S15a is performed on that candidate image. If there is no other candidate image, the process ends.

Effects of the embodiment will be described.
When reading the segment display, as described above, a candidate image that is a candidate for a portion where the segment display has been captured is extracted from the input image. Conventionally, for example, a correction such as distortion is appropriately performed on an extracted candidate image, and a numerical value of a segment display is read from the candidate image.
However, the candidate image may include , for example, a display similar to a segment display. In this case, the numerical value of another display is read as the numerical value of the segment display, and there is a possibility that erroneous detection may occur.

  The reading system 1 according to the embodiment includes the determination unit 15 in order to suppress such erroneous detection. Using the candidate image and a preset mask, the determination unit 15 calculates a precision that indicates the accuracy of the candidate image including the segment display. Then, the determination unit 15 determines that the candidate image is an image of the segment display when the matching rate is equal to or greater than a preset threshold.

  By performing such a determination, it is possible to read only a candidate image that is more likely to be a segment display. In particular, in the present embodiment, a mask corresponding to the segment display to be read is prepared in advance, and the candidate images are narrowed down using the mask. For this reason, even if a display similar to a segment display or another segment display is included in the input image, only the segment display to be read can be selected with higher accuracy, and the possibility of erroneous detection can be greatly reduced.

  In the example illustrated in FIG. 1, the reading system 1 according to the embodiment includes the imaging unit 11, but the reading system 1 may not include the imaging unit 11. For example, an image captured by another imaging device may be input to the reading system 1, and the reading system 1 may read a numerical value on the segment display from the input image.

In addition, the reading system 1 may not include the conversion unit 13 and the cutout unit 14. However, it is preferable that the reading system 1 include these components in order to improve reading accuracy.
For example, the distortion of the candidate image is corrected by the conversion unit 13, so that the accuracy of the matching rate can be improved. This allows the segment display to more accurately select the captured candidate images.
By cutting out a part of the candidate image by using the mask by the cutout unit 14, a subject other than the reading target can be excluded from the candidate image. Thereby, the accuracy of the precision can be improved.

  In the above description, the conversion unit 13 corrects the distortion of the candidate image when it is determined in the first processing that the candidate image is distorted. Instead of this method, when it is determined in the first processing that the candidate image is distorted, the conversion unit 13 may distort the mask in accordance with the distortion of the candidate image. Also in this case, the accuracy of the matching rate can be improved in the subsequent processing by the determination unit 15.

  FIG. 3 shows an example in which a numerical value displayed on the 7-segment display is read using a mask corresponding to the 7-segment display. However, the segment display that can be read by the reading system 1 according to the embodiment is not limited to the seven-segment display. The segment display to be read may be a 14-segment display or a 16-segment display.

  As a mask, an image of a segment display in which at least a part of a plurality of segments is turned on is used. More preferably, a binary image of a segment display in which all segments are turned on is used as a mask. By doing so, it is possible to more accurately select a candidate image captured by the segment display regardless of the display information of the segment display to be read.

  When a binary image in which all segments are turned on is used as a mask, the area of a white region in the mask increases as the number of segments increases. For example, if the candidate image contains a lot of noise and there are many white areas due to the noise, the matching rate between the candidate image and the mask may increase even though the candidate image does not contain a segment display. For this reason, it is desirable that the threshold value to be compared with the precision is adjusted according to the number of segments in the segment display to be read.

For example, when reading the numerical value of a 7-segment display, by setting the threshold to 0.6 or more, the candidate images on which the 7-segment display is captured can be more accurately selected.
Further, when the threshold value is high, even if the seven-segment display is captured in the candidate image, the possibility that the image is not determined to be an image of the seven-segment display increases. Therefore, it is desirable that the threshold value is 0.8 or less.
That is, when reading the numerical value of the 7-segment display, the accuracy of reading the numerical value can be improved by setting the threshold to 0.6 or more and 0.8 or less.

  Further, the processing in the extraction unit 12 can be appropriately changed as long as a candidate image can be extracted. For example, when the size of the input image is small, or when the input image is binarized in advance, the processing unit 12a or the binarization unit 12b is unnecessary. Further, the processing for extracting the candidate image from the input image can be changed as appropriate. These modifications will be described below.

(First Modification)
FIG. 4 is a block diagram illustrating a configuration of a reading system according to a first modification of the embodiment.
In the examples of FIGS. 1 to 3, the frame of the display area of the segment display is rectangular. Therefore, the extraction unit 12 is provided with a square extraction unit 12e according to the shape of the display area.

  When the frame of the display area is circular, the extraction unit 12 is provided with an ellipse extraction unit 12g instead of the square extraction unit 12e, as in the reading system 2 shown in FIG. The ellipse extraction unit 12g extracts an ellipse from the area selected by the first selection unit 12d. At this time, the ellipse extraction unit 12g acquires the coordinates of the extracted ellipse. The ellipse extraction unit 12g extracts an ellipse image corresponding to the coordinates from the original image and outputs the extracted image to the second selection unit 12f.

  The second selection unit 12f compares the input image of the ellipse with a preset condition and selects an ellipse that meets the condition. As the conditions, for example, the flatness of an ellipse, the distribution of luminance, and the like are used. The second selection unit 12f outputs the selected ellipse image to the conversion unit 13 as a candidate image. Subsequent processing is the same as in the example described with reference to FIGS. That is, the conversion unit 13 corrects the distortion of the candidate image, and the extraction unit 14 extracts a part of the candidate image. After the determination by the determination unit 15, the reading unit 16 reads the numerical value of the segment display.

  Generally, the numerical values of the segment displays are arranged side by side in a predetermined direction. Therefore, even when the frame of the display area of the segment display is a circle or an ellipse, the numerical value displayed on the segment display generally fits in a square. That is, even when the frame of the display area is a circle or an ellipse, the outer shape of the mask is set to a quadrangle. By making the outer shape of the mask a quadrangle, the ratio of the area where numerical values are displayed to the entire area of the mask can be increased. As a result, it is possible to reduce the influence of noise or the like in the candidate image on the precision, and improve the precision of the precision.

(Second Modification)
FIG. 5 is a block diagram illustrating a configuration of a reading system according to a second modification of the embodiment.
The reading system 3 illustrated in FIG. 5 includes an extraction unit 21 instead of the extraction unit 12, the conversion unit 13, and the extraction unit 14 in comparison with the reading system 1.

  In the reading system 3, before reading the numerical value of the segment display, the imaging unit 11 photographs only the segment display to be read from the front. Here, an image obtained by photographing only the segment display from the front is called a template image.

The extraction unit 21 includes a feature amount extraction unit 21a, a matching unit 21b, a conversion unit 21c, and a search unit 21d.
The feature amount extraction unit 21a extracts a feature amount of the input image. Further, the feature amount extraction unit 21a refers to the template image stored in the storage unit 18 and extracts the feature amount of the template image. Alternatively, the feature amount of the template image may be extracted and stored in the storage unit 18 in advance by the feature amount extracting unit 21a. In this case, the feature amount extraction unit 21a refers to the feature amount of the template image stored in the storage unit 18. The feature amount extraction unit 21a outputs the feature amount of the input image and the feature amount of the template image to the matching unit 21b.

The matching unit 21b calculates a correction amount of the input image necessary for matching the feature amount of the input image with the feature amount of the template image. For example, the matching unit 21b calculates a correction amount required to bring a distorted segment display in the input image closer to an image taken from the front. The matching unit 21b outputs the calculated correction amount and the input image to the conversion unit 13.
If the feature values cannot be matched, it is highly possible that the input image does not include the segment display. Therefore, in this case, for example, the process ends without calculating the correction amount.

  The conversion unit 21c converts the input image based on the input correction amount. Thereby, when the input image is distorted with respect to the template image, the distortion is corrected.

The search unit 21d refers to the template image stored in the storage unit 18. Then, the search unit 21d searches the input image for a part that matches the template image. The search unit 21d outputs the matched portion to the determination unit 15 as a candidate image.
Subsequent processes in the determination unit 15 and the reading unit 16 are the same as those in the reading system 1 shown in FIG.

  Note that the feature amount is extracted by the feature amount extracting unit 21a using, for example, KAZE, AKAZE (Accelerated KAZE), SIFT (Scale-invariant feature transform), or the like. The matching in the matching unit 21b is performed using, for example, KNN (K Nearest Neighbor) or FLANN (Fast Library for Approximate Nearest Neighbors). The search by the search unit 21d is performed using, for example, pattern matching based on the similarity between images.

  For example, when a plurality of segment displays are provided on a certain plane, and these segment displays are photographed, those segment displays are similarly distorted in an input image. In the reading system 3 according to the present modification, conversion of an input image using a feature amount and matching using a template image are performed. That is, the entire area of the input image is converted. For this reason, it is not necessary to correct distortion for each candidate image, and the processing can be performed faster than in the reading system 1 when a plurality of segment displays exist on the same plane.

(Third Modification)
FIG. 6 is a block diagram illustrating a configuration of a reading system according to a third modification of the embodiment.
The reading system 4 illustrated in FIG. 6 includes an extraction unit 31 instead of the extraction unit 12, the conversion unit 13, and the extraction unit 14 in comparison with the reading system 1. The extraction unit 31 includes a learning unit 31a, an input unit 31b, a detection unit 31c, and a processing unit 31d.

  In the reading system 4, a candidate image obtained by capturing a segment display is extracted from the input image using a neural network. For example, teacher data is input to the extraction unit 31 in advance, and learning of the neural network is performed. The teacher data includes, for example, a distorted image including the segment display, data indicating the position of the segment display in the image, and the distortion amount of the image.

  When the image data is input, the learning unit 31a responds (fires) to the data of the portion where the segment display is photographed, and responds to the neuron corresponding to the distortion amount of the image. Train a neural network. The learning unit 31a stores the learned neural network in the storage unit 18. Note that the learning by the learning unit 31a may be performed by a processing device in which the processing by the determination unit 15 or the reading unit 16 is performed, or may be performed by using another processing device different from this. For the learning, a processing device having a higher calculation performance is preferably used.

  After that, an image is acquired by the imaging unit 11 to read the numerical value of the segment display, and the image is input to the extraction unit 31. When an image is input, the input unit 31b refers to the learned neural network stored in the storage unit 18. Then, the input unit 31b inputs image data to the neural network.

  If there is a response of the neuron while the image data is being input to the neural network, the detection unit 31c detects the response. Then, the detecting unit 31c detects the coordinates and the amount of distortion of the image to which the neuron has reacted, and outputs the detected coordinates and the amount of distortion to the processing unit 31d.

The processing unit 31d extracts a candidate image from the input image based on the input coordinates. Further, the processing unit 31d corrects the distortion of the candidate image based on the input distortion amount. The processing unit 31d outputs the corrected candidate image to the determination unit 15.
Note that the order in which the candidate images are extracted and the distortion is corrected can be changed as appropriate. For example, the candidate image may be extracted after the distortion of the input image is corrected in the processing unit 31d.
Subsequent processes in the determination unit 15 and the reading unit 16 are the same as those in the reading system 1 shown in FIG.

  In the reading system 3 according to this modification, an image is cut out or corrected using a neural network. Therefore, even when the contrast ratio between the display area of the segment display and the outer frame is not large and it is difficult to extract the outline from the binarized image, the candidate image can be extracted with higher accuracy. Therefore, the numerical value of the segment display in the input image can be read with higher accuracy than the reading system 1.

FIG. 7 is a block diagram illustrating a hardware configuration for realizing the reading system according to the embodiment.
For example, the reading system according to the embodiment includes the reading device 5 and the imaging device 6 illustrated in FIG. The reading device 5 is, for example, a computer, and includes a ROM (Read Only Memory) 51, a RAM (Random Access Memory) 52, a CPU (Central Processing Unit) 53, and an HDD (Hard Disk Drive) 54.

  The ROM 51 stores a program for controlling the operation of the computer. The ROM 51 stores programs necessary for causing the computer to function as the extraction unit, the conversion unit, the extraction unit, the determination unit, the reading unit, the search unit, and the like in the above-described embodiment.

  The RAM 52 functions as a storage area in which programs stored in the ROM 51 are expanded. The CPU 53 reads the control program stored in the ROM 51, and controls the operation of the computer according to the control program. Further, the CPU 53 loads various data obtained by the operation of the computer into the RAM 52. The HDD 54 functions as the storage unit 18 in the above-described embodiment, and stores information necessary for reading and information obtained during the reading process.

  When the processing system 4 using a neural network is realized by using the reading device 5, the reading device 5 further includes a GPU (Graphics Processing Unit) and a dedicated chip specialized in processing of the neural network. Is also good. Further, the reading device 5 may include an embedded Multi Media Card (eMMC), a solid state drive (SSD), a solid state hybrid drive (SSHD), or the like, instead of the HDD 54.

The imaging device 6 photographs a subject (segment display) and transmits the acquired image to the reading device 5. The imaging device 6 is, for example, a camera.
The output device 7 outputs the data (the read numerical value of the segment display) output from the reading device 5 so that the user can recognize the data. The output device 7 is, for example, a monitor, a printer, a speaker, or the like.

  The reading device 5, the imaging device 6, and the output device 7 are interconnected, for example, by wire or wirelessly. Alternatively, they may be mutually connected via a network. Alternatively, at least two of the reading device 5, the imaging device 6, and the output device 7 may be incorporated in one device. For example, the reading device 5 may be integrated with an image processing unit of the imaging device 6 or the like.

  By using the reading system and the reading method according to the embodiment described above, it is possible to read the numerical value displayed on the segment display with higher accuracy. Similarly, by using a program for causing a computer to operate as a reading system, it becomes possible for a computer to read a numerical value displayed on a segment display with higher accuracy.

  While some embodiments of the present invention have been described above, these embodiments have been presented by way of example only, and are not intended to limit the scope of the inventions. These new embodiments can be implemented in other various forms, and various omissions, replacements, changes, and the like can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents. The above-described embodiments can be implemented in combination with each other.

  1-4 reading system, 5 reading device, 6 imaging device, 7 output device, 11 imaging unit, 12, 21, 31 extraction unit, 13 conversion unit, 14 cutout unit, 15 determination unit, 16 reading unit, 18 storage unit

Claims (33)

For a candidate image extracted from an input image and serving as a candidate for a portion where a segment display has been captured, a match rate indicating the likelihood that the candidate image includes the segment display using the candidate image and a preset mask. And a determination unit that determines the candidate image as an image of a segment display when the matching rate is equal to or greater than a preset threshold.   The reading system according to claim 1, wherein the mask is an image of a segment display in which all segments are turned on. When the candidate image is distorted, a first process of converting the candidate image closer to an image taken from the front,
When the size of the candidate image is different from a preset specified size, a second process of correcting the size of the candidate image to be closer to the specified size;
Further comprising a conversion unit for performing
The reading system according to claim 1, wherein the determination unit calculates the matching rate using the converted candidate image and the mask. When the candidate image is distorted, first processing for distorting the mask in accordance with the distortion of the candidate image;
A second process for correcting the size of the candidate image to be closer to the specified size when the size of the candidate image is different from a predetermined specified size;
Further comprising a conversion unit for performing
The reading system according to claim 1, wherein the determination unit calculates the matching rate using the converted candidate image and the distorted mask. A cutout unit that cuts out a part of the candidate image based on the size and shape of the mask,
The reading system according to claim 1, wherein the determination unit calculates the matching rate using the clipped candidate images.   An extraction unit that extracts the candidate image from the input image,
  From the candidate image determined to be an image of the segment display, a reading unit that reads characters displayed on the segment display,
  The reading system according to claim 1, further comprising: The extraction unit includes:
Extracting a contour from the input image,
Calculating the area of the region surrounded by the contour,
If the area is equal to or greater than a preset threshold, based on the contour, extract an image of a predetermined shape from the input image,
Outputting the extracted image satisfying a preset condition as the candidate image,
The reading system according to claim 6 . An extraction unit that extracts the candidate image from the input image,
From the candidate image determined to be an image of the segment display, a reading unit that reads characters displayed on the segment display,
Further comprising
3. The reading device according to claim 1, wherein the extraction unit searches the input image for a portion that matches a template image of a segment display to be read, which is prepared in advance, and extracts the matched portion as a candidate image. 4. system. The extraction unit includes:
Extracting a feature amount from the input image,
As the extracted feature amount matches the feature amount of the template image,
Correcting the input image,
Extracting the candidate image that matches the template image from the corrected input image,
The reading system according to claim 8 . An extraction unit that extracts the candidate image from the input image,
From the candidate image determined to be an image of the segment display, a reading unit that reads characters displayed on the segment display,
Further comprising
The extraction unit includes:
The data of the input image is input to a neural network learned in advance,
Based on the output result of the neural network, to detect the portion where the segment display was photographed in the input image, to extract as the candidate image,
The reading system according to claim 1. The reading system according to claim 10 , wherein the extraction unit further corrects a distortion of the candidate image based on the output result of the neural network.   The reading system according to any one of claims 6 to 11, further comprising an imaging unit configured to transmit the input image obtained by capturing an image of the segment display to the extraction unit.   13. The reading system according to claim 12, wherein the imaging unit captures a moving image in which a segment display is captured, and cuts out the input image from the moving image.   The reading system according to any one of claims 6 to 13, further comprising an output unit that outputs the read character. The determination unit includes:
Binarizing the candidate image,
Calculate the logical product by superimposing the binary candidate image and the mask,
Calculating the matching rate between the image obtained from the logical product and the candidate image as the matching rate,
The reading system according to any one of claims 1 to 14 . The mask is a binary image of a 7 segment display,
Reading system according to the threshold value, any one of claims 1 to 15 to be set to 0.6 or more.   An imaging unit that captures an input image by capturing a segment display;
  An extraction unit that extracts a candidate image that is a candidate for a portion where the segment display is captured from the input image,
  Using the candidate image and a preset mask, calculate a precision that indicates the likelihood that the candidate image includes a segment display, and when the precision is equal to or greater than a preset threshold, the candidate image A determination unit that determines that the image of the segment display
  From the candidate image determined to be an image of the segment display, a reading unit that reads characters displayed on the segment display,
  A reading device comprising:   18. The reading device according to claim 17, wherein the mask is an image of a segment display in which all segments are turned on.     When the candidate image is distorted, a first process of converting the candidate image closer to an image taken from the front,
    A second process of correcting the size of the candidate image to be closer to the specified size when the size of the candidate image is different from a predetermined specified size;
  Further comprising a conversion unit for performing
  19. The reading device according to claim 17, wherein the determination unit calculates the matching rate using the converted candidate image and the mask.     When the candidate image is distorted, first processing for distorting the mask in accordance with the distortion of the candidate image;
    A second process for correcting the size of the candidate image to be closer to the specified size when the size of the candidate image is different from a predetermined specified size;
  Further comprising a conversion unit for performing
  The reading device according to claim 17, wherein the determination unit calculates the matching rate using the converted candidate image and the distorted mask.   A cutout unit that cuts out a part of the candidate image based on the size and shape of the mask,
  The reading device according to any one of claims 17 to 20, wherein the determination unit calculates the matching rate using the clipped candidate images.   The determination unit includes:
    Binarizing the candidate image,
    Calculate the logical product by superimposing the binary candidate image and the mask,
    Calculating the matching rate between the image obtained from the logical product and the candidate image as the matching rate,
  A reading device according to any one of claims 17 to 21.   The mask is a binary image of a 7 segment display,
  23. The reading device according to claim 17, wherein the threshold is set to 0.6 or more.   24. The reading device according to claim 17, wherein the character displayed on the segment display indicates a numerical value. On the computer,
For a candidate image extracted from an input image and serving as a candidate for a portion where a segment display has been captured, a match rate indicating the likelihood that the candidate image includes the segment display using the candidate image and a preset mask. is calculated, if the matching degree is preset threshold or more, Ru is determined that the image of the candidate image segment display,
program.   26. The program according to claim 25, wherein the mask is an image of a segment display in which all segments are turned on.   On the computer,
    When the candidate image is distorted, a first process of converting the candidate image closer to an image taken from the front,
    A second process of correcting the size of the candidate image to be closer to the specified size when the size of the candidate image is different from a predetermined specified size;
  And execute
  In the determination, the matching rate is calculated using the converted candidate image and the mask,
  The program according to claim 25 or 26.   On the computer,
    When the candidate image is distorted, first processing for distorting the mask in accordance with the distortion of the candidate image;
    A second process for correcting the size of the candidate image to be closer to the specified size when the size of the candidate image is different from a predetermined specified size;
  And execute
  In the determination, the conversion ratio is calculated using the converted candidate image and the distorted mask,
  The program according to claim 25 or 26.   On the computer,
    Based on the size and shape of the mask, cut out a part of the candidate image,
  In the determination, the matching rate is calculated using the extracted candidate image,
  A program according to any one of claims 25 to 28.   In the computer, in the determination,
    Binarizing the candidate image,
    Superimposing the binarized candidate image and the mask to calculate a logical product,
    The coincidence rate between the image obtained from the logical product and the candidate image is calculated as the matching rate,
  A program according to any one of claims 25 to 29.   The mask is a binary image of a 7 segment display,
  31. The program according to claim 25, wherein the threshold is set to 0.6 or more.   The program according to any one of claims 25 to 31, wherein the computer causes the computer to read a character displayed on the segment display from the candidate image determined to be an image on the segment display. A storage medium storing the program according to any one of claims 25 to 32 .