JP6684947B2 - Reading system, reading device, program, and storage medium - Google Patents

Description

Embodiments of the present invention relate to a reading system, a reading device , a program, and a storage medium.

  There is a system that reads the numerical value displayed on the segment display. In this system, it is desirable that the numerical reading accuracy be high.

JP, 2017-10170, A

The problem to be solved by the present invention is to provide a reading system, a reading device , a program, and a storage medium that can improve the reading accuracy of numerical values of a segment display.

Reading system according to the embodiment includes a determine constant section. Extracted from the input image, the candidate image of the part-segment display is captured, the determination unit calculates an angle with respect to respective reference lines of a plurality of straight lines are detected from the candidate images, the said angle It is determined whether the candidate image is an image on a segment display based on a distribution indicating the relationship with the number of straight lines .

It is a block diagram showing the composition of the reading system concerning an embodiment. 6 is a flowchart showing the operation of the reading system according to the embodiment. It is a figure which illustrates the process in the reading system which concerns on embodiment. It is a block diagram showing the composition of the reading system concerning the 1st modification of an embodiment. It is a block diagram showing the composition of the reading system concerning the 2nd modification of an embodiment. It is a block diagram showing the composition of the reading system concerning the 3rd modification of an embodiment. It is a block diagram showing the hardware constitutions for realizing the reading system concerning an embodiment.

Each embodiment of the present invention will be described below with reference to the drawings.
In the specification and the drawings of the application, components similar to those described above are designated by like reference numerals, and a detailed description is omitted as appropriate.

FIG. 1 is a block diagram showing the configuration of the reading system according to the embodiment.
The reading system according to the embodiment is used to read a numerical value displayed on a segment display from an image including the segment display.
In the present embodiment, the segment display refers to a display including some display information such as numbers and characters configured by displaying a plurality of segments.

  As illustrated in FIG. 1, the reading system 1 according to the embodiment includes an imaging unit 11, an extraction unit 12, a conversion unit 13, a determination unit 15, a reading unit 16, and a storage unit 18.

  The image capturing unit 11 captures a still image by capturing an image of the segment display. The imaging unit 11 outputs the acquired image to the extraction unit 12. When a moving image is captured by the image capturing unit 11, a still image is cut out from the moving image and output to the extraction unit 12. The captured image may include something other than the segment display.

  The extraction unit 12 extracts a candidate for an image on the segment display from the input image. Here, the image captured by the imaging unit 11 and input to the extraction unit 12 is referred to as an input image. An image that is a part of the input image and is a candidate for the segment display is called a candidate image. The candidate image is a part of the input image determined by the extraction unit 12 as the segment display being photographed. There may be a plurality of candidate images output from the extraction unit 12.

  As a specific example, the extraction unit 12 includes a processing unit 12a, a binarization unit 12b, a contour extraction unit 12c, a first selection unit 12d, a quadrangle extraction unit 12e, and a second selection unit 12f.

  The processing unit 12a generates a processed image by converting the input image into a gray scale, for example. The processing unit 12a outputs the processed image to the binarization unit 12b. At this time, the processing unit 12a may output, for example, a processed image obtained by reducing the input image to a preset size to the binarization unit 12b. At that time, the processing unit 12 a stores the non-reduced full-size image in the storage unit 18.

  The binarizing unit 12b binarizes the input processed image to generate a binary image represented by white and black. The contour extraction unit 12c extracts the contour of a region represented by, for example, white in the binary image.

  The first selection unit 12d calculates the area of the region surrounded by the contour. When a plurality of contours are extracted, the area of each region is calculated. The first selection unit 12d compares each calculated area with a predetermined threshold value and selects only a region having an area equal to or larger than the threshold value. As a result, a region having an excessively small area is excluded from the candidates.

  The quadrangle extraction unit 12e linearly approximates the selected area and extracts a quadrangle. For example, the quadrangle extraction unit 12e acquires the coordinates of the vertices of the extracted quadrangle.

  The second selection unit 12f compares the extracted quadrangular area with preset conditions and selects an area that meets the conditions. The conditions are, for example, that the ratio of the vertical dimension to the horizontal dimension of the quadrangle is within a predetermined range, the angle of the apex is within a predetermined range, and any of the vertices of the quadrangle is near the outer frame of the image ( It is not within 3 pixels from the frame). The second selection unit 12f outputs the selected quadrangle to the conversion unit 13 as a final candidate image.

The conversion unit 13 performs the following first processing and second processing.
In the first process, the conversion unit 13 determines whether the candidate image input from the extraction unit 12 is distorted. When it is determined that the candidate image is distorted, the conversion unit 13 corrects the distortion so that the candidate image approaches the image captured from the front.
In the second process, the conversion unit 13 determines whether the size of the candidate image matches a preset specified size. When the size of the candidate image does not match the specified size, the conversion unit 13 corrects the candidate image so that the size of the candidate image approaches the specified size.
The conversion unit 13 corrects the distortion and the size by, for example, projectively converting the candidate image. The conversion unit 13 outputs the corrected candidate image to the determination unit 15. When it is determined that the candidate image is not distorted and the size of the candidate image matches the specified size, the conversion unit 13 outputs the input candidate image as it is to the determination unit 15.

  The determination unit 15 includes a straight line detection unit 15a, an angle calculation unit 15b, a distribution generation unit 15c, a first comparison unit 15d, an end point group detection unit 15e, an evaluation unit 15f, and a second comparison unit 15g.

The straight line detection unit 15a detects a plurality of straight lines included in the image subjected to the edge detection. For example, in the detection of a straight line, the start point and the end point of the line segment are detected. The straight line detection unit 15a outputs the detection result to the angle calculation unit 15b and stores it in the storage unit 18. Line detection unit 15 a detects a predetermined length or more line segments at a predetermined angular resolution.

  The angle calculator 15b calculates the angle of each straight line with respect to the reference line. The angle calculation unit 15b uses, for example, the image base (a straight line extending horizontally) as a reference line, and calculates an angle between the reference line and each straight line. The distribution generation unit 15c generates a distribution indicating the relationship between the angle and the number of straight lines based on the calculation result.

The first comparison unit 15d compares the generated distribution with a preset first condition. The first condition is, for example, that the sum of the total number of straight lines near the first angle and the total number of straight lines near the second angle inclined with respect to the first angle in the generated distribution is a preset value. That is all.
For example, the first angle is 0 degrees and the second angle is 80 degrees. The “vicinity” includes, for example, a range of −10 degrees or more and +10 degrees or less centered on the first angle or the second angle. In this case, the first comparison unit 15d calculates the sum of the total number of straight lines included in the range of −10 degrees to 10 degrees and the total number of straight lines included in the range of 70 degrees to 90 degrees in the distribution, Compare with a preset value.

For example, in an image, the angle of a line parallel to the horizontal direction is 0 degree, and the angle of a line parallel to the vertical direction is 90 degrees. When the segment display to be read is a 7-segment display, many straight lines of about 0 degrees and about 80 degrees are detected. When the segment display to be read is a 14-segment display or a 16-segment display, a straight line of about 0 degrees, a straight line of about 45 degrees, and a straight line of about 80 degrees are often detected.
The first condition is set based on this characteristic. If the distribution satisfies the first condition, the candidate image is likely to include a segmented display.

  When the distribution satisfies the first condition, the first comparison unit 15d outputs the determination result to the endpoint group detection unit 15e. When the distribution does not satisfy the first condition, there is a high possibility that the candidate image does not include the segment display, and thus the process ends, for example.

  When the determination result is input from the first comparison unit 15d, the endpoint group detection unit 15e refers to, for example, the straight line detection result stored in the storage unit 18. Then, the end point group detection unit 15e detects the end point group in which the end points (start point and end point) of each straight line are gathered from the detection result of the straight line.

  The end point group detection unit 15e divides the candidate image into a plurality of areas (in a matrix) at predetermined intervals in the horizontal and vertical directions, and extracts an area having a predetermined number or more of end points as an end point group. When the candidate image includes a segment display, a plurality of end point groups can be detected from the candidate image. The position where these end point groups are detected corresponds to the position of the end of the segment.

  For example, the evaluation unit 15f calculates, for each end point group, a vector from a certain end point group to an end point group adjacent thereto. The vector can be obtained, for example, by obtaining the amount of movement necessary for superimposing one end point group on another end point group. The evaluation unit 15f calculates the ratio of the sizes of the adjacent vectors and the angle between the vectors. The evaluation unit 15f outputs these evaluation values to the second comparison unit 15g.

  The second comparison unit 15g compares the evaluation value with the second condition. The second condition includes, for example, a first range regarding the ratio of the magnitudes and a second range regarding the angle. When the size ratio is within the first range and the angle is within the second range, the second comparing unit 15g determines that the candidate image is the image of the segment display, and the reading unit reads the candidate image. Output to 16.

  For example, a typical 7-segment display includes a segment extending vertically (vertical segment) and a segment extending horizontally (horizontal segment) when viewed from the front. Typically, the ratio of the lengths of the vertical segments is approximately 1. The ratio of the length of the horizontal segment to the length of the vertical segment is 1 or less. Further, the angle between the vertical segment and the horizontal segment is about 80 degrees or about 100 degrees. The angle between the vertical segments is 180 degrees.

  The ratio of the sizes of the vectors calculated by the evaluation unit 15f corresponds to the ratio of the lengths of the vertical segments or the ratio of the lengths of the vertical segments and the horizontal segments. The angle between the vectors calculated by the evaluation unit 15f corresponds to the angle between the vertical segment and the horizontal segment, or the angle between the vertical segments. Therefore, by comparing these evaluation values with the preset first range and second range, respectively, it is possible to determine whether or not the candidate image is the image of the segment display.

When the evaluation unit 15f calculates the plurality of size ratios and the plurality of angles, the second comparison unit 15g compares the respective size ratios with the first range and determines the respective angles. Compare with the second range.
The second comparison unit 15g detects, for example, the total number of calculated size ratios. The second comparison unit 15g detects the number of size ratios included in the first range. Then, the second comparison unit 15g calculates the ratio of the number of the size ratios included in the first range to the total number.
Similarly, the second comparison unit 15g calculates, for example, the ratio of the number of angles included in the second range to the total number of calculated angles.
The second comparing unit 15g determines that the candidate image is the image of the segment display when the calculated two ratios are equal to or more than the preset thresholds, respectively.

  The reading unit 16 reads the numerical value displayed on the segment display from the input candidate image. For example, the reading unit 16 reads a numerical value by cutting out a number from the input candidate image and detecting a lit segment. The reading unit 16 displays the read numerical value on a monitor or outputs it to a database, for example.

  The storage unit 18 stores information necessary for processing of the reading system 1 and data generated in the process of processing. For example, the storage unit 18 stores threshold values, conditions, and the like that are compared when making a determination.

The operation of the reading system 1 according to the embodiment will be described with reference to FIGS. 2 and 3.
FIG. 2 is a flowchart showing the operation of the reading system according to the embodiment.
FIG. 3 is a diagram illustrating a process in the reading system according to the embodiment.

  The image capturing unit 11 captures an image by capturing an image of the segment display (step S11 in FIG. 2). The processing unit 12a processes the input image (step S12a). As a result, a processed image in which the input image is reduced and grayscaled and a full-scale image in which the input image is grayscaled only are generated. FIG. 3A is an example of the processed image. The binarization unit 12b binarizes the processed image to generate a binary image as shown in FIG. 3B (step S12b). The contour extracting unit 12c extracts the contour of the binary image (step S12c).

  The first selection unit 12d selects a region surrounded by the contour (step S12d). Thereby, for example, as shown in FIG. 3C, the area surrounded by the frame A is selected. The quadrangle extraction unit 12e extracts a quadrangle from the full size image based on the extraction result of the quadrangle in the selected area (step S12e). The second selection unit 12f selects the extracted quadrangle (step S12f) and outputs it as a candidate image. FIG. 3D shows a quadrangle output as a candidate image. The conversion unit 13 converts the candidate image so as to correct the distortion, as illustrated in FIG. The conversion unit 13 corrects the distortion and the size by, for example, projectively converting the candidate image (step S13).

  As shown in FIG. 3F, the straight line detection unit 15a detects a plurality of straight lines SL from the candidate image (step S15a). Note that in FIG. 3F and subsequent figures, only a part of the candidate images is illustrated. The angle calculator 15b calculates the angle of each straight line SL (step S15b). As illustrated in FIG. 3G, the distribution generation unit 15c generates a distribution indicating the relationship between the angle θ and the number N of straight lines (step S15c).

  The first comparison unit 15d determines whether the distribution satisfies the first condition (step S15d). For example, the first comparing unit 15d folds a peak near 0 degree and a peak near 80 degrees at an intermediate angle between these peaks and overlaps them. As a result, the distribution illustrated by the broken line in FIG. In this distribution, the first comparison unit 15d calculates the total number N of straight lines included in the range of -10 degrees to 10 degrees, and compares it with a preset value.

  When the first condition is satisfied (when the total number is equal to or more than the value), the end point group detection unit 15e determines from the detection result of the straight line illustrated in FIG. 3F to that illustrated in FIG. Then, the end point group EG is detected (step S15e). The evaluation unit 15f calculates a vector connecting the end point groups adjacent to each other, and calculates an evaluation value (size ratio and angle) (step S15f).

  The second comparison unit 15g determines whether the evaluation value satisfies the second condition (step S15g). When the evaluation value satisfies the second condition, the reading unit 16 reads the numerical value of the segment display from the candidate image (step S16).

  The determination unit 15 determines whether there is another candidate image for which steps S15a to S15g have not been performed yet (step S17). If there is another candidate image, step S15a is performed for that candidate image. If there is no other candidate image, the process ends.

The effects of the embodiment will be described.
When reading the segment display, as described above, candidate images that are candidates for the portion where the segment display is photographed are extracted from the input image. Conventionally, for example, the extracted candidate image is appropriately corrected for distortion and the like, and the numerical value of the segment display is read from the candidate image.
However, the candidate image may include a display similar to a segment display , for example. In this case, the numerical value of another display may be read as the numerical value of the segment display, and erroneous detection may occur.

  In order to suppress such erroneous detection, the reading system 1 according to the embodiment determines whether the candidate image includes a segment display based on the result of the straight line detection and the result of the end point group detection for the candidate image. There is. By reading only the candidate images determined to include the segment display, the possibility of erroneous detection can be reduced.

  Further, the judgment based on the result of the straight line detection and the result of the detection of the end point group uses the features (segment length and angle) of a general segment display. Therefore, the reading system 1 can be applied to various segment displays and can read the numerical values with higher accuracy.

  In addition, in the example shown in FIGS. 1 to 3, the determination is performed by the first comparing unit 15d and the second comparing unit 15g. However, in the reading system 1, the first comparing unit 15d and the second comparing unit 15d are used. The determination may be performed by only one of the parts 15g. Even in this case, since the candidate images that are unlikely to include the segment display can be excluded before the reading by the reading unit 16, the numerical reading accuracy of the segment display can be improved. However, in order to further improve the reading accuracy, it is desirable that both the first comparison unit 15d and the second comparison unit 15g be provided.

  In the example illustrated in FIG. 1, the reading system 1 according to the embodiment includes the image capturing unit 11, but the reading system 1 does not have to include the image capturing unit 11. For example, an image captured by another imaging device may be input to the reading system 1, and the reading system 1 may read the numerical value of the segment display from the input image.

Further, the reading system 1 may not include the conversion unit 13, but it is desirable to include the conversion unit 13 in order to improve the reading accuracy.
For example, the conversion unit 13 corrects the distortion of the candidate image, so that the precision of the matching rate can be improved. This allows the segment display to more accurately select the photographed candidate images.

  Further, FIG. 3 shows an example of reading the numerical value displayed on the 7-segment display. However, the segment display that can be read by the reading system 1 according to the embodiment is not limited to the 7-segment display. The segment display to be read may be a 14-segment display or a 16-segment display.

  Further, the processing in the extraction unit 12 can be appropriately changed as long as the candidate image can be extracted. For example, when the size of the input image is small, or when the input image is binarized in advance, the processing unit 12a or the binarization unit 12b is unnecessary. Further, the process for extracting the candidate image from the input image can be changed as appropriate. These modifications will be described below.

(First modification)
FIG. 4 is a block diagram showing the configuration of the reading system according to the first modified example of the embodiment.
In the examples of FIGS. 1 to 3, the frame of the display area of the segment display is a quadrangle. Therefore, the extraction unit 12 is provided with the quadrilateral extraction unit 12e in accordance with the shape of the display area.

  When the frame of the display area is circular, as in the reading system 2 shown in FIG. 4, the extraction unit 12 is provided with an ellipse extraction unit 12g instead of the quadrangle extraction unit 12e. The ellipse extraction unit 12g extracts an ellipse from the area selected by the first selection unit 12d. At this time, the ellipse extraction unit 12g acquires the coordinates of the extracted ellipse. The ellipse extraction unit 12g extracts an image of an ellipse corresponding to the above coordinates in the full-scale image and outputs it to the second selection unit 12f.

  The second selection unit 12f compares the input ellipse image with a preset condition and selects an ellipse that meets the condition. As the conditions, for example, elliptic flatness, luminance distribution, etc. are used. The second selection unit 12f outputs the selected elliptical image to the conversion unit 13 as a candidate image. Subsequent processing is the same as the example described with reference to FIGS. That is, the conversion unit 13 corrects the distortion of the candidate image. Then, after the determination by the determination unit 15, the reading unit 16 reads the numerical value of the segment display.

(Second modified example)
FIG. 5 is a block diagram showing the configuration of the reading system according to the second modification of the embodiment.
The reading system 3 illustrated in FIG. 5 includes an extracting unit 21 instead of the extracting unit 12 and the converting unit 13 in comparison with the reading system 1.

  In the reading system 3, before reading the numerical value of the segment display, the imaging unit 11 photographs only the segment display to be read from the front. Here, an image obtained by shooting only the segment display from the front is called a template image.

The extraction unit 21 includes a feature amount extraction unit 21a, a matching unit 21b, a conversion unit 21c, and a search unit 21d.
The feature amount extraction unit 21a extracts the feature amount of the input image. Further, the feature amount extraction unit 21a refers to the template image stored in the storage unit 18 and extracts the feature amount of the template image. Alternatively, the feature quantity of the template image may be extracted in advance by the feature quantity extraction unit 21 a and stored in the storage unit 18. In this case, the feature quantity extraction unit 21a refers to the feature quantity of the template image stored in the storage unit 18. The feature amount extraction unit 21a outputs the feature amount of the input image and the feature amount of the template image to the matching unit 21b.

The matching unit 21b calculates the correction amount of the input image necessary for matching the feature amount of the input image with the feature amount of the template image. For example, the matching unit 21b calculates a correction amount necessary to bring the distorted segment display in the input image closer to the image captured from the front. The matching unit 21b outputs the calculated correction amount and input image to the conversion unit 13.
If the feature quantities cannot be matched with each other, there is a high possibility that the input image does not include the segment display. Therefore, in this case, for example, the process ends without calculating the correction amount.

  The conversion unit 21c converts the input image based on the input correction amount. Thereby, when the input image is distorted with respect to the template image, the distortion is corrected.

The search unit 21d refers to the template image stored in the storage unit 18. Then, the search unit 21d searches the input image for a portion that matches the template image. The search unit 21d outputs the matched portion to the determination unit 15 as a candidate image.
Subsequent processing in the determination unit 15 and the reading unit 16 is similar to that in the reading system 1 shown in FIG.

  Note that the extraction of the feature amount by the feature amount extraction unit 21a is performed using, for example, KAZE, AKAZE (Accelerated KAZE), SIFT (Scale-invariant feature transform), or the like. The matching in the matching unit 21b is performed using, for example, KNN (K Nearest Neighbors) or FLANN (Fast Library for Approximate Nearest Neighbors). The search by the search unit 21d is performed using, for example, pattern matching based on the similarity between images.

  In the reading system 3 according to this modification, conversion of an input image using a feature amount and matching using a template image are performed. That is, since the entire area of the input image is converted, it is effective when a plurality of segment displays are present on the same plane as compared with the reading system 1.

(Third modification)
FIG. 6 is a block diagram showing the configuration of the reading system according to the third modified example of the embodiment.
In comparison with the reading system 1, the reading system 4 illustrated in FIG. 6 includes an extracting unit 31 instead of the extracting unit 12 and the converting unit 13. The extraction unit 31 includes a learning unit 31a, an input unit 31b, a detection unit 31c, and a processing unit 31d.

  In the reading system 4, a neural network is used to extract a candidate image of which the segment display is captured from the input image. For example, teacher data is input to the extraction unit 31 in advance and learning of the neural network is performed. The teacher data includes, for example, a distorted image including the segment display, data indicating the position of the segment display in the image, and the distortion amount of the image.

  When the image data is input, the learning unit 31a causes the neurons to react (fire) to the data of the portion where the segment display is photographed, and the neurons corresponding to the distortion amount of the image to react. Train a neural network. The learning unit 31a stores the learned neural network in the storage unit 18. The learning by the learning unit 31a may be performed by a processing device in which the determination unit 15 and the reading unit 16 perform processing, or may be performed by another processing device different from this. For the learning, it is preferable to use a processing device having higher speed computing performance.

  After that, the image capturing unit 11 acquires an image for reading the numerical value of the segment display, and the image is input to the extracting unit 31. When the image is input, the input unit 31b refers to the learned neural network stored in the storage unit 18. Then, the input unit 31b inputs image data to the neural network.

  If there is a reaction of the neuron while the image data is being input to the neural network, the detection unit 31c detects the reaction. Then, the detection unit 31c detects the coordinates and the distortion amount of the image in which the neuron has reacted, and outputs the detection result to the processing unit 31d.

The processing unit 31d extracts a candidate image from the input image based on the input coordinates. Further, the processing unit 31d corrects the distortion of the candidate image based on the input distortion amount. The processing unit 31d outputs the corrected candidate image to the determination unit 15.
Note that the order in which the candidate image is extracted and the distortion is corrected can be changed as appropriate. For example, the processing unit 31d may extract the candidate image after the distortion of the input image is corrected.
Subsequent processing in the determination unit 15 and the reading unit 16 is similar to that in the reading system 1 shown in FIG.

  In the reading system 3 according to this modification, an image is cut out and corrected using a neural network. Therefore, even when the contrast ratio between the display area of the segment display and its outer frame is not large and it is difficult to extract the contour from the binarized image, the candidate image can be extracted with higher accuracy. Therefore, as compared with the reading system 1, the numerical value of the segment display in the input image can be read with higher accuracy.

FIG. 7 is a block diagram showing a hardware configuration for realizing the reading system according to the embodiment.
For example, the reading system according to the embodiment includes the reading device 5 and the imaging device 6 illustrated in FIG. 7. The reading device 5 is, for example, a computer, and includes a ROM (Read Only Memory) 51, a RAM (Random Access Memory) 52, a CPU (Central Processing Unit) 53, and an HDD (Hard Disk Drive) 54.

  The ROM 51 stores a program that controls the operation of the computer. The ROM 51 stores programs necessary for causing the computer to function as the extraction unit, the conversion unit, the determination unit, the reading unit, the search unit, and the like in the above-described embodiments.

  The RAM 52 functions as a storage area in which the programs stored in the ROM 51 are expanded. The CPU 53 reads the control program stored in the ROM 51 and controls the operation of the computer according to the control program. Further, the CPU 53 expands various data obtained by the operation of the computer in the RAM 52. The HDD 54 functions as the storage unit 18 in the above-described embodiment, and stores information necessary for reading and information obtained during the reading process.

  When the reading system 4 is realized by using the reading device 5, the reading device 5 further includes a GPU (Graphics Processing Unit) and a dedicated chip specialized for the processing of the neural network. Is also good. Further, the reading device 5 may include an eMMC (embedded Multi Media Card), an SSD (Solid State Drive), an SSHD (Solid State Hybrid Drive), or the like, instead of the HDD 54.

The imaging device 6 photographs a subject (segment display) and transmits the acquired image to the reading device 5. The imaging device 6 is, for example, a camera.
The output device 7 outputs the data output from the reading device 5 (the read numerical values of the segment display) so that the user can recognize the data. The output device 7 is, for example, a monitor, a printer, a speaker, or the like.

  The reading device 5, the imaging device 6, and the output device 7 are connected to each other, for example, by wire or wirelessly. Alternatively, these may be connected to each other via a network. Alternatively, at least two of the reading device 5, the imaging device 6, and the output device 7 may be incorporated in one device. For example, the reading device 5 may be integrated with the image processing unit of the imaging device 6 or the like.

  By using the reading system and the reading method according to the embodiment described above, it becomes possible to read the numerical value displayed on the segment display with higher accuracy. Similarly, by using a program for operating the computer as a reading system, it is possible to cause the computer to read the numerical value displayed on the segment display with higher accuracy.

  Although some embodiments of the present invention have been illustrated above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto. Further, the above-described embodiments can be implemented in combination with each other.

  1 to 4 reading system, 5 reading device, 6 imaging device, 7 output device, 11 imaging unit, 12, 21, 31 extraction unit, 13 conversion unit, 15 determination unit, 16 reading unit, 18 storage unit

Claims (31)

  With respect to the candidate image of the portion where the segment display is captured, which is extracted from the input image, the angles of the plurality of straight lines detected from the candidate image with respect to the reference line are calculated, and the relationship between the angle and the number of the straight lines is calculated. A reading system including a determination unit that determines whether the candidate image is an image of a segment display based on the distribution indicating.   The sum of the number of the straight lines in the vicinity of the first angle and the number of the straight lines in the vicinity of the second angle inclined with respect to the first angle in the distribution is preset. The reading system according to claim 1, wherein the candidate image is determined to be an image of a segment display when the value is equal to or more than a value. The determination unit further includes
Detecting a group of end points in the candidate image,
Calculate the vector that connects adjacent end point groups,
In addition to the distribution, the determination is performed based on at least one of the ratio of the sizes of the adjacent vectors and the angle between the adjacent vectors,
The reading system according to claim 1.   In the distribution, the determination unit presets a sum of the number of straight lines near the first angle and the number of straight lines near the second angle inclined with respect to the first angle. When a first condition that is equal to or more than a value and a second condition that the ratio of the magnitudes is in a preset first range and the angle is in a preset second range are satisfied. The reading system according to claim 3, wherein the candidate image is determined to be an image on a segment display. About the candidate image of the part where the segment display was taken, extracted from the input image,
Detecting a group of end points in the candidate image,
Calculate the vector that connects adjacent end point groups,
A reading system including a determination unit that determines whether or not the candidate image is an image of a segment display, based on at least one of the size ratio between adjacent vectors and the angle between adjacent vectors. .   The determination unit determines that the candidate image is an image of a segment display when the size ratio is in a preset first range and the angle is in a preset second range. The reading system according to claim 5. An extraction unit that extracts the candidate image from the input image,
From the candidate image determined to be an image of the segment display, a reading unit for reading the characters displayed on the segment display,
The reading system according to claim 1, further comprising:   The reading system according to claim 7, further comprising an imaging unit that transmits the input image obtained by capturing an image of a segment display to the extraction unit.   The reading system according to claim 8, wherein the imaging unit captures a moving image showing a segment display and cuts out the input image from the moving image.   The reading system according to any one of claims 7 to 9, further comprising an output unit that outputs the read characters. The extraction unit is
Extract the contour from the input image,
Calculate the area of the area surrounded by the contour,
If the area is greater than or equal to a preset threshold value, based on the contour, to extract an image of a predetermined shape from the input image,
Outputting the extracted image satisfying a preset condition as the candidate image,
Reading system according to any one of claims 7-10. The extraction unit, from the input image, prepared in advance, and searching for a portion matched with the reading target segment display of the template image, any one of claims 7 to 10 for extracting the portion of matched as candidate images The reading system described in 3. The extraction unit is
Extract the feature amount from the input image,
The input image is corrected so that the extracted feature amount matches the feature amount of the template image,
From the corrected input image, extract the candidate image that matches the template image,
The reading system according to claim 12 . The extraction unit is
Input the data of the input image to a pre-learned neural network,
Based on the output result of the neural network, detects the portion of the input image captured by the segment display, and extracts as the candidate image,
The reading system according to any one of claims 7 to 10. The reading system according to claim 14 , wherein the extraction unit further corrects the distortion of the candidate image based on the output result of the neural network. When the candidate image is distorted, a first process of converting the candidate image so as to be closer to an image captured from the front,
A second process of correcting the size of the candidate image so as to approach the specified size when the size of the candidate image is different from a specified size set in advance;
Further comprising a conversion unit for performing
The determination unit, a reading system according to any one of claims 1 to 15 for performing the determination using the transformed the candidate image. An imaging unit that captures an input image by shooting the segment display,
An extraction unit for extracting a candidate image of a portion where the segment display is photographed from the input image,
The angle of each of the plurality of straight lines detected from the candidate image with respect to the reference line is calculated, and it is determined whether the candidate image is a segment display image based on the distribution indicating the relationship between the angle and the number of straight lines. A determination unit to perform,
From the candidate image determined to be an image of the segment display, a reading unit for reading the characters displayed on the segment display,
A reader equipped with.   The sum of the number of the straight lines in the vicinity of the first angle and the number of the straight lines in the vicinity of the second angle inclined with respect to the first angle in the distribution is preset. 18. The reading device according to claim 17, wherein the candidate image is determined to be an image of a segment display when the value is equal to or more than a value. The determination unit further includes
Detecting a group of end points in the candidate image,
Calculate the vector that connects adjacent end point groups,
In addition to the distribution, the determination is performed based on at least one of the ratio of the sizes of the adjacent vectors and the angle between the adjacent vectors,
The reading device according to claim 17. An imaging unit that captures an input image by shooting the segment display,
An extraction unit for extracting a candidate image of a portion where the segment display is photographed from the input image,
Detecting a group of end points in the candidate image,
Calculate the vector that connects adjacent end point groups,
Based on at least one of the size ratio of adjacent vectors, and the angle between the adjacent vectors, to determine whether the candidate image is an image of a segment display, a determination unit,
From the candidate image determined to be an image of the segment display, a reading unit for reading the characters displayed on the segment display,
A reader equipped with.   The determination unit determines that the candidate image is an image of a segment display when the size ratio is in a preset first range and the angle is in a preset second range. The reading device according to claim 20.   22. The reading device according to claim 17, wherein the imaging unit captures a moving image showing a segment display and cuts out the input image from the moving image.   23. The reading device according to claim 17, wherein the character displayed on the segment display indicates a numerical value. On the computer,
With respect to the candidate image of the portion where the segment display is captured, which is extracted from the input image, the angles of the plurality of straight lines detected from the candidate image with respect to the reference line are calculated, and the relationship between the angle and the number of the straight lines is calculated. A determination is made based on the distribution indicating that the candidate image is an image of a segment display,
program. On the computer,
In the distribution, the sum of the number of straight lines near the first angle and the number of straight lines near the second angle inclined with respect to the first angle is equal to or greater than a preset value. To determine that the candidate image is an image of a segment display,
The program according to claim 24. In the computer,
Detect an end point group in which the end points are gathered in the candidate image,
Calculate the vector that connects adjacent end point groups,
In addition to the distribution, the determination is executed based on at least one of the ratio of the sizes of the adjacent vectors and the angle between the adjacent vectors,
The program according to claim 24. On the computer,
Extracted from the input image, with respect to the candidate image of the part where the segment display is photographed, the end point group in which the end points are gathered in the candidate image is detected
Calculate the vector that connects adjacent end point groups,
Based on at least one of the size ratio of the adjacent vectors and the angle between the adjacent vectors, it is determined whether the candidate image is an image of a segment display,
program.   Causing the computer to determine that the candidate image is an image of a segment display when the size ratio is in a preset first range and the angle is in a preset second range The program according to claim 27. The computer, from the candidate image determined as a segment display of the image, the program according to any one of claims 24-28 to read the characters displayed on the segment display.   30. The program according to claim 29, wherein the character displayed on the segment display indicates a numerical value.   A storage medium storing the program according to any one of claims 24 to 30.